turbot/steampipe-plugin-exec

Use SQL to instantly query & run shell commands on local & remote servers. Open source CLI. No DB required.

Exec Plugin for Steampipe

Use SQL to run commands locally or on remote Linux and Windows hosts. Then get the result as a table.

• Get started →
• Documentation: Table definitions & examples
• Community: Join #steampipe on Slack →
• Get involved: Issues

Quick start

Install

Download and install the latest Steampipe plugin:

steampipe plugin install exec

Configure your credentials and config file.

Configure your credential details in ~/.steampipe/config/exec.spc:

connection "exec" {
  plugin = "exec"

  protocol = "ssh"
  host = "my-remote-linux-host"
  user = "my-username"
  private_key = "~/.ssh/my-remote-linux-host.pem"
}

Run steampipe:

steampipe query

List disks on a Linux host

select
  output
from
  exec_command 
where
  command = 'df -h';

+------------------------------------------------------+
| output                                               |
+------------------------------------------------------+
| Filesystem      Size  Used Avail Use% Mounted on     |
| /dev/root       7.6G  3.4G  4.3G  44% /              |
| tmpfs           483M     0  483M   0% /dev/shm       |
| tmpfs           194M  872K  193M   1% /run           |
| tmpfs           5.0M     0  5.0M   0% /run/lock      |
| /dev/xvda15     105M  5.3M  100M   5% /boot/efi      |
| tmpfs            97M  4.0K   97M   1% /run/user/1001 |
+------------------------------------------------------+

Engines

This plugin is available for the following engines:

Engine	Description
Steampipe	The Steampipe CLI exposes APIs and services as a high-performance relational database, giving you the ability to write SQL-based queries to explore dynamic data. Mods extend Steampipe's capabilities with dashboards, reports, and controls built with simple HCL. The Steampipe CLI is a turnkey solution that includes its own Postgres database, plugin management, and mod support.
Postgres FDW	Steampipe Postgres FDWs are native Postgres Foreign Data Wrappers that translate APIs to foreign tables. Unlike Steampipe CLI, which ships with its own Postgres server instance, the Steampipe Postgres FDWs can be installed in any supported Postgres database version.
SQLite Extension	Steampipe SQLite Extensions provide SQLite virtual tables that translate your queries into API calls, transparently fetching information from your API or service as you request it.
Export	Steampipe Plugin Exporters provide a flexible mechanism for exporting information from cloud services and APIs. Each exporter is a stand-alone binary that allows you to extract data using Steampipe plugins without a database.
Turbot Pipes	Turbot Pipes is the only intelligence, automation & security platform built specifically for DevOps. Pipes provide hosted Steampipe database instances, shared dashboards, snapshots, and more.

Developing

Prerequisites:

• Steampipe
• Golang

Clone:

git clone https://github.com/turbot/steampipe-plugin-exec.git
cd steampipe-plugin-exec

Build, which automatically installs the new version to your ~/.steampipe/plugins directory:

make

Configure the plugin:

cp config/* ~/.steampipe/config
vi ~/.steampipe/config/exec.spc

Try it!

steampipe query
> .inspect exec

Further reading:

• Writing plugins
• Writing your first table

Open Source & Contributing

This repository is published under the Apache 2.0 (source code) and CC BY-NC-ND (docs) licenses. Please see our code of conduct. We look forward to collaborating with you!

Steampipe is a product produced from this open source software, exclusively by Turbot HQ, Inc. It is distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our Open Source FAQ.

Get Involved

Join #steampipe on Slack →

Want to help but don't know where to start? Pick up one of the help wanted issues:

• Steampipe
• Exec Plugin

steampipe-plugin-exec's Issues

Update docs with config arg information and which ones can be used in local/remote connection types

Is your feature request related to a problem? Please describe.
Some config args can be only be used in local or remote connection types, but it's difficult to tell which ones.

Describe the solution you'd like
Update the index doc with detailed config arg information.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Remote and local connections return different results for `exec_command` table

Describe the bug
When running this query on an Amazon Linux 2023 machine:

select
  *
from
  exec_command
where
  command = 'sudo auditctl -l | grep docker.service';

We get different results based on how the connection is configured (using remote vs. SSHing and using local):

SSHed in and using local connection:

+--------+-----------+---------+------+
| output | exit_code | command | _ctx |
+--------+-----------+---------+------+
+--------+-----------+---------+------+

Using remote connection:

+--------+-----------+----------------------------------------+----------------------------------+
| output | exit_code | command                                | _ctx                             |
+--------+-----------+----------------------------------------+----------------------------------+
|        | 1         | sudo auditctl -l | grep docker.service | {"connection_name":"exec_local"} |
+--------+-----------+----------------------------------------+----------------------------------+

Not sure if this issue also exists for exec_command_line too

Steampipe version (steampipe -v)
Example: v0.3.0

Plugin version (steampipe plugin list)
Example: v0.5.0

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

remote-linux-connection-through-ssh fail and defaulted to connect as root instead of ec2-user

Describe the bug
A clear and concise description of what the bug is.

https://hub.steampipe.io/plugins/turbot/exec#remote-linux-connection-through-ssh

This is my config setting in .spc

connection "exec_remote_linux" {
  plugin = "exec"
  host = "ec2-13-233-111-222.ap-south-1.compute.amazonaws.com"
  user = "ec2-user"
  private_key = "/Users/xyz/rvvv/priv-key/docker-cis.pem"
}

Even though I provide the user as ec2-user, every time it tries to connect as root

> select output from exec_remote_linux.exec_command where command = 'ls -al'
Error: timeout - last error: SSH authentication failed ([email protected]:22): ssh: handshake failed: ssh: unable to authenticate, attempted methods [none], no supported methods remain (SQLSTATE HV000)

Steampipe version (steampipe -v)
Example: v0.3.0

Plugin version (steampipe plugin list)
Example: v0.5.0

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Using exec_command crashes steampipe in some cases

Describe the bug
I am trying to use the exec_command plugin to run a dig +short against load balancer domain names to build a list of IP addresses in Postgres. When I try this query, it crashes. I'm not sure if it's the plugin crashing, or steampipe itself, but it doesn't crash with other plugins, hence starting here.

Steampipe version (steampipe -v)
v0.20.11

Plugin version (steampipe plugin list)
v0.0.3

To reproduce
Run this query:

select
    lb.dns_name,
    (select output from exec_command where command = 'dig +short ' || lb.dns_name) as ip_addresses
from
   aws_ec2_classic_load_balancer as lb

I understand this query may not be perfect, but I can't iterate on it as it crashes.

Expected behavior
I expect the query to return the load balancer DNS names along with the output from the dig command.

Additional context
I've attached the relevant log with the error.
database-2023-09-11.log

Should the `exec_command` table for remote connections always return exit codes 0 or 1?

Describe the bug
In https://github.com/turbot/steampipe-plugin-exec/blob/main/exec/utils.go#L116-L119, we only return exit codes 0 or 1. This may be because we aren't able to get the exact error code, but if that's the case, we should decide which exit codes to return and document it.

Steampipe version (steampipe -v)
Example: v0.3.0

Plugin version (steampipe plugin list)
Example: v0.5.0

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Update `exec_command` and `exec_command_line` tables to handle stdout and stderr better

Is your feature request related to a problem? Please describe.
When querying these tables, I'd like to see stdout and stderr and be able to distinguish which outputs belong to which.

In the exec_command_line table, if we don't already, we can show both stdout and stderr lines in order. A new column may be helpful as well that indicates if it's stdout or stderr.

For exec_command, instead of just output (or in addition to), maybe two new columns stdout_output and stderr_output could provide specific information.

In both tables, if we're unable to distinguish stdout vs. stderr, we should return all of stdout and stderr.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

query.docker_socket_not_mounted_inside_containers fails on linux?

query.docker_socket_not_mounted_inside_containers, used in Docker complance, runs the command shown here.

On my linux host I can do this in beta.4 with the general-improvements branch of the exec plugin.

docker ps --quiet --all | xargs docker inspect --format '{{ .Id }}: Volumes={{ .Mounts }}' | grep docker.sock

but not this:

> select * from exec_command where command = 'docker ps --quiet --all | xargs docker inspect --format ''{{ .Id }}: Volumes={{ .Mounts }}'' | grep docker.sock'
+---------------+-------------------------------------------------------+-----------+------------------------------------------------------->
| stdout_output | stderr_output                                         | exit_code | command                                               >
+---------------+-------------------------------------------------------+-----------+------------------------------------------------------->
|               |                                                       | 1         | docker ps --quiet --all | xargs docker inspect --forma>
|               | "docker inspect" requires at least 1 argument.        | 1         | docker ps --quiet --all | xargs docker inspect --forma>
|               | See 'docker inspect --help'.                          |           |                                                       >
|               |                                                       |           |                                                       >
|               | Usage:  docker inspect [OPTIONS] NAME|ID [NAME|ID...] |           |                                                       >
|               |                                                       |           |                                                       >
|               | Return low-level information on Docker objects        |           |                                                       >
|               |                                                       |           |                                                       >
+---------------+-------------------------------------------------------+-----------+-----------------------------------------------------