turbot/steampipe-plugin-datadog

Use SQL to instantly query Datadog resources across accounts. Open source CLI. No DB required.

Datadog Plugin for Steampipe

Use SQL to query dashboards, users, roles and more from Datadog.

• Get started →
• Documentation: Table definitions & examples
• Community: Join #steampipe on Slack →
• Get involved: Issues

Quick start

Install the plugin with Steampipe:

steampipe plugin install datadog

Run a query:

select name, type, overall_state from datadog_monitor where overall_state in ('Alert', 'Warn');

Engines

This plugin is available for the following engines:

Engine	Description
Steampipe	The Steampipe CLI exposes APIs and services as a high-performance relational database, giving you the ability to write SQL-based queries to explore dynamic data. Mods extend Steampipe's capabilities with dashboards, reports, and controls built with simple HCL. The Steampipe CLI is a turnkey solution that includes its own Postgres database, plugin management, and mod support.
Postgres FDW	Steampipe Postgres FDWs are native Postgres Foreign Data Wrappers that translate APIs to foreign tables. Unlike Steampipe CLI, which ships with its own Postgres server instance, the Steampipe Postgres FDWs can be installed in any supported Postgres database version.
SQLite Extension	Steampipe SQLite Extensions provide SQLite virtual tables that translate your queries into API calls, transparently fetching information from your API or service as you request it.
Export	Steampipe Plugin Exporters provide a flexible mechanism for exporting information from cloud services and APIs. Each exporter is a stand-alone binary that allows you to extract data using Steampipe plugins without a database.
Turbot Pipes	Turbot Pipes is the only intelligence, automation & security platform built specifically for DevOps. Pipes provide hosted Steampipe database instances, shared dashboards, snapshots, and more.

Developing

Prerequisites:

• Steampipe
• Golang

Clone:

git clone https://github.com/turbot/steampipe-plugin-datadog.git
cd steampipe-plugin-datadog

Build, which automatically installs the new version to your ~/.steampipe/plugins directory:

make

Configure the plugin:

cp config/* ~/.steampipe/config
vi ~/.steampipe/config/datadog.spc

Try it!

steampipe query
> .inspect datadog

Further reading:

• Writing plugins
• Writing your first table

Open Source & Contributing

This repository is published under the Apache 2.0 (source code) and CC BY-NC-ND (docs) licenses. Please see our code of conduct. We look forward to collaborating with you!

Steampipe is a product produced from this open source software, exclusively by Turbot HQ, Inc. It is distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our Open Source FAQ.

Get Involved

Join #steampipe on Slack →

Want to help but don't know where to start? Pick up one of the help wanted issues:

• Steampipe
• Datadog Plugin

steampipe-plugin-datadog's Issues

Add table datadog_<custom metric resource> using dynamic table creation

References
Add table to get details of metrics in realtime

https://github.com/DataDog/datadog-api-client-go/blob/master/api/v2/datadog/docs/MetricsApi.md#metricsapi
https://github.com/DataDog/datadog-api-client-go/blob/master/api/v1/datadog/docs/MetricsApi.md#metricsapi

Add table datadog_host

Hi there,
First thank you for this nice plugin!

Here a suggestion to add a table: to get the list of hosts (infrastructure_hosts)

Have a nice day,

References
https://docs.datadoghq.com/developers/guide/query-the-infrastructure-list-via-the-api/

Add table datadog_metrics data

References
Add table to get details of metrics in realtime

https://github.com/DataDog/datadog-api-client-go/blob/master/api/v2/datadog/docs/MetricsApi.md#metricsapi
https://github.com/DataDog/datadog-api-client-go/blob/master/api/v1/datadog/docs/MetricsApi.md#metricsapi

Paginate queries to https://api.datadoghq.com/api/v1/monitor

Because datadog_monitor does not perform paginated requests when calling the Datadog API /api/v1/monitor endpoint, the request can often fail for organizations with a large volume of monitors.

For example...

Using following query.sql:

SELECT
  name,
  'https://app.datadoghq.com/monitors/' || id AS URL
FROM
  datadog_monitor
WHERE
  query
LIKE
  '%ecs.fargate.cpu.limit%'

...results in an HTTP 504 from the Datadog API when the targeted organization has a large volume of monitors, which renders datadog_monitor unusable:

steampipe query query.sql

Error: datadog: 504 Gateway Time-out (SQLSTATE HV000)

+------+-----+
| name | url |
+------+-----+
+------+-----+

Describe the solution you'd like

I believe the HTTP 504 could be avoided if the steampipe-plugin-datadog issued the request(s) with a page_size query parameter, and fetched the monitors in smaller batches via distinct, per-page HTTP requests.

Some queries using `timestamp` or `filter_query` qual never complete

Describe the bug
When using these quals, some queries never finish:

> select
  id,
  title,
  timestamp,
  jsonb_pretty(attributes) as attributes
from
  datadog_security_monitoring_signal
where
  filter_query = 'status:(critical OR high OR medium)';
Error: execution cancelled

Time: 42.7s.
> select
  id,
  title,
  timestamp,
  message,
  jsonb_pretty(attributes) as attributes
from
  datadog_security_monitoring_signal
where
  timestamp >= (current_date - interval '5' day);
Error: execution cancelled

Time: 30.1s.
> select
  id,
  title,
  timestamp,
  message,
  jsonb_pretty(attributes) as attributes
from
  datadog_security_monitoring_signal
where
  timestamp >= (current_date - interval '5' day);
Error: execution cancelled

Time: 2,132.4s.

Steampipe version (steampipe -v)
v0.19.3

Plugin version (steampipe plugin list)
v0.5.0

To reproduce
See above

Expected behavior
Queries should complete by returning results or erroring appropriately

Additional context
Not sure if all tables have this issue or just the one above

Plugin not working with `api_url = "https://api.datadoghq.eu/"`

Describe the bug

> select name, id from datadog_eu.datadog_user
Error: rpc error: code = Internal desc = hydrate function listUsers failed with panic runtime error: invalid memory address or nil pointer dereference
> select name, id from datadog.datadog_user

Time: 7.954036ms
+-----------------------+--------------------------------------+
| name                  | id                                   |
+-----------------------+--------------------------------------+
| <null>                | 130f3218-38b8-11ec-9b20-da7ad0900002 |
| <null>                | 252dcb01-30e1-11ec-9255-da7ad0900002 |
| turbot-router-events  | f0717bea-30b4-405a-9c8f-34f0b3d28804 |
| Lalit Bhardwaj        | 45c9984e-30a0-11ec-9224-da7ad0900002 |
| Subhajit Kumar Mondal | 0f550528-30e1-11ec-9255-da7ad0900002 |
+-----------------------+--------------------------------------+

Steampipe version (steampipe -v)
Example: v0.9.0

Plugin version (steampipe plugin list)
Example: v0.0.1

To reproduce
setup connection config

connection "datadog_eu" {
  plugin  = "datadog"
  api_key = "reacted"
  app_key = "reacted"
  api_url = "https://api.datadoghq.eu/"
}

Add table datadog_user