amnk/dd2tf

Export DataDog configuration to Terraform

A simple utility to convert DataDog dashboards and/or monitors to Terraform format.

Requires DATADOG_API_KEY and DATADOG_APP_KEY environment variables.

Useful, if you had all dashboards configured adhoc and now want to follow DevOps style :)

How to build

Just run (GOPATH and sometimes GOBIN have to be set):

dep ensure
go generate && go build

go install

Examples

Export all dashboards:

DATADOG_API_KEY=xxx DATADOG_APP_KEY=xxx ./dd2tf dashboards --all

Export one particular dashboard (where 1111 is the ID of the dashboard):

DATADOG_API_KEY=xxx DATADOG_APP_KEY=xxx ./dd2tf dashboards --ids 111

Write dashboards to corresponding files:

DATADOG_API_KEY=xxx DATADOG_APP_KEY=xxx ./dd2tf dashboards --files --all

Datadog monitor can be exported with this command:

DATADOG_API_KEY=xxx DATADOG_APP_KEY=xxx ./dd2tf monitors --ids 1706011

And Datadog Screenboard:

DATADOG_API_KEY=xxx DATADOG_APP_KEY=xxx ./dd2tf screenboards --all

You can find api/app keys in settings, under Integrations -> API section.

dd2tf's Issues

Datadog Dashboards can have non-numeric IDs, but this isn't supported

🐛

ris-aew-hdc is an example. The Datadog docs now show this type of ID as well: https://docs.datadoghq.com/api/?lang=python#get-a-dashboard

Actual Behaviour

./dd2tf dashboards --ids ris-aew-hdc
invalid argument "ris-aew-hdc" for "--ids" flag: strconv.Atoi: parsing "ris-aew-hdc": invalid syntax
Usage of dashboards:
      --all        Export all available elements.
      --debug      Enable debug output.
      --files      Save each element into a separate file.
      --ids ints   IDs of the elements to fetch.
invalid argument "ris-aew-hdc" for "--ids" flag: strconv.Atoi: parsing "ris-aew-hdc": invalid syntax

Expected Behaviour

whatever working looks like 😅

Allow using tags to select datadog resources

Users of datadog tend to organize their monitors using tags. It would be nice to have an option to use those tags to import just the ones that use that tag.

e.g.

DATADOG_API_KEY=xxx DATADOG_APP_KEY=xxx ./dd2tf monitors --tags 'team:blue'

would fetch all monitors tagged with team:blue

Missing project LICENSE

As more people are using the project, it's important to have a LICENSE.md file. Github already provides templates for most common licensing model (https://help.github.com/en/articles/licensing-a-repository).

Given that I'd love to hear from you (@amnk) your thoughts about it and find out which license model should applicable here =)

Those articles can be also useful to help to take a decision:
https://github.blog/2015-03-09-open-source-license-usage-on-github-com/ (old but good)
https://resources.whitesourcesoftware.com/blog-whitesource/top-open-source-licenses-trends-and-predictions

Allow for automatic generation of terraform import statements for generated resources

Importing a large number of resources can create a lot of manual effort for importing, as the user has to manually terraform import the resources into the statefile. If the tool were to spit out terraform import statements that the users could just run, that would be sublime. e.g.

dd2tf monitors --ids 8651460

... generated tf

Import these resources using:
terraform import datadog_monitor.dd_8651460 8651460

Conditional format is missing

Hello,

First of all, thank you for this tool! It is pretty useful for my team since we do use Terraform a lot.

I'm able to export one dashboard that contains a toplist graph with conditional format but the conditional format is missing in the generated file.

I built the binary using the commit e312181.

`-dashboards` and `-monitors` do not work according to spec

It is said that -dashboards (or -monitors) when launched without params should render all elements, while in reality they though an error complaining about missing params

Screenboards unable to export

Timeboards seemed to export fine, though without the query part of it.

The problem is when exporting screenboards I get an error:

$ DATADOG_API_KEY=XXX DATADOG_APP_KEY=XXX ./dd2tf screenboards --ids 123
FATA[0000] json: cannot unmarshal string into Go struct field Screenboard.width of type int 
Usage: ./dd2tf <subcommand> <subcommand_options>
        where <subcommand> is one of: []
Environment variables DATADOG_API_KEY and DATADOG_APP_KEY are required

keys and id changed

I have built from master and the tagged version, same result.
Is something missing? The only thing that is not completely normal is this message during the install:

$ dep ensure
Warning: the following project(s) have [[constraint]] stanzas in Gopkg.toml:

  ✗  gopkg.in/zorkian/go-datadog-api.v2

However, these projects are not direct dependencies of the current project:
they are not imported in any .go files, nor are they in the 'required' list in
Gopkg.toml. Dep only applies [[constraint]] rules to direct dependencies, so
these rules will have no effect.

Either import/require packages from these projects so that they become direct
dependencies, or convert each [[constraint]] to an [[override]] to enforce rules
on these projects, if they happen to be transitive dependencies.

Sys info:

Linux thlarsen 4.18.0-18-generic #19-Ubuntu SMP Tue Apr 2 18:13:16 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Distributor ID: Ubuntu
Description:    Ubuntu 18.10
Release:        18.10
Codename:       cosmic
go version go1.12.5 linux/amd64

Generate build artefacts

Hi,

Pretty cool the tool. Thanks for sharing. How about we add Github release/tag and also add the binaries for download?

Feature: convert screenboards

Has there been any thought to also converting screenboards? I'd tried using dashconverter.py and then exporting that, but it reformatted everything.

monitor::no_data_timeframe is not exported

attribute no_data_timeframe is not exported with the monitor template.
Terraform will now try to restore to the defaumt no_data_timeframe = 10

Terraform v12doesn`t allow naming to start with a digit

dd2f is exporting resources with the datadog id as name

resource "datadog_monitor" "123" {

}

Terraform v12 doesn`t allow a name to start with a digit.

: resource "datadog_monitor" "123" {

A name must start with a letter and may contain only letters, digits,
underscores, and dashes.

Support SLO

Description

• I want to support terraform of SLO function.
  • https://www.terraform.io/docs/providers/datadog/r/service_level_objective.html

Cannot build

I'm using the tool for the first time. After cloning the repo into $GOPATH/src/dd2tf and resolving issues with not having dep installed and running dep ensure

> brew install dep && dep ensure

I move on to the second step of generating & building, at which point I see the following response:

> go generate && go build
dashboards.go:1: running "go-bindata": exec: "go-bindata": executable file not found in $PATH

My relevant environment variables are:

export DATADOG_APP_KEY="[my_app_key]"
export DATADOG_API_KEY="[my_api_key]"
export GOPATH="${HOME?}/go"
export GOBIN="/usr/local/bin/go"
export PATH=$HOME/bin:/usr/local/bin:$GOPATH:$GOBIN:$PATH

I'm not sure where to proceed from here.

Add possibility to export DD monitors

What are the differences between this tool vs terraform import?

Hi folks,

I am trying to decide between dd2tf and this to convert all manual DataDog monitors into TF.

https://registry.terraform.io/providers/DataDog/datadog/latest/docs/resources/monitor#import

Thanking you in advance.

• Sithu

Monitor escalation message with multiple lines breaks terraform output

Since monitor escalation message supports multiple lines, it is required to use EOT in the Monitor template otherwise it creates a broken terraform. I created a PR adding it (#18).

Cannot Build

I joined Datadog a month ago, did the laptop setup script at the time, but it seems that I don't have dep:

% dep ensure
zsh: command not found: dep

Are the build instructions in the README still up-to-date?

Missing a =

Hi!

Thank you for this tool!

I have one issue and that is that when i fetch data a = is missing.
It should be: thresholds = {
Look at it below.

Have a nice day!

`resource "datadog_monitor" "38794" {
name = "test"
type = "query alert"
tags = ["team:xxx","env:xxxx","service:xxxxx",]
message = <<EOT
@slack

This alarm means that.....
EOT
escalation_message = <<EOT

EOT

query = "sum(last_5m):avg:aws.xxxxxx.incoming_records{name:xxxxxxxxxx-metricstream}.as_count() > 40000"
notify_no_data = false
renotify_interval = 0
notify_audit = false
timeout_h = 0
include_tags = true
require_full_window = true
new_host_delay = 300
thresholds {
critical = 70000.0
}

}
`