Giter Site home page Giter Site logo

go-datadog-api's Introduction

GoDoc License Build status Go Report Card

Datadog API in Go

This is the v2.0 version of the API, and has breaking changes. Use the v1.0 branch if you need legacy code to be supported.

A Go wrapper for the Datadog API. Use this library if you need to interact with the Datadog system. You can post metrics with it if you want, but this library is probably mostly used for automating dashboards/alerting and retrieving data (events, etc).

The source API documentation is here: http://docs.datadoghq.com/api/

Installation

To use the default branch, include it in your code like:

    import "github.com/zorkian/go-datadog-api"

Or, if you need to control which version to use, import using gopkg.in. Like so:

    import "gopkg.in/zorkian/go-datadog-api.v2"

Using go get:

go get gopkg.in/zorkian/go-datadog-api.v2

USAGE

This library uses pointers to be able to verify if values are set or not (vs the default value for the type). Like protobuf there are helpers to enhance the API. You can decide to not use them, but you'll have to be careful handling nil pointers.

Using the client:

    client := datadog.NewClient("api key", "application key")

    dash, err := client.GetDashboard(*datadog.Int(10880))
    if err != nil {
        log.Fatalf("fatal: %s\n", err)
    }
    
    log.Printf("dashboard %d: %s\n", dash.GetId(), dash.GetTitle())

An example using datadog.String(), which allocates a pointer for you:

	m := datadog.Monitor{
		Name: datadog.String("Monitor other things"),
		Creator: &datadog.Creator{
			Name: datadog.String("Joe Creator"),
		},
	}

An example using the SetXx, HasXx, GetXx and GetXxOk accessors:

	m := datadog.Monitor{}
	m.SetName("Monitor all the things")
	m.SetMessage("Electromagnetic energy loss")

	// Use HasMessage(), to verify we have interest in the message.
	// Using GetMessage() always safe as it returns the actual or, if never set, default value for that type.
	if m.HasMessage() {
		fmt.Printf("Found message %s\n", m.GetMessage())
	}

	// Alternatively, use GetMessageOk(), it returns a tuple with the (default) value and a boolean expressing
	// if it was set at all:
	if v, ok := m.GetMessageOk(); ok {
		fmt.Printf("Found message %s\n", v)
	}

Check out the Godoc link for the available API methods and, if you can't find the one you need, let us know (or patches welcome)!

DOCUMENTATION

Please see: https://godoc.org/gopkg.in/zorkian/go-datadog-api.v2

BUGS/PROBLEMS/CONTRIBUTING

There are certainly some, but presently no known major bugs. If you do find something that doesn't work as expected, please file an issue on Github:

https://github.com/zorkian/go-datadog-api/issues

Thanks in advance! And, as always, patches welcome!

DEVELOPMENT

Running tests

  • Run tests tests with make test.
  • Integration tests can be run with make testacc. Run specific integration tests with make testacc TESTARGS='-run=TestCreateAndDeleteMonitor'

The acceptance tests require DATADOG_API_KEY and DATADOG_APP_KEY to be available in your environment variables.

Warning: the integrations tests will create and remove real resources in your Datadog account.

Regenerating code

Accessors HasXx, GetXx, GetOkXx and SetXx are generated for each struct field type type that contains pointers. When structs are updated a contributor has to regenerate these using go generate and commit these changes. Optionally there is a make target for the generation:

make generate

COPYRIGHT AND LICENSE

Please see the LICENSE file for the included license information.

Copyright 2013-2019 by authors and contributors.

go-datadog-api's People

Contributors

anatolebeuzon avatar bkabrda avatar bshelton229 avatar dharada1 avatar dragon3 avatar gzussa avatar jkorzekwa-atlassian avatar knyar avatar matthodge avatar mhintz avatar mnaboka avatar mzneos avatar nmuesch avatar nyanshak avatar ojongerius avatar paulomigalmeida avatar platinummonkey avatar rfairburn avatar rlhh avatar roughbits01 avatar sergey-meshkov avatar shawnps avatar takus avatar tariq1890 avatar teraken0509 avatar tjoyal avatar travisjeffery avatar tt810 avatar yfronto avatar zorkian avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-datadog-api's Issues

json: cannot unmarshal string into Go struct field Options.evaluation_delay of type int

Unmarshalling monitors created or modified from the Datadog Web UI fails as evaluation_delay is returned by the Datadog API a string, not an int, the empty string meaning not defined/or zero (the datadog default.

The Datadog doc says (http://docs.datadoghq.com/api/?lang=console#monitors):

evaluation_delay Time (in seconds) to delay evaluation, as a non-negative integer. For example, if the value is set to 300 (5min), the timeframe is set to last_5m and the time is 7:00, the monitor will evaluate data from 6:50 to 6:55. This is useful for AWS CloudWatch and other backfilled metrics to ensure the monitor will always have data during evaluation.

When not explicitly set, the API returns an empty string for the evaluation_delay field (appears as 0 in the datadog UI):

{"tags":["*"],"deleted":null,"query":"avg(last_5m):avg:docker.cpu.usage{*} > 0","message":"Test evaluation delay int or string","id":2363835,"multi":false,"name":"Test evaluation delay int or string","created":"2017-07-07T16:45:36.099033+00:00","created_at":1499445936000,"creator":{"id":543413,"handle":"[email protected]","name":null,"email":"[email protected]"},"org_id":113899,"modified":"2017-07-07T16:45:36.099033+00:00","state":{"groups":{}},"overall_state_modified":null,"overall_state":"No Data","type":"metric alert","options":{"notify_audit":false,"locked":false,"timeout_h":0,"silenced":{}
,"include_tags":false,"no_data_timeframe":10,"require_full_window":true,"new_host_delay":300,"notify_no_data":false,"renotify_interval":0,"evaluation_delay":"","escalation_message":"","thresholds":{"critical":0.0}}}

When set, the integer value in the evaluation_delay field is double-quoted:

{"tags":["*"],"deleted":null,"query":"avg(last_5m):avg:docker.cpu.usage{*} > 0","message":"Test evaluation delay int or string","id":2363835,"multi":false,"name":"Test evaluation delay int or string","created":"2017-07-07T16:45:36.099033+00:00","created_at":1499445936000,"creator": "id":543413,"handle":"[email protected]","name":null,"email":"[email protected]"},"org_id":113899,"modified":"2017-07-07T16:46:32.335298+00:00","state":{"groups":{}},"overall_state_modified":null,"overall_state":"No Data","type":"metric alert","options":{"notify_audit":false,"locked":false,"timeout_h":0,"silenced":{}
,"include_tags":false,"no_data_timeframe":10,"require_full_window":true,"new_host_delay":300,"notify_no_data":false,"renotify_interval":0,"evaluation_delay":"666","escalation_message":"","thresholds":{"critical":0.0}}}

Unfortunately, this breaks clients using go-datadog-api such as terraform with the following error:

* module.datadog.datadog_monitor.mymonitor: datadog_monitor.mymonitor: json: cannot unmarshal string into Go struct field Options.evaluation_delay of type int

The easiest way to trigger the issue is the edit the monitor from the Datadog Web UI.
Just saving a monitor without changing anything triggers it.

Remove duplicate POST /v1/series API wrapper

https://github.com/zorkian/go-datadog-api/blob/master/metrics.go
and
https://github.com/zorkian/go-datadog-api/blob/master/series.go
both wrap the same API. One should be removed to avoid confusion.

POST errors leak credentials

Discovered via: hashicorp/terraform#11107

When a POST request comes into the DataDog API via: https://github.com/hashicorp/terraform/blob/master/builtin/providers/datadog/resource_datadog_monitor.go#L227, the go-datadog-api calls *http.Client.Do().

Due to the fact that the API Key and Application Key are provided in the URI, and the URI is included inside the returned Error; this leads to leaking credentials on the CLI where the error is returned.

cc: @keymon who reported the issue to Terraform.

Consider updating library to use pointers for optional fields

Hi there,

So far using default values has worked for us but it will not work in the case where values can either be set or not, and not setting them can influence behaviour.

An example issue issue I ran into require_full_window which is a Boolean, and optional. If the option is not set Datadog uses custom behaviour, which is desirable in some cases.

Using the current API this field defaults to false, which is different to it not being set. I suspect, but have not checked if there are other cases where we ran into this issue.

A solution is to refactor the library to use pointers, as this allows us to check if a value is actually set. The upside is more precision, the downside is that clients using the library now have to do a little more work to use pointers.

A struct like Monitor currently looks like so:

// Monitor allows watching a metric or check that you care about,
// notifying your team when some defined threshold is exceeded
type Monitor struct {
    Creator Creator  `json:"creator,omitempty"`
    Id      int      `json:"id,omitempty"`
    Type    string   `json:"type,omitempty"`
    Query   string   `json:"query,omitempty"`
    Name    string   `json:"name,omitempty"`
    Message string   `json:"message,omitempty"`
    Tags    []string `json:"tags,omitempty"`
    Options Options  `json:"options,omitempty"`
}

And would look like this using pointers for optional fields only:

// Monitor allows watching a metric or check that you care about,
// notifying your team when some defined threshold is exceeded
type Monitor struct {
    Creator *Creator  `json:"creator,omitempty"`
    Id      *int      `json:"id,omitempty"`
    Type    string   `json:"type,omitempty"`
    Query   string   `json:"query,omitempty"`
    Name    *string   `json:"name,omitempty"`
    Message *string   `json:"message,omitempty"`
    Tags    []string `json:"tags,omitempty"`
    Options *Options  `json:"options,omitempty"`
}

Or like so using pointers for all fields:

// Monitor allows watching a metric or check that you care about,
// notifying your team when some defined threshold is exceeded
type Monitor struct {
    Creator *Creator  `json:"creator,omitempty"`
    Id      *int      `json:"id,omitempty"`
    Type    *string   `json:"type,omitempty"`
    Query   *string   `json:"query,omitempty"`
    Name    *string   `json:"name,omitempty"`
    Message *string   `json:"message,omitempty"`
    Tags    []string `json:"tags,omitempty"`
    Options *Options  `json:"options,omitempty"`
}

There are different ways to go about this:

Option Pros Cons
Do nothing No work to be done in library Users have to create their own structs
Pointers for optional only Precision. Less tedious to use than making everything a pointer. Less work to implement. Inconsistent, more work to check which fields are optional
Pointers for all Precision, easier to implement More work for users of library

Will Noris wrote an insightful blogpost about this at go-rest-apis-and-pointers, which relates to his adventures in issue 19 at go-github. Go-github did make the switch to use pointers for all the things, and created helper functions to make using it a little less tedious. The result is not very pleasant to read and use, but it is precise.
There is an interesting discussion making a similar choice in issue 2330 of swagger-api.

I'm leaning toward using pointers for optional fields. @zorkian thoughts?

Cheers,

Otto

Invalid screen board JSON structure

I'm experiencing several issues with the screen boards API which was implemented in #13. From what I can tell, the screen board JSON documents that are being sent to DataDog are invalid.
IMO the tests pass because the Datadog API is more or less a simple key-value store that saves everything you throw at it. However, I can't make it render screenboards created with the new data structure.

Compare those two JSON representations of the same widget:

The one that works (generated with the 'old' screen board code):

    {
      "title_size": 16,
      "title": true,
      "title_align": "left",
      "title_text": "# Containers",
      "height": 13,
      "tile_def": {
        "viz": "timeseries",
        "requests": [
          {
            "q": "...",
            "style": {
              "palette": "blue"
            },
            "type": "bars"
          }
        ],
        "events": [
          {
            "q": "tags:..."
          }
        ],
        "markers": []
      },
      "width": 36,
      "timeframe": "1d",
      "y": 5,
      "x": 111,
      "type": "timeseries",
      "legend": false
    }

vs. the one generated with the current go-datadog-api and the new Widget data structure. This one does not work:

    {
      "alert_graph": {},
      "iframe": {},
      "graph": {
        "tile_def": {}
      },
      "image": {
        "title_size": {
          "Auto": false,
          "Size": 0
        }
      },
      "alert_value": {},
      "change": {
        "tile_def": {}
      },
      "query_value": {
        "text_size": {
          "Auto": false,
          "Size": 0
        },
        "title_size": {
          "Auto": false,
          "Size": 0
        }
      },
      "hostmap": {
        "tile_def": {}
      },
      "free_text": {},
      "timeseries": {
        "title_size": {
          "Auto": false,
          "Size": 16
        },
        "title": true,
        "title_align": "left",
        "title_text": "# Containers",
        "height": 13,
        "tile_def": {
          "viz": "timeseries",
          "requests": [
            {
              "q": "...",
              "style": {
                "palette": "blue"
              },
              "type": "bars"
            }
          ],
          "events": [
            {
              "q": "tags:..."
            }
          ]
        },
        "width": 36,
        "timeframe": "1d",
        "y": 5,
        "x": 111,
        "type": "timeseries"
      },
      "check_status": {},
      "event_stream": {
        "title_size": {
          "Auto": false,
          "Size": 0
        }
      },
      "event_timeline": {},
      "frame": {},
      "toplist": {
        "tile_def": {},
        "title_size": {
          "Auto": false,
          "Size": 0
        }
      }
    }

Screenboards that have widgets in this 'new' syntax look like this when opening them via the UI:

screenshot 2016-01-04 15 05 21

The 'old' Widget structure was modelled after Datadog's API. I'm not sure how the new one can be mapped to the actual API without custom JSON marshalling code.

Also, the constructors I initially implemented in #7 where there to set the correct widget type. Now, one needs to pass the widget type every time a new widget is created:

widget := TimeseriesWidget{
  Type: "timeseries",

  X: x,
  Y: y,
   ...
}

Am I missing something here? Maybe @ojongerius can help out here? Thanks!

Improve documentation for posting metrics

Here is my code to post metrics. Metrics explorer could not find the sent metrics.
But there is no error shown

    datadogClient := datadog.NewClient(config.API_KEY,config.APP_KEY)

    data := []datadog.DataPoint{datadog.DataPoint{1.0,15.0}}

    test:= datadog.Metric{}
    test.Metric = "order"
    test.Host = "golife"
    test.Points = data

    fmt.Println(test)

    err = datadogClient.PostMetrics([]datadog.Metric{test})
    if err != nil{
        panic(err)
    }

Label open issues

@zorkian could you please add some labels on the open issues? Would make it easier to see which issues are awaiting response, which could do with some help, what is easy etc. Happy to do it for you if you give me the power to do so.

Labels I'd like to see are "awaiting response", "contributor friendly", "bug", and "enhancement".

Dashboards integration tests fail

I suspect since 5eb9734.

go test integration/* -v  -timeout 90m
# command-line-arguments
integration/dashboards_test.go:167: cannot use "warm" (type string) as type *string in assignment
integration/dashboards_test.go:192: cannot use 99.9 (type float64) as type json.Number in field value
integration/dashboards_test.go:196: cannot use 99 (type int) as type json.Number in field value
integration/dashboards_test.go:200: cannot use 99 (type int) as type json.Number in field value
FAIL    command-line-arguments [build failed]
make: *** [testacc] Error 2

@nyanshak have time to fix this?

Tag v2.2

Could we tag v2.2. We'd like to implement some features from #106 into terraform and I think we need a new tag for gopkg.in.

Datadog did update the documentation:
DataDog/documentation#1249

Thanks!

Remove acceptance test flake

Seen in https://travis-ci.org/zorkian/go-datadog-api/jobs/418580761 (Go 1.9):

--- PASS: TestDowntimeLinkedToMonitorCreateAndDelete (0.53s)
=== RUN   TestDowntimeUpdate
--- FAIL: TestDowntimeUpdate (0.32s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x670c93]
goroutine 134 [running]:
testing.tRunner.func1(0xc4200f61e0)
	/home/travis/.gimme/versions/go1.9.linux.amd64/src/testing/testing.go:711 +0x2d2
panic(0x6d22c0, 0x8c4a70)
	/home/travis/.gimme/versions/go1.9.linux.amd64/src/runtime/panic.go:491 +0x283
command-line-arguments.cleanUpDowntime(0xc4200f61e0, 0x1616ae41)
	/home/travis/gopath/src/github.com/zorkian/go-datadog-api/integration/downtime_test.go:127 +0xe3
command-line-arguments.TestDowntimeUpdate(0xc4200f61e0)
	/home/travis/gopath/src/github.com/zorkian/go-datadog-api/integration/downtime_test.go:70 +0x28a
testing.tRunner(0xc4200f61e0, 0x746d38)
	/home/travis/.gimme/versions/go1.9.linux.amd64/src/testing/testing.go:746 +0xd0
created by testing.(*T).Run
	/home/travis/.gimme/versions/go1.9.linux.amd64/src/testing/testing.go:789 +0x2de
exit status 2
FAIL	command-line-arguments	15.287s
make: *** [testacc] Error 1

Run gofmt in pipeline

Makefile contains make fmt which does formatting but we do not enforce formatting being correct. Something like this would do:

[ -n $(gofmt -l $(find . -name '*.go' | grep -v vendor)) ] && gofmt -d -e $(find . -name '*.go' | grep -v vendor) && exit 1
diff -u ./series.go.orig ./series.go
--- ./series.go.orig	2018-10-10 09:18:24.000000000 +1300
+++ ./series.go	2018-10-10 09:18:24.000000000 +1300
@@ -30,12 +30,12 @@

 // Unit represents a unit definition that we might receive when query for timeseries data.
 type Unit struct {
-	Family string       `json:"family"`
+	Family      string  `json:"family"`
 	ScaleFactor float32 `json:"scale_factor"`
-	Name string         `json:"name"`
-	ShortName string    `json:"short_name"`
-	Plural string       `json:"plural"`
-	Id int              `json:"id"`
+	Name        string  `json:"name"`
+	ShortName   string  `json:"short_name"`
+	Plural      string  `json:"plural"`
+	Id          int     `json:"id"`
 }

Accessor generation: Correct comment for GetFieldNameOk

Noticed this while reviewing #173.

In gen-accessors.go the comment is GetOk{{.FieldName}} but should be Get{{.FieldName}}Ok.

// GetOk{{.FieldName}} returns a tuple with the {{.FieldName}} field if it's non-nil, zero value otherwise
// and a boolean to check if the value has been set.
func ({{.ReceiverVar}} *{{.ReceiverType}}) Get{{.FieldName}}Ok() ({{.FieldType}}, bool){
  if {{.ReceiverVar}} == nil || {{.ReceiverVar}}.{{.FieldName}} == nil {
    return {{.ZeroValue}}, false
  }
  return *{{.ReceiverVar}}.{{.FieldName}}, true
}

Snapshot & QueryMetrics inconsistencies

I've been trying to use both the Snapshot func (from snapshot.go) and the QueryMetrics func (from series.go) and ran into issues trying to use QueryMetrics.

Example:

query := "sum:aws.elb.request_count{*}.as_count().rollup(sum)"

// assume client is valid initialized datadog api client and start / end are valid time.Time values
_, err := client.Snapshot(query, start, end, "")
if err != nil {
    fmt.Println(err)
}

_, err = client.QueryMetrics(start.Unix(), end.Unix(), query)
if err != nil {
    fmt.Println(err)
}

The implementation of Snapshot url-encodes the query value, whereas QueryMetrics just does:
..."?query="+query...
So unless you url-encode the query before sending to QueryMetrics, you'll get a 400 back from Datadog.

Possible solutions:

  • Document this different behavior in a comment / godoc somehow
  • Change implementation of QueryMetrics to match Snapshot
  • Change implementation of Snapshot to match QueryMetrics

I'd personally prefer option 2 (change implementation of QueryMetrics to match Snapshot) as it results in a simpler interface and easier to use for clients trying to use it.

I'll add a pull request with that approach.

make updatedeps fails

I'm not sure what the updatedeps function is trying to do (too much magic/too new to golang) so I couldn't try go get-ing things manually. Here's what make updatedeps handed me:

% make updatedeps                                                                                                                                                                                                                                                             โœน
go list ./... \
        | xargs go list -f '{{join .Deps "\n"}}'
can't load package: package _/Users/blaine/Development/github.com/zorkian/go-datadog-api: cannot find package "_/Users/blaine/Development/github.com/zorkian/go-datadog-api" in any of:
    /usr/local/opt/go/libexec/src/_/Users/blaine/Development/github.com/zorkian/go-datadog-api (from $GOROOT)
    /Users/blaine/.golang/src/_/Users/blaine/Development/github.com/zorkian/go-datadog-api (from $GOPATH)
can't load package: package _/Users/blaine/Development/github.com/zorkian/go-datadog-api/integration: cannot find package "_/Users/blaine/Development/github.com/zorkian/go-datadog-api/integration" in any of:
    /usr/local/opt/go/libexec/src/_/Users/blaine/Development/github.com/zorkian/go-datadog-api/integration (from $GOROOT)
    /Users/blaine/.golang/src/_/Users/blaine/Development/github.com/zorkian/go-datadog-api/integration (from $GOPATH)
make: *** [updatedeps] Error 1

Monitor has unsupported params in options key

When I curl a monitor, below parameters are exists.

  • API Response

    ...
    "options": {
      "notify_audit": false,
      "locked": false,
      "timeout_h": 0,
      "silenced": {},
      "enable_logs_sample": true,
      "thresholds": {
        "comparison": ">",
        "critical": 1000,
        "period": {
          "seconds": 300,
          "text": "5 minutes",
          "value": "last_5m",
          "name": "5 minute average",
          "unit": "minutes"
        },
        "timeAggregator": "avg"
      },
      "queryConfig": {
        "logset": {
          "id": "7748",
          "name": "main"
        },
        "timeRange": {
          "to": 1539675566736,
          "live": true,
          "from": 1539661166736
        },
        "queryString": "env:develop",
        "queryIsFailed": false
      }
    ...
    
  • Unsupported params

    • queryConfig in options
    • period in thresholds
    • comparison in thresholds
    • timeAggregator in thresholds

But these params are not written in monitor api documents.

Failed to unmarshal json because 'no_data_timeframe' sometimes returns false

Hi,

I hit the error json: cannot unmarshal bool into Go value of type int. when I called GetMonitors(). It seems that the value of options.NoDataTimeframe becomes false.

According to the official document, no_data_timeframe supposed to be int.

no_data_timeframe the number of minutes before a monitor will notify when data stops reporting. Must be at least 2x the monitor timeframe for metric alerts or 2 minutes for service checks.

But, it becomes sometimes bool. For example, GET ALL MONITOR DETAILS of Shell section in http://docs.datadoghq.com/api/ describes it like this:

    {
        "id": 91875,
        "message": "",
        "name": "**system.net.bytes_rcvd** over **host:host0** was **> 100** on average during the **last 1h**.",
        "options": {
            "escalation_message": "",
            "no_data_timeframe": false,
            "notify_audit": true,
            "notify_no_data": false,
            "renotify_interval": null,
            "silenced": {},
            "timeout_h": null
        },
        "org_id": 1499,
        "query": "avg(last_1h):sum:system.net.bytes_rcvd{host:host0} > 100",
        "type": "metric alert",
        "multi": false,
        "created": "2015-12-18T16:34:14.014039+00:00",
        "modified": "2015-12-18T16:34:14.014039+00:00"
    }

I'm asking Datadog supports whether this is expected or not. If this is not a bug, I'll make a pull request to manage this issue.

Broken Go 1.7 and 1.8 builds

Travis CI builds fail because golint no longer supports Go 1.7 and 1.8. Technically 1.9 is no longer supported, but still works. See golint release policy.

Proposed fixes:

  • Make Go-version-specific imports / pin golint version to support 1.7 and 1.8
  • Drop 1.7 and 1.8 from builds

I propose following option 2 (drop 1.7 / 1.8). In addition, I suggest we follow the Go release policy and support only the last two major releases (1.10 and 1.11).

I'll raise a PR to change the builds based on the decision in this thread.

cc @zorkian / @yfronto

integration/screen_widgets_test.go:244: cannot use datadog.JsonNumber("*") (type *json.Number) as type *string in field value

@nyanshak would you have time to fix acceptance tests that (I suspect) were broken after merging #179? My bad for letting it slip through, the build flake ( #166 ) is not helping here ๐Ÿ˜ž If not; let me know and I'll have a look.

Example build:https://travis-ci.org/zorkian/go-datadog-api/jobs/441403022#L753:

integration/screen_widgets_test.go:244: cannot use datadog.JsonNumber("*") (type *json.Number) as type *string in field value
FAIL	command-line-arguments [build failed]
make: *** [testacc] Error 2

JSON Marshalling of User Booleans Not Working Correctly

I ran into an interesting bug/feature of json.Marshal that does not pass false values from the User struct to the Datadog API using this library because of omitempty behavior. Seems exactly related to this StackOverflow post: http://stackoverflow.com/questions/37756236/json-golang-boolean-omitempty

The User struct is currently defined as follows, which means that users cannot be re-enabled or have their admin rights revoked (which I have verified while discovering this issue):

type User struct {
        //...
	IsAdmin  bool   `json:"is_admin,omitempty"`
	Verified bool   `json:"verified,omitempty"`
	Disabled bool   `json:"disabled,omitempty"`
}

Here's example Go code to show the behavior using Boolean pointers versus not:

package main

import (
	"encoding/json"
	"fmt"
)

type SomeStruct struct {
	SomeValue bool `json:"some_value,omitempty"`
}

type SomeStructPointers struct {
	SomeValue *bool `json:"some_value,omitempty"`
}

func main() {
	tBool := true
	fBool := false

	bad1, _ := json.Marshal(SomeStruct{tBool})
	bad2, _ := json.Marshal(SomeStruct{fBool})

	fmt.Println(string(bad1))
	fmt.Println(string(bad2))

	tBoolPointer := new(bool)
	*tBoolPointer = true
	fBoolPointer := new(bool)
	*fBoolPointer = false

	good1, _ := json.Marshal(SomeStructPointers{tBoolPointer})
	good2, _ := json.Marshal(SomeStructPointers{fBoolPointer})

	fmt.Println(string(good1))
	fmt.Println(string(good2))
}

And its output:

# go run bool_pointers.go
{"some_value":true}
{}
{"some_value":true}
{"some_value":false}

I would imagine changing to Boolean pointers (*bool) would probably be a fairly breaking change. What do you think would be the best path towards resolution? I can submit a PR if you point me in the direction you would like to move forward.

Dashboard yaxis assume floats

In dashboard.go, the struct is not quite correct:

type Yaxis struct {
	Min   *float64 `json:"min,omitempty"`
	Max   *float64 `json:"max,omitempty"`
	Scale *string  `json:"scale,omitempty"`
}

When the value of min and/or max is set to "auto" from the UI, GetDashboard will fail because demarshalling fails.

Support `access_role` field for user object

Proposal: Replace is_admin with access_role.

Reasons:

There are currently 3 types of users ro (read-only), st (standard), and adm (admin). The library as-is (with is_admin field) only supports st (is_admin set to false) and adm (is_admin set to true). This means that read-only users will automatically be changed to standard users when this library is used to update a user, even if that was not the intention.

Why not keep both is_admin and access_role?

This results in confusing behavior, because it's unclear to users what the behavior is if I set is_admin to true and access_role to st. Do I get a standard user that somehow has admin access? Do I get a standard user? Do I get an admin user? The correct answer is that access_role takes precedence and is_admin will be ignored if they conflict.

| is_admin | access_role | resulting user |
| false    | ro          | ro             |
| true     | ro          | ro             |
| false    | st          | st             |
| true     | st          | st             |
| false    | adm         | adm            |
| true     | adm         | adm            |

@zorkian Let me know what you think about the proposal.

Response from metrics endpoint may contain null values

Currently the values returned from the metrics endpoint are expected to be an array of ints with the first element being the timestamp and the second element being the value, But sometime the value is "null", which go is being set to 0 by golang due to the type being int in series.go:
type DataPoint [2]float64
I believe changing the array type to a pointer*float64 should fix the issue and allow the value to be null.

Errors reported in JSON response are ignored

It seems that in some cases Datadog API can return an error in a response with HTTP code 200. In this case the JSON response seems to have the status field set to error with the specific error returned in the error field. Here's an example: https://knyar.net/paste/raw/9d6b2d1a.txt

This behavior contradicts API documentation that says "API uses HTTP status codes to indicate the success or failure of a request". I have reported this issue to Datadog (issue 155764 that is probably only visible to me), and got the following response:

There is a larger discussion ongoing right now towards reworking the API I have added your notes and recommendations to this discussion, but as there is no development planned in the short term, I am going to mark this ticket as closed for now, but your details are still attached to this project and I will reach out once it is closer to development.

So it seems like they don't have any immediate plans to fix this. I've asked them to at least update API documentation to reflect this behavior, however this has not yet been done.

Currently this library does not handle errors communicated in such way, and returns an empty successful response. I believe we need to return an error instead.

Retry logic is flawed

Currently acceptance tests take a very long time, one test takes ~90 seconds:

github.com/zorkian/go-datadog-api  v2 โœ—                                            2h27m โš‘ โ—’
โ–ถ time make testacc
go test integration/* -v  -timeout 90m
=== RUN   TestInvalidAuth
--- PASS: TestInvalidAuth (1.27s)
=== RUN   TestValidAuth
--- PASS: TestValidAuth (0.31s)
=== RUN   TestDashboardCreateAndDelete
--- PASS: TestDashboardCreateAndDelete (86.53s)
=== RUN   TestDashboardCreateAndDeleteAdvancesTimeseries
--- PASS: TestDashboardCreateAndDeleteAdvancesTimeseries (86.80s)
=== RUN   TestDashboardUpdate
--- PASS: TestDashboardUpdate (65.49s)
=== RUN   TestDashboardGet
<SNIP>

What happened: github.com/cenkalti/backoff will retry every error. This resulted in all GETs (to confirm resources have been deleted) to retry until the max amount of time had passed.

After fixing up the backoff logic the total running time is now ~90 seconds:

--- PASS: TestInvalidAuth (1.21s)
=== RUN   TestValidAuth
--- PASS: TestValidAuth (0.26s)
=== RUN   TestDashboardCreateAndDelete
--- PASS: TestDashboardCreateAndDelete (1.45s)
=== RUN   TestDashboardCreateAndDeleteAdvancesTimeseries
--- PASS: TestDashboardCreateAndDeleteAdvancesTimeseries (1.40s)
=== RUN   TestDashboardUpdate
<SNIP>
=== RUN   TestScreenboardGet
--- PASS: TestScreenboardGet (2.67s)
=== RUN   TestUserCreateAndDelete
--- PASS: TestUserCreateAndDelete (2.02s)
PASS
ok      command-line-arguments  90.757s
make testacc  1.94s user 0.34s system 2% cpu 1:31.91 total
  • Cherry pick c4db413 into master if reviewed ok

List all dashboards does not include all fields

DashboardLite structure has very limited number of fields.

I need just one additional field which is modified, but ideally i think we need all of them.
Was there a reason not all fields are exported?

I can also open a PR to address this, but i want to hear your feedback first.

Thanks

State support

Follow up from #163

When reading up on the API doco at https://docs.datadoghq.com/api/?lang=bash#monitors I noticed there are quite a few changes possible in regards to states:

Input

  • group_states: could be "all" or "alert,warn" used as a an argument to getting monitor details

Output

  • state: empty or contains an array of groups, consisting of hosts
  • overall_state: *string, example "Alert" -as seen in example of "resolve monitor" #163
  • overall_state_modified: *string: "2017-08-11T12:21:05.949182+00:00" -as seen in example of "resolve monitor"

Graphs appearance customisation

Context
When user defines graph in Datadog it provides him a handful of ways to customise output. So devs want the same functionality to be reflected in the API's.

Problem
Currently devs can only provide "q" and "satcked" arguments to request clause. But what if we want something more?

Task
Expose other arguments to the APIs.
Example of rich query:

{
  "text_align": "left",
  "autoscale": false,
  "custom_unit": "%",
  "precision": 3,
  "viz": "query_value",
  "requests": [
    {
      "q": "<some_huge_query_here>",
      "aggregator": "sum",
      "conditional_formats": [
        {
          "palette": "white_on_green",
          "comparator": ">",
          "custom_bg_color": null,
          "value": 99.99,
          "invert": false,
          "custom_fg_color": null
        },
        {
          "palette": "white_on_yellow",
          "comparator": ">=",
          "custom_bg_color": null,
          "value": 99,
          "invert": false,
          "custom_fg_color": null
        },
        {
          "palette": "white_on_red",
          "comparator": "<",
          "custom_bg_color": null,
          "value": 99,
          "invert": false,
          "custom_fg_color": null
        }
      ]
    }
  ]
}

CreateUser API

Hi guys,

The API reference for DD shows a POST api/v1/user to create a new user. This client has the invite_users call in its stead.

I assume the DD API changed over time and that's why it's not here. Any reason for me not to add it in with a PR?

Thanks.

Integration Test Fails : TestMonitorCreateAndDelete IncludeTags

ran on my local, with my datadog api & app key

also master https://travis-ci.org/zorkian/go-datadog-api/builds/490935059 fails

=== RUN   TestMonitorCreateAndDelete
--- FAIL: TestMonitorCreateAndDelete (1.26s)
	monitors_test.go:21: 
			Error Trace:	monitors_test.go:21
			Error:      	Not equal: 
			            	expected: &datadog.Monitor{Creator:(*datadog.Creator)(0xc420246720), Id:(*int)(0xc42024a828), Type:(*string)(0xc420218450), Query:(*string)(0xc420218430), Name:(*string)(0xc420218440), Message:(*string)(0xc420218420), OverallState:(*string)(0xc420218460), OverallStateModified:(*string)(nil), Tags:[]string{}, Options:(*datadog.Options)(0xc420160180), State:datadog.State{Groups:map[string]datadog.GroupData(nil)}}
			            	actual  : &datadog.Monitor{Creator:(*datadog.Creator)(0xc420246680), Id:(*int)(0xc42024a718), Type:(*string)(0xc4202185f0), Query:(*string)(0xc420218580), Name:(*string)(0xc4202185a0), Message:(*string)(0xc420218590), OverallState:(*string)(0xc4202185e0), OverallStateModified:(*string)(nil), Tags:[]string{}, Options:(*datadog.Options)(0xc420160400), State:datadog.State{Groups:map[string]datadog.GroupData(nil)}}
			            	
			            	Diff:
			            	--- Expected
			            	+++ Actual
			            	@@ -29,3 +29,3 @@
			            	   ThresholdWindows: (*datadog.ThresholdWindows)(<nil>),
			            	-  IncludeTags: (*bool)(<nil>),
			            	+  IncludeTags: (*bool)(true),
			            	   RequireFullWindow: (*bool)(true),
			Test:       	TestMonitorCreateAndDelete
	monitors_test.go:27: 
			Error Trace:	monitors_test.go:27
			Error:      	Not equal: 
			            	expected: &datadog.Monitor{Creator:(*datadog.Creator)(0xc420246720), Id:(*int)(0xc42024a828), Type:(*string)(0xc420218450), Query:(*string)(0xc420218430), Name:(*string)(0xc420218440), Message:(*string)(0xc420218420), OverallState:(*string)(0xc420218460), OverallStateModified:(*string)(nil), Tags:[]string{}, Options:(*datadog.Options)(0xc420160180), State:datadog.State{Groups:map[string]datadog.GroupData(nil)}}
			            	actual  : &datadog.Monitor{Creator:(*datadog.Creator)(0xc420246ce0), Id:(*int)(0xc420162e80), Type:(*string)(0xc420218aa0), Query:(*string)(0xc420218a30), Name:(*string)(0xc420218a50), Message:(*string)(0xc420218a40), OverallState:(*string)(0xc420218a90), OverallStateModified:(*string)(nil), Tags:[]string{}, Options:(*datadog.Options)(0xc420160600), State:datadog.State{Groups:map[string]datadog.GroupData(nil)}}
			            	
			            	Diff:
			            	--- Expected
			            	+++ Actual
			            	@@ -29,3 +29,3 @@
			            	   ThresholdWindows: (*datadog.ThresholdWindows)(<nil>),
			            	-  IncludeTags: (*bool)(<nil>),
			            	+  IncludeTags: (*bool)(true),
			            	   RequireFullWindow: (*bool)(true),
			Test:       	TestMonitorCreateAndDelete

Monitor integration test fails

TestCreateAndDeleteMonitor currently fails:

=== RUN   TestCreateAndDeleteMonitor
--- FAIL: TestCreateAndDeleteMonitor (69.00s)
        Error Trace:    monitors_test.go:21
        Error:          Not equal: &datadog.Monitor{Creator:datadog.Creator{Email:"", Handle:"", Id:0, Name:""}, Id:1352361, Type:"metric alert", Query:"avg(last_15m):avg:system.disk.in_use{*} by {host,device}
                                != &datadog.Monitor{Creator:datadog.Creator{Email:"[email protected]", Handle:"[email protected]", Id:155928, Name:""}, Id:1352361, Type:"metric alert", Query:"avg

                        Diff:
                        --- Expected
                        +++ Actual
                        @@ -1,6 +1,6 @@
                        -(*datadog.Monitor)(0xc4206617a0)({
                        +(*datadog.Monitor)(0xc420661c20)({
                          Creator: (datadog.Creator) {
                        -  Email: (string) "",
                        -  Handle: (string) "",
                        -  Id: (int) 0,
                        +  Email: (string) (len=24) "[email protected]",
                        +  Handle: (string) (len=24) "[email protected]",
                        +  Id: (int) 155928,
                           Name: (string) ""
                        @@ -12,3 +12,4 @@
                          Message: (string) (len=12) "Test message",
                        - Tags: ([]string) <nil>,
                        + Tags: ([]string) {
                        + },
                          Options: (datadog.Options) {

        Error Trace:    monitors_test.go:27
        Error:          Not equal: &datadog.Monitor{Creator:datadog.Creator{Email:"", Handle:"", Id:0, Name:""}, Id:1352361, Type:"metric alert", Query:"avg(last_15m):avg:system.disk.in_use{*} by {host,device}
                                != &datadog.Monitor{Creator:datadog.Creator{Email:"[email protected]", Handle:"[email protected]", Id:155928, Name:""}, Id:1352361, Type:"metric alert", Query:"avg

                        Diff:
                        --- Expected
                        +++ Actual
                        @@ -1,6 +1,6 @@
                        -(*datadog.Monitor)(0xc4206617a0)({
                        +(*datadog.Monitor)(0xc42063c900)({
                          Creator: (datadog.Creator) {
                        -  Email: (string) "",
                        -  Handle: (string) "",
                        -  Id: (int) 0,
                        +  Email: (string) (len=24) "[email protected]",
                        +  Handle: (string) (len=24) "[email protected]",
                        +  Id: (int) 155928,
                           Name: (string) ""
                        @@ -12,3 +12,4 @@
                          Message: (string) (len=12) "Test message",
                        - Tags: ([]string) <nil>,
                        + Tags: ([]string) {
                        + },
                          Options: (datadog.Options) {

=== RUN   TestUpdateMonitor
--- PASS: TestUpdateMonitor (78.91s)

Guessing DD API changes make this fail at the moment. The test monitor will not include information about the creator, hence the diff fails.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.