cloudfoundry/honeycomb-ginkgo-reporter

Honeycomb Ginkgo Reporter

Getting Started

You should be able to get set up in less than 30 minutes. If it takes you longer, please open an issue, and I'll help you out and then update this README.

To use this reporter, you'll need a Honeycomb account. You should be able to make an account on Honeycomb.io and get a dataset and a write key following their documentation.

Then, in your suite test file, import this reporter and the honeycomb go library

"github.com/cloudfoundry/honeycomb-ginkgo-reporter/honeycomb"
"github.com/cloudfoundry/honeycomb-ginkgo-reporter/honeycomb/client"
"github.com/honeycombio/libhoney-go"

create a Honeycomb client with the write key and the data set name, and pass use that client to set up.

honeyCombClient := client.New(libhoney.Config{
	WriteKey: "write-key",
	Dataset:  "my-dataset",
})
honeyCombReporter := honeycomb.New(honeyCombClient)

Then run your Ginkgo specs with the custom reporter.

rs := []Reporter{}
rs = append(rs, honeyCombReporter)

RunSpecsWithDefaultAndCustomReporters(t, "suite-name", rs)

(This will replace the default RunSpecs function.)

Usage Example

We use this reporter to track flakes in the Cloud Foundry Acceptance Tests (CATs).

Here's the suite test code from that project.

The cf-deployment Concourse task for running CATs contains an example of passing in configuration, and adds some additional fields at runtime.

Running Queries

To get started, trying breaking down by Description Calculate COUNT per group add a Filter for state = failed and a Filter for Description != '' and order by COUNT desc.

This will generate a list of tests with the most failingest at the top.

Then, try filtering by that tests's description, and break down by ComponentCodeDescription to understand exactly how that test fails.

Adding Moar Data

You'll probably want to add some tags to your events, so you can break down your test failures by independent variables, and figure out what's causing your test failures.

You may also want to filter out tests from specific runs, from specific environments, or tests that were run under specific conditions.

You can add tags in two sets: Global and Custom.

globalTags := map[string]interface{}{
	"run_id":  os.Getenv("RUN_ID"),
	"env_api": Config.GetApiEndpoint(),
}

honeyCombReporter := honeycomb.New(honeyCombClient)
honeyCombReporter.SetGlobalTags(globalTags)
honeyCombReporter.SetCustomTags(reporterConfig.CustomTags)

These are two somewhat arbitrary buckets. In CATs, "global" tags are tags that the test suite expects to be present. "Custom" tags are passed in ad-hoc.

I recommend at least setting unique run_id one or more tags describing the environment the test was run in, and as much information about the version of the code under test as you have.

honeycomb-ginkgo-reporter's Issues

Please configure GITBOT

Pivotal provides the Gitbot service to synchronize issues and pull requests made against public GitHub repos with Pivotal Tracker projects.

If you are a Pivotal employee, you can configure Gitbot to sync your GitHub repo to your Pivotal Tracker project with a pull request.

Steps:

• Fork this repo: cfgitbot-config (an ask+cf@ ticket is the fastest way to get read access if you get a 404)
• Add the Toolsmiths-Bots team to have admin access to your repo
• Add the cf-gitbot ([email protected]) user to have owner access to your Pivotal Tracker project
• Add your new repo and or project to config-production.yml file
• Submit a PR, which will get auto-merged if you've done it right

If you are not a pivotal employee, you can request that [email protected] set up the integration for you.

You might also be interested in configuring GitHub's Service Hook for Tracker on your repo so you can link your commits to Tracker stories. You can do this yourself by following the directions at:

https://www.pivotaltracker.com/blog/guide-githubs-service-hook-tracker/

If you do not want to use Pivotal Tracker to manage this GitHub repo, Please add this repo to the Ignored repositories list

If there are any questions, please reach out to [email protected].

Please add a license

This repo is a dependency of software we'd like to release, which means it needs an OSS license file. We recommend Apache 2 for CFF repos.