palantir/pkg

A collection of stand-alone Go packages

pkg's Issues

go-metrics dependency requires dep override

In #142 we switched to a fork of go-metrics; in most cases this requires consuming projects to include a dep override of the form

[[override]]
  name = "github.com/rcrowley/go-metrics"
  source = "github.com/ashrayjain/go-metrics"
  branch = "aj/rescale-on-reads"

Ideally we would not need overrides in consuming projects

pkg/cli fails standard golint check

Implementing a cli.App.ContextConfig function causes the implementing package to fail standard golint checks:

func initContext(_ cli.Context, ctx context.Context) context.Context {
...
}

fails with:

[golint]        main.go:41:1: context.Context should be the first parameter of a function

Switch to go-metrics with decaying histograms

rcrowley/go-metrics#254

cli: --help should only show [subcommand] if visible subcommands exist

1. Define a CLI without any visible subcommands (assign the app's Command directly)
2. Run app --help

Expected

Help's "Usage" section should not reference subcommands

Actual

USAGE:
   app [global flags] command [command flags]

Reason

Right now, the CLI's Run function adds the _completion subcommand, and the template that determines whether subcommands are shown incorrectly considers hidden commands.

metrics Timer should use microseconds

Currently, the metrics.Registry interface has a Timer function that creates a timer metric. The unit for this metric is currently nanoseconds, but our tooling generally uses microseconds for durations. For consistency, the timer implementation returned by Timer should record its units in microseconds.

Note that this change will alter the semantic output of the metrics registry.

metrics registry doesn't remove metric ids on unregister

The root metric registry doesn't delete metric ids from idToMetricWithTags on Unregister()

See https://github.com/palantir/pkg/blob/master/metrics/registry.go#L255

cli: poor messaging on misconfigured duration flag

If I have a cli with a duration flag that is not required and has no default, it will always panic with the message:

panic: time: invalid duration

this happens whether or not the user provides a value on the command line. Turns out this panic arises in the default-parsing code:

pkg/cli/flag/duration_flag.go

Lines 51 to 54 in b9b4547

	func (f DurationFlag) Default() interface{} {
	dur, err := f.Parse(f.DefaultStr())
	if err != nil {
	panic(err)

We should have better messaging to ensure developers aren't confused when the panic happens.

Check for nil return from custom app.ContextConfig function

There exists a very specific NPE risk where a consumer of github.com/palantir/pkg/cli both implements a custom app.ContextConfig function, and has it return nil.

This call to context.WithCancel is not nil-safe, so we should probably wrap the call in a nil check.

https://play.golang.org/p/T0OOtNMPzO

app := cli.NewApp()
app.Name = "buggy-cli"

app.ContextConfig = func(ctx context.Context) context.Context {
	return nil
}

app.Run(os.Args) // This will NPE

add pkg/uuid package

There is no standard package for uuid, and there are multiple open source options to choose from. It is something worth considering for better code consistency.

metrics: deduplicate tags on insertion into registry

This library currently uses a slice to store tags [0]. However, Witchcraft converts the tags to a map when emitting them [1]. Even though the lossy behavior of converting the tag slice to a map is well-documented [2], it makes it difficult to debug issues involving multiple instances of the same tag key getting added to the metrics registry since the internal state diverges from what is logged. Since the Witchcraft spec defines a tag as a map<string, string> [3], let's move the deduplication of tags earlier in the pipeline such that it happens on insertion into the registry.

[0] https://github.com/palantir/pkg/blob/master/metrics/tag.go#L36
[1] https://github.com/palantir/witchcraft-go-server/blob/develop/witchcraft/server_metrics.go#L42
[2] https://github.com/palantir/pkg/blob/master/metrics/tag.go#L46-L49
[3] https://github.com/palantir/witchcraft-api/blob/master/witchcraft-logging-api/src/main/conjure/witchcraft-logging-api.yml#L369-L370

Recommend a better golang package for datetime

Carbon is a simple, semantic and developer-friendly golang package for datetime, it has been included by awesome-go

https://github.com/golang-module/carbon

retry: Default MaxBackoff is unintuitive

Issue

I would expect the following

func Test(t *testing.T) {
	err:=retry.Do(
		context.Background(),
		func() error {
			fmt.Println(time.Now())
			return errors.New("some errors")
		},
		retry.WithInitialBackoff(10*time.Second), retry.WithMultiplier(1), retry.WithMaxAttempts(5),
	)
	require.NoError(t,err)
}

To output one log line every 10 seconds. However what this actually outputs is

2019-01-16 14:51:16.025567248 -0800 PST m=+0.011095106
2019-01-16 14:51:18.092852159 -0800 PST m=+2.078377531
2019-01-16 14:51:20.357685855 -0800 PST m=+4.343208502
2019-01-16 14:51:22.459165577 -0800 PST m=+6.444685696
2019-01-16 14:51:24.425426817 -0800 PST m=+8.410944570

The reason for this output is the default for MaxBackoff is as follows:

// WithMaxBackoff sets upper limit on backoff duration.
//
// Max backoff of 0 indicates no limit.
//
// If max backoff option is not used, then default value of 2 seconds is used.

While the documentation is good, this caused us to write bugs that affected us in production as I think it's unintuitive.

Proposal

Set the default value of max backoff to 0 instead of 2 seconds, so that the max backoff will be unlimited.

I realize this is a breaking change, but I would believe that most people using this library would expect it to work this way

metrics: do not mutate context when adding tags

Currently, there are scenarios in which metrics.AddTags mutates the contents of the provided context: https://github.com/palantir/pkg/blob/master/metrics/context.go#L49

This behavior is documented, but is unintuitive/quite different from the usual pattern, which generally treats contexts as immutable and makes a copy of the entire context before updating state.

I'm generally fine with changing this, but just want to take a look at the Git history of the original change, as I believe that I chose this behavior based on requests from clients/users. However, if this is not the case (or if the clients can agree to migrate to the new behavior), then I'm all for changing this.