on stats, i always have an Unauthenticated error

[census-ecosystem]

on stats, i always have an Unauthenticated error

[odeke-em]

@sebglon thank you filing this issue and welcome to the OpenCensus project!

When starting your example code, did you try setting the environment variable GOOGLE_APPLICATION_CREDENTIALS i.e.
GOOGLE_APPLICATION_CREDENTIALS=<the_credentials_file.json> go run main.go?

Perhaps this codelab might be of use for you in the future https://opencensus.io/codelabs/stackdriver/

I would help also if you could paste in the "unauthenicated error" message that you get in its entirety but also a code snippet otherwise we are speculating while debugging the problem.

Thank you.

[odeke-em]

Kindly pinging you @sebglon!

[vaijab]

I ran into the same issue. After few hours of debugging I got to the same place as @sebglon.

I think the logic at https://github.com/census-ecosystem/opencensus-go-exporter-stackdriver/blob/master/stackdriver.go#L238-L251 is potentially incorrect.

Application default credentials do not have projectID set like service account json keys do, so the above code will never be able to figure out project ID. And if projectID is passed in the Options, then the code will not try to find application default credentials.

[dsansot-ru]

I also ran into this. As @vaijab says, my work around is to use a service account instead of my personal credentials.

Though, this is still not quite right, as I should be able to use a service account from ProjectA to connect to Stackdriver in ProjectB, as long as all the permissions are set up correctly.

[bmhatfield]

Chiming in here - running locally using per-user application-default-credentials, even with full permissions, did not work for stats. Tracing works fine. After creating a service account with the same credentials and exposing that as GOOGLE_APPLICATION_CREDENTIALS, stats exporting began working fine. We also experimented with GOOGLE_APPLICATION_CREDENTIALS pointing to our ~/.config/gcloud/application_default_credentials.json files; that also does not work.

While our running production environment uses per-application service accounts, engineers developing locally don't have full service accounts other than their Google Cloud account enabled in "service account mode" via gcloud auth application-default login. Every Google service except for stackdriver metrics works fine with this "user as a service account" mode.

I'd recommend considering this a bug. I see various reports of folks saying the example code doesn't work for them with an Unauthenticated error - they are likely using application-default credentials, which is broken for Stackdriver Metrics.

[ascherkus]

Based on my investigation I think one source of Unauthenticated errors for metrics is caused by a regression introduced by #58.

I was able to rule out the application default credentials authentication issue by building a stand-alone program that issues the same gRPC calls to the monitoring v3 API as the stackdriver exporter. I still need to specify the Project ID as expected.

The clue for me was the error messages:
2019/07/02 19:39:28.874369 stackdriver.go:420: Failed to export to Stackdriver: rpc error: code = Unauthenticated desc = transport: context canceled

Compare the client creation in newStatsExporter():

	ctx, cancel := o.newContextWithTimeout()
	defer cancel()
	client, err := monitoring.NewMetricClient(ctx, opts...)

... with newTraceExporter():

	ctx := o.Context
	if ctx == nil {
		ctx = context.Background()
	}
	client, err := tracingclient.NewClient(ctx, o.TraceClientOptions...)

Digging through the code I believe gRPC does the client connection in the background (i.e., is non-blocking), meaning defer cancel() will get executed once newStatsExporter() completes and the resulting client connection will now use a cancelled context.

Changing the code to use a simple context.Background() object fixes the Unauthenticated issue I was seeing and metrics now show up in Stackdriver.