Giter Site home page Giter Site logo

aws-xray-sdk-go's Introduction

Test Go Report Card

📣 OpenTelemetry Go with AWS X-Ray

AWS X-Ray supports using OpenTelemetry Go and the AWS Distro for OpenTelemetry (ADOT) Collector to instrument your application and send trace data to X-Ray. The OpenTelemetry SDKs are an industry-wide standard for tracing instrumentation. They provide more instrumentations and have a larger community for support, but may not have complete feature parity with the X-Ray SDKs. See choosing between the ADOT and X-Ray SDKs for more help with choosing between the two.

If you want additional features when tracing your Go applications, please open an issue on the OpenTelemetry Go Instrumentation repository.

AWS X-Ray SDK for Go

Screenshot of the AWS X-Ray console

Installing into GOPATH

The AWS X-Ray SDK for Go is compatible with Go 1.19 and above.

Install the SDK using the following command (The SDK's non-testing dependencies will be installed): Use go get to retrieve the SDK to add it to your GOPATH workspace:

go get github.com/aws/aws-xray-sdk-go

To update the SDK, use go get -u to retrieve the latest version of the SDK.

go get -u github.com/aws/aws-xray-sdk-go

If you also want to install SDK's testing dependencies. They can be installed using:

go get -u -t github.com/aws/aws-xray-sdk-go/...

Installing using Go Modules

The latest version of the SDK is the recommended version.

If you are using Go 1.11 and above, you can install the SDK using Go Modules (in project's go.mod), like so:

go get github.com/aws/aws-xray-sdk-go

To get a different specific release version of the SDK use @<tag> in your go get command. Also, to get the rc version use this command with the specific version.

go get github.com/aws/[email protected]

Getting Help

Please use these community resources for getting help. We use the GitHub issues for tracking bugs and feature requests.

Opening Issues

If you encounter a bug with the AWS X-Ray SDK for Go we would like to hear about it. Search the existing issues and see if others are also experiencing the issue before opening a new issue. Please include the version of AWS X-Ray SDK for Go, AWS SDK for Go, Go language, and OS you’re using. Please also include repro case when appropriate.

The GitHub issues are intended for bug reports and feature requests. For help and questions regarding the use of the AWS X-Ray SDK for Go please make use of the resources listed in the Getting Help section. Keeping the list of open issues lean will help us respond in a timely manner.

Documentation

The developer guide provides in-depth guidance on using the AWS X-Ray service and the AWS X-Ray SDK for Go.

See aws-xray-sdk-go-sample for a sample application that provides example of tracing SQL queries, incoming and outgoing request. Follow README instructions in that repository to get started with sample application.

Quick Start

Configuration

import "github.com/aws/aws-xray-sdk-go/xray"

func init() {
  xray.Configure(xray.Config{
    DaemonAddr:       "127.0.0.1:2000", // default
    ServiceVersion:   "1.2.3",
  })
}

Logger

xray uses an interface for its logger:

type Logger interface {
  Log(level LogLevel, msg fmt.Stringer)
}

const (
  LogLevelDebug LogLevel = iota + 1
  LogLevelInfo
  LogLevelWarn
  LogLevelError
)

The default logger logs to stdout at "info" and above. To change the logger, call xray.SetLogger(myLogger). There is a default logger implementation that writes to an io.Writer from a specified minimum log level. For example, to log to stderr at "error" and above:

xray.SetLogger(xraylog.NewDefaultLogger(os.Stderr, xraylog.LogLevelError))

Note that the xray.Config{} fields LogLevel and LogFormat are deprecated starting from version 1.0.0-rc.10 and no longer have any effect.

Plugins

Plugins can be loaded conditionally at runtime. For this purpose, plugins under "github.com/aws/aws-xray-sdk-go/awsplugins/" have an explicit Init() function. Customer must call this method to load the plugin:

import (
  "os"

  "github.com/aws/aws-xray-sdk-go/awsplugins/ec2"
  "github.com/aws/aws-xray-sdk-go/xray"
)

func init() {
  // conditionally load plugin
  if os.Getenv("ENVIRONMENT") == "production" {
    ec2.Init()
  }

  xray.Configure(xray.Config{
    ServiceVersion:   "1.2.3",
  })
}

Start a custom segment/subsegment Note that customers using xray.BeginSegment API directly will only be able to evaluate sampling rules based on service name.

  // Start a segment
  ctx, seg := xray.BeginSegment(context.Background(), "service-name")
  // Start a subsegment
  subCtx, subSeg := xray.BeginSubsegment(ctx, "subsegment-name")
  // ...
  // Add metadata or annotation here if necessary
  // ...
  subSeg.Close(nil)
  // Close the segment
  seg.Close(nil)

Generate no-op trace and segment id

X-Ray Go SDK will by default generate no-op trace and segment id for unsampled requests and secure random trace and entity id for sampled requests. If customer wants to enable generating secure random trace and entity id for all the (sampled/unsampled) requests (this is applicable for trace id injection into logs use case) then they achieve that by setting AWS_XRAY_NOOP_ID environment variable as False.

Disabling XRay Tracing

XRay tracing can be disabled by setting up environment variable AWS_XRAY_SDK_DISABLED . Disabling XRay can be useful for specific use case like if customer wants to stop tracing in their test environment they can do so just by setting up the environment variable.

  // Set environment variable TRUE to disable XRay
  os.Setenv("AWS_XRAY_SDK_DISABLED", "TRUE")

Capture

func criticalSection(ctx context.Context) {
  // This example traces a critical code path using a custom subsegment
  xray.Capture(ctx, "MyService.criticalSection", func(ctx1 context.Context) error {
    var err error

    section.Lock()
    result := someLockedResource.Go()
    section.Unlock()

    xray.AddMetadata(ctx1, "ResourceResult", result)
  })
}

HTTP Handler

func main() {
  http.Handle("/", xray.Handler(xray.NewFixedSegmentNamer("myApp"), http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("Hello!"))
  })))
  http.ListenAndServe(":8000", nil)
}

HTTP Client

func getExample(ctx context.Context) ([]byte, error) {
  resp, err := ctxhttp.Get(ctx, xray.Client(nil), "https://aws.amazon.com/")
  if err != nil {
    return nil, err
  }
  return ioutil.ReadAll(resp.Body)
}

AWS SDK Instrumentation

func main() {
  // Create a segment
  ctx, root := xray.BeginSegment(context.TODO(), "AWSSDKV1_Dynamodb")
  defer root.Close(nil)

  sess := session.Must(session.NewSession())
  dynamo := dynamodb.New(sess)
  // Instrumenting with AWS SDK v1:
  // Wrap the client object with `xray.AWS()`
  xray.AWS(dynamo.Client)
  // Use the `-WithContext` version of the `ListTables` method
  output := dynamo.ListTablesWithContext(ctx, &dynamodb.ListTablesInput{})
  doSomething(output)
}

Segment creation is not necessary in an AWS Lambda function, where the segment is created automatically

AWS SDK V2 Instrumentation

package main

import (
	"context"
	"log"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/dynamodb"
	"github.com/aws/aws-xray-sdk-go/instrumentation/awsv2"
	"github.com/aws/aws-xray-sdk-go/xray"
)

func main() {
	// Create a segment
	ctx, root := xray.BeginSegment(context.TODO(), "AWSSDKV2_Dynamodb")
	defer root.Close(nil)
  
	cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion("us-west-2"))
	if err != nil {
		log.Fatalf("unable to load SDK config, %v", err)
	}
	// Instrumenting AWS SDK v2
	awsv2.AWSV2Instrumentor(&cfg.APIOptions)
	// Using the Config value, create the DynamoDB client
	svc := dynamodb.NewFromConfig(cfg)
	// Build the request with its input parameters
	_, err = svc.ListTables(ctx, &dynamodb.ListTablesInput{
		Limit: aws.Int32(5),
	})
	if err != nil {
		log.Fatalf("failed to list tables, %v", err)
	}
}

Segment creation is not necessary in an AWS Lambda function, where the segment is created automatically

S3

aws-xray-sdk-go does not currently support *Request.Presign() operations and will panic if one is encountered. This results in an error similar to:

panic: failed to begin subsegment named 's3': segment cannot be found.

If you encounter this, you can set AWS_XRAY_CONTEXT_MISSING environment variable to LOG_ERROR. This will instruct the SDK to log the error and continue processing your requests.

os.Setenv("AWS_XRAY_CONTEXT_MISSING", "LOG_ERROR")

SQL

Any database/sql calls can be traced with X-Ray by replacing the sql.Open call with xray.SQLContext. It is recommended to use URLs instead of configuration strings if possible.

func main() {
  db, err := xray.SQLContext("postgres", "postgres://user:password@host:port/db")
  row, err := db.QueryRowContext(ctx, "SELECT 1") // Use as normal
}

Lambda

For Lambda support use version v1.0.0-rc.1 and higher

If you are using the AWS X-Ray Go SDK inside a Lambda function, there will be a FacadeSegment inside the Lambda context. This allows you to instrument your Lambda function using Configure, Capture, HTTP Client, AWS, SQL and Custom Subsegments usage. Segment operations are not supported.

func HandleRequest(ctx context.Context, name string) (string, error) {
    sess := session.Must(session.NewSession())
    dynamo := dynamodb.New(sess)
    xray.AWS(dynamo.Client)
    input := &dynamodb.PutItemInput{
        Item: map[string]*dynamodb.AttributeValue{
            "12": {
                S: aws.String("example"),
            },
        },
        TableName: aws.String("xray"),
    }
    _, err := dynamo.PutItemWithContext(ctx, input)
    if err != nil {
        return name, err
    }
    
    _, err = ctxhttp.Get(ctx, xray.Client(nil), "https://www.twitch.tv/")
    if err != nil {
        return name, err
    }
    
    _, subseg := xray.BeginSubsegment(ctx, "subsegment-name")
    subseg.Close(nil)
    
    db := xray.SQLContext("postgres", "postgres://user:password@host:port/db")
    row, _ := db.QueryRow(ctx, "SELECT 1")
    
    return fmt.Sprintf("Hello %s!", name), nil
}

gRPC

Note: aws-xray-sdk-go doesn't currently support streaming gRPC call.

Apply xray gRPC interceptors (xray.UnaryServerInterceptor or xray.UnaryClientInterceptor) to instrument gRPC unary requests/responses, and the handling code.

gRPC Client

conn, err := grpc.Dial(
    serverAddr,
    // use grpc.WithChainUnaryInterceptor instead to apply multiple interceptors
    grpc.WithUnaryInterceptor(
        xray.UnaryClientInterceptor(),
        // or xray.UnaryClientInterceptor(xray.WithSegmentNamer(xray.NewFixedSegmentNamer("myApp"))) to use a custom segment namer
    ),
)

gRPC Server

grpcServer := grpc.NewServer(
    // use grpc.ChainUnaryInterceptor instead to apply multiple interceptors
    grpc.UnaryInterceptor(
        xray.UnaryServerInterceptor(),
        // or xray.UnaryServerInterceptor(xray.WithSegmentNamer(xray.NewFixedSegmentNamer("myApp"))) to use a custom segment namer
    ),
)

fasthttp instrumentation

Support for incoming requests with valyala/fasthttp:

package main

import (
	"fmt"
	"log"
	"os"

	"github.com/aws/aws-xray-sdk-go/xray"
	"github.com/aws/aws-xray-sdk-go/xraylog"
	"github.com/fasthttp/router"
	"github.com/valyala/fasthttp"
)

func index(ctx *fasthttp.RequestCtx) {
	ctx.WriteString("Welcome!")
}

func hello(ctx *fasthttp.RequestCtx) {
	fmt.Fprintf(ctx, "Hello, %s!\n", ctx.UserValue("name"))
}

func middleware(name string, h fasthttp.RequestHandler, fh xray.FastHTTPHandler) fasthttp.RequestHandler {
	f := func(ctx *fasthttp.RequestCtx) {
		h(ctx)
	}

	return fh.Handler(xray.NewFixedSegmentNamer(name), f)
}

func init() {
	if err := xray.Configure(xray.Config{
		DaemonAddr:     "xray:2000",
		ServiceVersion: "0.1",
	}); err != nil {
		panic(err)
	}

	xray.SetLogger(xraylog.NewDefaultLogger(os.Stdout, xraylog.LogLevelDebug))
}

func main() {
	fh := xray.NewFastHTTPInstrumentor(nil)
	r := router.New()
	r.GET("/", middleware("index", index, fh))
	r.GET("/hello/{name}", middleware("hello", hello, fh))

	log.Fatal(fasthttp.ListenAndServe(":8080", r.Handler))
}

Oversampling Mitigation

Oversampling mitigation allows you to ignore a parent segment/subsegment's sampled flag and instead sets the subsegment's sampled flag to false. This ensures that downstream calls are not sampled and this subsegment is not emitted.

import (
    "context"
    "fmt"
    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-lambda-go/lambda"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/sqs"
    "github.com/aws/aws-xray-sdk-go/xray"
)

func HandleRequest(ctx context.Context, event events.SQSEvent) (string, error) {
    _, subseg := xray.BeginSubsegmentWithoutSampling(ctx, "Processing Event")

    sess := session.Must(session.NewSessionWithOptions(session.Options{
        SharedConfigState: session.SharedConfigEnable,
    }))

    svc := sqs.New(sess)

    result, _ := svc.ListQueues(nil)

    for _, url := range result.QueueUrls {
        fmt.Printf("%s\n", *url)
    }

    subseg.Close(nil)

    return "Success", nil
}

func main() {
    lambda.Start(HandleRequest)
}

The code below demonstrates overriding the sampled flag based on the SQS messages sent to Lambda.

import (
    "context"
    "fmt"
    "strconv"
    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-lambda-go/lambda"
    xrayLambda "github.com/aws/aws-xray-sdk-go/lambda"
    "github.com/aws/aws-xray-sdk-go/xray"
)

func HandleRequest(ctx context.Context, event events.SQSEvent) (string, error) {

    var i = 1

    for _, message := range event.Records {
        var subseg *xray.Segment

        if xrayLambda.IsSampled(message) {
            _, subseg = xray.BeginSubsegment(ctx, "Processing Message - " + strconv.Itoa(i))
        } else {
            _, subseg = xray.BeginSubsegmentWithoutSampling(ctx, "Processing Message - " + strconv.Itoa(i))
        }

        i++;

        // Do your procesing work here
        fmt.Println("Doing processing work")

        // End your subsegment
        subseg.Close(nil)
    }

    return "Success", nil
}

func main() {
    lambda.Start(HandleRequest)
}

License

The AWS X-Ray SDK for Go is licensed under the Apache 2.0 License. See LICENSE and NOTICE.txt for more information.

aws-xray-sdk-go's People

Contributors

aircraft-cerier avatar aloababa avatar atshaw43 avatar bhautikpip avatar carolabadeer avatar chrisradek avatar dependabot[bot] avatar gliptak avatar haotianw465 avatar jamesdbowman avatar jj22ee avatar johnwu20 avatar kaorimatz avatar luluzhao avatar marcomagdy avatar mxiamxia avatar narqo avatar nathanielrn avatar nicolascb avatar nikolaianohyn avatar okonos avatar raninho avatar rengers avatar shengxil avatar shogo82148 avatar srprash avatar sthulb avatar t-yuki avatar wangzlei avatar willarmiros avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-xray-sdk-go's Issues

Support sqlx package

Hi,

Thank you for creating this.

I would like to request adding support for github.com/jmoiron/sqlx in addition to database/sql.
I'd be happy to provide the implementation if you'd be ok with it.
Looking forward for your reply.

StreamingStrategy documentation missing

I'm doing some experiments with this sdk and found out that StreamingStrategy isn't documented yet.

Hard to understand what this is for (stream subsegment I guess) and if it is supported yet (returning true for RequiresStreaming doesn't seems to stream anything).

Add benchmarks

Before I can use AWS X-Ray in my high-throughput service, I need to have some expectation of how it will affect performance. It would be great to start writing benchmarks alongside the tests to get a baseline of CPU and heap usage.

"segment cannot be found" in lambda function

I'm new to X-Ray and am trying to integrate it into a Lambda function called from API Gateway as a Lambda proxy integration request. My code follows this example:

xray.Configure(xray.Config{LogLevel: "trace"})
sess := session.Must(session.NewSession())
dynamo := dynamodb.New(sess)
xray.AWS(dynamo.Client)

This results in the following error being logged Error Suppressing AWS X-Ray context missing panic: failed to begin subsegment named 'dynamodb': segment cannot be found., which is followed by a panic runtime error: invalid memory address or nil pointer dereference in the xray SDK when the first DynamoDB operation executes.

I can "fix" this by creating a new segment with xray.BeginSegment, which puts a new service node in the graph.

I have tried this both with "active tracing" enabled and disabled in the Lambda config. When enabled, I get 2 more nodes for the Lambda service and function in the graph, but they're not connected to the new segment I create in the function as described above.

I am not sure if I'm doing something wrong (in which case the example would seem wrong too) or if this is a bug.

It also worries me that this results in a panic that propagates to my Lambda, because even if I fix this particular problem, I don't like the idea of a library that just instruments my code potentially causing a fatal error of the lambda.

Do not use init() in ec2 and ecs packages

Adding the following two imports to my application:

	_ "github.com/aws/aws-xray-sdk-go/plugins/ec2"
	_ "github.com/aws/aws-xray-sdk-go/plugins/ecs"

will make it take up to 10 seconds to timeout before it can continue loading when I'm working on my local machine and not in an EC2 or ECS instance.

I would like to argue for removing the implicit behaviour those two imports have and manually adding an idempotent Init() function on each package which will allow users to call those functions whenever needed. It might not make for a "nice" experience but I'll take adding an extra couple of lines of code any day over 10 seconds of waiting time on application boot any single day.

Lambda to lambda tracing and displayed in the service map

I have the API Gateway that calls Lamdba function 1 and that invokes lambda function 2 in Go. I want to see these 2 functions joined in the service map.

The only way i have been able to do this so far is to create a custom segment eg called "parent" and the create a subsegment from this context eg called "child". Then using client.InvokeWithContext to invoke the function 2 passing the "child" segment context.

sess := session.Must(session.NewSession()) client := lambda.New(sess, &aws.Config{Region: aws.String(region)})

xray.Configure(xray.Config{LogLevel: "trace"}) xray.AWS(client.Client)

ctx, seg := xray.BeginSegment(context.Background(), "Parent") ctx, subseg := xray.BeginSubsegment(ctx, "Child") result, _ := client.InvokeWithContext(ctx, &lambda.InvokeInput{FunctionName: aws.String(functionName), Payload: nil})

subseg.Close(nil) seg.Close(nil)

Problem is that this creates trace parent -> child in sevice map but also has function 1 too.

What is the best way to join these 2 functions on the service map please ? Note. I have more than 2 that i want to see linked up on the service map to show me my whole flow through lambdas.

Please help.

Thanks Rick

README suggestion

After some experimentation I've been able to successfully integrate this into my application, thank you!

After the "Configuration" section and before the "Capture" section in the README's Quick Start can I recommend adding a simple example of how to start a segment and subsegment for those developers who aren't working with HTTPS based services:

// Start a segment:
ctx, seg := xray.BeginSegment(context.Background(), "my-service-name")

ctx, subSeg := xray.BeginSubsegment(ctx, "Some unit of work")
// ...
// some unit of work here
// ...
subSeg.Close(nil)

// Close the segment
seg.Close(nil)

Use of custom header parsing and creating in xray/handler.go instead of using header.Header

Hi,

I am relatively new to GO, coming from Node.js. Currently I'm trying to implement xray tracing for grpc (building client and server interceptors). Therefore I had a deep look into the http handler. As I said I am new to GO so I may be completely lost but this looks fishy to me:

		trace := parseHeaders(r.Header)
		if trace["Root"] != "" {
			seg.TraceID = trace["Root"]
			seg.RequestWasTraced = true
		}
		if trace["Parent"] != "" {
			seg.ParentID = trace["Parent"]
		}
		// Don't use the segment's header here as we only want to
		// send back the root and possibly sampled values.
		var respHeader bytes.Buffer
		respHeader.WriteString("Root=")
		respHeader.WriteString(seg.TraceID)
		switch trace["Sampled"] {
		case "0":
			seg.Sampled = false
			log.Trace("Incoming header decided: Sampled=false")
		case "1":
			seg.Sampled = true
			log.Trace("Incoming header decided: Sampled=true")
		default:
			seg.Sampled = privateCfg.SamplingStrategy().ShouldTrace(r.Host, r.URL.String(), r.Method)
			log.Tracef("SamplingStrategy decided: %t", seg.Sampled)
		}
		if trace["Sampled"] == "?" {
			respHeader.WriteString(";Sampled=")
			respHeader.WriteString(strconv.Itoa(btoi(seg.Sampled)))
		}
		w.Header().Set("x-amzn-trace-id", respHeader.String())
func parseHeaders(h http.Header) map[string]string {
	m := map[string]string{}
	s := h.Get("x-amzn-trace-id")
	for _, c := range strings.Split(s, ";") {
		p := strings.SplitN(c, "=", 2)
		k := strings.TrimSpace(p[0])
		v := ""
		if len(p) > 1 {
			v = strings.TrimSpace(p[1])
		}
		m[k] = v
	}
	return m
}

Why is there a custom parsing method for the Header? There already is a header package with a header.Header that can be constructed with a string. This one is even used right at the top traceHeader := header.FromString(r.Header.Get("x-amzn-trace-id"))

Also the response header can be constructed with the Header object.

If I can help with a pr I am more than happy to do so.

Regards from Germany,
Fabian

No KMS traces

I'm not seeing KMS service calls in my traces, despite wrapping the KMS client with xray.AWS().

I noticed the SDK has a whitelist that does not include KMS. Is it up to me to add KMS, etc to a custom whitelist in my code? Would you consider adding it to the default whitelist?

Supported Go version in readme

Just a quick note. I don't think this is compatible with Go 1.7.x

I was using 1.7.4 and saw this error:

# github.com/aws/aws-xray-sdk-go/xray
../../aws/aws-xray-sdk-go/xray/httptrace.go:193: unknown httptrace.ClientTrace field 'TLSHandshakeStart' in struct literal
../../aws/aws-xray-sdk-go/xray/httptrace.go:196: unknown httptrace.ClientTrace field 'TLSHandshakeDone' in struct literal

Updated to latest stable 1.9.1 and the errors went away.

Replace glide

As the documentation mentions, using Glide for dependency management is not recommended anymore as dep is available for some time now.

I would propose moving to dep or even better to Go modules (previously known as vgo) which are available since 1.11 release.

Wrapping clients (SQS, Lambda) makes them doing nothing?

When I wrap aws service's client with xray.AWS looks like it stops functioning normally (does no real AWS API calls), nor the xray traces are reported
I however do see the xray's tracing in the log (so i of course use WithContext versions of the calls)

error on aws codestar build of aws-xray

[Container] 2018/01/28 08:13:25 Running command go get -u github.com/aws/aws-xray-sdk-go/...

github.com/aws/aws-xray-sdk-go/xray

../github.com/aws/aws-xray-sdk-go/xray/httptrace.go:199: unknown httptrace.ClientTrace field 'TLSHandshakeStart' in struct literal
../github.com/aws/aws-xray-sdk-go/xray/httptrace.go:202: unknown httptrace.ClientTrace field 'TLSHandshakeDone' in struct literal

Is there anything that has to be included to get this right.

Use of internal package not allowed

When I try to test or run my app, I get the following error:

../../../github.com/awslabs/aws-xray-sdk-go/xray/segment.go:19:2: use of internal package not allowed

This is after simply getting the package, then adding it to a single endpoint:

  http.Handle("/status/check", xray.Handler(xray.NewFixedSegmentNamer("IYFSlackBot"), http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "<p style=\"color:green;\">Connection to Integrations API Healthy")
}))))

Getting "segment cannot be found" when using xray.Client

Hi there,
I'm trying to use xray.Client instead of http.DefaultClient for calling 3rd party services. I'm getting an error such as panic: failed to begin subsegment named 'api.github.com': segment cannot be found. when doing so.
Am I doing something wrong?
Thank you in advance!

Support for RFC to golang project for sqltrace subpackage.

I believe that the httptrace API for the HTTP client is very helpful for building various tools, not just tracing related and that a sqltrace package would have a similar effect.

I've started a google-groups thread at https://groups.google.com/forum/#!topic/golang-sql/nGh4Qn5KxCA where @kardianos ran through a lot of the work for Go 1.8 and Go 1.9 database/sql changes looking for any conversation on tidbits of information that are necessary to built a useful tracing interface for sqltrace. The intention is to use httptrace as an example, and hopefully, aim for Go 1.10 implementation and inclusion, but realistically it would likely be Go 1.11.

Return Context from xray.Handler

To capture incoming requests, we use xray.Handler to wrap HTTP Handler.
However, since xray.Handler does not return inner Context, we need to create another segment using another Context to capture other external calls in the wrapped handler.
As a result, segments generated in the handler are separated from request segments, which is not useful to trace each request-to-response flow.

If xray.Handler returns Context we can create a new subsegment inside the request segment in the wrapped handler by using the Context.
Could you consider it?
Thank you.

No way to access the current Config

As far as I can see there is currently no way to access the current Config object. privateCfg is of course private and Configure(c Config) sets the private state. But there is no way to read the config. This is needed for implementing custom incoming handlers in which we need decide the sampling strategy based on samplingStrategy sampling.Strategy.

Thanks so much for your help and consideration,
Fabian

Fully suppress X-Ray SDK logs

We are using X-Ray SDK inside Lambda. SDK version - 1.0.0-rc.5
Our config is:

xray.Configure(xray.Config{
        ContextMissingStrategy: ctxmissing.NewDefaultLogErrorStrategy(),
	LogLevel:               "off",
})

But we still continue to get messages such:

2018-07-10T21:51:41+02:00 [Error] Suppressing AWS X-Ray context missing panic: failed to begin subsegment named 'unmarshal': segment cannot be found.

Is it possible to completely suppress X-Ray SDK output?

Remove package global state

Hi,

I understand this is currently just in beta. Are you open to rethinking the API? I work at twitch, and forked a similar library to remove package global state. You can read more about package global state at https://dave.cheney.net/2017/06/11/go-without-package-scoped-variables or https://peter.bourgon.org/blog/2017/06/09/theory-of-modern-go.html

I worry the library, currently designed, forces a large amount of global mutable state which could limit usage.

The library also requires an import tree that's often too deep for many Go libraries. The library I originally forked (which is very similar to this one) I modified to remove those dependencies: particularly github.com/aws/aws-sdk-go

I would strongly prefer the following changes:

  1. Remove global mutable state. This includes configuration and (it appears) the logging library used.
  2. Remove dependency on github.com/aws/aws-sdk-go from xray core. There are many Go services that don't use the AWS sdk because they don't talk to DynamoDB, etc, and it would be a shame to force them to import such a heavy package chain.

Please let me know if you're open, and if I can be of help with either of these.

Thanks!

Restore *sql.DB compliance for Go 1.8+

The method signature for xray.DB for Go 1.8 seems like it should implement the context methods the same as database/sql to make it easy to interop with other packages, currently, the context is passed to the Exec/Query/QueryRow functions and not their {*}Context versions.

Remove log during package import.

xray/default_emitter.go

func (de *DefaultEmitter) RefreshEmitterWithAddress(raddr *net.UDPAddr) {
de.Lock()
de.conn, _ = net.DialUDP("udp", nil, raddr)
log.Infof("Emitter using address: %v", raddr) . <<<<<<<
de.Unlock()
}

Can we please get rid of the log line above?
That particular log shows up as part of package import and it is really annoying, for example if you build an app to print out json payloads for another script to consume it..... that line will always add a line .. "Emitter using address". And because it is done as part of package import it is hard to disable it.

Traced http.Client should allow requests without a segment to proceed

I'm currently migrating from our own home-grown (and less featured) xray sdk to the official one, and have hit a slightly annoying limitation. We had an http.Client that was mostly used for inline requests, but also would occasionally be provided with an out of context request. Those out of context requests all failed with this error:

	if opCtx == nil {
		return nil, errors.New("opCtx must be non-nil")
	}

While it certainly simplifies the implementation, it would be much nicer if we could reliably make HTTP requests and simply have the tracing skipped when there's no segment in the context.

Tests have issues when running with -count=2

I've noticed this while trying to fix #38 with master @ a53963a5ff9b8e55fe62e8f2f14afd6e40dcb627.

When running with a single, default run, we get the following output:

florin@hostname [02:26:07 PM] [~/go/src/github.com/aws/aws-xray-sdk-go] [master]
-> % /usr/local/go1.9/bin/go test -count 1 ./...
ok      github.com/aws/aws-xray-sdk-go/header   0.003s
?       github.com/aws/aws-xray-sdk-go/internal/plugins [no test files]
ok      github.com/aws/aws-xray-sdk-go/pattern  0.005s
?       github.com/aws/aws-xray-sdk-go/plugins/beanstalk        [no test files]
?       github.com/aws/aws-xray-sdk-go/plugins/ec2      [no test files]
?       github.com/aws/aws-xray-sdk-go/plugins/ecs      [no test files]
?       github.com/aws/aws-xray-sdk-go/resources        [no test files]
ok      github.com/aws/aws-xray-sdk-go/strategy/ctxmissing      0.003s
ok      github.com/aws/aws-xray-sdk-go/strategy/exception       0.004s
ok      github.com/aws/aws-xray-sdk-go/strategy/sampling        3.007s
ok      github.com/aws/aws-xray-sdk-go/xray     0.018s

Unfortunately, when running with -count=2, we get:

florin@hostname [02:29:05 PM] [~/go/src/github.com/aws/aws-xray-sdk-go] [master]
-> % /usr/local/go1.9/bin/go test -count 2 ./...
ok      github.com/aws/aws-xray-sdk-go/header   0.008s
?       github.com/aws/aws-xray-sdk-go/internal/plugins [no test files]
ok      github.com/aws/aws-xray-sdk-go/pattern  0.010s
?       github.com/aws/aws-xray-sdk-go/plugins/beanstalk        [no test files]
?       github.com/aws/aws-xray-sdk-go/plugins/ec2      [no test files]
?       github.com/aws/aws-xray-sdk-go/plugins/ecs      [no test files]
?       github.com/aws/aws-xray-sdk-go/resources        [no test files]
ok      github.com/aws/aws-xray-sdk-go/strategy/ctxmissing      0.004s
ok      github.com/aws/aws-xray-sdk-go/strategy/exception       0.004s
ok      github.com/aws/aws-xray-sdk-go/strategy/sampling        6.012s
1517322572747777229 [Trace] Beginning segment named Test
1517322572747820004 [Trace] Beginning subsegment named TestService
1517322572747827537 [Trace] Closing segment named Test
1517322572747831841 [Trace] Closing subsegment named TestService
1517322572748293034 [Trace] Beginning segment named Test
1517322572748320807 [Trace] Beginning subsegment named TestService
1517322572748342658 [Trace] Closing subsegment named TestService
1517322572748346166 [Trace] Closing segment named Test
1517322572748533672 [Trace] Beginning segment named Test
1517322572748565291 [Trace] Beginning subsegment named ErrorService
1517322572748573150 [Trace] Closing segment named Test
1517322572748576019 [Trace] Closing subsegment named ErrorService
1517322572748841472 [Trace] Beginning segment named Test
1517322572748880384 [Trace] Beginning subsegment named PanicService
1517322572748893972 [Trace] Closing subsegment named PanicService
1517322572748931369 [Trace] Closing segment named Test
1517322572749285148 [Trace] Beginning segment named Test
1517322572749330235 [Trace] Beginning subsegment named 127.0.0.1:39155
1517322572749364038 [Trace] Beginning subsegment named connect
1517322572749484793 [Trace] Beginning subsegment named dial
1517322572749588417 [Trace] Closing subsegment named dial
1517322572749613676 [Trace] Closing subsegment named connect
1517322572749621391 [Trace] Beginning subsegment named request
1517322572749660596 [Trace] Closing subsegment named request
1517322572749666467 [Trace] Beginning subsegment named response
1517322572749829332 [Trace] Closing subsegment named response
1517322572749862329 [Trace] Closing subsegment named 127.0.0.1:39155
1517322572749864939 [Trace] Closing segment named Test
1517322572750157995 [Trace] Beginning segment named Test
1517322572750190245 [Trace] Beginning subsegment named 127.0.0.1:37699
1517322572750207836 [Trace] Beginning subsegment named connect
1517322572750292901 [Trace] Beginning subsegment named dial
1517322572750395611 [Trace] Closing subsegment named dial
1517322572750411814 [Trace] Closing subsegment named connect
1517322572750415098 [Trace] Beginning subsegment named request
1517322572750484792 [Trace] Closing subsegment named request
1517322572750491576 [Trace] Beginning subsegment named response
1517322572750598729 [Trace] Closing subsegment named response
1517322572750622551 [Trace] Closing subsegment named 127.0.0.1:37699
1517322572750625457 [Trace] Closing segment named Test
1517322572750922561 [Trace] Beginning segment named Test
1517322572750948238 [Trace] Beginning subsegment named 127.0.0.1:39569
1517322572750962547 [Trace] Beginning subsegment named connect
1517322572751018923 [Trace] Beginning subsegment named dial
1517322572751109326 [Trace] Closing subsegment named dial
1517322572751127711 [Trace] Closing subsegment named connect
1517322572751130652 [Trace] Beginning subsegment named request
1517322572751183265 [Trace] Closing subsegment named request
1517322572751186717 [Trace] Beginning subsegment named response
1517322572751261045 [Trace] Closing subsegment named response
1517322572751276570 [Trace] Closing subsegment named 127.0.0.1:39569
1517322572751279152 [Trace] Closing segment named Test
1517322572751547112 [Trace] Beginning segment named Test
1517322572751573605 [Trace] Beginning subsegment named 127.0.0.1:38477
1517322572751588549 [Trace] Beginning subsegment named connect
1517322572751645536 [Trace] Beginning subsegment named dial
1517322572751740221 [Trace] Closing subsegment named dial
1517322572751754526 [Trace] Closing subsegment named connect
1517322572751757512 [Trace] Beginning subsegment named request
1517322572751815691 [Trace] Closing subsegment named request
1517322572751820095 [Trace] Beginning subsegment named response
1517322572751908404 [Trace] Closing subsegment named response
1517322572751927632 [Trace] Closing subsegment named 127.0.0.1:38477
1517322572751930276 [Trace] Closing segment named Test
1517322572752192115 [Trace] Beginning segment named Test
1517322572752220216 [Trace] Beginning subsegment named localhost:8000
1517322572752234352 [Trace] Closing subsegment named localhost:8000
1517322572752274765 [Trace] Closing segment named Test
--- FAIL: TestDefaultConfigureParameters (0.00s)
        Error Trace:    config_test.go:113
        Error:          Not equal: 
                        expected: &sampling.LocalizedStrategy{manifest:(*sampling.RuleManifest)(0xc4201b86c0)}
                        actual  : &sampling.LocalizedStrategy{manifest:(*sampling.RuleManifest)(0xc4203a6840)}
                    
                        Diff:
                        --- Expected
                        +++ Actual
                        @@ -10,3 +10,3 @@
                            reservoir: (*sampling.Reservoir)({
                        -    maskedCounter: (uint64) 0,
                        +    maskedCounter: (uint64) 151732257200000001,
                             perSecond: (uint64) 1,
        Test:           TestDefaultConfigureParameters
        Error Trace:    config_test.go:115
        Error:          Not equal: 
                        expected: ""
                        actual  : "TestVersion"
        Test:           TestDefaultConfigureParameters
--- FAIL: TestSQL (0.00s)
    --- FAIL: TestSQL/TestPasswordConnectionString (0.00s)
        Error Trace:    sql_test.go:49
                        sql_test.go:166
        Error:          Received unexpected error:
                        cannot create a new mock database with the same dsn: user=user password=password database=database
        Test:           TestSQL/TestPasswordConnectionString
    --- FAIL: TestSQL/TestPasswordURL (0.00s)
        Error Trace:    sql_test.go:49
                        sql_test.go:96
        Error:          Received unexpected error:
                        cannot create a new mock database with the same dsn: postgres://user:password@host:port/database
        Test:           TestSQL/TestPasswordURL
    --- FAIL: TestSQL/TestPasswordURLQuery (0.00s)
        Error Trace:    sql_test.go:49
                        sql_test.go:106
        Error:          Received unexpected error:
                        cannot create a new mock database with the same dsn: postgres://host:port/database?password=password
        Test:           TestSQL/TestPasswordURLQuery
    --- FAIL: TestSQL/TestPasswordURLSchemaless (0.00s)
        Error Trace:    sql_test.go:49
                        sql_test.go:116
        Error:          Received unexpected error:
                        cannot create a new mock database with the same dsn: user:password@host:port/database
        Test:           TestSQL/TestPasswordURLSchemaless
    --- FAIL: TestSQL/TestPasswordURLSchemalessUserlessQuery (0.00s)
        Error Trace:    sql_test.go:49
                        sql_test.go:126
        Error:          Received unexpected error:
                        cannot create a new mock database with the same dsn: host:port/database?password=password
        Test:           TestSQL/TestPasswordURLSchemalessUserlessQuery
    --- FAIL: TestSQL/TestPasswordlessConnectionString (0.00s)
        Error Trace:    sql_test.go:49
                        sql_test.go:156
        Error:          Received unexpected error:
                        cannot create a new mock database with the same dsn: user=user database=database
        Test:           TestSQL/TestPasswordlessConnectionString
    --- FAIL: TestSQL/TestPasswordlessURL (0.00s)
        Error Trace:    sql_test.go:49
                        sql_test.go:86
        Error:          Received unexpected error:
                        cannot create a new mock database with the same dsn: postgres://user@host:port/database
        Test:           TestSQL/TestPasswordlessURL
    --- FAIL: TestSQL/TestSemicolonPasswordConnectionString (0.00s)
        Error Trace:    sql_test.go:49
                        sql_test.go:176
        Error:          Received unexpected error:
                        cannot create a new mock database with the same dsn: odbc:server=localhost;user id=sa;password={foo}};bar};otherthing=thing
        Test:           TestSQL/TestSemicolonPasswordConnectionString
    --- FAIL: TestSQL/TestWeirdPasswordURL (0.00s)
        Error Trace:    sql_test.go:49
                        sql_test.go:136
        Error:          Received unexpected error:
                        cannot create a new mock database with the same dsn: user%2Fpassword@host:port/database
        Test:           TestSQL/TestWeirdPasswordURL
    --- FAIL: TestSQL/TestWeirderPasswordURL (0.00s)
        Error Trace:    sql_test.go:49
                        sql_test.go:146
        Error:          Received unexpected error:
                        cannot create a new mock database with the same dsn: user/password@host:port/database
        Test:           TestSQL/TestWeirderPasswordURL
FAIL
FAIL    github.com/aws/aws-xray-sdk-go/xray     0.031s

This makes me think that the tests are not very well isolated from each-other which makes it hard to reliably test changes, like the one I have for #38. I'll try and have a look at this but unfortunately I don't have too much time to spend on this right now.

Error when installing package

In our CI we can see this error:

[Container] 2019/01/07 10:50:59 Running command go get github.com/aws/aws-xray-sdk-go/...
# github.com/aws/aws-xray-sdk-go/strategy/exception
/go/src/github.com/aws/aws-xray-sdk-go/strategy/exception/default_exception_formatting_strategy.go:148:25: cannot convert frame (type "github.com/pkg/errors".Frame) to type uintptr

Running the same command inside official Go container for versions

  • 1.11
~/go/src/github.com/cosaquee/xray-demo(master ✗) docker run -it --rm golang:1.11
Unable to find image 'golang:1.11' locally
1.11: Pulling from library/golang
cd8eada9c7bb: Already exists
c2677faec825: Already exists
fcce419a96b1: Already exists
045b51e26e75: Already exists
70a7003e9fe9: Already exists
751b6f060321: Pull complete
462a907acb44: Pull complete
Digest: sha256:4f1cd313c3d59faa0ca93910f9d853bd74be7afbf6f6997aee06924415ec7dc9
Status: Downloaded newer image for golang:1.11
root@c2fdab63256d:/go# go get -u github.com/aws/aws-xray-sdk-go/...
# github.com/aws/aws-xray-sdk-go/strategy/exception
src/github.com/aws/aws-xray-sdk-go/strategy/exception/default_exception_formatting_strategy.go:148:25: cannot convert frame (type "github.com/pkg/errors".Frame) to type uintptr
root@c2fdab63256d:/go#
  • 1.10
/go/src/github.com/cosaquee/xray-demo(master ✗) docker run -it --rm golang:1.10
Unable to find image 'golang:1.10' locally
1.10: Pulling from library/golang
cd8eada9c7bb: Already exists
c2677faec825: Pull complete
fcce419a96b1: Pull complete
045b51e26e75: Pull complete
70a7003e9fe9: Pull complete
057d7cdb2121: Pull complete
a44377be7095: Pull complete
Digest: sha256:6d006a643599f44b08b0d692102bbb35e12dbd968c68cf0d6c10da9afda9d98d
Status: Downloaded newer image for golang:1.10
root@fc8f80227176:/go# go get -u github.com/aws/aws-xray-sdk-go/...
# github.com/aws/aws-xray-sdk-go/strategy/exception
src/github.com/aws/aws-xray-sdk-go/strategy/exception/default_exception_formatting_strategy.go:148:25: cannot convert frame (type "github.com/pkg/errors".Frame) to type uintptr

We are not using Glide for dependencies management so we are running plain go get commands to download all the packages. As we can see there is an error when downloading this package.

	// We also accept github.com/pkg/errors style stack traces for ease of use
	if err, ok := err.(interface {
		StackTrace() errors.StackTrace
	}); ok {
		for _, frame := range err.StackTrace() {
			s = append(s, uintptr(frame))
		}
	}

The current master of https://github.com/pkg/errors is breaking the code above. This commit has changed type of frame from uintptr to runtime.Frame.

The master is currently broken on Travis also, but I'm not sure if it is related to this issue.

I'm not sure how to handle this situation without rebuilding aws-xray-sdk-go with the pkg/errors version https://github.com/pkg/errors/releases/tag/v0.8.1

How to enable JSON log output?

Currently I have xray configured as follows

xray.Configure(xray.Config{
	LogLevel: "trace",
	LogFormat: "{\"time\":\"%UTCDate(2006-01-02T15:04:05.999999999Z07:00)\",\"level\":\"%Level\",\"message\":\"%Msg\"}%n",
})

This works ok for single line messages

{"time":"2018-12-04T07:28:46.968723844Z","level":"Trace","message":"Beginning segment named Hello"}

but doesn't work for multiline messages or messages with quotes.

{"time":"2018-12-04T07:28:46.968974696Z","level":"Trace","message":"{
"trace_id": "1-5c062cae-cf823bbc49bef3b723f2b4d6",
"id": "8fb80775d7a85f41",
"name": "Hello",
"start_time": 1543908526.9687552,
"end_time": 1543908526.968771
}"}

Also, each line is a separate entry in cloudwatch which is quite annoying.

Is there a better way to have logs print as JSON?

Custom Subsegment options

The current subsegment option is fine if you want to wrap you function inside xray and not return data. If you do, you'll have to scope the data outside the xray wrapper. Can I suggest a simpler subsegment record be introduced that allows for a function name to passed and perhaps annotations. A use case maybe reading data from a file and unmarshalling JSON, then processing that data.

Traced aws client should allow requests to proceed without Segments present

Wherever possible we try to include XRay tracing in our common wrapper libraries so that we can gradually benefit from tracing as more and more of our services and consumers start preparing segments. This works great with the http client, however the AWS instrumentation really doesn't like being passed requests which don't have segments embedded.

Ideally the instrumentation would be sufficiently unobtrusive that it silently continued when there is no active segment.

We did investigate embedding our own segments manually elsewhere but had the same issue reported in #51

Specifically endSubsegment assumes there's a segment in flight:

func endSubsegment(r *request.Request) {
seg := GetSegment(r.HTTPRequest.Context())
seg.Close(r.Error)
r.HTTPRequest = r.HTTPRequest.WithContext(context.WithValue(r.HTTPRequest.Context(), ContextKey, seg.parent))
}

And some of the hooks don't handle it at the beginning of the requests:

var xRayBeforeValidateHandler = request.NamedHandler{
Name: "XRayBeforeValidateHandler",
Fn: func(r *request.Request) {
ctx, opseg := BeginSubsegment(r.HTTPRequest.Context(), r.ClientInfo.ServiceName)
opseg.Namespace = "aws"
marshalctx, _ := BeginSubsegment(ctx, "marshal")
r.HTTPRequest = r.HTTPRequest.WithContext(marshalctx)
r.HTTPRequest.Header.Set("x-amzn-trace-id", opseg.DownstreamHeader().String())
},
}

By contrast, other parts of the codebase assume that missing segments can happen, and check for nil when working with segments.

aws-xray-sdk-go/xray/aws.go

Lines 178 to 180 in ddbaf76

if curseg == nil {
return
}

The specific crash we get is:

 panic: runtime error: invalid memory address or nil pointer dereference
 [signal SIGSEGV: segmentation violation code=0x1 addr=0xd8 pc=0x8eb5d5]
 
 goroutine 1 [running]:
 github.com/vend/aconsumer/vendor/github.com/aws/aws-xray-sdk-go/xray.glob..func1(0xc4203b4800)
 	/home/travis/gopath/src/github.com/vend/aconsumer/vendor/github.com/aws/aws-xray-sdk-go/xray/aws.go:53 +0xb5
 github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/aws/request.(*HandlerList).Run(0xc4203b4938, 0xc4203b4800)
 	/home/travis/gopath/src/github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/aws/request/handlers.go:195 +0x9d
 github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/aws/request.(*Request).Build(0xc4203b4800, 0x4298c9, 0x8)
 	/home/travis/gopath/src/github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/aws/request/request.go:355 +0x68
 github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/aws/request.(*Request).Sign(0xc4203b4800, 0xa89bb8, 0xc4203b4800)
 	/home/travis/gopath/src/github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/aws/request/request.go:376 +0x2f
 github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/aws/request.(*Request).Send(0xc4203b4800, 0x0, 0x0)
 	/home/travis/gopath/src/github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/aws/request/request.go:483 +0x132
 github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/service/s3/s3manager.(*uploader).singlePart(0xc42065e120, 0xb06700, 0xc42038d740, 0x301, 0x0, 0x0)
 	/home/travis/gopath/src/github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/service/s3/s3manager/upload.go:572 +0x11c
 github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/service/s3/s3manager.(*uploader).upload(0xc42065e120, 0x0, 0x0, 0x0)
 	/home/travis/gopath/src/github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/service/s3/s3manager/upload.go:463 +0x5b0
 github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/service/s3/s3manager.Uploader.UploadWithContext(0x500000, 0x5, 0x0, 0x2710, 0xb10aa0, 0xc4201f80f8, 0x0, 0x0, 0x0, 0x7f7e81acc180, ...)
 	/home/travis/gopath/src/github.com/vend/aconsumer/vendor/github.com/aws/aws-sdk-go/service/s3/s3manager/upload.go:379 +0x192

Versioning with Glide causing issues

It seems that the current glide.yaml is listing all dependencies without ranges. Which, in general, is absolutely acceptable. On the other hand, it creates a problem with cyclic, infinite combining of semver for dependency in a project which will try to use latest dependency of, for example, AWS SDK for GO, currently in version 1.2.70. I can assume that there are breaking changes between them (or maybe not), but in any case, this should be resolved this way or another. IMHO on the level of this project first...

Data race when use http client

Reproduced on:

  • SDK version: V1.0.0-rc.8.
  • GO version: go1.11.1 linux/amd64

When build with - race flag, data race warning occurs upon sending http request:

WARNING: DATA RACE
Read at 0x00c000a201b0 by goroutine 56:
vendor/github.com/aws/aws-xray-sdk-go/xray.(*HTTPSubsegments).GotFirstResponseByte()
vendor/github.com/aws/aws-xray-sdk-go/xray/httptrace.go:159 +0x47
privacyapi/vendor/github.com/aws/aws-xray-sdk-go/xray.NewClientTrace.func10()
vendor/github.com/aws/aws-xray-sdk-go/xray/httptrace.go:212 +0x41
net/http.http2traceFirstResponseByte()
/usr/local/go/src/net/http/h2_bundle.go:3035 +0x7d
net/http.(*http2clientConnReadLoop).processHeaders()
/usr/local/go/src/net/http/h2_bundle.go:8360 +0x52a
net/http.(*http2clientConnReadLoop).run()
/usr/local/go/src/net/http/h2_bundle.go:8292 +0x87d
net/http.(*http2ClientConn).readLoop()
/usr/local/go/src/net/http/h2_bundle.go:8189 +0x8c

Previous write at 0x00c000a201b0 by main goroutine:
vendor/github.com/aws/aws-xray-sdk-go/xray.(*HTTPSubsegments).WroteRequest()
vendor/github.com/aws/aws-xray-sdk-go/xray/httptrace.go:152 +0x156
vendor/github.com/aws/aws-xray-sdk-go/xray.NewClientTrace.func9()
vendor/github.com/aws/aws-xray-sdk-go/xray/httptrace.go:209 +0x55
net/http.http2traceWroteRequest()

What's the official way of tracing messages in sqs

Hi,

is there an official way that we should use to trace messages that are processed asynchronously in sqs? For example:

1.) request comes in triggers a task in sqs
2.) worker service get's message from sqs and processes the message.

My simple solution would be to build some kind of sender, that wraps the sqs functionality and attaches a tracing header to every sqs message in the MessageAttributes. The MessageAttributes header could then be unpacked in the message processor.

Of course, since we do have the context in

  queue.SendMessageWithContext(ctx, &sqs.SendMessageInput{
    MessageBody: aws.String("This is a test"),
    QueueUrl: aws.String("https://sqs.eu-central-1.amazonaws.com/***/some-queue"),
  })

the sdk could add the MessageAttribute automatically? Is this something that will happen at any time in the future?

This future would also be awesome for the SNS publish action. ;)

Thanks so much for your time and consideration,
Fabian

DynamoDB table information and other meta not set

I'm seeing aws metadata in segments however the table_name and other properties aren't being set:

"aws": {
  "consistent_read": null,
  "consumed_capacity": null,
  "operation": "GetItem",
  "projection_expression": null,
  "region": "eu-west-1",
  "request_id": "MNCJ6R6HEAMTOSCET28V5FD3MRVV4KQNSO5AEMVJF66Q9ASUAAJH",
  "retries": 0,
  "table_name": null
}

I'm following the guidance in this repos' README whereby calls to DynamoDB use the WithContext variation of the GetItem method.

I'm not sure if this is a problem with this library or with the AWS Golang SDK.

Unable to add annotations within a lambda handler

Given a very simple Lambda handler function like this:

func Handler(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	if err := xray.AddAnnotation(ctx, "PLEASE_WORK", "NOPE"); err != nil {
	        log.Printf("Error returned adding annotation: %v", err)
         }
        // do some work
}

This produces a 'unable to retrieve segment' error every time. I was under the impression that there was an implied segment started for the handler, but that doesn't seem to be the case. Am I supposed to BeginSubsegment at the start of the handler method?

Handle null segment in SQL

I have my missing context strategy to be a nop (I want to ignore instances where the context does not have a valid segment). Even with the right configuration the application crashes due to a null pointer.

Something like this would solve the problem.

diff --git a/xray/sql.go b/xray/sql.go
index d463672..88954cc 100644
--- a/xray/sql.go
+++ b/xray/sql.go
@@ -150,6 +150,9 @@ func (db *DB) SetMaxOpenConns(n int) { db.db.SetMaxOpenConns(n) }
 
 func (db *DB) populate(ctx context.Context, query string) {
        seg := GetSegment(ctx)
+       if seg == nil {
+               return
+       }
 
        seg.Lock()
        seg.Namespace = "remote"
@@ -191,6 +194,9 @@ func (stmt *Stmt) populate(ctx context.Context, query string) {
        stmt.db.populate(ctx, query)
 
        seg := GetSegment(ctx)
+       if seg == nil {
+               return
+       }
        seg.Lock()
        seg.GetSQL().Preparation = "statement"
        seg.Unlock()

logging library

The logging library used in this SDK is pretty terrible. I would to use a custom logger or at least have something that isn't using inline XML configuration, its 2017.

Go build errors : Missing xraySampling* definitions

@yogiraj07 This package is currently broken:

This is the output of go build in my current project :

vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling/centralized.go:61:26: undefined: xray.SamplingStatisticsDocument
vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling/centralized.go:61:64: undefined: xray.GetSamplingTargetsOutput
vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling/centralized.go:62:25: undefined: xray.SamplingRuleRecord
vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling/centralized.go:446:47: undefined: xray.SamplingStatisticsDocument
vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling/centralized.go:477:48: undefined: xray.SamplingTargetDocument
vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling/centralized_sampling_rule_manifest.go:40:48: undefined: xray.SamplingRule
vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling/centralized_sampling_rule_manifest.go:90:55: undefined: xray.SamplingRule
vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling/centralized_sampling_rule_manifest.go:145:75: undefined: xray.SamplingRule
vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling/centralized_sampling_rule_manifest.go:172:58: undefined: xray.SamplingRule
vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling/centralized_sampling_rule_manifest.go:219:58: undefined: xray.SamplingRule
vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling/centralized_sampling_rule_manifest.go:219:58: too many errors

To add more information I tried running the tests in vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling with command go test ./... and got :

./centralized.go:61:26: undefined: xray.SamplingStatisticsDocument
./centralized.go:61:64: undefined: xray.GetSamplingTargetsOutput
./centralized.go:62:25: undefined: xray.SamplingRuleRecord
./centralized.go:446:47: undefined: xray.SamplingStatisticsDocument
./centralized.go:477:48: undefined: xray.SamplingTargetDocument
./centralized_sampling_rule_manifest.go:40:48: undefined: xray.SamplingRule
./centralized_sampling_rule_manifest.go:90:55: undefined: xray.SamplingRule
./centralized_sampling_rule_manifest.go:145:75: undefined: xray.SamplingRule
./centralized_sampling_rule_manifest.go:172:58: undefined: xray.SamplingRule
./centralized_sampling_rule_manifest.go:219:58: undefined: xray.SamplingRule
./centralized_sampling_rule_manifest.go:219:58: too many errors
FAIL	github.com/lastmilelink/courierflow-service/vendor/github.com/aws/aws-xray-sdk-go/strategy/sampling [build failed]```

SQL Usage

The SQL wrapper is really great, especially when used with a normal web app. When you are doing a job that is not based on a http request, the usage becomes a little more complex.

The readme is slightly incorrect, it should have a context object passed into the query.

func main() {
  db := xray.SQL("postgres", "postgres://user:password@host:port/db")
  row, _ := db.QueryRow(ctx, "SELECT 1") // Use as normal
}

This calls capture, which is a new subsegment, but that assumes you have an existing segment. It will panic.

I found this when doing testing, that often you wont be using the xray handler wrapper, so the context will not be created. Could we change the approach to check for the existence of a segment and create one if there isn't one present?

question: how to handle async scenarios correctly?

I am using xray for http handlers, aws services and downstream http calls. Some handlers start async processes in a go routine (e.g. sending notifications).

func createMessage(c context.Context, m *types.Message, strategy messaging.CorrelationIdStrategy) (*types.Message, error) {
	msg, err := repo.Messages.Create(c, m, strategy)
	if err != nil {
		return msg, err
	}
	m.ConversationID = msg.ConversationID
	if m.Pushed {
		go func() {			
			if err := notif.PushMessage(m); err != nil { //TODO: xray instrumentation 
				logger.Errorf("%+v", err)
			}
		}()
	}
	return msg, nil
}

I want to instrument these processes with xray. What is the correct way / pattern for this?

Add log for completion http request in logs

When the loglevel has been set to trace like so:

xray.Configure(xray.Config{LogLevel: "trace"})

One would expect information about when the request is actually sent to be added to the logs. However, the last thing that gets logged is the json representation of the trace. What I would like to see is additional information such as the ip address the xray client sdk tried to send information to and whether the response was successful or failure. Something as brief as "Connection to X.X.X.X with protocol udp/tcp was successful/unsuccessful" would be great! The usecase is to just have more debugging information available if the xray client sdk is misconfigured. Right now everything just works, but if I were to say misconfigure my firewall/security group/iptables or apply too strict of a selinux policy, it would be great to have the additional xray sdk client logs to help troubleshoot this.

Subsegments from existing traceID

I have a situation where i'm a downstream application, I want to pass the traceID from my C# application to the Go app.

Looking at the SDK, this doesn't seem possible as subsegment has parent non exported. Unless i've missed something, is there a way of generating a new subsegment from an existing traceID?

Feature request: "Facade" Segment

Hi,

In my use-case I need to create a "facade" segment to mimic an existing Segment and send subsegment (and only subsegment) to X-Ray.
There is an equivalent in the nodejs package aws-cray-sdk-core

var facadeSegment = function facadeSegment() {
  var segment = new Segment('facade');
  var whitelistFcn = ['addNewSubsegment', 'addSubsegment', 'removeSubsegment', 'toString'];
  var silentFcn = ['incrementCounter', 'decrementCounter', 'isClosed', 'close', 'format', 'flush'];
  //...
  return segment;
}

Non-HTTP use cases?

Hi,

How might I achieve non-HTTP tracing?

My example: I have a long-running service that stands up, polls SQS for work to be done, processes the work items in parallel (interacting with other AWS services), and removes the queue items/reservations. It would be valuable to see how these SQS messages interact with the rest of our system.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.