Transactions started with `continue_trace()` do not inherit sampling

[jwhitaker-gridcog]

Transactions started with `continue_trace()` do not inherit sampling

[szokeasaurusrex]

@jwhitaker-gridcog Thank you for reporting, I am currently investigating this issue

[szokeasaurusrex]

Thanks for looking into this! Can I ask what is the use case for calling continue_trace and not wanting the transaction to be sent?

@jwhitaker-gridcog There is likely no use case for doing this.

But essentially, as I understand it, the continue_trace function's purpose is not, and never was, to start a transaction. Its purpose is only to create a child transaction object, which inherits certain information (like the sampling decision) from the parent transaction. To then start this transaction, the transaction must be passed to the start_transaction function, which handles the logic relevant to starting a transaction, like making sure the transaction is recorded on the correct hub/scope.

Essentially, the two functions can be summarized as follows:

• continue_trace creates a transaction, but does not start it
• start_transaction starts the the transaction that is passed to it
  • When no transaction object is passed to start_transaction, the function automatically creates a new transaction object and starts it. In other words, start_transaction(...) is essentially just syntactic sugar for start_transaction(Transaction(...)). Perhaps, this auto-transaction-generating feature is what is causing the confusion here?

[szokeasaurusrex]

Hey @jwhitaker-gridcog! After some investigation, it looks like the problem you have run into is different and more general than transactions started with continue_trace not inheriting the sampling decision. If the problem was only that the sampling decision was not inherited, we would still expect the child transaction to be sent because, within the worker, you initialized the Sentry SDK with traces_sample_rate=1.0.

In fact, the root cause of the issue is that transactions must always be started with the start_transaction function in order to be sent to Sentry. Transactions created via continue_trace (or any other method, such as calling the Transaction constructor) can only be sent to sentry if started via start_transaction. This is because start_transaction initializes the transaction's span recorder, which is needed in order to send the transaction to Sentry.

In your case, this means you can work around the issue by modifying your worker function as follows:

def worker(baggage_and_tp):
    sentry_sdk.init(traces_sample_rate=1.0, environment="jarrad-local", debug=True)
    baggage, traceparent = baggage_and_tp
    try:
        transaction = sentry_sdk.continue_trace(
            {"baggage": baggage, "sentry-trace": traceparent}, op="task", name="child"
        )

        # The transaction is created with continue_trace, but we start it with start_transaction
        with sentry_sdk.start_transaction(transaction):
            sentry_sdk.capture_message("hi")
    finally:
        sentry_sdk.Hub.current.flush()

I will need to discuss with the team to determine whether requiring transactions to be started via start_transaction is the desired behavior. If yes, we will need to update the docs to make this clear, and provide a better debug message than that we are discarding the transaction because sampled=False. Otherwise, we should change the behavior so that the transaction created by continue_trace can be started directly, without passing it to start_transaction

Update: I discussed with the team, and it is indeed the expected behavior that transactions only get sent to Sentry when started with start_transaction. So, you will need to use the workaround I suggested above. We will, however, document this behavior better, especially by improving the debug message when we drop a transaction.

[szokeasaurusrex]

Since we have determined that the behavior described here is as expected, I am closing this issue. I have opened the two issues below to improve the debug message and our docs, respectively:

• #3000
• getsentry/sentry-docs#9771

Hopefully, these changes will help us to avoid confusion in the future!

[jwhitaker-gridcog]

Thanks for looking into this! Can I ask what is the use case for calling continue_trace and not wanting the transaction to be sent?

[jwhitaker-gridcog]

thanks for following this up and detailed work on explanation and docs!