Adding only HttpClient instrumentation to ASP.NET Core app does not work

[alanwest]

Adding only HttpClient instrumentation to ASP.NET Core app does not work

[cijothomas]

@alanwest The root of the issue is some libraries (eg: Asp.Net Core) is creating activity irrespective of whether Otel asked is to do so. This is a "by design" behaviour for Asp.Net Core. Will try to see if Asp.Net Core has plans to disable this feature by default.

[cijothomas]

The best we can do is to document this behaviour. Tagging this as a documentation issue.

[cijothomas]

@mitoihs You can use a diff. sampler other than the default ParentBased sampler. This should give you the required solution for you.

[alanwest]

Did some investigation, the reason the HttpClient span does not show up without adding the ASP.NET Core instrumentation is because the ParentBasedSampler always makes a NotRecord decision.

1. It does have a parent (the ASP.NET Core HTTP IN activity), so parentContext.SpanId != default here
   https://github.com/open-telemetry/opentelemetry-dotnet/blob/32fb7c12f1269147bea1080f04ea4ba489aa0f63/src/OpenTelemetry/Trace/ParentBasedSampler.cs#L43-L49

2. The parent's TraceFlags do not indicate Recorded here
   https://github.com/open-telemetry/opentelemetry-dotnet/blob/32fb7c12f1269147bea1080f04ea4ba489aa0f63/src/OpenTelemetry/Trace/ParentBasedSampler.cs#L51-L55

3. So the sampling decision always falls through to here
   https://github.com/open-telemetry/opentelemetry-dotnet/blob/32fb7c12f1269147bea1080f04ea4ba489aa0f63/src/OpenTelemetry/Trace/ParentBasedSampler.cs#L69-L70

---

I guess this behavior is expected, but it does seem a bit unintuitive. That is, as a user I explicitly decide I only want HttpClient instrumentation, but that does not prevent ASP.NET Core from creating activities which will often be the root activity in a trace and influence downstream sampling decisions.

I suppose a work around would be to not use the ParentBasedSampler...

I also can see an argument that it is an unlikely scenario that someone wants to use OpenTelemetry with their ASP.NET Core and not enable ASP.NET Core instrumentation.

[cijothomas]

Same issue reported differently:
As reported in gitter channel this is an issue if you try to start own activity inside an AspNetCore application without enabling the AspNetCoreInstrumentation.

[mitoihs]

I've just hit that issue. I just need to trace my custom Activities (effectively I have AddSource in place of AddHttpClientInstrumentation of the example) and don't care about AspNetCoreInstrumentation but still, I can't turn it off or nothing is logged. I've tried to turn it on and filter everything out (.AddAspNetCoreInstrumentation(c => c.Filter = ctx => false)) but it's not working in that case also. That's sad. Is there a way to "turn on" tracing in ASP .NET Core without listening to "built-in" activities?

[CodeBlanch]

@cartersocha and I just ran into this too.

In a demo we have something like...

builder.Services.AddOpenTelemetryTracing(options =>
{
    options
        .SetResourceBuilder(resourceBuilder)
        .AddSource("MyCustomSource")  // Collect all traces from "MyCustomSource"
        //.AddAspNetCoreInstrumentation() // Not enabling AspNetCore instrumentation
        .AddConsoleExporter();
});

        public Task<SupportCase?> GetAsync(string id)
        {
            using Activity? activity = MyCustomSource.StartActivity(nameof(GetAsync), ActivityKind.Client); // <- Returns null Activity

            if (activity?.IsAllDataRequested == true)
            {
                activity.SetTag("entityId", id);
            }

            // Some logic here.
        }

The Activity from MyCustomSource is always null because the parent Activity created by AspNetCore has Recorded = false.

Just documenting this, not sure if we should/could do anything about it.

[cijothomas]

The demos could default to AlwaysOnSampler ?

[CodeBlanch]

In the actual demo AddAspNetCoreInstrumentation is added so it actually works fine. We were just messing with it to capture some screenshots and disabled AddAspNetCoreInstrumentation temporarily thinking that would cause just the custom stuff to show in the console when in reality it caused everything to disappear 😆

[martinjt]

I don't think we're going to do anything with this @alanwest @CodeBlanch ? if not, we should close this as not planned rather than keeping it open.

[cijothomas]

I don't think we're going to do anything with this @alanwest @CodeBlanch ? if not, we should close this as not planned rather than keeping it open.

We haven't done the documentation to call this out : #1779 (comment)

Maybe worth keeping it until someone helps add the documentation. (If we go native instrumentation route, then also this could be an issue?)