Sampling override with http.response.status_code doesn't work

[zwilling79]

Sampling override with http.response.status_code doesn't work

[jeanbisutti]

Only attributes set at the start of the span are available for sampling, so attributes such as http.response.status_code or request duration won't work for sampling.

Alternatively, you could try to use DCR. A tutorial: https://learn.microsoft.com/en-us/azure/azure-monitor/logs/tutorial-workspace-transformations-portal

[heyams]

@zwilling79 you can use OpenTelemetry Extension to filter telemetry based on http.reponse.status_code.
Here is an example how to filter out telemetry based on duration . You can do something similar.

[zwilling79]

Hm, this may work. Nonetheless, I would prefer to have this part of the configuration file so that it can be easily adjusted, especially if it is specific to certain environments. For instance, today I just want to filter out the health checks and the prometheus endpoint requests which have a response code of 200. Tomorrow I want to filter out some additional business application endpoints that have a response code of 200. To compile/package/distribute the otel extension JAR for such changes looks a bit overkill. Furthermore, if you want to use different configurations for different environments, you have to maintain different otel extension JARs or add more complexity to read/evaluate further configuration files.

I think, the problem in the code is that the values of the sampling override attributes are always treated as strings but the actual attribute is of type integer. So it is perhaps similar to #3378.

[potzkovge]

Only attributes set at the start of the span are available for sampling, so attributes such as http.response.status_code or request duration won't work for sampling.

It is very confusing which attributes are available for sampling since 3.5.0. The docs point you to the "exporting span" line but that line is basically useless as it includes the http.status_code and is not printing for example url.full which i am able to use even though it is not included in the "exporting span" line. While the next line warns you that only attributes at the start of the span are available for sampling it would be great to know which attributes are available when i set my loglevel to debug.

[zwilling79]

Have done exactly the same. Enabled the debug logging to see on which fields I can filter on. And because I saw http.response.status_code=200 in the attributes list, I thought I could filter on this.

[trask]

we are thinking to add a warning during startup if there are sampling override attributes used which are known not to be available at span start such as http.response.status_code