Trouble interpretting Jetbrains DotTrace backtraces involving Pipelines.Sockets.Unofficial via StackExchange.Redis

[mgravell]

Trouble interpretting Jetbrains DotTrace backtraces involving Pipelines.Sockets.Unofficial via StackExchange.Redis

[mgravell]

The fundamental issue here is that there is a fundamental disconnect here between requests and responses - the loop that dequeues network traffic, processes frames, and matches each response to the originating request - is not, and cannot be, directly tied into the same context that is sending requests, because the core of the lib is not "request, wait/await, response" - it heavily reuses minimal connections, sending lots of messages from separate threads down the same socket, and just keeping track of the request order.

So; if what you're asking is "how can I trace this from end to end" - I don't have a mechanism to make that work; we're not passing flow-context here - rather, we're simply signalling completion via TaskCompletionSource<T> as we identify responses.

---

Unrelated: I have some other work in the pipe to make protobuf-net work much more efficiently with pipelines, and this has already absorbed changes to make it work with the kinds of buffers that RedisValue makes available, so: there might be some other gains up ahead if you're using binary, although it looks like you're using base-64. I wonder whether there might be an alternative scheme possible there in the future where you lease an oversized array from ArrayPool, decode into that array, then pass the oversized array to protobuf (and finally release the array back to the pool)

[kieranbenton]

Thanks @mgravell - that does make sense, and at least confirms roughly what I thought, and thanks for the heads up on the protobuf support for pipelines as well. This isnt a super performant demo yet and I'm just tweaking as I go to get back into some skills (perf analysis) that I haven't used for a while.

Am I completely SOA here? Given its deeply entrenched position I'm assuming turning pipelining 'off' is a non-starter. Can I ask how you would approach finding the upstream hot path under these circumstances?

[mgravell]

did you mean SOL? yes, the codebase is written fully with that intent

[kieranbenton]

Ha, yes I meant SOL. Obviously having flashbacks today to other projects!