High latency when messages are not being send repeatedly

[plied]

High latency when messages are not being send repeatedly

[Lounarok]

Hi @plied ,
Just came across this issue.
Have you tried ZMQ_AFFINITY with zmq_ctx_set() and play with ZMQ_THREAD_SCHED_POLICY and ZMQ_THREAD_PRIORITY? There are many options there to bind a socket to specific IO thread with defined priority.

Btw, zmq_init() is deprecated by zmq_ctx_new().

[plied]

None of the above seem to make a big difference, I tried the following:

ZMQ_THREAD_SCHED_POLICY=SHED_FIFO
ZMQ_THREAD_PRIORITY=99
ZMQ_AFFINITY=1

It seems like the ZMQ_THREAD_SCHED_POLICY does improve the latency but minimally, but it still does not get anywhere close to the expected latency.

[hungpham2511]

Came across a similar issue which leads me to this issue. Could it be due to the processes are put to slept by the CPU scheduler and hence need to be re-scheduled for execution before the messages can be processed?

This can affect both the local and remote processes IMO.

One way to validate this hypothesis is to give the process high priority and use a real-time kernel.

[plied]

It could be something like that, however I did achieve super good and consistent performance using aeron on the same OS and kernel and without tweaking any scheduler configs. My hunch is that something within ZMQ itself is missconfigured but I cant find out what.