Max Message Size = 16mb

[kinghchan]

Max Message Size = 16mb

[JPWatson]

As it says in the Javadoc, it is a pragmatic length to encourage better design.
Large messages are a design smell. It is better to break them into chunks for transmission.

[kinghchan]

Hi @JPWatson, thanks for getting back to me.

I do understand that smaller messages are more 'pragmatic', but I was hoping for more clarification on exactly why on a hardware/OS level that using large messages is not pragmatic. My guess is because a large message would end up evicting other cached items.

For example, our L3 cache size is 64mb - does that mean we can safely increase the max message size to 32mb instead of 16mb (the current limit)?

Thanks again for your time!

[JPWatson]

Larger messages are more likely to increase the number of cache misses. You'll also have the cost of doing fragment assembly on the receiver size which involves additional copies. There is a bigger picture to consider. For instance, what are you optimising for? how would you build up the 32MB message to send in the first place?

[kinghchan]

Yes, I understand the cost of assembly.

The reason we have to consider large messages is because our source of data (from a grpc API) contains huge messages as large 60mb. The data would end up being built up in our application memory. This isn't something we can avoid, which is why we have to consider increasing the max message length of Aeron.

Yes, we can manually divide the messages into sub-messages in order to bypass Aeron's max message length when we receive the grpc message, but downstream we still have to manually join the sub-messages up. Our logic now has to consider both joining our sub-messages up as well as Aeron's own fragment assembler.

Hence my question as to whether it is safe to increase Aeron's max message length, and what the potential implications are.

Thanks again for your time.

[mjpt777]

If single messages get larger than what can practically fit in CPU cache along with other relevant data and code then performance can significantly degrade therefore we actively discourage designs that use large messages.

[plokhotnyuk]

@mjpt777 do you mean the last level cache or per core one?

[mjpt777]

@mjpt777 do you mean the last level cache or per core one?

LLC is the general idea for the limit. However things are best when significantly less than per core caches.