ArduinoJson 7 reserve memory pool

[Laxilef]

ArduinoJson 7 reserve memory pool

[bblanchon]

Hi @Laxilef,

ArduinoJson 7 has a very different memory management strategy which doesn't allow specifying the capacity anymore. I'm honestly quite surprised that someone asked for this after I received so many complaints about the fixed memory allocation of ArduinoJson 6.

JsonDocument allocates large blocks and only frees on destruction and reallocates in shrinkToFit(), so it shouldn't create heap fragmentation, at least in regular scenarios. The only place I expect to see a little fragmentation is in deserializeJson() because it needs to resize buffers for strings.
But what makes you think ArduinoJson 7 increases the heap fragmentation in your program?

Since it has to over-allocate, ArduinoJson 7 may have a peak memory consumption superior to version 6. You can check this with the ArduinoJson Assistant.

Best regards,
Benoit

[Laxilef]

The dynamic memory allocation feature is very convenient! I'm not saying that heap fragmentation is caused by the library :) I'm trying to find the reason.
If I understand correctly, the clear() method clears the JsonDocument but doesn't touch the memory pool. And shrinkToFit() reduces the pool strictly to the necessary size.
Is the memory pool preserved when destroying a JsonDocument?

I'm wondering whether the memory pool is being reused. For example, I have about 25 methods, each of which creates a JsonDocument ranging in size from 500 to 2000 bytes, at the end of each method the data is serialized and sent to mqtt. And these methods are called one after another. But I'm not using a static JsonDocument. Each call to one of the methods creates a new JsonDocument.

[bblanchon]

JsonDocument's destructor and clear() both release all the allocated heap memory.
Given all the tests I've run, I'd be very surprised if there were a memory leak in ArduinoJson 7, but we cannot exclude this possibility.
What hardware is this running on?
Can you print the available memory?

[Laxilef]

Thanks!
I'm testing on ESP8266 and ESP32.
I didn’t notice any problems on the ESP32, since there is more memory there than on the ESP8266. I continue to study the problem with memory. And it seems that the problem is not in your library. It looks like these are not really leaks, but heap fragmentation. That is, free heap = 22k, but max free block = 6k. The moment the max free block gets so small, the program crashes.

Although it would be great to try to allocate a pool at startup, for example 2kb. And then work only within this pool: all new JsonDocument will allocate memory only in this pool. Do you think it is possible to do this using a custom allocator?

[bblanchon]

On ArduinoJson 7, JsonDocument allocates blocks of 1KB, so it should be able to use the remaining heap, even if it's fragmented.
The allocation strategy is very different in v7 and doesn't allow reserving a pool at startup.
In theory, you could do it with a custom allocator, but I doubt it will improve the situation.

[Laxilef]

Ok, thanks for the information! I'll keep watching and let you know if I find something interesting.