Getting/Setting occasionally extremely slow

[siovene]

Getting/Setting occasionally extremely slow

[WisdomPill]

hello @siovene, if you're looking for performance I would strongly recommend to move away from pickle, maybe plain json would be a good. But if you want to take it further then my suggestion would be to try orjson, not only for redis but also for rendering if you're using django-rest-framework it would be fairly easy.

In some cases you could also decide to not serialise at all.

I do not know how Scout APM works, I have never used it, but it would be nicer to get more insights into which method is taking long.

Imho sentry does do a good job when it comes to performance measurements, I prefer using new relic for that.
Sentry can be misleading because of the number of traces it analyses and it does not do a good job when trying to understand the big picture especially with multiple services.

[siovene]

Hi @WisdomPill,

I appreciate the tip, I will try JSON or orson. However, do you really think that pickling would be so slow that occasionally it takes 5 seconds to deserialize a 20 byte strings?

[WisdomPill]

it depends on the complexity of the class if you're serialising a class... if you're serialising basic objects (int, string and so on) I would maybe even go so far to not even serialise in order to see if there are any differences.

[siovene]

Thanks @WisdomPill.

Is there a way to specify this on a per-key basis? I just looked around my code base and I cache some simple values, some python dictionaries, but also some entire Django models or even querysets. And I also cache template fragments.

[WisdomPill]

then I would definitely drop pickle and write serializers for your querysets. you could give a try to CacheRouter or use caches directly