node-persist reading all files to RAM ?

[Vance-ng-vn]

node-persist reading all files to RAM ?

[akhoury]

if you use any of these 4 functions, https://github.com/simonlast/node-persist/blob/master/src/local-storage.js#L133-L166 yes, it seems that is in fact reading everything to ram, temporarily, this should be garbage collected since we don't maintain a ref to data. But for most usage, getItem only reads the target file. Now, granted, if you are writing an insane number of files, that could increase the size of the q, can you share how exactly you're using it.

[akhoury]

Also, I dont recommend using this library for handling large or high concurrent transactions, maybe something like redis is better for your case. node-persist is designed as a light weight temporary localStorage, with a small limit, inlined with what the browsers currently offer for localStorage, which is 5mb

[Vance-ng-vn]

if you use any of these 4 functions, https://github.com/simonlast/node-persist/blob/master/src/local-storage.js#L133-L166 yes, it seems that is in fact reading everything to ram, temporarily, this should be garbage collected since we don't maintain a ref to data. But for most usage, getItem only reads the target file. Now, granted, if you are writing an insane number of files, that could increase the size of the q, can you share how exactly you're using it.

I am using it to cache some files in my Node.js app (subtitle files). I need to cache these files for 7 days. However, it seems that node-persist is not suitable for my needs.

Every time it clean up expired files, node-persist loads all of them into RAM. (The total amount of RAM needed remains unchanged compared to caching directly into RAM, even though it's only temporary)

I've switched to using SQLite for temporary storage, and everything is working fine now.

[akhoury]

you are right!, cleaning up expired does in fact load everything, that's a bug that i will fix.