Data caching issue

[ASDWQad]

Data caching issue

[edtechre]

Thanks @tsunamilx, I am not aware of any cache limit. I will need to test this myself and report back.

[edtechre]

Hi @ASDWQad,

You should be able to reuse the existing cache mechanism by extending DataSourceCacheMixin. Can you post your code here?

[tsunamilx]

Hi @edtechre

I have another question about the cache:
When I query for some small amount of symbols (like 100), the cache works, but when I query for large amount of symbols (like 1000), it always calls the fetch function (meaning cache is not used). Is there any limit on the cache (like max size limit) so that the early data is being evicted?

And when I query the 1000 symbols one by one, it actually loads everything from cache.

# querying 1000 symbols at once fails the cache
d = ds.query(symbols, start, end, timeframe)

# querying 1000 symbols one by one works
for s in symobls:
  d = ds.query(s, start, end, timeframe)

[edtechre]

Hi @tsunamilx,

Looking at the code, there is no difference between calling query with all symbols or individually for each symbol. See data.py.

Perhaps your cache was invalidated somehow?