Comments (6)
Expected behavior describes how the cache was supposed to work. That's precisely why I used Cache.get
instead of regular access operator and documented it in the comments.
I've just investigated the behavior of the code and @shiosai is right! Call to Cache.get
is not enough. We need to drop down to Cache.__getitem__
to completely skip the eviction algorithm.
I'll update tests to highlight this problem and prepare a fix.
from cachetools import LRUCache, Cache
print("Access with 'id_to_cluster[1])' changes the order.")
id_to_cluster = LRUCache(maxsize=2)
id_to_cluster[1] = 1
id_to_cluster[2] = 2
print(id_to_cluster._LRUCache__order)
id_to_cluster[1]
print(id_to_cluster._LRUCache__order)
print()
print("Access with 'Cache.get(id_to_cluster, 1)' changes the order.")
id_to_cluster = LRUCache(maxsize=2)
id_to_cluster[1] = 1
id_to_cluster[2] = 2
print(id_to_cluster._LRUCache__order)
Cache.get(id_to_cluster, 1)
print(id_to_cluster._LRUCache__order)
print()
print("Access with 'Cache.__getitem__(id_to_cluster, 1)' does NOT change the order.")
id_to_cluster = LRUCache(maxsize=2)
id_to_cluster[1] = 1
id_to_cluster[2] = 2
print(id_to_cluster._LRUCache__order)
Cache.__getitem__(id_to_cluster, 1)
print(id_to_cluster._LRUCache__order)
Access with 'id_to_cluster[1])' changes the order.
OrderedDict([(1, None), (2, None)])
OrderedDict([(2, None), (1, None)])
Access with 'Cache.get(id_to_cluster, 1)' changes the order.
OrderedDict([(1, None), (2, None)])
OrderedDict([(2, None), (1, None)])
Access with 'Cache.__getitem__(id_to_cluster, 1)' does NOT change the order.
OrderedDict([(1, None), (2, None)])
OrderedDict([(1, None), (2, None)])
from drain3.
I would like to get the opinion of @StanislawSwierc on that issue, as he contributed the cache functionality.
from drain3.
@StanislawSwierc
Great - I wasn't aware that it could be easily "fixed" by using __getitem__
. I just had a short test with my (already partly optimized) code and it still speed up the whole thing by nearly 30% for my logs.
(In this case I still used Cache.get
for a final cache update with the selected cluster)
from drain3.
@StanislawSwierc Kindly reminding you that the fix you proposed would be appreciated, as I see that the memory efficiency feature is widely used.
from drain3.
Thanks @davidohana for the reminder! I saw this issue shortly before going for vacation and it dropped it from the radar. I've just prepared a fix. It turned out to be slightly more complicated than replacing a single function call, but still manageable.
from drain3.
@StanislawSwierc Thanks so much. Preparing a new version soon.
from drain3.
Related Issues (20)
- What is the difference between drain3 with logstash in elasticsearch ?
- visualize drain parse tree (feature) HOT 1
- Hi, I've been trying to use drain for running log anomaly detection on some logs.
- Log Matching on new data HOT 2
- Chinese and English hybrid log template mining HOT 5
- Some DRAIN templates with <*> do not have parameters extracted HOT 7
- PermissionError when running with Persistance
- Is it possible to freeze templates when trainning? HOT 2
- Add a py.typed marker file
- `extra_delimiters` does not account for prefixed/suffixed delimiters
- Drain3 in golang HOT 2
- Masking Prefix and Suffix should not be escaped HOT 1
- A interesting issues. HOT 1
- big_file demo result's first cluster content is empty
- masking question,if i want to output the masking such as real info of the date,how can i putput
- Avoid creating many redis connections when you want to have seperate buckets of templates. HOT 2
- [Question] HOT 2
- one question, how to do Incremental learning in drain3 training?
- one question, how to do Incremental learning in drain3 training? HOT 6
- Release a new https://pypi.org/ version to update dependencies HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from drain3.