Comments (9)
as when I graduate from university. :)
Congratulations!
I would like to take a deeper look into Moka's internal functionalities,
I wish I had more time to write design docs, so all you can read now are:
- The one that I already mentioned: https://docs.rs/moka/latest/moka/#implementation-details
- A wiki page about admission and eviction policies: https://github.com/moka-rs/moka/wiki
- And the source code.
Moka's design is heavily inspired by the Caffeine library for Java. So you can learn the base design from Caffeine's publications: https://github.com/ben-manes/caffeine#in-the-news
I started to develop Moka because I needed a concurrent cache for the storage layer of a NoSQL database (as my hobby project) and found there was no such cache library in Rust that would meet my needs. I thought I will finish implementing Moka in a couple of months so I can go back to my original project quickly. I never thought it would take some years (but I am enjoying).
from moka.
I merged #169 into the master branch. I will release it as a part of v0.9.3.
Closing this issue.
from moka.
Sorry, it took longer than I thought. But I have published Moka v0.9.3 to crates.io.
from moka.
Hi. Thank you for using Moka.
-
The expiration is
lazy
by design, so expired entries will be preserved in the cache for a short period of time.- As of v0.8.3, they will be usually removed from the cache within 300 milliseconds after expiration, but this delay can get longer if the cache is receiving very heavy reads and/or writes from clients.
- The 300 ms delay (actually a clean-up interval) is currently hard-coded, but we can make it configurable in the future.
- If you are curious about internals, there is a (little, work-in-progress) doc about it: https://docs.rs/moka/latest/moka/#implementation-details
-
There is also a known issue in an underlying library (crossbeam-epoch) that will prevent entries from dropping for much longer period after the removal from the cache.
- Our cache internally uses a lock-free hash table to store the entries and it uses crossbeam-epoch for epoch-based garbage collection (GC): https://docs.rs/crossbeam-epoch/latest/crossbeam_epoch/
- crossbeam-epoch's garbage collector is not optimal for some use cases and may keep ~100 garbages (objects waiting to be dropped) in the heap. We are hitting this issue and some entries will not be dropped immediately.
As for 1, please let me know whether you are okay with the current delay or not. We could make the delay configurable per cache.
As for 2, let me explore some ideas to resolve/mitigate the GC issue.
from moka.
Hi. Thank you for using Moka.
1. The expiration is `lazy` by design, so expired entries will be preserved in the cache for a short period of time. * As of v0.8.3, they will be usually removed from the cache within 300 milliseconds after expiration, but this delay can get longer if the cache is receiving very heavy reads and/or writes from clients. * The 300 ms delay (actually a clean-up interval) is currently hard-coded, but we can make it configurable in the future. * If you are curious about internals, there is a (little, work-in-progress) doc about it: https://docs.rs/moka/latest/moka/#implementation-details 2. There is also a known issue in an underlying library (crossbeam-epoch) that will prevent entries from dropping for much longer period after the removal from the cache. * Our cache internally uses a lock-free hash table to store the entries and it uses crossbeam-epoch for epoch-based garbage collection (GC): https://docs.rs/crossbeam-epoch/latest/crossbeam_epoch/ * crossbeam-epoch's garbage collector is not optimal for some use cases and may keep ~100 garbages (objects waiting to be dropped) in the heap. We are hitting this issue and some entries will not be dropped immediately.
As for 1, please let me know whether you are okay with the current delay or not. We could make the delay configurable per cache.
As for 2, let me explore some ideas to resolve/mitigate the GC issue.
Thanks for your reply, I think preserving an evicted connection for less than 3s is ok (after all the connection pool is just a toy project), and it seems hardly occur.
And I would like to take a deeper look into Moka
's internal functionalities, as when I graduate from university. :)
from moka.
crossbeam-epoch's garbage collector is not optimal for some use cases and may keep ~100 garbages (objects waiting to be dropped) in the heap. We are hitting this issue and some entries will not be dropped immediately.
This one could be something we are actually noticing. Our cache entries are quite large and not that many so having about ~100 of them hanging around might look like a memory leak to us. Is there any way to mitigate this?
from moka.
Sorry for not getting back to you sooner.
I wonder if we could mitigate it by using the flush
method of crossbeam_epoch::Guard
.
https://docs.rs/crossbeam-epoch/0.9.9/crossbeam_epoch/struct.Guard.html#method.flush
Clears up the thread-local cache of deferred functions by executing them or moving into the global cache.
Call this method after deferring execution of a function if you want to get it executed as soon as possible. Flushing will make sure it is residing in in the global cache, so that any thread has a chance of taking the function and executing it.
We use some "deferred functions" to drop entries removed from cache.
Let me try it in next few days. I will measure performance impacts for both memory usage and speed.
from moka.
No worries, thanks for getting back to me.
Thank you for looking into it! Sounds like an approach that could work.
from moka.
As for the flush
stuff, I am working on it here: #169
from moka.
Related Issues (20)
- Conditional/Delayed Eviction HOT 1
- Update documentation HOT 1
- Flaky test `cht::segment::tests::drop_many_values` under Cargo Tarpaulin HOT 1
- high resource usage in the housekeeper HOT 9
- Bug in `moka::future::Cache::invalidate_all`? Elements not being invalidated immediatelly. HOT 3
- feat: Notifications on eviction, etc. HOT 1
- Vulnerable to CVE-2022-23639? HOT 7
- Implications of `Arc<K>: Borrow<Q>` HOT 2
- dash Cache has no `get_with` HOT 2
- CI: Run Linux AArch64 tests on real hardware HOT 3
- Memory leak after `moka::sync::Cache` is dropped HOT 4
- Moka counter & multiple cache integration HOT 3
- cache line optimized CountMin4
- Add new api similar to `try_get_with`, but return `Option` rather than `Result`? HOT 6
- Is it worth to modify the key of `get_with` as a `&K` rather than `K`? HOT 10
- Help with size based eviction that weighs with bytes HOT 2
- moka is too polymorphic (cargo-llvm-lines) HOT 3
- Setting number of segments will disable notifications HOT 2
- try_get_with always runs init future HOT 4
- Staggering expiry with TTL HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from moka.