Comments (2)
This sacrifices some memory but potentially beats replicating the whole cache per CPU.
If by this you mean that each cache entry would also be duplicated, I don't think that's good idea. It would pollute the shared cpu cache with a bunch of repeat data. This may not be something that shows up in a microbenchmark, but it certainly matters for real workloads.
I imagine the best architecture would be a multi-level cache system based on the master + proxy cache system you described. A worker would first check the local proxy cache for an item and if it didn't find it, it would then check the master cache. If it found the item there it would pull that into the local cache. Otherwise, it would perform the request and write the result into both the master and local cache. This means there wouldn't have to be any complicated mechanism for notify workers of a new cache entry and workers wouldn't do extra work adding unused entries.
The master hash table could also be replaced by something which allows multiple readers (and maybe writers) - something like this table allows multiple wait-free readers, but would require writer locking. A structure like a Hash Trie would allow multiple readers and writers with very good scaling and latency properties, but would have slower reads than the single-reader hash table and isn't technically O(1) on reads (although I believe reads are still wait-free).
I'll look into to that this week - using the same locking global cache for a multi-level architecture shouldn't be too hard, and I'll do some benchmarking to see if a lock-free global cache would have serious benefit.
from lwan.
Ah, no, I meant a proxy struct per item per thread that requested that item:
struct cache_entry_proxy {
cache_entry_t *entry;
int refs;
};
The entry
field would point to the shared cache item, and while refs
is more than zero, it would hold a reference to the shared cache entry. This would effectively remove fast path synchronization: no atomic operations to increment/decrement references, no locks, etc. These would only happen when whe cache is cold, or if the current thread does not have a proxied entry.
Anyway, I appreciate this work. Please keep me posted!
(Funny you mention a "hash trie", as I'm working on a data structure with that name, although it seems different from the linked article. It's not for concurrent use, as I pretend to use it for URL routing purposes.)
from lwan.
Related Issues (20)
- Problem with clean build and wsgi HOT 1
- Re-expose straitjackets HOT 1
- set POST request limitation above 64kb HOT 1
- Unexpected socket closing HOT 8
- HTTP request to third party within a handler HOT 4
- Centos็ผ่ฏๆฅ้ HOT 3
- max_post_data_size HOT 1
- WebSocket connection error with Apple Safari HOT 6
- Illegal instruction (core dumped) in latest docker image HOT 11
- Debian stable build fails not finding `lua.h` HOT 2
- Unexpected socket closing, round#2 HOT 3
- 12% of cycles spent percent-decoding URLs HOT 4
- Thread scheduler is broken on linux HOT 2
- Lost sockets HOT 13
- Error when building on Mac HOT 2
- get_number_of_cpus HOT 2
- socket activation fails with `Could not parse socket activation data from systemd: Invalid argument (error number 22)` HOT 3
- Problem with request HOT 1
- Is Gihtub embedded in smolsite? HOT 1
- No such file or directory lwan.h 166 | #include "lwan-http-status.h" HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lwan.