Comments (7)
typedef struct {
int futex;
const char *description;
} futex_t;
That's 16 bytes on a 64-bit computer (8 for int, 1 for description, 7 for padding), reducing that to 1 byte should probably improve things.
from gilectomy.
@jdahlin We can't use C++ features to implement locking. We might be able to lift some restrictions and allow C99 instead of C89. Is there an implementation in C99?
from gilectomy.
A proof of concept using C++ would be fine, it's currently unclear what the exact benefits are. If it turns out to be a significant advantage it could be ported over to C, or alternatively convince upstream that C++ is worth it, much like GCC also started to use C++ a few years back.
I haven't looked at CPython in a few years, but if IIRC, compilation with C++ compilers should be supported.
from gilectomy.
ints are 4 bytes on all the 64-bit platforms I know of. I don't understand how you conclude that description is 1 bytes; it's a pointer, which is 8 bytes on 64-bit platforms. So that's twelve bytes.
The description is a debugging tool while we do development, to give us a clue what the lock is used for. You should not assume it'll be around forever in release builds. The production "futex" on Linux (32- or 64-bit) should be 4 bytes, for the "int futex", and we're simply not going to do better than that.
The production "furtex" should be 12 bytes by the time we're done: 4-byte int futex, 4-byte int depth, 4-byte int threadid. I don't think we're going to do better than that. I don't think we're going to get much smaller than that.
I'm closing this because it's not actionable; it seems to be a complaint ("they're too big!") without any suggestion on how to make them smaller. If you have a suggestion on how to make a lock smaller feel free to reopen.
from gilectomy.
Well, WTF from WebKit uses a lock that is only one byte (WTF::Lock). Splitting out the required parts out of WebKit and try to use them instead of an OS futex would be a worthwhile exercise to see if it can improve cache hit/miss ratios.
I might take a look at experimenting with that, if time & energy permits.
from gilectomy.
I think one should first try to see if there are any real cache misses caused by big locks.
Like try profiling with perf
or some other tool which allows you to see cache misses.
So far my gut feeling tells me that most likely we are hurt by cache misses caused by incref-decref operations (especially given they're now implemented as atomics, thus CPU must flush its cache to the memory - a really slow thing to do).
from gilectomy.
@larryhastings
I just had an idea how we can try to make furtex smaller - I think we can double-utilize the futex field to store thread id instead of simply 0 or 1, that gives it the minimum size of 8 - much better alignment to cache lines than 12.
Still I think this would have a insignificant impact on the performance...
from gilectomy.
Related Issues (20)
- [idea] Thread-local incr/decr HOT 3
- [idea] liblfds -- lock-free data structure HOT 3
- Improve locking of certain dicts (e.g. __globals__)
- Current refcount tracking idea seems to be incomplete for borrowed references
- Daemon threads seem to be broken
- threading locks seem to be thread-unsafe (sic!) HOT 5
- Come talk Gilectemy on Talk Python Podcast? HOT 3
- Stop refcounting singleton values
- [DISCUSSION] Suggestions for a different approach HOT 2
- [idea] RCU HOT 1
- [idea] CAS for reference counts
- Failed to build _testcapi
- gilectomy performs worse than stock cpython HOT 1
- Maintain as cpython branch instead of separate repo?
- [idea] multiple granularity locking
- [talk] Spin / Competition HOT 1
- Benchmarking is currently invalid. Use division of work, not work duplication.
- Any Updates or new Commits? HOT 5
- set 'gilectomy' as default branch
- Any comments on nogil? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gilectomy.