Comments (7)
We do actually have that mode too, -X jit-all-static-functions
. I think the only reason I used a wildcard jit list for these benchmarks is that I wanted a fair comparison with running the non-static benchmark under the jit, and it didn't seem fair to expose only one of them to the wildcard jit list overhead.
Long-term for many applications the right answer is probably a dynamic mode where hot functions are jitted once they become hot in the process. It just hasn't been a priority because our application is a prefork webserver, so that mode wouldn't work for us. But we are picking up more workloads now, so it might happen sometime soon.
from cinder.
More runs looks better. Some benchmarks still slower.
$ time ../../python.exe fannkuch.py 20
real 0m29.641s
$ time ../../python.exe -X install-strict-loader -X jit fannkuch_static.py 20
real 0m17.892s
$ time ../../python.exe richards.py 200
real 0m30.914s
$ time ../../python.exe -X install-strict-loader -X jit richards_static.py 200
real 0m7.475s
$ time ../../python.exe nqueens.py 20
real 0m5.657s
$ time ../../python.exe -X install-strict-loader -X jit nqueens_static.py 20
real 0m12.813s
from cinder.
better with the jit-list constraints (static nqueens still much slower overall)
$ time ../../python.exe -X install-strict-loader -X jit -X jit-list-file=jitlist_richards_static.txt -X jit-enable-jit-list-wildcards richards_static.py 200
JIT: Jit/pyjit.cpp:925 -- Enabling wildcards in JIT list
JIT: Jit/jit_list.cpp:33 -- Jit-list file: jitlist_richards_static.txt
real 0m5.802s
$ time ../../python.exe nqueens.py 40
real 0m11.296s
$ time ../../python.exe -X install-strict-loader -X jit -X jit-list-file=jitlist_nqueens_static_basic.txt -X jit-enable-jit-list-wildcards nqueens_static_basic.py 40
JIT: Jit/pyjit.cpp:925 -- Enabling wildcards in JIT list
JIT: Jit/jit_list.cpp:33 -- Jit-list file: jitlist_nqueens_static_basic.txt
real 0m9.917s
$ time ../../python.exe -X install-strict-loader -X jit -X jit-list-file=jitlist_nqueens_static.txt -X jit-enable-jit-list-wildcards nqueens_static.py 40
JIT: Jit/pyjit.cpp:925 -- Enabling wildcards in JIT list
JIT: Jit/jit_list.cpp:33 -- Jit-list file: jitlist_nqueens_static.txt
real 0m24.740s
Suggest a benchmarks/README with some example invocations like this.
from cinder.
Yes, we definitely need a README for the benchmarks, thanks for the report! You seem to have converged on the right way to run them yourself, though. Static nqueens is new and under active development, I wouldn't worry too much about it yet. Richards, deltablue, and fannkuch should all be much faster under Static Python and with the JIT (and faster with SP+JIT than JIT alone.) SP without the JIT is more of a mixed bag; some of the arithmetic-heavy benchmarks (e.g. fannkuch) use primitives a lot in the static version, and we only actually keep primitives unboxed in the JIT. Also fannkuch has some uncharacteristic performance without the JIT because our bytecode quickening currently operates at function level based on number of calls, and fannkuch is just one very expensive function that is only called once, so bytecode quickening never kicks in.
I'll keep this open to track getting both a README on running the benchmarks, and our results from running them, added to the repo.
from cinder.
About managing jit-lists, it seems to be tedious in general for applications, and wildcards have known performance problems (#29).
I wonder if it would be easier all around to have a mode where only functions in static modules are jitted.
from cinder.
(Oh one gotcha for -X jit-all-static-functions
: it's additive, so -X jit -X jit-all-static-functions
is the same as -X jit
; to jit only static functions you need -X jit -X jit-list-file=/dev/null -X jit-all-static-functions
.)
from cinder.
a script to run benchmarks was added recently: 77d5d1f
from cinder.
Related Issues (20)
- Type errors in deltablue_static HOT 3
- Build error: C++20 ambiguous & forming offset 4 is out of the bounds HOT 1
- fannkuch benchmark & implicit returns
- type aliases no longer work? HOT 2
- better static checks on initializing CheckedDict variables with dictionary literals HOT 1
- A test in test_static forgot to import `Optional` HOT 1
- Add type checking of implicit None return HOT 1
- `and` fails to refine `Optional[T]`
- take infinite loops into account in type narrowing HOT 5
- A test in `test_static` should catch `AssertionError` rather than `TypeError` HOT 4
- questions about CheckedDict and unknown types HOT 5
- Benchmark Cinder vs. Cpython performance; improve pipeline and performance of the Cinder build HOT 15
- 15 failing tests fail when running `make test` on a Mac HOT 1
- build error: listnode.gcda missing-profile HOT 3
- Cinder vs. Python 3.11 HOT 9
- can't compare double to Literal HOT 1
- nonsensical TypeError from "-X jit -m compile --static" HOT 3
- No pattern found for opcode Fadd: Xxr HOT 2
- HPy support HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cinder.