Comments (7)
Thank you very much for including Datahike in the benchmark and reporting this performance issue with Windows. I would appreciate if you could update your reddit post to include your Ubuntu run numbers also. More data points help everyone. I will investigate the problem when I get hold of a Windows machine.
from datalevin.
No problem. Updated in reddit thread and on the datalevinbench repo README.
from datalevin.
Thank you. @joinr I appreciate it.
from datalevin.
It may not be a performance issue. Please take a look at joinr/datalevinbench#2
from datalevin.
That was it indeed. The assumption about the temp file directory, /tmp, doesn't hold on Windows. This led to an ever growing database. The interesting news is that a db about 13x the size was still reading about 3x slower than the datascript. With a fresh db, the results conform to your reported baseline. No problem with lmdb on windows, just tempfile expectations :)
I added new-db
to the benchmark to delete the old db (similar to what datahike is doing), which technically adds a little to the measurements, but the results are apparently identical (e.g. deletion isn't a problem). The "real" fix would be to implement a portable temp file pathway for the db (I think the storage/open function called with nil directory creates a tmp/uuid file, so that would probably be the only place to define a wrapper that leverages (System/getProperty "java.io.tmpdir") to construct a portable temp directory.
from datalevin.
Updated reddit thread and readme for posterity.
from datalevin.
Great. Thanks!
from datalevin.
Related Issues (20)
- Transaction Error: Entity cannot be retracted HOT 7
- [Client/Server] deal with cases when predicate definition contains references to DB
- Improve re-index
- collection bindings OOM for large data sets HOT 1
- Expose Clojure vars to inter-fn
- dump/load database in nippy binary format HOT 1
- Queries inside an inter-fn doesn't resolve a built-in properly HOT 1
- Support for Heterogenous tuples with :db/tupleTypes supporting :db.type/refs HOT 4
- The difference between :db.unique/identity and :db.unique/value for :db.type/tuple HOT 1
- Getting warnings for clojure.core/load being redefined HOT 1
- TX report prints differently HOT 1
- `:xform` not being called for `:cardinality/one` ref attributes in pull expression HOT 1
- Error transacting db.type/bigdec
- :validate-data? does not recognise homogeneous tuple :db/tupleType HOT 1
- DB cannot be started HOT 2
- deal with client calls when server search engine is timed out
- Datalevin doesn't play well with sci (through malli) HOT 6
- value unification in function expression result
- [QUESTION]: separate the client into a small lib? HOT 23
- Allow indexing of arbitrary data
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datalevin.