Comments (10)
4m is more than I would have expected, iI guess this is because genes, variants, etc make their way into the closure?
from golr-loader.
it does seem high, I could put the values in a file so we can debug.
from golr-loader.
Another solution is to only close over equivalentClass for entities such as genes, variants, etc.
from golr-loader.
from golr-loader.
This occurred after a new data load, and we've tried increasing the memory.
some more stats on the object_closure field:
sum: 819945513
max: 648
mean: 21.15496642741734
All values:
object_closure.txt.gz
Nothing I can see in the logs when the server is starting up.
from golr-loader.
Here is some more info after reindexing:
- Indexed on monarch5, moved files to solr-dev
- facets on object_closure work
- Optimized index
- Exception when faceting on object_closure
So oddly this is related to optimizing on the solr-dev VM, versus optimizing on monarch5, I changed this as we were running out of disk space on monarch5.
from golr-loader.
I'm uncertain of the cause of this, but at least understand how to avoid it. Propose we close since it's no longer an issue.
from golr-loader.
this is rearing its head again, although not for every load, and it seems re-indexing fixes it. Will play around with memory config.
from golr-loader.
from golr-loader.
I have to look back through my notes to see if I tried docValues. This has also been fixed in Solr 7 and above so another reason to push for an update: https://issues.apache.org/jira/browse/SOLR-11240
from golr-loader.
Related Issues (20)
- Investigate on slow queries HOT 1
- Queries should accept CURIEs in START statements
- chromose queries throw NPE
- Consider aligning getNeighbors and getClosure methods
- add flag to delete json as soon as they're indexed
- Use de-reification in evidence graphs HOT 3
- Extend golr pattern to include subj/obj equivalents and xrefs HOT 2
- Solr indexing sometimes fails HOT 13
- NotInTransactionException in golr loader HOT 2
- Rewrite golr loader to convert results to SolrInputDocument instead of JSON HOT 2
- Indexing behavior when a sub-object pair is linked by multiple relations HOT 6
- Version mismatch? HOT 3
- Consider indexing non clique leaders
- Remove IRI field from search core HOT 2
- Index all associations instead of merging subject-object pairs per query
- Index synonyms from non-clique leader IDs
- ArrayIndexOutOfBoundsException when splitting curies
- English spelling/synonyms HOT 3
- Filter out obsoleted terms from search HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from golr-loader.