Comments (8)
200M a day is about 2k per second. That should work fine for any kind of field even on a single node.
from vespa.
Just the fields that you want to update. That is why we call it a partial update. Numeric fields like byte, int, float etc is faster then string. Agree with @bratseth, 200M updates a day should be no match for even a single core machine.
from vespa.
Vespa supports partial updates of existing indexed documents, fastest is for fields defined with 'attribute' and of type numeric. See http://docs.vespa.ai/documentation/reference/document-json-update-format.html for update json syntax.
from vespa.
vespa’s partial update just reindex the updated fields ? es will reindex all the fields
from vespa.
@zhuxiang1981 solr has in-place-updates but with some caveats (non indexed etc) https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#UpdatingPartsofDocuments-In-PlaceUpdates
from vespa.
Any further questions on this topic @zhuxiang1981 ? Thanks
from vespa.
We recently saw that 16k updates/sec were successful in one of our experiments with a cluster having 3 nodes, although all were integer updates. It's a good enough for now. We want to achieve 100k/sec updates which we would horizontally scale and achieve. Though we found that update throughput got very low (4k/sec) after we simultaneously ran benchmarking and hit the system with lots of queries. Any suggestions ?
from vespa.
from vespa.
Related Issues (20)
- Short form for indexed tensors representing binary data requires "values" HOT 1
- Error when onnx model is fp16 HOT 3
- Generate a sample Vespa JSON payload given a Vespa tensor type
- Vespa CLI option to delete all documents in an application
- What's the difference between Tensor and Vector within other vector databases? HOT 1
- Vespa Deploy fails for application package with model files HOT 3
- Add matched-elements-only support for index fields
- Syntax support for configuring distance-metric within the field
- No 'input' query param when use "vespa query yql=xxx" HOT 4
- Support setting metrics-proxy heap size HOT 3
- All Search Nodes are crashing HOT 4
- Potential Memory Leak Issue in PrometheusModel class HOT 4
- Node coming up with latest deployed application version after being down HOT 5
- Configurable max token length
- Add embedding instruction prompt support for to hf-embedder HOT 1
- NPE while deploying HOT 1
- Cluster Crashes When Distribution Key Is Too High HOT 2
- Error code - ORT_INVALID_PROTOBUF HOT 2
- Vespa content pod created snapshot folder owned by nobody user in a high RAM situation HOT 3
- Add deploy time warning about combining paged attributes with index (hnsw) or fast-search
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vespa.