Comments (3)
Hi @AyWa, I see your point. Unfortunately the timing of adding index replicas is hard to get right. If you do it too late ES would have started to relocate indices for no reason, and if you do it too early (as we're doing it now) it will be colocated on a n already crowded node, and then moved over. None of it is ideal.
If you're in a lucky position to have single-shard indices, you might want to experiment with "index.auto_expand_replicas", this delegates the replica setting to ES.
Update: The patch you sent looks reasonable. We'll add it to the next release.
from es-operator.
From the issue it's not completely clear to me what the impact is of the shard replica being added before the ES node is ready. Can you elaborate on the impact? I.e. does ES go into a degraded state because of this?
from es-operator.
From the issue it's not completely clear to me what the impact is of the shard replica being added before the ES node is ready. Can you elaborate on the impact? I.e. does ES go into a degraded state because of this?
Sure, so the impact of the replica being added before the ES node is mainly that it duplicate some shards operations (and so use more cpu / increase latency when cluster is really small), and also it is likely to go over the max
shards per node that we set in the configuration
Lets make a simple example:
- 2 existing nodes with 1 shards each (no replica) (max shards 1)
- operator decide to auto scale because CPU is more than
50%
(add 2 instance) - Pod can take few minute to be ready
- the 2 existing nodes will send each other a shards
- New Node are ready. Need to wait for ES to re balance again the shards
- then shards are send to new node
If with the same configuration we wait for the node to be ready before creating the replica, we will not have a time where max shards is bigger than expected, and ES will likely move less shards.
Let me know if you need more information
from es-operator.
Related Issues (20)
- Unable to enable basic authentication HOT 3
- VCT example fails due to not setting max_map_count
- Custom service HOT 3
- [Feature] auto scaling configurations HOT 1
- [Bug] IP stays in exclude list when draining fails
- Feature request: Awareness of rolling restart requirement for version upgrade
- Apply minIndexReplicas on config change HOT 4
- Handling for system indices HOT 2
- Interest in contributing to OpenSearch HOT 12
- Unable to create "Persistent Volume Claims" HOT 3
- Infinite loop while updating EDS HOT 1
- Deprecation Warning "policy/v1beta1 PodDisruptionBudget is deprecated"
- Document spec.excludeSystemIndices
- Elasticsearch data pods don't come up HOT 7
- Document runbook in case of emergency
- Pod stuck in migration HOT 2
- Could it be used with the eck operator? HOT 1
- Auto-scaling does not work HOT 3
- Dependabot wasn't able to update github.com/go-resty/resty
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from es-operator.