Comments (5)
@bobh66 : Just wondering, why did you revert your changes to fix the partition assignment problems raised in this issue?
from faust.
Hi @jkgenser - I needed to do more testing with the partition assignment changes and I didn't want to hold up releasing the other two fixes which are also important. The existing partitioner logic will intentionally leave a consumer without any active partitions, which is also validated by existing tests, so in order to "fix" that I need to change the way the tests work. I'm reluctant to make too big a change in this area given how much of an impact it can have. I'm going to restrict the partitioner changes to only force a "balanced" configuration when there are no tables in use and the table_standby_replicas option is set to 0. That will limit the initial impact of the change so we can see how it works, and then decide whether to expand the implementation or not. Since the partitioner will default to promoting a standby partition to active, it is still possible to have empty assignments even with my changes, so it's better to leave that code alone for now. Hopefully this makes sense.
from faust.
Has this issue been resolved?
from faust.
Has this issue been resolved?
@taybin We're currently experiencing this so I guess this isn't resolved yet.
from faust.
At this point there will always be scenarios where this can happen - see this comment for more details on exactly what was fixed - #97 (comment)
Basically you need to "enable" the fix by setting table_standby_replicas to 0.
from faust.
Related Issues (20)
- Events getting skipped HOT 1
- Wrong offset commit in case of gaps
- Logic around sensor _on_stream_event_in differs between cython and python implementations
- Does Faust Streaming Consumer retry the offset commit if it fails? HOT 1
- Add a method to include the whole message when using the take method HOT 1
- ModuleNotFoundError: No module named 'mode.utils.typing' HOT 4
- 0.10.22 is broken HOT 4
- normal logging mode
- Question: Throttle Reconnect Attempts when kafka is unavailable HOT 1
- How to check liveness of agent? HOT 6
- StreamT has no filter abstract method
- Remove aredis
- Testing: timers
- Certain topic partitions are not sending fetch requests since initial start
- Error when creating a topic using aiokafka 0.11.0 HOT 5
- GlobalTable doesnt respect schema
- aiokafka create topic error > 0.10.0 HOT 8
- Worker logging doesn't work
- How Can I get stream one value in HTTP Request?
- setuptools removed setuptools.command.test and faust-streaming is failing pip install HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from faust.