Auto partition assignment issue when there are more consumers than there are partitions

[spring-cloud]

Auto partition assignment issue when there are more consumers than there are partitions

[sabbyanandan]

From @mbogoevici on September 14, 2016 18:14

Looks like a Kafka binder issue - let's pursue this in https://github.com/spring-cloud/spring-cloud-stream-binder-kafka

[sobychacko]

@mnadelson Could you give a bit more context around the issue? If you could share your configuration with us, we can try reproducing it on our end. I tried to reproduce and found that it worked as expected.

I created a simple ticktock (time | log) stream with time source writing into 3 partitions and created 4 different log-sink consumers. I found out that only 3 of them received data. When I restarted a consumer, Kafka did the auto rebalancing among partitions, i.e. still only 3 of them got any partitions but reshuffled. I tried this with Kafka 0.9.

What version of Kafka are you using?

[mnadelson]

I ran my code again (and in multiple variations) and could not reproduce. If I can I will comment back.

[sobychacko]

Closing this issue because we are not able to reproduce it at this point. If we will encounter the problems described here in the future, then we will re-open this issue.