Bigben Kafka consumer dies whenever topic has >1 partition

[walmartlabs]

Bigben Kafka consumer dies whenever topic has >1 partition

[sandeepmalik]

Looks like the consumer is trying to pause reading from partitions at a time when no partitions are assigned (or are being rebalanced). Could you specify the sequence of steps and app/topic profile (how many nodes, total partitions, etc) that lead to this error and I'll try to replicate it at my end.

[ajzwu8]

@sandeepmalik In terms of bigben setup, we didn't edit the bigben.yaml file too much beyond what the original defaults were and connecting it to our own cassandra/kafka clusters and setting enable.auto.commit to true. So bigben would have been initiated with the default 8 kafka consumers listening on the 'bigben' topic.

Topics were created using the kafka bin utils as such:
JMX_PORT=53000 ./kafka-topics.sh --zookeeper "<our zookeeper>" --create --topic bigben --replication-factor 3 --partitions 16

We have a simple node script that creates a producer for topic 'bigben', then registers a tenant creates a consumer for the output topic of the tenant. That's enough to reproduce the error message shown above if the partitions are greater than 1.

The kafka cluster is using kafka v2.0.0.

Please let me know if any additional details/code is needed.

[sandeepmalik]

Ahh I see the issue, thanks for pointing it out. Working on the fix.

[ajzwu8]

Thanks @sandeepmalik! Do you have an ETA for merging this into master?

[sandeepmalik]

Most likely by EoD Monday of coming week.

[sandeepmalik]

I merged the dev branch. Please test and let me know if the issue is still there.

[ajzwu8]

@sandeepmalik Looks good, ran the same test with more partitions. Thanks a lot.

[sandeepmalik]

sounds good, thanks!