Support OffsetCommit and OffsetFetch API for storing offsets in Kafka instead of Zookeeper

[elodina]

Support OffsetCommit and OffsetFetch API for storing offsets in Kafka instead of Zookeeper

[bosky101]

http://github.com/helpshift/ekaf 's producer also doesn't need zookeeper.

[t-pascal]

There seem to be several open issues related to Zookeeper and offsets. As of 0.8.2, kafka can support clients offsets storage inside a special topic without using zookeeper. We are already using 0.8.2 in production and would recommend upgrading to others.

It is not clear to me if go mirrormaker uses/supports the new API? I assume it does not. If there is anybody familiar with this project can point out where the upstream libraries are and whether they support the new API, that would be great. The only support needed in this project would a configuration directive like consumer.offset.storage = kafka for example (while still defaulting to zookeeper). See http://kafka.apache.org/082/documentation.html#distributionimpl

[joestein]

@t-pascal I think what you might be looking for is https://github.com/elodina/go_kafka_client/blob/master/docs/offset_storage.md