devopsbox/gafka

A full ecosystem that is built around kafka powered by golang

gafka

A full ecosystem that is built around kafka powered by golang.

Components

• ehaproxy

  Elastic haproxy that sits in front of kateway.

• kateway

  A fully-managed real-time secure and reliable RESTful Cloud Pub/Sub streaming message/job service.

• gk

  Unified multi-datacenter multi-cluster kafka swiss-knife management console.

• zk

  A handy zookeeper CLI that supports recursive query.

• kguard

  Kafka clusters body guard that emits health info to InfluxDB.

• gitlabtool

  A Pub/Sub sample application that watch all events on gitlab and evaluate KPI of programmers.

Install

export PATH=$PATH:$GOPATH/bin

#========================================
# install go-bindata first
#========================================
go get github.com/jteeuwen/go-bindata
cd $GOPATH/src/github.com/jteeuwen/go-bindata/go-bindata
go install

#========================================
# install gafka
#========================================
go get github.com/funkygao/gafka
cd $GOPATH/src/github.com/funkygao/gafka
go get ./...
./build.sh -h

#========================================
# install gafka command 'gk'
#========================================
./build -i -t gk

#========================================
# try the gafka command 'gk'
#========================================
gk -h

Status

Currently gafka manages:

• 4 data centers
• 50+ kafka clusters
• 100+ kafka brokers
• 500+ kafka topics
• 2000+ kafka partitions
• 10Billion messages per day
• peak load
  • 0.6Million message per second