scalalang2/kafka-faker

A fake message generator for apache kafka

Kafka Faker

Kafka Faker is a fake message generator for kafka. It generates fake messages based on a schema and sends them to a kafka topic.

It's useful for a service that consumes messages from kafka, stream processing with tools (e.g. Apache Flink) or testing.

Configuration

kafka:
  brokers: localhost:29092
generators:
  - topic: sample_topic
    partitionKey: any
    template: |-
      {
        "name": "{{ personName }}",
        "email": "{{ email }}",
        "from_address": "{{ ethereumAddress }}",
        "to_address": "{{ ethereumAddress }}",
        "amount": "{{ intBetween 1 1000 }}",
        "timestamp": "{{ timestamp }}"
      }
    number: 5
    loop: true
    delay: 2s

Test

First of all, you need to run kafka to test this app locally. Use docker-compose to run kafka and kafka-ui.

$ docker-compose up -d

Then, build & run kafka-faker with the config file.

$ go build -o ./bin/kafka-faker .
$ ./bin/kafka-faker --config-file=config.yml

Visit kafka-ui to see the generated messages. If you use docker-compose file. you can visit the kafka-ui at http://localhost:8080/

LICENSE

MIT