deividmarreiro/kafka-study

kafka-study

📘 Apache Kafka Study

This is my journey studying Apache Kafka along with another things, like Docker and Docker Compose
and some new programming languages like Go, Shell Scripting and JavaScript
relearn Java 😅

My Notion with my journey!

Setup

# Create Kafka and Zookeeper
docker compose up -d

# Check if the containers ar up
docker-compose ps

# Check if the zookeeper is running
docker-compose logs zookeeper | grep -i binding

# Check if the kafka is running 
# (this can take few seconds because will wait for the zookeeper start first)
docker-compose logs kafka | grep -i started

Kafka-utils Script

# See the help
./kafka-utils.sh --help

Utilities to use Kafka

Parameters: 
01 - Topic Name

Syntax: kafka-utils [--create-topic | --describe-topic | --test-prod | --test-cons ] TOPIC_NAME
options:
--create-topic     Create a topic if not exists in the kafka.
--describe-topic   Describe a kafka topic.
--test-prod        Test messages to kafka topic (sequence 1 to 100).
--test-cons        Test to print the messages sent in the --test-prod.
--help             Help with the script

Testing

# Create Topic
./kafka-utils.sh --create-topic "test-topic"

# Check if the topic was created
./kafka-utils.sh --describe-topic "test-topic"

# Testing Sending messages through a producer
./kafka-utils.sh --test-prod "test-topic"

# Testing Consuming messages through a consumer
./kafka-utils.sh --test-cons "test-topic"

Todo

• Create an project with Django

Related Links

Shell Script Study

• For Loop
• How to Use Command Line Arguments in a Bash Script
• Bash getopts
• Parse Command Line Arguments in Bash

Kafka Study

• Github: Confluent Inc. (Apache Kafka®)
• Medium: Aprendendo Kafka
• Kafka Cheat Sheet