oslabs-beta/kafkasaur

Apache Kafka client for Deno

A native Deno client for Apache Kafka

Table of Contents

• About the project
• Features
• Getting Started
  • Usage
• Developers
• Acknowledgements
• License

About the Project

KafkaSaur an Apache Kafka client for deno

Getting Started

https://deno.land/x/[email protected]

Prerequisites

Deno - https://deno.land/manual/getting_started/installation

Apache Kafka - https://kafka.apache.org/

Docker (for examples) -https://www.docker.com/

Usage

To run examples initiate Docker containers included in yaml file:

docker-compose up

Then run example producer/consumer files, in seperate terminals, with the following commands:

deno run --allow-all --unstable examples/example_producer.ts
deno run --allow-all --unstable examples/example_consumer.ts

Your two terminals (one Consuming, and one Producing) will now interact with the Broker and begin consuming and producing respectively.

With the Client imported into your application, you can write the producer/consumer logic like this:

//producer example
import {Kafkasaur} from "https://deno.land/x/kafkasaur/index.ts"

const kafka = new Kafkasaur({
  clientId: 'example-producer',
  brokers: ['localhost:9092']
})

const topic = 'topic-test';

const producer = kafka.producer();

const testmessage = {
  key: 'key',
  value: 'hello there',
  headers: {'correlation-id': `${Date.now()}`}
}

const messages: object[] = [];
messages.push(testmessage)

const sendMessage = () => {
  producer.send({
    topic,
    messages
  })
}

const run = async() => {
  await producer.connect();
  sendMessage();
}

run()

//consumer example
import {Kafkasaur} from "https://deno.land/x/kafkasaur/index.ts"

const kafka = new Kafkasaur({
  clientId: 'example-consumer',
  brokers: ['localhost:9092']
})

const topic = 'topic-test';

const consumer = kafka.consumer({ groupId: 'test-group' })

const run = async () => {
  await consumer.connect()
  await consumer.subscribe({ topic, fromBeginning: true })
  
  await consumer.run({
    eachMessage: async (message: any) => {
      console.log(message.value.toString())
    },
  })
}

run()

To run the instances of your Consumer/Producer be sure to pass the flags

--allow-all

and

--unstable

when issuing your deno run command. This ensures that Deno as the proper configuration to communicate with the Broker, and to log any errors.

Features

• 🛠 Built with [TypeScript][Deno]

• 🎬 Producer

• 🍴 Consumer

• 🤝 interactive producer with consumer

• 💂 deno's built in security; No file, network, or environment access, unless explicitly enabled

Want to Contribute?

If you'd like to contribute and help grow the Deno community, just reach out to one of us via LinkedIn or write some code, and make a PR here! We're super excited about getting the conversation started, and working to bring Kafka to Deno!

Developers

Sam Arnold

Wesley Appleget

Adam Blackwell

Stephanie Benitez

Acknowledgements

• Tommy Brunn - for his guidance and for trailblazing with KafkaJS
• Ryan Dahl - for building an awesome community with Node.js and then leveling it up even further with Deno
• Franz Kafka - for making us all remember, we could just be cockroaches.

License

This product is licensed under the MIT License - see the LICENSE.md file for details.

This is an open source product.

This product is accelerated by OS Labs.

Apache Kafka and Kafka are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries. KafkaJS has no affiliation with the Apache Software Foundation.