hussein-awala/async-batcher

A service to batch the http requests.

Async Batcher

This project provides a Python library to batch the asynchronous requests and handle them in batches.

How to use

To use the library, you need to install the package in your environment. You can install the package using pip:

pip install async-batcher

Then, you can create a new AsyncBatcher class by implementing the process_batch method:

from async_batcher.batcher import AsyncBatcher

class MyAsyncBatcher(AsyncBatcher):
    async def process_batch(self, batch):
        # Process the batch
        print(batch)

# Create a new instance of the `MyAsyncBatcher` class
async_batcher = MyAsyncBatcher(max_batch_size=20)
async_batcher.start()

Benchmark

The benchmark is available in the BENCHMARK.md file.

Use cases

The AsyncBatcher library can be used in any application that needs to handle asynchronous requests in batches, such as:

• Serving machine learning models that optimize the batch processing (e.g. TensorFlow, PyTorch, Scikit-learn, etc.)
• Storing multiple records in a database in a single query to optimize the I/O operations (or to reduce the cost of the database operations, e.g. AWS DynamoDB)
• Sending multiple messages in a single request to optimize the network operations (or to reduce the cost of the network operations, e.g. Kafka, RabbitMQ, AWS SQS, AWS SNS, etc.)

async-batcher's Issues

Implement an AsyncBatcher for MongoDB

MongoDB supports bulkWrite operation that allows batching different types of write operations (insertOne, updateOne, updateMany, deleteOne, deleteMany, and replaceOne).

To interact with MongoDB asynchronously, we can use Motor package, that supports the ordered bulk write operation which we need to process the batch and return the result.

Implement an AsyncBatcher for Pinecone

Pinecone is a vector database used in ML and AI projects. Pinecone supports inserting, updating, upserting, and even querying vectors in batches, which according to Pinecone's official site "significantly reduces the time it takes to process data."

We can use the official Python client to achieve this.

Implement an AsyncBatcher for AWS Kinesis

Like Kafka, producing AWS Kinesis messages helps optimize network utilization, improve overall throughput, and reduce cost.

We can use aiobotocore to produce the messages asynchronously in batches, similar to what we do with boto3.

Implement an AsyncBatcher for Apache Kafka

Producing Kafka messages in batches helps optimize network utilization and improve overall throughput.

aiokafka is the most used package to interact with Kafka asynchronously from Python, this package doesn't support an implicit batching mechanism. Instead, the user can create the batches explicitly and send them to Kafka.

Here is an example of how to produce messages in batches using aiokafka, which would be great to support in our package.