nmaquet / kasper Goto Github PK
View Code? Open in Web Editor NEWKasper is a lightweight library for processing Kafka topics.
License: MIT License
Kasper is a lightweight library for processing Kafka topics.
License: MIT License
I found your lib is pretty interesting and we share many similar ideas. Maybe we can collaborate somehow.
How about this distributed map reduce?
https://github.com/chrislusf/gleam/blob/master/examples/kafka_reader/kafka_reader.go
Greetings. I read your intro blog post and talk.
Very intrigued by this project (as well as goka), but noticed that it's still in beta and doesn't seem to be under active development.
So, curious ... are you still using it in production? Did the micro-batching / centralized state-management design prove to be problematic?
For testing, it would be nice to be able to use the noopMetric
as the MetricsProvider
but since it's not exported, you have to copy the implementation. It would be nice if this was exported so that you can use it.
You're defining two interfaces Map
and MultiMap
but in the concrete implementation of Redis
in the functions NewMultiRedis
and NewRedis
you're returning internal structure instead of the interface (MultiStore
and Store
respectfully). Same for elasticsearch.
This makes it difficult to leverage the interface when swapping out Store implementations.
For example, instead of:
func NewRedis(config *Config, conn redis.Conn, keyPrefix string) *Redis
It should be:
func NewRedis(config *Config, conn redis.Conn, keyPrefix string) Store
Since the Redis
struct implements the Store
interface.
If a user would like to use the concrete implementation (for some reason), they can always do something like the following to check the implementation and cast:
if r, ok := thestore.(*Redis); ok {
}
If we can make this change, it will make it much easier to actually use different implementations of these interfaces.
I'm happy to submit a PR if you'd like.
I use this for batch processing, but when multiple consumers are configured in the same consumer group, each consumer consumes the full amount of data;
is this design like this, or is there a bug?
i want use the consumer group , what should i do ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.