Comments (3)
Hi, I'm interested by the same feature.
Based on my understanding of the code unfortunately it looks like Gogstash filters only allow exactly 1 (one) event out (as in one event in -> one event out).
Can anyone confirm this ?
I have several use-cases where it'd be useful for filters to allow either zero, one, or multiple events out:
- "splitting" as requested by Mayur above is one of those where we would let multiple events out of the filter.
- "dropping" events would also be a use-case for a filter that returns zero events (although this can already be done through conditional outputs I feel it might be interesting to remove unnecessary processing of events we want to discard).
- Lastly, I'd like to implement event-pairing filters, similar to Logstash "ellapsed" and "aggregate" filters. Those filters hold events in a buffer before releasing the combined result, therefore such filters might need to return zero event first, then two at once.
from gogstash.
At the beginning, it's designed to process logs one by one. If you want to handle M inputs and N outputs, M and N are unsigned integers, some interfaces need to be changed.
from gogstash.
If you need something to split your input have a look at Gollum, also written in go.
Gollum is an n:m multiplexer that gathers messages from different sources and broadcasts them to a set of destinations.
It will not solve all of your issues but see if you can use it for something.
from gogstash.
Related Issues (20)
- Elastic output document type as deprecated HOT 1
- Can I use dynamic index with elasticsearch output? HOT 2
- Back pressure handling document HOT 3
- useragent filter should use uap-go regexes by default HOT 1
- Fields with dots in the name cannot be referenced HOT 3
- Filter for key-value and unix nano timestamps HOT 2
- How to log in to elasticsearch HOT 2
- Input - UDP packet mode HOT 3
- Issues with file output HOT 1
- How does output to loki work? HOT 9
- Improve error message for geoip filter HOT 1
- output module "http" failed HOT 4
- Do you have any plan to add plain codec? HOT 5
- Cannot stop program HOT 3
- Supporting Logstash config syntax HOT 3
- Ctrl+C does not kill worker processes HOT 1
- Using GOMAXPROC instead of Forking HOT 12
- testing with zinc HOT 1
- Loki and org_id HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gogstash.