Comments (5)
repeatedly
commented on August 28, 2024
I'm not sure why these files are not flushed but ".log" files contain buffered events.
from fluent-plugin-kafka.
repeatedly
commented on August 28, 2024
You have 2 pattern log files, "..b53d0adad2c926a4e.log" and "buffer.b53e210e8a540d529c7409b2af3b263c8.log".
I assume you updated fluentd to v0.14 or changed buffer_path parameter.
from fluent-plugin-kafka.
dtboctor
commented on August 28, 2024
Yeah, it looks like fluentd was bumped, would that have left those files. Is there any way to get those uploaded? Was the new version of fluentd incompatible with the old buffer files?
from fluent-plugin-kafka.
repeatedly
commented on August 28, 2024
Yeah, it looks like fluentd was bumped
We recommend to pin the fluentd / plugin version for stable deployment.
Is there any way to get those uploaded.
One way is moving old buffer files to other directly and using fluentd v0.12 to reload it.
Another way is reading buffer file and re-post it to fluentd via fluent-logger.
Was the new version of fluentd incompatible with the old buffer files?
Yes. fluentd v0.14 has new buffer mechanizm for more robustness and flexibility.
Hard to v0.12 buffers to v0.14 because v0.12 buffer doesn't have metadata in it.
from fluent-plugin-kafka.
dtboctor
commented on August 28, 2024
I'm going to close this since it isn't really an issue with this plugin.
from fluent-plugin-kafka.
Related Issues (20)
- Please enhance kafka2 output plugin to support ssl_client_cert_key_password HOT 5
- Output with kafka2 cannot dynamically define topic HOT 13
- Newly introduced share_producer setting in 0.18.0 for out_kafka2 plugin results in BufferOverflowError HOT 3
- rdkafka 0.12.0 is incompatible with fluent-plugin-kafka v0.18.0 or former HOT 7
- Fluent-bit failed to connecting to the Kafka brokers using Self Signed certs HOT 3
- Unexpected behavior by fluentd-collector from one Elastic search index to kafka HOT 4
- Output plugin kafka2 error: Kafka::BufferOverflow error="Cannot produce to XXX, max buffer size (1000 messages) reached HOT 3
- Support AWS MSK Authentication HOT 2
- Every buffer configuration returns Kafka::MessageSizeTooLarge or chunk bytes limit exceeds for an emitted event stream HOT 2
- Scram mechanism is not supported HOT 1
- SASL_PLAIN configuration does not work HOT 1
- encrypt / suppress sensitive data in the log HOT 2
- logging level HOT 2
- Implement `use_default_for_unknown_topic` for rdkafka2 output plugin HOT 3
- How to set topic_key with record's field in kafka2 output
- Request fluent-plugin-kafka gem version update HOT 3
- Expect a fluentd-plugin-rocketmq like this one
- ruby-kafka is no longer maintained and superseded by librdkafka via rdkafka-ruby bindings HOT 5
- Errors while sending data to Eventhub HOT 3
- source is kafka , not support kdc without ssl
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fluent-plugin-kafka.