Comments (17)
@nitantbhartia Hi, I've only gotten back to looking at this today, and it looks like there's a very simple solution.
just set spring.cloud.stream.kafka.binder.auto-create-topics=false
and the binder will never attempt to connect to Zookeeper. It still sets the default to localhost, but it isn't used.
I've tried this on 1.1.0.RELEASE
from spring-cloud-stream-binder-kafka.
Just to close this out; the upcoming 2.0 release has no dependencies on zookeeper at all. Even though the bindings have used the pure java client for a long time now, we still needed to go via Zookeeper to provision topics.
There is now a pure Java AdminClient
(it was added in 0.11 but was not fully functional - for the binder's needs - until 1.0.0).
from spring-cloud-stream-binder-kafka.
@barrycommins did you find a way around this? I am seeing the same issue and was wondering if you found a workaround
from spring-cloud-stream-binder-kafka.
@mbogoevici can we get some love here? Is there any way currently to bypass using zookeeper?
from spring-cloud-stream-binder-kafka.
@barrycommins Thanks for checking and responding! I suppose we can close this issue now?
from spring-cloud-stream-binder-kafka.
@mbogoevici Sure, I'll close it now, thanks.
from spring-cloud-stream-binder-kafka.
One thing that might be worth noting though is that even if spring.cloud.stream.kafka.binder.auto-create-topics
is set to false
, the KafkaProducer / KafkaConsumer will still autocreate the topic on the initial connection, using the default properties on the Kafka broker.
This is default behaviour in Kafka unless auto.create.topics.enable
is set to false
in server.properties
.
It will just be done by Kafka code instead of explicitly by the Spring Cloud Stream Kafka binder
from spring-cloud-stream-binder-kafka.
Yes - this is documented in the setting description (see http://docs.spring.io/spring-cloud-stream/docs/Brooklyn.BUILD-SNAPSHOT/reference/htmlsingle/#_configuration_options_2) . That is the behaviour of the Kafka broker and there's nothing that can be do on our side to control it - so the only thing we can do is to document it.
from spring-cloud-stream-binder-kafka.
Oops, RTFM I guess :-)
Sorry about that.
from spring-cloud-stream-binder-kafka.
No worries! Thanks for staying in touch! Looking forward to more :)!
from spring-cloud-stream-binder-kafka.
Just a heads up if you, like me, came upon this issue and still have problems... @barrycommins made a small typo in the property - 'kakfa.binder' should of course be 'kafka.binder'.
Serves me right for blindly copy-pasting but hopefully this saves somebody else the 15 minutes I spent wondering why this didn't work.
from spring-cloud-stream-binder-kafka.
@jensdt Sorry about that, I'd say I'd type 'kakfa' instead of 'kafka' about 50% of the time!
I've updated the original comment now to correct it.
from spring-cloud-stream-binder-kafka.
Thanks for spring.cloud.stream.kafka.binder.auto-create-topics=false
good to know!
from spring-cloud-stream-binder-kafka.
@garyrussell is this part of any of the current milestones? As of M5 I am still seeing zookeeper connections being opened.
from spring-cloud-stream-binder-kafka.
There is no M5; what do you mean?
2.0.0.M4 was released last week and no longer has zookeeper dependencies.
This commit: 50b8955
As you can see from the tag it is in M4
from spring-cloud-stream-binder-kafka.
Sorry I was not being accurate, M5 was referring to Spring Cloud Finchley.M5 release train.
Do you think M4 will be part of the next release train?
from spring-cloud-stream-binder-kafka.
See the project page.
Finchley.M5 includes Elmhurst.M3; you need to override the stream dependencies to Elmhurst.M4.
Presumably it will be in the next Finchley milestone.
from spring-cloud-stream-binder-kafka.
Related Issues (20)
- Live producer threads and dynamic-destination-cache-size property HOT 2
- How to wait for future inside Kafka Stream map()? HOT 2
- DefaultErrorHandler not exists HOT 4
- ClassCastException in pipeline: Kafka Streams Function -> Kafka Function -> Kafka Streams Consumer HOT 5
- NOT_ENOUGH_REPLICAS Errors After Upgrading The Spring Boot And Spring Cloud Version HOT 5
- KafkaBinderMetrics prevent JVM shutdown due to not shutting down scheduler thread pool! HOT 8
- Kafka bindings from previous test class remains active in the next test class HOT 2
- Kafka Streams Serdes having nested generic doesn't work HOT 3
- Support for DLQ in batch mode HOT 1
- spring.cloud.stream.bindings.process-in-0.consumer.concurrency not working in case of UpperCase letter in process name HOT 1
- ListenerContainerWithDlqAndRetryCustomizer template points to localhost:9092 HOT 5
- Looking for a sample with AWS MSK IAM
- [Feature Request] Allow to accept a list of customizers instead of a unique one for KafkaMessageChannelBinder HOT 1
- spring-kafka-test is compile (not test) dependency of spring-cloud-stream-binder-kafka-streams:3.2.3 and 3.2.4
- Batch mode is not working HOT 1
- Unknown Autoconfigured KafkaAvroSerializer and KafkaAvroDeSerializers HOT 2
- Spring Cloud stream binder kafka DLQ Handling for Batching doesn't work HOT 2
- Enable DLQ with destinations as pattern
- Fetch message timestamp
- Health Check for Binders Always Shows 'UNKNOWN' Status Due to Empty binderHealthContributors Map (Missing Bean Configuration for ReactorKafkaBinderHealthIndicator?) HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spring-cloud-stream-binder-kafka.