Comments (8)
Our recommended Node.js client is node-rdkafka. We don't test explicitly with kafka-node.
That said, we have some users running kafka-node and it used to work. I'm also not aware of a recent change that could have broken it. Can you share some logs to help us understand what's wrong with kafka-node?
To go back to your question, yes ssl.ca.location
is required with node-rdkafka and always have been. We also don't recommend disabling any TLS validations (rejectUnauthorized: false
) as these can allow a range of attacks that can compromise the security of your data.
from event-streams-samples.
Hi @mimaison . Thanks for the reply.
I was asking about kafka-node
, as I remembered that kafka-node
was recommended by IBM Event Stream before(if I recall it right). These days, I found the kafka-node
stopped working and I found this documentation, but not sure why these unsupported clients section is commented out.
With that said, when I have configuration:
{
kafkaHost: kafka_brokers_sasl,
sslOptions: { rejectUnauthorized: false },
sasl: {
mechanism: "plain",
username: user,
password: password
}
};
I get the errors:
{ TimeoutError: Request timed out after 30000ms
at new TimeoutError (/path/to/project/node_modules/kafka-node/lib/errors/TimeoutError.js:6:9)
at Timeout.timeoutId._createTimeout [as _onTimeout] (/path/to/project/node_modules/kafka-node/lib/kafkaClient.js:1012:16)
at ontimeout (timers.js:436:11)
at tryOnTimeout (timers.js:300:5)
at listOnTimeout (timers.js:263:5)
at Timer.processTimers (timers.js:223:10) message: 'Request timed out after 30000ms' }
However, I found a workaround here. And what we discussed there is the error we are facing. If I pass servername
under sslOptions
, I can successfully connect to IBM Event Streams
{
kafkaHost: kafka_brokers_sasl,
sslOptions: {
rejectUnauthorized: false,
servername: one of the kafka broker server name
},
sasl: {
mechanism: "plain",
username: user,
password: password
}
};
However, I need the luck to have a leader broker passed in. Therefore, I suppose it is not a workaround for the production environment.
Any ideas what is going wrong with the kafka-node
?
from event-streams-samples.
I guess you were previously using the Classic plan. This plan was retired earlier this year.
All our current plans (since early 2019) require Kafka clients to use SNI (Server Name Indication). In kafka-node
this is enabled via the servername
setting. This is not a workaround but the correct configuration. The Classic plan did not require SNI.
from event-streams-samples.
Thanks @mimaison
You are right, I was using the Classic plan. I am trying to migrate the project to use the standard plan. I thought using node-rdkafka
is the proper way until you mentioned that adding servername is the correct configuration instead of a workaround.
However, I have a concern, does the leader broker
change? If I do not provide a leader broker, I would get NotLeaderForPartition
. If that is the case, I will need to carefully handle that situation.
Thanks
from event-streams-samples.
I don't know kafka-node very well, but yes the fact that you can only provide a single servername is likely to be problematic. I see there's an open PR to address this: SOHU-Co/kafka-node#1375.
Our recommendation is still to use node-rdkafka. This is a client we continuously test and know works well.
from event-streams-samples.
Thank you very much, I think I will implement another API with node-rdkafka for my project.
from event-streams-samples.
If I've answered your questions, can we close this ticket?
from event-streams-samples.
Yes.
from event-streams-samples.
Related Issues (20)
- Server returned HTTP response code: 403 for URL HOT 2
- any way to avoid the ca_location flag? HOT 4
- ModuleNotFoundError: No module named 'confluent_kafka' HOT 2
- node-rdkafka build/run: dyld: Symbol not found: _timespec_get - MacOS
- Does the event-stream support the schema-registry in IBM public cloud? HOT 2
- starting node sample on windows
- Error: Cannot create 1 partition(s) for topic kafka-nodejs-console-sample-topic HOT 1
- Clarification regarding SASL Authentication with Event Streams HOT 6
- Connectors not listed HOT 2
- Does Event Stream support Spark Kafka Streaming? HOT 2
- Docker build fails for kafka-nodejs-console-sample due to ubuntu:latest base image HOT 2
- improve UX for kubernetes samples by creating secret using --from-file
- k8s lab kafka-java-console-schema-sample fails because image is not on dockerhub HOT 1
- kafka-nodejs-console-sample does not build on MacOS Catalina 10.15.6
- Kafka Connect REST API Security HOT 1
- Where Do You Download the PEM Certificate in IBM Cloud HOT 9
- java sample doesn't build with gradle 7 which is latest version as of Apr 09, 2021 HOT 2
- Upgrade the examples to use the ibm supplied APIs
- NodeJS documentation for local confusing when using Cloud Pak for for Integration HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from event-streams-samples.