Comments (9)
Which branch are you trying to build? I don't think we have any branches with any rc1
versions in the pom file. There's a release candidate tag that relies on a -cp1
version of Kafka, but this is pretty clearly labeled as a release candidate and those will sometimes rely on yet-unpublished dependencies as we build & test the platform as a whole, then push the release.
from kafka-connect-elasticsearch.
When I download something from releases, whether it is a release candidate or not, I expect it to be buildable. Looking back at closed issues I see that others have had this problem. May I suggest adding a note to the README to check the Kafka version in pom.xml
exists?
from kafka-connect-elasticsearch.
Hi @ewencp,
Thank you for your reply.
As others, I was also a little bit confused by the branch / tag and the fact that mast doesn't compile if you only have access to the public confluent repository.
Could you please give us more information about the below points?
- will the jar be released and published?
- are you planning to tag the code corresponding to the release kafka version (which tag should I use with Kafka 10.1?)?
This connector looks great, thank you for your work.
from kafka-connect-elasticsearch.
@matthew-d-jones There has not been an official release yet. We have opened up the repository early to allow people to have an early look. We provided the 0.10.0.0
branch on this repository to help those out who didn't want to build the SNAPSHOT dependencies themselves.
@bquartier Yes, the jars will be released and published, both in packaged form with the rest of Confluent and in Maven, although generally you'll want the former as it'll have all the dependencies bundled with it. We tag our releases of our projects with the version of Confluent that we release it with, i.e. you will see a v3.1.0
tag when we generate the 3.1.0 release.
from kafka-connect-elasticsearch.
@ewencp Ok, thank you. It is working nicely and we appreciate the work of the developers and Confluent making it available.
from kafka-connect-elasticsearch.
@matthew-d-jones were you able to make it work in distributed mode?
I keep getting an error saying:
org.apache.kafka.connect.errors.ConnectException: Sink tasks require a list of topics.
How did you define the mapping between topic and ES indices?
I tried a lot of different format in topic.index.map, without success.
from kafka-connect-elasticsearch.
@bquartier I've only tried the standalone connector so far, but that error suggests that you have not specified which topics to use in the .properties file. My elasticsearch-kafka-connect.properties
file looks like this:
name=elasticsearch-sink
connector.class=io.confluent.connect.elasticsearch.ElasticsearchSinkConnector
tasks.max=1
connection.url=http://localhost:9200
type.name=kafka-connect
# Custom config
topics=sample_log_test
key.ignore=true
from kafka-connect-elasticsearch.
is there any solution if the above error occurs in distriburted mode?
from kafka-connect-elasticsearch.
Closing because there are official releases of the connector now. Please reopen if there are any questions.
from kafka-connect-elasticsearch.
Related Issues (20)
- How to Convert JSON String field to ES Object?
- Capture Kafka key without using it as ID HOT 2
- Suggestion for INSERT operation "Ignoring EXTERNAL version conflict for operation INDEX on document"
- Used Elastic Java REST client is deprecated in 7.15.0 HOT 1
- Error with `"behavior.on.null.values": "delete"`
- Consumer paused indefinitely when using `AsyncOffsetTracker` with lot of null values
- Cannot use data stream with time_series mode HOT 2
- Error: Cannot infer mapping without schema HOT 1
- Connector fails with payloads >20 MB HOT 1
- Can't create a connector even if its loaded in Strimzi
- Support requests per second configuration options
- Log when there are too many requests errors
- [BUG] `TOO_MANY_REQUESTS` error craches the tasks with a unrecoverable exceptions without retries
- Ignore 'document_parsing_exception' HOT 1
- Inconsistent Logging for Tombstone Messages in Elastic Sink Connector
- abnormal data loss question
- Data Stream naming is far too restrictive HOT 1
- Creating index based on Timestamp doesn't work
- Limit retry backoff (and unlimited retries) HOT 4
- add support for index templates other than logs and metrics as types when using data streams
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kafka-connect-elasticsearch.