thomaskwscott/kafka-connect-shell-source

A Source connector for Kafka that runs shell commands and ingests the output

kafka-connect-shell-source's Issues

WARNING: The (sub)resource method listConnectors in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation.

Hi,
I am trying to use thekafka-connect-shell-source with the standalone mode.
When I try to start the connector with /usr/local/kafka/bin/connect-standalone.sh /usr/local/kafka/config/connect-standalone.properties /usr/local/kafka/config/connect-file-source.properties command, I have :
`[2020-09-30 13:48:56,948] INFO StandaloneConfig values:
cluster = connect
rest.advertised.host.name = null
task.shutdown.graceful.timeout.ms = 5000
rest.host.name = null
rest.advertised.port = null
bootstrap.servers = [localhost:9092]
offset.flush.timeout.ms = 5000
offset.flush.interval.ms = 10000
rest.port = 8083
internal.key.converter = class org.apache.kafka.connect.json.JsonConverter
internal.value.converter = class org.apache.kafka.connect.json.JsonConverter
value.converter = class org.apache.kafka.connect.json.JsonConverter
key.converter = class org.apache.kafka.connect.json.JsonConverter
(org.apache.kafka.connect.runtime.standalone.StandaloneConfig:165)
[2020-09-30 13:48:57,818] INFO Logging initialized @2332ms (org.eclipse.jetty.util.log:186)
[2020-09-30 13:48:58,117] INFO Kafka Connect starting (org.apache.kafka.connect.runtime.Connect:53)
[2020-09-30 13:48:58,130] INFO Worker starting (org.apache.kafka.connect.runtime.Worker:89)
[2020-09-30 13:48:58,218] INFO ProducerConfig values:
compression.type = none
metric.reporters = []
metadata.max.age.ms = 300000
metadata.fetch.timeout.ms = 60000
reconnect.backoff.ms = 50
sasl.kerberos.ticket.renew.window.factor = 0.8
bootstrap.servers = [localhost:9092]
retry.backoff.ms = 100
sasl.kerberos.kinit.cmd = /usr/bin/kinit
buffer.memory = 33554432
timeout.ms = 30000
key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
ssl.keystore.type = JKS
ssl.trustmanager.algorithm = PKIX
block.on.buffer.full = false
ssl.key.password = null
max.block.ms = 9223372036854775807
sasl.kerberos.min.time.before.relogin = 60000
connections.max.idle.ms = 540000
ssl.truststore.password = null
max.in.flight.requests.per.connection = 1
metrics.num.samples = 2
client.id =
ssl.endpoint.identification.algorithm = null
ssl.protocol = TLS
request.timeout.ms = 2147483647
ssl.provider = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
acks = all
batch.size = 16384
ssl.keystore.location = null
receive.buffer.bytes = 32768
ssl.cipher.suites = null
ssl.truststore.type = JKS
security.protocol = PLAINTEXT
retries = 2147483647
max.request.size = 1048576
value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
ssl.truststore.location = null
ssl.keystore.password = null
ssl.keymanager.algorithm = SunX509
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
send.buffer.bytes = 131072
linger.ms = 0
(org.apache.kafka.clients.producer.ProducerConfig:165)
[2020-09-30 13:48:58,659] INFO Kafka version : 0.9.0.0 (org.apache.kafka.common.utils.AppInfoParser:82)
[2020-09-30 13:48:58,659] INFO Kafka commitId : fc7243c2af4b2b4a (org.apache.kafka.common.utils.AppInfoParser:83)
[2020-09-30 13:48:58,662] INFO Starting FileOffsetBackingStore with file /tmp/connect.offsets (org.apache.kafka.connect.storage.FileOffsetBackingStore:53)
[2020-09-30 13:48:58,677] INFO Worker started (org.apache.kafka.connect.runtime.Worker:111)
[2020-09-30 13:48:58,677] INFO Herder starting (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:57)
[2020-09-30 13:48:58,678] INFO Herder started (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:58)
[2020-09-30 13:48:58,678] INFO Starting REST server (org.apache.kafka.connect.runtime.rest.RestServer:91)
[2020-09-30 13:48:59,507] INFO jetty-9.2.12.v20150709 (org.eclipse.jetty.server.Server:327)
sept. 30, 2020 1:49:02 PM org.glassfish.jersey.internal.Errors logErrors
AVERTISSEMENT: The following warnings have been detected: WARNING: The (sub)resource method listConnectors in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation.
WARNING: The (sub)resource method createConnector in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation.
WARNING: The (sub)resource method serverInfo in org.apache.kafka.connect.runtime.rest.resources.RootResource contains empty path annotation.

[2020-09-30 13:49:02,378] INFO Started o.e.j.s.ServletContextHandler@61e94def{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:744)
[2020-09-30 13:49:02,556] INFO Started ServerConnector@3f291286{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:266)
[2020-09-30 13:49:02,558] INFO Started @7103ms (org.eclipse.jetty.server.Server:379)
[2020-09-30 13:49:02,582] INFO REST server listening at http://127.0.1.1:8083/, advertising URL http://127.0.1.1:8083/ (org.apache.kafka.connect.runtime.rest.RestServer:132)
[2020-09-30 13:49:02,583] INFO Kafka Connect started (org.apache.kafka.connect.runtime.Connect:60)
[2020-09-30 13:49:02,587] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:91)
org.apache.kafka.common.config.ConfigException: Invalid value uk.co.threefi.connect.shell.ShellSourceConnector for configuration connector.class: Class uk.co.threefi.connect.shell.ShellSourceConnector could not be found.
at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:255)
at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:145)
at org.apache.kafka.common.config.AbstractConfig.(AbstractConfig.java:49)
at org.apache.kafka.common.config.AbstractConfig.(AbstractConfig.java:56)
at org.apache.kafka.connect.runtime.ConnectorConfig.(ConnectorConfig.java:71)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.startConnector(StandaloneHerder.java:194)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:145)
at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:85)
[2020-09-30 13:49:02,627] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect:68)
[2020-09-30 13:49:02,672] INFO Stopped ServerConnector@3f291286{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:306)
[2020-09-30 13:49:02,772] INFO Stopped o.e.j.s.ServletContextHandler@61e94def{/,null,UNAVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:865)
[2020-09-30 13:49:02,796] INFO Herder stopping (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:62)
[2020-09-30 13:49:02,803] INFO Herder stopped (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:77)
[2020-09-30 13:49:02,803] INFO Worker stopping (org.apache.kafka.connect.runtime.Worker:115)
[2020-09-30 13:49:02,804] INFO Stopped FileOffsetBackingStore (org.apache.kafka.connect.storage.FileOffsetBackingStore:61)
[2020-09-30 13:49:02,804] INFO Worker stopped (org.apache.kafka.connect.runtime.Worker:155)
[2020-09-30 13:49:02,804] INFO Kafka Connect stopped (org.apache.kafka.connect.runtime.Connect:74)
`

The worker.properties is :
`## worker.properties

bootstrap.servers=localhost:9092
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
offset.storage.file.filename=/tmp/connect.offsets
offset.flush.interval.ms=10000
plugin.path=/usr/local/share/kafka/plugins`

And the shell-source.properties :
`## shell sources properties

name=kafka-connect-shell-source-5.1.0
connector.class=uk.co.threefi.connect.shell.ShellSourceConnector
tasks.max=1
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter`

The curl localhost:8083/connector-plugins returns :
curl: (7) Failed to connect to localhost port 8083: Connection refused

Any idea ?
Thanks you :)