WorkerSincTask Error 5014

[snowflakedb]

WorkerSincTask Error 5014

[binglihub]

could you try to remove the snowpipe generated by connector and restart the connector?

[binglihub]

or, just restart connector again

[xiekevin]

Ah thanks for the help @binglihub. I've tried restarting the connector many times, but the message is the same. I've tried removing and adding the connector back in as well. I'm using it as part of a managed service for Kafka Connect (Aiven) so the amount of things I can change is fairly limited. There are some logs above the error message which are provided below:

[kafka-connect][SecurityManager-1(126)] INFO net.snowflake.ingest.connection.SecurityManager - Creating Token with issuer XXXX.USER.SHA256:HASH
[kafka-connect][SecurityManager-1(126)] INFO net.snowflake.ingest.connection.SecurityManager - Creating Token with subject XXXX.USER
[kafka-connect][SecurityManager-1(126)] INFO net.snowflake.ingest.connection.SecurityManager - Created new JWT - JWT_TOKEN
[kafka-connect][SF_KAFKA_CONNECTOR] SnowflakeSinkTask:stop (com.snowflake.kafka.connector.SnowflakeSinkTask:139)
[kafka-connect][2019-10-29 20:29:53,095] INFO
[kafka-connect][2019-10-29 20:29:53,094] ERROR WorkerSinkTask{id=SnowflakeSink-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:180)
[kafka-connect]at java.base/java.lang.Thread.run(Thread.java:834)
[kafka-connect]at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[kafka-connect]at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[kafka-connect]at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[kafka-connect]at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[kafka-connect]at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
[kafka-connect]at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
[kafka-connect]at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:196)
[kafka-connect]at org.apache.kafka.connect.runtime.WorkerSinkTask.closePartitions(WorkerSinkTask.java:590)
[kafka-connect]at org.apache.kafka.connect.runtime.WorkerSinkTask.commitOffsets(WorkerSinkTask.java:396)
[kafka-connect]at com.snowflake.kafka.connector.SnowflakeSinkTask.close(SnowflakeSinkTask.java:183)
[kafka-connect]at com.snowflake.kafka.connector.SnowflakeSinkTask.getSink(SnowflakeSinkTask.java:94)
[kafka-connect]at com.snowflake.kafka.connector.internal.SnowflakeErrors.getException(SnowflakeErrors.java:321)
[kafka-connect]at com.snowflake.kafka.connector.internal.SnowflakeErrors.getException(SnowflakeErrors.java:362)
[kafka-connect][SF_KAFKA_CONNECTOR] Detail: SinkTask hasn't been initialized before calling PUT function
[kafka-connect][SF_KAFKA_CONNECTOR] Error Code: 5014
[kafka-connect][SF_KAFKA_CONNECTOR] Exception: Failed to put records
[kafka-connect]com.snowflake.kafka.connector.internal.SnowflakeKafkaConnectorException:
[kafka-connect][2019-10-29 20:29:53,094] ERROR WorkerSinkTask{id=SnowflakeSink-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:179)
[kafkaconnect-9e4947f-2]2019-10-29T20:29:53.099094[kafka-connect][2019-10-29 20:29:53,098] INFO [Consumer clientId=connector-consumer-SnowflakeSink-0, groupId=connect-SnowflakeSink] Member connector-consumer-SnowflakeSink-0-9aa72fb1-2261-4084-b945-26bf310c7420 sending LeaveGroup request to coordinator 159.203.99.0:19154 (id: 2147483636 rack: null) (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:879)

Am I not authenticating correctly?

[binglihub]

Let me test it. it could be a server side issue.

[binglihub]

It is because of STOP function in SinkTask Timeout.
Fixed in 8550186

[xiekevin]

@binglihub thanks for looking into it! do you know when this change / version 0.5.4 will be released?

[binglihub]

@xiekevin it is available on Maven and Confluent Hub now

[ashinohara]

I have the most recent connector version 0.5.5 and I am still receiving a similar error when running the kafka connector:

[2019-11-19 12:50:59,491] ERROR WorkerSinkTask{id=Snowflake-7} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. (org.apache.kafka.connect.runtime.WorkerSinkTask:558)
com.snowflake.kafka.connector.internal.SnowflakeKafkaConnectorException:
[SF_KAFKA_CONNECTOR] Exception: Failed to put records
[SF_KAFKA_CONNECTOR] Error Code: 5014
[SF_KAFKA_CONNECTOR] Detail: SinkTask hasn't been initialized before calling PUT function
	at com.snowflake.kafka.connector.internal.SnowflakeErrors.getException(SnowflakeErrors.java:362)
	at com.snowflake.kafka.connector.internal.SnowflakeErrors.getException(SnowflakeErrors.java:321)
	at com.snowflake.kafka.connector.SnowflakeSinkTask.getSink(SnowflakeSinkTask.java:94)
	at com.snowflake.kafka.connector.SnowflakeSinkTask.put(SnowflakeSinkTask.java:195)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:538)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:321)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:224)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:192)
	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

[xiekevin]

I have seen that intermittently but I suspect it’s related to OOM issues. I upgraded the box the connect cluster is on currently to 32GB and it’s had no issues since after restarting it. Perhaps try that?

…

On Nov 19, 2019, at 12:53 PM, Andrew Shinohara ***@***.***> wrote:  I have the most recent connector version 0.5.5 and I am still receiving a similar error when running the kafka connector: [2019-11-19 12:50:59,491] ERROR WorkerSinkTask{id=Snowflake-7} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. (org.apache.kafka.connect.runtime.WorkerSinkTask:558) com.snowflake.kafka.connector.internal.SnowflakeKafkaConnectorException: [SF_KAFKA_CONNECTOR] Exception: Failed to put records [SF_KAFKA_CONNECTOR] Error Code: 5014 [SF_KAFKA_CONNECTOR] Detail: SinkTask hasn't been initialized before calling PUT function at com.snowflake.kafka.connector.internal.SnowflakeErrors.getException(SnowflakeErrors.java:362) at com.snowflake.kafka.connector.internal.SnowflakeErrors.getException(SnowflakeErrors.java:321) at com.snowflake.kafka.connector.SnowflakeSinkTask.getSink(SnowflakeSinkTask.java:94) at com.snowflake.kafka.connector.SnowflakeSinkTask.put(SnowflakeSinkTask.java:195) at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:538) at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:321) at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:224) at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:192) at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177) at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.