The Snowflake Avro Value Converter silently skips messages it can't decode

[snowflakedb]

The Snowflake Avro Value Converter silently skips messages it can't decode

[sfc-gh-zli]

This #171 PR would enable the connector to support non-snowflake converters. If you use native json/avro converters, DLQ is naturally supported. That could be a walk-around of this problem.
Add loggings of broken record to Snowflake Converters could be an improvement on our todo list.

[raphaelauv]

@yarmiganosca is this still occurring ?

[morhook]

I think this is not happening anymore.

[raphaelauv]

from the code I see that only in DEBUG mode the connect log when it can't decode an event

snowflake-kafka-connector/src/main/java/com/snowflake/kafka/connector/internal/FileNameUtils.java

Line 82 in 1f793ad

LOGGER.debug(Logging.logMessage("generated broken data file name: {}", fileName));

And the connector will continue to consume the partition without failing

snowflake-kafka-connector/src/main/java/com/snowflake/kafka/connector/internal/SnowflakeSinkServiceV1.java

Line 511 in 1f793ad

writeBrokenDataToTableStage(snowflakeRecord);

[raphaelauv]

@sfc-gh-tzhang could share with us why you closed that issue , thanks

[sfc-gh-tzhang]

@raphaelauv I believe the original question was answered and now we're uploading the broken record to a table stage, is there anything else that is not addressed in this thread?