Comments (3)
sfc-gh-tzhang
commented on June 23, 2024
I don't think we support this, but I guess you can do it with SMT?
from snowflake-kafka-connector.
ryzhyk
commented on June 23, 2024
Thanks! So to clarify, the current behavior is that JSON records will be parsed as if all columns of the table were configure in the AUTO mode. Is that correct?
from snowflake-kafka-connector.
sfc-gh-tzhang
commented on June 23, 2024
Yeah, I don't think we do anything fancy here, basically you create the TIMESTAMP column manually, then we use naming matching to put the data into corresponding column without any transformation. If you have any invalid timestamp format not in https://docs.snowflake.com/en/user-guide/data-load-snowpipe-streaming-overview#supported-java-data-types, the row will be rejected.
from snowflake-kafka-connector.
Related Issues (20)
- NDJSON support. HOT 4
- `buffer.flush.time` setting not working correctly. HOT 10
- Struct array elements are being serialized before writing to Array column HOT 4
- CVE-2023-39410 in Snowflake-kafka-connect JAR HOT 1
- Where is the 2.1.1 release? HOT 2
- Using configProviders other than file fails validation HOT 3
- google.cloud.storage.StorageException: 401 Unauthorized, causing RECORD_METADATA Exception: Invalid column name HOT 3
- Snowflake Sink committing offsets for null/tombstone messages HOT 4
- `google.cloud.storage.StorageException: 401 Unauthorized` HOT 3
- SNOW-989387 Connectors errored out after updating to v2.1.2 HOT 18
- Insert order when using snowpipe streaming HOT 2
- Streaming Channel Offset Migration (transient exception) HOT 3
- SCHEMA EVOLUTION: Converting a String to Number Column HOT 3
- Ingestion loop when Snowflake Ingestion fails HOT 3
- Log is flooded with INFO messages HOT 1
- Failure to properly deserialize decimal data type HOT 4
- Distribute "slim" version of the connector HOT 2
- Sporadic events causing lags? HOT 3
- Rollback on Prefix with channel name with connector name HOT 1
- Can i point Kafka record to snowflake table's column directly, just like we use snowflaketopic2table, do we have snowflakerecord2column config? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from snowflake-kafka-connector.