Comments (3)
Hey @drewbanin, thanks for taking the time to look into this.
Every single line is in one table - it seems to make use of BigQuery's nested & repeating column types.
No info yet available on the timeline for BigQuery support for the original web data model (hence me coming to this repo) - so I'll override the relevant models and macros until it works (as you suggest in README).
First step will be changing root_id
to contexts_com_snowplowanalytics_snowplow_web_page_1_0_0
!
Feel free to close this one; we can probably revisit once Snowplow themselves release BQ for the original model.
from snowplow.
Hey @wub - we find that the names of these columns are (confusingly!) different depending on how Snowplow is hosted. I checked out the schema you sent over above - is contexts_com_snowplowanalytics_snowplow_web_page_1_0_0
a table itself or is it a column in the event
table?
At a high level, I think there are two distinct things happening here:
Unrecognized name: root_id
- dbt is looking for a column called
root_id
, but it might be called something else in your setup. This columns should be the foreign key to theevent_id
in theevent
table. I'm no so familiar with the expected Snowplow schema on BigQuery. In something like Redshift, theweb_page
context will render out as a separate table with two columns:root_id
andid
. It looks to me like this is not the case in your setup!
No matching signature for function TIMESTAMP for argument types: TIMESTAMP
- I think this error is coming from these lines of code - BQ doesn't let you call
timestamp()
on atimestamp
field. I think the Snowplow datasets we've worked with have been providedcollector_tstamp
as adatetime
, and this code is intended to turn it into a timestamp.
In general, this package is based on Snowplow's original web data model. It sounds like some models for BQ are coming soon, and it would be super interesting to see how they end up recommending that these models work!
from snowplow.
Cool, thanks for the update @wub!
from snowplow.
Related Issues (20)
- Deduplication in snowplow_web_page_context drops all events that have a duplicate HOT 1
- NoneType has no len() error. HOT 7
- Support new snowplow data model HOT 2
- Compilation Error in model snowplow_web_ua_parser_context (models/page_views/optional/snowplow_web_ua_parser_context.sql) at path ['enabled']: None is not of type 'boolean' Code: 10004 HOT 3
- When are the todos in bigquery going to be implemented? Columns missing HOT 2
- no snowplow_users model - lookml should be removed HOT 1
- Testing for late-arriving facts HOT 1
- Update package readme
- Use enabled config to implement cross-database support HOT 2
- Map inferred_user_id in snowplow_page_views HOT 1
- Out of range exception HOT 4
- Performance issue with model snowplow_page_views HOT 9
- Fails to exclude page views from pingdom HOT 1
- snowplow_web_events_internal_fixed is not used
- Syntax Error in BigQuery snowplow_page_views
- Package tidy up
- Note undesired behavior from setting enabled: true in root project
- Add Spark support HOT 1
- Snowflake web page context is unnecessarily separated and is not incremental HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from snowplow.