Comments (3)
Sorry for the serious delay in responding here, @davehowell. This is something I've wanted to do for some time (#52). The potential time (+ cost) savings on Snowflake are immense. I think the required pieces are extensive enough that it could (+ should) be part of a larger refactor.
from snowplow.
I created a few override snowplow models in the main dbt project in order to bypass some of this, as well as adding those flattened out fields into some of the special "include" fields that are available as vars.
Honestly, this is what I've done every time I use Snowflake + Snowplow together. It would obviously be better if the package could "just work" on Snowflake in the way we know to be best.
from snowplow.
Unfortunately I have moved on from the previous project and don't have access to Snowplow data to look at this again, but I recall that I created a few override snowplow models in the main dbt project in order to bypass some of this, as well as adding those flattened out fields into some of the special "include" fields that are available as vars.
If you don't think it's too much to bite off at one time then sure include it as part of your larger refactor. I have to say the Snowplow models are pretty complicated, a lot to get your head around, and this particular issue is very Snowflake specific so might be easier to do ( and more importantly for the reviewer!) separately. If not then I guess this can be closed.
from snowplow.
Related Issues (20)
- Deduplication in snowplow_web_page_context drops all events that have a duplicate HOT 1
- NoneType has no len() error. HOT 7
- Support new snowplow data model HOT 2
- Compilation Error in model snowplow_web_ua_parser_context (models/page_views/optional/snowplow_web_ua_parser_context.sql) at path ['enabled']: None is not of type 'boolean' Code: 10004 HOT 3
- When are the todos in bigquery going to be implemented? Columns missing HOT 2
- no snowplow_users model - lookml should be removed HOT 1
- Testing for late-arriving facts HOT 1
- Update package readme
- Use enabled config to implement cross-database support HOT 2
- Map inferred_user_id in snowplow_page_views HOT 1
- Out of range exception HOT 4
- Performance issue with model snowplow_page_views HOT 9
- "Unrecognized name: root_id" and "No matching signature for function TIMESTAMP" HOT 3
- Fails to exclude page views from pingdom HOT 1
- snowplow_web_events_internal_fixed is not used
- Syntax Error in BigQuery snowplow_page_views
- Package tidy up
- Note undesired behavior from setting enabled: true in root project
- Add Spark support HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from snowplow.