Comments (7)
@jtcohen6 good shout. I've just tried emptying the packages.yml
then running dbt clean
and i'm getting the errors i expected eg. "model x depends on y" so i'll keep going with cleaning up my models till all dependancies are removed... and hopefully it was just that... :) FWIW I thought dbt clean would have cleaned my packages.yml too... but computers can't read my mind can they?
from snowplow.
Hey @joshuaLW, this is indeed a cryptic error. The question of removing the Snowplow package seems like a good place to start debugging.
- Do you have anything else in
packages.yml
? - Do you still have references to the
snowplow
package indbt_project.yml
?
from snowplow.
@jtcohen6 three things... I never actually checked that the snowplow modules were removed from packages.yml when i ran "dbt clean" but now having reversed these changes with "dbt deps" I've only this in that file:
packages:
- package: fishtown-analytics/snowplow
version: 0.11.0
I did comment out all variables and models referencing the snowplow package in the project.yml
I didn't remove all dependencies upon snowplow from my models but i expected to get an error of the kind model x refers to model y that does not exist.
I'm going to go at it that way now actually: i'm going to remove references to snowplow models in my models first and then try removing snowplow module again.
from snowplow.
I did comment out all variables and models referencing the snowplow package in the project.yml
If I had to pick one of the three as the likeliest cause, I'd say it's this one. Try removing snowplow
from packages.yml
, run dbt clean
, leave the vars
in place, and then see what error you get back. (I hope it will be of the missing-ref kind.)
from snowplow.
@jtcohen6 So the plot thickens: the error returns even with packages.yml
empty. It seems to be connected to one of the models I pinched from the package.
so to recap:
- i installed snowplow with dbt deps
- copied three models from the module into my models
- cleared packages.yml of its content
- ran dbt clean
- dbt docs generate (any other cmd get's the same result) and i get the "object of type 'NoneType' has no len()"
from snowplow.
@jtcohen6 yup i found the line that was causing the problem. It was a reference to a variable that was nested in the package i'd removed.
from snowplow.
There it is! Well found.
I think it would be an improvement to the snowplow package if its models handled better the case when var('snowplow:pass_through_columns')
is undefined in the project (None
), rather than merely empty ([]
).
Perhaps by rewriting that line to be:
{% if var('snowplow_pass_through_columns', []) | length > 0 %}
That way, if you do want to copy-paste the model code into your own project, and (sensibly) remove the snowplow-specific vars
from your project config, you won't be stymied by a cryptic error.
I'm going to transfer this issue to the snowplow package repository, since that's where the change should take place.
from snowplow.
Related Issues (20)
- Deduplication in snowplow_web_page_context drops all events that have a duplicate HOT 1
- Support new snowplow data model HOT 2
- Compilation Error in model snowplow_web_ua_parser_context (models/page_views/optional/snowplow_web_ua_parser_context.sql) at path ['enabled']: None is not of type 'boolean' Code: 10004 HOT 3
- When are the todos in bigquery going to be implemented? Columns missing HOT 2
- no snowplow_users model - lookml should be removed HOT 1
- Testing for late-arriving facts HOT 1
- Update package readme
- Use enabled config to implement cross-database support HOT 2
- Map inferred_user_id in snowplow_page_views HOT 1
- Out of range exception HOT 4
- Performance issue with model snowplow_page_views HOT 9
- "Unrecognized name: root_id" and "No matching signature for function TIMESTAMP" HOT 3
- Fails to exclude page views from pingdom HOT 1
- snowplow_web_events_internal_fixed is not used
- Syntax Error in BigQuery snowplow_page_views
- Package tidy up
- Note undesired behavior from setting enabled: true in root project
- Add Spark support HOT 1
- Snowflake web page context is unnecessarily separated and is not incremental HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from snowplow.