Comments (3)
@anaghshineh! 👋🏻 !
I'll say right away that I've never before encountered IAM Tags on BigQuery, so please enlighten me if I'm off as I try to figure it out and how you'd like to see it applied to dbt. I hope I'm providing helpful context
To me there are two questions to be answered:
How does one go about setting IAM tags programmatically in BigQuery (without dbt)
- Is it available possible via REST API?
- Is the REST API method surfaced within the
python-bigquery
client library?
After poking around the docs and got lost quickly. Help!
Can IAM tags be applied to individual models, or only to a Dataset or Project?
If there's a python-bigquery
way to do it, there's precedent for modifying/adding a new method to the BigQueryAdapter class. In virtually all cases it happens via a call to client.update_table
or client.update_dataset
(for example how PolicyTags are currently applied)
What should dbt UX look like?
- Should they work like grants?
models: - name: specific_model config: grants: select: ['SOME_IAM_TAG']
- Should they work like tags?
{{ config( materialized = "table", labels = {'SOME_IAM_TAG': ''} ) }}
from dbt-bigquery.
also @anaghshineh I'd appreciate your insight on dbt-labs/dbt-external-tables#263 if you have a chance.
from dbt-bigquery.
Hey,
I was also looking for this feature and stumbled upon this issue.
You can use these BigQuery tags to set conditional access (although it's still in preview I see).
To chip in on your question:
- Tags can technically be applied to both datasets and tables
- Programmatically, you can call
tables.update
and include tags inresource_tags
source - For datasets, there is no support for the REST API (but i can ask my Google contacts if needed)
Regarding the dbt UX:
- I like the label approach better -- with our org, we would like to have the flexibility to use our own tags / naming conventions (managed by terraform) -- for me it's not obvious how that would work with the grants approach;
from dbt-bigquery.
Related Issues (20)
- [Feature] Make `job_execution_timeout_seconds` configurable at the model level
- [Bug] hours to expiration timeformat is incorrect for materialized view
- [Feature] Keep grants for authorized views in sync when using the `grant_access_to` config HOT 8
- This is a test issue to see if GH project automation is woarking
- [feature] unit testing a recursive CTE fails HOT 8
- [Feature] Implicitly cast nested `STRUCT` properties during unit test fixture generation HOT 1
- [Feature] configure columns to be 'REQUIRED'
- [Feature] dbt manage multiple fields as primary key HOT 1
- BigQuery tags do not work HOT 1
- [Bug] State modified does not pick up changes to policy tags
- [Bug] dbt grant doesn't work for clone
- [Feature] Improvement in data processed/cost incurred in insert_overwrite method HOT 4
- [Release Improvements] Refresh workflows for the `pyproject.toml` migration
- [Bug] persist_docs not working for seeds HOT 2
- Import relevant pytest(s) for cross-database `cast` macro
- [Bug] `docs generate` appears to be returning no table metadata when run with the `--no-compile` option HOT 1
- [Tech Debt] `test_dbt_debug` is not dropping its test schema
- [Bug] `docs generate` does not find all schemas when there are more than 10K schemas
- Cross-database `date` macro
- [Bug] incremental run with __dbt_tmp table does not log the real bytes_billed in run_results.json HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dbt-bigquery.