Giter Site home page Giter Site logo

Comments (3)

dataders avatar dataders commented on June 6, 2024

@anaghshineh! 👋🏻 !

I'll say right away that I've never before encountered IAM Tags on BigQuery, so please enlighten me if I'm off as I try to figure it out and how you'd like to see it applied to dbt. I hope I'm providing helpful context

To me there are two questions to be answered:

How does one go about setting IAM tags programmatically in BigQuery (without dbt)

  1. Is it available possible via REST API?
  2. Is the REST API method surfaced within the python-bigquery client library?

After poking around the docs and got lost quickly. Help!

Can IAM tags be applied to individual models, or only to a Dataset or Project?

If there's a python-bigquery way to do it, there's precedent for modifying/adding a new method to the BigQueryAdapter class. In virtually all cases it happens via a call to client.update_table or client.update_dataset (for example how PolicyTags are currently applied)

What should dbt UX look like?

  1. Should they work like grants?
    models:
      - name: specific_model
        config:
          grants:
            select: ['SOME_IAM_TAG']
    
  2. Should they work like tags?
    {{
      config(
        materialized = "table",
        labels = {'SOME_IAM_TAG': ''}
      )
    }}

from dbt-bigquery.

dataders avatar dataders commented on June 6, 2024

also @anaghshineh I'd appreciate your insight on dbt-labs/dbt-external-tables#263 if you have a chance.

from dbt-bigquery.

Junobijlard avatar Junobijlard commented on June 6, 2024

Hey,

I was also looking for this feature and stumbled upon this issue.
You can use these BigQuery tags to set conditional access (although it's still in preview I see).

To chip in on your question:

  • Tags can technically be applied to both datasets and tables
  • Programmatically, you can call tables.update and include tags in resource_tags source
  • For datasets, there is no support for the REST API (but i can ask my Google contacts if needed)

Regarding the dbt UX:

  • I like the label approach better -- with our org, we would like to have the flexibility to use our own tags / naming conventions (managed by terraform) -- for me it's not obvious how that would work with the grants approach;

from dbt-bigquery.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.