godatadriven/pytest-dbt-core

Pytest plugin for dbt core

pytest-dbt-core's Issues

[Feature] Support dbt 'is_incremental' in pytest-dbt-core

Is this your first time opening an issue?

• I have read the expectations for open source contributors

Describe the Feature

dbt adds the 'this' somehow in the is_incremental macro:
https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/include/global_project/macros/materializations/models/incremental/is_incremental.sql
is_incremental.sql

{% macro is_incremental() %}
{#-- do not run introspective queries in parsing #}
{% if not execute %}
{{ return(False) }}

Describe alternatives you've considered

I would like to test the following macro that uses 'is_incremental' and 'this' argument

{% macro get_table_max_id(where_clause = '', id_column = '"id"') %}
    {% if is_incremental() %}
        {% set max_id_query %}
        select coalesce(max({{ id_column }}), 0) from {{ this }}
        {{ where_clause }}
        {% endset %}
        {% set max_id_res = run_query(max_id_query) %}
    {%endif%}
    {{ return(max_id) }}
{% endmacro %}

following error:
def raise_compiler_error(msg, node=None) -> NoReturn:

  raise CompilationException(msg, node)

E dbt.exceptions.CompilationException: Compilation Error in macro is_incremental (macros/materializations/models/incremental/is_incremental.sql)
E 'this' is undefined
E
E > in macro get_table_max_id (macros/unioner.sql)
E > called by macro test_table_max_id (macros/test_macro.sql)
E > called by macro is_incremental (macros/materializations/models/incremental/is_incremental.sql)

Who will this benefit?

All dbt users since is_incremental is very popular

Are you interested in contributing this feature?

No response

Anything else?

No response

[Feature] Add more test examples to docs

Is this your first time opening an issue?

• I have read the expectations for open source contributors

Describe the Feature

Add more examples to the docs, maybe as a separate page.

Describe alternatives you've considered

NA

Who will this benefit?

Users, the examples should help them get started quickly

Are you interested in contributing this feature?

yes

Anything else?

No response

[Feature] Test for Python 3.11

Is this your first time opening an issue?

• I have read the expectations for open source contributors

Describe the Feature

Test against Python 3.11

Describe alternatives you've considered

N.A.

Who will this benefit?

Those who use Python 3.11

Are you interested in contributing this feature?

yes

Anything else?

No response

[Feature] Run example tests from docs

Is this your first time opening an issue?

• I have read the expectations for open source contributors

Describe the Feature

The docs contain example tests, we should verify that those pass.

Note:

• The examples rely on macros. Where do we keep those? These should be stored in a place so that the tests can use them.

Describe alternatives you've considered

NA

Who will this benefit?

Me, as package maintainer I want to make sure that the examples in docs are accurate

Are you interested in contributing this feature?

yes

Anything else?

No response

[Feature] Extend the tox matrix to test for all minor versions of dbt

Is this your first time opening an issue?

• I have read the expectations for open source contributors

Describe the Feature

Extend the tox matrix to test for all minor versions of dbt.

Describe alternatives you've considered

N.A.

Who will this benefit?

Testing stability

Are you interested in contributing this feature?

No response

Anything else?

No response

[Feature] Run in DBT Cloud

Is this your first time opening an issue?

• I have read the expectations for open source contributors

Describe the Feature

Lots of my colleagues uses the DBT Cloud IDE. It lowers the entry barrier for anyone not experience with IDEs, git, etc. With that in mind, I would like them to also have the possibility to write unit tests and run the tests in that environment.

Describe alternatives you've considered

Running tests in CI pipeline, but this creates a very slow feedback loop.

Who will this benefit?

Any DBT developer using DBT Cloud IDE

Are you interested in contributing this feature?

Yes

Anything else?

I have now clue if this is even remotely possible. Does that DBT cloud runtime include Pytest? Is it possible to create a DBT package which pulls in additional packages from pypi?

[Feature] Speed up CI by not running redundant tox envs

Is this your first time opening an issue?

• I have read the expectations for open source contributors

Describe the Feature

The CI runs some tox environments multiple times.

Describe alternatives you've considered

N.A.

Who will this benefit?

Devs

Are you interested in contributing this feature?

yes

Anything else?

No response

[Feature] Flag to set the target

Is this your first time opening an issue?

• I have read the expectations for open source contributors

Describe the Feature

Add a flag to set the target.

Describe alternatives you've considered

Do not set a specific target, use the default target

Who will this benefit?

Everyone who does not want to use the default target

Are you interested in contributing this feature?

yes

Anything else?

Add flag to plugin:

# src/pytest_dbt_core/plugin.py
...
def pytest_addoption(...):
    ....
    
    parser.addoption(
        "--dbt-target",
        help="Which target to load for the given profile",
        type="string",
    )

And test in the example project in tests/

[Feature] Add examples for mocking macros

Is this your first time opening an issue?

• I have read the expectations for open source contributors

Describe the Feature

We are working on a use case where the macro internally maps to some classes. To perform the testing efficiently, classes need to be mocked instead of class functionality being used.
With macros directly pytest mocking cannot be used, some examples need to be added to showcase how to mock in the case of macros.

Describe alternatives you've considered

No response

Who will this benefit?

No response

Are you interested in contributing this feature?

No response

Anything else?

No response

[Feature] support dbt 1.6

Is this your first time opening an issue?

• I have read the expectations for open source contributors

Describe the Feature

hi @JCZuurmond ! dbt-core 1.6 released last week; do you plan on supporting it sometime soon?

if not, we're thinking about submitting a PR to support it since we use your library and we'd like to keep dbt up to date. just checking in before we do duplicative work.

FYI @ybressler

Are you interested in contributing this feature?

yes

[Feature] Unit testing models

Is this your first time opening an issue?

• I have read the expectations for open source contributors

Describe the Feature

Support unit testing models. Most likely we need another fixture to fetch a model, then similar to a macro, we use that fixture in a test.

To consider:

• models usually contain references to other models or sources: ref and source. How do we handle this? Could we for example mock them? If so, can we share mocks across tests.

Describe alternatives you've considered

Not unit testing models, stick to unit testing macros. You can put the complex SQL - or all SQL - in (modular) macros, then test those. This is not user friendly though.

Who will this benefit?

Those who want to test models, not just macros

Are you interested in contributing this feature?

Yes

Anything else?

See discussion on this topic in the dbt-core repo

[Feature] Add quickstart guide to docs

Is this your first time opening an issue?

• I have read the expectations for open source contributors

Describe the Feature

Add a quickstart guide to the docs

Describe alternatives you've considered

NA

Who will this benefit?

Package users, the quickstart should help them get started quickly.

Are you interested in contributing this feature?

yes

Anything else?

No response