Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support dbt 'is_incremental' in pytest-dbt-core #24

Open
1 task done
danaeder opened this issue Jan 25, 2023 · 1 comment
Open
1 task done

[Feature] Support dbt 'is_incremental' in pytest-dbt-core #24

danaeder opened this issue Jan 25, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@danaeder
Copy link

danaeder commented Jan 25, 2023

Is this your first time opening an issue?

Describe the Feature

dbt adds the 'this' somehow in the is_incremental macro:
https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/include/global_project/macros/materializations/models/incremental/is_incremental.sql
is_incremental.sql

{% macro is_incremental() %}
{#-- do not run introspective queries in parsing #}
{% if not execute %}
{{ return(False) }}

Describe alternatives you've considered

I would like to test the following macro that uses 'is_incremental' and 'this' argument

{% macro get_table_max_id(where_clause = '', id_column = '"id"') %}
    {% if is_incremental() %}
        {% set max_id_query %}
        select coalesce(max({{ id_column }}), 0) from {{ this }}
        {{ where_clause }}
        {% endset %}
        {% set max_id_res = run_query(max_id_query) %}
    {%endif%}
    {{ return(max_id) }}
{% endmacro %}

following error:
def raise_compiler_error(msg, node=None) -> NoReturn:

  raise CompilationException(msg, node)

E dbt.exceptions.CompilationException: Compilation Error in macro is_incremental (macros/materializations/models/incremental/is_incremental.sql)
E 'this' is undefined
E
E > in macro get_table_max_id (macros/unioner.sql)
E > called by macro test_table_max_id (macros/test_macro.sql)
E > called by macro is_incremental (macros/materializations/models/incremental/is_incremental.sql)

Who will this benefit?

All dbt users since is_incremental is very popular

Are you interested in contributing this feature?

No response

Anything else?

No response

@danaeder danaeder added the enhancement New feature or request label Jan 25, 2023
@JCZuurmond
Copy link
Collaborator

JCZuurmond commented Jan 26, 2023

Hi @danaeder, thanks for opening the issue. I see the this is missing; dbt adds this somehow; this package does not (yet).

Could you try the following for me?

{% macro get_table_max_id(where_clause = '', id_column = '"id"') %}
        {% set max_id_query %}
        select coalesce(max({{ id_column }}), 0) from {{ this }}
        {{ where_clause }}
        {% endset %}
        {% set max_id_res = run_query(max_id_query) %}
    {{ return(max_id) }}
{% endmacro %}

What happens with the this after the from (third line)? Is that compiled correctly now?

And, something that might unblock you quicker, is it possible to use the is_incremental outside the macro. I expect that you have something like:

SELECT * 
FROM my_table 
{{ get_table_max_id() }}

What happens if you move the is_incremental out of the macro:

SELECT * 
FROM my_table 
{% if is_incremental() %}
{{ get_table_max_id() }}
{%endif%}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants