Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQLMesh decorator processing fails with AttributeError for imported decorators #3640

Open
rbreejen opened this issue Jan 16, 2025 · 1 comment · May be fixed by #3687
Open

SQLMesh decorator processing fails with AttributeError for imported decorators #3640

rbreejen opened this issue Jan 16, 2025 · 1 comment · May be fixed by #3687
Assignees
Labels
Bug Something isn't working

Comments

@rbreejen
Copy link

Description

When using imported decorators in SQLMesh models, the processing fails with AttributeError: 'Attribute' object has no attribute 'id'. Possible cause: SQLMesh's AST processing assumes all decorators are Name nodes, but they can also be Attribute nodes depending on the import style.

Reproduction

Environment

  • Python 3.12
  • SQLMesh latest
  • tenacity (or other decorators?)

Steps

  1. Create a new file test_model.py:
from tenacity import retry, stop_after_attempt
from sqlmesh import model
import pandas as pd
from datetime import datetime
from sqlmesh import ExecutionContext
import typing as t


@retry(stop=stop_after_attempt(3))
def fetch_data():
    return "test data"

@model(
    name='test.raw_data',
    description='Test model with retry decorator',
    columns={
        "data": "text",
        "_sqlmesh__extracted_at": "timestamp"
    },
)
def execute(
    context: ExecutionContext,
    start: datetime,
    end: datetime,
    execution_time: datetime,
    **kwargs: t.Any,
) -> pd.DataFrame:
    data = fetch_data()
    return pd.DataFrame([{
        "data": data,
        "_sqlmesh__extracted_at": execution_time
    }])
  1. Run SQLMesh:
sqlmesh plan

Error

AttributeError: 'Attribute' object has no attribute 'id'

Expected Behavior

The model should process successfully, handling the @retry decorator.

@izeigerman izeigerman added the Bug Something isn't working label Jan 16, 2025
@rbreejen
Copy link
Author

I created a test around it in test_metaprogramming.py for debugging:

def test_decorator_name_with_attribute():
    @retry(stop=stop_after_attempt(3))
    def fetch_data():
        """Test function with retry decorator"""
        return "test data"

    # The test should fail when trying to process this function's decorators
    with pytest.raises(AttributeError, match="'Attribute' object has no attribute 'id'"):
        decorators(fetch_data)


def test_decorator_in_sqlmesh_model():
    # Mock the fetch_data function with retry decorator
    @retry(stop=stop_after_attempt(3))
    def fetch_data(url: str) -> str:
        return "<xml>test data</xml>"

    # Simplified version of your model
    @model(
        name='test.raw_data',
        description='Test model with retry decorator',
        kind=dict(name=ModelKindName.FULL),
        columns={
            "data": "text",
            "_sqlmesh__extracted_at": "timestamp"
        },
    )
    def execute(
        context: ExecutionContext,
        start: datetime,
        end: datetime,
        execution_time: datetime,
        **kwargs: t.Any,
    ) -> pd.DataFrame:
        # Use the decorated function
        data = fetch_data("https://api.example.com")
        return pd.DataFrame([{
            "data": data,
            "_sqlmesh__extracted_at": execution_time
        }])

    # Test that building the environment fails due to decorator
    env: t.Dict[str, t.Any] = {}
    with pytest.raises(AttributeError, match="'Attribute' object has no attribute 'id'"):
        build_env(execute, env=env, name="TEST_MODEL", path=Path("test_metaprogramming.py"))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants