Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DBT-747: Removing the describe table statement each tables rather use the show queries to gather metadata #189

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

vamshikolanu
Copy link
Collaborator

@vamshikolanu vamshikolanu commented Feb 8, 2024

Describe your changes

We have identified a bug where dbt-impala is trying to scan all schemas mentioned in dbt-project.yml and run a describe table on all tables of those schemas to figure out whether it's a view or a table that is causing large delays in the execution.

The problem was that Impala didn't implement the show views command which was fixed as part of https://issues.apache.org/jira/browse/IMPALA-3268.

Inorder to start using this adapter going further after this fix, we expect users to include this patch: https://issues.apache.org/jira/browse/IMPALA-3268

Internal Jira ticket number or external issue link

https://jira.cloudera.com/browse/DBT-747

Testing procedure/screenshots(if appropriate):

https://gist.github.com/vamshikolanu/977f79dc14acf16d512058410074718f

Checklist before requesting a review

  • I have performed a self-review of my code
  • I have formatted my added/modified code to follow pep-8 standards
  • I have checked suggestions from python linter to make sure code is of good quality.

@niteshy niteshy changed the title DBT-747: Issue with dbt-impala running describe table on all db all tables of DL DBT-747: Removing the describe table statement on all db all tables Feb 8, 2024
@niteshy niteshy changed the title DBT-747: Removing the describe table statement on all db all tables DBT-747: Removing the describe table statement each tables rather use the show queries to gather metadata Feb 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant