Created at snapshot #2
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dbt-labs#8865) (dbt-labs#8878) * fix * test * changelog (cherry picked from commit 35f46da) Co-authored-by: Chenyu Li <[email protected]>
* add test * fix test * first pass with constraint error * add back column checks for temp tables * changelog * Update .changes/unreleased/Fixes-20231024-145504.yaml (cherry picked from commit 98310b6) Co-authored-by: Emily Rockman <[email protected]>
…ion==0 (dbt-labs#8922) Co-authored-by: Kshitij Aranke <[email protected]>
* Fix issues around new get_catalog_by_relations macro (dbt-labs#8856) * Fix issues around new get_catalog_by_relations macro * Add changelog entry * Fix unit test. * Additional unit testing * Fix cased comparison in catalog-retrieval function (dbt-labs#8940) * Fix cased comparison in catalog-retrieval function. * Fix cased comparison in catalog-retrieval function.
(cherry picked from commit 211392c) Co-authored-by: Chenyu Li <[email protected]>
…atalog queries (dbt-labs#8945) (dbt-labs#8963) * changelog * write test case demonstrating the issue * update catalog query to reflect materialized views (cherry picked from commit bb21403) Co-authored-by: Mike Alfare <[email protected]>
…st during release process
* Fix back compat for run_results pre-v5 * Add type annotations * Add functional testing * Add inline annotations * Add changelog entry. * Consolidate upgrade_schema_version + upgrade_run_results_json * Restore accidentally reverted test cases * Pre-commit fixups --------- Co-authored-by: Jeremy Cohen <[email protected]>
…bt-labs#9021) (dbt-labs#9026) * changelog * use MANIFEST.in to identify package data (cherry picked from commit 839c720) Co-authored-by: Mike Alfare <[email protected]>
* wip * add tests * changelog * nits * pr feedback * nits (cherry picked from commit 01d481b) Co-authored-by: Chenyu Li <[email protected]>
…st during release process
(cherry picked from commit 3902137) Co-authored-by: Michelle Ark <[email protected]>
…-labs#9065) (dbt-labs#9074) * Add test asserting `SavedQuery` configs can be set from `dbt_project.yml` * Allow extraneous properties in Export configs This brings the Export config object more in line with how other config objects are specified in the unparsed definition. It allows for specifying of extra configs, although they won't get propagate to the final config. * Add `ExportConfig` options to `SavedQueryConfig` options This allows for specifying `ExportConfig` options at the `SavedQueryConfig` level. This also therefore allows these options to be specified in the dbt_project.yml config. The plan in the follow up commit is to merge the `SavedQueryConfig` options into all configs of `Exports` belonging to the saved query. There are a couple caveots to call out: 1. We've used `schema` instead of `schema_name` on the `SavedQueryConfig` despite it being called `schema_name` on the `ExportConfig`. This is because need `schema_name` to be the name of the property on the `ExportConfig`, but `schema` is the user facing specification. 2. We didn't add the `ExportConfig` `alias` property to the `SavedQueryConfig` This is because `alias` will always be specific to a single export, and thus it doesn't make sense to allow defining it on the `SavedQueryConfig` to then apply to all `Exports` belonging to the `SavedQuery` * Begin inheriting configs from saved query config, and transitively from project config Export configs will now inherit from saved query configs, with a preference for export config specifications. That is to say an export config will inherity a config attr from the saved query config only if a value hasn't been supplied on the export config directly. Additionally because the saved query config has a similar relationship with the project config, exports configs can inherit from the project config (again with a preference for export config specifications). * Correct conditional in export config building for map schema to schema_name I somehow wrote a really weird, but also valid, conditional statement. Previously the conditional was ``` if combined.get("schema") is not combined.get("schema_name") is None: ``` which basically checked whether `schema` was a boolean that didn't match the boolean of whether `schema_name` was None. This would pretty much always evaluate to True because `schema` should be a string or none, not a bool, and thus would never match the right hand side. Crazy. It has now been fixed to do the thing we want to it to do. If `schema` isn't `None`, and `schema_name` is `None`, then set `schema_name` to have the value of `schema`. * Update parameter names in `_get_export_config` to be more verbose (cherry picked from commit c2f7d75) Co-authored-by: Quigley Malcolm <[email protected]>
…errupt. (dbt-labs#9042) * During node execution, also treat SystemExit as an interrupt. (dbt-labs#8994) IDE worker process raises SystemExit in multiple scenarios, including user abort of a command. (cherry picked from commit 931b2db) * Add test asserting GraphRunnableTasks attempt to cancel connections on SystemExit (dbt-labs#9101) * Add test asserting GraphRunnableTasks attempt to cancel connections on SystemExit * Add test asserting GraphRunnableTasks attempt to cancel connections on KeyboardInterrupt * Add test asserting GraphRunnableNode doesn't try to cancel connections on generic Exception --------- Co-authored-by: Ben Mosher <[email protected]> Co-authored-by: Quigley Malcolm <[email protected]>
…st during release process
* Fixups for deps lock file (dbt-labs#9147) * Update git revision with commit SHA * Use PackageRenderer for lock file * add unit tests for git and tarball packages * deepcopy unrendered_packages_data before iteration, fix remaining tests * Add functional tests * Add changelog entries * Assert one more --------- Co-authored-by: Michelle Ark <[email protected]> * Restore warning on unpinned git packages --------- Co-authored-by: Michelle Ark <[email protected]>
… `dbt docs generate` (dbt-labs#9163) Co-authored-by: Kshitij Aranke <[email protected]>
…st during release process
…cts for manifest, catalog, sources, and run-results (dbt-labs#9229) * Drop `all_refs=True` from jsonschema-ization build process Passing `all_refs=True` makes it so that Everything is a ref, even the top level schema. In jsonschema land, this essentially makes the produced artifact not a full schema, but a fractal object to be included in a schema. Thus when `$id` is passed in, jsonschema tools blow up because `$id` is for identifying a schema, which we explicitly weren't creating. The alternative was to drop the inclusion of `$id`. Howver, we're intending to create a schema, and having an `$id` is recommended best practice. Additionally since we were intending to create a schema, not a fractal, it seemed best to create to full schema. * Explicity produce jsonschemas using DRAFT_2020_12 dialect Previously were were implicitly using the `DRAFT_2020_12` dialect through mashumaro. It felt wise to begin explicitly specifying this. First, it is closest in available mashumaro provided dialects to what we produced pre 1.7. Secondly, if mashumaro changes its default for whatever reason (say a new dialect is added, and mashumaro moves to that), we don't want to automatically inherit that. * Begin including schema dialect specification in produced jsonschema In jsonschema's documentation they state > It's not always easy to tell which draft a JSON Schema is using. > You can use the $schema keyword to declare which version of the JSON Schema specification the schema is written to. > It's generally good practice to include it, though it is not required. and > For brevity, the $schema keyword isn't included in most of the examples in this book, but it should always be used in the real world. Basically, to know how to parse a schema, it's important to include what schema dialect is being used for the schema specification. The change in this commit ensures we include that information. * Add change documentation for jsonschema schema production fix * Regenerate dbt jsonschemas with fixed mashumaro jsonschema production process Specifically we regenerated * catalog v1 * manifest v11 * run-results v5 * sources v3 using the command `scripts/collect-artifact-schema.py --path schemas`
…t-labs#9173 (dbt-labs#9255) * Move minimum DSI version to 0.4.2 We're backporting a feature "conversion metrics" to 1.7. Conversion metrics don't exist in DSI < 0.4.2 which is problematic if we allow for those versions. This ensures that those who are on a version of 1.7 that supports conversion metrics will also have the requisit version of DSI. * added ConversionTypeParams classes * updated parser for ConversionTypeParams * added step to populate input_measure for conversion metrics * added tests * added changelog * Regenerate v11 manifest jsonschema to include conversion metrics definition * Regenerate v11 manifest test artifact for testing version compatability --------- Co-authored-by: Will Deng <[email protected]>
…st during release process
* Fix full-refresh and vars for retry (dbt-labs#9328) Co-authored-by: Peter Allen Webb <[email protected]> (cherry picked from commit 1e4286a) * pr feedback * Update requires.py
…st during release process
(cherry picked from commit 50b85a0) Co-authored-by: Chenyu Li <[email protected]>
This reverts commit 5ec34cd.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
resolves #
Problem
Solution
Checklist