Make clean_stale_db_objects configurable #142
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
See title! This PR also updates local development instructions in README.
The primary concept here is params: https://airflow.apache.org/docs/apache-airflow/2.6.3/core-concepts/params.html#params
As of 2.0, Airflow has a new concept called task flows, that are less boilerplate-y and more declarative than traditional DAGs. They also allow you to pass the return value of one task directly to another. Here's more, specifically how to declare dependencies between tasks and traditional operators, which we need to here: https://airflow.apache.org/docs/apache-airflow/2.6.3/tutorial/taskflow.html#adding-dependencies-between-decorated-and-traditional-tasks
Checklist
Testing Instructions
clean_stale_db_objects
job with configuration. Use different values for window and max, or toggle on report, and confirm the DAG behaves as you've configured it.Closes #140, closes #141