Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/cele 119 #79

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from
Open

Feature/cele 119 #79

wants to merge 5 commits into from

Conversation

afonsobspinto
Copy link
Member

@afonsobspinto afonsobspinto commented Dec 19, 2024

closes https://metacell.atlassian.net/browse/CELE-119

  • Adds new populate_db endpoint, secured by a custom authenticator that uses django authenticate to login the user with username and password and verifies if the user exists and is a superuser

  • Adds @with_stdout_streaming decorator that:
    . Runs the decorated function in a separate thread
    . Captures anything it prints to stdout,
    . Streams that output asynchronously line-by-line as it's produced.

  • Updates ingestion script to trigger the database population (when the --populate-db flags is passed, we can also define the url via --populate-db-url") by using the gcp credentials to make a request to the aforementioned endpoint. Sub optimally, the request is made with curl via subprocess to surpass some buffering issues encountered when trying niquests (also happened with requests and httpx)

2024-12-19.11-51-46.mp4

@afonsobspinto afonsobspinto marked this pull request as ready for review December 19, 2024 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant