New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Edr/sql db #22

Merged

evanderiel merged 39 commits into main from edr/sql_db

Dec 15, 2023

Collaborator

evanderiel commented Dec 5, 2023

No description provided.

evanderiel requested review from movchan74 and HRashidi

December 5, 2023 09:29

movchan74 reviewed

View reviewed changes

aana/models/db/media.py Outdated Show resolved Hide resolved

movchan74 reviewed

View reviewed changes

aana/models/db/caption.py Outdated Show resolved Hide resolved

movchan74 reviewed

View reviewed changes

aana/models/db/transcript.py Outdated Show resolved Hide resolved

movchan74 reviewed

View reviewed changes

aana/models/db/transcript.py Show resolved Hide resolved

movchan74 reviewed

View reviewed changes

aana/configs/db.py Show resolved Hide resolved

movchan74 reviewed

View reviewed changes

aana/utils/db.py Outdated Show resolved Hide resolved

movchan74 reviewed

View reviewed changes

aana/tests/db/datastore/test_utils.py Outdated Show resolved Hide resolved

movchan74 reviewed

View reviewed changes

aana/repository/datastore/caption_repo.py Outdated Show resolved Hide resolved

movchan74 reviewed

View reviewed changes

aana/exceptions/database.py Outdated Show resolved Hide resolved

movchan74 reviewed

View reviewed changes

aana/utils/db.py Outdated Show resolved Hide resolved

movchan74 requested changes

View reviewed changes

Contributor

movchan74 left a comment

I left some comments with questions and changes.
I would also like to know how you see it being integrated with the pipeline.

movchan74 reviewed

View reviewed changes

README.md Outdated Show resolved Hide resolved

evanderiel force-pushed the edr/sql_db branch from cfd137b to f9afb38 Compare

December 5, 2023 15:30

evanderiel requested a review from movchan74

December 7, 2023 21:25

movchan74 requested changes

View reviewed changes

Contributor

movchan74 left a comment

Need to be merged with the main because it has changes related to the PR. For example, I added generate_frames_decord that would need frame_ids as output as well.

aana/utils/db.py Outdated Show resolved Hide resolved

aana/utils/db.py Show resolved Hide resolved

aana/utils/video.py Outdated Show resolved Hide resolved

aana/models/db/media.py Outdated Show resolved Hide resolved

aana/configs/pipeline.py Show resolved Hide resolved

evanderiel added 13 commits

December 12, 2023 15:57


          Add models, repos, config

4517a1e


          Add db util functions for saving

7ee5a4a


          Try to resolve sqlalchemy typing trilemma

7ea7d99

There appears to be a problem with how SQLAlchemy's type annotations
work. Column[int] is incompatible with int, unless you install
sqlalchemy-stubs, which breaks the type annotations for reference()
fields. But you can manually cast things to int (id_type), so
we do that.


          Add alembic + tests

c1ef1de


          Fix tests

58f45f7


          Commit poetry.lock

8ec53a7


          Fix "Args" in docstrings

c9e0db6


          PR feedback

f2724ad

rename SqlAlchemy models to ...Entity


          Fixes for tests and configs (PR)

6c6de89


          Add check constraints to columns

9261aa5


          Replace old alembic init

7f5c9e8


          Make TranscriptEntity.segments a JSON column

e59b104


          Add frame_ids to frame extraction output

20c13f5


          PR: fixes for batching, dict_output

af68a85

evanderiel force-pushed the edr/sql_db branch from 30f0d2a to af68a85 Compare

December 12, 2023 14:58

evanderiel and others added 7 commits

December 12, 2023 19:28


          Fix transcription stuff

5e58869


          Fix bug in blip2 video endpoint definition

b1eedb0


          PR: de-batchify video storage

d6f6306


          Use a separate Video and Media entities

bda03c5


          Fix tests

8e95a54


          PR: cleanup

99e92a2


          Re-add batch transcript saving

4bf5b67

evanderiel requested a review from movchan74

December 14, 2023 10:27

movchan74 requested changes

View reviewed changes

Contributor

movchan74 left a comment

Noticed a few issues.
Also, I think I figured out the problem with tests, please check if it solves the issue.
And run the ruff check, there are a few problems it reports.

aana/models/db/base.py Outdated Show resolved Hide resolved

.vscode/settings.json Outdated Show resolved Hide resolved

aana/models/db/video.py Outdated Show resolved Hide resolved


          PR feedback: rename aana.models.db.BaseModel to BaseEntity

94ba449

evanderiel force-pushed the edr/sql_db branch from 8f92d47 to 94ba449 Compare

December 14, 2023 13:06

evanderiel added 4 commits

December 14, 2023 13:07


          Add __init__.py file to remove --import-mode=importlib

334c3a3


          Ruff fixes

e355d20


          Update README & precommit hook

afe6076


          Ruff format for db init

763f6ee

evanderiel requested a review from movchan74

December 14, 2023 13:28

movchan74 reviewed

View reviewed changes

.github/workflows/python-package.yml Outdated Show resolved Hide resolved

movchan74 reviewed

View reviewed changes

aana/api/app.py Outdated Show resolved Hide resolved

evanderiel added 2 commits

December 14, 2023 21:05


          PR feedback: remove need for video_id in nodes

271661e


          Pr feedback: disable importlib mode in CI tests

5144b91

evanderiel requested a review from movchan74

December 14, 2023 21:18

movchan74 approved these changes

View reviewed changes

evanderiel added 4 commits

December 15, 2023 10:37


          Merge branch 'main' into edr/sql_db

7e9cac8


          Fix db tests and type annotations

129f72d


          merge from origin/main

c038c34


          Ruff fix

28eb1a9

evanderiel merged commit 302e313 into main

1 check passed

evanderiel deleted the edr/sql_db branch

May 22, 2024 07:47

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet