Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/cele 78 #30

Merged
merged 72 commits into from
Oct 11, 2024
Merged

Feature/cele 78 #30

merged 72 commits into from
Oct 11, 2024

Commits on Aug 28, 2024

  1. CELE-78 Work on the ingestion logic

    @dvcorreia
    dvcorreia committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    5c77304 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. CELE-78 Move extraction in to celegans ingestion CLI

    @dvcorreia
    dvcorreia committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    e144b3d View commit details
    Browse the repository at this point in the history

  2. CELE-78 Separate subcommands in its own files

    @dvcorreia
    dvcorreia committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    68be4cd View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2024

  1. CELE-78 Add segmentation file finder and tile grid metadata

    @dvcorreia
    dvcorreia committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    cf7b93f View commit details
    Browse the repository at this point in the history

  2. CELE-78 Code format

    @dvcorreia
    dvcorreia committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    e379eeb View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. CELE-78 Segmentations and tiles GCP storage upload

    @dvcorreia
    dvcorreia committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    e6843bd View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. CELE-78 Add suggestions from Vincent

    @dvcorreia
    dvcorreia committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    06032ad View commit details
    Browse the repository at this point in the history

  2. CELE-78 Fix regression on data ingestion directory

    @dvcorreia
    dvcorreia committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    afedf7e View commit details
    Browse the repository at this point in the history

  3. CELE-78 Unit tests for cli data validation and small code tweaks

    @dvcorreia
    dvcorreia committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    917d0ea View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2024

  1. CELE-78 Unit tests for cli segmentation upload to GCS

    @dvcorreia
    dvcorreia committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    4da554e View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. CELE-78 Add RFC bucket ingestion storage pattern docs

    @dvcorreia
    dvcorreia committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    9a6393f View commit details
    Browse the repository at this point in the history

  2. CELE-78 Add ingestion POC for 3D files

    @dvcorreia
    dvcorreia committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    647da8f View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. CELE-78 Some more format ingestion reviews

    @dvcorreia
    dvcorreia committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    7327788 View commit details
    Browse the repository at this point in the history

  2. CELE-78 Extract tileset metadata

    @dvcorreia
    dvcorreia committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    1df9667 View commit details
    Browse the repository at this point in the history

  3. CELE-78 Add new blob names and unit tests

    @dvcorreia
    dvcorreia committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    953a59e View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. CELE-78 If the prunning of the bucket is interrupted it removes the l… 

    …ifecycle rule
    @dvcorreia
    dvcorreia committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    cd42ff1 View commit details
    Browse the repository at this point in the history

  2. CELE-78 Code tweaks and remove of unused code

    @dvcorreia
    dvcorreia committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    09e5087 View commit details
    Browse the repository at this point in the history

  3. CELE-78 Upload EM tile metadata do GCP bucket

    @dvcorreia
    dvcorreia committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    f659884 View commit details
    Browse the repository at this point in the history

  4. CELE-78 Code format

    @dvcorreia
    dvcorreia committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    6af78bf View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2024

  1. CELE-78 Ingestion CLI review changes: multiple datasets

    @dvcorreia
    dvcorreia committed Sep 12, 2024
    Configuration menu
    Copy the full SHA
    b04d79f View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. CELE-78 Fix ingestion CLI bug

    @dvcorreia
    dvcorreia committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    e22207e View commit details
    Browse the repository at this point in the history

  2. CELE-78 Enforce dataset id to be present in datasets.json

    @dvcorreia
    dvcorreia committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    2c676a6 View commit details
    Browse the repository at this point in the history

  3. CELE-78 Tweaks to EM tiles metadata

    @dvcorreia
    dvcorreia committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    8526960 View commit details
    Browse the repository at this point in the history

  4. CELE-78 Fix bug in unit test

    @dvcorreia
    dvcorreia committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    5113fb9 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. CELE-78 Upload EM tiles metadata

    @dvcorreia
    dvcorreia committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    4d7747b View commit details
    Browse the repository at this point in the history

  2. CELE-78 Upload segmentation image resolution metadata

    @dvcorreia
    dvcorreia committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    1295251 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. CELE-78 Fix ingestion uploading synapses STL files when it should not

    @dvcorreia
    dvcorreia committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    45b660e View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. CELE-78 Display help when root command is called with no args

    @dvcorreia
    dvcorreia committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    0cdc3a4 View commit details
    Browse the repository at this point in the history

  2. CELE-78 Clarify ingestion overwrite flag usage

    @dvcorreia
    dvcorreia committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    15057c5 View commit details
    Browse the repository at this point in the history

  3. CELE-78 Add dry run ingestion flag

    @dvcorreia
    dvcorreia committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    c16af28 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. CELE-78 Upload ingestion raw data to GCP bucket

    @dvcorreia
    dvcorreia committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    db52b6b View commit details
    Browse the repository at this point in the history

  2. CELE-78 Fix --data ingestion flag typo

    @dvcorreia
    dvcorreia committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    dccc13e View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2024

  1. CELE-78 Update dry-run option

    @aranega
    aranega committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    80b039a View commit details
    Browse the repository at this point in the history

  2. CELE-78 Change the raw-directory localisation

    @aranega
    aranega committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    9368346 View commit details
    Browse the repository at this point in the history

  3. CELE-78 Move up the argument for raw-data ingestion

    @aranega
    aranega committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    585031a View commit details
    Browse the repository at this point in the history

  4. CELE-78 Fix some debug message

    @aranega
    aranega committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    a96c37b View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. CELE-78 Remove debug level from gcp module

    @aranega
    aranega committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    91cf7c6 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. CELE-99 Add way of pulling information from local FS to populate the DB

    @aranega
    aranega committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    0046eeb View commit details
    Browse the repository at this point in the history

  2. CELE-99 Add cleaning of the DB before DB population

    @aranega
    aranega committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    9b4d122 View commit details
    Browse the repository at this point in the history

  3. CELE-78 Change directory for db-raw-data on the bucket

    @aranega
    aranega committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    5fcc821 View commit details
    Browse the repository at this point in the history

  4. CELE-99 Add fetching of raw db files from bucket

    @aranega
    aranega committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    d104bf2 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'feature/CELE-78' into feature/CELE-99

    @aranega
    aranega committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    4bd83d5 View commit details
    Browse the repository at this point in the history

  6. CELE-99 Add summary.txt generation

    @aranega
    aranega committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    44265f9 View commit details
    Browse the repository at this point in the history

  7. CELE-99 Update .gitignore for ingestion

    @aranega
    aranega committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    64e8738 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'develop' of github.com:MetaCell/c-elegans-app into feat… 

    …ure/CELE-78
    @aranega
    aranega committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    55bcf93 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'feature/CELE-78' into feature/CELE-99

    @aranega
    aranega committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    291c66e View commit details
    Browse the repository at this point in the history

  10. CELE-78 Documentation for the ingestion script

    @dvcorreia
    dvcorreia committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    dea760a View commit details
    Browse the repository at this point in the history

  11. CELE-78 Fix ingestion prune logic

    @dvcorreia
    dvcorreia committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    1796d8e View commit details
    Browse the repository at this point in the history

  12. CELE-78 Ignore mypy error in FakeBucket

    @dvcorreia
    dvcorreia committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    028e072 View commit details
    Browse the repository at this point in the history

  13. CELE-78 Fix bug in check prune rule

    @dvcorreia
    dvcorreia committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    d783bec View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. CELE-78 Code review changes

    @dvcorreia
    dvcorreia committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    a89de2f View commit details
    Browse the repository at this point in the history

  2. CELE-78 Improve prune_bucket doc comment

    @dvcorreia
    dvcorreia committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    71f3a9c View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2024

  1. CELE-78 Update description on bucket file structure

    @dvcorreia
    dvcorreia committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    be9e958 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2024

  1. CELE-97 Fix EM viewer sliding window undefined reference

    @dvcorreia
    dvcorreia committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    0346ecf View commit details
    Browse the repository at this point in the history

  2. CELE-97 Fix request for tiles at inexistent minzoom

    @dvcorreia
    dvcorreia committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    03a3ff7 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2024

  1. CELE-99 Add missing library for the backend

    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    2a9d1a0 View commit details
    Browse the repository at this point in the history

  2. CELE-78 Add github action tests for ingestion

    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    0d9d652 View commit details
    Browse the repository at this point in the history

  3. CELE-78 Fix bad import

    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    7960e00 View commit details
    Browse the repository at this point in the history

  4. CELE-78 Change the way metadata are represented for EM slices

    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ad44b6d View commit details
    Browse the repository at this point in the history

  5. Merge branch 'feature/CELE-99' into feature/em-viewer-config

    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    28da8e0 View commit details
    Browse the repository at this point in the history

  6. Add transmission of EM data information to the frontend

    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    a00d7f2 View commit details
    Browse the repository at this point in the history

  7. Change the way missing em metadata are handled

    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    5f24887 View commit details
    Browse the repository at this point in the history

  8. Generate new typescript binding

    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    3e53729 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'bug/CELE-97' into feature/em-viewer-config

    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    15e6b3b View commit details
    Browse the repository at this point in the history

  10. Plug backend information to the EM viewer

    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    c1c1a0b View commit details
    Browse the repository at this point in the history

  11. Merge branch 'develop' of github.com:MetaCell/c-elegans-app into feat… 

    …ure/CELE-78
    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    6e58e31 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'feature/CELE-78' into feature/em-viewer-config

    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    b4849cb View commit details
    Browse the repository at this point in the history

  13. Fix linting issue

    @aranega
    aranega committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    e2f7004 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2024

  1. Merge pull request #48 from MetaCell/feature/CELE-99 

    CELE-99 Add way of pulling information from local FS or the bucket to populate the DB and clean the tables before DB population
    @aranega
    aranega authored Oct 9, 2024
    Configuration menu
    Copy the full SHA
    92d6edd View commit details
    Browse the repository at this point in the history

  2. Merge branch 'feature/CELE-78' into feature/em-viewer-config

    @aranega
    aranega authored Oct 9, 2024
    Configuration menu
    Copy the full SHA
    00895c0 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. Merge pull request #59 from MetaCell/feature/em-viewer-config 

    EM Viewer Config
    @ddelpiano
    ddelpiano authored Oct 10, 2024
    Configuration menu
    Copy the full SHA
    3b8946e View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. CELE-78 Multithreading blob upload and retry

    @dvcorreia
    dvcorreia committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    e44249a View commit details
    Browse the repository at this point in the history