Skip to content

Commit

Permalink
CELE-78 Update description on bucket file structure
Browse files Browse the repository at this point in the history
  • Loading branch information
dvcorreia committed Oct 1, 2024
1 parent 71f3a9c commit be9e958
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 5 deletions.
26 changes: 21 additions & 5 deletions ingestion/format-ingestion.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,18 @@ The cloud storage of the ingested files will be organized in the following patte

```console
.
├── db-raw-data
│ ├── annotations
│ │ ├── complete.annotations.json
│ │ └── head.annotations.json
│ ├── connections
│ │ ├── <dataset_id>.json
│ │ ...
│ ├── datasets.json
│ ├── neurons.json
│ └── trajectories
│ ├── <dataset_id>.json
│ ...
├── dataset-1
│   ├── 3d
│   │   ├── nervering.stl
Expand All @@ -177,25 +189,29 @@ The cloud storage of the ingested files will be organized in the following patte
│   │   ├── ADEL.stl
│ │ │ ...
│   ├── em
│   │   ├── metadata.json
│   │   ├── ...
│   │   ├── 13
│   │   │   ├── 0_0_5.jpg
│   │   │   ├── 0_1_4.jpg
│   │   │   ├── 0_1_5.jpg
│ │ │ ...
│   │   ├── ...
│   │   └── metadata.json
│   │   ...
│   └── segmentations
│   ├── metadata.json
│   ├── s000.json
│   ── s001.json
│   ── s001.json
│   └── ...
├── dataset-2
├── dataset-3
...
```

Each dataset will have its own base directory with the name being the dataset identifier. Inside each dataset directory we will find 3 subdirectories:
A `db-raw-data` directory containing a copy of the data ingested in the dabase.

Then, each dataset will have its own base directory with the name being the dataset identifier. Inside each dataset directory we will find 3 subdirectories:

- `3d`: containing the 3D models for the neurons with the file name following `<neuron name>.stl`, with the exception of `nervering.stl`.
- `em`: storing each slice tileset in its own subdirectory and a `metadata.json` file with information required to represent the tiles in the frontend application _(TODO: define `metadata.json` format)_.
- `segmentations`: stores all the segmentation json files following the namming schema `s<slice>.json`, where `slice` is a positive integer (can contain left padding zeros).
- `em`: storing each slice tileset in its own subdirectory and a `metadata.json` file with information required to represent the tiles in the frontend application (for an example see: [./tests/fixtures/em-tiles/metadata.json](./tests/fixtures/em-tiles/metadata.json)).
- `segmentations`: stores all the segmentation json files following the namming schema `s<slice>.json`, where `slice` is a positive integer (can contain left padding zeros). It also contains a metadata file (for an example see: [./tests/fixtures/segmentation/metadata.json](./tests/fixtures/segmentation/metadata.json))
1 change: 1 addition & 0 deletions ingestion/tests/fixtures/segmentation/metadata.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"resolution":[19968,11008]}

0 comments on commit be9e958

Please sign in to comment.