datalad container fails when running the exercise

[djarecka]

The command failed on my OSX. Installed datalad and datalad-container with pip (datalad 0.16.3, Python 3.8.11)

(ReproOHBM22) MacBook-Pro:my_analysis dorota$ datalad containers-run \
>   -n containers/repronim-simple-workflow \
>   --input 'rawdata/sub-RC4*/ses-1/anat/sub-*_ses-1_T1w.nii.gz' \
>   code/simple_workflow/run_demo_workflow.py \
>     -o . -w data/workdir --plugin_args 'dict(n_procs=10)' '{inputs}'
[INFO   ] Making sure inputs are available (this may take some time) 
get(ok): rawdata/sub-RC4106/ses-1/anat/sub-RC4106_ses-1_T1w.nii.gz (file) [from s3-PUBLIC...]                      
get(ok): rawdata/sub-RC4227/ses-1/anat/sub-RC4227_ses-1_T1w.nii.gz (file) [from s3-PUBLIC...]                      
get(ok): containers/images/repronim/repronim-simple-workflow--1.1.0.sing (file) [from datalad...]                  
[INFO   ] == Command start (output follows) =====                                                                  
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
ERROR  : Failed to spawn stage 1
[INFO   ] == Command exit (modification check follows) ===== 
run(error): /Users/dorota/tmp/ReproNim_OHBM2022/my_analysis (dataset) [containers/scripts/singularity_cmd run c...]
action summary:                                                                                                    
  get (notneeded: 2, ok: 3)
  run (error: 1)
  save (notneeded: 4)

Full history here

(ReproOHBM22) MacBook-Pro:ReproNim_OHBM2022 dorota$ datalad create -c text2git my_analysis
[INFO ] Running procedure cfg_text2git
[INFO ] == Command start (output follows) =====
[INFO ] == Command exit (modification check follows) =====
run(ok): /Users/dorota/tmp/ReproNim_OHBM2022/my_analysis (dataset) [/Users/dorota/miniconda3/bin/python /Use...]
create(ok): /Users/dorota/tmp/ReproNim_OHBM2022/my_analysis (dataset)
action summary:
create (ok: 1)
run (ok: 1)
(ReproOHBM22) MacBook-Pro:ReproNim_OHBM2022 dorota$ cd my_analysis
(ReproOHBM22) MacBook-Pro:my_analysis dorota$ datalad install -d . -s https://github.com/djarecka/ds001907-EDU.git rawdata
[INFO ] Remote origin not usable by git-annex; setting annex-ignore
[INFO ] https://github.com/djarecka/ds001907-EDU.git/config download failed: Not Found
[INFO ] access to 1 dataset sibling s3-PRIVATE not auto-enabled, enable with:
| datalad siblings -d "/Users/dorota/tmp/ReproNim_OHBM2022/my_analysis/rawdata" enable -s s3-PRIVATE
install(ok): rawdata (dataset)
add(ok): rawdata (file)
add(ok): .gitmodules (file)
save(ok): . (dataset)
add(ok): .gitmodules (file)
save(ok): . (dataset)
action summary:
add (ok: 3)
install (ok: 1)
save (ok: 2)
(ReproOHBM22) MacBook-Pro:my_analysis dorota$ datalad install -d . -s https://github.com/ReproNim/containers containers
[INFO ] Remote origin not usable by git-annex; setting annex-ignore
[INFO ] https://github.com/ReproNim/containers/config download failed: Not Found
install(ok): containers (dataset)
add(ok): containers (file)
add(ok): .gitmodules (file)
save(ok): . (dataset)
add(ok): .gitmodules (file)
save(ok): . (dataset)
action summary:
add (ok: 3)
install (ok: 1)
save (ok: 2)
(ReproOHBM22) MacBook-Pro:my_analysis dorota$ mkdir code
(ReproOHBM22) MacBook-Pro:my_analysis dorota$ datalad install -d . -s https://github.com/proj-nuisance/simple_workflow code/simple_workflow
install(ok): code/simple_workflow (dataset)
add(ok): code/simple_workflow (file)
add(ok): .gitmodules (file)
save(ok): . (dataset)
add(ok): .gitmodules (file)
save(ok): . (dataset)
action summary:
add (ok: 3)
install (ok: 1)
save (ok: 2)
(ReproOHBM22) MacBook-Pro:my_analysis dorota$ mkdir workdir
(ReproOHBM22) MacBook-Pro:my_analysis dorota$ echo workdir > .gitignore
(ReproOHBM22) MacBook-Pro:my_analysis dorota$ datalad save -m "ignore workdir" .gitignore
add(ok): .gitignore (file)
save(ok): . (dataset)
action summary:
add (ok: 1)
save (ok: 1)

(ReproOHBM22) MacBook-Pro:my_analysis dorota$ datalad containers-run \

-n containers/repronim-simple-workflow
--input 'rawdata/sub-RC4*/ses-1/anat/sub-*_ses-1_T1w.nii.gz'
code/simple_workflow/run_demo_workflow.py
-o . -w data/workdir --plugin_args 'dict(n_procs=10)' '{inputs}'
[INFO ] Making sure inputs are available (this may take some time)
get(ok): rawdata/sub-RC4106/ses-1/anat/sub-RC4106_ses-1_T1w.nii.gz (file) [from s3-PUBLIC...]
get(ok): rawdata/sub-RC4227/ses-1/anat/sub-RC4227_ses-1_T1w.nii.gz (file) [from s3-PUBLIC...]
get(ok): containers/images/repronim/repronim-simple-workflow--1.1.0.sing (file) [from datalad...]
[INFO ] == Command start (output follows) =====
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
ERROR : Failed to spawn stage 1
[INFO ] == Command exit (modification check follows) =====
run(error): /Users/dorota/tmp/ReproNim_OHBM2022/my_analysis (dataset) [containers/scripts/singularity_cmd run c...]
action summary:
get (notneeded: 2, ok: 3)
run (error: 1)
save (notneeded: 4)

[yarikoptic]

Well, I guess new/fancy m1 isn't supported in that we don't have an image built for it: neither our docker shim, nor workflow. Worth checking if there is a way to use that rosetta 2 emulator works with docker

[djarecka]

can try.. I wasn't sure if this is the issue or not. I got the warnings regarding the platform more often, but sometimes it just works...

[dnkennedy]

So, we'll be on a JupyterHub, so M1 will not be an issue there. So, for the course, I would not worry about that.

That said, if we are missing some large part of the community that can not run simple1 in general, we might want that fixed (and confirmed that you get the same answer, which would be cool!), but that is an issue for the simple1 project, not this course...

[yarikoptic]

We've troubleshooted this issue with @djarecka . I've filed the dedicated ReproNim/containers#67 and will buzz here whenever I think this issue is resolved.

[dnkennedy]

I'm going to close this as an issue for the course (where we will be running from a linux-based JupyterHub), but hope that the topic of making the M1's work properly will live on independently at ReproNim/containers...