Launchcontainers: A Python tool for launching containerized analysis on HPC

[yongninglei]

Title

Launchcontainers: A Python tool for launching containerized analysis on HPC

Leaders

Yongning Lei

email: [email protected]

Collaborators

No response

Brainhack Global 2023 Event

Brainhack Donostia

Project Description

Launchcontainers is a Python-based program built to automatically launch containerlized MRI processing pipelines. This program takes one config.ymal file, one container.json file, and one subject-session-list.txt file as inputs. Using 1 line of bash command, it will automatically send jobs to HPC clusters regarding your computing demands.

This program is well-suited for multi-subject, multi-scan datasets. And it will save a lot of time if you need analysis your entire dataset with different parameters multiple times.

In future versions, we are trying to add more functionality to this repository so that you can launch all the MRI data analysis pipelines such as Heudiconv, fMRIprep, pRF pipelines and etc. Please keep track of this repo and if you have any questions or suggestions, don't hesitate to contact Gari: [email protected] and Tiger: [email protected]

Link to project repository/sources

https://github.com/garikoitz/launchcontainers

Goals for Brainhack Global

• Let more people get to know the advantages of using containers to process MRI data to improve the data availability and reproducibility
• Let beginner level users get familiar with the UNIX operation and git operation
• Add new features to launchcontainer so that it supports more MRI processing pipeline (ie. HeuDiconv, fMRIprep, PRF-analyze)
• Make a better documentation on the project Wiki

Good first issues

1. issue one:
   Add heudiconv garikoitz/launchcontainers#28
2. issue two:
   add instructions and link to code to do NORDIC preprocessing garikoitz/launchcontainers#32

Communication channels

https://mattermost.bcbl.eu/bhdonostia/channels/launchcontainer

Skills

• Python intermediate
• Unix intermediate

Onboarding documentation

https://docs.google.com/document/d/1XMmMpp3w7i2o8wVyHv_kUgoTgYVJ7n-qvgIQilfEBEU/edit?usp=sharing

What will participants learn?

• How to use Unix to run python script, navigate to directory, and edit files.
• How to use git to collaborate with other's, create branch to fix issues, pull-request
• How to use launchcontainers to launch DWI pipeline on SGE, SLURM and local machine

Data to use

I will prepare the converted nifti DWI scan folder in BIDS format and zipped it for use

Number of collaborators

3

Credit to collaborators

Project contributers will be listed in the launchcontainer repo

Image

Type

documentation coding_methods

Development status

2_releases_existing

Topic

reproducible_scientific_methods

Tools

BIDS

Programming language

Python

Modalities

DWI

Git skills

1_commit_push, 2_branches_PRs

Anything else?

No response

Things to do after the project is submitted and ready to review.

• Add a comment below the main post of your issue saying: Hi @brainhackorg/project-monitors my project is ready!
• Twitter-sized summary of your project pitch.

[yongninglei]

Hi @brainhackorg/project-monitors my project is ready!

[Remi-Gau]

hey @yongninglei
could you add or tag at least 2 issues in https://github.com/garikoitz/launchcontainers that you want to work on during the hackathon ?

for example have issues that match your "Goals for Brainhack Global" section, so that people who want to help you can at elast say they want to work on this and discuss progress / problems...

[yongninglei]

Hey @Remi-Gau
I have add a tag in the repo called for brainhackdonostia 2023 and there are 7 issues ready.

[Remi-Gau]

perfect

[yongninglei]

Hi @Remi-Gau , I attached the launchcontainer logo to this issue just now :DDD maybe it can be added to the web?

[Remi-Gau]

OK testing if relabelling the issue will just reopen a PR with image (that's me being lazy here).

[Remi-Gau]

The image appears in the project page but not as a thumbnail on the listing of projects here: http://brainhack.org/global2023/project/project_44/

Not sure why

[likeajumprope]

quick q: does the project need a localisation label? Such as donostia?

[Remi-Gau]

I think we had them on the previous years but I have not generated the labels

ideally it would be better to use the same labels as we did in the previous years to help do stats across years