Skip to content

Commit

Permalink
updated submission.md et FAQ"
Browse files Browse the repository at this point in the history
  • Loading branch information
Nicolas HOMBERG committed Nov 6, 2024
1 parent 3e2562d commit 17d213d
Show file tree
Hide file tree
Showing 2 changed files with 56 additions and 53 deletions.
52 changes: 51 additions & 1 deletion phase-0-smoothie/bundle/FAQ.md
Original file line number Diff line number Diff line change
@@ -1 +1,51 @@
To complete
# Problems with creating or submiting your code ? Try the following :


## Submissions fail ? -> Have a look at the logs on Codabench

If your submission fails on Codabench, don't panic—review the logs! You can access them via the Codabench user interface:

1. Go to the "Submission" tab and scroll to the end of the page to find your submission.
2. Click on the row corresponding to your submission.
3. Select the "Logs" tab.
4. Browse the various types of logs available: `stdout`, `stderr`, `Ingestion stdout`, and `Ingestion stderr`, for both the ingestion and scoring steps.

## Cannot create your submissions ? Try our conda environnement: (Especially useful for python user !)

A conda evnironnement is provided follow this steps to install and activate it:
- retrieve the file environment-r.yml from github : [environment-r.yml](https://github.com/bcm-uga/hadaca3/blob/main/docker/codabench_hadaca3_pyr/environment/environment-r.yml) :
- `wget https://raw.githubusercontent.com/bcm-uga/hadaca3/main/docker/codabench_hadaca3_pyr/environment/environment-r.yml`
- and run: `conda env create -f environment-r.yml && conda activate h3`



## Running Submissions ingestion and scoring locally with Docker

To speed up the debugging process and avoid the full submission workflow, you can run your submission program locally using **FAKE** data. The fake data serves two purposes: first, it allows testing without relying on the actual scoring program (which requires the ground truth), and second, it speeds up computation.

To test locally, follow these steps:

1. Clone the challenge repository:
```
git clone https://github.com/bcm-uga/hadaca3.git
cd hadaca3
```
2. Set up the environment and activate it:
```
conda env create -f environment-r.yml && conda activate h3
```
3. Run the automated Docker test script:
```
sh automated_docker_test.sh
```

Please note that this script will use the `submission_script.R` from the `hadaca3/starting_kit/` folder. **Ensure that your script is placed inside the `hadaca3/starting_kit/` folder or modify the existing script at `hadaca3/starting_kit/submission_script.R`**.


The script `automated_docker_test.sh` first executes the R submission script locally, then re-executes it within the Docker container (ingestion phase), followed by running the scoring program.

If you wish to test each step independently or run the Python script locally, refer to the `README.md` in the `hadaca3` GitHub repository.

## Include an external file to be used in the submission script
It is possible to include an external file that is sourced in the submission script and correctly included in the zip file to be submitted on the Codabench platform.
Beware, when unziped the file 'program.R' has to be on the current directory and not inside a folder. Aditionnal files can in other folder with the correct relative path
57 changes: 5 additions & 52 deletions phase-0-smoothie/bundle/submission.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
If this page does not answer your questions try the FAQ page :)

## How to generate a prediction of the data (R)?

[1] On your local machine, unzip the starting_kit.zip.
Expand All @@ -13,7 +15,7 @@ The unziped starting_kit directory contains now:
- From terminal run `Rscript submission_script.R`
- Launch an R console and execute the following command : `source("submission_script.R")`
- Run the `submission_script.R` in RStudio/ Vscode
- From docker :`sudo docker run -v .:/hadaca3 -w /hadaca3 hombergn/hadaca3_pyr Rscript submission_script.R`
- From docker :`sudo docker run -v .:/hadaca3 -w /hadaca3 hombergn/hadaca3_pyr Rscript submission_script.R` and regain ownership of the files generated with `sudo chown -R $USER submissions`


[3] The code of the `submission_script.R` generates the files:
Expand All @@ -35,7 +37,7 @@ Edit the code inside the following chunk (i.e. the `program` function):


## How to generate a prediction of the data (Python)?
Please note that this challenge was primarily designed with R in mind, and as such, R functionality and compatibility have been tested more extensively than those in Python. When using Python, keep in mind that your scripts will need to interact with R data, which requires the rpy2 library. To facilitate this, a Conda environment has been provided. For more details, please refer to the "Conda Environment" chapter under "Troubleshooting".
Please note that this challenge was primarily designed with R in mind, and as such, R functionality and compatibility have been tested more extensively than those in Python. When using Python, keep in mind that your scripts will need to interact with R data, which requires the rpy2 library. To facilitate this, a Conda environment has been provided. For more details, please refer to the "Conda Environment" chapter under the "FAQ" page.

[1] On your local machine, unzip the starting_kit.zip.

Expand All @@ -50,7 +52,7 @@ The unzipped starting_kit directory contains now:
- From terminal run `python submission_script.py`
- Launch an interactive python shell by executing : `python -i submission_script.py`
- Run the `submission_script.py` in Spider/ Vscode
- From docker : `sudo docker run -v .:/hadaca3 -w /hadaca3 hombergn/hadaca3_pyr python submission_script.py`
- From docker : `sudo docker run -v .:/hadaca3 -w /hadaca3 hombergn/hadaca3_pyr python submission_script.py` and regain ownership of the files generated with `sudo chown -R $USER submissions`


[3] The code of the `submission_script.R` generates the files:
Expand Down Expand Up @@ -96,52 +98,3 @@ By clicking on your submission in the submissions summary table, you will access
- a submission metadata edition menu

[3] Check the leaderboard in the *Results* menu

## Troubleshooting

### Conda environnement:

A conda evnironnement is provided follow this steps to install and activate it:
- retrieve the file environment-r.yml from github : [environment-r.yml](https://github.com/bcm-uga/hadaca3/blob/main/docker/codabench_hadaca3_pyr/environment/environment-r.yml) :
- `wget https://raw.githubusercontent.com/bcm-uga/hadaca3/main/docker/codabench_hadaca3_pyr/environment/environment-r.yml`
- and run: `conda env create -f environment-r.yml && conda activate h3`


### Logs on Codabench

If your submission fails on Codabench, don't panic—review the logs! You can access them via the Codabench user interface:

1. Go to the "Submission" tab and scroll to the end of the page to find your submission.
2. Click on the row corresponding to your submission.
3. Select the "Logs" tab.
4. Browse the various types of logs available: `stdout`, `stderr`, `Ingestion stdout`, and `Ingestion stderr`, for both the ingestion and scoring steps.

### Running Submissions ingestion and scoring locally with Docker

To speed up the debugging process and avoid the full submission workflow, you can run your submission program locally using **FAKE** data. The fake data serves two purposes: first, it allows testing without relying on the actual scoring program (which requires the ground truth), and second, it speeds up computation.

To test locally, follow these steps:

1. Clone the challenge repository:
```
git clone https://github.com/bcm-uga/hadaca3.git
cd hadaca3
```
2. Set up the environment and activate it:
```
conda env create -f environment-r.yml && conda activate h3
```
3. Run the automated Docker test script:
```
sh automated_docker_test.sh
```

Please note that this script will use the `submission_script.R` from the `hadaca3/starting_kit/` folder. **Ensure that your script is placed inside the `hadaca3/starting_kit/` folder or modify the existing script at `hadaca3/starting_kit/submission_script.R`**.


The script `automated_docker_test.sh` first executes the R submission script locally, then re-executes it within the Docker container (ingestion phase), followed by running the scoring program.

If you wish to test each step independently or run the Python script locally, refer to the `README.md` in the `hadaca3` GitHub repository.

### Including an external file to be used in the submission script
It is possible to include an external file that is sourced in the submission script and correctly included in the zip file to be submitted on the Codabench platform.

0 comments on commit 17d213d

Please sign in to comment.