Skip to content

Commit

Permalink
fix: add --parsable to sbatch call for a more robust output parsing (#…
Browse files Browse the repository at this point in the history
…125)

Following discussion in
#121 (comment)
, this implementation seems more robust than match a word with only
numbers (which could break in future versions of slurm, or if a cluster
name is made of only numbers).
  • Loading branch information
nigiord authored Aug 7, 2024
1 parent 03d474f commit 5e41d05
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 6 deletions.
2 changes: 1 addition & 1 deletion docs/further.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ You can use the following specifications:
| `--ntasks` | `tasks` | number of concurrent tasks / ranks |
| `--cpus-per-task` | `cpus_per_task` | number of cpus per task (in case of SMP, rather use `threads`) |
| `--nodes` | `nodes` | number of nodes |
| `--clusters` | `cluster` | comma separated string of clusters |
| `--clusters` | `clusters` | comma separated string of clusters |

Each of these can be part of a rule, e.g.:

Expand Down
15 changes: 10 additions & 5 deletions snakemake_executor_plugin_slurm/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,11 @@ def run_job(self, job: JobExecutorInterface):
else:
comment_str = f"rule_{job.name}_wildcards_{wildcard_str}"
call = (
f"sbatch --job-name {self.run_uuid} --output {slurm_logfile} --export=ALL "
f"sbatch "
f"--parsable "
f"--job-name {self.run_uuid} "
f"--output {slurm_logfile} "
f"--export=ALL "
f"--comment {comment_str}"
)

Expand Down Expand Up @@ -205,10 +209,11 @@ def run_job(self, job: JobExecutorInterface):
)

# multicluster submissions yield submission infos like
# "Submitted batch job <id> on cluster <name>".
# To extract the job id in this case we need to match any number
# in between a string - which might change in future versions of SLURM.
slurm_jobid = re.search(r"\d+", out).group()
# "Submitted batch job <id> on cluster <name>" by default, but with the
# --parsable option it simply yields "<id>;<name>".
# To extract the job id we split by semicolon and take the first element
# (this also works if no cluster name was provided)
slurm_jobid = out.split(";")[0]
slurm_logfile = slurm_logfile.replace("%j", slurm_jobid)
self.logger.info(
f"Job {job.jobid} has been submitted with SLURM jobid {slurm_jobid} "
Expand Down

0 comments on commit 5e41d05

Please sign in to comment.