Skip to content

Latest commit

 

History

History
54 lines (46 loc) · 2.09 KB

README.md

File metadata and controls

54 lines (46 loc) · 2.09 KB
Raw

ClusterMonitor

PyPI versionDownloads

What is it?

A very simple python script to log the CPU and Memory consumption of submitted cluster jobs.

Installation

Simplest option is to install via PyPi:

pip install ClusterMonitor

Alternatively you can also clone this repository and use the scripts inside the ClusterMonitor directory.

Usage

To start recording use the cpu_ram_log.py script. It requires 3 arguments:

  • -u which corresponds to your username.
  • -o which specifies the output file (in tsv format)
  • --interval which specifies the time (in seconds) over which the CPU and RAM usage gets averaged

Example:

python ClusterMonitor/cpu_ram_log.py -u nickhir -o cpu_ram.log --interval 5

The script can simply be included in your cluster job submittion like this:

#!/bin/bash
#SBATCH --job-name=Example_run

sambamba sort -t 3 some_bam.bam &
MainScript=$!
python cpu_ram_log.py -u nickhir -o cpu_ram.log --interval 5 &
wait $MainScript
pkill -P $$

If you used pip to install the package, cpu_ram_log and visualize_log will be added to your PATH variable, thus you can simple use:

cpu_ram_log -u nickhir --interval 5 -o cpu_ram.log

Afterwards the resulting log file can be plotted (for that matplotlib is needed). For that use the visualize_log.py script which needs 2 arguments:

  • -i which specifies the path to the log file which was created with cpu_ram_log.py
  • -o which specifies the path to the resulting PDF which contains a plot for the CPU and RAM usage.

Example:

python visualize_log.py -i cpu_ram.log -o cpu_ram_visualization.pdf

The resulting plots will look somewhat like this: exampleImage