XSEDE 2013 SAGA Tutorial

Preparation

This tutorial is part of the XSEDE13 SAGA-BigJob Tutorial will use the TACC Virtual Machine (VM) repex1. Log-in via SSH using the username/password you received at the beginning of the tutorial:

ssh <username>@repex1.tacc.utexas.edu

Installation

Next, you need to install SAGA-Python in your user account on repex1. Since saga-python - as the name suggests - is written in Python, you can use virtualenv to create a local installation:

virtualenv $HOME/sagaenv
. $HOME/sagaenv/bin/activate

The SAGA-Python package on PyPi (Python Package Index) that we are using is called saga-python and can be installed via pip:

pip install saga-python

Please, validate the installation by typing:

$ python -c "import saga; print saga.version"
0.9.9

Code Example 1: Local Job Submission

Preparation

1. Take a look at the full example code on GitHub.

2. Create a new file in your home directory, copy & paste the code into it and save it, e.g., as saga_example_local.py.

Execution

Execute the Python script:

python saga_example_local.py

The output will look something like this:

Job ID    : None
Job State : New

...starting job...

Job ID    : [fork://localhost]-[31981]
Job State : Done

...waiting for job...

Job State : Done
Exitcode  : 0

Code Example 2: Remote Job Submission

Preparation

1. Take a look at the full example code on GitHub.

2. Create a new file in your home directory, copy & paste the code into it and save it, e.g., as saga_example_remote.py.

Execution

Execute the Python script:

python saga_example_remote.py

The output will look something like this:

Job ID    : None
Job State : New

...starting job...

Job ID    : [slurm+ssh://stampede.tacc.utexas.edu]-[32704]
Job State : Done

...waiting for job...

Job State : Done
Exitcode  : 0

Discussion

Where is your output? Let's login to Stampede - ssh [email protected] and cd into the working directory that we just created.

According to the code, our working directory is: jd.working_directory = "/home1/02554/sagatut/XSEDETutorial/%s/SAGA" % USER_NAME

Note that USER_NAME corresponds to your tutorial account on repex1, i.e. if you logged in as tutorial-04, the path on Stampede would be: /home1/02554/sagatut/XSEDETutorial/tutorial-04/SAGA

We told saga-python to name our output files:

    jd.output            = "examplejob.out"
    jd.error             = "examplejob.err"

SAGA-Python provides many other backend adaptors besides slurm, ssh, and fork. Check out the adaptor section of the documentation if you want to learn more: http://saga-project.github.io/saga-python/doc/adaptors/saga.adaptor.index.html.

Code Example 3: Remote Job Submission + File Staging

Preparation

1. Take a look at the full example code on GitHub.

2. Create a new file in your home directory, copy & paste the code into it and save it, e.g., as saga_example_remote_staging.py.

Execution

Execute the Python script:

python saga_example_remote_staging.py

The output will look something like this:

Job ID    : None
Job State : New

...starting job...

Job ID    : [slurm+ssh://stampede.tacc.utexas.edu]-[4319]
Job State : Done

...waiting for job...

Job State : Done
Exitcode  : 0

Staged out sftp://[email protected]/tmp/mysagajob-tutorial-00.stdout to file:///home/tutorial-00/ (size: 16 bytes)

You will find a .stdout file in your working directory that contains the output of your job.

Discussion

This example shows you how to stage data out (i.e., back) automatically once the job is finished. Here is a link for more information.

Code Example 4: Sequentially Executing Many Tasks (Mandelbrot Set)

Please refer to the instructions and discussion in the SAGA-Python documentation: Mandelbrot Example