Skip to content

Commit

Permalink
Add salloc steps in Virtual Environment section RT 514756
Browse files Browse the repository at this point in the history
  • Loading branch information
yulgroper committed Mar 12, 2024
1 parent 4efb87c commit d64840a
Showing 1 changed file with 11 additions and 9 deletions.
20 changes: 11 additions & 9 deletions doc/speed-manual.tex
Original file line number Diff line number Diff line change
Expand Up @@ -531,17 +531,17 @@ \subsection{Creating Virtual Environments}
Virtual environments typically instantiated via Conda or Python.
Another option is Singularity detailed in \xs{sect:singularity-containers}.
%
Usually, virtual environments are created once and before sending any job to the
scheduler, so when sending the job to the scheduler we (1) activate the virtual environment,
(2) use it, and (3) close it at the end of the job.

Usually, virtual environments are created once during an interactive session before submitting
a batch job to the scheduler. The job script submitted to the scheduler is then written to
(1) activate the virtual environment, (2) use it, and (3) close it at the end of the job.

% ------------------------------------------------------------------------------
\subsubsection{Anaconda}
\label{sect:conda-venv}

To create an anaconda environment in your speed-scratch directory, use the \texttt{\-\-prefix}
option when executing \texttt{conda create}. For example, to create an anaconda environment for
Request an interactive session in the queue you wish to submit your jobs to (e.g., salloc -p pg --gpus=1 for GPU jobs).
Once your interactive has started, create an anaconda environment in your speed-scratch directory by using
the \texttt{\-\-prefix} option when executing \texttt{conda create}. For example, to create an anaconda environment for
\texttt{a\_user}, execute the following at the command line:

\begin{verbatim}
Expand Down Expand Up @@ -586,8 +586,8 @@ \subsubsection{Anaconda}
A consolidated example using Conda:
\small
\begin{verbatim}
salloc -p pg --gpus=1 --mem=10GB -A <slurm account name>
cd /speed-scratch/$USER
srun --partition=p(s/g) -A Your_account --mem=10Gb --gpus=1 --pty /encs/bin/tcsh
module load python/3.11.0/default
conda create -p /speed-scratch/$USER/pytorch-env
conda activate /speed-scratch/$USER/pytorch-env
Expand All @@ -596,7 +596,7 @@ \subsubsection{Anaconda}
https://download.pytorch.org/whl/cu117
....
conda deactivate
exit
exit # end the salloc session
\end{verbatim}
\normalsize
\end{itemize}
Expand All @@ -611,15 +611,17 @@ \subsubsection{Python}
\label{sect:python-venv}

Setting up a Python virtual environment is fairly straightforward.
The first step is to request an interactive session in the queue you wish to submit your jobs to.

We have a simple example that use a Python virtual environment:

\begin{itemize}
\item
Using Python Venv
\small
\begin{verbatim}
salloc -p pg --gpus=1 --mem=10GB -A <slurm account name>
cd /speed-scratch/$USER
srun --partition=p(s/g) -A Your_account --mem=10Gb --gpus=1 --pty /encs/bin/tcsh
module load python/3.9.1/default
mkdir -p /speed-scratch/$USER/tmp
setenv TMPDIR /speed-scratch/$USER/tmp
Expand Down

0 comments on commit d64840a

Please sign in to comment.