Instructions for Conda environment and L40S GPU node setup for Hurricane data GAN project

[caiqile]

Step by Step instructions to setup a conda environment and run model on L40S GPU node for Hurricane data GAN project

1. SSH into keeling:
   ssh -Y "netID"@keeling.earth.illinois.edu

"netID" is your Illinois account netID

2. Activate L40S GPU node
   qlogin -p l40s -N 1 -n 96 --gres=gpu:L40S:1 --mem=250000 --time=2:00:00

3. Find hostname of node: (This will be used later on in the Jupyter Notebook Server setup)
   hostname

4. Activate conda installation:
   source .bashrc

5. Create conda environment
   conda create --name "my-env"

"my-env" is the name you create for your environment

6. Install packages

Pytorch Install Instructions: https://pytorch.org/get-started/locally/
Code: conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

• Make sure pytorch build version has the word cuda in it

Jupyter Notebook: conda install jupyter notebook

Matplotlib: conda install matplotlib

Xarray: conda install -c conda-forge xarray netCDF4 h5netcdf

*** These steps below are for interactive computing ***

7. Host Jupyter Notebook Server
   jupyter notebook --port=XXXX --no-browser --ip=127.0.0.1
   Set the port number yourself to a random port number. Check the statements printed after running this to confirm if your server is on the port you selected. Copy the URL the terminal provides to access the Jupyter Notebook Server.

8. SSH into Jupyter Notebook Server (step is necessary in order to access the server when you run VS code)
   Open a new terminal window and run:
   ssh -L XXXX:127.0.0.1:XXXX [email protected] ssh -L XXXX:127.0.0.1:XXXX "hostname"

XXXX is the port number you selected, and "hostname" is the hostname you received in Step 3.

9. Open up Visual Studio Code and select the kernel to an Existing Jupyter Notebook Server. Paste the URL you copied in Step 7.

The instructions are now complete, you will be able to run the workflow on the L40S GPU node in an interactive Jupyter Notebook Session.