Skip to content

Commit

Permalink
update
Browse files Browse the repository at this point in the history
- add email to author frontmatter\n- add ascii logo text file\n- version 0.1.1 declared
  • Loading branch information
Christian Ramirez committed Dec 26, 2024
1 parent 23a20f5 commit c3b775c
Show file tree
Hide file tree
Showing 5 changed files with 47 additions and 27 deletions.
9 changes: 9 additions & 0 deletions ascii_logo.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
.-') _ ('-. .-. .-')
( OO) ) ( OO ).-.\ ( OO )
,(_)----. ,--. / . --. / ;-----.\
| | | |.-') | \-. \ | .-. |
'--. / | | OO ).-'-' | | | '-' /_)
(_/ / | |`-' | \| |_.' | | .-. `.
/ /___(| '---.' | .-. | | | \ |
| || | | | | | | '--' /
`--------'`------' `--' `--' `------'
18 changes: 10 additions & 8 deletions computing_resources.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,23 @@
---
title: Computing Resources
subtitle: Technical guides for commonly used software tools
author: Christian S. Ramirez
authors:
- name: Christian S. Ramirez
email: [email protected]
---
# Secure Jupyter
## Why?
HTTPS encrypts data between your browser and JupyterLab to prevent eavesdropping/tampering, while password authentication ensures only authorized users can access your notebooks and execute code on your system.
## Setting up your HTTPS and `jupyter_server_config.py`
### Make jupyter password and `certfile` directory
```
```bash
mkdir -p ~/.jupyter/certfiles
jupyter lab --generate-config
jupyter lab password
```
### Create your private (.key) and public key (.pem)
Make sure to replace `HOSTNAME` with the machine you are on (i.e. z014).
```
```bash
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-keyout ~/.jupyter/certfiles/jupyter.key \
-out ~/.jupyter/certfiles/jupyter.pem \
Expand All @@ -24,7 +26,7 @@ chmod 600 ~/.jupyter/certfiles/jupyter.key
```
### Add config options to your `jupyter_server_conifg.py`
Make sure to replace `HOSTNAME` with the machine you are on (i.e. z014) and `USERNAME` with your username.
```
```python
c.ServerApp.certfile = '/Users/USERNAME/.jupyter/certfiles/jupyter.pem'
c.ServerApp.keyfile = '/Users/USERNAME/.jupyter/certfiles/jupyter.key'
c.ServerApp.ip = 'HOSTNAME'
Expand All @@ -37,11 +39,11 @@ c.ServerApp.notebook_dir = '/data/GROUP/USERNAME'
Now start jupyter lab using your preferred method. [See Getting Started](getting_started.md) if you are unsure. Navigate to `https://127.0.0.1:8888/lab`. If this is you first time doing this, your browser may return a self-signed certificate warning. Ignore this error, and connect to the url. Now you can login using the password you made at the beginning.

Defining our config options in `jupyter_server_config.py` allows you be less declarative when starting jupyter in the terminal. For example, when using singularity we can start jupyter lab with:
```
```bash
singularity exec --writable -B /data,/zata /zata/zippy/$(whoami)/bin/bioinformatics jupyter lab
```
As opposed to:
```
```bash
singularity exec --writable -B /data,/zata /zata/zippy/$(whoami)/bin/bioinformatics jupyter lab --port=8888 --ip=HOSTNAME --no-browser --notebook-dir=/data/GROUP/$(whoami)
```
# Screen
Expand Down Expand Up @@ -123,7 +125,7 @@ Micromamba is a standalone version of Mamba which is an alternative to Conda. Re
This wrapper script simplifies the usage of `python -m venv` for creating native python virtual environments. This is especially handy if you are using Homebrew on MacOS since, by default, Homebrew disallows the usage of pip in the global environment.
### Wrapper Script
Add the following to your `.zshrc` or `.bashrc`:
```
```bash
# Usage
# $ mkvenv myvirtualenv # creates venv with default python3
# $ mkvenv myvirtualenv 3.8 # creates venv with python3.8
Expand Down Expand Up @@ -188,7 +190,7 @@ Make sure to run `source ~/.bashrc` or `source ~/.zshrc` after adding the script
Please refer to the [getting started page](https://hpc.umassmed.edu/doc/index.php?title=Getting_started) on the HPC wiki at `hpc.umassmed.edu`. Make sure that you are connected to the UMMS network (on-premises) or [[VPN|Getting Started#computing-access##vpn-setup]].
## Useful bash aliases
Add the following to you bashrc.
```
```bash
alias gpu='bsub -q gpu -gpu "num=1:gmodel=TeslaV100_SXM2_32GB" -W 1440 -n 10 -R "rusage[mem=8000]" -R "span[hosts=1]" -Is bash'
alias cpu='bsub -q interactive -n 10 -R "rusage[mem=8000]" -R "span[hosts=1]" -Is bash'
```
Expand Down
35 changes: 19 additions & 16 deletions getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@
title: Getting Started
subtitle: Onboarding tutorials for new lab members
authors:
- name: Christian S. Ramirez
- name: Greg Andrews
- name: Christian S. Ramirez
email: [email protected]
---
# Computing Access

Expand All @@ -21,7 +22,7 @@ If on windows, you will have to install [WSL](https://learn.microsoft.com/en-us/
[WSL basic commands](https://learn.microsoft.com/en-us/windows/wsl/basic-commands)

From your powershell start your linux VM with:
```
```bash
wsl.exe -d ubuntu
```
## Update ssh config file
Expand Down Expand Up @@ -56,7 +57,9 @@ Host z010 z011 z012 z013 z014
User USERNAME
```
## Logging in
`ssh [email protected]`
```bash
ssh [email protected]
```

If it's your first time, you will be prompted to scan the QR code with any two-factor authentication app. You should also be prompted to change your password.
```{important}
Expand All @@ -67,7 +70,7 @@ I personally recommend using Microsoft Authenticator, since this is the 2FA app
From this point forward, every time you login, you will provide your password + two-factor code with no spaces.
```
From bastion, you can then `ssh` into any of the ZLab servers:
```
```bash
ssh HOSTNAME
```
Replace `HOSTNAME` with one of the hostnames defined in your ssh config file (i.e. z011, z012 ... z014).
Expand All @@ -76,7 +79,7 @@ Replace `HOSTNAME` with one of the hostnames defined in your ssh config file (i.
## Install conda
Conda is a great way to quickly install software and create separate environments for projects requiring different, and potentially conflicting, pieces of software. The conda (Miniforge3) installation instructions have been adapted from default installation instructions so it is available across all our machines.

```
```bash
curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh -p /zata/zippy/$(whoami)/Miniforge3
source ~/.bashrc
Expand All @@ -86,7 +89,7 @@ source ~/.bashrc
While this is optional, you may want to use the `mamba` dependency resolver which tends to be faster than native `conda`. You can still use conda if you are prefer, but in the following examples will use `mamba`. You may need to run `mamba init` if this is your first time using `mamba` and then update your bashrc file with `source ~/.bashrc`.

Create your first conda environment:
```
```bash
mamba create -n myenv jupyterlab numpy pandas matplotlib bedtools
```
## Singularity
Expand All @@ -97,12 +100,12 @@ Containers are useful in bioinformatics because they ensure reproducibility by p
The remote docker image `clarity001/bioinformatics` contains a full suite of bioinformatics software that you will most likely need.

Create the destination folder and build the singularity sandbox container:
```
```bash
mkdir -p /zata/zippy/$(whoami)/bin/
singularity build --sandbox /zata/zippy/$(whoami)/bin/bioinformatics docker://clarity001/bioinformatics
```
To start an interactive shell in the *writable* container (optional):
```
```bash
singularity shell --writable -B /data,/zata /zata/zippy/$(whoami)/bin/bioinformatics
```
# JupyterLab
Expand All @@ -113,21 +116,21 @@ In the following examples:

## Start a JupyterLab server using singularity
Using the sandbox container you just built, start the JupyterLab server with the following command:
```
```bash
singularity exec --writable -B /data,/zata /zata/zippy/$(whoami)/bin/bioinformatics jupyter lab --port=8888 --ip=HOSTNAME --no-browser --notebook-dir=/data/GROUP/$(whoami)
```
## Start a JupyterLab server using docker
First see [rootless docker setup](rootless_docker_setup)
```
```bash
docker container run -it --rm -p 8888:8888 --mount type=bind,src=/data,target=/data --mount type=bind,src=/zata,target=/zata clarity001/bioinformatics jupyter-lab --port=8888 --ip=* --no-browser --allow-root --notebook-dir=/data/GROUP/$(whoami)/
```
## Start a JupyterLab server in a conda environment
Activate your environment from the previous example:
```
```bash
conda activate myenv
```
Start your jupyterlab server:
```
```bash
jupyter-lab --port=8888 --ip=HOSTNAME --no-browser
```
## Accessing your JupyterLab server
Expand All @@ -148,15 +151,15 @@ You can then open your notebook server in your favorite web browser by navigatin
## Common issues with JupyterLab
### Unable to delete files
This is an issue that has come up before. To fix, you first need to find you jupyter config file. It will most likely be located at `/home/USERNAME/.jupyter`. First, change into that directory:
```
```bash
cd /home/$(whoami)/.jupyter
```
If you cannot find `jupyter_server_config.py` in that directory, run:
```
```bash
jupyter server --generate-config
```
Edit `jupyter_server_config.py` with your preferred text editor:
```
```bash
vim jupyter_server_config.py
```
Look for the config option `c.FileContentsManager.delete_to_trash`. If it is set to `True`, change it to `False`. Now you should be able to delete files in JupyterLab
Expand All @@ -165,7 +168,7 @@ Look for the config option `c.FileContentsManager.delete_to_trash`. If it is set
## Why use rootless Docker?
Rootless Docker enables users to run containers without administrator privileges, making it ideal for shared HPC environments where granting root access to multiple users would pose security risks.
## Setup
```
```bash
mkdir -p ~/.config/docker/
echo '{"data-root":"/rootless/docker/'$(whoami)'/docker"}' > ~/.config/docker/daemon.json
dockerd-rootless-setuptool.sh install
Expand Down
6 changes: 4 additions & 2 deletions learning_resources.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
---
title: Learning Resources
subtitle: Curated collection of bioinformatics learning materials
author: Christian S. Ramirez
authors:
- name: Christian S. Ramirez
email: [email protected]
---
# Foundation Skills
## Programming Essentials (Python)
Expand Down Expand Up @@ -110,4 +112,4 @@ project_name/
├── scripts/ # Workflow-specific scripts
└── logs/ # Log files
```
This is not an absolute standard to follow by any means. The above directory tree is meant more as a guideline for how to approach project organization.
This is not an absolute standard to follow. The above directory tree is meant more as a guideline for how to approach project organization.
6 changes: 5 additions & 1 deletion root.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,6 @@
# Welcome
# Welcome to the Zlab!
```{literalinclude} ascii_logo.txt
:start-line: 0
:lineno-match:
```
This wiki is intended for onboarding new students into the lab, by providing tutorials and guides about common software used in our lab and research.

0 comments on commit c3b775c

Please sign in to comment.