Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: cloud instructions #50

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 84 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,13 @@

- **[Documentation](https://probcomp.github.io/bayes3d/)**
- **[Installation](#installation-guide)**
- **[Get Assets](#get-assets)**
- **[Google Cloud Instance Setup](#gcp-setup)**
- **[Google Cloud Setup](#running-on-a-google-cloud-instance)**

# Installation Guide

Set up a fresh Python environment:
These instructions require a GPU. To run on a cloud instance, follow [Google Cloud Setup](#running-on-a-google-cloud-instance) and then return here.

First, set up a fresh Python environment:

```bash
conda create -n bayes3d python=3.9
Expand Down Expand Up @@ -106,22 +107,94 @@ To check your CUDA version:
```
nvcc --version
```
# Running on a Google Cloud Instance

Bayes3D has high compute/GPU requirements, so working in a Cloud VM is a great option. After you've set up a [Google Cloud Platform]([url](https://cloud.google.com)) account, you can follow these instructions to get up and running.

## Configuring the `gcloud` CLI

Install the [Google Cloud command line
tools](https://cloud.google.com/sdk/docs/install).

- Follow the instructions on the [installation page](https://cloud.google.com/sdk/docs/install)
- run `gcloud init` as described [in this guide](https://cloud.google.com/sdk/docs/initializing) and configure the tool with the ID of a Cloud project.

## Launching a GPU-Configured VM

To launch a Cloud VM, run the following commands at your terminal:

```bash
export ZONE="us-west1-b"
export INSTANCE_NAME="bayes3d-template"

gcloud compute instances create $INSTANCE_NAME \
--zone=$ZONE \
--image-family="common-gpu-debian-11-py310" \
--image-project=deeplearning-platform-release \
--maintenance-policy=TERMINATE \
--boot-disk-size=300GB \
--machine-type n1-standard-8 \
--accelerator="type=nvidia-tesla-v100,count=1" \
--metadata="install-nvidia-driver=True" \
--scopes=https://www.googleapis.com/auth/cloud-platform
```

- Make sure to customize `INSTANCE_NAME`, these are shared across the project / region.
- You may need to increase `--boot-disk-size`, but don't go lower.

# GCP Setup
Of course you can customize anything you like, but don't change the
`--image-project` or `--image-family` arguments.

- Start new VM instance (see
[link](https://cloud.google.com/compute/docs/instances/create-start-instance)).
Select GPU - NVIDIA V100 and Machine Type 8vCPU 4 Core 30GB.
## Accessing the VM

-From the VM instances page, searched for public image `c2-deeplearning-pytorch-2-0-gpu-v20230925-debian-11-py310`. Increase storage to 1000GB.
After a few minutes your VM will be available for access via SSH. You can reach
a terminal in a few different ways:

- Note that public image names get updated frequently, so it is possible you may not find the one mentioned above. To find the latest public image, go to the [public list](https://cloud.google.com/compute/docs/images#console), and look for an image as close to the one above (Debian 11, CUDA 11.8, Python 3.10, Pytorch 2.0 etc.).
- Locate your image on the [Cloud
Console](https://console.cloud.google.com/compute/instances) and click the
"SSH" button
- Log in via the `gcloud` command line tool with the following command:

- SSH into instance and when prompted, install the NVIDIA drivers.
```bash
# These environment variables were set in the code block above:
gcloud compute ssh --zone $ZONE $INSTANCE_NAME
```

- Configure your `ssh` credentials so the normal `ssh` command works by running

```bash
gcloud compute config-ssh
```

- Follow [installation guide](#installation-guide).
Then you should be able to log in like:

```bash
ssh $INSTANCE_ID.$ZONE.$PROJECT_ID
```

The `gcloud compute config-ssh` command needs to be re-run after instances have been stopped/started, as they are often assigned new IP addresses.

## Port-forwarding from a VM

Configure your ssh credentials:

```bash
gcloud compute config-ssh
```

Then `ssh` into the instance using this command:

```bash
ssh $INSTANCE_ID.$ZONE.$PROJECT_ID -L <local_port>:localhost:<remote_port>
```

For example, to forward port 8888 on the VM to my local port 8888:

```
ssh my-image.us-west1-b.probcomp-caliban -L 8888:localhost:8888
```

---
## License

Distributed under the [Apache 2.0](LICENSE) license. See [LICENSE](LICENSE).
Expand Down
Loading