diff --git a/README.md b/README.md index 0c69a28..859ba5e 100644 --- a/README.md +++ b/README.md @@ -32,6 +32,7 @@ Linux Ubuntu. | Status | Client OS | Server hardware | Runner | | --- | --- | --- | --- | | :heavy_check_mark: Completed | Linux Ubuntu | local machine via Docker | [link](/ubuntu-docker/README.md) | +| :heavy_check_mark: Completed | Linux Ubuntu | local machine via Singularity | [link](/ubuntu-singularity/README.md) | | :heavy_check_mark: Completed | Linux Ubuntu | local machine via Vagrant | [link](/ubuntu-vagrant/README.md) | | :heavy_check_mark: Completed | Linux Ubuntu | local machine via VirtualBox | [link](/ubuntu-virtualbox/README.md) | | :heavy_check_mark: Completed | Linux Ubuntu | remote machine at [SURF HPC Cloud] | [link](/ubuntu-surf-hpc-cloud/README.md) | diff --git a/ubuntu-docker/Dockerfile b/ubuntu-docker/Dockerfile index 5e17304..59af782 100644 --- a/ubuntu-docker/Dockerfile +++ b/ubuntu-docker/Dockerfile @@ -8,16 +8,16 @@ ENV RUNNER_VERSION=2.267.1 ENV AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache RUN apt-get update && \ - apt-get install -y --no-install-recommends ca-certificates \ - curl \ - jq \ - libcurl4-openssl-dev \ - sudo \ - tar && \ + apt-get install -y --no-install-recommends \ + ca-certificates \ + curl \ + jq \ + libcurl4-openssl-dev \ + tar && \ rm -rf /var/lib/apt/lists/* RUN printf "\n\033[0;44m---> Creating SSH user.\033[0m\n" -RUN useradd -rm -d /home/${DOCKER_USER} -s /bin/bash -G sudo -u 1001 ${DOCKER_USER} +RUN useradd -rm -d /home/${DOCKER_USER} -s /bin/bash -G ${DOCKER_USER} -u 1001 ${DOCKER_USER} RUN echo "${DOCKER_USER}:${DOCKER_PASS}" | chpasswd WORKDIR /home/${DOCKER_USER}/actions-runner diff --git a/ubuntu-singularity/README.md b/ubuntu-singularity/README.md new file mode 100644 index 0000000..fc4e479 --- /dev/null +++ b/ubuntu-singularity/README.md @@ -0,0 +1,228 @@ +# Setting up a CI server for a GitHub Action runner Singularity from Linux Ubuntu + +After following this guide, you'll have a simple GitHub action workflow on a GitHub repository of your choice. When new commits are made to your repository, the workflow delegates work to a server which runs in a [Singlularity](https://sylabs.io/singularity/) container. You can use this Singularity container to on a HPC cluster as a regular user and you will not need root permissions. + +This guide distinguishes between the _client_ and the _server_; the client is your own machine; the server is whichever +machine will run the tests. This document describes the case where the server is a Singularity container running on your own machine. + +For guides on how to configure other features in addition to just the runner, go [here](/README.md). + +## Prerequisites + +1. Install Singularity: https://sylabs.io/guides/3.5/user-guide/quick_start.html#quick-installation-steps + +2. Follow a [short tutorial](https://sylabs.io/guides/3.5/user-guide/quick_start.html#overview-of-the-singularity-interface) (Optional) + +### Testing your Singularity setup + +Check Singularity version +```shell +> singularity version + +3.5.3 +``` + +Pull an example image + +```shell +singularity pull library://sylabsed/examples/lolcow +``` + +Start a shell in the Singlarity container + +```shell +singularity shell lolcow_latest.sif +``` + +Run some test commands + +```shell +Singularity> id + +uid=1000(fdiblen) gid=985(users) groups=985(users),98(power),108(vboxusers),972(docker),988(storage),998(wheel) + +Singularity> hostname + +archlinux +``` + +## Server side configuration + +### Build image + +Now we are ready to build our Singularity image. The following command will use [Definition file](github-actions-runner-singularity.def) to build the image. It will create a system install necessary system packages and dependencies for the runner. In order to create a Singularity image, you will need root permission (or sudo) on your system. + +```shell +sudo singularity build github-actions-runner-singularity.sif github-actions-runner-singularity.def +``` + +This command will generate ``github-actions-runner-singularity.sif`` (SIF stands for Singularity Image Format) image which we will use to set up the runner. + +## Client side configuration + +### Generate an OAuth token + +We're almost ready to use our Docker image to set up a GitHub Runner, but first we need to +generate an OAuth token, as follows: + +1. Go to [https://github.com/settings/tokens](https://github.com/settings/tokens) and click the ``Generate new token`` button. +2. Provide your GitHub password when prompted +3. Fill in a description for the token, for example _GitHub runner for github.com/<your organization>/<your repository>_ +4. Enable the ``repo`` scope and all of its checkboxes, like so: + + ![Token permissions](/images/token_permissions.png) + +5. Click ``Generate`` at the bottom. Make sure to copy its value because we'll need it in the next step + +### Run the server + +#### Preperation +Before using the Singularity image we need to set some environment variables. The Singularity container will use these environment variables to set up the runner. + +```shell +export SINGULARITYENV_PERSONAL_ACCESS_TOKEN="" +export SINGULARITYENV_RUNNER_NAME="" +export SINGULARITYENV_RUNNER_WORKDIR="/tmp/actions-runner-repo" +export SINGULARITYENV_GITHUB_ORG="" +export SINGULARITYENV_GITHUB_REPO="" +``` + +Create an envionment file which will be user by the runner to save some variables. + +```shell +cp env.template env +``` + +#### Instance mode + +Alternatively, you can start it as an instance (service). + +```shell +singularity instance start github-actions-runner-singularity.sif github-actions-runner --writable-tmpfs --bind ./env:/opt/actions-runner/.env +``` + +For more information about Singularity services see [this link](https://sylabs.io/guides/3.5/user-guide/running_services.html). + + +To list the running instances: + +```shell +singularity instance list +``` + +To stop the running Singularity instance: + +```shell +singularity instance stop github-actions-runner +``` + +To start the Singularity instance again: + +```shell +singularity instance start github-actions-runner +``` + +#### Temporary mode + +Now we can run Singularity container with the following command. + +```shell +singularity run \ + --writable-tmpfs \ + --bind ./env:/opt/actions-runner/.env \ + github-actions-runner-singularity.sif +``` + +Singularity containers by-default starts in ``read-only`` mode so you cannot make changes. While setting up the runner, some scripts needs to create a few files so we need a write access. This is achieved by adding ``--writable-tmpfs`` argument. + +If you stop the running container or interrupt it by pressing to ``CTRL+C``, the Github actions runner will stop and it will be unregistered from your Github repository. + +#### Accessing the logs + +The singularity instances save the logs in +`~/.singularity/instances/logs/INSTANCE_NAME` folder. + +## Using on a HPC Cluster + +In most of the cases, the HPC user does not have root persmissions s it wont be possible to build the Singularity image. The Singularity image can be built locally and copied to +the cluster. We will use `scp` command to copy the singularity image. + +```shell +scp github-actions-runner-singularity.sif USERNAME@CLUSTER_IP_ADDRESS:$REMOTE_FOLDER +``` + +The `$REMOTE_FOLDER` is typically your home folder which you can figure out using the command below. + +```shell +echo $HOME +``` + +In oder to use the Singularity image on a HPC cluster, we first need to create a jobscript. The job script will be handled by the scheduler and eventually it will set up the runner when it is executed. + + +Example: +``` +#!/bin/bash +#SBATCH -t 1:00:00 +#SBATCH -n 480 + +cd $HOME/work + +export SINGULARITYENV_PERSONAL_ACCESS_TOKEN="" +export SINGULARITYENV_RUNNER_NAME="" +export SINGULARITYENV_RUNNER_WORKDIR="/tmp/actions-runner-repo" +export SINGULARITYENV_GITHUB_ORG="" +export SINGULARITYENV_GITHUB_REPO="" + +srun singularity run \ + --writable-tmpfs \ + github-actions-runner-singularity.sif +``` + +To submit the job script using ``sbatch``: + +```shell +sbatch jobscript +``` + +## GPU support + +See [Singularity documentation](https://sylabs.io/guides/3.5/user-guide/gpu.html) + +## Limitations + +GitHub owned servers have some pre-installed software (see [here](https://docs.github.com/en/actions/reference/software-installed-on-github-hosted-runners)). Self hosted runners are able to download and use these software. However, this is only possible if self-hosted runner is being run on one of the supported platforms such as Ubuntu. Otherwise, users have to install the required software by themselves. +Also actions based on Docker images will not work as running Docker containers inside a Singularity container does not work. + +### Extras + +#### Get Singularity image details + +Use `singularity inspect` to display details of the image + +```shell +> singularity inspect github-actions-runner-singularity.sif* +WARNING: No SIF metadata partition, searching in container... +org.label-schema.build-date: Monday_6_July_2020_16:55:58_CEST +org.label-schema.schema-version: 1.0 +org.label-schema.usage.singularity.deffile.bootstrap: library +org.label-schema.usage.singularity.deffile.from: ubuntu:19.10 +org.label-schema.usage.singularity.deffile.mirrorurl: http://us.archive.ubuntu.com/ubuntu/ +org.label-schema.usage.singularity.deffile.osversion: eoan +org.label-schema.usage.singularity.deffile.stage: build +org.label-schema.usage.singularity.version: 3.5.3 +``` + +#### Accessing Singularity container + +If you need access to a shell on the running Singularity container: + +```shell +singularity shell \ + --writable-tmpfs \ + github-actions-runner-singularity.sif +``` + +### What's next + +Find instructions for provisioning additional functionality [here](../README.md). diff --git a/ubuntu-singularity/entrypoint.sh b/ubuntu-singularity/entrypoint.sh new file mode 100644 index 0000000..32b1bbe --- /dev/null +++ b/ubuntu-singularity/entrypoint.sh @@ -0,0 +1,40 @@ +#!/bin/bash + +export RUNNER_USERNAME=$(id -un) +export RUNNER_USERGROUP=$(id -gn) + +export AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache + +cd /opt/actions-runner + +if [[ -z "${RUNNER_NAME}" ]]; then + RUNNER_NAME="singularity-$(hostname)" +fi + +ACTIONS_URL="https://api.github.com/repos/${GITHUB_ORG}/${GITHUB_REPO}/actions/runners/registration-token" +echo "Requesting registration URL at '${ACTIONS_URL}'" + +PAYLOAD=$(curl -sX POST -H "Authorization: token ${PERSONAL_ACCESS_TOKEN}" ${ACTIONS_URL}) +export RUNNER_TOKEN=$(echo $PAYLOAD | jq .token --raw-output) + +printf "\n\033[0;44m---> Configuring the runner.\033[0m\n" +./config.sh \ + --name ${RUNNER_NAME} \ + --token ${RUNNER_TOKEN} \ + --url https://github.com/${GITHUB_ORG}/${GITHUB_REPO} \ + --work ${RUNNER_WORKDIR} \ + --labels "singularity,github" \ + --unattended \ + --replace + +remove_runner() { + printf "\n\033[0;44m---> Removing the runner.\033[0m\n" + ./config.sh remove --unattended --token "${RUNNER_TOKEN}" +} + +# run remove_runner function if "./run.sh" script is interrupted +trap "remove_runner" EXIT SIGINT SIGTERM KILL + +printf "\n\033[0;44m---> Starting the runner.\033[0m\n" +./run.sh "$*" & +wait $! diff --git a/ubuntu-singularity/env.template b/ubuntu-singularity/env.template new file mode 100644 index 0000000..16c0740 --- /dev/null +++ b/ubuntu-singularity/env.template @@ -0,0 +1 @@ +AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache diff --git a/ubuntu-slurm/ga-runner-singularity.def b/ubuntu-singularity/github-actions-runner-singularity.def similarity index 55% rename from ubuntu-slurm/ga-runner-singularity.def rename to ubuntu-singularity/github-actions-runner-singularity.def index eeba2e5..91a11c0 100644 --- a/ubuntu-slurm/ga-runner-singularity.def +++ b/ubuntu-singularity/github-actions-runner-singularity.def @@ -1,21 +1,21 @@ Bootstrap: library -From: ubuntu:19.10 +From: ubuntu:20.04 Stage: build -# %setup -# touch /file1 -# touch ${SINGULARITY_ROOTFS}/file2 %files ./entrypoint.sh / - # /entrypoint.sh / %environment export LC_ALL=C export DEBIAN_FRONTEND=noninteractive export RUNNER_VERSION=2.267.1 + export AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache %post + export RUNNER_VERSION=2.267.1 + chmod +x /entrypoint.sh + apt-get install -y --no-install-recommends software-properties-common add-apt-repository universe add-apt-repository multiverse @@ -24,7 +24,16 @@ Stage: build ca-certificates \ curl \ jq \ + git \ libcurl4-openssl-dev + + echo ${RUNNER_VERSION} + mkdir /opt/actions-runner && cd /opt/actions-runner + curl -L "https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz" > actions_runner.tar.gz && \ + tar -zxf actions_runner.tar.gz && \ + rm -f actions_runner.tar.gz && \ + ./bin/installdependencies.sh + NOW=`date` echo "export NOW=\"${NOW}\"" >> $SINGULARITY_ENVIRONMENT @@ -32,22 +41,3 @@ Stage: build echo "Container was created $NOW" echo "Arguments received: $*" exec /bin/bash /entrypoint.sh "$@" - -# %startscript -# nc -lp $LISTEN_PORT - -# %test -# grep -q NAME=\"Ubuntu\" /etc/os-release -# if [ $? -eq 0 ]; then -# echo "Container base is Ubuntu as expected." -# else -# echo "Container base is not Ubuntu." -# fi - -# %labels -# Author d@sylabs.io -# Version v0.0.1 - -# %help -# This is a demo container used to illustrate a def file that uses all -# supported sections. diff --git a/ubuntu-slurm/README.md b/ubuntu-slurm/README.md deleted file mode 100644 index 9a1e0b5..0000000 --- a/ubuntu-slurm/README.md +++ /dev/null @@ -1,280 +0,0 @@ -# Setting up a CI server for a GitHub Action runner on a HPC cluster from Linux Ubuntu - -After following this guide, you'll have a simple GitHub action workflow on a GitHub repository of your choice. When new -commits are made to your repository, the workflow delegates work to a server which runs in a HPC cluster.The guide assumes that you have an acess to a HPC cluster which uses `slurm` as a scheduler. - -This guide distinguishes between the _client_ and the _server_; the client is your own machine; the server is whichever -machine will run the tests. This document describes the case where the server is a HPC cluster. - -For guides on how to configure other features in addition to just the runner, go [here](/README.md). - -## Prerequisites - -1. Install singularity - -```shell -> singularity version - -3.5.3 -``` - -## Server side configuration - -E.g. how to configure VirtualBox, how to run docker container, how to configure HPC cloud machine - -### Install SSH Client - -e.g. - -- sudo apt install openssh-client -- install putty -- homebrew install ssh - -### Generate SSH key pair - -Generate a key pair (files ``id_rsa`` and ``id_rsa.pub``) in directory -[``something/something``](something/something) using RSA encryption: - -**Note: ``id_rsa`` is the private half of the SSH key pair; don't share it with anybody else.** - -**e.g.** - -```shell -cd something/something/ -ssh-keygen -t rsa -f ./id_rsa -N '' -``` -Make sure that the permissions are set correctly: - -```shell -chmod 600 id_rsa -chmod 644 id_rsa.pub -``` - -Note you can use ``stat``'s ``%a`` option to see a file's permissions as an octal number, e.g. - -```shell -stat -c "%a %n" -stat -c "%a %n" `ls -1` -``` - - -### Copy the key pair to server - -Copy the public half of the key pair (i.e. ``id_rsa.pub``) to the server. - -**e.g.** - -```shell -ssh-copy-id -i ./id_rsa.pub -p 2222 tester@127.0.0.1 -``` - - -### Test connection with server using ``ssh`` - -Test if you can SSH into the server using the other half of the key pair (i.e. ``id_rsa``) - -**e.g.** - -```shell -ssh -i ./id_rsa -p 2222 tester@127.0.0.1 -``` - -If you get a ``Host key verification failed`` error, clear the existing key with - -```shell -ssh-keygen -R "[127.0.0.1]:2222" -``` - -and try again. - - -Log out of the server with - -```shell -exit -``` - -### Troubleshooting SSH - -Getting SSH connections to work can be tricky. Check out [this document](/docs/troubleshooting-ssh.md) if you're -experiencing difficulties. - -### The inventory file - -Ansible uses so-called _inventory_ files to define how to connect to remote machines. The inventory file is typically -called ``hosts``. The following inventory file is equivalent to the ``ssh`` command line we just used: - -```yaml -all: - hosts: - ci-server: - ansible_connection: ssh - ansible_host: 127.0.0.1 - ansible_port: 2222 - ansible_ssh_private_key_file: ./id_rsa - ansible_user: tester -``` - -This inventory file defines a group ``all`` with just one machine in it, which we labeled ``ci-server``. ``ci-server`` -has a bunch of variables that define how to connect to it. For more information on inventory files, read -[Ansible's documentation](https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html). - -### The Ansible configuration file - -In addition to the inventory file, it's often convenient to use a configuration file. The default filename for this file -is ``ansible.cfg``, and it can be used to specify Ansible's behavior. The configuration option documentation can be -found [here](https://docs.ansible.com/ansible/latest/reference_appendices/config.html#the-configuration-file). - -### Test connection with server using ``ansible`` - -We're about ready to test if we can connect to the server using Ansible. For this we will use the ``ping`` module, and -we'll instruct Ansible to run the module on all hosts as defined in the inventory, as follows: - -```shell -ansible all -m ping -``` - -Which should return: - -```text -ci-server | SUCCESS => { - "changed": false, - "ping": "pong" -} -``` - -### Install the runner using the playbook - -For more complicated tasks than ``ping``, it's often inconvenient having to put everything on the command line. Instead, -a better option is to create a so-called _playbook_ containing all the steps that you want to include in your -provisioning. The playbook is a YAML file that defines a series of ``tasks``. When creating new tasks, one can start -from scratch, or make use of tasks that have been published by others (see [https://galaxy.ansible.com/](https://galaxy.ansible.com/)). - -We're almost ready to use ``ansible-playbook`` to set up a GitHub Runner on your own server, but first we need to -generate an OAuth token, as follows: - -1. Go to [https://github.com/settings/tokens](https://github.com/settings/tokens) and click the ``Generate new token`` button. -1. Provide your GitHub password when prompted -1. Fill in a description for the token, for example _GitHub runner for github.com/<your organization>/<your repository>_ -1. Enable the ``repo`` scope and all of its checkboxes, like so: - - ![Token permissions](/images/token_permissions.png) - -1. Click ``Generate`` at the bottom. Make sure to copy its value because we'll need it in the next step - -Configuring your server such that it can run continuous integration requires 4 pieces of information, for which you will be prompted: - -1. Because our playbook requires elevated permissions, the command uses the ``--ask-become-pass`` option to prompt for -the root password. Fill in the password ``password`` to become ``root`` in the server. -1. Fill in the GitHub organization (which might be simply your GitHub user name) and ... -1. ...the repository name for which you want to run workflows on a self-hosted server -1. Finally, you need to supply the Personal Access Token - -Now run this command to provision the GitHub Action runner on your server: - -```shell -ansible-playbook playbook.yml --ask-become-pass -``` - -If you now go to GitHub [https://github.com/<your organization>/<your repository>/settings/actions](https://github.com/%3Cyour%20organization%3E/%3Cyour%20repository%3E/settings/actions), -you should see a self-hosted runner with status "Idle": - -![Self hosted runner status is Idle](/images/github-self-hosted-runners-status-idle.png) - -Add the following simple workflow as ``.github/workflows/self_hosted_ci.yml`` in your repository: - -```yaml -name: Self-hosted CI example - -on: [push, pull_request] - -jobs: - test: - name: test - runs-on: self-hosted - steps: - - name: Show directory listing - shell: bash -l {0} - run: | - ls -la -``` - -Now try making a change to one of the files in your repository to see if you can trigger running the simple workflow -on your self-hosted server. If successful, the status will change to "Active" while the workflow is running. You can -get an overview of previous GitHub actions by navigating to [https://github.com/<your organization>/<your repository>/actions](https://github.com/%3Cyour%20organization%3E/%3Cyour%20repository%3E/actions). - - -### Monitoring the runner service's logs - -The log of the runner can be viewed with - -```shell -ssh -i -p @ -``` - -Then - -```shell -journalctl -u actions.runner.* -``` - -### Start the runner each time the machine boots - -```shell -ansible-playbook playbook.yml --tags enable -``` - -### Start the runner - -```shell -ansible-playbook playbook.yml --tags start -``` - -### Managing the runner service through the playbook - -```shell -ansible-playbook playbook.yml --tags start -ansible-playbook playbook.yml --tags stop -ansible-playbook playbook.yml --tags restart -ansible-playbook playbook.yml --tags status -ansible-playbook playbook.yml --tags enable -ansible-playbook playbook.yml --tags disable -``` - -Uninstalling the runner - -```shell -ansible-playbook playbook.yml --tags uninstall -``` - -### Verify that your newly configured runner is triggered - -Add the following simple workflow as ``.github/workflows/self_hosted_ci.yml`` in your repository -[https://github.com/<your organization>/<your repository>](https://github.com/%3Cyour%20organization%3E/%3Cyour%20repository%3E): - -```yaml -name: Self-hosted CI example - -on: [push, pull_request] - -jobs: - test: - name: test - runs-on: self-hosted - steps: - - name: Show directory listing - shell: bash -l {0} - run: | - ls -la -``` - -With this workflow in place, new pushes and new pull requests should trigger your self-hosted server. -Try making a change to one of the files in your repository to see if you can trigger running the simple workflow -on your self-hosted server. If successful, the status will change to "Active" while the workflow is running. -You can see a record of past and current GitHub Actions by pointing your browser to -[https://github.com/<your organization>/<your repository>/actions?query=workflow:"Self-hosted+CI+example"](https://github.com/%3Cyour%20organization%3E/%3Cyour%20repository%3E/actions?query=workflow%3A%22Self-hosted+CI+example%22). - - -### What's next - -Find instructions for provisioning additional functionality [here](../README.md).