Skip to content

Commit

Permalink
Rebuild
Browse files Browse the repository at this point in the history
  • Loading branch information
@RosalynHatcher
RosalynHatcher committed Nov 1, 2023
1 parent 66cd7a5 commit 5f02a70
Show file tree
Hide file tree
Showing 29 changed files with 1,739 additions and 3,333 deletions.
2 changes: 1 addition & 1 deletion .buildinfo
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: 08abfb2f80a9202fc2ea94d0ca8a0d80
config: 635f7983675768b911a1fe17844e1fb7
tags: 645f666f9bcd5a90fca523b33c5a78b7
196 changes: 112 additions & 84 deletions _sources/getting-setup-selfstudy.rst.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,59 +2,48 @@ Getting Set Up (Self-Study Instructions)
========================================

.. warning::
You **MUST** have PUMA, ARCHER2 and MOSRS accounts setup before starting this section.
You **MUST** have PUMA2, ARCHER2 and MOSRS accounts setup before starting this section.

.. warning::
If you have moved your account over to PUMA2 from the old PUMA server (pumanew), then you should follow these instructions: https://cms.ncas.ac.uk/puma2/

.. _archer2:

Setup connection to PUMA & ARCHER2
----------------------------------
Set up your ARCHER2 connection
------------------------------

To use the UM Introduction Tutorials you will first need to ensure you can connect from your local desktop to a both PUMA & ARCHER2. There a multiple ways in which you can do this depending on your desktop platform:
To use the UM Introduction Tutorials you will first need to ensure you can connect from your local desktop to ARCHER2. There a multiple ways in which you can do this depending on your desktop platform:

* via `Terminal <terminal_>`_ on GNU/Linux & macOS
* via `MobaXTerm <mobaxterm_>`_ on Windows

SSH key files
^^^^^^^^^^^^^

Before you try and connect to PUMA or ARCHER2, you need to make sure that you have the ssh-keys for both platforms available on your computer.
Before you try and connect to ARCHER2, you need to make sure that you have the ssh-keys available on your computer.

.. _terminal:

Connecting via a Terminal (GNU/Linux & macOS)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
It is possible to connect via a terminal with an X11 connection (`XQuartz <https://www.xquartz.org/>`_ is also required when using macOS)

Login to PUMA: ::

ssh -Y -i ~/.ssh/id_rsa_puma <puma-username>@192.171.169.138
We suggest adding an entry to the ``~/.ssh/config`` file so you don't need to keep typing in the IP address. For example: ::

Host puma
Hostname 192.171.169.138
User <puma_username>
IdentityFile ~/.ssh/id_rsa_puma
ForwardX11 yes
ForwardX11Trusted yes
so that you can then connect using the command: ::

ssh puma

In a new terminal window, login to ARCHER2: ::
Login to ARCHER2: ::

ssh -Y -i ~/.ssh/id_rsa_archer <archer2-username>@login.archer2.ac.uk
ssh -Y -i ~/.ssh/id_rsa_archer2 <archer2-username>@login.archer2.ac.uk

It is also possible to define a ``~/.ssh/config`` file entry for each with the necessary information, if desired. For example: ::
To simplify the login process, you can define a ``~/.ssh/config`` file entry containing the necessary information. For example: ::

Host login.archer2.ac.uk
Host archer2
Hostname login.archer2.ac.uk
User <archer2_username>
IdentityFile ~/.ssh/id_rsa_archer
IdentityFile ~/.ssh/id_rsa_archer2
ForwardX11 yes
ForwardX11Trusted yes

so that you could then just connect using the command: ::
so that you can connect using just the command: ::
ssh login.archer2.ac.uk
ssh archer2

.. _mobaxterm:

Expand All @@ -63,12 +52,12 @@ Connecting via MobaXTerm

* From Chrome, go to page: https://mobaxterm.mobatek.net/download.html
* Under “Home Edition” select “Download now”
* | On next page select **“MobaXterm Home Edition v21.4 (Portable edition)”**.
* | On next page select **“MobaXterm Home Edition v23.3 (Portable edition)”**.
| This should download the package.
* Click the download icon in the bottom left hand corner.
* | Double-click on the **MobaXterm_Personal_21.4** application file, and select “Extract all”.
* | Double-click on the **MobaXterm_Personal_23.3** application file, and select “Extract all”.
| A new directory window will open up.
* Double-click **MobaXterm_Personal_21.4** to launch the application.
* Double-click **MobaXterm_Personal_23.3** to launch the application.

Next time, navigate to “Downloads” to open the application.

Expand All @@ -84,94 +73,133 @@ Change the permissions on your ``/home`` and ``/work`` directories to enable the
chmod -R g+rX /home/n02/n02/<your-username>
chmod -R g+rX /work/n02/n02/<your-username>

Set up your PUMA environment
.. _puma2:

Set up your PUMA2 connection
----------------------------

Login to PUMA from your local desktop.
PUMA2 is accessed from the ARCHER2 login nodes, and you will use the same username and password.

Configure ``~/.profile``
^^^^^^^^^^^^^^^^^^^^^^^^
If this is the first time you have used your PUMA account, you will need to create a ``.profile``. Copy our standard one: ::
From an ARCHER2 terminal type: ::

puma$ cd
puma$ cp ~um/um-training/setup/.profile .
archer2$ ssh -Y puma2

(If you already have a ``.profile``, make sure it includes the lines from the standard file.)
and type your ARCHER2 password when prompted.

Configure access to MOSRS
^^^^^^^^^^^^^^^^^^^^^^^^^
Run the ``mosrs-setup`` script which will take you through the set up process to access the Met Office Science Repository Service (Remember your MOSRS username is one word; usually firstnamelastname, all in lowercase): ::
You should now be logged into PUMA2. To go back to the ARCHER2 login nodes, type ``exit``.

puma$ ~um/um-training/mosrs-setup
Set up passwordless access to PUMA2
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Log out of PUMA and back in again (you will get a warning about not being able to find ``~/.ssh/ssh-setup`` this can be ignored and will be resolved in the next step). You should be prompted for your Met Office Science Repository Service password. A new window should then pop up (it may be hidden behind other windows) for ``Rosie`` asking for **Username for 'u' - 'https://code.metoffice.gov.uk/rosie/u'** . Enter your MOSRS username again.
You can set up a passphrase-less ssh-key to allow you to connect to PUMA2 without typing a password or passphrase.

.. note:: The cached password is configured to expire after 12 hours. Simply run the command ``mosrs-cache-password`` to re-cache it if this happens. Also if you know you won't need access to the repositories during a login session then just press return when asked for your MOSRS password.
.. note:: We would never normally advise using an ssh-key without a passphrase, but in this case it is safe to do so since we are already authenticated within the ARCHER2 system.

Configure connection to ARCHER2
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
From the ARCHER2 login nodes, type: ::

Due to ARCHER2 security and the UM workflow it is necessary to generate a special ssh-key that allows submission of UM suite from PUMA.
archer2$ ssh-keygen -t rsa -f ~/.ssh/id_rsa_puma2

**i. Generate UM workflow ssh-key**
At the prompt, press enter for an empty passphrase.

Run the following command to generate your ``id_rsa_archerum`` ssh key: ::
Copy the key over to PUMA2: ::

puma$ ssh-keygen -t rsa -b 4096 -C "ARCHER2 UM Workflow" -f ~/.ssh/id_rsa_archerum
archer2$ ssh-copy-id -i ~/.ssh/id_rsa_puma2 puma2

When prompted to **Enter passphrase**, this should be a fairly complicated and unguessable passphrase. You can use spaces in the passphrase if it helps you to remember it more readily. It is recommended that you don't use your password in case it is hacked.
Type in your ARCHER2 password when prompted.

Your ``id_rsa_archerum`` key will be automatically detected and sent to ARCHER2 to be installed. This may take up to 48 hours, excluding weekends, to become activated and you will receive an email confirmation.
Next, create a file called ``~/.ssh/config`` (if it doesn’t already exist), and add the following lines: ::

.. warning::
* **DO NOT** use an empty passphrase. This presents a security issue.
* **DO NOT** regenerate your ``id_rsa_archerum`` key once you have a working one in place, unless absolutely necessary.
Host puma2
IdentityFile ~/.ssh/id_rsa_puma2
ForwardX11 yes

**ii. Update ssh config file**
Test it works by typing: ::

In your PUMA ``~/.ssh/config`` file add the following section: ::
archer2$ ssh puma2

Host login.archer2.ac.uk
User <archer2_username>
IdentityFile ~/.ssh/id_rsa_archerum
ForwardX11 no
ForwardX11Trusted no
You should not be prompted for your password. Note that this should have set up X11 forwarding, so you no longer need the ``-Y`` option.

.. warning:: You should never use a passphrase-less key to access the ARCHER2 login nodes, as this is a serious security risk.

Set up your PUMA2 environment
-----------------------------

Copy our standard ``.profile`` and ``.bashrc`` files: ::

puma2$ cd
puma2$ cp ~um1/um-training/puma2/.bash_profile .
puma2$ cp ~um1/um-training/puma2/.bashrc .

Logout of PUMA2 and back in again to pick up these changes. You will get a warning about not being able to find ``~/.ssh/ssh-setup``. This can be ignored and will be resolved in the next step.

Where ``<archer2_username>`` should be replaced with your ARCHER2 username. If you don't have a ``~/.ssh/config`` file create one.
You should then be prompted for your Met Office Science Repository Service password, then username. Note that it asks for your **password** first. Remember your MOSRS username is one word; usually firstnamelastname, all in lowercase.

If the password caching works, you should see: ::

Subversion password cached
Rosie password cached

This means you can now access the code and roses suites stored in the Met Office respositories.

.. note:: The cached password is configured to expire after 12 hours. Simply run the command ``mosrs-cache-password`` to re-cache it if this happens. Also if you know you won't need access to the repositories during a login session then just press return when asked for your MOSRS password.

Finally, change the permission on your PUMA2 ``/home`` space: ::

chmod -R g+rX /home/n02/n02/<your-username>

.. _ssh-setup:

**iii. Set up ssh-agent**
Set up your ssh-agent
---------------------

Setting up an ``ssh-agent`` allows caching of your ``id_rsa_archerum`` key passphrase for a period of time. ::
In order to submit jobs to ARCHER2 from PUMA2, you will need to set up an ``ssh-agent`` and use it to cache the passphrase to your ARCHER2.

puma$ cp ~um/um-training/setup/ssh-setup ~/.ssh
**i. Copy your ARCHER2 ssh-key pair to PUMA2**

Log out of PUMA and back in again to start up the ``ssh-agent`` process.
Your ARCHER2 key is the one that you use to ssh into the ARCHER2 login nodes. You need to copy both the public and private keys into your ``.ssh/`` directory on PUMA2.

Add your ``id_rsa_archerum`` key to your ``ssh-agent`` by running: ::
Open a new terminal from wherever you originally connected to ARCHER2 in :ref:archer2:, and run the following command ::

puma$ ssh-add ~/.ssh/id_rsa_archerum
Enter passphrase for /home/<puma-username>/.ssh/id_rsa:
[TYPE_YOUR_PASSPHRASE]
scp ~/.ssh/id_rsa_archer2* <archer2-username>@login.archer2.ac.uk:/home/n02/n02-puma/<archer2-username>/.ssh

Enter your passphrase when prompted. The ``ssh-agent`` will continue to run even when you log out of PUMA, however, it may stop from time to time, for example if PUMA is rebooted. For instructions on what to do in this situation see :ref:`restarting-agent` in the Appendix.
**ii. Start up your ssh-agent**

**iv. Verify the setup is correct**
Setting up an ``ssh-agent`` allows caching of your ARCHER2 key passphrase for a period of time.

First copy the ``ssh-setup`` script to your ``.ssh/`` directory. ::

puma2$ cp ~um/um-training/setup/ssh-setup ~/.ssh

.. note:: Only proceed to this step once your ``id_rsa_archerum`` key has been installed on ARCHER2.
Next log out of PUMA2 and back in again to start up the ``ssh-agent`` process. You should see the following message ::

Log in to ARCHER2 with: ::
Initialising new SSH agent...

puma$ ssh login.archer2.ac.uk
**iii. Add your ARCHER2 key**

Add your ARCHER2 key to the ``ssh-agent``, by running ::

puma2$ ssh-add ~/.ssh/id_rsa_archer2

Enter your passphrase when prompted. If the passphrase has been cached successfully you should see a message like this: ::

Identity add: /home/n02/n02/<archer2-username>/.ssh/id_rsa_archer2

The ``ssh-agent`` will continue to run even when you log out of PUMA2, however, it may stop from time to time, for example if PUMA2 is rebooted. For instructions on what to do in this situation see :ref:`restarting-agent` in the Appendix.

**iv. Configure access to the ARCHER2 login nodes**

Create a file ``.ssh/config`` (if it doesn't already exist), and add the following lines: ::

# ARCHER2 login nodes
Host ln*
IdentityFile ~/.ssh/<archer-key>

**iv. Verify the setup is correct**

You should not be prompted for your passphrase. The response from ARCHER2 should be: ::
To test this is all working correctly, run: ::

puma$ ssh login.archer2.ac.uk
PTY allocation request failed on channel 0
Comand rejected by policy. Not in authorised list
Connection to login.archer2.ac.uk closed.
puma2$ rose host-select archer2

.. note:: It is not possible to start an interactive login session on ARCHER2 from PUMA. For an interactive session you need to login from your local desktop or via your host institution.
This should return one of the login nodes, e.g. ``ln01``. If it returns a message like ``[WARN] ln03: (ssh failed)`` then something has gone wrong with the ssh setup.

You are now ready to try running a UM suite!
Loading

0 comments on commit 5f02a70

Please sign in to comment.