From e07b06a1d0cb7a879e20cb6a1e32fb1cc1681d56 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hampus=20N=C3=A4sstr=C3=B6m?= Date: Mon, 30 Sep 2024 16:53:57 +0200 Subject: [PATCH 1/8] Added migration instructions --- README.md | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/README.md b/README.md index 8195619..e373d0b 100644 --- a/README.md +++ b/README.md @@ -12,3 +12,62 @@ This repository is a template for creating your own custom NOMAD Oasis distribut Click [here](https://github.com/new?template_name=nomad-distribution-template&template_owner=FAIRmat-NFDI) to use this template, or click the `Use this template` button in the upper right corner of the main GitHub page for this template. + +# Updating an Existing Distribution + +In order to update an existing distribution with any potential changes in the template you can add a new `git remote` for the template and merge with that one while allowing for unrelated histories: + +``` +git remote add template https://github.com/FAIRmat-NFDI/nomad-distribution-template +git fetch template +git merge template/main --allow-unrelated-histories +``` + +Most likely this will result in some merge conflicts which will need to be resolved. At the very least the `Dockerfile` and GitHub workflows should be taken from "theirs": + +``` +git checkout --theirs Dockerfile +git checkout --theirs .github/workflows/docker-publish.yml +``` + +## Migrating to building the full docker image (Version 2) +When moving from the first to the second version of the distribution image you can choose to keep your `docker-compose.yaml`: + +``` +git checkout --ours docker-compose.yaml +``` + +You can choose whether to keep your `README.md` but we recommend updating it as it contains the updated instructions for how to use the distribution repository: +``` +git checkout --theirs README.md +``` + +The plugins are now listed in the `pyproject.toml` instead of the `plugins.txt` and can be added there with `uv` and the `plugins.txt` removed: + +``` +uv add --optional plugins -r plugins.txt --no-sync +rm plugins.txt +``` + +The `nomad.yaml` is now moved to the `configs` directory, please copy over any changes you wish to keep and remove the old one: + +``` +rm nomad.yaml +``` + +Finally, the new recommendation is to either `git clone` the distribution repository or to `curl` the whole repository so the `nomad-oasis.zip` can also be removed: + +``` +rm nomad-oasis.zip +``` + +## Commit the Changes and Run the Workflows + +Ones the merge conflict is resolved you should add the changes and commit them + +``` +git add -A +git commit -m "Updated to new distribution version" +``` + +Ideally all workflows should be triggered automatically but you might need to run the initialization one manually by navigating to the "Actions" tab at the top, clicking "Template Repository Initialization" on the left side, and triggering it by clicking "Run workflow" under the "Run workflow" button on the right. \ No newline at end of file From 0a3e93e313dc928f128e5c2533465e534f9da3b8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hampus=20N=C3=A4sstr=C3=B6m?= Date: Mon, 30 Sep 2024 17:03:01 +0200 Subject: [PATCH 2/8] Changed to advice moving nomad.yaml --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index e373d0b..a65af4b 100644 --- a/README.md +++ b/README.md @@ -49,10 +49,10 @@ uv add --optional plugins -r plugins.txt --no-sync rm plugins.txt ``` -The `nomad.yaml` is now moved to the `configs` directory, please copy over any changes you wish to keep and remove the old one: +The `nomad.yaml` is now moved to the `configs` directory, please copy over your existing `nomad.yaml` there: ``` -rm nomad.yaml +mv nomad.yaml configs/nomad.yaml ``` Finally, the new recommendation is to either `git clone` the distribution repository or to `curl` the whole repository so the `nomad-oasis.zip` can also be removed: @@ -63,7 +63,7 @@ rm nomad-oasis.zip ## Commit the Changes and Run the Workflows -Ones the merge conflict is resolved you should add the changes and commit them +Once the merge conflict is resolved you should add the changes and commit them ``` git add -A From 842e9e223eb05a17785b1d20ed7b1e85de1120ac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hampus=20N=C3=A4sstr=C3=B6m?= Date: Mon, 30 Sep 2024 17:45:17 +0200 Subject: [PATCH 3/8] Combine template_README and README --- README.md | 188 ++++++++++++++++++++++++++++++++++++++------- template_README.md | 164 --------------------------------------- 2 files changed, 160 insertions(+), 192 deletions(-) delete mode 100644 template_README.md diff --git a/README.md b/README.md index a65af4b..a0f2a66 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,11 @@ ![docker image](https://github.com/FAIRmat-NFDI/nomad-distribution-template/actions/workflows/docker-publish.yml/badge.svg) +# NOMAD Oasis Distribution *Template* +This repository is a template for creating your own custom NOMAD Oasis distribution image. +Click [here](https://github.com/new?template_name=nomad-distribution-template&template_owner=FAIRmat-NFDI) +to use this template, or click the `Use this template` button in the upper right corner of +the main GitHub page for this template. + > [!IMPORTANT] > The templated repository will run a GitHub action on creation which might take a few minutes. > After the workflow finishes you should refresh the page and this message should disappear. @@ -7,67 +13,193 @@ > "Actions" tab at the top, clicking "Template Repository Initialization" on the left side, > and triggering it by clicking "Run workflow" under the "Run workflow" button on the right. -# NOMAD Oasis Distribution *Template* -This repository is a template for creating your own custom NOMAD Oasis distribution image. -Click [here](https://github.com/new?template_name=nomad-distribution-template&template_owner=FAIRmat-NFDI) -to use this template, or click the `Use this template` button in the upper right corner of -the main GitHub page for this template. +# FAIRmat-NFDI's NOMAD Oasis Distribution -# Updating an Existing Distribution +This is the NOMAD Oasis distribution of FAIRmat-NFDI. +Below are instructions for how to [deploy this distribution](#deploying-the-distribution) +and how to customize it through [adding plugins](#adding-a-plugin). -In order to update an existing distribution with any potential changes in the template you can add a new `git remote` for the template and merge with that one while allowing for unrelated histories: +> [!IMPORTANT] +> Depending on the settings of the owner of this repository, the distributed image might +> be private and require authentication to pull. +> If you want to keep the image private you need to configure and use a personal access +> token (PAT) according to the instructions in the GitHub docs [here](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic). +> If you want to make the image public (recommended), you should make sure that your +> organization settings allow public packages and make this package public after building it. +> You can read more about this in the GitHub docs [here](https://docs.github.com/en/packages/learn-github-packages/configuring-a-packages-access-control-and-visibility). + +> [!TIP] +> In order for others to find and learn from your distribution we in FAIRmat would +> greatly appreciate it if you would add the topic `nomad-distribution` by clicking the +> ⚙️ next to "About" on the main GitHub page for this repository. + +## Deploying the distribution + +Below are instructions for how to deploy this NOMAD Oasis distribution +[for a new Oasis](#for-a-new-oasis) and [for an existing Oasis](#for-an-existing-oasis) + +### For a new Oasis + +1. Make sure you have [docker](https://docs.docker.com/engine/install/) installed. + Docker nowadays comes with `docker compose` built in. Prior, you needed to + install the stand-alone [docker-compose](https://docs.docker.com/compose/install/). + +2. Clone the repository or download the repository as a zip file. +```sh +git clone https://github.com/FAIRmat-NFDI/nomad-distribution-template.git +cd nomad-distribution-template ``` -git remote add template https://github.com/FAIRmat-NFDI/nomad-distribution-template -git fetch template -git merge template/main --allow-unrelated-histories + +or + +```sh +curl-L -o nomad-distribution-template.zip "https://github.com/FAIRmat-NFDI/nomad-distribution-template/archive/main.zip" +unzip nomad-distribution-template.zip +cd nomad-distribution-template ``` -Most likely this will result in some merge conflicts which will need to be resolved. At the very least the `Dockerfile` and GitHub workflows should be taken from "theirs": +3. _On Linux only,_ recursively change the owner of the `.volumes` directory to the nomad user (1000) +```sh +sudo chown -R 1000 .volumes ``` -git checkout --theirs Dockerfile -git checkout --theirs .github/workflows/docker-publish.yml + +4. Pull the images specified in the `docker-compose.yaml` + +Note that the image needs to be public or you need to provide a PAT (see "Important" note above). + +```sh +docker compose pull +``` + +5. And run it with docker compose in detached (--detach or -d) mode + +```sh +docker compose up -d +``` + +6. Optionally you can now test that NOMAD is running with + +``` +curl localhost/nomad-oasis/alive ``` -## Migrating to building the full docker image (Version 2) -When moving from the first to the second version of the distribution image you can choose to keep your `docker-compose.yaml`: +7. Finally, open [http://localhost/nomad-oasis](http://localhost/nomad-oasis) in your browser to start using your new NOMAD Oasis. +Whenever you update your image you need to shut down NOMAD using + +```sh +docker compose down ``` -git checkout --ours docker-compose.yaml + +and then repeat steps 4. and 5. above. + +#### NOMAD Remote Tools Hub (NORTH) + +To run NORTH (the NOMAD Remote Tools Hub), the `hub` container needs to run docker and +the container has to be run under the docker group. You need to replace the default group +id `991` in the `docker-compose.yaml`'s `hub` section with your systems docker group id. +Run `id` if you are a docker user, or `getent group | grep docker` to find your +systems docker gid. The user id 1000 is used as the nomad user inside all containers. + +You can find more details on setting up and maintaining an Oasis in the NOMAD docs here: +[nomad-lab.eu/prod/v1/docs/oasis/install.html](https://nomad-lab.eu/prod/v1/docs/oasis/install.html) + +### For an existing Oasis + +If you already have an Oasis running you only need to change the image being pulled in +your `docker-compose.yaml` with `ghcr.io/FAIRmat-NFDI/nomad-distribution-template:main` for the services +`worker`, `app`, `north`, and `logtransfer`. + +If you want to use the `nomad.yaml` from this repository you also need to comment out +the inclusion of the `nomad.yaml` under the volumes key of those services in the +`docker-compose.yaml`. + +```yaml +volumes: + # - ./configs/nomad.yaml:/app/nomad.yaml ``` -You can choose whether to keep your `README.md` but we recommend updating it as it contains the updated instructions for how to use the distribution repository: +To run the new image you can follow steps 5. and 6. [above](#for-a-new-oasis). + +## Adding a plugin + +To add a new plugin to the docker image you should add it to the plugins table in the `pyproject.toml` file. + +Here you can put either plugins distributed to PyPI, e.g. + ``` -git checkout --theirs README.md +nomad-material-processing ``` -The plugins are now listed in the `pyproject.toml` instead of the `plugins.txt` and can be added there with `uv` and the `plugins.txt` removed: +or plugins in a git repository with either the commit hash ``` -uv add --optional plugins -r plugins.txt --no-sync -rm plugins.txt +"nomad-measurements @ git+https://github.com/FAIRmat-NFDI/nomad-measurements.git@71b7e8c9bb376ce9e8610aba9a20be0b5bce6775" ``` -The `nomad.yaml` is now moved to the `configs` directory, please copy over your existing `nomad.yaml` there: +or with a tag ``` -mv nomad.yaml configs/nomad.yaml +"nomad-measurements @ git+https://github.com/FAIRmat-NFDI/nomad-measurements.git@v0.0.4" ``` -Finally, the new recommendation is to either `git clone` the distribution repository or to `curl` the whole repository so the `nomad-oasis.zip` can also be removed: +To add a plugin in a subdirectory of a git repository you can use the `subdirectory` option, e.g. ``` -rm nomad-oasis.zip +"ikz_pld_plugin @ git+https://github.com/FAIRmat-NFDI/AreaA-data_modeling_and_schemas.git@30fc90843428d1b36a1d222874803abae8b1cb42#subdirectory=PVD/PLD/jeremy_ikz/ikz_pld_plugin" ``` -## Commit the Changes and Run the Workflows +If the plugin is new, you also need to add it under `plugins` in the [nomad.yaml](nomad.yaml) +config file that will be included in the image. +For example, if you have added a schema plugin `legacy_plugin` you should add +the following: + +```yaml +plugins: + options: + schemas/legacy_plugin: + python_package: legacy_plugin +``` -Once the merge conflict is resolved you should add the changes and commit them +Once the changes have been committed to the main branch, the new image will automatically +be generated. + +# Updating an Existing Distribution + +In order to update an existing distribution with any potential changes in the template you can add a new `git remote` for the template and merge with that one while allowing for unrelated histories: + +``` +git remote add template https://github.com/FAIRmat-NFDI/nomad-distribution-template +git fetch template +git merge template/main --allow-unrelated-histories +``` + +Most likely this will result in some merge conflicts which will need to be resolved. At the very least the `Dockerfile` and GitHub workflows should be taken from "theirs": + +``` +git checkout --theirs Dockerfile +git checkout --theirs .github/workflows/docker-publish.yml +``` + +For detailed instructions on how to resolve the merge conflicts between different version we refer you to the latest template release [notes](https://github.com/FAIRmat-NFDI/nomad-distribution-template/releases/latest) + +Once the merge conflicts are resolved you should add the changes and commit them ``` git add -A git commit -m "Updated to new distribution version" ``` -Ideally all workflows should be triggered automatically but you might need to run the initialization one manually by navigating to the "Actions" tab at the top, clicking "Template Repository Initialization" on the left side, and triggering it by clicking "Run workflow" under the "Run workflow" button on the right. \ No newline at end of file +Ideally all workflows should be triggered automatically but you might need to run the initialization one manually by navigating to the "Actions" tab at the top, clicking "Template Repository Initialization" on the left side, and triggering it by clicking "Run workflow" under the "Run workflow" button on the right. + +## FAQ/Trouble shooting + +_I get an_ `Error response from daemon: Head "https://ghcr.io/v2/FAIRmat-NFDI/nomad-distribution-template/manifests/main": unauthorized` +_when trying to pull my docker image._ + +Most likely you have not made the package public or provided a personal access token (PAT). +You can read how to make your package public in the GitHub docs [here](https://docs.github.com/en/packages/learn-github-packages/configuring-a-packages-access-control-and-visibility) +or how to configure a PAT (if you want to keep the distribution private) in the GitHub +docs [here](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic). \ No newline at end of file diff --git a/template_README.md b/template_README.md deleted file mode 100644 index 3bf2422..0000000 --- a/template_README.md +++ /dev/null @@ -1,164 +0,0 @@ -![docker image](https://github.com/GITHUB_REPOSITORY/actions/workflows/docker-publish.yml/badge.svg) - -# GITHUB_REPOSITORY_OWNER's NOMAD Oasis Distribution - -This is the NOMAD Oasis distribution of GITHUB_REPOSITORY_OWNER. -Below are instructions for how to [deploy this distribution](#deploying-the-distribution) -and how to customize it through [adding plugins](#adding-a-plugin). - -> [!IMPORTANT] -> Depending on the settings of the owner of this repository, the distributed image might -> be private and require authentication to pull. -> If you want to keep the image private you need to configure and use a personal access -> token (PAT) according to the instructions in the GitHub docs [here](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic). -> If you want to make the image public (recommended), you should make sure that your -> organization settings allow public packages and make this package public after building it. -> You can read more about this in the GitHub docs [here](https://docs.github.com/en/packages/learn-github-packages/configuring-a-packages-access-control-and-visibility). - -> [!TIP] -> In order for others to find and learn from your distribution we in FAIRmat would -> greatly appreciate it if you would add the topic `nomad-distribution` by clicking the -> ⚙️ next to "About" on the main GitHub page for this repository. - -## Deploying the distribution - -Below are instructions for how to deploy this NOMAD Oasis distribution -[for a new Oasis](#for-a-new-oasis) and [for an existing Oasis](#for-an-existing-oasis) - -### For a new Oasis - -1. Make sure you have [docker](https://docs.docker.com/engine/install/) installed. - Docker nowadays comes with `docker compose` built in. Prior, you needed to - install the stand-alone [docker-compose](https://docs.docker.com/compose/install/). - -2. Clone the repository or download the repository as a zip file. - -```sh -git clone https://github.com/GITHUB_REPOSITORY.git -cd GITHUB_REPOSITORY_NAME -``` - -or - -```sh -curl-L -o GITHUB_REPOSITORY_NAME.zip "https://github.com/GITHUB_REPOSITORY/archive/main.zip" -unzip GITHUB_REPOSITORY_NAME.zip -cd GITHUB_REPOSITORY_NAME -``` - -3. _On Linux only,_ recursively change the owner of the `.volumes` directory to the nomad user (1000) - -```sh -sudo chown -R 1000 .volumes -``` - -4. Pull the images specified in the `docker-compose.yaml` - -Note that the image needs to be public or you need to provide a PAT (see "Important" note above). - -```sh -docker compose pull -``` - -5. And run it with docker compose in detached (--detach or -d) mode - -```sh -docker compose up -d -``` - -6. Optionally you can now test that NOMAD is running with - -``` -curl localhost/nomad-oasis/alive -``` - -7. Finally, open [http://localhost/nomad-oasis](http://localhost/nomad-oasis) in your browser to start using your new NOMAD Oasis. - -Whenever you update your image you need to shut down NOMAD using - -```sh -docker compose down -``` - -and then repeat steps 4. and 5. above. - -#### NOMAD Remote Tools Hub (NORTH) - -To run NORTH (the NOMAD Remote Tools Hub), the `hub` container needs to run docker and -the container has to be run under the docker group. You need to replace the default group -id `991` in the `docker-compose.yaml`'s `hub` section with your systems docker group id. -Run `id` if you are a docker user, or `getent group | grep docker` to find your -systems docker gid. The user id 1000 is used as the nomad user inside all containers. - -You can find more details on setting up and maintaining an Oasis in the NOMAD docs here: -[nomad-lab.eu/prod/v1/docs/oasis/install.html](https://nomad-lab.eu/prod/v1/docs/oasis/install.html) - -### For an existing Oasis - -If you already have an Oasis running you only need to change the image being pulled in -your `docker-compose.yaml` with `ghcr.io/GITHUB_REPOSITORY:main` for the services -`worker`, `app`, `north`, and `logtransfer`. - -If you want to use the `nomad.yaml` from this repository you also need to comment out -the inclusion of the `nomad.yaml` under the volumes key of those services in the -`docker-compose.yaml`. - -```yaml -volumes: - # - ./configs/nomad.yaml:/app/nomad.yaml -``` - -To run the new image you can follow steps 5. and 6. [above](#for-a-new-oasis). - -## Adding a plugin - -To add a new plugin to the docker image you should add it to the plugins table in the `pyproject.toml` file. - -Here you can put either plugins distributed to PyPI, e.g. - -``` -nomad-material-processing -``` - -or plugins in a git repository with either the commit hash - -``` -"nomad-measurements @ git+https://github.com/FAIRmat-NFDI/nomad-measurements.git@71b7e8c9bb376ce9e8610aba9a20be0b5bce6775" -``` - -or with a tag - -``` -"nomad-measurements @ git+https://github.com/FAIRmat-NFDI/nomad-measurements.git@v0.0.4" -``` - -To add a plugin in a subdirectory of a git repository you can use the `subdirectory` option, e.g. - -``` -"ikz_pld_plugin @ git+https://github.com/FAIRmat-NFDI/AreaA-data_modeling_and_schemas.git@30fc90843428d1b36a1d222874803abae8b1cb42#subdirectory=PVD/PLD/jeremy_ikz/ikz_pld_plugin" -``` - -If the plugin is new, you also need to add it under `plugins` in the [nomad.yaml](nomad.yaml) -config file that will be included in the image. -For example, if you have added a schema plugin `legacy_plugin` you should add -the following: - -```yaml -plugins: - options: - schemas/legacy_plugin: - python_package: legacy_plugin -``` - -Once the changes have been committed to the main branch, the new image will automatically -be generated. - -## FAQ/Trouble shooting - -_I get an_ `Error response from daemon: Head "https://ghcr.io/v2/GITHUB_REPOSITORY/manifests/main": unauthorized` -_when trying to pull my docker image._ - -Most likely you have not made the package public or provided a personal access token (PAT). -You can read how to make your package public in the GitHub docs [here](https://docs.github.com/en/packages/learn-github-packages/configuring-a-packages-access-control-and-visibility) -or how to configure a PAT (if you want to keep the distribution private) in the GitHub -docs [here](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic). From 618502c1a381c571af5539c18314db2144aed513 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hampus=20N=C3=A4sstr=C3=B6m?= Date: Mon, 30 Sep 2024 18:33:23 +0200 Subject: [PATCH 4/8] Update initialization workflow to run on the new README --- .github/workflows/initialize.yml | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/.github/workflows/initialize.yml b/.github/workflows/initialize.yml index 38b6695..401c556 100644 --- a/.github/workflows/initialize.yml +++ b/.github/workflows/initialize.yml @@ -25,11 +25,13 @@ jobs: # Replaces the template repository name in the README with the new one - name: Update README run: | - sed -i "s|GITHUB_REPOSITORY_OWNER|${{ github.repository_owner }}|g" template_README.md - sed -i "s|GITHUB_REPOSITORY_NAME|${{ github.event.repository.name }}|g" template_README.md - sed -i "s|GITHUB_REPOSITORY|${{ github.repository }}|g" template_README.md - sed -i "s|GITHUB_REPOSITORY_NAME|${{ github.event.repository.name }}|g" template_README.md - mv template_README.md README.md + sed -i "/# NOMAD Oasis Distribution/,/button on the right./d" README.md + sed -i "s|FAIRmat-NFDI/nomad-distribution-template|${{ github.repository }}|g" README.md + sed -i "s|FAIRmat-NFDI|${{ github.repository_owner }}|g" README.md + sed -i "s|nomad-distribution-template|${{ github.event.repository.name }}|g" README.md + sed -i "s|template https://github.com/${{ github.repository }}|template https://github.com/FAIRmat-NFDI/nomad-distribution-template|g" README.md + sed -i "s|${{ github.repository }}/releases/latest|FAIRmat-NFDI/nomad-distribution-template/releases/latest|g" README.md + sed -i "s|@ git+https://github.com/${{ github.repository_owner }}|@ git+https://github.com/FAIRmat-NFDI|g" README.md # Replaces the template repository name in the docker config file with the new one - name: Update docker-compose.yaml From 5f6cab388794ad9b0251091da7832fe3e9256e36 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hampus=20N=C3=A4sstr=C3=B6m?= Date: Mon, 30 Sep 2024 18:39:31 +0200 Subject: [PATCH 5/8] Added indentation to enumerated code blocks --- README.md | 56 +++++++++++++++++++++++++++---------------------------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/README.md b/README.md index a0f2a66..1c1c404 100644 --- a/README.md +++ b/README.md @@ -46,54 +46,54 @@ Below are instructions for how to deploy this NOMAD Oasis distribution 2. Clone the repository or download the repository as a zip file. -```sh -git clone https://github.com/FAIRmat-NFDI/nomad-distribution-template.git -cd nomad-distribution-template -``` + ```sh + git clone https://github.com/FAIRmat-NFDI/nomad-distribution-template.git + cd nomad-distribution-template + ``` -or + or -```sh -curl-L -o nomad-distribution-template.zip "https://github.com/FAIRmat-NFDI/nomad-distribution-template/archive/main.zip" -unzip nomad-distribution-template.zip -cd nomad-distribution-template -``` + ```sh + curl-L -o nomad-distribution-template.zip "https://github.com/FAIRmat-NFDI/nomad-distribution-template/archive/main.zip" + unzip nomad-distribution-template.zip + cd nomad-distribution-template + ``` 3. _On Linux only,_ recursively change the owner of the `.volumes` directory to the nomad user (1000) -```sh -sudo chown -R 1000 .volumes -``` + ```sh + sudo chown -R 1000 .volumes + ``` 4. Pull the images specified in the `docker-compose.yaml` -Note that the image needs to be public or you need to provide a PAT (see "Important" note above). + Note that the image needs to be public or you need to provide a PAT (see "Important" note above). -```sh -docker compose pull -``` + ```sh + docker compose pull + ``` 5. And run it with docker compose in detached (--detach or -d) mode -```sh -docker compose up -d -``` + ```sh + docker compose up -d + ``` 6. Optionally you can now test that NOMAD is running with -``` -curl localhost/nomad-oasis/alive -``` + ``` + curl localhost/nomad-oasis/alive + ``` 7. Finally, open [http://localhost/nomad-oasis](http://localhost/nomad-oasis) in your browser to start using your new NOMAD Oasis. -Whenever you update your image you need to shut down NOMAD using + Whenever you update your image you need to shut down NOMAD using -```sh -docker compose down -``` + ```sh + docker compose down + ``` -and then repeat steps 4. and 5. above. + and then repeat steps 4. and 5. above. #### NOMAD Remote Tools Hub (NORTH) From fdfd8f00d75132b24cd3b4a1b9d91142e17a06eb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hampus=20N=C3=A4sstr=C3=B6m?= Date: Mon, 30 Sep 2024 18:41:35 +0200 Subject: [PATCH 6/8] Remove legacy note about adding plugins to nomad.yaml --- README.md | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/README.md b/README.md index 1c1c404..4ecb1d1 100644 --- a/README.md +++ b/README.md @@ -151,18 +151,6 @@ To add a plugin in a subdirectory of a git repository you can use the `subdirect "ikz_pld_plugin @ git+https://github.com/FAIRmat-NFDI/AreaA-data_modeling_and_schemas.git@30fc90843428d1b36a1d222874803abae8b1cb42#subdirectory=PVD/PLD/jeremy_ikz/ikz_pld_plugin" ``` -If the plugin is new, you also need to add it under `plugins` in the [nomad.yaml](nomad.yaml) -config file that will be included in the image. -For example, if you have added a schema plugin `legacy_plugin` you should add -the following: - -```yaml -plugins: - options: - schemas/legacy_plugin: - python_package: legacy_plugin -``` - Once the changes have been committed to the main branch, the new image will automatically be generated. From 44aecd6fc9246ac7589eca0f5a919e0372c29f57 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hampus=20N=C3=A4sstr=C3=B6m?= Date: Mon, 30 Sep 2024 18:48:47 +0200 Subject: [PATCH 7/8] Clarified the toml plugin inclusion --- README.md | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 4ecb1d1..3baa4f3 100644 --- a/README.md +++ b/README.md @@ -129,26 +129,38 @@ To add a new plugin to the docker image you should add it to the plugins table i Here you can put either plugins distributed to PyPI, e.g. -``` -nomad-material-processing +```toml +[project.optional-dependencies] +plugins = [ + "nomad-material-processing>=1.0.0", +] ``` or plugins in a git repository with either the commit hash -``` -"nomad-measurements @ git+https://github.com/FAIRmat-NFDI/nomad-measurements.git@71b7e8c9bb376ce9e8610aba9a20be0b5bce6775" +```toml +[project.optional-dependencies] +plugins = [ + "nomad-measurements @ git+https://github.com/FAIRmat-NFDI/nomad-measurements.git@71b7e8c9bb376ce9e8610aba9a20be0b5bce6775", +] ``` or with a tag -``` -"nomad-measurements @ git+https://github.com/FAIRmat-NFDI/nomad-measurements.git@v0.0.4" +```toml +[project.optional-dependencies] +plugins = [ + "nomad-measurements @ git+https://github.com/FAIRmat-NFDI/nomad-measurements.git@v0.0.4" +] ``` To add a plugin in a subdirectory of a git repository you can use the `subdirectory` option, e.g. -``` -"ikz_pld_plugin @ git+https://github.com/FAIRmat-NFDI/AreaA-data_modeling_and_schemas.git@30fc90843428d1b36a1d222874803abae8b1cb42#subdirectory=PVD/PLD/jeremy_ikz/ikz_pld_plugin" +```toml +[project.optional-dependencies] +plugins = [ + "ikz_pld_plugin @ git+https://github.com/FAIRmat-NFDI/AreaA-data_modeling_and_schemas.git@30fc90843428d1b36a1d222874803abae8b1cb42#subdirectory=PVD/PLD/jeremy_ikz/ikz_pld_plugin" +] ``` Once the changes have been committed to the main branch, the new image will automatically From 2b35c29e6a3b16ede86e6fc5627097b8fc997d4e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hampus=20N=C3=A4sstr=C3=B6m?= Date: Mon, 30 Sep 2024 18:56:12 +0200 Subject: [PATCH 8/8] Added small TOC --- README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 3baa4f3..0a8e68d 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,12 @@ and how to customize it through [adding plugins](#adding-a-plugin). > greatly appreciate it if you would add the topic `nomad-distribution` by clicking the > ⚙️ next to "About" on the main GitHub page for this repository. +In this README you will find instructions for: +1. [Deploying the distribution](#deploying-the-distribution) +2. [Adding a plugin](#adding-a-plugin) +3. [Updating the distribution from the template](#updating-the-distribution-from-the-template) +4. [Solving common issues](#faqtrouble-shooting) + ## Deploying the distribution Below are instructions for how to deploy this NOMAD Oasis distribution @@ -166,7 +172,7 @@ plugins = [ Once the changes have been committed to the main branch, the new image will automatically be generated. -# Updating an Existing Distribution +## Updating the distribution from the template In order to update an existing distribution with any potential changes in the template you can add a new `git remote` for the template and merge with that one while allowing for unrelated histories: