From 2575fa5388c37a24e71a17ffa533c1f207827da0 Mon Sep 17 00:00:00 2001 From: Ayaz Salikhov Date: Mon, 9 Sep 2024 14:36:46 +0200 Subject: [PATCH 01/10] Create a separate doc page on how to build custom set of images --- docs/contributing/features.md | 1 + docs/using/custom-images.md | 74 +++++++++++++++++++++++++++++++++++ docs/using/recipes.md | 57 ++------------------------- 3 files changed, 78 insertions(+), 54 deletions(-) create mode 100644 docs/using/custom-images.md diff --git a/docs/contributing/features.md b/docs/contributing/features.md index b4d681976a..b45dc3e775 100644 --- a/docs/contributing/features.md +++ b/docs/contributing/features.md @@ -12,6 +12,7 @@ Please follow the process below to suggest a new feature for inclusion in one of describing the feature you'd like to contribute. 2. Discuss with the maintainers whether the addition makes sense in [one of the core stacks](../using/selecting.md#core-stacks), + as a [way to build a custom set of images](custom-images.md), as a [recipe in the documentation](recipes.md), as a [community stack](stacks.md), or as something else entirely. diff --git a/docs/using/custom-images.md b/docs/using/custom-images.md new file mode 100644 index 0000000000..6f09807e39 --- /dev/null +++ b/docs/using/custom-images.md @@ -0,0 +1,74 @@ +# Building custom set of images + +This section describes how to build a custom set of images. +It may be helpful if you need to change Ubuntu or Python version, or make a significant change to the build process itself. + +## Building stack images with custom arguments + +A selection of prebuilt images are available from Quay.io, +however, it's impossible to cater to everybody's needs. +For extensive customization with an automated build pipeline, +you may wish to create a [community-maintained stack](https://jupyter-docker-stacks.readthedocs.io/en/latest/contributing/stacks.html), +however, for minor customizations, this may be overkill. +For example, you may wish to use the same jupyter stacks but built on a different base image, +or build with a different Python version. + +To achieve this you can use [Docker Bake](https://docs.docker.com/build/bake/) +to build the stacks locally with custom arguments. + +```{note} +Custom arguments may result in build errors due to incompatibility. +If so your use-case may require a fully customized stack. +``` + +As a basic example, if you want to build a custom image based on the `minimal-notebook` image using `Python 3.12`, +then with a Dockerfile like: + +```{code-block} Dockerfile +:caption: Dockerfile + +ARG BASE_CONTAINER=minimal-notebook +FROM $BASE_CONTAINER +... +``` + +Include the below file in your project: + +```{literalinclude} recipe_code/docker-bake.python312.hcl +:force: +:language: hcl +:caption: docker-bake.hcl +``` + +To build this stack, in the same directory run: + +```bash +docker buildx bake +``` + +Docker Bake then determines the correct build order from the `contexts` parameters +and builds the stack as requested. + +This image can then be run using: + +```bash +docker run custom-jupyter +``` + +or referenced in a docker compose file. + +## Forking our repository + +The project tries to make it's best for users, who would like to fork the repository, and change a few things and build their set of images. + +Existing customization points: + +- `ROOT_CONTAINER` - a docker argument of `docker-stacks-foundation` image +- `PYTHON_VERSION` - a docker argument of `docker-stacks-foundation` image +- `REGISTRY`, `OWNER`, `BASE_CONTAINER` - docker arguments for all the other images we build +- `REGISTRY`, `OWNER` - part of `env` in most of our GitHub workflows + +## Automating your build using template cookiecutter project + +We have created a cookiecutter project, helping you to automate building a custom image. +Please, take a look at [the documentation](../contributing/stacks.md). diff --git a/docs/using/recipes.md b/docs/using/recipes.md index 46c6edb6aa..69ecf9e180 100644 --- a/docs/using/recipes.md +++ b/docs/using/recipes.md @@ -4,6 +4,9 @@ Users sometimes share interesting ways of using the Jupyter Docker Stacks. We encourage users to [contribute these recipes](../contributing/recipes.md) to the documentation in case they prove helpful to other community members by submitting a pull request to `docs/using/recipes.md`. The sections below capture this knowledge. +All the recipes here assume you would like to use an image built by this project and install some things on top of it. +If you would like to build a custom set of images, [take a look at the docs](custom-images.md). + ## Using `sudo` within a container Password authentication is disabled for the `NB_USER` (e.g., `jovyan`). @@ -517,57 +520,3 @@ they may be explained in the "Installation instructions" section of the Download ```{literalinclude} recipe_code/oracledb.dockerfile :language: docker ``` - -## Building stack images with custom arguments - -A selection of prebuilt images are available from Quay.io, -however, it's impossible to cater to everybody's needs. -For extensive customization with an automated build pipeline, -you may wish to create a [community-maintained stack](https://jupyter-docker-stacks.readthedocs.io/en/latest/contributing/stacks.html), -however, for minor customizations, this may be overkill. -For example, you may wish to use the same jupyter stacks but built on a different base image, -or build with a different Python version. - -To achieve this you can use [Docker Bake](https://docs.docker.com/build/bake/) -to build the stacks locally with custom arguments. - -```{note} -Custom arguments may result in build errors due to incompatibility. -If so your use-case may require a fully customized stack. -``` - -As a basic example, if you want to build a custom image based on the `minimal-notebook` image using `Python 3.12`, -then with a Dockerfile like: - -```{code-block} Dockerfile -:caption: Dockerfile - -ARG BASE_CONTAINER=minimal-notebook -FROM $BASE_CONTAINER -... -``` - -Include the below file in your project: - -```{literalinclude} recipe_code/docker-bake.python312.hcl -:force: -:language: hcl -:caption: docker-bake.hcl -``` - -To build this stack, in the same directory run: - -```bash -docker buildx bake -``` - -Docker Bake then determines the correct build order from the `contexts` parameters -and builds the stack as requested. - -This image can then be run using: - -```bash -docker run custom-jupyter -``` - -or referenced in a docker compose file. From c82b3d3f70630d28b7eb2467aa9d8cc48df3417c Mon Sep 17 00:00:00 2001 From: Ayaz Salikhov Date: Mon, 9 Sep 2024 14:40:44 +0200 Subject: [PATCH 02/10] Fix link --- docs/contributing/features.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/contributing/features.md b/docs/contributing/features.md index b45dc3e775..7da372de13 100644 --- a/docs/contributing/features.md +++ b/docs/contributing/features.md @@ -12,7 +12,7 @@ Please follow the process below to suggest a new feature for inclusion in one of describing the feature you'd like to contribute. 2. Discuss with the maintainers whether the addition makes sense in [one of the core stacks](../using/selecting.md#core-stacks), - as a [way to build a custom set of images](custom-images.md), + as a [way to build a custom set of images](../using/custom-images.md), as a [recipe in the documentation](recipes.md), as a [community stack](stacks.md), or as something else entirely. From 1367bd9b6beccd8a5caf66cea457e832139b2549 Mon Sep 17 00:00:00 2001 From: Ayaz Salikhov Date: Mon, 9 Sep 2024 14:42:29 +0200 Subject: [PATCH 03/10] Include new page in toctree --- docs/index.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/index.rst b/docs/index.rst index 896ca0f7da..23aa0e90de 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -13,6 +13,7 @@ Table of Contents using/common using/specifics using/recipes + using/custom-images using/troubleshooting using/faq From cf2445280705b08b13109217560c7a752ca7fb8f Mon Sep 17 00:00:00 2001 From: Ayaz Salikhov Date: Mon, 9 Sep 2024 14:46:23 +0200 Subject: [PATCH 04/10] Minor fix --- docs/using/custom-images.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using/custom-images.md b/docs/using/custom-images.md index 6f09807e39..12e3c915d0 100644 --- a/docs/using/custom-images.md +++ b/docs/using/custom-images.md @@ -1,4 +1,4 @@ -# Building custom set of images +# Building a custom set of images This section describes how to build a custom set of images. It may be helpful if you need to change Ubuntu or Python version, or make a significant change to the build process itself. From 586671c39839c31d69331fc88f3bca45ca0f4a81 Mon Sep 17 00:00:00 2001 From: Ayaz Salikhov Date: Mon, 9 Sep 2024 14:52:29 +0200 Subject: [PATCH 05/10] Rewrite --- docs/using/custom-images.md | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/docs/using/custom-images.md b/docs/using/custom-images.md index 12e3c915d0..4a65fa65d9 100644 --- a/docs/using/custom-images.md +++ b/docs/using/custom-images.md @@ -3,6 +3,20 @@ This section describes how to build a custom set of images. It may be helpful if you need to change Ubuntu or Python version, or make a significant change to the build process itself. +## Automating your build using template cookiecutter project + +If you wish to build your own image on top of one of our images and automate your build process, +please, [take a look at cookiecutter template](../contributing/stacks.md). + +## Custom arguments + +Existing customization points: + +- `ROOT_CONTAINER` - a docker argument of `docker-stacks-foundation` image +- `PYTHON_VERSION` - a docker argument of `docker-stacks-foundation` image +- `REGISTRY`, `OWNER`, `BASE_CONTAINER` - docker arguments for all the other images we build +- `REGISTRY`, `OWNER` - part of `env` in most of our GitHub workflows + ## Building stack images with custom arguments A selection of prebuilt images are available from Quay.io, @@ -59,16 +73,5 @@ or referenced in a docker compose file. ## Forking our repository -The project tries to make it's best for users, who would like to fork the repository, and change a few things and build their set of images. - -Existing customization points: - -- `ROOT_CONTAINER` - a docker argument of `docker-stacks-foundation` image -- `PYTHON_VERSION` - a docker argument of `docker-stacks-foundation` image -- `REGISTRY`, `OWNER`, `BASE_CONTAINER` - docker arguments for all the other images we build -- `REGISTRY`, `OWNER` - part of `env` in most of our GitHub workflows - -## Automating your build using template cookiecutter project - -We have created a cookiecutter project, helping you to automate building a custom image. -Please, take a look at [the documentation](../contributing/stacks.md). +If for some reason, you need to change more things in our images, feel free to fork it and change it the way you like it. +If your customization is easy to backport to the main repo and might be helpful for other users, feel free to create a PR. From 9dbc65794848c37d0778c4702b54937d146342b5 Mon Sep 17 00:00:00 2001 From: Ayaz Salikhov Date: Mon, 9 Sep 2024 14:54:40 +0200 Subject: [PATCH 06/10] Rewrite --- docs/using/custom-images.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/using/custom-images.md b/docs/using/custom-images.md index 4a65fa65d9..0b2fe8718f 100644 --- a/docs/using/custom-images.md +++ b/docs/using/custom-images.md @@ -73,5 +73,7 @@ or referenced in a docker compose file. ## Forking our repository -If for some reason, you need to change more things in our images, feel free to fork it and change it the way you like it. +If for some reason, you need to change more things in our images, feel free to fork it and change it any way you want. If your customization is easy to backport to the main repo and might be helpful for other users, feel free to create a PR. + +It is almost always a great idea to keep your diff as small as possible and to merge/rebase the latest version of our repo in your project. From 8c11c3b7bf2b7a6f7cc49cb29e64c30eac3c95ed Mon Sep 17 00:00:00 2001 From: Ayaz Salikhov Date: Mon, 9 Sep 2024 14:58:40 +0200 Subject: [PATCH 07/10] Apply suggestions from code review Co-authored-by: Simon Li --- docs/using/custom-images.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/using/custom-images.md b/docs/using/custom-images.md index 0b2fe8718f..dfb42b170f 100644 --- a/docs/using/custom-images.md +++ b/docs/using/custom-images.md @@ -1,7 +1,7 @@ # Building a custom set of images This section describes how to build a custom set of images. -It may be helpful if you need to change Ubuntu or Python version, or make a significant change to the build process itself. +It may be helpful if you need to change the Ubuntu or Python version, or to make a significant change to the build process itself. ## Automating your build using template cookiecutter project @@ -19,13 +19,13 @@ Existing customization points: ## Building stack images with custom arguments -A selection of prebuilt images are available from Quay.io, +A selection of prebuilt images are available from [quay.io](https://quay.io/organization/jupyter), however, it's impossible to cater to everybody's needs. For extensive customization with an automated build pipeline, -you may wish to create a [community-maintained stack](https://jupyter-docker-stacks.readthedocs.io/en/latest/contributing/stacks.html), +you may wish to create a [community-maintained stack](../contributing/stacks), however, for minor customizations, this may be overkill. -For example, you may wish to use the same jupyter stacks but built on a different base image, -or build with a different Python version. +For example, you may wish to use the same Jupyter stacks but built on a different base image, +or built with a different Python version. To achieve this you can use [Docker Bake](https://docs.docker.com/build/bake/) to build the stacks locally with custom arguments. @@ -69,7 +69,7 @@ This image can then be run using: docker run custom-jupyter ``` -or referenced in a docker compose file. +or referenced in a Docker Compose file. ## Forking our repository From 8c3d2b0d3be04c785a300a3258ec8e801fcb6745 Mon Sep 17 00:00:00 2001 From: Ayaz Salikhov Date: Mon, 9 Sep 2024 14:59:17 +0200 Subject: [PATCH 08/10] Capitalize Quay.io --- docs/using/custom-images.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using/custom-images.md b/docs/using/custom-images.md index dfb42b170f..94d9720406 100644 --- a/docs/using/custom-images.md +++ b/docs/using/custom-images.md @@ -19,7 +19,7 @@ Existing customization points: ## Building stack images with custom arguments -A selection of prebuilt images are available from [quay.io](https://quay.io/organization/jupyter), +A selection of prebuilt images are available from [Quay.io](https://quay.io/organization/jupyter), however, it's impossible to cater to everybody's needs. For extensive customization with an automated build pipeline, you may wish to create a [community-maintained stack](../contributing/stacks), From e4129493531fc97384e779b8a9c1d0387670ebc0 Mon Sep 17 00:00:00 2001 From: Ayaz Salikhov Date: Mon, 9 Sep 2024 15:43:37 +0200 Subject: [PATCH 09/10] Add info about old images --- docs/using/custom-images.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/using/custom-images.md b/docs/using/custom-images.md index 94d9720406..5a3e3641d1 100644 --- a/docs/using/custom-images.md +++ b/docs/using/custom-images.md @@ -3,6 +3,9 @@ This section describes how to build a custom set of images. It may be helpful if you need to change the Ubuntu or Python version, or to make a significant change to the build process itself. +This project only builds one set of images at a time. +If you want to use older images, take a look [here](https://jupyter-docker-stacks.readthedocs.io/en/latest/#using-old-images). + ## Automating your build using template cookiecutter project If you wish to build your own image on top of one of our images and automate your build process, From 36bfa67a64bf50999ed666f1fbc22aebf1249236 Mon Sep 17 00:00:00 2001 From: Ayaz Salikhov Date: Mon, 9 Sep 2024 21:20:35 +0200 Subject: [PATCH 10/10] Better text --- docs/using/custom-images.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/using/custom-images.md b/docs/using/custom-images.md index 5a3e3641d1..ad2ca87b87 100644 --- a/docs/using/custom-images.md +++ b/docs/using/custom-images.md @@ -66,10 +66,10 @@ docker buildx bake Docker Bake then determines the correct build order from the `contexts` parameters and builds the stack as requested. -This image can then be run using: +This image can then be run the same way as any other image provided by this project, for example: ```bash -docker run custom-jupyter +docker run -it --rm -p 8888:8888 custom-jupyter ``` or referenced in a Docker Compose file.