From 1347d86333dabeff2edabfca05baf63a3b66e1d8 Mon Sep 17 00:00:00 2001 From: Alex Lowe Date: Thu, 17 Oct 2024 16:18:47 -0400 Subject: [PATCH] docs: add howto guides for switching from the charm plugin This adds howto guides to switch from the charm plugin to the python and poetry plugins. --- docs/howto/charm-to-poetry.rst | 107 +++++++++++++++++++ docs/howto/charm-to-python.rst | 126 +++++++++++++++++++++++ docs/howto/index.rst | 10 ++ docs/index.rst | 11 ++ docs/reference/plugins/python_plugin.rst | 1 - 5 files changed, 254 insertions(+), 1 deletion(-) create mode 100644 docs/howto/charm-to-poetry.rst create mode 100644 docs/howto/charm-to-python.rst create mode 100644 docs/howto/index.rst diff --git a/docs/howto/charm-to-poetry.rst b/docs/howto/charm-to-poetry.rst new file mode 100644 index 000000000..53b76a764 --- /dev/null +++ b/docs/howto/charm-to-poetry.rst @@ -0,0 +1,107 @@ +.. _howto-migrate-to-poetry: + +Migrate from the ``charm`` plugin to Poetry +=========================================== + +Many charms use `Poetry`_ to manage their Python projects. For these charms, Charmcraft +has a :ref:`craft_parts_poetry_plugin`. Migrating from the charm plugin provides some +benefits, such as no longer having to maintain a ``requirements.txt`` file. If the +charm to be migrated does not currently use poetry, refer to the +`Poetry documentation `_ for instructions +on how to use poetry for a Python project. + +Update ``charmcraft.yaml`` +-------------------------- + +The first step is to update ``charmcraft.yaml`` to include the correct parts definition. +Depending on the history of a specific charm, it may not have an explicitly-included +``parts`` section determining how to build the charm. In this case, a ``parts`` section +can be created as follows: + +.. code-block:: yaml + + parts: + my-charm: # This can be named anything you want + plugin: poetry + source: . + +Ensure a recent enough version of ``pip`` and ``poetry`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``poetry`` plugin requires at least pip 22.3, released in October 2022. If the +charm's base uses an older version of pip, a newer version can be installed in the +build environment using a dependency part. Likewise, a charm may require a newer +version of poetry than is available in the distribution's repositories. The following +``parts`` section can be used in place of the section above to upgrade pip and Poetry +for charms that build on Ubuntu 22.04 or earlier: + +.. code-block:: yaml + :emphasize-lines: 2-9,11 + + parts: + poetry-deps: + plugin: nil + build-packages: + - curl + override-build: | + /usr/bin/python3 -m pip install pip==24.2 + curl -sSL https://install.python-poetry.org | python3 - + ln -sf $HOME/.local/bin/poetry /usr/local/bin/poetry + my-charm: # This can be named anything you want + after: [poetry-deps] + plugin: poetry + source: . + +Add optional dependency groups +------------------------------ + +If the charm has optional `dependency groups`_ that should be included when creating +the virtual environment, the ``poetry-with`` key can be used to include those groups +when creating the virtual environment. + +.. note:: + This is useful and encouraged, though not mandatory, for keeping track of + library dependencies (see below). For an example, see + `postgresql-operator`_. + +Include charm library dependencies +---------------------------------- + +Unlike the ``charm`` plugin, the ``poetry`` plugin does not install the dependencies +for included charmlibs. If any of the charm libraries used have PYDEPS, these will +need to be added to the charm's dependencies, potentially as their own +`dependency group `_. + +To find these dependencies, check each library file for its ``PYDEPS``. A command +that can find these is:: + + find lib -name "*.py" -exec awk '/PYDEPS = \[/,/\]/' {} + + +If run from the base directory of a charm, this will show all the PYDEPS declarations +from all loaded charm libs. + +Include extra files +------------------- + +A ``poetry`` plugin only includes the contents of the ``src`` and ``lib`` directories +as well as the generated virtual environment. If other files were previously included +from the main directory, they can be included again using the +:ref:`craft_parts_dump_plugin`: + +.. code-block:: yaml + :emphasize-lines: 5-9 + + parts: + my-charm: # This can be named anything you want + plugin: poetry + source: . + version-file: + plugin: dump + source: . + stage: + - charm_version + + +.. _dependency groups: https://python-poetry.org/docs/managing-dependencies/#dependency-groups +.. _Poetry: https://python-poetry.org +.. _postgresql-operator: https://github.com/canonical/postgresql-operator/blob/3c7c783d61d4bee4ce64c190a9f7d4a78048e4e7/pyproject.toml#L22-L35 diff --git a/docs/howto/charm-to-python.rst b/docs/howto/charm-to-python.rst new file mode 100644 index 000000000..4c9816dfc --- /dev/null +++ b/docs/howto/charm-to-python.rst @@ -0,0 +1,126 @@ +.. _howto-migrate-to-python: + +Migrate from the ``charm`` plugin to the ``python`` plugin +========================================================== + +The ``python`` Charmcraft plugin offers a faster, stricter means of packing an operator +charm with a virtual environment. This guide shows how to migrate from a charm using +the default ``charm`` plugin to using the ``python`` plugin. + +Update ``charmcraft.yaml`` +-------------------------- + +The first step is to update ``charmcraft.yaml`` to include the correct parts definition. +Depending on the history of a specific charm, it may not have an explicitly-included +``parts`` section determining how to build the charm. In this case, a ``parts`` section +can be created as follows: + +.. code-block:: yaml + + parts: + my-charm: # This can be named anything you want + plugin: python + source: . + python-requirements: + - requirements.txt # Or whatever your requirements file is called. + +Ensure a recent enough version of ``pip`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``python`` plugin requires at least pip 22.3, released in October 2022. If the +charm's base uses an older version of pip, a newer version can be installed in the +build environment using a dependency part. The following ``parts`` section can be +used in place of the section above to upgrade pip for charms that build on Ubuntu +22.04 or earlier: + +.. code-block:: yaml + :emphasize-lines: 2-5,7 + + parts: + python-deps: + plugin: nil + override-build: | + /usr/bin/python3 -m pip install pip==24.2 + my-charm: # This can be named anything you want + after: [python-deps] + plugin: python + source: . + python-requirements: + - requirements.txt # Or whatever your requirements file is called. + +Flatten ``requirements.txt`` +---------------------------- + +One difference between the ``python`` plugin and the ``charm`` plugin is that the +``python`` plugin does not install dependencies, so the ``requirements.txt`` file +must be a complete set of packages needed in the charm's virtual environment. + +.. note:: + There are several tools for creating an exhaustive ``requirements.txt`` file. + Charmcraft works with any as long as it generates a requirements file that ``pip`` + understands. Because different versions of packages may have different + dependencies, it is recommended that the requirements file be generated using a + tool that will lock the dependencies to specific versions. + A few examples include: + + - `uv export `_ + - `pip-compile `_ + - `pip freeze `_ + +A basic ``requirements.txt`` file for a charm with no dependencies other than the +Operator framework may look something like:: + + ops==2.17.0 + pyyaml==6.0.2 + websocket-client==1.8.0 + +To check that the virtual environment for the charm would be valid, activate an +empty virtual environment and then run:: + + pip install --no-deps -r requirements.txt + pip check + +Include charm library dependencies +---------------------------------- + +Unlike the ``charm`` plugin, the ``python`` plugin does not install the dependencies +for included charmlibs. If any of the charm libraries used have PYDEPS, these will +need to be added to a requirements file as well. + +.. note:: + All requirements files are included in the same ``pip`` command to prevent + conflicting requirements from overriding each other. However, this means + that a charm will fail to build if it has conflicting requirements. A single + ``requirements.txt`` file, while not mandatory, is recommended. + +To find these dependencies, check each library file for its ``PYDEPS``. A command +that can find these is:: + + find lib -name "*.py" -exec awk '/PYDEPS = \[/,/\]/' {} + + +If run from the base directory of a charm, this will show all the PYDEPS declarations +from all loaded charm libs, which can be used to help generate the input for a tool +that generates ``requirements.txt``. + +Include extra files +------------------- + +A ``python`` plugin only includes the contents of the ``src`` and ``lib`` directories +as well as the generated virtual environment. If other files were previously included +from the main directory, they can be included again using the +:ref:`craft_parts_dump_plugin`: + +.. code-block:: yaml + :emphasize-lines: 7-11 + + parts: + my-charm: # This can be named anything you want + plugin: python + source: . + python-requirements: + - requirements.txt # Or whatever your requirements file is called. + version-file: + plugin: dump + source: . + stage: + - charm_version diff --git a/docs/howto/index.rst b/docs/howto/index.rst new file mode 100644 index 000000000..46ccff52b --- /dev/null +++ b/docs/howto/index.rst @@ -0,0 +1,10 @@ +.. _howto: + +How-To +****** + +.. toctree:: + :maxdepth: 2 + + charm-to-poetry + charm-to-python diff --git a/docs/index.rst b/docs/index.rst index b0bd33862..4e9931d9f 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -10,9 +10,20 @@ Most of Charmcraft's documentation is available there. :maxdepth: 1 :hidden: + howto/index reference/index explanation/index +.. grid:: 1 1 2 2 + + .. grid-item-card:: `Tutorial `_ + + **Get started** with a hands-on introduction to Charmcraft + + .. grid-item-card:: :ref:`How-to guides ` + + **Step-by-step guides** covering key operations and common tasks + .. grid:: 1 1 2 2 :reverse: diff --git a/docs/reference/plugins/python_plugin.rst b/docs/reference/plugins/python_plugin.rst index 911c94c23..3b9c8a256 100644 --- a/docs/reference/plugins/python_plugin.rst +++ b/docs/reference/plugins/python_plugin.rst @@ -49,7 +49,6 @@ During the build step, the plugin performs the following actions: 4. It copies any existing ``src`` and ``lib`` directories from your charm project into the final charm. - Example -------