Skip to content

Commit

Permalink
docs: add howto guides for switching from the charm plugin
Browse files Browse the repository at this point in the history
This adds howto guides to switch from the charm plugin to the python
and poetry plugins.
  • Loading branch information
lengau committed Oct 17, 2024
1 parent 408650c commit 95567bd
Show file tree
Hide file tree
Showing 5 changed files with 254 additions and 1 deletion.
107 changes: 107 additions & 0 deletions docs/howto/charm-to-poetry.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
.. _howto-migrate-to-poetry:

Migrate from the ``charm`` plugin to Poetry
===========================================

Many charms use `Poetry`_ to manage their Python projects. For these charms, Charmcraft
has a :ref:`craft_parts_poetry_plugin`. Migrating from the charm plugin provides some
benefits, such as no longer having to maintain a ``requirements.txt`` file. If the
charm to be migrated does not currently use poetry, refer to the
`Poetry documentation <https://python-poetry.org/docs/basic-usage/>`_ for instructions
on how to use poetry for a Python project.

Update ``charmcraft.yaml``
--------------------------

The first step is to update ``charmcraft.yaml`` to include the correct parts definition.
Depending on the history of a specific charm, it may not have an explicitly-included
``parts`` section determining how to build the charm. In this case, a ``parts`` section
can be created as follows:

.. code-block:: yaml
parts:
my-charm: # This can be named anything you want
plugin: poetry
source: .
Ensure a recent enough version of ``pip`` and ``poetry``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The ``poetry`` plugin requires at least pip 22.3, released in October 2022. If the
charm's base uses an older version of pip, a newer version can be installed in the
build environment using a dependency part. Likewise, a charm may require a newer
version of poetry than is available in the distribution's repositories. The following
``parts`` section can be used in place of the section above to upgrade pip and Poetry
for charms that build on Ubuntu 22.04 or earlier:

.. code-block:: yaml
:emphasize-lines: 2-9,11
parts:
poetry-deps:
plugin: nil
build-packages:
- curl
override-build: |
/usr/bin/python3 -m pip install pip==24.2
curl -sSL https://install.python-poetry.org | python3 -
ln -sf $HOME/.local/bin/poetry /usr/local/bin/poetry
my-charm: # This can be named anything you want
after: [poetry-deps]
plugin: poetry
source: .
Add optional dependency groups
------------------------------

If the charm has optional `dependency groups`_ that should be included when creating
the virtual environment, the ``poetry-with`` key can be used to include those groups
when creating the virtual environment.

.. note::
This is useful and encouraged, though not mandatory, for keeping track of
library dependencies (see below). For an example, see
`postgresql-operator`_.

Include charm library dependencies
----------------------------------

Unlike the ``charm`` plugin, the ``poetry`` plugin does not install the dependencies
for included charmlibs. If any of the charm libraries used have PYDEPS, these will
need to be added to the charm's dependencies, potentially as their own
`dependency group <dependency groups>`_.

To find these dependencies, check each library file for its ``PYDEPS``. A command
that can find these is::

find lib -name "*.py" -exec awk '/PYDEPS = \[/,/\]/' {} +

If run from the base directory of a charm, this will show all the PYDEPS declarations
from all loaded charm libs.

Include extra files
-------------------

A ``poetry`` plugin only includes the contents of the `src` and `lib` directories
as well as the generated virtual environment. If other files were previously included
from the main directory, they can be included again using the
:ref:`craft_parts_dump_plugin`:

.. code-block:: yaml
:emphasize-lines: 5-9
parts:
my-charm: # This can be named anything you want
plugin: poetry
source: .
version-file:
plugin: dump
source: .
stage:
- charm_version
.. _dependency groups: https://python-poetry.org/docs/managing-dependencies/#dependency-groups
.. _Poetry: https://python-poetry.org
.. _postgresql-operator: https://github.com/canonical/postgresql-operator/blob/3c7c783d61d4bee4ce64c190a9f7d4a78048e4e7/pyproject.toml#L22-L35
126 changes: 126 additions & 0 deletions docs/howto/charm-to-python.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
.. _howto-migrate-to-python:

Migrate from the ``charm`` plugin to the ``python`` plugin
==========================================================

The ``python`` Charmcraft plugin offers a faster, stricter means of packing an operator
charm with a virtual environment. This guide shows how to migrate from a charm using
the default ``charm`` plugin to using the ``python`` plugin.

Update ``charmcraft.yaml``
--------------------------

The first step is to update ``charmcraft.yaml`` to include the correct parts definition.
Depending on the history of a specific charm, it may not have an explicitly-included
``parts`` section determining how to build the charm. In this case, a ``parts`` section
can be created as follows:

.. code-block:: yaml
parts:
my-charm: # This can be named anything you want
plugin: python
source: .
python-requirements:
- requirements.txt # Or whatever your requirements file is called.
Ensure a recent enough version of ``pip``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The ``python`` plugin requires at least pip 22.3, released in October 2022. If the
charm's base uses an older version of pip, a newer version can be installed in the
build environment using a dependency part. The following ``parts`` section can be
used in place of the section above to upgrade pip for charms that build on Ubuntu
22.04 or earlier:

.. code-block:: yaml
:emphasize-lines: 2-5,7
parts:
python-deps:
plugin: nil
override-build: |
/usr/bin/python3 -m pip install pip==24.2
my-charm: # This can be named anything you want
after: [python-deps]
plugin: python
source: .
python-requirements:
- requirements.txt # Or whatever your requirements file is called.
Flatten ``requirements.txt``
----------------------------

One difference between the ``python`` plugin and the ``charm`` plugin is that the
``python`` plugin does not install dependencies, so the ``requirements.txt`` file
must be a complete set of packages needed in the charm's virtual environment.

.. note::
There are several tools for creating an exhaustive ``requirements.txt`` file.
Charmcraft works with any as long as it generates a requirements file that ``pip``
understands. Because different versions of packages may have different
dependencies, it is recommended that the requirements file be generated using a
tool that will lock the dependencies to specific versions.
A few examples include:

- `uv export <https://docs.astral.sh/uv/reference/cli/#uv-export>`_
- `pip-compile <https://pip-tools.readthedocs.io/en/stable/cli/pip-compile/>`_
- `pip freeze <https://pip.pypa.io/en/stable/cli/pip_freeze/>`_

A basic ``requirements.txt`` file for a charm with no dependencies other than the
Operator framework may look something like::

ops==2.17.0
pyyaml==6.0.2
websocket-client==1.8.0

To check that the virtual environment for the charm would be valid, activate an
empty virtual environment and then run::

pip install --no-deps -r requirements.txt
pip check

Include charm library dependencies
----------------------------------

Unlike the ``charm`` plugin, the ``python`` plugin does not install the dependencies
for included charmlibs. If any of the charm libraries used have PYDEPS, these will
need to be added to a requirements file as well.

.. note::
All requirements files are included in the same ``pip`` command to prevent
conflicting requirements from overriding each other. However, this means
that a charm will fail to build if it has conflicting requirements. A single
``requirements.txt`` file, while not mandatory, is recommended.

To find these dependencies, check each library file for its ``PYDEPS``. A command
that can find these is::

find lib -name "*.py" -exec awk '/PYDEPS = \[/,/\]/' {} +

If run from the base directory of a charm, this will show all the PYDEPS declarations
from all loaded charm libs, which can be used to help generate the input for a tool
that generates ``requirements.txt``.

Include extra files
-------------------

A ``python`` plugin only includes the contents of the `src` and `lib` directories
as well as the generated virtual environment. If other files were previously included
from the main directory, they can be included again using the
:ref:`craft_parts_dump_plugin`:

.. code-block:: yaml
:emphasize-lines: 7-11
parts:
my-charm: # This can be named anything you want
plugin: python
source: .
python-requirements:
- requirements.txt # Or whatever your requirements file is called.
version-file:
plugin: dump
source: .
stage:
- charm_version
10 changes: 10 additions & 0 deletions docs/howto/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.. _howto:

How-To
******

.. toctree::
:maxdepth: 2

charm-to-poetry
charm-to-python
11 changes: 11 additions & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,20 @@ Most of Charmcraft's documentation is available there.
:maxdepth: 1
:hidden:

howto/index
reference/index
explanation/index

.. grid:: 1 1 2 2

.. grid-item-card:: `Tutorial <https://juju.is/docs/sdk/tutorials>`_

**Get started** with a hands-on introduction to Charmcraft

.. grid-item-card:: :ref:`How-to guides <howto>`

**Step-by-step guides** covering key operations and common tasks

.. grid:: 1 1 2 2
:reverse:

Expand Down
1 change: 0 additions & 1 deletion docs/reference/plugins/python_plugin.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@ During the build step, the plugin performs the following actions:
4. It copies any existing ``src`` and ``lib`` directories from your charm project into
the final charm.


Example
-------

Expand Down

0 comments on commit 95567bd

Please sign in to comment.