Skip to content

Commit

Permalink
Update xtrigger plugins docs
Browse files Browse the repository at this point in the history
  • Loading branch information
MetRonnie committed Feb 19, 2024
1 parent d88206d commit f1230d2
Show file tree
Hide file tree
Showing 3 changed files with 73 additions and 54 deletions.
4 changes: 4 additions & 0 deletions src/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,10 @@
autosummary_generate_overwrite = True
autosummary_imported_members = False

# Autodoc type hints can look very messy if they are included in the signature,
# so we only include them in the description instead:
autodoc_typehints = 'description'

# Mapping to other Sphinx projects we want to import references from.
# NOTE: To search available references, use:
# $ python -m sphinx.ext.intersphinx <url>/objects.inv | less
Expand Down
40 changes: 33 additions & 7 deletions src/plugins/xtriggers/index.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
Xtrigger Plugins
======================================

Xtrigger plugins allow you to install and use xtriggers without them being
in your ``CYLC_PYTHONPATH``.
.. versionadded:: 8.3

Xtrigger plugins allow you to install and use
:ref:`xtriggers <Section External Triggers>` without them being
in ``<workflow-dir>/lib/python/`` or ``$CYLC_PYTHONPATH``.

.. seealso::

* :ref:`Built-in Clock Triggers`
* :ref:`Built-in Workflow State Triggers`


Built In Plugins
Expand All @@ -15,6 +23,7 @@ Cylc Flow provides the following xtriggers.
:template: docstring_only.rst

cylc.flow.xtriggers.echo.echo
cylc.flow.xtriggers.wall_clock.wall_clock
cylc.flow.xtriggers.workflow_state.workflow_state
cylc.flow.xtriggers.xrandom.xrandom

Expand All @@ -26,20 +35,37 @@ Cylc Flow provides the following xtriggers.
Developing ``xtrigger`` plugins
-------------------------------

Cylc uses entry points registered by setuptools to search for xtrigger
plugins.
Cylc uses the ``cylc.xtriggers`` entry point registered by setuptools to search
for xtrigger plugins. Each xtrigger is registered individually.

Example
^^^^^^^

Plugins are registered by registering them with the ``cylc.xtriggers``
entry points. Each xtrigger is registered individually.
Consider a package called ``my_package`` with the following structure:

.. code-block:: python
:caption: ``my_package/foo.py``
def foo():
...
def bar():
...
.. code-block:: python
:caption: ``my_package/baz.py``
def baz():
...
These xtriggers can be registered in the package's ``setup.cfg`` or
``pyproject.toml`` file.

.. code-block:: ini
:caption: ``setup.cfg``
[options.entry_points]
cylc.xtriggers =
cylc.xtriggers =
foo = my_package.foo
bar = my_package.foo
baz = my_package.baz
Expand Down
83 changes: 36 additions & 47 deletions src/user-guide/writing-workflows/external-triggers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,11 @@ broker.
Cylc has several built-in external trigger functions:

- clock triggers - see :ref:`Built-in Clock Triggers`
- inter-workflow triggers - see :ref:`Built-in Workflow State Triggers`
- :ref:`Built-in Clock Triggers`
- :ref:`Built-in Workflow State Triggers`

Trigger functions are normal Python functions, with certain constraints as
described below in:

- custom trigger functions - see :ref:`Custom Trigger Functions`
described below in :ref:`Custom Trigger Functions`.

External triggers are configured in the
:cylc:conf:`flow.cylc[scheduling][xtriggers]` section.
Expand All @@ -62,9 +60,7 @@ in :ref:`ClockTriggerTasks`.
Clock triggers, unlike other trigger functions, are executed synchronously in
the main process. The clock trigger function signature looks like this:

.. code-block:: python
wall_clock(offset=None)
.. autofunction:: cylc.flow.xtriggers.wall_clock.wall_clock

The ``offset`` argument is a datetime duration (``PT1H`` is 1
hour) relative to the dependent task's cycle point (automatically passed to the
Expand Down Expand Up @@ -130,10 +126,7 @@ tasks off of remote task statuses or messages in other workflows.

The workflow state trigger function signature looks like this:

.. code-block:: python
workflow_state(workflow, task, point, offset=None, status='succeeded',
message=None, cylc_run_dir=None, debug=False)
.. autofunction:: cylc.flow.xtriggers.workflow_state.workflow_state

The first three arguments are compulsory; they single out the target workflow name
(``workflow``) task name (``task``) and cycle point
Expand Down Expand Up @@ -240,12 +233,7 @@ properties:
- In ``<workflow-dir>/lib/python/``;
- Anywhere in your ``$CYLC_PYTHONPATH``;
- Defined using the ``cylc.xtriggers`` entry point for an installed
package.

.. seealso::

:ref:`developing.xtrigger.plugins` for more information on writing
xtrigger plugins.
package - see :ref:`developing.xtrigger.plugins`

- They can take arbitrary positional and keyword arguments
- Workflow and task identity, and cycle point, can be passed to trigger
Expand All @@ -255,10 +243,11 @@ properties:
- If a trigger function depends on files or directories (for example)
that might not exist when the function is first called, just return
unsatisfied until everything required does exist.
- The module containing the xtrigger function may also contain a "validate"
function taking arguments ``args``, ``kwargs`` and ``signature``, and
raising ``cylc.flow.exceptions import WorkflowConfigError`` if validation
fails. See :ref:`xrandom.validate` for an example of a validate function.
- The module containing the xtrigger function may also contain a ``validate``
function taking a single argument, which is a dictionary of all the arguments
passed to the xtrigger function in ``flow.cylc``. It should raise an
exception if validation fails. See :ref:`xrandom.validate` for an example
of a validate function.

.. note::

Expand Down Expand Up @@ -356,9 +345,7 @@ time (useful for testing the effect of a long-running trigger function
- which should be avoided) and has a configurable random chance of
success. The function signature is:

.. code-block:: python
xrandom(percent, secs=0, _=None, debug=False)
.. autofunction:: cylc.flow.xtriggers.xrandom.xrandom

The ``percent`` argument sets the odds of success in any given call;
``secs`` is the number of seconds to sleep before returning; and the
Expand All @@ -377,38 +364,40 @@ An example xrandom trigger workflow:
Validation example using xrandom
""""""""""""""""""""""""""""""""

The xrandom xtrigger module might also contain a ``validate`` function.
The ``xrandom`` xtrigger module contains a ``validate`` function.

This will be run on the inputs to an xtrigger, and should raise
a ``WorkflowConfigError`` with a meaningful description if a condition
is not met.
This will be run on the inputs to the xtrigger when calling
``cylc validate`` or before the workflow starts, and should raise an exception
with a meaningful description if a condition is not met.

A simplified example looks like this:

.. code-block:: python
from cylc.flow.exceptions import WorkflowConfigError
def validate(args: List, kwargs: Dict, signature: str) -> None:
# Total number of args & kwargs makes sense:
totalargs = len(args) + len(kwargs)
if totalargs < 1 or totalargs > 4:
raise WorkflowConfigError(
f'{signature} xtrigger should have between 1 and 4 arguments')
# Check whether we have a percentage argument:
if not args and not 'percent' in kwargs:
raise WorkflowConfigError(
f'{signature} xtrigger should have a percentage argument')
def validate(args: Dict[str, Any]) -> None:
# Check that percentage is a reasonable value:
percentage = args[0] if args else kwargs['percentage']
percent = args['percent']
if (
not isinstance(percentage, (int, float))
or percentage > 100
or percentage < 0
not isinstance(percent, (float, int))
or not (0 <= percent <= 100)
):
raise WorkflowConfigError(
f'{signature} xtrigger percentage argument'
f' must be a number between 0 and 100, not {percentage}')
"'percent' should be a float between 0 and 100"
)
See below for a link to the full ``validate`` function:

.. autofunction:: cylc.flow.xtriggers.xrandom.validate

.. tip::

The arguments you call the xtrigger function with are automatically
validated against the function signature, so you don't necessarily need
to check for the presence of arguments or their types in the validate
function. However, you may want to check that the values are of the correct
type or within a certain range.


.. _Current Trigger Function Limitations:
Expand Down

0 comments on commit f1230d2

Please sign in to comment.