Xtriggers: Doc Fixes

.github/workflows/test.yml

@@ -79,7 +79,7 @@ jobs:
 
       - name: build & test
         run: |
-          make html spelling linkcheck doctest \
+          make html spelling linkcheck \
             SPHINXOPTS='-Wn --keep-going' FORCE_COLOR=true
 
       - name: debug sphinx failure

Makefile

@@ -20,6 +20,7 @@ clean:
 	# remove auto-generated content
 	rm -rf src/plugins/main-loop/built-in
 	rm -rf src/plugins/install/built-in
+	rm -rf src/plugins/xtriggers/built-in
 	rm -rf src/user-guide/task-implementation/job-runner-handlers
 
 watch: clean

README.md

@@ -183,7 +183,7 @@ allows it to pick up changes to autodocumented items in the source code.
 
 ```console
 # note: -W tells Sphinx to fail on warnings
-$ make html linkcheck doctest SPHINXOPTS='-W'
+$ make html linkcheck SPHINXOPTS='-W'
 ```
 
 ### Deploying

bin/version

@@ -27,7 +27,6 @@ from pkg_resources import parse_version
 DUMMY_FORMATS = [
     'doctrees',
     'linkcheck',
-    'doctest',
     'coverage'
 ]
 

src/conf.py

@@ -34,7 +34,6 @@
     # sphinx built-in extensions
     'sphinx.ext.autodoc',
     'sphinx.ext.autosummary',
-    'sphinx.ext.doctest',
     'sphinx.ext.graphviz',
     'sphinx.ext.intersphinx',
     'sphinx.ext.napoleon',
@@ -111,6 +110,10 @@
 autosummary_generate_overwrite = True
 autosummary_imported_members = False
 
+# Autodoc type hints can look very messy if they are included in the signature,
+# so we only include them in the description instead:
+autodoc_typehints = 'description'
+
 # Mapping to other Sphinx projects we want to import references from.
 # NOTE: To search available references, use:
 # $ python -m sphinx.ext.intersphinx <url>/objects.inv | less

src/dictionaries/words

@@ -46,6 +46,7 @@ filesystem
 filesystems
 fillchar
 foo
+fpath
 freeform
 fullmatch
 globals
@@ -128,6 +129,7 @@ py
 pythonically
 queueing
 qux
+randint
 rc
 refactor
 regex
@@ -147,6 +149,7 @@ ret
 retrigger
 retriggered
 retriggering
+rmtree
 roadmap
 ruleset
 rulesets
@@ -155,10 +158,12 @@ runnable
 runtime
 schd
 screenshot
+secs
 setup
 setups
 shebang
 shutdown
+shutil
 situ
 speedup
 ssh
@@ -185,6 +190,7 @@ symlink
 symlinked
 symlinking
 symlinks
+sys
 taskdef
 templated
 templating
@@ -194,6 +200,7 @@ timeouts
 timestep
 timezone
 timezones
+tuple
 uiserver
 unpaused
 unpausing
@@ -208,6 +215,7 @@ vs
 wallclock
 websocket
 websockets
+whitelist
 whitelisting
 whitespace
 workflow

src/plugins/xtriggers/index.rst

@@ -1,52 +1,69 @@
 Xtrigger Plugins
 ======================================
 
-Xtrigger plugins allow you to install and use xtriggers without them being
-in your ``CYLC_PYTHONPATH``.
+.. versionadded:: 8.3
 
+   Xtrigger plugins allow you to install and use
+   :ref:`xtriggers <Section External Triggers>` without them being
+   in ``<workflow-dir>/lib/python/`` or ``$CYLC_PYTHONPATH``.
 
-Built In Plugins
-----------------
+.. seealso::
 
-Cylc Flow provides the following xtriggers.
+   * :ref:`Built-in Clock Triggers`
+   * :ref:`Built-in Workflow State Triggers`
+   * :ref:`Built-in Toy Xtriggers`
 
-.. autosummary::
-   :toctree: built-in
-   :template: docstring_only.rst
 
-   cylc.flow.xtriggers.echo
-   cylc.flow.xtriggers.workflow_state
-   cylc.flow.xtriggers.xrandom
-
-.. Note: Autosummary generates files in this directory, these are cleaned
-         up by `make clean`.
+.. _developing.xtrigger.plugins:
 
 Developing ``xtrigger`` plugins
 -------------------------------
 
-Cylc uses entry points registered by setuptools to search for xtrigger
-plugins.
+Cylc uses the ``cylc.xtriggers`` entry point registered by setuptools to search
+for xtrigger plugins. Each xtrigger is registered individually.
 
 Example
 ^^^^^^^
 
-Plugins are registered by registering them with the ``cylc.xtriggers``
-entry points. Each xtrigger is registered individually.
+Consider a package called ``my_package`` with the following structure:
+
+.. code-block:: python
+   :caption: ``my_package/foo.py``
+
+   def foo():
+       ...
+
+   def bar():
+       ...
+
+.. code-block:: python
+   :caption: ``my_package/baz.py``
+
+   def baz():
+       ...
+
+These xtriggers can be registered in the package's ``setup.cfg`` or
+``pyproject.toml`` file.
 
 .. code-block:: ini
    :caption: ``setup.cfg``
 
    [options.entry_points]
-       cylc.xtriggers =
-           foo = my_package.foo:foo
-           bar = my_package.foo:bar
-           baz = my_package.baz:baz
+   cylc.xtriggers =
+       foo = my_package.foo
+       bar = my_package.foo
+       baz = my_package.baz
 
 .. code-block:: toml
    :caption: ``pyproject.toml``
 
    [project.entry-points."cylc.xtriggers"]
-   foo = "my_package.foo:foo"
-   bar = "my_package.foo:bar"
-   baz = "my_package.baz:baz"
+   foo = "my_package.foo"
+   bar = "my_package.foo"
+   baz = "my_package.baz"
+
+.. tip::
 
+   It is recommended to implement only one xtrigger per module. This allows
+   you to write a ``validate`` function for each xtrigger - see
+   :ref:`xrandom.validate` for an example.

src/user-guide/writing-workflows/external-triggers.rst

@@ -34,13 +34,11 @@ broker.
 
 Cylc has several built-in external trigger functions:
 
-- clock triggers - see :ref:`Built-in Clock Triggers`
-- inter-workflow triggers - see :ref:`Built-in Workflow State Triggers`
+- :ref:`Built-in Clock Triggers`
+- :ref:`Built-in Workflow State Triggers`
 
 Trigger functions are normal Python functions, with certain constraints as
-described below in:
-
-- custom trigger functions - see :ref:`Custom Trigger Functions`
+described below in :ref:`Custom Trigger Functions`.
 
 External triggers are configured in the
 :cylc:conf:`flow.cylc[scheduling][xtriggers]` section.
@@ -62,9 +60,7 @@ in :ref:`ClockTriggerTasks`.
 Clock triggers, unlike other trigger functions, are executed synchronously in
 the main process. The clock trigger function signature looks like this:
 
-.. code-block:: python
-
-   wall_clock(offset=None)
+.. autofunction:: cylc.flow.xtriggers.wall_clock.wall_clock
 
 The ``offset`` argument is a datetime duration (``PT1H`` is 1
 hour) relative to the dependent task's cycle point (automatically passed to the
@@ -130,10 +126,7 @@ tasks off of remote task statuses or messages in other workflows.
 
 The workflow state trigger function signature looks like this:
 
-.. code-block:: python
-
-   workflow_state(workflow, task, point, offset=None, status='succeeded',
-                  message=None, cylc_run_dir=None, debug=False)
+.. autofunction:: cylc.flow.xtriggers.workflow_state.workflow_state
 
 The first three arguments are compulsory; they single out the target workflow name
 (``workflow``) task name (``task``) and cycle point
@@ -269,29 +262,34 @@ Custom Trigger Functions
 Trigger functions are just normal Python functions, with a few special
 properties:
 
-- they must:
+- They must:
 
-  - be defined in a module with the same name as the function, unless
+  - Be defined in a module with the same name as the function, unless
     provided using the ``cylc.xtriggers`` entry point;
   - be compatible with the same Python version that runs the scheduler
     (see :ref:`Requirements` for the latest version specification).
 
-- they can be located either:
+- They can be located either:
 
-  - in ``<workflow-dir>/lib/python/``;
-  - anywhere in your ``$CYLC_PYTHONPATH``;
-  - or defined using the ``cylc.xtriggers`` entry point for an installed
-    package.
+  - In ``<workflow-dir>/lib/python/``;
+  - Anywhere in your ``$CYLC_PYTHONPATH``;
+  - Defined using the ``cylc.xtriggers`` entry point for an installed
+    package - see :ref:`developing.xtrigger.plugins`
 
-- they can take arbitrary positional and keyword arguments
+- They can take arbitrary positional and keyword arguments
   (except ``sequential``, which is reserved - see :ref:`Sequential Xtriggers`)
-- workflow and task identity, and cycle point, can be passed to trigger
+- Workflow and task identity, and cycle point, can be passed to trigger
   functions by using string templates in function arguments (see below)
-- integer, float, boolean, and string arguments will be recognized and
+- Integer, float, boolean, and string arguments will be recognized and
   passed to the function as such
-- if a trigger function depends on files or directories (for example)
+- If a trigger function depends on files or directories (for example)
   that might not exist when the function is first called, just return
   unsatisfied until everything required does exist.
+- The module containing the xtrigger function may also contain a ``validate``
+  function taking a single argument, which is a dictionary of all the arguments
+  passed to the xtrigger function in ``flow.cylc``. It should raise an
+  exception if validation fails. See :ref:`xrandom.validate` for an example
+  of a validate function.
 
 .. note::
 
@@ -343,6 +341,7 @@ The :term:`scheduler` manages trigger functions as follows:
   the workflow log in debug mode (stdout is needed to communicate return values
   from the sub-process in which the function executes)
 
+.. _Built-in Toy Xtriggers:
 
 Toy Examples
 ^^^^^^^^^^^^
@@ -353,14 +352,9 @@ echo
 The trivial built-in ``echo`` function takes any number of positional
 and keyword arguments (from the workflow configuration) and simply prints
 them to stdout, and then returns False (i.e. trigger condition not
-satisfied). Here it is in its entirety.
-
-.. code-block:: python
+satisfied).
 
-   def echo(*args, **kwargs):
-       print("echo: ARGS:", args)
-       print("echo: KWARGS:", kwargs)
-       return False, {}
+.. autofunction:: cylc.flow.xtriggers.echo.echo
 
 Here's an example echo trigger workflow:
 
@@ -389,9 +383,7 @@ time (useful for testing the effect of a long-running trigger function
 - which should be avoided) and has a configurable random chance of
 success. The function signature is:
 
-.. code-block:: python
-
-   xrandom(percent, secs=0, _=None, debug=False)
+.. autofunction:: cylc.flow.xtriggers.xrandom.xrandom
 
 The ``percent`` argument sets the odds of success in any given call;
 ``secs`` is the number of seconds to sleep before returning; and the
@@ -405,6 +397,47 @@ An example xrandom trigger workflow:
 .. literalinclude:: ../../workflows/xtrigger/xrandom/flow.cylc
    :language: cylc
 
+.. _xrandom.validate:
+
+Validation example using xrandom
+""""""""""""""""""""""""""""""""
+
+The ``xrandom`` xtrigger module contains a ``validate`` function.
+
+This will be run on the inputs to the xtrigger when calling
+``cylc validate`` or before the workflow starts, and should raise an exception
+with a meaningful description if a condition is not met.
+
+A simplified example looks like this:
+
+.. code-block:: python
+
+   from cylc.flow.exceptions import WorkflowConfigError
+
+   def validate(args: Dict[str, Any]) -> None:
+       # Check that percentage is a reasonable value:
+       percent = args['percent']
+       if (
+           not isinstance(percent, (float, int))
+           or not (0 <= percent <= 100)
+       ):
+           raise WorkflowConfigError(
+               "'percent' should be a float between 0 and 100"
+           )
+
+See below for a link to the full ``validate`` function:
+
+.. autofunction:: cylc.flow.xtriggers.xrandom.validate
+
+.. tip::
+
+   The arguments you call the xtrigger function with are automatically
+   validated against the function signature, so you don't necessarily need
+   to check for the presence of arguments or their types in the validate
+   function. However, you may want to check that the values are of the correct
+   type or within a certain range.
+
+
 .. _Current Trigger Function Limitations:
 
 Current Limitations

src/user-guide/writing-workflows/jinja2.rst

@@ -227,14 +227,6 @@ In the argument list of a filter or test function, the first argument is
 the variable value to be filtered or tested, and subsequent arguments can be
 whatever is needed. Currently three custom filters are supplied:
 
-.. import the filters to allow their doctests to pass (make doctest)
-
-.. testsetup::
-
-   from cylc.flow.jinja.filters.pad import pad
-   from cylc.flow.jinja.filters.strftime import strftime
-   from cylc.flow.jinja.filters.duration_as import duration_as
-
 .. autosummary::
    :nosignatures: