Plugin API for Usage

Overview

We need a way to automate testing of Galaxy and q2galaxy integration. Creating a usage syntax in the plugin API will allow us to create test-cases for our testing plugin.

Main challenges include:

• Registration (what does it look like)
• Specific syntax (what is registered)
• Data (where does it come from)
• Expected output (how to describe for a test, in a galaxy-compatible way)

Registration and Syntax

Unknown. However, initial prototypes of this only need to involve the framework and this repo, so we can iteratively design this as needed before making it "production ready".

Input/output data for usage

Data-free usage

Data-enabled usage

Potential Reuse

This registration could be used for:

• Automated framework integration tests per-plugin
• Documentation

This syntax could be re-used:

• as a backing structure for parsed provenance.
• as a description of user-generated workflows (non-pipeline)
• as interface agnostic tutorial commands (provided a interface-specific driver exists)

Notes from 2019.03.13 Discussion

• Usage examples will be defined per-action --- actions will accept a list of usage examples that are relevant to the action.
• An API for defining usage data will need to be drafted, this will likely look like some kind of hash with named references, and some way of "registering" the actual data implementation.
• Usage examples will be defined "abstractly" --- this will likely look a lot like the Artifact API or Pipeline. Interfaces will define one or more "drivers" that will contain the interface-specific formatting rules. This will be some kind of interface that needs to be implemented by the interface. Drivers will be responsible for rendering the abstract usage examples into some kind of interface-specific output (e.g. q2cli renders a shell script).
• Tutorials will be implemented separately from this initial take on the Usage API, but will likely re-use much of the machinery. Tutorials will be implemented as RST documents with one or more custom RST directives defined by us that will allow an interface driver to be utilized as part of the RST render. This will most likely be a job for Library.
• Interface drivers will need to implement minimum functions: how to render an import data (this will use the data API referenced above), an export, and an Action runner. Optionally, drivers can define how to indicate comments, and a handful of simple test assertion behavior.
• There are a few parts of the Framework that might benefit from being refactored or implemented against the Usage API, including: Provenance Capture, Provenance Parsing, Pipelines.
• Drivers will be configurable by the interface to allow for different modes of rendering (for example, q2cli could render with or without line breaks, or with --verbose).
• Action "defaults" (and how they are rendered by a driver) could be implemented as some driver-specific config, but, it is probably more of a special-case that the Framework could/should know more about, and communicate that to the interface.
• The goal here is to be able to generate tool descriptions for Galaxy that include unit tests specified by the Usage API. We could also leverage this on Busywork with q2cli --- unit tests could include running the new UX-testing plugin created for q2/Galaxy.