Possible improvements to plugin functionality

[lbianchi-lbl]

While working on @ksbeattie's PR #906, an issue was encountered with the FOQUS plugin discovery mechanism, where adding a comment header at the top of a plugin .py file causes the file to stop being recognized as a plugin. This prompted a discussion on the current implementation of the plugin system implementation, along with possible ways to improve it.

Usage and Requirements

• Plugins in FOQUS can be defined as Python modules that are imported at runtime based on some condition
• There are two main purposes for which plugins are used:
  • Modules that are part of the FOQUS codebase (i.e. .py files that are installed when installing FOQUS), but that rely on optional dependencies, i.e. third-party packages that are not part of FOQUS's core dependencies
    • e.g. BFGS.py
  • Modules that are dynamically generated at runtime, i.e. .py files that do not exist when FOQUS is installed, but are automatically generated by FOQUS as part of certain workflows
    • e.g. SM optimizer plugins built from the options specified by users in the GUI
• The plugin functionality is not user-facing, i.e. users are not expected to create or manage their own custom plugins (except as an internal implementation detail)
• Importing Python modules causes the code defined within them to be executed; therefore, to avoid accidentally executing arbitrary code from spurious .py files in the plugin search path, the plugin functionality should offer a way of identifying plugins (i.e. discriminating Python files that are plugins from Python files that are not plugins) that does not rely on importing the file as a module

Current solution

• Plugins are identified using a sentinel comment that must be defined within the first n characters of a .py file
  • This fulfills the requirement of identifying plugins without having to import the file
  • n is configurable but still requires manual adjustment if any content (code, docstring, comments) are added to the top of the file

Possible alternatives

A: directory/filename pattern

• Identify plugins by requiring plugins to be placed in a particular directory or having a particular file name pattern
  • e.g. for optional dependencies plugins, this could be foqus_lib/plugins (possibly with subdirectories to further distinguish between different plugin categories)
  • for dynamically generated plugins, this could be either:
    • A subdirectory of the working directory plus a .py file with an unique identifier (if needed), e.g. foqus_plugins/sm_optimizer_123456.py
    • A .py file in the working directory with a prefix, e.g. foqus_plugin_sm_optimizer_123456.py

B: Python code defined in file

• Identify plugins by requiring a sentinel Python construct to be defined in the file
  • This could be e.g. a constant (FOQUS_PLUGIN = True) or a function (def register_foqus_plugin(): pass)
  • The file could be inspected to search for the sentinel definition without importing it using e.g. the ast module of the standard library

Further reading

• https://packaging.python.org/guides/creating-and-discovering-plugins
• https://dev.to/charlesw001/plugin-architecture-in-python-jla
• https://alysivji.github.io/simple-plugin-system.html

[eslickj]

I think for user-written of dynamically generated plugins, it probably makes sense to either have a plugins sub-directory of the working directory where we put only plugin files, or maybe a specific file name pattern to identify plugins anywhere in the working directory and sub-directories.