List of Utility Tools

This list is intended to be a starting point for documenting useful supporting tools for developers and users of Pyomo and IDAES related modeling tools. It is hoped that by informing developers and users of the available tools, duplication of effort (and proliferation of similar tools) can be avoided.

Pyomo

pyomo.util

• calc_var_value
  • Performs a single variable Newton solve to calculate the value of a variable from a constraint
• slices
  • slice_component_along_sets - A utility for taking a component data object and replacing indices corresponding to certain sets by slices. Indices may be in the component or any of its parent blocks.
• subsystems
  • create_subsystem_block - Returns a block that contains the user's constraints and variables as references. This block may be solved as a subsystem containing only the variables and constraints the user is interested in.
  • TemporarySubsystemManager - A context manager that temporarily fixes and deactivates specified variables and constraints. Useful for fixing inputs then solving a square problem.
  • ParamSweeper - An iterable context manager meant for testing that a solve calculates the values it is meant to calculate. A better name may be ParamSweepTester

pyomo.contrib

• incidence_analysis
  • maximum_matching - Performs a maximum matching of rows and columns in a sparse matrix.
  • block_triangularize - Partitions the rows and columns of a square sparse matrix into strongly connected components in a topological order. These are exactly the diagonal blocks that may be solved/factorized independently of one another in an order such that this decomposition is valid.
  • IncidenceGraphInterface - An interface into the maximum_matching and block_triangularize functionality for Pyomo models

IDAES

General Comments

• apps and surrogates packages are targeted for overhauls when resources are available. surrogates will hopefully be improved later this year.
• documentation of many of these tools is non-existent
• Tools are poorly organized, making it hard to find things. E.g. there are multiple modules for plotting related functions which are spread across multiple modules.
  • Suggest a series of subfolders to organize tools based on use
    • tools to manipulate models
    • tools to analyze model structure
    • tools to work with model results (plotting, save/load, other UI)
• There appears to be some duplication of tools even within IDAES alone

Tools

• apps

  • caprese - tools for NMPC
    • categorize - categorize variables and equations into differential and algebraic. This may be useful beyond Caprese.
    • dynamic_block - an extension of Pyomo blocks that contains methods for working with time-indexed variables
    • controller - an extension of dynamic_block that contains methods for setting up piecewise constant inputs and tracking objective functions
    • rolling - tools for storing data from rolling horizon simulations. Very rough. Should be replaced with Pandas data structures.
    • util - contains a tool for integrating a model over a subset of the time domain
  • ddm_learning - why is this still here? DDM learning tools have their own package now
  • matopt - tools for materials optimization. Somewhat orphaned from rest of project?
  • uq - tools for uncertainty propagation and quantification
    • sens - tools for calcuating model sensitivity using sipopt and/or kaug

• commands - command line tools for IDAES

  • convergence, get_extensions, get_examples

• convergence - application of convergence tool

• core/util

  • convergence - base methods for convergence tool

  • config - validation methods for ConfigDicts

  • constants - class containing definitions of common physical constants with units

  • dyn_utils - tools for working with time indexed IDAES models

    • lots of useful methods for manipulating dynamic models
    • Most of these have been superseded by (a) the Pyomo DAE flatten module and (b) the ability to get a ComponentUID of a slice.

  • exceptions - IDAES custom Exception types

  • expr_doc - Tools for documenting Expressions (?). Converts Pyomo Expression objects to sympy representations.

  • homotopy - IPOPT-based homotopy solver for Pyomo models

  • initialization - common methods for initializing IDAES models

    • propagate_state - ** appears to be duplicated by misc.copy_port_values
    • initialize_by_time_element - "Integrates" a dynamic IDAES model

  • math - implementations of smooth approximations for abs, min, max, sqrt and log functions for use in Pyomo models

  • misc - miscellaneous utility functions from IDAES

    Available functions

      * get_solver - wrapper to return solver objects with ability to define defaults
      * add_object_reference - method to work around Pyomo's single-owner limitation. `References` are generally preferred, but this method is necessary when dealing with scalar components as a `Reference` always has an index.
      * extract_data
      * TagReference
      * svg_tag
      * copy_port_values - **appears to duplicate `initialization.propagate_state`**
      * set_params_from_config - method to lookup ConfigBlock for parameter values, with unit conversion. **Some aspects of this maybe obsolete with updates to how Pyomo handles units.**
      * VarLikeExpression - extension of Pyomo `Expression` object which adds Exception messages for common activities on `Vars`. This is intended for specialized use in outlet Ports where we sometime use `Expressions` instead of `Vars` and uses might not be aware of this.
    

  • model_diagnostics - tools for debugging and assessing potential points of failure in a model
    • DegeneracyHunter
  • model_serializer - save and load model data from json
  • model_statistics - functions for counting number of variables, constraints and blocks on a model with various sub-categories
  • phase_equilibrium - tool for developing Txy plots
  • plot - tools for plotting dynamic results
  • scaling - tools for scaling models
  • tables - tools for generating stream tables
  • testing - tools to assist with writing tests for IDAES models
  • unit_costing - tools for developing costing models of equipment I think this should really be an IDAES sub-package in its own right

• dmf - IDAES DMF tools

• surrogates

  • helmet
    • DataManipulation
    • GAMSDataWrite, GAMSWrite - how do these differ from the Pyomo GAMS writer?
    • Plotting
  • pysmo - I see FeatureScaling classes in a number of places. Could this be made common?
    • sampling - sampling tools. This looks like a good candidate for generalisation

• ui

  • flowsheet.py - I see a FlowsheetSerializer function here. Should check for overlap with model serializer in utils.

ProteusLib / NAWI-AML

• parameter_sweep - tool for doing parameter sweeps on IDAES/Pyomo models (essentially sensitivity studies?)
  • supports mpi4py (parallelization)
  • likely duplicates features available in parmest, and other Python tools
  • probably duplicates a lot of what is in the Convergence Tester tool

DISPATCHES

Other Useful Tools