List of Utility Tools

This list is intended to be a starting point for documenting useful supporting tools for developers and users of Pyomo and IDAES related modeling tools. It is hoped that by informing developers and users of the available tools, duplication of effort (and proliferation of similar tools) can be avoided.

Pyomo

IDAES

General Comments

• apps and surrogates packages are targeted for overhauls when resources are available. surrogates will hopefully be improved later this year.
• documentation of many of these tools is non-existent
• Tools are poorly organized, making it hard to find things. E.g. there are multiple modules for plotting related functions which are spread across multiple modules.
  • Suggest a series of subfolders to organize tools based on use
    • tools to manipulate models
    • tools to analyze model structure
    • tools to work with model results (plotting, save/load, other UI)
• There appears to be some duplication of tools even within IDAES alone

Tools

• apps
  • capresse - tools for NMPC
    • categorize - categorize Vars in differential and algebraic -This looks like it could go in model statistics
    • controller - check for overlap with IDAES PID block
    • dynamic_block - how does this differ from standard IDAES blocks? Should it inherit from PrcoessBase?
    • rolling - tools for rolling horizon models
    • util - is naything here more general than just Capresse?
  • ddm_learning - why is this still here? DDM learning tools have their own package now
  • matopt - tools for materials optimization. Somewhat orphaned from rest of project?
  • uq - tools for uncertainty propagation and quantification
    • sens - tools for calcuating model sensitivity using sipopt and/or kaug
• commands - command line tools for IDAES
  • convergence
  • get_extensions
  • get_examples
• convergence - application of convergence tool
• core/util
  • convergence - base methods for convergence tool
  • config - validation methods for ConfigDicts
  • constants - class containing definitions of common physical constants with units
  • dyn_utils - tools for working with time indexed IDAES models
    • get_activity_dict - map activated and deactivated Blocks and Constraints in model
    • get_fixed_dict - map fixed and unfixed Vars in model
    • deactivate_model_at - deactivate Blocks and Constraints at specified index
    • deactivate_constraint_unindexed_by - deactivate all Constraints which are not indexed by given Set
    • fix_vars_unindexed_by - fix all Vars which are not indexed by given Set
    • get_location_of_coordinate_set - For a SetProduct and some 1-dimensional coordinate set of that SetProduct, returns the location of an index of the coordinate set within the index of the setproduct.
    • get_index_of_set- For some data object of an indexed component, gets the value of the index corresponding to given 1-dimensional Pyomo set.
    • get_implicit_index_of_set(comp, wrt) - For some data object contained (at some level of the hierarchy) in a block indexed by wrt, returns the index corresponding to wrt in that block.
    • get_derivatives_at(b, time, pts) - Finds derivatives with respect to time at points specified. No distinction made for multiple derivatives or mixed partials.
    • path_from_block(comp, blk, include_comp=False) - Returns a list of tuples with (local_name, index) pairs required to locate comp from blk
    • find_comp_in_block(tgt_block, src_block, src_comp, allow_miss=False) - This function finds a component in a source block, then uses the same local names and indices to try to find a corresponding component in a target block. This is used when we would like to verify that a component of the same name exists in the target block, as in model predictive control where certain variables must be correlated between plant and controller model.
    • find_comp_in_block_at_time(tgt_block, src_block, src_comp, time, t0, allow_miss=False) - This function finds a component in a source block, then uses the same local names and indices to try to find a corresponding component in a target block, with the exception of time index in the target component, which is replaced by a specified time point. This is used for validation of a component by its name in the case where blocks may differ by at most time indices, for example validating a steady-state model or a model with a different time discretization.
    • copy_non_time_indexed_values(fs_tgt, fs_src, copy_fixed=True, outlvl=idaeslog.NOTSET) - Function to set the values of all variables that are not (implicitly or explicitly) indexed by time to their values in a different flowsheet.
    • copy_values_at_time(fs_tgt, fs_src, t_target, t_source, copy_fixed=True, outlvl=idaeslog.NOTSET) - Function to set the values of all (explicitly or implicitly) time-indexed variables in a flowsheet to similar values (with the same name) but at different points in time and (potentially) in different flowsheets.
  • exceptions - IDAES custom Exception types
  • expr_doc - Tools for documenting Expressions (?). Converts Pyomo Expression objects to sympy representations.
  • homotopy - IPOPT-based homotopy solver for Pyomo models
  • initialization - common methods for initializing IDAES models
    • fix_state_vars - iterate over a Port and fix all members. Return a map of what members were fixed.
    • revert_state_vars - Take a map of members of a Port and unfix those that were fixed.
    • propagate_state(stream, direction="forward") - This method propagates values between Ports along Arcs. Values can be propagated in either direction using the direction argument.
    • solve_indexed_block - Work around to allow IndexedBlocks to be sent to a solver.
    • initialize_by_time_element(fs, time, **kwargs) - Function to initialize Flowsheet fs element-by-element along ContinuousSet time. Assumes sufficient initialization/correct degrees of freedom such that the first finite element can be solved immediately and each subsequent finite element can be solved by fixing differential and derivative variables at the initial time point of that finite element.
  • math - implementations of smooth approximations for abs, min, max, sqrt and log functions for use in Pyomo models
  • misc - miscellaneous utility functions from IDAES
    • get_solver - wrapper to return solver objects with ability to define defaults
    • add_object_reference - method to work around Pyomo's single-owner limitation. References are generally preferred, but this method is necessary when dealing with scalar components as a Reference always has an index.
    • extract_data
    • TagReference
    • svg_tag
    • copy_port_values - appears to duplicate initialization.propagate_state
    • set_params_from_config - method to lookup ConfigBlock for parameter values, with unit conversion. Some aspects of this maybe obsolete with updates to how Pyomo handles units.
    • VarLikeExpression - extension of Pyomo Expression object which adds Exception messages for common activities on Vars. This is intended for specialized use in outlet Ports where we sometime use Expressions instead of Vars and uses might not be aware of this.
  • model_diagnostics - tools for debugging and assessing potential points of failure in a model
    • DegeneracyHunter
  • model_serializer - save and load models from json
  • model_statistics - functions for counting number of variables, constraints and blocks on a model with various sub-categories
  • phase_equilibrium - tool for developing Txy plots
  • plot - tools for plotting dynamic results
  • scaling - tools for scaling models
  • tables - tools for generating stream tables
  • testing - tools to assist with writing tests for IDAES models
  • unit_costing - tools for developing costing models of equipment I think this should really be an IDAES sub-package in its own right
• dmf - IDAES DMF tools
• surrogates
  • helmet
    • DataManipulation
    • GAMSDataWrite, GAMSWrite - how do these differ from the Pyomo GAMS writer?
    • Plotting
  • pysmo - I see FeatureScaling classes in a number of places. Could this be made common?
    • sampling - sampling tools. This looks like a good candidate for generalisation
• ui
  • flowsheet.py - I see a FlowsheetSerializer function here. Should check for overlap with model serializer in utils.

Proteus/Lib / NAWI-AML

DISPATCHES

Other Useful Tools