…3.03.09

Boverhof.user plugins backend.2023.03.09

* adding attribute check to avoid throwing exception

* fixed code formatting

…error (CCSI-Toolset#1149)

* Try adding isinstance check

* Fix import name

* Changes to allow install for arm64 darwin.

* Changes to allow install for arm64 darwin.

* fixed logic for conditional install of pyQt5

* update rank function to use pure python library and move away from mlrose-hiive

* tweak the way we pass permutation argument

* pinning python-tsp version

* tweaking ranked_df in the test_rank function. The new ordering algorithm always start in the node zero and goes reversed to what the mlrose TSP used to do.

* update documentation to reflect change in the library used for ordering designs

* Update text in Example USF-1

* Update text in Example USF-2

* Update text in Example USF-3

* Add updated figures for Example USF-1

* Add updated figures for Example USF-3

* Update figures in Example USF-1

* Update figures and captions for Example USF-3

* Update text in Example NUSF-1

* Add new screenshots to Example NUSF-1

* Add example file for Example NUSF-1

* Update figures and captions in Example NUSF-1

* Update text in Example IRSF-1

* Add new screenshots for Example IRSF-1

* Update figures in Example IRSF-1

…error (CCSI-Toolset#1159)

…t#1164)

* Brady's gradient generation script

* Adding helpful docstrings and comments

* Brady's working files for gradient model regression

* Integrate Brady's regression script into main file, clean up

* fix formatting for black and pylint

* Generalize for multiple outputs, add settings optimization loop

* Add documentation for gradient tool

* Manually run spellchecker on ML AI Plugin files

* Fix links and file names

* Add option to use simple partial derivative in lieu of chain rule formula

* Fix comment

* run black again

* minor typo

* Give docs page reference name

---------

Co-authored-by: Keith Beattie <[email protected]>

…CCSI-Toolset#1169)

* Example training script, model file

* Framework to support SMT models

* Update documentation

* Update tests, rename test model

* correct text

* run black

* run typos manually on doc file

* review comment

---------

Co-authored-by: Ludovico Bianchi <[email protected]>

…#1177)

* Add NLOpt installation instructions back

* Fix code block syntax

---------

Co-authored-by: Ludovico Bianchi <[email protected]>

* first attempt at spellchecker, listing all file types and excluding some known exceptions

* Move file to repo root

* Exclude JSON and R files as we are less sure if they can be corrected

* Add common abbreviations to avoid false positives

* Fix "releative"

* Fix "permision"

* Fix "Setings"

* Fix "untill"

* Fix "buttom"

* Fix "alloted"

* Fix "controling"

* Fix "auxiliary"

* Fix "constriant"

* Fix most typos in non-code files

* Change config file path in GHA workflow

* Fix "seperate"

* Fix "excpetion"

* Fix "handeling"

* Fix "deafult(s)"

* Fix typos in code comments or strings

* Add exclusions for typos for identifiers in non-internal scopes

* Fix typos in code

* Fix typo in figure name

* Fix "thershold"

* Format code

* Fix "NoInteractiveSer"

* Fix "ghostOuput"

* Uncommit and ignore Pipfile

---------

Co-authored-by: Ludovico Bianchi <[email protected]>
Co-authored-by: Keith Beattie <[email protected]>

…pport for Python 3.7 (CCSI-Toolset#1173)

* Add push trigger while WIP

* Add Python 3.9 to compare

* Remove PR jobs while WIP

* Add pull_request trigger while WIP

* Try removing version constraint for Pywin32

* Modify job to run from PR branch

* Add back stable matrix item to avoid failures

* Add back pywin32 version constraint to verify correlation to failures

* Revert "Add back pywin32 version constraint to verify correlation to failures"

This reverts commit c78e3ae.

* Revert "Remove PR jobs while WIP"

This reverts commit e3e428d.

* Add exact version constraint for pywin32

* Add pywin32 version to conda install commands in docs and CI

* Remove Python 3.7 from installation docs and CI matrices

* Revert temporary changes to nightlies CI workflow

* fix FixedFormatter issue when plotting with matplotlib

* embedding all SDoE plots into QDialog windows

* fixing test_plot_utils.py

* fixing test_plot_utils.py

* fixing test_plot_utils.py

* showing the 2 NUSF plots in the same QDialog window

* adding QDialog window titles

* fix previous commit

* fixing sdoe gui test

* importing Window class from sdoePreview

* adding tests for plot_weights and plot_pareto functions

* copy_from_package nusf and irsf candidate files

* adding NUSF and IRSF results in JSON files to test plotting functions

* adding more SDoE GUI testing to cover sdoePreview dialog

* Add MVP for ML/AI surrogate plugin

* Add copy of data file for testing

* Formatting changes

* Resolve pylint error

* Initial implementation of GUI tests for surrogate plugins

* Skip tests if tensorflow not available

* Add utility method to export input and output vars directly as dataframe

* Add utility method to get currently selected input and output vars data

* Add example code for accessing selected input and output vars and data

* Format

* Remove f-string syntax incompatible with Python 3.7

* Updated to use variables from flowsheet

* Remove unused and/or redundant local variables for training data

* Intercept stdout so that training output can be shown in GUI

* Change model serialization file format to resolve deprecation warning

* Use flowsheet for pytorch_nn.py and scikit_nn.py

* Formatting

* Displays pytorch_nn.py output in FOQUS GUI

* Handle pylint error

* Generalize importorskip using parametrization

* Use different session file for NN training

* Add variable selection before executing

* Format code

* Draft description for pytorch_nn.py and scikit_nn.py

Also fixed spacing in keras_nn.py description

* Added input validation and custom file name option

* Add ml_ai subdirectory to default path for output file

* Fix typos identified by Spell Checker

---------

Co-authored-by: Ludovico Bianchi <[email protected]>
Co-authored-by: Keith Beattie <[email protected]>
Co-authored-by: Brandon Paul <[email protected]>
Co-authored-by: Brandon Paul <[email protected]>

* Initial documentation for keras_nn plugin

* Fix doc issues

* Make docs generic to include pytorch and sci-kit learn

* Add note mentioning planned updates

---------

Co-authored-by: Ludovico Bianchi <[email protected]>

…et#1170)

* Enable wrong-import-order Pylint check

* Investigate why enabled check is not enforced

* Edit CLI flags on the command that actually gets run

* Make pytest jobs dependent on pylint

* Run isort 5.12.0

* Run Black

* Move conditional imports after normal imports to allow isort to work

* Run isort --profile=black

* Fix remaining file

* Run isort on recently added or modified files

* Add Sphinx extension adding role to render and validate repository paths

* Convert existing data paths in flowsheet tutorial to use repopath role

* Convert most of the examples path references to use :path: Sphinx role

* Add sphinxcontrib.jquery to extensions to restore search functionality

* Format with Black

* Remove unused first iteration

* Use os.fspath() consistently

* Add more specific validation for files

* Format code

* change objection with objective

* fix typo 

fixing typo caught by spelling checker

* Enable CI checks for Python 3.10 and 3.11

* Address Pyqt set value type error

* Formatting

* Update Pylint/Astroid requirements

* Temporarily skip Pylint for Python 3.8

* Resolve TypeError in SDOE widget

* Try adding macos-arm64 with exclusions for unavailable combinations

* Remove job-level if due to matrix context being (ironically) unavailable

* Try streamlining logic for pytest steps with and without coverage

* Add step for ad-hoc Conda setup on macOS ARM

* Try using .bash_profile to resolve failures in Conda setup step

* Add PyQt 5 requirement for macOS ARM

* Ensure step outcome matches command exit code

* Apply strict error shell options to setup-foqus Action as well

* Try requiring a newer matplotlib version for Python 3.11+

* Fix PEP508 syntax

* Make code using matplotib.Axes.lines compatible with 3.6+ API

* Try streamlining requirements

* Remove matrix.unavailable include as not needed

* Update to non-deprecated Action versions

* Update actions/upload-artifact

* Update hopefully last Action with deprecated version

* Add loose xfail marker for test being tracked in CCSI-Toolset#1195

* Try updating codecov-action to v3

* Make ARM default for macOS

* Try installing Miniconda version through setup-miniconda Action

* Remove joblib requirement that shouldn't be needed anymore

* Try to see what breaks with Python 3.12 while we're at it

* Replace deprecated/removed imp with importlib

* Actually remove the imp import

* Update default Python version

* Hardcode Python version since env context is not available in matrix def

* Revert accidental indentation

* Try if pytest version is causing issues

* Try adding the tensorflow constraint with outer quotes for YAML

* Use full 3.8-3.12 matrix for Pylint

* Try if crash on 3.8 is resolved with Pylint 3.1.0

* Use Pylint 2 with Python 3.8 to avoid crash

* Format with Black

* Fix Pylint directive after formatting

* Specify Python version

* Use requirements file for ML/AI dependencies

---------

Co-authored-by: Ludovico Bianchi <[email protected]>

…mSinter and TurbineLite (CCSI-Toolset#1204)

Co-authored-by: Keith Beattie <[email protected]>

* adding type annotations to SDoE

* fixing typo in session.py

* using typing.List and typing.Dict

* fix issues in df_utils.py

* add TypedDict to irsf

* add TypedDict to nusf

* add TypedDict to plot_utils

* add TypedDict to usf

---------

Co-authored-by: Ludovico Bianchi <[email protected]>

…olset#1203)

* Add Turbine Configuration section

* try file role

Co-authored-by: Ludovico Bianchi <[email protected]>

* flip slashes

* flip slashes back, try double slash

* try just double backslash, no file role

* Try if using :file: role renders correctly for RTD build

---------

Co-authored-by: Ludovico Bianchi <[email protected]>

* parallelize USF algorithm using Dask

* code formatting

* removing import that is not yet ready

* put back update best_val removed during refactor

* adding minimax mode to USF test

* do timing in benchmark

* fixing import order per linter

* adding dataframe extenstion from dask and pinning to <2024.3 due to bug reported to dask

* cleaning up seed usage in dask USF test and benchmark

* tweak to work as expected with dask 2024.3

* restrict pytest version

* NUSF algorithm parallelization using Dask

* fixing up dask tests that load files

* Format with Black

* make dask CLI flag `kebab-kase`

Co-authored-by: Ludovico Bianchi <[email protected]>

* adding type annotations and reusing nusf functions

* adding logging when loading dask client fails

* fixing type hints to work with python 3.8

* fix type annotation for python 3.8

---------

Co-authored-by: Ludovico Bianchi <[email protected]>

…lset#1194)

* implementing difficulty column and ordering depending on input difficulty.

* remove order.py

* add test for order_blocks function

* add difficulty to config files so tests pass

* save one final design

* fix test_order

* Format files with Black

---------

Co-authored-by: Ludovico Bianchi <[email protected]>

* Bump black from 22.3.0 to 24.3.0

Bumps [black](https://github.com/psf/black) from 22.3.0 to 24.3.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](psf/black@22.3.0...24.3.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <[email protected]>

* Format using Black 24.3.0

* Run Black 24.3.0 also on files outside foqus_lib

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ludovico Bianchi <[email protected]>

* Run addheader after updating copyright year to 2024

* Update copyright year outside headers

* Remove spurious trailing newline in header_text.txt

* Found a few more files that needed the copyright year updated

* Adding usage to .addheader.yml and running it again

* restricting matplotlib due to issues when using 3.8

* cast parameter for setValue to int

* Add link to CCSI-Toolset#1211

---------

Co-authored-by: Ludovico Bianchi <[email protected]>

…SI-Toolset#1215)

* Add arange as known exception

* Update mainWindow.py

* Update graph.py

* Update ALAMO.py

* Update surrogate.py

* Update irsf.py

 probably initials, add periods so it's not flagged

* Add yhat, iy as known variable names to except

* Update session.py

* update smt genn to new syntax, create example of jenn and retrain models

* update code, tests and dependency list

* update documentation

* run black

* Update copyright year in new file

* fix pandas deprecation for UQ

* fix pandas deprecation for SDoE and ODoE

* casting "time" column in UQ result as str instead of datetime

* casting column dtypes in the dataframe initialization

…foqus_cloud

* Define logger variable once

* Add verification for PSUADE executable and exit with error if not found

* reorganize ML_AI example files and regenerate models with latest H5-compatible TensorFlow

* update tests to verify new file structure is working

* first attempt at updating examples, methods, tests to support Keras 3

* pickup copied zip file

* run black

* unfix TensorFlow to use latest Keras

* add example of mea column model with no custom layer

* add handling for no custom layer and for TFSM models

* run black

* update documentation

* test running ML AI tests on Python 3.9 and 3.11

* add prints to help debugging

* more print statements and items for debugging

* run black, again

* revert debugging changes, remove file size condition

* Update checks.yml

* Temporarily set pytest to exit after 3 failures to debug macOS timeouts

* Remove job dependencies to work around repeated spell-check errors

* Try increasing timeout causing test failures on macOS

* Try requiring different TF version for macOS

* Avoid installing TF at all on macOS

* Update expected exception for TF import

---------

Co-authored-by: Ludovico Bianchi <[email protected]>

* Use dedicated dependent job for Codecov upload

* Empty commit to trigger CI

* Fix indentation

* Tweaking test_plot_utils and test_sdoe. Adding test_df_utils

* clean up imports

---------

Co-authored-by: Keith Beattie <[email protected]>
Co-authored-by: Ludovico Bianchi <[email protected]>

* Update settings.rst

* Update install_optional.rst

* Update alamo.rst

* TestNUSF in progress

For issue CCSI-Toolset#1218

* Working NUSF Dropdown Selection

* Formatting

* Implement Run SDoE in Test

* Add Test for NUSF Plotting

* Check Generated Design rows instead of run button

* Working test for NUSF-2 Tutorial

* Placing NUSF and NUSF2 tests together causes other error

* NUSF and NUSF2 now in separate files

The two tests succeed when run individually, but not when all tests are run together

* Change window fixture scope to "module"

* Run black .

* Revert conftest.py change

* Move NUSF tests to test_sdoe.py

* Try uploading coverage report even if pytest step fails

---------

Co-authored-by: Pedro Sotorrio <[email protected]>
Co-authored-by: Ludovico Bianchi <[email protected]>