Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create CI/CD for SLICOT #2

Open
bnavigator opened this issue Feb 9, 2021 · 2 comments
Open

Create CI/CD for SLICOT #2

bnavigator opened this issue Feb 9, 2021 · 2 comments

Comments

@bnavigator
Copy link

Originally posted by @andreasvarga in #1 (comment)

  1. It would be great if we could provide a testing facility for SLICOT for
    the purpose of CI. Such facility exists also for LAPACK, but we lack
    experience in this area.

  2. I see one of the main usages for SLICOT as a computational kernel for
    control related computations in conjunction with environments like Matlab,
    Python, Julia, Octave. Bulding interfaces to these tools (wrappers,
    mex-files, etc.) involves (probably) the generation of appropriate shared
    libraries for different architectures and compilers. It would be usefull
    for many potential users to have this process largely automated.

@bnavigator
Copy link
Author

#1 (comment) by @ilayn

Just to give a very basic introduction to the CI tools. Currently a few major vendors provide free CI/CD tools to open source codebases. GitHub itself, Azure Pipelines, CircleCI and AppVeyor are the most common ones. TravisCI was the most prominent player but lately they removed their free support so in case you saw their name that's why.

The main workflow is the Issue reports and Pull Requests and this is the annoying part that any change to be made should better be first committed as a pull request. The reason for this is that CI tools and GitHub are integrated each other hence when a new pull request is opened, CI tools receives a webhook signal and starts incorporating the changes to the build and run all the existing tests. This is very helpful to catch the initial tirivial problems should there be any and to see any existing behavior is broken. Once every test passes the CI check then codebase owners, in this case you both, can decide whether a piece of code should go in or not and click the merge button.

Obviously, the more tests there are the better it is. Hence all kinds of trivial input tests wrong strings, wrong options, wrong calls etc. can be tested and get out of the way.

Another advantage of these CI tools is the possibility of running builds against different machine architectures Linux, Win, OSX are very typical choices and some other esoteric options such as ARM and Raspbian etc. are also available.

  1. I see one of the main usages for SLICOT as a computational kernel for control related computations in conjunction with environments like Matlab, Python, Julia, Octave. Bulding interfaces to these tools (wrappers, mex-files, etc.) involves (probably) the generation of appropriate shared libraries for different architectures and compilers. It would be useful for many potential users to have this process largely automated.

Yes indeed one of the nice byproducts of these CI/CD tools is that you can save the so-called artifacts, the resulting files after the build and use them as releasing a new version. This is a bit trickier so I'll skip the details for now.

@bnavigator
Copy link
Author

  1. It would be great if we could provide a testing facility for SLICOT for
    the purpose of CI. Such facility exists also for LAPACK, but we lack
    experience in this area.

The Slycot build process and CI is obviously very Python centric. We use NumPy's F2PY to create a Python specific library, called an 'extension', from the Fortran sources, and then test our wrapper functions written in Python against it using Pytest. Unfortunately, none of this applies directly to this repository.

A first step towards a SLICOT CI/CD is to use one of the cloud facilities mentioned by @ilayn to compile the Fortran sources into a (Fortran) library. You already provide a makefile, but there is no configuration helper how to find the compiler and the BLAS/LAPACK libraries and headers. The paths to these could be hardcoded into the CI recipes for the various machines. A better option would be to use autoconf/automake or CMake to find all the build requirements and construct the build commands. Slycot's machinery uses CMake (calling f2py and wrapped by yet another layer named Scikit-buid for the Python setuptools system). I see the LAPACK repository on Github does also use CMake. Providing CMake recipes will also benefit potential users of SLICOT, who want to build it on their own.

Once the library is built, CMake's ctest (also used by LAPACK) could compile test programs which use the library and check their outputs. Many SLICOT routines already come with example code and example result data. These could be reused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant