Add automated testing / CI

[zachmullen]

No description provided.

[hmeine]

Yes, that'd be a good idea as well. For real testing, one would've to actually call CLIs, though, which makes things difficult. Probably it's worth starting with parsing XML.

[zachmullen]

Agreed, starting with parsing XML gets us most of the way there. For the executable part, I'd suggest just mocking subprocess.Popen and set the return_value on the mock (here's an example of doing that).

[hmeine]

Mocking could be an idea, yes, OTOH the most interesting cases would be stuff like CLIs missing DLLs when not called from within Slicer, etc. ;-)

[hmeine]

I would like to introduce tests, but could use some help with the infrastructure. My preferred tool would be pytest, BTW.

• Where should tests be located?
• Enabling Travis CI for Python 2+3, with coverage analysis?

If you can help, know someone who can help, or know a good resource to look at (e.g., a role model project's repository or some short piece of documentation), please speak up! (ping: @jcfr ?)

[zachmullen]

Where should tests be located?

I think the most common place to put tests is as a sibling of the package directory, i.e. a top level tests directory in the repo, making it so you don't have to exclude it from the packaging step in setup.py.

Enabling Travis CI for Python 2+3, with coverage analysis?

This is easy in either Travis or CircleCI, whichever you happen to like more (personally I favor CircleCI, but they are very very similar.) In travis, this just involves including two different python versions in your travis.yml (example), and in Circle you can do two parallel builds, one for each python version (example). For coverage, the coverage.py tool is the one to use, and for coverage reporting and visualization I'd recommend submitting to codecov, it has really nice visualizations including diff-based coverage on PRs.

[zachmullen]

I'd also strongly recommend adding static analysis as part of CI, with flake8 being my tried-and-true favorite tool there.

[jcfr]

Here is a work in progress topic to add build of the package across multiple version of python.

By using travis, we can easily leverage the matrix build feature whereas for circleci you have to make use of circleci-matrix (that we are taking over with @thewtex for continuing its maintenance)

I will now fix the remaining flake8 error

[hmeine]

Thanks to both of you; I am already in love with pyflakes (and had locally started setting up automatic analysis with it), so flake8 looks awesome.