-
Notifications
You must be signed in to change notification settings - Fork 5
run multiple OpenBSD regression tests and collect results
License
bluhm/regress-all
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Run a bunch of OpenBSD regression tests in a row. The intension is to have a reliable set of tests and run them on a regular basis. The result summary is written to a file, log output of each test is available. A second prupose is to run OpenBSD performance tests. A certain cvs checkout corresponding to a date is created. The build environment is based on the previous release and adapted with commands from current.html. This is mostly updating the build tools to compile a kernel from that cvs checkout date. The results are grouped by run date, cvs checkout date, and repetitions. The tests are run multiple times to detect deviations. Tests consist of iperf3, tcpbench, and kernel compile time. Additional features are running ports tests, building release, and testing network interfaces in multiple configurations. Buildquirks.pm - create tool chain to compile kernel from certain date Hostctl.pm - run commands on all tests hosts to configure them Html.pm - utility functions to create html pages with results Logcmd.pm - common log funtions for run.pl, setp.pl, and setup.pl Machine.pm - functions to setup a single machine Netstat.pm - parse netstat statistics and show coounter differences Testvars.pm - variables with test names for perform and netlink bsdcons.pl - collect kernel output from console of selected machine build-ot14.list - setup builds special tools needed on test machine ot14 cleanup.pl - remove old test logs to recover disk space cmd-ot1.list - setup executes commands after installation of ot1 cvsbuild.pl - checkout kernel at certain cvs date and install it cvslog.pl - collect all cvs commit messages between two dates env-ot1.sh - environment variables to be set on the test machine ot1 favicon.ink.svg - favicon for bluhm.genua.de created with inkscape favicon.svg - favicon converted to small SVG force-ssh.pl - restrict ssh commands on linux perform servers gnuplot.pl - generate gnuplot graphics for perform html keys.sh - generate gnuplot icons for legend in html graphics this is started by gnuplot.pl mktestlist.pl - find possible names of tests in regress directory make.pl - run make build and release makealign.sh - align object files of kernel on perform machine mktestlist.pl - print the Makefile leaves of the regress directory net.pl - run network regression tests netlink-html.pl - convert netlink test results into html table netlink.pl - set up and run one network regression test on test machine this is started by net.pl once.pl - run performance test once, used for manual testing perform-html.pl - convert performance test results into html table perform.pl - run one set up performace test on test machine, this is started by step.pl or once.pl pkg-ot1.list - setup installs packages on test machine ot1 plot.gp - gnuplot script used by gnuplot.pl ports.list - list of /usr/ports subdirectories that should be tested portstest.pl - run port test for set of ports on test machine this is started by test.pl power.pl - power up and down machine to save cooling power reboot.pl - reboot machine, after optionally reordering kernel regress-html.pl - convert all regression test results into html table regress.pl - run one openbsd regression test on test machine, this is started by run.pl release.pl - build release from /usr/src this is started by make.pl run.pl - do regression tests, add this to cron job running-html.pl - create html table with all tests currently running setup-html.pl - convert machine setup logs into html table setup.pl - install one test machine, this is started by run.pl and step.pl site.list - list of config files that must be installed on test machine step.pl - do performance tests, add this to cron job test.pl - do ports tests, add this to cron job test.list - list of /usr/src/regress subdirectories that should be tested testsuite-html.pl - create html of test suites run during regress Several modes for run.pl and setup.pl exist to prepare the test machines. build - build system from source /usr/src cvs - cvs update /usr/src and make obj install - install from snapshot keep - keep existing software on machine and skip setup kernel - build kernel from source /usr/src/sys upgrade - upgrade with snapshot Daily test results are published here: http://bluhm.genua.de/regress/results/regress.html Performance timelines are published here: http://bluhm.genua.de/perform/results/perform.html Thanks to genua GmbH https://www.genua.de/ for sponsoring this work.
About
run multiple OpenBSD regression tests and collect results
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published