This folder contains a basic set of utilities for orchestrating massive automated evaluations of Pyret programs.
The two utilities course staff will find helpful are evaluate.sh, which evaluates a single test suite against an single implementation, and evaluate-many.sh which schedules evaluations on a GridEngine cluster.
The evaluate-many.sh consumes its joblist from stdin. Each line of input should be a space-delmited (1) path to an implementation, (2) path to a test suite, and (3) path to a folder to store its output. For a complete example of invoking evaluate-many.sh, see evaluate-example.sh.
Be sure to modify evaluate.sh such that PATH includes an appropriate version of node.