OVERVIEW:
Labour is a suite of libraries and test-drivers to benchmark the durability of WSGI web servers under various stress conditions pertaining to misbehaving workers. Labour facilitates the chores of benchmarking WSGI Web Servers, such as starting/shutting servers, generating HTTP requests and reporting results. In addition, Labour provides a WSGI Application which does all sorts of things ("Behaviours") to stress its containing server.

Labour was written by Yaniv Aknin ([email protected]).

QUICK START:
In the 'bin' directory you will find various executables, try them with '-h'. The simplest test you can run is 'basic', but probably the more interesting one is 'multitest'.

Here's a sample exeuction of basic:
    $ bin/basic 
    Running basic test against WSGIRef.

    Test of 512 requests complete in 8.37 seconds (61.16req/s).
    512 requests (100.00%) returned OK and 0 (0.00%) failed.

    $ 

A small clarification about wording: when Labour says the request returned "OK" it means the request returned "as expected". Returning as expected is not the same thing as returning HTTP_OK (status 200). This is because some test behaviours can, for example, dictate the request should return with a status code of INTERNAL_SERVER_ERROR. Labour knows kind of result to expect, so if HTTP_OK was returned when INTERNAL_SERVER_ERROR was expected, this will be shown as a failure.

QUICK CODE START:
Labour's main components are a Server, a Client, a Behaviour, a Policy and a Report. A Server is an object which abstracts away the details of spawning a WSGI Server, shutting it down and wiring it to a test application. The Client is a rather simple HTTP client based on a home-brewed parallelism solution (multiprocessed, not multithreaded) and urllib2. A Behaviour is an object describing what the Client can request the WSGI Application to do (return a page, sleep and return a page, wedge, segfault, etc) as well as what the client should expect following that request (HTTP OK, HTTP Not Found, socket error, etc). After instantiation, the Client should be configured with one or more Behaviours, along with a 'weight' assigned to each Behaviour. The Policy dictates how the Client will choose Behaviours from the weighted list of Behaviours, for example, sequentially or at random. A Report is an object which takes one or more of the result tuples which the client returns after execution, formats and displays them.

Here's a minimalist piece of code which demonstrates code usage of Labour:
    import sys
    import httplib

    from labour import servers
    from labour import tester
    from labour import behaviours
    from labour import reports
    from labour.tester import policies

    client = tester.Client(policy=policies.Random)
    client.add(behaviours.PlainResponse(), weight=98)
    client.add(behaviours.Sleeping(sleep_duration=0.5), weight=1)
    client.add(behaviours.PlainResponse(status=httplib.INTERNAL_SERVER_ERROR),
                                        weight=1)

    with servers.WSGIRef() as server:
        result = client.execute(iterations=512, policy=policies.Random)

    reports.PlainReport(result).emit('ascii')


TODO: 
 * Add support for more servers.
   Aspen, Gunicorn, MagnumPy, uWSGI and of course, mod_wsgi
 * Add pretty HTML output support to reports.
 * Enrich server creation API (specify number of processes, etc).
 * Research occasional very-low failure rate of some servers;
    I already found two bugs in WSGI Server (see Concurrence's and FAPWS3's
    code), but I suspect a bug of my own as well.
 * Grep source for FIXME (and HACK!) and resolve them all :)

ACKNOWLEDGEMENTS:
This software was inspired by a blog post by Ian Bicking (http://blog.ianbicking.org/2010/03/16/web-server-benchmarking-we-need/) but is in no way affiliated with him.

This software contains trivial server wiring code-fragments by Nicholas Piel, as published here (http://nichol.as/benchmark-of-python-web-servers), but is in no way affiliated with him.