Test Plan

If not specifically mentioned, each test shall have an issue, or belong to an issue on RedMine to track its progress. The results of the tests may be placed in the comments of the Redmine issue (recommended) or placed on the blog at the tester's discretion. However, all test results must be made visible to all members of the team, and the members of the team must acknowledge results before an issue can be closed.

Unit Testing

Currently, our application does not have a wide coverage of automated tests. Our intention is to have unit tests in place for key methods, such as 'GetNextAvailableIP'.

Unit testing aims to isolate the different layers of the application. VMAT runs on a Model View Controller framework, so testing will address the Model and Controller layers. It is too difficult to automate testing for the View layer (ie. the web interface) using unit tests, therefore it will have to suffice to test them using the acceptance tests.

The development team plans to make good use of mocking in the unit tests. The application interfaces with a number of different external systems, such as a SQL database and VMware. These systems can be slow to interface with; speeding up these actions by mocking the systems will make testing much more effective and painless.

Results of these tests may be placed in the comments of the Redmine issues that are associated with them or the acceptance tests, or they may be placed on the blog at the discretion of the tester. Test results must be made visible to all members of the team. If a major feature has broken, then an email to the team is necessary to ensure visibility.

Below is a list of items to test and their current testing status:

• Controller actions
• Model actions
• Create virtual machines
• Edit virtual machines
• Get IP address from pool
• Archive
• Backup
• Toggle virtual machines power status
• Undo pending operation
• Domain login

Integration Testing

Integration testing will ensure that each use-case executes correctly across all layers of the application. Testing all functionality across the entire system is overkill for the application, so much of it will be done by simply running the application and manually performing each use-case to determine whether it meets specification. In the case of creating/editing VMs and scheduler actions, we will manually check the data to ensure the data was properly stored.

Results of these tests may be placed in the comments of the Redmine issues that are associated with them or the acceptance tests, or they may be placed on the blog at the discretion of the tester. Test results must be made visible to all members of the team. If a major feature has broken, then an email to the team is necessary to ensure visibility.

Below is a list of major items to consider when integration testing:

• Successful connection to VMware server
• Successful connection to Domain
• Creation/Editing propagates all functionality correctly through the application
• Scheduler performs operations and updates data objects

Deployment Testing

Local testing for deployment is done using 'VMAT2' as a test bed, which runs on Windows Sever 2008. 'VMAT' is used to replicate the client's VMware server. There are currently plans to add another server, appropriately titled 'VMAT3', which will run on Ubuntu rather than Windows Server, in order to more closely resemble the client's environment. We shall deploy the build to VMAT2, then test to ensure the deployment worked.

Due to the problems the client found during his functional tests, we have created a new policy which dictates that we shall test all the major and minor features of VMAT before we deploy the application to the client (see acceptance testing below).

Acceptance Testing

The client provides feedback for each release based on his own testing of the application. Preferably, the release packages should be installed to the client's own servers. The development team is able to provide access to their local 'VMAT2' server via VPN, should deployment to the client's servers fail.

The client has provided a list of operations/use-cases he will run to determine if the application is up to specification. The client is likely to follow the same pattern he used the first time, which involves manually running through each test.

The team will do a round of acceptance testing before delivery to the client. The team member(s) that perform this test must adhere to these rules when performing the test.

1. Use a clean master branch and deploy to vmat2.csse.rose-hulman.edu. If in doubt as to whether the branch is clean or not (consider you might have git ignored some local changes), you should git clone [email protected]:tiger12506/VMAT.git into a separate directory in order to receive a clean copy of the master branch before deploying.

2. Run each of the below acceptance tests in each of the supported browsers (as per requirements document). The results should be structured in such a way as to place each browser in a separate column, with each row dedicated to a particular test below. Results may be either pass, fail, or -- which would indicate that comments are present in the rightmost column.

3. Before deployment occurs, every team member must review acceptance tests and okay the result. During this process, issues may be created concerning the outcome of the tests.

The client's tests are as follows:

• Admin Button
• Save Configuration Button
• Check Web.config Settings Link
• See Exception Log/Info Link
• Run all scheduled tasks Link
• Links to all pages (in the footer)
• Create Button
• New VMs with New Project
• New VMs with Existing Project
• Undo create VM Button
• Show More VM Info
• Show Less VM Info
• Start/Shutdown VM
• Close Project Button

In addition, we also suggest these items, based on updated functionality/bug fixes:

• Archive VM
• Undo archive VM
• Archive Project
• Backups Daily
• Backups Weekly
• Set scheduling times
• Set IP Address pool, check with create page
• Synchronize with VMware
• Edit IP Address