AWS setup for QIIME Workshops
QIIME 2 has succeeded QIIME 1 as of 1 January 2018. QIIME 1 is no longer officially supported, as our development and support efforts are now focused entirely on QIIME 2. For more information, see our blog post: QIIME 2 has succeeded QIIME 1.
View previous content of this page
These instructions are written for QIIME 1.9.1. This document describes how to set up AWS instances for use in a QIIME workshop, and assume that users will be working through the Illumina Overview Tutorial using a Jupyter (IPython) Notebook server running on AWS. Other analyses are possible using this setup, so these instructions are a good starting point whenever you want to teach QIIME 1.9.1.
This section covers steps that you should take in the weeks or months leading up to your QIIME workshop.
Consider requesting workshop participants to fill out a questionnaire to gauge their familiarity with the command line, previous use of QIIME and QIIME installation. You can find a questionnaire that was previously used here, though some parts of this are not relevant (e.g., requirement that users have installed QIIME before the workshop).
TODO: develop a standard pre-workshop questionnaire?
Students don't need to install QIIME on their laptops, the workshop will use AWS. Students should use Google Chrome during the workshop, which should be installed before they arrive.
The organizers will want to have an agenda and schedule for the workshop before it begins, so they can share it with attendees and to confirm that you'll be covering what they want you to cover. Here is an example of what this might look like for a two-day workshop. You should discuss the break and lunch schedule with the organizers, as there are often constraints that you'll need to work around. Especially for multi-day workshops, it's convenient to do this in Google Docs (or similar) so that you can update this as the schedule changes.
This section covers steps that should be taken on the day before your workshop. Do not put this off until the last minute, as issues do come up and you'll need time to address them. These issues usually have to do with the facilities that are available for the workshop - most often wifi-related.
Before the workshop begins you'll want to boot up all of the AWS instances that you'll use, and test the full tutorial using the same network that the students will be using. This will help you to identify any connectivity issues, etc. For example, conference and university "guest" networks often will not have port 22 open for ssh connections. If ssh connectivity is important for your workshop, be sure to figure out if this will be an issue. If you're only using the IPython notebook, ssh is not necessary for students but instructors will need ssh access to configure the instances, and students will need to be able to connect to the port assigned to the IPython Notebook server in their web browsers.
You should start one m2.4xlarge instance per ~15 workshop participants to avoid having connections to the notebook be rejected. We also recommend starting a dedicated instance for the instructor - that way if connections get rejected for some users it doesn’t affect the instructors' presentation.
Start your instances via the AWS interface, following the instructions in QIIME's Working with AWS documentation. Be sure to configure the security group to allow for use of the IPython Notebook, as described here.
Once the instances are running, you'll need to change to a new directory so that you'll have enough storage for multiple users to be able to run the tutorial. If you skip this step, your students will start getting failures when they're almost done with the tutorial. Run the following command:
sudo mkdir --mode a+rwx /mnt/workshop
cd /mnt/workshop
You'll next follow the instructions for connecting to the EC2 instances using the IPython Notebook, but be sure to start the notebook server from the directory you just created (/mnt/workshop) and to be running as the ubuntu user when starting the notebook server.
You should now be able to access the IPython Notebook servers on the AWS instances. Confirm this by opening each via its URL. You should then load illumina_overview_tutorial_workshop_template.ipynb on each instance, as students will start by creating a copy of this notebook. It's a good idea to keep a copy of this notebook on your local desktop as well, as someone may accidentally delete or edit the copy that you load on to the instance.
One comment we received multiple years in a row at the Cesky Krumlov Workshop on Genomics is that students are confused by what the Notebook is. In the case of this particular workshop, students have typically been operating in a terminal for a week and the change for this tutorial leads to confusion. As a result, we developed a light tutorial focused on covering the basics of what the Notebook is and interaction. This notebook does assume that fortune and cowsay are available in the environment, which can be installed on the instances by issuing sudo apt-get install fortune cowsay.
You'll want to make the following information available to all students in the class. I often do this by publishing a google doc, and creating a shortened bit.ly link which I write on the board. For example, see http://bit.ly/acadia-rcn-qiime, which is a published Google Doc. This makes it easy to drop extra text in the document as necessary through-out the workshop.
Workshop schedule:
[Insert link to workshop schedule defined above]
Logging into QIIME's IPython notebook:
[Insert IPython Notebook URL(s) here]
Password:
qiime
We'll work through this tutorial:
http://qiime.org/tutorials/illumina_overview_tutorial.html
Getting help with QIIME scripts:
http://scripts.qiime.org
Other useful QIIME links:
http://www.qiime.org
http://help.qiime.org
http://qiime.org/tutorials/working_with_aws.html
Interested in using AWS for research or education? AWS grants are available:
http://aws.amazon.com/education/
As you prepare your slides for the workshop, ask the workshops hosts if there are funding sources that you should include in your acknowledgements, and when and how they would like you to acknowledge that source (e.g., in some cases, the workshop host may want to introduce you and acknowledge their funding source at the beginning of the workshop, or they may just want you to include their logo at the end of your presentation).
You should acknowledge the QIIME development group early in your presentation. It is important that you let the audience know that QIIME is a team effort with many developers.
This section contains steps that should be taken during the workshop.
Make the link to the Information for students documentation that you created easily accessible to the students throughout the workshop, for example by writing the URL on the board.
Each student should open the illumina_overview_tutorial_workshop_template link in the IPython notebook. They should immediately go to File > Save a Copy and save with a descriptive name (e.g., including their name in the notebook name). They should then execute all cells in the notebook to configure their environment. At this point, they're ready to begin working through the tutorial. You should have students refer to the Illumina Overview Tutorial to find the relevant commands. They should copy/paste commands from that notebook into their copy of the template notebook. While commands are running, you should be describing what they're doing.
If you run into serious issues where students can't execute certain steps of the tutorial, or you run out of time and want to show some of the results that the students haven't yet been able to run, have the students execute the command in the Precomputed Results section of the Illumina Overview Tutorial. This will generate links to the precomputed output of all of the commands run throughout the tutorial.
TODO: Develop standard feedback questionnaire to provide to students.
Students sometimes will need a completion certificate when they process their reimbursements. The text below has been adapted based on the information that students often need. You can adapt this text, or write your own to provide this documentation. You should send to students as a PDF on an as-needed basis.
[Letterhead from your primary affiliation]
[your name, title, and contact information]
Dear Participant,
Thank you for registering for the QIIME workshop in [location] on [date]
(cost of $[cost] USD). This document confirms that you paid in full, attended
the whole workshop, participated fully, and completed all the exercises
satisfactorily. We hope the skills you learned are useful for your ongoing
studies.
All the best,
[Image of your signature]
[your name] (on behalf of the organizers)
