SenateCrawler

##Class AvailableRecords

Package - edu.usc.cssl.nlputils.plugins.senatecrawler.process

###Method Summary #####static String[] getAllCongresses() Description
Returns an array of Strings containing all the congresses currently available in the senate website.

#####static String[] getSenators(String congressString) Parameters - congressString represents the congress that the user has currently selected through the GUI.
Returns - An array of strings containing the names of all the senators of the selected congress.

#####static String[] getAllSenators(String[] congresses) Parameters - An array of all the congress IDs.
Returns - An array of strings containing the names of all the senators.

Description
AvailableRecords class is used to populate the dropdown items in SenateCrawlerSettings user interface, so that the users can specify the search criteria for the crawler.

##Class SenateCrawler

Package - edu.usc.cssl.nlputils.plugins.senatecrawler.process

###Method Summary

#####void initialize(int maxDocs, String dateFrom, String dateTo, String outputDir) Parameters - maxDocs is the maximum records to be extracted per senator, dateFrom and dateTo are date strings specifying the date constraint and outputDir is the path of the output directory.
Description
Initializes the maximum number of records per senator, date constraints and output directory

#####void searchSenatorRecords(int congress,String senText) Parameters - An integer representing the congress number and a string specifying the senators to consider.
Description
Extracts records for the given congress and senator and calls writeToFile with the filename and contents.

#####void writeToFile(String fileName, String[] contents) Description
Writes the contents to a file with fileName <filename>

#####String[] extract(String extractLink, String lastName) Description
Extracts the text from the transcript corresponding to the senator with the specified lastname.

#####void writeReadMe(String location) Description
Writes a readme file to the location containing details about the entire process, parameters and output.

#####void run() Description
For implementing Runnable interface. Calls crawl()

#####void crawl() Description
If the user wants to collect All Senators/All Republicans/All Democrats/All Independents, this function calls getAll function with the required congress numbers and the senators to consider. If the user only wants the records of a particular senator, this function calls searchSenatorRecords directly.

#####void getAll(int congressNum, String senText) Description
Depending on the input, getAll searches the website for All Senators/All Republicans/All Democrats/All Independents for the specific congress <congressNum$gt;. getAll internally calls searchSenatorRecords.

#####void terminate() Description
Sets the boolean flag terminate to True, which will terminate the crawling in a graceful manner.

#####void appendLog(String message) Description
Appends the log to the console in the part org.eclipse.e4.ide.partstackdisplay of the main application. Internally calls the append function of the imported class edu.usc.cssl.nlputils.utilities.Log