Learn%2Bbasics%2Babout%2Bnotebooks%2Band%2BSpark%2BLocal.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Learn the basics about notebooks and Apache Spark\n",
    "\n",
    "This notebook introduces you to the basics of analytics notebooks and explains what Apache Spark is and how to use Spark in notebooks. The notebook shows you how to load data into the notebook, parse and explore the data, run queries on the data to extract information, plot your analysis results, and save your result in Object Storage.\n",
    "\n",
    "This notebooks runs on Python 2 with Spark 2.0.2\n",
    "\n",
    "## Table of contents\n",
    "- [What is Apache Spark](#apache_spark)\n",
    "- [Get data](#data_set)\n",
    "- [Load data](#load_data)\n",
    "- [Access data](#access_data)\n",
    "- [Add and parse header](#add_header)\n",
    "- [Explore data](#explore_data)\n",
    "- [Use Spark SQL](#use_spark_sql)\n",
    "- [Save results in Object Storage](#save)\n",
    "- [Summary](#summary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id=\"apache_spark\"></a>\n",
    "## What is Apache Spark\n",
    "\n",
    "[Spark](http://spark.apache.org/) is a fast open-source engine for large-scale data processing. It is built for speed and ease of use. Through the advanced DAG execution engine that supports cyclic data flow and in-memory computing, programs can run up to 100 times faster than Hadoop MapReduce in memory, or 10 times faster on disc.\n",
    "\n",
    "Spark consists of multiple components:\n",
    "\n",
    "* Spark Core is the underlying computation engine with the fundamental programming abstraction called resilient distributed datasets (RDDs)\n",
    "* Spark SQL provides a new data abstraction called DataFrames for structured data processing with SQL and domain-specific language\n",
    "* MLlib is a scalable machine learning framework for delivering fast distributed algorithms for mining big data\n",
    "* Streaming leverages Spark's fast scheduling capability to perform real-time analysis on streams of new data\n",
    "* GraphX is the graph processing framework for the analysis of graph structured data\n",
    "\n",
    "In this sample, you will focus on Spark Core and Spark SQL by using the Python API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id=\"data_set\"></a>\n",
    "## Get data\n",
    "\n",
    "In the sample, you will use Apache Spark to analyze weather data collected from weather stations in 2015. This data is provided by the NOAA National Climatic Data Center (NCDC).\n",
    "\n",
    "To get at the raw data from the NOAA National Climatic Data Center (NCDC), run the following code cell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2017-11-01 22:01:01--  ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/by_year/2015.csv.gz\n",
      "           => ‘2015.csv.gz’\n",
      "Resolving ftp.ncdc.noaa.gov (ftp.ncdc.noaa.gov)... 205.167.25.101, 2610:20:8040:2::101\n",
      "Connecting to ftp.ncdc.noaa.gov (ftp.ncdc.noaa.gov)|205.167.25.101|:21... connected.\n",
      "Logging in as anonymous ... Logged in!\n",
      "==> SYST ... done.    ==> PWD ... done.\n",
      "==> TYPE I ... done.  ==> CWD (1) /pub/data/ghcn/daily/by_year ... done.\n",
      "==> SIZE 2015.csv.gz ... 196294128\n",
      "==> PASV ... done.    ==> RETR 2015.csv.gz ... done.\n",
      "Length: 196294128 (187M) (unauthoritative)\n",
      "\n",
      "2015.csv.gz         100%[=====================>] 187.20M  1.65MB/s   in 1m 55s \n",
      "\n",
      "2017-11-01 22:03:00 (1.62 MB/s) - ‘2015.csv.gz’ saved [196294128]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget 'ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/by_year/2015.csv.gz'\n",
    "!gunzip -c '2015.csv.gz' > '../datasets/2015.csv'\n",
    "!rm '2015.csv.gz'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "\n",
    "The data in the base file has the following format:\n",
    "\n",
    "<table border=\"1\" style=\"width:90%\">\n",
    "  <tr>\n",
    "    <th>STATION</th><th>DATE</th><th>METRIC</th><th>VALUE</th><th>C5</th><th>C6</th><th>C7</th><th>C8</th>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>US1FLSL0019</td><td>20150101</td><td>PRCP</td><td>173</td><td></td><td></td><td>N</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>US1TXTV0133</td><td>20150101</td><td>PRCP</td><td>119</td><td></td><td></td><td>N</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>USC00178998</td><td>20150101</td><td>TMAX</td><td>-33</td><td></td><td></td><td>7</td><td>100</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td>\n",
    "  </tr>\n",
    "</table>\n",
    "<p>\n",
    "\n",
    "Each row contains a weather station identifier, a date, a metric which is collected (like precipitation, daily maximum and minimum temperatures, temperature at the time of observation, snowfall, snow depth, and so on) and some additional values.\n",
    "\n",
    "**Note**: The header is not included in the CSV file and is documented in **ghcn-daily-by_year-format.rtf**. It was added to the table above for sake of vividness."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Load data \n",
    "To load the data into a notebook\n",
    "\n",
    "1. Click the **Find Data** icon on the top right of the notebook screen. \n",
    "1. Click **Insert to Code** for the `2015.csv` file.\n",
    "1. You can skip this step for this notebook. The generated code to insert the data is included below.\n",
    "\n",
    "The data file is stored in the filesystem.\n",
    "\n",
    "**Note**: Because the CSV file is relatively large, it might take a few minutes to load the data file. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id=\"access_data\"></a>\n",
    "## Access data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "To add the code to access the data file which is stored in the filesystem, run the next code cell.\n",
    "\n",
    "This function inserts the setup code for the preconfigured [`SparkContext`](https://spark.apache.org/docs/1.6.0/api/python/pyspark.html#pyspark.SparkContext). Then the data is loaded into a `Spark RDD` which is created by using the `SparkContext`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<div class=\"alert alert-block alert-info\">Note: The `Create DataFrame` will automatically add a cell at the top to import library and set up project context. Please run that cell first before loading data in the following cell.</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-----------+--------+----+---+----+----+---+----+\n",
      "|        _c0|     _c1| _c2|_c3| _c4| _c5|_c6| _c7|\n",
      "+-----------+--------+----+---+----+----+---+----+\n",
      "|US1MTMH0019|20150101|PRCP|  0|null|null|  N|null|\n",
      "|US1MTMH0019|20150101|SNOW|  0|null|null|  N|null|\n",
      "|US1MTMH0019|20150101|SNWD|127|null|null|  N|null|\n",
      "|US1MTMH0019|20150101|WESD|137|null|null|  N|null|\n",
      "|ASN00015643|20150101|TMAX|373|null|null|  a|null|\n",
      "|ASN00015643|20150101|TMIN|222|null|null|  a|null|\n",
      "|ASN00015643|20150101|PRCP|  0|null|null|  a|null|\n",
      "|ASN00085296|20150101|TMAX|254|null|null|  a|null|\n",
      "|ASN00085296|20150101|TMIN|110|null|null|  a|null|\n",
      "|ASN00085296|20150101|PRCP|  0|null|null|  a|null|\n",
      "|ASN00085280|20150101|TMAX|262|null|null|  a|null|\n",
      "|ASN00085280|20150101|TMIN| 88|null|null|  a|null|\n",
      "|ASN00085280|20150101|PRCP|  0|null|null|  a|null|\n",
      "|ASN00040209|20150101|PRCP|  0|null|null|  a|null|\n",
      "|US1MNCV0008|20150101|PRCP|  0|null|null|  N|null|\n",
      "|US1MNCV0008|20150101|SNOW|  0|null|null|  N|null|\n",
      "|US1MNCV0008|20150101|WESF|  0|null|null|  N|null|\n",
      "|US1MISW0005|20150101|PRCP|  0|   T|null|  N|null|\n",
      "|US1KSSG0097|20150101|PRCP|  0|null|null|  N|null|\n",
      "|US1KSSG0097|20150101|SNWD| 13|null|null|  N|null|\n",
      "+-----------+--------+----+---+----+----+---+----+\n",
      "only showing top 20 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from pyspark.sql import SQLContext\n",
    "# Add asset from file system\n",
    "df_data_1 = SQLContext(sc).read.csv('../datasets/2015.csv', header='false')\n",
    "df_data_1.show(20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<div class=\"alert alert-block alert-info\">Note: The code below creates an RDD from the dataframe loaded above. Please update the dataframe variable name of \"df_1\" to make it consist with the variable name in the cell above.</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Row(_c0=u'US1MTMH0019', _c1=u'20150101', _c2=u'PRCP', _c3=u'0', _c4=None, _c5=None, _c6=u'N', _c7=None),\n",
       " Row(_c0=u'US1MTMH0019', _c1=u'20150101', _c2=u'SNOW', _c3=u'0', _c4=None, _c5=None, _c6=u'N', _c7=None),\n",
       " Row(_c0=u'US1MTMH0019', _c1=u'20150101', _c2=u'SNWD', _c3=u'127', _c4=None, _c5=None, _c6=u'N', _c7=None),\n",
       " Row(_c0=u'US1MTMH0019', _c1=u'20150101', _c2=u'WESD', _c3=u'137', _c4=None, _c5=None, _c6=u'N', _c7=None),\n",
       " Row(_c0=u'ASN00015643', _c1=u'20150101', _c2=u'TMAX', _c3=u'373', _c4=None, _c5=None, _c6=u'a', _c7=None)]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weather = df_data_1.rdd\n",
    "weather.take(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now you can access the data by using the preconfigured `SparkContext` function in your notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The RDD you created is a collection of strings corresponding to the individual lines in the raw data file. It is also important to remember that the RDD is defined but not instantiated. By applying an action like `count` to the RDD, you effectively instantiate the RDD."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total records in the 2015.csv dataset: 34355109\n"
     ]
    }
   ],
   "source": [
    "print \"Total records in the 2015.csv dataset:\", weather.count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Apply another action to the same RDD that reads the first row of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The first row in the 2015.csv dataset: Row(_c0=u'US1MTMH0019', _c1=u'20150101', _c2=u'PRCP', _c3=u'0', _c4=None, _c5=None, _c6=u'N', _c7=None)\n"
     ]
    }
   ],
   "source": [
    "print \"The first row in the 2015.csv dataset:\", weather.first()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id=\"add_header\"></a>\n",
    "## Add and parse header"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Since the header is not included in the CSV file, it has to be added programmatically, as well be parsed into columns. You can do this by mapping each line in the RDD to a function that splits the line by commas.\n",
    "\n",
    "The lambda notation in Python is used to create anonymous functions, in other words, functions which are not bound to a name. This concept is used in the following code cell to pass a function as a parameter to the `map` function. The anonymous function receives the header as string RDD and splits it at comma boundaries. As a result, the new `weatherParse` RDD is a list of lists. Each list in `weatherParse` corresponds to a line in `weather`, and the strings in each list are the individual elements of the row."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['STATION', 'DATE', 'METRIC', 'VALUE', 'C5', 'C6', 'C7', 'C8'],\n",
       " Row(_c0=u'US1MTMH0019', _c1=u'20150101', _c2=u'PRCP', _c3=u'0', _c4=None, _c5=None, _c6=u'N', _c7=None)]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "header = sc.parallelize(['STATION,DATE,METRIC,VALUE,C5,C6,C7,C8']).map(lambda line : line.split(','))\n",
    "union = header.union(weather)\n",
    "weatherParse = union\n",
    "weatherParse.take(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Run the next cell for a quick look at the first list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['STATION', 'DATE', 'METRIC', 'VALUE', 'C5', 'C6', 'C7', 'C8']"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weatherParse.first()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now take a look at the individual elements of this first list where the first entry starts at offset zero."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'STATION'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weatherParse.first()[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "You can also pull elements by index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'METRIC'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weatherParse.first()[2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id=\"explore_data\"></a>\n",
    "## Explore data\n",
    "To better consume the precipitation data, it has to be converted or mapped from one raw form into another format.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Calculate the average precipitation by weather station\n",
    "To calculate the average precipitation that was recorded at each weather station in the data set, reduce the data set by selecting only those rows with precipitation data values, in other words, those rows where the METRIC column equals `PRCP`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "weatherPrecp = weatherParse.filter(lambda x: x[2] == \"PRCP\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The `weatherPrecp` RDD that you created contains a list of pairs (v1, v2), where v1 is a weather station identifier and v2 is one precipitation data point (one day) for that station. Table 1. depicts this structure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "#### Table 1.\n",
    "\n",
    "<table border=\"1\" style=\"width:80%\">\n",
    "  <tr>\n",
    "    <th>Key</th><th>Value</th>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>Station 1</td><td>Value 1</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>Station 2</td><td>Value 2</td>\n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>Station 1</td><td>Value 3</td>\n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>Station 2</td><td>Value 4</td>\n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>Station 3</td><td>Value 5</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>...</td><td>...</td>\n",
    "  </tr>\n",
    "</table>\n",
    "<p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Next, transform (map) this data set into a new one where each row (data pair) is augmented with the value `1`. Table 2. shows this new structure."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "#### Table 2.\n",
    "\n",
    "<table border=\"1\" style=\"width:80%\">\n",
    "  <tr>\n",
    "    <th>Key</th><th>Value</th>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>Station 1</td><td>(Value 1,1)</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>Station 2</td><td>(Value 2,1)</td>\n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>Station 1</td><td>(Value 3,1)</td>\n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>Station 2</td><td>(Value 4,1)</td>\n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>Station 3</td><td>(Value 5,1)</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>...</td><td>...</td>\n",
    "  </tr>\n",
    "</table>\n",
    "<p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The reason for this mapping is to reduce the table into the form represented by Table 3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "#### Table 3.\n",
    "\n",
    "<table border=\"1\" style=\"width:80%\">\n",
    "  <tr>\n",
    "    <th>Key</th><th>Value</th>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>Station 1</td><td>(Value 1 + Value 3,2)</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>Station 2</td><td>(Value 2 + Value 4,2)</td>\n",
    "  </tr>\n",
    "    <tr>\n",
    "    <td>Station 3</td><td>(Value 5,1)</td>\n",
    "  </tr>\n",
    "  <tr>\n",
    "    <td>...</td><td>...</td>\n",
    "  </tr>\n",
    "</table>\n",
    "<p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Using this table, you can compute the average precipitation for each station by dividing the summation of the values by the corresponding count.\n",
    "\n",
    "Run the next code cell to create `weatherPrecpCountByKey`. The outcome is the equivalent to what you see in Table 2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# x[0] is the station\n",
    "# x[3] is the precipitation value\n",
    "weatherPrecpCountByKey = weatherPrecp.map(lambda x : (x[0], (int(x[3]), 1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The same Python lambda function notation is used to pass a function into the mapping function which transforms `weatherPrecp` into the new RDD. \n",
    "\n",
    "To confirm that the mapping produced the expected results, run the next cell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(u'US1MTMH0019', (0, 1))"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weatherPrecpCountByKey.first()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Calculate the total precipitation by weather station\n",
    "\n",
    "To calculate the total precipitation by weather station, sum (reduce) the precipitation amounts and total readings for every station. Use the `reduceByKey` function for this purpose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "weatherPrecpAddByKey = weatherPrecpCountByKey.reduceByKey(lambda v1,v2 : (v1[0]+v2[0], v1[1]+v2[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Using the `first` function, you can inspect the precipitation values and read the totals for the first station ID. Note that this operation might take some time to complete as the whole chain of RDDs that you created are reinstantiated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(u'USC00477015', (10129, 309))"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weatherPrecpAddByKey.first()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Compute the average values per station\n",
    "\n",
    "Now that you have transformed the data into the format you need it in, you can finally compute the average precipitation values per weather station. You create the `weatherAverages` RDD by mapping the `weatherPrecpAddByKey` RDD through a function that divides the precipitation total by the total number of readings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "weatherAverages = weatherPrecpAddByKey.map(lambda k: (k[0], k[1][0] / float(k[1][1] ) ) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(u'USC00477015', 32.77993527508091)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weatherAverages.first()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now print the first ten stations and their average precipitation values. The station ID is the sort order in the `top` function because it appears first in the tuple (station ID, average precipitation) in the RDD."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Station ZI000067983 had average precipitations of 43.362637\n",
      "Station ZI000067975 had average precipitations of 42.397260\n",
      "Station ZI000067775 had average precipitations of 98.833333\n",
      "Station ZA000067743 had average precipitations of 170.142857\n",
      "Station WZ004455110 had average precipitations of 57.678899\n",
      "Station WQW00041606 had average precipitations of 29.680628\n",
      "Station WFM00091754 had average precipitations of 70.784768\n",
      "Station WF000917530 had average precipitations of 87.988669\n",
      "Station WA012084750 had average precipitations of 81.787879\n",
      "Station WA010517310 had average precipitations of 14.275862\n"
     ]
    }
   ],
   "source": [
    "for pair in weatherAverages.top(10):\n",
    "    print \"Station %s had average precipitations of %f\" % (pair[0],pair[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "If you want to output the ten weather stations with the highest average precipitation, you need to reverse the order of the tuple to (average precipitation, station ID). You can do this with a `map` function that switches the pair order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Station US1TXLR0005 had average precipitations of 1996.000000\n",
      "Station US1TXHND002 had average precipitations of 1842.000000\n",
      "Station US1TXWM0106 had average precipitations of 1768.000000\n",
      "Station US1TXCMR017 had average precipitations of 1765.000000\n",
      "Station US1FLBW0047 had average precipitations of 1740.000000\n",
      "Station US1TXBND040 had average precipitations of 1715.000000\n",
      "Station US1SDMH0043 had average precipitations of 1702.000000\n",
      "Station US1INJN0014 had average precipitations of 1402.000000\n",
      "Station US1IAPK0065 had average precipitations of 1394.000000\n",
      "Station IDM00096237 had average precipitations of 1161.000000\n"
     ]
    }
   ],
   "source": [
    "precTop10=[]\n",
    "stationsTop10=[]\n",
    "for pair in weatherAverages.map(lambda (x,y) : (y,x)).top(10):\n",
    "    precTop10.append(pair[0])\n",
    "    stationsTop10.append(pair[1])\n",
    "    print \"Station %s had average precipitations of %f\" % (pair[1],pair[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Plot your results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAFZCAYAAABg/aPVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe8HFX9//HXmwTpASKRlkAAKYIU5QrYEASlqIBYAAVC\nUfQHCqhfFZSv6FdRLKBiQUGRpjQRBQWlSJEmXEIIhBqaSWih9yDw+f1xzpLJZu/e3XvvzNwk7+fj\nsY+7OzM7nzN3y2fnnDPnKCIwMzPrxkJ1F8DMzOY9Th5mZtY1Jw8zM+uak4eZmXXNycPMzLrm5GFm\nZl1z8pgPSPqkpAvrLkcrkp6VtHqb9fdJ2noQ+z9R0ncG+vymfV0m6VN9rFslH8uIDvYzXlJIGjkU\n5bJqSHq3pDuGetsu4k+RtMVQ7rNMTh5tSPqcpF5JsySd2GL9VpJul/S8pEslrTqAGHtJurKL7ef6\nYoqI30fE+7uNXYWIWDIi7oHBf9F3+78aShHxn3wsr9QRH9onNxu8iPhXRKw9kG27/RHU6rMQEetF\nxGUdF7hmTh7tPQB8BziheYWk5YA/Af8LjAZ6gTMqLZ1ZDYbzGdVwLtt8JyJ86+dGSiAnNi3bD7i6\n8HgJ4AVgnT72sRdwD/AMcC/wSeBNwIvAK8CzwJN52w8ANwJPA9OAbxb28x8g8vbPAm/P+76ysM07\ngOuBp/LfdxTWXQZ8G7gql+VCYLm8blHgVOAx4Mn83OVbHMvewHmFx3cBZxUeTwM2yvcDeGP+f/0X\neCmX+7y8/j7gf4DJubxnAIu2iNnX/+pE4BfA3/Lx/BtYo/C8dYCLgMeBO4CPt3md2/1vxudjGZkf\nrwZckbe7OJfh1KZtJ+TX61Hg64U4CwGHAHfn//WZwOh2rwFwRD72F/Px/7yPYzgLeCj/L68A1svL\nN83LRxS2/TAwuYMyNY5n33w8V7SLlde9HjiP9B6+nvQZunIQr8v3gOvy/v7SQdk2A67O/8ObgC0K\n+xsN/I704/AJ4M95+RbA9MJ29wGHArfm7X5Hfm8WtwVOAV4lff6fBb7Sz2vR7rOwdb6/CPCTXMYH\n8v1FirGBLwGPAA8Ce1f+vVh1wHnxRuvk8VPg2KZlNwMfafH8JfKbfu38eMXCG2mv4oeq8OZYn/SB\n3gB4GNgpr2t8WEYWtn9tH/mD8QSwBzAS2C0/fn1efxnpC2ItYLH8+Mi87jOkD/ziwAhgY2BUi+NZ\nPX8oFwJWAu4vfJBWz/EWyo8DeGO+fyLwnaZ93Uf6Ulgpl/024LN9vA6t/lcnkr7sNsnH+3vg9ML/\nfRop2Y0E3kL6Il+3j/23+9/M8X8HrgF+BLwOeFd+fZuTx/F5PxsCs4A35fUHAdcCY0lfEr8GTuvv\nNcjl+VQ/79V9gKWY/eUzqbDubuB9hcdnAYd0UKbG8Zyc/6eLdRDr9HxbHFg3vw5XDuJ1mQG8OT/3\n7Bb/69fKBqyc3xPbk96j78uPx+Tn/I30I2VZYGHgPYXPXXPyuAUYR3pvXkV+//ax7dZdvBYn0vqz\n0Ege/5dfjzcAY0iJ8NuF2C/nbRbOx/k8sGyl34tVBptXb7ROHr8lf7EUll0F7NXi+UuQvmw/0vjg\nFdbtRdMXYovn/wT4cb7f+LD0lTz2AK5rev41jXLlD+JhhXX7A3/P9/fJb9INOvifTAPeCuwKHEdK\nAOuQvhDOLWzXSfLYvfD4B8Cv+og51/8q7/M3hcfbA7fn+7sA/2ra/tfA4X3sv93/5rX/O7BK/vAu\nXtj2VOb+QhtbWH8dsGu+fxuwVWHdiqRfoiPbvQZ0kDyatl8ml2Ppwvv4hHx/KeA5YNUOytQ4ntU7\niUVKev8l/1gqxG68RwfyuhxZeLwu6Vf7iFZlA74KnNK0j3+QzgRXJJ0lzPVFS+uE8NnC4+2Bu9ts\nu3Wr8vfxWpxI++RxN7B9Yd02wH2F2C8w53fAI8Bmnb43huLmNo+BexYY1bRsaVI1xhwi4jnSB+az\nwIOS/iZpnb52LGnT3AA/U9JT+XnLdViuxplA0f2kX2MNDxXuPw8sme+fQvqQnS7pAUk/kLRwH3Eu\nJ72JN8/3LwPek2+Xd1jW/soz2OevCmwq6cnGjVRduMIgy7IS8HhEPF9YNq3Lcp1TKNNtpCqp5enu\nNZiDpBGSjpR0t6SnSV9GMPu98wdgZ0mLADsDEyOi8V5pV6a5jrGfWGNISWdaq+cysNel+Pz7Sb+4\nl+tj/arAx5r2/y5S4hhHeu2eaBOrXdyVOnlSB69Ff5o/x82xH4uIlwuPB/K5GRQnj4GbQqqOAEDS\nEsAaeflcIuIfEfE+0hv4dlKVBqRfI83+AJwLjIuIpYFfAWqzfdEDpA9P0Sqk0/62IuK/EfGtiFiX\n1G7yQWDPPjZvJI935/uX03/y6K/s/Raxy+2nAZdHxDKF25IR8f8GWY4HgdGSFi8sG9dlubZrKtei\nETGjn9egv+P/BLAjsDXph8z4vFwAEXEr6Utou7ztHzopU2GbYvx2sWaSzszGFrYv/n8G8roUn78K\n6czm0T7KNo105lHc/xIRcWReN1rSMm1itYv7QB/bNb82bV+LFts3a/4ct4tdCyePNiSNlLQo6fR4\nhKRFC705zgHeLOkjeZvDgZsi4vYW+1le0o45wcwinbW8mlc/DIyV9LrCU5Yi/Tp6UdImpDdiw8z8\n3L6unTgfWEvSJ3L5dyGd5v+1g+PdUtL6+VqGp0kf0Ff72PxyYEtSNdx04F/AtqSG0hv7eM7Dbcrd\niVb/q3b+Svpf7CFp4Xx7m6Q3DaIM5F/rvcA3Jb1O0tuBD3Wxi18BRzS6dksaI2nHfL/da9Df/28p\n0vvrMVJbw3dbbPMHUvvG5qQ2j37L1G2sSN2Z/0T6/yyez7KLP0IG8rrsLmndnLD/D/hj9N1t+lTg\nQ5K2yWcAi0raQtLYiHgQuAD4paRlc+zN28Q9QNJYSaOBr9N3j8rm16a/16K/1/I04LD8OiwHfCMf\n17Dh5NHeYaS6xUOA3fP9wwAiYiapDeMIUgPxJqT6/1YWAr5I+uXwOOnXeeNX1j9JZysPSWr8ktof\n+D9Jz5DeNGc2dpSrSo4Arsqn5JsVA0XEY6Rfq18ivXG/AnwwIoq/0vqyAvBH0pfWbaQEcUqrDSPi\nTlIS/Fd+/DSpN9lVbT7UvwXWzeX+cwfladbqf9WniHgGeD/pdXmAVI30fVID5mB9ktTT7TFSff4Z\npC+LTvyUdGZ5YX6NryX1hoL2r8FPgY9KekLSMS32ezLpzGIGqYfQtS22OY30/vtn03uiXZla6S/W\n50i/uB/K5T+N/P8Z4OtyCqmd4CFSj7QD+9owIqaRfvV/jfRjaxrwZWZ/3+1BSsq3k9oKDm4T9w+k\nXnf3kNoh+rpO6XukL/snJf0P/f9/+vssfIf0A2UyqSPOxDaxa6Hc2GJmgyDpDFJD/eF1l2U4kvR9\nYIWImDCA515G6ozwmyEvWPu495E6KFxcZdx5hc88zAYgV7OsIWkhSduSfukO5GxqviRpHUkbKNmE\ndB3GOXWXy4aOr8Y0G5gVSPX6ryddsPX/IqKvtp4F0VKkqqqVSPX7R5Eu7rP5hKutzMysa662MjOz\nrs231VbLLbdcjB8/vu5imJnNU2644YZHI2JMf9vNt8lj/Pjx9Pb21l0MM7N5iqTmESpacrWVmZl1\nzcnDzMy65uRhZmZdc/IwM7OuOXmYmVnXSkseksblOSlulTRF0kF5+WhJF0m6K/9dtvCcQyVNlXSH\npG0KyzeWdHNed4wktYppZmbVKPPM42XgS3legs1IQxuvSxqh9pKIWBO4JD8mr9sVWI80tPcv87DU\nAMcCnwbWzLdtSyy3mZn1o7TkEREPRsTEfP8Z0vDSK5MGkDspb3YSsFO+vyNp7ulZEXEvMBXYRNKK\npDmcr400lsrJheeYmVkNKmnzkDSeNMn9v4Hl84QskMbmb0xzuTJzTvk4PS9bOd9vXt4qzn6SeiX1\nzpw5c8jKb2Zmcyo9eUhaEjgbODhPGPSafCYxZCMzRsRxEdETET1jxvR7dX2fpMHdzMzmd6UmD0kL\nkxLH7yPiT3nxw7kqivz3kbx8BnPOFzw2L5vBnHMhN5abmVlNyuxtJdJUi7dFxNGFVecCjdnEJjB7\njP9zgV0lLSJpNVLD+HW5iutpSZvlfe6J5wUwM6tVmQMjvpM0V/DNkiblZV8DjgTOlLQvaY7fjwNE\nxBRJZ5Lm+30ZOKAwF/b+pPmLFyNNXn9BieU2M7N+zLeTQfX09MRAR9UdbLvFfPovNbMFgKQbIqKn\nv+18hbmZmXXNycPMzLrm5GFmZl1z8jAzs645eZiZWdecPMzMrGtOHmZm1jUnDzMz65qTh5mZdc3J\nw8zMuubkYWZmXStzYEQbII+tZWbDnc88zMysa04eZmbWNScPMzPrmpOHmZl1zcnDzMy6VuYc5idI\nekTSLYVlZ0ialG/3NaanlTRe0guFdb8qPGdjSTdLmirpmDyPuZmZ1ajMrronAj8HTm4siIhdGvcl\nHQU8Vdj+7ojYqMV+jgU+DfwbOB/YFs9hXip3FTaz/pR25hERVwCPt1qXzx4+DpzWbh+SVgRGRcS1\nkSZbPxnYaajLasOLNLibmZWvrjaPdwMPR8RdhWWr5SqryyW9Oy9bGZhe2GZ6XtaSpP0k9UrqnTlz\n5tCX2szMgPquMN+NOc86HgRWiYjHJG0M/FnSet3uNCKOA44D6OnpceWJDYir7cz6V3nykDQS2BnY\nuLEsImYBs/L9GyTdDawFzADGFp4+Ni8zM7Ma1VFttTVwe0S8Vh0laYykEfn+6sCawD0R8SDwtKTN\ncjvJnsBfaiizmZkVlNlV9zTgGmBtSdMl7ZtX7crcDeWbA5Nz190/Ap+NiEZj+/7Ab4CpwN24p5XN\n59xhwOYFivm0granpyd6e3sH9Ny667wd3/EX5PhWL0k3RERPf9v5CnMzM+uak4eZmXXNycPMzLrm\n5GFmZl1z8jAzs645eZiZWdecPMzMrGtOHmZm1jUnDzMz65qTh5mZdc3Jw8zMuubkYWZmXXPyMDOz\nrjl5mJlZ15w8zMysa04eZmbWNScPMzPrWpnT0J4g6RFJtxSWfVPSDEmT8m37wrpDJU2VdIekbQrL\nN5Z0c153TJ7L3MzMalTmmceJwLYtlv84IjbKt/MBJK1Lmtt8vfycX0oakbc/Fvg0sGa+tdqnmZlV\nqLTkERFXAI93uPmOwOkRMSsi7gWmAptIWhEYFRHXRpps/WRgp3JKbGZmnaqjzePzkibnaq1l87KV\ngWmFbabnZSvn+83LW5K0n6ReSb0zZ84c6nKbmVlWdfI4Flgd2Ah4EDhqKHceEcdFRE9E9IwZM2Yo\nd21mZgWVJo+IeDgiXomIV4HjgU3yqhnAuMKmY/OyGfl+83IzM6tRpckjt2E0fBho9MQ6F9hV0iKS\nViM1jF8XEQ8CT0vaLPey2hP4S5VlNjOzuY0sa8eSTgO2AJaTNB04HNhC0kZAAPcBnwGIiCmSzgRu\nBV4GDoiIV/Ku9if13FoMuCDfzMysRkqdmOY/PT090dvbO6DnDvZKksH+Sx3f8Rfk+FYvSTdERE9/\n2/kKczMz65qTh5mZdc3Jw8zMuubkYWZmXSutt5WZ2UC4wX7e4ORhZlbg5NUZV1uZmVnXnDzMzKxr\nTh5mZtY1Jw8zM+taR8lD0jslLZHv7y7paEmrlls0MzMbrjo98zgWeF7ShsCXgLtJs/qZmdkCqNPk\n8XKeBnZH4OcR8QtgqfKKZWZmw1mn13k8I+lQYHdgc0kLAQuXVywzMxvOOj3z2AWYBewbEQ+RZvT7\nYWmlMjOzYa2jM4+cMI4uPP4PbvMwM1tgddrbamdJd0l6StLTkp6R9HTZhTMzs+Gp02qrHwA7RMTS\nETEqIpaKiFHtniDpBEmPSLqlsOyHkm6XNFnSOZKWycvHS3pB0qR8+1XhORtLulnSVEnH5LnMzcys\nRp0mj4cj4rYu930isG3TsouAN0fEBsCdwKGFdXdHxEb59tnC8mOBTwNr5lvzPs3MrGKd9rbqlXQG\n8GdSwzkAEfGnvp4QEVdIGt+07MLCw2uBj7YLKmlFYFREXJsfnwzsBFzQYbnNzKwEnSaPUcDzwPsL\nywLoM3l0YB/gjMLj1SRNAp4CDouIfwErA9ML20zPy1qStB+wH8Aqq6wyiKKZmVk7nfa22nsog0r6\nOvAy8Pu86EFglYh4TNLGwJ8lrdftfiPiOOA4gJ6engVkVH0zs+p12ttqbG7gfiTfzpY0diABJe0F\nfBD4ZL5qnYiYFRGP5fs3kIY/WQuYQbqmpGFsXmZmZjXqtMH8d8C5wEr5dl5e1hVJ2wJfIfXcer6w\nfIykEfn+6qSG8Xsi4kHgaUmb5V5WewJ/6TaumZkNrU6Tx5iI+F1EvJxvJwJj2j1B0mnANcDakqZL\n2hf4OWlMrIuauuRuDkzObR5/BD4bEY/ndfsDvwGmks5I3FhuZlazThvMH5O0O3Bafrwb8Fi7J0TE\nbi0W/7aPbc8Gzu5jXS/w5g7LaWZmFej0zGMf4OPAQ6TG7Y8CQ9qIbmZm845Oe1vdD+xQclnMzGwe\n0TZ5SPpKRPxA0s9I13XMISIOLK1kZmY2bPV35tEYkqS37IKYmRkMdvS+qOgKt7bJIyLOy3efj4iz\niuskfay0UpmZ2bDWaYP5oR0uMzOzBUB/bR7bAdsDK0s6prBqFGl4ETMzWwD11+bxAKm9YwfghsLy\nZ4AvlFUoMzMb3vpr87gJuEnSHyLivxWVyczMhrlOrzAfL+l7wLrAoo2FEbF6KaUyM7NhrZuBEY8l\ntXNsCZwMnFpWoczMbHjrNHksFhGXAIqI+yPim8AHyiuWmZkNZ51WW82StBBwl6TPkebUWLK8YpmZ\n2XDW6ZnHQcDiwIHAxsDupLk1zMxsAdRp8hgfEc9GxPSI2DsiPgJ4knAzswWUrzA3M7Ou+QpzMzPr\nWn9nHo0rzF8kXWHeuJ0LbNPuiZJOkPSIpFsKy0ZLukjSXfnvsoV1h0qaKukOSdsUlm8s6ea87pg8\nl7mZmdWobfKIiJsi4iRgjYg4qXD7U0Q80c++TwS2bVp2CHBJRKwJXJIfI2ldYFdgvfycX0oakZ9z\nLPBpYM18a96nmZlVrG3ykHRmvnujpMnNt3bPjYgrgMebFu8InJTvnwTsVFh+ekTMioh7ganAJpJW\nBEZFxLUREaSLE3fCzMxq1d91Hgflvx8conjLR8SD+f5DwPL5/srAtYXtpudl/833m5e3JGk/YD+A\nVVZxZzAzs7L0V231YP57PzAL2BDYAJiVlw1YPpMY0jmvIuK4iOiJiJ4xY8YM5a7NzKygo666kj4F\nXAfsDHwUuFbSPgOI93CuiiL/fSQvnwGMK2w3Ni+bke83Lzczsxp1ep3Hl4G3RMReETGBdJX5VwcQ\n71xgQr4/AfhLYfmukhaRtBqpYfy6fObztKTNci+rPQvPMTOzmnQ6ttVjpAmgGp7Jy/ok6TRgC2A5\nSdOBw4EjgTMl7QvcD3wcICKm5Mb5W0nXjxwQEa/kXe1P6rm1GHBBvpmZWY2Umh762Ug6GVif9Ks/\nSL2jJucbEXF0iWUckJ6enujt7R3Qcwd7JUkH/1LHd3zHd/xhGl83RERPf9t1euZxd741NKqOluq2\nYGZmNu/rKHlExLfKLoiZmc07+hvb6icRcbCk82jRrTYidiitZGZmNmz1d+ZxSv77o7ILYmZm8462\nySMibsh3e4EXIuJVgDzu1CIll83MzIapTq/zuIQ0k2DDYsDFQ18cMzObF3SaPBaNiGcbD/L9xdts\nb2Zm87FOk8dzkt7aeCBpY+CFcopkZmbDXafXeRwMnCXpAUDACsAupZXKzMyGtU6v87he0jrA2nnR\nHRHx3/KKZWZmw1mno+ouThoI8aCIuAUYL2mo5vgwM7N5TKdtHr8DXgLenh/PAL5TSonMzGzY6zR5\nrBERPyDN7EdEPE9q+zAzswVQp8njJUmLkYcokbQGaWZBMzNbAHXa2+pw4O/AOEm/B94J7FVWoczM\nbHjrN3nkGfxuJ01BuxmpuuqgiHi05LKZmdkw1W/yiIiQdH5ErA/8rYIymZnZMNdpm8dESW8bioCS\n1pY0qXB7WtLBkr4paUZh+faF5xwqaaqkOyRtMxTlMDOzgeu0zWNTYHdJ9wHPkaquIiI26DZgRNwB\nbASvjc47AzgH2Bv4cUTMMfy7pHWBXYH1gJWAiyWtVZjj3MzMKtZp8ijr1/5WwN0Rcb/6nrh3R+D0\niJgF3CtpKrAJcE1JZTIzs360rbaStKikg4EvA9sCMyLi/sZtCOLvCpxWePx5SZMlnSBp2bxsZWBa\nYZvpeVmr8u4nqVdS78yZM4egeGZm1kp/bR4nAT3AzcB2wFFDFVjS64AdgLPyomOB1UlVWg8OJFZE\nHBcRPRHRM2bMmKEqqpmZNemv2mrd3MsKSb8FrhvC2NsBEyPiYYDG3xzreOCv+eEMYFzheWPzMjMz\nq0l/Zx6vjZwbES8PcezdKFRZSVqxsO7DwC35/rnArpIWkbQasCZDm8TMzKxL/Z15bCjp6XxfwGL5\ncaO31aiBBJW0BPA+4DOFxT+QtBFpCJT7GusiYoqkM4FbgZeBA9zTysysXm2TR0SMKCNoRDwHvL5p\n2R5ttj8COKKMspiZWfc6vUjQzMzsNU4eZmbWNScPMzPrmpOHmZl1zcnDzMy65uRhZmZdc/IwM7Ou\nOXmYmVnXnDzMzKxrTh5mZtY1Jw8zM+uak4eZmXXNycPMzLrm5GFmZl1z8jAzs645eZiZWdecPMzM\nrGu1JA9J90m6WdIkSb152WhJF0m6K/9dtrD9oZKmSrpD0jZ1lNnMzGar88xjy4jYKCJ68uNDgEsi\nYk3gkvwYSesCuwLrAdsCv5RUyvS4ZmbWmeFUbbUjcFK+fxKwU2H56RExKyLuBaYCm9RQPjMzy+pK\nHgFcLOkGSfvlZctHxIP5/kPA8vn+ysC0wnOn52VzkbSfpF5JvTNnziyj3GZmBoysKe67ImKGpDcA\nF0m6vbgyIkJSdLvTiDgOOA6gp6en6+ebmVlnajnziIgZ+e8jwDmkaqiHJa0IkP8+kjefAYwrPH1s\nXmZmZjWpPHlIWkLSUo37wPuBW4BzgQl5swnAX/L9c4FdJS0iaTVgTeC6akttZmZFdVRbLQ+cI6kR\n/w8R8XdJ1wNnStoXuB/4OEBETJF0JnAr8DJwQES8UkO5zcwsqzx5RMQ9wIYtlj8GbNXHc44Ajii5\naGZm1qHh1FXXzMzmEU4eZmbWNScPMzPrmpOHmZl1zcnDzMy65uRhZmZdc/IwM7OuOXmYmVnXnDzM\nzKxrTh5mZtY1Jw8zM+uak4eZmXXNycPMzLrm5GFmZl1z8jAzs645eZiZWdecPMzMrGt1zGE+TtKl\nkm6VNEXSQXn5NyXNkDQp37YvPOdQSVMl3SFpm6rLbGZmc6pjDvOXgS9FxERJSwE3SLoor/txRPyo\nuLGkdYFdgfWAlYCLJa3leczNzOpT+ZlHRDwYERPz/WeA24CV2zxlR+D0iJgVEfcCU4FNyi+pmZn1\npdY2D0njgbcA/86LPi9psqQTJC2bl60MTCs8bTp9JBtJ+0nqldQ7c+bMkkptZma1JQ9JSwJnAwdH\nxNPAscDqwEbAg8BR3e4zIo6LiJ6I6BkzZsyQltfMzGarJXlIWpiUOH4fEX8CiIiHI+KViHgVOJ7Z\nVVMzgHGFp4/Ny8zMrCZ19LYS8Fvgtog4urB8xcJmHwZuyffPBXaVtIik1YA1geuqKq+Zmc2tjt5W\n7wT2AG6WNCkv+xqwm6SNgADuAz4DEBFTJJ0J3ErqqXWAe1qZmdWr8uQREVcCarHq/DbPOQI4orRC\nmZlZV3yFuZmZdc3Jw8zMuubkYWZmXXPyMDOzrjl5mJlZ15w8zMysa04eZmbWNScPMzPrmpOHmZl1\nzcnDzMy65uRhZmZdc/IwM7OuOXmYmVnXnDzMzKxrTh5mZtY1Jw8zM+uak4eZmXVtnkkekraVdIek\nqZIOqbs8ZmYLsnkieUgaAfwC2A5YlzTf+br1lsrMbME1TyQPYBNgakTcExEvAacDO9ZcJjOzBdbI\nugvQoZWBaYXH04FNmzeStB+wX374rKQ7KijbXKR+N1kOeNTxHd/xHX8Yxl+1kzjzSvLoSEQcBxxX\ndzn6I6k3Inoc3/Ed3/Hn1fjzSrXVDGBc4fHYvMzMzGowrySP64E1Ja0m6XXArsC5NZfJzGyBNU9U\nW0XEy5I+B/wDGAGcEBFTai7WYNRdteb4ju/4jj8oioih2I+ZmS1A5pVqKzMzG0acPMzMrGtOHmZm\n1jUnDzMz65qTR8kkbVu4v7Sk30qaLOkPkpavIP4oSd+TdIqkTzSt+2XZ8fsoz8aSlq0o3oGSxvW/\nZWnxj5b0zrri5zJI0qaSds63TaUOrkMemtiLS/qKpC9LWlTSXpLOlfQDSUtWUYYWZfpuhbHq/vzv\nIGnRUvbt3lblkjQxIt6a7/8GeAg4HtgZeE9E7FRy/LOBu4BrgX2A/wKfiIhZxbKVGP9U4OCIeFTS\nNqRjvxNYE/ifiDir5PhPAc8BdwOnAWdFxMwyYzbFnwncD4wBzgBOi4gbK4z/fuCXpPdA48LascAb\ngf0j4sKS459JGlpoMWBt4DbS/2EHYIWI2KPk+Mc0LwL2AE4GiIgDS45f9+f/BdL7/wLS+/8fEfHK\nkOw8Inwr8QZMLNyf1LRuUgXxm2N+HbgKeH2xbCXGv7lw/2pgfL6/HHBTBfFvJJ1hvx/4LTAT+Dsw\nAViqivj571rA/wJTgNuBw4G1Koh/W+N/3rR8NeC2CuJPyn9F+uJU4fHkCuJPA04F9syv+YT8HpgA\nTKggft2f/xuBZYFPA5cADwO/IiWuQe3b1Vble4OkL0r6ErB0U3VBFf//RSS9FicijiD98rmClEDK\ntpCkUfn+q8B/cjkepZqLVCMiXo2ICyNiX2Al0i/xbYF7qoifC3FnRHw7ItYDPg4sCpxfQfyRpIFE\nm80AFq4gPpBeBOD8/LfxuIpqj3VJgwBuC1wUEScBz0TESfl+2er+/EdEPBERx0fEVsCGwK3AkZKm\n9fPctuaDnSncAAAbYUlEQVSJK8zncccDS+X7J5J+cc+UtAIwqYL45wHvBS5uLIiIEyU9BPysgvjf\nAi6V9AvSGc9Zks4FtiSdAZRtjrr9iPgvaWibcyUtXnX8XIbJwGTg0ArinwBcL+l0Zo9MvQqwC+lM\nrGy9kpaMiGcjYp/GQklrAM+UHTwingEOlrQx8HtJf6Patt66P//N7/+HgGOAYyR1NHpunzvOPwTM\nSiPpjaTT5rWY/Uv4zxHxjwpirxURd5Ydp038JSPi2bri5zK8iTT/zcp50Qzg3Ii4tb5SpYb8qPAL\nKP/q3x94e0TsXlXcOknaIiIuK2XfTh7lk7QOrT+8ty0I8Yerur7YJe0fEZX3dCvEf0NEPFJj/O9G\nxNfqil8lSZuS2paelrQYcAjwVlLV0Xcj4qlaCzgITh4lk/RVYDfS7IeNuuexpJGBT4+II+fn+LkM\n2wA7MWfy+ktEVFFt1SdJ/4mIVUqO8cXmRaTqqu8CRMTRJccf3WLxROAtpM//4yXHr7W3UzuSbo6I\n9UuOMQXYMNLgrscBzwN/BLbKy3cuOf46wI9J7Y0Hkjpt7ETq8ThhMD8g3eZRvn2B9XJd+2skHU3q\neVP2l3et8SX9hFRddTJzJq8DJW0XEQeVHL/5y/u1VUAV1xl8i9QwPoXZ9c8jmF0PXrZHSV2Fi1Ym\nJZAAVi85/oeBy4ELmX38uwI3lBwXAEl9fTkLWKGCIiwUES/n+z0xu2v8lZKqaPM4Dvgh6b3+T+Cr\nwN7AB4Gfk5LYgPjMo2SSbge2iYj7m5avClwYEWvP5/HvjIi1WiwXcGdErFly/BdJH56XW6z+QkQs\nU3L8VYCjSD27vhURz0u6JyLK/tJuxP8S8D7gyxFxc152b0SsVlH8pYBvA28gXdfzQMXH/1/g97Tu\n2fXRiCg1iUs6i9TL7HeSfgf8IiJ6Ja0F/D4i3lZy/Bsj4i35/tSIeGNh3aCu8/KZR/kOBi6RdBdz\n9nZ5I/C5BSD+i5LeFhHXNy1/G/BiBfEnkhrn5/qlK+lTZQePiP8AH5O0I3CRpB+XHbMp/lGSzgB+\nnLtmHk41XWQb8evu7TQZ+FFE3NK8QtLWFcT/FPBTSYeRzgKvya/DtLyubCMK95urSF83mB37zKMC\n+TqLTZizzv/6GKorPYdxfElvBY4lVdM0qq3GAU8BB7T6Uh/i+GsDj0eLq8olLR8RD5cZvynekqQv\n700jYvOq4hbi7wB8jXTRYBVVNs3xK+/tJOndwP05iTev64mI3orKMYp0YeZIYHpV7ztJnyGd4Tzb\ntPyNwOci4uAB79vJoxp5HJvXvryr/NIaJvFXaIr/UJXxLck9ftZo9Uu8gtijAcpupB+O6v78lcHV\nViWTtBFpOIClSb+8BYyV9CRpbKGJ83P8hpws5kgYktaJiNvLjCtpJKnTwIdJV5dD7u0F/La5I0EJ\n8Zcm9a7aCVie1OvlkRz/yIh4ssz4uQxzddWW9EoVXbVzm88PSA2zT6ZFGkVqvD0kIu6roAy19fZr\n+vy9NrZYlZ+/fPxjgUuK/29J+0TECQPer888ypV7VHwmIv7dtHwz4NcRseH8HL+dirrKnkb60jqJ\nOXt7TQBGR8QuJcf/B+mL8qTG2VY+C5sAbBUR7y85ft1dxa8BfgL8sVFNKmkE8DHSgJmblRy/r95+\newJ3VdDbr+7P//eAd5La/j4E/CQifpbXDarB3MmjZJLu6qtHUXPvh/k0fnM//9dWkfqZj+pj/VDF\nb9nbq791Qxj/jr56tLVbN4Tx76R1V+3XAVMq6O3W7v3X57ohjF93b7+6P383A2/J15ksA/wBuCMi\nvlDsiTUQrrYq3wW5h8nJzO7tNI70y6eKi+Tqjr838CVgVot1u1UQ/3FJHwPOjohX4bUOBB8Dnqgg\n/v2SvkI683g4x18e2IvZr0eZXiVV1zVf67FiXle2G5TmjTmJOd9/E0gjvpat7t5+dX/+RjauM4mI\nJyV9CDgudyF2b6vhTtJ2tB4epIpRVWuNL+mfwGERcXWLdaVfbyBpPPB90uCQjWSxDHApqc793pLj\nL0sakmJHUpsHpLafc4HvV3CF97aki8FadtUuu94/n+HsS4v3H6nNqdWPiqGMX2tvv1yGOj9/fwV+\nGBGXNy3/DvC1iBhwt2knDytV7mHzYkQ8PwzK8nqAiHis7rJUqe6u4sPBgtrbL/euIyJeaLFu5YiY\nMfezOuNqq5IVevvM1duDCnr75DJMII1rs05edBtwTEScXHbs4dQts5g0JL0P+EpEvK/suK16O1Hh\nwJQR8aqk55h9cd6zVSaOOns75fgCVi3EHynp4ajgl/Mw+Py/Qpo9tFGeLckDM0bEBYPZsc88SjYM\nevtMIF1l/kVSjwuR3jw/JPW8OKXk+MWuqm8gXd1cWVdVSe8ldZVcCfgzqQrrd6T/wxER8aeS49fd\n22lp0v96HOlqawHrkybl2jEini45ft29neqehrfuz/9NwBYR8YSkL5O6rJ8PvAfojYiBzykTJU+D\nuKDfSD06ul43hPGvpfU0pOOBayuI/w/SYGwrFJatkJddWEH8G4EtgEVICexZUl1/Za8/sHCL5a8j\nfXmWHf8Y4EekAfoayxYiXXvxsyqOv4/lquj4656Gt+7P/y2F+73AYvn+SAY5DbCnoS3f45I+psJU\nsJIWkrQL1fT2GRUtLsTKy0rtJpuNj4jvR6GOOSIeiojvk6oSyhYRcVlEzIqIP5Pqu39eQdyGRm+n\nZlX1dtqa1DHgtVj5/tfyurK9KKnV4H9V9Xaqexreuj//T0t6c77/KGn6Y0j/l0F9/7vNo3y7kqpK\nfimpubfPrhXEn6uhrMN1Q6XurqrLaM5huUcWH0fJ1VbUPzDlSzF7SPDXROr3X2pPp2xv0nu/VW+n\nvSqI32oa3nGkz14V0/DW/fn/LGlAyptI1cW9kq4gVV1+dzA7dptHhero7SPpeWBqq1XA6hGxRMnx\nm7uqBvAw1XVV/V2b1RGFebVLLEOdA1PeTmpzaZ5LXcCpEfGmssuQy1FbbydJ6wI7UMM0vJIWjtwo\n3vz5l7RalNxVPMcZAbyfOaeB/kcMsr3RyaMmVfX2UT+T3EfTPB82tHKD8dXAVTGIbpGDiH8ZbYZg\nj4gtS47fdviLqGhstbpIOh/YKSJealq+ASmBja+lYEPA1VYl66+3T9nx604O6nsmP6CSaVhrjU86\n69sJ+EHqMcrV+XYVcFOxLaIMEbFFmfvvwFGF+xuTGm0bZ0FBunizNJIupe/kGREx4Jn0OjSRdJX5\nhyJf6yRpC+BUUpVeqZSmAfgK8BFSL6+XgLuBX0XEiYPat888yiXpRuALwDXAdqQ3zSFVNdpKupc5\nPzwqPI6IWKPk+K8Ck4ALSEOUzFF9EhHfmp/jN5VlJeAd+bYD8IYof2yvtvOGRMQVZcZvKsugxlIa\nYMyNWyzejPSF+kiUPJNfLsNhwDakz//7SQNF7hwVzCUi6S/AOcDFwMeBJUjdxg8jVR9+bcD7dvIo\nl5pGrqxiMLym+K9vWrQQ6U30P8DEiPhIyfE3JNW5b0uat/o00tDQlbzx6o6fy9C4tuIdpBFO1wVm\nAtdUkDzPa7E4gA2AcRExosX6ssoyqFFchyD+e4D/JfU4OiIGeZFcl7G/CHyG9ONl+4ho1Q5ZRtyb\nojByr6TrI+JtuR3u1ohYp83T23K1Vflq7e1TaJxbCNgD+DLpl/gHqmgwjIibgJuAQyS9g/RF/jNJ\nX42Ic+f3+JIuInWJnkS65ua7UdGV5QAR8aGm8ryT9KvzIeDzVZWjTvkK98NIZ55HRMSlFcY+j5Ss\nBYwhVWMenaswiYgdSi7Cc5LeFRFXKs0k+XiO+6oahRggJ4/yXU4aR7/hisLjAMq+wnlhYB9S1dmV\npMa7Sn71NJVjDPAW0i/w6aRugwtC/HtIv/LXBB4DHpU0MyIerSg+AJK2Iv3qDlICu6iiuD9jdjXp\nWDUN0R8RB5Yc/3rSl/YPSVXHczTiV9Bg/6M+7lfls8BvJK0F3EIaKqXxefjFYHbsaqsaSfpIRJxd\ncozpwMuketa55nEu+8xH0j6karJFgT8CZ0ZEZYmj7viFcowi1bW/I/8dQ7r6d0LJcT8AfJ10XcUR\nEXFlmfFaxG97fBFxUsnxL6N9g3mpDfa5DBuRruuZUuVZZ9mcPGqkambSO5H2H55Sr3PIDda3MHs+\niTnKUvZpe93xC+VYhHRV9TuZnUAeiYj1S477KulM6yZavA+qOv4FlaRvALuT2ts2Bb4XEcdXXIZS\nBuZ08qiRpGkRMa7ucpQpN1L2KZrmGZgP4/+YlCzWJI2zdQ2pm+41g71Iq8P4dR9/23alCn487Nxm\n9Szg7oi4vcT4U4C3RcTzufPK36vo4VWIX9rAnG7zqFcVQ0LXfZ3DssDVdVQVZbcCY5o7B+SrjmdW\nEP9eUvfsSVVcUd6s7OTQgbeThgU5Dfg3c1/pXrYPtVk3EniTpKtLbHuZ1bi+IyIeK45xVZF9aT0N\n8dHAFMDJY7hSmkO4VZIQs2eWK9NShfufAX5dQcyi3YFf5GFSGhfHXR0Rt1QU/2ekIbmbvZ7UA+cT\nZQaPiGOU5nRoTIE7jlR9cXdElD4Nq6Q1SYMgPgEcDRwPvJt0odi+FVxrsALwPtKv308AfwNOi4gp\nJccFICLaXoiXv8xvLrEIqxfOvgSsUTwbq6DasLRpiF1tVbLhNDxIHRdpFWKPZ/YFcm8nDQ54fURs\nX3Lc3ojo6WPdLRHx5lbrhjD+p0jDnz8LfJvUVXoiqefXCZFGFy4z/pWkuTRGkXrcHQycR0og34mI\nTcuM31SWRUhJ5IfAt6q4UFbSnm1WR0ScImnFiHiwpPh1VxuWNg2xzzzKFxExVy8nAEnvZu5fBKWW\npcJYcwaOuE/SosBi+da4X7al2qyrYkjuLwBr5HLcBqwaEY9KWhy4njRcTZmWjIjjACR9NiLOyssv\nkvTDkmOT4y4CfICUOMaT5hg5p4rYpE4KrTQGSjylrMQBfSeHfAa6K6krf2ki4u+5m+6QD8zp5FG+\nyyT9Cjiq8WIpDUl+FGla2Ja/iucXkr5GOtMYA9xBulDu58B+FbUBTJW0fUSc31Su7UjXYJTtpYh4\nAnhC0tTG9R25AfWlfp47FIpVE82zBpY+n4ikk4E3k2av+1aF1ZUARMRrF0Lmi+I+SZqI7FoqGFuu\nKF9b8TFSEl2JihJoviDwXtK4VpCGJRn0Z8/VViVTGpL8SFJ1zUGki9S+SKrKODZKHhivqc3ljcwe\nnl2ks6INSo5/O/AcqarkauDfEfFUmTGb4q8F/DXHviEv7iEltA9GxJ0lx28Mib4QqeH8E6T/fSVD\nomv2kPwinQEVX/8qhuR/lfT6NzTei433X+kTkuU2p71IQ/JcS+oue0fZcXPspYCdSa/7WqSLgneJ\niLEVxd+INDDr0qTeViL1tnqSNA3vgC+SdPKoiKSDgB8DDwCbRUSr2c3KiFt7m4uk0cxu79gMWJJ0\n3cHVEdFuvo2hir8I6cPbaN+YAvwhIkqfyU5pVNe+lH6R2nB4/esk6QDSj7ZLSPPH3Fdx/BeA60id\nM66MiJB0T0SsXlH8ScBnIuLfTcs3A34dhXGvut63k0e5JC1DqtfelDSS5/bAVsBBEfHPCuKv0q7N\nJSL+VXYZCvFGkobl3pzU82u1KHlgvuF0/C3ib9r8oS4hxrA4fknrk6ppIQ3IV0lvq3zm8wipW/Zc\no0tXcOZ9MKltYwlSd+UzgIsqTB53RcSafaybGhFvHPC+nTzKJekeUlfRn0SeDjSfSv4SuD8idqsg\nfp9tLn31RBrC+DswezTZ9Ui/+q8iXSx3dUSUeq1F3cffT9mqGGGg7td/aeAvpB4+N5G+tNcnDZWz\nY0Q0t8MMdfxhceYlaXVSEtmNdMHo4cA5FVSbHkOqrjyZOafh3RO4NyIGPBWyk0fJ+vnl9+myhyoY\nBm0ufyJf2wHcEE0zqpWt7uPvp2yljzBQ9/HnL6+XSLNmNq51WSiXabFig/aCQtKbSdWoHx/ML/8u\n4m1H6+FJzu/7WR3s18mjXKp5DoNCOepqcxku1Sa1HH87VZx5FGLV9frfCmzQOOsuLB8J3FxBh4Fn\n6Psi3Uoa7OdX7qpbvqqHY5gz+JxtLtuS2lwukFRJmws1d1Wu+/g1ez6HuVaRrnIvO37dr/9LzYkD\nICJeljSr7OAR0e46n9LVnbxykt6XNBVy8czjL8Bvo2nYkq727TOPcikNid7n+FFR/hzedbe51F1t\nUvfx132Fcd3H3+iq3PwjqpKuyu1IWjIinq0rfhUknUbqlnsScw6MOAEYHRG7DHjfTh7lkvQgcCx9\nnIFE+dOQ1trmUog1HKvNqmhz6jN+FYbB8V9Gm5ENImLLMuO3U2W1YR/xS09eku6MiLW6XdfRvp08\nylV3m8cwiF93V+W6j/+1+JLOjpLnjG8Xf0GkvkeVFvD1iBhdZXnmKEA1ve2uJVURn93UYeFjwBdj\nEGObuc2jfLW2eQyD+BNJVSQH5GqTCxvVJpJKrzah/uMvxq+kb3+b+NUHl94GTIuIh/LjPYGPkMZ0\n+2ZEPF5yEb5LGohxrnYX0lX/peoneS1ZdnxS9+Dvkz5vT+S4SwOX5nUD5jOPkkka3dcHpKLT1rrb\nXOquNqn7+ItnHpWfBQyH4we2jojHJW1OmpTo88BGwJsi4qMlx78a+HxE3NBiXRVdpV+k7+T1hYhY\npsz4TWV5PaR5RYZifz7zKFk/v6xuJV08VaYRpF84df0C/TPQ8guzovaWuo9/Q0lP5/iL5ftQXVfR\nuo9/ROEzsAtwXEScDZydh84o295AX1+WVVwgOhH4cx/J61NlB1eLmRSl2W+FiPjTQPft5FGyYXDa\n+mBE/F8FcfpSd7VRrccfJQ+/0oG6X/8RkkbmKsutgP0K60r//okWAyDmHoBPRsTDZcen/uT1R2BS\nvsGcn8cgDdQ4IE4e5au1zpX6v7zHtEmgVUyDW/fx96mirqJ1H/9pwOWSHgVeAP4FIOmNQOmjK0v6\nBnBmRNyuNEDm34ENgZclfSIiLi4z/jBIXjuT2jY2IF3bcVpETG3/lM64zaNkw6DOte42l7q7Ktd6\n/O1U1Num9uNXGsF1ReDCiHguL1uLNFHVgIcE7zD2FODNERGS9iNdc7I1aXj0kyJik5Lj95m8gNKT\nV6EcS5CGKNmFdHHq1wd7jZHPPMpX62nrMGhzqbvaqNbjr7vasu7jz2W4tsWyOyVVUW37Usz+hbwN\ncHoe6eC2fPV12XYhTT8M6cI8SBOjrUW6cK+S5AG8SDrTexpYlTST56A4eZSs7tPWur+8qL+raN3H\nv6B3FW2niuQ1Kw9E+DCwJWlCqIbFS44NNScvSe8lVVttQkpUP42I3qHYt5NHyequc6X+Npet+lpR\nUbVJ3cdfa28bnLwOIjUajwGOjoh7c7m2B26sIH7dyetiYDJwJbAIsGe+1gaAiDhwoDt28ihf3aet\ntX55DYNqk7q/vOvubVP38dedvN8JNLqEh6QvAI+SZvUr+wJVqD957V3Wjt1gXjJJN0bEW/L9s0mN\nhr/Oj0u/aEzS2sBjEfFoi3XLl111VvfwEHUffx9lalRblv7hq/v4h0GHkcNbLB5NqkL6ZkScXnL8\nLzK76jbyrZG87i0zdtmcPEqWx5b5FOm09Q5g48Kvj9sjYp12zy+pTFV+eQ2bK2wbKj7+YdHbpqlM\nC0zy6ouk0cDFFfx4qzV55TJMIJ0BrZ0X3QYcExEnD2rHEeFbiTfSgIC3k6ouDiss357U57rs+N8g\nTTcKqc7zUuBx0rzOW1cQ/2pSwmy1btoCcPxTmP0jbb8cfwTwJuC6+f34+yjTso3/SZ034MYaY48G\nJlYQZwKpemxL0phWywDvBW4A9hjMvquoc1zQNepcvws8L+kLkvYAbotq6lx3IZ3xwJxtLu/JZSrb\n3qRB8Fqpos6/7uNv2dsmIm6jmjbHWo9f0jckrZPvLyLpUuBu4GFJW5cdv025tgSeqCt+pLbAKnoi\n/j/gwxFxaUQ8FRFPRhrN+iPAAYPZsRvMy9dqJrPxwNclfTPKP22ttatg1H+Fbd39/OvubVP38dfa\nYUTSzcw9n8ho0rwye879jGpUmLxGRcR9zQsj4j5JgxpXzcmjZNHHFdSNOlfSKKNlqvXLaxh0Va77\ny7vu3jZ1H3/dyeuDTY+D1AbzXAWxh0PyemGA6/rl5FGTSENUV3HaWveXV91dles+/gW9q2itySsi\n+qoyrUqtyQt4k6TJLZaLQc4v4+RRkwpPW+v+8qr7l2fdx193tWXdx1938qrVMEhepc0R7666Jevv\ntDUibi85ft393Gvtqlz38fdlQekqOj9f57Cgc/IomaRVmxZVfdraUoVfXpuSqqfGAD+OiO/k5duT\nugpW8eu3VbkqOf5+yvDaBaQ1xF4gkteCTtIzzP3jFYZgMjInjwVYFV9ew/mXZ81f3lsC/xsR760j\nfi7DfJ+8rDxu81hAVdjmUnedf0tVHf8w6G3T0nC4zqGiDiNWEieP+VzdX151d1Wu+/hxV9GW6k5e\nNniutprPDdc2F6is2mzYHn8V6j7+ujuMWHl85jGfGwZdBVuq6pfncD3+qgyD46/7OgcriZOHlWq4\nVptYNYZB8rKSuNrKSlV3tYmZlcPJw8zMuuYh2c3MrGtOHmZm1jUnD7MOSPq6pCmSJkuaJGlTSQdL\n6ndk2ObtJJ0vqfLpd82Gkts8zPoh6e3A0cAWETFL0nLA60hT7PZEi/m5m55/Xyfbmc1LfOZh1r8V\ngUcjYhZATgIfBVYCLs1TqyLpWEm9+QzlW3nZgS22uy8nICR9UdIt+XZwXjZe0m2Sjs/7ulDSYo39\nSbo1nwF5UEGrjc88zPohaUngStLkRRcDZ0TE5c1nFJJG5zGbRgCXAAdGxOQW291Hmr99VeBEYDPS\n4JH/BnYnXTw5NT9nkqQzgXMj4lRJDwCr5TOgZSLiyWr+C2Zz8pmHWT8i4llgY2A/YCZwhqS9Wmz6\ncUkTSZMcrQes28+u3wWcExHP5Rh/At6d190bEZPy/RtIg0kCTAZ+L2l34OWBHZHZ4PkKc7MO5NkP\nLwMuy1fNTyiul7QaaYrVt0XEE5JOBBYdRMhZhfuvAIvl+x8ANgc+RBqZeP2IcBKxyvnMw6wfktaW\ntGZh0UbA/cAzzB5yfhTwHPCUpOWB7QrbF7cr+hewk6TFJS0BfDgv66scCwHjIuJS4KvA0sCSAzsq\ns8HxmYdZ/5YEfpa7175Mao/YD9gN+LukByJiS0k3ArcD04CrCs8/rrhdY2FETMxnKNflRb+JiBsl\nje+jHCOAUyUtTWojOcZtHlYXN5ibmVnXXG1lZmZdc/IwM7OuOXmYmVnXnDzMzKxrTh5mZtY1Jw8z\nM+uak4eZmXXt/wPuOjQX3wSmVwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2908c90190>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "N = 10\n",
    "index = np.arange(N)  \n",
    "bar_width = 0.5\n",
    "\n",
    "plt.bar(index, precTop10, bar_width,\n",
    "                 color='b')\n",
    "plt.xlabel('Stations')\n",
    "plt.ylabel('Precipitations')\n",
    "plt.title('10 stations with the highest average precipitation')\n",
    "plt.xticks(index + bar_width, stationsTop10, rotation=90)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id=\"use_spark_sql\"></a>\n",
    "## Use Spark SQL\n",
    "\n",
    "`Spark SQL` lets you query structured data, for example, data in a relational table and can be a very powerful tool for performing complex aggregations.\n",
    "\n",
    "To create a relational table that you can query using `Spark SQL` and fill it with snowfall data, you'll use the `Row` class from the `pyspark.sql` package. You will use every line in the `weatherSnow` RDD to create a row object. Each of the row's attributes will be used to access the value of each column."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "To filter the weather data to show only those weather stations that contain the keyword SNOW, you need to reduce the data set to lines with SNOW in the third column. To do this, run the following commands:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4271464"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weatherSnow = weatherParse.filter(lambda x: x[2]==\"SNOW\")\n",
    "weatherSnow.count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The next commands convert each line of the `weatherSnow` RDD into a row object, infer and apply a schema to an RDD of row objects, and register the table name `snow2015`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "from pyspark.sql import SQLContext, Row\n",
    "from datetime import datetime\n",
    "\n",
    "# instantiate SQLContext object\n",
    "sqlContext = SQLContext(sc)\n",
    "# Convert each line of snowWeather RDD into a Row object\n",
    "snowRows= weatherSnow.map(lambda p: Row(station=p[0], month=datetime.strptime(str(p[1]), '%Y%m%d').month, date=datetime.strptime(str(p[1]), '%Y%m%d').day,metric=p[2], value=int(p[3])))\n",
    "# Apply Row schema\n",
    "snowSchema = sqlContext.createDataFrame(snowRows)\n",
    "# Register 'snow2015' table with 5 columns: station, month, date, metric, and value\n",
    "snowSchema.registerTempTable(\"snow2015\")\n",
    "sqlContext.cacheTable(\"snow2015\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Compare the number of snow days between two stations\n",
    "\n",
    "In this section, you'll calculate the number of snow days for each month of the year at the `US10chey021` \n",
    "and `USW00094985` weather stations. With that information, you'll plot a bar chart to compare the number of snow days for each month at the two stations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "First, find out on how many days of every month it snowed at the `US10chey021` weather station:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "snow_US10chey021 = sqlContext.sql(\"SELECT month, COUNT(*) AS snowdays FROM snow2015 WHERE station='US10chey021' GROUP BY month ORDER BY month\").collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[31, 27, 31, 30, 31, 30, 31, 31, 30, 31, 27, 25]\n"
     ]
    }
   ],
   "source": [
    "US10chey021_snowdays_y=[0] * 12\n",
    "for row in snow_US10chey021:\n",
    "    US10chey021_snowdays_y[row.month - 1]=row.snowdays\n",
    "    \n",
    "print US10chey021_snowdays_y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Next, find out how many days of every month it snowed at the `USW00094985` weather station and plot the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "snow_USW00094985 = sqlContext.sql(\"SELECT  month, COUNT(*) AS snowdays FROM snow2015 WHERE station='USW00094985' GROUP BY month ORDER BY month\").collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[31, 28, 31, 30, 30, 30, 31, 30, 30, 31, 30, 31]\n"
     ]
    }
   ],
   "source": [
    "USW00094985_snowdays_y=[0] * 12\n",
    "for row in snow_USW00094985:\n",
    "    USW00094985_snowdays_y[row.month -1]=row.snowdays\n",
    "    \n",
    "print USW00094985_snowdays_y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEWCAYAAABbgYH9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYVcX9x/H3R0BRQRBEo6Ai1ohREGzR2Es0JjZisAvW\nRI2x5KcxzRJrNMZYY8cSNXaixhIVGwYFRRQbGlEpSlEEVBDw+/tjZpfLsnt3Fzhb4PN6nn32njpz\n6vfMzLlzFRGYmZktaks1dgbMzGzx5ABjZmaFcIAxM7NCOMCYmVkhHGDMzKwQDjBmZlYIB5gCSbpG\n0u8bOx+LO0n/lnRYY+ejsUk6XNLzjZ0PswqLJMBI2kbSYElfSPpM0guSNlsU617A/AySNEPSNElT\nJQ2TdLqkZRoyHxFxbEScsyDLSrpY0qi8DW9LOrTK9B55u77K/3uUTNtI0mOSJkma74tOJftnev57\nZ0HymNc1WtLOtcxzhqQPclpjJN1VJS9H1iO9MyXdVjouInaPiAH1z/2CqelGXrovJHWRdG8+Bl9I\nekPS4SXzXivpHUnflo4vmX6SpE/y+XtjQ5+71eRnp3wefiXpaUlrlkz7dd6+afk4/7rKsudIel3S\nbElnNkLe5ztn8viQtE7+3F3S4/n+NSVfU3vkae9I+lnJclvnZauOmyapZR4+UNKHkr6U9ICkDiXz\nLpOP6dR8jE+ukq9y1/Yyki6VNE7S55KuktSqmm1bN1/jt1UZf6Sk9/K1+Kik1aqs+xpJn+b98C9J\nnUumj5b0dcl94/Ha9v1CBxhJKwAPAZcDHYDOwFnAzIVd90I6PiLaAqsCpwB9gUckqXGzVWdfAj8G\n2gGHAZdJ+j6ApKWBB4HbgBWBAcCDeTzALOCfwBFl1n98RLTJf+sXtA0olSwOAXaOiDZAb+DJotJr\nQm4FPgbWBDqS9sGnJdNfA34BvFJ1QUm7AacDO+Xlu5GuqUYhaSXgPuD3pGt8KHBX6SzAoaRz8YfA\n8ZL6lkx/D/g/4OEGyfCC+RfwBPAdYGXgl8DUPO1ZYNuSebcF3q5m3IsRMVtSd+DvpGO+CvAVcFXJ\nvGcC65KO7Q7A/0n6IdTp2j6ddA1tBKwHbAr8rprtuRJ4uXSEpO2B84C9SMfxA+COkllOBLYCNgZW\nAz4n3ddL/bjkvrFrNenOKyIW6i9v7JQy0w8Hngcuzhn+ANi9ZPpqwEDgM9KJeFQe3xr4GlgpD/8W\nmA2skIfPAf5aQ5qDgCOrjFuDdKD3zMObAy8CU4DxwBXA0nnalcAlVZYfCJyUP58GjAWmAe8AO9WQ\nj5uBP+XP2wNjSMFuQk6zXz3280DglPx515y+SqZ/BPywyjLrpENc+/4pk+7awFPAZGAScDvQPk+7\nFfg2H6fpwP9Vs/wVZY7TucAcYEZe/oo8/jLSzXkqMAz4QR7/Q+AbUgCdDrxWdXtID02/Az7M+/kW\noF2e1hUIUsD+KG/Pb0vysznp5jmVFAz+Uu6crmb8aFIgJeevRx327/PA4VXG/QM4r2R4R+CTkuHV\nSTf8ifm4XFGaL2q+1toBN+RzbyzwJ6AFsDTp+vteybwrk66XTsDRwOCSacvnY75BDdv0N+Dyasbf\nBpxZZl+sltfboWRcz3ycWpHO52eAL/K4u+p4Dp8J3FbN+MjrXCl/bl/D8ocAr5cMP5L3ddVxv8uf\nzwP+UeUa+gZom4fHAbuWTD8buLMu13Y+P/cvmXYg8HGV/PYlPWDOs935vLiqyv4OYO08fDVwUcn0\nHwHvVHd+1/VvUVSRvQvMkTRA0u6SVqxmni1IN+KVgIuAG0pKEneSbryrAX2A8yTtGBEzSBF4uzzf\ndqSbxtYlw8/UNZMR8RHp4Pwgj5oDnJTztBXpafEXedoA4ABJS0HlE9zOwD8krQ8cD2wWqYS0G2nH\n18V3SBd5Z1Lp4soa9tc8JC0LbAaMzKO6AyMiH/XstTy+rs7P1Tcv5CebGpMHzicdn++Sbm5nAkTE\nIaSTv+Kp5qJqlv8vcGiuRuktqUXFhIj4LfAcc0tTx+dJLwM9SE9Z/wDultQ6Ih4lXbx35fk3qSa9\nw/PfDqQn/zakIFdqG2B90jH/g6Tv5vGXAZdFxAqkm8I/y+yX2vyXdHz7Slqjnst2Jx3PCq8Bq0jq\nmPffQ6RroSvpXLqzZN5y19rNpIe0dUg37l1JgfmbvI6DS9ZzAPBkREysmp+I+JL0MDjf+ZbT+gFz\nz9U6i4hxpIe+/UpGHwjcExGzSA+Vj5Oe7Lsw/9P1gppM2p7bJO0taZUq058FukvqkO8JvUkluPYl\n47bO88H8++t9Uo3Oevl6X5X5j2/3kmXrc20L6CKpHVTWKJ0NnFzD/FWXhVQagvTwsbWk1SQtBxwE\n/LvKMrdLmpirE6u7/uax0AEmIqaSLtgArgMmShpY5SB9GBHXRcQc0s17VdIFszrpwJwWETMiYjhw\nPam4DSmAbJfrNTcmPRltJ6k16Yb7LPUzjnTTIiKGRcR/I2J2RIwmFWm3y9NeIj0l7ZSX6wsMiohP\nSYFpGWBDSa0iYnQ+gepiFnB2RMyKiEdIT7l1qZ66hnSSPZaH2+T8lZoKtK1jPk4j3Xw7A9cC/5K0\ndnUzRsR7EfFERMzMN5u/MDfo1yoibgNOIAXiZ4AJkk6rbZmImJyPzSWk/V3XaryDSCWP/0XEdOA3\nQN+KuvHsrIj4OiJeI+3XigtlFrCOpJUiYnpE/Leu21mNn5KC5++BDyQNV93bJase34qqmrakUtZq\nwK8j4st83ZS2B9V0ra0C7AH8Ki83AbiUdG7D3IeqipvOIaQSanX5qchTdefbmaT7yk113Naq/kEK\nbhXBqm8eB+n4rAmsVs12L7B8M9+B9KB4CTBe0rOS1s3TPyQ9SP2AdK6MioivgRdKxi0NDMmrLLe/\n2uThqse3Yl/Wtq8fBU6U1EnSd0hVeQDL5f/nADdExJhqNvVR4KeSNs4PrX8g3bcrlh1FqjkYm9P8\nLilYVTiI9FCzJvA08Jik9tWkU2mRNPJHxFsRcXhEdCFFw9WAv5bM8knJvF/lj23yfJ9FxLSSeT8k\n3fgg3ZC2J9Uzvk6qI90O2BJ4LyIm1zOrnUlVAUhaT9JDuZFtKunJeKWSeQcw94nuYPLFFhHvAb8i\nXUgTJN1Z2lBWi8kRMbtk+CvmnnDVkvRn0j7dv+SpZjqwQpVZ25Gq7GoVEUMiYloOGgNIF8oeNaS/\nSt7GsXk/3ca8+6ku6d0eETsD7YFjgXNyO0O1JJ0q6S2lxvEpedvqmuZqpHOowodAS1JdeIVPSj6X\nHoMjSPXab0t6WdKeNaQxm1RlU1Ur0k2QiPg8Ik6PiO457eHAA3VsA6x6fNvl/9NIJcgPq5xHpWq6\n1tbM+Ruv1Ig9hfRQtXKedwhpX2wvaQNSKWdgDfmpyNM855uk40kPhz+KiAVtg70X2ErSqqR2jW9J\ngRpSO46AlySNlNS/juuc73iVNIxXHK8xEXF8RKxN2ldfkqpXK1S0w2xbkp/nS8a9VLLN5fbX9Dxc\n9fhW7Mva9vW5wKuk82kw8EDehk+VXgbYmfTgMJ+I+A/pvnUvKZiOzuutCEZXkpomOpKqQe+jpAQT\nES/kB7OvIuJ8UvNCRY1QtRb5a8oR8TapKL5RLbNCLlFIKn0SWoMUQSHtwPWBfYBnIuLNPH0P6lE9\nBpBLS72Ye3JcTWqoWzdXiZzB3CIjpBvpXrkY+F3SgQQgIv4REduQTsQALqxPXuqR57OA3Un1tVNL\nJo0ENq5ys9qYBaiWyIJ5t73UeXn69/J+OrjKvHXujjuX3O4GRjD3/JhneUk/IN1I9gdWjIj2pCc6\nVTd/NcaRjkuFNUg3mE+rn32e/I2KiANIN90LgXskLV/NrB8Ba5Tu/1ylsDLzBreK9U4i1X+vRi5B\n12Ikc0tV5M+f5geqj3PaLatdsmYfk6ppVoqI9vlvhRwAK1Q8VB1CqpaaUV1+8j5Zm5LzLd/sTye1\nR1b39FwnEfE5qRrsZ6TqsTsrHqwi4pOIOCoiVgOOAa5SfgusFh+RnrxLrUU6L8ZWnTkiPibdbEvv\nYRUB5gfMvYc8VzKutDal6v5am1TCeTdv33jmP74jS5at8drON/jjI6JzRHQjVe8Ni4hvSQ/jXYGP\nJH0CnArsJ6nyRZKIuDIi1o2IVUiBpiXwRp7cA7gpIj7LwfJyYPPcRFCdcvcNYNG8RbaBpFMkdcnD\nq5OKuLVWL+QDOZjUHtBa0sakp8jb8vSvSI28xzE3oAwmPQXXKcBIWk7SdqQ3M14iNcZBKnJOBabn\nJ7afV8nbGFJbwK3AvblIjKT1Je2o9NroDFKj5Ld1yUt9SPoN6QLbuZqS2iBSVd0vlV4t/CXpYD+V\nl1WuRlw6D7fO+UVSe0m75XEtJR1EukgerSErbUlPVV8ovbL46yrTPyVVt9W0HYdL+pGktpKWkrQ7\nqT55SA3LtyVd+BOBlpL+wLxPdJ8CXXO9d3XuAE6StJakNsxts6npib80rwdL6pQv1il5dHXHdgjp\n2J+e9+PywAWkNr4P87ouVHpdvGV+gPo5JaVuSUvnYySgVV5PxTbdAhwhaUOlOvvfkx7aIJ3D44EL\nJC2fl6tol6xRRIwn3bgvkbRCPhZr52ujwm2kh7mDmffp/X5gI0n75Tz/kfSCxdt5Ww4i7eddIuJ/\n1ezXVnm5pUjHtLVK2uKq8Q9SSagPc6vHkPTTivsM6SWGoG7X3qPABpIOyXnpkPN7b6S3vlaUdJak\ndfJ+WQnoz7z3sGdJ7Vbbkkr8kGpV1iJVr5UGmNuBH0v6QT43zgHuK6mpuQX4XU73u8BRzD2+gyh/\nbXdWaiORpC1J58Yf87LXkgJ/j/x3DenNvd3ysq3zOSmldsFrSW2On+flXya1l7bLJbxfAOMiYpKk\nNZRexV46r+fXpFqFin1RvajHGwHV/ZGqnf5JehL4Mv//O3Pf9jqcKm/c5B22Tv7chdRo+RnwPnBs\nlXnPJ93El8nDx+flVymTp0GkG8C0/Pcq6S201iXzVLxqOJ30JHJ2Nfk8OKe1Q8m4jUkX+bSc54dI\ndcLV5eNmqrxFVmX6aGp4KyOnOzPnr+LvjJLpPUnB92vSq649S6Z1zcuX/o3O0zrlE2ka6Sb6X9KN\noaZ92T2nM51ULD+ldDtIrzx+lNd1ajXL70s6CT8nBfTXKXlrivSCxbt5+t9IbzXdmOcdTyrNVO4n\nUvH9+Tz/KyXHu/Qtsj+Qntgnkl/3rLJfWlY5VyqWvY305tl00hPj3mX2y4akNrFJpKB3D7B6yfTL\nSXXa03M+HgK+WyXdqsdo+5LpJ+f1TiW1ZyxTMm0NUom64s2+v9XxWmtHKrmPIZUKXwX6Vpn/P3l/\nq8r4nUnXy9c5711Lpn3A3Df7Kv6uqXIdVN3Ww8vs22VJ5+fIKuMvIt1fppPuFUeXTBsJHFRmnd8v\nOW/Gkdp6K86L5Umlt9F53Z+QHlQ6V1nHeFIDfOm4R/K2L19l/IGk6+JL0sNt6ZtxyzD3HP8UOLnK\nsuWu7W1zPr8ivcxRbpvPZN63yNqTag++zNt4PtCiZHpHUnCcQLqenwc2L7kPVCw7mfRVg941pV3x\np7ywVUPStqSbzprhHWVLAEk3kp5aq/tuhVm91Lced4mRi4gnAtc7uNiSQFJXUomzZ+PmxBYX7ous\nGrledArpFc+/1jK7WbMn6RxSY++fI+KDxs6PLR5cRWZmZoVwCcbMzArRLNpgVlpppejatWtjZ8PM\nrFkZNmzYpIjo1FjpN4sA07VrV4YOHdrY2TAza1YkzffF34bkKjIzMyuEA4yZmRXCAcbMzArRLNpg\nzKxhzJo1izFjxjBjxozaZ7Ymo3Xr1nTp0oVWrarr5LvxOMCYWaUxY8bQtm1bunbtiprNr4sv2SKC\nyZMnM2bMGNZaa63Gzs48XEVmZpVmzJhBx44dHVyaEUl07NixSZY6HWDMbB4OLs1PUz1mDjBmZlYI\nt8GYWY101qJ9Mo4/1t734ejRo9lzzz154403KsedeeaZtGnThm222YYTTzyRmTNnMnPmTH72s59x\n5pln8vbbb9OvXz9eeeUVzj33XE499dTKZR999FFOPPFE5syZw5FHHsnpp59eY9qDBg3i4osv5qGH\nHlq4DQU++OAD+vbty+TJk+nVqxe33norSy+9NLfffjsXXnghEUHbtm25+uqr2WST9AOX/fv356GH\nHmLllVeeZ/ubq8U+wNTlAnl6u1pnYfvtm0anoEvi9tTlptRUNNTxWdz229Bxc3vqGPfpOL6e/fW8\n46aNY81Wy3LIIT9jwIDz+d731mPOnDmMGvUh06YNZemlP+P884/l4Yfn/aHbOXPmcNxxx/HEE0/Q\npUsXNttsM37yk5+w4YYbFr5Np512GieddBJ9+/bl2GOP5YYbbuDnP/85a621Fs888wwtW77P44+/\nwBFHHMTTT98MwP77b0m/fjtyzDF/ZNq0tP1t2/YuPK9FcRWZmTUbkyZ9zne+k34ivkWLFmywQfq1\n7U6dOtCrV3datpz3mfmll15inXXWoVu3biy99NL07duXBx98EID33nuPnXfemU022YRNN92U999/\nH4Dp06fTp08fNthgAw466KCKX3tk2LBhbLfddvTq1YvddtuN8ePH8/7777PppptWpjdq1Cg23XRT\nIoKnnnqKPn36AHDYYYfxwAMPAPD973+fFVdcEYDNNvse48ZNqFx+6603ZcUVS38hvHlzgDGzZuMX\nvziAXr36cOCBv+bGG+9jxoyZZecfO3Ysq6++euVwly5dGDt2LAAHHXQQxx13HK+99hqDBw9m1VVX\nBeDVV1/lr3/9K2+++Sb/+9//eOGFF5g1axYnnHAC99xzD8OGDaN///789re/Ze2116Zdu3YMHz4c\ngJtuuol+/foxefJk2rdvXxnwStMtdeutD7LLLt9fJPumKSqsikxSa+BZ0u9PtwTuiYg/SuoA3EX6\nffTRwP4R8XlR+TCz5qWmN6IkcfrpR7H//rvz1FP/5e67H+Weex7jkUf+Xu80pk2bxtixY9lnn32A\n9EXFCptvvjldunQBoEePHowePZr27dvzxhtvsMsuuwCp6q0iIB155JHcdNNN/OUvf+Guu+7ipZde\noi6/s/Xss0O55ZaBPPbYdfXOf3NRZBvMTGDHiJief374eUn/Jv0k65MRcYGk04HTgdMKzIeZNSPt\nVmzHtC+mzTNu6pSpdFh3NQC6detCt259OPzwvenWbVcmT55Cx47tq11X586d+fjjjyuHx4wZQ+fO\nncumv8wyy1R+btGiBbNnzyYi6N69Oy+++OJ88++3336cddZZ7LjjjvTq1YuOHTsSEUyZMoXZs2fT\nsmXL+dIdMWIExx//J+6997Ia8744KKyKLJLpebBV/gtgL2BAHj8A2LuoPJhZ87Pc8sux0sor8fLz\nLwPwxedf8OLTL7LVVj149NHnK0sH77//MUsttRTt27etcV2bbbYZo0aN4oMPPuCbb77hzjvv5Cc/\n+Qlt27alS5cule0iM2fO5KuvvqpxPeuvvz4TJ06sDDCzZs1i5MiRQCr97Lbbbvz85z+nX79+QCpt\n7bDDDtxzzz0ADBgwgL322guAjz76iH333ZfrrjuLddddc2F2VZNX6FtkkloAw4B1gCsjYoikVSJi\nfJ7lE2CVGpY9GjgaYI011igym1aNpvS22qBBtedlUbx11ZzevquLRbHfXj7q5VrXsX7N9/dK9X0T\n6szLzuSiMy7i0rMuBeDIk4+kW7cunH32VfzmN5ey3HLL0LJlS66//hxatGjBp59OYrvtDmPatC9Z\naqmWlW0oK6ywAldccQW77bYbc+bMoX///nTv3h2AW2+9lWOOOYY//OEPtGrVirvvvhuA2bO/qHyD\n65tvJjBjxgfMnDmCAQPO4tRTj2Pq1OnMnj2bk0/+TeW6DjroIO6//3523XXXym248MIL6du3L7/7\n3e/o2bMnRxxxBABnn302kydP5uSTLwSgZcuWPPPMLQD06/dbnn9+GJMnT2GDDX7EGWcczXHHNd+3\nyAoNMBExB+ghqT1wv6SNqkwPSdWe4RFxLXAtQO/evZvPVW1mC63bet245p5r5ht/883nVTv/Kqus\nxNtvPwzMH8z22GMP9thjj/mWWXfddXnqqafmTbdbN3r1urRy+JJL/q/y88Ybr8+jj15bOVyazvPP\nP0+/fv1o0aLFPOt66aWX5kv3+uuv5/rrr68MYqVuuuncarevuWqQ78FExBRJTwM/BD6VtGpEjJe0\nKjChlsXNzJqsffbZh/fff3++YGUFtsFI6pRLLkhaFtgFeBsYCByWZzsMeLCoPJiZFe3+++9nxIgR\nrLTSSo2dlSanyBLMqsCA3A6zFPDPiHhI0ovAPyUdAXwI7F9gHszMrJEUFmAiYgTQs5rxk4GdikrX\nzMyaBn+T38zMCuEAY2ZmhVjse1M2swU3/d3Nap1nWD3WV5fvEY37eBwnHXYSdz11V+W4ay+5ljU7\nLMuWW/bgtNMu4ZtvvmHmzFnsu+8u/OY3R7HWWrvw6qv3seKKKzB+/HhWW201nnvuObbZZhsAOnXq\nxNtvv02bNm049NBDGTZsGB07duSuu+6ia9euQPoy5J/+9Ce+/XYmv/51fw46aE8ARo8eS79+v+Wz\nz76gZ88NuPbas1l66VZ8/vnn9O/fn/fff5/WrVtz4403stFGc7+JMWfOHHr37k3nzp0ru/9/7bXX\nOPbYY5k+fTpdurTn+uvPYYUV2jBr1myOP/5PvPba28yePYcDDtiDU07pl/fZ9owfP55ll10WgMcf\nf5yVV165Hnu98bgEY2bNxrHHnsnf/nYGL7zwD4YMuZN9990ZSWy22fd46aXXARg8eDA9e/Zk8ODB\nALzzzjt07NiRjh07csMNN7Diiivy3nvvcdJJJ3HaaamXqs8++4yzzjqLIUOG8PTTN3Phhdfx+edT\nAfjjH6/guOMO5LXX7qd9+xW45Zb04ut5551Hjx49GDFiBLfccgsnnnjiPHm97LLL+O53vzvPuCOP\nPJILLriA119/nR//eAcuu+xWAO6//z/MnPkN//3vnTz77K3cdNP9fPjhuMrlbr/9doYPH87w4cOb\nTXABBxgza0Zq6q5/iy02ZsiQEUAKMCeddFJlty6DBw9m6623BuDBBx/ksMPStyT69OnDk08+SUTw\n2GOPscsuu9ChQwdWXHEFdthhC/7znxeJCJ555mX23ntHAA444Ec89FD6zZk333yTHXdM4zfYYANG\njx7Np59+CqQ+zx5++GGOPPLIefL/7rvvsu222wKwww6bM3Dg00DqWuarr75m9uzZfP31DFq1akXb\ntssXsAcblgOMmTUbNXXXv+WWG/PSSynAvPTSS+yzzz6VnVwOHjyY738/dYlf2n1/y5YtadeuHZMn\nT56vW//VVluZ8eMn8NlnX9CuXdvKbvc7d07jATbZZBPuu+++yjQ//PBDxowZA8CvfvUrLrroIpZa\nat5bbPfu3St/j+aBB55k7NgUkPbeeyeWW25Z1l13d7p3/zG//OVBdOjQrnK5ww47jB49enDOOefU\nqafmpsIBxsyalNq66x806BZ23HEL7r77Ufbd95cAbLppd0aMeIcvv/yaWbNm0aZNG7p168Z77703\nTwlmUTr99NOZMmUKPXr04PLLL6dnz560aNGi8iePe/XqNd8yN954I1dddRW9evVi2rSvaNWqFQDD\nho2kRYulePfdf/P66w9y+eW388EHKVjdfvvtjBw5kueee47nnnuOW2+9dZFvS1EcYMysSamxu/4O\nqVv7bt26cOSRffjXv67ijTdGMXnyFJZbrjXduq3OrbcOrPyFyS233JJHHnmECRMmsP766wPzdt8/\ne/ZsvvjiCzp27Dhft/7jxk1g1VVXpkOHdnzxxTRmz54NwNixaTzACiuswE033cTw4cO55ZZbmDhx\nIt26deOFF15g4MCBdO3alb59+/LUU09x8MEHA6kq7fHHH2fYsGH06bMra62VuvD/5z8fZeedv0+r\nVi3p1KkDW265Ca+++lZlngHatm3LgQceWG3/Zk2VA0wDGjRItf6ZLekWtLv+LbbYmKuuuoOtttoK\ngK222orLLruMLbfcsrJU9JOf/IQBA9Kvhdxzzz3suOOOSGK33Xbj8ccf5/PPP+fzz6fy1FND2Gmn\ntNy22/bmgQdSP2N33PEwP/pRakOZMmUK33zzDZA6sNx2221ZYYUVOP/88xkzZgyjR4/mzjvvZMcd\nd+S2224DYMKEVL327bff8uc/38gRR+wHwOqrf4dnn03b++WXX/Pyy2+w3npdmT17NpMmTQLSTwQ8\n9NBD87yp1tT5NWUzq1Gb9ZpHd/0AW265CVdffWdlgNl0000ZM2bMPA3tRxxxBIcccgjrrLMOHTp0\n4M477wSgQ4cO/P73v2ezzTbj229nctppR1S2gZx11vH06/dbzjnnajbZZH0OPTT9rstbb73FYYcd\nhiS6d+/ODTfcUOt23XHHHVx55ZUA7LnnVhx88I8BOOqon/KLX5zN5pvvTwQcfPCP2Wijdfnyy6/5\n0Y92Y9asWcyZM4edd96Zo446ql77sjE5wJhZk1Pf7voB9tlnZ/bZZ2fatl0HSL9MOXPmzHnmad26\ndeXvvlTVv39/+vfvP183+mut1YVBgwbMN/9WW23Fu+++W3Y7tt9+e7bffvvK4RNPPLHydebSdNq0\nWY5bbrlgvuWXX35Zhg2rzzeNmhZXkZmZWSEcYMzMrBAOMGY2j+b0PQtLmuoxcxvMIrKofvPdrDG1\nbt2ayZMn07Fjxxq/j9KcDR03/88UV1WXlxaakohg8uTJtG7durGzMh8HGDOr1KVLF8aMGcPEiRMB\nmDRlUq3LtKjDfa1167fKTl9U6dSWVkOlUxczZtSel7qm07p1a7p06bJQ+SmCA4yZVWrVqhVrrbVW\n5fCGZ21Y6zJ1KZn37Fm+CmdRpVNbWg2VTl0MGlR7XhZFOo3JbTBmZlYIBxgzMyuEA4yZmRXCAcbM\nzArhAGNmZoVwgDEzs0I4wJiZWSEcYMzMrBCFBRhJq0t6WtKbkkZKOjGPP1PSWEnD898eReXBzMwa\nT5Hf5J8Rsmg9AAAUoklEQVQNnBIRr0hqCwyT9ESedmlEXFxg2mZm1sgKCzARMR4Ynz9Pk/QW0Lmo\n9MzMrGlpkL7IJHUFegJDgK2BEyQdCgwllXI+r2aZo4GjAdZYY42GyKaZ2SLh3tWTwhv5JbUB7gV+\nFRFTgauBbkAPUgnnkuqWi4hrI6J3RPTu1KlT0dk0M7NFrNAAI6kVKbjcHhH3AUTEpxExJyK+Ba4D\nNi8yD2Zm1jiKfItMwA3AWxHxl5Lxq5bMtg/wRlF5MDOzxlNkG8zWwCHA65KG53FnAAdI6gEEMBo4\npsA8mJlZIynyLbLngepauh4pKk0zM2s6/E1+MzMrhAOMmZkVwgHGzMwK4QBjZmaFcIAxM7NCOMCY\nmVkhHGDMzKwQDjBmZlYIBxgzMyuEA4yZmRXCAcbMzArhAGNmZoVwgDEzs0I4wJiZWSEcYMzMrBAO\nMGZmVggHGDMzK4QDjJmZFcIBxszMCuEAY2ZmhXCAMTOzQjjAmJlZIRxgzMysEA4wZmZWiBoDjKQf\nS1qzZPgPkl6TNFDSWrWtWNLqkp6W9KakkZJOzOM7SHpC0qj8f8VFsylmZtaUlCvBnAtMBJC0J3Aw\n0B8YCFxTh3XPBk6JiA2BLYHjJG0InA48GRHrAk/mYTMzW8yUCzAREV/lz/sCN0TEsIi4HuhU24oj\nYnxEvJI/TwPeAjoDewED8mwDgL0XNPNmZtZ0lQswktRG0lLATqTSRoXW9UlEUlegJzAEWCUixudJ\nnwCr1LDM0ZKGSho6ceLE+iRnZmZNQLkA81dgODAUeCsihgJI6gmML7PcPCS1Ae4FfhURU0unRUQA\nUd1yEXFtRPSOiN6dOtVaYDIzsyamZZlpA4DHgJWB10rGfwL0q8vKJbUiBZfbI+K+PPpTSatGxHhJ\nqwIT6p9tMzNr6sqVYIYBa0TEqxHxbcXI3LbyUW0rliTgBlLp5y8lkwYCh+XPhwEP1j/bZmbW1JUL\nMMcAl0m6bgFfJd4aOATYUdLw/LcHcAGwi6RRwM552MzMFjM1VpFFxBBJWwDHAkMl/RsoLcn8styK\nI+J5QDVM3mkB8mpmZs1IuTYYgA7AZqTvwwyjJMCYmZmVU2OAkXQs8Gvgz8AR+Y0vMzOzOilXgtkG\n2Coi/JaXmZnVW42N/BFxcNXgImltSb+XNLL4rJmZWXNWa2/KklaTdJKkl4GReZm+hefMzMyatXK9\nKR8t6WlgENAROAIYHxFnRcTrDZQ/MzNrpsq1wVwBvAgcWNJNjBv6zcysTsoFmFWBnwKXSPoO8E+g\nVYPkyszMmr1yjfyTI+KaiNiO9I37KaR+xN6SdF6D5dDMzJqlOv1kckR8HBGXRERv0u+5zCg2W2Zm\n1tyV/Sa/pI7AgcAGedRbwB0RcXbRGTMzs+at3Ftk3wXeAHoB7wKjSN3GvC5p/YbJnpmZNVflSjDn\nACdGxD9LR0raDzgP2K/IjJmZWfNWrg3me1WDC0BE3AtsVFyWzMxscVAuwHy5gNPMzMzKVpGtLOnk\nasYL6FRQfszMbDFRLsBcB7StYdr1BeTFzMwWI+V+0fKshsyImZktXur0RUszM7P6coAxM7NCOMCY\nmVkhamyDkTQZGAK8AAwGhkTEVw2VMTMza97KlWDWAv5K6qL/N8DHkoZKukzS/g2SOzMza7bKvUU2\nFXg8/yFpeaAf8CvgeNLvw5iZmVWrXGeXq0nqI+kvkp4DHgXWAX4HdKttxZJulDRB0hsl486UNFbS\n8Py3x6LYCDMza3rKfdFyDPAKcClwekR8U89130z62eVbqoy/NCIurue6zMysmSkXYLYGtgL2AU6W\nNBp4Mf8NjYiZ5VYcEc9K6rposmlmZs1NuZ9MfjEi/hIRfSKiF3AKMBMYAHyxEGmeIGlErkJbcSHW\nY2ZmTVjZ78FI2kBSf0nXA/8GzgBeJ7XDLIirSe03PYDxwCVl0j46v7U2dOLEiQuYnJmZNZZy34OZ\nBIwjVYk9C1wQEe8tTGIR8WnJ+q8DHioz77XAtQC9e/eOhUnXzMwaXrk2mLUjYmGqwuYjadWIGJ8H\n9yH9JLOZmS2GyrXBfCFpd0nPSpqU/56p66vFku4glX7WlzRG0hHARZJelzQC2AE4aZFshZmZNTnl\nqsiOAo4B/g8Ymkf3Bi6Q1CVXYdUoIg6oZvQNC5pRMzNrXspVkZ0EbBMRn5WMe0rS7sDz5PYRMzOz\n6pR7i0xVggsAETG5wPyYmdliolyAmSppk6oj87hpxWXJzMwWB+WqyE4BBkq6CRiWx/UGDgMOLjpj\nZmbWvJV7i+x5YIs8z+H5bylgyzzNzMysRuVKMETEJ8AfGigvZma2GCn3mvLTQE3foI+I2KmYLJmZ\n2eKgXAnm1GrGbUn6XsyEYrJjZmaLi3K/aFnRsI+k7YDfA62BYyPi3w2QNzMza8bKtsFI2o3Uc/JM\n4NyIeLpBcmVmZs1euTaYl4FOwJ9JfYohadOK6RHxSuG5MzOzZqtcCeZLYDrQB9gPUMm0AHYsMF9m\nZtbMlWuD2b4B82FmZouZGr9oKWkzSd8pGT5U0oOS/iapQ8Nkz8zMmqtyfZH9HfgGQNK2wAXALcAX\nuCdlMzOrRbk2mBYlvSn/DLg2Iu4F7pU0vPismZlZc1auBNNCUkUA2gl4qmRa2debzczMygWKO4Bn\nJE0CvgaeA5C0DqmazMzMrEbl3iI7V9KTwKrA4xFR0S/ZUsAJDZE5MzNrvmrrTfm/1Yx7t7jsmJnZ\n4qJcG4yZmdkCc4AxM7NCOMCYmVkhHGDMzKwQDjBmZlaIwgKMpBslTZD0Rsm4DpKekDQq/1+xqPTN\nzKxxFVmCuRn4YZVxpwNPRsS6wJN52MzMFkOFBZiIeBb4rMrovYAB+fMAYO+i0jczs8bV0G0wq0TE\n+Pz5E2CVmmaUdLSkoZKGTpw4sWFyZ2Zmi0yjNfLnrmeizPRrI6J3RPTu1KlTA+bMzMwWhYYOMJ9K\nWhUg/5/QwOmbmVkDaegAMxA4LH8+DHiwgdM3M7MGUuRryncALwLrSxoj6QjSr2LuImkUsHMeNjOz\nxVBhPxwWEQfUMGmnotI0M7Omw9/kNzOzQjjAmJlZIRxgzMysEA4wZmZWCAcYMzMrhAOMmZkVwgHG\nzMwK4QBjZmaFcIAxM7NCOMCYmVkhHGDMzKwQDjBmZlYIBxgzMyuEA4yZmRXCAcbMzArhAGNmZoVw\ngDEzs0I4wJiZWSEcYMzMrBAOMGZmVggHGDMzK4QDjJmZFcIBxszMCuEAY2ZmhWjZGIlKGg1MA+YA\nsyOid2Pkw8zMitMoASbbISImNWL6ZmZWIFeRmZlZIRorwATwH0nDJB1d3QySjpY0VNLQiRMnNnD2\nzMxsYTVWgNkmInoAuwPHSdq26gwRcW1E9I6I3p06dWr4HJqZ2UJplAATEWPz/wnA/cDmjZEPMzMr\nToMHGEnLS2pb8RnYFXijofNhZmbFaoy3yFYB7pdUkf4/IuLRRsiHmZkVqMEDTET8D9ikodM1M7OG\n5deUzcysEA4wZmZWCAcYMzMrhAOMmZkVwgHGzMwK4QBjZmaFcIAxM7NCOMCYmVkhHGDMzKwQDjBm\nZlYIBxgzMyuEA4yZmRXCAcbMzArhAGNmZoVwgDEzs0I4wJiZWSEcYMzMrBAOMGZmVggHGDMzK4QD\njJmZFcIBxszMCuEAY2ZmhXCAMTOzQjjAmJlZIRxgzMysEI0SYCT9UNI7kt6TdHpj5MHMzIrV4AFG\nUgvgSmB3YEPgAEkbNnQ+zMysWI1RgtkceC8i/hcR3wB3Ans1Qj7MzKxAioiGTVDqA/wwIo7Mw4cA\nW0TE8VXmOxo4Og+uD7zToBmFlYBJTsfpOB2n04zTWTMiOi3C9dVLy8ZKuDYRcS1wbWOlL2loRPR2\nOk7H6TidxS2dhtIYVWRjgdVLhrvkcWZmthhpjADzMrCupLUkLQ30BQY2Qj7MzKxADV5FFhGzJR0P\nPAa0AG6MiJENnY86aKjqOafjdJyO02nodBpEgzfym5nZksHf5Dczs0I4wJiZWSGW+AAjaXrB658j\naXjJX9cy824v6aEFSCMk3VYy3FLSxAVZVx3T2zunuUFB62/Q7clpFHoe1CctSYMkLdCrqkUfm5J0\nfitppKQR+bzeosC0ukh6UNIoSe9Luiy/IFTT/L+StFw91h+SLikZPlXSmQuZ7erSqbgXjJT0mqRT\nJC3W9+DFeuOaiK8jokfJ3+gC0vgS2EjSsnl4F+r56rek+rzwcQDwfP5fnzRa1HHWhd6eJdgCHZv6\nkLQVsCewaURsDOwMfFxQWgLuAx6IiHWB9YA2wLllFvsVUOcAA8wE9pW00gJntG4q7gXdSef07sAf\nC06zUTnAAJLaSHpS0iuSXpe0Vx7fVdJbkq7LTx2Pl9z0Fia9FpL+LOnl/AR4TMnkFSQ9nDsDvaYe\nTziPAD/Knw8A7ihJb3NJL0p6VdJgSevn8YdLGijpKeDJOua9DbANcATpFfOKktez1eVb0nRJl0h6\nDdiqjtuyoNvzrKQeJfM9L2mTuiZYtQQp6QpJh+fPoyWdVXKOLFQJoVxaC7HOmo5NTdu0h6S3JQ2T\n9Ld6lBBXBSZFxEyAiJgUEeMk9ZL0TF7fY5JWzekMyqWO4ZLekLR5PTZrR2BGRNyU05oDnAT0l7S8\npIvzOkdIOkHSL4HVgKclPV3HNGaT3t46qeqEfA94Kq//SUlrSGon6cOSc3x5SR9LalXXjYqICaSe\nSo5XUuM9QdJp+Zx7TdIFdU2jKXCASWYA+0TEpsAOwCX5yQlgXeDK/NQxBdivnuteVnOrx+7P444A\nvoiIzYDNgKMkrZWnbQ6cQOoIdG1g3zqmcyfQV1JrYGNgSMm0t4EfRERP4A/AeSXTNgX6RMR2dUxn\nL+DRiHgXmCypVy35Xh4YEhGbRMTzdUxjQbfnBuBwAEnrAa0j4rV6pFmbSfkcuRo4dRGud1Gp6djM\nJ+/XvwO7R0QvoD7diTwOrC7pXUlXSdou31wvJ51LvYAbmbeUsVxE9AB+kafVVXdgWOmIiJgKfAQc\nCXQFeuSS1O0R8TdgHLBDROxQj3SuBA6S1K7K+MuBARXrB/4WEV8Aw4GKa2ZP4LGImFWP9IiI/5G+\nqrEyNdwTJO1OOq5bRMQmwEX1SaOxOcAkAs6TNAL4D9AZWCVP+yAihufPw0gndH2UVpHtk8ftChwq\naTjpxtmRFMgAXsodgc4hPbVvU5dEImJEztsBpKf/Uu2AuyW9AVxKumgrPBERn9Vjew4g3fzJ/yuq\nYmrK9xzg3nqsH1jg7bkb2DPf7PoDN9c33Vrcl/8vyHnQEGo6NtXZAPhfRHyQh+8oM+88ImI60Iv0\nBD4RuAs4BtgIeCKf178j9dJR4Y687LOkUnr7uqZXxvbA3yNidl53fc7jeeSgdQvwyyqTtgL+kT/f\nytzz+i7gZ/lz3zy8MGq6J+wM3BQRX+V8LvA2NoYm2xdZAzuI9ATXKyJmSRoNtM7TZpbMNwdY6Coy\nUkA7ISIem2ektD1Q9YtJ9fmi0kDgYtKF17Fk/DnA0xGxj9JLBoNKpn1Z15VL6kCqsviepCA9fQXw\ncJl8z8hBZ0HUa3si4itJT5Ce+PYn3QTrYzbzPnS1rjK94lyYw8JfO7WlVS9ljs2DizKdCvmYDgIG\nSXodOA4YGRE1VYMu6Hn9JtCndISkFYA1gNF1zW8d/RV4BbipDvMOJD2UdiCdZ0/VNzFJ3Ujn0gRq\nvifsVt/1NiUuwSTtgAk5uOwArFlweo8BP6+os5W0nqTl87TNc9F4KdITUn2qlW4EzoqI16uMb8fc\nRvLDFzzb9AFujYg1I6JrRKwOfAD8YCHzXZMF2Z7rgb8BL0fE5/VM70NgQ0nL5Cfsneq5fGOmVdOx\nWaqGdN4BumnuW40/q7rCmkhaX9K6JaN6AG8BnZReAEBSK0mlJeWf5fHbkKqCvqhjck8Cy0k6NC/f\nAriEVDp9DDhG+QWVfLMHmAa0rev2VMilg3+SqqsqDCa3Z5EeRJ/L804ndXt1GfBQfR+iJHUCrgGu\niPRt95ruCU8A/ZTfiivZxmZhiQ4w+cScSapb7Z2fxA4l1fEX6XrSk9kruZrn78x9In4ZuIJ0wX4A\n3F/tGqoREWNyHXRVFwHnS3qVhXvyPqCa/Nybxy9wvmuyINsTEcOAqdTtKRSYex5ExMekG8wb+f+r\nC5r3RkirpmPTt7p0IuJrUnvIo5KGkW7Kdb3ptwEGSHozVytvSGoL6wNcqPRCx3Dg+yXLzMjH6xrm\nvYGXlW+++wA/lTQKeJfUZnoG6Tr6CBiR0zwwL3Zt3q66NvKXuoTUZX6FE0g3+BHAIcCJJdPuAg6m\n7tVjFe2xI0lV8Y8DZ+Vp1d4TIuJRUmlpaK4+a4ptfzVaoruKUXrD6LqIqM9bLVZFrto7NSL2bAJ5\nWY1UdbNBRHxbx2Ua7DxoSuecpDYRMT2/0HIlMCoiLi0gnUGk82Pool63NW1LbAlG0rGkhsffNXZe\nbNHI1ShDgN/WI7g02HnQBM+5o/JT8UhStePfGzk/tphZokswZmZWnCW2BGNmZsVygDEzs0I4wJiZ\nWSEcYGyJpTr02qzUO/EIpT7pXpe0d8m0myWNlbRMHl5Jqc+y75V0D/SZpA/y5/8o9W31RpV8nCmp\nWb1+alYX/ia/Lckqe23O3wuZp9fm/ErxxcAuEfGBUn9xT0j6X+7KBtI3sfuT+icDIH8xtEdex82k\nL+Ldk4e7Fr1RZk2FSzC2pKux12bSl9rOq+ivK/8/H/h1yTx/BU5S/X7uwGyJ4ABjS7pyvTbP15Mv\nMJR5Owv9iNQtziH1SHPtkiq04cCx9c+2WdPnpy5bokXEiFxtVV2vzXV1PqlTyYfrOP/7uet6ILXB\nLGC6Zk2aSzBmc3ttrtpl/ZvM3yNzL9I33ytFxChS31v7F5VBs+bIJRiz1GvzlIh4PferVuFi0u/O\nPBURo3NJ5wyqdB+fnUvdSzBmSwQHGFviRcQYUhf/VccPl3Qa8K/cjfos4P9KfoCudN6Rkl4h/UKo\nmeG+yMzMrCBugzEzs0I4wJiZWSEcYMzMrBAOMGZmVggHGDMzK4QDjJmZFcIBxszMCvH/j2L+op3m\nNYsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2902f6acd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "N=12\n",
    "ind=np.arange(N)\n",
    "width = 0.35\n",
    "pUS10chey021 = plt.bar(ind, US10chey021_snowdays_y, width, color='g', label='US10chey021')\n",
    "pUSW00094985 = plt.bar(ind+width, USW00094985_snowdays_y, width, color='y', label='USW00094985')\n",
    "\n",
    "plt.ylabel('SNOW DAYS')\n",
    "plt.xlabel('MONTH')\n",
    "plt.title('Snow Days in 2015 at Stations US10chey021 vs. USW00094985')\n",
    "plt.xticks(ind+width, ('Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'))\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Determine the number of snow days at each US weather station\n",
    "To determine how many snow days there were at each of the US weather stations in 2015, run the following command to query the `snow2015` table, using the `COUNT(*)` function to get the total snow days. The table is sorted by station name and limited to only 100 stations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "snowStations = sqlContext.sql(\"SELECT  station, COUNT(*) AS snowdays FROM snow2015 WHERE station LIKE 'US%' GROUP BY station ORDER BY station LIMIT 100\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Now print the first 5 rows of the `snowStations` table, including the station name and number of snow days at that station:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Row(station=u'US10adam001', snowdays=79),\n",
       " Row(station=u'US10adam002', snowdays=192),\n",
       " Row(station=u'US10adam004', snowdays=353),\n",
       " Row(station=u'US10adam006', snowdays=58),\n",
       " Row(station=u'US10adam007', snowdays=36)]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "snowStations.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Save the query results into a new table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In this step, you will save the query result of the above `SELECT` query in a new table called `snowdays_2015`. The new table has two columns: STATION (name of station) and SNOWDAYS (number of snow days at the station). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "sqlContext.registerDataFrameAsTable(snowStations, \"snowdays_2015\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Output the five stations with the highest number of snow days in 2015:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Row(station=u'US10adam056', snowdays=361)\n",
      "Row(station=u'US10buff036', snowdays=360)\n",
      "Row(station=u'US10adam051', snowdays=355)\n",
      "Row(station=u'US10adam004', snowdays=353)\n",
      "Row(station=u'US10adam032', snowdays=352)\n"
     ]
    }
   ],
   "source": [
    "snowStations_new = sqlContext.sql(\"SELECT station, snowdays FROM snowdays_2015 ORDER BY snowdays DESC LIMIT 5\").collect()\n",
    "for row in snowStations_new:\n",
    "    print row"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Identify the stations with the same number of snow days in 2015\n",
    "\n",
    "In this section, you'll identify the stations that had the same number of snow days in 2015. First, you need to query the table `snowdays_2015` again. \n",
    "\n",
    "Then, by using the `map` function, you'll transform each row into a key-value pair where the key is the number of snow days and the value is the station name.\n",
    "\n",
    "Next, you'll apply the `reduceByKey` function to each pair, and the value of those pairs that have the same key are concatenated. As a result, the RDD contains the number of snow days, and the list of all stations that have the same number of snow days.\n",
    "\n",
    "The `for` loop prints the number of snow days, and the list of stations that have that number of snow days in 2015."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Snow days:128 Stations:US10butl008\n",
      "Snow days:64 Stations:US10chas025,US10cher016\n",
      "Snow days:3 Stations:US10cher019\n",
      "Snow days:135 Stations:US10chas034\n",
      "Snow days:8 Stations:US10chas009\n",
      "Snow days:300 Stations:US10buff008\n",
      "Snow days:141 Stations:US10cass011\n",
      "Snow days:15 Stations:US10bann007\n",
      "Snow days:17 Stations:US10chas027\n",
      "Snow days:147 Stations:US10ceda006\n",
      "Snow days:90 Stations:US10boon010\n",
      "Snow days:69 Stations:US10buff007,US10chas023\n",
      "Snow days:26 Stations:US10chas032\n",
      "Snow days:28 Stations:US10adam057\n",
      "Snow days:29 Stations:US10adam013\n",
      "Snow days:30 Stations:US10box_007\n",
      "Snow days:31 Stations:US10box_020,US10buff039,US10chas021,US10chas035\n",
      "Snow days:32 Stations:US10chas019\n",
      "Snow days:161 Stations:US10buff004,US10chas001\n",
      "Snow days:34 Stations:US10chas013\n",
      "Snow days:36 Stations:US10adam007,US10box_009\n",
      "Snow days:37 Stations:US10chas033,US10cher013\n",
      "Snow days:66 Stations:US10cher018\n",
      "Snow days:40 Stations:US10adam012\n",
      "Snow days:41 Stations:US10adam017,US10adam024,US10ceda015\n",
      "Snow days:42 Stations:US10buff025\n",
      "Snow days:43 Stations:US10ceda026\n",
      "Snow days:44 Stations:US10ceda001\n",
      "Snow days:45 Stations:US10cher009\n",
      "Snow days:49 Stations:US10adam035\n",
      "Snow days:306 Stations:US10butl011\n",
      "Snow days:115 Stations:US10box_023\n",
      "Snow days:52 Stations:US10box_008\n",
      "Snow days:54 Stations:US10adam046,US10cher005\n",
      "Snow days:56 Stations:US10bann009\n",
      "Snow days:180 Stations:US10box_024\n",
      "Snow days:58 Stations:US10adam006\n",
      "Snow days:287 Stations:US10buff006\n",
      "Snow days:60 Stations:US10brow008\n",
      "Snow days:61 Stations:US10boon002\n",
      "Snow days:63 Stations:US10chas029\n",
      "Snow days:192 Stations:US10adam002\n",
      "Snow days:83 Stations:US10adam034,US10ceda017\n",
      "Snow days:71 Stations:US10ante003\n",
      "Snow days:324 Stations:US10boon005\n",
      "Snow days:325 Stations:US10ceda013\n",
      "Snow days:33 Stations:US10bann005\n",
      "Snow days:328 Stations:US10chas002\n",
      "Snow days:73 Stations:US10adam010\n",
      "Snow days:75 Stations:US10adam008\n",
      "Snow days:333 Stations:US10adam022\n",
      "Snow days:79 Stations:US10adam001,US10blai004\n",
      "Snow days:81 Stations:US10blai012,US10burt002,US10chas011\n",
      "Snow days:82 Stations:US10chas016\n",
      "Snow days:339 Stations:US10ceda002\n",
      "Snow days:84 Stations:US10box_004\n",
      "Snow days:85 Stations:US10adam036\n",
      "Snow days:343 Stations:US10cass001\n",
      "Snow days:88 Stations:US10chas012\n",
      "Snow days:89 Stations:US10butl005\n",
      "Snow days:346 Stations:US10ceda005\n",
      "Snow days:335 Stations:US10box_013,US10buff002\n",
      "Snow days:314 Stations:US10box_001\n",
      "Snow days:222 Stations:US10box_014\n",
      "Snow days:182 Stations:US10cass005\n",
      "Snow days:352 Stations:US10adam032\n",
      "Snow days:353 Stations:US10adam004\n",
      "Snow days:50 Stations:US10adam029,US10chas017\n",
      "Snow days:101 Stations:US10buff034,US10ceda009\n",
      "Snow days:360 Stations:US10buff036\n",
      "Snow days:194 Stations:US10chas003\n",
      "Snow days:111 Stations:US10butl020\n",
      "Snow days:229 Stations:US10adam019\n",
      "Snow days:59 Stations:US10adam045\n",
      "Snow days:19 Stations:US10cher004\n",
      "Snow days:355 Stations:US10adam051\n",
      "Snow days:361 Stations:US10adam056\n",
      "Snow days:72 Stations:US10ceda020\n",
      "Snow days:212 Stations:US10box_030,US10cass010,US10chas036\n",
      "Snow days:351 Stations:US10chas028\n"
     ]
    }
   ],
   "source": [
    "snowdays_100 = sqlContext.sql(\"SELECT station, snowdays FROM snowdays_2015\")\n",
    "snowday_stations=snowdays_100.rdd.map(lambda x:  (x.snowdays,x.station)).reduceByKey(lambda x, y: x + ',' + y)\n",
    "for snowday in snowday_stations.collect():\n",
    "    print 'Snow days:' + str(snowday[0]) + ' Stations:' + str(snowday[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id=\"save\"></a>\n",
    "## Save results in Filesystem\n",
    "\n",
    "In this section, you'll save the `snowStations` DataFrame, which is the query result of the `snow2015` table for 100 US weather stations in the filesystem. Each row contains the name of the weather station and the number of snow days at that station.\n",
    "\n",
    "The data will be saved in CSV file format. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "from os import listdir\n",
    "\n",
    "# Save as csv file, requires spark 2.0+\n",
    "if 'snowStations.csv' not in listdir('../datasets'):\n",
    "    snowStations.write.csv(\"../datasets/snowStations.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "After the DataFrame has been saved to the filesystem, you will see a new data source called `snowStations.csv` in the `Data Sets` pane. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id=\"summary\"></a>\n",
    "## Summary\n",
    "\n",
    "In this sample, you learned how to use Apache Spark, how to use PySpark, the Python API for Spark and the matplotlib plotting library. You also learned how to load data from Object Storage as an RDD, how to define Spark data schemas, and how to save intermediate results in parquet format."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "<a id=\"resources\"></a>\n",
    "### Resources\n",
    "[Apache Spark 2.0.2 Programming Guide](https://spark.apache.org/docs/2.0.2/programming-guide.html)\n",
    "\n",
    "[Apache Spark 2.0.2 SQL and DataFrames](https://spark.apache.org/docs/2.0.2/sql-programming-guide.html)\n",
    "\n",
    "[PySpark 2.0.2 - Python API for Spark](https://spark.apache.org/docs/2.0.2/api/python/)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Author\n",
    "Sven Hafeneger is a member of the Data Science Experience development team at IBM Analytics in Germany. He holds a M.Sc. in Bioinformatics and is passionate about data analysis, machine learning and the Python ecosystem for data science. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "<div class=\"alert alert-block alert-info\"> Note: To save resources and get the best performance please use the code below to stop the kernel before exiting your notebook.</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%javascript\n",
    "Jupyter.notebook.session.delete();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "<hr>\n",
    "Copyright &copy; IBM Corp. 2017. Released as licensed Sample Materials."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  },
  "name": "cds_ax_spark.ipynb"
 },
 "nbformat": 4,
 "nbformat_minor": 1
}