Ch02.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Women in Data Science - ATX Meetup\n",
    "# Ch. 2: A Crash Course in Python\n",
    "Notes on \"Data Science from Scratch\" by Joel Grus\n",
    "\n",
    "- Download the Jupyter notebook for these slides [here](https://github.com/mepa/datasci-from-scratch-notes/blob/master/Ch02.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The Basics\n",
    "- Getting Python\n",
    "- The Zen of Python\n",
    "- Whitespace Formatting\n",
    "- Modules\n",
    "- Arithmetic\n",
    "- Functions\n",
    "- Strings\n",
    "- Exceptions\n",
    "- Lists\n",
    "- Tuples\n",
    "- Dictionaries\n",
    "    - <code>defaultdict</code>\n",
    "    - <code>Counter</code>\n",
    "- Sets\n",
    "- Control Flow\n",
    "- Truthiness"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Getting Python\n",
    "- We recommend installing [Anaconda](https://store.continuum.io/cshop/anaconda/)\n",
    "- Otherwise, you can install the following to get started:\n",
    "    - [Python](https://www.python.org/)\n",
    "    - Python package manager: [pip](https://pypi.python.org/pypi/pip)\n",
    "    - Nicer Python shell: [IPython](http://ipython.org/)\n",
    "    - Web app for interactive data sci: [Jupyter Notebook](http://jupyter.org/)\n",
    "        - <code>pip install jupyter</code> (Python 2)\n",
    "        - <code>pip3 install jupyter</code> (Python 3)\n",
    "    - etc..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### The Zen of Python\n",
    "- Python design principles: [The Zen of Python (PEP 20)](https://www.python.org/dev/peps/pep-0020/)\n",
    "- Python code readability: [Style Guide for Python Code (PEP 8)](https://www.python.org/dev/peps/pep-0008/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Whitespace Formatting\n",
    "In Python, whitespace is important. Other languages, like C++, use curly braces {} to delimit blocks of code.\n",
    "<code> \n",
    "for( int i = 0; i < 5; i++ )\n",
    "   {\n",
    "       cout << i << endl;\n",
    "   }</code>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "# Python uses indentation to delimit blocks of code.\n",
    "for i in range(5):\n",
    "    print i"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Modules\n",
    "- `import` modules (= libraries = packages) that aren't in the [Python 2.7 standard library](https://docs.python.org/2/library/)\n",
    "- Python has a *huge* variety and number of extra packages that can be installed, if not already, with `pip` or `conda`, then imported:\n",
    "    - [PyPI (Python Package Index)](https://pypi.python.org/pypi)\n",
    "    - [Anaconda 2.5.0 Package List](https://docs.continuum.io/anaconda/pkg-docs); additionally, check out these docs:\n",
    "        - Also, check out \"Packages available in Anaconda\" and \"Managing Packages in Anaconda\" from [here](https://docs.continuum.io/anaconda/index#packages-available-in-anaconda)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Let's try importing the [Matplotlib pyplot module](http://matplotlib.org/api/pyplot_api.html), which \"provides a MATLAB-like plotting framework.\"\n",
    "\n",
    "*Note*:\n",
    "- Sometimes you'll want to use [IPython magic commands](http://ipython.readthedocs.org/en/stable/interactive/magics.html) in your Jupyter notebooks.\n",
    "- Try not to do the following since you may inadvertently overwrite variables you've already defined: `from matplotlib.pyplot import *`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10e551e10>]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADs9JREFUeJzt3W+oXHedx/HPp3a3cSkb2BWiJLbFS0uNUGKFpqFCZgWx\nWYQ+SemDglmfNGwjCfhQFnIDPvGZzaNUsc12RUgo0lTbsghmEJXeleYG26QRiwVLIDcPapEYWZR+\nfXDPjSeTmXvOzJwz55zfeb9g6Pz53Zkfh9Nf3vndmYkjQgCANN3W9AQAAPVhkQeAhLHIA0DCWOQB\nIGEs8gCQMBZ5AEhY4SJv+w7bK7ZXbb9p++iEccdt/9b2edu7qp8qAGBatxcNiIj/t/1vEXHd9kck\n/cL2axHxfxtjbO+TtBQR99reLemEpIfrmzYAoIxS2zURcT27eofW/2AY/QTVY5JeyMauSNpqe1tV\nkwQAzKbUIm/7Nturkq5I+klE/GpkyHZJ7+VuX87uAwA0qGzJfxgRn5W0Q9Ju2zvrnRYAoAqFe/J5\nEfFH22clPSrpYu6hy5I+mbu9I7vvJrb5ohwAmEFEeJafK/Pumo/Z3ppd/6ikL0q6NDLsZUlfycY8\nLOmDiFibMFEuFV2OHj3a+BxSunA8OZZtuKythfbvD33606HXX1+/bx5ltms+Iems7fOSViT9b0S8\navug7aeyhftVSe/afkfSs5KenmtWANBDp09LDzwgLS1J585Ju3fP/5xl3kL5pqQHx9z/7Mjtr80/\nHQDon6tXpUOHpAsXpDNnqlncN/CJ1w4bDAZNTyEpHM/qcCzLq6Pe8zzvfs9UL2bHIl8PANoqX+/P\nP7/54m5bUdcvXgEA1aq73vOmegslAGB2de69T0LJA8ACLLLe8yh5AKhRE/WeR8kDQE2aqvc8Sh4A\nKtZ0vedR8gBQoTbUex4lDwAVaFO951HyADCnttV7HiUPADNqa73nUfIAMIM213seJQ8AU+hCvedR\n8gBQUlfqPY+SB4ACXav3PEoeADbRxXrPo+QBYIwu13seJQ8AI7pe73mUPABkUqn3PEoeAJRWvedR\n8gB6LcV6z6PkAfRWqvWeR8kD6J3U6z2PkgfQK32o9zxKHkAv9Kne8yh5AMnrW73nUfIAktXXes+j\n5AEkqc/1nkfJA0gK9X4zSh5AMqj3W1HyADqPep+MkgfQadT75ih5AJ20Ue9vvUW9b4aSB9A5G/X+\nqU9Jq6ss8Juh5AF0BvU+PUoeQCdQ77Oh5AG0GvU+H0oeQGtR7/Oj5AG0DvVeHUoeQKtQ79Wi5AG0\nAvVeD0oeQOOo9/pQ8gAaQ73Xj5IH0AjqfTEoeQALRb0vFiUPYGGo98UrLHnbOyS9IGmbpA8lfTci\njo+M2SvpjKTfZXf9MCK+WfFcAXQU9d6cMiX/V0lfj4jPSNoj6ZDt+8eM+1lEPJhdWOABSKLem1ZY\n8hFxRdKV7Po1229L2i7p0shQVz89AF1FvbfDVHvytu+RtEvSypiH99g+b/sV2zsrmBuAjqLe26P0\nu2ts3ynpRUlHIuLayMNvSLorIq7b3ifpJUn3jXue5eXlG9cHg4EGg8GUUwbQVtR7NYbDoYbDYSXP\n5YgoHmTfLunHkl6LiGdKjH9X0uci4v2R+6PM6wHontOnpcOHpQMHpGPHpC1bmp5ROmwrImbaEi9b\n8s9Jujhpgbe9LSLWsusPaf0Pj/fHjQWQFuq93Qr35G0/IulJSV+wvWr7nO1HbR+0/VQ2bL/tt2yv\nSvq2pCdqnDOAlmDvvf1KbddU9mJs1wBJyNf7yZMs7nWbZ7uGT7wCmAr13i18dw2AUth77yZKHkAh\n6r27KHkAE1Hv3UfJAxiLek8DJQ/gJtR7Wih5ADdQ7+mh5AFQ7wmj5IGeo97TRskDPUW99wMlD/QQ\n9d4flDzQI9R7/1DyQE9Q7/1EyQOJo977jZIHEka9g5IHEkS9YwMlDySGekceJQ8kgnrHOJQ8kADq\nHZNQ8kCHUe8oQskDHUW9owxKHugY6h3ToOSBDqHeMS1KHugA6h2zouSBlqPeMQ9KHmgp6h1VoOSB\nFqLeURVKHmgR6h1Vo+SBlqDeUQdKHmgY9Y46UfJAg6h31I2SBxpAvWNRKHlgwah3LBIlDywI9Y4m\nUPLAAlDvaAolD9SIekfTKHmgJtQ72oCSBypGvaNNKHmgQtQ72oaSBypAvaOtKHlgTtQ72oySB2ZE\nvaMLKHlgBtQ7uoKSB6ZAvaNrKHmgJOodXUTJAwWod3QZJQ9sgnpH1xWWvO0dkl6QtE3Sh5K+GxHH\nx4w7LmmfpD9J+o+IOF/xXIGFod6RijIl/1dJX4+Iz0jaI+mQ7fvzA2zvk7QUEfdKOijpROUzBRaE\nekdKCks+Iq5IupJdv2b7bUnbJV3KDXtM67WviFixvdX2tohYq2HOQC2od6Roqj152/dI2iVpZeSh\n7ZLey92+nN0HdAL1jlSVfneN7TslvSjpSERcm/UFl5eXb1wfDAYaDAazPhUwN+odbTQcDjUcDit5\nLkdE8SD7dkk/lvRaRDwz5vETks5GxKns9iVJe0e3a2xHmdcDFuH0aenwYenAAenYMWnLlqZnBIxn\nWxHhWX62bMk/J+niuAU+87KkQ5JO2X5Y0gfsx6OtqHf0SeGevO1HJD0p6Qu2V22fs/2o7YO2n5Kk\niHhV0ru235H0rKSna501MCP23tE3pbZrKnsxtmvQkHy9nzzJ4o5umWe7hk+8InnUO/qM765Bsth7\nByh5JIp6B9ZR8kgK9Q7cjJJHMqh34FaUPDqPegcmo+TRadQ7sDlKHp1EvQPlUPLoHOodKI+SR2dQ\n78D0KHl0AvUOzIaSR6tR78B8KHm0FvUOzI+SR+tQ70B1KHm0CvUOVIuSRytQ70A9KHk0jnoH6kPJ\nozHUO1A/Sh6NoN6BxaDksVDUO7BYlDwWhnoHFo+SR+2od6A5lDxqRb0DzaLkUQvqHWgHSh6Vo96B\n9qDkURnqHWgfSh6VoN6BdqLkMRfqHWg3Sh4zo96B9qPkMTXqHegOSh5Tod6BbqHkUQr1DnQTJY9C\n1DvQXZQ8JqLege6j5DEW9Q6kgZLHTah3IC2UPG6g3oH0UPKg3oGEUfI9R70DaaPke4p6B/qBku8h\n6h3oD0q+R6h3oH8o+Z6g3oF+ouQTR70D/UbJJ4x6B0DJJ4h6B7CBkk8M9Q4gr7DkbX9P0pclrUXE\nA2Me3yvpjKTfZXf9MCK+WeksUYh6BzBOmZJ/XtKXCsb8LCIezC4s8AtGvQOYpLDkI+Lntu8uGOaK\n5oMpUO8AilS1J7/H9nnbr9jeWdFzYoII6h1AOVW8u+YNSXdFxHXb+yS9JOm+SYOXl5dvXB8MBhoM\nBhVMoT+uXpWeflq6cIF6B1I1HA41HA4reS5HRPGg9e2aH437xeuYse9K+lxEvD/msSjzerjVRr0f\nOSIdOCAdOyZt2dL0rAAsgm1FxEzb4mVL3pqw7257W0SsZdcf0vofHLcs8Jgd9Q5gVoV78rZ/IOmX\nku6z/XvbX7V90PZT2ZD9tt+yvSrp25KeqHG+vRIhnTq1vve+tMTeO4DpldquqezF2K4pLV/vJ0+y\nuAN9Ns92DZ94bRnqHUCV+O6aFmHvHUDVKPkWoN4B1IWSbxj1DqBOlHxDqHcAi0DJN4B6B7AolPwC\nUe8AFo2SXxDqHUATKPmaUe8AmkTJ14h6B9A0Sr4G1DuAtqDkK0a9A2gTSr4i1DuANqLkK0C9A2gr\nSn4O1DuAtqPkZ0S9A+gCSn5K1DuALqHkp0C9A+gaSr4E6h1AV1HyBah3AF1GyU9AvQNIASU/BvUO\nIBWUfA71DiA1lHyGegeQot6XPPUOIGW9LnnqHUDqelny1DuAvuhdyVPvAPqkNyVPvQPoo16UPPUO\noK+SLnnqHUDfJVvy1DsAJFjy1DsA/F1SJU+9A8DNkih56h0Axut8yVPvADBZZ0ueegeAYp0seeod\nAMrpVMlT7wAwnc6UPPUOANNrfclT7wAwu1aXPPUOAPNpZclT7wBQjdaVPPUOANVpTclT7wBQvVaU\nPPUOAPUoLHnb37O9ZvvXm4w5bvu3ts/b3lX2xal3AKhXme2a5yV9adKDtvdJWoqIeyUdlHSizAtf\nvSo9/ri0vLxe79/6lrRlS5mfxIbhcNj0FJLC8awOx7I9Chf5iPi5pD9sMuQxSS9kY1ckbbW9bfLz\nUe9V4X+kanE8q8OxbI8q9uS3S3ovd/tydt/auMGPP87eOwAsysJ/8bq0JH3/+2zNAMAiOCKKB9l3\nS/pRRDww5rETks5GxKns9iVJeyPilpK3XfxiAIBbRIRn+bmyJe/sMs7Lkg5JOmX7YUkfjFvgpdkn\nCQCYTeEib/sHkgaS/tX27yUdlfSPkiIivhMRr9r+d9vvSPqTpK/WOWEAQHmltmsAAN1U+dca1Pnh\nqT4qOp6299r+wPa57PJfi55jV9jeYfunti/YftP24QnjOD9LKHM8OT/Ls32H7RXbq9nxPDph3HTn\nZ0RUepH0eUm7JP16wuP7JL2SXd8t6fWq55DSpcTx3Cvp5abn2YWLpI9L2pVdv1PSbyTdPzKG87Pa\n48n5Od0x/afsvx+R9Lqkh0Yen/r8rLzko+IPT/VdieMpTf6lOHIi4kpEnM+uX5P0ttY/05HH+VlS\nyeMpcX6WFhHXs6t3aP13pqP76VOfn018C+WkD09hdnuyv7q9Yntn05PpAtv3aP1vSCsjD3F+zmCT\n4ylxfpZm+zbbq5KuSPpJRPxqZMjU52crvoUSc3lD0l0RcT37HqGXJN3X8Jxazfadkl6UdCQrUMyh\n4Hhyfk4hIj6U9Fnb/yzpJds7I+LiPM/ZRMlflvTJ3O0d2X2YQURc2/grXkS8JukfbP9Lw9NqLdu3\na31B+p+IODNmCOfnFIqOJ+fnbCLij5LOSnp05KGpz8+6FvmiD099RZKKPjyFGyYez/x+nO2HtP62\n2PcXNbEOek7SxYh4ZsLjnJ/T2fR4cn6WZ/tjtrdm1z8q6YuSLo0Mm/r8rHy7hg9PVavoeErab/s/\nJf1F0p8lPdHUXNvO9iOSnpT0ZrbvGZK+IelucX5OrczxFOfnND4h6b9t36b1AD+VnY8HNcf5yYeh\nACBhrfk3XgEA1WORB4CEscgDQMJY5AEgYSzyAJAwFnkASBiLPAAkjEUeABL2Nyd8jIN/neVAAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c01df10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Tip: use IPython magic command \"%matplotlib inline\" to display plots in notebook\n",
    "%matplotlib inline \n",
    "import matplotlib.pyplot # import module and reference by its super long name\n",
    "\n",
    "matplotlib.pyplot.plot([1,2,3], [1,2,3]) # must type the whole name to access methods, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10eb03090>]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADs9JREFUeJzt3W+oXHedx/HPp3a3cSkb2BWiJLbFS0uNUGKFpqFCZgWx\nWYQ+SemDglmfNGwjCfhQFnIDPvGZzaNUsc12RUgo0lTbsghmEJXeleYG26QRiwVLIDcPapEYWZR+\nfXDPjSeTmXvOzJwz55zfeb9g6Pz53Zkfh9Nf3vndmYkjQgCANN3W9AQAAPVhkQeAhLHIA0DCWOQB\nIGEs8gCQMBZ5AEhY4SJv+w7bK7ZXbb9p++iEccdt/9b2edu7qp8qAGBatxcNiIj/t/1vEXHd9kck\n/cL2axHxfxtjbO+TtBQR99reLemEpIfrmzYAoIxS2zURcT27eofW/2AY/QTVY5JeyMauSNpqe1tV\nkwQAzKbUIm/7Nturkq5I+klE/GpkyHZJ7+VuX87uAwA0qGzJfxgRn5W0Q9Ju2zvrnRYAoAqFe/J5\nEfFH22clPSrpYu6hy5I+mbu9I7vvJrb5ohwAmEFEeJafK/Pumo/Z3ppd/6ikL0q6NDLsZUlfycY8\nLOmDiFibMFEuFV2OHj3a+BxSunA8OZZtuKythfbvD33606HXX1+/bx5ltms+Iems7fOSViT9b0S8\navug7aeyhftVSe/afkfSs5KenmtWANBDp09LDzwgLS1J585Ju3fP/5xl3kL5pqQHx9z/7Mjtr80/\nHQDon6tXpUOHpAsXpDNnqlncN/CJ1w4bDAZNTyEpHM/qcCzLq6Pe8zzvfs9UL2bHIl8PANoqX+/P\nP7/54m5bUdcvXgEA1aq73vOmegslAGB2de69T0LJA8ACLLLe8yh5AKhRE/WeR8kDQE2aqvc8Sh4A\nKtZ0vedR8gBQoTbUex4lDwAVaFO951HyADCnttV7HiUPADNqa73nUfIAMIM213seJQ8AU+hCvedR\n8gBQUlfqPY+SB4ACXav3PEoeADbRxXrPo+QBYIwu13seJQ8AI7pe73mUPABkUqn3PEoeAJRWvedR\n8gB6LcV6z6PkAfRWqvWeR8kD6J3U6z2PkgfQK32o9zxKHkAv9Kne8yh5AMnrW73nUfIAktXXes+j\n5AEkqc/1nkfJA0gK9X4zSh5AMqj3W1HyADqPep+MkgfQadT75ih5AJ20Ue9vvUW9b4aSB9A5G/X+\nqU9Jq6ss8Juh5AF0BvU+PUoeQCdQ77Oh5AG0GvU+H0oeQGtR7/Oj5AG0DvVeHUoeQKtQ79Wi5AG0\nAvVeD0oeQOOo9/pQ8gAaQ73Xj5IH0AjqfTEoeQALRb0vFiUPYGGo98UrLHnbOyS9IGmbpA8lfTci\njo+M2SvpjKTfZXf9MCK+WfFcAXQU9d6cMiX/V0lfj4jPSNoj6ZDt+8eM+1lEPJhdWOABSKLem1ZY\n8hFxRdKV7Po1229L2i7p0shQVz89AF1FvbfDVHvytu+RtEvSypiH99g+b/sV2zsrmBuAjqLe26P0\nu2ts3ynpRUlHIuLayMNvSLorIq7b3ifpJUn3jXue5eXlG9cHg4EGg8GUUwbQVtR7NYbDoYbDYSXP\n5YgoHmTfLunHkl6LiGdKjH9X0uci4v2R+6PM6wHontOnpcOHpQMHpGPHpC1bmp5ROmwrImbaEi9b\n8s9Jujhpgbe9LSLWsusPaf0Pj/fHjQWQFuq93Qr35G0/IulJSV+wvWr7nO1HbR+0/VQ2bL/tt2yv\nSvq2pCdqnDOAlmDvvf1KbddU9mJs1wBJyNf7yZMs7nWbZ7uGT7wCmAr13i18dw2AUth77yZKHkAh\n6r27KHkAE1Hv3UfJAxiLek8DJQ/gJtR7Wih5ADdQ7+mh5AFQ7wmj5IGeo97TRskDPUW99wMlD/QQ\n9d4flDzQI9R7/1DyQE9Q7/1EyQOJo977jZIHEka9g5IHEkS9YwMlDySGekceJQ8kgnrHOJQ8kADq\nHZNQ8kCHUe8oQskDHUW9owxKHugY6h3ToOSBDqHeMS1KHugA6h2zouSBlqPeMQ9KHmgp6h1VoOSB\nFqLeURVKHmgR6h1Vo+SBlqDeUQdKHmgY9Y46UfJAg6h31I2SBxpAvWNRKHlgwah3LBIlDywI9Y4m\nUPLAAlDvaAolD9SIekfTKHmgJtQ72oCSBypGvaNNKHmgQtQ72oaSBypAvaOtKHlgTtQ72oySB2ZE\nvaMLKHlgBtQ7uoKSB6ZAvaNrKHmgJOodXUTJAwWod3QZJQ9sgnpH1xWWvO0dkl6QtE3Sh5K+GxHH\nx4w7LmmfpD9J+o+IOF/xXIGFod6RijIl/1dJX4+Iz0jaI+mQ7fvzA2zvk7QUEfdKOijpROUzBRaE\nekdKCks+Iq5IupJdv2b7bUnbJV3KDXtM67WviFixvdX2tohYq2HOQC2od6Roqj152/dI2iVpZeSh\n7ZLey92+nN0HdAL1jlSVfneN7TslvSjpSERcm/UFl5eXb1wfDAYaDAazPhUwN+odbTQcDjUcDit5\nLkdE8SD7dkk/lvRaRDwz5vETks5GxKns9iVJe0e3a2xHmdcDFuH0aenwYenAAenYMWnLlqZnBIxn\nWxHhWX62bMk/J+niuAU+87KkQ5JO2X5Y0gfsx6OtqHf0SeGevO1HJD0p6Qu2V22fs/2o7YO2n5Kk\niHhV0ru235H0rKSna501MCP23tE3pbZrKnsxtmvQkHy9nzzJ4o5umWe7hk+8InnUO/qM765Bsth7\nByh5JIp6B9ZR8kgK9Q7cjJJHMqh34FaUPDqPegcmo+TRadQ7sDlKHp1EvQPlUPLoHOodKI+SR2dQ\n78D0KHl0AvUOzIaSR6tR78B8KHm0FvUOzI+SR+tQ70B1KHm0CvUOVIuSRytQ70A9KHk0jnoH6kPJ\nozHUO1A/Sh6NoN6BxaDksVDUO7BYlDwWhnoHFo+SR+2od6A5lDxqRb0DzaLkUQvqHWgHSh6Vo96B\n9qDkURnqHWgfSh6VoN6BdqLkMRfqHWg3Sh4zo96B9qPkMTXqHegOSh5Tod6BbqHkUQr1DnQTJY9C\n1DvQXZQ8JqLege6j5DEW9Q6kgZLHTah3IC2UPG6g3oH0UPKg3oGEUfI9R70DaaPke4p6B/qBku8h\n6h3oD0q+R6h3oH8o+Z6g3oF+ouQTR70D/UbJJ4x6B0DJJ4h6B7CBkk8M9Q4gr7DkbX9P0pclrUXE\nA2Me3yvpjKTfZXf9MCK+WeksUYh6BzBOmZJ/XtKXCsb8LCIezC4s8AtGvQOYpLDkI+Lntu8uGOaK\n5oMpUO8AilS1J7/H9nnbr9jeWdFzYoII6h1AOVW8u+YNSXdFxHXb+yS9JOm+SYOXl5dvXB8MBhoM\nBhVMoT+uXpWeflq6cIF6B1I1HA41HA4reS5HRPGg9e2aH437xeuYse9K+lxEvD/msSjzerjVRr0f\nOSIdOCAdOyZt2dL0rAAsgm1FxEzb4mVL3pqw7257W0SsZdcf0vofHLcs8Jgd9Q5gVoV78rZ/IOmX\nku6z/XvbX7V90PZT2ZD9tt+yvSrp25KeqHG+vRIhnTq1vve+tMTeO4DpldquqezF2K4pLV/vJ0+y\nuAN9Ns92DZ94bRnqHUCV+O6aFmHvHUDVKPkWoN4B1IWSbxj1DqBOlHxDqHcAi0DJN4B6B7AolPwC\nUe8AFo2SXxDqHUATKPmaUe8AmkTJ14h6B9A0Sr4G1DuAtqDkK0a9A2gTSr4i1DuANqLkK0C9A2gr\nSn4O1DuAtqPkZ0S9A+gCSn5K1DuALqHkp0C9A+gaSr4E6h1AV1HyBah3AF1GyU9AvQNIASU/BvUO\nIBWUfA71DiA1lHyGegeQot6XPPUOIGW9LnnqHUDqelny1DuAvuhdyVPvAPqkNyVPvQPoo16UPPUO\noK+SLnnqHUDfJVvy1DsAJFjy1DsA/F1SJU+9A8DNkih56h0Axut8yVPvADBZZ0ueegeAYp0seeod\nAMrpVMlT7wAwnc6UPPUOANNrfclT7wAwu1aXPPUOAPNpZclT7wBQjdaVPPUOANVpTclT7wBQvVaU\nPPUOAPUoLHnb37O9ZvvXm4w5bvu3ts/b3lX2xal3AKhXme2a5yV9adKDtvdJWoqIeyUdlHSizAtf\nvSo9/ri0vLxe79/6lrRlS5mfxIbhcNj0FJLC8awOx7I9Chf5iPi5pD9sMuQxSS9kY1ckbbW9bfLz\nUe9V4X+kanE8q8OxbI8q9uS3S3ovd/tydt/auMGPP87eOwAsysJ/8bq0JH3/+2zNAMAiOCKKB9l3\nS/pRRDww5rETks5GxKns9iVJeyPilpK3XfxiAIBbRIRn+bmyJe/sMs7Lkg5JOmX7YUkfjFvgpdkn\nCQCYTeEib/sHkgaS/tX27yUdlfSPkiIivhMRr9r+d9vvSPqTpK/WOWEAQHmltmsAAN1U+dca1Pnh\nqT4qOp6299r+wPa57PJfi55jV9jeYfunti/YftP24QnjOD9LKHM8OT/Ls32H7RXbq9nxPDph3HTn\nZ0RUepH0eUm7JP16wuP7JL2SXd8t6fWq55DSpcTx3Cvp5abn2YWLpI9L2pVdv1PSbyTdPzKG87Pa\n48n5Od0x/afsvx+R9Lqkh0Yen/r8rLzko+IPT/VdieMpTf6lOHIi4kpEnM+uX5P0ttY/05HH+VlS\nyeMpcX6WFhHXs6t3aP13pqP76VOfn018C+WkD09hdnuyv7q9Yntn05PpAtv3aP1vSCsjD3F+zmCT\n4ylxfpZm+zbbq5KuSPpJRPxqZMjU52crvoUSc3lD0l0RcT37HqGXJN3X8Jxazfadkl6UdCQrUMyh\n4Hhyfk4hIj6U9Fnb/yzpJds7I+LiPM/ZRMlflvTJ3O0d2X2YQURc2/grXkS8JukfbP9Lw9NqLdu3\na31B+p+IODNmCOfnFIqOJ+fnbCLij5LOSnp05KGpz8+6FvmiD099RZKKPjyFGyYez/x+nO2HtP62\n2PcXNbEOek7SxYh4ZsLjnJ/T2fR4cn6WZ/tjtrdm1z8q6YuSLo0Mm/r8rHy7hg9PVavoeErab/s/\nJf1F0p8lPdHUXNvO9iOSnpT0ZrbvGZK+IelucX5OrczxFOfnND4h6b9t36b1AD+VnY8HNcf5yYeh\nACBhrfk3XgEA1WORB4CEscgDQMJY5AEgYSzyAJAwFnkASBiLPAAkjEUeABL2Nyd8jIN/neVAAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x104a45310>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt # OR, alias it as a shorter, more fun-to-type name\n",
    "\n",
    "plt.plot([1,2,3], [1,2,3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Arithmetic\n",
    "- Python 2.7 (like Fortran) uses integer division by default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n"
     ]
    }
   ],
   "source": [
    "print 5 / 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- To force floating point division, specify at least one value in equation as float:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<type 'int'> <type 'float'>\n",
      "2.5\n"
     ]
    }
   ],
   "source": [
    "print type(5), type(2.0)\n",
    "print 5 / 2.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Or, make floating point division default:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.5\n",
      "2\n"
     ]
    }
   ],
   "source": [
    "from __future__ import division\n",
    "print 5 / 2   # now floating point division is default\n",
    "print 5 // 2  # use double slash for integer division"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Functions\n",
    "- Define functions with `def`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def double(x):\n",
    "    \"\"\"This is where you put an optional docstring that \n",
    "    explains what the function does.\n",
    "    For example, this function multiplies its input by 2.\"\"\"\n",
    "    return x * 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "def apply_to_one(f):\n",
    "    \"\"\"Calls the function f with 1 as its argument\"\"\"\n",
    "    return f(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Python functions are [first-class](https://en.wikipedia.org/wiki/First-class_function)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n"
     ]
    }
   ],
   "source": [
    "my_double = double          # refers to the previously defined function\n",
    "x = apply_to_one(my_double)\n",
    "print x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### `lambda` functions\n",
    "- These are short, [anonymous functions](https://en.wikipedia.org/wiki/Anonymous_function)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n"
     ]
    }
   ],
   "source": [
    "y = apply_to_one(lambda x: x + 4)\n",
    "print y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# but use `def` instead of assigning a lambda function to a variable\n",
    "another_double = lambda x: 2 * x     # don't do this\n",
    "def another_double(x): return 2 * x  # do this instead"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### default function arguments\n",
    "- define a default arg; specify arg only if you want different value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hello\n"
     ]
    }
   ],
   "source": [
    "def my_print(message=\"my default message\"):\n",
    "    print message\n",
    "    \n",
    "my_print(\"hello\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "my default message\n"
     ]
    }
   ],
   "source": [
    "my_print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data science is the same as data science\n"
     ]
    }
   ],
   "source": [
    "# Delimit strings with single OR double quotation marks\n",
    "single_quoted_string = 'data science'\n",
    "double_quoted_string = \"data science\"\n",
    "print single_quoted_string, \"is the same as\", double_quoted_string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Use backslashes to encode special characters\n",
    "tab_string = \"\\t\"  # represents the tab character\n",
    "len(tab_string)    # string length is 1 (not 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Use raw strings to represent backslashes\n",
    "not_tab_string = r\"\\t\"  # represents the characters '\\' and 't'\n",
    "len(not_tab_string)     # now length is 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is the first line.\n",
      "and this is the second line\n",
      "and this is the third line\n"
     ]
    }
   ],
   "source": [
    "# Create multiline strings using triple-quotes\n",
    "multi_line_string = \"\"\"This is the first line.\n",
    "and this is the second line\n",
    "and this is the third line\"\"\"\n",
    "print multi_line_string"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Exceptions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "ename": "ZeroDivisionError",
     "evalue": "division by zero",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mZeroDivisionError\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-18-ccf175f7fa0e>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# This makes your code crash\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0;32mprint\u001b[0m \u001b[0;36m0\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mZeroDivisionError\u001b[0m: division by zero"
     ]
    }
   ],
   "source": [
    "# This makes your code crash\n",
    "print 0 / 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "[Here's](https://docs.python.org/2/library/exceptions.html) a list of built-in exceptions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cannot divide by zero\n"
     ]
    }
   ],
   "source": [
    "# This will handle the exception by printing an error message\n",
    "try:\n",
    "    print 0 / 0\n",
    "except ZeroDivisionError:\n",
    "    print \"Cannot divide by zero\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Lists\n",
    "- Ordered [collections](https://en.wikipedia.org/wiki/Collection_(abstract_data_type%29)\n",
    "- Similar to an array in other languages, but holds heterogeneous data (e.g., floats + ints + strings)\n",
    "- *Note*: Use NumPy arrays ([here's](https://docs.scipy.org/doc/numpy-dev/user/quickstart.html) a tutorial) for larger amounts of homogeneous data (e.g., just floats)\n",
    "- Specify with brackets `[]`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3\n",
      "6\n"
     ]
    }
   ],
   "source": [
    "integer_list = [1, 2, 3]\n",
    "heterogeneous_list = [\"string\", 0.1, True]\n",
    "list_of_lists = [ integer_list, heterogeneous_list, [] ]\n",
    "\n",
    "print len(integer_list)  # get the length of a list\n",
    "print sum(integer_list)  # get the sum of the elements in a list (if addition is defined for those elements)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n",
      "0\n",
      "1\n",
      "9\n",
      "8\n"
     ]
    }
   ],
   "source": [
    "# Use square brackets to get the n^{th} element of a list\n",
    "x = range(10)\n",
    "print x\n",
    "print x[0]\n",
    "print x[1]\n",
    "print x[-1]\n",
    "print x[-2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0, 1, 2]\n",
      "[3, 4, 5, 6, 7, 8, 9]\n",
      "[1, 2, 3]\n",
      "[7, 8, 9]\n",
      "[1, 2, 3, 4, 5, 6, 7, 8]\n",
      "[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n"
     ]
    }
   ],
   "source": [
    "# Use square brackets to \"slice\" lists\n",
    "print x[:3]   # up to but not including 3\n",
    "print x[3:]   # 3 and up\n",
    "print x[1:4]  # 1 up to but not including 4\n",
    "print x[-3:]  # last 3\n",
    "print x[1:-1] # without 1 and 9\n",
    "print x[:]    # all elements of the list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "# Use the `in` operator to check for list membership; use only for small lists or if run time is not a concern\n",
    "print 1 in [1, 2, 3]\n",
    "print 0 in [1, 2, 3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 3]\n",
      "[1, 2, 3, 4, 5, 6] \n",
      "\n",
      "[1, 2, 3, 4, 5, 6]\n"
     ]
    }
   ],
   "source": [
    "# Concatenate lists like this\n",
    "x = [1, 2, 3]\n",
    "\n",
    "y = x + [4, 5, 6]   # creates a new list leaving \"x\" unchanged\n",
    "print x\n",
    "print y, \"\\n\"\n",
    "\n",
    "x.extend([4, 5, 6]) # changes \"x\"\n",
    "print x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 3] \n",
      "\n",
      "[1, 2, 3, 0]\n"
     ]
    }
   ],
   "source": [
    "# Append to lists like this\n",
    "x = [1, 2, 3]\n",
    "print x, \"\\n\"\n",
    "\n",
    "x.append(0)\n",
    "print x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<type 'int'> x = 1\n",
      "<type 'int'> y = 2\n",
      "<type 'list'> z = [1, 2]\n"
     ]
    }
   ],
   "source": [
    "# It's convenient (and common) to unpack lists like this\n",
    "z = [1, 2]\n",
    "x, y = z\n",
    "print type(x), \"x =\", x \n",
    "print type(y), \"y =\", y \n",
    "print type(z), \"z =\", z "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "4\n",
      "[3, 4]\n"
     ]
    }
   ],
   "source": [
    "# It's also common to use an underscore for a value you're going to throw away\n",
    "z = [3, 4]\n",
    "_, y = z\n",
    "print x\n",
    "print y\n",
    "print z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Tuples\n",
    "- Similar to list, but immutable (can't be modified)\n",
    "- Specify with parentheses`()` or nothing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 3]\n"
     ]
    }
   ],
   "source": [
    "my_list = [1, 2]\n",
    "my_tuple = (1, 2)\n",
    "other_tuple = 3, 4\n",
    "my_list[1] = 3\n",
    "\n",
    "print my_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cannot modify a tuple\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    my_tuple[1] = 3\n",
    "except TypeError:\n",
    "    print \"Cannot modify a tuple\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(5, 6)\n",
      "15\n",
      "50\n"
     ]
    }
   ],
   "source": [
    "# Use tuples to return multiple values from functions\n",
    "def sum_and_product(x, y):\n",
    "    return (x + y), (x * y)\n",
    "\n",
    "sp = sum_and_product(2, 3)\n",
    "s, p = sum_and_product(5, 10)\n",
    "print sp\n",
    "print s \n",
    "print p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x = 1\n",
      "y = 2\n"
     ]
    }
   ],
   "source": [
    "# Use tuples (and lists) for multiple assignments\n",
    "x, y = 1, 2\n",
    "print \"x =\", x\n",
    "print \"y =\", y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x = 2\n",
      "y = 1\n"
     ]
    }
   ],
   "source": [
    "x, y = y, x  # Pythonic way to swap variables\n",
    "print \"x =\", x\n",
    "print \"y =\", y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Dictionaries\n",
    "- Dictionaries associate *values* with *keys*\n",
    "- Allow quick retrieval of value given a key\n",
    "- Specify with curly braces `{}` or `dict()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "80\n"
     ]
    }
   ],
   "source": [
    "empty_dict = {}                    # Pythonic\n",
    "empty_dict2 = dict()               # less Pythonic\n",
    "grades = { \"Joel\": 80, \"Tim\": 95}\n",
    "\n",
    "# Use square brackets to look up value(s) for a key\n",
    "print grades[\"Joel\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No grade for Kate!\n"
     ]
    }
   ],
   "source": [
    "# KeyError exception raised if key not found\n",
    "try: \n",
    "    kates_grade = grades[\"Kate\"]\n",
    "except KeyError:\n",
    "    print \"No grade for Kate!\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "# Use `in` to check for existence of a key\n",
    "joel_has_grade = \"Joel\" in grades\n",
    "kate_has_grade = \"Kate\" in grades\n",
    "\n",
    "print joel_has_grade\n",
    "print kate_has_grade"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "80\n",
      "0\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# Use `get` method of dictionaries when you want to return a default value (rather than raise exception)\n",
    "print grades.get(\"Joel\", 0)\n",
    "print grades.get(\"Kate\", 0)\n",
    "print  grades.get(\"No One\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of students: 3\n"
     ]
    }
   ],
   "source": [
    "# Use square brackets to assign key-value pairs, e.g., dict_name[key] = value\n",
    "grades[\"Tim\"] = 99\n",
    "grades[\"Kate\"] = 100\n",
    "\n",
    "print \"Number of students:\", len(grades)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Use dictionaries to represent structured data, such as in a tweet\n",
    "tweet = {\n",
    "    \"user\": \"joelgrus\",\n",
    "    \"text\": \"Data Science is Awesome\",\n",
    "    \"retweet_count\": 100,\n",
    "    \"hashtags\": [\"#data\", \"#science\", \"#datascience\", \"#awesome\"]\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['text', 'retweet_count', 'hashtags', 'user']\n",
      "['Data Science is Awesome', 100, ['#data', '#science', '#datascience', '#awesome'], 'joelgrus']\n",
      "[('text', 'Data Science is Awesome'), ('retweet_count', 100), ('hashtags', ['#data', '#science', '#datascience', '#awesome']), ('user', 'joelgrus')]\n"
     ]
    }
   ],
   "source": [
    "print tweet.keys()    # list of keys\n",
    "print tweet.values()  # list of values\n",
    "print tweet.items()   # list of (key, value) tuples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "True\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "print \"user\" in tweet.keys()       # list `in` is slow\n",
    "print \"user\" in tweet              # dict `in` is fast (and more Pythonic)\n",
    "print \"joelgrus\" in tweet.values() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "You *cannot* use lists as keys.  If that's needed, then:\n",
    "- use a `tuple`, or\n",
    "- represent the key as a string"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Let's grab some data to use in the upcoming examples on `defaultdict`\n",
    "- Clone the UW Intro to Data Science course materials repository:\n",
    "\n",
    "`git clone https://github.com/uwescience/datasci_course_materials.git`\n",
    "\n",
    "- Look in `datasci_course_materials/assignment1/` for a file named `three_minutes_tweets.json`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total # of tweets: 8299\n",
      "type(tweets[0]): <type 'dict'>\n"
     ]
    }
   ],
   "source": [
    "# Load the data into a list called `tweets`\n",
    "import json\n",
    "# Substitute the local path to your three_minutes_tweets.json file within the quotes\n",
    "tweet_file = open(\"/Users/mepa/repos/datasci_course_materials/assignment1/three_minutes_tweets.json\")\n",
    "\n",
    "tweets = []\n",
    "for line in tweet_file:\n",
    "    tweets.append(json.loads(line)) # each list element contains all data pertaining to a single tweet\n",
    "print \"Total # of tweets:\", len(tweets)\n",
    "print \"type(tweets[0]):\", type(tweets[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- For tweets that contain a \"text\" key we can print someone's tweet:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@1voodoochild thanks for the follow💯\n"
     ]
    }
   ],
   "source": [
    "print tweets[7]['text']  # text of 8th tweet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "إنّ العرب إذا تغلبوا على أوطان أسرع إليها الخراب والسبب في ذلك أنها أمة وحشية بإستحكام عوائد التوحش وأسبابه فصار لهم خلقة وجبلة\n",
      "\n",
      "- ابن خلدون\n"
     ]
    }
   ],
   "source": [
    "print tweets[3]['text']  # text of 4th tweet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "*Aside*: The text is attributed to [Ibn Khaldun](https://en.wikipedia.org/wiki/Ibn_Khaldun) who lived 1332 - 1406.  Google Translate gives the following:\n",
    "\n",
    "\"The Arabs overcame their homelands if faster to ruin and why they are a nation and brutal savagery Bastgam returns and causes them became habitus and protoplasm\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@morgancollins42 I don't like you\n"
     ]
    }
   ],
   "source": [
    "print tweets[54]['text']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Newf opened up and ate a whole bar of Ivory soap. Some things about the canine diet I will never understand.\n"
     ]
    }
   ],
   "source": [
    "print tweets[7000]['text']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "contributors : \tNone\n",
      "truncated : \tFalse\n",
      "text : \tI'm starving I can't wait to leave work and eat Wing Barn 😋🍗\n",
      "in_reply_to_status_id : \tNone\n",
      "id : \t633030783796514816\n",
      "favorite_count : \t0\n",
      "source : \t<a href=\"http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>\n",
      "retweeted : \tFalse\n",
      "coordinates : \tNone\n",
      "timestamp_ms : \t1439761274657\n",
      "entities : \t{u'user_mentions': [], u'symbols': [], u'trends': [], u'hashtags': [], u'urls': []}\n",
      "in_reply_to_screen_name : \tNone\n",
      "id_str : \t633030783796514816\n",
      "retweet_count : \t0\n",
      "in_reply_to_user_id : \tNone\n",
      "favorited : \tFalse\n",
      "user : \t{u'follow_request_sent': None, u'profile_use_background_image': True, u'default_profile_image': False, u'id': 509749471, u'verified': False, u'profile_image_url_https': u'https://pbs.twimg.com/profile_images/603406649433268224/wfMu2EpQ_normal.jpg', u'profile_sidebar_fill_color': u'DDEEF6', u'profile_text_color': u'333333', u'followers_count': 149, u'profile_sidebar_border_color': u'C0DEED', u'id_str': u'509749471', u'profile_background_color': u'C0DEED', u'listed_count': 2, u'profile_background_image_url_https': u'https://abs.twimg.com/images/themes/theme1/bg.png', u'utc_offset': None, u'statuses_count': 6211, u'description': u'21 years old, Aquarius \\u2652\\ufe0f, Instagram _BENNYAGUIRRE, taken', u'friends_count': 206, u'location': u'harlingen, tx', u'profile_link_color': u'0084B4', u'profile_image_url': u'http://pbs.twimg.com/profile_images/603406649433268224/wfMu2EpQ_normal.jpg', u'following': None, u'geo_enabled': False, u'profile_banner_url': u'https://pbs.twimg.com/profile_banners/509749471/1423998507', u'profile_background_image_url': u'http://abs.twimg.com/images/themes/theme1/bg.png', u'name': u'Ben\\u270c', u'lang': u'en', u'profile_background_tile': False, u'favourites_count': 3447, u'screen_name': u'BenitoAguirre1', u'notifications': None, u'url': None, u'created_at': u'Thu Mar 01 05:16:54 +0000 2012', u'contributors_enabled': False, u'time_zone': None, u'protected': False, u'default_profile': True, u'is_translator': False}\n",
      "geo : \tNone\n",
      "in_reply_to_user_id_str : \tNone\n",
      "possibly_sensitive : \tFalse\n",
      "lang : \ten\n",
      "created_at : \tSun Aug 16 21:41:14 +0000 2015\n",
      "filter_level : \tlow\n",
      "in_reply_to_status_id_str : \tNone\n",
      "place : \tNone\n"
     ]
    }
   ],
   "source": [
    "# Print all key-value pairs in the 54th tweet\n",
    "for key in tweets[53].keys():\n",
    "    print key, \": \\t\", tweets[53][key]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### `defaultdict`\n",
    "- Like a regular dictionary, except that when you try to look up a key that is not present in the dictionary, it first adds a value for it using a no-argument function that you provide when creating it.\n",
    "- Imagine creating a dictionary to count the words in a document.  Here are 3 approaches for doing that with a regular dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Approach 1 - if/else statement\n",
    "word_counts = {}\n",
    "for word in document:\n",
    "    if word in word_counts:\n",
    "        word_counts[word] += 1\n",
    "    else:\n",
    "        word_counts[word] = 1\n",
    "        \n",
    "# Approach 2 - handle the exception\n",
    "word_counts = {}\n",
    "for word in document:\n",
    "    try:\n",
    "        word_counts[word] += 1\n",
    "    except KeyError:\n",
    "        word_counts[word] = 1\n",
    "        \n",
    "# Approach 3 - use `get`\n",
    "word_counts = {}\n",
    "for word in document:\n",
    "    previous_count = word_counts.get(word, 0)\n",
    "    word_counts[word] = previous_count + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Instead we can use `defaultdict` (needs to be imported from `collections`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "from collections import defaultdict\n",
    "\n",
    "word_counts = defaultdict(int)  # int(0) produces 0\n",
    "for word in document:\n",
    "    word_counts[word] += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RT @HCeretto: I don't love you I'm just passing the time\n",
      "You could love me if I knew how to lie\n",
      "But who could love me? I am out of my mind \n",
      "\n",
      "defaultdict(<type 'int'>, {'love': 3, 'just': 1, \"don't\": 1, 'am': 1, 'me?': 1, 'You': 1, 'if': 1, 'RT': 1, 'lie': 1, 'how': 1, 'But': 1, 'to': 1, 'you': 1, 'out': 1, 'knew': 1, 'I': 3, 'mind': 1, 'who': 1, \"I'm\": 1, 'passing': 1, 'me': 1, '@HCeretto:': 1, 'of': 1, 'could': 2, 'time': 1, 'the': 1, 'my': 1})\n"
     ]
    }
   ],
   "source": [
    "# Let's implement the example above, but substitute a tweet for \"document\"\n",
    "from collections import defaultdict\n",
    "\n",
    "tweet_text = tweets[2736]['text'].encode('utf-8') # returns one long string & encodes the text as unicode\n",
    "tweet_words = tweet_text.split()                  # split() returns a list of words\n",
    "\n",
    "word_counts = defaultdict(int)\n",
    "for word in tweet_words:\n",
    "    word_counts[word] += 1\n",
    "        \n",
    "print tweet_text, \"\\n\"\n",
    "print word_counts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Optional Exercises\n",
    "\n",
    "Read over the [UW Intro to Data Science - Assignment 1](http://htmlpreview.github.io/?https://github.com/uwescience/datasci_course_materials/blob/master/assignment1/assignment1.html) description.\n",
    "\n",
    "Skip\n",
    "- Problem 1: Get the Twitter Data\n",
    "\n",
    "and just use \"three_minutes_tweets.json\" for the following problems:\n",
    "\n",
    "- Problem 2: Derive the sentiment of each tweet\n",
    "- Problem 3: Derive the sentiment of new terms\n",
    "- Problem 4: Compute term frequency\n",
    "\n",
    "Can skip the last two problems:\n",
    "- Problem 5: Which state is happiest?\n",
    "- Problem 6: Top ten hash tags"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "`defaultdict` can also be useful with `list` or `dict` or your own functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "defaultdict(<type 'list'>, {2: [1]}) \n",
      "\n",
      "defaultdict(<type 'dict'>, {'Joel': {'City': 'Seattle'}}) \n",
      "\n",
      "defaultdict(<function <lambda> at 0x11c4e5410>, {2: [0, 1]})\n"
     ]
    }
   ],
   "source": [
    "dd_list = defaultdict(list)          # list() produces an empty list\n",
    "dd_list[2].append(1) \n",
    "print dd_list, \"\\n\"\n",
    "\n",
    "dd_dict = defaultdict(dict)          # dict() produces an empty dict\n",
    "dd_dict[\"Joel\"][\"City\"] = \"Seattle\"\n",
    "print dd_dict, \"\\n\"\n",
    "\n",
    "dd_pair = defaultdict(lambda: [0,0]) # use a lambda function\n",
    "dd_pair[2][1] = 1\n",
    "print dd_pair"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### `Counter`\n",
    "- Instead of using any of the approaches mentioned above to compute word-counts, we could have rather used a simpler built-in counter offered by Python. `Counter` turns a sequence of values into defaultdict(int) like objects mapping keys to its corresponding counts. This gives a very simple way to solve our word-count problem.\n",
    "- A `Counter` instance has a `most_common` method to find most common keys and their counts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counter({0: 2, 1: 1, 2: 1})\n"
     ]
    }
   ],
   "source": [
    "from collections import Counter\n",
    "c = Counter([0, 1, 2, 0])\n",
    "print c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "word_counts = Counter(document)\n",
    "\n",
    "# Print the 10 most common words and their counts\n",
    "for word, count in word_counts.most_common(10):\n",
    "    print word, count"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RT @HCeretto: I don't love you I'm just passing the time\n",
      "You could love me if I knew how to lie\n",
      "But who could love me? I am out of my mind \n",
      "\n",
      "love 3\n",
      "I 3\n",
      "could 2\n",
      "just 1\n",
      "don't 1\n"
     ]
    }
   ],
   "source": [
    "print tweet_text, \"\\n\"\n",
    "\n",
    "word_counts = Counter(tweet_words)\n",
    "\n",
    "# Print the 5 most common words and their counts\n",
    "for word, count in word_counts.most_common(5):\n",
    "    print word, count"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Sets\n",
    "- Represents a collection of distinct elements\n",
    "- Used for two main reasons: \n",
    "    1. the `in` operation is much faster on sets than on lists, for example \n",
    "    2. sometimes we want to find *distinct* items in a list "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "s is set([1])\n",
      "s is now set([1, 2])\n",
      "s is still set([1, 2])\n",
      "There are 2 elements in s.\n",
      "True\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "s = set()\n",
    "s.add(1)\n",
    "print \"s is\", s\n",
    "s.add(2)\n",
    "print \"s is now\", s\n",
    "s.add(2)\n",
    "print \"s is still\", s\n",
    "print \"There are\", len(s), \"elements in s.\"\n",
    "print 2 in s\n",
    "print 3 in s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "78668\n"
     ]
    }
   ],
   "source": [
    "# Let's use the tweets we read in earlier from \"three_minutes_tweets.json\"\n",
    "all_tweets_words = []\n",
    "for tweet in tweets:\n",
    "    if 'text' in tweet: # since not all \"tweets\" contain text\n",
    "        tweet_text = tweet['text'].encode('utf-8')\n",
    "        tweet_words_list = tweet_text.split()\n",
    "        all_tweets_words.extend(tweet_words_list)\n",
    "print len(all_tweets_words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 49 µs, sys: 11 µs, total: 60 µs\n",
      "Wall time: 62 µs\n",
      "CPU times: user 5 µs, sys: 2 µs, total: 7 µs\n",
      "Wall time: 7.87 µs\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%time \"love\" in all_tweets_words # another IPython magic command!\n",
    "\n",
    "all_tweets_words_set = set(all_tweets_words)\n",
    "%time \"love\" in all_tweets_words_set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6\n",
      "set([1, 2, 3])\n",
      "3\n",
      "3\n"
     ]
    }
   ],
   "source": [
    "item_list = [1, 2, 3, 1, 2, 3]\n",
    "print len(item_list)\n",
    "print set(item_list)\n",
    "print len(set(item_list))\n",
    "print len(list(set(item_list)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Control Flow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "#### Conditional looping: if-else statements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Negative changed to zero\n"
     ]
    }
   ],
   "source": [
    "x = - 4\n",
    "\n",
    "if x < 0:\n",
    "    x = 0\n",
    "    print 'Negative changed to zero'\n",
    "elif x == 0:\n",
    "     print 'Zero'\n",
    "elif x == 1:\n",
    "     print 'Single'\n",
    "else:\n",
    "    print 'More'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "odd\n"
     ]
    }
   ],
   "source": [
    "# Can also write if-then-else statements on one line\n",
    "x = 5\n",
    "parity = \"even\" if x % 2 == 0 else \"odd\" \n",
    "print parity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### While loops\n",
    "- Repeatedly executes a target statement as long as a given condition is true."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The count is: 0\n",
      "The count is: 1\n",
      "The count is: 2\n",
      "The count is: 3\n",
      "The count is: 4\n",
      "The count is: 5\n",
      "The count is: 6\n",
      "The count is: 7\n",
      "The count is: 8\n",
      "Good bye!\n"
     ]
    }
   ],
   "source": [
    "count = 0\n",
    "while (count < 9):\n",
    "    print 'The count is:', count\n",
    "    count = count + 1\n",
    "\n",
    "print \"Good bye!\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### For loops\n",
    "- Iterates over the items of any sequence, such as a list or a string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dogs 4\n",
      "cats 4\n",
      "elephant 8\n",
      "tiger 5\n",
      "lion 4\n"
     ]
    }
   ],
   "source": [
    "animal_kingdom = ['dogs', 'cats', 'elephant', 'tiger', 'lion']\n",
    "\n",
    "for animal in animal_kingdom:\n",
    "    print animal, len(animal)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### `continue` and `break`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "for x in range(10):\n",
    "    if x == 3:\n",
    "        continue   # go immediately to the next iteration\n",
    "    if x == 5:\n",
    "        break      # quit the loop entirely\n",
    "    print x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Truthiness"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Booleans work in Python as in most other languages, except that they are capitalized\n",
    "- Python uses value 'None' to indicate a non-existent value. It is similar to 'null' in other languages. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "one_is_less_than_two: True\n",
      "true_equals_false: False\n",
      "x_equals_none: True\n",
      "x is none? True\n"
     ]
    }
   ],
   "source": [
    "one_is_less_than_two = 1 < 2\n",
    "print 'one_is_less_than_two:', one_is_less_than_two\n",
    "\n",
    "true_equals_false = True == False\n",
    "print 'true_equals_false:', true_equals_false\n",
    "\n",
    "x = None\n",
    "print 'x_equals_none:', x == None # non-Pythonic way\n",
    "print 'x is none?', x is None     # Pythonic way"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- Python lets you use any value where it expects a boolean. These all count as `False`:\n",
    "    - `False`\n",
    "    - `None`\n",
    "    - `[]`\n",
    "    - `{}`\n",
    "    - `\"\"`\n",
    "    - `set()`\n",
    "    - `0`\n",
    "    - `0.0`\n",
    "- Almost everything else gets treated as `True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f\n"
     ]
    }
   ],
   "source": [
    "# Use `if` statements to check for empty lists, strings, dicts, etc.\n",
    "s = \"foo\"\n",
    "if s:\n",
    "    first_char = s[0]\n",
    "else:\n",
    "    first_char = \"\"\n",
    "print first_char"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Python also has 'all' and 'any' functions that take a list and return True precisly when 'all' or 'any' elements of the list are truthy respectively. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "False\n",
      "True\n",
      "True\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "print all([True, 1, {3}])\n",
    "print all([True, \"\", {3}])\n",
    "print any([False, 1, [2]])\n",
    "print all([])\n",
    "print any([])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The Not-So-Basics\n",
    "- Sorting\n",
    "- List Comprehensions\n",
    "- Generators and Iterators\n",
    "- Randomness\n",
    "- Regular Expressions\n",
    "- Object-Oriented Programming\n",
    "- Functional Tools\n",
    "- <code>enumerate</code>\n",
    "- <code>zip</code> and Argument Unpacking\n",
    "- args and kwargs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Sorting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "You can sort a list using two functions:\n",
    "- sort(): this sorts list in place\n",
    "- sorted(): this creates a new sorted list and so, the original list remains unchanged\n",
    "\n",
    "Usually, the elements are sorted by values in ascending order. However, you can change this default behaviour by setting argument 'reverse' = True in the sorting function. \n",
    "\n",
    "Also, you can specify key by which you would like your collection to be sorted. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 3, 4]\n",
      "[4, 1, 2, 3]\n",
      "[1, 2, 3, 4]\n"
     ]
    }
   ],
   "source": [
    "x = [4,1,2,3]\n",
    "print sorted(x)\n",
    "print x         # x list is still the same\n",
    "x.sort()        \n",
    "print x         # x list is now changed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4, 3, 2, 1]\n",
      "[4, 3, 2, 1]\n"
     ]
    }
   ],
   "source": [
    "print sorted(x, reverse=True)\n",
    "x.sort(reverse=True)\n",
    "print x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-4, 3, -2, 1]\n"
     ]
    }
   ],
   "source": [
    "#sort a list by abs value in descending order\n",
    "x = sorted([-4, 1, -2, 3], key = abs, reverse = True)\n",
    "print x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### List Comprehensions\n",
    "- List Comprehensions is a very powerful tool, which creates a new list based on another list, in a single, readable line.\n",
    "- You can also tyrn lists into dictionaries or sets using list comprehensions. \n",
    "- We can also use multiple for loops in list comprehension. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "list of even numbers below 5: [0, 2, 4]\n",
      "list of squares of numbers less than 5: [0, 1, 4, 9, 16]\n",
      "list of squares of even numbers less than 5: [0, 4, 16]\n",
      "\n",
      "dictionary of squares of numbers less than 5: {0: 0, 1: 1, 2: 4, 3: 9, 4: 16}\n",
      "set of squares of numbers less than 5: set([0, 1, 4, 16, 9])\n",
      "\n",
      "[(0, 0), (0, 1), (0, 2), (0, 3), (0, 4), (1, 0), (1, 1), (1, 2), (1, 3), (1, 4), (2, 0), (2, 1), (2, 2), (2, 3), (2, 4), (3, 0), (3, 1), (3, 2), (3, 3), (3, 4), (4, 0), (4, 1), (4, 2), (4, 3), (4, 4)]\n"
     ]
    }
   ],
   "source": [
    "even_numbers = [x for x in range(5) if x%2 == 0]\n",
    "print 'list of even numbers below 5:', even_numbers\n",
    "\n",
    "squares = [x**2 for x in range(5)]\n",
    "print 'list of squares of numbers less than 5:', squares\n",
    "\n",
    "even_squares = [x**2 for x in even_numbers]\n",
    "print 'list of squares of even numbers less than 5:', even_squares\n",
    "\n",
    "print\n",
    "#turning list into dict or sets\n",
    "square_dict = {x : x**2 for x in range(5)}\n",
    "print 'dictionary of squares of numbers less than 5:', square_dict\n",
    "\n",
    "square_set =  set(x**2 for x in range(5))\n",
    "print 'set of squares of numbers less than 5:', square_set\n",
    "\n",
    "print\n",
    "#list comprehension with multiple for statements\n",
    "pairs = [(x, y) \n",
    "         for x in range(5)\n",
    "         for y in range(5)\n",
    "        ]\n",
    "print pairs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Generators and Iterators"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "5\n",
      "6\n",
      "7\n",
      "8\n",
      "9\n",
      "squares_list: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]\n"
     ]
    }
   ],
   "source": [
    "def squares(n):\n",
    "    return n**2\n",
    "\n",
    "def lazy_range(n):  # this is`xrange` in Python2 and `range` in Python3\n",
    "    \"\"\"a lazy version of range\"\"\"\n",
    "    i = 0\n",
    "    while i < n:\n",
    "        print i\n",
    "        yield i\n",
    "        i += 1\n",
    "        \n",
    "squares_list = []\n",
    "\n",
    "for i in lazy_range(10):\n",
    "    squares_list.append(squares(i))\n",
    "\n",
    "print 'squares_list:', squares_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Create an infinite sequence (but don't use without some kind of `break` statement)\n",
    "def natural_numbers():\n",
    "    n = 1\n",
    "    while True:\n",
    "        yield n\n",
    "        n += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Recall that every dict has items() that returns a list of its key-value pairs. Dictionaries also have `iteritems()` method that lazily yields the key value pairs one at a time as we iterate over it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Randomness"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We can generate random numbers using random module from Python. Some of the important methods from random package that we will often use are as follows:\n",
    "    \n",
    "- `random.seed` - Set random seed in case you want to have reproducible results with random numbers\n",
    "- `randrange()` - allows you to produce random number from within given range\n",
    "- `shuffle()` - randomly shuffles elements of a collection and gives an output\n",
    "- `choice()` - in case you want to randomly select an element from a collection\n",
    "- `sample()` - if you want to randomly choose a sample of elements without replacement (sampling without any duplicates)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "four_uniform_random numbers: [0.4506916641123372, 0.5200461255446185, 0.39018952392156003, 0.5616870973196527]\n",
      "\n",
      "reproducible random number: 0.57140259469\n",
      "reproducible random number again: 0.57140259469\n",
      "\n",
      "random number between 0 and 10: 4\n",
      "random number between 3 and 6: 4\n",
      "\n",
      "ordered list: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n",
      "shuffled list: [8, 3, 5, 1, 9, 0, 4, 6, 7, 2]\n",
      "\n",
      "my_best_friend: Skinny Pete\n",
      "\n",
      "winning lottery numbers: [4, 86, 60, 38, 28, 67]\n",
      "\n",
      "four_with_replacement [4, 6, 6, 1]\n"
     ]
    }
   ],
   "source": [
    "import random \n",
    "\n",
    "four_uniform_randoms = [random.random() for _ in range(4)]\n",
    "print 'four_uniform_random numbers:', four_uniform_randoms\n",
    "print\n",
    "\n",
    "#set seed for reproducible results\n",
    "random.seed(10)\n",
    "print 'reproducible random number:', random.random()\n",
    "random.seed(10) #reset the seed to 10\n",
    "print 'reproducible random number again:',random.random()\n",
    "print\n",
    "\n",
    "#create range of random numbers \n",
    "print 'random number between 0 and 10:', random.randrange(10)\n",
    "print 'random number between 3 and 6:',random.randrange(3, 6)\n",
    "print\n",
    "\n",
    "#shuffle a list in order to get random order of its elements\n",
    "up_to_10 = range(10)\n",
    "print 'ordered list:', up_to_10\n",
    "random.shuffle(up_to_10)\n",
    "print 'shuffled list:', up_to_10\n",
    "print\n",
    "\n",
    "#randomly pick one element from list\n",
    "my_best_friend = random.choice(['Heisenberg', 'Saul', 'Jesse', 'Skinny Pete'])\n",
    "print 'my_best_friend:', my_best_friend\n",
    "print \n",
    "\n",
    "#sample numbers without replacement(without dups)\n",
    "lottery_numbers = range(100)\n",
    "winning_numbers = random.sample(lottery_numbers, 6)\n",
    "print 'winning lottery numbers:', winning_numbers\n",
    "print\n",
    "\n",
    "#sample with replacement(with dups)\n",
    "four_with_replacement = [random.choice(range(10)) for _ in range(4)]\n",
    "print 'four_with_replacement', four_with_replacement"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Regular Expressions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Regular expressions provide a way of searching text and are used extensively in NLP. They range from easy to extremely complicated. Some of the important functions from regular expressions (re) package in python are as follows:\n",
    "\n",
    "- `re.match()` - tries to match pattern with the string and returns True or False\n",
    "- `re.search()` - looks for pattern in the string and returns the matching pattern in string\n",
    "- `re.split()` - splits the string based on pattern\n",
    "- `re.sub()` - replaces / substitutes pattern with replacement value and changes the string value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "does a match cat? None\n",
      "does cat have an a? <_sre.SRE_Match object at 0x11c6b2370>\n",
      "does dog have an a? None\n",
      "split carbs on a and b ['c', 'r', 's']\n",
      "replace any numbers with to from here to there\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "print 'does a match cat?', re.match('a', 'cat')\n",
    "print 'does cat have an a?', re.search('a', 'cat')\n",
    "print 'does dog have an a?', re.search('a', 'dog')\n",
    "print 'split carbs on a and b', re.split('[ab]', 'carbs')\n",
    "print 'replace any numbers with to', re.sub('[0-9]+', 'to', 'from here 2 there')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Object-Oriented Programming"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Python allows you to create classes that encapsulate data and functions that operate on them. \n",
    "\n",
    "For example: Let us say we did not have an in-built implementation of sets in python and we would like to build one. So, we can start by constructing elements of Set class. \n",
    "\n",
    "In our set class, we would like to have following functions: \n",
    "- add : to add items to set\n",
    "- remove: to remove items from set\n",
    "- contains: to check if a given element is present in the set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "class Set:\n",
    "    \n",
    "    def __init__(self, values = None):\n",
    "        self.dict = {}\n",
    "        if values is not None:\n",
    "            for value in values:\n",
    "                self.add(value)\n",
    "    \n",
    "    def __repr__(self):\n",
    "        return 'Set:', str(self.dict.keys())\n",
    "    \n",
    "    def add(self, value):\n",
    "        self.dict[value] = True\n",
    "    \n",
    "    def remove(self, value):\n",
    "        del self.dict[value]\n",
    "    \n",
    "    def contains(self, value):\n",
    "        return value in self.dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "set contains 3? True\n",
      "set contains 3? False\n"
     ]
    }
   ],
   "source": [
    "s = Set([1, 2, 3])\n",
    "s.add(4)\n",
    "print 'set contains 3?', s.contains(3)\n",
    "s.remove(3)\n",
    "print 'set contains 3?', s.contains(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Functional Tools"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "When passing functions around, sometimes we want to apply a function only partially to create new functions. For this purpose we can use various functional tools offered by Python. Some of the functions that we would be using are as follows:\n",
    "\n",
    "- partial(): allows you to partially fill function with default values and create new functions\n",
    "- map(): allows you to apply (or map) a function to every element of a collection\n",
    "- filter(): returns elements of a list that satisfy a pre-defined condition or filter\n",
    "- reduce(): combines all elements of a collection from left to right"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "two to the power 3: 8\n",
      "square of 3: 9\n"
     ]
    }
   ],
   "source": [
    "#use of partial() function:\n",
    "def exp(base, power):\n",
    "    return base ** power\n",
    "\n",
    "#compute two to the power without using partial\n",
    "def two_to_the_power(power):\n",
    "    return exp(2, power)\n",
    "\n",
    "#use partial() function to compute results of 2 raised to a power\n",
    "from functools import partial\n",
    "two_to_the_power = partial(exp, 2) #two_to_the_power is now a function of just one variable\n",
    "print 'two to the power 3:', two_to_the_power(3)\n",
    "\n",
    "#use partial() function to compute any base number raised to a power\n",
    "square_of = partial(exp, power = 2)\n",
    "print 'square of 3:', square_of(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "twice_xs created using list comprehension method: [2, 4, 6, 8]\n",
      "twice_xs created using map method: [2, 4, 6, 8]\n"
     ]
    }
   ],
   "source": [
    "#use map() function\n",
    "def double(x):\n",
    "    return 2 * x\n",
    "\n",
    "xs = [1, 2, 3, 4]\n",
    "twice_xs = [double(x) for x in xs] #double every element of list using list comprehension\n",
    "print 'twice_xs created using list comprehension method:', twice_xs\n",
    "twice_xs = map(double, xs) #double every element of list by using map() function\n",
    "print 'twice_xs created using map method:', twice_xs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_evens created using list comprehension method: [2, 4]\n",
      "x_evens created using filter method: [2, 4]\n"
     ]
    }
   ],
   "source": [
    "#use filter() function\n",
    "def is_even(n):\n",
    "    return n%2 == 0\n",
    "\n",
    "x_evens = [x for x in xs if is_even(x)] #find even numbers in the list using list-comprehension method\n",
    "print 'x_evens created using list comprehension method:', x_evens\n",
    "x_evens = filter(is_even, xs) #double every element of list by using map() function\n",
    "print 'x_evens created using filter method:', x_evens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "product of all elements of list: 24\n"
     ]
    }
   ],
   "source": [
    "#use reduce() function\n",
    "def multiply(x, y): return x*y\n",
    "\n",
    "x_product = reduce(multiply, xs) #computes 1 * 2 * 3 * 4\n",
    "print 'product of all elements of list:', x_product"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `enumerate`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The `enumerate()` function can be use to iterate over indices and items of a list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 a\n",
      "1 b\n",
      "2 c\n",
      "0 a\n",
      "1 b\n",
      "2 c\n"
     ]
    }
   ],
   "source": [
    "a = [\"a\", \"b\", \"c\"]\n",
    "\n",
    "#non-pythonic way\n",
    "for i in range(len(a)):\n",
    "    print i, a[i]\n",
    "\n",
    "#pythonic way\n",
    "for index, value in enumerate(a):\n",
    "    print index, value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `zip` and Argument Unpacking"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The `zip()` function can be used to iterate over two or more lists in parallel. `zip()` transforms multiple lists into a single list of tuples of corresponding elements. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "list1 and list2 zipped together: [('a', 1), ('b', 2), ('c', 3)]\n",
      "1 3 6\n",
      "2 4 7\n",
      "3 5 8\n",
      "0 a1 b1\n",
      "1 a2 b2\n",
      "2 a3 b3\n"
     ]
    }
   ],
   "source": [
    "#example1:\n",
    "list1 = ['a', 'b', 'c']\n",
    "list2 = [1, 2, 3]\n",
    "print 'list1 and list2 zipped together:', zip(list1, list2)\n",
    "\n",
    "#example2:\n",
    "a = [1, 2, 3]\n",
    "b = [3, 4, 5]\n",
    "c = [6, 7, 8]\n",
    "\n",
    "for i, j, k in zip(a, b, c):\n",
    "    print i, j, k\n",
    "\n",
    "\n",
    "### Use zip() and enumerate() together\n",
    "alist = ['a1', 'a2', 'a3']\n",
    "blist = ['b1', 'b2', 'b3']\n",
    "\n",
    "for i, (a,b) in enumerate(zip(alist, blist)):\n",
    "    print i, a, b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "You can also `unzip` a list using *. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "numbers in list: (1, 2, 3)\n",
      "letters in list: ('one', 'two', 'three')\n",
      "add 1 and 2 by using simple function call: 3\n",
      "add 1 and 2 by passing unpacked list [1,2] to add() method: 3\n"
     ]
    }
   ],
   "source": [
    "pairs = [(1, 'one'), (2, 'two'), (3, 'three')]\n",
    "numbers, letters = zip(*pairs)\n",
    "print 'numbers in list:', numbers\n",
    "print 'letters in list:', letters\n",
    "\n",
    "#using argument unpacking with any function. \n",
    "def add(a, b): return a + b\n",
    "\n",
    "print 'add 1 and 2 by using simple function call:', add(1,2)\n",
    "# print 'add 1 and 2 by passing list [1,2] to add() method:' add([1, 2]) #produces error\n",
    "print 'add 1 and 2 by passing unpacked list [1,2] to add() method:', add(*[1, 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### args and kwargs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "`*args`  allows us to pass variable number of arguments to the function. Let’s take an example to make this clear.\n",
    "\n",
    "In our following example, we first build a function to add two numbers. As you can see the first add() only accepts two numbers, what if you want to pass more than two arguments, this is where *args  comes into play."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "add 3 numbers using add_variable() 6\n",
      "add 6 numbers using add_variable() 24\n"
     ]
    }
   ],
   "source": [
    "#function to add numbers:\n",
    "def add_fixed(a, b):\n",
    "    return a + b\n",
    "\n",
    "def add_variable(*args):\n",
    "    sum = 0\n",
    "    for num in args:\n",
    "        sum += num\n",
    "    return sum\n",
    "\n",
    "# print 'add 3 numbers using add_fixed():', add_fixed(3, 2, 1) #gives an error\n",
    "print 'add 3 numbers using add_variable()', add_variable(3, 2, 1)\n",
    "print 'add 6 numbers using add_variable()', add_variable(3, 2, 1, 5, 6, 7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "**Note:** name of `*args`  is just a convention you can use anything that is a valid identifier. For example, `*myargs` is perfectly valid."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "`**kwargs` allows us to pass variable number of keyword argument like this `func_name(name='tim', team='school')`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('sport', 'football')\n",
      "('name', 'tim')\n",
      "('roll', 19)\n"
     ]
    }
   ],
   "source": [
    "def my_func(**kwargs):\n",
    "    for i, j in kwargs.items():\n",
    "        print(i, j)\n",
    "        \n",
    "        \n",
    "my_func(name='tim', sport='football', roll=19)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## For Further Exploration\n",
    "- Python\n",
    "    - [Official Python.org tutorial](https://docs.python.org/2/tutorial) (good)\n",
    "- IPython\n",
    "    - [Official IPython.org tutorial](https://ipython.org/ipython-doc/2/interactive/tutorial.html) (not quite as good)\n",
    "    - [IPython.org videos](https://ipython.org/videos.html) (better)\n",
    "    - [Python for Data Analysis](http://shop.oreilly.com/product/0636920023784.do?cmp=af-prog-books-videos-lp-na_afp_book_mckinney_cj_12307942_7040302) by Wes McKinney (original author of *pandas*)\n",
    "    \n",
    "Note: Get the code and examples from the book for Chapters 3 - 24 [here](https://github.com/joelgrus/data-science-from-scratch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Finally, thanks for coming and reach out to Meghann or Chai if you have any questions on these notes! "
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}