boston_housing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#Machine Learning Engineer Nanodegree\n",
    "##Model Evaluation & Validation\n",
    "##Project 1: Predicting Boston Housing Prices\n",
    "\n",
    "Welcome to the first project of the Machine Learning Engineer Nanodegree! In this notebook, some template code has already been written. You will need to implement additional functionality to successfully answer all of the questions for this project. Unless it is requested, do not modify any of the code that has already been included. In this template code, there are four sections which you must complete to successfully produce a prediction with your model. Each section where you will write code is preceded by a **STEP X** header with comments describing what must be done. Please read the instructions carefully!\n",
    "\n",
    "In addition to implementing code, there will be questions that you must answer that relate to the project and your implementation. Each section where you will answer a question is preceded by a **QUESTION X** header. Be sure that you have carefully read each question and provide thorough answers in the text boxes that begin with \"**Answer:**\". Your project submission will be evaluated based on your answers to each of the questions.  \n",
    "\n",
    "A description of the dataset can be found [here](https://archive.ics.uci.edu/ml/datasets/Housing), which is provided by the **UCI Machine Learning Repository**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#Getting Started\n",
    "To familiarize yourself with an iPython Notebook, **try double clicking on this cell**. You will notice that the text changes so that all the formatting is removed. This allows you to make edits to the block of text you see here. This block of text (and mostly anything that's not code) is written using [Markdown](http://daringfireball.net/projects/markdown/syntax), which is a way to format text using headers, links, italics, and many other options! Whether you're editing a Markdown text block or a code block (like the one below), you can use the keyboard shortcut **Shift + Enter** or **Shift + Return** to execute the code or text block. In this case, it will show the formatted text.\n",
    "\n",
    "Let's start by setting up some code we will need to get the rest of the project up and running. Use the keyboard shortcut mentioned above on the following code block to execute it. Alternatively, depending on your iPython Notebook program, you can press the **Play** button in the hotbar. You'll know the code block executes successfully if the message *\"Boston Housing dataset loaded successfully!\"* is printed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Boston Housing dataset loaded successfully!\n"
     ]
    }
   ],
   "source": [
    "# Importing a few necessary libraries\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as pl\n",
    "from sklearn import datasets\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "\n",
    "# Make matplotlib show our plots inline (nicely formatted in the notebook)\n",
    "%matplotlib inline\n",
    "\n",
    "# Create our client's feature set for which we will be predicting a selling price\n",
    "CLIENT_FEATURES = [[11.95, 0.00, 18.100, 0, 0.6590, 5.6090, 90.00, 1.385, 24, 680.0, 20.20, 332.09, 12.13]]\n",
    "\n",
    "# Load the Boston Housing dataset into the city_data variable\n",
    "city_data = datasets.load_boston()\n",
    "\n",
    "# Initialize the housing prices and housing features\n",
    "housing_prices = city_data.target\n",
    "housing_features = city_data.data\n",
    "\n",
    "print \"Boston Housing dataset loaded successfully!\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#Statistical Analysis and Data Exploration\n",
    "In this first section of the project, you will quickly investigate a few basic statistics about the dataset you are working with. In addition, you'll look at the client's feature set in `CLIENT_FEATURES` and see how this particular sample relates to the features of the dataset. Familiarizing yourself with the data through an explorative process is a fundamental practice to help you better understand your results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Step 1\n",
    "In the code block below, use the imported `numpy` library to calculate the requested statistics. You will need to replace each `None` you find with the appropriate `numpy` coding for the proper statistic to be printed. Be sure to execute the code block each time to test if your implementation is working successfully. The print statements will show the statistics you calculate!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Boston Housing dataset statistics (in $1000's):\n",
      "\n",
      "Total number of houses: 506\n",
      "Total number of features: 13\n",
      "Minimum house price: 5.0\n",
      "Maximum house price: 50.0\n",
      "Mean house price: 22.533\n",
      "Median house price: 21.2\n",
      "Standard deviation of house price: 9.188\n"
     ]
    }
   ],
   "source": [
    "# Number of houses in the dataset\n",
    "total_houses = len(housing_prices)\n",
    "\n",
    "# Number of features in the dataset\n",
    "total_features = len(housing_features[0])\n",
    "\n",
    "# Minimum housing value in the dataset\n",
    "minimum_price = np.min(housing_prices)\n",
    "\n",
    "# Maximum housing value in the dataset\n",
    "maximum_price = np.max(housing_prices)\n",
    "\n",
    "# Mean house value of the dataset\n",
    "mean_price = np.mean(housing_prices)\n",
    "\n",
    "# Median house value of the dataset\n",
    "median_price = np.median(housing_prices)\n",
    "\n",
    "# Standard deviation of housing values of the dataset\n",
    "std_dev = np.std(housing_prices)\n",
    "\n",
    "# Show the calculated statistics\n",
    "print \"Boston Housing dataset statistics (in $1000's):\\n\"\n",
    "print \"Total number of houses:\", total_houses\n",
    "print \"Total number of features:\", total_features\n",
    "print \"Minimum house price:\", minimum_price\n",
    "print \"Maximum house price:\", maximum_price\n",
    "print \"Mean house price: {0:.3f}\".format(mean_price)\n",
    "print \"Median house price:\", median_price\n",
    "print \"Standard deviation of house price: {0:.3f}\".format(std_dev)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Question 1\n",
    "As a reminder, you can view a description of the Boston Housing dataset [here](https://archive.ics.uci.edu/ml/datasets/Housing), where you can find the different features under **Attribute Information**. The `MEDV` attribute relates to the values stored in our `housing_prices` variable, so we do not consider that a feature of the data.\n",
    "\n",
    "*Of the features available for each data point, choose three that you feel are significant and give a brief description for each of what they measure.*\n",
    "\n",
    "Remember, you can **double click the text box below** to add your answer!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer: **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Question 2\n",
    "*Using your client's feature set `CLIENT_FEATURES`, which values correspond with the features you've chosen above?*  \n",
    "**Hint: ** Run the code block below to see the client's data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[11.95, 0.0, 18.1, 0, 0.659, 5.609, 90.0, 1.385, 24, 680.0, 20.2, 332.09, 12.13]]\n"
     ]
    }
   ],
   "source": [
    "print CLIENT_FEATURES"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer: **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#Evaluating Model Performance\n",
    "In this second section of the project, you will begin to develop the tools necessary for a model to make a prediction. Being able to accurately evaluate each model's performance through the use of these tools helps to greatly reinforce the confidence in your predictions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Step 2\n",
    "In the code block below, you will need to implement code so that the `shuffle_split_data` function does the following:\n",
    "- Randomly shuffle the input data `X` and target labels (housing values) `y`.\n",
    "- Split the data into training and testing subsets, holding 30% of the data for testing.\n",
    "\n",
    "If you use any functions not already acessible from the imported libraries above, remember to include your import statement below as well!  \n",
    "Ensure that you have executed the code block once you are done. You'll know if the `shuffle_split_data` function is working if the statement *\"Successfully shuffled and split the data!\"* is printed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Successfully shuffled and split the data!\n"
     ]
    }
   ],
   "source": [
    "# Put any import statements you need for this code block here\n",
    "from sklearn import cross_validation\n",
    "\n",
    "def shuffle_split_data(X, y):\n",
    "    \"\"\" Shuffles and splits data into 70% training and 30% testing subsets,\n",
    "        then returns the training and testing subsets. \"\"\"\n",
    "\n",
    "    # Shuffle and split the data\n",
    "    X_train, X_test, y_train, y_test = cross_validation.train_test_split(X, y, test_size=0.30)\n",
    "    \n",
    "    # Return the training and testing data subsets\n",
    "    return X_train, y_train, X_test, y_test\n",
    "\n",
    "\n",
    "# Test shuffle_split_data\n",
    "try:\n",
    "    X_train, y_train, X_test, y_test = shuffle_split_data(housing_features, housing_prices)\n",
    "    print \"Successfully shuffled and split the data!\"\n",
    "except:\n",
    "    print \"Something went wrong with shuffling and splitting the data.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Question 4\n",
    "*Why do we split the data into training and testing subsets for our model?*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer: **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Step 3\n",
    "In the code block below, you will need to implement code so that the `performance_metric` function does the following:\n",
    "- Perform a total error calculation between the true values of the `y` labels `y_true` and the predicted values of the `y` labels `y_predict`.\n",
    "\n",
    "You will need to first choose an appropriate performance metric for this problem. See [the sklearn metrics documentation](http://scikit-learn.org/stable/modules/classes.html#sklearn-metrics-metrics) to view a list of available metric functions. **Hint: ** Look at the question below to see a list of the metrics that were covered in the supporting course for this project.\n",
    "\n",
    "Once you have determined which metric you will use, remember to include the necessary import statement as well!  \n",
    "Ensure that you have executed the code block once you are done. You'll know if the `performance_metric` function is working if the statement *\"Successfully performed a metric calculation!\"* is printed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Something went wrong with performing a metric calculation.\n"
     ]
    }
   ],
   "source": [
    "# Put any import statements you need for this code block here\n",
    "from sklearn.metrics import mean_absolute_error\n",
    "\n",
    "def performance_metric(label, prediction):\n",
    "    \"\"\" Calculates and returns the total error between true and predicted values\n",
    "        based on a performance metric chosen by the student. \"\"\"\n",
    "\n",
    "    meanAbsoluteError = mean_absolute_error(label, prediction)\n",
    "    \n",
    "    return meanAbsoluteError\n",
    "\n",
    "\n",
    "# Test performance_metric\n",
    "try:\n",
    "    total_error = performance_metric(label, prediction)\n",
    "    print \"Successfully performed a metric calculation!\"\n",
    "except:\n",
    "    print \"Something went wrong with performing a metric calculation.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Question 4\n",
    "*Which performance metric below did you find was most appropriate for predicting housing prices and analyzing the total error. Why?*\n",
    "- *Accuracy*\n",
    "- *Precision*\n",
    "- *Recall*\n",
    "- *F1 Score*\n",
    "- *Mean Squared Error (MSE)*\n",
    "- *Mean Absolute Error (MAE)*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer: **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Step 4 (Final Step)\n",
    "In the code block below, you will need to implement code so that the `fit_model` function does the following:\n",
    "- Create a scoring function using the same performance metric as in **Step 2**. See the [sklearn `make_scorer` documentation](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.make_scorer.html).\n",
    "- Build a GridSearchCV object using `regressor`, `parameters`, and `scoring_function`. See the [sklearn documentation on GridSearchCV](http://scikit-learn.org/stable/modules/generated/sklearn.grid_search.GridSearchCV.html).\n",
    "\n",
    "When building the scoring function and GridSearchCV object, *be sure that you read the parameters documentation thoroughly.* It is not always the case that a default parameter for a function is the appropriate setting for the problem you are working on.\n",
    "\n",
    "Since you are using `sklearn` functions, remember to include the necessary import statements below as well!  \n",
    "Ensure that you have executed the code block once you are done. You'll know if the `fit_model` function is working if the statement *\"Successfully fit a model to the data!\"* is printed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Put any import statements you need for this code block\n",
    "\n",
    "def fit_model(X, y):\n",
    "    \"\"\" Tunes a decision tree regressor model using GridSearchCV on the input data X \n",
    "        and target labels y and returns this optimal model. \"\"\"\n",
    "\n",
    "    # Create a decision tree regressor object\n",
    "    regressor = DecisionTreeRegressor()\n",
    "\n",
    "    # Set up the parameters we wish to tune\n",
    "    parameters = {'max_depth':(1,2,3,4,5,6,7,8,9,10)}\n",
    "\n",
    "    # Make an appropriate scoring function\n",
    "    scoring_function = None\n",
    "\n",
    "    # Make the GridSearchCV object\n",
    "    reg = None\n",
    "\n",
    "    # Fit the learner to the dataset to obtain the optimal model with tuned parameters\n",
    "    reg.fit(X, y)\n",
    "\n",
    "    # Return the optimal model\n",
    "    return reg\n",
    "\n",
    "\n",
    "# Test fit_model\n",
    "try:\n",
    "    reg = fit_model(housing_features, housing_prices)\n",
    "    print \"Successfully fit a model!\"\n",
    "except:\n",
    "    print \"Something went wrong with fitting a model.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Question 5\n",
    "*What is the grid search algorithm and when is it applicable?*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer: **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Question 6\n",
    "*What is cross-validation, and how is it performed on a model? Why would cross-validation be helpful when using grid search?*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer: **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#Checkpoint!\n",
    "You have now successfully completed your last code implementation section. Pat yourself on the back! All of your functions written above will be executed in the remaining sections below, and questions will be asked about various results for you to analyze. To prepare the **Analysis** and **Prediction** sections, you will need to intialize the two functions below. Remember, there's no need to implement any more code, so sit back and execute the code blocks! Some code comments are provided if you find yourself interested in the functionality."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def learning_curves(X_train, y_train, X_test, y_test):\n",
    "    \"\"\" Calculates the performance of several models with varying sizes of training data.\n",
    "        The learning and testing error rates for each model are then plotted. \"\"\"\n",
    "    \n",
    "    print \"Creating learning curve graphs for max_depths of 1, 3, 6, and 10. . .\"\n",
    "    \n",
    "    # Create the figure window\n",
    "    fig = pl.figure(figsize=(10,8))\n",
    "\n",
    "    # We will vary the training set size so that we have 50 different sizes\n",
    "    sizes = np.round(np.linspace(1, len(X_train), 50))\n",
    "    train_err = np.zeros(len(sizes))\n",
    "    test_err = np.zeros(len(sizes))\n",
    "\n",
    "    # Create four different models based on max_depth\n",
    "    for k, depth in enumerate([1,3,6,10]):\n",
    "        \n",
    "        for i, s in enumerate(sizes):\n",
    "            \n",
    "            # Setup a decision tree regressor so that it learns a tree with max_depth = depth\n",
    "            regressor = DecisionTreeRegressor(max_depth = depth)\n",
    "            \n",
    "            # Fit the learner to the training data\n",
    "            regressor.fit(X_train[:s], y_train[:s])\n",
    "\n",
    "            # Find the performance on the training set\n",
    "            train_err[i] = performance_metric(y_train[:s], regressor.predict(X_train[:s]))\n",
    "            \n",
    "            # Find the performance on the testing set\n",
    "            test_err[i] = performance_metric(y_test, regressor.predict(X_test))\n",
    "\n",
    "        # Subplot the learning curve graph\n",
    "        ax = fig.add_subplot(2, 2, k+1)\n",
    "        ax.plot(sizes, test_err, lw = 2, label = 'Testing Error')\n",
    "        ax.plot(sizes, train_err, lw = 2, label = 'Training Error')\n",
    "        ax.legend()\n",
    "        ax.set_title('max_depth = %s'%(depth))\n",
    "        ax.set_xlabel('Number of Data Points in Training Set')\n",
    "        ax.set_ylabel('Total Error')\n",
    "        ax.set_xlim([0, len(X_train)])\n",
    "    \n",
    "    # Visual aesthetics\n",
    "    fig.suptitle('Decision Tree Regressor Learning Performances', fontsize=18, y=1.03)\n",
    "    fig.tight_layout()\n",
    "    fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def model_complexity(X_train, y_train, X_test, y_test):\n",
    "    \"\"\" Calculates the performance of the model as model complexity increases.\n",
    "        The learning and testing errors rates are then plotted. \"\"\"\n",
    "    \n",
    "    print \"Creating a model complexity graph. . . \"\n",
    "\n",
    "    # We will vary the max_depth of a decision tree model from 1 to 14\n",
    "    max_depth = np.arange(1, 14)\n",
    "    train_err = np.zeros(len(max_depth))\n",
    "    test_err = np.zeros(len(max_depth))\n",
    "\n",
    "    for i, d in enumerate(max_depth):\n",
    "        # Setup a Decision Tree Regressor so that it learns a tree with depth d\n",
    "        regressor = DecisionTreeRegressor(max_depth = d)\n",
    "\n",
    "        # Fit the learner to the training data\n",
    "        regressor.fit(X_train, y_train)\n",
    "\n",
    "        # Find the performance on the training set\n",
    "        train_err[i] = performance_metric(y_train, regressor.predict(X_train))\n",
    "\n",
    "        # Find the performance on the testing set\n",
    "        test_err[i] = performance_metric(y_test, regressor.predict(X_test))\n",
    "\n",
    "    # Plot the model complexity graph\n",
    "    pl.figure(figsize=(7, 5))\n",
    "    pl.title('Decision Tree Regressor Complexity Performance')\n",
    "    pl.plot(max_depth, test_err, lw=2, label = 'Testing Error')\n",
    "    pl.plot(max_depth, train_err, lw=2, label = 'Training Error')\n",
    "    pl.legend()\n",
    "    pl.xlabel('Maximum Depth')\n",
    "    pl.ylabel('Total Error')\n",
    "    pl.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#Analyzing Model Performance\n",
    "In this third section of the project, you'll take a look at several models' learning and testing error rates on various subsets of training data. Additionally, you'll investigate one particular algorithm with an increasing `max_depth` parameter on the full training set to observe how model complexity affects learning and testing errors. Graphing your model's performance based on varying criteria can be beneficial in the analysis process, such as visualizing behavior that may not have been apparent from the results alone."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "learning_curves(X_train, y_train, X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Question 7\n",
    "*Choose one of the learning curve graphs that are created above. What is the max depth for the chosen model? As the size of the training set increases, what happens to the training error? What happens to the testing error?*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer: **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Question 8\n",
    "*Look at the learning curve graphs for the model with a max depth of 1 and a max depth of 10. When the model is using the full training set, does it suffer from high bias or high variance when the max depth is 1? What about when the max depth is 10?*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer: **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model_complexity(X_train, y_train, X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Question 9\n",
    "*From the model complexity graph above, describe the training and testing errors as the max depth increases. Based on your interpretation of the graph, which max depth results in a model that best generalizes the dataset? Why?*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer: **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#Model Prediction\n",
    "In this final section of the project, you will make a prediction on the client's feature set using an optimized model from `fit_model`. *To answer the following questions, it is recommended that you run the code blocks several times and use the median or mean value of the results.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Question 10\n",
    "*Using grid search, what is the optimal `max_depth` parameter for your model? How does this result compare to your intial intuition?*  \n",
    "**Hint: ** Run the code block below to see the max depth produced by your optimized model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "print \"Final model optimal parameters:\", reg.best_params_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer: **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Question 11\n",
    "*With your parameter-tuned model, what is the best selling price for your client's home? How does this selling price compare to the basic statistics you calculated on the dataset?*  \n",
    "\n",
    "**Hint: ** Run the code block below to have your parameter-tuned model make a prediction on the client's home."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "sale_price = reg.predict(CLIENT_FEATURES)\n",
    "print \"Predicted value of client's home: {0:.3f}\".format(sale_price[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer: **"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Question 12 (Final Question):\n",
    "*In a few sentences, discuss whether you would use this model or not to predict the selling price of future clients' homes in the Greater Boston area.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Answer: **"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}