Data_Science_Making_Predictions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Science: Making Predictions\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn\n",
    "%matplotlib inline \n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE**: If running the cell above produces a `ModuleNotFoundError` or and `ImportError`, you will need to install the missing package using [these instructions](https://docs.anaconda.com/anaconda/user-guide/tasks/install-packages/). E.g. If you get the error `ModuleNotFoundError: No module named 'seaborn'`, then you can fix it by installing the package named \"seaborn\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1- Introduction\n",
    "### We can use data to learn, infer or predict future outcomes\n",
    "---\n",
    "__Prediction:__ With Data Science we make predictions by using the past outcome information.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Your Turn__: What you would like to predict?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Past information from Titanic dataset:__ We know who survided or not and its characteristics, e.g. gender, age and class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Prediction:__ Imagine we build another Titanic, an accident happens and Titanic starts to sink.  \n",
    "We can predict if you would survive or not based on the past outcomes from the first accident (in the first Titanic).  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### We will use a reduced Titanic dataset for this task"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we load in the data from a CSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pandas.read_csv('data/reduced_titanic.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 - Data Science Models\n",
    "### Learning from the data to make a prediction\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Data Science Models:__\n",
    "They use a set of rules and learns from the data to produce a result of interest  \n",
    "For example, if a new passanger would survive based on his/her characteristics...\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/DT_Titanic.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decision Trees \n",
    "### Models possible consequences for the observations using a tree structure\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/tree.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* A decision tree uses an upside down tree structure to represent a number of decision paths, and an outcome for each path   \n",
    "* Decision trees are sort of a more complex \"if-and-else statement\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/tree2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On the code below we can choose the \"path\" or part of code to use depending on a \"decision\" or condition."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Your Turn__: Run the code below  \n",
    "If it is cold today, which code block are we accessing in the code below? \n",
    "Which part of the code we access if we writte sunny instead?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What is happening to the weather today?\n",
    "\n",
    "weather = input(\"What is the weather like: \")\n",
    "\n",
    "if weather == \"raining\":\n",
    "    print(\"Take an umbrella\")\n",
    "    \n",
    "elif weather == \"cold\":\n",
    "    print(\"Take a coat\")\n",
    "    \n",
    "elif weather == \"sunny\":\n",
    "    print(\"Wear sunscreen\")\n",
    "    \n",
    "else:\n",
    "    print(\"Have a good day, whatever the weather!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Back to decision trees: Have you played Guess who game?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/guesswho1.JPG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Your Turn__: What questions would eliminate most candidates?\n",
    "\n",
    "a) Is it efficient to ask \"Does he/she has a big nose\"?\n",
    "\n",
    "b) Is it efficient to ask \"Is he/she bald\"? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Decision Tree Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " __Guess who game and feature ordering in a decision tree__\n",
    "\n",
    "Just like how much information you can get depending on what are your first  questions in the game above, \n",
    "efficiency on trainning the model is mostly dictated by which feature goes on the top node, second level nodes, and so on.  \n",
    "Information gain strategies: http://www.saedsayad.com/decision_tree.htm\n",
    "\n",
    "Decision tree models are used to predict the value of a target variable based on several features (input variables).   \n",
    "Each interior node corresponds to one of the features. Branches represent feature values.  \n",
    "Each leaf represents a possible outcome of the target. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/forecast.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 3- Training our Model\n",
    "### Since we don't have new data we can split our data into training and test sets:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* training (to train our model)\n",
    "* test (used as new data to test our model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The test data can check if our model works well for a new dataset, how close our predictions get to the real results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we split the data and drop the results column \"Survived\" since that is what we aim to predict "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split the data into X_train, X_test, y_train, and y_test\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "                                   data.drop('Survived', axis=1), \n",
    "                                   data['Survived'], \n",
    "                                   test_size=0.33, \n",
    "                                   random_state=42)\n",
    "X_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we train our model using the training data. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import tree\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "decision_tree = DecisionTreeClassifier()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training our model with two features, Gender and Class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "features = ['Gender', 'Class']\n",
    "model = decision_tree.fit(X_train[features], y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# How good is the model?\n",
    "model.score(X_test[features], y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4- Making Predictions\n",
    "### Let's check if you would survive or not... :O"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " \n",
    "__Your Turn:__ For the cells below run all for different cases:   \n",
    "e.g. use female (denominated by 0) and class 1  \n",
    "or use male (denominated by 1) and class 3   \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Gender = ? \n",
    "Class = ? # how much can you pay for the tickets?  \n",
    "\n",
    "Result = decision_tree.predict([[Gender, Class]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "Survive_Prob = decision_tree.predict_proba([[Gender, Class]])[0,1]*100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Would our new passanger survive or not?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "####################### Decision Tree Prediction #######################\n",
    "if Result==1: \n",
    "    print('He/She has a {0:.2f}% chance of survival! :)'.format(Survive_Prob))\n",
    "else: \n",
    "    print('He/She has a {0:.2f}% chance of survival :('.format(Survive_Prob))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see the model is predicting what would happend to a new person based on what happened to all the passengers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "__Exercise__: Train your model using almost all the features: 'Class','Gender','Age','Parents', 'Fare'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "features = ['Gender', # complete this list]\n",
    "model = decision_tree.fit(# give the paremeters needed here)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# How good is our model? Is it better or worse than the one with less features? Why?\n",
    "model.score(X_test[features], y_test)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}