fcc_cat_dog.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "fcc_cat_dog.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ArieRS/30-Days-of-Code/blob/master/fcc_cat_dog.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XeCllzC77-P5",
        "colab_type": "text"
      },
      "source": [
        "**For Development Only: [Solution](https://colab.research.google.com/drive/12tIgKycuvrOna5tjPtdyBKWawQ8eSqnN)**\n",
        "\n",
        "*Note: You are currently reading this using Google Colaboratory which is a cloud-hosted version of Jupyter Notebook. This is a document containng both text cells for documentation and runnable code cells. If you are unfamiliar with Jupyter Notebook, watch this 3-minute introduction before starting this challenge: https://www.youtube.com/watch?v=inN8seMm7UI*\n",
        "\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "\n",
        "For this challenge, you will complete the code below to classify images of dogs and cats. You will use Tensorflow 2.0 and Karas to create a convolutional neural network that correctly classifies images of cats and dogs at least 63% of the time. (Extra credit if you get it to 70% accuracy!)\n",
        "\n",
        "Some of the code is given to you but some code you must fill in to complete this challenge. Read the instruction in each text cell so you will know what you have to do in each code cell.\n",
        "\n",
        "The first code cell imports the required libraries. The second code cell downloads the data and sets key variables. The third cell is the first place you will write your own code.\n",
        "\n",
        "The structure of the dataset files that are downloaded looks like this (You will notice that the test directory has no subdirectories and the images are not labeled):\n",
        "```\n",
        "cats_and_dogs\n",
        "|__ train:\n",
        "    |______ cats: [cat.0.jpg, cat.1.jpg ...]\n",
        "    |______ dogs: [dog.0.jpg, dog.1.jpg ...]\n",
        "|__ validation:\n",
        "    |______ cats: [cat.2000.jpg, cat.2001.jpg ...]\n",
        "    |______ dogs: [dog.2000.jpg, dog.2001.jpg ...]\n",
        "|__ test: [1.jpg, 2.jpg ...]\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "la_Oz6oLlub6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "try:\n",
        "  # This command only in Colab.\n",
        "  %tensorflow_version 2.x\n",
        "except Exception:\n",
        "  pass\n",
        "import tensorflow as tf\n",
        "\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPooling2D\n",
        "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
        "\n",
        "import os\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jaF8r6aOl48C",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "URL = 'https://github.com/beaucarnes/fcc_python_curriculum/raw/master/cats_and_dogs.zip'\n",
        "\n",
        "path_to_zip = tf.keras.utils.get_file('cats_and_dogs.zip', origin=URL, extract=True)\n",
        "\n",
        "PATH = os.path.join(os.path.dirname(path_to_zip), 'cats_and_dogs')\n",
        "\n",
        "train_dir = os.path.join(PATH, 'train')\n",
        "validation_dir = os.path.join(PATH, 'validation')\n",
        "test_dir = os.path.join(PATH, 'test')\n",
        "\n",
        "# Get number of files in each directory. The train and validation directories\n",
        "# each have the subdirecories \"dogs\" and \"cats\".\n",
        "total_train = sum([len(files) for r, d, files in os.walk(train_dir)])\n",
        "total_val = sum([len(files) for r, d, files in os.walk(validation_dir)])\n",
        "total_test = len(os.listdir(test_dir))\n",
        "\n",
        "# Variables for pre-processing and training.\n",
        "batch_size = 128\n",
        "epochs = 15\n",
        "IMG_HEIGHT = 150\n",
        "IMG_WIDTH = 150"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "54bSbGlK9-6T"
      },
      "source": [
        "Now it is your turn! Set each of the variables below correctly. (They should no longer equal `None`.)\n",
        "\n",
        "Create image generators for each of the three image data sets (train, validation, test). Use `ImageDataGenerator` to read / decode the images and convert them into floating point tensors. Use the `rescale` argument (and no other arguments for now) to rescale the tensors from values between 0 and 255 to values between 0 and 1.\n",
        "\n",
        "For the `*_data_gen` variables, use the `flow_from_directory` method. Pass in the batch size, directory, target size (`(IMG_HEIGHT, IMG_WIDTH)`), class mode, and anything else required. `test_data_gen` will be the trickiest one. For `test_data_gen`, make sure to pass in `shuffle=False` to the `flow_from_directory` method. This will make sure the final predictions stay is in the order that our test expects.\n",
        "\n",
        "After you run the code, the output should look like this:\n",
        "```\n",
        "Found 2000 images belonging to 2 classes.\n",
        "Found 1000 images belonging to 2 classes.\n",
        "Found 50 images belonging to 1 classes.\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EOJFeEfumns6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_image_generator = None\n",
        "validation_image_generator = None\n",
        "test_image_generator = None\n",
        "\n",
        "train_data_gen = None\n",
        "val_data_gen = None\n",
        "test_data_gen = None"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "77TvZVETP_b2",
        "colab_type": "text"
      },
      "source": [
        "The `plotImages` function will be used a few times to plot images. It takes an array of images and a probabilities list, although the probabilities list is optional. This code is given to you. If you created the `train_data_gen` variable correctly, then running the cell below will plot five random training images."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TP0WA8j1mt7Q",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def plotImages(images_arr, probabilities = False):\n",
        "    fig, axes = plt.subplots(len(images_arr), 1, figsize=(5,len(images_arr) * 3))\n",
        "    if probabilities is False:\n",
        "      for img, ax in zip( images_arr, axes):\n",
        "          ax.imshow(img)\n",
        "          ax.axis('off')\n",
        "    else:\n",
        "      for img, probability, ax in zip( images_arr, probabilities, axes):\n",
        "          ax.imshow(img)\n",
        "          ax.axis('off')\n",
        "          if probability > 0.5:\n",
        "              ax.set_title(\"%.2f\" % (probability*100) + \"% dog\")\n",
        "          else:\n",
        "              ax.set_title(\"%.2f\" % ((1-probability)*100) + \"% cat\")\n",
        "    plt.show()\n",
        "\n",
        "sample_training_images, _ = next(train_data_gen)\n",
        "plotImages(sample_training_images[:5])\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GlLF4j7hkxEp",
        "colab_type": "text"
      },
      "source": [
        "Recreate the `train_image_generator` using `ImageDataGenerator`. \n",
        "\n",
        "Since there are a small number of training examples there is a risk of overfitting. One way to fix this problem is by creating more training data from existing training examples by using random transformations.\n",
        "\n",
        "Add 4-6 random transformations as arguments to `ImageDataGenerator`. Make sure to rescale the same as before.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-32RRLY_3voj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_image_generator = None\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JLzS54VmqkRg",
        "colab_type": "text"
      },
      "source": [
        "You don't have to do anything for the next cell. `train_data_gen` is created just like before but with the new `train_image_generator`. Then, a single image is plotted five different times using different variations."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pkwq2LFvqabS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size,\n",
        "                                                     directory=train_dir,\n",
        "                                                     target_size=(IMG_HEIGHT, IMG_WIDTH),\n",
        "                                                     class_mode='binary')\n",
        "\n",
        "augmented_images = [train_data_gen[0][0][0] for i in range(5)]\n",
        "\n",
        "plotImages(augmented_images)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wUPWeKtESzA9",
        "colab_type": "text"
      },
      "source": [
        "In the cell below, create a model for the neural network that outputs class probabilities. It should use the Keras Sequential model. It will probably involve a stack of Conv2D and MaxPooling2D layers and then a fully connected layer on top that is activated by a ReLU activation function.\n",
        "\n",
        "Compile the model passing the arguments to set the optimizer and loss. Also pass in `metrics=['accuracy']` to view training and validation accuracy for each training epoch."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k8aZkwMam4UY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model = Sequential()\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "model.summary()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-rYmKuT1dRdy",
        "colab_type": "text"
      },
      "source": [
        "Use the `fit` method on your `model` to train the network. Make sure to pass in arguments for `x`, `steps_per_epoch`, `epochs`, `validation_data`, and `validation_steps`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1niQDz5x6K7y",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "history = None"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Imux8tZWsWp-",
        "colab_type": "text"
      },
      "source": [
        "Run the next cell to visualize the accuracy and loss of the model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5xS51mB56OAC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "acc = history.history['accuracy']\n",
        "val_acc = history.history['val_accuracy']\n",
        "\n",
        "loss = history.history['loss']\n",
        "val_loss = history.history['val_loss']\n",
        "\n",
        "epochs_range = range(epochs)\n",
        "\n",
        "plt.figure(figsize=(8, 8))\n",
        "plt.subplot(1, 2, 1)\n",
        "plt.plot(epochs_range, acc, label='Training Accuracy')\n",
        "plt.plot(epochs_range, val_acc, label='Validation Accuracy')\n",
        "plt.legend(loc='lower right')\n",
        "plt.title('Training and Validation Accuracy')\n",
        "\n",
        "plt.subplot(1, 2, 2)\n",
        "plt.plot(epochs_range, loss, label='Training Loss')\n",
        "plt.plot(epochs_range, val_loss, label='Validation Loss')\n",
        "plt.legend(loc='upper right')\n",
        "plt.title('Training and Validation Loss')\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2bWrqi-usz3y",
        "colab_type": "text"
      },
      "source": [
        "Now it is time to use your model to predict whether a brand new image is a cat or a dog.\n",
        "\n",
        "In this final cell, get the probability that each test image (from `test_data_gen`) is a dog or a cat. `probabilities` should be a list of integers. \n",
        "\n",
        "Call the `plotImages` function and pass in the test images and the probabilities corresponding to each test image.\n",
        "\n",
        "After your run the cell, you should see all 50 test images with a label showing the percentage sure that the image is a cat or a dog. The accuracy will correspond to the accuracy shown in the graph above (after running the previous cell). More training images could lead to a higher accuracy."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vYrSifOit2aK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "222ZZUup6SVO",
        "colab_type": "text"
      },
      "source": [
        "Run this final cell to see if you passed the challenge or if you need to keep trying."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4IH86Ux_u7TZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "answers =  [1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0,\n",
        "            1, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0,\n",
        "            1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1,\n",
        "            1, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, \n",
        "            0, 0, 0, 0, 0, 0]\n",
        "\n",
        "correct = 0\n",
        "\n",
        "for probability, answer in zip(probabilities, answers):\n",
        "  if round(probability) == answer:\n",
        "    correct +=1\n",
        "\n",
        "percentage_identified = (correct / len(answers))\n",
        "\n",
        "passed_challenge = percentage_identified > 0.63\n",
        "\n",
        "print(f\"Your model correctly identified {rount(percentage_identified, 2)}% of the images of cats and dogs.\")\n",
        "\n",
        "if passed_challenge:\n",
        "  print(\"You passed the challenge!\")\n",
        "else:\n",
        "  print(\"You haven't passed yet. Your model should identify at least 63% of the images. Keep trying. You will get it!\")"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}