digit-recognizer-10.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Digit recognition system is the working of a machine to train itself or recognizing the digits from different sources like emails, bank cheque, papers, images, etc. and in different real-world scenarios for online handwriting recognition on computer tablets or system, recognize number plates of vehicles, processing bank cheque amounts, numeric entries in forms filled up by hand (say-tax forms) and so on. The handwritten digits are not always of same size, width, orientation and justified to margins as they differ from writing of person to person, so the general problem would be while classifying the digits due to the similarity between digits such as 1 and 7,5 and 6,3 and 8,2 and 5,2 and 7,etc. This problem is faced more when many people write a single digit with a variety of different handwritings. Lastly, the uniqueness and variety in the handwriting of different individuals also influence the formation and appearance of the digits.\n",
    "\n",
    "This dataset includes handwriting digits total of 60,000 images consisting of 49,000 examples in training set with labelled images from 10 digits (0 to 9) and 21,000 examples in testing set which are unlabelled images.\n",
    "\n",
    "Handwritten digits are images in the form of 28*28 gray scale intensities of images representing an image. The size of an image is 28 by 28, so there are 784 (28*28) values for the label. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 to 255 , inclusive. Our goal is to correctly identify digits from a dataset of tons of thousands of handwritten images.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5"
   },
   "outputs": [],
   "source": [
    "# This Python 3 environment comes with many helpful analytics libraries installed\n",
    "# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n",
    "# For example, here's several helpful packages to load\n",
    "\n",
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
    "\n",
    "# Input data files are available in the read-only \"../input/\" directory\n",
    "# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n",
    "\n",
    "import os\n",
    "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))\n",
    "\n",
    "# You can write up to 5GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n",
    "# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as mpimg\n",
    "import seaborn as sns\n",
    "%matplotlib inline\n",
    "\n",
    "np.random.seed(2)\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import confusion_matrix\n",
    "import itertools\n",
    "\n",
    "from keras.utils.np_utils import to_categorical # convert to one-hot-encoding\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D\n",
    "from keras.optimizers import RMSprop\n",
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "from keras.callbacks import ReduceLROnPlateau\n",
    "\n",
    "\n",
    "sns.set(style='white', context='notebook', palette='deep')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
    "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://s3.amazonaws.com/img-datasets/mnist.npz\n",
      "11493376/11490434 [==============================] - 0s 0us/step\n"
     ]
    }
   ],
   "source": [
    "#loading the dataset\n",
    "from keras.datasets import mnist\n",
    "(x_train,y_train),(x_test,y_test)=mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD7CAYAAAClmULcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAMPUlEQVR4nO3dX8hV9Z7H8ffTc2AugppBap7+oRn5HYhOkyGB4gkGBoU4MKeiEsubuUgLujmEWNiNRNp0M6Vi3kmZdBFENyqci+qIFF0kgyf6VicNqSTHaiI4eWGei2c5mDzP2o/uvfbe+n2/YLMef1/W3l9+8HHtvX577TVx5swZJNVzxagbkDQahl8qyvBLRRl+qSjDLxX1m1G9cET8A7AE+AY4Pao+pMvYJHAd8GFmnjq/2Hf4I2IRsAuYB5wE1mTmZ3PYdQnw535fX1JPy4ED5w8O4si/A9iWma9FxCPAK8C/zWG/bwB2797N1NTUANqQdK7jx4+zevVqaLJ2vr7CHxHXAouBf2+G9gBbI+KazDzRY/fTAFNTU9x44439tCGp3Ywfq/s94XcT8FVmngZotl8345LGmGf7paL6Df8x4IaImARottc345LGWF/hz8xvgUPAqmZoFfDRHD7vSxqxQZztXwvsiohnge+BNQN4Tkkd6zv8mfkJcPcAepE0RJ7wk4oy/FJRhl8qyvBLRRl+qSjDLxVl+KWiDL9UlOGXijL8UlGGXyrK8EtFGX6pKMMvFWX4paIMv1SU4ZeKMvxSUYZfKsrwS0UZfqkowy8VZfilogy/VJThl4oy/FJRhl8qyvBLRRl+qSjDLxXV9y26I+Io8HPzAFifmfv7fV5J3eo7/I0HMvPwgJ5L0hD4tl8qalBH/t0RMQEcAJ7OzB8G9LySOjKII//yzLwDWAJMAFsH8JySOtZ3+DPzWLM9BWwHlvX7nJK611f4I+LKiLi6+XsCeBg4NIjGJHWr38/8/wy8GRGTwCTwMfB4311JHfjxxx9nrW3fvr1133fffbe1vm/fvtb6ypUrW+t79+5trXehr/Bn5hfAnQPqRdIQudQnFWX4paIMv1SU4ZeKMvxSUYP6eq/Ut4MHD7bW33vvvdZ6v8txXVqwYMHIXns2Hvmlogy/VJThl4oy/FJRhl8qyvBLRRl+qSjX+XVB2i6Lhd6Xxu7cuXPW2pEjRy6qp0Hodcntxo0bW+tLly4dZDtD4ZFfKsrwS0UZfqkowy8VZfilogy/VJThl4pynV8XZP369a31HTt2XPRzr127trX+6KOPXvRzw6W5Ft8lj/xSUYZfKsrwS0UZfqkowy8VZfilogy/VJTr/PqVzZs3t9b379/fWu+1Vv/UU0/NWlu4cGHrvhqsnuGPiBeB+4EFwO2ZebgZXwTsAuYBJ4E1mflZd61KGqS5vO1/C/gd8OV54zuAbZm5CNgGvDLg3iR1qGf4M/NAZh47dywirgUWA3uaoT3A4oi4ZvAtSurCxZ7wuwn4KjNPAzTbr5txSZcAz/ZLRV1s+I8BN0TEJECzvb4Zl3QJuKjwZ+a3wCFgVTO0CvgoM08MqjFJ3ZrLUt9LwH3AFPCniDiZmbcBa4FdEfEs8D2wptNONRAHDx5srW/YsKG13uv37bds2dJav+qqq1rrGp6e4c/MJ4EnZxj/BLi7i6Ykdc8TflJRhl8qyvBLRRl+qSjDLxXlJb3FbNq0qa/977nnntb64cOHW+tTU1Oz1rykd7g88ktFGX6pKMMvFWX4paIMv1SU4ZeKMvxSUa7zX4bWrVs3a23fvn2t+/a6ZHf+/Pmt9UceeaS1vmLFillrXg48XB75paIMv1SU4ZeKMvxSUYZfKsrwS0UZfqko1/nH0BdffNFaf+KJJ1rrbWv5r7/+euu+9957b2u911r7l1+efz/XX2v7afBe6/waLI/8UlGGXyrK8EtFGX6pKMMvFWX4paIMv1SU6/xj6IMPPmit97omv20tf9WqVRfVky4/PcMfES8C9wMLgNsz83AzfhT4uXkArM/M/Z10KWng5nLkfwv4b+DPM9QeOPufgaRLS8/wZ+YBgIjovhtJQ9PvZ/7dETEBHACezswfBtCTpCHo52z/8sy8A1gCTABbB9OSpGG46PBn5rFmewrYDiwbVFOSundR4Y+IKyPi6ubvCeBh4NAgG5PUrbks9b0E3AdMAX+KiJPA74E3I2ISmAQ+Bh7vstFKeq3F93vNfZd27tw5stfWhZnL2f4ngSdnKN05+HYkDYtf75WKMvxSUYZfKsrwS0UZfqkoL+m9BI1yKW/Pnj2t9SNHjrTWn3/++Vlr3oJ7uDzyS0UZfqkowy8VZfilogy/VJThl4oy/FJRrvPP4uDBg631pUuXDqmT4eq1jv/MM8+01m+++ebW+uOPe+X3uPDILxVl+KWiDL9UlOGXijL8UlGGXyrK8EtFuc4/i2XL2u9BsnLlyllrGzdubN236+8IbN68edbahg0b+nrutWvXtta3bNnSWvea/fHhkV8qyvBLRRl+qSjDLxVl+KWiDL9UlOGXinKdfxa9rkvft2/frLXMbN13xYoVrfX9+/e31nv9Nn6btu8nALzxxhutddfpLx89wx8R84BXgVuAU8DnwGOZeSIiFgG7gHnASWBNZn7WYb+SBmQub/vPAC9kZmTmb4G/Ame/QrYD2JaZi4BtwCvdtClp0HqGPzO/y8x3zhl6H5gfEdcCi4Gzv/u0B1gcEdcMvEtJA3dBJ/wi4gpgHfA2cBPwVWaeBmi2XzfjksbchZ7tfxn4CdjaQS+ShmjO4Y+IF4FbgYcy8xfgGHBDREw29Ung+mZc0pib01JfRDwH3AXcm5mnADLz24g4BKwCXmu2H2Xmia6aHabXXnuttb5p06ZZa23LgAA7duxorfe6bHb+/Pmt9QcffHDW2sKFC1v3VR1zWeq7DXga+BQ4GBEARzLzD8BaYFdEPAt8D6zpsFdJA9Qz/Jn5F2BiltonwN2DbkpS9/x6r1SU4ZeKMvxSUYZfKsrwS0V5Se8sev289t69e4fUidQNj/xSUYZfKsrwS0UZfqkowy8VZfilogy/VJThl4oy/FJRhl8qyvBLRRl+qSjDLxVl+KWiDL9UlOGXijL8UlGGXyrK8EtFGX6pKMMvFWX4paIMv1TUXG7RPQ94FbgFOAV8DjyWmSci4ijwc/MAWJ+Z+7tpVdIgzeWmHWeAFzLzHYCI+C9gM/CfTf2BzDzcTXuSutIz/Jn5HfDOOUPvA+u6akjScFzQ7boi4gqmg//2OcO7I2ICOAA8nZk/DLA/SR250BN+LwM/AVubfy/PzDuAJcDEOeOSxtycwx8RLwK3Ag9l5i8AmXms2Z4CtgPLumhS0uDN6W1/RDwH3AXc2wSdiLgS+E1m/l/ztv9h4FBnnUoaqLks9d0GPA18ChyMCIAjwB+BNyNiEpgEPgYe765VSYM0l7P9f2H68/xM7hxsO5KGxW/4SUUZfqkowy8VZfilogy/VJThl4oy/FJRhl8qyvBLRRl+qSjDLxVl+KWiDL9U1AX9jNeATQIcP358hC1Il69zsjU5U32U4b8OYPXq1SNsQSrhOuCv5w+OMvwfAsuBb4DTI+xDulxNMh38D2cqTpw5c2a47UgaC57wk4oy/FJRhl8qyvBLRRl+qSjDLxVl+KWiRvkln/8XEYuAXcA84CSwJjM/G21X0yLiKPBz8wBYn5n7R9DHi8D9wALg9sw83IyPfO5aejvKiOcuIuYBrwK3AKeAz4HHMvPEqOeuR29H6XjuxuXIvwPYlpmLgG3AKyPu53wPZOa/No+hB7/xFvA74Mvzxsdh7mbrDUY/d2eAFzIzMvO3TH/NdXNTG/XctfUGHc/dyMMfEdcCi4E9zdAeYHFEXDO6rsZPZh44e1fks8Zl7mbqbVxk5neZ+c45Q+8D88dh7mbrbVivP/LwAzcBX2XmaYBm+3UzPi52R8T/RMT2iPjHUTdzDufuAkTEFcA64G3GbO7O6+2sTuduHMI/7pZn5h3AEqZvWLp1xP1cSsZt7l4GfhqDPmZyfm+dz904hP8YcENzq2+a7fXN+MidfTubmaeA7cCy0Xb0K87dHDUnJW8FHsrMXxijuZuht6HM3cjDn5nfAoeAVc3QKuCjzDwxuq6mRcSVEXF18/cE8DDTvY4F527OvTwH3AX8RxOmsZm7mXob1tyNxSW9EfEvTC+5/BPwPdNLLjnariAiFgJvMn1d9CTwMfBkZn4zgl5eAu4DpoD/BU5m5m3jMHcz9Qb8njGYu4i4DTgMfAr8rRk+kpl/GPXczdYb8EeGMHdjEX5Jwzfyt/2SRsPwS0UZfqkowy8VZfilogy/VJThl4oy/FJRfwdGjZYoHNPRPwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#viewing the image of the digit at index 35\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt \n",
    "image_index=35\n",
    "print(y_train[image_index])\n",
    "plt.imshow(x_train[image_index],cmap='Greys')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60000, 28, 28)\n",
      "(10000, 28, 28)\n"
     ]
    }
   ],
   "source": [
    "#checking the shape of the training data\n",
    "print(x_train.shape)\n",
    "print(x_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('uint8')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#checking the type of the data\n",
    "type(x_train)\n",
    "x_train.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data type is unsigned 8-bit integer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#checking for null values in training data\n",
    "np.isnan(x_train).any()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#checking for null values in test data\n",
    "np.isnan(x_test).any()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#checking for null values in training labels data\n",
    "np.isnan(y_train).any()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#checking for null values in test labels data\n",
    "np.isnan(y_test).any()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hence,there are no missing values in the whole dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 784)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#reshaping 3-D array to 2-D array\n",
    "x_train1=x_train.reshape(60000,784)\n",
    "x_train1.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 784)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#converting the numpy array into pandas Datframe for further analysis\n",
    "df=pd.DataFrame.from_records(x_train1)\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>774</th>\n",
       "      <th>775</th>\n",
       "      <th>776</th>\n",
       "      <th>777</th>\n",
       "      <th>778</th>\n",
       "      <th>779</th>\n",
       "      <th>780</th>\n",
       "      <th>781</th>\n",
       "      <th>782</th>\n",
       "      <th>783</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 784 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   0    1    2    3    4    5    6    7    8    9    ...  774  775  776  777  \\\n",
       "0    0    0    0    0    0    0    0    0    0    0  ...    0    0    0    0   \n",
       "1    0    0    0    0    0    0    0    0    0    0  ...    0    0    0    0   \n",
       "2    0    0    0    0    0    0    0    0    0    0  ...    0    0    0    0   \n",
       "3    0    0    0    0    0    0    0    0    0    0  ...    0    0    0    0   \n",
       "4    0    0    0    0    0    0    0    0    0    0  ...    0    0    0    0   \n",
       "\n",
       "   778  779  780  781  782  783  \n",
       "0    0    0    0    0    0    0  \n",
       "1    0    0    0    0    0    0  \n",
       "2    0    0    0    0    0    0  \n",
       "3    0    0    0    0    0    0  \n",
       "4    0    0    0    0    0    0  \n",
       "\n",
       "[5 rows x 784 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>774</th>\n",
       "      <th>775</th>\n",
       "      <th>776</th>\n",
       "      <th>777</th>\n",
       "      <th>778</th>\n",
       "      <th>779</th>\n",
       "      <th>780</th>\n",
       "      <th>781</th>\n",
       "      <th>782</th>\n",
       "      <th>783</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>60000.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>...</td>\n",
       "      <td>60000.000000</td>\n",
       "      <td>60000.000000</td>\n",
       "      <td>60000.000000</td>\n",
       "      <td>60000.000000</td>\n",
       "      <td>60000.000000</td>\n",
       "      <td>60000.0000</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.200433</td>\n",
       "      <td>0.088867</td>\n",
       "      <td>0.045633</td>\n",
       "      <td>0.019283</td>\n",
       "      <td>0.015117</td>\n",
       "      <td>0.0020</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>6.042472</td>\n",
       "      <td>3.956189</td>\n",
       "      <td>2.839845</td>\n",
       "      <td>1.686770</td>\n",
       "      <td>1.678283</td>\n",
       "      <td>0.3466</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>254.000000</td>\n",
       "      <td>254.000000</td>\n",
       "      <td>253.000000</td>\n",
       "      <td>253.000000</td>\n",
       "      <td>254.000000</td>\n",
       "      <td>62.0000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 784 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           0        1        2        3        4        5        6        7    \\\n",
       "count  60000.0  60000.0  60000.0  60000.0  60000.0  60000.0  60000.0  60000.0   \n",
       "mean       0.0      0.0      0.0      0.0      0.0      0.0      0.0      0.0   \n",
       "std        0.0      0.0      0.0      0.0      0.0      0.0      0.0      0.0   \n",
       "min        0.0      0.0      0.0      0.0      0.0      0.0      0.0      0.0   \n",
       "25%        0.0      0.0      0.0      0.0      0.0      0.0      0.0      0.0   \n",
       "50%        0.0      0.0      0.0      0.0      0.0      0.0      0.0      0.0   \n",
       "75%        0.0      0.0      0.0      0.0      0.0      0.0      0.0      0.0   \n",
       "max        0.0      0.0      0.0      0.0      0.0      0.0      0.0      0.0   \n",
       "\n",
       "           8        9    ...           774           775           776  \\\n",
       "count  60000.0  60000.0  ...  60000.000000  60000.000000  60000.000000   \n",
       "mean       0.0      0.0  ...      0.200433      0.088867      0.045633   \n",
       "std        0.0      0.0  ...      6.042472      3.956189      2.839845   \n",
       "min        0.0      0.0  ...      0.000000      0.000000      0.000000   \n",
       "25%        0.0      0.0  ...      0.000000      0.000000      0.000000   \n",
       "50%        0.0      0.0  ...      0.000000      0.000000      0.000000   \n",
       "75%        0.0      0.0  ...      0.000000      0.000000      0.000000   \n",
       "max        0.0      0.0  ...    254.000000    254.000000    253.000000   \n",
       "\n",
       "                777           778         779      780      781      782  \\\n",
       "count  60000.000000  60000.000000  60000.0000  60000.0  60000.0  60000.0   \n",
       "mean       0.019283      0.015117      0.0020      0.0      0.0      0.0   \n",
       "std        1.686770      1.678283      0.3466      0.0      0.0      0.0   \n",
       "min        0.000000      0.000000      0.0000      0.0      0.0      0.0   \n",
       "25%        0.000000      0.000000      0.0000      0.0      0.0      0.0   \n",
       "50%        0.000000      0.000000      0.0000      0.0      0.0      0.0   \n",
       "75%        0.000000      0.000000      0.0000      0.0      0.0      0.0   \n",
       "max      253.000000    254.000000     62.0000      0.0      0.0      0.0   \n",
       "\n",
       "           783  \n",
       "count  60000.0  \n",
       "mean       0.0  \n",
       "std        0.0  \n",
       "min        0.0  \n",
       "25%        0.0  \n",
       "50%        0.0  \n",
       "75%        0.0  \n",
       "max        0.0  \n",
       "\n",
       "[8 rows x 784 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Conclusions from the description**\n",
    "1. Each figure is a 28x28 sized image,so there are 784 columns,each depicting the intensity value of a pixel.\n",
    "2. Many pixels have 0 intensity value.\n",
    "3. Some pixels have values other than 0 as in the last few columns,the mean,std and max are not 0.\n",
    "4. Most of the pixels in each column have 0 value as 25%,75% and the median for all the columns are 0.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Int64Index([4928], dtype='int64')"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#checking the index of a pixel with intensity 254 \n",
    "df[df.iloc[:,774]==254].index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000,)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#checking the shape of the training labels data \n",
    "y_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    5\n",
       "1    0\n",
       "2    4\n",
       "3    1\n",
       "4    9\n",
       "Name: 0, dtype: int64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#converting the training labels data into a dataframe\n",
    "df_y=pd.DataFrame.from_records(y_train.reshape(60000,1))\n",
    "df_y[0].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    6742\n",
       "7    6265\n",
       "3    6131\n",
       "2    5958\n",
       "9    5949\n",
       "0    5923\n",
       "6    5918\n",
       "8    5851\n",
       "4    5842\n",
       "5    5421\n",
       "Name: 0, dtype: int64"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#displaying the count of each value (0,1,2,3,4,5,6,7,8,9) in the labels\n",
    "df_y[0].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD5CAYAAAAqaDI/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAQvUlEQVR4nO3dfZBddX3H8fe6ERRQHrYJMQ+KBvarMBQMZYIVfOiY8keHitUZuoNdrNPSQFtapw+2WkPVofUBRitZmnRa2kUwM7YiinQKpaNT0VrtkIxQmi9RHswDIUuIrVbFuGz/OGedS0y6d8/ePXfJ7/2a2bm7v+85e783e3M+93fOuecOTE1NIUkq13P63YAkqb8MAkkqnEEgSYUzCCSpcAaBJBXOIJCkwi2aaYGIOAW4rWPoBOCFmXlSRAwD48AQsA8Yzczt9XqNal30czRwLvAYMNnNOpIkBoEXAV/LzKc6CzMGQWY+Apw9/XNEfLRjvY3AWGbeHBFvBTYBPzfH2kzOBb7Y5bKSpGe6ALinc2BgNm8oi4ijgF3AhcBO4EFgKDMnI2KQ6tX9acBAk1pmTnTRwyrgG7fccgtLly7tundJKtmePXu49NJLAU7NzG921macERzkF4FdmXlvRJxTfz8JUG/UdwMrqTb2TWozBgH17qClS5eyYsWKWbYvScX7iV3qsz1Y/Hbgxt70IklaCLoOgohYBrwWuKUe2gEsr3ftUN8uq8eb1iRJLZvNjOBtwB2ZuQ8gM/cCW4GRuj4CbMnMiaa1OT0SSVIjszlG8DbgqoPG1gHjEbEe2A+M9qAmSWpR10GQmcOHGNsGrDnM8o1qkqR2+c5iSSqcQSBJhTsig+CHB+Z+5Yle/A5JejaY7RvKnhWOeu4gF/3eZ+b0O26/7o096kaSFrYjckYgSeqeQSBJhTMIJKlwBoEkFc4gkKTCGQSSVDiDQJIKZxBIUuEMAkkqnEEgSYUzCCSpcAaBJBXOIJCkwhkE82iul7L2UtiS2nBEXoZ6oZjr5bC9FLakNjgjkKTCGQSSVDiD4Ajnx3ZKmklXxwgi4nnAR4A3AD8A/i0zL4+IYWAcGAL2AaOZub1ep1FNveXHdkqaSbczgg9RBcBwZp4JvKce3wiMZeYwMAZs6linaU2S1KIZZwQRcRwwCqzIzCmAzHw8IpYAq4G19aKbgQ0RsRgYaFLLzInePCxJUre6mRGsotp9c3VE/EdEfCEizgdWArsycxKgvt1djzetSZJa1k0QLAJeBmzJzJ8B3gncChw3n43pyOKb66SFq5uDxY8CP6LahUNm/ntEPAF8H1geEYOZORkRg8AyYAfV7p8mNR2hFsKb6354YJKjnjvY19+xEHqQDjZjEGTmExHxeap9+nfVZ/wsAR4EtgIjwM317Zbp/fwR0agmzZeFcAbVQugBDCQ9U7eXmFgH3BgR1wEHgF/JzG9HxDpgPCLWA/upDip3rtOkJmmeLZRA0sLQVRBk5kPA6w4xvg1Yc5h1GtUklWGhzEoWyu/oJy86J6kvFsqsZKH00U9eYkKSCmcQSFLhDAJJKpxBIEmFMwgkqXAGgSQVziCQpMIZBJJUOINAkgpnEEhS4QwCSSqcQSBJfdaLD16ay+/wonOS1Gf9vvCdMwJJKpxBIEmFMwgkqXAGgSQVziCQpMIZBJJUOINAkgpnEEhS4bp6Q1lEPAL8oP4CeGdm3hkRw8A4MATsA0Yzc3u9TqOaJKlds5kRvCUzz66/7qzHNgJjmTkMjAGbOpZvWpMktajxJSYiYgmwGlhbD20GNkTEYmCgSS0zJ5r2I0lqZjYzglsi4usRcUNEnACsBHZl5iRAfbu7Hm9akyS1rNsguCAzzwLOpXpFv2H+WpIktamrIMjMHfXtU8ANwKuBHcDyiBgEqG+X1eNNa5Kkls0YBBFxbEQcX38/APwysDUz9wJbgZF60RFgS2ZONK316kFJkrrXzcHik4FP1a/cB4EHgCvr2jpgPCLWA/uB0Y71mtYkSS2aMQgy8yHglYepbQPW9LImSWqX7yyWpMIZBJJUOINAkgpnEEhS4QwCSSqcQSBJhTMIJKlwBoEkFc4gkKTCGQSSVDiDQJIKZxBIUuEMAkkqnEEgSYUzCCSpcAaBJBXOIJCkwhkEklQ4g0CSCmcQSFLhDAJJKtyi2SwcEVcDfwqcmZn3R8QwMA4MAfuA0czcXi/bqCZJalfXM4KIWA2cB3yrY3gjMJaZw8AYsKkHNUlSi7oKgog4mmqDfSUwVY8tAVYDm+vFNgOrI2Jx01oPHo8kaZa6nRG8D7g5Mx/uGFsJ7MrMSYD6dnc93rQmSWrZjEEQEa8CzgVumP92JElt62ZG8Frg5cDDEfEIsAK4E1gFLI+IQYD6dhmwo/5qUpMktWzGIMjMD2Tmssw8JTNPAXYCF2bmJ4GtwEi96AiwJTMnMnNvk1rPHpUkqWuzOn30ENYB4xGxHtgPjPagJklq0ayDoJ4VTH+/DVhzmOUa1SRJ7fKdxZJUOINAkgpnEEhS4QwCSSqcQSBJhTMIJKlwBoEkFc4gkKTCGQSSVDiDQJIKZxBIUuEMAkkqnEEgSYUzCCSpcAaBJBXOIJCkwhkEklQ4g0CSCmcQSFLhDAJJKpxBIEmFMwgkqXCLulkoIm4DXgo8DXwX+O3M3BoRw8A4MATsA0Yzc3u9TqOaJKld3c4ILsvMszLzlcC1wI31+EZgLDOHgTFgU8c6TWuSpBZ1NSPIzP/u+PF44OmIWAKsBtbW45uBDRGxGBhoUsvMibk8GEnS7HV9jCAi/joivgVcA1wGrAR2ZeYkQH27ux5vWpMktazrIMjMX8vMFwPvAj48fy1Jkto067OGMvPjwOuBncDyiBgEqG+XATvqryY1SVLLZgyCiDguIlZ2/HwR8CSwF9gKjNSlEWBLZk5kZqNaLx6QJGl2ujlYfCzw9xFxLDBJFQIXZeZURKwDxiNiPbAfGO1Yr2lNktSiGYMgMx8HzjtMbRuwppc1SVK7fGexJBXOIJCkwhkEklQ4g0CSCmcQSFLhDAJJKpxBIEmFMwgkqXAGgSQVziCQpMIZBJJUOINAkgpnEEhS4QwCSSqcQSBJhTMIJKlwBoEkFc4gkKTCGQSSVDiDQJIKZxBIUuEMAkkq3KKZFoiIIeDjwCrgKeAbwG9k5kREDAPjwBCwDxjNzO31eo1qkqR2dTMjmAI+lJmRmT8NfBP4QF3bCIxl5jAwBmzqWK9pTZLUohlnBJn5JPCFjqGvAFdExBJgNbC2Ht8MbIiIxcBAk1pmTszt4UiSZmtWxwgi4jnAFcBngZXArsycBKhvd9fjTWuSpJbN9mDx9cB3gQ3z0IskqQ+6DoKIuBY4DbgkM58GdgDLI2Kwrg8Cy+rxpjVJUsu6CoKIuAY4B7g4M58CyMy9wFZgpF5sBNiSmRNNa714QJKk2enm9NEzgHcBDwJfjgiAhzPzTcA6YDwi1gP7gdGOVZvWJEkt6uasof+kOtPnULVtwJpe1iRJ7fKdxZJUOINAkgpnEEhS4QwCSSqcQSBJhTMIJKlwBoEkFc4gkKTCGQSSVDiDQJIKZxBIUuEMAkkqnEEgSYUzCCSpcAaBJBXOIJCkwhkEklQ4g0CSCmcQSFLhDAJJKpxBIEmFWzTTAhFxLfBm4BTgzMy8vx4fBsaBIWAfMJqZ2+dSkyS1r5sZwW3Aa4BHDxrfCIxl5jAwBmzqQU2S1LIZZwSZeQ9ARPx4LCKWAKuBtfXQZmBDRCwGBprUMnNizo9GkjRrTY8RrAR2ZeYkQH27ux5vWpMk9YEHiyWpcE2DYAewPCIGAerbZfV405okqQ8aBUFm7gW2AiP10AiwJTMnmtaaPgBJ0tx0c/rox4BfApYCd0fEvsw8A1gHjEfEemA/MNqxWtOaJKll3Zw1dBVw1SHGtwFrDrNOo5okqX0eLJakwhkEklQ4g0CSCmcQSFLhDAJJKpxBIEmFMwgkqXAGgSQVziCQpMIZBJJUOINAkgpnEEhS4QwCSSqcQSBJhTMIJKlwBoEkFc4gkKTCGQSSVDiDQJIKZxBIUuEMAkkqnEEgSYVb1K87johhYBwYAvYBo5m5vV/9SFKp+jkj2AiMZeYwMAZs6mMvklSsvswIImIJsBpYWw9tBjZExOLMnJhh9UGAPXv2/L8LHfjek3PqcefOnXNavxd9LIQeFkofC6GHXvWxEHpYKH0shB4WSh/z3UPHNnPw4NrA1NTUnO68iYg4B7gpM8/oGHsAeGtm3jvDuucDX5znFiXpSHVBZt7TOdC3YwRz8DXgAuAxYLLPvUjSs8Ug8CKqbegz9GtGsAR4EBjKzMmIGKQ6YHxaF7uGJEk91JeDxZm5F9gKjNRDI8AWQ0CS2teXGQFARLyc6vTRE4H9VKePZl+akaSC9S0IJEkLg+8slqTCGQSSVDiDQJIKZxBIUuEMAkkqnEEgSYV7Nl5iQj0UESdm5v5+99EvEXES8GLgAPBQZn6/T30MASvrH3dk5r5+9FH3chwwDHwjM/+nj30cA7wC+GZmfruPfbwhM+/u1/23wfcRtCQingA+AdyYmVv71MNZwI1U12i6DLgWeD3V5T0uaquveqP3QaoN8Gcyc6yj9qnMfHMLPbyE6lLoFwJTwLeB5wN/CfxxZv5wvnuo+1gF/BXV1Xh318PLgHuBdW18RkdEbATek5kTEfFq4FbgCWAx1YUg75rvHuo+3kT1JtPdVM/PTwL/C5wMvC0zb2+hh9MPMXwn8PPAQGY+MN891H2szcx/rr8/HtgA/CzVFRmuzMzHe3l/xe8aioj7Wrqr71BtgO+KiHsj4rci4sSW7nvax4D3Uj2p/gn4RGYeA1xJFQpt2QQ8SbUhvjgibo2I6dnpy1rq4e+Am6k+GOl3qf5NTgGOBz7SUg8AN1GF81BmnlFfkXcI+Nu61oZXdVze5f1ULwrOAM4H/qylHgCuBl4NXA7cAYxk5ul1H+9rqYf76/vu/FoK/CPwuZZ6gOqF0rRrqLYfbwS2Uf0/7qkidg0dJuWnDbXUxv7MfEdE/CHVH/RXgT+PiDuAv5lO/3n2gsz8LEBEvD8zbwHIzNsjoq3/aACnZuZb6j4+TbUR/lxEXNxiDydNP37g+oj4amZeHRGXA21e6mSoow8AMvNp4OaI+JOWenh+x/cvyMyv1n08GBFHtdQDwFRm3gcQEd/NzC/XffxXRLTVw3uBNcAVmflo3cvDmfnSthqoDXR8fz5wbmYeAN49Hy9eS5kR3E+V5gcn/R3AT7XZSGYeyMx/yMxfoNoP+3Xg+pbuvvPJdfB0v83nwtHT32TmVGb+JnAf1d/jeS318KN6t8z052M8VffzNNXxgrY8GREjEfHjv01EDETEpVS7q9pwd0RcV++T/3xEXFL3sZZqt2FbpiLiFRHxKuDYiDiv7mOYQ3yYynzIzPcC7wY2R8S66b7auO+DHF3/W5xOFZCdz8meX36/iBkB8AjVhzHsOrgQETta6mHg4IHMfIxq6t3W9PuRiHhBZn4nM399ejAiVgDfa6kHgIci4jWZ+a/TA5n5BxFxDfBHLfWwHvhKROyhmvpPb/xOBr7UUg9Q7QvfCIxFxPTzcznVvuDLWurhHcCHgV1UG/7fj4ibgM8Db2+pB6j+Jl+i2tBdArw/Il4ErACuaKuJzNwSEa8D3hcR/wK0OSuadgzVC6MBgIhYnpm7IuKFwNO9vrMiDhZHxIeBT09PNQ+q/UVm/k4LPbxkeqq50ETEscCx9eXB27i/k6he5fzE2UoRcXqLB+ROAE4FHuzn2TF1L4t55llDrV+SvX4erKJ6gfhoP89cqvsZBM4Gdvb64OgsejgPeG1mfnDGhVtQz9pOzsyHe/l7iwgCSdLhlXKMQJJ0GAaBJBXOIJCkwhkEklS4/wOnGzOQoiiYEgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#plotting a bar plot to show the frequency of each digit in the training data\n",
    "df_y[0].value_counts().plot(kind='bar')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All the digits occur with almost the same frequency in the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>774</th>\n",
       "      <th>775</th>\n",
       "      <th>776</th>\n",
       "      <th>777</th>\n",
       "      <th>778</th>\n",
       "      <th>779</th>\n",
       "      <th>780</th>\n",
       "      <th>781</th>\n",
       "      <th>782</th>\n",
       "      <th>783</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 784 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   0    1    2    3    4    5    6    7    8    9    ...  774  775  776  777  \\\n",
       "0    0    0    0    0    0    0    0    0    0    0  ...    0    0    0    0   \n",
       "1    0    0    0    0    0    0    0    0    0    0  ...    0    0    0    0   \n",
       "2    0    0    0    0    0    0    0    0    0    0  ...    0    0    0    0   \n",
       "3    0    0    0    0    0    0    0    0    0    0  ...    0    0    0    0   \n",
       "4    0    0    0    0    0    0    0    0    0    0  ...    0    0    0    0   \n",
       "\n",
       "   778  779  780  781  782  783  \n",
       "0    0    0    0    0    0    0  \n",
       "1    0    0    0    0    0    0  \n",
       "2    0    0    0    0    0    0  \n",
       "3    0    0    0    0    0    0  \n",
       "4    0    0    0    0    0    0  \n",
       "\n",
       "[5 rows x 784 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_test1=x_test.reshape(10000,784)\n",
    "df_2=pd.DataFrame.from_records(x_test1)\n",
    "df_2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Preprocessing \n",
    "#Reshaping the images to a single color channel\n",
    "x_train = df.values.reshape((-1, 28, 28, 1))\n",
    "x_test = df_2.values.reshape((-1, 28, 28, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60000, 28, 28, 1)\n",
      "(10000, 28, 28, 1)\n"
     ]
    }
   ],
   "source": [
    "#Checking the shape after re-shaping\n",
    "print(x_train.shape)\n",
    "print(x_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 10)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#One-hot encoding on target\n",
    "from keras.utils.np_utils import to_categorical\n",
    "y_train = to_categorical(y_train, num_classes = 10)\n",
    "y_train.shape\n",
    "y_test = to_categorical(y_test, num_classes = 10)\n",
    "y_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "#preparing pixel data\n",
    "#converting integers into float\n",
    "train_norm_x = x_train.astype('float32')\n",
    "test_norm_x = x_test.astype('float32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "#normalizing by dividing by the highest value i.e. 255\n",
    "train_norm_x=train_norm_x/255.0\n",
    "test_norm_x=test_norm_x/255.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "#splitting into training and validation data\n",
    "X_train, X_val, Y_train, Y_val = train_test_split(train_norm_x, y_train, test_size = 0.1, random_state=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 54000 samples, validate on 6000 samples\n",
      "Epoch 1/10\n",
      "54000/54000 [==============================] - 13s 236us/step - loss: 0.1930 - accuracy: 0.9415 - val_loss: 0.0542 - val_accuracy: 0.9830\n",
      "Epoch 2/10\n",
      "54000/54000 [==============================] - 9s 163us/step - loss: 0.0834 - accuracy: 0.9751 - val_loss: 0.0409 - val_accuracy: 0.9880\n",
      "Epoch 3/10\n",
      "54000/54000 [==============================] - 9s 160us/step - loss: 0.0634 - accuracy: 0.9807 - val_loss: 0.0341 - val_accuracy: 0.9898\n",
      "Epoch 4/10\n",
      "54000/54000 [==============================] - 8s 155us/step - loss: 0.0507 - accuracy: 0.9847 - val_loss: 0.0375 - val_accuracy: 0.9888\n",
      "Epoch 5/10\n",
      "54000/54000 [==============================] - 8s 155us/step - loss: 0.0448 - accuracy: 0.9864 - val_loss: 0.0376 - val_accuracy: 0.9887\n",
      "Epoch 6/10\n",
      "54000/54000 [==============================] - 8s 152us/step - loss: 0.0375 - accuracy: 0.9885 - val_loss: 0.0371 - val_accuracy: 0.9902\n",
      "Epoch 7/10\n",
      "54000/54000 [==============================] - 9s 158us/step - loss: 0.0338 - accuracy: 0.9895 - val_loss: 0.0315 - val_accuracy: 0.9908\n",
      "Epoch 8/10\n",
      "54000/54000 [==============================] - 9s 165us/step - loss: 0.0288 - accuracy: 0.9913 - val_loss: 0.0349 - val_accuracy: 0.9905\n",
      "Epoch 9/10\n",
      "54000/54000 [==============================] - 9s 161us/step - loss: 0.0274 - accuracy: 0.9914 - val_loss: 0.0362 - val_accuracy: 0.9912\n",
      "Epoch 10/10\n",
      "54000/54000 [==============================] - 9s 162us/step - loss: 0.0245 - accuracy: 0.9921 - val_loss: 0.0314 - val_accuracy: 0.9917\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.callbacks.History at 0x7fc1d09a9150>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#making a basic cnn model of 2 convolutional layers without data augmentation\n",
    "##model building\n",
    "model_b = Sequential()\n",
    "#convolutional layer with rectified linear unit activation\n",
    "model_b.add(Conv2D(32, kernel_size=(3, 3),\n",
    "                 activation='relu',\n",
    "                 input_shape=(28,28,1)))\n",
    "#32 convolution filters used each of size 3x3\n",
    "#again\n",
    "model_b.add(Conv2D(64, (3, 3), activation='relu'))\n",
    "#64 convolution filters used each of size 3x3\n",
    "#choose the best features via pooling\n",
    "model_b.add(MaxPool2D(pool_size=(2, 2)))\n",
    "#randomly turn neurons on and off to improve convergence\n",
    "model_b.add(Dropout(0.25))\n",
    "#flatten since too many dimensions, we only want a classification output\n",
    "model_b.add(Flatten())\n",
    "#fully connected to get all relevant data\n",
    "model_b.add(Dense(128, activation='relu'))\n",
    "#one more dropout for convergence' sake :) \n",
    "model_b.add(Dropout(0.5))\n",
    "#output a softmax to squash the matrix into output probabilities\n",
    "model_b.add(Dense(10, activation='softmax'))\n",
    "model_b.compile(optimizer = 'adam' , loss = \"categorical_crossentropy\", metrics=[\"accuracy\"])\n",
    "model_b.fit(X_train,Y_train, batch_size=32, epochs=10,verbose=1,validation_data = (X_val,Y_val))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using a simple 2D CNN we achieved 99.17% accuracy ,let's try the same model with data augmentation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Data Augmentation \n",
    "datagen = ImageDataGenerator(\n",
    "        featurewise_center=False,  # set input mean to 0 over the dataset\n",
    "        samplewise_center=False,  # set each sample mean to 0\n",
    "        featurewise_std_normalization=False,  # divide inputs by std of the dataset\n",
    "        samplewise_std_normalization=False,  # divide each input by its std\n",
    "        zca_whitening=False,  # apply ZCA whitening\n",
    "        rotation_range=10,  # randomly rotate images in the range (degrees, 0 to 180)\n",
    "        zoom_range = 0.1, # Randomly zoom image \n",
    "        width_shift_range=0.1,  # randomly shift images horizontally (fraction of total width)\n",
    "        height_shift_range=0.1,  # randomly shift images vertically (fraction of total height)\n",
    "        horizontal_flip=False,  # randomly flip images\n",
    "        vertical_flip=False)  # randomly flip images\n",
    "datagen.fit(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      " - 22s - loss: 0.1880 - accuracy: 0.9456 - val_loss: 0.0331 - val_accuracy: 0.9888\n",
      "Epoch 2/10\n",
      " - 22s - loss: 0.1190 - accuracy: 0.9657 - val_loss: 0.0307 - val_accuracy: 0.9885\n",
      "Epoch 3/10\n",
      " - 21s - loss: 0.1049 - accuracy: 0.9698 - val_loss: 0.0262 - val_accuracy: 0.9908\n",
      "Epoch 4/10\n",
      " - 21s - loss: 0.0938 - accuracy: 0.9723 - val_loss: 0.0266 - val_accuracy: 0.9928\n",
      "Epoch 5/10\n",
      " - 22s - loss: 0.0891 - accuracy: 0.9742 - val_loss: 0.0279 - val_accuracy: 0.9915\n",
      "Epoch 6/10\n",
      " - 21s - loss: 0.0820 - accuracy: 0.9767 - val_loss: 0.0248 - val_accuracy: 0.9915\n",
      "Epoch 7/10\n",
      " - 21s - loss: 0.0778 - accuracy: 0.9774 - val_loss: 0.0273 - val_accuracy: 0.9912\n",
      "Epoch 8/10\n",
      " - 22s - loss: 0.0784 - accuracy: 0.9769 - val_loss: 0.0243 - val_accuracy: 0.9925\n",
      "Epoch 9/10\n",
      " - 21s - loss: 0.0728 - accuracy: 0.9790 - val_loss: 0.0253 - val_accuracy: 0.9927\n",
      "Epoch 10/10\n",
      " - 21s - loss: 0.0735 - accuracy: 0.9789 - val_loss: 0.0272 - val_accuracy: 0.9908\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.callbacks.History at 0x7fc1d0be7e90>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_b.fit_generator(datagen.flow(X_train,Y_train, batch_size=32),\n",
    "                      epochs =10, validation_data = (X_val,Y_val),verbose = 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We achieved a 99.28% accuracy with data augmentation ,let's try to improve it further with a more connected CNN."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}