Keras5.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/vn322/Sport-University/blob/main/Keras5.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q9lphOLwnZgW"
      },
      "source": [
        "## Детали метода fit в Keras"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lsgGTfpbH4-M"
      },
      "source": [
        "Сегодня обсудим:\n",
        "1. Как обучаться по батчам (batch_size)\n",
        "2. Как при обучении считать метрики и на тесте (validation_data)\n",
        "3. Как обучаться на генераторах\n",
        "4. И как визуализировать процесс обучения\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4MA2CdPbKl0O"
      },
      "source": [
        "## Поддержка\n",
        "Платформа boosty https://boosty.to/machine_learrrning\n",
        "\n",
        "\n",
        "<img src='https://drive.google.com/uc?id=1WX1sZjw2nHwpifsOQnCiE-SmyoSZ1ChJ'>\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5wJj0qVRMUhf"
      },
      "source": [
        "Получение данных\n",
        "\n",
        "Описание датасета:\n",
        "\n",
        "- CRIM     per capita crime rate by town\n",
        "- ZN       proportion of residential land zoned for lots over 25,000 sq.ft.\n",
        "- INDUS    proportion of non-retail business acres per town\n",
        "- CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)\n",
        "- NOX      nitric oxides concentration (parts per 10 million)\n",
        "- RM       average number of rooms per dwelling\n",
        "- AGE      proportion of owner-occupied units built prior to 1940\n",
        "- DIS      weighted distances to five Boston employment centres\n",
        "- RAD      index of accessibility to radial highways\n",
        "- TAX      full-value property-tax rate per 10,000\n",
        "- PTRATIO  pupil-teacher ratio by town\n",
        "- B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town\n",
        "- LSTAT    % lower status of the population\n",
        "- MEDV     Median value of owner-occupied homes in $1000's"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "22XUy6MVL66Z",
        "outputId": "d4838d9b-aea1-4bd5-bba6-e0df9e32d70c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/boston_housing.npz\n",
            "57026/57026 [==============================] - 0s 0us/step\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((404, 13), (102, 13))"
            ]
          },
          "metadata": {},
          "execution_count": 1
        }
      ],
      "source": [
        "from keras.datasets import boston_housing\n",
        "\n",
        "(X_train, y_train), (X_test, y_test) = boston_housing.load_data()\n",
        "\n",
        "X_train.shape, X_test.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Q5Ds6QFBMiAX",
        "outputId": "b08238dd-db72-491c-b5ee-4935c3a7077b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[  1.23247,   0.     ,   8.14   ,   0.     ,   0.538  ,   6.142  ,\n",
              "         91.7    ,   3.9769 ,   4.     , 307.     ,  21.     , 396.9    ,\n",
              "         18.72   ]])"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ],
      "source": [
        "X_train[:1]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CSYSvyorMWTO"
      },
      "source": [
        "Масштабирование данных"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xAepFcolMM93",
        "outputId": "7153315b-eb33-4f12-972c-d11b5b7a2f2e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([3.74511057e+00, 1.14801980e+01, 1.11044307e+01, 6.18811881e-02,\n",
              "        5.57355941e-01, 6.26708168e+00, 6.90106436e+01, 3.74027079e+00,\n",
              "        9.44059406e+00, 4.05898515e+02, 1.84759901e+01, 3.54783168e+02,\n",
              "        1.27408168e+01]),\n",
              " array([9.22929073e+00, 2.37382770e+01, 6.80287253e+00, 2.40939633e-01,\n",
              "        1.17147847e-01, 7.08908627e-01, 2.79060634e+01, 2.02770050e+00,\n",
              "        8.68758849e+00, 1.66168506e+02, 2.19765689e+00, 9.39946015e+01,\n",
              "        7.24556085e+00]))"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ],
      "source": [
        "mean = X_train.mean(axis=0)\n",
        "std = X_train.std(axis=0)\n",
        "\n",
        "mean, std"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "qu2qimTJMMyT"
      },
      "outputs": [],
      "source": [
        "X_train -= mean\n",
        "X_train /= std\n",
        "\n",
        "X_test -= mean\n",
        "X_test /= std"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "p3vd2WHqMMoa",
        "outputId": "7343a8ef-c595-4550-c58b-8e9ef83f13a0"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([-1.01541438e-16,  1.09923072e-17,  1.74337992e-15, -1.26686340e-16,\n",
              "       -5.25377321e-15,  6.41414864e-15,  2.98441140e-16,  4.94653823e-16,\n",
              "        1.12671149e-17, -1.98136337e-16,  2.36686358e-14,  5.95679996e-15,\n",
              "        6.13920356e-16])"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "X_train.mean(axis=0)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PSOWwTi7MMhI",
        "outputId": "43cbe5cc-091d-4096-ee75-73ce03b1b0b3"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "X_train.std(axis=0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z1XTMfjpNU8S"
      },
      "source": [
        "### Архитектура сети\n",
        "\n",
        "\n",
        "Определение сети через класс Sequential и добавление слоев в него через add"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "P734IyAUNWpM",
        "outputId": "722857de-56e6-4280-af3b-e12ac72ecec8"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " dense (Dense)               (None, 32)                448       \n",
            "                                                                 \n",
            " dense_1 (Dense)             (None, 16)                528       \n",
            "                                                                 \n",
            " dense_2 (Dense)             (None, 1)                 17        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 993\n",
            "Trainable params: 993\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ],
      "source": [
        "from keras.models import Sequential\n",
        "from keras.layers import Dense\n",
        "import tensorflow as tf\n",
        "tf.random.set_seed(9)\n",
        "\n",
        "\n",
        "model = Sequential()\n",
        "model.add(Dense(32, activation='relu', input_shape=(X_train.shape[1], )))\n",
        "model.add(Dense(16, activation='relu'))\n",
        "model.add(Dense(1))\n",
        "\n",
        "model.summary()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "6bxv11s1NnJu"
      },
      "outputs": [],
      "source": [
        "model.compile(optimizer='adam', loss='mse', metrics=['mae'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QmoBr-uXRcy2"
      },
      "source": [
        "### Обучение сети"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vCI--E5vN3HG",
        "outputId": "af5ab4a3-3a1d-439a-bc9d-9fb45b5b974b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "13/13 [==============================] - 3s 8ms/step - loss: 577.6455 - mae: 22.2771\n",
            "Epoch 2/10\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 552.1992 - mae: 21.7273\n",
            "Epoch 3/10\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 527.0702 - mae: 21.1845\n",
            "Epoch 4/10\n",
            "13/13 [==============================] - 0s 25ms/step - loss: 499.5734 - mae: 20.5741\n",
            "Epoch 5/10\n",
            "13/13 [==============================] - 0s 15ms/step - loss: 466.0179 - mae: 19.8282\n",
            "Epoch 6/10\n",
            "13/13 [==============================] - 0s 17ms/step - loss: 423.3872 - mae: 18.8689\n",
            "Epoch 7/10\n",
            "13/13 [==============================] - 0s 10ms/step - loss: 371.7760 - mae: 17.5955\n",
            "Epoch 8/10\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 314.9016 - mae: 16.0498\n",
            "Epoch 9/10\n",
            "13/13 [==============================] - 0s 8ms/step - loss: 254.3791 - mae: 14.1603\n",
            "Epoch 10/10\n",
            "13/13 [==============================] - 0s 10ms/step - loss: 198.6382 - mae: 12.1582\n",
            "CPU times: user 2.07 s, sys: 107 ms, total: 2.18 s\n",
            "Wall time: 5.82 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f6a3541c9a0>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "%%time\n",
        "\n",
        "num_epochs = 10\n",
        "\n",
        "model.fit(X_train, y_train,\n",
        "          epochs=num_epochs)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EtQKfdyWOXB5"
      },
      "source": [
        "Сейчас прошло 13 батчей через сеть"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KPRwWBzXOmAA"
      },
      "source": [
        "#### **batch_size**\n",
        "> batch_size: Integer или None.<br>\n",
        "Количество сэмплов за один шаг градиентного спуска<br>\n",
        "Если None, то batch_size примет дефолтное значение 32<br>\n",
        "Не указывайте batch_size, если ваши данные представлены в виде генератора или keras.utils.Sequence (из-за того, что они сами генерируют данные батчами)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ekb4bBmeOHFl",
        "outputId": "dee58d12-3ab5-4a82-9630-f367d1c93794"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(404, 13)"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "X_train.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "k4mULIhJOZUY",
        "outputId": "7275ef83-2f29-4c52-b7a8-1157455785cb"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "12.625"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "404 / 32"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3fOpPUEDOE2T"
      },
      "source": [
        "<img src='https://drive.google.com/uc?export=view&id=1j8SxKYEi12jzJXi_bPO28q5SV9emuu0Y'>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2TEVf7n6OxA6",
        "outputId": "7a2a2c98-59a7-4989-fbb2-942536cf27d8"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "7/7 [==============================] - 0s 8ms/step - loss: 161.8336 - mae: 10.6805\n",
            "Epoch 2/10\n",
            "7/7 [==============================] - 0s 11ms/step - loss: 139.9734 - mae: 9.7196\n",
            "Epoch 3/10\n",
            "7/7 [==============================] - 0s 10ms/step - loss: 121.6629 - mae: 8.9444\n",
            "Epoch 4/10\n",
            "7/7 [==============================] - 0s 8ms/step - loss: 107.0853 - mae: 8.2484\n",
            "Epoch 5/10\n",
            "7/7 [==============================] - 0s 20ms/step - loss: 94.6019 - mae: 7.6389\n",
            "Epoch 6/10\n",
            "7/7 [==============================] - 0s 20ms/step - loss: 84.4375 - mae: 7.1095\n",
            "Epoch 7/10\n",
            "7/7 [==============================] - 0s 16ms/step - loss: 75.8428 - mae: 6.6329\n",
            "Epoch 8/10\n",
            "7/7 [==============================] - 0s 16ms/step - loss: 68.1057 - mae: 6.2129\n",
            "Epoch 9/10\n",
            "7/7 [==============================] - 0s 15ms/step - loss: 61.7162 - mae: 5.8566\n",
            "Epoch 10/10\n",
            "7/7 [==============================] - 0s 14ms/step - loss: 55.6949 - mae: 5.5247\n",
            "CPU times: user 513 ms, sys: 19.3 ms, total: 532 ms\n",
            "Wall time: 1.54 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f6a322ce430>"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "%%time\n",
        "num_epochs = 10\n",
        "\n",
        "model.fit(X_train, y_train,\n",
        "          epochs=num_epochs,\n",
        "          batch_size=64)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XREUhIiMUELz",
        "outputId": "33104084-e6b7-41b0-cf8b-25b55b824fb1"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "6.3125"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "404 / 64"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1ZGrjXFdOxPA",
        "outputId": "40a1e5d3-f4a6-40ac-ddbc-74048f5ac1ee"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "1/1 [==============================] - 0s 28ms/step - loss: 52.2837 - mae: 5.3354\n",
            "Epoch 2/10\n",
            "1/1 [==============================] - 0s 32ms/step - loss: 51.6250 - mae: 5.2975\n",
            "Epoch 3/10\n",
            "1/1 [==============================] - 0s 32ms/step - loss: 50.9782 - mae: 5.2599\n",
            "Epoch 4/10\n",
            "1/1 [==============================] - 0s 31ms/step - loss: 50.3434 - mae: 5.2223\n",
            "Epoch 5/10\n",
            "1/1 [==============================] - 0s 42ms/step - loss: 49.7210 - mae: 5.1856\n",
            "Epoch 6/10\n",
            "1/1 [==============================] - 0s 61ms/step - loss: 49.1120 - mae: 5.1500\n",
            "Epoch 7/10\n",
            "1/1 [==============================] - 0s 41ms/step - loss: 48.5154 - mae: 5.1148\n",
            "Epoch 8/10\n",
            "1/1 [==============================] - 0s 39ms/step - loss: 47.9312 - mae: 5.0800\n",
            "Epoch 9/10\n",
            "1/1 [==============================] - 0s 17ms/step - loss: 47.3591 - mae: 5.0457\n",
            "Epoch 10/10\n",
            "1/1 [==============================] - 0s 15ms/step - loss: 46.7985 - mae: 5.0120\n",
            "CPU times: user 285 ms, sys: 14.5 ms, total: 300 ms\n",
            "Wall time: 829 ms\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f6a322f1d30>"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "%%time\n",
        "num_epochs = 10\n",
        "\n",
        "model.fit(X_train, y_train,\n",
        "          epochs=num_epochs,\n",
        "          batch_size=404)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jskYwkWFPX8e"
      },
      "source": [
        "#### **steps_per_epoch**\n",
        "\n",
        ">  steps_per_epoch: Integer or `None`.<br>\n",
        "        Общее количество шагов (батчей сэмплов) на одной эпохе. <br>\n",
        "        Если стоит None, то steps_per_epoch равняется количеству сэмплов в датасете деленное на batch size"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mbCJjK6qUxY-",
        "outputId": "9a4e5ee6-b1cd-4603-f8ae-98036a1c5089"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "40.4"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "404 / 10"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "elDajMhxPXv3",
        "outputId": "60b46841-8d1b-4eea-faaf-6cdc0c2fa32b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "10/10 [==============================] - 0s 5ms/step - loss: 44.0470 - mae: 4.8423\n",
            "Epoch 2/10\n",
            "10/10 [==============================] - 0s 7ms/step - loss: 39.9519 - mae: 4.5770\n",
            "Epoch 3/10\n",
            "10/10 [==============================] - 0s 13ms/step - loss: 36.3724 - mae: 4.3588\n",
            "Epoch 4/10\n",
            "10/10 [==============================] - 0s 9ms/step - loss: 33.8860 - mae: 4.2046\n",
            "Epoch 5/10\n",
            "10/10 [==============================] - 0s 11ms/step - loss: 31.8988 - mae: 4.0674\n",
            "Epoch 6/10\n",
            "10/10 [==============================] - 0s 9ms/step - loss: 30.2303 - mae: 3.9560\n",
            "Epoch 7/10\n",
            "10/10 [==============================] - 0s 8ms/step - loss: 28.9526 - mae: 3.8639\n",
            "Epoch 8/10\n",
            "10/10 [==============================] - 0s 9ms/step - loss: 27.7988 - mae: 3.7890\n",
            "Epoch 9/10\n",
            "10/10 [==============================] - 0s 10ms/step - loss: 26.9376 - mae: 3.7334\n",
            "Epoch 10/10\n",
            "10/10 [==============================] - 0s 8ms/step - loss: 26.1212 - mae: 3.6789\n",
            "CPU times: user 591 ms, sys: 23.4 ms, total: 614 ms\n",
            "Wall time: 1.51 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f6a33021880>"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ],
      "source": [
        "%%time\n",
        "num_epochs = 10\n",
        "\n",
        "model.fit(X_train, y_train,\n",
        "          epochs=num_epochs, \n",
        "          steps_per_epoch=10)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IaDwXXa9QSWO"
      },
      "source": [
        "#### **validation_split**\n",
        "\n",
        "> validation_split: Float между 0 и 1.<br>\n",
        "        Доля обучающих данных, которая будет использоваться как валидационная часть<br>\n",
        "        Модель возьмет эту долю и не будет на ней обучаться, а будет подсчитывать метрики и функцию потерь в конце каждой эпохи<br>\n",
        "        Данные для валидации берутся из данных с конца в `x` и `y` **до перемешивания**."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VJgIZmddVP8c",
        "outputId": "cb2e27f5-121a-4aca-d7c2-2677e9a209fa"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "323"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "404 - 81"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ncS1dP4rQR9f",
        "outputId": "5e01cb03-5544-4fc7-c580-8c6027c0772b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "323/323 [==============================] - 4s 11ms/step - loss: 23.4202 - mae: 3.3994 - val_loss: 21.0647 - val_mae: 3.4214\n",
            "Epoch 2/10\n",
            "323/323 [==============================] - 3s 9ms/step - loss: 17.8573 - mae: 2.9863 - val_loss: 17.1680 - val_mae: 3.0921\n",
            "Epoch 3/10\n",
            "323/323 [==============================] - 2s 5ms/step - loss: 15.1941 - mae: 2.6903 - val_loss: 17.1426 - val_mae: 3.2252\n",
            "Epoch 4/10\n",
            "323/323 [==============================] - 2s 5ms/step - loss: 13.2552 - mae: 2.5279 - val_loss: 14.8308 - val_mae: 2.9081\n",
            "Epoch 5/10\n",
            "323/323 [==============================] - 3s 11ms/step - loss: 12.2521 - mae: 2.4049 - val_loss: 14.1870 - val_mae: 2.8837\n",
            "Epoch 6/10\n",
            "323/323 [==============================] - 1s 5ms/step - loss: 11.2367 - mae: 2.3210 - val_loss: 15.7526 - val_mae: 2.7817\n",
            "Epoch 7/10\n",
            "323/323 [==============================] - 1s 2ms/step - loss: 10.5615 - mae: 2.3518 - val_loss: 13.4374 - val_mae: 2.7194\n",
            "Epoch 8/10\n",
            "323/323 [==============================] - 1s 2ms/step - loss: 10.3781 - mae: 2.2417 - val_loss: 14.3366 - val_mae: 2.7130\n",
            "Epoch 9/10\n",
            "323/323 [==============================] - 1s 2ms/step - loss: 9.7806 - mae: 2.1499 - val_loss: 17.6216 - val_mae: 3.0384\n",
            "Epoch 10/10\n",
            "323/323 [==============================] - 1s 2ms/step - loss: 9.5508 - mae: 2.1753 - val_loss: 17.2772 - val_mae: 2.8888\n",
            "CPU times: user 10.8 s, sys: 488 ms, total: 11.3 s\n",
            "Wall time: 18.2 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f6a33043a60>"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "%%time\n",
        "num_epochs = 10\n",
        "\n",
        "model.fit(X_train, y_train,\n",
        "          epochs=num_epochs, \n",
        "          batch_size=1,\n",
        "          validation_split=0.2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uISMy0AEQp0t"
      },
      "source": [
        "#### **validation_data**\n",
        "\n",
        "> validation_data: Данные, на которых считается функция потерь и метрики в конце каждой эпохи<br>\n",
        "Модель не будет обучаться на этих данных\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NSL_MVEFRG1I",
        "outputId": "25de6777-9edf-4d11-e3f0-4c8c170dd7ab"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 10.1883 - mae: 2.2577 - val_loss: 21.0811 - val_mae: 3.0041\n",
            "Epoch 2/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 10.0301 - mae: 2.2425 - val_loss: 20.9901 - val_mae: 3.1147\n",
            "Epoch 3/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 9.5694 - mae: 2.2213 - val_loss: 23.0623 - val_mae: 3.0973\n",
            "Epoch 4/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 9.6029 - mae: 2.2154 - val_loss: 24.1658 - val_mae: 3.1759\n",
            "Epoch 5/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 9.2423 - mae: 2.1600 - val_loss: 20.3745 - val_mae: 3.0282\n",
            "Epoch 6/10\n",
            "404/404 [==============================] - 1s 3ms/step - loss: 8.8284 - mae: 2.1138 - val_loss: 19.8209 - val_mae: 2.8851\n",
            "Epoch 7/10\n",
            "404/404 [==============================] - 2s 4ms/step - loss: 8.4609 - mae: 2.0930 - val_loss: 21.3264 - val_mae: 3.0055\n",
            "Epoch 8/10\n",
            "404/404 [==============================] - 1s 3ms/step - loss: 8.7313 - mae: 2.1072 - val_loss: 20.4849 - val_mae: 3.1316\n",
            "Epoch 9/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 8.2316 - mae: 2.0467 - val_loss: 23.1979 - val_mae: 3.1851\n",
            "Epoch 10/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 8.3622 - mae: 2.0854 - val_loss: 18.5759 - val_mae: 2.8878\n",
            "CPU times: user 10.2 s, sys: 490 ms, total: 10.7 s\n",
            "Wall time: 10.3 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f6a32fa9100>"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ],
      "source": [
        "%%time\n",
        "num_epochs = 10\n",
        "\n",
        "model.fit(X_train, y_train,\n",
        "          epochs=num_epochs, \n",
        "          batch_size=1,\n",
        "          validation_data=(X_test, y_test))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3WgOrPa_Ru1B"
      },
      "source": [
        "#### **validation_batch_size**\n",
        "\n",
        ">validation_batch_size: Integer или `None`.<br>\n",
        "        Количество сэмплов в один валидационный батч<br>\n",
        "        Если None, то по дефолту равняется `batch_size`.<br>\n",
        "        Не указывайте `validation_batch_size`, если ваши данные представлены в виде генератора или keras.utils.Sequence (из-за того, что они сами генерируют данные батчами)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ThsEzmj_SKWr",
        "outputId": "017482f5-1067-459c-daff-dd1ed140f9fb"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(102, 13)"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "X_test.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wrg5omkiRj1B",
        "outputId": "ad93fba9-dbbe-4be9-dca6-f7a089c8c3c3"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 7.9101 - mae: 1.9924 - val_loss: 17.2365 - val_mae: 2.7353\n",
            "Epoch 2/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 7.9717 - mae: 2.0023 - val_loss: 17.6940 - val_mae: 2.8572\n",
            "Epoch 3/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 7.6244 - mae: 1.9923 - val_loss: 19.7891 - val_mae: 2.8949\n",
            "Epoch 4/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 7.8231 - mae: 2.0013 - val_loss: 21.1397 - val_mae: 2.9782\n",
            "Epoch 5/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 7.6040 - mae: 1.9705 - val_loss: 17.2456 - val_mae: 2.8175\n",
            "Epoch 6/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 7.3185 - mae: 1.9319 - val_loss: 17.4146 - val_mae: 2.7172\n",
            "Epoch 7/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 6.9537 - mae: 1.8915 - val_loss: 19.0834 - val_mae: 2.8516\n",
            "Epoch 8/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 7.3251 - mae: 1.9370 - val_loss: 18.5906 - val_mae: 3.0220\n",
            "Epoch 9/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 6.8284 - mae: 1.8708 - val_loss: 21.9550 - val_mae: 3.1440\n",
            "Epoch 10/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 7.0220 - mae: 1.9217 - val_loss: 17.2014 - val_mae: 2.8298\n",
            "CPU times: user 8.28 s, sys: 362 ms, total: 8.64 s\n",
            "Wall time: 10.3 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f6a32fc20d0>"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ],
      "source": [
        "%%time\n",
        "num_epochs = 10\n",
        "\n",
        "model.fit(X_train, y_train,\n",
        "          epochs=num_epochs, \n",
        "          batch_size=1,\n",
        "          validation_data=(X_test, y_test),\n",
        "          validation_batch_size=X_test.shape[0])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w_ceE1SIR8Vt"
      },
      "source": [
        "#### **validation_freq**\n",
        ">validation_freq: Только нужен, если предоставлены validation data. Integer\n",
        "        или `collections.abc.Container` (например, list, tuple, и др.).<br>\n",
        "        Если integer, то столько эпох с обучением пройдет, до одной валидации, например, `validation_freq=2` будет запускать валидацию каждые 2 эпохи.<br>\n",
        "        Если это Container, то валидация будет запущена в эти эпохи, к примеру,`validation_freq=[1, 2, 10]` запускает валидацию на 1, 2 и 10 эпохах."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2O7qWnJKSQwz",
        "outputId": "e2d0fe74-1bbb-4483-b8cf-8428f3d38020"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 6.6431 - mae: 1.8431\n",
            "Epoch 2/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 6.7405 - mae: 1.8414 - val_loss: 16.4107 - val_mae: 2.7393\n",
            "Epoch 3/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 6.4514 - mae: 1.8212\n",
            "Epoch 4/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 6.6401 - mae: 1.8433 - val_loss: 19.4960 - val_mae: 2.8574\n",
            "Epoch 5/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 6.4832 - mae: 1.8431\n",
            "Epoch 6/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 6.2846 - mae: 1.8026 - val_loss: 15.7963 - val_mae: 2.5866\n",
            "Epoch 7/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 5.9123 - mae: 1.7571\n",
            "Epoch 8/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 6.2981 - mae: 1.7892 - val_loss: 17.7605 - val_mae: 2.9508\n",
            "Epoch 9/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 5.8583 - mae: 1.7322\n",
            "Epoch 10/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 6.0068 - mae: 1.7779 - val_loss: 16.4085 - val_mae: 2.7389\n",
            "CPU times: user 7.83 s, sys: 423 ms, total: 8.25 s\n",
            "Wall time: 7.56 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f6a32f2a8e0>"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "%%time\n",
        "num_epochs = 10\n",
        "\n",
        "model.fit(X_train, y_train,\n",
        "          epochs=num_epochs, \n",
        "          batch_size=1,\n",
        "          validation_data=(X_test, y_test),\n",
        "          validation_batch_size=X_test.shape[0],\n",
        "          validation_freq=2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eIjSjGqaST76",
        "outputId": "56d90abb-2ff6-4cc0-8f88-4139b04f8ec9"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 5.7400 - mae: 1.7239 - val_loss: 15.0133 - val_mae: 2.5423\n",
            "Epoch 2/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 5.8994 - mae: 1.7280\n",
            "Epoch 3/10\n",
            "404/404 [==============================] - 1s 3ms/step - loss: 5.6313 - mae: 1.7001\n",
            "Epoch 4/10\n",
            "404/404 [==============================] - 1s 3ms/step - loss: 5.8083 - mae: 1.7404\n",
            "Epoch 5/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 5.7340 - mae: 1.7402 - val_loss: 14.7606 - val_mae: 2.5953\n",
            "Epoch 6/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 5.5479 - mae: 1.7065\n",
            "Epoch 7/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 5.1876 - mae: 1.6520\n",
            "Epoch 8/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 5.5650 - mae: 1.6932\n",
            "Epoch 9/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 5.1685 - mae: 1.6363\n",
            "Epoch 10/10\n",
            "404/404 [==============================] - 1s 2ms/step - loss: 5.3498 - mae: 1.6770 - val_loss: 15.7357 - val_mae: 2.7314\n",
            "CPU times: user 8.78 s, sys: 374 ms, total: 9.16 s\n",
            "Wall time: 10.3 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f6a32fa6eb0>"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ],
      "source": [
        "%%time\n",
        "num_epochs = 10\n",
        "\n",
        "model.fit(X_train, y_train,\n",
        "          epochs=num_epochs, \n",
        "          batch_size=1,\n",
        "          validation_data=(X_test, y_test),\n",
        "          validation_batch_size=X_test.shape[0],\n",
        "          validation_freq=[1, 5, 10])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8EEzRgS6bDR-"
      },
      "source": [
        "### Обучение нейросети на генераторах"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oE4SheQsddUk"
      },
      "source": [
        "Про Sequence больше [тут](https://www.tensorflow.org/api_docs/python/tf/keras/utils/Sequence)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "Bf-nfJSZbS2W"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "#from tensorflow.keras.utils import Sequence\n",
        "from tensorflow.keras.utils import Sequence\n",
        "\n",
        "class DataGenerator(Sequence):\n",
        "    def __init__(self, data, labels, batch_size=32):\n",
        "        self.batch_size = batch_size\n",
        "        self.data = data\n",
        "        self.labels = labels\n",
        "        \n",
        "    def __len__(self):\n",
        "        return int(np.round(len(self.data) / self.batch_size))\n",
        "\n",
        "    def __getitem__(self, index):\n",
        "        X = self.data[index * self.batch_size : (index+1) * self.batch_size]\n",
        "        y = self.labels[index * self.batch_size : (index+1) * self.batch_size]\n",
        "\n",
        "        return X, y"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "id": "YJ7LVBucds4z"
      },
      "outputs": [],
      "source": [
        "train_datagen = DataGenerator(X_train, y_train)\n",
        "test_datagen = DataGenerator(X_test, y_test)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8JEIEtjdED9B",
        "outputId": "773457d5-b3e8-40cd-f345-51f7ad27c335"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "13"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ],
      "source": [
        "len(train_datagen)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-fvtDpq4W-cO",
        "outputId": "670a64ff-aae2-4fa5-cf9a-64f1ecee973d"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ],
      "source": [
        "len(test_datagen)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TlBqpp-jd2e2",
        "outputId": "e169764c-ef91-455f-fcc8-a72924f24931"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(32, 13)\n",
            "(32,)\n"
          ]
        }
      ],
      "source": [
        "for X, y in train_datagen:\n",
        "    print(X.shape)\n",
        "    print(y.shape)\n",
        "    break"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oOpzLoZUenDC",
        "outputId": "4aef7cc9-3228-4bf0-f667-5c3036507773"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "12.625"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ],
      "source": [
        "404 / 32"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "foAOq_mUds91",
        "outputId": "9fe3b5e8-b25e-4fa3-90de-86dacb596cf5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "13/13 [==============================] - 1s 18ms/step - loss: 4.9604 - mae: 1.5813 - val_loss: 10.8587 - val_mae: 2.4586\n",
            "Epoch 2/10\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.4441 - mae: 1.4696 - val_loss: 10.1958 - val_mae: 2.3908\n",
            "Epoch 3/10\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.3174 - mae: 1.4666 - val_loss: 10.1397 - val_mae: 2.3839\n",
            "Epoch 4/10\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.2951 - mae: 1.4711 - val_loss: 10.2045 - val_mae: 2.3924\n",
            "Epoch 5/10\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.2593 - mae: 1.4626 - val_loss: 10.2958 - val_mae: 2.4062\n",
            "Epoch 6/10\n",
            "13/13 [==============================] - 0s 10ms/step - loss: 4.2390 - mae: 1.4578 - val_loss: 10.3441 - val_mae: 2.4119\n",
            "Epoch 7/10\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.2343 - mae: 1.4580 - val_loss: 10.3665 - val_mae: 2.4133\n",
            "Epoch 8/10\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.2264 - mae: 1.4541 - val_loss: 10.2918 - val_mae: 2.3990\n",
            "Epoch 9/10\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.2183 - mae: 1.4519 - val_loss: 10.2630 - val_mae: 2.3944\n",
            "Epoch 10/10\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.2121 - mae: 1.4522 - val_loss: 10.3042 - val_mae: 2.4017\n",
            "CPU times: user 1.7 s, sys: 63.3 ms, total: 1.76 s\n",
            "Wall time: 2.28 s\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f6a356f8070>"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ],
      "source": [
        "%%time\n",
        "num_epochs = 10\n",
        "\n",
        "model.fit(train_datagen,\n",
        "          epochs=num_epochs, \n",
        "          validation_data=test_datagen)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "52wMWVdWfKy-"
      },
      "source": [
        "### History\n",
        "\n",
        "Больше [тут](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/History) "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "q5KOlVn4fMJG",
        "outputId": "251fbefb-aae1-4edb-fcac-f9a1f78aad5b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/50\n",
            "13/13 [==============================] - 0s 9ms/step - loss: 4.2033 - mae: 1.4516 - val_loss: 10.3204 - val_mae: 2.4028\n",
            "Epoch 2/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1984 - mae: 1.4504 - val_loss: 10.2717 - val_mae: 2.3950\n",
            "Epoch 3/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.1977 - mae: 1.4516 - val_loss: 10.2973 - val_mae: 2.4012\n",
            "Epoch 4/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1947 - mae: 1.4495 - val_loss: 10.2410 - val_mae: 2.3904\n",
            "Epoch 5/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1824 - mae: 1.4461 - val_loss: 10.2734 - val_mae: 2.3955\n",
            "Epoch 6/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1868 - mae: 1.4510 - val_loss: 10.2458 - val_mae: 2.3918\n",
            "Epoch 7/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1795 - mae: 1.4489 - val_loss: 10.2879 - val_mae: 2.3986\n",
            "Epoch 8/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1715 - mae: 1.4476 - val_loss: 10.2213 - val_mae: 2.3867\n",
            "Epoch 9/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1650 - mae: 1.4439 - val_loss: 10.2135 - val_mae: 2.3849\n",
            "Epoch 10/50\n",
            "13/13 [==============================] - 0s 7ms/step - loss: 4.1632 - mae: 1.4432 - val_loss: 10.2651 - val_mae: 2.3940\n",
            "Epoch 11/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1605 - mae: 1.4442 - val_loss: 10.2750 - val_mae: 2.3966\n",
            "Epoch 12/50\n",
            "13/13 [==============================] - 0s 8ms/step - loss: 4.1528 - mae: 1.4457 - val_loss: 10.2676 - val_mae: 2.3958\n",
            "Epoch 13/50\n",
            "13/13 [==============================] - 0s 11ms/step - loss: 4.1480 - mae: 1.4431 - val_loss: 10.2269 - val_mae: 2.3886\n",
            "Epoch 14/50\n",
            "13/13 [==============================] - 0s 11ms/step - loss: 4.1480 - mae: 1.4426 - val_loss: 10.2244 - val_mae: 2.3857\n",
            "Epoch 15/50\n",
            "13/13 [==============================] - 0s 11ms/step - loss: 4.1557 - mae: 1.4446 - val_loss: 10.2972 - val_mae: 2.3988\n",
            "Epoch 16/50\n",
            "13/13 [==============================] - 0s 12ms/step - loss: 4.1417 - mae: 1.4407 - val_loss: 10.2173 - val_mae: 2.3852\n",
            "Epoch 17/50\n",
            "13/13 [==============================] - 0s 8ms/step - loss: 4.1404 - mae: 1.4398 - val_loss: 10.2062 - val_mae: 2.3833\n",
            "Epoch 18/50\n",
            "13/13 [==============================] - 0s 11ms/step - loss: 4.1358 - mae: 1.4394 - val_loss: 10.2007 - val_mae: 2.3823\n",
            "Epoch 19/50\n",
            "13/13 [==============================] - 0s 8ms/step - loss: 4.1354 - mae: 1.4408 - val_loss: 10.2497 - val_mae: 2.3904\n",
            "Epoch 20/50\n",
            "13/13 [==============================] - 0s 11ms/step - loss: 4.1320 - mae: 1.4409 - val_loss: 10.2102 - val_mae: 2.3842\n",
            "Epoch 21/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1259 - mae: 1.4395 - val_loss: 10.2673 - val_mae: 2.3925\n",
            "Epoch 22/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.1238 - mae: 1.4379 - val_loss: 10.2217 - val_mae: 2.3839\n",
            "Epoch 23/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.1304 - mae: 1.4405 - val_loss: 10.2254 - val_mae: 2.3836\n",
            "Epoch 24/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.1195 - mae: 1.4361 - val_loss: 10.2263 - val_mae: 2.3833\n",
            "Epoch 25/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1231 - mae: 1.4377 - val_loss: 10.2465 - val_mae: 2.3886\n",
            "Epoch 26/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.1167 - mae: 1.4365 - val_loss: 10.2049 - val_mae: 2.3831\n",
            "Epoch 27/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.1179 - mae: 1.4400 - val_loss: 10.2939 - val_mae: 2.3978\n",
            "Epoch 28/50\n",
            "13/13 [==============================] - 0s 9ms/step - loss: 4.1071 - mae: 1.4376 - val_loss: 10.2357 - val_mae: 2.3885\n",
            "Epoch 29/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1103 - mae: 1.4340 - val_loss: 10.1708 - val_mae: 2.3765\n",
            "Epoch 30/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1058 - mae: 1.4357 - val_loss: 10.2229 - val_mae: 2.3843\n",
            "Epoch 31/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.1046 - mae: 1.4396 - val_loss: 10.2155 - val_mae: 2.3826\n",
            "Epoch 32/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1059 - mae: 1.4357 - val_loss: 10.2504 - val_mae: 2.3867\n",
            "Epoch 33/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.1008 - mae: 1.4370 - val_loss: 10.2731 - val_mae: 2.3932\n",
            "Epoch 34/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.1027 - mae: 1.4366 - val_loss: 10.1416 - val_mae: 2.3716\n",
            "Epoch 35/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.0973 - mae: 1.4336 - val_loss: 10.2563 - val_mae: 2.3872\n",
            "Epoch 36/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.0895 - mae: 1.4332 - val_loss: 10.1942 - val_mae: 2.3785\n",
            "Epoch 37/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.0925 - mae: 1.4309 - val_loss: 10.1857 - val_mae: 2.3778\n",
            "Epoch 38/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.0776 - mae: 1.4297 - val_loss: 10.1859 - val_mae: 2.3785\n",
            "Epoch 39/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.0721 - mae: 1.4298 - val_loss: 10.2112 - val_mae: 2.3811\n",
            "Epoch 40/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.0720 - mae: 1.4316 - val_loss: 10.2703 - val_mae: 2.3889\n",
            "Epoch 41/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.0822 - mae: 1.4376 - val_loss: 10.3019 - val_mae: 2.3959\n",
            "Epoch 42/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.0775 - mae: 1.4304 - val_loss: 10.1860 - val_mae: 2.3765\n",
            "Epoch 43/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.0631 - mae: 1.4269 - val_loss: 10.1668 - val_mae: 2.3746\n",
            "Epoch 44/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.0595 - mae: 1.4290 - val_loss: 10.2174 - val_mae: 2.3816\n",
            "Epoch 45/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.0585 - mae: 1.4303 - val_loss: 10.2757 - val_mae: 2.3890\n",
            "Epoch 46/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.0550 - mae: 1.4310 - val_loss: 10.2666 - val_mae: 2.3883\n",
            "Epoch 47/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 4.0643 - mae: 1.4320 - val_loss: 10.1595 - val_mae: 2.3723\n",
            "Epoch 48/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.0483 - mae: 1.4254 - val_loss: 10.2822 - val_mae: 2.3887\n",
            "Epoch 49/50\n",
            "13/13 [==============================] - 0s 9ms/step - loss: 4.0495 - mae: 1.4263 - val_loss: 10.1798 - val_mae: 2.3754\n",
            "Epoch 50/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 4.0430 - mae: 1.4273 - val_loss: 10.1989 - val_mae: 2.3803\n",
            "CPU times: user 5.37 s, sys: 272 ms, total: 5.64 s\n",
            "Wall time: 8.11 s\n"
          ]
        }
      ],
      "source": [
        "%%time\n",
        "num_epochs = 50\n",
        "\n",
        "history = model.fit(train_datagen,\n",
        "                    epochs=num_epochs, \n",
        "                    validation_data=test_datagen)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1Ju1W7FVfc2F",
        "outputId": "b9f490e8-defa-4c08-805e-f4480bd76b5e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f6a32fa8b50>"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ],
      "source": [
        "history"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ODbUIevHfSPg",
        "outputId": "4b19142f-ffcf-441c-eda4-d2af6e8d8590"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'loss': [4.203336715698242,\n",
              "  4.198439598083496,\n",
              "  4.19774866104126,\n",
              "  4.194680690765381,\n",
              "  4.182363510131836,\n",
              "  4.186787128448486,\n",
              "  4.1794843673706055,\n",
              "  4.171491622924805,\n",
              "  4.165003299713135,\n",
              "  4.163246154785156,\n",
              "  4.160518646240234,\n",
              "  4.152813911437988,\n",
              "  4.148003101348877,\n",
              "  4.147953033447266,\n",
              "  4.1557183265686035,\n",
              "  4.14171838760376,\n",
              "  4.140407562255859,\n",
              "  4.135788917541504,\n",
              "  4.135436534881592,\n",
              "  4.132004261016846,\n",
              "  4.125892639160156,\n",
              "  4.1238274574279785,\n",
              "  4.130366325378418,\n",
              "  4.1194586753845215,\n",
              "  4.123128890991211,\n",
              "  4.116687297821045,\n",
              "  4.117896556854248,\n",
              "  4.107085227966309,\n",
              "  4.110342502593994,\n",
              "  4.10580587387085,\n",
              "  4.1046366691589355,\n",
              "  4.105923652648926,\n",
              "  4.100790023803711,\n",
              "  4.102657318115234,\n",
              "  4.097344398498535,\n",
              "  4.08951997756958,\n",
              "  4.0925469398498535,\n",
              "  4.077587127685547,\n",
              "  4.072129249572754,\n",
              "  4.072021961212158,\n",
              "  4.08218240737915,\n",
              "  4.077493667602539,\n",
              "  4.063054084777832,\n",
              "  4.059482574462891,\n",
              "  4.05851936340332,\n",
              "  4.05497932434082,\n",
              "  4.064332485198975,\n",
              "  4.048338413238525,\n",
              "  4.049499034881592,\n",
              "  4.043018817901611],\n",
              " 'mae': [1.4515800476074219,\n",
              "  1.4504485130310059,\n",
              "  1.4516156911849976,\n",
              "  1.449548602104187,\n",
              "  1.4460790157318115,\n",
              "  1.4510449171066284,\n",
              "  1.4489179849624634,\n",
              "  1.447642207145691,\n",
              "  1.4438565969467163,\n",
              "  1.4432435035705566,\n",
              "  1.4441531896591187,\n",
              "  1.4457210302352905,\n",
              "  1.4430837631225586,\n",
              "  1.4426480531692505,\n",
              "  1.4446494579315186,\n",
              "  1.4407455921173096,\n",
              "  1.439757227897644,\n",
              "  1.4393786191940308,\n",
              "  1.4407551288604736,\n",
              "  1.4408857822418213,\n",
              "  1.4395126104354858,\n",
              "  1.4379158020019531,\n",
              "  1.4405467510223389,\n",
              "  1.4360593557357788,\n",
              "  1.4376775026321411,\n",
              "  1.4364922046661377,\n",
              "  1.4399652481079102,\n",
              "  1.4375919103622437,\n",
              "  1.4339786767959595,\n",
              "  1.435675024986267,\n",
              "  1.439587116241455,\n",
              "  1.4357290267944336,\n",
              "  1.4370373487472534,\n",
              "  1.4365521669387817,\n",
              "  1.4336450099945068,\n",
              "  1.4332157373428345,\n",
              "  1.4308794736862183,\n",
              "  1.4297161102294922,\n",
              "  1.429750919342041,\n",
              "  1.4315824508666992,\n",
              "  1.4375571012496948,\n",
              "  1.4303722381591797,\n",
              "  1.4269239902496338,\n",
              "  1.4290471076965332,\n",
              "  1.4303481578826904,\n",
              "  1.4310059547424316,\n",
              "  1.4320038557052612,\n",
              "  1.4254347085952759,\n",
              "  1.4263041019439697,\n",
              "  1.427328109741211],\n",
              " 'val_loss': [10.320406913757324,\n",
              "  10.271661758422852,\n",
              "  10.297274589538574,\n",
              "  10.240973472595215,\n",
              "  10.273421287536621,\n",
              "  10.245831489562988,\n",
              "  10.287934303283691,\n",
              "  10.221267700195312,\n",
              "  10.213512420654297,\n",
              "  10.265114784240723,\n",
              "  10.274977684020996,\n",
              "  10.267577171325684,\n",
              "  10.226874351501465,\n",
              "  10.224422454833984,\n",
              "  10.297240257263184,\n",
              "  10.217310905456543,\n",
              "  10.206155776977539,\n",
              "  10.200675964355469,\n",
              "  10.249688148498535,\n",
              "  10.210160255432129,\n",
              "  10.267317771911621,\n",
              "  10.221673965454102,\n",
              "  10.225382804870605,\n",
              "  10.226346015930176,\n",
              "  10.246500968933105,\n",
              "  10.204936027526855,\n",
              "  10.293879508972168,\n",
              "  10.235679626464844,\n",
              "  10.170833587646484,\n",
              "  10.222911834716797,\n",
              "  10.215519905090332,\n",
              "  10.25042724609375,\n",
              "  10.273083686828613,\n",
              "  10.141554832458496,\n",
              "  10.256333351135254,\n",
              "  10.194159507751465,\n",
              "  10.185675621032715,\n",
              "  10.185897827148438,\n",
              "  10.211185455322266,\n",
              "  10.27026653289795,\n",
              "  10.301911354064941,\n",
              "  10.186015129089355,\n",
              "  10.16677188873291,\n",
              "  10.217395782470703,\n",
              "  10.275717735290527,\n",
              "  10.266616821289062,\n",
              "  10.159539222717285,\n",
              "  10.28216552734375,\n",
              "  10.179810523986816,\n",
              "  10.198859214782715],\n",
              " 'val_mae': [2.402756690979004,\n",
              "  2.3949546813964844,\n",
              "  2.401182174682617,\n",
              "  2.3903539180755615,\n",
              "  2.395486831665039,\n",
              "  2.3917500972747803,\n",
              "  2.3986198902130127,\n",
              "  2.386711597442627,\n",
              "  2.384948492050171,\n",
              "  2.3939743041992188,\n",
              "  2.3965647220611572,\n",
              "  2.3957901000976562,\n",
              "  2.3886377811431885,\n",
              "  2.385704755783081,\n",
              "  2.398794412612915,\n",
              "  2.385237216949463,\n",
              "  2.3833329677581787,\n",
              "  2.3823325634002686,\n",
              "  2.3903579711914062,\n",
              "  2.384185791015625,\n",
              "  2.392514944076538,\n",
              "  2.383890151977539,\n",
              "  2.3836333751678467,\n",
              "  2.383296012878418,\n",
              "  2.388572931289673,\n",
              "  2.383052110671997,\n",
              "  2.3978123664855957,\n",
              "  2.3885180950164795,\n",
              "  2.376469850540161,\n",
              "  2.38425612449646,\n",
              "  2.3825628757476807,\n",
              "  2.3867247104644775,\n",
              "  2.39316725730896,\n",
              "  2.3716325759887695,\n",
              "  2.3872101306915283,\n",
              "  2.3785064220428467,\n",
              "  2.377782106399536,\n",
              "  2.378450393676758,\n",
              "  2.381114959716797,\n",
              "  2.388939142227173,\n",
              "  2.395909547805786,\n",
              "  2.3764517307281494,\n",
              "  2.374570846557617,\n",
              "  2.3816416263580322,\n",
              "  2.3889577388763428,\n",
              "  2.3882803916931152,\n",
              "  2.3722758293151855,\n",
              "  2.3887462615966797,\n",
              "  2.3754351139068604,\n",
              "  2.3802638053894043]}"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ],
      "source": [
        "history.history"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 430
        },
        "id": "Rx7FzE21fmmV",
        "outputId": "8180e84a-1ba9-4fba-bbb8-ea6d3c18cdc0"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA5LElEQVR4nO3deXxU9b3/8feZNfsChCUSNmXfVFCKaK2Simi5SL3qpfQqemmvNlbR2lZ+reIOtep1uda2the12mLbK+itWkRlqSgIsirI1kBQdoRMEpLJZOb8/vjOTBLWhMycIfB6Ph7ncWbOzJzzzZnJnPf5fL8zY9m2bQsAAMAhrlQ3AAAAnF4IHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAAR3lS3YBDRSIRbd++XdnZ2bIsK9XNAQAATWDbtioqKlRYWCiX69i1jZMufGzfvl1FRUWpbgYAADgB27ZtU+fOnY95n5MufGRnZ0syjc/JyUlxawAAQFMEAgEVFRXFj+PHctKFj1hXS05ODuEDAIBWpilDJhhwCgAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABwFOEDAAA4ivABAAAcRfgAAACOInwAAABHET4AAICjTp/wEYlIf5koffQrae9GybZT3SIAAE5LJ92v2ibNzlXSZ7PMNGeKlNdVOqtY6vlNqdtFkj+raes5+JW0b7NUWyl1+ZrkTU9uu5E44Tpp+wopr4uU3SHVrUFzhOukupqm/5/i1BCJSPtLpV2fSrvWSvndpEHXSa7T57z5VGXZ9slVAggEAsrNzVV5eblycnISt+KKXdKaP0ub3pW2fiiFa+tvc/ukLsNNEDmr2AST/aWmQrJvkwkb+zaZqfqr+sd5M6Vel0n9xko9L5N8mc1vVzgkuTxSE36C+IiP3b5C2vKB+ZtkSwOvkfr+i+TLaP76EiUSMeEsWGHa5M0w+8btO7G/s6Wq9kqfvCAt+x8p8KVpx6DrpBG3S+16Ot8eNF1d0Dx3Cx+TqvZIXUdIA8ZJfcdKWQWpbl3i2HZq/jdOJrVVJmDsWiPt/FTauUbavda8lzTU5QLpqmelNj1S086GAtulULXkzzHB2JOWvOdx70ZpwxzzHn+Snjw15/h9+oSPhoKV0pZ/mCCyca50YGvzHp9zhpkHvqxf5kmXehabN8Veo6S0I7S9cre0c7X5p9oRne/bZF64Bb2kdr0bz/O6Si53/eNDNdKXy0zQ2PKB9MVSKXTw8O34c6SB/yqde73U6ezE/jOEaqTShdI/50mVu6SagBQMmKDR8LKO8LKy3CaExMKIL8MEuOwOUtHXTCWp4yDJnaCC3JefSB8/L336v/Vh05sphapiDZL6XCldeIfUeWhitpkKoRpp8/vS+jclyyV1v9hMrfngHA5JK/8oLXhUCnxx+O2Wy1QsB3zbhO2MNs63sTnCIal8m7R/yyHTVjO3bemSKdKwm1v2/1p9IBr0vYlodfLtWW/+P9e+Ie35XEd833D7pfZ9pHa9pPVvmzDizZC++YA09D+cr4IcKDMV9E9fk3asbHybyyP5syVftpn7s8y88BxpxOQTq9zZtjlxmvMzqS4adL4xRTr/eyfd80z4aA7bNpWNTXNNENnygRQOSmm5Utue5sy47ZlS27PM1KaH+ee2bVN1WPu6mfaX1q/T7ZPOHCmdNdIk451rzFS5s3lt86SZbbbraYLLF8tM2xpKbyN1vUDqdqEJVSv+0DhMdRgonfvvJi2f6Bt0xS5p4xxp/d9N6DhS4DkSl0eSJUVCTd+WN1MqOs+c3XQdLp0xtHlVnLqg9Nls6ePfmqAWU3iuNOw/pf7jpO0rpUVPSuvfqr+920WmEnJWsTNnoOGQtG2Jec1tX2FeY0Vfk7oMM6HzeG0I1Uib3zN/6/q3pdqKw+/TYaDU42KpxyVmX55IZa45wnVSxQ5zkC3/wswDO6S8IlOx6DT4+G+WkbA5GM2fJn31T7Msu5P09R9LPb4hff6m9NlrZp/FWG5z24BvmzCZnp+sv7BpKnaa9sWm3Z+bAGVHjv/YfmOlf/nvI5+8HEtdrfSPx6R/PG7euwZcbap7Zww5+SoqX5Wa5/DT10x3SkNZHaQOA6SOA8zrt+MA8z4cOyHZv1V6vcScPEpS96+b/ZXfNbltDmw3/2ufvWZO+mIslwlCh1ZnjiTnDOmKX5rXaFNV7ZXe+GH9e1VGW+ngPnO5fT9p9KNS94uavr4kI3y0RKjaTOn5Tf+ntW0TLmJBZN/Go9zRMmGi40AzdRpkXkDV+80ZwN4N9fO9Gw8PGpKU2V7qNsK8mXe70FRJGib/SETaslBa/gdp3f/Vr8Ptl/p+y4SQrPbmIO9rMDXsEon9PRv+bg5s25c3bkPOGaa6065XtNyYbd4s/dEpLbosVoIMh0xJtbbKBJf4/KCpQuzbLJUtlrYtlmrKG2/L5THVm8Kzzfgal8ccbFye6OSqX1a1R1r5iplL5m/qP046/z+lzkMO35e7P5c+fFpa/aoUqTPLOgwwIaTfVZLHd/Tn/EQEtpuwsWmu9M8Fpkp0JFkdTQiJhZGOg8xBO1RjqnVrZ5sg2DBwZBeaA5fLbda9a03jdbq8UtH55iDdcaA5yEdC5rkJh0xlKFJn5uGQuc1W9IBpm7ltN7hum/tW7IwGjS+kiu3HPsB6M0wbulxgAnPnofVjpmzbvF7nPSLtWWeWZbSTLvqRNPTGw8dWffXP+oPBzgZ/q8sr9R4tnfNdcwJwIlW0YKVU9pEZYxKr0jWaRyt2bo85ODQMGttXmAB2JJ40M2Yhr6uZ53czB838blLpP6R3fm72e5szpev+IHXo37T27lgtzf7B4c+5ZNY16Dpp0LVSm+7N3xeJEtgerRb8r6lIxri85iRtwNXmtZnV/vjrikSkpb+T3p1q3kd8WdKoh6Vzb0hs0KrYZd7PP5tlXg/xqoxl3nv7jzNVt6wC8/9UW2WqvrEu59hUtUda9FT9SWHvK6UrHpVyOx97+5vek2bfYirMbp9UfJ95L1v5ivTuffVDAAZcLV32kJRTmLi//QQRPlLJtk35cO3r5oCa18WEjI7RoNHUslskbF6sezaYMJKWI3W90JwhN/Uf7OBX0pq/SstfOvIbU0MuT30giYTqD+AxZwyReo02oaPjwOScTUUi5sCz9UOz78o+aty11VTZhdLQm6QhNzTtzaz8S2nxr6RlMxp0yci8qaXlmTPJ9Oi84XV/tuTxm4NKoym6zJtmguWmd6WN70q7P2u83Yy25gDZ5WsmgG1bLO1YVR+EYjzp5gxw9+eNA0fOGSZw9LtK6nxe4xBauUcqXSD9c76Zyrc1Ywe2gNtn2pXbWcotMl1qezdKWxeZfdGQy2teV0Xnm7buWGWWp+WaAHj+fzbt/2Xvpuhg8tfMGIGYrA7mwHv2BFO2P5byL6UNb5tQV7rwyMH/SH9rw7FjMZZLKuhjSu2F55hA26aHeS0e6/9m21LzibzAF+Y5/9YT0tnfOfr9G1Y7InWmCnrFL81rdPWr0ud/a1ylLPqaCSH9xyWvmyoSNt0SX22OjpXbbLqayxYrfvC2XKZiMeBqqc+3Trwt+zab0LVtsbl+5kjpX56Rcs84sfXFTro2zjFjK75YpkbdQEVfM9W1fmOl7I7NW3ftQWnhL83JTqTOvNde8v9MN9uhAbkuKL17v7T4WXO9XW/p6t+Z40jMwa+keQ+b7hg7YtZ38U+kr/2gaSdNtQfNyU9z/47jIHygMds2fZPL/2AO7LWV0eng0d9kvRmmXN/7cqnnqNQMcLJtc9Dc+pEJdJFQ9Iw9bP6B7eg8tsxymfb2+daJ9YUe/Epa9ntpyW8OD18JYZmDbc9vSmd901RzGo7pkcxzsn15tBL0semaqTlQf3tOZ/Pm1/8q0yXVlP5u2zaVgtIF0uZ55uDg9pl95PaaEOD2mTdBt89MLrfZn7LM3LIaX7ZcZsruVB80cjtLmQVHblMkIu1db0LI1g+lLYsO74b0ZZk3z+ElJtydiJ1rzFiR1a/Wl6cls9/PnmAOHun59f8T6/9uSto7VzdeT15XE16OVKlrVN2xTLdoLGgUnmPC+Yl2cVXtk177nulSk8y4rdGPHl75ObTa0fdfpCufaDzOJ1hhuqlWzTTPfazdLq9poz86LsGXafa9LzN6PXq54TYbHSYaXA4dNK+tWNDYX3rkQCaZQf0Drjav36acFDRFJCwtfk56/8Hop6FyzUG902ATQrI7Hfu9oLbKVAo3zpE2vGOqdw2dMTQaOK468VDT0K610t/uqA9MHQdKY54yr0/JnGD876T65/W8SdI3Hzx61/OOVdKbd0lffGyut+0pXT7dVNMCXzaYtpuAHdhuwm31flNR/s8FLf+bGiB8oOnCdeYNtbbKvLnWVpoDeocB5sz9dBQJm+6f6v3mwF9Tbgby1ZSb67HLwQoT3uqC5o0vVGPmset1QVNR6jbChI0zL5Uy2zazLRFT+dqxypTNmxo4Tna2bQ5UsSpXdidzFtjc/XM0dbXSxndMiXrDHBNUJdP92P3r0q7PDjnQWKYC0+tyqfcVUkHvI1cpbNs8r6GDZop1MyZSJGIqGvMekWSbA9S1L5nqSV2tqXT847HG1Y4BVx+7qhLYIX36V2nVq8evgraU22/a2vbM+vFyZ156/G6GltizwXRRNBznJUmyzNl9TmF9RS6n0HTTbn7PdHc1PAHzZpjun16jzCcYk9GVEYlIK16S5k6NnlhYZvBomzNNV1JdjamKjn3WdCE2ZX2rZ0pz723eSVNeV2ny6uPfrxkIHwAQU7lbWv1nE0Qadst4M6UzLzFho+dlJ9+ngzbPM2fBB/eakHPpz031MhYe+nxL+tZ/Nb+KsPtzU62IVUCDldGTj+hYhdjluhpJDQJNo3ATvez2SvndGweNnDMOr+g5IVwnLfm1qWSVf2HO8psy2D2vqwkbvUaZrm2nTroq90jv/MxU6Ro681Lpquea3yVSUy7Nny4t/b056ck9Ixq6ooEr9wzz3OREl6flJrz7nPABAIeybVNB+ud8M5iz20Unf3UvsF36y431ZXrJdBtd8djxqx2nu0jEBLfyLxp0PUQv11aZQc89Rx29yuWUzfOkN39k2lZ8n6kAtqS6GeuCTsHfRPgAgFNFOGQ+3bD4OanPFdGxHQkaM4GTQyQc7cbLTnVLWoTwAQCnmlDNyV+pwWmtOcfvU2DkGgCcBggeOIU0O3wsXLhQY8aMUWFhoSzL0uzZsxvdbtu27r33XnXq1Enp6ekqLi7Wxo1H+9ItAABwuml2+KiqqtLgwYP17LPPHvH2Rx99VE8//bR+/etfa8mSJcrMzNSoUaNUU1PT4sYCAIDWr9nfPTx69GiNHn3kzx7btq0nn3xSP//5zzV27FhJ0ksvvaQOHTpo9uzZ+rd/+7eWtRYAALR6CR3zUVpaqp07d6q4uDi+LDc3V8OGDdNHH32UyE0BAIBWKkG/XW7s3Gm+LrlDh8Zfxd2hQ4f4bYcKBoMKBuu/YS4QOMqPbQEAgFNCyj/tMm3aNOXm5sanoqKiVDcJAAAkUULDR8eO5utgd+3a1Wj5rl274rcdasqUKSovL49P27Y59OubAAAgJRIaPrp3766OHTvqvffeiy8LBAJasmSJhg8ffsTH+P1+5eTkNJoAAMCpq9ljPiorK7Vp06b49dLSUq1cuVJt2rRRly5dNHnyZD300EPq2bOnunfvrnvuuUeFhYW66qqrEtluAADQSjU7fCxbtkyXXHJJ/Pqdd94pSbrhhhv0wgsv6Cc/+Ymqqqr0/e9/XwcOHNCFF16ov//970pL49v5AAAAv+0CAAASgN92AQAAJy3CBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABwFOEDAAA4ivABAAAcRfgAAACOInwAAABHET4AAICjCB8AAMBRhA8AAOAowgcAAHAU4QMAADiK8AEAABxF+AAAAI4ifAAAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABwFOEDAAA4ivABAAAcRfgAAACOInwAAABHET4AAICjCB8AAMBRhA8AAOAowgcAAHAU4QMAADgqKeGjoqJCkydPVteuXZWenq4LLrhAS5cuTcamAABAK5OU8DFp0iTNnTtXf/jDH7RmzRpddtllKi4u1pdffpmMzQEAgFbEsm3bTuQKq6urlZ2drddff11XXnllfPmQIUM0evRoPfTQQ8d8fCAQUG5ursrLy5WTk5PIpgEAgCRpzvHbk+iN19XVKRwOKy0trdHy9PR0ffDBB4fdPxgMKhgMxq8HAoFENwkAAJxEEt7tkp2dreHDh+vBBx/U9u3bFQ6H9fLLL+ujjz7Sjh07Drv/tGnTlJubG5+KiooS3SQAAHASSXi3iyRt3rxZN910kxYuXCi3261zzz1XvXr10ieffKJ169Y1uu+RKh9FRUV0uwAA0IqktNtFks4880wtWLBAVVVVCgQC6tSpk6677jr16NHjsPv6/X75/f5kNAMAAJyEkvo9H5mZmerUqZP279+vOXPmaOzYscncHAAAaAWSUvmYM2eObNtW7969tWnTJv34xz9Wnz59dOONNyZjcwAAoBVJSuWjvLxcJSUl6tOnj66//npdeOGFmjNnjrxebzI2BwAAWpGkDDhtCb7nAwCA1qc5x29+2wUAADiK8AEAABxF+AAAAI4ifAAAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABwFOEDAAA4ivABAAAcRfgAAACOInwAAABHET4AAICjCB8AAMBRhA8AAOAowgcAAHAU4QMAADiK8AEAABxF+AAAAI4ifAAAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABwFOEDAAA4ivABAAAcRfgAAACOSnj4CIfDuueee9S9e3elp6frzDPP1IMPPijbthO9KQAA0Ap5Er3CX/ziF3ruuef04osvqn///lq2bJluvPFG5ebm6rbbbkv05gAAQCuT8PDx4YcfauzYsbryyislSd26ddOf/vQnffzxx4neFAAAaIUS3u1ywQUX6L333tOGDRskSatWrdIHH3yg0aNHH/H+wWBQgUCg0QQAAE5dCa983H333QoEAurTp4/cbrfC4bAefvhhTZgw4Yj3nzZtmu6///5ENwMAAJykEl75+POf/6xXXnlFf/zjH7V8+XK9+OKLeuyxx/Tiiy8e8f5TpkxReXl5fNq2bVuimwQAAE4ilp3gj6EUFRXp7rvvVklJSXzZQw89pJdfflmff/75cR8fCASUm5ur8vJy5eTkJLJpAAAgSZpz/E545ePgwYNyuRqv1u12KxKJJHpTAACgFUr4mI8xY8bo4YcfVpcuXdS/f3+tWLFCTzzxhG666aZEbwoAALRCCe92qaio0D333KNZs2Zp9+7dKiws1Pjx43XvvffK5/Md9/F0uwAA0Po05/id8PDRUoQPAABan5SO+QAAADgWwgcAAHAU4QMAADiK8AEAABxF+AAAAI4ifAAAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHOVJdQMAAKefcDisUCiU6magmXw+n1yultctCB8AAMfYtq2dO3fqwIEDqW4KToDL5VL37t3l8/latB7CBwDAMbHg0b59e2VkZMiyrFQ3CU0UiUS0fft27dixQ126dGnRc0f4AAA4IhwOx4NH27ZtU90cnICCggJt375ddXV18nq9J7weBpwCABwRG+ORkZGR4pbgRMW6W8LhcIvWQ/gAADiKrpbWK1HPHeEDAAA4ivABAICDunXrpieffDLl60glBpwCAHAM3/jGN3T22Wcn7GC/dOlSZWZmJmRdrRXhAwCAFrJtW+FwWB7P8Q+rBQUFDrTo5Ea3CwAARzFx4kQtWLBATz31lCzLkmVZ2rJli+bPny/LsvT2229ryJAh8vv9+uCDD7R582aNHTtWHTp0UFZWls477zy9++67jdZ5aJeJZVn63e9+p3HjxikjI0M9e/bUG2+80ax2lpWVaezYscrKylJOTo6uvfZa7dq1K377qlWrdMkllyg7O1s5OTkaMmSIli1bJknaunWrxowZo/z8fGVmZqp///566623TnynNQGVDwBASti2repQyz6yeaLSve4mfXLjqaee0oYNGzRgwAA98MADkkzlYsuWLZKku+++W4899ph69Oih/Px8bdu2TVdccYUefvhh+f1+vfTSSxozZozWr1+vLl26HHU7999/vx599FH98pe/1DPPPKMJEyZo69atatOmzXHbGIlE4sFjwYIFqqurU0lJia677jrNnz9fkjRhwgSdc845eu655+R2u7Vy5cr493SUlJSotrZWCxcuVGZmptauXausrKzjbrclCB8AgJSoDoXV7945Kdn22gdGKcN3/ENgbm6ufD6fMjIy1LFjx8Nuf+CBB/TNb34zfr1NmzYaPHhw/PqDDz6oWbNm6Y033tCtt9561O1MnDhR48ePlyQ98sgjevrpp/Xxxx/r8ssvP24b33vvPa1Zs0alpaUqKiqSJL300kvq37+/li5dqvPOO09lZWX68Y9/rD59+kiSevbsGX98WVmZrr76ag0cOFCS1KNHj+Nus6XodgEA4AQNHTq00fXKykrddddd6tu3r/Ly8pSVlaV169aprKzsmOsZNGhQ/HJmZqZycnK0e/fuJrVh3bp1KioqigcPSerXr5/y8vK0bt06SdKdd96pSZMmqbi4WNOnT9fmzZvj973tttv00EMPacSIEZo6dapWr17dpO22BJUPAEBKpHvdWvvAqJRtOxEO/dTKXXfdpblz5+qxxx7TWWedpfT0dP3rv/6ramtrj7meQ7+q3LIsRSKRhLRRku677z595zvf0Ztvvqm3335bU6dO1cyZMzVu3DhNmjRJo0aN0ptvvql33nlH06ZN0+OPP64f/vCHCdv+oQgfAICUsCyrSV0fqebz+Zr8deKLFi3SxIkTNW7cOEmmEhIbH5Isffv21bZt27Rt27Z49WPt2rU6cOCA+vXrF79fr1691KtXL91xxx0aP368ZsyYEW9nUVGRbr75Zt18882aMmWKnn/++aSGD7pdAAA4hm7dumnJkiXasmWL9u7de8yKRM+ePfXaa69p5cqVWrVqlb7zne8ktIJxJMXFxRo4cKAmTJig5cuX6+OPP9b111+viy++WEOHDlV1dbVuvfVWzZ8/X1u3btWiRYu0dOlS9e3bV5I0efJkzZkzR6WlpVq+fLnmzZsXvy1ZCB8AABzDXXfdJbfbrX79+qmgoOCY4zeeeOIJ5efn64ILLtCYMWM0atQonXvuuUltn2VZev3115Wfn6+vf/3rKi4uVo8ePfTqq69Kktxut/bt26frr79evXr10rXXXqvRo0fr/vvvl2R+JK6kpER9+/bV5Zdfrl69eulXv/pVctts27ad1C00UyAQUG5ursrLy5WTk5Pq5gAAEqSmpkalpaXq3r270tLSUt0cnIBjPYfNOX5T+QAAAI4ifAAAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AABwkvrGN76hyZMnp7oZCUf4AADgGJIRACZOnKirrroqoetsTRIePrp16ybLsg6bSkpKEr0pAADQCiU8fCxdulQ7duyIT3PnzpUkXXPNNYneFAAASTVx4kQtWLBATz31VPxkesuWLZKkTz/9VKNHj1ZWVpY6dOigf//3f9fevXvjj/3rX/+qgQMHKj09XW3btlVxcbGqqqp033336cUXX9Trr78eX+f8+fOb1J79+/fr+uuvV35+vjIyMjR69Ght3LgxfvvWrVs1ZswY5efnKzMzU/3799dbb70Vf+yECRNUUFCg9PR09ezZUzNmzEjYvmoOT6JXWFBQ0Oj69OnTdeaZZ+riiy9O9KYAAK2ZbUuhg6nZtjdDsqzj3u2pp57Shg0bNGDAAD3wwAOSzHHuwIEDuvTSSzVp0iT913/9l6qrq/XTn/5U1157rd5//33t2LFD48eP16OPPqpx48apoqJC//jHP2Tbtu666y6tW7dOgUAgfvBv06ZNk5o9ceJEbdy4UW+88YZycnL005/+VFdccYXWrl0rr9erkpIS1dbWauHChcrMzNTatWuVlZUlSbrnnnu0du1avf3222rXrp02bdqk6urqE9yBLZPw8NFQbW2tXn75Zd15552yjvIkB4NBBYPB+PVAIJDMJgEAThahg9IjhanZ9v/bLvkyj3u33Nxc+Xw+ZWRkqGPHjvHl//3f/61zzjlHjzzySHzZ//zP/6ioqEgbNmxQZWWl6urq9O1vf1tdu3aVJA0cODB+3/T0dAWDwUbrPJ5Y6Fi0aJEuuOACSdIrr7yioqIizZ49W9dcc43Kysp09dVXx7fVo0eP+OPLysp0zjnnaOjQoZLMMIlUSeqA09mzZ+vAgQOaOHHiUe8zbdo05ebmxqeioqJkNgkAgBZbtWqV5s2bp6ysrPjUp08fSdLmzZs1ePBgjRw5UgMHDtQ111yj559/Xvv372/RNtetWyePx6Nhw4bFl7Vt21a9e/fWunXrJEm33XabHnroIY0YMUJTp07V6tWr4/e95ZZbNHPmTJ199tn6yU9+og8//LBF7WmJpFY+fv/732v06NEqLDx6sp0yZYruvPPO+PVAIEAAAYDTgTfDVCBSte0WqKys1JgxY/SLX/zisNs6deokt9utuXPn6sMPP9Q777yjZ555Rj/72c+0ZMkSde/evUXbPpZJkyZp1KhRevPNN/XOO+9o2rRpevzxx/XDH/5Qo0eP1tatW/XWW29p7ty5GjlypEpKSvTYY48lrT1Hk7TKx9atW/Xuu+9q0qRJx7yf3+9XTk5OowkAcBqwLNP1kYqpCeM9Ynw+n8LhcKNl5557rj777DN169ZNZ511VqMpMzMz+udZGjFihO6//36tWLFCPp9Ps2bNOuo6j6dv376qq6vTkiVL4sv27dun9evXq1+/fvFlRUVFuvnmm/Xaa6/pRz/6kZ5//vn4bQUFBbrhhhv08ssv68knn9Rvf/vbZrUhUZIWPmbMmKH27dvryiuvTNYmAABIum7dumnJkiXasmWL9u7dq0gkopKSEn311VcaP368li5dqs2bN2vOnDm68cYbFQ6HtWTJEj3yyCNatmyZysrK9Nprr2nPnj3q27dvfJ2rV6/W+vXrtXfvXoVCoeO2o2fPnho7dqy+973v6YMPPtCqVav03e9+V2eccYbGjh0rSZo8ebLmzJmj0tJSLV++XPPmzYtv895779Xrr7+uTZs26bPPPtPf/va3+G1OS0r4iEQimjFjhm644QZ5PEnt2QEAIKnuuusuud1u9evXTwUFBSorK1NhYaEWLVqkcDisyy67TAMHDtTkyZOVl5cnl8ulnJwcLVy4UFdccYV69eqln//853r88cc1evRoSdL3vvc99e7dW0OHDlVBQYEWLVrUpLbMmDFDQ4YM0be+9S0NHz5ctm3rrbfektfrlSSFw2GVlJSob9++uvzyy9WrVy/96le/kmSqLVOmTNGgQYP09a9/XW63WzNnzkzOTjsOy7ZtO9ErfeeddzRq1CitX79evXr1atZjA4GAcnNzVV5eThcMAJxCampqVFpaqu7duystLS3VzcEJONZz2Jzjd1LKEpdddpmSkGkAAMApgN92AQAAjiJ8AAAARxE+AACAowgfAADAUYQPAICj+EBC65Wo547wAQBwROy7KA4eTNEv2aLFamtrJUlut7tF6+EbwAAAjnC73crLy9Pu3bslSRkZGUf9xXOcfCKRiPbs2aOMjIwWf4Eo4QMA4JjYT8jHAghaF5fLpS5durQ4NBI+AACOsSxLnTp1Uvv27Zv0eyY4ufh8PrlcLR+xQfgAADjO7Xa3eNwAWi8GnAIAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABwFOEDAAA4ivABAAAcRfgAAACOInwAAABHET4AAICjCB8AAMBRhA8AAOAowgcAAHAU4QMAADiK8AEAABxF+AAAAI4ifAAAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARyUlfHz55Zf67ne/q7Zt2yo9PV0DBw7UsmXLkrEpAADQyngSvcL9+/drxIgRuuSSS/T222+roKBAGzduVH5+fqI3BQAAWqGEh49f/OIXKioq0owZM+LLunfvnujNAACAVirh3S5vvPGGhg4dqmuuuUbt27fXOeeco+eff/6o9w8GgwoEAo0mAABw6kp4+PjnP/+p5557Tj179tScOXN0yy236LbbbtOLL754xPtPmzZNubm58amoqCjRTQIAACcRy7ZtO5Er9Pl8Gjp0qD788MP4sttuu01Lly7VRx99dNj9g8GggsFg/HogEFBRUZHKy8uVk5OTyKYBAIAkCQQCys3NbdLxO+GVj06dOqlfv36NlvXt21dlZWVHvL/f71dOTk6jCQAAnLoSHj5GjBih9evXN1q2YcMGde3aNdGbAgAArVDCw8cdd9yhxYsX65FHHtGmTZv0xz/+Ub/97W9VUlKS6E0BAIBWKOHh47zzztOsWbP0pz/9SQMGDNCDDz6oJ598UhMmTEj0pgAAQCuU8AGnLdWcASsAAODkkNIBpwAAAMdC+AAAAI4ifAAAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABwFOEDAAA4ivABAAAcRfgAAACOInwAAABHET4AAICjCB8AAMBRhA8AAOAowgcAAHAU4QMAADiK8AEAABxF+AAAAI4ifAAAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABwFOEDAAA4ivABAAAcRfgAAACOSnj4uO+++2RZVqOpT58+id4MAABopTzJWGn//v317rvv1m/Ek5TNAACAVigpqcDj8ahjx47JWDUAAGjlkjLmY+PGjSosLFSPHj00YcIElZWVHfW+wWBQgUCg0QQAAE5dCQ8fw4YN0wsvvKC///3veu6551RaWqqLLrpIFRUVR7z/tGnTlJubG5+KiooS3SQAAHASsWzbtpO5gQMHDqhr16564okn9B//8R+H3R4MBhUMBuPXA4GAioqKVF5erpycnGQ2DQAAJEggEFBubm6Tjt9JHwmal5enXr16adOmTUe83e/3y+/3J7sZAADgJJH07/morKzU5s2b1alTp2RvCgAAtAIJDx933XWXFixYoC1btujDDz/UuHHj5Ha7NX78+ERvCgAAtEIJ73b54osvNH78eO3bt08FBQW68MILtXjxYhUUFCR6UwAAoBVKePiYOXNmolcJAABOIfy2CwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABwFOEDAAA4ivABAAAcRfgAAACOInwAAABHET4AAICjCB8AAMBRhA8AAOAowgcAAHAU4QMAADiK8AEAABxF+AAAAI4ifAAAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABwFOEDAAA4ypPqBjilti6isq+q5HO75fO45Pe45ItOHpcly7JS3UQAAE4Lp0342H6gWsVPLDzibS5LJoi4XfJ73SaMSPFAYlnRSVZ0LrksSx63JbfLJa/bksdlyeNyyeO25HGbQON1W8pN9yovw6fcdK/yM3zKy/AqL7osL8Or3HSvJCkUjqi2LqLa6DwUtqPXw6qts+V2WcrwuZXucyvD51aG16N0n1teN8EJANC6nDbhoy5iKzfdGz/AhyN2/LaILdWEIqoJRaSauhS2svncLksZXhNKMv0epXvdyvS7leHz1M99bqVH5xl+j9K8LqV73Ur3upXmcyvNYx6f7nUrzetSmtctt8uSy7Lktiy5XCZsuV0mfLnjlwk9AIDms2zbto9/N+cEAgHl5uaqvLxcOTk5SdtOXbi+ylBbF1EwOtXWRVQXici2JVuSbdvRuSTZ8eWRiK1wxFYoYqsuHFFdxFZd2FZdJKK6sLktWBdWoKZOBw7W6sDBkPYfDKm8uvHlULh+91uW5HO74t1C3uhlr9ulcMRWdW1YB2vrVB0KN3pcqrgsyeNyye0ylR93tALkblAF8ntcSvd5lBENRenxMBSt4PhMYIp1gfnjFShXvIvM7ANLph51dLZs1YTCqgqGdTC6r2LzqmBY1SFzOcPnUZtMn9pk+tQ2PverTZZPmT43oQoATkBzjt+nTeXjUB63Sx63Sxm+1LXBtm0drA3HQ4fH3fTxv6FwRAdrw/FAYg6y0XBSG1ZVg4Nuo3ltWDWh+qk6FFZNKKLq2sbLIk3INhFbqg1HpHALdsJJxudxqU2GTxl+d7x7zWWZio+rQRXIsqx415rP4zaBKTaOyO2S12PJ53bL66kPTbFMYzW6HK0muSx5o911HnesK8/Mve5YkKuvTKU1vNygYuVtxmtIMq+j6lBYNbXR10H0+a8LR5Sd5lVOuke56V6lewllABLntA0fJwPLspTpP7GnwOt2KTfdFR8zkmi2bao8YdtUcRpejkRshW0zr4tWgMy8vgJklkUUDJmQVNUgFFUfGpZCEdXWheNdYsHQ4VWpUDjSpHanR6spGdHKSmbscrQLKs3r1sFgnb6qqtW+qlp9FZ32VQVVEzLb2xmoSco+dYrbVd9dZuZWg2WWbFvxkBluSsqU5HFZykn3KifNhJGcdK+y/B5FbPN814ZNxS8UjigUsRWKVhDrwrY8bivapWemWPdeus8tf7TLz21ZsmUrYkuR6GvPthtfb9gGMzftyEn3xC9n+tzx12Mo1qZoO2KXIxEzhipebYuGxOaeAAA4cYQPHJEVO9uXJa871a1xRnVtWPuqgvqqqlY1oYgith0/8B1pHooedGMhqeGg4WCDZZKi3XV2/HJDEduOhrbYwdJcDoXNwOO6iJkH6yIKxqtWEdXU1V9uKByxFZbdrIqUy1KjgOBxW6qoqVOgOmTaFrHjQe1UFht8HqtCWZYll6X6uRpcd0Wf10OCUsSu766N2LYsKdql6I5Xx+rn7ngXayQW7uNzxS/HTgBijzXdktHHe6PLouuKBfBMvzs6BswT7+LMiI4Nc1uWQpFIPKgdqevY7bLULsuvNpk++TyEMiQW4QOISve51dmXoc75GaluSrPYdiyYmODT8IDV8HKscmXJfHIqXoXwmQPZkbpVbNtWdSis8uqQAtV1CtSEVH4wpEBNSJXBOrksq0HXkEu+WHeRxyVvtAupLta106BbJxgyXYax65GIHT3Am4O7y1Xf1RXr/gqFIwrUmDYEqkMK1NSpotq0pbw6dNRxUJYleWNjk9ymChQO2wpGw2JDjQafIy4nzaN22X61y/SrbZZPbbN8apflV36GT+neBgHI624UhNK85nJTeuyqa80YuYqaUHxe0Whep3DEjlesDu3qjC3P8LnVJtbO6JiuLL+HbsOTDOEDaOUsy4pXLJKxbnPG7FGn3ISvPmFiAawqWGcGQbtjY3JM6DjW40LR7qKGH3UP1kXi1QxbtiKRBt1B0e6hcMRuVBVpNDaoQcUkHg7j3Yjhwwa5h8KReOhyxx7vsuR2NVyfCWDBunCjsBkMhePrinWnNRpoHe32PFgbVlWwTsEGgcsd7ZLzxgaKx8YduSzVhm3tP1ircMSOhr46/XNPlRNPZ8L5PC61zTShqU2mX/kZ3ng3my/29QjuaNdbdLC8123Fx3eZAFz/XKtBKPZEQ6234VitBuHb67ZUF7FVWVOnimCdKmvqVBkMHXK9ThHbVrssvwqyo1OWX+2i84Jsf1L+v1Mp6eFj+vTpmjJlim6//XY9+eSTyd4cgNPQiQYwy7Lk85jxH5n+JDXuJFMXjshW/XigY4lEbJVXh7SvKqg9FWZs1L7KWu2rDGpPZa0OHKyNBp+wgtGuwGAoUr8sGpJiXY4xh3Y92pIyfG5lp3mU7feaeZoZ3xO7nJ3mkcftindzmu9AOryr82BtWPuqTBu/qqrVwVpzvx3lNdpR3nrHc2WnmU/pWdJRx0ZFop/KjHXHxb9SITbeyudWWrRrrkNOmkouOStlf09Sw8fSpUv1m9/8RoMGDUrmZgAATdScQbUul6X8TJ/yM306q30SG5VEsbFc+yrNmKW9lUGVV4caD5IONxhn1WD8VXwMj8zBXYeM+zIVsMghA5wbD7gORSJyW5ay08wg7aw0j7L9HmVGL2f5TcCSpL2VtdpTEdSeiqD2Vpr5nsqgausi8a6nROlRkHlqho/KykpNmDBBzz//vB566KFkbQYAgKNqrWO5YmzbdHvtqQiqvNoM+G44Pir2cf1Yt59kfk6k/qsUGoy3qg2bgeq1YeUk6ZOSTZW08FFSUqIrr7xSxcXFxwwfwWBQwWAwfj0QCCSrSQAAtCqWZX6mI1lfq5AqSQkfM2fO1PLly7V06dLj3nfatGm6//77k9EMAABwEkr4h7e3bdum22+/Xa+88orS0tKOe/8pU6aovLw8Pm3bti3RTQIAACeRhP+2y+zZszVu3Di53fWjzsPhcPRLeVwKBoONbjuUU7/tAgAAEielv+0ycuRIrVmzptGyG2+8UX369NFPf/rTYwYPAABw6kt4+MjOztaAAQMaLcvMzFTbtm0PWw4AAE4/fGE/AABwlCNfrz5//nwnNgMAAFoBKh8AAMBRhA8AAOAowgcAAHAU4QMAADiK8AEAABxF+AAAAI5y5KO2zRH7tnd+3RYAgNYjdtxuyq+2nHTho6KiQpJUVFSU4pYAAIDmqqioUG5u7jHvk/AflmupSCSi7du3Kzs7W5ZlJXTdgUBARUVF2rZtGz9a5wD2t7PY385ifzuL/e2sE9nftm2roqJChYWFcrmOParjpKt8uFwude7cOanbyMnJ4cXrIPa3s9jfzmJ/O4v97azm7u/jVTxiGHAKAAAcRfgAAACOOq3Ch9/v19SpU+X3+1PdlNMC+9tZ7G9nsb+dxf52VrL390k34BQAAJzaTqvKBwAASD3CBwAAcBThAwAAOIrwAQAAHHXahI9nn31W3bp1U1pamoYNG6aPP/441U06ZSxcuFBjxoxRYWGhLMvS7NmzG91u27buvfdederUSenp6SouLtbGjRtT09hWbtq0aTrvvPOUnZ2t9u3b66qrrtL69esb3aempkYlJSVq27atsrKydPXVV2vXrl0panHr9txzz2nQoEHxL1oaPny43n777fjt7Ovkmj59uizL0uTJk+PL2OeJc99998myrEZTnz594rcnc1+fFuHj1Vdf1Z133qmpU6dq+fLlGjx4sEaNGqXdu3enummnhKqqKg0ePFjPPvvsEW9/9NFH9fTTT+vXv/61lixZoszMTI0aNUo1NTUOt7T1W7BggUpKSrR48WLNnTtXoVBIl112maqqquL3ueOOO/R///d/+stf/qIFCxZo+/bt+va3v53CVrdenTt31vTp0/XJJ59o2bJluvTSSzV27Fh99tlnktjXybR06VL95je/0aBBgxotZ58nVv/+/bVjx4749MEHH8RvS+q+tk8D559/vl1SUhK/Hg6H7cLCQnvatGkpbNWpSZI9a9as+PVIJGJ37NjR/uUvfxlfduDAAdvv99t/+tOfUtDCU8vu3bttSfaCBQts2zb71uv12n/5y1/i91m3bp0tyf7oo49S1cxTSn5+vv273/2OfZ1EFRUVds+ePe25c+faF198sX377bfbts3rO9GmTp1qDx48+Ii3JXtfn/KVj9raWn3yyScqLi6OL3O5XCouLtZHH32UwpadHkpLS7Vz585G+z83N1fDhg1j/ydAeXm5JKlNmzaSpE8++UShUKjR/u7Tp4+6dOnC/m6hcDismTNnqqqqSsOHD2dfJ1FJSYmuvPLKRvtW4vWdDBs3blRhYaF69OihCRMmqKysTFLy9/VJ98NyibZ3716Fw2F16NCh0fIOHTro888/T1GrTh87d+6UpCPu/9htODGRSESTJ0/WiBEjNGDAAElmf/t8PuXl5TW6L/v7xK1Zs0bDhw9XTU2NsrKyNGvWLPXr108rV65kXyfBzJkztXz5ci1duvSw23h9J9awYcP0wgsvqHfv3tqxY4fuv/9+XXTRRfr000+Tvq9P+fABnKpKSkr06aefNuqjReL17t1bK1euVHl5uf7617/qhhtu0IIFC1LdrFPStm3bdPvtt2vu3LlKS0tLdXNOeaNHj45fHjRokIYNG6auXbvqz3/+s9LT05O67VO+26Vdu3Zyu92HjdDdtWuXOnbsmKJWnT5i+5j9n1i33nqr/va3v2nevHnq3LlzfHnHjh1VW1urAwcONLo/+/vE+Xw+nXXWWRoyZIimTZumwYMH66mnnmJfJ8Enn3yi3bt369xzz5XH45HH49GCBQv09NNPy+PxqEOHDuzzJMrLy1OvXr20adOmpL++T/nw4fP5NGTIEL333nvxZZFIRO+9956GDx+ewpadHrp3766OHTs22v+BQEBLlixh/58A27Z16623atasWXr//ffVvXv3RrcPGTJEXq+30f5ev369ysrK2N8JEolEFAwG2ddJMHLkSK1Zs0YrV66MT0OHDtWECRPil9nnyVNZWanNmzerU6dOyX99t3jIaiswc+ZM2+/32y+88IK9du1a+/vf/76dl5dn79y5M9VNOyVUVFTYK1assFesWGFLsp944gl7xYoV9tatW23btu3p06fbeXl59uuvv26vXr3aHjt2rN29e3e7uro6xS1vfW655RY7NzfXnj9/vr1jx474dPDgwfh9br75ZrtLly72+++/by9btswePny4PXz48BS2uvW6++677QULFtilpaX26tWr7bvvvtu2LMt+5513bNtmXzuh4addbJt9nkg/+tGP7Pnz59ulpaX2okWL7OLiYrtdu3b27t27bdtO7r4+LcKHbdv2M888Y3fp0sX2+Xz2+eefby9evDjVTTplzJs3z5Z02HTDDTfYtm0+bnvPPffYHTp0sP1+vz1y5Eh7/fr1qW10K3Wk/SzJnjFjRvw+1dXV9g9+8AM7Pz/fzsjIsMeNG2fv2LEjdY1uxW666Sa7a9euts/nswsKCuyRI0fGg4dts6+dcGj4YJ8nznXXXWd36tTJ9vl89hlnnGFfd9119qZNm+K3J3NfW7Zt2y2vnwAAADTNKT/mAwAAnFwIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABw1P8HkLouw1DpZ28AAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.plot(history.history['loss'], label='train loss')\n",
        "plt.plot(history.history['val_loss'], label='test loss')\n",
        "plt.legend();"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 430
        },
        "id": "lPUO7Rz1hJdv",
        "outputId": "1b5bb313-883b-4fcb-e6d9-08e37703be94"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA+SklEQVR4nO3deXxU5aH/8e8smcm+AdkgkKggIouAipGiqBRES0WvP71qRarVWqGVWtpqqyi2t6HW2mJrbW+9xXqvFisVsFoXXAiyKgoqYkGWmCgJYcuezHp+fzyZCZEEkpDksHzeL87rzJw5Z84zT4Y53/M8z5lxWJZlCQAAwCZOuwsAAABOboQRAABgK8IIAACwFWEEAADYijACAABsRRgBAAC2IowAAABbEUYAAICt3HYXoD3C4bB27dqlpKQkORwOu4sDAADawbIs1dTUKCcnR05n2+0fx0UY2bVrl3Jzc+0uBgAA6ITS0lL169evzcePizCSlJQkybyY5ORkm0sDAADao7q6Wrm5udHjeFuOizAS6ZpJTk4mjAAAcJw50hALBrACAABbEUYAAICtCCMAAMBWhBEAAGArwggAALAVYQQAANiKMAIAAGxFGAEAALYijAAAAFsRRgAAgK06FEYKCwt1zjnnKCkpSRkZGZo6daq2bNnS7u0XLlwoh8OhqVOndrScAADgBNWhMFJUVKQZM2Zo7dq1WrZsmQKBgCZOnKi6urojbltcXKzZs2dr3LhxnS4sAAA48XToh/JeeeWVFveffPJJZWRk6L333tMFF1zQ5nahUEg33HCD5s6dq7fffluVlZWdKmyX2/QP6bM10qBJUt5XpJg4u0sEAMBJ56h+tbeqqkqSlJ6eftj1HnzwQWVkZOiWW27R22+/fcTn9fl88vl80fvV1dVHU8y2ffCs9Omr0rt/ltxxUv4F0sCvmnCS2v/w2zZUSqXvSKVrpZK10q6NUnq+NHq6NPwaKTale8rcXpYl1e+X4tIkJ0OD2hRoMFP84d/DAIDu47Asy+rMhuFwWF//+tdVWVmplStXtrneypUr9Z//+Z/auHGjevfurenTp6uyslJLlixpc5sHHnhAc+fOPWR5VVWVkpOTO1Pc1n26TPr3S9Knr0nVX7R8rM9gaeBEE0xyx0jVu6TSdVLJGqlknVSxWVIbVRcTLw39D+nsm6W+ozpWpnBYCvmlmNiOv54Dn0nFb0s7V5ippkzqPUg67w5pxH/2fMuPZUmNVVLtbqmm3MwbDkhOt+T2Su5YyeUxc7dHcnnN8pg4Kf1Us6y7NFZJ6/5bWvuYKdPgr0nj75ayhnXfPtE1wiFp4zPSxqel1AHSsKulU8ZLrhi7S4aeFGiU6vZIKf2kI/w8PexTXV2tlJSUIx6/Ox1GvvOd7+jll1/WypUr1a9fv1bXqamp0fDhw/WHP/xBkydPlqR2hZHWWkZyc3O7PoxEWJa0+2MTSj59zYQOK9z8uDNGCgcO3S79FKl/gQkrOSNNUFn/F2nPv5vXyR5hQsnQqyVvYsvtg36zbvmHUtmHZl6+SfLXSEnZ5vnT8k2LS/opTVN+c6tLdVnL8FH5WduvMb6XdPYt0jnfkpIyO19XrWmskra8LH2+Xqotl2p2m3lthRRs7NxzxiSYrrNTxkunXmTCYVd84NTvl9b9UVr7R8lXdejjZ3xdGn+PlDnk6PeFrrftdem1OVLFxy2Xx/eShlxh/p/1Lzi+WwNDAWnPFiltgORNsrs0x45QUCr7QNq53Hzelaw1ny+nTZCmPi4lZthdQrSiW8PIzJkztXTpUq1YsUL5+fltrrdx40aNHDlSLpcruiwcNgd5p9OpLVu26NRTTz3i/tr7YrpMwwFp2xum5WTbMql+nzmbzx4h5Z4n9W+aWnvzW1ZTKFkgbV5iWjkkyZNkum/6DG4KHx+YIBJ5vCPie5kPqQPFLZc73VLOKNPdlH+BlDFE+ug5ad3jUmWJWcflkYZdIxXcIWWe2fF9R/hqpC2vSB8vNnV0uNcRmyIlZpkQFJcuhYNm/aDPTCFf8+2gT/JVm+lgiZkmmJxykXTKhVJyTsfKW7dXWvOY9M6fTdiTpN6nSxf80NTD2w9Lm56Xae1ySGdeaVpK+pzesf0crUgwDvqkrKGmpagz2//7JenfL5rb/c+TzphippTWTxyOeWUfSsvmSDveMvdjU6Tzv2da2z5ebM6QI5L7mr/fsKul7LOO/bPmQIMJ8p+tlkpWm+7fQL3pYv1/T5r3/cnIsqSKT5pOtoqk4lWtn0BIUkKGdNWfpFMv7tky4oi6JYxYlqXvfve7Wrx4sZYvX66BAwcedv3GxkZt27atxbJ7771XNTU1mj9/vgYNGiSP58hN8T0eRg4WDkn7d5gPOE98x7at2yd98IwJJvu3t76ON0XKHi5lDW+eJ2ZKlcXS/p1m39H5Dqmu4qCNHSYgRcJH//NaP5MKBc2Bae0fTKtPxCkXSQUzzH9gp+vQ7b7MV2vG2Gx63gS1UHPrlXoPkgZdasbaJGY0h4/EzI53D4XD0u5N0o7l5uDz2epDW1h6n27qKznH/G2S+zbfTujTfGZcs1ta/ahpsQrUm2WZQ6ULZktnXNHyDHr3ZqlonrR5adMChzmgXfhjqffh3+tHJdBgPnC3vCxtfVWq2WWWu7ymmy93jPnb5o5pfWxLKGjGLkUCSCR4tiZnlDTk66YFqNeRTwS6RCgo7fvUjKsq+8CE+6yhpiw5Zx3+7L/qC+nNn0sf/E2SZcL0ubdJ437QXBehoFS8QvroH9In/2x5wEo/VTrretM6eayMC4qMNytZbd7bX7x/aMtrpDXW4ZIm/lw67zudC1WBRhPYUvsf+6FMMieC29+Utr5m5i0+72RCaN64ps+8C00L9qKbpT2fmMfH3ilddG/3dvG2prrMfFZtf9OcZDrd5r3q8pgTishtl8eUzZssjfzG0Z0QVpZKy+dJcanmc7yjJ2g9pFvCyB133KFnnnlGS5cu1emnN58xpqSkKC7OHHCmTZumvn37qrCwsNXnaE83zZfZGka6gmWZg82G/zNdGlnDzIE0e4Tp8+7Ih4SvxrSI1O8z28eldawspe+acRKbl7bsivImS7GpUlyKmcemmDd5bNO0+yPzARFsaN4m/VRp6FXSmVdJGWd034ddoNGEqEg42bVRbY7XkcwHQVKOlJRlWqEiQSb7LOnCH0mDJh++Gb98k7S80BzYJcnhNGfamUNNfcelmQNb5HZcuglcHXn9NeUmeGx9xbyuSFCSzJijmDjzN/6y3oOaw4k3ybRObX1FatjfvI471gTMwZebA/7OImnzC6bF7uB6yzizqcXkaybEOZwmlDqc5iDY4n47X1soYD6MI8GjbKOpz4PfNy04zGvqO8p0deaMMv8/Qj5p5W9NgI78/c68SrpkjumqbEvQZ4LypkWmbiL7dcdJI28w46c6E8Ii/++8Seb/hjf58AHessw4s71bpb2fmjAWuf3l8WmS6ZYdcL7pYhowVkrLk176gTmZkaTh10pT5rc/2FuW9OHfpdfvN2PHUvubADrkCqnv2cdON1ak9ePTV83nS+k6yQo1P+6OkwYUNIeP7BGH1nugQXr1p9L6/zH3c0ZJV/+P6dbuLv56Eya3NwWQis0dfw6HSxrzbdMC25ELHkIB8/9i+bzmzw2XVxp9k/SV7x9zoaRbwoijjQ+kBQsWaPr06ZKk8ePHKy8vT08++WSr656UYeRYdOAz6Z3/lt5/6tAukcNJy28KIE0HZzvOtur3S5+tMi1G1bvMh3v1LjPVlrcMWZLU7xzpgh+ZK6U6Ut6yD8x/+C3/OvK6Lq8JJp54EybcsebAEZncTXNXjPT5u9KuDS23T+5nBkufPtmc+bm90r7tzVdrla4zB7O2xKWbbU+/zIyx8SQcuk5thQlYn/zThONwsP11IZkPT6fLhD2HyxzQIsscTcvr9rRsMYuISWgK4GeZbsbyD6QvNkjVnx+6rtNt6ivSndb/fNM60G90x8rrqzGvde0fpPKPIi/ChLTzv2tCXVvvB8syweHTV804ss/WHNp6EQnwsSnN4d0VY96X+7ZJ/tq2y5Z+inldA5qmtLxDy2JZ5v/oK/eYA3T2COnap6XU3MO/7i/el17+sfT5O60/npTdFEK/bvbdnlbRrmJZpvXj83dNGP/0NamqtOU6kYsHBk6Ucs9tf1flJ/+Uls6UGitNt/jXHjFd410h2lrbFD4+W/Ol97nDhOlTLzZllpq7okMBs24o0HTfL33xXvPJTkKGNPFnJnAe6fOpZK304vebw0//ArPvktXmvssjjZpmQskx0iXb7QNYexJhpBuFgubDobHStNo0VJrbkWWR+wkZ5qwqe8Sx3dwbCppm6UhISco6/EGnPb5434xLqN9n6uXgqX5/64Ob26PvaNNKc/ql7Qt2dfvMASYSThoOSKdeIg2+zIxlcnXgSv36/aZF5ZN/mg/Xzg40bo032bxPskeY8JE9wrRGtHbQq60w9bvrfRPQvnhfqt9rHut1mvTVB03AOpq/X6Rlcs3vzcEvou/Z0vkzpcFTTN0FGqTilWadra8eOiA8Ls200rXZyvMlDpcJHb0HmW6+3gPN7V6ndazLaOfb0nM3mfdffG/pmqekvLGHrldbIb0xV9rwtCTLBMBxd0nn3GJe1+alprUoEvIk06U5+HJTxwl9mroUvAdd3XbQVW5Ol6nLkL9pOujgGpmCPtNKV1PePNVGbjcNbP/y+DJ3rAnggyaZAJI2oP1182WVpdLztzUfnEdcJ132q84NBN6/07Qq7lhu3j9fbqlM7meC/6kXm3E9He0G3PaG9PKPTHCVzP/hyx9u/Yq++v1mzNSG/zX343tJX/2Z6YKUzIUMy+eZkzTJhJKRN5q/f2uhJBQ0+929qWn62ExX/8W0unYhwgjQEyxL8tc1h5NAffN3lwQbmm8HGswBP1BvDlADJ3X9VU2dZVmmNckKmzFSVticiUfvW+Z+ONQ0DzavF266H3k8LlVKzet8N4BlmTPlmnJzptnVl+tW/NuEkg+fbT4opg4wIaF4Zcug4fKYLpPIQTLSvRP0SY3VJrw3VjYH+cYq83eOPF9aXteNXagskRbeYLodnW7p0nnmyjiHw1yV986fpKKHmls5h18rTXjg0Cb7oM8cXDe/YM7MGyvbXwaH89BWx85K7S+d1vSdTnnjOj4e73BCQTMgveiXprzpp5gDc3x6U1d0WsvJk2DqsW5vU/hoCiBfDqORK/xOvdhMvQce/YlZ0GcG1q/4lflscDilc26VLvqJ+b8UDpuuutfua+6KHTVNmjC39fCzMxJKmr5uwxljxqYMvty0rO7+2LQS7tnSegvm5b8276suRBgBgLbUVpgrq959ouV4m+S+pjtv4CQzTuHLl+PbyV8v/fN75go5qekg8zVzoNr3qVmWfZY0+SGp/5gjP18oYM6oN79ggligoenKNn/zFW6HG5slmdafyKDMyADNuDQzcD0p2wTupOyW9xMzO36VWGd8tlr6x62tdwUezBljutkiLXLR5W7TxXvKeDP1Hd1932VT9bkZ97J5ibmf0McM0t68tGmsl8wYr6890r6Wi+KVJpQUH+ZLRj2J5orLzDPNgPLMoeZ2F19OThgBgCPx15ufhWisNFeXZZ55bHdDWpZp2Vk2p2UrRUIf6ZL7pbNu6LrBqZZlWr0O7opxus0BORI8enK8SWc0HJDW/cm0LLWnizVzqBkoe8p4M56mp8Po9rdM183B48NiEqSL7pHG3N7xMFS8Slr5iBkjmDFYyhxm3uOZZ5oWvB4YyEwYAYAT1fY3zSWtvhpzkLrwR/b/BMXxxrJM10gknCRmHhtfnBb0m++GWjXfBKJJhUcetHwMI4wAwImsodK0WhwrY4+AVrT3+H1UP5QHALBJXKrdJQC6zDHyzTcAAOBkRRgBAAC2IowAAABbEUYAAICtCCMAAMBWhBEAAGArwggAALAVYQQAANiKMAIAAGxFGAEAALYijAAAAFsRRgAAgK0IIwAAwFaEEQAAYCvCCAAAsBVhBAAA2IowAgAAbEUYAQAAtiKMAAAAWxFGAACArQgjAADAVoQRAABgK8IIAACwFWEEAADYijACAABsRRgBAAC2IowAAABbEUYAAICtCCMAAMBWhBEAAGArwggAALAVYQQAANiKMAIAAGxFGAEAALYijAAAAFsRRgAAgK0IIwAAwFaEEQAAYCvCCAAAsBVhBAAA2IowAgAAbEUYAQAAtiKMAAAAWxFGAACArQgjAADAVoQRAABgK8IIAACwFWEEAADYijACAABsRRgBAAC2IowAAABbEUYAAICtOhRGCgsLdc455ygpKUkZGRmaOnWqtmzZctht/vznP2vcuHFKS0tTWlqaJkyYoHfeeeeoCg0AAE4cHQojRUVFmjFjhtauXatly5YpEAho4sSJqqura3Ob5cuX67rrrtNbb72lNWvWKDc3VxMnTtQXX3xx1IUHAADHP4dlWVZnN96zZ48yMjJUVFSkCy64oF3bhEIhpaWl6fe//72mTZvWrm2qq6uVkpKiqqoqJScnd7a4AACgB7X3+O0+mp1UVVVJktLT09u9TX19vQKBwGG38fl88vl80fvV1dWdLyQAADimdXoAazgc1qxZszR27FgNHTq03dv9+Mc/Vk5OjiZMmNDmOoWFhUpJSYlOubm5nS0mAAA4xnU6jMyYMUObNm3SwoUL273NvHnztHDhQi1evFixsbFtrnfPPfeoqqoqOpWWlna2mAAA4BjXqW6amTNn6sUXX9SKFSvUr1+/dm3z8MMPa968eXr99dc1fPjww67r9Xrl9Xo7UzQAAHCc6VAYsSxL3/3ud7V48WItX75c+fn57druoYce0n/913/p1Vdf1dlnn92pggIAgBNTh8LIjBkz9Mwzz2jp0qVKSkpSeXm5JCklJUVxcXGSpGnTpqlv374qLCyUJP3yl7/UnDlz9MwzzygvLy+6TWJiohITE7vytQAAgONQh8aMPP7446qqqtL48eOVnZ0dnZ599tnoOiUlJSorK2uxjd/v19VXX91im4cffrjrXgUAADhudbib5kiWL1/e4n5xcXFHdgEAAE4y/DYNAACwFWEEAADYijACAABsRRgBAAC2IowAAABbEUYAAICtCCMAAMBWhBEAAGArwggAALAVYQQAANiKMAIAAGxFGAEAALYijAAAAFsRRgAAgK0IIwAAwFaEEQAAYCvCCAAAsBVhBAAA2IowAgAAbEUYAQAAtiKMAAAAWxFGAACArQgjAADAVoQRAABgK8IIAACwFWEEAADYijACAABsRRgBAAC2IowAAABbEUYAAICtCCMAAMBWhBEAAGArwggAALAVYQQAANiKMAIAAGxFGAEAALYijAAAAFsRRgAAgK0IIwAAwFaEEQAAYCvCCAAAsBVhBAAA2IowAgAAbEUYAQAAtiKMAAAAWxFGAACArQgjAADAVoQRAABgK8IIAACwFWEEAADYijACAABsRRgBAAC2IowAAABbEUYAAICtCCMAAMBWhBEAAGArwggAALAVYQQAANiqQ2GksLBQ55xzjpKSkpSRkaGpU6dqy5YtR9zuueee0+DBgxUbG6thw4bpX//6V6cLDAAATiwdCiNFRUWaMWOG1q5dq2XLlikQCGjixImqq6trc5vVq1fruuuu0y233KINGzZo6tSpmjp1qjZt2nTUhQcAAMc/h2VZVmc33rNnjzIyMlRUVKQLLrig1XWuvfZa1dXV6cUXX4wuO++883TWWWfpj3/8Y7v2U11drZSUFFVVVSk5ObmzxQUAAD2ovcfvoxozUlVVJUlKT09vc501a9ZowoQJLZZNmjRJa9asaXMbn8+n6urqFhMAADgxdTqMhMNhzZo1S2PHjtXQoUPbXK+8vFyZmZktlmVmZqq8vLzNbQoLC5WSkhKdcnNzO1tMAABwjOt0GJkxY4Y2bdqkhQsXdmV5JEn33HOPqqqqolNpaWmX7wMAABwb3J3ZaObMmXrxxRe1YsUK9evX77DrZmVlaffu3S2W7d69W1lZWW1u4/V65fV6O1M0AABwnOlQy4hlWZo5c6YWL16sN998U/n5+UfcpqCgQG+88UaLZcuWLVNBQUHHSgoAAE5IHWoZmTFjhp555hktXbpUSUlJ0XEfKSkpiouLkyRNmzZNffv2VWFhoSTpzjvv1IUXXqhf//rXuvzyy7Vw4UKtX79e//3f/93FLwUAAByPOhRGHn/8cUnS+PHjWyxfsGCBpk+fLkkqKSmR09nc4HL++efrmWee0b333quf/OQnGjhwoJYsWXLYQa8AgBNbKBRSIBCwuxg4SjExMXK5XEf9PEf1PSM9he8ZAYATg2VZKi8vV2Vlpd1FQRdJTU1VVlaWHA7HIY+19/jdqQGsAAB0RiSIZGRkKD4+vtUDGI4PlmWpvr5eFRUVkqTs7OxOPxdhBADQI0KhUDSI9OrVy+7ioAtExotWVFQoIyOj0102/GovAKBHRMaIxMfH21wSdKXI3/NoxgARRgAAPYqumRNLV/w9CSMAAMBWhBEAAHpYXl6efvvb39pdjGMGA1gBADiC8ePH66yzzuqyAPHuu+8qISGhS57rREAYAQCgC1iWpVAoJLf7yIfWPn369ECJjh900wAAcBjTp09XUVGR5s+fL4fDIYfDoeLiYi1fvlwOh0Mvv/yyRo8eLa/Xq5UrV2r79u264oorlJmZqcTERJ1zzjl6/fXXWzznl7tpHA6HnnjiCV155ZWKj4/XwIED9cILLxy2XHl5efr5z3+uadOmKTExUQMGDNALL7ygPXv26IorrlBiYqKGDx+u9evXR7fZt2+frrvuOvXt21fx8fEaNmyY/va3v7V43nA4rMLCQuXn5ysuLk4jRozQokWLjr4iD4MwAgCwjWVZqvcHbZna+wXk8+fPV0FBgW699VaVlZWprKxMubm50cfvvvtuzZs3T5988omGDx+u2tpaXXbZZXrjjTe0YcMGXXrppZoyZYpKSkoOu5+5c+fqmmuu0YcffqjLLrtMN9xwg/bv33/YbX7zm99o7Nix2rBhgy6//HLdeOONmjZtmr7xjW/o/fff16mnnqpp06ZFX2tjY6NGjx6tl156SZs2bdJtt92mG2+8Ue+88070OQsLC/XUU0/pj3/8oz7++GN9//vf1ze+8Q0VFRW1q746g6+DBwD0iMbGRu3cuVP5+fmKjY2VJNX7gxoy51VbyrP5wUmK97RvtEJrY0aWL1+uiy66SEuWLNEVV1xx2O2HDh2q22+/XTNnzpRkWjVmzZqlWbNmSTItI/fee69+9rOfSZLq6uqUmJiol19+WZdeemmrz5mXl6dx48bpf//3fyWZb7fNzs7WfffdpwcffFCStHbtWhUUFKisrExZWVmtPs/XvvY1DR48WA8//LB8Pp/S09P1+uuvq6CgILrOt771LdXX1+uZZ545ZPvW/q4RfB08AAA94Oyzz25xv7a2Vg888IBeeukllZWVKRgMqqGh4YgtI8OHD4/eTkhIUHJycvSr1tuzTWZmpiRp2LBhhyyrqKhQVlaWQqGQfvGLX+jvf/+7vvjiC/n9fvl8vugXl23btk319fX66le/2mI/fr9fI0eOPGxZjgZhBABgm7gYlzY/OMm2fXeFL18VM3v2bC1btkwPP/ywTjvtNMXFxenqq6+W3+8/7PPExMS0uO9wOBQOh9u9TeTLx1pbFnmeX/3qV5o/f75++9vfatiwYUpISNCsWbOiZautrZUkvfTSS+rbt2+LfXm93sOW5WgQRgAAtnE4HO3uKrGTx+NRKBRq17qrVq3S9OnTdeWVV0oyB/ji4uJuLF37rVq1SldccYW+8Y1vSDIhZevWrRoyZIgkaciQIfJ6vSopKdGFF17YY+U69t8BAADYLC8vT+vWrVNxcbESExOVnp7e5roDBw7U888/rylTpsjhcOi+++47YgtHTxk4cKAWLVqk1atXKy0tTY888oh2794dDSNJSUmaPXu2vv/97yscDusrX/mKqqqqtGrVKiUnJ+umm27qlnJxNQ0AAEcwe/ZsuVwuDRkyRH369Dns+I9HHnlEaWlpOv/88zVlyhRNmjRJo0aN6sHStu3ee+/VqFGjNGnSJI0fP15ZWVmaOnVqi3V+9rOf6b777lNhYaHOOOMMXXrppXrppZeUn5/fbeXiahoAQI843FUXOH51xdU0tIwAAABbEUYAAICtCCMAAMBWhBEAAGArwggAALAVYQQAANiKMAIAAGxFGAEAALYijAAAAFsRRgAAgK0IIwAAHMH48eM1a9asLn3O6dOnH/K7MCcrwggAALAVYQQAgMOYPn26ioqKNH/+fDkcDjkcDhUXF0uSNm3apMmTJysxMVGZmZm68cYbtXfv3ui2ixYt0rBhwxQXF6devXppwoQJqqur0wMPPKC//vWvWrp0afQ5ly9f3ur+x48fr+9+97uaNWuW0tLSlJmZqT//+c+qq6vTN7/5TSUlJem0007Tyy+/HN0mFArplltuUX5+vuLi4nT66adr/vz5hzz3E088oTPOOEOxsbEaPHiw/vCHP3Rp3bWX25a9AgAgSZYlBert2XdMvORwHHG1+fPna+vWrRo6dKgefPBBSVKfPn1UWVmpiy++WN/61rf0m9/8Rg0NDfrxj3+sa665Rm+++abKysp03XXX6aGHHtKVV16pmpoavf3227IsS7Nnz9Ynn3yi6upqLViwQJKUnp7eZhn++te/6kc/+pHeeecdPfvss/rOd76jxYsX68orr9RPfvIT/eY3v9GNN96okpISxcfHKxwOq1+/fnruuefUq1cvrV69Wrfddpuys7N1zTXXSJKefvppzZkzR7///e81cuRIbdiwQbfeeqsSEhJ00003dUEFt5/DsiyrR/fYCe39CWIAwLGr1Z+a99dJv8ixp0A/2SV5Etq16vjx43XWWWfpt7/9bXTZz3/+c7399tt69dVXo8s+//xz5ebmasuWLaqtrdXo0aNVXFysAQMGHPKc06dPV2VlpZYsWXLEfYdCIb399tuSTKtHSkqKrrrqKj311FOSpPLycmVnZ2vNmjU677zzWn2emTNnqry8XIsWLZIknXbaafrZz36m6667rsVr+te//qXVq1e3q16kNv6uTdp7/KZlBACATvjggw/01ltvKTEx8ZDHtm/frokTJ+qSSy7RsGHDNGnSJE2cOFFXX3210tLSOryv4cOHR2+7XC716tVLw4YNiy7LzMyUJFVUVESXPfbYY/rLX/6ikpISNTQ0yO/366yzzpIk1dXVafv27brlllt06623RrcJBoNKSUnpcPmOFmEEAGCfmHjTQmHXvo9CbW2tpkyZol/+8peHPJadnS2Xy6Vly5Zp9erVeu211/S73/1OP/3pT7Vu3Trl5+d3rKgxMS3uOxyOFsscTd1N4XBYkrRw4ULNnj1bv/71r1VQUKCkpCT96le/0rp166Jll6Q///nPGjNmTIvndrlcHSpbVyCMAADs43C0u6vETh6PR6FQqMWyUaNG6R//+Ify8vLkdrd+OHU4HBo7dqzGjh2rOXPmaMCAAVq8eLHuuuuuVp+zq6xatUrnn3++7rjjjuiy7du3R29nZmYqJydHO3bs0A033NAtZegIrqYBAOAI8vLytG7dOhUXF2vv3r0Kh8OaMWOG9u/fr+uuu07vvvuutm/frldffVXf/OY3FQqFtG7dOv3iF7/Q+vXrVVJSoueff1579uzRGWecEX3ODz/8UFu2bNHevXsVCAS6rLwDBw7U+vXr9eqrr2rr1q2677779O6777ZYZ+7cuSosLNSjjz6qrVu36qOPPtKCBQv0yCOPdFk52oswAgDAEcyePVsul0tDhgxRnz59VFJSopycHK1atUqhUEgTJ07UsGHDNGvWLKWmpsrpdCo5OVkrVqzQZZddpkGDBunee+/Vr3/9a02ePFmSdOutt+r000/X2WefrT59+mjVqlVdVt5vf/vbuuqqq3TttddqzJgx2rdvX4tWEkn61re+pSeeeEILFizQsGHDdOGFF+rJJ5/scBdSV+BqGgBAjzjcVRc4fnXF1TS0jAAAAFsRRgAAgK0IIwAAwFaEEQAAYCvCCAAAsBVhBADQo46DizjRAV3x9ySMAAB6ROTry+vrbfqVXnSLyN/zy19Z3xF8HTwAoEe4XC6lpqZGf8wtPj4++psqOP5YlqX6+npVVFQoNTX1qH7ThjACAOgxWVlZklr+uiyOb6mpqdG/a2cRRgAAPcbhcCg7O1sZGRld+lsssEdMTEyX/MovYQQA0ONcLpctP1WPYxMDWAEAgK0IIwAAwFaEEQAAYCvCCAAAsBVhBAAA2IowAgAAbEUYAQAAtiKMAAAAWxFGAACArTocRlasWKEpU6YoJydHDodDS5YsOeI2Tz/9tEaMGKH4+HhlZ2fr5ptv1r59+zpTXgAAcILpcBipq6vTiBEj9Nhjj7Vr/VWrVmnatGm65ZZb9PHHH+u5557TO++8o1tvvbXDhQUAACeeDv82zeTJkzV58uR2r79mzRrl5eXpe9/7niQpPz9f3/72t/XLX/6yo7sGAAAnoG4fM1JQUKDS0lL961//kmVZ2r17txYtWqTLLruszW18Pp+qq6tbTAAA4MTU7WFk7Nixevrpp3XttdfK4/EoKytLKSkph+3mKSwsVEpKSnTKzc3t7mICAACbdHsY2bx5s+68807NmTNH7733nl555RUVFxfr9ttvb3Obe+65R1VVVdGptLS0u4sJAABs0uExIx1VWFiosWPH6oc//KEkafjw4UpISNC4ceP085//XNnZ2Yds4/V65fV6u7toAADgGNDtLSP19fVyOlvuxuVySZIsy+ru3QMAgGNch8NIbW2tNm7cqI0bN0qSdu7cqY0bN6qkpESS6WKZNm1adP0pU6bo+eef1+OPP64dO3Zo1apV+t73vqdzzz1XOTk5XfMqAADAcavD3TTr16/XRRddFL1/1113SZJuuukmPfnkkyorK4sGE0maPn26ampq9Pvf/14/+MEPlJqaqosvvphLewEAgCTJYR0HfSXV1dVKSUlRVVWVkpOT7S4OAABoh/Yev/ltGgAAYCvCCAAAsBVhBAAA2IowAgAAbEUYAQAAtiKMAAAAWxFGAACArQgjAADAVoQRAABgK8IIAACwFWEEAADYijACAABsRRgBAAC2IowAAABbEUYAAICtCCMAAMBWhBEAAGArwggAALAVYQQAANiKMAIAAGxFGAEAALYijAAAAFsRRgAAgK0IIwAAwFaEEQAAYCvCCAAAsBVhBAAA2IowAgAAbEUYAQAAtiKMAAAAWxFGAACArQgjAADAVoQRAABgK8IIAACwFWEEAADYijACAABsRRgBAAC2IowAAABbEUYAAICtCCMAAMBWhBEAAGArwggAALAVYQQAANiKMAIAAGxFGAEAALYijAAAAFsRRgAAgK0IIwAAwFaEEQAAYCvCCAAAsBVhBAAA2IowAgAAbEUYAQAAtiKMAAAAWxFGAACArQgjAADAVoQRAABgqw6HkRUrVmjKlCnKycmRw+HQkiVLjriNz+fTT3/6Uw0YMEBer1d5eXn6y1/+0pnyAgCAE4y7oxvU1dVpxIgRuvnmm3XVVVe1a5trrrlGu3fv1v/8z//otNNOU1lZmcLhcIcLCwAATjwdDiOTJ0/W5MmT273+K6+8oqKiIu3YsUPp6emSpLy8vI7uFgAAnKC6fczICy+8oLPPPlsPPfSQ+vbtq0GDBmn27NlqaGhocxufz6fq6uoWEwAAODF1uGWko3bs2KGVK1cqNjZWixcv1t69e3XHHXdo3759WrBgQavbFBYWau7cud1dNAAAcAzo9paRcDgsh8Ohp59+Wueee64uu+wyPfLII/rrX//aZuvIPffco6qqquhUWlra3cUEAAA26faWkezsbPXt21cpKSnRZWeccYYsy9Lnn3+ugQMHHrKN1+uV1+vt7qIBAIBjQLe3jIwdO1a7du1SbW1tdNnWrVvldDrVr1+/7t49AAA4xnU4jNTW1mrjxo3auHGjJGnnzp3auHGjSkpKJJkulmnTpkXXv/7669WrVy9985vf1ObNm7VixQr98Ic/1M0336y4uLiueRUAAOC41eEwsn79eo0cOVIjR46UJN11110aOXKk5syZI0kqKyuLBhNJSkxM1LJly1RZWamzzz5bN9xwg6ZMmaJHH320i14CAAA4njksy7LsLsSRVFdXKyUlRVVVVUpOTra7OAAAoB3ae/zmt2kAAICtCCMAAMBWhBEAAGArwggAALAVYQQAANiKMAIAAGxFGAEAALYijAAAAFsRRgAAgK0IIwAAwFaEEQAAYCvCCAAAsBVhBAAA2IowAgAAbEUYAQAAtiKMAAAAWxFGAACArQgjAADAVoQRAABgK8IIAACwFWEEAADYijACAABsRRgBAAC2IowAAABbEUYAAICtCCMAAMBWhBEAAGArwggAALAVYQQAANiKMAIAAGxFGAEAALYijAAAAFsRRgAAgK0IIwAAwFaEEQAAYCvCCAAAsBVhBAAA2IowAgAAbEUYAQAAtiKMAAAAWxFGAACArQgjAADAVoQRAABgK7fdBbDT8+9/rs27qhXvcSnO41ZcjFPxHrfiPK6mZS7FxbgUG+OSPxhWvT+khkBQ9f6Q6n0h1fuDqg+E1OA3U2yMS2kJHqXFxygtwaP0eI/SEzxKS/AoweOSw+E46jLX+YL6d3mNPj9Qr+RYs59eXbwPAAB60kkdRt7askf//GBXj+zL43IqNT5GGcle5abFq396vPqlxys3LU656fHqmxqn2BhXdH3LsrS72qfNZVXavKtan5TVaHNZtYr31cmy2tiH26n0eBNM0hNilBbvUVJsjBI8LsV73S3mCV63EjxuxXtdykqOVVZyrJxOggwAoOed1GFk4pBM5aTGqsEfMq0e/pAaAqbFI7Ks3h+SLxiSx+VUXNNBPC7GtJzEe9xNc5diPS41+kPaXx9QZb1f++v8OlDn1/56vxoDYflDYVXU+FRR49OmL6pbLU9Wcqxy0+Pkdjr17/JqHagPtLpeRpJXeb0TVOcL6kCdX/vq/PIFw/IHwyqvblR5dWOH68LrdmpAr3gN6JWgvOg8QXm945WdEidXFwWVYCisWl9QNY1BVTcGVNMYVG1jUDU+c7umMag6X1DJcTHKSPIqMzlWGUleZSTFKjnO3WUtP5Zlqd4f0v46v8KWpZzUOMW46LUEADs4LKut8+xjR3V1tVJSUlRVVaXk5GS7i9NhDf6QDjQFlPKqRpUeqFfJ/nqV7m/Q5wfqVbq/XnX+0CHbuZwOndonQUOyk3VGdrKG5Jh570Rvi/Usy1JDwBxYvzzV+0Oq8wVV5w+q3hcyc39ItT5zv9YX1O7qRgXDbb8NPC6neiV6FONyyu1yyONyKsblVIzL0TQ3t50OhxqDITUGwmoMhOQLmnljICxf031/KNzpevS6ncpINsEkI8mrBK9bbqdDLqcph8vpiN43c6d8wZAONAXEA/V+HagL6EC9X5X1gRZlcTsdyk2PV16veOX3TlR+73jl9U5Qfu8E5aTE0WoEAJ3Q3uM3YeQYYFmW9tf5VXqgQaX769UYCGlwVrIGZia26LrpLsFQWLsqG7VzX50+21en4r31Zr6vTqX7G44qQLQlNsappNgYJcW6lRQbo+RYt7ntjVGcx6WqhoAqahpVUW1ak6oaWm8lOloet1MOSb5g26/R43aqX1qcEr1uxTaNIYqLcZrbbjO2yBvjVKzbpRiXCUFup0POplDkPCgkuRwO+YJh1fmCqvUFo/PI7bqmgOgPhRW2LMn8k2VZsiSFLUuWJVmWCavxHpcSvW4lxrqV4HUr0dM095pWvMRYtxK9biXHRerY1HlybIzivzTGqM4XNK131Y3a3TTf09Sat6fGp+Q4twZmJGlQZpIGZSYqr3dCj7YmNfhDJrwfMEF+V1WD4mPc6p3kUe9Er3onRubeQ15bd/IHwyrZX6+de+tUUdOo7JRY5fVKUG56PK1tOOkRRtAlQmFLuyobVFkfUCAcViAYViBkKRAyrRyB6GQpHLaaDtZOeZsO1JGDdGzk4B1jDp4ed8c+pBsDoaYDY3NAaQiEFApbCoYshcJhBcJWi/vBsKUYl1Np8R6lJcQoNb5pcHHTuJq0+BjFxbhkWdLumkbt3FunnXvrVNw037m3TiX76xUIHfP/RTrF5XSYION1q7Le32rr3OHEuBzK752ggZlJGpiRqEGZSUqL96imMaDqxqCZN5j5wd1yvqAZ7B3vcSmhacB4i+7PphavXZUNTS2I9So90KA9Nb52ly02xhkNJjmpscpNj9eA9AT1T4/XgF7xyk6JlbudQcGyLDUGwtpT49POfXXauadWxfvqtaPpvfL5gXq11rDocjrULy1Oeb1MC9uAXqa1bUB6vCQ1t1D6g6r1NbVgNgXSen9QYcuS2+VUTFMrn9tlQq27qSUy8vfLTolTdkqsMpK98rq7/+TFbhXVjXq/5IDeL6nUzr11OjMnWeMG9tbwfqmEv2MQYQToApFWo88rTYtVg990PTUEQtGuqAZ/KLosFLYUDJtgFmwKR6HobROQvG5Xc8uF17RkRFozEr0xSvC65HE75XQ45JDkdJq5wyE5HJHbDoXCVvQAVtuipaX5wFbjaw4DBweCUBvdcvEelzKTY9UnyRsdq5OR7FWfRK/21/m1dXeNtlbUatvumg6Hl66Q5HVHB37npMapMRDS3lq/9tb6tK/Op701fjUEjlwut9Ohvmlx0XCS6I1RdWNAVQ0BVTc0hamGpvuNgSMG0gSPS3m9E5SZHKuyqkYV761rVzm6Wq8Ej7JSYpWdEqusFDMwvXeiV26XsynINHdhHtylGeM2ob1PordLx2YdrUAorE/KqvX+ZyZ8vF9yQJ8faGh13USvW+ed0kvjBvbWVwb21im9E46Z13EyI4wAaFVkjFFNY3NQSYmLUUZyrBK97RvTblmWdlU1auvuGn26u0Zbd9fq0901qm0afNzc9WbmZpnpHvK4nWaAeCCkel+w6ZJ5E6Aa/GZcUyBkKTvFtGjkpsUrN90Eh5S4mCMeYOp8Qe2r9WtPrU97a3364oBpYflsn2npKj3QIP9huuXa4nE7ld80qDuvd4JO6W0Geef3SVCfRG+LclmWpYoaX3NL2z4zL95ruplcDocSvOZqtkSvO9rdFt/UzZbgccnpdCgYshRsCrHBULjpvlkWCFmqbghod3WjyqoaD9vV2BExLofSE0yXV69Er3oneNQ7yav0BI+SYk1Z42KaB+/HfWkwv9tpWidMx2KkPlruozEQUlVDQJVNga+q3swrI/MGvz4/0KAPP69UY6Dl63I4pNMzkzSyf5pO7ZOgDaWVWr1t7yED/nNSYjX2NBNMzshOVr+0OMV7uvaaDV8wFB2LV9c0b/CH5A+ZCwoiLcf+YFj+kKVA07g5/0Hj6RoCIfkCITUGQ00nNmaZPxhWfp8EnZefrjGn9NLAjMTjMlwRRgCgFeGwpfLqRpXsr1fJvnp9tr9OjYGwUuKag1NybIxS4s08Oa71MTbHEsuyVFkfUFlVo8qrG1RW1ajdVSak7K/zR1vpguHwQS11lgJNXZq+YFj76/yqaQza/VIOkRIXo5H9UzWqf5pG9U/TiNwUJcXGtFgnHLb08a5qvb1tj1Zt26t3iw+0Gjh7J3rULy2+KeTGtQi7Djm0v94fvUIxcjXkwferGgItutd6sgs3PcGjc/PSdW5+usackq4zspI7NbDesqzmFt1g81WkjYGQ8nsnKj3B06XlJowAADrEFzRX5e2t8WtvnU/7Il1gteZ2rS/Y9PUHka9DCKqu6WsRzDiX9u8rKdatlLgYpcbHmHmcR8kH3e+T6NWI3FSd0juhwwfdBn9I6z/br5Wf7tXaHfu0c2+dqrsxaHndzqbuVpfiY8yYuMjVhua201yF2LTc43IeNBjejKmL87haLHM5pU1fVGvdzn1677MDh7QQJce6dW5+urJSYuULhNUYbL5q8eCrGSOtMA0HtcS05bHrR+ny4dldWjftPX6f1N8zAgBo5nW7mgbExnV428gZd/ig81szwqlZpGEpMhC3u8R5XBo3sI/GDewTXVbVEFDp/vqmr1NoGhzd9NUKkXEovRI8Sk/0KC3y7dnxzd9wnZ7gUUpcTHR8V6RLLd7j6raBsxcPzpQ0UP5gWB99Ual1O/dr3Y79Wl+8X9WNQb3+ScVRPb/H5ZQ3xqm4GNPdFuOyr+WPlhEAwEnNsqxjtguuNcFQWB/vqta7xftV0xiMXrV48NzbdBWj1+2S1+00X84ZaXnxuBTrdnZrIIygZQQAgHY4noKIJLldTo3ITdWI3FS7i9JluCgbAADYijACAABs1eEwsmLFCk2ZMkU5OTlyOBxasmRJu7ddtWqV3G63zjrrrI7uFgAAnKA6HEbq6uo0YsQIPfbYYx3arrKyUtOmTdMll1zS0V0CAIATWIcHsE6ePFmTJ0/u8I5uv/12XX/99XK5XB1qTQEAACe2HhkzsmDBAu3YsUP3339/u9b3+Xyqrq5uMQEAgBNTt4eRTz/9VHfffbf+7//+T253+xpiCgsLlZKSEp1yc3O7uZQAAMAu3RpGQqGQrr/+es2dO1eDBg1q93b33HOPqqqqolNpaWk3lhIAANipW7/0rKamRuvXr9eGDRs0c+ZMSVI4HJZlWXK73Xrttdd08cUXH7Kd1+uV1+vtzqIBAIBjRLeGkeTkZH300Uctlv3hD3/Qm2++qUWLFik/P787dw8AAI4DHQ4jtbW12rZtW/T+zp07tXHjRqWnp6t///6655579MUXX+ipp56S0+nU0KFDW2yfkZGh2NjYQ5YDAICTU4fDyPr163XRRRdF7991112SpJtuuklPPvmkysrKVFJS0nUlBAAAJzR+tRcAAHSLE+pXeyN5ie8bAQDg+BE5bh+p3eO4CCM1NTWSxPeNAABwHKqpqVFKSkqbjx8X3TThcFi7du1SUlKSHA5Hlz1vdXW1cnNzVVpaSvdPD6C+exb13bOo755Fffe8ztS5ZVmqqalRTk6OnM62v9rsuGgZcTqd6tevX7c9f3JyMm/mHkR99yzqu2dR3z2L+u55Ha3zw7WIRPTIb9MAAAC0hTACAABsdVKHEa/Xq/vvv5+vnu8h1HfPor57FvXds6jvnteddX5cDGAFAAAnrpO6ZQQAANiPMAIAAGxFGAEAALYijAAAAFud1GHkscceU15enmJjYzVmzBi98847dhfphLBixQpNmTJFOTk5cjgcWrJkSYvHLcvSnDlzlJ2drbi4OE2YMEGffvqpPYU9ARQWFuqcc85RUlKSMjIyNHXqVG3ZsqXFOo2NjZoxY4Z69eqlxMRE/cd//Id2795tU4mPb48//riGDx8e/eKngoICvfzyy9HHqevuM2/ePDkcDs2aNSu6jPruWg888IAcDkeLafDgwdHHu6u+T9ow8uyzz+quu+7S/fffr/fff18jRozQpEmTVFFRYXfRjnt1dXUaMWKEHnvssVYff+ihh/Too4/qj3/8o9atW6eEhARNmjRJjY2NPVzSE0NRUZFmzJihtWvXatmyZQoEApo4caLq6uqi63z/+9/XP//5Tz333HMqKirSrl27dNVVV9lY6uNXv379NG/ePL333ntav369Lr74Yl1xxRX6+OOPJVHX3eXdd9/Vn/70Jw0fPrzFcuq765155pkqKyuLTitXrow+1m31bZ2kzj33XGvGjBnR+6FQyMrJybEKCwttLNWJR5K1ePHi6P1wOGxlZWVZv/rVr6LLKisrLa/Xa/3tb3+zoYQnnoqKCkuSVVRUZFmWqd+YmBjrueeei67zySefWJKsNWvW2FXME0paWpr1xBNPUNfdpKamxho4cKC1bNky68ILL7TuvPNOy7J4b3eH+++/3xoxYkSrj3VnfZ+ULSN+v1/vvfeeJkyYEF3mdDo1YcIErVmzxsaSnfh27typ8vLyFnWfkpKiMWPGUPddpKqqSpKUnp4uSXrvvfcUCARa1PngwYPVv39/6vwohUIhLVy4UHV1dSooKKCuu8mMGTN0+eWXt6hXifd2d/n000+Vk5OjU045RTfccINKSkokdW99Hxc/lNfV9u7dq1AopMzMzBbLMzMz9e9//9umUp0cysvLJanVuo88hs4Lh8OaNWuWxo4dq6FDh0oyde7xeJSamtpiXeq88z766CMVFBSosbFRiYmJWrx4sYYMGaKNGzdS111s4cKFev/99/Xuu+8e8hjv7a43ZswYPfnkkzr99NNVVlamuXPnaty4cdq0aVO31vdJGUaAE9WMGTO0adOmFn286Hqnn366Nm7cqKqqKi1atEg33XSTioqK7C7WCae0tFR33nmnli1bptjYWLuLc1KYPHly9Pbw4cM1ZswYDRgwQH//+98VFxfXbfs9KbtpevfuLZfLdcgI4N27dysrK8umUp0cIvVL3Xe9mTNn6sUXX9Rbb72lfv36RZdnZWXJ7/ersrKyxfrUeed5PB6ddtppGj16tAoLCzVixAjNnz+fuu5i7733nioqKjRq1Ci53W653W4VFRXp0UcfldvtVmZmJvXdzVJTUzVo0CBt27atW9/fJ2UY8Xg8Gj16tN54443osnA4rDfeeEMFBQU2luzEl5+fr6ysrBZ1X11drXXr1lH3nWRZlmbOnKnFixfrzTffVH5+fovHR48erZiYmBZ1vmXLFpWUlFDnXSQcDsvn81HXXeySSy7RRx99pI0bN0ans88+WzfccEP0NvXdvWpra7V9+3ZlZ2d37/v7qIa/HscWLlxoeb1e68knn7Q2b95s3XbbbVZqaqpVXl5ud9GOezU1NdaGDRusDRs2WJKsRx55xNqwYYP12WefWZZlWfPmzbNSU1OtpUuXWh9++KF1xRVXWPn5+VZDQ4PNJT8+fec737FSUlKs5cuXW2VlZdGpvr4+us7tt99u9e/f33rzzTet9evXWwUFBVZBQYGNpT5+3X333VZRUZG1c+dO68MPP7Tuvvtuy+FwWK+99pplWdR1dzv4ahrLor672g9+8ANr+fLl1s6dO61Vq1ZZEyZMsHr37m1VVFRYltV99X3ShhHLsqzf/e53Vv/+/S2Px2Ode+651tq1a+0u0gnhrbfesiQdMt10002WZZnLe++77z4rMzPT8nq91iWXXGJt2bLF3kIfx1qra0nWggULous0NDRYd9xxh5WWlmbFx8dbV155pVVWVmZfoY9jN998szVgwADL4/FYffr0sS655JJoELEs6rq7fTmMUN9d69prr7Wys7Mtj8dj9e3b17r22mutbdu2RR/vrvp2WJZlHV3bCgAAQOedlGNGAADAsYMwAgAAbEUYAQAAtiKMAAAAWxFGAACArQgjAADAVoQRAABgK8IIAACwFWEEAADYijACAABsRRgBAAC2IowAAABb/X9MwtJK+mgjRwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "plt.plot(history.history['mae'], label='train mae')\n",
        "plt.plot(history.history['val_mae'], label='test mae')\n",
        "plt.legend();"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5SXsFcMUp84_"
      },
      "source": [
        "History - Callback, про них поговорим в следующих занятиях"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cKL1g6ILJiNk"
      },
      "source": [
        "## Summary\n",
        "\n",
        "\n",
        "Вот мы и разобрались в деталях, что можно передавать в метод fit в Keras."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cj4ciEg13c7A"
      },
      "source": [
        "**validation_data** и **validation_batch_size**\n",
        "\n",
        "<img src='https://media0.giphy.com/media/yOhgNO9YsmdsQ/giphy.gif?cid=ecf05e473qx1a618m4t3o922udc1mcfmqtfwdyi4mrqg2gcf&rid=giphy.gif&ct=g'>\n",
        "\n",
        "___________________"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 379
        },
        "id": "swLaCh1dhhzb",
        "outputId": "76a877df-7e40-4dda-872c-79dc54629b46"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
            "11490434/11490434 [==============================] - 0s 0us/step\n",
            "1875/1875 [==============================] - 241s 128ms/step - loss: 0.0312\n",
            "1/1 [==============================] - 0s 100ms/step\n",
            "1/1 [==============================] - 0s 148ms/step\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAEOCAYAAAApP3VyAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAccklEQVR4nO3df2xV9f3H8ddtpZci5ZaC7eWGghU2MUNwY4IdaHA0IC5OtEumMwsuBjZ3IcFu6kgEh5p0ssU5NsR/NjozEcMiOImpP6qUGCkLVcdQV6Fjo4a2itp7S4FSe8/3D+b99kr5nN7eez/33t7nI/kk9L7OPffDaXnz7rnnfo7HcRxHAAAAluSlewIAACC30HwAAACraD4AAIBVNB8AAMAqmg8AAGAVzQcAALCK5gMAAFhF8wEAAKyi+QAAAFZdlO4JfFkkEtHx48dVVFQkj8eT7ukAOclxHHV3dysQCCgvLzt+R6F2AOkVV91wUuQPf/iDM3XqVMfr9Tpz58519u/fP6TntbW1OZIYDEYGjLa2tlSViEENt244DrWDwciUMZS6kZLmY/v27U5BQYHzpz/9yXn33XedFStWOMXFxU5nZ6frc7u6utJ+4BgMxrnR1dWVihIxqETqhuPE1g6PxzPoSPfxZDByYQylbqSk+Zg7d64TDAajX/f39zuBQMCpra11fW4oFEr7gWMwGOdGKBRKRYkYVCJ1w3H+v3Z4PB4nLy9v0JHu43mhpmioI93zZ/A9NB2DL/Kh1I2kv5l79uxZNTc3q6qqKvpYXl6eqqqqtG/fvvO27+3tVTgcjhkAcku8dUOidgDZLOnNx4kTJ9Tf36+ysrKYx8vKytTR0XHe9rW1tfL5fNFRXl6e7CkByHDx1g2J2gFks7Rfxr527VqFQqHoaGtrS/eUAGQBageQvZL+UduJEycqPz9fnZ2dMY93dnbK7/eft73X65XX6032NABkkXjrhkTtALJZ0s98FBQUaM6cOWpoaIg+FolE1NDQoMrKymS/HIARIJl1w3EcRSKRQQeQKOfcBzWGPUaCZPzdUrLIWE1NjZYvX65vfvObmjt3rh5//HH19PToRz/6USpeDsAIQN0AckdKmo/vf//7+vjjj7V+/Xp1dHToqquuUn19/XkXkwHAF6gbQO7wOBl2HigcDsvn86V7GgAkhUIhjRs3Lt3TGJJsqB2JLvueYeUaGNRQ6kbaP+0CAAByC80HAACwiuYDAABYRfMBAACsSsmnXQAA8eOCUuQKznwAAACraD4AAIBVNB8AAMAqmg8AAGAVzQcAALCK5gMAAFhF8wEAAKxinQ8AGKK8PPPva27rdHBjOeAcznwAAACraD4AAIBVNB8AAMAqmg8AAGAVzQcAALCK5gMAAFhF8wEAAKxinQ8AOcNtnQ23dTQSzd2wjgdyBWc+AACAVTQfAADAKpoPAABgFc0HAACwiuYDAABYRfMBAACsovkAAABWsc4HgBHHbT2P4WIdDyA5kn7m45e//KU8Hk/MmDFjRrJfBsAIQt0AcktKznx87Wtf06uvvvr/L3IRJ1gAmFE3gNyRkn/dF110kfx+fyp2DWCEom4AuSMlF5wePnxYgUBAl112me644w4dO3bsgtv29vYqHA7HDAC5J566IVE7gGyW9OZj3rx5qqurU319vbZs2aKjR4/q2muvVXd396Db19bWyufzRUd5eXmypwQgw8VbNyRqB5DNPE6KL7/u6urS1KlT9dhjj+muu+46L+/t7VVvb2/063A4TBEBMkQoFNK4ceOsv65b3ZDMtWO4n3bh0yhA4oZSN1J+RVdxcbG++tWv6siRI4PmXq9XXq831dMAkEXc6oZE7QCyWcqbj5MnT6q1tVU//OEPU/1SAEaIbK0bbmdc8vLM73S75UM5o9Pf32/M3c7ujPS1TNyOYaLzT8YaM5l+DJMh6dd8/PznP1djY6P+85//6M0339Qtt9yi/Px83X777cl+KQAjBHUDyC1JP/Px4Ycf6vbbb9cnn3yiSy65RAsWLFBTU5MuueSSZL8UgBGCugHklpRfcBqvcDgsn8+X7mkAUPouOB2OgbUjXRec8rZL5r9lwNsuqTeUusGN5QAAgFU0HwAAwCqaDwAAYBXNBwAAsIrbRqbA9773PWO+YsUK130cP37cmJ85c8aYP/3008a8o6PDmJsWdwKyVaIX8rldEJqfn2/M3S7CGz9+vDEvLi425pL73YDdaoebiRMnGvOPP/7YmLtdkNnV1WXM+/r6Etp/aWmpMT916pQxd7uH0OnTp435yZMnjbkkff7558Y82y9IlTjzAQAALKP5AAAAVtF8AAAAq2g+AACAVTQfAADAKpoPAABgFc0HAACwiuYDAABYxV1tU+Df//63Mb/00kvtTMSgu7vbmL/77ruWZpKZPvzwQ2O+ceNG130cOHAgWdNJm1y7q62bwsJCY+62CNjcuXONuVttmDZtmjEfyhzc9hEIBIz5mDFjjLnbImEnTpww5qFQyJh/8MEHxrykpMSYX3XVVcbc7a7Ara2txryurs6Yv/HGG8Zccj8GkUjEdR/pxF1tAQBAxqH5AAAAVtF8AAAAq2g+AACAVTQfAADAKpoPAABgFc0HAACw6qJ0T2AkWrFihTGfNWuW6z7ef/99Y37FFVcY82984xvGfOHChcb8mmuuMeZtbW3GvLy83Jgn6vPPPzfmH3/8sTGfNGlSQq9/7Ngx121GwjofucZtfRC3dT7c1phwW/tg1KhRxjwcDhvzoejp6THmp0+fTihvb2835n19fcbc7e/o9j1yW7rK7Xvots7JhAkTjLnbOiDNzc3GXHJf52Mk4MwHAACwiuYDAABYRfMBAACsovkAAABW0XwAAACraD4AAIBVNB8AAMAq1vlIgYaGhoTyoaivr0/o+ePHjzfmV111lTF3+6z61VdfHe+U4nLmzBlj/sEHHxhzt3VU3NZrcPssP9LLba2H4XJb48JtfZm33nrLmOfn5xtztzUyJKm/v9+Yu/3bz8sz/07q9m/v008/NeZuent7jbnbMRgzZowxr66uNua33XabMR87dqwx7+7uNuZuaxRJqfv5zSRxn/nYu3evbrrpJgUCAXk8Hu3atSsmdxxH69ev16RJk1RYWKiqqiodPnw4WfMFkIWoGwAGirv56Onp0ezZs7V58+ZB840bN2rTpk168skntX//fl188cVasmSJa7cMYOSibgAYKO63XZYuXaqlS5cOmjmOo8cff1wPPPCAbr75ZknSU089pbKyMu3atWvQ01m9vb0xp9mSsXwwgMyS7LohUTuAbJbUC06PHj2qjo4OVVVVRR/z+XyaN2+e9u3bN+hzamtr5fP5oiPV9wQBkFmGUzckageQzZLafHR0dEiSysrKYh4vKyuLZl+2du1ahUKh6HC7YRmAkWU4dUOidgDZLO2fdvF6vfJ6vemeBoAsQ+0AsldSz3z4/X5JUmdnZ8zjnZ2d0QwABqJuALknqWc+Kioq5Pf71dDQEF0nIhwOa//+/br77ruT+VJI0GeffWbMX3/99YT2n4y1TBLh9ll+t7UO/vnPfxrzZ599Nu45YXCZVDfc1ldwW4PCbQ2HU6dOGXO3NTrcckmKRCKu22Ty8xPlVttefvllY75gwQJjXlpaaszb29uNudvPUK6Iu/k4efKkjhw5Ev366NGjeuedd1RSUqIpU6ZozZo1euSRR/SVr3xFFRUVWrdunQKBgJYtW5bMeQPIItQNAAPF3XwcOHBA119/ffTrmpoaSdLy5ctVV1en++67Tz09PVq5cqW6urq0YMEC1dfXa/To0cmbNYCsQt0AMFDczcfChQuNpyY9Ho8eeughPfTQQwlNDMDIQd0AMBA3lgMAAFbRfAAAAKtoPgAAgFU0HwAAwKq0r3AKDIfbZ+2feOIJY56XZ+673S58/PTTT405Ria3NSzc1glxe36i+7chE+aQiMmTJxvzGTNmGPMTJ04Y8/fee8+Yu631ImX/MR4KznwAAACraD4AAIBVNB8AAMAqmg8AAGAVzQcAALCK5gMAAFhF8wEAAKxinQ9kpWAwaMwvueQSY/7ZZ58Z85aWlrjnBCS6zkcy1ndw24fH40n5HNLJ6/Ua8/vvv9+YT58+3Zi3tbUZ8yNHjhjzvr4+Y54rOPMBAACsovkAAABW0XwAAACraD4AAIBVNB8AAMAqmg8AAGAVzQcAALCKdT6QkebPn2/Mf/GLXyS0/2XLlhnzQ4cOJbR/YDBua2jYWGMj29fxcHPZZZcZ88rKSmPudnx27NhhzE+cOGHMcQ5nPgAAgFU0HwAAwCqaDwAAYBXNBwAAsIrmAwAAWEXzAQAArKL5AAAAVrHOBzLSjTfeaMxHjRplzBsaGoz5vn374p4T4Gakr6GRCS66yPzf1h133GHMR48ebcxbWlqM+QsvvGDMI5GIMcc5cZ/52Lt3r2666SYFAgF5PB7t2rUrJr/zzjvl8Xhixg033JCs+QLIQtQNAAPF3Xz09PRo9uzZ2rx58wW3ueGGG9Te3h4dzzzzTEKTBJDdqBsABor7bZelS5dq6dKlxm28Xq/8fv+wJwVgZKFuABgoJRec7tmzR6Wlpbr88st1991365NPPrngtr29vQqHwzEDQO6Jp25I1A4gmyW9+bjhhhv01FNPqaGhQY8++qgaGxu1dOlS9ff3D7p9bW2tfD5fdJSXlyd7SgAyXLx1Q6J2ANks6Z92ue2226J/vvLKKzVr1ixNmzZNe/bs0aJFi87bfu3ataqpqYl+HQ6HKSJAjom3bkjUDiCbpXydj8suu0wTJ07UkSNHBs29Xq/GjRsXMwDkNre6IVE7gGyW8nU+PvzwQ33yySeaNGlSql8KWaSwsNCYu33M8uzZs8b8wQcfNOZ9fX3GHOlF3cCFBAIBY/7d737XmH/++efG/De/+Y0xd7sWCUMTd/Nx8uTJmN9Gjh49qnfeeUclJSUqKSnRhg0bVF1dLb/fr9bWVt13332aPn26lixZktSJA8ge1A0AA8XdfBw4cEDXX3999Osv3nNdvny5tmzZooMHD+rPf/6zurq6FAgEtHjxYj388MPyer3JmzWArELdADBQ3M3HwoULjUsIv/TSSwlNCMDIQ90AMBA3lgMAAFbRfAAAAKtoPgAAgFU0HwAAwKqUr/MBDObee+815l//+teNeX19vTF/8803454TgPQbNWqUMf/Zz35mzKdPn27MDx8+bMz/+te/GnPTkv8YOs58AAAAq2g+AACAVTQfAADAKpoPAABgFc0HAACwiuYDAABYRfMBAACsYp0PpMR3vvMdY75u3TpjHg6HjflDDz0U95yAVPN4PMbcdHO9XOF2jK688kpjfssttxjzU6dOGfMNGzYY81AoZMyRHJz5AAAAVtF8AAAAq2g+AACAVTQfAADAKpoPAABgFc0HAACwiuYDAABYxTofGJYJEyYY802bNhnz/Px8Y/7iiy8a86amJmMOpAPreLjz+XzG/He/+50xLygoMObbt2835rt37zbmfA/t4MwHAACwiuYDAABYRfMBAACsovkAAABW0XwAAACraD4AAIBVNB8AAMAq1vnAoNzW4aivrzfmFRUVxry1tdWYr1u3zpgD6cAaEO7y8sy/0waDQWM+a9YsY/7BBx8Y8w0bNhjznp4eYw474jrzUVtbq6uvvlpFRUUqLS3VsmXL1NLSErPNmTNnFAwGNWHCBI0dO1bV1dXq7OxM6qQBZBdqB4CB4mo+GhsbFQwG1dTUpFdeeUV9fX1avHhxTCd5zz336IUXXtCOHTvU2Nio48eP69Zbb036xAFkD2oHgIHietvly6fa6+rqVFpaqubmZl133XUKhUL64x//qG3btunb3/62JGnr1q264oor1NTUpGuuuSZ5MweQNagdAAZK6ILTUCgkSSopKZEkNTc3q6+vT1VVVdFtZsyYoSlTpmjfvn2D7qO3t1fhcDhmABjZqB1Abht28xGJRLRmzRrNnz9fM2fOlCR1dHSooKBAxcXFMduWlZWpo6Nj0P3U1tbK5/NFR3l5+XCnBCALUDsADLv5CAaDOnTokOsdBN2sXbtWoVAoOtra2hLaH4DMRu0AMKyP2q5atUq7d+/W3r17NXny5Ojjfr9fZ8+eVVdXV8xvMJ2dnfL7/YPuy+v1yuv1DmcaALIMtQOAFGfz4TiOVq9erZ07d2rPnj3nreUwZ84cjRo1Sg0NDaqurpYktbS06NixY6qsrEzerJFy06ZNM+Zz5sxJaP81NTXG3G0dEGQXasfI4fF4jLnbOh1r1qwx5pFIxJg//PDDxvxCb9Mhs8TVfASDQW3btk3PP/+8ioqKot9kn8+nwsJC+Xw+3XXXXaqpqVFJSYnGjRun1atXq7KykqvVgRxG7QAwUFzNx5YtWyRJCxcujHl869atuvPOOyVJv/3tb5WXl6fq6mr19vZqyZIleuKJJ5IyWQDZidoBYKC433ZxM3r0aG3evFmbN28e9qQAjCzUDgADcWM5AABgFc0HAACwiuYDAABYRfMBAACsovkAAABWDWuFU2S/qVOnGvOXX345of3fe++9xnz37t0J7R9AargtIlZaWmrM169fb8xHjx5tzP/2t78Z85deesmYIztw5gMAAFhF8wEAAKyi+QAAAFbRfAAAAKtoPgAAgFU0HwAAwCqaDwAAYBXrfOSolStXGvMpU6YktP/GxkZjPpS7nAJIPrd1PIqKioz5I488Ysyvv/56Y97b22vM//KXvxjzs2fPGnNkB858AAAAq2g+AACAVTQfAADAKpoPAABgFc0HAACwiuYDAABYRfMBAACsYp2PEWrBggXGfPXq1ZZmAiCT5OWZf+ecM2eOMb/xxhuN+cUXX2zM29vbjXlLS4sxZ42gkYEzHwAAwCqaDwAAYBXNBwAAsIrmAwAAWEXzAQAArKL5AAAAVtF8AAAAq+Ja56O2tlbPPfec/vWvf6mwsFDf+ta39Oijj+ryyy+PbrNw4UI1NjbGPO/HP/6xnnzyyeTMGENy7bXXGvOxY8cmtP/W1lZjfvLkyYT2j5GF2mGPx+Mx5sXFxcZ84cKFxtxtHY/e3l5jfuDAAWMeDoeNudvfj3VAskNcZz4aGxsVDAbV1NSkV155RX19fVq8eLF6enpitluxYoXa29ujY+PGjUmdNIDsQu0AMFBcZz7q6+tjvq6rq1Npaamam5t13XXXRR8fM2aM/H5/cmYIIOtROwAMlNA1H6FQSJJUUlIS8/jTTz+tiRMnaubMmVq7dq1OnTp1wX309vYqHA7HDAAjG7UDyG3DvrdLJBLRmjVrNH/+fM2cOTP6+A9+8ANNnTpVgUBABw8e1P3336+WlhY999xzg+6ntrZWGzZsGO40AGQZageAYTcfwWBQhw4d0htvvBHz+MqVK6N/vvLKKzVp0iQtWrRIra2tmjZt2nn7Wbt2rWpqaqJfh8NhlZeXD3daADIctQPAsJqPVatWaffu3dq7d68mT55s3HbevHmSpCNHjgxaQLxer7xe73CmASDLUDsASHE2H47jaPXq1dq5c6f27NmjiooK1+e88847kqRJkyYNa4IAsh+1A8BAcTUfwWBQ27Zt0/PPP6+ioiJ1dHRIknw+nwoLC9Xa2qpt27bpxhtv1IQJE3Tw4EHdc889uu666zRr1qyU/AWQGv/4xz+M+aJFi4z5p59+mszpIMtRO+zJyzN/jmD8+PHG/P333zfmL774ojHPz8835nV1dcb8i4uRL4R1PEaGuJqPLVu2SDp/EZqtW7fqzjvvVEFBgV599VU9/vjj6unpUXl5uaqrq/XAAw8kbcIAsg+1A8BAcb/tYlJeXn7eCoUAQO0AMBD3dgEAAFbRfAAAAKtoPgAAgFU0HwAAwCqaDwAAYJXHybAPTYfDYfl8vnRPA4DOrbkwbty4dE9jSAbWDo/HM+g2iZa7C+03WdzmN5TXd1vno6CgIKE5FBcXG/P+/n5jbrpZ4FDyDPsvKyUy/efsQj9jjuMoEokMqW5w5gMAAFhF8wEAAKyi+QAAAFbRfAAAAKtoPgAAgFU0HwAAwKq4bixnQy58jArIFtn073HgXFM173Qfj6G8vts2ieaRSCShPNHXzwXpPgbD/R598fhQ5p9xzUd3d3e6pwDgf7q7u7Nm3R1qxzlu//mfOXMmof1/9NFHCT0f2c/tZ2wodSPjFhmLRCI6fvy4ioqK5PF4FA6HVV5erra2tqxZ7CjTcAwTk4vHz3EcdXd3KxAIuC5alSmoHcnF8Utcrh3DeOpGxp35yMvL0+TJk897fNy4cTnxzUsljmFicu34ZcsZjy9QO1KD45e4XDqGQ60b2fErDQAAGDFoPgAAgFUZ33x4vV49+OCD8nq96Z5K1uIYJobjl534viWG45c4juGFZdwFpwAAYGTL+DMfAABgZKH5AAAAVtF8AAAAq2g+AACAVTQfAADAqoxvPjZv3qxLL71Uo0eP1rx58/T3v/893VPKWHv37tVNN92kQCAgj8ejXbt2xeSO42j9+vWaNGmSCgsLVVVVpcOHD6dnshmotrZWV199tYqKilRaWqply5appaUlZpszZ84oGAxqwoQJGjt2rKqrq9XZ2ZmmGeNCqBtDR91IDHVjeDK6+Xj22WdVU1OjBx98UG+99ZZmz56tJUuWcGOjC+jp6dHs2bO1efPmQfONGzdq06ZNevLJJ7V//35dfPHFWrJkScI3mhopGhsbFQwG1dTUpFdeeUV9fX1avHixenp6otvcc889euGFF7Rjxw41Njbq+PHjuvXWW9M4a3wZdSM+1I3EUDeGyclgc+fOdYLBYPTr/v5+JxAIOLW1tWmcVXaQ5OzcuTP6dSQScfx+v/PrX/86+lhXV5fj9XqdZ555Jg0zzHwfffSRI8lpbGx0HOfc8Ro1apSzY8eO6Dbvv/++I8nZt29fuqaJL6FuDB91I3HUjaHJ2DMfZ8+eVXNzs6qqqqKP5eXlqaqqSvv27UvjzLLT0aNH1dHREXM8fT6f5s2bx/G8gFAoJEkqKSmRJDU3N6uvry/mGM6YMUNTpkzhGGYI6kZyUTfiR90YmoxtPk6cOKH+/n6VlZXFPF5WVqaOjo40zSp7fXHMOJ5DE4lEtGbNGs2fP18zZ86UdO4YFhQUqLi4OGZbjmHmoG4kF3UjPtSNobso3RMAMlEwGNShQ4f0xhtvpHsqALIEdWPoMvbMx8SJE5Wfn3/eFcGdnZ3y+/1pmlX2+uKYcTzdrVq1Srt379brr7+uyZMnRx/3+/06e/asurq6YrbnGGYO6kZyUTeGjroRn4xtPgoKCjRnzhw1NDREH4tEImpoaFBlZWUaZ5adKioq5Pf7Y45nOBzW/v37OZ7/4ziOVq1apZ07d+q1115TRUVFTD5nzhyNGjUq5hi2tLTo2LFjHMMMQd1ILuqGO+rGMKX7ileT7du3O16v16mrq3Pee+89Z+XKlU5xcbHT0dGR7qllpO7ubuftt9923n77bUeS89hjjzlvv/2289///tdxHMf51a9+5RQXFzvPP/+8c/DgQefmm292KioqnNOnT6d55pnh7rvvdnw+n7Nnzx6nvb09Ok6dOhXd5ic/+YkzZcoU57XXXnMOHDjgVFZWOpWVlWmcNb6MuhEf6kZiqBvDk9HNh+M4zu9//3tnypQpTkFBgTN37lynqakp3VPKWK+//roj6byxfPlyx3HOfWxu3bp1TllZmeP1ep1FixY5LS0t6Z10Bhns2Elytm7dGt3m9OnTzk9/+lNn/PjxzpgxY5xbbrnFaW9vT9+kMSjqxtBRNxJD3Rgej+M4jr3zLAAAINdl7DUfAABgZKL5AAAAVtF8AAAAq2g+AACAVTQfAADAKpoPAABgFc0HAACwiuYDAABYRfMBAACsovkAAABW0XwAAACr/g9Mq0xfZZIpGQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "import os\n",
        "os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'\n",
        "\n",
        "import tensorflow as tf\n",
        "from tensorflow.keras.layers import Dense, Input, Conv2D, MaxPooling2D, Flatten, Conv2DTranspose\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras.datasets import cifar10, mnist\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "enc_input = Input(shape=(28, 28, 1))\n",
        "x = Conv2D(32, 3, activation='relu')(enc_input)\n",
        "x = MaxPooling2D(2, padding='same')(x)\n",
        "x = Conv2D(64, 3, activation='relu')(x)\n",
        "x = MaxPooling2D(2, padding='same')(x)\n",
        "x = Flatten()(x)\n",
        "enc_output = Dense(8, activation='linear')(x)\n",
        "\n",
        "encoder = keras.Model(enc_input, enc_output, name=\"encoder\")\n",
        "\n",
        "dec_input = keras.Input(shape=(8,), name=\"encoded_img\")\n",
        "x = Dense(7 * 7 * 8, activation='relu')(dec_input)\n",
        "x = keras.layers.Reshape((7, 7, 8))(x)\n",
        "x = Conv2DTranspose(64, 5, strides=(2, 2), activation=\"relu\", padding='same')(x)\n",
        "x = keras.layers.BatchNormalization()(x)\n",
        "x = Conv2DTranspose(32, 5, strides=(2, 2), activation=\"linear\", padding='same')(x)\n",
        "x = keras.layers.BatchNormalization()(x)\n",
        "dec_output = Conv2DTranspose(1, 3, activation=\"sigmoid\", padding='same')(x)\n",
        "\n",
        "decoder = keras.Model(dec_input, dec_output, name=\"decoder\")\n",
        "\n",
        "autoencoder_input = Input(shape=(28, 28, 1), name=\"img\")\n",
        "x = encoder(autoencoder_input)\n",
        "autoencoder_output = decoder(x)\n",
        "\n",
        "autoencoder = keras.Model(autoencoder_input, autoencoder_output, name=\"autoencoder\")\n",
        "#autoencoder.summary()\n",
        "\n",
        "\n",
        "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
        "\n",
        "x_train = x_train.astype(\"float32\") / 255.0\n",
        "x_test = x_test.astype(\"float32\") / 255.0\n",
        "y_train = keras.utils.to_categorical(y_train, 10)\n",
        "y_test = keras.utils.to_categorical(y_test, 10)\n",
        "\n",
        "autoencoder.compile(optimizer='adam', loss='mean_squared_error')\n",
        "autoencoder.fit(x_train, x_train, batch_size=32, epochs=1)\n",
        "\n",
        "h = encoder.predict(tf.expand_dims(x_test[0], axis=0))\n",
        "img = decoder.predict(h)\n",
        "\n",
        "plt.subplot(121)\n",
        "plt.imshow(x_test[0], cmap='gray')\n",
        "plt.subplot(122)\n",
        "plt.imshow(img.squeeze(), cmap='gray')\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cOloD1B8j_Ji",
        "outputId": "ba9e2cc6-6709-4374-a90f-641f749ccaef"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz\n",
            "170498071/170498071 [==============================] - 4s 0us/step\n",
            "Epoch 1/15\n",
            "625/625 [==============================] - 224s 355ms/step - loss: 1.8336 - accuracy: 0.2839 - val_loss: 1.5420 - val_accuracy: 0.4100\n",
            "Epoch 2/15\n",
            "625/625 [==============================] - 230s 368ms/step - loss: 1.3686 - accuracy: 0.4907 - val_loss: 1.1847 - val_accuracy: 0.5582\n",
            "Epoch 3/15\n",
            "625/625 [==============================] - 233s 373ms/step - loss: 1.1304 - accuracy: 0.5887 - val_loss: 1.0540 - val_accuracy: 0.6098\n",
            "Epoch 4/15\n",
            "625/625 [==============================] - 236s 377ms/step - loss: 0.9899 - accuracy: 0.6442 - val_loss: 0.8885 - val_accuracy: 0.6792\n",
            "Epoch 5/15\n",
            "625/625 [==============================] - 224s 358ms/step - loss: 0.8789 - accuracy: 0.6864 - val_loss: 0.8821 - val_accuracy: 0.6917\n",
            "Epoch 6/15\n",
            "625/625 [==============================] - 229s 367ms/step - loss: 0.8047 - accuracy: 0.7128 - val_loss: 0.7837 - val_accuracy: 0.7197\n",
            "Epoch 7/15\n",
            "625/625 [==============================] - 230s 368ms/step - loss: 0.7384 - accuracy: 0.7398 - val_loss: 0.7285 - val_accuracy: 0.7417\n",
            "Epoch 8/15\n",
            "625/625 [==============================] - 232s 370ms/step - loss: 0.6794 - accuracy: 0.7630 - val_loss: 0.7667 - val_accuracy: 0.7350\n",
            "Epoch 9/15\n",
            "625/625 [==============================] - 231s 370ms/step - loss: 0.6236 - accuracy: 0.7820 - val_loss: 0.6972 - val_accuracy: 0.7616\n",
            "Epoch 10/15\n",
            "625/625 [==============================] - 228s 364ms/step - loss: 0.5755 - accuracy: 0.7969 - val_loss: 0.7201 - val_accuracy: 0.7535\n",
            "Epoch 11/15\n",
            "625/625 [==============================] - 228s 365ms/step - loss: 0.5400 - accuracy: 0.8116 - val_loss: 0.6841 - val_accuracy: 0.7720\n",
            "Epoch 12/15\n",
            "625/625 [==============================] - 230s 369ms/step - loss: 0.5038 - accuracy: 0.8238 - val_loss: 0.6632 - val_accuracy: 0.7769\n",
            "Epoch 13/15\n",
            "625/625 [==============================] - 222s 355ms/step - loss: 0.4627 - accuracy: 0.8409 - val_loss: 0.6784 - val_accuracy: 0.7837\n",
            "Epoch 14/15\n",
            "625/625 [==============================] - 230s 368ms/step - loss: 0.4295 - accuracy: 0.8509 - val_loss: 0.7357 - val_accuracy: 0.7684\n",
            "Epoch 15/15\n",
            "625/625 [==============================] - 232s 371ms/step - loss: 0.4134 - accuracy: 0.8550 - val_loss: 0.7043 - val_accuracy: 0.7709\n",
            "313/313 [==============================] - 14s 46ms/step - loss: 0.7334 - accuracy: 0.7665\n",
            "[0.7334170341491699, 0.7664999961853027]\n"
          ]
        }
      ],
      "source": [
        "import os\n",
        "os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'\n",
        "\n",
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras.datasets import cifar10, mnist\n",
        "\n",
        "tf.random.set_seed(1)\n",
        "\n",
        "\n",
        "(x_train, y_train), (x_test, y_test) = cifar10.load_data()\n",
        "\n",
        "x_train = x_train / 255\n",
        "x_test = x_test / 255\n",
        "\n",
        "y_train = keras.utils.to_categorical(y_train, 10)\n",
        "y_test = keras.utils.to_categorical(y_test, 10)\n",
        "\n",
        "inputs = keras.Input(shape=(32, 32, 3), name=\"img\")\n",
        "x = layers.Conv2D(32, 3, activation=\"relu\")(inputs)\n",
        "x = layers.Conv2D(64, 3, activation=\"relu\")(x)\n",
        "block_1_output = layers.MaxPooling2D(3)(x)\n",
        "\n",
        "x = layers.Conv2D(64, 3, activation=\"relu\", padding=\"same\")(block_1_output)\n",
        "x = layers.Conv2D(64, 3, activation=\"relu\", padding=\"same\")(x)\n",
        "block_2_output = layers.add([x, block_1_output])\n",
        "\n",
        "x = layers.Conv2D(64, 3, activation=\"relu\", padding=\"same\")(block_2_output)\n",
        "x = layers.Conv2D(64, 3, activation=\"relu\", padding=\"same\")(x)\n",
        "block_3_output = layers.add([x, block_2_output])\n",
        "\n",
        "x = layers.Conv2D(64, 3, activation=\"relu\")(block_3_output)\n",
        "x = layers.GlobalAveragePooling2D()(x)\n",
        "x = layers.Dense(256, activation=\"relu\")(x)\n",
        "x = layers.Dropout(0.5)(x)\n",
        "outputs = layers.Dense(10, activation='softmax')(x)\n",
        "\n",
        "model = keras.Model(inputs, outputs, name=\"toy_resnet\")\n",
        "#model.summary()\n",
        "\n",
        "model.compile(optimizer='adam',\n",
        "             loss='categorical_crossentropy',\n",
        "             metrics=['accuracy'])\n",
        "\n",
        "model.fit(x_train, y_train, batch_size=64, epochs=15, validation_split=0.2)\n",
        "\n",
        "print( model.evaluate(x_test, y_test) )"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.8"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}