Keras6.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.8"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/vn322/Sport-University/blob/main/Keras6.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Callbacks Keras"
      ],
      "metadata": {
        "id": "Q9lphOLwnZgW"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Сегодня обсудим:\n",
        "1. Что такое callback?\n",
        "2. Для чего callback нужен при обучении нейросети?\n",
        "3. Рассмотрим основные callback на Keras"
      ],
      "metadata": {
        "id": "lsgGTfpbH4-M"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Получение данных\n",
        "\n",
        "Описание датасета:\n",
        "\n",
        "- CRIM     per capita crime rate by town\n",
        "- ZN       proportion of residential land zoned for lots over 25,000 sq.ft.\n",
        "- INDUS    proportion of non-retail business acres per town\n",
        "- CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)\n",
        "- NOX      nitric oxides concentration (parts per 10 million)\n",
        "- RM       average number of rooms per dwelling\n",
        "- AGE      proportion of owner-occupied units built prior to 1940\n",
        "- DIS      weighted distances to five Boston employment centres\n",
        "- RAD      index of accessibility to radial highways\n",
        "- TAX      full-value property-tax rate per 10,000\n",
        "- PTRATIO  pupil-teacher ratio by town\n",
        "- B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town\n",
        "- LSTAT    % lower status of the population\n",
        "- MEDV     Median value of owner-occupied homes in $1000's"
      ],
      "metadata": {
        "id": "5wJj0qVRMUhf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from keras.datasets import boston_housing\n",
        "\n",
        "(X_train, y_train), (X_test, y_test) = boston_housing.load_data()\n",
        "\n",
        "X_train.shape, X_test.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "22XUy6MVL66Z",
        "outputId": "d8e75708-2e3b-4b56-afd2-f3ebdf6b058f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((404, 13), (102, 13))"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "X_train[:1]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Q5Ds6QFBMiAX",
        "outputId": "5654ba98-757d-48fa-b1ca-3ce03898ffe3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[  1.23247,   0.     ,   8.14   ,   0.     ,   0.538  ,   6.142  ,\n",
              "         91.7    ,   3.9769 ,   4.     , 307.     ,  21.     , 396.9    ,\n",
              "         18.72   ]])"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Масштабирование данных"
      ],
      "metadata": {
        "id": "CSYSvyorMWTO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "mean = X_train.mean(axis=0)\n",
        "std = X_train.std(axis=0)\n",
        "\n",
        "mean, std"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xAepFcolMM93",
        "outputId": "05de93e9-fabc-4fa4-e4c7-58d1318929df"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([3.74511057e+00, 1.14801980e+01, 1.11044307e+01, 6.18811881e-02,\n",
              "        5.57355941e-01, 6.26708168e+00, 6.90106436e+01, 3.74027079e+00,\n",
              "        9.44059406e+00, 4.05898515e+02, 1.84759901e+01, 3.54783168e+02,\n",
              "        1.27408168e+01]),\n",
              " array([9.22929073e+00, 2.37382770e+01, 6.80287253e+00, 2.40939633e-01,\n",
              "        1.17147847e-01, 7.08908627e-01, 2.79060634e+01, 2.02770050e+00,\n",
              "        8.68758849e+00, 1.66168506e+02, 2.19765689e+00, 9.39946015e+01,\n",
              "        7.24556085e+00]))"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "X_train -= mean\n",
        "X_train /= std\n",
        "\n",
        "X_test -= mean\n",
        "X_test /= std"
      ],
      "metadata": {
        "id": "qu2qimTJMMyT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "X_train.mean(axis=0)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "p3vd2WHqMMoa",
        "outputId": "ecafd8d7-8916-4ac3-82a0-8c8c3acf1eda"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([-1.01541438e-16,  1.09923072e-17,  1.74337992e-15, -1.26686340e-16,\n",
              "       -5.25377321e-15,  6.41414864e-15,  2.98441140e-16,  4.94653823e-16,\n",
              "        1.12671149e-17, -1.98136337e-16,  2.36686358e-14,  5.95679996e-15,\n",
              "        6.13920356e-16])"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "X_train.std(axis=0)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PSOWwTi7MMhI",
        "outputId": "3dcc057b-1b4e-498c-9611-a88d52a21b52"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Архитектура сети\n",
        "\n",
        "\n",
        "Определение сети через класс Sequential и добавление слоев в него через add"
      ],
      "metadata": {
        "id": "Z1XTMfjpNU8S"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from keras.models import Sequential\n",
        "from keras.layers import Dense\n",
        "import tensorflow as tf\n",
        "tf.random.set_seed(9)\n",
        "\n",
        "\n",
        "model = Sequential()\n",
        "model.add(Dense(32, activation='relu', input_shape=(X_train.shape[1], )))\n",
        "model.add(Dense(16, activation='relu'))\n",
        "model.add(Dense(1))\n",
        "\n",
        "model.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "P734IyAUNWpM",
        "outputId": "d8e861c7-4607-4fac-b4e0-ea4f1c6352e0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " dense (Dense)               (None, 32)                448       \n",
            "                                                                 \n",
            " dense_1 (Dense)             (None, 16)                528       \n",
            "                                                                 \n",
            " dense_2 (Dense)             (None, 1)                 17        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 993\n",
            "Trainable params: 993\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model.compile(optimizer='adam', loss='mse', metrics=['mae'])"
      ],
      "metadata": {
        "id": "6bxv11s1NnJu"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Обучение сети"
      ],
      "metadata": {
        "id": "QmoBr-uXRcy2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "history = model.fit(X_train, y_train,\n",
        "                    epochs=50, \n",
        "                    validation_data=(X_test, y_test))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "U2FULpqdqOqk",
        "outputId": "c533d628-1696-4893-8638-ab90524d7fe4"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/50\n",
            "13/13 [==============================] - 1s 19ms/step - loss: 568.1458 - mae: 21.9958 - val_loss: 586.9446 - val_mae: 22.4357\n",
            "Epoch 2/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 548.3544 - mae: 21.5271 - val_loss: 566.4633 - val_mae: 21.9536\n",
            "Epoch 3/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 526.5145 - mae: 20.9990 - val_loss: 542.5834 - val_mae: 21.3806\n",
            "Epoch 4/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 500.7500 - mae: 20.3578 - val_loss: 514.4185 - val_mae: 20.6882\n",
            "Epoch 5/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 470.4189 - mae: 19.5854 - val_loss: 480.9787 - val_mae: 19.8343\n",
            "Epoch 6/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 434.6254 - mae: 18.6555 - val_loss: 442.5670 - val_mae: 18.8151\n",
            "Epoch 7/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 394.5962 - mae: 17.5325 - val_loss: 398.4568 - val_mae: 17.6801\n",
            "Epoch 8/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 349.9172 - mae: 16.2734 - val_loss: 348.3925 - val_mae: 16.4060\n",
            "Epoch 9/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 299.5508 - mae: 14.7792 - val_loss: 294.2163 - val_mae: 14.9413\n",
            "Epoch 10/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 246.4727 - mae: 13.1105 - val_loss: 235.7321 - val_mae: 13.1803\n",
            "Epoch 11/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 193.1914 - mae: 11.1790 - val_loss: 182.4164 - val_mae: 11.4719\n",
            "Epoch 12/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 146.8641 - mae: 9.4199 - val_loss: 143.0115 - val_mae: 10.0086\n",
            "Epoch 13/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 115.7502 - mae: 8.1897 - val_loss: 116.1516 - val_mae: 8.7589\n",
            "Epoch 14/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 95.5486 - mae: 7.3156 - val_loss: 98.5280 - val_mae: 7.8929\n",
            "Epoch 15/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 81.5020 - mae: 6.6943 - val_loss: 85.5257 - val_mae: 7.2363\n",
            "Epoch 16/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 70.6172 - mae: 6.1313 - val_loss: 74.1513 - val_mae: 6.6325\n",
            "Epoch 17/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 60.8533 - mae: 5.6348 - val_loss: 65.4592 - val_mae: 6.1901\n",
            "Epoch 18/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 53.4683 - mae: 5.2305 - val_loss: 57.9930 - val_mae: 5.8007\n",
            "Epoch 19/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 46.8867 - mae: 4.8706 - val_loss: 51.7539 - val_mae: 5.4857\n",
            "Epoch 20/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 41.8914 - mae: 4.6025 - val_loss: 46.6215 - val_mae: 5.2817\n",
            "Epoch 21/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 37.7009 - mae: 4.3923 - val_loss: 42.7632 - val_mae: 5.0911\n",
            "Epoch 22/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 34.5108 - mae: 4.2171 - val_loss: 39.7050 - val_mae: 4.9577\n",
            "Epoch 23/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 31.8019 - mae: 4.0639 - val_loss: 37.2880 - val_mae: 4.8103\n",
            "Epoch 24/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 29.6347 - mae: 3.9275 - val_loss: 35.2174 - val_mae: 4.7216\n",
            "Epoch 25/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 28.0028 - mae: 3.8202 - val_loss: 33.4373 - val_mae: 4.6348\n",
            "Epoch 26/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 26.5075 - mae: 3.7177 - val_loss: 32.1551 - val_mae: 4.5525\n",
            "Epoch 27/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 25.2216 - mae: 3.6267 - val_loss: 30.8402 - val_mae: 4.4789\n",
            "Epoch 28/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 24.2512 - mae: 3.5587 - val_loss: 30.0257 - val_mae: 4.4232\n",
            "Epoch 29/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 23.2691 - mae: 3.4848 - val_loss: 29.0684 - val_mae: 4.3508\n",
            "Epoch 30/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 22.4343 - mae: 3.4155 - val_loss: 28.3075 - val_mae: 4.2850\n",
            "Epoch 31/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 21.7021 - mae: 3.3578 - val_loss: 27.5971 - val_mae: 4.2218\n",
            "Epoch 32/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 21.1741 - mae: 3.3128 - val_loss: 27.1976 - val_mae: 4.1695\n",
            "Epoch 33/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 20.5393 - mae: 3.2514 - val_loss: 26.6274 - val_mae: 4.1050\n",
            "Epoch 34/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 20.0258 - mae: 3.2170 - val_loss: 26.2300 - val_mae: 4.0632\n",
            "Epoch 35/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 19.5465 - mae: 3.1775 - val_loss: 25.8447 - val_mae: 4.0075\n",
            "Epoch 36/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 18.9933 - mae: 3.1256 - val_loss: 25.2771 - val_mae: 3.9403\n",
            "Epoch 37/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 18.6208 - mae: 3.0973 - val_loss: 25.0819 - val_mae: 3.9148\n",
            "Epoch 38/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 18.0341 - mae: 3.0557 - val_loss: 24.7810 - val_mae: 3.8625\n",
            "Epoch 39/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 17.6707 - mae: 3.0163 - val_loss: 24.4798 - val_mae: 3.8176\n",
            "Epoch 40/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 17.2502 - mae: 2.9785 - val_loss: 24.2330 - val_mae: 3.7825\n",
            "Epoch 41/50\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 16.9081 - mae: 2.9516 - val_loss: 23.9965 - val_mae: 3.7487\n",
            "Epoch 42/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 16.5542 - mae: 2.9185 - val_loss: 23.8272 - val_mae: 3.7099\n",
            "Epoch 43/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 16.2783 - mae: 2.8894 - val_loss: 23.5236 - val_mae: 3.6661\n",
            "Epoch 44/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 15.9892 - mae: 2.8640 - val_loss: 23.9089 - val_mae: 3.6682\n",
            "Epoch 45/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 15.6189 - mae: 2.8369 - val_loss: 23.5805 - val_mae: 3.6141\n",
            "Epoch 46/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 15.3247 - mae: 2.8043 - val_loss: 23.4106 - val_mae: 3.5792\n",
            "Epoch 47/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 15.1319 - mae: 2.7935 - val_loss: 23.1997 - val_mae: 3.5546\n",
            "Epoch 48/50\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 14.8406 - mae: 2.7601 - val_loss: 22.7984 - val_mae: 3.5048\n",
            "Epoch 49/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 14.5706 - mae: 2.7252 - val_loss: 23.0147 - val_mae: 3.5025\n",
            "Epoch 50/50\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 14.3121 - mae: 2.7021 - val_loss: 23.2137 - val_mae: 3.5020\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "history.history"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8AyLNLxsqaiH",
        "outputId": "0dd3023a-6cf5-47ca-8bb0-b16f96afbd11"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'loss': [568.145751953125,\n",
              "  548.3544311523438,\n",
              "  526.5145263671875,\n",
              "  500.7500305175781,\n",
              "  470.41888427734375,\n",
              "  434.62542724609375,\n",
              "  394.5962219238281,\n",
              "  349.917236328125,\n",
              "  299.5508117675781,\n",
              "  246.4727325439453,\n",
              "  193.19140625,\n",
              "  146.8640899658203,\n",
              "  115.75017547607422,\n",
              "  95.54859924316406,\n",
              "  81.50202941894531,\n",
              "  70.6172103881836,\n",
              "  60.85331726074219,\n",
              "  53.46826171875,\n",
              "  46.8867073059082,\n",
              "  41.89142608642578,\n",
              "  37.700889587402344,\n",
              "  34.51081848144531,\n",
              "  31.80191421508789,\n",
              "  29.634727478027344,\n",
              "  28.002803802490234,\n",
              "  26.507484436035156,\n",
              "  25.221569061279297,\n",
              "  24.251182556152344,\n",
              "  23.269088745117188,\n",
              "  22.434255599975586,\n",
              "  21.70208168029785,\n",
              "  21.174062728881836,\n",
              "  20.539318084716797,\n",
              "  20.025815963745117,\n",
              "  19.54646110534668,\n",
              "  18.993345260620117,\n",
              "  18.620819091796875,\n",
              "  18.03410530090332,\n",
              "  17.670673370361328,\n",
              "  17.250192642211914,\n",
              "  16.908100128173828,\n",
              "  16.55420684814453,\n",
              "  16.278331756591797,\n",
              "  15.989215850830078,\n",
              "  15.61890697479248,\n",
              "  15.324657440185547,\n",
              "  15.131909370422363,\n",
              "  14.840559005737305,\n",
              "  14.570591926574707,\n",
              "  14.312091827392578],\n",
              " 'mae': [21.995834350585938,\n",
              "  21.527118682861328,\n",
              "  20.999004364013672,\n",
              "  20.357776641845703,\n",
              "  19.585378646850586,\n",
              "  18.655542373657227,\n",
              "  17.532506942749023,\n",
              "  16.27336311340332,\n",
              "  14.779224395751953,\n",
              "  13.110522270202637,\n",
              "  11.179001808166504,\n",
              "  9.41987419128418,\n",
              "  8.189728736877441,\n",
              "  7.315576076507568,\n",
              "  6.69431734085083,\n",
              "  6.131317615509033,\n",
              "  5.634832859039307,\n",
              "  5.23053503036499,\n",
              "  4.870604991912842,\n",
              "  4.6025471687316895,\n",
              "  4.392312526702881,\n",
              "  4.217063903808594,\n",
              "  4.063925743103027,\n",
              "  3.927515983581543,\n",
              "  3.820192337036133,\n",
              "  3.7177364826202393,\n",
              "  3.6266896724700928,\n",
              "  3.558689832687378,\n",
              "  3.484762668609619,\n",
              "  3.4155027866363525,\n",
              "  3.357825517654419,\n",
              "  3.3127782344818115,\n",
              "  3.251358985900879,\n",
              "  3.216977834701538,\n",
              "  3.177464485168457,\n",
              "  3.125622034072876,\n",
              "  3.0972585678100586,\n",
              "  3.055713415145874,\n",
              "  3.0163118839263916,\n",
              "  2.9785029888153076,\n",
              "  2.951610803604126,\n",
              "  2.9185428619384766,\n",
              "  2.8893959522247314,\n",
              "  2.86403751373291,\n",
              "  2.8369197845458984,\n",
              "  2.804344892501831,\n",
              "  2.7935402393341064,\n",
              "  2.760113000869751,\n",
              "  2.725160598754883,\n",
              "  2.702106475830078],\n",
              " 'val_loss': [586.944580078125,\n",
              "  566.4632568359375,\n",
              "  542.5834350585938,\n",
              "  514.41845703125,\n",
              "  480.9786682128906,\n",
              "  442.5669860839844,\n",
              "  398.45684814453125,\n",
              "  348.3924865722656,\n",
              "  294.2163391113281,\n",
              "  235.73211669921875,\n",
              "  182.41636657714844,\n",
              "  143.01148986816406,\n",
              "  116.1516342163086,\n",
              "  98.52796936035156,\n",
              "  85.52571868896484,\n",
              "  74.15132904052734,\n",
              "  65.45916748046875,\n",
              "  57.99304962158203,\n",
              "  51.753936767578125,\n",
              "  46.621524810791016,\n",
              "  42.76323699951172,\n",
              "  39.704986572265625,\n",
              "  37.28800582885742,\n",
              "  35.21744155883789,\n",
              "  33.437286376953125,\n",
              "  32.155052185058594,\n",
              "  30.840164184570312,\n",
              "  30.025671005249023,\n",
              "  29.06844711303711,\n",
              "  28.30754280090332,\n",
              "  27.597091674804688,\n",
              "  27.197614669799805,\n",
              "  26.62737274169922,\n",
              "  26.23000144958496,\n",
              "  25.84467315673828,\n",
              "  25.277149200439453,\n",
              "  25.08193588256836,\n",
              "  24.781044006347656,\n",
              "  24.479843139648438,\n",
              "  24.23297691345215,\n",
              "  23.996517181396484,\n",
              "  23.827150344848633,\n",
              "  23.523637771606445,\n",
              "  23.908916473388672,\n",
              "  23.58051109313965,\n",
              "  23.410594940185547,\n",
              "  23.199731826782227,\n",
              "  22.79837417602539,\n",
              "  23.014711380004883,\n",
              "  23.2137451171875],\n",
              " 'val_mae': [22.435688018798828,\n",
              "  21.95355796813965,\n",
              "  21.380645751953125,\n",
              "  20.688249588012695,\n",
              "  19.834287643432617,\n",
              "  18.81509780883789,\n",
              "  17.680091857910156,\n",
              "  16.406009674072266,\n",
              "  14.9413480758667,\n",
              "  13.1802978515625,\n",
              "  11.471870422363281,\n",
              "  10.008591651916504,\n",
              "  8.758904457092285,\n",
              "  7.892892837524414,\n",
              "  7.236267566680908,\n",
              "  6.632521152496338,\n",
              "  6.190135478973389,\n",
              "  5.80069637298584,\n",
              "  5.485721588134766,\n",
              "  5.281671047210693,\n",
              "  5.091116905212402,\n",
              "  4.957669734954834,\n",
              "  4.810307025909424,\n",
              "  4.7216057777404785,\n",
              "  4.634791851043701,\n",
              "  4.552516937255859,\n",
              "  4.478949546813965,\n",
              "  4.423216342926025,\n",
              "  4.350799560546875,\n",
              "  4.285003185272217,\n",
              "  4.221817493438721,\n",
              "  4.169478416442871,\n",
              "  4.105000019073486,\n",
              "  4.063167095184326,\n",
              "  4.007526397705078,\n",
              "  3.9403321743011475,\n",
              "  3.9148337841033936,\n",
              "  3.862520217895508,\n",
              "  3.817568302154541,\n",
              "  3.7825114727020264,\n",
              "  3.748728036880493,\n",
              "  3.709853172302246,\n",
              "  3.6661040782928467,\n",
              "  3.6681525707244873,\n",
              "  3.614068031311035,\n",
              "  3.579241991043091,\n",
              "  3.554636001586914,\n",
              "  3.5048089027404785,\n",
              "  3.5025365352630615,\n",
              "  3.502004384994507]}"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.plot(history.history['loss'], label='train loss')\n",
        "plt.plot(history.history['val_loss'], label='test loss')\n",
        "plt.legend();"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "qhgLo3lVqczd",
        "outputId": "3891c829-7a23-480c-db72-125e8b88d1f1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXwV5dn/8c91lmxkIUBkx4CAELaggCgqO7Io4FqrVLS29KnW5WdLweexaBcfsbZurdqiUmn1oW51A1RQQcQFCGEV0IRFSViSAAnZz3b//jgTCBBIIMvknHO9X6/zmpl7Zs5cg8fvTO4zM0eMMSillAovDrsLUEop1fA03JVSKgxpuCulVBjScFdKqTCk4a6UUmHIZXcBAG3atDGpqal2l6GUUiFl3bp1BcaYlJrmNYtwT01NJSMjw+4ylFIqpIjId6eap90ySikVhuoU7iLSUkTeEJHtIrJNRC4WkVYiskxEsqxhsrWsiMjTIpItIptE5ILG3QWllFInquuZ+1PAB8aYXsAAYBswG/jYGNMD+NiaBpgA9LBeM4DnGrRipZRStaq1z11EkoDLgVsBjDEewCMiU4AR1mILgBXALGAK8E8TfK7BV9ZZf3tjzL4Gr14p1ex5vV5ycnKoqKiwu5SQFRMTQ6dOnXC73XVepy5fqHYF8oF/iMgAYB1wD9C2WmDvB9pa4x2BPdXWz7Hajgt3EZlB8MyeLl261LlgpVRoycnJISEhgdTUVETE7nJCjjGGgwcPkpOTQ9euXeu8Xl26ZVzABcBzxpiBQCnHumCqNm6AM3oCmTFmnjFmkDFmUEpKjVfyKKXCQEVFBa1bt9ZgP0siQuvWrc/4L5+6hHsOkGOMWW1Nv0Ew7A+ISHtr4+2BPGt+LtC52vqdrDalVITSYK+fs/n3qzXcjTH7gT0icr7VNBrYCrwLTLfapgPvWOPvArdYV80MBYoarb/94A5Y9iDoY4uVUuo4db1a5i7gFRHZBKQD/wvMBcaKSBYwxpoGWALsBLKB54E7GrTi6rYvhs+fhE9+32ibUEqFtsLCQp599tmzWnfixIkUFhbWefmHHnqIP/3pT2e1rYZWpztUjTEbgEE1zBpdw7IGuLOeddXNJXfBwSz47M+Q1BkG3dYkm1VKhY6qcL/jjpPPM30+Hy7XqWNwyZIljVlaowrtO1RFYNIT0H0sLL4Pvl1qd0VKqWZm9uzZ7Nixg/T0dGbOnMmKFSu47LLLmDx5MmlpaQBMnTqVCy+8kD59+jBv3ryj66amplJQUMDu3bvp3bs3P/3pT+nTpw/jxo2jvLz8tNvdsGEDQ4cOpX///lx99dUcPnwYgKeffpq0tDT69+/PjTfeCMCnn35Keno66enpDBw4kOLi4nrvd7N4tky9OF1w/Uvw0kR4/Va4bTF0GGh3VUqpGvz2va/ZuvdIg75nWodEHryqzynnz507ly1btrBhwwYAVqxYQWZmJlu2bDl6aeH8+fNp1aoV5eXlDB48mGuvvZbWrVsf9z5ZWVksXLiQ559/nhtuuIE333yTadOmnXK7t9xyC3/5y18YPnw4c+bM4be//S1PPvkkc+fOZdeuXURHRx/t8vnTn/7EM888w7BhwygpKSEmJqa+/ywhfuZeJToebnoN4lrDKzfA4d12V6SUasaGDBly3DXjTz/9NAMGDGDo0KHs2bOHrKysk9bp2rUr6enpAFx44YXs3r37lO9fVFREYWEhw4cPB2D69OmsXLkSgP79+3PzzTfz8ssvH+0SGjZsGPfddx9PP/00hYWFp+0qqquQPnM3xrD/SAXtk2IhoR1MewNeHAsvXwe3L4W4VnaXqJSq5nRn2E2pRYsWR8dXrFjBRx99xJdffklcXBwjRoyo8Zry6Ojoo+NOp7PWbplTWbx4MStXruS9997j4YcfZvPmzcyePZtJkyaxZMkShg0bxocffkivXr3O6v2rhPSZ+3Of7mD8k5/x7QGrfyrlfLhxIRR+B/++Cbx6u7NSkS4hIeG0fdhFRUUkJycTFxfH9u3b+eqrr+q9zaSkJJKTk/nss88A+Ne//sXw4cMJBALs2bOHkSNH8uijj1JUVERJSQk7duygX79+zJo1i8GDB7N9+/Z61xDS4X5V/w5EuRzc8uIacguto2jqMJj6HHz/JSz9H3sLVErZrnXr1gwbNoy+ffsyc+bMk+aPHz8en89H7969mT17NkOHDm2Q7S5YsICZM2fSv39/NmzYwJw5c/D7/UybNo1+/foxcOBA7r77blq2bMmTTz5J37596d+/P263mwkTJtR7+2KawQ1AgwYNMmf7Yx3b9h3hhr9/yTkJ0bz+X5fQqkVUcMaH/wNf/hWumw99r23AapVSZ2Lbtm307t3b7jJCXk3/jiKyzhhT02XqoX3mDtC7fSIv3DKIPYfL+fFLaynz+IIzxjwEnYbAu/cE72RVSqkIEvLhDnBRt9b85YcD2ZRTyB2vZOL1B8Dphuv/EbxU8rXp4D27Lz+UUioUhUW4A1zRpx3/e3U/VnyTz6/f2EQgYCCpE1w9Dw5shg9m1/4mSikVJsIm3AFuHNKFX43ryVvrc3nk/W3Bxp7jYNi9sO4l2PS6rfUppVRTCenr3Gty58juFJR4eP6zXXRKjmP6Jakw6jewZzW8dw+0HwApPe0uUymlGlVYnblD8LnHc65MY0zvtvxu0VY+zy4I9rtfNx/cMfD6dPCU2V2mUko1qrALdwCHQ3jyxnS6p8RzxyuZ7CoohcQOcM3zkLcNPnrI7hKVUk2kPo/8BXjyyScpK6v5hHDEiBGc7WXcjS0swx0gPtrFC9MH4RD4yYK1HKnwQvfRMGQGrJkHe9baXaJSqgk0Zrg3Z2Eb7gCdW8Xx7M0X8t3BMu5euB5/wMDo30BiR3j3LvB57C5RKdXITnzkL8Bjjz3G4MGD6d+/Pw8++CAApaWlTJo0iQEDBtC3b19effVVnn76afbu3cvIkSMZOXLkabezcOFC+vXrR9++fZk1axYAfr+fW2+9lb59+9KvXz+eeOIJoObH/ja0sPtC9UQXn9eahyb34YG3t/DoB9v574m9YdKfYeEP4POnYPjJtyMrpRrJ+7Nh/+aGfc92/WDC3FPOPvGRv0uXLiUrK4s1a9ZgjGHy5MmsXLmS/Px8OnTowOLFi4HgM2eSkpJ4/PHHWb58OW3atDnlNvbu3cusWbNYt24dycnJjBs3jrfffpvOnTuTm5vLli1bAI4+4remx/42tLA+c68ybei5/GjoucxbuZM31uXA+eOhzzWw8o+Q/63d5SmlmtDSpUtZunQpAwcO5IILLmD79u1kZWXRr18/li1bxqxZs/jss89ISkqq83uuXbuWESNGkJKSgsvl4uabb2blypV069aNnTt3ctddd/HBBx+QmJgI1PzY34YW9mfuVeZclUZ2Xgn//Z/NnJfSgoETHoUdnwQvj7x1MTgi4jinlL1Oc4bdVIwx3H///fzsZz87aV5mZiZLlizhgQceYPTo0cyZM6de20pOTmbjxo18+OGH/O1vf+O1115j/vz5NT72t6FDPmISze108OzNF5CSEM29r26g1N0KrngYvv8CMl+yuzylVCM58ZG/V1xxBfPnz6ekpASA3Nxc8vLy2Lt3L3FxcUybNo2ZM2eSmZlZ4/o1GTJkCJ9++ikFBQX4/X4WLlzI8OHDKSgoIBAIcO211/KHP/yBzMzMUz72t6FFzJk7QHKLKJ74QTo/mPclv1+0lbnX3AybXoVlD0LPCZDY3u4SlVINrPojfydMmMBjjz3Gtm3buPjiiwGIj4/n5ZdfJjs7m5kzZ+JwOHC73Tz33HMAzJgxg/Hjx9OhQweWL19e4zbat2/P3LlzGTlyJMYYJk2axJQpU9i4cSO33XYbgUAAgEceeeToY3+Lioowxhx97G9DC/lH/p6Nue9v52+f7uD5WwYxtm0JPHcJdB8DN77SZDUoFSn0kb8NI+Ie+Xs27hvbk7T2icx+cxP5UZ1gxP2wfRFsfdfu0pRSqkFEZLhHuRw8eWM6xZU+Zr25CXPxndC2X/CXm3yVdpenlFL1FpHhDtCzbQKzx/fik+15LMzYB+N+B4XfQ8Z8u0tTKuw0h+7fUHY2/351CncR2S0im0Vkg4hkWG2tRGSZiGRZw2SrXUTkaRHJFpFNInLBGVfVRG69JJVh3Vvz+0Vb2ZV0EXQbAZ/+ESqK7C5NqbARExPDwYMHNeDPkjGGgwcPEhMTc0br1ekLVRHZDQwyxhRUa/sjcMgYM1dEZgPJxphZIjIRuAuYCFwEPGWMueh079/UX6hWt6+onCueWEm3lHjemBKL64WRcPlMGPWALfUoFW68Xi85OTlUVFTYXUrIiomJoVOnTrjd7uPaT/eFan0uhZwCjLDGFwArgFlW+z9N8KjxlYi0FJH2xph99dhWo2mfFMvDV/fjroXrefabc7i7zzXw5TMw+CeQ0M7u8pQKeW63m65du9pdRsSpa5+7AZaKyDoRmWG1ta0W2PuBttZ4R2BPtXVzrLbjiMgMEckQkYz8/PyzKL3hXDWgA5P6t+evy7PZe8GvwO8Jds8opVSIqmu4X2qMuQCYANwpIpdXn2mdpZ9Rh5oxZp4xZpAxZlBKSsqZrNoofjMpDZdDmLOqHC68LfizfAXZdpellFJnpU7hbozJtYZ5wFvAEOCAiLQHsIZ51uK5QOdqq3ey2pq1dkkx3DWqBx9tO8Cqjj8GVwx88nu7y1JKqbNSa7iLSAsRSagaB8YBW4B3genWYtOBd6zxd4FbrKtmhgJFzbW//UQ/vjSVbm1a8JuP8vANvQO2vg256+wuSymlzlhdztzbAqtEZCOwBlhsjPkAmAuMFZEsYIw1DbAE2AlkA88DdzR41Y0k2uXkwcl92FVQyj/MlRDXJvjcGb2ESykVYmq9WsYYsxMYUEP7QWB0De0GuLNBqrPB8J4pjEtry+Of7ueG0feStOIB2PFx8NkzSikVIiL2DtXT+c2VaQSMYU7uEGh5Lnz0Wz17V0qFFA33GnRuFcfPR5zHO5sLyE67E/ZvguyP7S5LKaXqTMP9FP5r+Hl0So7lni3dMYkdYNUTdpeklFJ1puF+CjFuJ3OuTOPrvApWt7sJvlsFe9bYXZZSStWJhvtpjE1ry+U9U7jn2/4EYpJh1ZN2l6SUUnWi4X4aIsKvrzifAxUu1qRcB98shrztdpellFK10nCvRd+OSYxLa8vMPRdh3HHw+VN2l6SUUrXScK+De8f0ZE9FHOvbTIbNr0HhntpXUkopG2m410Fah0TG92nHrL2XB5+O9uUzdpeklFKnpeFeR/eM6UFWZUu2tr4CMhdA6UG7S1JKqVPScK+j3u0TmdC3HffnjQZvGayZZ3dJSil1ShruZ+CeMT3YVNmO7FaXw5q/Q2WJ3SUppVSNNNzPQK92iUzs1445B8dB+WHI/KfdJSmlVI003M/QPaN78kVlN75PvAC+/Cv4PHaXpJRSJ9FwP0Pnt0tgUr/2zC0cA0dygzc2KaVUM6PhfhbuGdODD739KYpqDxnz7S5HKaVOouF+Fnq2TWBCv478o2I47FoJBVl2l6SUUsfRcD9LvxjVnZc9w/GLEzL+YXc5Sil1HA33s9SrXSK9e5zHJ1yE2fAKeMvtLkkppY7ScK+Hn17WjRcrRyEVhfD1W3aXo5RSR2m418NlPdpQmDKE7x2dMPrFqlKqGdFwrwcR4fbLuvGPypFIzlrYt8nukpRSCtBwr7fJ6R34NHYMHonSyyKVUs2Ghns9RbucXDusL+94h+Lf9BpUFttdklJKabg3hJsv6sLrcgVObylsetXucpRSSsO9IbSMi6LXhcP52qTiXf0CGGN3SUqpCFfncBcRp4isF5FF1nRXEVktItki8qqIRFnt0dZ0tjU/tXFKb15+fGk3XvaNwV2wDfassbscpVSEO5Mz93uAbdWmHwWeMMZ0Bw4Dt1vttwOHrfYnrOXCXmqbFpT2nEoJsfjWvGB3OUqpCFencBeRTsAk4AVrWoBRwBvWIguAqdb4FGsaa/5oa/mwN31EH970XYpsfRvKDtldjlIqgtX1zP1J4NdAwJpuDRQaY3zWdA7Q0RrvCOwBsOYXWcsfR0RmiEiGiGTk5+efZfnNy4XntiLznKk4Ax4CG/WLVaWUfWoNdxG5EsgzxqxryA0bY+YZYwYZYwalpKQ05Fvb6oqRo/k6cC5H1v6f3aUopSJYXc7chwGTRWQ38G+C3TFPAS1FxGUt0wnItcZzgc4A1vwk4GAD1tysjUtry8fuEbQ8tAkKsu0uRykVoWoNd2PM/caYTsaYVOBG4BNjzM3AcuA6a7HpwDvW+LvWNNb8T4yJnGsDXU4H0QN/QMAIRWtesbscpVSEqs917rOA+0Qkm2Cf+otW+4tAa6v9PmB2/UoMPVddegFfmD7BfvfIOa4ppZoRV+2LHGOMWQGssMZ3AkNqWKYCuL4BagtZHVrG8sE5E7m04I/4vl+N69yhdpeklIoweodqIzlv+I2UmyhyP11Q+8JKKdXANNwbyaV9urHKOYRWuxeBz2N3OUqpCKPh3kicDqG893UkBI5wYP1iu8tRSkUYDfdGdNGY6zhoEjj45b/sLkUpFWE03BtR2+QENiaNptuhz6gsOWx3OUqpCKLh3siSLrqZGDxs/USveVdKNR0N90Y2cOgY9kh73Ftet7sUpVQE0XBvZA6ng71dJpNWuZHdu761uxylVITQcG8C3cf8GIcYsj9+ye5SlFIRQsO9CbTu3IudMWl0zllEhddvdzlKqQig4d5EAv1+wPl8xxeff2p3KUqpCKDh3kS6DZ+GDydl6xbaXYpSKgJouDcRR3wb9rQcTP8jy8krKre7HKVUmNNwb0Jx6dfRRfL5fNXHdpeilApzGu5NqO2Qa/HjwL/5LbtLUUqFOQ33phTXin2tLmJw2UqyDxTbXY1SKoxpuDexxAuv41xHHl9+/ondpSilwpiGexNLTL8aPw5k69tE0E/LKqWamIZ7U2vRmoI2Q7nU8zmZ3x2yuxqlVJjScLdB0uDrSXUcYPUXekOTUqpxaLjbIKbvFPw4iP72Xbz+gN3lKKXCkIa7HVq0prDtUEYFvmDlN3l2V6OUCkMa7jZJGnQ9XR0HWPPVSrtLUUqFIQ13m7jSJuPHScvdiymu8NpdjlIqzGi426VFG0rbD+UKvuLDLfvtrkYpFWZqDXcRiRGRNSKyUUS+FpHfWu1dRWS1iGSLyKsiEmW1R1vT2db81MbdhdCVcMF1dHPsZ/3az+wuRSkVZupy5l4JjDLGDADSgfEiMhR4FHjCGNMdOAzcbi1/O3DYan/CWk7VQHpfRQAH7XM/JO9Ihd3lKKXCSK3hboJKrEm39TLAKOANq30BMNUan2JNY80fLSLSYBWHk/gUKjpezATHat7dkGt3NUqpMFKnPncRcYrIBiAPWAbsAAqNMT5rkRygozXeEdgDYM0vAlrX8J4zRCRDRDLy8/PrtxchLC79Ws5z7GNT5hd2l6KUCiN1CndjjN8Ykw50AoYAveq7YWPMPGPMIGPMoJSUlPq+XejqPZkADroXfMR3B0vtrkYpFSbO6GoZY0whsBy4GGgpIi5rViegql8hF+gMYM1PAg42SLXhKD4FT6eLGe9Yy6JN++yuRikVJupytUyKiLS0xmOBscA2giF/nbXYdOAda/xdaxpr/idGH394WjF9r6KnI5d169fZXYpSKkzU5cy9PbBcRDYBa4FlxphFwCzgPhHJJtin/qK1/ItAa6v9PmB2w5cdZs6fCEC3g5+yI7+kloWVUqp2rtoWMMZsAgbW0L6TYP/7ie0VwPUNUl2kSD4Xb5s0xuatY/Gmfdw9uofdFSmlQpzeodpMuNMmMdjxLSs3bLe7FKVUGNBwby7On4iDAOceXMW3+vuqSql60nBvLjoMxB/fnnHOdXrVjFKq3jTcmwsRnL0mMty1iQ837tbfV1VK1YuGe3PSayIxppIOh9awbZ92zSilzp6Ge3OSehmBqHiucK5j0aa9dlejlAphGu7NiSsaR/cxjHdvYMmmXO2aUUqdNQ335qbXJFoGDpF8eDNbco/YXY1SKkRpuDc3PcZixMk4V6Z2zSilzpqGe3MTm4ykDmNyzAYWbdqnXTNKqbOi4d4cnT+Jjt7vcBXtYsOeQrurUUqFIA335uj8CQBMcGXqDU1KqbOi4d4cJZ8LbftydYtNLN60j0BAu2aUUmdGw725On8iPSu2UHkkj8zvD9tdjVIqxGi4N1e9JiIEGOfeqF0zSqkzpuHeXLVPh4QO3Ji4mSWb9+HXrhml1BnQcG+uRKDXRPpVrONI8RHW7j5kd0VKqRCi4d6cpU3B5S/nCvdGFmvXjFLqDGi4N2fnDoMW5zA9aT3vb9mHzx+wuyKlVIjQcG/OHE5Im8KA8tWUlRxhzS7tmlFK1Y2Ge3PX9xqc/gomRG3gPe2aUUrVkYZ7c9d5KMS345bE9XygXTNKqTrScG/uHA7oM5W+ZavxlhXx5c6DdleklAoBGu6hoM81OAMeJkVvZNFG7ZpRStVOwz0UdBoMiR2ZlpDJB1/vx6tdM0qpWmi4hwKHA/pcTZ+yNZjyQlZlF9hdkVKqmas13EWks4gsF5GtIvK1iNxjtbcSkWUikmUNk612EZGnRSRbRDaJyAWNvRMRoc/VOAJerozZoDc0KaVqVZczdx/wS2NMGjAUuFNE0oDZwMfGmB7Ax9Y0wASgh/WaATzX4FVHoo4XQlIXpsWv48Ov91Pp89tdkVKqGas13I0x+4wxmdZ4MbAN6AhMARZYiy0AplrjU4B/mqCvgJYi0r7BK480ItBnKr1KM5CKQlZladeMUurUzqjPXURSgYHAaqCtMaaqf2A/0NYa7wjsqbZajtV24nvNEJEMEcnIz88/w7IjVJ+rcRgfU63fV1VKqVOpc7iLSDzwJnCvMeZI9Xkm+CvOZ/RMWmPMPGPMIGPMoJSUlDNZNXJ1GAjJqdwcn8GyrQeo8GrXjFKqZnUKdxFxEwz2V4wx/7GaD1R1t1jDPKs9F+hcbfVOVpuqLxHoczU9S9fhqjzMsq0H7K5IKdVM1eVqGQFeBLYZYx6vNutdYLo1Ph14p1r7LdZVM0OBomrdN6q++lyNGD83xm/k1bV7al9eKRWR6nLmPgz4ETBKRDZYr4nAXGCsiGQBY6xpgCXATiAbeB64o+HLjmDt+kOr87gpPoNV2QXsOVRmd0VKqWbIVdsCxphVgJxi9ugaljfAnfWsS52K1TXTedXjpEghr2fs4b5x59tdlVKqmdE7VENR+k2IMTyQ8jmvr8vR31dVSp1Ewz0UtT4Pek1iQsUiCosK+SxLLyVVSh1Pwz1UXXI3UZ4ibo1dpV+sKqVOouEeqrpcBJ0v4mfu91m+bS8FJZV2V6SUakY03EPZJXfT0rOPMWY1b2XqrQRKqWM03EPZ+ROhdXfujXufV9d+T/BCJaWU0nAPbQ4HXPwLuvuySTm4hszvC+2uSCnVTGi4h7oBP8TEpfBz92JeXfu93dUopZoJDfdQ545BLprB5bKB7ZtWU1Lps7sipVQzoOEeDgb/BL8rllvMeyzauNfuapRSzYCGeziIa4Vj4DSmOL9g2er1dlejlGoGNNzDhFzyC5wEGHzgdb7ZX2x3OUopm2m4h4vkVLznX8XNzo949sNMu6tRStlMwz2MRF9+LwlSTu+s5/k8W39jValIpuEeTjpegG/ANH7qWsyrb72Fzx+wuyKllE003MOMa8L/4ok9h7uLH+fVL7PsLkcpZRMN93ATk0TMtc/S3bEX70d/oLDMY3dFSikbaLiHIek+msLeN/Ej8x5vvP2f2ldQSoUdDfcw1XLKoxRHncOobx7i25w8u8tRSjUxDfdwFZOIc+ozdJN9ZP17tj4xUqkIo+EexhL6jGV7p+uZUPwf1q5cYnc5SqkmpOEe5s676c/kOVJov+JXVJbrnatKRQoN9zDnjksib9Sf6Wz2smP+TyHgt7skpVQT0HCPAP0vm8ySlJ+Qlv8+373wI/DrY4GVCnca7hFizM8e49XE2zh372LyFvwI/F67S1JKNaJaw11E5otInohsqdbWSkSWiUiWNUy22kVEnhaRbBHZJCIXNGbxqu6iXA4m3vEYz8f+mHO+X0LRv34EPr3BSalwVZcz95eA8Se0zQY+Nsb0AD62pgEmAD2s1wzguYYpUzWEhBg3U37+CE+4bidp9/uUvXIz+CrtLksp1QhqDXdjzErg0AnNU4AF1vgCYGq19n+aoK+AliLSvqGKVfV3TmIMV834HQ/LT4jbtRTPKz8Eb4XdZSmlGtjZ9rm3Ncbss8b3A22t8Y7AnmrL5VhtJxGRGSKSISIZ+fn5Z1mGOhvdz4ln/K3/w//4Z+Da9Qn+f06FI/rzfEqFk3p/oWqCtz6e8e2Pxph5xphBxphBKSkp9S1DnaELz23F8Bt/yf/z3ok3J5PAs5fA9sV2l6WUaiBnG+4HqrpbrGHVw0tygc7VlutktalmaFyfdlx+zc+Z4nmErMpk+PdNsOg+8JbbXZpSqp7ONtzfBaZb49OBd6q132JdNTMUKKrWfaOaoWsv7MSj/3UtM9yP8GJgEmS8CPNGwoGv7S5NKVUPdbkUciHwJXC+iOSIyO3AXGCsiGQBY6xpgCXATiAbeB64o1GqVg0qvXNL/nP3SD7pcje3eGZRfPgAZt5IWD0PAvprTkqFImkOTwscNGiQycjIsLuMiOcPGP689BteXZHJ3xNeZJA3AzoPhauegnN62V2eUuoEIrLOGDOopnl6h6o6yukQfj2+F4/8aBS3eWbyG+6gcv82zN8uhU8e1ksmlQohGu7qJOP6tOPduy4jq8NkLimey6fuYbDyj/C3YbB7ld3lKaXqQMNd1ahrmxYs/OlQ5tw4nJmBu5jumcWh4lJ4aRK8c6deF69UM6fhrk5JRJiS3pGPfzmc8y6eymUlj/CSTCGwYSHmqQHw3r1weLfdZSqlaqBfqKo627r3CHPe2cL+77/h/oQPmeD7CIcJQP8b4NL7IKWn3SUqFVFO94Wqhrs6I4GA4b1Ne3nsw2/wHs7loTYfc0X5+zj8lZA2BS65CzpeCCJ2l6pU2NNwVw2u0ufn5a++56+fZOEoK2BuxzeZ3ZUAAAwjSURBVFWMPvIODm8JtE+HwT+BvtdCVJzdpSoVtjTcVaM5UuHl75/u4MVVu4gNlPFA581MrFhMbOG3EJME6dNg8O3Q+jy7S1Uq7Gi4q0a3v6iCZ5Zn85/MHEo9Pm5I2cOd8SvocuAjJOCD1MuCZ/JpUyCuld3lKhUWNNxVkymp9PHW+lxe+eo7tu8vpmt0MQ90yODS8k+ILtwBDhecNwr6Xge9JkJ0gt0lKxWyNNxVkzPGsO67w7z81Xcs2bwfj9/PlSkFzEjOpM/hj3AW54IrFrqPhh7joPsYSKrx0f9KqVPQcFe2OlTqYdGmvbyZmcvGPYU4JcBtXfK5KW4tqQUrcBRbN0Sd0wd6jIHuY6HLUHC67S1cqWZOw101G9l5Jby1Poe31+8lt7CcaJdwTccjTE3YRr+yNcTuX4sEvOCOgw4Dg5dVdhoEHQfpmb1SJ9BwV81OIGBYvesQS7fuZ+W3+ezILwXgvETDLe12c5lrG53LvsadtwUC3uBKCe2DYd+2b/ApleekQatueoavIpaGu2r2cgvLWfltPiu/zWdVdgHFFT4AuiW7mNz2IJfGfkdP33YSDm5GDu8CYz1n3uGGNj2DYd+6OySnQnLX4DC+LTj0CRsqfGm4q5Di8wfYsvcIGbsPkbH7MBnfHaagpBKAxBgXA9pHMyzpEAOi99E18D1tynfhKtgGRTnHQh/AFRMM+ZZdILFjsFsnqbM13gkSO4Ar2p6dVKoBaLirkGaM4buDZazdfYjM7wvZvv8I3+4vptTjP7pMl1ZxnJ8STXpCMb1jDpLqyKOdfz+xpd8jhXvgSC6UHTz5zWOTg9098W0hoV3wFd8WYlsF58UmB6/Lj00O3pTlcDbhnit1eqcLd1dTF6PUmRIRUtu0ILVNC64fFPz99UDAkHO4nO37j/DN/mK27y9mR34Jn+0MUOFNApKAHsRHu+jcKo6ObWPpmiT0iDlCF9dhOkgBrfwFxFXm4yjNg+J9UJAFJQeO9fHXJCYpGPxVgV81Hp0AUS3A3SI4jIqDqPjgKzo+OD86MTjtimqSfzcV2TTcVUhyOIQurePo0jqOcX3aHW0PBAz7jlSwK7+UnQUl7MwvZc+hMnIOl7F6ZznFlT4gGugIdEQEkuOiaBMfRZuEaFLauekc56Gdu5xz3GW0cZSS7Cgl0RQTHzhClKcIKT8M5YegtCB4QCg/DJXFQB3/CnZGBwPf3QLcscEDgds6ILirXrHWKy7Y7ooNHhScUcHvGZzu4LjTHexacsWcPHRGg9NlLRelf3VEGA13FVYcDqFjy1g6tozl0h5tTppfVO5lb2E5uYfL2XekgoLiSgpKKsm3hutzyllWUkmZxw8IEG+92gLgcggt49wkxbppGRdFcrKbpA5RJMY4aen209LlIcnlIdHhIdFZSbxU0oIK4kwZMaaMaH8pLm8JVJaAtxy8peApA28ZlOQFh95ya1gRHNb1oFErORb0VaFf44Ei9oQDRXTwwCDOY0NxBMcdzuBdx6d6OatPu49fR6rWrfa+J02fsD1xHHudajtI8K8vvwf8PmvoCX4fI47gflbtt8MVHIqj2nbC40t4DXcVUZJig8Hcu33iaZcr9/g5WFrJoVIPB0s9HCrxcKjUw+EyD4XlXorKvBwu87C3sIKte49QXOmjpNLHyV9huTh2gAiKcjmIj3YRF+WkRZSLuGhrGOskLslJbFRwXqzbSazbQbzTT4LTQ4zDT4zDT5QEiHH4iBI/0XiJFj/R4iUKL1F4iDIe3MYbfAyz3wq5QFXIeY8NA15r2hr3ecBfCb5KqCgCXx74KoLTAR8YPwT8wZA0fghUDX3B92iwg1AzcFzYVx1oHMcOOFUHhaN/FbmO/XUU8B//b3XcdLV/t6p5Y38H6T9s8F3QcFeqBrFRTjpFxdEpue6PLDbGUObxU2IFfXGFj1JrWFLpo6TCG2yv9FFS4aPc46fU46PM46e00kdBSSWlHh/lngAVXj9lHh+BM8pLl/UK1ux2CtEuJ1EuB9EuB1EuB1FOa1htPNrlwF01HuMg2uUk2uUg2u04ur7b6SDKKbicDlwOIcrlwOU4tn6M20m0E2KchhhHgChHAJf4cRPAJQHc+HFJAKfxVQs5azzgP/7gceL8qiA05oSDS9Uy1V5VBxlHtb9Gqs7UHS5rOa+1rO/YAa7q/c0JgXx0/IQ2f9V7eKodNK22E//qqH5QOKndEbyaqxFouCvVQESEFtEuWkS7rE6c+jHG4PEHqPAEKPP68PgCVPoCVHoDePx+Kr3WtM9PhTd4QKjw+qnwVY0H8PiCy1atWzX0+oPD4gofB6tNe6z3qxr3ndnRpVYiWAcKB26n4HYeO7C4HILT4cTtdONyCm6HA5d1QHE75ITx4PpOh+ByOKyhVBsG1z2x/ejBye3AFR1cxu0MvsfR5ZzHxqteDgGHVI1XLXeqbQvSDH6sRsNdqWZKRKyzaCdJ2HMXrs8KfZ8/eKDxBYLjXn8Ar98cPRhUeI8dFCq8wYOJN2Dw+autW7VeIIDXFxz3BQJ4fFXzgweTqqHXHzzAlHr8R9/Ha22/ahl/wFQbBvAHDF6//d1DzhPC/rhxEZxOa+gQ7h3Tk6sGdGjwGjTclVKn5HI6cDlD7wvGwClC/8SDU3AYIGCC848dLILzAwGD3xgCJvie/oAhYE48qBxb3ue3lq/WHjxgHd9+dL4xtIxrnAO3hrtSKuw4HEKUo6prJDIvAW2UQ7KIjBeRb0QkW0RmN8Y2lFJKnVqDh7uIOIFngAlAGvBDEUlr6O0opZQ6tcY4cx8CZBtjdhpjPMC/gSmNsB2llFKn0Bjh3hHYU206x2o7jojMEJEMEcnIz89vhDKUUipy2fY1uDFmnjFmkDFmUEpKil1lKKVUWGqMcM8FOleb7mS1KaWUaiKNEe5rgR4i0lVEooAbgXcbYTtKKaVOocGvczfG+ETkF8CHBC8wnW+M+bqht6OUUurUmsUvMYlIPvDdWa7eBihowHJCRaTuN0Tuvut+R5a67Pe5xpgav7RsFuFeHyKScaqfmQpnkbrfELn7rvsdWeq736H30AillFK10nBXSqkwFA7hPs/uAmwSqfsNkbvvut+RpV77HfJ97koppU4WDmfuSimlTqDhrpRSYSikwz1SnhsvIvNFJE9EtlRrayUiy0Qkyxom21ljYxCRziKyXES2isjXInKP1R7W+y4iMSKyRkQ2Wvv9W6u9q4istj7vr1p3gIcdEXGKyHoRWWRNh/1+i8huEdksIhtEJMNqq9fnPGTDPcKeG/8SMP6EttnAx8aYHsDH1nS48QG/NMakAUOBO63/xuG+75XAKGPMACAdGC8iQ4FHgSeMMd2Bw8DtNtbYmO4BtlWbjpT9HmmMSa92bXu9PuchG+5E0HPjjTErgUMnNE8BFljjC4CpTVpUEzDG7DPGZFrjxQT/h+9ImO+7CSqxJt3WywCjgDes9rDbbwAR6QRMAl6wpoUI2O9TqNfnPJTDvU7PjQ9jbY0x+6zx/UBbO4tpbCKSCgwEVhMB+251TWwA8oBlwA6g0BjjsxYJ18/7k8CvgYA13ZrI2G8DLBWRdSIyw2qr1+dcfyA7DBhjjIiE7TWtIhIPvAnca4w5EjyZCwrXfTfG+IF0EWkJvAX0srmkRiciVwJ5xph1IjLC7nqa2KXGmFwROQdYJiLbq888m895KJ+5R/pz4w+ISHsAa5hncz2NQkTcBIP9FWPMf6zmiNh3AGNMIbAcuBhoKSJVJ2Th+HkfBkwWkd0Eu1lHAU8R/vuNMSbXGuYRPJgPoZ6f81AO90h/bvy7wHRrfDrwjo21NAqrv/VFYJsx5vFqs8J630UkxTpjR0RigbEEv29YDlxnLRZ2+22Mud8Y08kYk0rw/+dPjDE3E+b7LSItRCShahwYB2yhnp/zkL5DVUQmEuyjq3pu/MM2l9QoRGQhMILgI0APAA8CbwOvAV0IPi75BmPMiV+6hjQRuRT4DNjMsT7Y/ybY7x62+y4i/Ql+geYkeAL2mjHmdyLSjeAZbStgPTDNGFNpX6WNx+qW+ZUx5spw329r/96yJl3A/xljHhaR1tTjcx7S4a6UUqpmodwto5RS6hQ03JVSKgxpuCulVBjScFdKqTCk4a6UUmFIw10ppcKQhrtSSoWh/w9ugKdKC1+WywAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "plt.plot(history.history['mae'], label='train mae')\n",
        "plt.plot(history.history['val_mae'], label='test mae')\n",
        "plt.legend();"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "t_AbpIN0qd3I",
        "outputId": "1687aab3-586d-43f4-dfd3-b3f70ca3f837"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9dn//9eVZLInkISw78i+BQibgIDIIiru1gUrt96i1vZubW+rVqtt7/ZXe9evtbZWRaXWW0WoilBFBSm4IAIBAUGQRVnClkDIvs7k+v1xJhBCAiELJ5m5no/HPObMWWaug/F9znzOmc9HVBVjjDGBK8TtAowxxjQuC3pjjAlwFvTGGBPgLOiNMSbAWdAbY0yAC3O7gOq0atVKu3bt6nYZxhjTbKxfv/6oqiZXt6xJBn3Xrl1JS0tzuwxjjGk2RGRvTcus6cYYYwKcBb0xxgQ4C3pjjAlwTbKN3hgTWMrKykhPT6e4uNjtUpq9yMhIOnbsiMfjqfU2Zw16EekEvAK0ARSYo6p/FpE/AlcApcBu4D9UNbua7fcAeYAP8Kpqaq2rM8YEhPT0dOLi4ujatSsi4nY5zZaqcuzYMdLT0+nWrVutt6tN040X+Jmq9gNGAfeKSD9gGTBAVQcBO4CHzvAeE1U1xULemOBUXFxMUlKShXw9iQhJSUnn/M3orEGvqodUdYN/Og/YBnRQ1aWq6vWv9gXQ8RxrNsYEEQv5hlGXf8dzuhgrIl2BIcCaKotuB96vYTMFlorIehGZfa4FnpOP/xcObWrUjzDGmOam1kEvIrHAW8BPVDW30vyHcZp3Xqth07GqOhS4FKfZ56Ia3n+2iKSJSFpmZmatd+CEwixY/zK8OBm+fPXctzfGBKzs7Gz+9re/1Wnb6dOnk5192uXHZqVWQS8iHpyQf01V3640fxZwOXCL1jCCiaoe8D9nAAuBETWsN0dVU1U1NTm52l/xnll0Itz1CXQZDYvuhcX/BWV2hd8Yc+ag93q91c6vsGTJElq2bNkYZZ03Zw16cRqEXgK2qeqTleZPA34OzFDVwhq2jRGRuIppYAqwpSEKr1ZMK5j5Noz7GWz4B8ydCsdr/FWwMSZIPPjgg+zevZuUlBTuv/9+Vq5cybhx45gxYwb9+vUD4KqrrmLYsGH079+fOXPmnNi2a9euHD16lD179tC3b1/uvPNO+vfvz5QpUygqKjrts2bNmsU999zDqFGj6N69OytXruT222+nb9++zJo168R699xzD6mpqfTv35/HHnvsxPz169czfvx4hg0bxtSpUzl06FC9978299GPAW4FvhKRjf55vwCeBiKAZf6LA1+o6t0i0h54UVWn49ySudC/PAx4XVU/qHfVNdiVkU/XpGjCJj0KHVJh4d0wZzxc8yL0vKSxPtYYcw5+/a+tfH0w9+wrnoN+7eN57Ir+NS5//PHH2bJlCxs3OhG2cuVKNmzYwJYtW07cpjh37lwSExMpKipi+PDhXHvttSQlJZ3yPjt37mTevHm88MIL3HDDDbz11lvMnDnztM87fvw4q1evZvHixcyYMYNVq1bx4osvMnz4cDZu3EhKSgq/+93vSExMxOfzMWnSJDZv3kzfvn350Y9+xKJFi0hOTmb+/Pk8/PDDzJ07t17/PmcNelX9DKjuMu+SGtY/CEz3T38LDK5PgbWVXVjKdc99Tp+2cTx94xBa95kOs1fAgu/Da9fBpF/C2J+CXfk3xgAjRow45V70p59+moULFwKwf/9+du7ceVrQd+vWjZSUFACGDRvGnj17qn3vK664AhFh4MCBtGnThoEDBwLQv39/9uzZQ0pKCgsWLGDOnDl4vV4OHTrE119/TUhICFu2bGHy5MkA+Hw+2rVrV+99DZhfxraMDueRy/rxyDtfMf3pz3j6xhQuvKAH3LEM/vVfsPw3kJ8BU38PIdbzgzFuOdOZ9/kUExNzYnrlypV89NFHrF69mujoaCZMmFDtveoREREnpkNDQ6ttuqm8XkhIyCnbhISE4PV6+e6773jiiSdYt24dCQkJzJo1i+LiYlSV/v37s3r16obaTedzG/TdXHbdsI4suncsLaLCuOWlNfz5o534wqLg6jkw6l5Y8xy8czf4ytwu1RhzHsXFxZGXl1fj8pycHBISEoiOjmb79u188cUXjVpPbm4uMTExtGjRgiNHjvD++87d6b179yYzM/NE0JeVlbF169Z6f15ABT1A77ZxLP7hWK5K6cCfPtrBrL+v5WhhGUz9HVz8S9g8H964BUqrvX5sjAlASUlJjBkzhgEDBnD//feftnzatGl4vV769u3Lgw8+yKhRoxq1nsGDBzNkyBD69OnDzTffzJgxYwAIDw/nzTff5IEHHmDw4MGkpKTw+eef1/vzpIa7Il2Vmpqq9R14RFWZv24/jy7eSkK0h7/ePJThXRMhbS68+1PoPApuegOimvdtU8Y0B9u2baNv375ulxEwqvv3FJH1NXUzE3Bn9BVEhBtHdOadH4whOjyMm1/4gjfXp0Pq7XDdXEhPg5cvh7wjbpdqjDGNKmCDvkK/9vG8c+8YRnRL5L//uYknPvyG8n5Xw81vQNZu+PulkF+HX+IaY0wzEfBBD9AiysPL/zGCG4d34q8rdvGjN76kuMtEuHUh5B6E16+Hkny3yzTGmEYRFEEP4AkN4ffXDOShS/uw5KtD3DjnCzIThsD1f3c6QlvwfbsbxxgTkIIm6MFpt79rfA+evWUY2w/nctUzq/imxVi4/CnYvRwW/wia4MVpY4ypj6AK+grTBrRlwV2jKfWVc91zn7Ol7VUw4RewaR4s/7Xb5RljTIMKyqAHGNSxJQt/cCHxkR5ufWkN23vfDcP+Az77E6x53u3yjDENqD7dFAM89dRTFBY239/eBG3QA3RMiOb1O0cSHhbCzJfWsmv4r6H3ZfD+A7B1odvlGWMaiAV9kOuSFMPrd44ChFvmrmPfxU9DpxHw9mw4+KXb5RljGkDVbooB/vjHPzJ8+HAGDRp0opvggoICLrvsMgYPHsyAAQOYP38+Tz/9NAcPHmTixIlMnDjxtPfu2rUrDz30ECkpKaSmprJhwwamTp1Kjx49eO655wDIz89n0qRJDB06lIEDB7Jo0aIT27/66quMGDGClJQU7rrrLnw+X4Pvf8B0alYfPZJjee0/R3LjnNXc9PfN/PP7c2n/xhR483ZnMJOIOLdLNCZwvP8gHP6qYd+z7UC49PEaF1ftpnjp0qXs3LmTtWvXoqrMmDGDTz75hMzMTNq3b897770HOH3gtGjRgieffJIVK1bQqlWrat+/c+fObNy4kfvuu49Zs2axatUqiouLGTBgAHfffTeRkZEsXLiQ+Ph4jh49yqhRo5gxYwbbt29n/vz5rFq1Co/Hww9+8ANee+01vv/97zfoP0/Qn9FX6N02jv+7YyR5xWXc+OpOjk19Bo7vgSU/d7s0Y0wDW7p0KUuXLmXIkCEMHTqU7du3s3PnTgYOHMiyZct44IEH+PTTT2nRokWt3m/GjBkADBw4kJEjRxIXF0dycjIRERFkZ2ejqvziF79g0KBBXHLJJRw4cIAjR46wfPly1q9fz/Dhw0lJSWH58uV8++23Db6/dkZfyYAOLfjH7SO49aW1XP9BBO+N+ilRq5+AHhNh0A1ul2dMYDjDmff5oqo89NBD3HXXXact27BhA0uWLOGRRx5h0qRJPProo2d9v7N1S/zaa6+RmZnJ+vXr8Xg8dO3a9US3xLfddhu///3vG27nqlGboQQ7icgKEflaRLaKyI/98xNFZJmI7PQ/J9Sw/W3+dXaKyG0NvQMNbUjnBObOGk56VhF37ZuEdhrpdIKW1fBHWWPM+VG1m+KpU6cyd+5c8vOdX8QfOHCAjIwMDh48SHR0NDNnzuT+++9nw4YN1W5/rnJycmjdujUej4cVK1awd68zxOmkSZN48803ycjIACArK+vEsoZUm6YbL/AzVe0HjALuFZF+wIPAclXtCSz3vz6FiCQCjwEjcQYFf6ymA0JTMqJbIr+9egCf7D7OM4kPOgOVvHkHeEvdLs0YUwdVuymeMmUKN998M6NHj2bgwIFcd9115OXl8dVXX524MPrrX/+aRx55BIDZs2czbdq0ai/G1sYtt9xCWloaAwcO5JVXXqFPnz4A9OvXj9/+9rdMmTKFQYMGMXny5AYZI7aqc+6mWEQWAX/1Pyao6iERaQesVNXeVda9yb/OXf7Xz/vXm3emz2iIboobwqOLtvDK6r0sGHeEEevugzE/gcn2gypjzpV1U9ywGrWbYhHpCgwB1gBtVLXi0HMYZyDwqjoA+yu9TvfPaxZ+eXk/RnRL5NbV7cjqczOsegp2/9vtsowx5pzUOuhFJBZ4C/iJqp4yhLs6Xwvq1UmMiMwWkTQRScvMbBrdBntCQ/jbLUNJjAnnuu9m4E3sBQvvtm6NjTHNSq2CXkQ8OCH/mqq+7Z99xN9kg/85o5pNDwCdKr3u6J93GlWdo6qpqpqanJxc2/obXavYCJ6/dRjpBfBw6H1o0XFY+rDbZRnT7DTF0eyao7r8O9bmrhsBXgK2qeqTlRYtBiruorkNWFR1W+BDYIqIJPgvwk7xz2tWBnVsye+vHsj8/S34NPlmZ9zZ7z51uyxjmo3IyEiOHTtmYV9PqsqxY8eIjIw8p+1qcx/9GOBW4CsR2eif9wvgcWCBiNwB7AVuABCRVOBuVf1PVc0Skf8B1vm3+42qZp1ThU3EtcM6svVgLneumsCGxI+Iee9ncPdnEBbudmnGNHkdO3YkPT2dptIs25xFRkbSsWPHc9omYAcHbwxeXznXPbeazkc/4Wl9HC75NYz9idtlGWNMcA4O3hjCQkN44vpBfFCWwsboC9GP/wDZ+8++oTHGuMiC/hxd0DqOn07uxb1Z38NXXg4fnPY7MWOMaVIs6OvgznHdSe7Uk2fLr4Ht78KOpW6XZIwxNbKgr4PQEOGJ6wfxXNl0Dnk6o+/fD2VFbpdljDHVsqCvowtax/Gjyf34acGtyPE9zhCExhjTBFnQ18Od47pT1GEM7zMG/exPcGy32yUZY8xpLOjroaIJ57femRSXh6EfPuR2ScYYcxoL+nq6oHUct04eydOlM5AdH8Kez9wuyRhjTmFB3wDuHNeddW1v4AhJlH/4S2iCP0IzxgQvC/oGEBoi3H9ZCn8su46QQxtg60K3SzLGmBMs6BvIyO5J5Pa6lh3aGd+yX9loVMaYJsOCvgE9ML0/v/feRGjOXkib63Y5xhgDWNA3qB7JsXRMvYJV5QPwrXwcinPcLskYYyzoG9qPJ/fizzKT0OLj8NlTbpdjjDEW9A2tVWwEF42/hHd8F1K++hnIqXZALWOMOW8s6BvBHWO784/IW/H5fOiK37ldjjEmyFnQN4Ko8FBunjqOl71TYOPrcGSr2yUZY4JYbcaMnSsiGSKypdK8+SKy0f/YU2mIwarb7hGRr/zrNb0hoxrRNUM78mHiTPKJxrfsMbfLMcYEsdqc0b8MTKs8Q1W/p6opqpoCvAW8fYbtJ/rXrXaIq0AVGiL8+IoRPFt2GaG7lsGBDW6XZIwJUmcNelX9BKh2QG8REZxBwec1cF0BYVzPZHZ2vYlcYvCu/F+3yzHGBKn6ttGPA46o6s4aliuwVETWi8jsM72RiMwWkTQRSQukkeLvmTqEud6phO18Hw5vOfsGxhjTwOob9Ddx5rP5sao6FLgUuFdELqppRVWdo6qpqpqanJxcz7KajqGdE9je+WYKiMT78R/dLscYE4TqHPQiEgZcA8yvaR1VPeB/zgAWAiPq+nnN2Z1Th/EP7xRCty2CzB1ul2OMCTL1OaO/BNiuqunVLRSRGBGJq5gGpgBB2XYxrEsiX3W6hRI8+D55wu1yjDFBpja3V84DVgO9RSRdRO7wL7qRKs02ItJeRJb4X7YBPhORTcBa4D1V/aDhSm9eZk0ezqveSciWNyHrW7fLMcYEkbCzraCqN9Uwf1Y18w4C0/3T3wKD61lfwBjZPYm/t5/JrRkfEfrJk4Rd9Ve3SzLGBAn7Zex59P0po5jvnYBsmgfZ+90uxxgTJCzoz6PRPZL4vO1MVBWf9WxpjDlPLOjPIxHh5iljeNM7Dja8AnmH3S7JGBMELOjPs3E9W7Gy9a1ouRffqqfdLscYEwQs6M8zEeF7Uy5ike9CdN1cKKy2dwljjGkwFvQumNA7mZWJ3yPMV0T5l6+6XY4xJsBZ0LtARJh+yWTWlPeh+PPnodzndknGmABmQe+Syf3asMgzneiCdNi5zO1yjDEBzILeJWGhIbQbeQOHNYGiVc+6XY4xJoBZ0LvohlHdmOe7hKh9K+FoTT09G2NM/VjQu6hNfCRHet5IGaF4v5jjdjnGmABlQe+yq8YN4V3fKMo3vg4leW6XY4wJQBb0LhvZLZEVcVcS7s2HTW+4XY4xJgBZ0LtMREgdO4VN5d0p/vw5UHW7JGNMgLGgbwKuHtqRN5hGZPYu+O4Tt8sxxgQYC/omIC7SQ/jga8nSOEo/f87tcowxAaY2I0zNFZEMEdlSad6vROSAiGz0P6bXsO00EflGRHaJyIMNWXiguWlML+b5JhK26wPI3ud2OcaYAFKbM/qXgWnVzP+Tqqb4H0uqLhSRUOAZ4FKgH3CTiPSrT7GBrE/beLa0uw4Fp7MzY4xpIGcNelX9BKhLF4sjgF2q+q2qlgJvAFfW4X2CxqVjh7PUN4yydS9DWZHb5RhjAkR92uh/KCKb/U07CdUs7wBUHi8v3T+vWiIyW0TSRCQtMzOzHmU1X9P6t2Wx51LCS4/D9vfcLscYEyDqGvTPAj2AFOAQ8P/qW4iqzlHVVFVNTU5Oru/bNUvhYSH0GDGdQ5pI0Yb5bpdjjAkQdQp6VT2iqj5VLQdewGmmqeoA0KnS647+eeYMbhzZhcW+0YTvWW6DkhhjGkSdgl5E2lV6eTWwpZrV1gE9RaSbiIQDNwKL6/J5waRjQjR72k0nVH3o14vcLscYEwBqc3vlPGA10FtE0kXkDuB/ReQrEdkMTATu86/bXkSWAKiqF/gh8CGwDVigqlsbaT8CSurI8ewqb0/eunlul2KMCQBhZ1tBVW+qZvZLNax7EJhe6fUS4LRbL82ZTRvYjhcXj+XHRxZATjq06Oh2ScaYZsx+GdsExUSEUdDragDKNv3T5WqMMc2dBX0TNWH0CL4sv4CC9dajpTGmfizom6hR3ZJYGT6eljnbIWO72+UYY5oxC/omKiREiEq5Fp8K+XZWb4ypBwv6JmzaqBRWlQ/At+mf1k+9MabOLOibsK6tYtiUMJkWxeloeprb5RhjmikL+iau7cjrKVEPR1e/6nYpxphmyoK+iZs6rCcrdChROxaBz+t2OcaYZsiCvomLj/Swv+PlxHqPU7prhdvlGGOaIQv6ZqD3uGvI1WgyPrfmG2PMubOgbwbG9O7AytBRJO1bagOSGGPOmQV9MxAaIuT3uoYoLSR387tul2OMaWYs6JuJ4eOv4LAmkP2FNd8YY86NBX0z0bNdSz6LnED7zE+h4Jjb5RhjmhEL+makqO/1hOEjb70NM2iMqT0L+mYkdeQ4tpV3pmSDDUhijKm92owwNVdEMkRkS6V5fxSR7SKyWUQWikjLGrbd4x+JaqOI2G/466lP2zhWRF5Mq+zNcHSX2+UYY5qJ2pzRvwxMqzJvGTBAVQcBO4CHzrD9RFVNUdXUupVoKogIvv5Oj5bFG153uxxjTDNx1qBX1U+ArCrzlvrHhAX4ArCx7s6TsUMGOj1abpxvPVoaY2qlIdrobwfer2GZAktFZL2IzD7Tm4jIbBFJE5G0zMzMBigrMA3u2JJ/h08kpjAd9q9xuxxjTDNQr6AXkYcBL/BaDauMVdWhwKXAvSJyUU3vpapzVDVVVVOTk5PrU1ZACwkRPP2voFAjKPvSLsoaY86uzkEvIrOAy4FbVKtvQ1DVA/7nDGAhMKKun2dOmji4Ox+Wp6JbFoK3xO1yjDFNXJ2CXkSmAT8HZqhqYQ3rxIhIXMU0MAXYUt265tyM6JrIR56JhJflwI4P3S7HGNPE1eb2ynnAaqC3iKSLyB3AX4E4YJn/1snn/Ou2F5El/k3bAJ+JyCZgLfCeqn7QKHsRZMJCQ4jvO4lMbYlvk40na4w5s7CzraCqN1Uz+6Ua1j0ITPdPfwsMrld1pkZTBnbknU0XcsfOpVCYBdGJbpdkjGmi7JexzdSFFyTxQegEQsrLYOvbbpdjjGnCLOibqYiwUDr0Hs5OOqObrO8bY0zNLOibsUsHtuPNsjFI+lo4ttvtcowxTZQFfTM2vncy78tYFIHNC9wuxxjTRFnQN2PR4WH06dWHNTIY3fAK+Lxn38gYE3Qs6Ju5aQPa8kLJJCTvIGy3YQaNMaezoG/mJvVpw6cMJTuiHax9we1yjDFNkAV9M9ci2sOFPVvzqm8K7P0MDtuPj40xp7KgDwBXprTnhfwx+EIjYZ2d1RtjTmVBHwAm92tLiSeeL1tMdu6+KTrudknGmCbEgj4AxEaEcUnfNvwxezyUFcKXNfUabYwJRhb0AeKqlA6sKWxPdnKq03xT7nO7JGNME2FBHyAu6pVMiygP73gug+N7YNdHbpdkjGkiLOgDRHhYCNMHtuPJ9F6Ux7aFNc+7XZIxpomwoA8gV6a0J7dU2NHpeti9HI7ucrskY0wTYEEfQEZ0TaRdi0iezxsHIR671dIYA9Qy6EVkrohkiMiWSvMSRWSZiOz0PyfUsO1t/nV2ishtDVW4OV1IiDBjcHv+9a2Pkt5XwsbXoSTP7bKMMS6r7Rn9y8C0KvMeBJarak9guf/1KUQkEXgMGIkzMPhjNR0QTMOYkdIeb7myosWVUJILNtSgMUGvVkGvqp8AWVVmXwn8wz/9D+CqajadCixT1SxVPQ4s4/QDhmlA/drFc0HrWObuaQXthzgXZe1WS2OCWn3a6Nuo6iH/9GGcwcCr6gDsr/Q63T/vNCIyW0TSRCQtMzOzHmUFNxHhysHtWbvnOFkpd8OxnfD1O26XZYxxUYNcjFVVBbSe7zFHVVNVNTU5ObkhygpaV6Y4x9IFhcMguQ98/EcoL3e5KmOMW+oT9EdEpB2A/zmjmnUOAJ0qve7on2caUeekaIZ0bsmiTYfhovshcxtsW+R2WcYYl9Qn6BcDFXfR3AZUlyQfAlNEJMF/EXaKf55pZFcObs+2Q7nsaHUJtOoFH/+vndUbE6Rqe3vlPGA10FtE0kXkDuBxYLKI7AQu8b9GRFJF5EUAVc0C/gdY53/8xj/PNLLLBrUnNERYtPkwXPRzyPgatv/L7bKMMS4Qp3m9aUlNTdW0tDS3y2j2vj93LTuP5PHxzy4i/PnREBYBd30KIfY7OWMCjYisV9XU6pbZ//EB7PYxXTmUU8w/vzzotNUf2QLfvOd2WcaY88yCPoCN75XMkM4teebfuyjpexUk9oCP/wBN8FucMabxWNAHMBHhvkt6cTCnmAUb/HfgHP4KvlnidmnGmPPIgj7AjevZimFdEvjbil2U9LsGErvDysftrN6YIGJBH+AqzuoP5RQzf/0hGPffcHgz7PjA7dKMMeeJBX0QGHNBEsO7JvDMil0U97sOEro6Z/V2X70xQcGCPghUnNUfyS3hjbSDMP5BOLQR1tooVMYEAwv6IDG6RxIjuiXyt5W7Ke53PfSeDssedS7OGmMCmgV9kKg4q8/IK+H1tfthxl8hKhHe+k8oK3K7PGNMI7KgDyKjeyQxqnsiz368m+LwlnD1s5C5HZY+4nZpxphGZEEfZO67pBeZeSW8+sVe6HExjP4hrHsRvnnf7dKMMY3Egj7IjOyexIU9knju493kFJXBpEeh7SBYdC/kHXa7PGNMI7CgD0IPXtqH44VlPLpoi9PR2bUvQWkhLLzbbrk0JgBZ0AehQR1b8uNJPVm08SDvfHkAknvBtN/Dtyvgi7+5XZ4xpoFZ0AepH0zowbAuCfzynS3szyqEYbOgz+Xw0a9g3xq3yzPGNCAL+iAVFhrCU99LQYGfLtiIT4EZf4GWneC16+HQZrdLNMY0kDoHvYj0FpGNlR65IvKTKutMEJGcSus8Wv+STUPplBjNb67sz7o9x3l25S6IToTvL4KIOPi/q+HoTrdLNMY0gDoHvap+o6opqpoCDAMKgYXVrPppxXqq+pu6fp5pHFcP6cAVg9vz1Ec72bg/G1p2dsJeBF65Eo7vdbtEY0w9NVTTzSRgt6paKjQzIsJvrxpAm/hIfvLGlxSUeKHVBXDrO1Ca74S93XZpTLPWUEF/IzCvhmWjRWSTiLwvIv1regMRmS0iaSKSlpmZ2UBlmdpoEeXh/90wmL1ZhfzPu187M9sOgFvegvwMeOUqKLQx3Y1pruod9CISDswA/lnN4g1AF1UdDPwFeKem91HVOaqaqqqpycnJ9S3LnKNR3ZO4Z3wP3li33/nVLECn4XDTPMj6Fl69Bopz3S3SGFMnDXFGfymwQVWPVF2gqrmqmu+fXgJ4RKRVA3ymaQT3Te7FxX1a88g7W3htjT/su4+HG/7h9HL56rVQnONukcaYc9YQQX8TNTTbiEhbERH/9Aj/5x1rgM80jcATGsKzM4dycZ/WPLywUtj3vhSufxkOfum02VszjjHNSr2CXkRigMnA25Xm3S0id/tfXgdsEZFNwNPAjao2WGlTFhEWekrYv75mn7Og7xXwvVfhyFZ4ZQYU2PHamOZCmmLupqamalpamttlBLUSr497Xt3Av7dn8P9dPZCbR3Z2Fuz6CN64BRK6wW2LIba1u4UaYwAQkfWqmlrdMvtlrKlWxZn9xN7J/GLhVyfP7C+4BG5eANl74eXLIPeQu4UaY87Kgt7UKCIslOduHXYi7F9ZvcdZ0H08zHwLcg/Cy9MhJ93NMo0xZ2FBb86oIuwv6duGRxdt5cllO1BV6HIh3LoQCo7CnAmwY6nbpRpjamBBb84qIiyU52YO5YbUjjy9fCcPv7MFX7lCpxFwx1KIaQ2vXw9Lfm7jzxrTBFnQm1oJCw3hD9cO4gcTevD6mn3c+9oGist80Lov3PlvGHkPrH0eXrjYuTPHGNNkWNCbWhMRfj6tD49e3o8Pth7mtrlryS0uA08kXPq402VCwVGYMxG+eON6DqIAABI6SURBVA6a4B1dxgQjC3pzzm4f240/35jChn3H+d7zX5CRW+ws6HkJ3PM5dJ8AHzzg/JLWer80xnUW9KZOrkzpwEu3DWfvsQKuemYVm9OznQWxyXDzfJj+BOz7Av42ClY9DT6vuwUbE8Qs6E2dXdQrmQV3jUZEuO651by13n+bpQiMuBPuXQPdxsOyX8ILE+DAelfrNSZYWdCbehnQoQWLfziGYZ0T+Nk/N/GrxVsp85U7C1t2cnq/vOEVyM+EFyY5d+ZYL5jGnFcW9KbekmIj+L87RnD7mG68/PkeZr64hqP5Jc5CEeh3JfxwLQz/T1g7B54ZAWlzwVvqbuHGBAkLetMgwkJDePSKfjx5w2A27s9mxl8+O9luDxDZAi57Au5YBi06wrv3wV+GwYZXwFfmXuHGBAELetOgrhnakbfuuRAR4dpnP+epj3ZQ6i0/uUKn4U7Y3/IWxLSCxT+Cvw6HjfPsgq0xjcR6rzSNIquglF//ayuLNh6kT9s4/nDtIAZ3annqSqqw40NY8Ts4vBmSLoAhtzpNPYnd3CncmGbqTL1XWtCbRvXR10d45J0tZOQVc+e47tw3uReRntBTV1KF7e/CZ0/BAf9/97aDoP9V0O8qSOpx/gs3ppmxoDeuyi0u4/dLtjFv7X66JkXzh2sHMbJ7UvUrH98L2/4FX78D6euceW0GQJ/Loc905wDgDFpmjKmkUYNeRPYAeYAP8Fb9IP9Qgn8GpgOFwCxV3XCm97SgD0yf7zrKg29/xb6sQq4e0oEHL+1Dm/jImjfISfeH/iLnx1coxHd0hjbsMx26jIWw8PNWvzFN2fkI+lRVPVrD8unAj3CCfiTwZ1Udeab3tKAPXIWlXp5ZsYsXPvmOsFDh3okXcMfYbqc351SVnwk7P4TtS2D3v8FbBBHx0O0i6DAMOqZC+yEQEXd+dsSYJsbtoH8eWKmq8/yvvwEmqGqNQxNZ0Ae+fccK+e17X7P06yN0Tozm4cv6MqVfG6Q2zTKlhfDdx7D9Pdi7CrK+deZLCCT3cUK/43DoPNq5wGtNPSYINHbQfwccBxR4XlXnVFn+LvC4qn7mf70ceEBV06qsNxuYDdC5c+dhe/daZ1jB4LOdR/nNu1vZcSSfsRe04r8m9WR414TaBX6Fwiyne4X0NOdibnoaFPvv4Y9Jhs6joPOF0GU0tBkIoWGNszPGuKixg76Dqh4QkdbAMuBHqvpJpeW1CvrK7Iw+uHh95by2Zh9/+mgH2YVl9G8fz3+M6cYVg9sREXaWJp3qqMLRnbDvc9i7Gvatdsa4BfBEO3fxJHaHRP9zUg9nOra1nf2bZuu83XUjIr8C8lX1iUrzrOnG1EphqZeFXx7g5VV72JmRT6vYcG4e0ZmZo7rQ+kwXbWsj54AT+AfWw7FdTnPP8T1QXulHWp4Y/wGgq//Z/0joBvHtIaQOBx1jzpNGC3oRiQFCVDXPP70M+I2qflBpncuAH3LyYuzTqjriTO9rQR/cVJVVu47x8uffsXx7BmEhwsTerbl8cHsu6dua6PAGanrxeSFnnxP6x771h/93Jw8Cvkp98YR4oGVnSOh68tGys9NxW4tOThORfRswLjpT0Nf3/5g2wEJ/e2oY8LqqfiAidwOo6nPAEpyQ34Vze+V/1PMzTYATEcb2bMXYnq3Yc7SAV7/Yy782H2Tp10eI9IQwqU8bLh/Ujol9Wp/9bp0zCQ07edZ+QZVl5T7IPQDHdjvNPsf3nHwc3ABFx6u8V4TTh0/FIybZaQqKae300R/TGmLbQHSiHRDMeWc/mDLNQnm5sm5PFu9uPsT7Ww5xNL+U6PBQJvZuzfjeyUzolVz/5p1zUZQNOfshe79zv3/Ofv8j3WkmKsiE8mo6awuLPPWA0KITxHdwmobi20NcO6cDODsYmHNkv4w1AcXrK2fNd1m8u/kgy7dlkJHndInct10843slM6F3MsO6JOAJdbHPPlXnrL8gE/IzoCDDec5JP/WRf/j0bT3RTuBXBH98O+c5ru3J59g24Ik6//tlmiwLehOwVJXth/NY+U0mH+/IIG3PcbzlSnR4KMO6JDCqexIjuyUyqGNLwsOaYGet3hLIPQh5h059rpjOOwR5h0+9XlAhPA5ikpxmophkiE5ymotOHBTaOweJmNZ2S2kQsKA3QSOvuIzVu4/x2a6jrPk2i2+O5AEQ6QlhWJcERnRNYliXBAZ3akFcpMflamup4ttBRfDnHnK+CRQcg8KjzreGgmPOc+HRU+8kAueHZDHJEJUIUS0hKgEiW56cjk70X0NoffLaQnisNR81Mxb0JmhlFZSy9rtjfPFtFmu+y2L74VxUnQzr1TqOoV1aMqRTAkM6t6R7ciyhIc083MrLnbDPPeh8E8ireD7kHCyKsp1HcbbzujS/+vcJi3K+FVRcO4hvf/JaQkyy07wUHuM0H3minYf1O+QqC3pj/HKKyti0P5sv92WzYd9xvtx3nNxi5ww4IiyEnm1i6dUmjt5t4ujV1nlu1yLy3H6p25z4yqDgqP8aQubJawkFmc4BIvegc/dR3qHqm48qC/E4g8lEt3KeY1qdbFKKTjr57SEq8eS0XWdoMBb0xtSgvFz59mgBG/dn883hXLYfzmPHkTyO5JacWCcuIowL2sTSs3UsPVvH0bNNLD3bxNE+kA8AValC4TEn9AsyoazIeZQW+KcLoCTfWafg6MlmpIKjNX9rAAgNdzqni4hzHpEtTk6f8og/+RzZ4tRHRJw1M9G499Eb06yFhAgXtI7lgtaxp8zPLixlx5F8vjmcy44j+ezMyOPf2zNYkJZ+Yp3o8FC6JMXQvVUM3VrF0LXiOSmaxJjwwDoIiJw8Sz9XZUVOM1FhFhRlnTpdnAsluVCS55/Og+x9/nn5znPVaw6n1RbiHACiEk4+ohNPXosICXUOVOjJZ3CuQ1RcyI5p5VybiG4FnvN4m+55Ymf0xpyDrIJSdh7JY0dGPt9m5rPnaAHfHS1g//EifOUn/18KDwuhXYtI2sZHOs8tomjXIpL2LaPo4H/ER4UF1sGgMag6dyaV5PnDP9c5IBTnONcZinOcR+XrDicOJMedZVTNuIp/8xqyzxMDkfGnfpOIjHfucgqPPvW6hCfKuVYRleAcJKITnYNGRPx5/5ZhZ/TGNJDEmHBGdk86bYSsMl85+7MK+e5oAXuPFXIkt5hDOcUcyikibe9xjuQeosx3arDERoTRvmUkHVpG0a5lFO39BwTnOZJ2LaKICg/y/nVEnDNsT6TzC+NzVV5+8n2qBm9pgf96hP8aRUGmv8kpq9KBJc955B50mqDKCp1vKN7iM39uiMcJ/dAI/3Gl4vMrPVfdT3CuZdyx9Nz38yws6I1pAJ7QELonx9I9Obba5eXlytGCEg5mF3Mwu4gDx4s4kF3kTGcXsSk9h6yC0y92xkeGkRwX4X9EkhzrTCfFhtMyykNCjPPcMjqcltEed38k1hSFnOHfIzzGGYS+LgPRl/sqXafId5qhCrOcg0ah/7bXwmPOxe6qzUantaJUeh0Rf+611IIFvTHnQUiI0DouktZxkaR0alntOsVlPg7nFHMwp4jDOc43gozcYjLzS8jMK2HLgRwy80rIL6m5zTouIozk+Ahax0XQJj7yxHNyXATxUR7iIsKIjQwjLtJDbEQYsRFhzf+WUjeEhEJErPMgGajDweI8sqA3pomI9ITS1X9R90wKS70cyy8lp6iM44WlHC8sI8f/nFVQSkZeMRm5JWzYd5yM3BJKvOVnfL+Y8FBaRHmIj/IQH+khPiqM+CgPLaPCSYoNJykmnMSYcJJiI0iKCSchOpzoiFD79tCMWNAb08xEh4cRnRhGp1qsq6rkFnvJzCsmt9hLfrGX/BLnOa/ES15xGXnFXnKKysgtKiO3uIyD2cVsO5RHdmEpBaW+Gt87PDSE6IhQoj2hREeEERMRRnxkGC2iPNU+Kg4kzrTzrcK+TZwfFvTGBDARORG0dVFc5uNYQSlZ+aUcLSghK7+U7KIyCku8FJb5KCzxUlDqo6jUR16Jl9yiMg4cLyK7qIycorJT7kSqTqz/4BBX8U0i0kNcpNO8FB0eRpQnlOhw5xEVHkZMeCixkWEnmp0qpmPCwwixg0aNLOiNMTWK9ISeuB30XKkqBaW+k98WisrILXYOBjn+R16xl9ziMvKKy8gt8nI4t5gdGWXkF3spLPWdtdmpsihPKDERoc43nvBQYiLCTh4kPM6BouLAERV+6gEkutL8GP+3k0C6hmFBb4xpFCJyIizrcqAA8JUrRWU+Cku9FJX6KCjxUVBaqQmqxEtBiZe8Yi9FZT7yS7wnvmUUljrzM3JL/O/ho9j/Xmf5onEK5wDiHDTCQgVPSAhhoUJYiBAWGkJYiLOfMRUXuqt824irdPG74ttKTHgYkZ7Q83YQqXPQi0gn4BWcUaYUmKOqf66yzgRgEfCdf9bbqvqbun6mMSa4hIacPFg0FFWlxFvuD30n+J1nZ7qgxEdByakHkvwSZ5m3XPH6yvGVK2U+xVeulPrKOZJXTH6ms35esbfW30TCQ0OI9IQQ6XG+TbSJi2TB3aMbbF8r1Odfzwv8TFU3iEgcsF5Elqnq11XW+1RVL6/H5xhjTIMRESI9oUR6QmkZ3TifUeotP3GQyC0uO+UbSG6xl6JSL0Wl5RSVOd8yKh71GhrzDOoc9Kp6CDjkn84TkW1AB6Bq0BtjTFAJDwshMcy5LbUpaJAbYUWkKzAEWFPN4tEisklE3heR/md4j9kikiYiaZmZmQ1RljHGGBog6EUkFngL+Imq5lZZvAHooqqDgb8A79T0Pqo6R1VTVTU1ObkOfVoYY4ypVr2CXkQ8OCH/mqq+XXW5quaqar5/egngEZE69HNqjDGmruoc9OL0r/oSsE1Vn6xhnbb+9RCREf7PO1bXzzTGGHPu6nPXzRjgVuArEdnon/cLoDOAqj4HXAfcIyJeoAi4UZtiB/jGGBPA6nPXzWec1qnyaev8FfhrXT/DGGNM/Vn3c8YYE+As6I0xJsA1yTFjRSQT2FvHzVsBRxuwnObC9ju42H4Hl9rsdxdVrfbe9CYZ9PUhImk1DZAbyGy/g4vtd3Cp735b040xxgQ4C3pjjAlwgRj0c9wuwCW238HF9ju41Gu/A66N3hhjzKkC8YzeGGNMJRb0xhgT4AIm6EVkmoh8IyK7RORBt+tpTCIyV0QyRGRLpXmJIrJMRHb6nxPcrLGhiUgnEVkhIl+LyFYR+bF/fkDvN4CIRIrIWv+4DltF5Nf++d1EZI3/b36+iDSNUS4akIiEisiXIvKu/3XA7zOAiOwRka9EZKOIpPnn1flvPSCCXkRCgWeAS4F+wE0i0s/dqhrVy8C0KvMeBJarak9guf91IKkYurIfMAq41//fOND3G6AEuNg/rkMKME1ERgF/AP6kqhcAx4E7XKyxsfwY2FbpdTDsc4WJqppS6f75Ov+tB0TQAyOAXar6raqWAm8AV7pcU6NR1U+ArCqzrwT+4Z/+B3DVeS2qkanqIVXd4J/Ow/mfvwMBvt8A6sj3v/T4HwpcDLzpnx9w+y4iHYHLgBf9r4UA3+ezqPPfeqAEfQdgf6XX6f55waSNfxxfgMNAGzeLaUxVhq4Miv32N2FsBDKAZcBuIFtVvf5VAvFv/ing50C5/3USgb/PFRRYKiLrRWS2f16d/9br0x+9aaJUVUUkIO+brTp0pX9cGyCw91tVfUCKiLQEFgJ9XC6pUYnI5UCGqq4XkQlu1+OCsap6QERaA8tEZHvlhef6tx4oZ/QHgE6VXnf0zwsmR0SkHYD/OcPlehpcDUNXBvx+V6aq2cAKYDTQUkQqTtYC7W9+DDBDRPbgNMVeDPyZwN7nE1T1gP85A+fAPoJ6/K0HStCvA3r6r8iHAzcCi12u6XxbDNzmn74NWORiLQ3uDENXBvR+A4hIsv9MHhGJAibjXKNYgTOKGwTYvqvqQ6raUVW74vz//G9VvYUA3ucKIhIjInEV08AUYAv1+FsPmF/Gish0nDa9UGCuqv7O5ZIajYjMAybgdF16BHgMeAdYgDOU417gBlWtesG22RKRscCnwFecbLP9BU47fcDuN4CIDMK5+BaKc3K2QFV/IyLdcc52E4EvgZmqWuJepY3D33Tz36p6eTDss38fF/pfhgGvq+rvRCSJOv6tB0zQG2OMqV6gNN0YY4ypgQW9McYEOAt6Y4wJcBb0xhgT4CzojTEmwFnQG2NMgLOgN8aYAPf/A6tBmqARer9TAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Callbacks\n",
        "\n",
        "> callbacks: List of `keras.callbacks.Callback` instances.<br>\n",
        "        List of callbacks to apply during training.<br>\n",
        "        See `tf.keras.callbacks`. Note `tf.keras.callbacks.ProgbarLogger`\n",
        "        and `tf.keras.callbacks.History` callbacks are created automatically\n",
        "        and need not be passed into `model.fit`."
      ],
      "metadata": {
        "id": "pnCQ79WXe3sF"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Callback (обратные вызовы) – набор функций, применяемых в определенные моменты во время процедуры обучения. Вы можете использовать функции callback чтобы получить информацию о внутреннем состоянии модели в процессе обучения. Нужно передавать список callback’ов (именованным аргументом callbacks) методу .fit() модели. Подходящие методы callback будут вызваны на каждой стадии обучения."
      ],
      "metadata": {
        "id": "MfX3flrYsVjM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Рассмотрим основные callback'и, с остальными можете ознакомиться в [документации](https://ru-keras.com/callbacks/)."
      ],
      "metadata": {
        "id": "X810Ko7_hcEY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### ModelCheckpoint\n",
        "\n",
        "\n",
        "Сохранение модели после каждой эпохи.\n",
        "\n",
        "Filepath может содержать именованные опции форматирования, заполняемые значениями epoch и ключами в logs (передаваемыми on_epoch_end).\n",
        "\n",
        "К примеру: если filepath назван weights.{epoch:02d}-{val_loss:.2f}.hdf5, тогда модель будет сохраняться с номером эпохи и validation_loss в имени файла.\n",
        "\n",
        "**Аргументы:**\n",
        "\n",
        "— filepath: строка, путь сохранения модели\n",
        "\n",
        "— monitor: параметр для мониторинга\n",
        "\n",
        "— verbose: режим отображения, 0 или 1\n",
        "\n",
        "— save_best_only: если save_best_only=True, если результат текущей эпохи хуже предыдущей, он не будет сохранен.\n",
        "\n",
        "— save_weights_only: если True, тогда будут сохраняться только веса модели (model.save_weights(filepath)), в противном случае будет сохраняться вся модель (model.save(filepath)).\n",
        "\n",
        "— mode: один из {auto, min, max}. Если save_best_only=True, решение о перезаписи текущего файла будет приниматься в зависимости от уменьшения/увеличения параметра мониторинга. Для val_acc, необходим max, для val_loss необходим min. В auto режиме, mode выбирается в зависимости от имени monitor.\n",
        "\n",
        "— save_freq: `'epoch'` или integer. Если `'epoch'`, то сохраняется модель после каждой эпохиh. Когда integer, то сохранение модели через это кол-во батчей.\n"
      ],
      "metadata": {
        "id": "-GcPBpbEhfjk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from keras import callbacks\n",
        "\n",
        "model_checkpoint = callbacks.ModelCheckpoint(filepath='model_best_{epoch}.h5',\n",
        "                                             monitor='val_mae',\n",
        "                                             verbose=1,\n",
        "                                             save_best_only=True,\n",
        "                                             save_weights_only=False,\n",
        "                                             mode='auto',\n",
        "                                             save_freq='epoch'\n",
        "                                             ) "
      ],
      "metadata": {
        "id": "eUCR9tBYfIF3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.compile(optimizer='adam', loss='mse', metrics=['mae'])"
      ],
      "metadata": {
        "id": "GPfyGgufihsv"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.fit(X_train, y_train,\n",
        "          epochs=5, \n",
        "          validation_data=(X_test, y_test),\n",
        "          callbacks=[model_checkpoint])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "K_k-zgKghvz1",
        "outputId": "187151c5-19b6-4cf1-9de6-3c3c04ef7037"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            " 1/13 [=>............................] - ETA: 4s - loss: 10.0320 - mae: 2.2673\n",
            "Epoch 1: val_mae improved from inf to 3.47105, saving model to model_best_1.h5\n",
            "13/13 [==============================] - 1s 18ms/step - loss: 14.3296 - mae: 2.6998 - val_loss: 23.0815 - val_mae: 3.4710\n",
            "Epoch 2/5\n",
            " 1/13 [=>............................] - ETA: 0s - loss: 11.8018 - mae: 2.4359\n",
            "Epoch 2: val_mae improved from 3.47105 to 3.42411, saving model to model_best_2.h5\n",
            "13/13 [==============================] - 0s 7ms/step - loss: 13.8708 - mae: 2.6862 - val_loss: 22.9685 - val_mae: 3.4241\n",
            "Epoch 3/5\n",
            " 1/13 [=>............................] - ETA: 0s - loss: 12.5643 - mae: 2.7318\n",
            "Epoch 3: val_mae improved from 3.42411 to 3.37504, saving model to model_best_3.h5\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 13.4311 - mae: 2.6055 - val_loss: 22.6860 - val_mae: 3.3750\n",
            "Epoch 4/5\n",
            " 1/13 [=>............................] - ETA: 0s - loss: 6.7863 - mae: 2.0111\n",
            "Epoch 4: val_mae improved from 3.37504 to 3.35504, saving model to model_best_4.h5\n",
            "13/13 [==============================] - 0s 8ms/step - loss: 13.0470 - mae: 2.5661 - val_loss: 22.6854 - val_mae: 3.3550\n",
            "Epoch 5/5\n",
            " 1/13 [=>............................] - ETA: 0s - loss: 6.8940 - mae: 1.9689\n",
            "Epoch 5: val_mae improved from 3.35504 to 3.34110, saving model to model_best_5.h5\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 12.8665 - mae: 2.5670 - val_loss: 22.8322 - val_mae: 3.3411\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f5e1ad6fe10>"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### EarlyStopping\n",
        "\n",
        "\n",
        "Прекращение обучения, когда параметр monitor перестает улучшаться.\n",
        "\n",
        "**Аргументы**\n",
        "\n",
        "— monitor: параметр для мониторинга\n",
        "\n",
        "— min_delta: минимальное значение изменения величины monitor, расцениваемое как улучшение, то есть, если абсолютное изменение меньше min_delta, то улучшение не засчитывается\n",
        "\n",
        "— patience: число эпох, за которые величина monitor не улучшается, после которых обучение будет остановлено. Проверочные величины могут производиться не после каждой эпохи если validation_freq (model.fit(validation_freq=5)) больше единицы.\n",
        "\n",
        "— verbose: режим отображения, 0 или 1.\n",
        "\n",
        "— mode: один из {auto, min, max}. В режиме min, обучение остановится когда величина monitor перестанет уменьшаться; в режиме max, обучение остановится когда величина monitor перестанет увеличиваться; в режиме auto, mode выбирается в зависимости от имени monitor.\n",
        "\n",
        "— baseline: значение, которое должна достичь величина monitor. Обучение прекратится, если модель не достигла baseline.\n",
        "\n",
        "— restore_best_weights: восстанавливать ли веса модели с эпохи с лучшем значением параметра monitor. Если False, веса модели будут загружены из последней шага обучения."
      ],
      "metadata": {
        "id": "vMqUCfSWiwah"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "early_stop = callbacks.EarlyStopping(monitor='val_loss',\n",
        "                                     min_delta=0,\n",
        "                                     patience=2,\n",
        "                                     verbose=1,\n",
        "                                     mode='auto',\n",
        "                                    #  baseline=0.006,\n",
        "                                     restore_best_weights=True)\n",
        "\n",
        "\n",
        "model.fit(X_train, y_train,\n",
        "          epochs=10, \n",
        "          validation_data=(X_test, y_test),\n",
        "          callbacks=[early_stop])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4MN7i1yUi17a",
        "outputId": "a1536595-5663-4b21-9317-2ca658e12ab9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 11.4339 - mae: 2.4067 - val_loss: 22.8170 - val_mae: 3.2265\n",
            "Epoch 2/10\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 11.2441 - mae: 2.3857 - val_loss: 22.9058 - val_mae: 3.2160\n",
            "Epoch 3/10\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 11.0545 - mae: 2.3535 - val_loss: 22.4483 - val_mae: 3.1733\n",
            "Epoch 4/10\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 10.7794 - mae: 2.3241 - val_loss: 22.5810 - val_mae: 3.1742\n",
            "Epoch 5/10\n",
            " 1/13 [=>............................] - ETA: 0s - loss: 7.0549 - mae: 2.0822Restoring model weights from the end of the best epoch: 3.\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 10.6989 - mae: 2.3302 - val_loss: 22.8186 - val_mae: 3.1609\n",
            "Epoch 5: early stopping\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f5e14af8250>"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### ReduceLROnPlateau\n",
        "\n",
        "Уменьшение скорости обучения, когда метрика перестала улучшаться.\n",
        "\n",
        "Модели зачастую работают лучше если уменьшать скорость обучения. Этот callback следит за параметром monitor и, если не происходит улучшения за patience эпох, уменьшает скорость обучения.\n",
        "\n",
        "\n",
        "**Аргументы**\n",
        "\n",
        "— monitor:  параметр для мониторинга\n",
        "\n",
        "— factor: коэффициент уменьшения скорости обучения. new_lr = lr * factor\n",
        "\n",
        "— patience: число эпох, за которые величина monitor не улучшается, после которых обучение будет остановлено. Проверочные величины могут производиться не после каждой эпохи если validation_freq (model.fit(validation_freq=5)) больше единицы.\n",
        "\n",
        "— verbose: int. 0: «тихий» режим, 1: выводить сообщения\n",
        "\n",
        "— mode: один из {auto, min, max}. В режиме min, скорость обучения уменьшится когда величина monitor перестанет уменьшаться; в режиме max, скорость обучения уменьшится когда величина monitor перестанет увеличиваться; в режиме auto, mode выбирается в зависимости от имени monitor.\n",
        "\n",
        "— min_delta: минимальное значение изменения величины monitor, расцениваемое как улучшение, то есть, если абсолютное изменение меньше min_delta, то улучшение не засчитывается\n",
        "\n",
        "— cooldown: число эпох после уменьшения скорости обучения, которые должны пройти, прежде чем стандартный процесс уменьшения возобновится.\n",
        "\n",
        "— min_lr: нижняя граница скорости обучения"
      ],
      "metadata": {
        "id": "EwqAsmgFiztP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model.compile(optimizer='adam', loss='mse', metrics=['mae'])"
      ],
      "metadata": {
        "id": "tqbZuvWGjXWQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "reduce_lr = callbacks.ReduceLROnPlateau(monitor='val_mae',\n",
        "                                        factor=0.1,\n",
        "                                        patience=0,\n",
        "                                        verbose=1,\n",
        "                                        mode='auto',\n",
        "                                        min_delta=0,\n",
        "                                        cooldown=2,\n",
        "                                        min_lr=1e-10) \n",
        "\n",
        "model.fit(X_train, y_train,\n",
        "          epochs=10, \n",
        "          validation_data=(X_test, y_test),\n",
        "          callbacks=[reduce_lr])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BmWcW6_XhuS6",
        "outputId": "b310771b-e32b-47f1-eb21-0f336be04843"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "13/13 [==============================] - 1s 16ms/step - loss: 9.5661 - mae: 2.1952 - val_loss: 22.8341 - val_mae: 3.0329 - lr: 0.0010\n",
            "Epoch 2/10\n",
            " 1/13 [=>............................] - ETA: 0s - loss: 8.5618 - mae: 2.1496\n",
            "Epoch 2: ReduceLROnPlateau reducing learning rate to 0.00010000000474974513.\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 9.3436 - mae: 2.1768 - val_loss: 22.9169 - val_mae: 3.0523 - lr: 0.0010\n",
            "Epoch 3/10\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 9.1911 - mae: 2.1516 - val_loss: 22.8268 - val_mae: 3.0453 - lr: 1.0000e-04\n",
            "Epoch 4/10\n",
            " 1/13 [=>............................] - ETA: 0s - loss: 6.0728 - mae: 1.7557\n",
            "Epoch 4: ReduceLROnPlateau reducing learning rate to 1.0000000474974514e-05.\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 9.1711 - mae: 2.1488 - val_loss: 22.7865 - val_mae: 3.0400 - lr: 1.0000e-04\n",
            "Epoch 5/10\n",
            "13/13 [==============================] - 0s 4ms/step - loss: 9.1600 - mae: 2.1474 - val_loss: 22.7855 - val_mae: 3.0393 - lr: 1.0000e-05\n",
            "Epoch 6/10\n",
            " 1/13 [=>............................] - ETA: 0s - loss: 10.2622 - mae: 2.1920\n",
            "Epoch 6: ReduceLROnPlateau reducing learning rate to 1.0000000656873453e-06.\n",
            "13/13 [==============================] - 0s 7ms/step - loss: 9.1582 - mae: 2.1474 - val_loss: 22.7870 - val_mae: 3.0394 - lr: 1.0000e-05\n",
            "Epoch 7/10\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 9.1570 - mae: 2.1472 - val_loss: 22.7865 - val_mae: 3.0393 - lr: 1.0000e-06\n",
            "Epoch 8/10\n",
            " 1/13 [=>............................] - ETA: 0s - loss: 4.8981 - mae: 1.7305\n",
            "Epoch 8: ReduceLROnPlateau reducing learning rate to 1.0000001111620805e-07.\n",
            "13/13 [==============================] - 0s 7ms/step - loss: 9.1568 - mae: 2.1472 - val_loss: 22.7863 - val_mae: 3.0392 - lr: 1.0000e-06\n",
            "Epoch 9/10\n",
            "13/13 [==============================] - 0s 5ms/step - loss: 9.1567 - mae: 2.1472 - val_loss: 22.7862 - val_mae: 3.0392 - lr: 1.0000e-07\n",
            "Epoch 10/10\n",
            " 1/13 [=>............................] - ETA: 0s - loss: 9.5503 - mae: 2.0736\n",
            "Epoch 10: ReduceLROnPlateau reducing learning rate to 1.000000082740371e-08.\n",
            "13/13 [==============================] - 0s 6ms/step - loss: 9.1566 - mae: 2.1472 - val_loss: 22.7863 - val_mae: 3.0392 - lr: 1.0000e-07\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f5e14857b90>"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### Кастомный callback"
      ],
      "metadata": {
        "id": "N9xh6p7ojsz0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import keras\n",
        "\n",
        "class CustomCallback(keras.callbacks.Callback):\n",
        "    def on_train_begin(self, logs=None):\n",
        "        keys = list(logs.keys())\n",
        "        print(f\"Starting training; got log keys: {keys}\")\n",
        "\n",
        "    def on_train_end(self, logs=None):\n",
        "        keys = list(logs.keys())\n",
        "        print(f\"Stop training; got log keys: {keys}\")\n",
        "\n",
        "    def on_epoch_begin(self, epoch, logs=None):\n",
        "        keys = list(logs.keys())\n",
        "        print(f\"Start epoch {epoch} of training; got log keys: {keys}\")\n",
        "\n",
        "    def on_epoch_end(self, epoch, logs=None):\n",
        "        keys = list(logs.keys())\n",
        "        print(f\"End epoch {epoch} of training; got log keys: {keys}\")\n",
        "\n",
        "    def on_test_begin(self, logs=None):\n",
        "        keys = list(logs.keys())\n",
        "        print(f\"Start testing; got log keys: {keys}\")\n",
        "\n",
        "    def on_test_end(self, logs=None):\n",
        "        keys = list(logs.keys())\n",
        "        print(f\"Stop testing; got log keys: {keys}\")\n",
        "\n",
        "    def on_train_batch_begin(self, batch, logs=None):\n",
        "        keys = list(logs.keys())\n",
        "        print(f\"...Training: start of batch {batch}; got log keys: {keys}\")\n",
        "\n",
        "    def on_train_batch_end(self, batch, logs=None):\n",
        "        keys = list(logs.keys())\n",
        "        print(f\"...Training: end of batch {batch}; got log keys: {keys}\")\n",
        "\n",
        "    def on_test_batch_begin(self, batch, logs=None):\n",
        "        keys = list(logs.keys())\n",
        "        print(f\"...Evaluating: start of batch {batch}; got log keys: {keys}\")\n",
        "\n",
        "    def on_test_batch_end(self, batch, logs=None):\n",
        "        keys = list(logs.keys())\n",
        "        print(f\"...Evaluating: end of batch {batch}; got log keys: {keys}\")"
      ],
      "metadata": {
        "id": "Uwk2zI0Yjuml"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.fit(X_train, y_train,\n",
        "          epochs=1, \n",
        "          validation_data=(X_test, y_test),\n",
        "          callbacks=[CustomCallback()],\n",
        "          verbose=0);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mJ3by3ggj0pM",
        "outputId": "8c5d93c9-ca2e-4dc2-b1ec-a3dadbcaf943"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Starting training; got log keys: []\n",
            "Start epoch 0 of training; got log keys: []\n",
            "...Training: start of batch 0; got log keys: []\n",
            "...Training: end of batch 0; got log keys: ['loss', 'mae']\n",
            "...Training: start of batch 1; got log keys: []\n",
            "...Training: end of batch 1; got log keys: ['loss', 'mae']\n",
            "...Training: start of batch 2; got log keys: []\n",
            "...Training: end of batch 2; got log keys: ['loss', 'mae']\n",
            "...Training: start of batch 3; got log keys: []\n",
            "...Training: end of batch 3; got log keys: ['loss', 'mae']\n",
            "...Training: start of batch 4; got log keys: []\n",
            "...Training: end of batch 4; got log keys: ['loss', 'mae']\n",
            "...Training: start of batch 5; got log keys: []\n",
            "...Training: end of batch 5; got log keys: ['loss', 'mae']\n",
            "...Training: start of batch 6; got log keys: []\n",
            "...Training: end of batch 6; got log keys: ['loss', 'mae']\n",
            "...Training: start of batch 7; got log keys: []\n",
            "...Training: end of batch 7; got log keys: ['loss', 'mae']\n",
            "...Training: start of batch 8; got log keys: []\n",
            "...Training: end of batch 8; got log keys: ['loss', 'mae']\n",
            "...Training: start of batch 9; got log keys: []\n",
            "...Training: end of batch 9; got log keys: ['loss', 'mae']\n",
            "...Training: start of batch 10; got log keys: []\n",
            "...Training: end of batch 10; got log keys: ['loss', 'mae']\n",
            "...Training: start of batch 11; got log keys: []\n",
            "...Training: end of batch 11; got log keys: ['loss', 'mae']\n",
            "...Training: start of batch 12; got log keys: []\n",
            "...Training: end of batch 12; got log keys: ['loss', 'mae']\n",
            "Start testing; got log keys: []\n",
            "...Evaluating: start of batch 0; got log keys: []\n",
            "...Evaluating: end of batch 0; got log keys: ['loss', 'mae']\n",
            "...Evaluating: start of batch 1; got log keys: []\n",
            "...Evaluating: end of batch 1; got log keys: ['loss', 'mae']\n",
            "...Evaluating: start of batch 2; got log keys: []\n",
            "...Evaluating: end of batch 2; got log keys: ['loss', 'mae']\n",
            "...Evaluating: start of batch 3; got log keys: []\n",
            "...Evaluating: end of batch 3; got log keys: ['loss', 'mae']\n",
            "Stop testing; got log keys: ['loss', 'mae']\n",
            "End epoch 0 of training; got log keys: ['loss', 'mae', 'val_loss', 'val_mae']\n",
            "Stop training; got log keys: ['loss', 'mae', 'val_loss', 'val_mae']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from IPython.display import clear_output\n",
        "import time\n",
        "\n",
        "\n",
        "class LossAndErrorPrintingCallback(keras.callbacks.Callback):\n",
        "    def on_train_batch_end(self, batch, logs=None):\n",
        "        print(\n",
        "            f'...Train: batch {batch}, the average loss is {logs[\"loss\"]:5.2f}'\n",
        "        )\n",
        "\n",
        "    def on_test_batch_end(self, batch, logs=None):\n",
        "        print(\n",
        "            f'...Test: batch {batch}, the average loss is {logs[\"loss\"]:7.2f}'\n",
        "        )\n",
        "\n",
        "    def on_epoch_begin(self, epoch, logs=None):\n",
        "        print(\n",
        "            f\"Start epoch {epoch}; You're doing great, honey!\\n\"\n",
        "        )\n",
        "    \n",
        "    def on_epoch_end(self, epoch, logs=None):\n",
        "        print(\n",
        "            f'\\nThe average loss for epoch {epoch} is {logs[\"loss\"]:7.2f} '\n",
        "            f'and mean absolute error is {logs[\"mae\"]:7.2f}.'\n",
        "        )"
      ],
      "metadata": {
        "id": "UQ_rAI4wo9Kr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.fit(X_train, y_train,\n",
        "          epochs=1, \n",
        "          validation_data=(X_test, y_test),\n",
        "          callbacks=[LossAndErrorPrintingCallback()],\n",
        "          verbose=0);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oBpGWEINo_lG",
        "outputId": "aa0f0ff2-7a9f-4a2f-98a6-c7a2aac2919a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Start epoch 0; You're doing great, honey!\n",
            "\n",
            "...Train: batch 0, the average loss is  7.40\n",
            "...Train: batch 1, the average loss is 16.97\n",
            "...Train: batch 2, the average loss is 13.57\n",
            "...Train: batch 3, the average loss is 11.92\n",
            "...Train: batch 4, the average loss is 11.17\n",
            "...Train: batch 5, the average loss is 10.25\n",
            "...Train: batch 6, the average loss is 10.07\n",
            "...Train: batch 7, the average loss is  9.91\n",
            "...Train: batch 8, the average loss is  9.37\n",
            "...Train: batch 9, the average loss is  9.46\n",
            "...Train: batch 10, the average loss is  9.29\n",
            "...Train: batch 11, the average loss is  9.27\n",
            "...Train: batch 12, the average loss is  9.16\n",
            "...Test: batch 0, the average loss is   12.33\n",
            "...Test: batch 1, the average loss is   13.73\n",
            "...Test: batch 2, the average loss is   14.28\n",
            "...Test: batch 3, the average loss is   22.79\n",
            "\n",
            "The average loss for epoch 0 is    9.16 and mean absolute error is    2.15.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#####**self.model.stop_training = True**\n",
        "\n",
        "прекращает обучение модели"
      ],
      "metadata": {
        "id": "2lf4HsfxnSp4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class LossAndErrorPrintingCallback(keras.callbacks.Callback):\n",
        "    def on_train_batch_end(self, batch, logs=None):\n",
        "        print(\n",
        "            f'...Train: batch {batch}, the average loss is {logs[\"loss\"]:5.2f}'\n",
        "        )\n",
        "\n",
        "        if batch == 4:\n",
        "            self.model.stop_training = True\n",
        "            print('Model is tired. Stop it')\n",
        "\n",
        "    def on_test_batch_end(self, batch, logs=None):\n",
        "        print(\n",
        "            f'...Test: batch {batch}, the average loss is {logs[\"loss\"]:7.2f}'\n",
        "        )\n",
        "\n",
        "    def on_epoch_begin(self, epoch, logs=None):\n",
        "        print(\n",
        "            f\"Start epoch {epoch}; You're doing great, honey!\\n\"\n",
        "        )\n",
        "    \n",
        "    def on_epoch_end(self, epoch, logs=None):\n",
        "        print(\n",
        "            f'\\nThe average loss for epoch {epoch} is {logs[\"loss\"]:7.2f} '\n",
        "            f'and mean absolute error is {logs[\"mae\"]:7.2f}.'\n",
        "        )"
      ],
      "metadata": {
        "id": "LWAgNP8QlAzU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.fit(X_train, y_train,\n",
        "          epochs=2, \n",
        "          validation_data=(X_test, y_test),\n",
        "          callbacks=[LossAndErrorPrintingCallback()],\n",
        "          verbose=0);"
      ],
      "metadata": {
        "id": "pSvusXX7lCV6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b1fff346-7e2b-4820-aec7-8db9ffe1fb06"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Start epoch 0; You're doing great, honey!\n",
            "\n",
            "...Train: batch 0, the average loss is  7.40\n",
            "...Train: batch 1, the average loss is 16.97\n",
            "...Train: batch 2, the average loss is 13.57\n",
            "...Train: batch 3, the average loss is 11.92\n",
            "...Train: batch 4, the average loss is 11.17\n",
            "Model is tired. Stop it\n",
            "...Test: batch 0, the average loss is   12.33\n",
            "...Test: batch 1, the average loss is   13.73\n",
            "...Test: batch 2, the average loss is   14.28\n",
            "...Test: batch 3, the average loss is   22.79\n",
            "\n",
            "The average loss for epoch 0 is   11.17 and mean absolute error is    2.20.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##### **self.model.predict()**\n",
        "\n",
        "Получать предсказания модели\n"
      ],
      "metadata": {
        "id": "eaEXWbkjpNFY"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "class LossAndErrorPrintingCallback(keras.callbacks.Callback):\n",
        "    def on_train_batch_end(self, batch, logs=None):\n",
        "        print(\n",
        "            f'...Train: batch {batch}, the average loss is {logs[\"loss\"]:5.2f}'\n",
        "        )\n",
        "\n",
        "    def on_test_batch_end(self, batch, logs=None):\n",
        "        print(\n",
        "            f'...Test: batch {batch}, the average loss is {logs[\"loss\"]:7.2f}'\n",
        "        )\n",
        "\n",
        "    def on_epoch_begin(self, epoch, logs=None):\n",
        "\n",
        "        pred = self.model.predict(X_test[:1])\n",
        "\n",
        "        print(f'Model prediction is {pred[0][0]}. True is {y_test[0]}')\n",
        "        print(\n",
        "            f\"Start epoch {epoch}; You're doing great, honey!\\n\"\n",
        "        )\n",
        "    \n",
        "    def on_epoch_end(self, epoch, logs=None):\n",
        "        print(\n",
        "            f'\\nThe average loss for epoch {epoch} is {logs[\"loss\"]:7.2f} '\n",
        "            f'and mean absolute error is {logs[\"mae\"]:7.2f}.'\n",
        "        )"
      ],
      "metadata": {
        "id": "hxqFKeOFoaW0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.fit(X_train, y_train,\n",
        "          epochs=2, \n",
        "          validation_data=(X_test, y_test),\n",
        "          callbacks=[LossAndErrorPrintingCallback()],\n",
        "          verbose=0);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kFC5kDq2od9_",
        "outputId": "4c194da2-6738-45fc-f95f-246133fa1a9d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model prediction is 7.978826522827148. True is 7.2\n",
            "Start epoch 0; You're doing great, honey!\n",
            "\n",
            "...Train: batch 0, the average loss is  7.40\n",
            "...Train: batch 1, the average loss is 16.97\n",
            "...Train: batch 2, the average loss is 13.57\n",
            "...Train: batch 3, the average loss is 11.92\n",
            "...Train: batch 4, the average loss is 11.17\n",
            "...Train: batch 5, the average loss is 10.25\n",
            "...Train: batch 6, the average loss is 10.07\n",
            "...Train: batch 7, the average loss is  9.91\n",
            "...Train: batch 8, the average loss is  9.37\n",
            "...Train: batch 9, the average loss is  9.46\n",
            "...Train: batch 10, the average loss is  9.29\n",
            "...Train: batch 11, the average loss is  9.27\n",
            "...Train: batch 12, the average loss is  9.16\n",
            "...Test: batch 0, the average loss is   12.33\n",
            "...Test: batch 1, the average loss is   13.73\n",
            "...Test: batch 2, the average loss is   14.28\n",
            "...Test: batch 3, the average loss is   22.79\n",
            "\n",
            "The average loss for epoch 0 is    9.16 and mean absolute error is    2.15.\n",
            "Model prediction is 7.978828430175781. True is 7.2\n",
            "Start epoch 1; You're doing great, honey!\n",
            "\n",
            "...Train: batch 0, the average loss is  8.51\n",
            "...Train: batch 1, the average loss is  9.64\n",
            "...Train: batch 2, the average loss is 10.08\n",
            "...Train: batch 3, the average loss is  9.08\n",
            "...Train: batch 4, the average loss is  8.25\n",
            "...Train: batch 5, the average loss is  9.51\n",
            "...Train: batch 6, the average loss is 11.30\n",
            "...Train: batch 7, the average loss is 10.57\n",
            "...Train: batch 8, the average loss is  9.77\n",
            "...Train: batch 9, the average loss is  9.54\n",
            "...Train: batch 10, the average loss is  9.02\n",
            "...Train: batch 11, the average loss is  9.33\n",
            "...Train: batch 12, the average loss is  9.16\n",
            "...Test: batch 0, the average loss is   12.33\n",
            "...Test: batch 1, the average loss is   13.73\n",
            "...Test: batch 2, the average loss is   14.28\n",
            "...Test: batch 3, the average loss is   22.79\n",
            "\n",
            "The average loss for epoch 1 is    9.16 and mean absolute error is    2.15.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##### Делать визуализации и чистить экран"
      ],
      "metadata": {
        "id": "mjz-HnompPGN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from IPython.display import clear_output\n",
        "import time\n",
        "\n",
        "\n",
        "class LossAndErrorPrintingCallback(keras.callbacks.Callback):\n",
        "    def on_train_batch_end(self, batch, logs=None):\n",
        "        print(\n",
        "            f'...Train: batch {batch}, the average loss is {logs[\"loss\"]:5.2f}'\n",
        "        )\n",
        "        time.sleep(0.5)\n",
        "\n",
        "    def on_test_batch_end(self, batch, logs=None):\n",
        "        print(\n",
        "            f'...Test: batch {batch}, the average loss is {logs[\"loss\"]:7.2f}'\n",
        "        )\n",
        "\n",
        "    def on_epoch_begin(self, epoch, logs=None):\n",
        "        print(\n",
        "            f\"Start epoch {epoch}; You're doing great, honey!\\n\"\n",
        "        )\n",
        "    \n",
        "    def on_epoch_end(self, epoch, logs=None):\n",
        "        print(\n",
        "            f'\\nThe average loss for epoch {epoch} is {logs[\"loss\"]:7.2f} '\n",
        "            f'and mean absolute error is {logs[\"mae\"]:7.2f}.'\n",
        "        )\n",
        "        print('\\nIn 2 seconds screen will be clear')\n",
        "        time.sleep(2)\n",
        "        clear_output(wait=True)"
      ],
      "metadata": {
        "id": "O7yHAp20ofMx"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.fit(X_train, y_train,\n",
        "          epochs=2, \n",
        "          validation_data=(X_test, y_test),\n",
        "          callbacks=[LossAndErrorPrintingCallback()],\n",
        "          verbose=0);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-CnapMSLokFf",
        "outputId": "2f702eb2-e1f1-4d1b-8f99-ce71b0516d85"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Start epoch 1; You're doing great, honey!\n",
            "\n",
            "...Train: batch 0, the average loss is  8.51\n",
            "...Train: batch 1, the average loss is  9.64\n",
            "...Train: batch 2, the average loss is 10.08\n",
            "...Train: batch 3, the average loss is  9.08\n",
            "...Train: batch 4, the average loss is  8.25\n",
            "...Train: batch 5, the average loss is  9.51\n",
            "...Train: batch 6, the average loss is 11.30\n",
            "...Train: batch 7, the average loss is 10.57\n",
            "...Train: batch 8, the average loss is  9.77\n",
            "...Train: batch 9, the average loss is  9.54\n",
            "...Train: batch 10, the average loss is  9.02\n",
            "...Train: batch 11, the average loss is  9.33\n",
            "...Train: batch 12, the average loss is  9.16\n",
            "...Test: batch 0, the average loss is   12.33\n",
            "...Test: batch 1, the average loss is   13.73\n",
            "...Test: batch 2, the average loss is   14.28\n",
            "...Test: batch 3, the average loss is   22.79\n",
            "\n",
            "The average loss for epoch 1 is    9.16 and mean absolute error is    2.15.\n",
            "\n",
            "In 2 seconds screen will be clear\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Можно изменять параметры оптимизатора **self.model.optimizer.learning_rate**\n"
      ],
      "metadata": {
        "id": "0yyAH6xWpf7H"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Summary\n",
        "\n",
        "\n",
        "Вот мы и разобрались, что такое callback и как им пользоваться при обучении нейронной сети через keras."
      ],
      "metadata": {
        "id": "cKL1g6ILJiNk"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Callback (обратные вызовы)** – набор функций, применяемых в определенные моменты во время процедуры обучения."
      ],
      "metadata": {
        "id": "bX9poU4vs0f8"
      }
    }
  ]
}