4.4.模型读取和存储.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "4.4. 模型读取和存储.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/chongzicbo/Dive-into-Deep-Learning-tf.keras/blob/master/4.4.%E6%A8%A1%E5%9E%8B%E8%AF%BB%E5%8F%96%E5%92%8C%E5%AD%98%E5%82%A8.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PvUzDSeFeIE0",
        "colab_type": "text"
      },
      "source": [
        "##4.4. 模型读取和存储\n",
        "&emsp;&emsp;到目前为止，我们介绍了如何处理数据以及如何构建、训练和测试深度学习模型。然而在实际中，我们有时需要把训练好的模型部署到很多不同的设备。在这种情况下，我们可以把内存中训练好的模型参数存储在硬盘上供后续读取使用。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FlVU6tS_dYb0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 62
        },
        "outputId": "5268a883-6690-4591-87b2-3edad1c8bbf6"
      },
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers,Sequential"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<p style=\"color: red;\">\n",
              "The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.<br>\n",
              "We recommend you <a href=\"https://www.tensorflow.org/guide/migrate\" target=\"_blank\">upgrade</a> now \n",
              "or ensure your notebook will continue to use TensorFlow 1.x via the <code>%tensorflow_version 1.x</code> magic:\n",
              "<a href=\"https://colab.research.google.com/notebooks/tensorflow_version.ipynb\" target=\"_blank\">more info</a>.</p>\n"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kB3z8XYBd3v0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "tf.enable_eager_execution()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dw1SJalJd6Fa",
        "colab_type": "text"
      },
      "source": [
        "### 4.4.1. 模型的存储\n",
        "&emsp;&emsp;我们先来定义一个模型"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WmMX9vnyuCJj",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 289
        },
        "outputId": "f9773cf3-dbe3-4ab5-efc0-d184f9816133"
      },
      "source": [
        "x=tf.random.normal(shape=(100,100))\n",
        "y=tf.random.normal(shape=(100,5))\n",
        "model=Sequential([layers.Dense(10,input_dim=100,activation='relu'),layers.Dense(5)])\n",
        "model.compile(optimizer='adam',loss=keras.losses.categorical_crossentropy,metrics=['accuracy'])\n",
        "model.fit(x=x,y=y,batch_size=10,epochs=5)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 100 samples\n",
            "Epoch 1/5\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/math_grad.py:1394: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
            "100/100 [==============================] - 3s 33ms/sample - loss: 4.5191 - acc: 0.2200\n",
            "Epoch 2/5\n",
            "100/100 [==============================] - 0s 250us/sample - loss: 2.2676 - acc: 0.2200\n",
            "Epoch 3/5\n",
            "100/100 [==============================] - 0s 251us/sample - loss: 1.0065 - acc: 0.2100\n",
            "Epoch 4/5\n",
            "100/100 [==============================] - 0s 298us/sample - loss: 0.9531 - acc: 0.2100\n",
            "Epoch 5/5\n",
            "100/100 [==============================] - 0s 466us/sample - loss: 1.0109 - acc: 0.2200\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7f8f319df160>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Hqp-RQZ51wFM",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 218
        },
        "outputId": "1674c49f-2bd4-46ef-c219-7e9c14eb80d5"
      },
      "source": [
        "model.summary()"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense (Dense)                (None, 10)                1010      \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 5)                 55        \n",
            "=================================================================\n",
            "Total params: 1,065\n",
            "Trainable params: 1,065\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wyMWczDnuo_Q",
        "colab_type": "text"
      },
      "source": [
        "你可以使用model.save(filepath)将Keras模型和权重保存在一个HDF5文件中，该文件将包含：\n",
        "* 模型的结构，以便重构该模型\n",
        "* 模型的权重\n",
        "* 训练配置（损失函数，优化器等）\n",
        "* 优化器的状态，以便于从上次训练中断的地方开始"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iE2z9D-nuczD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model.save(filepath='my_model.h5')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HDcT2rhEuh4N",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "436ace41-40dd-40c6-a9ea-1ad72384a909"
      },
      "source": [
        "!ls ./"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "my_model.h5  sample_data\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yodY3aGGuka1",
        "colab_type": "text"
      },
      "source": [
        "&emsp;&emsp;如果你只是希望保存模型的结构，而不包含其权重或配置信息，可以使用："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "39av7CyAvRAT",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "json_string=model.to_json()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wYa1ZS2wvT_B",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "yaml_string=model.to_yaml()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ucz7vlw-vYB1",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        },
        "outputId": "dfd29c32-f5dd-4652-c62e-c41ad20f8e26"
      },
      "source": [
        "json_string"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'{\"class_name\": \"Sequential\", \"config\": {\"name\": \"sequential\", \"layers\": [{\"class_name\": \"Dense\", \"config\": {\"name\": \"dense\", \"trainable\": true, \"batch_input_shape\": [null, 100], \"dtype\": \"float32\", \"units\": 10, \"activation\": \"relu\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"GlorotUniform\", \"config\": {\"seed\": null, \"dtype\": \"float32\"}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {\"dtype\": \"float32\"}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}}, {\"class_name\": \"Dense\", \"config\": {\"name\": \"dense_1\", \"trainable\": true, \"dtype\": \"float32\", \"units\": 5, \"activation\": \"linear\", \"use_bias\": true, \"kernel_initializer\": {\"class_name\": \"GlorotUniform\", \"config\": {\"seed\": null, \"dtype\": \"float32\"}}, \"bias_initializer\": {\"class_name\": \"Zeros\", \"config\": {\"dtype\": \"float32\"}}, \"kernel_regularizer\": null, \"bias_regularizer\": null, \"activity_regularizer\": null, \"kernel_constraint\": null, \"bias_constraint\": null}}]}, \"keras_version\": \"2.2.4-tf\", \"backend\": \"tensorflow\"}'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VExXT8JUvZuj",
        "colab_type": "text"
      },
      "source": [
        "&emsp;&emsp;这项操作将把模型序列化为json或yaml文件，这些文件对人而言也是友好的，如果需要的话你甚至可以手动打开这些文件并进行编辑。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zxBf79UXvq7W",
        "colab_type": "text"
      },
      "source": [
        "&emsp;&emsp;如果需要保存模型的权重，可通过下面的代码利用HDF5进行保存。注意，在使用前需要确保你已安装了HDF5和其Python库h5py"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g_eZY3C_vxYI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model.save_weights('my_model_weights.h5')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DdMtemJuv1oV",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "98359b98-600e-4c67-a40a-4e029e4c12d9"
      },
      "source": [
        "!ls ./"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "my_model.h5  my_model_weights.h5  sample_data\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mwUGN1esxDU0",
        "colab_type": "text"
      },
      "source": [
        "### 4.4.2. 模型的读取\n",
        "&emsp;&emsp;如果之前使用了model.save()保存了完整的模型结构和权重参数，那么可以使用keras.models.load_model(filepath)来重新实例化你的模型，如果文件中存储了训练配置的话，该函数还会同时完成模型的编译\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CB4uAA_PyEDf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.keras.models import load_model\n",
        "model=load_model('my_model.h5')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GbIgFMsmyOg3",
        "colab_type": "text"
      },
      "source": [
        "&emsp;&emsp;你也可以根据之前保存的json或者yaml文件来载入模型结构"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ps4TLzbWzLKn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.keras.models import model_from_json,model_from_yaml\n",
        "model=model_from_json(json_string)\n",
        "model_yaml=model_from_yaml(yaml_string)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a0RA9ZH3zZ_v",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 50
        },
        "outputId": "00fbf81e-35c4-4a4f-fd23-5a88b3eed5d7"
      },
      "source": [
        "model.layers"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tensorflow.python.keras.layers.core.Dense at 0x7f8f209c2a90>,\n",
              " <tensorflow.python.keras.layers.core.Dense at 0x7f8f31a60320>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KXPiFV7HzcPN",
        "colab_type": "text"
      },
      "source": [
        "&emsp;&emsp;然后再加载之前保存的模型权重"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mILfemCfz8e8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model_yaml.load_weights('my_model_weights.h5')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mMzxjiqB0gOI",
        "colab_type": "text"
      },
      "source": [
        "&emsp;&emsp;打印模型结构看看"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9gLO54p40EnJ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 218
        },
        "outputId": "982d0f65-c1bb-40c7-a117-25644ca2da40"
      },
      "source": [
        "model_yaml.summary()"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense (Dense)                (None, 10)                1010      \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 5)                 55        \n",
            "=================================================================\n",
            "Total params: 1,065\n",
            "Trainable params: 1,065\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v2RWijXt2mZe",
        "colab_type": "text"
      },
      "source": [
        "&emsp;&emsp;如果你需要加载权重到不同的网络结构（有些层一样）中，例如fine-tune或transfer-learning，你可以通过层名字来加载模型："
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iwpG-Han0e16",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model=Sequential()\n",
        "model.add(layers.Dense(10,input_dim=100,name='dense'))\n",
        "model.add(layers.Dense(3,name='new_dense'))\n",
        "model.load_weights(filepath='my_model_weights.h5',by_name=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9SomJqz91WsK",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 218
        },
        "outputId": "94a11dc9-a001-4926-d7e7-7d90ce491ccd"
      },
      "source": [
        "model.summary()"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_3\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense (Dense)                (None, 10)                1010      \n",
            "_________________________________________________________________\n",
            "new_dense (Dense)            (None, 3)                 33        \n",
            "=================================================================\n",
            "Total params: 1,043\n",
            "Trainable params: 1,043\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ASWQ8d8c2PfM",
        "colab_type": "text"
      },
      "source": [
        "###4.4.3. 小结\n",
        "* model.save()可以保存完整的模型：包括模型的结构和参数以及优化器等；model.to_json()和model.to_yaml()可以保存模型的网络结构。model.save_weights()可以保存模型的参数。\n",
        "\n",
        "* load_model()方法可以加载save()方法保存的完整模型；model_from_json(),model_from_yaml()方法可以恢复模型的网络结构，然后可以根据load_weights()方法加载模型。"
      ]
    }
  ]
}