5.12. 稠密连接网络(DenseNet).ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "5.12. 稠密连接网络(DenseNet).ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/chongzicbo/Dive-into-Deep-Learning-tf.keras/blob/master/5.12.%20%E7%A8%A0%E5%AF%86%E8%BF%9E%E6%8E%A5%E7%BD%91%E7%BB%9C(DenseNet).ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MEtloEJUs-uX",
        "colab_type": "text"
      },
      "source": [
        "##5.12. 稠密连接网络（DenseNet）\n",
        "ResNet中的跨层连接设计引申出了数个后续工作。本节我们介绍其中的一个：稠密连接网络（DenseNet） [1]。 它与ResNet的主要区别如图5.10所示。\n",
        "<img src=\"https://zh.gluon.ai/_images/densenet.svg\" width=\"500\"/>\n",
        "\n",
        "<center>图 5.10 ResNet（左）与DenseNet（右）在跨层连接上的主要区别：使用相加和使用连结</center>\n",
        "\n",
        "图5.10中将部分前后相邻的运算抽象为模块 $A$ 和模块 $B$ 。与ResNet的主要区别在于，DenseNet里模块 $B$ 的输出不是像ResNet那样和模块 $A$ 的输出相加，而是在通道维上连结。这样模块 $A$ 的输出可以直接传入模块 $B$ 后面的层。在这个设计里，模块 $A$ 直接跟模块 $B$ 后面的所有层连接在了一起。这也是它被称为“稠密连接”的原因。\n",
        "\n",
        "DenseNet的主要构建模块是稠密块（dense block）和过渡层（transition layer）。前者定义了输入和输出是如何连结的，后者则用来控制通道数，使之不过大。\n",
        "\n",
        "###5.12.1. 稠密块\n",
        "DenseNet使用了ResNet改良版的“批量归一化、激活和卷积”结构（参见上一节的练习），我们首先在conv_block函数里实现这个结构。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xRLweL6LpoiG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers,Sequential\n",
        "from tensorflow import data\n",
        "from tensorflow.keras import losses,optimizers\n",
        "import time\n",
        "import numpy as np\n",
        "from tensorflow import image\n",
        "import os\n",
        "import sys"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9hha8XQdeFCg",
        "colab_type": "code",
        "outputId": "6f891567-6dea-4a29-8378-1b0bfa0894ad",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# !pip install tensorflow==2.0.0\n",
        "tf.__version__"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'2.0.0'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hFWnF8MBtfN5",
        "colab_type": "text"
      },
      "source": [
        "稠密块由多个conv_block组成，每块使用相同的输出通道数。但在前向计算时，我们将每块的输入和输出在通道维上连结。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FQSzJUNrtZyU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def conv_block(num_channels):\n",
        "  blk=Sequential([\n",
        "      layers.BatchNormalization(),\n",
        "      layers.Activation('relu'),\n",
        "      layers.Convolution2D(num_channels,kernel_size=3,padding='same')\n",
        "  ])\n",
        "  return blk"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yLTSKfkZtOac",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# tf.enable_eager_execution() #启用动态图计算，tf2.x不用调用该函数"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "04DHrJUYt3u3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class DenseBlock(keras.Model):\n",
        "  def __init__(self,num_convs,num_channels,**kwargs):\n",
        "    super(DenseBlock,self).__init__()\n",
        "    self.num_convs=num_convs #卷积模块数量\n",
        "    self.conv_blocks=[] #存放所有的卷积模块\n",
        "    for _ in range(self.num_convs):\n",
        "      self.conv_blocks.append(conv_block(num_channels))\n",
        "\n",
        "  def call(self,X):\n",
        "    #遍历所有的卷积模块并传入上层输出作为输入\n",
        "    for i in range(int(self.num_convs)):\n",
        "      output=self.conv_blocks[i](X)\n",
        "      X=tf.concat([X,output],axis=-1)#在通道维上将输入和输出连接\n",
        "    return X"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ixWbowU1tnBn",
        "colab_type": "text"
      },
      "source": [
        "在下面的例子中，我们定义一个有2个输出通道数为10的卷积块。使用通道数为3的输入时，我们会得到通道数为 $3 + 2 \\times 10 =23$ 的输出。卷积块的通道数控制了输出通道数相对于输入通道数的增长，因此也被称为增长率（growth rate）。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n1N262qf0E31",
        "colab_type": "code",
        "outputId": "486680ca-c088-4f6d-a5d6-035fa3c69329",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "blk=DenseBlock(2,10)\n",
        "X=tf.random.uniform(shape=(4,8,8,3))\n",
        "Y=blk(X)\n",
        "Y.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([4, 8, 8, 23])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xj-MgN8htwf2",
        "colab_type": "text"
      },
      "source": [
        "###5.12.2. 过渡层\n",
        "由于每个稠密块都会带来通道数的增加，使用过多则会带来过于复杂的模型。过渡层用来控制模型复杂度。它通过$ 1\\times1$ 卷积层来减小通道数，并使用步幅为2的平均池化层减半高和宽，从而进一步降低模型复杂度。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2ZrLb5HG1Egb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def transition_block(num_channels):\n",
        "  blk=Sequential([\n",
        "    layers.BatchNormalization(),\n",
        "    layers.Activation('relu'),\n",
        "    layers.Convolution2D(num_channels,kernel_size=1),\n",
        "    layers.AveragePooling2D(pool_size=2,strides=2)                  \n",
        "  ])\n",
        "  return blk"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yBZtapG5t3Ze",
        "colab_type": "text"
      },
      "source": [
        "对上一个例子中稠密块的输出使用通道数为10的过渡层。此时输出的通道数减为10，高和宽均减半。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "y8eo9Br-1pjR",
        "colab_type": "code",
        "outputId": "4a1082c5-3dce-4253-a09e-88eff916fb6b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "blk=transition_block(10)\n",
        "blk(Y).shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([4, 4, 4, 10])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7R7AtZGYt87u",
        "colab_type": "text"
      },
      "source": [
        "###5.12.3. DenseNet模型\n",
        "我们来构造DenseNet模型。DenseNet首先使用同ResNet一样的单卷积层和最大池化层。\\\n",
        "类似于ResNet接下来使用的4个残差块，DenseNet使用的是4个稠密块。同ResNet一样，我们可以设置每个稠密块使用多少个卷积层。这里我们设成4，从而与上一节的ResNet-18保持一致。稠密块里的卷积层通道数（即增长率）设为32，所以每个稠密块将增加128个通道。\n",
        "\n",
        "ResNet里通过步幅为2的残差块在每个模块之间减小高和宽。这里我们则使用过渡层来减半高和宽，并减半通道数。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gdwPWsH_1wxa",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "net=Sequential(\n",
        "    [\n",
        "     layers.Convolution2D(64,kernel_size=7,strides=2,padding='same',input_shape=(224,224,1)),\n",
        "     layers.BatchNormalization(),\n",
        "     layers.Activation('relu'),\n",
        "     layers.MaxPool2D(pool_size=3,strides=2,padding='same')\n",
        "    ]\n",
        ")\n",
        "num_channels,growth_rate=64,32 #num_channels为当前的通道数\n",
        "num_convs_in_dense_blocks=[4,4,4,4]\n",
        "for i,num_convs in enumerate(num_convs_in_dense_blocks):\n",
        "  net.add(DenseBlock(num_convs,growth_rate))\n",
        "  #上一个稠密块的输出通道数\n",
        "  num_channels+=num_convs*growth_rate\n",
        "  #在稠密块之间加入通道数减半的过渡层\n",
        "  if i!=len(num_convs_in_dense_blocks)-1:\n",
        "    num_channels //2\n",
        "    net.add(transition_block(num_channels))\n",
        "\n",
        "net.add(layers.BatchNormalization())\n",
        "net.add(layers.Activation('relu'))\n",
        "net.add(layers.GlobalAveragePooling2D())\n",
        "net.add(layers.Dense(10))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f7ckNMG7uCCX",
        "colab_type": "text"
      },
      "source": [
        "###5.12.4. 获取数据并训练模型\n",
        "由于这里使用了比较深的网络，本节里我们将输入高和宽从224降到96来简化计算。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pvsBLJkh3SbB",
        "colab_type": "code",
        "outputId": "7912a019-66ff-4620-a5ac-b60e23bae797",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 306
        }
      },
      "source": [
        "lr,num_epochs,batch_size=0.05,5,128\n",
        "optimizer=optimizers.SGD(learning_rate=lr)\n",
        "loss=losses.SparseCategoricalCrossentropy(from_logits=True)\n",
        "buffer_size=1000\n",
        "def load_data_fashion_mnist(batch_size,buffer_size):\n",
        "  (x_train,y_train),(x_test,y_test)=keras.datasets.fashion_mnist.load_data()\n",
        "  x_train=x_train[:,:,:,np.newaxis]#将三维张量增加一个channel维\n",
        "  x_test=x_test[:,:,:,np.newaxis]\n",
        "  train_iter=data.Dataset.from_tensor_slices((x_train,y_train)).map(lambda x,y:(x/255,y)).shuffle(buffer_size).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
        "  test_iter=data.Dataset.from_tensor_slices((x_test,y_test)).map(lambda x,y:(x/255,y)).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
        "  return train_iter,test_iter\n",
        "train_iter,test_iter=load_data_fashion_mnist(batch_size=batch_size,buffer_size=buffer_size)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00ac51510> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00ac51510>. It was defined on this line: train_iter=data.Dataset.from_tensor_slices((x_train,y_train)).map(lambda x,y:(x/255,y)).shuffle(buffer_size).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING: Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00ac51510> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00ac51510>. It was defined on this line: train_iter=data.Dataset.from_tensor_slices((x_train,y_train)).map(lambda x,y:(x/255,y)).shuffle(buffer_size).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING:tensorflow:Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00aa8aae8> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00aa8aae8>. It was defined on this line: train_iter=data.Dataset.from_tensor_slices((x_train,y_train)).map(lambda x,y:(x/255,y)).shuffle(buffer_size).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING: Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00aa8aae8> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00aa8aae8>. It was defined on this line: train_iter=data.Dataset.from_tensor_slices((x_train,y_train)).map(lambda x,y:(x/255,y)).shuffle(buffer_size).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING:tensorflow:Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00aa8ac80> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00aa8ac80>. It was defined on this line: test_iter=data.Dataset.from_tensor_slices((x_test,y_test)).map(lambda x,y:(x/255,y)).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING: Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00aa8ac80> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00aa8ac80>. It was defined on this line: test_iter=data.Dataset.from_tensor_slices((x_test,y_test)).map(lambda x,y:(x/255,y)).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING:tensorflow:Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00aa72048> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00aa72048>. It was defined on this line: test_iter=data.Dataset.from_tensor_slices((x_test,y_test)).map(lambda x,y:(x/255,y)).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n",
            "WARNING: Entity <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00aa72048> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Unable to identify source code of lambda function <function load_data_fashion_mnist.<locals>.<lambda> at 0x7fa00aa72048>. It was defined on this line: test_iter=data.Dataset.from_tensor_slices((x_test,y_test)).map(lambda x,y:(x/255,y)).batch(batch_size).map(lambda x,y:(image.resize(images=x,size=(224,224)),y))\n",
            ", which must contain a single lambda with matching signature. To avoid ambiguity, define each lambda in a separate expression.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FUzhfncQ3irt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "net.compile(optimizer=optimizer,loss=loss,metrics=['accuracy'])\n",
        "history=net.fit_generator(train_iter,validation_data=test_iter,epochs=num_epochs)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "45wgL7-J3jSy",
        "colab_type": "code",
        "outputId": "d2d51843-8fdd-4d7d-84ae-e87ecfabbbbe",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 269
        }
      },
      "source": [
        "X=tf.random.uniform(shape=(1,224,224,1))\n",
        "for layer in net.layers:\n",
        "  X=layer(X)\n",
        "  print(layer.name,X.shape)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "conv2d_3 (1, 112, 112, 64)\n",
            "batch_normalization_3 (1, 112, 112, 64)\n",
            "activation_3 (1, 112, 112, 64)\n",
            "max_pooling2d (1, 56, 56, 64)\n",
            "dense_block_2 (1, 56, 56, 192)\n",
            "sequential_8 (1, 28, 28, 192)\n",
            "dense_block_3 (1, 28, 28, 320)\n",
            "sequential_13 (1, 14, 14, 320)\n",
            "dense_block_4 (1, 14, 14, 448)\n",
            "sequential_18 (1, 7, 7, 448)\n",
            "dense_block_5 (1, 7, 7, 576)\n",
            "batch_normalization_23 (1, 7, 7, 576)\n",
            "activation_23 (1, 7, 7, 576)\n",
            "global_average_pooling2d (1, 576)\n",
            "dense (1, 10)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SaX2q1eG6VTt",
        "colab_type": "code",
        "outputId": "56f2ef14-8f36-4067-fea6-433581239244",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 655
        }
      },
      "source": [
        "net.summary()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_3\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "conv2d_3 (Conv2D)            (None, 112, 112, 64)      3200      \n",
            "_________________________________________________________________\n",
            "batch_normalization_3 (Batch (None, 112, 112, 64)      256       \n",
            "_________________________________________________________________\n",
            "activation_3 (Activation)    (None, 112, 112, 64)      0         \n",
            "_________________________________________________________________\n",
            "max_pooling2d (MaxPooling2D) (None, 56, 56, 64)        0         \n",
            "_________________________________________________________________\n",
            "dense_block_2 (DenseBlock)   (None, 56, 56, 192)       130944    \n",
            "_________________________________________________________________\n",
            "sequential_8 (Sequential)    (None, 28, 28, 192)       37824     \n",
            "_________________________________________________________________\n",
            "dense_block_3 (DenseBlock)   (None, 28, 28, 320)       280448    \n",
            "_________________________________________________________________\n",
            "sequential_13 (Sequential)   (None, 14, 14, 320)       104000    \n",
            "_________________________________________________________________\n",
            "dense_block_4 (DenseBlock)   (None, 14, 14, 448)       429952    \n",
            "_________________________________________________________________\n",
            "sequential_18 (Sequential)   (None, 7, 7, 448)         202944    \n",
            "_________________________________________________________________\n",
            "dense_block_5 (DenseBlock)   (None, 7, 7, 576)         579456    \n",
            "_________________________________________________________________\n",
            "batch_normalization_23 (Batc (None, 7, 7, 576)         2304      \n",
            "_________________________________________________________________\n",
            "activation_23 (Activation)   (None, 7, 7, 576)         0         \n",
            "_________________________________________________________________\n",
            "global_average_pooling2d (Gl (None, 576)               0         \n",
            "_________________________________________________________________\n",
            "dense (Dense)                (None, 10)                5770      \n",
            "=================================================================\n",
            "Total params: 1,777,098\n",
            "Trainable params: 1,764,170\n",
            "Non-trainable params: 12,928\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GvFSYAtv1F1p",
        "colab_type": "text"
      },
      "source": [
        "###5.12.5. 小结\n",
        "* 在跨层连接上，不同于ResNet中将输入与输出相加，DenseNet在通道维上连结输入与输出。\n",
        "DenseNet的主要构建模块是稠密块和过渡层。"
      ]
    }
  ]
}