3.3 线性回归的简洁实现.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"3.3 线性回归的简洁实现.ipynb","provenance":[],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"markdown","metadata":{"id":"N9CZ60wiV3r7","colab_type":"text"},"source":["##3.3. 线性回归的简介实现\n","&emsp;&emsp;随着深度学习框架的发展，开发深度学习应用变得越来越便利。实践中，我们通常可以用比上一节更简洁的代码来实现同样的模型。在本节中，我们将介绍如果使用tensorflow提供的keras模块更方便地实现线性回归的训练。"]},{"cell_type":"code","metadata":{"id":"riDMBU80sm3g","colab_type":"code","colab":{}},"source":["%matplotlib inline\n","from IPython import display\n","from matplotlib import pyplot as pyplot\n","import tensorflow as tf\n","import random\n","import numpy as np\n","import tensorflow.keras as keras\n","\n","tf.enable_eager_execution() #启用动态图计算"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"KnlpZW9AWwFp","colab_type":"text"},"source":["###3.3.1.生成数据集\n","&emsp;&emsp;我们生成与上一节中相同的数据集。其中features是训练数据特征，labels是标签。"]},{"cell_type":"code","metadata":{"id":"T3CdJQeWs4ju","colab_type":"code","colab":{}},"source":["num_inputs=2\n","num_examples=1000\n","true_w=[2,-3.4]\n","true_b=4.2\n","features=tf.random.normal(shape=(num_examples,num_inputs),stddev=1)\n","labels=true_w[0]*features[:,0]+true_w[1]*features[:,1]+true_b\n","labels+=tf.random.normal(shape=labels.shape,stddev=0.01)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"NObCcfJYqz5A","colab_type":"code","outputId":"0e38dc1a-a7aa-4430-a9e0-ebcea02b5d23","executionInfo":{"status":"ok","timestamp":1574742260501,"user_tz":-480,"elapsed":854,"user":{"displayName":"程波","photoUrl":"","userId":"08277046305423172038"}},"colab":{"base_uri":"https://localhost:8080/","height":35}},"source":["features.numpy().shape,labels.numpy().shape"],"execution_count":9,"outputs":[{"output_type":"execute_result","data":{"text/plain":["((1000, 2), (1000,))"]},"metadata":{"tags":[]},"execution_count":9}]},{"cell_type":"markdown","metadata":{"id":"uu6otVuhXHuT","colab_type":"text"},"source":["###3.3.2.读取数据\n","&emsp;&emsp;我们使用tensorflow中的data模块中的Dataset类来读取数据。Dataset.from_tensor_slices()方法传入张量并返回Dataset实例。设置batch_size为10，在每一次迭代中，我们将随机读取包含10个数据样本的小批量。"]},{"cell_type":"code","metadata":{"id":"I-hVg4H4q8fN","colab_type":"code","colab":{}},"source":["batch_size=10\n","buffer_size=1000\n","dataset=tf.data.Dataset.from_tensor_slices((features,labels))\n","dataset=dataset.shuffle(1000).batch(batch_size)"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"sJf5D6l9s1Cb","colab_type":"code","outputId":"9b1fe991-a3c1-428d-ad2b-be872355fc36","executionInfo":{"status":"ok","timestamp":1574742260503,"user_tz":-480,"elapsed":839,"user":{"displayName":"程波","photoUrl":"","userId":"08277046305423172038"}},"colab":{"base_uri":"https://localhost:8080/","height":35}},"source":["type(dataset)"],"execution_count":11,"outputs":[{"output_type":"execute_result","data":{"text/plain":["tensorflow.python.data.ops.dataset_ops.DatasetV1Adapter"]},"metadata":{"tags":[]},"execution_count":11}]},{"cell_type":"code","metadata":{"id":"RCW9QxmLs_Xv","colab_type":"code","outputId":"f3289076-c2ef-4822-ea25-62d3152284b9","executionInfo":{"status":"ok","timestamp":1574742260503,"user_tz":-480,"elapsed":830,"user":{"displayName":"程波","photoUrl":"","userId":"08277046305423172038"}},"colab":{"base_uri":"https://localhost:8080/","height":35}},"source":["for x,y in dataset:\n","  print(x.shape,y.shape)\n","  break"],"execution_count":12,"outputs":[{"output_type":"stream","text":["(10, 2) (10,)\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"oeSSQ5s6YR2v","colab_type":"text"},"source":["### 3.3.3定义模型\n","&emsp;&emsp;\n","在上一节从零开始的实现中，我们需要定义模型参数，并使用它们一步步描述模型是怎样计算的。当模型结构变得更复杂时，这些步骤变得更繁琐。其实，keras模块提供了大量预定义的层，这使我们只需要关注用哪些层来构造模型。下面将介绍如何使用keras更简洁地定义线性回归。\\\n","&emsp;&emsp;首先，从keras中导入Sequential类，可以将它看作是一个串联各个层的容器。在构造模型时，我们在该容器中依次添加层。当给定输入数据时，容器中的每一层将依次计算并将输出作为下一层的输入。"]},{"cell_type":"code","metadata":{"id":"FFPPukddtGML","colab_type":"code","colab":{}},"source":["from tensorflow.keras import Sequential"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"Rm67aEVrtWJF","colab_type":"code","colab":{}},"source":["net=Sequential()"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"E-EG9gpWfzMJ","colab_type":"text"},"source":["&emsp;&emsp;回顾图3.1中线性回归在神经网络图中的表示。作为一个单层神经网络，线性回归输出层中的神经元和输入层中各个输入完全连接。因此，在线性回归的输出层又叫全连接层。在keras中，全连接层是一个Dense实例。我们定义该层输出个数为1."]},{"cell_type":"code","metadata":{"id":"aLxBHYV9tY_S","colab_type":"code","colab":{}},"source":["net.add(keras.layers.Dense(1))"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"jZplHZM0gW_5","colab_type":"text"},"source":["&emsp;&emsp;值得一提的是，在keras中我们无须指定每一层输入的形状，例如线性回归的输入个数。当模型得到数据时，例如后面执行net(X)时，模型将自动推断出每一层的输入个数。"]},{"cell_type":"markdown","metadata":{"id":"PPE6J9ohgm14","colab_type":"text"},"source":["###3.3.4 定义损失函数\n","&emsp;&emsp;在keras中，losses模块定义了各种损失。我们使用平方误差损失作为模型的损失函数。"]},{"cell_type":"code","metadata":{"id":"t2rqwLdDtfuc","colab_type":"code","colab":{}},"source":["loss=keras.losses.mean_squared_error"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"yXtD-CiSg714","colab_type":"text"},"source":["###3.3.5 定义优化算法\n","&emsp;&emsp;同样，我们也无须实现小批量随机梯度下降。在导入keras后，使用optimizer模块创建一个SGD实例，并指定学习率为0.03。该优化算法将用来迭代net实例所有通过add函数嵌套的层所包含的全部参数。"]},{"cell_type":"code","metadata":{"id":"SqopGnqWt9pj","colab_type":"code","colab":{}},"source":["optimizer=keras.optimizers.SGD(learning_rate=0.03) #定义优化器"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"o6aH4AAZ7M1N","colab_type":"text"},"source":["###3.3.6. 训练模型\n","&emsp;&emsp;在使用keras训练模型时，需要先调用compile()函数来编译模型，在该函数中指定optimizer优化器、损失函数已经评价指标。然后调用fit_generator()函数指定训练数据和epoch进行训练。"]},{"cell_type":"code","metadata":{"id":"GXgk0pgBuICh","colab_type":"code","outputId":"51ef97f8-9efe-4c03-db20-77b5c7b28849","executionInfo":{"status":"ok","timestamp":1574742262814,"user_tz":-480,"elapsed":3110,"user":{"displayName":"程波","photoUrl":"","userId":"08277046305423172038"}},"colab":{"base_uri":"https://localhost:8080/","height":138}},"source":["num_epochs=3\n","net.compile(optimizer=optimizer,loss=loss,metrics=[keras.metrics.mae])\n","net.fit_generator(dataset,epochs=3)"],"execution_count":18,"outputs":[{"output_type":"stream","text":["Epoch 1/3\n","100/100 [==============================] - 1s 9ms/step - loss: 3.2955 - mean_absolute_error: 2.0419\n","Epoch 2/3\n","100/100 [==============================] - 1s 6ms/step - loss: 1.1064e-04 - mean_absolute_error: 0.0091\n","Epoch 3/3\n","100/100 [==============================] - 1s 6ms/step - loss: 1.0312e-04 - mean_absolute_error: 0.0081\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/plain":["<tensorflow.python.keras.callbacks.History at 0x7f3afb4c6c18>"]},"metadata":{"tags":[]},"execution_count":18}]},{"cell_type":"markdown","metadata":{"id":"0fWP7nW57vEt","colab_type":"text"},"source":["&emsp;&emsp;让我们打印出模型结构看看，可以看到模型仅包含一个Dense层，并且参数数量为3."]},{"cell_type":"code","metadata":{"id":"YK1rFKVIu2Lw","colab_type":"code","outputId":"8d74e806-3a1e-4050-b20a-f0b419ce82e1","executionInfo":{"status":"ok","timestamp":1574742262823,"user_tz":-480,"elapsed":3107,"user":{"displayName":"程波","photoUrl":"","userId":"08277046305423172038"}},"colab":{"base_uri":"https://localhost:8080/","height":190}},"source":["net.summary()"],"execution_count":19,"outputs":[{"output_type":"stream","text":["Model: \"sequential\"\n","_________________________________________________________________\n","Layer (type)                 Output Shape              Param #   \n","=================================================================\n","dense (Dense)                multiple                  3         \n","=================================================================\n","Total params: 3\n","Trainable params: 3\n","Non-trainable params: 0\n","_________________________________________________________________\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"58jpFfT28Hem","colab_type":"text"},"source":["&emsp;&emsp;下面我们比较学习到的模型参数和真实的模型参数。我们从net.get_weights()函数获取权重（weight）和偏差(bias)。学到的参数和真实的参数很接近"]},{"cell_type":"code","metadata":{"id":"jhRQepY_u-hV","colab_type":"code","outputId":"74ff3c0b-4eb0-4b12-dab6-1606e36aadf1","executionInfo":{"status":"ok","timestamp":1574742262825,"user_tz":-480,"elapsed":3096,"user":{"displayName":"程波","photoUrl":"","userId":"08277046305423172038"}},"colab":{"base_uri":"https://localhost:8080/","height":52}},"source":["train_w,train_b=net.get_weights()\n","true_w,train_w"],"execution_count":20,"outputs":[{"output_type":"execute_result","data":{"text/plain":["([2, -3.4], array([[ 2.0006943],\n","        [-3.399984 ]], dtype=float32))"]},"metadata":{"tags":[]},"execution_count":20}]},{"cell_type":"code","metadata":{"id":"yVPZiXTyvFgI","colab_type":"code","outputId":"cc081bdd-f4fd-4fe9-9f66-05a03b19a552","executionInfo":{"status":"ok","timestamp":1574742263100,"user_tz":-480,"elapsed":3364,"user":{"displayName":"程波","photoUrl":"","userId":"08277046305423172038"}},"colab":{"base_uri":"https://localhost:8080/","height":35}},"source":["true_b,train_b"],"execution_count":21,"outputs":[{"output_type":"execute_result","data":{"text/plain":["(4.2, array([4.1989474], dtype=float32))"]},"metadata":{"tags":[]},"execution_count":21}]},{"cell_type":"markdown","metadata":{"id":"nA5PHNrh8eZf","colab_type":"text"},"source":["###3.3.7. 小结\n","* 在keras中可以更简洁地实现模型。\n","* tf.data模块提供了数据处理的工具，keras中定义了大量的神经网络层，keras.losses模块定义了各种损失函数。keras.optimizers模块中定义了各种优化算法。"]},{"cell_type":"code","metadata":{"id":"OkW6_tASwUwt","colab_type":"code","colab":{}},"source":[""],"execution_count":0,"outputs":[]}]}