Chapter11_Part1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ce2d6e30",
   "metadata": {},
   "source": [
    "# Chapter 11. Deep Learning in Chemistry"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d09f3a0",
   "metadata": {},
   "source": [
    "## 11.1. Perceptron"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa334069",
   "metadata": {},
   "source": [
    "In this section, we will explore the structure of a perceptron, learn about how it work, and use single-perceptron networks to solve simple linear regression problems."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a096bd7c",
   "metadata": {},
   "source": [
    "### 11.1.1. Structure of Perceptron"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3d39a7a-07c6-4b0b-82b8-22b841a6a779",
   "metadata": {},
   "source": [
    "The structure of a perceptron can be described as follow:\n",
    "![Single layer perceptron](images/perceptron_structure.jpg)\n",
    "\n",
    "**Key points to remember about perceptron**\n",
    "\n",
    "- A perceptron can have 1 or multiple inputs, and only 1 output\n",
    "- A perceptron contains the weights (w) for each of the inputs, and a bias (b)\n",
    "- The output of a perceptron is calculated using the following equation:\n",
    "\n",
    "$$\\text{output} = \\sum_{i}(w_i \\times x_i) + b$$\n",
    "\n",
    "Here, $w_i$ represents the weight associated with the i<sup>th</sup> input, $x_i$ is the i<sup>th</sup> input, and $b$ is the bias term. The perceptron computes a weighted sum of its inputs, adds a bias, and then applies an activation function to produce the output."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5972708",
   "metadata": {},
   "source": [
    "### 11.1.2. How Single-Perceptron Network Works"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d201e568",
   "metadata": {},
   "source": [
    "#### 11.1.2.1. General Concept"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26814b2b",
   "metadata": {},
   "source": [
    "Single-perceptron network, or any other artificial neural network (ANN), works on the basic of **gradient descent**.\n",
    "You can think of an ANN as a mathematic expression that maps the inputs to the outputs:\n",
    "![ANN](images/ANN.png)\n",
    "In general, to use an ANN to solve a regression problem, you need to follow these step:\n",
    "- **Step 1.** Initialize the parameters (weights and biases) of each neuron in the network randomly.\n",
    "- **Step 2.** Forward pass: Pass the input values to the network, calculate the output of each neuron in the network in order, until we get the output values.\n",
    "- **Step 3.** Calculate the value of the loss function.\n",
    "- **Step 4.** Backward pass: Calculate the gradients for all weights and biases in the network with respect to the loss function\n",
    "- **Step 5.** Update the values of weights and biases so that they can lower the loss function.\n",
    "- Return to **Step 2**, repeat after a number of iterations (epochs) or until the loss function is low enough.\n",
    "\n",
    "The loop from step 2 to step 5 is called the **training process**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0bc1bb1",
   "metadata": {},
   "source": [
    "#### 11.1.2.2. The Workflow of a Single-Perceptron Network"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1479b306",
   "metadata": {},
   "source": [
    "An example of a single-perceptron network with 3 inputs is below:\n",
    "![Single-perceptron network](images/single-perceptron_network_3_inputs.png)\n",
    "We will test this network for the regression of function $y = x_1 - 3.x_2 + 2.x_3$, using 1 single data point: $x_1 = 3.4$, $x_2 = 2.4$, $x_3 = -1.7$ → $y = -7.2$.\n",
    "\n",
    "Following the general workflow:\n",
    "- **Step 1.** Initialize the parameters:\n",
    "![Single-perceptron network](images/single-perceptron_network_3_inputs_step1.png)\n",
    "\n",
    "- **Step 2.** Forward:\n",
    "![Single-perceptron network](images/single-perceptron_network_3_inputs_step2.png)\n",
    "\n",
    "- **Step 3.** Calculate the loss function. We use MSE loss function in this case:\n",
    "\n",
    "$$\\text{MSE Loss} = \\frac{1}{n} \\sum_{i=1}^{n} (y_i - \\hat{y}_i)^2$$\n",
    "\n",
    "with $n = 1$ (1 data point), $y = -7.2$ (observed value), $\\hat{y} = 0.38$ (predicted value) → $MSE Loss = 57.5$\n",
    "\n",
    "- **Step 4.** Backward:\n",
    "\n",
    "We cannot calculate the gradients for weights and bias directly. We have to use the chain rule.\n",
    "\n",
    "$$\\frac{\\partial \\text{MSE}}{\\partial \\hat{y}} = -2 \\times (y - \\hat{y}) = 15.16$$\n",
    "\n",
    "$$\\frac{\\partial \\hat{y}}{\\partial w_1} = x_1 = 3.4 \\rightarrow \\frac{\\partial \\text{MSE}}{\\partial w_1} = \\left(\\frac{\\partial \\text{MSE}}{\\partial \\hat{y}}\\right) \\times \\left(\\frac{\\partial \\hat{y}}{\\partial w_1}\\right) = 15.16 \\times 3.4 = 51.544$$\n",
    "\n",
    "$$\\frac{\\partial \\hat{y}}{\\partial w_2} = x_2 = 2.4 \\rightarrow \\frac{\\partial \\text{MSE}}{\\partial w_2} = \\left(\\frac{\\partial \\text{MSE}}{\\partial \\hat{y}}\\right) \\times \\left(\\frac{\\partial \\hat{y}}{\\partial w_2}\\right) = 15.16 \\times 2.4 = 36.384$$\n",
    "\n",
    "$$\\frac{\\partial \\hat{y}}{\\partial w_3} = x_3 = -1.7 \\rightarrow \\frac{\\partial \\text{MSE}}{\\partial w_3} = \\left(\\frac{\\partial \\text{MSE}}{\\partial \\hat{y}}\\right) \\times \\left(\\frac{\\partial \\hat{y}}{\\partial w_3}\\right) = 15.16 \\times (-1.7) = -25.772$$\n",
    "\n",
    "$$\\frac{\\partial \\hat{y}}{\\partial b} = 1 \\rightarrow \\frac{\\partial \\text{MSE}}{\\partial b} = \\frac{\\partial \\text{MSE}}{\\partial \\hat{y}} = 15.16$$\n",
    "\n",
    "![Single-perceptron network](images/single-perceptron_network_3_inputs_step4.png)\n",
    "\n",
    "- **Step 5.** Update weights and bias:\n",
    "\n",
    "Choose a learning rate: $lr = 0.001$\n",
    "\n",
    "$$w_1 = w_1 - \\left( \\frac{\\partial \\text{MSE}}{\\partial w1} \\right) \\times \\text{lr} = w1 - 51.544 \\times 0.001 = 0.248$$\n",
    "\n",
    "$$w_2 = w_2 - \\left( \\frac{\\partial \\text{MSE}}{\\partial w2} \\right) \\times \\text{lr} = w2 - 36.384 \\times 0.001 = -0.536$$\n",
    "\n",
    "$$w_3 = w_3 - \\left( \\frac{\\partial \\text{MSE}}{\\partial w3} \\right) \\times \\text{lr} = w3 - (-25.772) \\times 0.001 = 0.226$$\n",
    "\n",
    "$$b = b - \\left( \\frac{\\partial \\text{MSE}}{\\partial b} \\right) \\times \\text{lr} = b - 15.16 \\times 0.001 = 0.885$$\n",
    "\n",
    "![Single-perceptron network](images/single-perceptron_network_3_inputs_step5.png)\n",
    "\n",
    "- Return to **Step 2** and continue\n",
    "\n",
    "**Note:** If we continue running this loop, the values of weights and bias will approaching their target values ($w_1 → 1$, $w_2 → -3$, $w_3 → 2$, $b → 0$), the output will get closer to the observed values, and the MSE loss will approach its mininum."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87cf1436",
   "metadata": {},
   "source": [
    "### 11.1.3. PyTorch Implementation of Perceptron"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06c4ccc2",
   "metadata": {},
   "source": [
    "PyTorch is an open-source machine learning library developed by Facebook's AI Research lab, widely used for applications in deep learning and artificial intelligence. Known for its flexibility and dynamic computational graph, it provides a rich ecosystem for developing and training neural network models efficiently and intuitively. PyTorch is particularly favored for its ease of use, simplicity in debugging, and strong support for GPU acceleration, making it a popular choice among researchers and developers in the field of AI.\n",
    "\n",
    "To install PyTorch, visit their [website](https://pytorch.org/) for instruction."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf5d20fa",
   "metadata": {},
   "source": [
    "#### 11.1.3.1. Single-perceptron network with 1 input, no scaling"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "170f1606",
   "metadata": {},
   "source": [
    "In this section, we will use a single-perceptron network to perform linear regression for function $y = f(x)$:\n",
    "\n",
    "*Network structure:*\n",
    "![Single-perceptron network](images/single-perceptron_network_1_input.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc5c3d1a",
   "metadata": {},
   "source": [
    "**a. Import modules**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e06c9d90-8488-4aac-90ee-96b688865741",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install matplotlib extension for JupyterLab\n",
    "!conda install -c conda-forge ipympl"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "429c8af5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Import modules\n",
    "import math\n",
    "import numpy as np\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.animation import FuncAnimation\n",
    "%matplotlib widget\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from tqdm import tqdm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e75ea2ca",
   "metadata": {},
   "source": [
    "**b. Load data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ebfccc2b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define the data points\n",
    "x = torch.tensor([1.01, 2.03, 3.01, 4.07, 5.09])\n",
    "y = torch.tensor([2.02, 4.04, 6.05, 8.03, 10.08])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbf300a3",
   "metadata": {},
   "source": [
    "**c. Create model**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f34e3598",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define the linear regression class\n",
    "class LinearRegressionModel(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(LinearRegressionModel, self).__init__()\n",
    "        self.linear = nn.Linear(1, 1)  # One input feature and one output\n",
    "\n",
    "    def forward(self, x):\n",
    "        return self.linear(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a93267b0",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Instantiate the model\n",
    "model = LinearRegressionModel()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "737fcf88",
   "metadata": {},
   "source": [
    "**d. Training**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "59aaafbf",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define the loss function and optimizer\n",
    "criterion = nn.MSELoss()\n",
    "optimizer = optim.SGD(model.parameters(), lr=0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "abf487e8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Define the train function\n",
    "def train(x, y):\n",
    "    # Forward pass\n",
    "    outputs = model(x.unsqueeze(1))\n",
    "    loss = criterion(outputs, y.unsqueeze(1))\n",
    "    \n",
    "    # Backward and optimize\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "    \n",
    "    return loss.item()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51168a91",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create lists of parameters and losses for visualization\n",
    "w_values = []\n",
    "b_values = []\n",
    "losses = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a1340db",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Train the model\n",
    "num_epochs = 100\n",
    "progress_bar = tqdm(range(num_epochs))\n",
    "for epoch in progress_bar:\n",
    "    train_loss = train(x, y)\n",
    "    \n",
    "    # Get the trained parameters\n",
    "    w = model.linear.weight.item()\n",
    "    b = model.linear.bias.item()\n",
    "    \n",
    "    # Add trained parameters to their lists for visualization\n",
    "    w_values.append(w)\n",
    "    b_values.append(b)\n",
    "    losses.append(train_loss)\n",
    "    \n",
    "    # Print progress\n",
    "    progress_bar.set_description(f'Epoch [{epoch+1}/{num_epochs}], Train loss: {train_loss:.4f}') "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ecdd638",
   "metadata": {},
   "source": [
    "**e. Visualization**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6012dd11",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create the axis\n",
    "x_min = math.floor(torch.min(x).item())\n",
    "x_max = math.ceil(torch.max(x).item())\n",
    "y_min = math.floor(torch.min(y).item())\n",
    "y_max = math.ceil(torch.max(y).item())\n",
    "\n",
    "# Create the initial plot\n",
    "fig = plt.figure()\n",
    "plt.xlim(x_min, x_max)\n",
    "plt.ylim(y_min, y_max)\n",
    "plt.scatter(x, y, color='red')\n",
    "plt.xlabel('x')\n",
    "plt.ylabel('y')\n",
    "x_values = np.linspace(x_min, x_max, num=101)\n",
    "y_values = w_values[0]*x_values + b_values[0]\n",
    "lines = plt.plot(x_values, y_values)\n",
    "line = lines[0]\n",
    "\n",
    "def update(frame):\n",
    "    # Get the x and y coordinates for the current frame\n",
    "    y_values = w_values[frame]*x_values + b_values[frame]\n",
    "\n",
    "    # Update the scatter plot with the current point\n",
    "    line.set_data(x_values, y_values)\n",
    "\n",
    "# Create the animation\n",
    "animation = FuncAnimation(fig, update, frames=num_epochs, interval=200, blit=False)\n",
    "\n",
    "# Display the animation\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19d2ea3c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualize MSE loss values over time\n",
    "fig = plt.figure()\n",
    "plt.plot(losses)\n",
    "plt.xlabel('epoch')\n",
    "plt.ylabel('MSE loss')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37055600",
   "metadata": {},
   "source": [
    "**f. Make prediction**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f89d4829",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Input for prediction\n",
    "x_pred = 5\n",
    "\n",
    "# Convert to tensor\n",
    "x_pred = torch.tensor([x_pred]).float()\n",
    "\n",
    "# Run model forward\n",
    "y_pred = model(x_pred)\n",
    "\n",
    "# Convert back to number\n",
    "y_pred = y_pred.item()\n",
    "\n",
    "# Show result\n",
    "print(y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3dd410c1",
   "metadata": {},
   "source": [
    "**g. Save and load model**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3712059d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save model\n",
    "model_name = 'model_1'\n",
    "file_name = f'./{model_name}_{num_epochs}.ckpt'\n",
    "torch.save(model.state_dict(), file_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51f233a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load model\n",
    "file_name = f'./{model_name}_100.ckpt'\n",
    "loaded_model = LinearRegressionModel()\n",
    "loaded_model.load_state_dict(torch.load(file_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c355a6d8",
   "metadata": {},
   "source": [
    "#### 11.1.3.2. Single-perceptron network with 2 inputs, with scaling"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f852a0e3",
   "metadata": {},
   "source": [
    "In this section, we will use a single-perceptron network to perform linear regression for function with multiple variables $y = f(x_1, x_2)$:\n",
    "\n",
    "*Network structure:*\n",
    "![Single-perceptron network](images/single-perceptron_network_2_inputs.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3de7c90a",
   "metadata": {},
   "source": [
    "**b. Load data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55a6cefc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the data points\n",
    "x1 = np.array([1.01, 2.03, 3.01, 4.07, 5.09])\n",
    "x2 = np.array([9.02, 7.04, 5.05, 3.03, 1.08])\n",
    "y = 2*x1 - 3*x2 +5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "296dd874",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Combine x1 and x2 into 1 array\n",
    "x = np.column_stack((x1, x2))\n",
    "\n",
    "# Define input and output scalers\n",
    "input_scaler = MinMaxScaler(feature_range=(0, 1))\n",
    "output_scaler = MinMaxScaler(feature_range=(0, 1))\n",
    "\n",
    "# Scale the data to the range from 0 to 1\n",
    "x_scaled = input_scaler.fit_transform(x)\n",
    "y_scaled = output_scaler.fit_transform(y.reshape(-1, 1)).reshape(-1)\n",
    "\n",
    "# Convert to tensor\n",
    "x_scaled = torch.tensor(x_scaled).float()\n",
    "y_scaled = torch.tensor(y_scaled).float()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1467b6f6",
   "metadata": {},
   "source": [
    "**c. Create model**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a48cea8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the linear regression class\n",
    "class LinearRegressionModel(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(LinearRegressionModel, self).__init__()\n",
    "        self.linear = nn.Linear(2, 1)  # One input feature and one output\n",
    "\n",
    "    def forward(self, x):\n",
    "        return self.linear(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a89af10e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiate the model\n",
    "model = LinearRegressionModel()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7a5086a",
   "metadata": {},
   "source": [
    "**d. Training**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91b7d264",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the loss function and optimizer\n",
    "criterion = nn.MSELoss()\n",
    "optimizer = optim.SGD(model.parameters(), lr=0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d4d1267f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the train function\n",
    "def train(x, y):\n",
    "    # Forward pass\n",
    "    outputs = model(x)\n",
    "    loss = criterion(outputs, y.unsqueeze(1))\n",
    "    \n",
    "    # Backward and optimize\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "    \n",
    "    return loss.item()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91d84ed7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create lists of parameters and losses for visualization\n",
    "w_values = []\n",
    "b_values = []\n",
    "losses = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "013a5dcc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train the model\n",
    "num_epochs = 1000\n",
    "progress_bar = tqdm(range(num_epochs))\n",
    "for epoch in progress_bar:\n",
    "    train_loss = train(x_scaled, y_scaled)\n",
    "    \n",
    "    # Get the trained parameters\n",
    "    w1 = model.linear.weight[0][0].item()\n",
    "    w2 = model.linear.weight[0][1].item()\n",
    "    b = model.linear.bias.item()\n",
    "    \n",
    "    # Add trained parameters to their lists for visualization\n",
    "    w_values.append((w1, w2))\n",
    "    b_values.append(b)\n",
    "    losses.append(train_loss)\n",
    "    \n",
    "    # Print progress\n",
    "    progress_bar.set_description(f'Epoch [{epoch+1}/{num_epochs}], Train loss: {train_loss:.4f}') "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3d18638",
   "metadata": {},
   "source": [
    "**e. Visualization**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33a76692",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualize MSE loss values over time\n",
    "fig = plt.figure()\n",
    "plt.plot(losses)\n",
    "plt.xlabel('epoch')\n",
    "plt.ylabel('MSE loss')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcb03fb9",
   "metadata": {},
   "source": [
    "**f. Make prediction**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9daeefaf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Input for prediction\n",
    "x_pred = [8, 1]\n",
    "\n",
    "# Scale input to range 0 to 1\n",
    "x_pred_scaled = input_scaler.transform([x_pred])\n",
    "\n",
    "# Convert to tensor\n",
    "x_pred_scaled = torch.tensor(x_pred_scaled).float()\n",
    "\n",
    "# Run model forward\n",
    "y_pred_scaled = model(x_pred_scaled)\n",
    "\n",
    "# Convert back to number\n",
    "y_pred_scaled = y_pred_scaled.item()\n",
    "\n",
    "# Scale output back to original range\n",
    "y_pred = output_scaler.inverse_transform([[y_pred_scaled]])[0][0]\n",
    "\n",
    "# Show result\n",
    "print(y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd0ccdf1",
   "metadata": {},
   "source": [
    "**g. Save and load model**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c7ea803f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save model\n",
    "model_name = 'model_2'\n",
    "file_name = f'./{model_name}_{num_epochs}.ckpt'\n",
    "torch.save(model.state_dict(), file_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5f8dfad4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load model\n",
    "file_name = f'./{model_name}_2000.ckpt'\n",
    "loaded_model = LinearRegressionModel()\n",
    "loaded_model.load_state_dict(torch.load(file_name))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "358c1974-9373-46f4-91dc-0817e0f1e034",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  },
  "toc": {
   "base_numbering": "",
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Table of Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "244.333px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}