0_Intro_PyTorch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CSE6250BDH Deep Learning Labs\n",
    "## 0. Introduction to PyTorch\n",
    "\n",
    "In this chapter, we will learn basic usage of PyTorch.\n",
    "There are many good tutorials on PyTorch on web.\n",
    "We highly recommend you to follow the official [tutorial](http://pytorch.org/tutorials/) even though this tutorial is also mainly from it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After installing PyTorch, you can import `torch` in Python to use PyTorch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check the version of PyTorch, and it should be 1.0 or higher."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0.0\n"
     ]
    }
   ],
   "source": [
    "print(torch.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tensor Creation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch is very similar with Numpy as they say it is a replacement for Numpy to use the power of GPUs. Although there are still missing components, it has many same/similar functions for constructing or manipulating 'Tensor's.\n",
    "\n",
    "A basic object used in PyTorch is 'Tensor' which is equivalent to 'ndarray' in Numpy. Similarly to Numpy, there are multiple types of Tensors, e.g. Float, Double, Int, Long, etc. Most of time, however, we will use FloatTensor mainly (and it is a default type for the most of functions) to utilize GPU and LongTensor sometime for target/label values.\n",
    "\n",
    "Lets try to create a Tensor. If you call `torch.Tensor(rows, cols)`, it will return a FloatTensor without initialization (with garbage values)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[2.6135e+16, 3.0624e-41, 5.7453e-44],\n",
       "        [0.0000e+00,        nan, 0.0000e+00],\n",
       "        [1.3733e-14, 6.4076e+07, 2.0706e-19],\n",
       "        [7.3909e+22, 2.4176e-12, 1.1625e+33],\n",
       "        [8.9605e-01, 1.1632e+33, 5.6003e-02]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.Tensor(5, 3) # same result with torch.FloatTensor(5,3)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar to `numpy.ndarray()`, you can create a Tensro with values using `torch.tensor(values)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 2.],\n",
       "        [3., 4.]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_manual = torch.tensor([[1.0, 2.0], [3.0, 4.0]])\n",
    "x_manual"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also, you can create an initialized Tensor filled with 1s, 0s, or random numbers from a uniform distribtution by using `torch.ones`, `torch.zeros`, or `torch.rand` repectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.],\n",
      "        [1., 1., 1.]])\n",
      "tensor([[0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.],\n",
      "        [0., 0., 0.]])\n",
      "tensor([[0.6617, 0.6074, 0.7318],\n",
      "        [0.4898, 0.9365, 0.4340],\n",
      "        [0.4794, 0.5075, 0.9448],\n",
      "        [0.8782, 0.4373, 0.2330],\n",
      "        [0.6070, 0.9320, 0.0694]])\n"
     ]
    }
   ],
   "source": [
    "x_ones = torch.ones(5,3)\n",
    "print(x_ones)\n",
    "\n",
    "x_zeros = torch.zeros(5,3)\n",
    "print(x_zeros)\n",
    "\n",
    "x_uniform = torch.rand(5,3)\n",
    "print(x_uniform)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise: Try `torch.eye`, `torch.linspace`, `torch.logspace`, etc.\n",
    "### Exercise: Try other random functions from [here](http://pytorch.org/docs/master/torch.html#random-sampling)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Converting from/to Numpy ndarray"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also create a Tensor from Numpy ndarray or vice versa. In fact, we may do this many times in a project since we want to utilize many Numpy-based libraries (e.g., Pandas, Scikit-learn, Matplotlib, etc.) as well as GPU computation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can simply call `torch.from_numpy(ndarray)` to create a `Tensor` from a `numpy.ndarray`. **Be careful that the returned Tensor and original ndarray share the same memory**. Therefore, if you modify the Tensor, it will be reflected in the ndarray."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1. 2. 3.]\n",
      "tensor([1., 2., 3.])\n",
      "tensor([-1.,  2.,  3.])\n",
      "[-1.  2.  3.]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "np_array = np.array([1., 2., 3.], dtype=np.float32)  # set dtype=np.float32 to get a FloatTensor\n",
    "print(np_array)\n",
    "torch_tensor = torch.from_numpy(np_array)\n",
    "print(torch_tensor)\n",
    "\n",
    "# Modify the Tensor\n",
    "torch_tensor[0] = -1.0\n",
    "print(torch_tensor)\n",
    "# np_array has also been modified\n",
    "print(np_array)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the reverse way of conversion, you can call `numpy()` on a Tensor. Again, resulting ndarray shares the memory with the Tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([0.3749, 0.5540, 0.2663])\n",
      "[0.37494987 0.553994   0.26629055]\n",
      "tensor([0.7499, 0.5540, 0.2663])\n"
     ]
    }
   ],
   "source": [
    "another_torch_tensor = torch.rand(3)\n",
    "print(another_torch_tensor)\n",
    "another_np_array = another_torch_tensor.numpy()\n",
    "print(another_np_array)\n",
    "\n",
    "# Modify ndarray\n",
    "another_np_array[0] *= 2.0\n",
    "print(another_torch_tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To extract the value from a single-element Tensor, e.g., Tensor storing a loss value, you can use `item()` on a Tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([1.2300])\n",
      "1.2300000190734863\n"
     ]
    }
   ],
   "source": [
    "single_element_tensor = torch.Tensor([1.23])\n",
    "print(single_element_tensor)\n",
    "single_value = single_element_tensor.item()\n",
    "print(single_value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basic Operations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Indexing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use standard numpy-like indexing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.9285, 0.0259, 0.4460],\n",
      "        [0.1122, 0.8947, 0.6859],\n",
      "        [0.9858, 0.0953, 0.9046]])\n",
      "tensor([0.0259, 0.8947, 0.0953])\n",
      "tensor([[0.9285, 0.0259, 0.4460],\n",
      "        [0.1122, 0.8947, 0.6859]])\n"
     ]
    }
   ],
   "source": [
    "A = torch.rand(3,3)\n",
    "print(A)\n",
    "print(A[:, 1])  # get the 1st column\n",
    "print(A[:2, :])  # get the rows upto the 2nd row"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Arithmetic Operations\n",
    "Arithmetic operations with `+-*/` operators are all element-wise computation. Therefore, if you want to do some matrix computations such as matrix-matrix (or vector) multiplication, you need to call separate functions.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1.1311, 0.9856, 0.5035],\n",
      "        [0.1141, 1.5496, 1.2764],\n",
      "        [1.1244, 0.3644, 1.1973]])\n",
      "tensor([[1.8818e-01, 2.4884e-02, 2.5658e-02],\n",
      "        [2.0847e-04, 5.8597e-01, 4.0501e-01],\n",
      "        [1.3658e-01, 2.5652e-02, 2.6476e-01]])\n",
      "tensor([[1.8818e-01, 2.4884e-02, 2.5658e-02],\n",
      "        [2.0847e-04, 5.8597e-01, 4.0501e-01],\n",
      "        [1.3658e-01, 2.5652e-02, 2.6476e-01]])\n",
      "tensor([[0.2500, 1.0280, 0.1993],\n",
      "        [0.1194, 0.8782, 0.7355],\n",
      "        [0.3253, 1.2520, 0.3778]])\n",
      "tensor([1.0280, 0.8782, 1.2520])\n"
     ]
    }
   ],
   "source": [
    "B = torch.rand(3,3)\n",
    "print(A+B)\n",
    "print(A*B)\n",
    "# Another elementwise multiplication\n",
    "print(torch.mul(A,B))\n",
    "\n",
    "# Matrix-Matrix multiplication\n",
    "print(torch.mm(A,B))\n",
    "# Matrix-Vector multiplication\n",
    "print(torch.mv(A,B[:,1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many predefined operations for your convenience such as batch multiplication with addition, etc. Please read [PyTorch Docs](http://pytorch.org/docs/master/torch.html#math-operations) for more information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GPU Acceleration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we have NVIDIA GPU(s), we can accelerate computation once we move Tensors onto GPU.\n",
    "Let's compare how much GPU can accelerate especially matrix operations.\n",
    "We will do a matrix-matrix multiplication between two 5k-by-5k matrices on both CPU and GPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.2130, 0.6759, 0.2104,  ..., 0.1507, 0.7708, 0.7172],\n",
       "        [0.1117, 0.4493, 0.6779,  ..., 0.1140, 0.3263, 0.9410],\n",
       "        [0.3007, 0.0067, 0.9703,  ..., 0.3241, 0.1301, 0.4399],\n",
       "        ...,\n",
       "        [0.3882, 0.5549, 0.4373,  ..., 0.3492, 0.5614, 0.1680],\n",
       "        [0.2265, 0.5140, 0.8682,  ..., 0.8785, 0.3173, 0.6228],\n",
       "        [0.3648, 0.9283, 0.0312,  ..., 0.7357, 0.2212, 0.9060]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mat_cpu = torch.rand(5000, 5000)\n",
    "mat_cpu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([5000, 5000])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mat_cpu.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 6.21 s, sys: 1.04 s, total: 7.25 s\n",
      "Wall time: 290 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([[1695.7577, 1285.5846, 1256.9727,  ..., 1270.6284, 1257.8542,\n",
       "         1269.5782],\n",
       "        [1285.5846, 1683.9097, 1256.9144,  ..., 1264.7170, 1253.2314,\n",
       "         1267.9730],\n",
       "        [1256.9727, 1256.9144, 1638.8047,  ..., 1244.1445, 1239.7261,\n",
       "         1252.8428],\n",
       "        ...,\n",
       "        [1270.6284, 1264.7170, 1244.1445,  ..., 1669.2982, 1247.4923,\n",
       "         1262.1633],\n",
       "        [1257.8541, 1253.2316, 1239.7261,  ..., 1247.4923, 1668.7012,\n",
       "         1263.3229],\n",
       "        [1269.5781, 1267.9731, 1252.8429,  ..., 1262.1633, 1263.3229,\n",
       "         1677.7290]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "torch.mm(mat_cpu.t(), mat_cpu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### We need a GPU for this comparison\n",
    "We can check its availability like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "if torch.cuda.is_available():\n",
    "    cuda = True\n",
    "else:\n",
    "    cuda = False\n",
    "cuda"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.6907, 0.9506, 0.7832,  ..., 0.0178, 0.2380, 0.7641],\n",
       "        [0.6972, 0.5160, 0.4726,  ..., 0.1372, 0.0847, 0.6451],\n",
       "        [0.4686, 0.7244, 0.0956,  ..., 0.9701, 0.7684, 0.5043],\n",
       "        ...,\n",
       "        [0.4801, 0.7096, 0.2673,  ..., 0.0248, 0.8264, 0.7739],\n",
       "        [0.5860, 0.4998, 0.2829,  ..., 0.5636, 0.7892, 0.2528],\n",
       "        [0.2868, 0.1217, 0.2432,  ..., 0.0665, 0.8304, 0.6522]],\n",
       "       device='cuda:0')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mat_gpu = torch.rand(5000, 5000)\n",
    "if cuda:\n",
    "    mat_gpu = mat_gpu.cuda()\n",
    "mat_gpu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([5000, 5000])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mat_gpu.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 0 ns, sys: 3.51 ms, total: 3.51 ms\n",
      "Wall time: 2.98 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([[1670.1737, 1253.2634, 1260.9958,  ..., 1259.3188, 1263.1609,\n",
       "         1244.2560],\n",
       "        [1253.2634, 1665.1344, 1274.3370,  ..., 1264.2490, 1260.3347,\n",
       "         1248.6567],\n",
       "        [1260.9958, 1274.3370, 1683.3060,  ..., 1274.7039, 1263.1434,\n",
       "         1256.4707],\n",
       "        ...,\n",
       "        [1259.3188, 1264.2490, 1274.7039,  ..., 1682.9913, 1250.9548,\n",
       "         1264.6089],\n",
       "        [1263.1609, 1260.3347, 1263.1434,  ..., 1250.9548, 1685.4556,\n",
       "         1265.3738],\n",
       "        [1244.2560, 1248.6567, 1256.4707,  ..., 1264.6089, 1265.3738,\n",
       "         1682.0267]], device='cuda:0')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "torch.mm(mat_gpu.t(), mat_gpu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Can you see the speed-up? It will be much critical if we use larger matrices, more matrix computations, and a deeper neural network model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ~~Variable~~ Autograd\n",
    "\n",
    "**Variable wrapping is deprecated from PyTorch 0.4.** You can use just regular Tensor to use autograd functionality, and you can control the flag `requires_grad` in the Tensor. \n",
    "\n",
    "PyTorch provide a functionality of automatic differentiation with a package `autograd` and now `torch.Tensor` (instead of `Variable` in the old versions) is the key class for utilizing it.\n",
    "\n",
    "A Tensor keeps its value and the gradient with respect to this Tensor value. Also, almost all of built-in operations in PyTorch supports automatic differentiation. Therefore, we can call `.backward()` on a computation graph, e.g. neural network, after we finish our computation on the graph, then we can get automatically accumulated gradient for each Tensor (which has `requires_grad=True`) related with the graph.\n",
    "\n",
    "Let's try a simple example for easier understanding."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### old code using Variable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "from torch.autograd import Variable\n",
    "\n",
    "# Create some Tensors and a Variable\n",
    "x = Variable(torch.FloatTensor([2.0]), requires_grad=False)\n",
    "w = Variable(torch.FloatTensor([0.5]), requires_grad=True)\n",
    "b = Variable(torch.FloatTensor([0.1]), requires_grad=True)\n",
    "print(x)\n",
    "print(w)\n",
    "print(b)\n",
    "\n",
    "# Define a computational graph\n",
    "y = w*x + b # Currently, y = 0.5x + 0.1 and y(2) = 1.1\n",
    "print(y)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Current one with PyTorch >= 0.4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(2.)\n",
      "tensor(0.5000, requires_grad=True)\n",
      "tensor(0.1000, requires_grad=True)\n",
      "tensor(1.1000, grad_fn=<AddBackward0>)\n"
     ]
    }
   ],
   "source": [
    "# Create some Tensors\n",
    "x = torch.tensor(2.0, requires_grad=False)\n",
    "w = torch.tensor(0.5, requires_grad=True)\n",
    "b = torch.tensor(0.1, requires_grad=True)\n",
    "print(x)\n",
    "print(w)\n",
    "print(b)\n",
    "\n",
    "# Define a computational graph\n",
    "y = w*x + b # Currently, y = 0.5x + 0.1 and y(2) = 1.1\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's compute gradients on the graph y and print the gradient w.r.t each Variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n",
      "tensor(2.)\n",
      "tensor(1.)\n"
     ]
    }
   ],
   "source": [
    "# Compute gradients\n",
    "y.backward()\n",
    "\n",
    "print(x.grad)\n",
    "print(w.grad)\n",
    "print(b.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we set `requires_grad=False` for Tensor `x`, it has `None` value.\n",
    "Also, if we do a simple math to differentiate it manually, we can easily get:\n",
    "$$\n",
    "\\frac{\\partial y}{\\partial w} = \\frac{\\partial}{\\partial w}\\left(wx + b\\right) = x\\\\\n",
    "\\text{and}\\\\\n",
    "\\displaystyle \\frac{\\partial y}{\\partial w}\\Bigr|_{x=2} = 2 \n",
    "$$\n",
    "Similarly,\n",
    "$$\n",
    "\\frac{\\partial y}{\\partial b} = \\frac{\\partial}{\\partial b}\\left(wx + b\\right) = 1\\\\\n",
    "\\text{and}\\\\\n",
    "\\displaystyle \\frac{\\partial y}{\\partial b}\\Bigr|_{x=2} = 1 \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thanks to the functionality of automatic differentiation, we can build a very complex computational graph such as a neural network with many layers without manually computing the gradients of parameters.\n",
    "\n",
    "Please refer to the official [tutorial](https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html) for more details.\n",
    "\n",
    "In the next chapter, we will build a simple feed-forward neural network by using these components of PyTorch we have learnt."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:pytorch1]",
   "language": "python",
   "name": "conda-env-pytorch1-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}