diff --git a/preparatory_notebooks/F2_linear_regression.ipynb b/preparatory_notebooks/F2_linear_regression.ipynb new file mode 100644 index 0000000..e0fa5c7 --- /dev/null +++ b/preparatory_notebooks/F2_linear_regression.ipynb @@ -0,0 +1,469 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "czDam0lOe-Xx" + }, + "source": [ + "# Notebook: F2 -- Linear Regression\n", + "\n", + "This notebook is complementary to lecture F2 about linear regressoin in order to highlight its key concepts to refresh your knowledge and gain intuition. The focus will be one\n", + "1. **Generating data** for supervised machine learning problems\n", + "2. **Fit linear models** to this data\n", + "3. **Evaluate** the fitted models to see how it performs on new data\n", + "\n", + "Please read the instructions and play around with the notebook where it is described.\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "okNobqIurFRS" + }, + "source": [ + "---\n", + "\n", + "We start by importing necessary libraries. These libraries will be used throughout the course. If you are unfamiliar with them or need to refresh your knowledge, we recommended to take a look at the \"Introduction to Python\" material available on Studium." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "C8P18Lggc55a" + }, + "outputs": [], + "source": [ + "# Import necessary libraries\n", + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "from sklearn.model_selection import train_test_split" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Tn5lIINVc9bc" + }, + "source": [ + "---\n", + "\n", + "## 1. Data Generation\n", + "\n", + "The first step for a supervised machine learning problem is to **get a dataset**. Each input $x_i$ comes with a corresponding output or label $y_i$. Here, $i$ denotes the index of a particular sample, and we collect $n$ samples in total. Compactly, we denote our dataset as $\\mathcal{T} = \\{x_i, y_i\\}_{i=1}^{n}$.\n", + "\n", + "Now we:\n", + "1. Generate a synthetic dataset $\\mathcal{T}$.\n", + "2. Split the dataset into one train dataset and one test dataset. The train dataset will be used to fit a model to the data, and the test dataset will be used to evaluate our model. \n", + "\n", + "The **goal** of our supervised machine learning method is to find a model that performs well the unseen test data. So it is important to leave out a part of the data (the test dataset) from the training process to be able to evaluate how well our model will perform on new input datapoints $x$ in the future.\n", + "\n", + "Below, we have some helper function to generate synthetic data, split the data and then plot them. Skip over and go to the next box." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "HHjHft5Iga81" + }, + "outputs": [], + "source": [ + "# Generate synthetic data\n", + "def generate_synthetic_data():\n", + " np.random.seed(0)\n", + " X = np.random.rand(100, 1) # Feature (independent variable)\n", + " y = np.e * X + np.pi/2 + np.random.normal(0, 0.1, (100,1)) # Target (dependent variable)\n", + "\n", + " # Split the data into training and testing sets\n", + " X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n", + "\n", + "\n", + " # Print table header\n", + " print(\" | #x | #y |\")\n", + " print(\"-----------|-----|-----|\")\n", + " # Print train data row\n", + " print(f\"Train Data | {np.shape(X_train)[0]}{' ' * (3 - len(str(np.shape(X_train)[0])))} | {np.shape(y_train)[0]}{' ' * (3 - len(str(np.shape(y_train)[0])))} |\")\n", + " # Print test data row\n", + " print(f\"Test Data | {np.shape(X_test)[0]}{' ' * (3 - len(str(np.shape(X_test)[0])))} | {np.shape(y_test)[0]}{' ' * (3 - len(str(np.shape(y_test)[0])))} |\")\n", + " \n", + " return X_train, y_train, X_test, y_test\n", + "\n", + "# Plot the train data and test data\n", + "def plot_data():\n", + " plt.scatter(X_train, y_train, label='Training Data', alpha=0.5)\n", + " plt.scatter(X_test, y_test, label='Testing Data', alpha=0.5)\n", + " plt.xlabel('X')\n", + " plt.ylabel('y')\n", + " plt.legend()\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "W9oSQ59uhW7U" + }, + "source": [ + "Now we can generate our dataset and plot them to get an understanding of what our data looks like. We plot both our train data (in blue) and our test data (in orange). \n", + "\n", + "Task:\n", + "- Run the cell below to visualize the synthetic train- and test datasets.\n", + "- Check if the test data are representative of the train data.\n", + "- Is there some relationship between $x$ and $y$?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "AzuGzLTZfsrk" + }, + "outputs": [], + "source": [ + "# generate data\n", + "X_train,y_train, X_test, y_test = generate_synthetic_data()\n", + "\n", + "# plot the data\n", + "plot_data()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "HoYjDXdJPlwK" + }, + "source": [ + "## Explore the Family of Linear Models\n", + "From our plot, we notice that there seems to be a linear pattern: $y$ increases linearly with $x$. Hence, as model it might be suitable to use a **linear model** on the form:\n", + "\n", + "$$\n", + "y=θ_0+θ_1x + ϵ\n", + "$$\n", + "\n", + "We call $θ_0$ and $θ_1$ the **parameters** of our model, and $ϵ$ is a noise term capturing random errors in our data that our model does not account for.\n", + "\n", + "Finding a **good model**: This amount to fitting our model to the data. Meaning, finding good values of $θ_0$ and $θ_1$, so that $y_i\\approxθ_0+θ_1x_i$ holds for the samples in our training dataset $\\mathcal{T}_{train} = \\{x_i, y_i\\}_{i=1}^{m}$. Here, $m$ denotes the number of samples in our train set, i.e. $m=80$. \n", + "\n", + "Below is a helper function that plots the linear models. Skip over and go to the next box." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def plot_linear_models(\n", + " X, y, label='Training',\n", + " model1_params=[],\n", + " model2_params=[],\n", + " model3_params=[],\n", + "):\n", + " # model 1\n", + " if not all(element is None for element in model1_params):\n", + " y_model1 = model1_params[0] + X * model1_params[1]\n", + " plt.plot(X, y_model1, 'r', label='Model 1', alpha=0.5)\n", + " \n", + " # model 2\n", + " if not all(element is None for element in model2_params):\n", + " print('aaa')\n", + " y_model2 = model2_params[0] + X * model2_params[1]\n", + " plt.plot(X, y_model2, 'm', label='Model 2', alpha=0.5)\n", + " \n", + " # model 3\n", + " if not all(element is None for element in model3_params):\n", + " y_model3 = model3_params[0] + X * model3_params[1]\n", + " plt.plot(X, y_model3, 'g', label='Model 3', alpha=0.5)\n", + "\n", + " # Plot the training data\n", + " plt.scatter(X, y, label=label+' Data', alpha=0.5)\n", + " plt.xlabel('X')\n", + " plt.ylabel('y')\n", + " plt.legend()\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this section we want to find a good linear model. We plot the training data, as well as the linear models which are fully described by $θ_0$ and $θ_1$. If a model fits the data, we can use our parameters along with the inputs of the data (variable $\\mathtt{X\\_train}$) to calculate predicted y-values which are close to the true y-values.\n", + "\n", + "Tasks:\n", + "\n", + "1. Run the code below and visualize model 1 with the given parameters. Does it fit the data?\n", + "2. Try to optimize the parameters of model 2 and model 3 to obtain better fits to the data. Replace the $\\mathtt{None}$ values with what you think are better parameters.\n", + "3. Which set of parameters fit the data best? \n", + "4. What does $θ_0$ and $θ_1$ stand for?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# model 1:\n", + "theta0 = 3\n", + "theta1 = -4\n", + "model1_params = [theta0, theta1]\n", + "\n", + "# model 2\n", + "theta0 = None\n", + "theta1 = None\n", + "model2_params = [theta0, theta1]\n", + "\n", + "\n", + "# model 3:\n", + "theta0 = None\n", + "theta1 = None\n", + "model3_params = [theta0, theta1]\n", + "\n", + "# plot model fits\n", + "plot_linear_models(X_train, y_train, 'Training',\n", + " model1_params, model2_params, model3_params)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "o7GqnLWy7dm0" + }, + "source": [ + "---\n", + "\n", + "## 2. Model evaluation\n", + "\n", + "Above you visually fit the linear model to the data. But how can we determine quantitatively which model is better? A common metric is the mean squared error (MSE):\n", + "\n", + "$$\n", + "\\frac{1}{m} \\sum_{i=1}^{m} {(y_i - f_{\\theta}(x_i))}^2\n", + "$$\n", + "\n", + "Here, $y_i$ denotes the true value for each input $x_i$ in the train dataset, and $f_{\\theta}(x_i) = \\theta_0 + \\theta_{1}x_i$ is the output of the model parameterized by our particular choice of $\\theta_0$ and $\\theta_1$.\n", + "\n", + "Below is a helper function which compute model predictions and the mean squared error of that model on the given data points. Skip over and go to the next box." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def model_prediction(X, model_params):\n", + " return model_params[0] + X * model_params[1]\n", + "\n", + "def MSE(y, pred):\n", + " m = len(y)\n", + " return np.sum((y - pred)**2)/m" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Following there are two code cells which perform the following:\n", + "- Compare the MSE of the three models on the **train dataset**.\n", + "- Compare the MSE of the three models on the **test dataset** and plot the function with the test data.\n", + "\n", + "Tasks:\n", + "1. Do you want to minimize or maximise MSE?\n", + "2. Given the train MSE, which model would you choose? Does this align with you visual impression from above?\n", + "3. Does the model generalize to unseen test data? Or more specifically: Does the MSE on train data and test data match?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "WNDAkUdSnaPz" + }, + "outputs": [], + "source": [ + "print('MSE on train data')\n", + "\n", + "pred1 = model_prediction(X_train, model1_params)\n", + "mse1 = MSE(y_train, pred1)\n", + "print(f'Model 1: {mse1:.3f}')\n", + "\n", + "pred2 = model_prediction(X_train, model2_params)\n", + "mse2 = MSE(y_train, pred2)\n", + "print(f'Model 2: {mse2:.3f}')\n", + "\n", + "pred3 = model_prediction(X_train, model3_params)\n", + "mse3 = MSE(y_train, pred3)\n", + "print(f'Model 3: {mse3:.3f}')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "HXPpgsGvpJbk" + }, + "outputs": [], + "source": [ + "plot_linear_models(X_test, y_test, 'Test',\n", + " model1_params, model2_params, model3_params)\n", + "\n", + "print('MSE on test data')\n", + "\n", + "pred1 = model_prediction(X_test, model1_params)\n", + "mse1 = MSE(y_test, pred1)\n", + "print(f'Model 1: {mse1:.3f}')\n", + "\n", + "pred2 = model_prediction(X_test, model2_params)\n", + "mse2 = MSE(y_test, pred2)\n", + "print(f'Model 2: {mse2:.3f}')\n", + "\n", + "pred3 = model_prediction(X_test, model3_params)\n", + "mse3 = MSE(y_test, pred3)\n", + "print(f'Model 3: {mse3:.3f}')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ET6GtSDwrXGm" + }, + "source": [ + "---\n", + "\n", + "## 3. Finding the \"optimal\" linear model:\n", + "\n", + "Now we can not just fit a model by visual inspection but also select a model quantitatively by it's lowest MSE.\n", + "\n", + "But is there a **systematic way** to select the model parameters $\\theta_0$ and $\\theta_1$? We define the \"best possible linear model\" as the model generating the smallest MSE. Finding the model parameters that minimize the MSE is equivalent to finding the parameters that minimize the squared L2-norm of the residual vector. Thus, to find the best linear model we want to solve the following optimization problem with respect to $\\theta=[\\theta_0, \\theta_1]^\\top$:\n", + "\n", + "$$\n", + "\\hat{\\mathbf{\\theta}} = \\text{arg}\\min_{\\mathbf{\\theta}} \\frac{1}{m} \\sum_{i=1}^{m} {(y_i - f_{\\theta}(x_i))}^2 = \\text{arg}\\min_{\\mathbf{\\theta}} ||{(\\mathbf{y} - \\mathbf{X}\\mathbf{\\theta})}||_2^2\n", + "$$\n", + "\n", + "where\n", + "\\begin{align*}\n", + "\\mathbf{y} &= \\begin{bmatrix}\n", + " y_1 \\\\\n", + " y_2 \\\\\n", + " \\vdots\\\\\n", + " y_m\n", + "\\end{bmatrix}\n", + "& \\mathbf{X} &= \\begin{bmatrix}\n", + " 1 & x_1 \\\\\n", + " 1 & x_2 \\\\\n", + " \\vdots & \\vdots \\\\\n", + " 1 & x_m\n", + "\\end{bmatrix}\n", + "& \\mathbf{θ} &= \\begin{bmatrix}\n", + " θ_0 \\\\\n", + " θ_1 \\\\\n", + "\\end{bmatrix}\n", + "\\end{align*}\n", + "\n", + "We use $\\hat{\\mathbf{\\theta}}$ to denote our estimates of the true parameters.The solution to this optimization problem finds the least squares solution $\\hat{\\mathbf{\\theta}}$ to the following (overdetermined) linear system of equations:\n", + "\n", + "$$\n", + "\\mathbf{y}=\\mathbf{X}\\mathbf{\\theta}\n", + "$$\n", + "\n", + "\n", + "We say that we find the solution that minimizes the **least squares cost**. So when we say that our model is the **optimal** linear model, we mean that it is optimal in a least squares sense given the data.\n", + "\n", + "When working with linear models, the optimization problem above has a closed-form solution that can be found by solving the normal equations for $\\hat{\\mathbf{\\theta}}$:\n", + "\n", + "$$\n", + "\\mathbf{X}^T\\mathbf{X}\\hat{\\mathbf{\\theta}}=\\mathbf{X}^T\\mathbf{y}\n", + "$$\n", + "\n", + "In the cell below, we solve the normal equations using our train data:\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "cVJhyY80Ie1K" + }, + "outputs": [], + "source": [ + "# construct the matrix X\n", + "n = len(X_train) # number of samples in our training data\n", + "X = np.ones((n, 2))\n", + "X[:,1] = X_train[:,0]\n", + "\n", + "# solve the normal equations\n", + "theta_ls = np.linalg.inv(X.T@X)@(X.T@y_train)\n", + "print('Least Squares Solution:')\n", + "print(f'theta_0 = {theta_ls[0][0]}')\n", + "print(f'theta_1 = {theta_ls[1][0]}')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "aFfFDLDZJ9o8" + }, + "source": [ + "Task:\n", + "1. Compare the optimal linear model with the best one that you found\n", + "2. What is the train and test MSE of this optimal model?" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "gu5bHt0zMOa9" + }, + "source": [ + "---\n", + "\n", + "# Take-home message\n", + "\n", + "\n", + "* Collect a dataset and split the dataset into a train set (for training your model) and a test set (for evaluating your model).\n", + "* Define the model $f_{\\theta}(x)$ you want to fit to the data. In the case of linear regression, we let $f_{\\theta}(x)$ be the family of linear models parameterized by $\\theta$.\n", + "* Choose an error metric and set up an optimization problem to find the optimal parameters $\\theta$. Here, we choose the MSE.\n", + "* Solve the optimization problem using the train dataset.\n", + "* Evaluate your model on the test dataset.\n", + "\n", + "**Recommendation for further reading:** The material covered in this notebook is well-covered in the beginning of Chapter 3.1 in the course book.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.3" + } + }, + "nbformat": 4, + "nbformat_minor": 1 +} diff --git a/preparatory_notebooks/F3_logistic_regression.ipynb b/preparatory_notebooks/F3_logistic_regression.ipynb new file mode 100644 index 0000000..413d3ff --- /dev/null +++ b/preparatory_notebooks/F3_logistic_regression.ipynb @@ -0,0 +1,351 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Notebook: F3 -- Logistic Regression\n", + "\n", + "This notebook is complementary to lecture F3 about Logistic Regression in order to highlight the key concepts. The focus will be on\n", + "1. Understanding and visualizing different loss functions: **Misclassification** and **Logistic Loss**\n", + "2. A basic classifier and its **Misclassification Loss** and modifying the parameters to see the effects on the loss.\n", + "3. Finally, the same classifier with its **Logistic Loss** and visualizing the loss surface.\n", + "\n", + "Please read the instructions and play around with the notebook where it is described." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pip install -q ipywidgets" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# imports necessary libraries\n", + "%matplotlib inline\n", + "\n", + "import scipy.stats\n", + "import numpy as np\n", + "import matplotlib\n", + "import matplotlib.pyplot as plt\n", + "from matplotlib import cm\n", + "from mpl_toolkits.mplot3d import Axes3D\n", + "from ipywidgets import interact, widgets\n", + "\n", + "np.random.seed(42) # fix the random seed" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "\n", + "## 1. Loss Functions: Misclassification and Logistic Loss\n", + "\n", + "In this section we will look at two loss functions: the misclassification loss and the logistic loss.\n", + "\n", + "The **misclassification loss** is defined as\n", + "\n", + "$$\n", + "\\ell_{\\text{misclass}}(y, \\hat{y}) = \\begin{cases}\n", + "0 & y\\hat{y} \\ge 0 \\\\\n", + "1 & y\\hat{y} \\lt 0\n", + "\\end{cases}\n", + "$$\n", + "\n", + "where $y$ is the true label and $\\hat{y}$ is the predicted label.\n", + "\n", + "Moreover, the **logistic loss** is defined as\n", + "\n", + "$$\n", + "\\ell_{\\text{logistic}}(y, \\hat{y}) = \\ln(1 + \\exp(-y\\hat{y}))\n", + "$$\n", + "\n", + "where $y$ is the true label and $\\hat{y}$ is the predicted label. \n", + "\n", + "The logistic loss is basically a *continuous* approximation to the misclassification loss, taking into account also **how far away** our predictions are from the real labels.\n", + "\n", + "Below, we have some helper functions for visualizing each of these loss functions. Skip over and go to the next box." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def plot_misclassification_loss():\n", + " yyhat = np.linspace(-5, 5, 100)\n", + " loss = np.where(yyhat < 0, 1, 0)\n", + " plt.figure(figsize=(5, 3))\n", + " plt.plot(yyhat, loss)\n", + " plt.xlabel('$y \\cdot \\hat{y}$')\n", + " plt.ylabel('Misclassification Loss')\n", + " plt.show()\n", + "\n", + "def plot_logistic_loss():\n", + " yyhat = np.linspace(-5, 5, 100)\n", + " loss = np.log(1 + np.exp(-yyhat))\n", + " plt.figure(figsize=(5, 3))\n", + " plt.plot(yyhat, loss, c='orange') # try also with semilogy\n", + " plt.xlabel('$y \\cdot \\hat{y}$')\n", + " plt.ylabel('Logistic Loss')\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Below, we visualize each of the loss functions in a 2D plot. The x-axis is the product of the real value $y$ and the predicted one $\\hat{y}$, and the y-axis is the loss $\\ell(y, \\hat{y})$." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Plot the misclassification loss\n", + "plot_misclassification_loss()\n", + "\n", + "# Plot the logistic loss\n", + "plot_logistic_loss()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "\n", + "## 2. A basic classifier + the Misclassification Loss\n", + "\n", + "In this section, we will look at a basic classifier and its misclassification loss. We will also modify the parameters to see the effects on the loss and the decision boundary.\n", + "\n", + "We model our basic classifier as follows:\n", + "\n", + "$$\n", + "\\hat{y} = \\text{sign}(\\theta_1 x_1 + \\theta_2 x_2)\n", + "$$\n", + "\n", + "where $\\theta_1$ and $\\theta_2$ are the weights. If we consider $\\theta= [\\theta_1, \\theta_2]$, then we can rewrite the above equation in a more compact form:\n", + "\n", + "$$\n", + "\\hat{y} = \\text{sign}(\\theta^T x)\n", + "$$\n", + "\n", + "Using this model, we can compute the average misclassification loss given a set of parameters $\\theta$. This will be our cost function:\n", + "\n", + "$$\n", + "J_{\\text{misclass}}(w) = \\frac{1}{N} \\sum_{i=1}^N \\ell_{\\text{misclass}}(y_i, \\hat{y}_i ; \\theta)\n", + "$$\n", + "\n", + "where $N$ is the number of samples in the dataset.\n", + "\n", + "Below we generate our dataset and there are some helper functions to visualize the decision boundary and calculate the misclassification loss. Skip over and go to the next box." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# generate synthetic dataset\n", + "x = np.random.rand(100, 2) * 4\n", + "y = np.where(x[:, 1] > x[:, 0], 1, -1)\n", + "\n", + "def calculate_misclassification_cost(x, y, theta):\n", + " return np.sum(np.where(y * np.dot(x, theta) < 0, 1, 0)) / len(y)\n", + "\n", + "def plot_decision_boundary(x, y, theta):\n", + " plt.figure(figsize=(5, 3))\n", + " plt.scatter(x[:, 0], x[:, 1], c=y, cmap=cm.coolwarm)\n", + " x1 = np.linspace(0, 4, 100)\n", + " x2 = -theta[0] / theta[1] * x1\n", + " plt.plot(x1 , x2, c='black')\n", + "\n", + " mesh = np.meshgrid(np.linspace(0, 4, 100),\n", + " np.linspace(0, 4, 100))\n", + "\n", + " Z = np.sign(np.dot(np.c_[mesh[0].ravel(), mesh[1].ravel()], theta))\n", + " Z = Z.reshape(mesh[0].shape)\n", + " plt.pcolormesh(mesh[0], mesh[1], Z, cmap=cm.coolwarm, alpha=0.2)\n", + "\n", + " plt.xlim([0, 4])\n", + " plt.ylim([0, 4])\n", + " plt.xlabel('$x_1$')\n", + " plt.ylabel('$x_2$')\n", + " plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Below, we plot the decision boundary for a classifier with the initialized $w$ parameters, alongside our data points which are colored according to their true label. Moreover, the misclassification cost is also calculated for the classifier and printed.\n", + "\n", + "Tasks:\n", + "1. Play around with the parameters $\\theta_1$ and $\\theta_2$ to: \n", + " - Observe how the decision boundary changes. \n", + " - Observe how the misclassification cost changes. \n", + "2. Try to minimize the cost by changing $\\theta_1$ and $\\theta_2$ in order to separate the data points as best as possible." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Our initial weight vector, i.e. theta1 and theta2\n", + "theta = [1, -0.3]\n", + "\n", + "# Plot the decision boundary\n", + "plot_decision_boundary(x, y, theta)\n", + "\n", + "# Calculate the misclassification cost\n", + "print(\"The misclassification rate: \", calculate_misclassification_cost(x, y, theta))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "\n", + "## 2. The same classifier + the Logistic Loss\n", + "\n", + "In this section, we will look at the same classifier as before, but this time we will use the logistic loss instead of the misclassification loss. We will also visualize the loss surface in addition to the decision boundary.\n", + "\n", + "Remembering our definition of the logistic loss, we can compute the average logistic loss given a set of parameters $w$. This will be our cost function:\n", + "\n", + "$$\n", + "J_{\\text{logistic}}(w) = \\frac{1}{N} \\sum_{i=1}^N \\ell_{\\text{logistic}}(y_i, \\hat{y}_i ; \\theta)\n", + "$$\n", + "\n", + "where $N$ is the number of samples in the dataset.\n", + "\n", + "Below are some helper functions to calculate and visualize the logistic loss function. Skip over and go to the next box." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def calculate_logistic_cost(x, y, theta):\n", + " return np.sum(np.log(1 + np.exp(-y * np.dot(x, theta)))) / len(y)\n", + "\n", + "def plot_logistic_loss(x, y, azimuth, elevation):\n", + " theta1 = np.linspace(-1, 1, 50)\n", + " theta2 = np.linspace(-1, 1, 50)\n", + " theta1, theta2 = np.meshgrid(theta1, theta2)\n", + " loss = np.zeros(theta1.shape)\n", + " for i in range(len(theta1)):\n", + " for j in range(len(theta2)):\n", + " theta = [theta1[i, j], theta2[i, j]]\n", + " loss[i, j] = calculate_logistic_cost(x, y, theta)\n", + " fig = plt.figure(figsize=(8, 6))\n", + " ax = fig.add_subplot(projection='3d')\n", + " ax.plot_surface(theta1, theta2, loss, cmap=cm.viridis)\n", + " ax.set_xlabel(r'$\\theta_1$')\n", + " ax.set_ylabel(r'$\\theta_2$')\n", + " ax.set_zlabel('Logistic Loss')\n", + " ax.view_init(elevation, azimuth)\n", + " ax.tick_params(axis='x', which='major', pad=3)\n", + " ax.tick_params(axis='y', which='major', pad=3)\n", + " ax.set_xticks(np.linspace(-1, 1, 5))\n", + " ax.set_yticks(np.linspace(-1, 1, 5))\n", + " plt.show()\n", + "\n", + "def plot_log_loss_interactive(x, y):\n", + " interact(plot_logistic_loss, x=widgets.fixed(x), y=widgets.fixed(y), \n", + " azimuth=widgets.FloatSlider(min=0, max=360, step=10, value=0), \n", + " elevation=widgets.FloatSlider(min=0, max=90, step=10, value=20))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Below, we again draw the decision boundary of our classifier for the same dataset. However, this time we calculate and print the logistic loss instead of the misclassification loss. Moreover, the loss surface is also plotted, where you can see how the loss changes for different values of $w_1$ and $w_2$, for this specific dataset.\n", + "\n", + "Task:\n", + "1. Try again to minimize the cost by changing $\\theta_1$ and $\\theta_2$ in order to separate the data points as best as possible. Note how the best decision boundary does not yield a cost of 0, but rather a small value now. What does this mean for the classifier?\n", + "2. Inspect the loss surface and see how the loss changes for different values of $\\theta_1$ and $\\theta_2$. What parameters yield the lowest loss? Is it the same as the one you found?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Our initial weight vector, i.e. theta1 and theta2\n", + "theta = [1, -0.3]\n", + "\n", + "# Plot the decision boundary\n", + "plot_decision_boundary(x, y, theta)\n", + "print(\"The logistic loss: \", calculate_logistic_cost(x, y, theta))\n", + "\n", + "# Plot the logistic loss\n", + "plot_log_loss_interactive(x, y)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "\n", + "# Take-home message\n", + "\n", + "* Logistic regression is used for classification problems.\n", + "* Logisitc regression is a linear model with a certain decision boundary.\n", + "* We can use the misclassification loss or the logistic loss. The latter gives a better notion of the distance of a sample to the decision boundary.\n", + "\n", + "**Recommendation for further reading:** The material covered in this notebook is well-covered in the beginning of Chapter 3.2 in the course book." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.3" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/preparatory_notebooks/F4_lda_qda.ipynb b/preparatory_notebooks/F4_lda_qda.ipynb new file mode 100644 index 0000000..257b3d2 --- /dev/null +++ b/preparatory_notebooks/F4_lda_qda.ipynb @@ -0,0 +1,783 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "wZut3XSZ0ryK" + }, + "source": [ + "# Notebook: F4 -- LDA, QDA\n", + "\n", + "This notebook is complementary to lecture F4 about LDA/QDA in order to highlight its key concepts. The focus will be on\n", + "1. Visualizing **multivariate Gaussian distributions**\n", + "2. **LDA**: Fitting Gaussian with the same covariance to data\n", + "3. **QDA**: Fitting Gaussian with varying covariance to data\n", + "\n", + "Please read the instructions and play around with the notebook where it is described." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "6B-KxYey_GUo", + "outputId": "85d9ce60-cdc4-45fe-b569-b1cd61eaa4f1" + }, + "outputs": [], + "source": [ + "pip install -q ipywidgets" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "NgfYHQxpoPvo" + }, + "outputs": [], + "source": [ + "# imports necessary libraries\n", + "%matplotlib inline\n", + "\n", + "import scipy.stats\n", + "import numpy as np\n", + "import matplotlib\n", + "import matplotlib.pyplot as plt\n", + "from matplotlib import cm\n", + "from mpl_toolkits.mplot3d import Axes3D\n", + "from ipywidgets import interact, widgets\n", + "\n", + "np.random.seed(42) # fix the random seed" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "YghGdL_lD0XT" + }, + "source": [ + "---\n", + "\n", + "## 1. Multivariate Gaussians\n", + "\n", + "A multivariate gaussian distribution for $k$ dimensions is given by the following probability density function (PDF):\n", + "\n", + "$f(x) = \\frac{1}{\\sqrt{(2 \\pi)^k \\det \\Sigma}}\\exp\\left( -\\frac{1}{2} (x - \\mu)^T \\Sigma^{-1} (x - \\mu) \\right)$\n", + "\n", + "with $\\Sigma$ as the covariance matrix and $\\mu$ as the mean vector.\n", + "Here, we will investigate how such PDFs look and what the influence of $\\mu$, $\\Sigma$ is.\n", + "\n", + "Below are helper functions to plot the results. Skip over those and go to the next text box." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "cMXLIHQ9_W04" + }, + "outputs": [], + "source": [ + "def plot_multivariate_gaussian(mu,Sigma,elevation=0, azimuth=90):\n", + " N = 100\n", + " X = np.linspace(-5, 5, N)\n", + " Y = np.linspace(-5, 5, N)\n", + " X, Y = np.meshgrid(X, Y)\n", + "\n", + " pos = np.empty(X.shape + (2,))\n", + " pos[:, :, 0] = X\n", + " pos[:, :, 1] = Y\n", + "\n", + " Z = scipy.stats.multivariate_normal.pdf(pos, mu, Sigma)\n", + "\n", + " fig = plt.figure()\n", + " ax = fig.add_subplot(projection='3d')\n", + " ax.plot_surface(X, Y, Z, rstride=3, cstride=3, linewidth=1, antialiased=True,\n", + " cmap=cm.viridis)\n", + " cset = ax.contourf(X, Y, Z, zdir='z', offset=-0.15, cmap=cm.viridis)\n", + " ax.set_xlabel('x1')\n", + " ax.set_ylabel('x2')\n", + " ax.set_title('Multivariate Gaussians')\n", + "\n", + " ax.set_zlim(-0.15,0.1)\n", + " ax.set_zticks(np.linspace(0,0.1,5))\n", + " ax.view_init(elev=elevation,azim=azimuth)\n", + "\n", + " plt.show()\n", + "\n", + "def plot_multivariate_gaussian_interactive(mu,Sigma, initial_elevation=90, initial_azimuth=0):\n", + " assert mu.shape == (2,), 'mu must be of shape (2,)'\n", + " assert Sigma.shape == (2,2), 'Sigma must be of shape (2,2)'\n", + " assert np.allclose(Sigma, Sigma.T), 'Sigma must be symmetric'\n", + " assert np.all(np.linalg.eigvals(Sigma) > 0), 'Sigma must be positive definite'\n", + "\n", + " interact(plot_multivariate_gaussian,\n", + " mu=widgets.fixed(mu),\n", + " Sigma=widgets.fixed(Sigma),\n", + " elevation=widgets.FloatSlider(min=0, max=90, step=1, value=initial_elevation),\n", + " azimuth=widgets.FloatSlider(min=0, max=360, step=1, value=initial_azimuth)\n", + " )\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lqzuMaB12gmj" + }, + "source": [ + "Below, we plot a multivariate Gaussian for a specific mean vector $\\mu$ and covariance matrix $\\Sigma$. We visualize both, the PDF and a contour plot below.\n", + "\n", + "Tasks:\n", + "1. Change the elevation/azimuth using the sliders to familiarize yourself with the PDF and contour plot. Note: You first have to run all code cells until here to change the view.\n", + "2. Understand the effect of the mean vector $\\mu$: change $\\texttt{mu}$ and observe the change in the PDF\n", + "3. Understand the effect of the covariance matrix $\\Sigma$: change $\\texttt{Sigma}$ and observe the change. Some ideas:\n", + "- What happens with a diagonal $\\Sigma$?\n", + "- What happens if $\\Sigma_{11}>\\Sigma_{22}$ or vice versa?\n", + "- What happens when you add/remove off diagonal values? Note $\\Sigma$ has to be symmetric $\\Sigma_{12}=\\Sigma_{21}$." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 492, + "referenced_widgets": [ + "eb475a8cde92418d97e8604866d0a587", + "0632b737574b46d1968bd198902f0c35", + "1463125f5dd04e80a394067acb5f0462", + "4030a6224a1f4e629c0b8c764f68c80b", + "57980998a5874ce19178b2464c67584d", + "97184c81f3674bcc9d5d9146cfebbe01", + "5b57540dd7284c82a69e1282eda41ce0", + "924cd572d4cc4e9e9751cf4f82ab349c", + "288b525666bf40e7aa241b1a32d249a6", + "083a18e7851d43909c146404ec6f253d" + ] + }, + "id": "TOvRWhgkoUHm", + "outputId": "958de684-2fd3-48b5-c98e-f77cdc7a53c6" + }, + "outputs": [], + "source": [ + "# mean mu is a vector of size 2\n", + "mu = np.array([0., 0.])\n", + "# covariance Sigma is a matrix of size 2x2\n", + "Sigma = np.array([[1. , 0.5],\n", + " [0.5 , 1.]])\n", + "\n", + "# plot the multivariate gaussian\n", + "plot_multivariate_gaussian_interactive(\n", + " mu,Sigma,\n", + " initial_elevation=90, initial_azimuth=90,\n", + " )" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "V7wN-NU-D9VP" + }, + "source": [ + "\n", + "\n", + "---\n", + "\n", + "\n", + "## 2. Linear discriminant analysis (LDA)\n", + "\n", + "In linear discriminant analysis we fit $m$ Gaussians with the same covariance matrix to data with $m$ classes. Here we focus on $m=2$, the binary case.\n", + "\n", + "Below is a helper function to plot the results. Skip over it and go to the next text box." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "dbx8Vo4iBEsf" + }, + "outputs": [], + "source": [ + "def plot_lda(Sigma=None):\n", + " N = 150\n", + " limit = 4.5\n", + " X = np.linspace(-limit, limit, N)\n", + " Y = np.linspace(-limit, limit, N)\n", + " X, Y = np.meshgrid(X, Y)\n", + "\n", + " pos = np.empty(X.shape + (2,))\n", + " pos[:, :, 0] = X\n", + " pos[:, :, 1] = Y\n", + "\n", + " pi = np.array([0.5, 0.5])\n", + " mu_true = np.array([[-1., -1.],\n", + " [1., 1.]])\n", + " Sigma_true = np.array([[[ 1. , -0.7], [-0.7, 1.]],\n", + " [[ 1. , 0.8], [ 0.8, 1.3]]])\n", + "\n", + "\n", + " fig = plt.figure()\n", + " ax = fig.add_subplot()\n", + " if Sigma is not None:\n", + " assert Sigma.shape == (2,2), 'Sigma must be of shape (2,2)'\n", + " assert np.allclose(Sigma, Sigma.T), 'Sigma must be symmetric'\n", + " assert np.all(np.linalg.eigvals(Sigma) > 0), 'Sigma must be positive definite'\n", + " Z = pi[0]*scipy.stats.multivariate_normal.pdf(pos, mu_true[0], Sigma) + \\\n", + " pi[1]*scipy.stats.multivariate_normal.pdf(pos, mu_true[1], Sigma)\n", + " cset = ax.contourf(X, Y, Z, cmap=cm.viridis)\n", + "\n", + " s1 = scipy.stats.multivariate_normal.rvs(mu_true[0], Sigma_true[0], int(0.6*N), random_state=42)\n", + " s2 = scipy.stats.multivariate_normal.rvs(mu_true[1], Sigma_true[1], int(0.4*N), random_state=42)\n", + "\n", + " ax.scatter(s1[:, 0], s1[:, 1], marker='x', color='red')\n", + " ax.scatter(s2[:, 0], s2[:, 1], marker='o', facecolors='none', edgecolors='blue')\n", + "\n", + " ax.set_xlabel('x1')\n", + " ax.set_ylabel('x2')\n", + " ax.set_title('Linear discriminant analysis')\n", + " ax.set_aspect('equal')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vprBT1r44y-g" + }, + "source": [ + "Below we have some data for two classes: red 'x' and blue 'o' represent each class. Tasks:\n", + "1. View the data by running the code cell below\n", + "2. Comment line 5 and run the cell with line 8 instead. You will see that two Gaussians with the same covariance specified by $\\texttt{Sigma}$ are fit to the two data clusters.\n", + "3. Try to modify $\\texttt{Sigma}$ such that the Gaussians fit the data as good as possible.\n", + "\n", + "LDA will use the two Gaussians and then add a **linear** decision boundary between the two fitted Gaussians to classify the data." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 507 + }, + "id": "iCqd-5GdMXsz", + "outputId": "79d6f711-b9ed-4280-9b18-df39504e0e42" + }, + "outputs": [], + "source": [ + "Sigma = np.array([[1., 0.],\n", + " [0., 1.]])\n", + "\n", + "# First: plot the data with the following line to visualize it\n", + "# plot_lda(Sigma=None)\n", + "\n", + "# Second: comment the line above and replace with the one below to fit your Sigma\n", + "plot_lda(Sigma=Sigma)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "xz7LXT-fS_IS" + }, + "source": [ + "---\n", + "\n", + "## 3. Quadratic discriminant analysis (QDA)\n", + "\n", + "\n", + "In quadratic discriminant analysis we extend LDA such that we use Gaussians with different covariance matrices $\\Sigma$.\n", + "\n", + "Below is a helper function to plot the results. Skip over it and go to the next text box." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "jk2-BR2CNA8K" + }, + "outputs": [], + "source": [ + "def plot_qda(mu, Sigma):\n", + " assert mu.shape == (2,), 'mu must be of shape (2,)'\n", + " assert Sigma.shape == (2,2), 'Sigma must be of shape (2,2)'\n", + " assert np.allclose(Sigma, Sigma.T), 'Sigma must be symmetric'\n", + " assert np.all(np.linalg.eigvals(Sigma) > 0), 'Sigma must be positive definite'\n", + "\n", + " N = 150\n", + " limit = 4.5\n", + " X = np.linspace(-limit, limit, N)\n", + " Y = np.linspace(-limit, limit, N)\n", + " X, Y = np.meshgrid(X, Y)\n", + "\n", + " pos = np.empty(X.shape + (2,))\n", + " pos[:, :, 0] = X\n", + " pos[:, :, 1] = Y\n", + "\n", + " pi = np.array([0.5, 0.5])\n", + " mu_true = np.array([[-1., -1.],\n", + " [1., 1.]])\n", + " Sigma_true = np.array([[[ 1. , -0.7], [-0.7, 1.]],\n", + " [[ 1. , 0.8], [ 0.8, 1.3]]])\n", + "\n", + "\n", + " Z = pi[0]*scipy.stats.multivariate_normal.pdf(pos, mu, Sigma) + \\\n", + " pi[1]*scipy.stats.multivariate_normal.pdf(pos, mu_true[1], Sigma_true[1])\n", + "\n", + " fig = plt.figure()\n", + " ax = fig.add_subplot()\n", + " cset = ax.contourf(X, Y, Z, cmap=cm.viridis)\n", + "\n", + " s1 = scipy.stats.multivariate_normal.rvs(mu_true[0], Sigma_true[0], int(0.6*N), random_state=42)\n", + " s2 = scipy.stats.multivariate_normal.rvs(mu_true[1], Sigma_true[1], int(0.4*N), random_state=42)\n", + "\n", + " ax.scatter(s1[:, 0], s1[:, 1], marker='x', color='red')\n", + " ax.scatter(s2[:, 0], s2[:, 1], marker='o', facecolors='none', edgecolors='blue')\n", + "\n", + " ax.set_xlabel('x1')\n", + " ax.set_ylabel('x2')\n", + " ax.set_title('Linear discriminant analysis')\n", + " ax.set_aspect('equal')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "eC7EcI4_6EOs" + }, + "source": [ + "We use the same data as for LDA above. Now the Gaussian for the blue class is well fit already. Your task:\n", + "1. Change the mean $\\texttt{mu}$ to place the second Gaussian well for the red class.\n", + "2. Change the covariance $\\texttt{Sigma}$ such that the second gaussian fits the red class well.\n", + "\n", + "QDA will use the two Gaussians and then add a **quadratic** decision boundary between the two fitted Gaussians to classify the data. It is therefore more flexible than LDA." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 507 + }, + "id": "5YxMtvb8UZQw", + "outputId": "bac19a34-5abc-4841-aab3-cbfa40a3aa0c" + }, + "outputs": [], + "source": [ + "mu = np.array([-3., -3.])\n", + "Sigma = np.array([[1., 0.],\n", + " [0., 1.]])\n", + "\n", + "\n", + "plot_qda(mu, Sigma)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "\n", + "# Take-home message\n", + "\n", + "* Multivariate Gaussians are determined by the mean vector and a symmetric, positive-definite covariance matrix. \n", + "* Understand how the mean vector and covariance matrix influene the shapre of the Gaussian\n", + "* LDA fits a Gaussian with the same covariance to each class to the data.\n", + "* QDA fits a separate Gaussian (with different covariance) for each class to the data.\n", + "* Both, LDA and QDA are generative models since we can in principle sample from the fitted Gaussians to obtain new samples.\n", + "\n", + "**Recommendation for further reading:** The material covered in this notebook is well-covered in the beginning of Chapter 10.1 in the course book." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.3" + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "0632b737574b46d1968bd198902f0c35": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatSliderModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatSliderModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "FloatSliderView", + "continuous_update": true, + "description": "elevation", + "description_tooltip": null, + "disabled": false, + "layout": "IPY_MODEL_97184c81f3674bcc9d5d9146cfebbe01", + "max": 90, + "min": 0, + "orientation": "horizontal", + "readout": true, + "readout_format": ".2f", + "step": 1, + "style": "IPY_MODEL_5b57540dd7284c82a69e1282eda41ce0", + "value": 90 + } + }, + "083a18e7851d43909c146404ec6f253d": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "1463125f5dd04e80a394067acb5f0462": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatSliderModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatSliderModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "FloatSliderView", + "continuous_update": true, + "description": "azimuth", + "description_tooltip": null, + "disabled": false, + "layout": "IPY_MODEL_924cd572d4cc4e9e9751cf4f82ab349c", + "max": 360, + "min": 0, + "orientation": "horizontal", + "readout": true, + "readout_format": ".2f", + "step": 1, + "style": "IPY_MODEL_288b525666bf40e7aa241b1a32d249a6", + "value": 91 + } + }, + "288b525666bf40e7aa241b1a32d249a6": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "SliderStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "SliderStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "", + "handle_color": null + } + }, + "4030a6224a1f4e629c0b8c764f68c80b": { + "model_module": "@jupyter-widgets/output", + "model_module_version": "1.0.0", + "model_name": "OutputModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/output", + "_model_module_version": "1.0.0", + "_model_name": "OutputModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/output", + "_view_module_version": "1.0.0", + "_view_name": "OutputView", + "layout": "IPY_MODEL_083a18e7851d43909c146404ec6f253d", + "msg_id": "", + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAGbCAYAAAAr/4yjAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABObElEQVR4nO3deZwU1bk//s+p6m32YRkGkZ0IqIAaCKiogCCg4r2iIkZUQEWjYDSa5LpFE01EE64aucb1grlGFPEbJRo3giBGIfJTUcGAgIDjsA3b7DPdXXV+f/R00T19TkOPPVM9M5/36zXK1Pr0MufpqvP0OUJKKUFERATAcDsAIiLKHEwKRETkYFIgIiIHkwIRETmYFIiIyMGkQEREDiYFIiJyMCkQEZGDSYGIiBxMCu3Ur3/9awghjmrb5557DkIIbN++vXmD0ujduzdmzJjhyrnbuu3bt0MIgeeee87tUChDMClkoGgjLITAP//5z4T1Ukr06NEDQghMmjQpbed94IEH8Nprr6XteG7buXMnfv3rX2PdunXNcvyKigr87ne/w7Bhw1BQUAC/349evXph6tSp+Pvf/94s5yRqbh63AyC9QCCARYsW4Ywzzohb/v777+O7776D3+9P6/keeOABXHLJJbjwwgvjll955ZW47LLL0n6+o7Vp0yYYRuqfX3bu3Inf/OY36N27N04++eS0xrRlyxZMmDABO3bswOTJk3HVVVchNzcXJSUlePPNNzFp0iT83//9H6688sq0njfdevXqhdraWni9XrdDoQzBpJDBzjvvPCxZsgSPPfYYPJ7DL9WiRYswdOhQ7Nu3r0XiME0Tpmm2yLmipJSoq6tDVlaWa8lIJxwOY/LkydizZw/ef/99jBw5Mm79vffei3fffReWZbkU4dETQiAQCLgdBmUQ3j7KYD/+8Y+xf/9+LFu2zFkWDAbxyiuv4PLLL0/YfuXKlRBCYOXKlXHLj+a+sRAC1dXV+POf/+zcuorex2/cpzBp0iT07dtXeZzTTjsNw4YNc35fuHAhzj77bHTp0gV+vx8nnHACnnjiiYT9evfujUmTJuGdd97BsGHDkJWVhaeeespZF9uncODAAfz85z/H4MGDkZubi/z8fJx77rn4/PPP456LH/3oRwCAmTNnOo8p9jn417/+hYkTJ6KgoADZ2dkYNWoUPvzwQ+1zFLVkyRKsX78ev/rVrxISQtT48eNx7rnnphQzoO+/Ub22mzdvxsUXX4yuXbsiEAige/fuuOyyy1BeXu5ss2zZMpxxxhkoLCxEbm4uBgwYgDvvvNNZr3pvfPHFF5gxYwb69u2LQCCArl274uqrr8b+/fvjYor2S23ZsgUzZsxAYWEhCgoKMHPmTNTU1MRte6Q4KHPwSiGD9e7dG6eddhpefPFFp4F56623UF5ejssuuwyPPfZY2s71/PPP49prr8Xw4cNx3XXXAQD69eun3Hbq1Km46qqrsHbtWqfhBYAdO3ZgzZo1+MMf/uAse+KJJ3DiiSfiP/7jP+DxePD666/jxhtvhG3bmD17dtxxN23ahB//+Me4/vrrMWvWLAwYMEB5/m+++QavvfYapkyZgj59+mDPnj146qmnMGrUKHz11Vfo1q0bjj/+eNx333245557cN111+HMM88EAJx++ukAgPfeew/nnnsuhg4dinvvvReGYTgJ7IMPPsDw4cO1z9Xrr78OALjiiiuO9LSmFHMqgsEgJkyYgPr6etx0003o2rUrSktL8cYbb+DQoUMoKCjAhg0bMGnSJAwZMgT33Xcf/H4/tmzZcsTEt2zZMnzzzTeYOXMmunbtig0bNuDpp5/Ghg0bsGbNmoQChUsvvRR9+vTB3Llz8emnn+LZZ59Fly5d8NBDDwFAk+Mgl0jKOAsXLpQA5Nq1a+X//M//yLy8PFlTUyOllHLKlClyzJgxUkope/XqJc8//3xnvxUrVkgAcsWKFXHH27ZtmwQgFy5c6Cy79957ZeOXPycnR06fPl0bz7Zt26SUUpaXl0u/3y9vu+22uO1+//vfSyGE3LFjh7MsGnesCRMmyL59+8Yt69WrlwQg33777YTte/XqFRdXXV2dtCwr4TH6/X553333OcvWrl2b8LillNK2bXncccfJCRMmSNu242Lt06ePPOeccxJiiHXKKafIwsLChOVVVVWyrKzM+SkvL0855sbPdVTj1/azzz6TAOSSJUu0cT7yyCMSgCwrK9Nuo3pvqF6zF198UQKQq1atcpZF30NXX3113LaTJ0+WnTp1SikOyhy8fZThLr30UtTW1uKNN95AZWUl3njjDeWto5YUvfXx8ssvQ8bM0bR48WKceuqp6Nmzp7MsKyvL+Xd5eTn27duHUaNG4Ztvvom7zQEAffr0wYQJE454fr/f73Q8W5aF/fv3O7ckPv300yPuv27dOmzevBmXX3459u/fj3379mHfvn2orq7G2LFjsWrVKti2rd2/oqICubm5CcvvuusuFBUVOT+xr9P3jbmxgoICAMA777yTcKsmqrCwEACwdOnSpI+nsdjXrK6uDvv27cOpp54KAMpYf/KTn8T9fuaZZ2L//v2oqKj4XnGQO5gUMlxRURHGjRuHRYsW4a9//Sssy8Ill1zidliYOnUqSkpKsHr1agDA1q1b8cknn2Dq1Klx23344YcYN24ccnJyUFhYiKKiIudesiopHA3btvHII4/guOOOg9/vR+fOnVFUVIQvvvgi4ZgqmzdvBgBMnz49rhEvKirCs88+i/r6+qTHycvLQ1VVVcLyG2+8EcuWLcOyZctQXFyc1pgb69OnD2699VY8++yz6Ny5MyZMmIDHH3887lhTp07FyJEjce2116K4uBiXXXYZXn755SM2zAcOHMDNN9+M4uJiZGVloaioyHltVLHGfggAgA4dOgAADh48+L3iIHewT6EVuPzyyzFr1izs3r0b5557rvPJqzHdl9GaowrmggsuQHZ2Nl5++WWcfvrpePnll2EYBqZMmeJss3XrVowdOxYDBw7Eww8/jB49esDn8+HNN9/EI488ktAoxH5CTeaBBx7Ar371K1x99dW4//770bFjRxiGgVtuueWoGproNn/4wx+0paqqK4GogQMHYt26dSgtLcWxxx7rLO/fvz/69+8PAAkVPUcbcyqv4X//939jxowZWLp0Kd5991389Kc/xdy5c7FmzRp0794dWVlZWLVqFVasWIG///3vePvtt7F48WKcffbZePfdd7UVZZdeeik++ugj/OIXv8DJJ5+M3Nxc2LaNiRMnKp9f3XGiV5FNjYPcwaTQCkyePBnXX3891qxZg8WLF2u3i35CO3ToUNzyHTt2HNV5jvYbzgCQk5ODSZMmYcmSJXj44YexePFinHnmmXEdpq+//jrq6+vxt7/9Le7T5IoVK476PCqvvPIKxowZg//93/+NW37o0CF07tzZ+V33eKId6Pn5+Rg3blzK5580aRJeeuklvPDCC/jlL3+Z1phTfQ0HDx6MwYMH4+6778ZHH32EkSNH4sknn8Rvf/tbAIBhGBg7dizGjh2Lhx9+GA888ADuuusurFixQvnYDx48iOXLl+M3v/kN7rnnHmd59OqqqVKNg9zD20etQG5uLp544gn8+te/xgUXXKDdrlevXjBNE6tWrYpb/qc//emozpOTk5PQGCUzdepU7Ny5E88++yw+//zzhFtH0U+Asf0O5eXlWLhw4VGfQ8U0zbhjApEy0dLS0rhlOTk5ABIb2KFDh6Jfv36YN2+e8jZQWVlZ0vNfeumlOOGEE3D//fdjzZo1ym0ax3e0MUcTVuxraFkWnn766bjtKioqEA6H45YNHjwYhmGgvr4eQOQ2UGPRK6PoNo2pXjMAePTRR5XbH42mxEHu4ZVCKzF9+vQjblNQUIApU6Zg/vz5EEKgX79+eOONN7B3796jOsfQoUPxj3/8Aw8//DC6deuGPn36YMSIEdrtzzvvPOTl5eHnP/85TNPExRdfHLd+/Pjx8Pl8uOCCC3D99dejqqoKzzzzDLp06YJdu3YdVUwqkyZNwn333YeZM2fi9NNPx5dffokXXngh4bsT/fr1Q2FhIZ588knk5eUhJycHI0aMQJ8+ffDss8/i3HPPxYknnoiZM2fi2GOPRWlpKVasWIH8/Hyn7FTF6/Xi1VdfxYQJE3DGGWfgoosuwplnnomcnByUlpbib3/7G7799lucf/75Kcd84okn4tRTT8Udd9yBAwcOoGPHjnjppZcSEsB7772HOXPmYMqUKejfvz/C4TCef/75uNfhvvvuw6pVq3D++eejV69e2Lt3L/70pz+he/fuCd+Sj8rPz8dZZ52F3//+9wiFQjj22GPx7rvvYtu2bSm9RrGaEge5yMXKJ9KILUlNpnFJqpRSlpWVyYsvvlhmZ2fLDh06yOuvv16uX7/+qEpSN27cKM866yyZlZUlAThloLoySSmlnDZtmgQgx40bp4zxb3/7mxwyZIgMBAKyd+/e8qGHHpILFixIOJ7qscSua1ySetttt8ljjjlGZmVlyZEjR8rVq1fLUaNGyVGjRsXtu3TpUnnCCSdIj8eT8Bx89tln8qKLLpKdOnWSfr9f9urVS1566aVy+fLlyjgaO3TokLzvvvvkKaecInNzc6XP55M9evSQl1xyiXz99dfjtk0l5q1bt8px48ZJv98vi4uL5Z133imXLVsWV5L6zTffyKuvvlr269dPBgIB2bFjRzlmzBj5j3/8wznO8uXL5X/+53/Kbt26SZ/PJ7t16yZ//OMfy6+//trZRlWS+t1338nJkyfLwsJCWVBQIKdMmSJ37twpAch7773X2S76Hmpcatr4/XI0cVDmEFI2uk4kIqJ2i30KRETkYFIgIiIHkwIRETmYFIiIyMGkQEREDiYFIiJyMCkQEZGDSYGIiBxMCkRE5GBSICIiB5MCERE5mBSIiMjBpEBERA4mBSIicjApEBGRg0mBiIgcTApERORgUiAiIgeTAhEROZgUiIjIwaRAREQOJgUiInIwKRARkYNJgYiIHEwKRETkYFIgIiIHkwIRETmYFIiIyMGkQEREDiYFoiR27dqFyy+/HP3794dhGLjlllvcDomoWTEpECVRX1+PoqIi3H333TjppJPcDoeo2TEpULtWVlaGrl274oEHHnCWffTRR/D5fFi+fDl69+6NP/7xj7jqqqtQUFDgYqRELcPjdgBEbioqKsKCBQtw4YUXYvz48RgwYACuvPJKzJkzB2PHjnU7PKIWx6RA7d55552HWbNmYdq0aRg2bBhycnIwd+5ct8MicgVvHxEBmDdvHsLhMJYsWYIXXngBfr/f7ZCIXMGkQARg69at2LlzJ2zbxvbt290Oh8g1vH1E7V4wGMQVV1yBqVOnYsCAAbj22mvx5ZdfokuXLm6HRtTimBSo3bvrrrtQXl6Oxx57DLm5uXjzzTdx9dVX44033gAArFu3DgBQVVWFsrIyrFu3Dj6fDyeccIKLURM1DyGllG4HQeSWlStX4pxzzsGKFStwxhlnAAC2b9+Ok046CQ8++CBuuOEGCCES9uvVqxdvM1GbxKRAREQOdjQTEZGDSYGIiBxMCkRE5GBSICIiB5MCERE5mBSIiMjBpEBERA4mBSIicjApEBGRg0mBiIgcTApERORgUiAiIgeTAhEROZgUiIjIwaRAREQOJgUiInIwKbQDDz74IIQQuOWWW9wOBQAwd+5c/OhHP0JeXh66dOmCCy+8EJs2bXI7rDiPP/44evfujUAggBEjRuDjjz92OySt7du345prrkGfPn2QlZWFfv364d5770UwGHQ7tDj19fU4+eSTIYRwpjilzMOk0MatXbsWTz31FIYMGeJ2KI73338fs2fPxpo1a7Bs2TKEQiGMHz8e1dXVbocGAFi8eDFuvfVW3Hvvvfj0009x0kknYcKECdi7d6/boSlt3LgRtm3jqaeewoYNG/DII4/gySefxJ133ul2aHF++ctfolu3bm6HQUciqc2qrKyUxx13nFy2bJkcNWqUvPnmm90OSWnv3r0SgHz//ffdDkVKKeXw4cPl7Nmznd8ty5LdunWTc+fOdTGq1Pz+97+Xffr0cTsMx5tvvikHDhwoN2zYIAHIzz77zO2QSINXCm3Y7Nmzcf7552PcuHFuh5JUeXk5AKBjx44uRwIEg0F88skncc+ZYRgYN24cVq9e7WJkqSkvL8+I5xMA9uzZg1mzZuH5559Hdna22+HQEXjcDoCax0svvYRPP/0Ua9eudTuUpGzbxi233IKRI0di0KBBboeDffv2wbIsFBcXxy0vLi7Gxo0bXYoqNVu2bMH8+fMxb948t0OBlBIzZszAT37yEwwbNgzbt293OyQ6Al4pHCUpJSzLgpTS7VCOqKSkBDfffDNeeOEFBAIBt8NJavbs2Vi/fj1eeuklt0PJOLfffjuEEEl/Gieq0tJSTJw4EVOmTMGsWbNcj23+/PmorKzEHXfc0WyxUHoJ2RpauQxgWRbq6uqwY8cODBgwAKZpuh2S1muvvYbJkyfHxWhZFoQQMAwD9fX1GRH/nDlzsHTpUqxatQp9+vRxOxwAkdtH2dnZeOWVV3DhhRc6y6dPn45Dhw5h6dKlLRZLWVkZ9u/fn3Sbvn37wufzAQB27tyJ0aNH49RTT8Vzzz0Hw2i+z3xHG9ull16K119/HUIIZ7llWTBNE9OmTcOf//znZouRmoZJ4ShZloXq6mqsWLEC/fr1w/HHH58RDatKZWUlduzYEbds5syZGDhwIP7rv/7L9ds0UkrcdNNNePXVV7Fy5Uocd9xxrsbT2IgRIzB8+HDMnz8fQOQWV8+ePTFnzhzcfvvtLkenVlpaijFjxmDo0KH4y1/+kjHvzW+//RYVFRXO7zt37sSECRPwyiuvYMSIEejevbuL0ZEK+xSaYOvWrQCQsYkhLy8voeHPyclBp06dXE8IQOSW0aJFi7B06VLk5eVh9+7dAICCggJkZWW5HB1w6623Yvr06Rg2bBiGDx+ORx99FNXV1Zg5c6bboSmVlpZi9OjR6NWrF+bNm4eysjJnXdeuXV2MDOjZs2fc77m5uQCAfv36MSFkKCaFJsr0xJDJnnjiCQDA6NGj45YvXLgQM2bMaPmAGpk6dSrKyspwzz33YPfu3Tj55JPx9ttvJ3Q+Z4ply5Zhy5Yt2LJlS0JDyxsBlCrePjpKsbePYmX6rSQiolSw+uh72rp1K8LhsNthEBGlBZNCGoTDYdi27XYYCaSUqKioyNhbCIzv+8nk+DI5NkqOSSFNQqFQxiWGyspKFBQUoLKy0u1QlBjf95PJ8WVybJQck0IaZWJiICJKBauP0iwUCsHj8WTEJ6RoffihQ4cyMlkxvu8nk+PLpNiklKiqqsKxxx7brF/oayuYFJrBvn37XK8PBw7XqPfq1cvlSNSGDh0KIHPji1aU9e3bF5ZluRxNPMMwcN111wHIzOevR48eADIrtpKSEn434igwKTSDnJwcfPvttzBNM+7r/S1t165dGDp0KDZt2oS8vDzX4tDZuHEjzj777IyN79ChQzjhhBPw5ZdforCw0O1w4ti2jVWrVuHJJ5/MyOfvu+++w/DhwzMitsrKSgwYMMD1OFoLJoVmIIRAfn4+PB6Pq0khOiR1fn4+8vPzXYtDJzpYX6bGF706yMvLQ0FBgcvRxIu9JZOJz9+BAwcAZEZs0b9BN/8WWxPeYGvDcnNz8f/+3//L2E9IOTk5eO+99zI2vry8PCxatChj48vKysL27dszMr7c3Fy89tprGRkbJcek0IYJIZCTk5Oxn5CEEMjNzc3o+LKzszM6vvz8/IyML9Pfe6THpEBERA4mBSIicjApEBGRg0mBiIgcTApEROTgfApHSTefAgCMGzcOHs/hr3yEw2GEw+Ejfk/hP3OnN0usRO3F0qojz/FcUVGBY445BuXl5a5/Z6I14JUCERE5mBSIiMjBpEBERA4mBSIicjApEBGRg0mBiIgcTApERORgUiAiIgeTAhEROZgUiIjIwaRAREQOztFM5AaP5k/PNIGGuaGPajkACACKEcyEaUIq9tEtT3osjwkZVhzL64GsrVMfi1olJgUiN4TD6uUiyTrdcg2p2Ue3POmxhFAfi9Nttjm8fURERA4mBSIicjAppMC2bbdDICJqVkwKKVi3bp3bIRARNSt2NKdgwIAB+Pjjj90Og9oCr+ZPTwj1Ot1yINJzrOzvTcexxOH/eb2J2xvsaG5rmBRSkJOT43YIlIk8HnU1j245AIQ0y70e9bpkx0pnXDpeLxAKKY5lpnYcyni8fZQGtbW1bodARJQWTAppsGbNGpSWlkJKxbd+iIhaESaFNBg0aBA2bdqEdevWIRgMuh0OEVGTMSmkQVFREUaOHAkpJT788EPs37/f7ZCIiJqEHc1p4vf7ccopp6C0tBRffPEF/H4/BIcAaJ2Ur5u2xCeyTvdap/oekJp9ZJJzaGPT7JPsWLp12uWaeKP7NLOPPvpIu05KiZqaGuTl5QEAKioqnHV5eXn8+9RgUkgjIQS6d++OwsJClJeXwzRNvvEyVZMqc3SNnNA3gKk2jEKzj0hyDm1smn2OdKxU9tHFezTnSYP+/ftr11VWVmLQoEHO7z169HD+XV5ejvz8/GaNrbViUmgGgUAAHo8HHo+HSYGoGXXu3Fm7rlOnTti1axcqKyvRv39/lJSUOIkgevVAidinQERtkhAC+fn5TgLIz893ftL1Ya2+vh4nn3wyhBBtZsQDJgWiTMKq5lbll7/8Jbp16+Z2GGnFpEBE1ARvvfUW3n33XcybN8/tUNKKfQpERCnas2cPZs2ahddeew3Z2dluh5NWTArUNqgGawMAqRnuXAj1uD2GZnmyilTdPrpz6EgAhgF4NAPPqZYDiY8xGmdT4tI9TgHNsQx9K6I7lmkCduJ9MuExW8XUnlJKzJgxAz/5yU8wbNgwbN++3e2Q0opJgdoG1WBtyXgEoJhzWLu8pY5laOZiFknmaNYlvnTG5TU0x4L+WNoyVkP5erk9TMztt9+Ohx56KOk2//73v/Huu++isrISd9xxRwtF1rKYFIiIANx2222YMWNG0m369u2L9957D6tXr4bf749bN2zYMEybNg1//vOfmzHK5sekQESEyHA1RUVFR9zusccew29/+1vn9507d2LChAlYvHgxRowY0ZwhtggmBSKiFPTs2TPu99zcXABAv3790L17dzdCSiuWpBIRkYNXCtQ2qKqPpB3p1Exln6THSlJJ5PUlLjMAYSiWJ5N0Ok51+ZOUUvkNXQk0PJZG64QmXqDhcSrOo5uOUyDSCa0OLKVjCdOATHVGuAzQu3dv1zvJ04lJgVoPj6mudNFNFQkA0FTGNGVAPKH5w9fsI7wefcWQjkezj0dffaSrlBWmCamsGNI8jwAAqa4YSjYdZ6rVR5pjScnmKBPw9hERETmYFIiIyMGkQNRmcdh2Sh2TAlFr0Hb6MdHGHkybw54daj10bYlumIekx0p1qkpxeLlqtaH5fKVbriOEeh9hAEaqjamMOZaMW5wkAM3MnrqZ5ZKdPsV9eGGTEZgUKLMkm8JRW2ZjQFtlpGMY6kHZzCRlp5ryVuEx1ac3PYCtSViG5sGYhjopGQagrdbUVAzFPcaYY5oGZNKqKFUZqSa5JWvIk03hqcILiIzA20dERORgUiDKKOm8h8L7MZQ6JgUiInIwKRC1VbxQoCZgUiDKJG1oDB1qnVh9RM1KZGUpx+yRRhNm/xIiMmuYcrnqrSzjq2Zi2lthmurST8OAMNV/FkIzhaU0TXUVpxAQupJUzad4KXWrJGCpK5mkZWmqSCWEYo0UgNCNh2fZSabj1AzUp3pNIgFoBsRTv16RsZpa34B4bQ2TAjUrGQqpB5jzagaeS1aSmmxAPN2xdHWOpqFsZIVh6MtIpaYkVb01hKbk/3BsmmMpHr9Mci9Itw+EUD4WYRiRxl/FttXH0g4g2JQB8dTH4jVSZuDtIyI6jP0Q7R6TAhEROZgUiL4vfrqmNoRJgai5tMqb5Mxw7R07mql5accS0lUMJWEY6qkfhQHhSyynkQBEbPVRTHsnTFM9vaVhKKuSBAD4NNN3mgZgNHqcApFxh+IqlmLOJ6V6/CPLit8umllsqe/QDYUOd1zHdOwK24ZUxSwlROyffmzysq1Gr5k8/F/V62UkqT6KvAKK5aw+ymRMCtS8wmH1NJJNqj7yKo8lfIaywRRCxI/jFluS6vWqz2MaEI0beCfmJElB1WDatn4f21aXxIZC6qSkq4gCIg25M+7d4UZYAhCq514I/YB40tYOFKhssFl91Obw9hFRq8fxkih9mBSI3MCPxZShmBRSsGPHDrdDoDajBbKC5gtyRMkwKaTA7/e7HQK1edrvR7doFEeNVzxtDjuaU9C1a1f8+9//Tlj++eef49hjj0VRURHMZDN3tUupTuOYpJVxBgYSiuWaSqLGy6NjTxhG4vEEIlVETsVQw7aiISqfGb8tIsNPSEMAXlWVkR3phI6eN1YYjTqUGx63MCCkiFsWDVFEjxX3FGmeEyDSma14XqQqHmel+vmXsc/xUQ/ax+k4WyMmhTTIzc3F119/jS+//BLFxcXo0qULsrKyYFkWRHu4hDeEsmIlQvP4tU+Lod/FNABLURnj8WjH/lGVqgIAsgLqwepME8gOJC6XEnZBlvJQthWGzFHsEwxBZqmvLkVNPeBPrEwSVXUwjPg/SwkAtg2zul55rEgQiscfNpTPS2REqNTel0KIw/vEvqeTToWqGXsqjdNx1tfrnxMpJSorK1FTUwMAqKiocNbl5eW1j7/NJmBSSIN+/fqhf//+qKiowK5du/Dvf/876ZuV2gfldwSi6/xeddvYpHYqycB/bdyqVau062pqanD55Zc7v/fo0cP5d3l5OfLz85s1ttaKSSFNhBAoKChAQUEB+vXrh1AoBI/Hk/TTyP/gLy0YITUX3dVAk49Vp/5uASUaO3asdp2UEqWlpaiqqsKAAQNQUlLiJIK8vLyWCrHVYVJoBkIIGIYBwzB4iUqp4dslJYZuvooGhYWFzjb5+fm8OjgKrD4iiqXrz3A7uQsR6exVrmvZUNq77du345prrkGfPn2QlZWFfv364d5770UwGHQ7tLTglQKRgoyZAk2akSomyxTOxyjplAQBtldAmtEZdQ630DYAETe+UUyjHhaRQ8nDqwQAaUsIM6aPoGF0C2HbkSVSup+g2rmNGzfCtm089dRT+MEPfoD169dj1qxZqK6uxrx589wO73tjUqAEIlszhSaEehwjQH/NKYRmSkYD8EYrg2SjXTSNntcD4VW8ZT2eSAVS/FEirWxujnK5neOLDHAX07gDApawEe7ceJ+I+kJTXeJp2wjnJJYiC0tGkoXqoVSGD5erxjDqbcga9RWBDw0D/EWvGCJZAkY4DGFJKDubLVWJdOOB+mKEcfgxxoUeO4hd7CBSSQY2THE6TpimejysDDNx4kRMnDjR+b1v377YtGkTnnjiCSYFaptkMJjatJfJSlJNUzMgnld9DtXIoQ30paeGci5iCKEfpdU09Q1jqnTl+E35QJ9kH2kYEJGhX2O2FZHnTCbuLDTPpYR6mk4AkcH1VOt0DXayhjzFAfFas/LycnTs2NHtMNKCfQpEMew0VhIlv9efvltA7bMYNXNs2bIF8+fPx/XXX+92KGnBpEDtUxpvy1uB9P0ZJb26YFdCs7r99tsjw60n+dm4cWPcPqWlpZg4cSKmTJmCWbNmuRR5evH2EbVh7lcM6e6rh7MEPMpilSRfRNM+HN0+zCKpuO222zBjxoyk2/Tt29f5986dOzFmzBicfvrpePrpp5s5upbDpEB0lKQ43MzagPOLbQBSRMcuarwTICDj2+xk93uOeMuJN4uaS1FREYqKio5q29LSUowZMwZDhw7FwoULj/h9idaESYESmR4oWycBfaWJ7m/CMADVlJiGAaGaDhORWb6UvF7FAHeIzKIWHeMoulpEBqqzcnzOoHZO5VG2H/UdvLCjU3uKhts2QsD2SNR1NBqWCdgxT0WwAyANJMRgBCXCmi/ImtUAFBVInkogrqy9oa0364DDda8SIrZktc6AacX8XlsPSMAQsqGjv1HC0FylSCkVgwVGK5pM9YxwgPq1T/aeiK5PWCYinc2NGa2j+qi0tBSjR49Gr169MG/ePJSVlTnrunbt6mJk6cGkQImscPqqj5JNoamqSkryiUt4NcOG+H1AduJgdVIAdgd1eWk424RUzPdsmzasLHUMUmhKLJuiIQnF/Q7ACgDSc3hh7LMqDOPw8yxweBC+ulBDtmrElspGOfJ9CE31kZWk+iikuN/V1Ok4Q6qpPdWHyTTLli3Dli1bsGXLFnTv3j1unfYLhq1I27nmofYrjV/mCmsSQjJWkoIl2RIjqesePrsUmsWMGTMgpVT+tAVMCtQ+aRJJi/1ZN6Ehb9L3HohSxKRAbZdIcxOfxiuS9EbGbEHpw6RArZ/uU79fM8FOBrD9bMgpM7WSrh1qc7QjfiZpLBuPo9RQUSRFo35WAdh+L6yAiXB2tJIITkURBFDdBYCJw+uMho7pgAWr0IYVgFO1FN03N7cWhifaeXu4E9euMyECis5ZCdQc8iMsjMilQXQoioZKIik8MOukc7jo5UOooWDKrI3OdRZZYSPxU5yoD0U6hp1O+9i62TRej7SR++V0ZEwKlCiNf/9CQDmekfB41OMSmSYMv6bntiBPuY+V44XVITthecgvUNNNfbVQ30U9WJ2dC9gd1JU5Zk4YhjfxybG9EqZH/aSJGglpaL6kUAVYOYqS0TIAQiSUufoqLMCO315m+4HqYCTBJZw8SVVYqgkjnSOzqiqPgFZRjtoe8PYRtSKa20RN6bRtyumb8NcifZrSTw5nQRmKSYFavzSPRtoyWuJ2DG/5UOqYFKj1SDr2TyrbH2Gdhq6JbdJgqK4nJSI1JgVKA3dbOJmt7jdIXtevXmmrOoyPwOfV3wtvUlVsyk8nrwgofdjRTK2e1HSChrOMw9NqxlQRWV7A8FsNQzJJQMjIiAwBGzk5tcjJqYchJAwhISAhhESWP4TCQDU8hnV4eKWG/4ekiRrbB9mQhaQEaoM+SAlIH1DdaLQHETQQOXKjGecaCogkAKGbtYyomTEptHFGQT5kXX3Ccimgn1ozOptXwmLNQGYSDYPoKXjMmA++Mn65oao+MoCcbCeM2JhCBYGGgdwajub3AgBqi7wI5RiIlmNGS0zrC4GabqqgJDoNPKgM91h/OX7QaZ9yXSezAtmeUMLy8lAAB61c5T4bcAyE4kLCqjFRVRFQ7hM3LpKUTrmqtAFY8VcyIhiKvI6Kih4ZbhjHqNFLKSX0Y0xJW1nhJXUD3wkBeFJMXqwyymhMCm2cDIch6xOTAgxDPyVjsik0VeWEhtHQYiUSpvo8wqdplIShTjwAZJZP2ZjZXqEc3C6+FLT5iKS3b75nDOLw9w4a0l48nzeyRtXIG+JwUok9JABpqRtmadnq94VHM4VmUwbEo4zGPgXKLEk7h1PsUNZtnuRGfyCgnPkGQNM6mgN+zfFa4s4Q22NqAiYFaj10w1lolzfhFEnWBUz1p+tkSaZJ3QIpN+Zs/Sl9mBQow2ga+GS3IVJteD36CqOmtOFZZmI/w/c5nh4bf2p+TArUOjRhSOlw4rw7RzxW2jWhJlUbXovkBCae9o4dzW2eSDKFpq6zV6jve2irj0RkVjAV01SPcWQYkU7KxofymofHJIoJwcoJIJjTaArJhn/WdQJsZ6C6yPJwloCnYz2yOgSdAexkIFLGk2UEMbrzZvi9ofjSUwDHB3ajt/9AQxWrbChoijSUNZYXWUb8VYEEUGMf/p5EreWFlKKhYEhgj5EDU1gIhjwxza0A6gyIRt0NZrQewAJEw9hEZt3hqxphybgxi0Rtww4hS90JbFnxGUbG/E835amE/qOi6n1kJKs+klCmOFYfZTQmhTZPaqbQbEL1kc8HhDVTNWqOJQxDOyWj8HoV5/DALkycQlOaQLCDYnsA9R0AqZjv2Z8fgq9jYuVVthFEjzx1SWq2GUS+qajWAmDZAtmG+laR2XBFkOuJb+m9woINAx5v/PNjBQDR6BInmls8QQmncMo83EIbIRtmMOY4DWXAwpYJpaqRA9rKajEhBGzdoHTSVg+Wx+qjdoO3j8gdqVYSJbvno60yUi/3eZJ8A1l/FvdvrOhGG3c/MmpDmBSodUj2TtWWpKobSyPNM7I1qYuiCV9taJHGn/ml3WNSoIwiddVHTeho1o+T17QS0jxT/x2G1GlOJKX+j1IXNhtySiMmBcos2gY+9dtHMqC+1538SiG9LawuybEhJwD461//inPOOQdFRUXIz8/HaaedhnfeecfVmNjRTM2s6TN8xXad2kI431VwvqwmGoYFig5617AMAKQp4RE2PLAASHhM21ntN0LwI+RUF8Xshq5mJbxS/VkpCNt5NJXW4YqjoPTAirlcqbO8ztVI2DZgWSLuHLABOywOPzV2TBwSkJDqO1+6r1foOnPZyZvxVq1ahXPOOQcPPPAACgsLsXDhQlxwwQX417/+hVNOOcWVmJgU2gpNFYgM6r9Y1RRCOVAe1IPbAYBPM7VmIAAEEoe8trNMBDslDhQX8gM1xyjGN4INq5f6ts6JXXajOK8yYXkBqjE6d6tyn07Cj0JTHfO3oRqERKRlLoipMqq2gZ3Bjsp9DtXloLw8sZpK1BjIdgqgYp5TWyJwUH2F46moh9FQZSQBp9EX9UEgqC5JlSmOVRUZK0k137Smwihp3mFSKisrw+DBg/HTn/4Ud955JwDgo48+wujRo/HWW2/h0Ucfjdv+gQcewNKlS/H6668zKdD31AJ/f9o7OEcYy0G5NnYk0BgpT62Z5NS620QF3jr9TmlWW6+e66Fpr1fcNx2O+Lw37S2h6+U2ACgSwxHnrGjfiaGoqAgLFizAhRdeiPHjx2PAgAG48sorMWfOHIwdOzZhe9u2UVlZiY4d1R8yWgKTQgpsTS1+uFH9djgcdpYJjomvlmrpqe7WvKl/fnVJId1VPMnu0ujWabs1ko3mYdlQPhHtuN1t/LcXS0qJyspKVFZWwjRNVFRUOOvy8vJa7G/zvPPOw6xZszBt2jQMGzYMOTk5mDt3rnLbefPmoaqqCpdeemmLxKbCpJCCHTt2KJevXLmyZQNpE1Js/FNcDugHqks2gF1TJDuarQ1cvdgINyG2dpwUVqxYoV1XU1ODyy+/3Pm9R48ezr/Ly8uRn5/frLHFmjdvHgYNGoQlS5bgk08+gd+feIty0aJF+M1vfoOlS5eiS5cuLRZbY0wKKejVqxe2b9+esHz06NHwxAwBEL1S8Hg8ST+N/A/+0hxhtmp2tqYPQidJA59smsx00lYYJVuX6hVEdB/l4dpvVhgzZox2nZQSJSUlqKysxODBg7F9+3YnEeTl5bVUiACArVu3YufOnbBtG9u3b8fgwYPj1r/00ku49tprsWTJEowbN65FY2uMJakpMDTj+3g8nib9kIJ2cDvNWzXJO1h3myjdNw2SJQXdlYJZq9tBdxLJW5EKyf6+vF4vOnbsiA4dOsCyLOTn5zs/qT6Xjz/+OHr37o1AIIARI0bg448/Pup9g8EgrrjiCkydOhX3338/rr32Wuzdu9dZ/+KLL2LmzJl48cUXcf7556cUV3Ngy9RWeEzNFJrQzGQm4sbViV9lAKpqImEAihnOAAGhmS0NHlPZIWp7jMi7T0aDBOxsH8J+E5a/0fYSCAcAyxvT/EogHBCATyIbkQor2x+fBAxpOWMCSQF08NYAALwIYW8oN3pyFHmqnX1qYTXqTz18zHLbxG6rg9NPEF1TY/mwszrf+V0CqKvzQQKoqvYDtQaMuvjnzTwk4KlqqB6KPlU1NoyQhFETf4Vj1DSMxVSj7iCPTLup6AS2beV7QkLqB8QD9B3YqQ6I10YGvlu8eDFuvfVWPPnkkxgxYgQeffRRTJgwAZs2bTqq2zx33XUXysvL8dhjjyE3Nxdvvvkmrr76arzxxhtYtGgRpk+fjj/+8Y8YMWIEdu/eDQDIyspCQUFBcz80JSaFtiJspT6Fpm5API9HPcCa4VUvN019j6qpTgrS7wUa3SoSAKTPgJWd2GDZASCcl3gcIyARUAx6BwBdAxXI9yauyzJC6OypOhxLzLpqGUYN1I3Z/1fXF7UysZroUDALm8uLlPvUV2bBcyBxH6Newl+VsBRGyIK3ttH5hRn5LoNuTu1wWLtOWZJqGurlkT00Axh6Ux8Qr414+OGHMWvWLMycORMA8OSTT+Lvf/87FixYgNtvvz3pvitXrsSjjz6KFStWOLeunn/+eZx00kl44oknsHjxYoTDYcyePRuzZ8929ps+fTqee+65ZntMyTApUArSePuiCSWm6TpFUzVllAlRp74aE/p5fppwFmouwWAQn3zyCe644w5nmWEYGDduHFavXn3E/UePHo1QKP67Qr1790Z5eTkA4IYbbkhvwGnAPgXKLKlWGSWbCrOFGlJthREAmWL1UVNKVan57Nu3D5Zlobi4OG55cXGxc6unrWFSoO+vSR/JU9wp1ZG2m3AKACgL5+pX6kaTSDotnHpx6o1/y43XRO0bkwK5ownfO0jlOM2wk1Y6rxT0B2riOvpeOnfuDNM0sWfPnrjle/bsQdeuXV2KqnmxT6Gt0HToQojUp+M0jMRjRWvkFVNoCsNIMrWnJl7bjnR2CkD6YuKTAKzDA8TZDdUzNuAMiPd9SzOTt6H6tborgvqgekY453CRUfuc3wUiU2uK2H5bGTm6EbIbxjFqVOJkWZC6wgDd9Km2ra4wE0L5Ojr7pFJ9JNp29ZHP58PQoUOxfPlyXHjhhQAiIxssX74cc+bMcTe4ZsKk0FZYSaqPUp2OU1N9JDyaaTeTJQWfT12SmuWDzEqsyrECBkK5ioHv/BKyYZw8GVMdIwwbliUOF8zISOPt9Vooqe0AvxF2RlMVEMj31KLG8OBf6NWoiY8coNby4rM6r7NESgMSQEUogHWHuqHOjs7BLGBJgWDIg9o6Hw6UFgK2ABpGRJUyctLc7QKRoZbiz5a1Nwx/ueL1ClkwKxNLT6VtA7WaktT6euXrIg1DPSCiaQLhFAdKbMfVR7feeiumT5+OYcOGYfjw4Xj00UdRXV3tVCO1NUwKlFGOakC82H8bQpGPJKQ0IISA0WhspCo7CwBw0E4cuRQAKi0/DlnqfoWy+nzUWomJrN72QAZVn6IBU9f26oa5qFGX1ya/fZTq8OSpbd7eTZ06FWVlZbjnnnuwe/dunHzyyXj77bcTOp/bCiYFStQS96hboGHSPYyKcBY6+6o1a1M/XtIHo+1oTmenQvp2IbU5c+a02dtFjbGjmdohfWt5MJSdZD914y/q9Ekh5SojToxDLmNSoMyS6hVEsmGrm3I5kuR4YasJJVNNGfgubZhgKHVMCuSS5r9/FAppKmySNOLJokprkuGX1ChDsU+hrdCVJUbXpbBcmKJhpq3GK4yYjxEx+xqGdoA1Ee0Fls5/Ir/aNhAKR8ZAit0+LGHWRipprMDhc4gQ4GmYWdOKztYpAGkIhCq8kH47/iEJiYr6ACQAn9dyvt0sEKleqgr7nLiijb0EUGt5YNsmJASCQQ/shkojWwoEa70ISzMydIUEICNVT+EKLzyVgKfWOXWk9NQGfJUWPLW2syx6Is+hOpi1VmQqTSdJSYjaesDpbI4tS5WQQfW0o1LakbGRGi83DCizjG4qzmR0++hmZKNWi0mhrZCagcyi6xoTIsn2mn0iLWjMRjGH08WlK1X1eiADifX90mPAUg2TLSSkN3IWI6YNsoUZaeAV1ZKb0AVer6K0VthYi97KsKqrfajXTKFp7PLBCCfGZtQBeTsVO0iJ7P3qBtOss2DWhpBQViWhLxXWvV62+rWPJsBETbjiYePfbvD2EaUg82oZjzgpjYL2thKSP0Kh6VPw1GnmbUjygdyo19SqajuaeV+JWgaTArUOTWoTNbfHvlcgCk1px9n2U4ZiUqBWQduQJ6s+0q1raulPC3y1gMhtTAr0/bWhaSKN+nSWsXIUO2p92NHcyoisLOVyaVma6ThF6tNxmpqxjAxxuMootk0TSDx3dL1mUDRRH4oM8pblj7sMEGE70jBHB8TzRv+BSIeqQHwSSlZeWm8A/sTO0SansGiFj4ypMJKAEZYw6w53wgsZ+bcRlBChyPlFTb1TRSSkiFQZyUad/TJSYSRVY1jJJAPVqQYwRMOhvYrB+oRmuXMexWsf0vSB6JZTq8Wk0MrIWs2M77oB7poyHSeEep1pKksfleWozsTDmreY1wT8XqcBdcKyADOmTTTDkZXShJPEnIoaAYQ9AOA/fEvIKUkF6rwm6vcF4pcDgA0YdsOC6OmjyaVOIFAjIoPbNTT6wopslLVHwFQ8fF+5BV+lqiGX8O1JmHMzojaoHsQuFAbqFeMfGSIy8J2C1FWLeUx1o+3xHGEEU1YZtWdMCu1Vht7xOZphIWKHzvaEJGxbQPWAbCnUeU8CMqjphA4CZsJgpKLhv+m85cPbR5SZ2KdAbVcT2l1deWnS4+mGstCVnSY7VpMwwVD6MCmQOzK1kidDr6CIWgqTAmUWTePvqVP3f4gk38xOvA0U3Un3Td/kUh7uOq2JLPVB94iagn0KrY1mjCHt2EdCs48QgCeF6TgRaXtEXFXSUbRGsTf1446ZagN7hEYxTZ/wm9S+NqogcsZ5su34xx8zRIjUDUsiZeJzH01iutcemsok3bSbyabj1D2XbXx2NTqMSaG1UZUrAvpqIsNQ72MakTJPFaGuPhIej+YcUl2VBCQ2mNFd6ixIKxQpw4wmGgPwmAIi3OhYQsAybWQ1NFbhgIhvuCwRiQEN7e7h8eWcQeoOx9BwyHrAiBlwLtrBbdTZ8NQhMohddPuGH9+BEIyGxlxIRCeOhgiG4SmvjRwntnEOh4GqGvXTUqNeDkA98J1pRiqTlDskmVZV1Zh7BBt50mJSINeIhqsbpxm1G77y0PiqQAKA0VBlBPgatafChvqTsupTdwOzVirvnZrVEp6gOsF56izlPqK6HgIibVcrRG5inwK5I9WxfzgjGVGLYFIgl6TYyCf5FG6EmtBprLvdlew8KZ+FqPXh+5yaV7omqG/ChYKZ5DsHab3TwxFPqQ1hUiB3uP3l4JRzFVt4ah/Y0dzaJJtaU2g6O1UD5R2pY1Q5wJpmlmIpI9NrqtQH4ytyYkKSoXDD9GDR80l4ag2EO+c2xCbiQpVV4cS4BSBMQCo6h0XYhhFuiCt2mAwJmHWRXm1nWI2GCiMjaMGstyP/rql3koGQgKiqc7YTMbPQyfogZMhC3IkkAGlD1qum0JSRQe80M+Ip00+ypBT7XCmOl9JyJr92j0mhtUk25WajweUAACY0g9gptj3CeYRuoDzjCHEpCCkjx4ttlCEAy4a3OrH00jYA01K/XaUh1NOcSantOxDBMAzlLhKeSsXAc1ICtbqRQsPqQQcBfQmxbrlu0EED+tLTSICaxSkup3aPt48ow+g+wbbM2Y0a1Sf7TMB6V2oZTArkjlRLT/nBlqhF8PZRmlVXV2Pnzp2orq6GoZqohppE2HbyCc5S/CAtgmH9XA/UamzYsEG7TkqJ6upq+P1+AEBFRYWzLi8vT9nXRUwKaVFXV4d9+/Zh165dqKioQKdOnZCTkwPDMPjGS5ckz6OorYfMCSQurw9pZp1DkiuPJlyS8CrGNYFA4useVVVVhbPPPtv5vUePHs6/y8vLkZ+f36yxtVZMCmnwz3/+Ex07dkT37t3RtWtXCCEQDofh8XjSnxR00ygKAFJxZZJsOk7dhYyhmY4TUE/hmewh6gbEazzzV7TQSAjIumDMMuHsKpzvHYi4/yFsAXWNO4ElELIgLNWAdIAIhQHLPjx4XfR/wTAQrViKrTCSipijKxSVRBIN40HpXn/TTFwnG8b9Uw5iB32Cs6W+wky1j2keYea11qNfv37adVJK7Nq1C5WVlejfvz9KSkqcRJCXl9dSIbY6TAopsDQVI2eccQZyc3Od38PN+QenmxM31ek4TSPSKKp4vepj+Qz1QGra0TuPsE5xfiHUU14CgFmjHr0V4TDgUTwvugofAFCWijbso6sk0k6HKdWvi2lABtWvl9R9o9o0gXATptBUPVTNPkIY7eLiRggRdzWQn5+f8tXB3Llz8de//hUbN25EVlYWTj/9dDz00EMYMGBAusPNGLzpnYIvv/xSuTzZJSy1Z7x12Nq9//77mD17NtasWYNly5YhFAph/PjxqK6udju0ZsMrhRQMGjQIH3zwgdthuKg9fL4kOuztt9+O+/25555Dly5d8Mknn+Css85yKarmxSuFFHhYrZI+TfnyVMq78JM6pVd5eTkAoGPHji5H0nyYFKgVSd/IqkSpsm0bt9xyC0aOHIlBgwa5HU6z4Uff1ibV6Th1+wgBmJrtDQHV5wUJ0Wg6zkbHU+xx+L+q1fohMJIf9+jp52LWDPORbPtk4wUpnpfI3EAxyxuN2aRkIMnrpeu013xRQ7d9ss5/0po9ezbWr1+Pf/7zn26H0qyYFFqbdE3HqdsekcZfOR2nYUYGcmvMNAFNlY0zpWVjmnGMIIS+MkhZdtlwfEVjLgH989V4/uSY82srhpI891JVfeTx6KvFdDwedcxCJK+mUtFsL9tIOWpLmjNnDt544w2sWrUK3bt3dzucZsWkQM0sOgwqUesjpcRNN92EV199FStXrkSfPn3cDqnZMSlQ68e8Q81k9uzZWLRoEZYuXYq8vDzs3r0bAFBQUICsrCyXo2se7GgmItJ44oknUF5ejtGjR+OYY45xfhYvXux2aM2GVwpERBr6QoW2i1cKRETk4JVCa6OrwBFCv061XLm90KyL/FtqZv4Sth2Z/UxVFpms0kU3dUKyx6g7kG4sId00obLh8TTeTQj1DGdSasKVkXMoK3Jt/WuSTCqvI6B/7ERNwKTQ2ugaAEPTMOqm41Ru3/C7MBqta/i3qS5vlaYJYWniEqa29FM5IJ8hAN2sk4bQPBbNQH1CALamLFO3j8dMUsaqeYweTXmv2fh5PAqG5jy65URpxttHlIjfBCZqt5gU6OjxgypRm8ekQEfP9SsI1wMgavOYFKj10+YKXtoQpYodza2NbjpOQD+Qmm6fZIPrqQa+U03tGR2LTag/X0ghIFRTeEpoBpGT2ilMpdSsiztWbCJIMuUoEPP4Gw0op5gOU0pEOu11VM9xsudeR2iOZRhtZgpNymxMCq1Nk6bjVOyTZEA8mJoKHJ9PPVWlYUDa6gZLeExI1Xl000sKof/CkKkZkC/ZsXRTjuoeo8cDhDQD8unozu/16Kf2TPFYwu/jdQ+1CN4+IiIiB5MCERE5mBSofWIhE5ESkwKlAe92E7UV7GhubdI1HeeRjqXdR/U5QkSGzVDRTO0ZWaeqcAIgk43vpNlHeyzj8L8bH0v1WARSn65S+3wleR51VyqmCWWS5RSa1EKYFFqbFpiOU1uZoyuLTDq1p+ZYXp9+vCJt9ZHUlGVqKqySHStZxVCq017qpso0mjCFpuo5ASDDtakdh6iJePuIiIgcTArkknT2Q7BPgyhdmBSIiMjBpEBERA4mBUoD3r4haitYfdRWKEtFG6jKNZOViur20S0XSUpSo2WkqhJM3emTfbEs+jjjTqcpVY2eX0e5T5JjJaPaxzRTH/uIyGVMCm2FLTVTQmqmw8QRSlJTOZZumkygYUpMXalsiscyNVN4mprHkqwkVVdGqztWMppjCclrKGp9ePuIiIgcTApERORgUiAiIgeTAhEROdjRnGbBYBAbNmxAeXl5i57XyM5Sz3Bm2frOzmSDrGmn49RN7ZksuDQdSzvwXVOmvdTso5sOE4Dw6c4hAL8vcanHA1lfn1pclJIPPvhAu05KiZqaGhQWFgIAKioqnHV5eXnaaV/bOyaFNCorK8P69euRn5+PgQMHwjTNFnvj2ZVVyuUiJxuoq0tc0ZQB8bRTeyarGNIdSzNVZTqP1dQB8TRlpFI3FaoGK4+a36BBg7TrKisrcfzxxzu/9+jRw/l3eXk58vPzmzW21opJIQ0sy8LXX3+N0tJSHH/88SguLoZlWfB4PPw0QtSMOnTooF1XWFiIXbt2obKyEv3790dJSYmTCPLy8pp0vgcffBB33HEHbr75Zjz66KNNOkamY1JIgzVr1sDv92PkyJHIzs5GWDm8MxG1JCFE3NVAfn7+97o6WLt2LZ566ikMGTIkHeFlLHY0p0G3bt0wYsQIZGdnux0KETWDqqoqTJs2Dc8880zSq5O2gEkhDfr06cPbRERt2OzZs3H++edj3LhxbofS7Hj7KAVS12mZwYRhQKqqjAxDPy6QbtrL6H7K7Y8w9lFKy1M9lqGupDrSMBe650V5LGhnRaO27aWXXsKnn36KtWvXuh1Ki2BSSMGGDRvcDiFluqqkpNNEelpg7COpOVayhlxzLGGakMGgeh8d3eNPdfpMatNKSkpw8803Y9myZQgEAm6H0yJ4+ygF3bp1czsEImpBn3zyCfbu3Ysf/vCH8Hg88Hg8eP/99/HYY4/B4/HAaoMfInilkIKOHTu6HQIRtaCxY8fiyy+/jFs2c+ZMDBw4EP/1X/8FM9kXQFspJgUiIo28vLyEL8jl5OSgU6dOSb8415rx9hERETl4pUBElIKVK1e6HUKz4pUCJdKVXurG/tFVHgH6ah7dsZKV/WqOJWtr9fsQUUqYFIiIyMGkQEREDiYFIiJyMCkQEZGDSYGIiBxMCkRE5GBSICIiB5MCERE5mBSIiMjBpEBERA4mBSIicjApEBGRg6Okumhp1Z+b9fgHDx7E+vXrceaZZzbreZpq69atqKurw4knnuh2KErhcBgrVqzAmDFj4PFk1p+KbdtYvnw5zjrrLPj9frfDSbBv3z58/fXXOP30090OhVLEKwUiInIwKRARkYNJgYiIHEwKRETkYFIgIiIHkwIRETmYFIiIyJFZxddtRDgchm3bCOkmp28h0fMHg0FX49CxLAu2bWdsfOFwGEDk+bNt2+Vo4kXjCYVCEEK4HE2icDgMKaXrr61t23jkkUdcjaG1YVJIs2AwmBF/qFJKlJSUIC8vD5ZluRqLTmVlZUbHF43LsizXX8/GpJTw+XyoqKhAhw4d3A4nQSAQQH19Pfbt2+dqfKFQCJ9//jkA4MCBA8jPz3ctltaCSSGNgsEg6uvrYRiG69+A3bVrF6qqqjB8+HB4vV5XY1EJhUI4ePAg+vfvn5HxAXASgdfrdf31VCkuLsb+/fvRpUsXt0NJ4PV68YMf/ACbN2929T3o9Xrx/PPP46qrrsLkyZOxfPlydOzY0ZVYWgv2KaRJbELwer0QQrj2U1dXh82bN+P444+Hz+dzNRbdT1lZGfLy8pCdne16LMl+ALgeg+6nuLgYZWVlsG3b9VhUP926dUN+fj42bdrk6vMYCATwl7/8BT169MC4ceNw4MABN5uKjMekkAaNE4KbpJT46quv0LVrV3Tq1MnVWJLZs2cPiouL3Q6jVcvLy4PP58P+/fvdDkVJCIGBAwfi0KFD2L17t6ux+Hw+LFy4EN27d2diOAImhTTIlIQAADt27EAoFMIPfvADt0PRqq+vR3l5eUbe9mhNolcLe/bscTsULZ/Ph+OPPx5ff/01amtrXY+FieHIMu9GaSskpYQQwqlWcUtlZSW2bduGk08+GVJK1+PR2bVrFwoKCmCaZsbGCByuPsrkGDt16oQdO3agrq4uI/s9AKCwsBBFRUX46quvMGTIEOe2nBsMw8DTTz+N6667DuPHj8eSJUsSrqjz8vJcjdFtQkop3Q6iNbAsC9XV1VixYkXcciEE+BQStR41NTW4/PLLtetLSkpw7LHHttvEwNtH3wMTAlHrk5WVhWeffVa7vkePHigtLW23f9tMCk3EhEDUOgkh0KlTJyYGDSaFFETvLTMhELVuTAx6TAopsG0bhmG0uzcJUVt0tImhsrKyBaNyHzuaUxAKhZzxeqjts20b27ZtQ58+fWAY/PzUVkkpUVlZiVAohM2bN2PYsGFx69tbNRKTAhEROfjxh4iIHEwKRETkYFIgIiIHkwIRETmYFIiIyMGkQEREDiYFIiJyMCkQEZGDSYGIiBxMCkRE5GBSICIiB5MCERE5mBSIiMjBpEBERA4mBSIicjApEBEBePzxx9G7d28EAgGMGDECH3/8cdLtlyxZgoEDByIQCGDw4MF4880349ZLKXHPPffgmGOOQVZWFsaNG4fNmzfHbXPgwAFMmzYN+fn5KCwsxDXXXIOqqqq0P7aUSCKidu6ll16SPp9PTps2TXbr1k2apilN05RvvfWWcvsPP/xQmqYpr7jiCtmnTx9pmqYEIB9//HFnm9/97nfS5/PJnj17ykAgIP1+v8zNzZVbt251tpk4caL0er0SQNzP3Llzm/0x6zApEFG7N3z4cDl+/Pi4xABACiGUieHSSy+Vp512WlxiiDbojz/+uLRtW3bp0kUed9xxctSoUQmN/oQJE+RXX30lAchjjjlG3n777XLy5MkyEAhIAPKSSy6RlZWVLjwTUnI6TiJq14LBILKzs9G3b1/06dMHK1euxJNPPolXX30Vb775JrxeL3bs2IEuXbo4+/Ts2ROdOnVCVlYWPv74Y8ydOxfbtm3DM888A9u28cYbb+C8887DZ599hkcffRR79uzBwoULcc4552D9+vX44osvsHbtWtx2220oKChAVlYW/H4/Hn/8cZx11lkoLi7G6NGjsWjRohZ/PtinQETt2r59+2BZFr755hts27YNs2bNwsyZM1FTUwMpJerq6jB06NC4Pobdu3ejpKQEtbW1mDhxIn7xi18gGAzCsizYto2pU6cCAIqLiwEAfr8fxxxzDNavXw8AGDJkCK655hocOnQIBw8exMaNG7Ft2zaMGTMGtm1j165dePHFFyGEwIMPPtiizweTAhER4CSGcePGYfHixVi5ciV8Ph+EEPjuu+9w+umn4+2333a2P3ToEEpKSjBu3Dh89NFHWLBggbOusrISALBx40YAwMqVK9GpUycIIeD1etGtWzcYRqT5HTRoEHJzc/HBBx8gLy8PQHwyWb16Nfbs2dMizwHApEBE7Vznzp2dBtqyLBQXF+Phhx9Gly5dUFdXh7y8PAwaNAh+vx+TJ0/G3r170bVrVwCRxFBcXIw//vGP6N69O7Kzs+H3+yGEABCpaJo4cSIWLFiAQYMGwePxIBQKwefzoX///hBC4NNPP0WXLl0wZMgQJyns2bMHBQUF+NnPfoa9e/fioosuarHng0mBiNo1n8+HH/7wh87voVAIn3zyCfbu3YvOnTujqKgI2dnZuPjiiwEACxYswGmnnQav1+vss3r1apimiQEDBjhXBD6fDx999BEuvvhiPP/889i/fz/C4TDOOOMMbN++HZs3b4aUEqZpoqKiAgBQU1PjHLOqqgqbNm3CM888g48++ghr1qxpkeeDSYGI2r2f//znAAAhBFatWgXLspyf/Px87Ny5E19//TVycnKwevVq3Hzzzaivr4cQAl988QVKS0tRUlICy7LQtWtXmKaJ/v37Y/fu3TjzzDPx5Zdfolu3bpBS4qabbnKuKM455xwAkSuODz/8EPX19U5Mtm3j3XffxYIFC9CzZ0+sXr26RZ4LJgUiavemTp2KGTNmQEqJX/3qV87yYDCI3bt3w+v1YuPGjairq8Pu3btx+umn47777kM4HMaDDz4I27YxYsQIfPXVV7AsC36/H8cddxwMw8DatWvx7bff4uDBgwAiVwM1NTUYM2YMbrvtNvj9foTDYZxzzjno1q0bAGDcuHEAgPvuuw/z589Hly5dsHv37hZ5LpgUiIgALFy4EDNmzIhbNnToUNTW1mL16tUIBAKor69HSUkJAODuu+/G/fff72z79ddfY8qUKfjqq6+Qn5/vVDXZto1QKIS1a9cCAGbOnAkAuPfee1FUVAS/3w8AqKurw759+2AYBrZv347LLrsMEyZMQDgcjruCaG5MCkREDRYuXIj58+c7/QW7du3C22+/jaKiIng8HgQCAXg8Hmf7u+++Gy+//DJyc3Oxf/9+fP7553jttdfg9XpRUlKCa665BuPHj3f2B4BjjjkGQ4YMwY9+9CP88Ic/xDnnnIOJEyfisssuQ1VVFWzbxvDhw/H0009j3bp1MAwDBw8edDq3mxuTAhFRjDlz5uD555+HEAI9e/ZEfn4+brjhBlRVVSEvLw833ngjrrrqKtxxxx0AgClTpuCdd96BaZqYOXMm+vbti0OHDuHbb7/FLbfcgnfeeQd79+5FKBTC0KFDUVZWhhtuuAFApN9g+fLlGDVqFG666Sbcc8898Hg8GDlyJJYuXYqf/exnuOCCC/Ddd9/htNNOa5knwJXvURMRZbgZM2ZIANI0TTl48GB54YUXysLCQrl79245atQo2bdvX3n77bc72y9atEj27NlTejwe6fF45EUXXSQ/++wzuXnzZmeba6+9VgKQf/rTn+RXX30lr7vuOueYn3zyiRwxYoT0+XwSgOzZs6ecPXu2PPXUU+Vpp53WYo+bSYGISGP+/PmyZ8+e0ufzyeHDh8s1a9Y460aNGiWnT5/u/L5t27aEMY4AyFGjRh31MaWUsra2Vt54442yQ4cOMjs7W06ePFnu2rWrOR9mHI59REREDvYpEBGRg0mBiIgcTApERORgUiAiIgeTAhEROZgUiIjIwaRAREQOJgUiInIwKRARkYNJgYiIHEwKRETkYFIgIiIHkwIRETmYFIiIyMGkQEREDiYFIiJyMCkQEZGDSYGIiBxMCkRE5GBSICIiB5MCERE5mBSIiMjBpEBERA4mBSIicjApEBGRg0mBiIgcTApERORgUiAiIgeTAhEROZgUiIjIwaRAREQOJgUiInIwKRARkYNJgYiIHEwKRETkYFIgIiIHkwIRETmYFIiIyMGkQEREDiYFIiJyMCkQEZGDSYGIiBz/P/hca+HkJi7YAAAAAElFTkSuQmCC\n", + "text/plain": "
" + }, + "metadata": {}, + "output_type": "display_data" + } + ] + } + }, + "57980998a5874ce19178b2464c67584d": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "5b57540dd7284c82a69e1282eda41ce0": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "SliderStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "SliderStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "", + "handle_color": null + } + }, + "924cd572d4cc4e9e9751cf4f82ab349c": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "97184c81f3674bcc9d5d9146cfebbe01": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "eb475a8cde92418d97e8604866d0a587": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "VBoxModel", + "state": { + "_dom_classes": [ + "widget-interact" + ], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "VBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "VBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_0632b737574b46d1968bd198902f0c35", + "IPY_MODEL_1463125f5dd04e80a394067acb5f0462", + "IPY_MODEL_4030a6224a1f4e629c0b8c764f68c80b" + ], + "layout": "IPY_MODEL_57980998a5874ce19178b2464c67584d" + } + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 1 +}