main.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ADM-HW4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Group #25.\n",
    "\n",
    "Alessio Sampieri, Salwa Afrah and Gamza Bakhishaliyeva."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Hashing task\n",
    "For this task we have The first dataset which represents the \"pool\" of passwords and we have to find how many passwords of the second dataset are already present in the pool by implementing :\n",
    "- Hash function\n",
    "- Bloom Filter structur"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import math \n",
    "import numpy as np\n",
    "import time\n",
    "import json\n",
    "import matplotlib.pyplot as plt\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Read dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "password1=[]\n",
    "with open('./DATA/passwords1.txt', 'r') as fh:\n",
    "    line=fh.readlines()\n",
    "    for i in range(len(line)):\n",
    "        password1.append(line[i].strip().split('\\n'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100000000"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(password1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['JV#0$S;D@j+c96/8C(W%'],\n",
       " [\"J,)e+LH'ENmdV5YGWu4B\"],\n",
       " ['tR0V2@$SNuU9;<&DafxO'],\n",
       " ['/DXhI;GNl*nbCzKso=?@']]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "password1[:4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "password2=[]\n",
    "with open('./DATA/passwords2.txt', 'r') as f2:\n",
    "    line2=f2.readlines()\n",
    "    for i in range(len(line2)):\n",
    "        password2.append(line2[i].strip().split('\\n'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "39000000"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(password2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[')cKy&3MG+V?J@Ps=,p0k'],\n",
       " ['PHxf(Xu/eWTtIR;-,<bd'],\n",
       " ['oZs,l!:b4LqHC#y\"S.?m'],\n",
       " ['C+sTrO#>6Gw\"RmlWboAi']]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "password2[:4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The bloom filter \n",
    "\n",
    "#### At first we need to calculate :\n",
    "\n",
    "The error rate p =(1-e^-kn/m)^k\n",
    "\n",
    "The size of the filter m = n log P/(log2)^2\n",
    "\n",
    "The number of hash functions k = m/n log2\n",
    "\n",
    "n= the number of elements here we have 100,000,000 Passwords\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "filter size table =  814236334 \n",
      "Number of hash functions needed =  6 \n",
      "Starting Error rate =  0.02 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "n=len(password1)\n",
    "# As a start we will set error rate to 0.02 so we can calculate the m\n",
    "p=0.02\n",
    "m = round(-(n*math.log(p))/(math.log(2)**2))\n",
    "bloomTable=['' for i in range(m)] #The bloom table which has the size of m filled with empty space\n",
    "k = round((m/n)*math.log(2))\n",
    "\n",
    "print(\"filter size table = \", m ,'\\nNumber of hash functions needed = ', k ,\"\\nStarting Error rate = \", p, \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hash functions\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Different implementation \n",
    "def hash_func_1(password):\n",
    "    hashed_1 = 0\n",
    "    for i in range(len(password)):\n",
    "        hashed_1 += ord(password[i])\n",
    "    return (hashed_1%m) #first hashing function which thake the ord of each char\n",
    "\n",
    "def hash_func_2(password):\n",
    "    hashed_2 = 0\n",
    "    for i in range(len(password)):\n",
    "        hashed_2 += hashed_2+(ord(password[i]))**2 #Second hashing function which thake the ord of each char power 2\n",
    "    return (hashed_2%m)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### For now we are Using  small fraction of the DS to test the code"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "x1=password1[:300]\n",
    "x2=password2[:300]\n",
    "n1=len(x1)\n",
    "# As a start we will set error rate to 0.02 so we can calculate the m\n",
    "p1=0.02\n",
    "m1 = round(-(n*math.log(p))/(math.log(2)**2))\n",
    "bloomTable1=['' for i in range(m1)]\n",
    "k1 = round((m/n)*math.log(2))\n",
    "print(\"filter size table = \", m1 ,'\\nNumber of hash functions needed = ', k1 ,\"\\nStarting Error rate = \", p1, \"\\n\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bloom filter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bloom_filter(password1,password2):\n",
    "    \n",
    "    start=time.time()\n",
    "    duplicates_counter = 0\n",
    "    not_duplicates=0\n",
    "    \n",
    "    for j in password1:\n",
    "        for u in j:\n",
    "            pass1_1 = hash_func_1(u)\n",
    "            pass1_2 = hash_func_2(u)\n",
    "        #Fill the corresponding index in the bloom filter table with one \n",
    "        if bloomTable[pass1_1] != 1:\n",
    "            bloomTable[pass1_1] = 1       \n",
    "        if bloomTable[pass1_2] != 1:\n",
    "            bloomTable[pass1_2] = 1 \n",
    "         \n",
    "    # Now we work on the second data set \n",
    "    for i in password2:\n",
    "        for y in i:\n",
    "            pass2_1=hash_func_1(y)\n",
    "            pass2_2=hash_func_2(y)\n",
    "      \n",
    "        \n",
    "        if bloomTable[pass2_1] == 1 and bloomTable[pass2_2] == 1: # iF the bloom table index already filled with 1 meaning the hashed value of password1 occupied it\n",
    "            duplicates_counter += 1\n",
    "        else:\n",
    "            not_duplicates+=1\n",
    "    \n",
    "    end = time.time()\n",
    "    \n",
    "    print('Number of hash function used: ', k) # K form the bloom filter formula the nneeded hashing function ,but we used two hashing functions\n",
    "    #print('Number tot pass ', len(password1), len(password2))\n",
    "    print('Not_duplicates ',not_duplicates)\n",
    "    print('Number of duplicates detected: ', duplicates_counter )\n",
    "    print('Probability of false positives: ', p)\n",
    "    print('Execution time: ', end-start)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of hash function used:  6\n",
      "Not_duplicates  22110216\n",
      "Number of duplicates detected:  16889784\n",
      "Probability of false positives:  0.02\n",
      "Execution time:  3860.2183072566986\n"
     ]
    }
   ],
   "source": [
    "bloom_filter(password1,password2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Alphabetical Sort"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Counting sort"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function gives in input an array of integers and returns the sorted array and the order of them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def counting_sort(array):\n",
    "    aux_array = [0]*1000 # make auxilary array of size 1000, we suppose the ordinal are between 0 and 999, however it can be any range\n",
    "    order = [0]*len(array)\n",
    "    \n",
    "    \n",
    "    # calculate the frequencies \n",
    "    for i in array:\n",
    "        aux_array[i] += 1\n",
    "        \n",
    "        \n",
    "    # calculate the cumulative values of the frequencies\n",
    "    for i in range(1,len(aux_array)):\n",
    "        aux_array[i] += aux_array[i-1]\n",
    "    sorted_array = [None]*len(array)\n",
    "    \n",
    "    \n",
    "    # put tha values in array to sorted array on the right position and store the order\n",
    "    for j,i in enumerate(array):\n",
    "        order[aux_array[i]-1] = j\n",
    "        sorted_array[aux_array[i]-1] = i\n",
    "        aux_array[i] += -1\n",
    "    return sorted_array, order"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([0, 1, 2, 3, 6, 9], [5, 4, 3, 0, 1, 2])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "counting_sort([3,6,9,2,1,0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Counting sort for letters\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function gives in input an array of letters and returns them in alphabetical order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def counting_sort_alpha(arr):\n",
    "    \n",
    "    count=[ 0 for i in range(256)] #fill counter with zeros 256 index because we are using ord\n",
    "    alpha_=['' for _ in arr] # to fill the ordered letters \n",
    "    \n",
    "    for a in arr:\n",
    "        count[ord(a)] += 1 # count occurrence of each character and fill Corresponding ord index with letter counter\n",
    "        #print(count)\n",
    "    \n",
    "    for i in range(len(count)): \n",
    "        count[i] = count[i] + count[i-1] #put each letter in its actual Position the count[i] \n",
    "        \n",
    "        #print(count)\n",
    "        \n",
    "    for i in range(len(arr)): # for each index number from above minus 1 put the letter in the index number       \n",
    "        count[count[ord(arr[i])]-1] = arr[i] \n",
    "        count[ord(arr[i])] -= 1\n",
    "        \n",
    "        #print(count)\n",
    "    for i in range(len(arr)): \n",
    "        alpha_[i] = count[i] \n",
    "        #print(count[i] )\n",
    "    return (print(alpha_ , end=\"\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z']"
     ]
    }
   ],
   "source": [
    "counting_sort_alpha(['z','y','x','w','v','u','t','s','r','q','p','o','n','m','l','k','i','h','g','f','e','d','c','b','a'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running time for Counting Sort for letters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Theoretical"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The counting sort alpha is basically 3 for-loops. The first and third loops look at the entire array of letters and positions them in a certain position. The second loop is only concerned with putting each letter in its actual position the count[i]. Then first and third we will have a complexity of O (n), for the second O (k). Adding these values ​​we will have that the complexity will be O (n + k).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Empirical"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_count_sort():    \n",
    "    time_tot = list()\n",
    "    # change m from 10 to 100000 with steps of 10000\n",
    "    for m in range(10,100000,10000):\n",
    "        time_ = list()\n",
    "        let = np.random.randint(ord('A'), ord('z')+1, m)\n",
    "        start = time.time()\n",
    "        sorted_words_list = counting_sort(let) # same thing of counting_sort_alpha\n",
    "        end_time = time.time()\n",
    "        time_.append(end_time - start)\n",
    "        time_tot.append(sum(time_))\n",
    "   #PLOT\n",
    "    fig = plt.figure(figsize=(16,4))\n",
    "    plt.plot(range(10,100000,10000),time_tot)\n",
    "    plt.xlabel('Letter lenght')\n",
    "    plt.ylabel('Running time (secs)')\n",
    "    plt.title('Running time for counting sort')         "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA8AAAAEWCAYAAACpLv6AAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xl8VdW5//HPk5ORQMKQMAXCHFREVCI4VFFbh7YqWrVQvLb2emsnO/xabe29Hayd1Ku1Wu1grbX1FrVibbHWotaKWhUIKijKPIYxgRBC5pzz/P44O3AIGQ7IyQnJ9/165XXO3nvtdZ6d2JJv1l5rm7sjIiIiIiIi0t2lJLsAERERERERkc6gACwiIiIiIiI9ggKwiIiIiIiI9AgKwCIiIiIiItIjKACLiIiIiIhIj6AALCIiIiIiIj2CArCIiHR7ZvaMmX2qkz7rv83sgU76rMvMbJOZ7TWzkzrjMztDcD2jk12HiIh0P6bnAIuISGcws/XAICAM7AX+AVzv7nuTWdf7YWZnA//n7sOS9PlrgK+5+1+T8flHgpm9SPR72Cl/NDhSgv+e/8vdn092LSIiEj+NAIuISGe62N17AycCJwHfSnI9R7sRwLLDOdHMQke4lh7BzFKTXYOIiBw+BWAREel07r4NmEc0CAPRkUAz+6+Y7WvM7JWYbTezz5nZKjOrMLP7zMxi25rZHcGxdWb24db6jqPtKDN7ycyqzOz54HP+r+U1mFk28AwwNLhld6+ZDTWzm5vbm9nIoO5PB7cqVwTXcIqZLTWz3WZ2b4t+/9PM3gvazjOzEa18doaZ7QVCwJJgJBgzOza41t1mtszMLok55yEz+6WZ/d3MqoFzWum3v5n9zsy2BJ//l5hjnzGz1Wa2y8zmmtnQFteYGtM2ru+3mf0IOBO4N/j+3Rvzsx4bU/d9ZvZ08DNZYGZjYj7rfDNbYWaVZvYLM5sf+99Ri+ubYmYlZrbHzLab2U9jjl0SfM92B/UfG3NsvZl908yWAtVm9ghQCDwV1P2N1j5PRES6HgVgERHpdGY2DPgwsPoQT70IOAWYBHwcuCDm2FRgBZAH3A78tjkgt6K9trOBhcAA4Gbg6tY6cPfq4Bq2uHvv4GtLO583DpgB/Az4H+BDwATg42Y2DcDMLgX+G/gYkA+8DDzSymfXByPpAJPcfYyZpQFPAc8CA4EvAX80s/Exp84CfgT0AV7hYA8DvYK6BgJ3BXWdC/yE6Pd8CLABeLSNa23r+g/6frv7/wTXeH3w/bu+jfM/AXwf6Ef0v5kfBXXlAXOI3kkwIPiM09up427gbnfPAcYAfwr6KSL6ff4q0e/734mG2/QWNXwU6OvunwA2EtzR4O63H8L3QkREkkgBWEREOtNfzKwK2ATsAL53iOff6u673X0j8C9iRpCBDe7+G3cPA78nGtQGtdFPq23NrJBowP6uuze4+yvA3EOssTU/cPc6d38WqAYecfcd7r6ZaABsXsDqs8BP3P09d28Cfgyc2NoocCtOBXoT/R41uPsLwN+IBrdmf3X3f7t7xN3rYk82syFEA/3n3L3C3RvdfX5w+CrgQXd/w93riQbO08xsZJzXfyg/m9b82d0XBt+TP7L/5/4RYJm7/zk4dg+wrZ1+GoGxZpbn7nvd/fVg/wzgaXd/zt0bgTuALA4M0/e4+yZ3rz2EukVEpItRABYRkc50qbv3Ac4GjiE6IngoYsNNDdHAd9Axd68J3sYeb7WfFm2HArti9kE0rL9f22Pe17ay3VznCODu4Dbc3cAuwICCOD5jKLDJ3SMx+za0OLe9axlO9Nor2uh7Q/NGsHDZzjjrgkP72bR7Pgf+3IcSc00eXdmztJ1+rgWKgOVmtsjMLorpJ/b6IkG/8X7vRETkKKEALCIinS4YWXyI6Ehbs2qit982G9yZNQW2Av3NLLaO4e20P9KPUtgEfNbd+8Z8Zbn7q3GcuwUYbmax/7YXAptjtturdxPRa+/bRt/7RqGD+c8Dgr6rg92H+7N7P9/DrcC+FbiD29jbXJHb3VcFty8PBG4D5gTX0vL6jOjPvb3vnR6jISJyFFIAFhGRZPkZcJ6ZNd/O+hbwMTPrFSyAdG1nF+TuG4AS4GYzSzez04CL2zllOzDAzHKPUAm/Ar5lZhMAzCzXzK6M89wFRMPoN8wszaKPaLqYOOfquvtWoot6/cLM+gV9nBUcng182sxONLMMordmL3D39e5eRjQo/oeZhczsP4nOr43XduBwn/n7NDDRzC4NFuH6Iu2EbzP7DzPLD0Z4dwe7w0TnAn/UzD4YzKX+OlAPtPeHh/dTt4iIJIkCsIiIJEUQnP4AfCfYdRfQQDRY/J7oXM9kuAo4jegtvj8EHiMahg7i7suJLp60Nrhteej7+WB3f5LoyOSjZrYHeIfovNx4zm0ALgnalwO/AD4Z1Bivq4nOk11OdI72V4O+/0n05/QE0VHXMcDMmPM+A9xI9Hs2gfaDY0t3A1cEK0Tfcwjn4e7lwJVEF9baCRxH9A8Yrf68gAuBZRZdQftuYGYwN3sF8B/Az4l+7y4musBVQzsf/xPg28HP/YZDqVtERJLHotNlREREpDVm9hiw3N0PdcEu6WTB7d+lwFXu/q9k1yMiIl2PRoBFRERiWPQZvWPMLMXMLgSmA3/p6DxJDjO7wMz6Brdm/zfRRcNe7+A0ERHpoVI7biIiItKjDAb+THSRp1Lg8+7+ZnJLknacRnSOcjrwLtGVxvWoIhERaZVugRYREREREZEeQbdAi4iIiIiISI/QI26BzsvL85EjRya7DBERERERETnC8vLymDdv3jx3v7Cjtj0iAI8cOZKSkpJklyEiIiIiIiIJYGZ58bTTLdAiIiIiIiLSIygAi4iIiIiISI+gACwiIiIiIiI9ggKwiIiIiIiI9AgKwCIiIiIiItIjKACLiIiIiIhIj5DQAGxmF5rZCjNbbWY3tXI8w8weC44vMLORLY4XmtleM7sh3j5FREREREREWpOw5wCbWQi4DzgPKAUWmdlcd383ptm1QIW7jzWzmcBtwIyY43cBzxxinyIiIiIiInKY6pvCbNpVw9qyataVVzOkbxaXTBqa7LKOiIQFYGAKsNrd1wKY2aPAdCA2rE4Hbg7ezwHuNTNzdzezS4G1QPUh9ikiIiIiIiLtiEScLZW1rCuPhtzmsLuuvJrSihoivr/tR08YogAchwJgU8x2KTC1rTbu3mRmlcAAM6sFvkl0pPeG1tq306eIiIiIiEiP5+7srG6IBtuyataWV7OufC/ryqtZv7OGhqbIvrbZ6SFG5WczaXhfLj2pgNF52YzKy2ZkXja5WWlJvIojK5EB2FrZ53G2+T5wl7vvNbN42h/84WbXAdcBFBYWdlisiIiIiIjI0WhvfRPry4OAW7Y/5K4tr6aqrmlfu7SQUdi/F6PyenP2+IGMCkLu6Lxs8vtk0CJ7dUuJDMClwPCY7WHAljbalJpZKpAL7CI6qnuFmd0O9AUiZlYHLI6jTwDc/X7gfoDi4uJWQ7KIiIiIiMjRoOW83OaAu668mrKq+n3tzGBobhaj87O57KSCfSF3VF42BX2zSA317AcBJTIALwLGmdkoYDMwE5jVos1c4FPAa8AVwAvu7sCZzQ3M7GZgr7vfG4TkjvoUERERERE56oQjzpbdtcEtyu3Py83rnc7IAdmcXZTPqPzs4Jbl3owY0IvMtFDyLqKLS1gADub0Xg/MA0LAg+6+zMxuAUrcfS7wW+BhM1tNdOR35uH0mahrEBEREREROZI0Lze5LDrg2r0VFxd7SUlJsssQEREREZEeoqqukfXlNawt38v68pq45uWOzs/ukfNyjwQzW+zuxR21S+Qt0CIiIiIiIt3W+52XOzqvN0P7Zvb4ebmdSQFYRERERESkDbHzclt+tTYvd1Se5uV2ZQrAIiIiIiLSo2lebs+hACwiIiIiIj1C7Lzc5lHc9W3Myx0xIBpse+rzcrsrBWAREREREek2NC9X2qMALCIiIiIiR51tlXWs3F4V97zcc8bnMyqvN6Pyemlebg+mACwiIiIiIl1eXWOYRet3MX9FGfNXlrFqx959xzQvV+KlACwiIiIiIl2Ou7N+Zw3zV+xg/soyXlu7k7rGCOmhFKaO7s/Hi4czcViu5uXKIVEAFhERERGRLqG6volX1+zkpZXRUd6Nu2oAGJWXzcxTCplWlM/U0f3pla4YI4dH/+WIiIiIiEhSuDvLt1Uxf2UZ81eUUbJhF41hp1d6iNPHDOAzZ47irKJ8RgzITnap0k0oAIuIiIiISKfZXdPAK6vLmb+ijJdWlbF9T3Rl5mMG9+E/zxjFtKJ8Jo/sR0aqFqiSI08BWEREREREEiYccd7eXBksXrWDtzbtJuKQk5nKmUX5TCvK56xx+QzOzUx2qdIDKACLiIiIiMgRtaOqjpdXljN/ZRkvryqjoqYRMzhhWF+uP3cc04rymTQsV8/alU6nACwiIiIiIu9LYzjCGxsqonN5V5axbMseIPoM3nOOGci0onzOHJdP/+z0JFcqPZ0CsIiIiIiIHLLSihpeWlnO/JU7+PfqneytbyI1xTh5RD9uvGA804ryOW5IDikpejyRdB0KwCIiIiIi0qG6xjAL1+3aN8q7esdeAAr6ZnHxpKFMK8rn9LEDyMlMS3KlIm1LaAA2swuBu4EQ8IC739rieAbwB2AysBOY4e7rzWwKcH9zM+Bmd38yOGc9UAWEgSZ3L07kNYiIiIiI9ETuzrry6n2B9/W1O6lrjJCemsLUUf2Zecpwzh6fz5j83phplFeODgkLwGYWAu4DzgNKgUVmNtfd341pdi1Q4e5jzWwmcBswA3gHKHb3JjMbAiwxs6fcvSk47xx3L09U7SIiIiIiPdHe+iZeW7OT+St3MH9lGZt21QIwOi+bmacUMm18PqeOGkBWuh5RJEenRI4ATwFWu/taADN7FJgOxAbg6cDNwfs5wL1mZu5eE9MmE/AE1ikiIiIi0iO5O8u3VUVHeVeUUbJhF41hp1d6iNPH5HHdWWOYNi6fwgG9kl2qyBGRyABcAGyK2S4FprbVJhjtrQQGAOVmNhV4EBgBXB0z+uvAs2bmwK/d/X5aYWbXAdcBFBYWHpkrEhERkSOuvinM5opaBuZk0jtDy5OIJNrumgZeXlXOS8GtzTuq6gE4dkgO135gNNOK8pk8oh/pqXpEkXQ/ifxXprWJAC1Hctts4+4LgAlmdizwezN7xt3rgDPcfYuZDQSeM7Pl7v7SQZ1Eg/H9AMXFxRpBFhER6YLe2FjBDX9awtryagD6ZKQyKDeTIbmZDM4JXnOzgtfodm5WmuYbihyCcMRZWrp731zeJZt2E3HIzUrjzHF5TCvK56yifAblZCa7VJGES2QALgWGx2wPA7a00abUzFKBXGBXbAN3f8/MqoHjgRJ33xLs32FmTxK91fqgACwiIiJdV11jmLueX8lvXlrLkNwsfnjp8VTXN7G1so7te+rYWlnHqu3l7KiqI9Liz9gZqSkxgThrXzCOBuYsBuVmkJedoUevSI+2o6oueERRGS+vKmN3TSNmMGlYX7507jimjc9n0rC+hPS/E+lhEhmAFwHjzGwUsBmYCcxq0WYu8CngNeAK4AV39+CcTcFt0SOA8cB6M8sGUty9Knh/PnBLAq9BREREjrClpbv5+p+WsGrHXmaeMpz/+eix9GnjsSlN4Qhle+vZWlnHtsq6AwLytspaFq3fxfY9dTSGD0zJaSFjYJ/MA0aOm0eSBwUjywP7ZJAa0i2e0j00hiMs3lCxby7vu1v3AJDXO4MPHjOIaePzOXNsHv2y05NcqUhyJSwAB+H1emAe0ccgPejuy8zsFqIjuXOB3wIPm9lqoiO/M4PTPwDcZGaNQAT4gruXm9lo4MngtqdUYLa7/yNR1yAiIiJHTkNThJ+/sIpfvLiG/N4ZPPTpUzh7/MB2z0kNpTAkN4shuVlttolEnJ3VDUFAro0JyNHXZVv28Px726lrjBxwXopBfp+MaDDOiQ3KB44mZ6RqtVvpmkoravYF3lfX7GRvfROpKcbkEf34xoXjmVaUz7GDc3Q3hEgMc+/+02OLi4u9pKQk2WWIiIj0WMu2VPL1Py1h+bYqLj95GN+9+Dhys1of9U0Ed6eytjEajPfsD8fbKmv3heVtlXVU1TcddO6A7PSDwnHLecm90rV4lyReXWOYBet2MX9FGfNX7mBNWXTufEHfLKaNz2daUT6njxnQ5h0VIt2ZmS129+KO2un/rUVERCRhGsMRfvniGu755yr6Zafzm08Wc95xgzq9DjOjb690+vZK59ghOW2221vftC8Mb62sjb4GgXnz7joWb6igoqbxoPNyMlP3zUce3GI0uXl/TmaqFu+SQ+LurC2vDgJvGa+v3Ul9U4T01BROHT2AWVNHMK0onzH52fpvSyROCsAiIiKSECu2VXHD40t4e3Mll0wayvcvmdDl5x/2zkhl7MDejB3Yu802dY3hg26z3jeSvKeO97buoWxvPS1vsuuVHmolIB94+3X/7HQFmR5ub30Tr64u37dic2lFLQCj87OZNbWQaUX5TB01gKx03ZovcjgUgEVEROSIagpHuP/ltfzsuVX0zkzll1edzIcnDkl2WUdMZlqIEQOyGTEgu802jeEIO6rqD7jFOvb26wVro4t3NbVY4jo9lNLGKPL+267zemdo5d5uxN15b2tVEHh3ULK+gqaIk50e4oyxeXxu2himFeUzvH+vZJcq0i0oAIuIiMgRs6ZsL1//0xLe2rSbDx8/mB9cejx5vTOSXVanSwulUNA3i4K+bS/eFY44O4MVrptHkbft2R+al5Tu5h/L6mhoOnDxrlCKMbBPRszjn7JaBOVMBvbJJD1VK1x3VRXVDbwSM8pbVlUPwHFDcvjMWaOZVpTPyYX99DMUSQAFYBEREXnfwhHnd/9ex//OW0FWeoh7PnESF58wRLfztiOUYgzMyWRgTiaThrfext2pqGncNx/5wAW86lixrYoXV5RR0xA+6Ny83hkHBeP83hmkp6aQmpJCKMVITTFSQ0ZqSkrwaqSGUmL22762aaHmVwvO3X+Ofs7tC0ecJaW7mb+ijJdWlbFk024iDn17pXHmuOjiVWeNy2NgTmaySxXp9hSARURE5H1ZX17NjXOWsGh9BR86diA//thEBvbRL/JHgpnRPzud/tnpTBia22obd6cqZvGu/bdbR0eSN+2qYeG6XVTWHrx415ESSomG4rTm19CBoTk2TKeGmtu2OLYveAevMeF8f1hvEc5j2oZCKa18vhFK2X/OAcdSDgz0B9ba4lhKyiE/SmjHnrp9I7yvrC5nd00jZnDi8L58+YPjmFaUzwnD+up2dpFOpgAsIiIihyUScR5+fQO3PrOc1JBx55WT+NjJBRoN7GRmRk5mGjmZaRQN6tNmu9qGMOV762kIRwhHnMZ9r0444jSFIzRF/MBjEScciRzUpinswWvMOZEI4eb9kchBbZq3w5HY7Qh1TQf2d3BfMX0E7yNJeIqnGe2E9iDcBwG8IRxhbfCIovw+GXzo2EFMK8rnA2PzuvxCcCLdnQKwiIiIHLJNu2r4xpylvLZ2J9OK8rn18okMyW17vqskX1Z6qNsspBSJ+EHBuzESCUL6/qDdHNxbhv2WYX3/sYPPiQ3rB/2BIDinqUXwB7hy8nCmFeVz7JA++qOQSBeiACwiIiJxc3ceWbiJHz39LmbGbZdP5OPFw/ULvnSqlBQjPbh1OAs9DkhE4qcALCIiInHZsruWbz6xlJdXlXPG2AHcdvkJDOvXPUYURUSkZ1AAFhERkXa5O48vLuUHT71LU8T5wfQJXDV1xCEvCiQiIpJsCsAiIiLSpu176vjvP7/NP5fvYMqo/txxxSQKB2jUV0REjk4KwCIiInIQd+evb23he3OXUd8U5rsXHcc1p4/UqK+IiBzVFIBFRETkAGVV9Xz7L28zb9l2Ti7syx1XTmJ0fu9klyUiIvK+KQCLiIjIPk8v3cq3//I21Q1hvvXhY/ivM0cT0qiviIh0EwrAIiIiwq7qBr7713f429KtTBqWyx1XTmLcoD7JLktEROSISklk52Z2oZmtMLPVZnZTK8czzOyx4PgCMxsZ7J9iZm8FX0vM7LJ4+xQREZFDM2/ZNs6/az7zlm3jxgvG88TnT1f4FRGRbilhI8BmFgLuA84DSoFFZjbX3d+NaXYtUOHuY81sJnAbMAN4Byh29yYzGwIsMbOnAI+jTxEREYlDZU0jNz+1jCff3MxxQ3J4+NqpHDskJ9lliYiIJEwib4GeAqx297UAZvYoMB2IDavTgZuD93OAe83M3L0mpk0m0eAbb58iIiLSgX8t38E3n1jKruoGvvLBcXzxnLGkpyb0xjAREZGk6zAAm9lA4AxgKFBLdHS2xN0jHZxaAGyK2S4FprbVJhjtrQQGAOVmNhV4EBgBXB0cj6fP5rqvA64DKCws7OgyRUREeoQ9dY388G/v8qeSUsYP6sOD15zC8QW5yS5LRESkU7QZgM3sHOAmoD/wJrCD6GjspcAYM5sD3Onue9rqopV9Hm8bd18ATDCzY4Hfm9kzcfZJcP79wP0AxcXFrbYRERHpSV5eVcY35yxl2546vnD2GL7yoXFkpIaSXZaIiEinaW8E+CPAZ9x9Y8sDZpYKXER0Lu4TbZxfCgyP2R4GbGmjTWnQZy6wK7aBu79nZtXA8XH2KSIiIjH21jfx47+/x+wFGxmdn80Tnz+dkwr7JbssERGRTtdmAHb3G9s51gT8pYO+FwHjzGwUsBmYCcxq0WYu8CngNeAK4AV39+CcTcFtzyOA8cB6YHccfYqIiEjgtTU7uXHOEjbvruUzZ47i6+ePJzNNo74iItIzxTMH+CvA74Aq4AHgJOAmd3+2vfOC8Ho9MA8IAQ+6+zIzu4XoHOK5wG+Bh81sNdGR35nB6R8AbjKzRiACfMHdy4N6DurzUC9aRESku6tpaOL2f6zgoVfXM3JALx7/7GkUj+yf7LJERESSytzbnx5rZkvcfZKZXQB8EfgO8Dt3P7kzCjwSiouLvaSkJNlliIiIdIqS9bu44fElrN9ZwzWnj+QbF46nV3oiH/wgIiKSXGa22N2LO2oXz7+GzQtPfYRo8F1iZq0tRiUiIiJJVNcY5s5nV/DAK+so6JvFI585ldPGDEh2WSIiIl1GPAF4sZk9C4wCvmVmfYjeliwiIiJdxJsbK7jh8SWsKavmqqmFfOsjx9I7Q6O+IiIiseL5l/Fa4ERgrbvXmNkA4NOJLUtERETiUd8U5u7nV/Gr+WsYnJPJw9dO4cxx+ckuS0REpEuKJwBPJ7o6c2WwHQZGA0sTVpWIiIh06J3NlXz9T0tYsb2KjxcP49sXHUdOZlqyyxIREemy4gnA33P3J5s33H23mX2Pjh+DJCIiIgnQ0BTh3n+t5r5/rWZAdjoPXlPMuccMSnZZIiIiXV48ATjlMM8TERGRI+y9rXv4+p+W8O7WPVx2UgE3XzyB3F4a9RUREYlHPEG2xMx+CtwHOPAlYHFCqxIREZEDNIUj/Gr+Gu7+5ypys9L49dWTuWDC4GSXJSIiclSJJwB/ieizfx8Ltp8Fvp2wikREROQAq7ZX8fXHl7C0tJKLThjCLdOPp392erLLEhEROep0GIDdvRq4ycx6u/veTqhJREREgHDEeeDltdz53Eqy00PcN+tkPnrCkGSXJSIictTqMACb2enAA0BvoNDMJgGfdfcvJLo4ERGRnmpt2V5ueHwJb2zczfnHDeJHl00kv09GsssSERE5qsVzC/RdwAXAXAB3X2JmZyW0KhERkR4qEnEeenU9t89bTkZqiJ/NOJHpJw7FzJJdmoiIyFEvrtWc3X1Ti394w4kpR0REpOfauLOGG+YsYeG6XZx7zEB+8rGJDMrJTHZZIiIi3UY8AXhTcBu0m1k68GXgvcSWJSIi0nNEIs4fF2zgJ88sJ2TG7VecwJWTh2nUV0RE5AiLJwB/DrgbKABKia4C/cVEFiUiItJTlFbU8M0nlvLv1Ts5c1wet11+AkP7ZiW7LBERkW4pnlWgy4GrOqEWERGRHsPdeWzRJn749Hu4Oz++bCKfmDJco74iIiIJlNJRAzO73cxyzCzNzP5pZuVm9h/xdG5mF5rZCjNbbWY3tXI8w8weC44vMLORwf7zzGyxmb0dvJ4bc86LQZ9vBV8D479cERGR5NtaWcs1v1vETX9+m+MLcvjHV89i1tRChV8REZEEi+cW6PPd/RtmdhnRW6CvBP4F/F97J5lZCLgPOC84b5GZzXX3d2OaXQtUuPtYM5sJ3AbMAMqBi919i5kdD8wjegt2s6vcvSS+SxQREeka3J0/v7GZm59aRlPY+f4lE7j61BGkpCj4ioiIdIZ4AnBa8PoR4BF33xXnX6inAKvdfS2AmT0KTAdiA/B04Obg/RzgXjMzd38zps0yINPMMty9Pp4PFhER6Wp2VNXx339+m+ff20HxiH7cceUkRuZlJ7ssERGRHiWeAPyUmS0HaoEvmFk+UBfHeQXAppjtUmBqW23cvcnMKoEBREeAm10OvNki/P7OzMLAE8AP3d1bfriZXQdcB1BYWBhHuSIiIkeeuzN3yRa+N3cZtQ1hvv3RY/n0GaMIadRXRESk08WzCNZNZnYbsMfdw2ZWQ3TktiOt/cveMqi228bMJhC9Lfr8mONXuftmM+tDNABfDfyhlbrvB+4HKC4uPiggi4iIJNrOvfV8+y/v8Mw72zhxeF/uuHISYwf2TnZZIiIiPVabAdjMPuDurwC4e0XzfnevBqrNLAcodPd32uiiFBgesz0M2NJGm1IzSwVygV3B5w8DngQ+6e5rYj5/c/BaZWazid5qfVAAFhERSaZn3t7Kt//yDlV1TXzjwvFcd+ZoUkMdrj0pIiIiCdTeCPDlZnY78A9gMVAGZAJjgXOAEcDX2zl/ETDOzEYBm4GZwKwWbeYCnwJeA64AXnB3N7O+wNPAt9z9382Ng5Dc193LzSwNuAh4Pt6LFRERSbSK6ga+N3cZc5ds4fiCHGZfeSLjB/dJdlkiIiJCOwHY3f+fmfUjGkyvBIYQnQf8HvDr5tHhds5vMrPria7gHAIedPdlZnYLUOLuc4EhvDClAAAgAElEQVTfAg+b2WqiI78zg9OvJxq0v2Nm3wn2nQ9UA/OC8BsiGn5/cxjXLSIicsQ9/+52vvXk21RUN/C184r4/NljSNOor4iISJdhrawf1e0UFxd7SYmemiQiIolRWdvILU+9yxNvlHLM4D7c+fFJTBiam+yyREREegwzW+zuxR21i2cVaBEREWnDiyt2cNMTb1O2t54vnTuWL507jvRUjfqKiIh0RQrAIiIih6GqrpEfPf0ejy7axLiBvfn11ZOZNLxvsssSERGRdigAi4iIHKJ/ry7nG3OWsrWyls9NG8NXPzSOzLRQsssSERGRDnQYgM2sF9HVngvd/TNmNg4Y7+5/S3h1IiIiXUh1fRO3PrOch1/fwOi8bB7/3OlMHtEv2WWJiIhInOIZAf4d0ccgnRZslwKPAwrAIiLSYyxYu5Mb5yxlU0UN/3nGKG68YDxZ6Rr1FREROZrEE4DHuPsMM/sEgLvXmpkluC4REZEuobYhzO3zlvPQq+sZ3q8Xj37mVKaOHpDsskREROQwxBOAG8wsC3AAMxsD1Ce0KhERkU7QFI5Q0ximtiFMdX0TNQ1hahrCVDc0UVMfZk9dI/e/tJZ15dV88rQR3PThY+iVruUzREREjlbx/Cv+PeAfwHAz+yNwBnBNIosSERGJFY44tY1hauqbqA7Cam3jgaG1pqGJ6vowtQ3RNjUN0WPV9cGxhuBY/f5j9U2RDj+7oG8Wf/yvqZwxNq8TrlREREQSqcMA7O7PmdkbwKmAAV9x9/KEVyYiIkedSBBUm0dQ9wXTVsJndcP+QNsytNbUR/uoDUZj6xo7DqrNzKBXWoheGalkp4folZ5Kr/QQuVlpDM3NJCs9RHZ6Kr0yQvRKSyU7I9omOyNEVlqI7Ixo++bzBuVk6rm+IiIi3US893EVAKGg/Vlmhrv/OXFliYhIIrkHQbU+vC9kxjNiWtPR6Gtj+JDqiA2avdKj4bN3RioD+2TsD6nB8ez01Gh4bQ6sMdv73qenkpmWgpaqEBERkdbE8xikB4ETgGVA85/gHVAAFhFJkpXbq1hXXt3miGnz6Gv1vlDbtC/w1jRE37vH/3lZadGA2isIm83BNa93BtkZzeEzJsy2GH3NzogNutHR1szUECkpCqoiIiLSeeIZAT7V3Y9LeCUiItKhlduruPPZFcxbtr3V4xmpKTG38O4Pm/2ze+3fbj4WhNSs5n37tkMHjL5mpYUIKaiKiIhINxBPAH7NzI5z93cTXo2IiLRq484afvb8Sp58azO901P52nlFnHvMwIPCroKqiIiISNviCcC/JxqCtxF9/JEB7u4nJLQyERFh+546fv7CKh5duIlQinHdWaP53Flj6JednuzSRERERI468QTgB4GrgbfZPwdYREQSqKK6gV/NX8NDr64nHHE+MaWQ688dy6CczGSXJiIiInLUiicAb3T3uQmvRERE2FvfxG9fXsdvXl5LdUMTl51UwFc/WEThgF7JLk1ERETkqBdPAF5uZrOBp4jeAg0Q12OQzOxC4G6ij1B6wN1vbXE8A/gDMBnYCcxw9/Vmdh5wK5AONAA3uvsLwTmTgYeALODvRJ9LfAhrmYqIdD11jWH+7/UN/OLFNeyqbuDCCYP52vlFFA3qk+zSRERERLqNeAJwFtHge37Mvg4fg2RmIeA+4DygFFhkZnNbLKZ1LVDh7mPNbCZwGzADKAcudvctZnY8MI/os4gBfglcB7xONABfCDwTx3WIiHQ5jeEIj5eUcs8/V7FtTx1njsvjhvPHM2l432SXJiIiItLtdBiA3f3Th9n3FGC1u68FMLNHgelAbACeDtwcvJ8D3Gtm5u5vxrRZBmQGo8X9gRx3fy3o8w/ApSgAi8hRJhJxnlq6hZ8+t5INO2uYPKIfd804kdPGDEh2aSIiIiLdVpsB2My+4e63m9nPiY74HsDdv9xB3wXAppjtUmBqW23cvcnMKoEBREeAm10OvOnu9WZWEPQT22cBrTCz64iOFFNYWNhBqSIincPdef69Hdz57AqWb6vi2CE5PHhNMeeMH4iZHmEkIiIikkjtjQC/F7yWHGbfrf0m1zJIt9vGzCYQvS36/HjaH7DT/X7gfoDi4mLNERaRpHt1dTm3z1vBW5t2Myovm59/4iQ+OnEIKXp2r4iIiEinaDMAu/tTwdsad3889piZXRlH36XA8JjtYcCWNtqUmlkqkAvsCj5jGPAk8El3XxPTflgHfYqIdClvbqzgjmdX8O/VOxmam8ltl0/k8pOHkRpKSXZpIiIiIj1KPItgfQt4PI59LS0CxpnZKGAzMBOY1aLNXOBTwGvAFcAL7u5m1hd4GviWu/+7ubG7bzWzKjM7FVgAfBL4eRzXICLS6ZZv28Odz67kuXe3MyA7ne9edByzphaSmRZKdmkiIiIiPVJ7c4A/DHwEKDCze2IO5QBNHXUczOm9nugKziHgQXdfZma3ACXBs4V/CzxsZquJjvzODE6/HhgLfMfMvhPsO9/ddwCfZ/9jkJ5BC2CJSBezYWc1dz23kr8u2ULvjFRuOL+IT58xiuyMeP7mKCIiIiKJYm09QtfMJgEnArcA3405VAX8y90rEl/ekVFcXOwlJYc7lVlEJD7bKuu454VV/GnRJlJDxqfPGMVnzxpN317pyS5NREREpFszs8XuXtxRu/bmAC8BlpjZbHdvPKLViYh0I7uqG/jli6v5/WsbcHeumlrIF88Zy8CczGSXJiIiIiIx4nkOsMKviEgrquoaeeDldTzw8lpqG8N87ORhfOWD4xjev1eySxMRERGRVmhCmojIIaprDPOH19bzixfXsLumkY9MHMzXziti7MA+yS5NRERERNqhACwiEqeGpgh/KtnEz19YxfY99UwryueG88czcVhusksTERERkTh0GIDN7Cmg5UpZlUAJ8Gt3r0tEYSIiXUU44sxdspm7nlvFxl01FI/oxz0zT2Lq6AHJLk1EREREDkE8I8BrgXzgkWB7BrAdKAJ+A1ydmNJERJLL3Xn23e3c+ewKVm7fy3FDcvjdp0/h7KJ8zCzZ5YmIiIjIIYonAJ/k7mfFbD9lZi+5+1lmtixRhYmIJIu78+/VO/nfectZUlrJ6Pxs7pt1Mh8+fjApKQq+IiIiIkereAJwvpkVuvtGADMrBPKCYw0Jq0xEJAkWb6jgjnkreG3tTgr6ZnH7FSfwsZMKSA2lJLs0EREREXmf4gnAXwdeMbM1gAGjgC+YWTbw+0QWJyLSWd7buoc7n13B8+/tIK93OjdffByfmFpIRmoo2aWJiIiIyBESz3OA/25m44BjiAbg5TELX/0skcWJiCTauvJq7npuJU8t3UKfjFRuvGA815w+kuwMLZIvIiIi0t3E+xveZGBk0P4EM8Pd/5CwqkREEmzL7lp+/sIq/lRSSnoohc9PG8NnzxpDbq+0ZJcmIiIiIgkSz2OQHgbGAG8B4WC3AwrAInLU2bm3nl+8uIaHX9+Au3P1qSP4wjljGNgnM9mliYiIiEiCxTMCXAwc5+4tnwUsInLU2FPXyAMvreW3r6yjtjHM5ScP4ysfGsewfr2SXZqIiIiIdJJ4AvA7wGBga4JrERE54mobwvz+tfX88sU1VNY28tGJQ/h/5xUxdmDvZJcmIiIiIp0sngCcB7xrZguB+uad7n5JwqoSEXmfGpoiPLZoI/e8sJqyqnrOHp/PDeeP5/iC3GSXJiIiIiJJEk8AvjnRRYiIHCnhiPOXNzdz1/MrKa2o5ZSR/bhv1slMGdU/2aWJiIiISJLF8xik+YfbuZldCNwNhIAH3P3WFscziC6mNRnYCcxw9/VmNgCYA5wCPOTu18ec8yIwBKgNdp3v7jsOt0YR6R7cnXnLtnHnsytZtWMvE4bm8MNLj2daUT5mluzyRERERKQLaDMAm9kr7v4BM6siuurzvkOAu3tOex2bWQi4DzgPKAUWmdlcd383ptm1QIW7jzWzmcBtwAygDvgOcHzw1dJV7l7S8eWJSHfn7ry8qpw7nl3B0tJKxuRn84urTubCCYNJSVHwFREREZH92gzA7v6B4LXPYfY9BVjt7msBzOxRYDoQG4Cns/8W6znAvWZm7l4NvGJmYw/zs0WkB1i8YRe3/2MFC9btoqBvFv97xQlcdlIBqaGUZJcmIiIiIl1QPHOAm0dzB8W2d/eNHZxWAGyK2S4FprbVxt2bzKwSGACUd9D378wsDDwB/LC1RzSZ2XXAdQCFhYUddCciR5NlWyq589mVvLB8B3m9M/j+JROYOWU4GamhZJcmIiIiIl1YhwHYzL4EfA/YDkSC3Q6c0NGprexrGVTjadPSVe6+2cz6EA3AVxOdR3xgJ+73A/cDFBcX6xnGIt3A2rK9/PS5lfxt6VZyMlP5xoXjueb0kfRKj+tveSIiIiLSw8XzW+NXgPHuvvMQ+y4FhsdsDwO2tNGm1MxSgVxgV3uduvvm4LXKzGYTvdX6oAAsIt3Hlt213PPPVTy+uJSM1BSuP2csnzlrNLlZackuTURERESOIvEE4E1A5WH0vQgYZ2ajgM3ATGBWizZzgU8BrwFXAC+0djtzsyAk93X3cjNLAy4Cnj+M2kTkKFC+t55f/GsN//f6BgA+edoIvnD2WPL7ZCS5MhERERE5GsUTgNcCL5rZ00B98053/2l7JwVzeq8H5hF9DNKD7r7MzG4BStx9LvBb4GEzW0105Hdm8/lmth7IAdLN7FLgfGADMC8IvyGi4fc38V6siBwdKmsbeeDltfz2lXXUNYa5cvJwvvyhcRT0zUp2aSIiIiJyFIsnAG8MvtKDr7i5+9+Bv7fY992Y93XAlW2cO7KNbicfSg0icvSoaWji969u4Ffz11BZ28hFJwzh/51XxJj83skuTURERES6gQ4DsLt/vzMKEZGeq6EpwqOLNvLzF1ZTVlXPuccM5OvnFzFhaG6ySxMRERGRbiSeVaCLgBuAkRz4GKRzE1eWiPQE4Yjz5Jub+dnzKymtqGXKqP788qqTKR7ZP9mliYiIiEg3FM8t0I8DvwIeAMKJLUdEegJ35x/vbOPO51ayesdeJhbk8uPLJnLmuDzMWns6moiIiIjI+xdPAG5y918mvBIR6fbcnZdWlXPHvBW8vbmSsQN786v/OJkLJgxW8BURERGRhIsnAD9lZl8AnuTAVaDbfV6viEisRet38b/zVrBw3S6G9cvizisncelJBYRSFHxFREREpHPEE4A/FbzeGLPPgdFHvhwR6W7e2VzJnc+u4F8rysjvk8EPpk9gximFpKemJLs0EREREelh4lkFelRnFCIi3cuasr389LmVPL10K7lZadz04WP41GkjyUoPJbs0EREREemh4lkF+pOt7Xf3Pxz5ckTkaFdaUcM9/1zFnMWlZKaF+PK5Y7n2zNHkZqUluzQRERER6eHiuQX6lJj3mcAHgTcABWARASAScV5ds5PZCzfw7LLtpKQYnz5jFJ8/ewx5vTOSXZ6IiIiICBDfLdBfit02s1zg4YRVJCJHjfK99TxeUsqjizayYWcNfXulcc3pI/nPD4xiaN+sZJcnIiIiInKAeEaAW6oBxh3pQkTk6ODuvLZmJ39cuJFnl22jMexMGdWfr51XxAUTBpOZpjm+IiIiItI1xTMH+Cmiqz4DpADHAY8nsigR6Xp2VTcwZ/EmHlm4iXXl1eRmpXH1qSOZNXU4Ywf2SXZ5IiIiIiIdimcE+I6Y903ABncvTVA9ItKFuDuvr93F7IUbmffONhrCEYpH9ONL547lIxOHaLRXRERERI4q8cwBnh+7bWYhM7vK3f+YuLJEJJkqqht44o1SZi/cyNqyanIyU5k1tZBZUwspGqTRXhERERE5OrUZgM0sB/giUADMBZ4Ltm8E3gIUgEW6EXdn0foKZi/YwN/f2UZDU4STC/tyx5WT+OjEIXp+r4iIiIgc9dobAX4YqABeA/6LaPBNB6a7+1vxdG5mFwJ3AyHgAXe/tcXxDKKPU5oM7ARmuPt6MxsAzCH6CKaH3P36mHMmAw8BWcDfga+4uyMih2V3TQN/fmMzsxduZPWOvfTJSGXmKcOZNbWQYwbnJLs8EREREZEjpr0APNrdJwKY2QNAOVDo7lXxdGxmIeA+4DygFFhkZnPd/d2YZtcCFe4+1sxmArcBM4A64DvA8cFXrF8C1wGvEw3AFwLPxFOTiES5O4s3VDB7wUaefnsr9U0RThzel9svP4GLJg2hV/rhLBAvIiIiItK1tfdbbmPzG3cPm9m6eMNvYAqw2t3XApjZo8B0IDYATwduDt7PAe41M3P3auAVMxsb26GZDQFy3P21YPsPwKUoAIvEpbK2kSffKOWRhZtYsb2K3hmpXFk8jE9MKWTC0NxklyciIiIiklDtBeBJZrYneG9AVrBtgLt7R/dGFgCbYrZLgalttXH3JjOrBAYQHW1uq8/YFahLg30HMbPriI4UU1hY2EGpIt2Xu/Pmpt3MXrCRvy3dQl1jhBOG5XLrxyZy8aShZGdotFdEREREeoY2f/N19/e74o211u1htDms9u5+P3A/QHFxseYIS4+zp66Rv7y5mdkLNrJ8WxXZ6SEuO2kYV00t5PgCjfaKiIiISM+TyKGfUmB4zPYwYEsbbUrNLBXIBXZ10OewDvoU6bHcnSWllcxesIGnlmyltjHM8QU5/PiyiVxy4lB6a7RXRERERHqwRP42vAgYZ2ajgM3ATGBWizZzgU8RXWn6CuCF9lZ0dvetZlZlZqcCC4BPAj9PRPEiR5Oqukb++tYWZi/YyLtb99ArPcT0E4cya2ohJwzrm+zyRERERES6hIQF4GBO7/XAPKKPQXrQ3ZeZ2S1AibvPBX4LPGxmq4mO/M5sPt/M1gM5QLqZXQqcH6wg/Xn2PwbpGbQAlvRgS0t388jCjfz1rS3UNIQ5dkgOP7j0eC49cSh9MtOSXZ6IiIiISJdiPeERusXFxV5SUpLsMkSOiL31Tcx9awuzF27gnc17yExL4eIToqO9Jw7vi1lrU+VFRERERLovM1vs7sUdtdOEQJGjxDubK5m9cCN/fXMz1Q1hxg/qw/cvmcClJxWQm6XRXhERERGRjigAi3RhNQ1NPLUkOrd3SWklGakpXBSM9p5cqNFeEREREZFDoQAs0gW9u2UPsxdu4C9vbmFvfRPjBvbmexcfx8dOGkZuL432ioiIiIgcDgVgkS6itiHMU0ujo71vbdpNemoKH504hFlTCyke0U+jvSIiIiIi75MCsEiSrdhWxewFG/jzm5upqmtiTH4237noOD52UgH9stOTXZ6IiIiISLehACySBHWNYf62dCuPLNzI4g0VpIdS+PDEwcyaUsiUUf012isiIiIikgAKwCKdaNX2Kv64YCN/fqOUPXVNjM7L5n8+ciyXTx5Gf432ioiIiIgklAKwSILVNYZ55p2tzF6wkUXrK0gLGRdMGMysqYWcNnqARntFRERERDqJArBIgqzesZdHFm7kiTdK2V3TyMgBvfjWh4/hisnDGNA7I9nliYiIiIj0OArAIkdQfVOYf7yzjT8u2MjCdbtITTlwtDclRaO9IiIiIiLJogAscgSsLYuO9s5ZXEpFTSOF/XvxjQvHc+Xk4eT30WiviIiIiEhXoAAscpgamiLMW7aN2Qs28tranaSmGOcdN4hZUws5Y0yeRntFRERERLoYBWCRQ7S+vJpHFm1kTkkpO6sbGNYvixsvGM+Vk4cxMCcz2eWJiIiIiEgbFIBF4tDQFOG5d7fzyMKNvLK6nFCK8cFjBjJraiFnjcvXaK+IiIiIyFFAAVikHRt31vDIoo08XlJK+d56Cvpm8bXziphxynAGabRXREREROSoogAs0kJjOMI/39vOHxds5OVV5aQYnHvMIK6aWshZRfmENNorIiIiInJUSmgANrMLgbuBEPCAu9/a4ngG8AdgMrATmOHu64Nj3wKuBcLAl919XrB/PVAV7G9y9+JEXoP0HJt21fDYok08VrKJsqp6huRm8tUPjWPGKcMZkpuV7PJEREREROR9SlgANrMQcB9wHlAKLDKzue7+bkyza4EKdx9rZjOB24AZZnYcMBOYAAwFnjezIncPB+ed4+7liapdeo6mcIR/Lt/B7AUbeWlVGQacPX4gs6YUcvb4fFJDKckuUUREREREjpBEjgBPAVa7+1oAM3sUmA7EBuDpwM3B+znAvWZmwf5H3b0eWGdmq4P+XktgvdKDbN5dy2MLN/JYySa276lnUE4GXzo3Otpb0FejvSIiIiIi3VEiA3ABsClmuxSY2lYbd28ys0pgQLD/9RbnFgTvHXjWzBz4tbvf39qHm9l1wHUAhYWF7+9KpFtoCkd4cUUZsxdu5MUVO3BgWlE+P5heyLnHDNRor4iIiIhIN5fIANzaSkEeZ5v2zj3D3beY2UDgOTNb7u4vHdQ4GozvByguLm75udIDuDulFbUsWLeLhet28tLKcrbtqSO/TwZfOHssM04ZzvD+vZJdpoiIiIiIdJJEBuBSYHjM9jBgSxttSs0sFcgFdrV3rrs3v+4wsyeJ3hp9UACWnsfdWVdeHQTeXSxYu5MtlXUA5GalMXVUf24++Tg+eOwg0jTaKyIiIiLS4yQyAC8CxpnZKGAz0UWtZrVoMxf4FNG5vVcAL7i7m9lcYLaZ/ZToIljjgIVmlg2kuHtV8P584JYEXoN0YZGIs3JHVRB2d/3/9u48uK6rPuD49yd5i63E9pPjLHZsPSdOiYEhcYRkCCEZSEPaMpiWMJgyEKhn0jAJS1raQjsdGOh0oGWgUJYOJSnLQNNglrq0xQ0J7bTTRl4SshPisezEWQi2bMVO4kXSr3+8Y/fV8Rrr6Wn5fmbe6J1zzz33SP756P107z2Xnt4+tu3eC8Cctil0V9v53WqF7kUVzp97Ki0+vkiSJEma0BqWAJd7em8A1lB7DNLNmflARHwcWJ+Zq4GbgG+WRa76qCXJlHa3UlswawC4PjMHI+IM4Pu1dbKYBHw7M3/UqO9Bo8vA4BAPPbmLnt7t9PT2sW5zHzuf2w/AWTOn8Zrz2ule1E5XtcKiOTMocSJJkiRJAETm+L89trOzM9evX9/sYegE7RsY4r7H++np3c7a3j7Wb97B7r0DACxsn053tUJXtZ3uaoX5s08x4ZUkSZImqIjYkJmdx2rXyEugpROyZ/8gdz+6k7W9fazdvJ0NW3awZ/8QAOfNbWP5hWfTVa3QXW3nzJnTmjxaSZIkSWONCbCa5tm9A2zYsqN2D2/vdu55rJ99g0NEwAVnnsaKVy5g2aIKnR0V5rRNbfZwJUmSJI1xJsAaMf3P72f95tpiVT29fdz/eD+DQ0lrS/CyeTN59yUddFcrdC6sMHP65GYPV5IkSdI4YwKshtm+ey/rNvdx56baY4keeuoZMmFKawuvOGcm773sXLqqFZYunE3bVENRkiRJUmOZdWjY/OKZPdy5aXu5pLmPjU/vBmDa5BYuXjibD77+fLqqFS5aMItpk1ubPFpJkiRJE40JsF6UzGTrjufp6e1jbXks0ZbtzwHQNnUSnR2z+a2l8+iutvPyeTOZMqmlySOWJEmSNNGZAOu4ZCabtj1bO7tbzvI+0b8HgFnTJ/PKjgrvXLaQ7mo7F5x1KpNaTXglSZIkjS4mwDqsoaHk50/voqfcv9vT28e23XsBmNM2le5FFa6rVuiqVjh/7qm0tPgMXkmSJEmjmwmwABgYHOKhJ3fRUy5nXre5j53P7Qfg7JnTuHTxnPIM3grVOTOIMOGVJEmSNLaYAE9Q+waGuO/xnbVHEm3qY8OWHezeOwBAR/t0rlxyBt3VdrqqFc6pTG/yaCVJkiTp5JkATxB79g9y96M76emt3b9716M72LN/CIDFc9tYfuHZdC9qp6ujwpkzpzV5tJIkSZI0/EyAx6ndewe4a8uOgwnvPY/1s29wiAi44MzTeHvXArqrFV7ZUaG9bWqzhytJkiRJDWcCPE70P7efdZv7WLu5tkrz/U88w+BQ0toSvHzeTN5zSQdd1QqdHRVmnjK52cOVJEmSpBFnAjxGbdu9l3Vldeae3j5+9tQzZMKU1hYuPGcW773sXLoXVVi6YDYzpvrPLEmSJElmRmPEU/17Dq7QvLa3j41P7wZg2uQWLl44mxuvOJ+uaoULz5nFtMmtTR6tJEmSJI0+JsCjUGaydcfz3Lmpdv/u2s19bNn+HABtUyfR2TGbtyydT1e1wsvnzWTKpJYmj1iSJEmSRr+GJsARcRXwOaAV+GpmfvKQ7VOBbwAXA9uBt2Xm5rLtI8BKYBB4f2auOZ4+x6LMZNO2Z+nZ1Mfacpb3yf49AMyaPpmujgrvXLaQZYvaueCs02ht8Rm8kiRJknSiGpYAR0Qr8EXgV4GtwLqIWJ2ZD9Y1WwnsyMzzImIF8CngbRGxBFgBvBQ4G/hxRJxf9jlWn2PKt3se5TO3Pcy23fsAmNM2le5FFZZVK3RV21k8t40WE15JkiRJOmmNPAPcBWzMzE0AEXELsByoT1aXAx8r71cBX4iIKPW3ZOZeoDciNpb+OI4+x5S5p07l0sWn012t0FWtUJ0zg9qPQJIkSZI0nBqZAM8DHqsrbwW6j9QmMwcioh9oL/V3HrLvvPL+WH0CEBHXAtcCLFiw4MV9ByPgiiVncMWSM5o9DEmSJEka9xq5etLhTmPmcbY50foXVmZ+JTM7M7Pz9NNPP+pAJUmSJEnjXyMT4K3AOXXl+cATR2oTEZOAmUDfUfY9nj4lSZIkSXqBRibA64DFEVGNiCnUFrVafUib1cA15f3VwB2ZmaV+RURMjYgqsBhYe5x9SpIkSZL0Ag27B7jc03sDsIbaI4tuzswHIuLjwPrMXA3cBHyzLHLVRy2hpbS7ldriVgPA9Zk5CHC4Phv1PUiSJEmSxo+onXAd3zo7O3P9+vXNHoYkSZIkqQEiYkNmdh6rXSMvgZYkSZIkadQwAZYkSZIkTQgmwJIkSZKkCWFC3AMcEYkW0+0AAAf0SURBVL8EtjR7HEcxB9jW7EFIw8iY1nhiPGs8MZ413hjTghIDmXnVsRpOiAR4tIuI9cdzw7Y0VhjTGk+MZ40nxrPGG2NaJ8pLoCVJkiRJE4IJsCRJkiRpQjABHh2+0uwBSMPMmNZ4YjxrPDGeNd4Y0zoh3gMsSZIkSZoQPAMsSZIkSZoQTIAlSZIkSROCCXCTRcRVEfFwRGyMiA83ezzSARFxTkT8JCIeiogHIuIDpb4SEbdFxCPl6+xSHxHx+RLL90bE0rq+rintH4mIa+rqL46I+8o+n4+IGPnvVBNJRLRGxN0R8cNSrkZET4nNf4iIKaV+ailvLNs76vr4SKl/OCLeUFfvfK4RExGzImJVRPyszNOvcn7WWBYRN5bPG/dHxN9HxDTnaDWCCXATRUQr8EXg14AlwNsjYklzRyUdNAD8fmZeACwDri/x+WHg9sxcDNxeylCL48XldS3wZaglzMBHgW6gC/jogQ9lpc21dfsd8+Hl0kn6APBQXflTwGdLPO8AVpb6lcCOzDwP+GxpR/k/sAJ4KbV4/VJJqp3PNdI+B/woM18CvIJaXDs/a0yKiHnA+4HOzHwZ0EptrnWO1rAzAW6uLmBjZm7KzH3ALcDyJo9JAiAzn8zMu8r7XdQ+XM2jFqNfL82+Dry5vF8OfCNr7gRmRcRZwBuA2zKzLzN3ALcBV5Vtp2Xm/2RtNb5v1PUlDbuImA/8BvDVUg7gdcCq0uTQeD4Q56uA15f2y4FbMnNvZvYCG6nN5c7nGjERcRrwWuAmgMzcl5k7cX7W2DYJOCUiJgHTgSdxjlYDmAA31zzgsbry1lInjSrl0qKLgB7gjMx8EmpJMjC3NDtSPB+tfuth6qVG+SvgD4GhUm4HdmbmQCnXx+DBuC3b+0v7E41zqREWAb8E/q5c0v/ViJiB87PGqMx8HPg08Ci1xLcf2IBztBrABLi5Dnc/jc+l0qgSEW3Ad4EPZuYzR2t6mLp8EfXSsIuINwJPZ+aG+urDNM1jbDOeNRpMApYCX87Mi4Bn+b/LnQ/HeNaoVi69Xw5UgbOBGdQuVz6Uc7ROmglwc20FzqkrzweeaNJYpBeIiMnUkt9vZeb3SvUvyuVxlK9Pl/ojxfPR6ucfpl5qhEuAN0XEZmqXvr2O2hnhWeVyO/j/MXgwbsv2mUAfJx7nUiNsBbZmZk8pr6KWEDs/a6y6AujNzF9m5n7ge8CrcY5WA5gAN9c6YHFZ4W4KtZv2Vzd5TBJw8P7Im4CHMvMzdZtWAwdWCr0G+Me6+neV1UaXAf3lErw1wJURMbv8hfdKYE3ZtisilpVjvauuL2lYZeZHMnN+ZnZQm2vvyMx3AD8Bri7NDo3nA3F+dWmfpX5FWYG0Sm1xoLU4n2sEZeZTwGMR8Sul6vXAgzg/a+x6FFgWEdNLzB2IaedoDbtJx26iRsnMgYi4gdovoFbg5sx8oMnDkg64BHgncF9E/LTU/THwSeDWiFhJ7RfWW8u2fwF+ndqCE88B7wHIzL6I+AS1Xz4AH8/MvvL+vcDXgFOAfy0vaST9EXBLRPwZcDdlUaHy9ZsRsZHaWYUVAJn5QETcSu2D2QBwfWYOAjifa4S9D/hW+TC/idqc24Lzs8agzOyJiFXAXdTm1ruBrwD/jHO0hlnU/lgiSZIkSdL45iXQkiRJkqQJwQRYkiRJkjQhmABLkiRJkiYEE2BJkiRJ0oRgAixJkiRJmhBMgCVJaoCI2H0CbS+PiFfXld8cEUtO8viXR8QPT6aPI/T7tYi4+jD1HRHx28N9PEmShpMJsCRJzXc58Oq68puBE0qAI2LScA7oRegATIAlSaOaCbAkSSMkIk6PiO9GxLryuiQiOoDrgBsj4qcRcRnwJuAvS/nc8vpRRGyIiP+MiJeU/r4WEZ+JiJ8AnzrKcWdExM3lmHdHxPJS/+6I+F7p+5GI+Iu6fVZGxM8j4t8j4m8j4gt1Xb42Iv47IjbVnQ3+JHBpGfONw/lzkyRpuDT7r8WSJE0knwM+m5n/FRELgDWZeUFE/A2wOzM/DRARq4EfZuaqUr4duC4zH4mIbuBLwOtKn+cDV2Tm4FGO+yfAHZn5OxExC1gbET8u2y4ELgL2Ag9HxF8Dg8CfAkuBXcAdwD11/Z0FvAZ4CbAaWAV8GPhQZr7xRf90JElqMBNgSZJGzhXAkog4UD4tIk492g4R0Ubt8ujv1O03ta7Jd46R/AJcCbwpIj5UytOABeX97ZnZX471ILAQmAP8R2b2lfrvUEu0D/hBZg4BD0bEGcc4tiRJo4YJsCRJI6cFeFVmPl9fWZfYHmmfnZl54RG2P3scxw3gLZn58CHH7aZ25veAQWqfDY46oEP2OVZbSZJGDe8BliRp5PwbcMOBQkQcSGp3AfVngg+WM/MZoDci3lr2iYh4xQkedw3wviiZdkRcdIz2a4HLImJ2WVzrLcdxjEO/B0mSRh0TYEmSGmN6RGyte/0e8H6gMyLuLZcbX1fa/hPwm2UBqUuBW4A/KAtWnQu8A1gZEfcADwDLT3AsnwAmA/dGxP2lfESZ+Tjw50AP8GPgQaD/GMe4FxiIiHtcBEuSNFpFZjZ7DJIkaZSJiLbM3F3OAH8fuDkzv9/scUmSdDI8AyxJkg7nYxHxU+B+oBf4QZPHI0nSSfMMsCRJkiRpQvAMsCRJkiRpQjABliRJkiRNCCbAkiRJkqQJwQRYkiRJkjQhmABLkiRJkiaE/wU0sHQNRO3USgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1152x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_count_sort()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can observe the complexity is linear"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Alphabetical sort based on counting sort"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The goal is to sort an array of strings alphabetically.\n",
    "What we will do first is to create a matrix containing for each position a value that corresponds to a letter.\n",
    "Then we will order this matrix starting from the first column to the last one.\n",
    "We must be careful NOT to reorder the columns already ordered because in that case we could remix."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to convert all letter of our words in number and store them in a matrix to compare them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_matrix(words):  \n",
    "    \n",
    "    m = max(map(len, words))\n",
    "    n = len(words)\n",
    "    arrays = np.zeros((n, m+1), dtype=int)\n",
    "    arrays[:,-1] = range(n)\n",
    "    for i,word in enumerate(words):\n",
    "        arrays[i,range(len(word))] = list(map(ord, word.lower()))\n",
    "        \n",
    "    return arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given an ordered array, if there is 2 or more equal value, merge them in a 'group'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_group(array, index):\n",
    "    problem_list = list()\n",
    "    problem = [index[0]]\n",
    "    for i in range(1,len(array)):\n",
    "        if array[i] == array[i-1]:\n",
    "            problem.append(index[i])\n",
    "        else:\n",
    "            problem_list.append(problem)\n",
    "            problem = [index[i]]\n",
    "    problem_list.append(problem)\n",
    "    return problem_list\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[5], [4, 3], [0], [1], [2]]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "make_group([0, 1, 1, 3, 6, 9], [5, 4, 3, 0, 1, 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you need to see how many equal values there are for each 'column'. \n",
    "If it is one it means that the value is in the correct position, if it is not so, then we apply the previous function to those 'column' and we'll order them.\n",
    "This is a recursive function becouse if there are further equal values after those found then I call the function and I also order and concaten them.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sort_eq_col(arrays, i):\n",
    "    \n",
    "    if arrays.shape[0] == 1:\n",
    "        return arrays\n",
    "    else:\n",
    "        # reorder equal columns (splitted by 'group') \n",
    "        sorted_array, order = counting_sort(arrays[:,i])\n",
    "        arrays = arrays[order,:]\n",
    "        \n",
    "        if (i+1) < arrays.shape[1] :\n",
    "            list_groups = make_group(sorted_array,list(range(arrays.shape[0])))\n",
    "            list_array = list()\n",
    "            for problem in list_groups:\n",
    "                list_array.append(sort_eq_col(arrays[problem,:], i+1))\n",
    "            arrays = np.concatenate(list_array, axis = 0)\n",
    "        return arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Finally we have our function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sort words in alphabetical order"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def alpha_counting_sort():\n",
    "    words=list(map(str,input(\"Enter words: \").split()))\n",
    "    arrays = create_matrix(words)\n",
    "   \n",
    "    sorted_array = sort_eq_col(arrays, 0)\n",
    "    order = list(map(int, sorted_array[:,-1]))\n",
    "    return([words[i] for i in order])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter words: ciao hello english italian Roma Rome\n",
      "['ciao', 'english', 'hello', 'italian', 'Roma', 'Rome']\n"
     ]
    }
   ],
   "source": [
    "sorted_words_list = alpha_counting_sort()\n",
    "print(sorted_words_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running Time for alphabetical sort"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Theoretical"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This algorithm was implemented through a recursive function. In particular, putting the words 'one above the other', when there are two identical letters on the same 'column', a group of letters to be ordered is created and the function is called.\n",
    "The worst case occurs if the words are all the same so we have to call the function L times, ie the word length.\n",
    "So, since the counting sort algorithm has a running time of O (k + n), we have to repeat this L times.\n",
    "So for this algorithm the complexity will be O(Lk + Ln)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Empirical"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Adjustment to the algorithm that first took the words that the user wrote, now the words are already present in an external array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def alpha_counting_sortC(words):\n",
    "    arrays = create_matrix(words)\n",
    "    sorted_array = sort_eq_col(arrays, 0)\n",
    "    order = list(map(int, sorted_array[:,-1]))\n",
    "    return([words[i] for i in order])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "chr_vec = np.vectorize(chr)\n",
    "def plot_alpha_sort():\n",
    "    # we need to fix the number of words\n",
    "    m = 50\n",
    "    time_tot = list()\n",
    "    # change n from 10 to 10000 with steps of 1000\n",
    "    for n in range(10,10000,1000):\n",
    "        time_ = list()\n",
    "        \n",
    "        ordinals_list = np.random.randint(ord('A'), ord('z')+1, (m,n))\n",
    "        chr_lists = chr_vec(ordinals_list)\n",
    "        words_list = list()\n",
    "        # convert ordinals to words\n",
    "        for i in range(chr_lists.shape[0]):\n",
    "            words_list.append(\"\".join(chr_lists[i,:]))\n",
    "        start_time = time.time()\n",
    "        sorted_words_list = alpha_counting_sortC(words_list)\n",
    "        end_time = time.time()\n",
    "        time_.append(end_time - start_time)\n",
    "        time_tot.append(sum(time_))\n",
    "   \n",
    "    # PLOT\n",
    "    fig = plt.figure(figsize=(16,4))\n",
    "    plt.plot(range(10,100000,10000),time_tot)\n",
    "    plt.xlabel('Number of words')\n",
    "    plt.ylabel('Running time (secs)')\n",
    "    plt.title('Running time for fixed word lenght')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7kAAAEWCAYAAACjclDSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xl8nWd95/3PT5Ilb5KsxYsk25J3J3acOLGzkBDWEEICSShLytKU0tIp0Hb6tGXKPO1Al2cKvDpD6QPtDMMyLB3oU5rQLEDYaUlCEi+JY2ch3m1Z3iRZkiVb6/X8cd8WivEiO5ZlS5/36+WXdc65z31+R75zoq+u6/pdkVJCkiRJkqSxoGC0C5AkSZIk6Vwx5EqSJEmSxgxDriRJkiRpzDDkSpIkSZLGDEOuJEmSJGnMMORKkiRJksYMQ64k6YITEd+OiLvP02v954j43Hl6rTsjYldEHI6IlefgfDMj4t8ioiMi/ttIvZeI+PWI+Om5Pu9Z1PHjiPjNkzz20Yj46nmo4by8jiTp7BWNdgGSpAtTRGwHZgL9wGHgO8AHU0qHR/q1U0q3jMR5I+KVwFdTSrOHvNZ/HYnXOom/Ifse/us5Ot/7gINAWXLj+wtC/t/Nb6aUvj/atUjSeOVIriTpVN6YUpoKXAGsBD48yvVc7OqBTWfzxIgoPMn5nhlrATci/CW8JOmsGXIlSaeVUtoLPEQWdoFfnjp6/JTWiEgR8R8i4oWIaI2Iz0REDD02Iv4mf2xbRNxyonMP49h5Q6bsfj9/nV+aThoRU4BvA7X5dOHDEVE7dPppRDTkdb8nn1bcmr+H1RGxISIORcSnjzvvb0TEs/mxD0VE/QleuyQiDgOFwFMRsSW//5L8vR6KiE0R8aYhz/nfEfEPEfGtiOgEXnXcOf83cDfwofy9vPa49/L2iNgaEWX57VsiYm9ETM9vL42I70VES0Q8HxFvG3Luqoi4LyLaI+JxYMHx72nIsV+KiD/Mv67Lv3/vz28vzM9/7N/9tyJic37ffRFRO+Q8KSI+EBEvAC/k990UEc9FRFv+fY+T1XGCuq6NiEfy7+1T+Sj+scd+HBF/GREP59fNdyOiesjjvxYROyKiOSL+LCK2R8Rrh5y+OCK+nD93U0Ssyp/3FWAucH/+b/Kh4dYrSTp3DLmSpNOKiNnALcDmM3zqbcBq4HLgbcDNQx67BngeqAY+AXz+WBg6gVMd+3+Ax4Eq4KPAu090gpRSZ/4e9qSUpuZ/9pzi9RYBbwf+Fvi/gdcCy4C3RcQrACLiDuA/A28GpgP/DnztBK/dnY+IA1yeUloQEROA+4HvAjOA3wX+MSKWDHnqO4D/BygFfnrcOX8d+EfgE/l7+f5xj/8T8CjwdxFRBXyebBrtgTzwfy//3s0AfhX4+4hYlj/9M8BRoAb4jfzPyfwEeGX+9SuArfnfADcC/55SShHxauCvya6DGmAH8PXjznUH2ff+0jx0/gvwp2T/7luA609Rx6CIqAMeBP4KqAT+CPiXYwE/9w7gPfn7L86PISIuBf4eeGdeZzlQd9xLvCmvfRpwH/BpgJTSu4Gd5DMgUkqfGE69kqRzy5ArSTqVb0ZEB7AL2A985Ayf/7GU0qGU0k7gRwwZCQZ2pJT+V0qpH/gSWaCYeZLznPDYiJhLFqL/S0qpJ6X0U7LQ8VL9ZUrpaErpu0An8LWU0v6UUiNZkD3WNOq3gb9OKT2bUuoD/itwxYlGc0/gWmAq2feoJ6X0Q+ABssB5zL+mlB5OKQ2klI6exfv4APBq4MfA/SmlB/L7bwO2p5S+mFLqSymtIwuUb4lsWvSvkH1PO1NKG8m+5yfzE+DlEVFAFmo/wS/C6CvyxyELjV9IKa1LKXWTTX2/LiIahpzrr1NKLSmlI8AbyKZifyOl1Ev2y4a9w3zf7wK+lVL6Vv69+x6wJj/nMV9MKf08f63/j19cm2/Jv1c/TSn1AP8FOH46+E/zc/cDXyH7JY4k6QJhyJUkncodKaVSspG6pWQjamdiaCjpIgt1v/RYSqkr/3Lo4yc8z3HH1gItQ+6DLJC/VPuGfH3kBLeP1VkPfCqfEnsIaCGbUnv8yN+J1AK7UkoDQ+7bcdxzX9J7SSkdAv4ZWA78tyEP1QPXHKs7r/2dwCyyEemi4157xyleYwtZY7IrgJeTBfU9+Yj00JBbO/Q8eQOzZk7+fmuH3s7XHQ/3+1EPvPW493cD2S9HjjnZtXn863bldQ51/HMnhuuIJemCYciVJJ1WSuknwP8m6w58TCcwecjtWeezplwTUBkRQ+uYc4rjz3WDpl3Ab6eUpg35Myml9MgwnrsHmJOPgB4zF2gccvsl1RsRV5BNNf4a8HfH1f2T4+qemlL6HeAA0MeLv49zT/NSPyEbAS3OR7t/AvwaUAE8mR+zhyx8HqttCtkU85O936ahNeTT00/1bzvULuArx72/KSmljw3juU3AYPftiJiU1zlcY6oJmCRdjAy5kqTh+lvgpjw4QRZe3hwRkyNiIfDe811QSmkH2TTUj0ZEcURcB7zxFE/ZB1RFRPk5KuF/AB8+tpY1Isoj4q3DfO5jZL8o+FBETMgbI72RX16nelYiYiLwVbI1w+8B6o41hCIbbV0cEe/OX3tCZM21Lsmn4N5D9j2dnK9RPd2exT8BPgj8W377x2RrjH+anw+y9b/viYgrIqKEbGr3Yyml7Sc554PAsoh4cz5K+nsM/xcpXwXeGBE3R0RhREyMiFfma8tP5xv5c18WEcXAn3MGDa/IrrH5Z3C8JOkcM+RKkoYlpXQA+DLwZ/ldnwR6yH6o/xJZE6TR8E7gOrIppX8F/BPQfaIDU0rPkY1qbs2nsdae6LjhSindC3wc+HpEtAMbyZpbDee5PWQNjG4h2+v274Ffy2s8F/4a2J1S+od8Dey7gL+KiEUppQ7gdcBdZCOse/P3UZI/94Nk03f3ko3gf/E0r/UTsuZYx0LuT8lG+Y/dJqX0A7Jr51/IRksX5K9/Qimlg8BbgY+R/dsuAh4exvsmpbQLuJ0s4B8gG9n9Y4bxc09KaRNZQP96XmcH2Xr0E15TJ/DXwJ/m19cfDfM5kqRzKMbY1nqSpHEuIv4JeC6ldKZNsqRfEhFTgUPAopTSttGuR5J0eo7kSpIuavk02wURURARrycbwfvmaNeli1dEvDGfqj2FbB3608D20a1KkjRchlxJ0sVuFtka0MNkzZV+J6W0flQr0sXudrJp3HvIpknflZz6JkkXDacrS5IkSZLGDEdyJUmSJEljxpjZuLy6ujo1NDSMdhmSJEmSpBGwdu3agyml6ac7bsyE3IaGBtasWTPaZUiSJEmSRkBE7BjOcU5XliRJkiSNGYZcSZIkSdKYYciVJEmSJI0ZhlxJkiRJ0phhyJUkSZIkjRmGXEmSJEnSmGHIlSRJkiSNGYZcSZIkSRqnDnX18KPn9vM3Dz3P7tau0S7nnCga7QIkSZIkSSMvpcTWg52s3dHKuh2trN3Rygv7DwNQWBBcPmcasysmj3KVL50hV5IkSZLGoKO9/Ty16xBrd/4i1LZ29QJQPmkCV9VXcMfKOq6cW8Hlc8qZXDw24uHYeBeSJEmSNM7taz/K2h2trNneytqdrWxqbKNvIAEwf/oUbrp0JlfVV3BVfQXzq6dSUBCjXPHIMORKkiRJ0kWmr3+A5/Z2sG5nHmp3tNJ46AgAJUUFXD5nGu+7cT5X1Vewcm4FlVOKR7ni88eQK0mSJEkXuLYjvazPpx2v2dHKk7sO0dXTD8DMshJW1Vfy3hvmcVV9BZfUlFFcNH57DBtyJUmSJOkCklJie3MXa/N1tGt3tPDC/sOklDWIuqSmlLdeNZurGiq5qr6C2vKJRIzNqcdnw5ArSZIkSaPoaG8/Tze2DYbadTtaae7sAaBsYhFX1lfwxhW1XFVfweVzpjGlxBh3Kn53JEmSJOk82p83iFq7I2sQtbGxjd7+vEFU9RRetXTGYIOohdPHboOokWLIlSRJkqQR0j+QeH5vB2t3trJ2ewtrd7ayq2VIg6jZ03jvDVmDqCvnTqNqaskoV3zxM+RKkiRJ0jnSfrSXJ3ceGhypfXLXIQ539wEwvbSEVfUV3H1dA1fVV7CstnxcN4gaKYZcSZIkSToLKSV2tgxtENXK8/s6SAkKApbOKuPOlXWsaqjgyrkVzK6YZIOo88CQK0mSJEnD0N3Xz8YhDaLW7jjEwcPdAJSWFLGyvoJbltewqiFrEDXVBlGjwu+6JEmSJJ3AgY7urNvxzizUPr27jZ7+AQAaqiZz4+LqwQZRi2aUUmiDqAuCIVeSJEnSuNc/kHhhf0c2Qrs963q8o7kLgOLCAlbMLuc91zdwZX029Xh6qQ2iLlSGXEmSJEnjzuHuvsEGUWt2tPDkzkN05A2iqqdmDaLedU09V9ZXsLyujJKiwlGuWMNlyJUkSZI0pqWU2N16ZHAt7ZodrTy/t52BBBGwZGYpt6+szaYez61kTqUNoi5mhlxJkiRJY0pP3wAb97SxbkjX4/0dWYOoqSVFrJw7jde9ehGrGiq4Ys40SidOGOWKdS4ZciVJkiRd1JoPZw2i1u5sZd2OVp7a3UZPX9Ygam7lZK5fWM2V9RVcNbeCJbNsEDXWGXIlSZIkXTQGBhKbDxxmzfbWwc7H2w52AlmDqOV1Zdx9XT1X5Q2iZpRNHOWKdb4ZciVJkiRdsDq7+3hq16HBtbTrdrbScTRrEFU1pZir6iu4a/UcrqqvYHldORMn2CBqvDPkSpIkSbpgHO7uY832Fh7d2szPtrawsbGN/oFEBCyeUcptK2pZle9NW1812QZR+iWGXEmSJEmjpqunjzXbW/NQ28yG3VmonVAYXDFnGu9/5QJWNVRyxZxplE+yQZROz5ArSZIk6bw50tPP2h2t/GxrM49ubeapXYfoG0gUFQSXz5nG77xiAdfOr+Kq+gomFTv1WGfOkCtJkiRpxBzt7WfdzlZ+tiWbfrx+Vyu9/YnCgmDF7HJ+68b5XJeH2iklxhO9dCN6FUXE64FPAYXA51JKHzvu8RLgy8BVQDPw9pTS9oiYAHwOuDKv8csppb8eyVolSZIkvXTdff2s33mIR7dk04/X7zpET98ABQGX1ZXzGzfM49r5VaxuqGSqoVYjYMSuqogoBD4D3ATsBp6IiPtSSs8MOey9QGtKaWFE3AV8HHg78FagJKV0WURMBp6JiK+llLaPVL2SJEmSzlxP3wBP7jqUTT/e0sy6na1056F2WW05d19Xz3ULslBbOtE1tRp5I/mrk6uBzSmlrQAR8XXgdmBoyL0d+Gj+9TeAT0fWHi0BUyKiCJgE9ADtI1irJEmSpGHo6Rvg6cZjI7UtrNnRwtHeASLg0poy3nVtPdfNr2L1vEobRWlUjGTIrQN2Dbm9G7jmZMeklPoiog2oIgu8twNNwGTgD1JKLce/QES8D3gfwNy5c891/ZIkSdK419s/wNONbYMjtWu2t3Kktx+ApbNK+dWr53Lt/CqumVfJtMnFo1ytNLIh90QbVqVhHnM10A/UAhXAv0fE94+NCg8emNJngc8CrFq16vhzS5IkSTpDff0DbNrTzqODobaFzp4s1C6ZWcrbVs3mugVVXDOvioophlpdeEYy5O4G5gy5PRvYc5JjdudTk8uBFuAdwHdSSr3A/oh4GFgFbEWSJEnSOdM/kHhmTzuPbj3Iz7a28MS2Fjq6+wBYOGMqb74yC7VXz6ukemrJKFcrnd5IhtwngEURMQ9oBO4iC69D3QfcDTwKvAX4YUopRcRO4NUR8VWy6crXAn87grVKkiRJ48LAQOKZpnZ+tjXrfvzYthY6jmahdv70KbzpilqunV/FtfOrmF5qqNXFZ8RCbr7G9oPAQ2RbCH0hpbQpIv4CWJNSug/4PPCViNhMNoJ7V/70zwBfBDaSTWn+Ykppw0jVKkmSJI1VAwOJ5/d18OiWZh7d2szj21poO9ILwLzqKdy2omYw1M4smzjK1UovXaQ0Npayrlq1Kq1Zs2a0y5AkSZJG1cBA4oX9h3l0Szb9+LFtzbR2ZaF2buVkrptfxbULKrl2fhU15ZNGuVpp+CJibUpp1emOc/dlSZIk6SKWUmLz/sNZ9+Ot2bY+LZ09AMyumMRrLpmZB9sq6qYZajX2GXIlSZKki0hKia0HOwenHz+2tZmDh7NQW1s+kVcumZ6F2vlVzKmcPMrVSuefIVeSJEm6gKWU2N7cxaNbmgebRe3v6AZgVtlEXr5oOtfOr+S6+dXMqZxExIl26ZTGD0OuJEmSdAFJKbGzpSubfrwlm368t/0oANNLS7hufhXXLchGahuqJhtqpeMYciVJkqRRtqulK1tPm4/W7mnLQm311JJslDYPtfOrpxhqpdMw5EqSJEnnWeOhI4PTjx/d0kzjoSMAVE0p5tr5VfxOHmwXTJ9qqJXOkCFXkiRJGmFNbUdeNP14Z0sXABWTJ3DNvCred+N8rp1fxeKZhlrppTLkSpIkSefYvvajQ0JtM9ubs1BbPmkC18yr5Ndf1sB1C6pYMrOUggJDrXQuGXIlSZKkl2h/x1F+trUl6368pZmtBzsBKJ1YxDXzKnnXtfVct6CKpbPKKDTUSiPKkCtJkiSdoQMd3Ty+rYVHtx7kZ1tb2Lz/MABTS4q4el4lv3r1XK6dX8WltYZa6Xwz5EqSJEmncGxLn8e3tfDE9hae2N7KtnykdkpxIavnVfKWq2Zz3fwqltWWUVRYMMoVS+ObIVeSJEkaYmAg8dzeDp7Y3sLj21tYs72Ffe3dQLamdnVDBXetnsPqeZVcVlfOBEOtdEEx5EqSJGlc6+7r5+ndbTy+vYUntrWwZkcrHUf7AKgpn8g186pYPa+SqxsqWTRjqo2ipAucIVeSJEnjyuHuPtbuaOWJbdlI7VO7DtHdNwDAgulTuG1FDasbKlndUMnsiklu6SNdZAy5kiRJGtMOHu4eDLRPbG/hmT3tDCQoLAiW1Zbxrmvr81BbQdXUktEuV9JLZMiVJEnSmJFSYlfLkcGpx09sbxnczqekqICVc6fxwVctZPW8Sq6cW8GUEn8clsYa/6uWJEnSRWtgIPH8vrxJVB5qjzWJKptYxOqGSt62eg6rG7ImUcVFNomSxjpDriRJki4aPX0DPN14iMe3tfJE3vm4PW8SNatsIlfPq+LqhgpWz6tk8YxSm0RJ49BpQ25EzACuB2qBI8BGYE1KaWCEa5MkSdI4d7i7j3U7WgdHap88rknUGy7LmkRdPc8mUZIyJw25EfEq4E+ASmA9sB+YCNwBLIiIbwD/LaXUfj4KlSRJ0th38HA3a7a3DI7UbtrTZpMoSWfkVCO5bwB+K6W08/gHIqIIuA24CfiXEapNkiRJY1hKid2tRwbX0j6+vYWtB07cJGrl3Aqm2iRK0jCc9JMipfTHp3isD/jmiFQkSZKkMelYk6g121t4fHu2T+3e9qPAkCZRq2wSJemlGc6a3N8Hvgh0AJ8DVgJ/klL67gjXJkmSpIvY6ZpErZ5XaZMoSefccOZ8/EZK6VMRcTMwHXgPWeg15EqSJGnQqZpEzbdJlKTzZDgh99inzxuAL6aUngo/kSRJksa945tEPdPUTv9AoiBgWW0577ymnqvnVbCqoZJqm0RJOk+GE3LXRsR3gXnAhyOiFHD7IEmSpHHkdE2irpgzjfe/cgGrGyq5st4mUZJGz3A+fd4LXAFsTSl1RUQV2ZRlSZIkjVEDA4mf7+/giW0nbhK1qqGSt141h6vnVbC8rpySosJRrliSMsMJubcDP0wpteW3+4H5wIYRq0qSJEnnVdYkqo0ntrfwxLYW1uxope1ILwAzy0oG19KubqhkyUybREm6cA0n5H4kpXTvsRsppUMR8RHcQkiSJOmi1dndx7qdrflIbdYk6mjvL5pE3bJ8FqsaKrm6oZI5lTaJknTxGE7IPdEGZS6ykCRJuog0H+7mie1Zg6gntrewac+Lm0S942qbREkaG4YTVtdExH8HPgMk4HeBtSNalSRJks7asSZRxwLt49ta2GKTKEnjxHA+0X4X+DPgn/Lb3wX+dMQqkiRJ0hnr6Rvg4S0HeXBDEw9vPkhT24ubRL3FJlGSxonThtyUUifwJxExNaV0+DzUJEmSpGHo6x/gkS3NPLihie9s2kvbkV5KJxZx4+LpXGOTKEnj1GlDbkS8DPgcMBWYGxGXA7+dUnr/SBcnSZKkF+sfSDy2tZn7NzTx0Ka9tHT2MKW4kJsuncltK2p5+eJqR2oljWvDma78SeBm4D6AlNJTEXHjiFYlSZKkQf0DiSe2t/Dghia+vbGJg4d7mFxcyGsumcmtl9XwyiXTmTjBYCtJMMwuySmlXce1je8fmXIkSZIEMDCQWLezlQc2NPGtp5vY39HNxAkFvHrpDG5bUcurlsxgUrHBVpKON5yQuyufspwiohj4PeDZkS1LkiRp/Ekp8eSuQ4PBtqntKMVFBbxy8XRuu7yW1yydwRQ7IUvSKQ3nU/I/AJ8C6oDdZN2VPzCSRUmSJI0XKSWebmzjwQ1NPLChicZDR5hQGLxi8XQ+9PolvPaSmZROnDDaZUrSRWM43ZUPAu88m5NHxOvJAnIh8LmU0seOe7wE+DJwFdAMvD2ltD1/bAXwP4EyYABYnVI6ejZ1SJIkXUhSSjzT1M4DG5p4cEMTO1u6KCoIblhUzR/ctJibLp1J+SSDrSSdjeF0V/4E8FfAEeA7wOXAf0wpffU0zysEPgPcRDYC/ERE3JdSembIYe8FWlNKCyPiLuDjwNsjogj4KvDuvNFVFdB75m9PkiTpwpBS4vl9HYMjttsOdlJYELxsQRUfeNUCbl42i2mTi0e7TEm66A1nuvLrUkofiog7ycLqW4EfkYXQU7ka2JxS2goQEV8HbgeGhtzbgY/mX38D+HRkHa5eB2xIKT0FkFJqHt7bkSRJurBs3t/B/U818eDTTWzef5iCgGvnV/FbL5/PzctmUjW1ZLRLlKQxZTgh99hcmTcAX0sptRzXaflk6oBdQ27vBq452TEppb6IaAOqgMVkja4eAqYDX08pfeL4F4iI9wHvA5g7d+5wapIkSRpxWw8cHhyxfX5fBxFwdUMld9++jNcvr2F6qcFWkkbKcELu/RHxHNl05fdHxHRgOGtjT5SE0zCPKQJuAFYDXcAPImJtSukHLzowpc8CnwVYtWrV8eeWJEk6b3Y0dw6usX2mqR2AVfUVfPSNl3LLZTXMLJs4yhVK0vgwnMZTfxIRHwfaU0r9EdFFNs34dHYDc4bcng3sOckxu/N1uOVAS37/T/KmV0TEt4ArgR8gSZJ0gdjV0sW3ns5GbJ9ubANg5dxp/Omtl3DrihpqyieNcoWSNP6cNORGxA0ppZ8CpJRaj92fUuoEOiOiDJibUtp4klM8ASyKiHlAI3AX8I7jjrkPuBt4FHgL8MOU0rFpyh+KiMlAD/AK4JNn8wYlSZLOpT2HjgwG2yd3HQJgxexy/vMblvKGy2qYXTF5lCuUpPHtVCO5v5J3Vv4OsBY4AEwEFgKvAuqBPzzZk/M1th8EHiLbQugLKaVNEfEXwJqU0n3A54GvRMRmshHcu/LntkbEfycLygn4VkrpwZf2ViVJks7Ovvajg8F27Y7sd//Lasv40OuXcOtlNdRXTRnlCiVJx0RKJ1/KGhEVZCOs1wM1ZOtynwUePDbKe6FYtWpVWrNmzWiXIUmSxogDHd18Z2MT929o4ontLaQES2eVcutlNdy6oob506eOdomSNK7kfZpWne64U67Jzacp/6/8jyRJ0pjWfLib72zay4MbmvjZ1mYGEiycMZXff80ibltRw8IZpaNdoiTpNIbTXVmSJGnMOtTVw0Ob9vLAhiYe2dJM/0BiXvUUPvCqhdy2opbFM6cyzO0TJUkXAEOuJEkad9qO9PLdTXt58OkmfvrCQfoGEnMrJ/PbN87n1hU1XFpTZrCVpIuUIVeSJI0LHUd7+f6z+3jgqSb+7YUD9PYn6qZN4r03zOO2FbUsrzPYStJYcNqQm2/j84dk2wX9VkQsApaklB4Y8eokSZJegs7uPr7/7D4e3NDEj39+gJ6+AWrKJ3L3dQ3cuqKGK+ZMM9hK0hgznJHcL5JtIXRdfns38M+AIVeSJF1wunr6+NFzB3hgwx5++Nx+uvsGmFFawjuunssbL69h5ZwKCgoMtpI0Vg0n5C5IKb09In4VIKV0JPyVpyRJuoAc7e3nx8/v54ENTfzg2f0c6e2nemoxb1s1h9tW1LC6odJgK0njxHBCbk9ETAISQEQsALpHtCpJkqTT6O7r599+fpAHNuzh+8/so7Onn8opxdx5ZR23XVbDNfOrKDTYStK4M5yQ+xHgO8CciPhH4Hrg10eyKEmSpBPp6Rvgp5sP8MCGJr63aR8d3X2UT5rAbStque3yGq6bX0VRYcFolylJGkWnDbkppe9FxDrgWiCA308pHRzxyiRJkoDe/gEe2dLMgxv28NCmfbQd6aV0YhE3L5/FrStquGFhNRMMtpKk3HC3EKoDCvPjb4wIUkr3jFxZkiRpPOvrH+CxbS08sGEP39m4l9auXqaWFHHTpTO5bUUNNyyqpqSocLTLlCRdgIazhdAXgBXAJmAgvzsBhlxJknTO9A8kHt/WwoNPZ8H24OEeJhcX8tpLZnLrihpesXg6EycYbCVJpzackdxrU0qXjnglkiRp3BkYSKzd2coDT+3hWxv3cqCjm4kTCnjN0mzE9pVLZjCp2GArSRq+4YTcRyPi0pTSMyNejSRJGvNSSqzbeYgHNzTxraeb2Nt+lJKiAl61ZAa3rqjhNZfMYHLxcFdUSZL0YsP5P8iXyILuXrKtgwJIKaUVI1qZJEkaM/r6B9jQ2MZ3Nu7lwQ1NNB46QnFhATcuns6H37CU11wyk6klBltJ0ks3nP+bfAF4N/A0v1iTK0mSdFIpJX6+7zAPbz7II1sO8tjWFjq6+5hQGLx80XT+r5sWc9OymZRNnDDapUqSxpjhhNydKaX7RrwSSZJ0UdvV0pWH2mYe2dLMwcPdAMytnMxtl9fwsgXVvHxRNdMmF49ypZKksWw4Ife5iPg/wP1k05UB3EJIkqRx7kBHN49ubeaRzQd5eMtBdrUcAaB6agnXL6ziZQuqeNmCauaj3CIvAAAgAElEQVRUTh7lSiVJ48lwQu4ksnD7uiH3uYWQJEnjTMfRXh7b2sLDWw7yyOZmnt/XAUDpxCKunV/Fb1w/j+sXVrNoxlQiYpSrlSSNV6cNuSml95yPQiRJ0oXlaG8/63a08vCWgzy8uZmnG9voH0iUFBWwqqGCN12xhOsXVrO8toyiwoLRLleSJOAUITciPpRS+kRE/L9kI7cvklL6vRGtTJIknVd9/QM83djGI1uaeXjzQdbsaKWnb4DCgmDF7HJ+5xULeNnCKq6cW8HECe5dK0m6MJ1qJPfZ/O8156MQSZJ0fr24A3Izj21tpqO7D4Cls0p51zX1XL+wiqvnVVJqF2RJ0kXipCE3pXR//mVXSumfhz4WEW8d0aokSdKI2NXSxSP59OMTdUC+bkE1L1tQRfXUklGuVJKkszOcxlMfBv55GPdJkqQLzMHD3dmWPifogPyyBVV5F2Q7IEuSxo5Trcm9BXgDUBcRfzfkoTKgb6QLkyRJZ25oB+RHtzTz3N68A3JJEdfYAVmSNA6caiR3D9l63DcBa4fc3wH8wUgWJUmShudYB+RHtjTz8JaDbNj94g7If3yzHZAlSePLqdbkPgU8FRH/J6XUex5rkiRJJ9HXP8DGPe15s6iDrNneSrcdkCVJGjScfXINuJIkjZKUEi/szzogP7y5mce2NdNx9BcdkN95TT0vW1DFNfPtgCxJEgyv8ZQkSTqPTtUB+dbLanjZwmqum1/F9FI7IEuSdDxDriRJo+xYB+RH82C7s6ULsAOyJEln47QhNyLuB9Jxd7eRNaX6nymloyNRmCRJY1XH0V4e39aSj9Qe/KUOyO+5vsEOyJIknaXhjORuBaYDX8tvvx3YBywG/hfw7pEpTZKkseFobz/rdrbyyOYXd0AuLipgdd4B+WULqrisrtwOyJIkvUTDCbkrU0o3Drl9f0T8W0rpxojYNFKFSZJ0seofSDzd2MbDm7O9ap/Y3vKiDsj/4RXzuX5BNVfW2wFZkqRzbTghd3pEzE0p7QSIiLlAdf5Yz4hVJknSReJYB+RHNh/k4S3N/Gzrizsgv+OauVy/oJqr51dSZgdkSZJG1HBC7h8CP42ILUAA84D3R8QU4EsjWZwkSReq3a1dg9OPH9nSzIGOrAPynMpJdkCWJGkUDWef3G9FxCJgKVnIfW5Is6m/HcniJEm6UDTnHZAfsQOyJEkXtOFuIXQV0JAfvyIiSCl9ecSqkiRplJ2uA/KvvyzrgLx4ph2QJUm6kAxnC6GvAAuAJ4H+/O4EnDbkRsTrgU8BhcDnUkofO+7xkvw8VwHNwNtTStuHPD4XeAb4aErpb4bxfiRJOitDOyA/suUgTw3pgLyq3g7IkiRdLIYzkrsKuDSldPxeuacUEYXAZ4CbgN3AExFxX0rpmSGHvRdoTSktjIi7gI+TbVF0zCeBb5/J60qSdDoDA4kdLV1sbGxj4542nt7dxtodrXT3DVAQsGL2NDsgS5J0kRpOyN0IzAKazvDcVwObU0pbASLi68DtZCOzx9wOfDT/+hvApyMiUkopIu4g26O38wxfV5KkQX39A2w+cJhNje1s3NPGpsZ2nmlq53B31v14QmGweKYdkCVJGiuGE3KrgWci4nGg+9idKaU3neZ5dcCuIbd3A9ec7JiUUl9EtAFVEXEE+E9ko8B/dLIXiIj3Ae8DmDt37jDeiiRpLDva28/P93Ww8Vig3dPOc03tdPcNADBxQgGX1pTx5ivrWFZbxrLachbPLKW4yOnHkiSNFcMJuR89y3OfqAvH8VOeT3bMnwOfTCkdPlUzj5TSZ4HPAqxateqMplNLki5uh7v7eLapnU2NbWzc087GxjY27z9M30D2v4PSiUUsry3n3dfWs7yunOV1ZcyrnkphgU2iJEkay4azhdBPzvLcu4E5Q27PBvac5JjdEVEElAMtZCO+b4mITwDTgIGIOJpS+vRZ1iJJuogd6uphUx5kN+5pZ9OeNrYd7ORYt4jqqcUsqy3nNZfMYFltOctry5lTOcmux5IkjUMnDbkR8dOU0g0R0cGLR2ADSCmlstOc+wlgUUTMAxqBu4B3HHfMfcDdwKPAW4Af5g2uXj6kjo8Chw24kjQ+7G8/Orh2duOeNjY2ttN46Mjg43XTJrGstozbL69jeV0Zy+vKmVFaYqCVJEnAKUJuSumG/O/Sszlxvsb2g8BDZFsIfSGltCki/gJYk1K6D/g88JWI2Ew2gnvX2byWJOnik1Jid+sRNuVBdtOebJT2QMdg+wfmVU9h5dxpvPu6+sE1tJVTikexakmSdKGL4ewMlG8HNJMhoTiltHME6zpjq1atSmvWrBntMiRJJ9A/kNh2sJNNeTOojY3Z321HegEoLAgWzZjKstpyltVmo7OX1JRSapdjSZKUi4i1KaVVpzvutGtyI+J3gY8A+4CB/O4ErHhJFUqSxqTe/gFe2Hc4n3Kchdlnmtrp6ukHoLiogEtmlfKGy2qy6ca15SyZVepetJIk6ZwYTnfl3weWpJSaR7oYSdLF5Whvf9bhOG8GtbGxnef3dtDTn/1OdEpxIZfWlvG2VXMGR2gXzpjKhEK37JEkSSNjOCF3F9A20oVIki5sHUd7eWZPe9bdOB+h3XzgMP35lj3TJk9geW0577m+gWV15SyvLaOhagoFbtkjSZLOo+GE3K3AjyPiQWCwG0hK6b+PWFWSpFHVfLg7H51tH5x2vL25a/DxGaUlLK8r5+ZlM7m0NtuDtm6aW/ZIkqTRN5yQuzP/U5z/kSSNESkl9rV35/vPZtONn9nTxp62o4PHzKmcxPLact5y1WyW1WWNoWaUThzFqiVJkk7utCE3pfTn56MQSdLISimxs6XrRdv1bGpso7mzB4AImF89hdXzKlleW86yujKW1ZRTPtkOx5Ik6eIxnO7Ki4E/Ahp48RZCrx65siRJL0X/QGLrgcODo7MbG9t4pqmdjqN9ABQVBItnlvKaS2awLJ9uvHRWGVNKhjPBR5Ik6cI1nJ9m/hn4H8DngP6RLUeSdKa6+/qzLXvyZlAb97TxbFM7R3uzDscTJxSwdFYZt19Rm43Q1pazeNZUSorcskeSJI09wwm5fSmlfxjxSiRJp9XV08ezTR35dj3ZKO0L+zvo7c86HJeWFHFpbRnvvKZ+cMue+dVTKHLLHkmSNE4MJ+TeHxHvB+7lxd2VW0asKkkSrZ09PLu3Pdu2pzFbQ7v1wGHyHXuonFLM8rpyXrFkOsvzKcdzKia7ZY8kSRrXhhNy787//uMh9yVg/rkvR5LGn+6+frbs7+S5ve08v7eDZ/d28Pzedva1D/5ekZryiSyrLee2FTWDTaFmlU10yx5JkqTjDKe78rzzUYgkjXUpJfa0HeW5pnae29vBc3mY3Xqgk758eLa4sIBFM6dy/cJqLplVxpJZpSyrLaNqaskoVy9JknRxGE535V870f0ppS+f+3IkaWxoP9rLz4eMyj6fh9pj3Y0BZldMYumsUl536SyWzCrlkppSGqpcPytJkvRSDGe68uohX08EXgOsAwy5ksa9vv4Bth3sHAyzzzVlYbbx0JHBY0onFnHJrDLuuKJuMMwunllK6UT3n5UkSTrXhjNd+XeH3o6IcuArI1aRJF2AUkoc6Oj+pTC7ef9hevqzrXqKCoIF06dyVX0F77x2LktnlbJ0Vhk15a6dlSRJOl+GM5J7vC5g0bkuRJIuFF09ffx832Ge39vOs00d+VTjdlq7egePmVU2kSWzSnn54urBMDt/+hT3npUkSRplw1mTez9ZN2WAAuBS4J9HsihJOh/6BxI7W7p+KczuaOki5Z96k4sLWTyzlNcvn8WSmaUsrSlj6axSpk0uHt3iJUmSdELDGcn9myFf9wE7Ukq7R6geSRoRLZ09Q7oaZ42gnt/XwdHebKpxQUBD9RQurS3jzpWzWVpTytJZpe47K0mSdJEZzprcnwy9HRGFEfHOlNI/jlxZknR2jvb2s3n/4cFR2WNb9Rzo+MWes1VTillaU8o7r6nPGkHNKmPRzKlMnOBUY0mSpIvdSUNuRJQBHwDqgPuA7+W3/xh4EjDkSho1KSV2tx4Z3Gv2WJjddrCT/nzP2ZKibM/ZVyyePrhudsmsUqaXuuesJEnSWHWqkdyvAK3Ao8BvkoXbYuD2lNKT56E2SQKg7UhvNr14b3ve3Tj7c7j7F3vOzqmcxNJZZdyyfNZgmG2omuyes5IkSePMqULu/JTSZQAR8TngIDA3pdRxXiqTNO709g+w9UDn4DTj5/d28FxTO3vajg4eUzaxiKU1Zbz5yrrBMLtkVilTS86mWbwkSZLGmlP9VDi4V0ZKqT8ithlwJZ0LKSX2tXe/KMw+29TOlgOH6e3PphoXFQQLZ0xl9bxKls7KOhovrSllVpl7zkqSJOnkThVyL4+I9vzrACbltwNIKaWyEa9O0kWvs7uPn+/reFGYfX5fB4eG7DlbU57tOfvKJTMGw+z86qkUFznVWJIkSWfmpCE3pWSbUUnD1j+Q2N7cOTjF+FgjqJ0tXYPHTCkuZMmsUm5ZXsMlNaXZvrOzyiifPGEUK5ckSdJY4iI2SWfsaG8/a3e0ZqOyeZh9Yf+L95ydVz2Fy+rKeetVs7NtemrKqJs2yT1nJUmSNKIMuZKGZWAg8bNtzdy7rpFvb9w72Nm4emoJS2eV8q5je87WlLFwhnvOSpIkaXQYciWd0s/3dXDPukb+9clGmtqOMrWkiFuWz+INK2q4rK6c6qnuOStJkqQLhyFX0i/Z33GU+57cw73rG9m0p53CguDGRdV8+A2XcNMlM5lU7CitJEmSLkyGXEkAdPX08b1n9nHPukb+/YUDDCS4rK6c/3Lbpbzx8lqmlzpiK0mSpAufIVcax/oHEo9uaeae9bt5aONeOnv6qZs2id955QLuXFnHwhmlo12iJEmSdEYMudI49Nzedu5d18i/PrmHve1HKS0p4rYVtdx5ZR1XN1TaAVmSJEkXLUOuNE7sa8/W2d6zvpFnm9opKghesXg6f3rbJbz2kpl2Q5YkSdKYYMiVxrDO7j4e2rSXe9c38vDmgwwkuHzONP78Tcu4bUUNVXZGliRJ0hhjyJXGmP6BxMObD3Lv+kYe2rSXrp5+ZldM4gOvWsgdK+tYMH3qaJcoSZIkjRhDrjQGpJR4pilbZ3vfU3vY39FN6cQibr+iljtXzmZVfYXrbCVJkjQuGHKli9jetqN888lG7l3XyPP7OphQGLxyyQzevLKOVy2d4TpbSZIkjTuGXOkic7i7j+9s3Mu963fzyJZmUoKVc6fxl7cv47YVtVRMKR7tEiVJkqRRM6IhNyJeD3wKKAQ+l1L62HGPlwBfBq4CmoG3p5S2R8RNwMeAYqAH+OOU0g9HslbpQtbXP8C/bz7IN/N1tkd7B5hbOZnfffUi7lxZx7zqKaNdoiRJknRBGLGQGxGFwGeAm4DdwBMRcV9K6Zkhh70XaE0pLYyIu4CPA28HDgJvTCntiYjlwENA3UjVKl2IUkps2tPOPfk624OHuymfNIFfuXI2b76yjivnVhDhOltJkiRpqJEcyb0a2JxS2goQEV8HbgeGhtzbgY/mX38D+HREREpp/ZBjNgETI6IkpdQ9gvVKF4Q9h44MrrN9Yf9hJhQGr146gztXzuZVS6dTUuQ6W0mSJOlkRjLk1gG7htzeDVxzsmNSSn0R0QZUkY3kHvMrwPoTBdyIeB/wPoC5c+eeu8ql86zjaC/ffnov96zfzWPbWkgJVtVX8Fd3LOe2FTVMm+w6W0mSJGk4RjLknmgeZTqTYyJiGdkU5ted6AVSSp8FPguwatWq488tXdB6+wf49xcOcM+6Rr73zD66+wZoqJrMf3zNYu5cWcfcqsmjXaIkSZJ00RnJkLsbmDPk9mxgz0mO2R0RRUA50AIQEbOBe4FfSyltGcE6pfMmpcTTjW3cs66R+5/aQ3NnD9MmT+Btq+Zw55V1rJwzzXW2kiRJ0kswkiH3CWBRRMwDGoG7gHccd8x9wN3Ao8BbgB+mlFJETAMeBD6cUnp4BGuUzovdrV18c30j965vZMuBTooLC3jtpTO444o6XrlkBsVFBaNdoiRJkjQmjFjIzdfYfpCsM3Ih8IWU0qaI+AtgTUrpPuDzwFciYjPZCO5d+dM/CCwE/iwi/iy/73Uppf0jVa90rrUd6eXbTzdxz/pGHt/WAsDVDZX85svn84bLaiifNGGUK5QkSZLGnkhpbCxlXbVqVVqzZs1ol6Fxrrd/gJ88f4B71zfyvWf30dM3wPzqKdy5so47VtYxp9J1tpIkSdLZiIi1KaVVpztuJKcrS+NCSokndx3im+sbuX9DEy2dPVROKeYdV8/ljpV1XD673HW2kiRJ0nliyJXO0q6WLu5d38g31zey9WAnxUUF3HTpTN68so4bF09nQqHrbCVJkqTzzZArnYG2rl4efLqJe9fv5ontrQBcM6+S337FfG65rIayia6zlSRJkkaTIVc6jZ6+AX70/H6+ub6RHzy7n57+ARbOmMof37yE26+oZXaF62wlSZKkC4UhVzqBlBLrdh7i3vW7eWBDE4e6eqmeWsw7r53Lm1fOZnldmetsJUmSpAuQIVcaYkdz5+A62+3NXZQUFfC6ZbN488o6blhU7TpbSZIk6QJnyNW4d6irh/s3NPHN9Y2s3dFKBFw7r4r3v2ohtyyfRanrbCVJkqSLhiFX41J3Xz8/em4/96xr5EfP76e3P7F45lT+0+uXcvsVtdROmzTaJUqSJEk6C4ZcjRspJdbuaOWe9Y08uKGJtiO9VE8t4deua+DOlXUsq3WdrSRJknSxM+RqzNt2sJN71+3m3icb2dVyhEkTCrl52UzuWFnHDQurKXKdrSRJkjRmGHI1JrV09vDAhj3cs66RJ3cdIgKuX1DNf3zNYm5ePoupJV76kiRJ0ljkT/q66PX1D9B46Ajbm7vY0dzJv/38ID9+fj99A4mls0r58C1Luf2KOmaVTxztUiVJkiSNMEOuLgrdff3sajnCjuZOtjd3sTP/e0dzJ7tbj9A3kAaPnVFawm/cMI87V9ZxSU3ZKFYtSZIk6Xwz5OqCcaSnn50tXWxv7hwMszuaO9l+sIs9bUdIv8ixlJYUUV89mWV15dy6oob6qik0VE2hoWoy00tLbCAlSZIkjVOGXJ1XHUd72dHcxY7mLMzubD4WarvY2370RcdWTJ5AfdUUVjdUMLdqNg1Vk/MwO5nKKcUGWUmSJEm/xJCrc66tq5ftzZ2D4fXY3zuaOzl4uOdFx1ZPLaGhajLXL6zOQmx1FmLrK6dQPnnCKL0DSZIkSRcrQ67OWEqJ5s6ewanEO5o72dHSNTi9+FBX74uOrymfSH3VZF57yczBkdi5+aisXY4lSZIknUsmDJ1QSon9Hd1sP/ji0dhjfx/u7hs8tiCgdtokGqqmcOtlNTRUTaG+ajIN1VOYWzmZiRMKR/GdSJIkSRpPDLnjWP9AoqntyAnXx25v7uRo78DgsUUFwZzKydRXTWZ1Q2UWYvMwO7tiMsVFBaP4TiRJkiQpY8gd447fQ/bY9OLtzZ3sajlCT/8vgmxxUQH1eZAdXCObdy2unTaRokKDrCRJkqQLmyF3DDiTPWQnTSikvmoyi2aU8tpLZ/5ianHVFGaVTaSgwI7FkiRJki5ehtyLxJnuIdtQPcU9ZCVJkiSNO4bcC8jxe8juGNx659R7yNZXzaah+hdTiysmTzDISpIkSRqXDLnn2ZnsITu9NNtD9oZF1fm2O+4hK0mSJEmnYsg9T97zxcdZv+vQsPaQrc/XyU5xD1lJkiRJOiOmqPOkoXoKdRWT8kZPWZid4x6ykiRJknROGXLPk4+8cdlolyBJkiRJY54bn0qSJEmSxgxDriRJkiRpzDDkSpIkSZLGDEOuJEmSJGnMMORKkiRJksYMQ64kSZIkacww5EqSJEmSxgxDriRJkiRpzIiU0mjXcE5ExAFgx2jXcRrVwMHRLkI6R7yeNdZ4TWss8XrWWOL1rGPqU0rTT3fQmAm5F4OIWJNSWjXadUjngtezxhqvaY0lXs8aS7yedaacrixJkiRJGjMMuZIkSZKkMcOQe359drQLkM4hr2eNNV7TGku8njWWeD3rjLgmV5IkSZI0ZjiSK0mSJEkaMwy5kiRJkqQxw5B7HkTE6yPi+YjYHBF/Mtr1SMdExJyI+FFEPBsRmyLi9/P7KyPiexHxQv53RX5/RMTf5dfyhoi4csi57s6PfyEi7h5y/1UR8XT+nL+LiDj/71TjSUQURsT6iHggvz0vIh7Lr81/ioji/P6S/Pbm/PGGIef4cH7/8xFx85D7/TzXeRUR0yLiGxHxXP5ZfZ2f0bpYRcQf5D9vbIyIr0XERD+jNRIMuSMsIgqBzwC3AJcCvxoRl45uVdKgPuAPU0qXANcCH8ivzz8BfpBSWgT8IL8N2XW8KP/zPuAfIAvFwEeAa4CrgY8c+6ErP+Z9Q573+vPwvjS+/T7w7JDbHwc+mV/PrcB78/vfC7SmlBYCn8yPI/9v4C5gGdn1+vd5cPbzXKPhU8B3UkpLgcvJrm0/o3XRiYg64PeAVSml5UAh2Wetn9E65wy5I+9qYHNKaWtKqQf4OnD7KNckAZBSakoprcu/7iD74amO7Br9Un7Yl4A78q9vB76cMj8DpkVEDXAz8L2UUktKqRX4HvD6/LGylNKjKety9+Uh55LOuYiYDdwKfC6/HcCrgW/khxx/PR+7zr8BvCY//nbg6yml7pTSNmAz2We5n+c6ryKiDLgR+DxASqknpXQIP6N18SoCJkVEETAZaMLPaI0AQ+7IqwN2Dbm9O79PuqDk04BWAo8BM1NKTZAFYWBGftjJrudT3b/7BPdLI+VvgQ8BA/ntKuBQSqkvvz30Ghy8bvPH2/Ljz/Q6l0bKfODA/9/e/cdsWdVxHH9/BG1pW5puTYMNI7JaWz+cQqlJ5qiVU2vQdC6KsZaaFDW2Zn8wM//QyNYfZc1IWY1mzHSwZcPNSJgVkIDij1nmrzBSC4dCiyK+/XGdx25vH3488Dz8ePZ+bey+rnPOfc65n52d6/7e51wXwK1tC/7CJMfhHK0jUFU9C3wbeIYuuN0K3I9ztEaAQe7IG+zeFv/fJh1WkrwB+AUwt6pe2lPRQdJqP9KlYZfkAuD5qrq/N3mQorWXPMezDhdjgfcDP6iq9wHb+f/W5ME4pnXYalvkLwJOBU4BjqPbWtzPOVoHzCB35G0CxvecjwP+eoj6Ir1GkqPpAtzFVXVHS36ubWOjvT7f0nc3nveUPm6QdGkknAVcmOQpum1q59Gt7B7ftsbBq8fgK+O25b8R2MLQx7k0UjYBm6pqdTu/nS7odY7Wkeh84MmqeqGq/gPcAXwQ52iNAIPckbcWmNSeHHcM3Y3yyw5xnyTglfsVfww8WlXf6claBgw8ffOzwNKe9JntCZ5TgK1tq9xyYFqSE9ovtdOA5S3v5SRTWlsze+qShlVVXV1V46pqAt1c++uqugxYAUxvxfrH88A4n97KV0u/pD3Z81S6h/GswflcB1lV/Q34S5LTWtJHgEdwjtaR6RlgSpJj23gbGM/O0Rp2Y/deRAeiqnYmuYruAjMGuKWqHj7E3ZIGnAV8BtiYZENL+zpwPbAkyWy6i9KMlncX8HG6hzz8E5gFUFVbknyT7gIDcG1VbWnHVwCLgNcDv2r/pIPpa8BtSa4D1tMe4tNef5rkcbrVgUsAqurhJEvovnztBL5YVf8FcD7XITAHWNy+tD9BN+8ehXO0jjBVtTrJ7cA6url1PXAz8EucozXM0v0gIkmSJEnSkc/typIkSZKkUcMgV5IkSZI0ahjkSpIkSZJGDYNcSZIkSdKoYZArSZIkSRo1DHIlSeqTpJLc2HM+L8k1w1T3oiTT917ygNuZkeTRJCtGuq3W3ueSfO9gtCVJ0p4Y5EqS9Fo7gE8lOelQd6RXkjFDKD4buLKqPjwC/UgSv0NIkg5LXqAkSXqtncDNwFf6M/pXYpNsa69Tk9ybZEmSPya5PsllSdYk2ZhkYk815ydZ1cpd0N4/JsmCJGuTPJjkCz31rkjyM2DjIP25tNX/UJIbWtp84Gzgh0kW9JW/KcmF7fjOJLe049lJrmvHX231PZRkbkub0FaGbwLWAeOTzGqf4V7grJ42ZrT3PpBk5RD/9pIkHZCxh7oDkiQdpr4PPJjkW0N4z3uAdwJbgCeAhVV1ZpIvA3OAua3cBOBcYCKwIsnbgJnA1qo6I8nrgPuS3N3Knwm8u6qe7G0sySnADcDpwIvA3Ukurqprk5wHzKuqP/T1cSVwDrAMeAtwcks/G7gtyenALGAyEGB1C2JfBE4DZlXVlUlOBr7R2t4KrADWt7rmAx+tqmeTHD+Ev58kSQfMlVxJkgZRVS8BPwG+NIS3ra2qzVW1A/gzMBCkbqQLbAcsqapdVfUnumD4HcA0YGaSDcBq4ERgUiu/pj/Abc4AflNVL1TVTmAx8KG99HEVcE6SdwGPAM+1gPUDwG/pgt07q2p7VW0D7qALigGerqrft+PJPW3/G/h5Txv3AYuSfB4YyhZrSZIOmCu5kiTt3nfptube2pO2k/YjcZIAx/Tk7eg53tVzvotXX3Orr52iWzWdU1XLezOSTAW276Z/2esn6G+oW109AfgY3arum4BPA9uq6uX2mXanvx/9n2OgjcuTTAY+AWxI8t6q+sdQ+ypJ0v5wJVeSpN2oqi3AErqHOA14im6LLsBFwNH7UfWMJEe1+3TfCjwGLAeuSHI0QJK3JzluL/WsBs5NclJ7KNWlwL370P7v6LZOr6Rb2Z3XXmlpFyc5trX/yZ68/ranJjmx9XnGQEaSiVW1uqrmA38Hxu9DnyRJGhau5EqStGc3Alf1nP8IWJpkDXAPu19l3ZPH6ILRNwOXV9W/kiyk29K8rq2mvgBcvKdKqmpzkqvp7ocNcFdVLd2H9lcB06rq8SRP063mrpoUY0EAAAB7SURBVGp1rkuyCFjTyi6sqvVJJgzS9jV0AfNmuhXvga3JC5JMan26B3hgH/okSdKwSNWgO40kSZIkSTriuF1ZkiRJkjRqGORKkiRJkkYNg1xJkiRJ0qhhkCtJkiRJGjUMciVJkiRJo4ZBriRJkiRp1DDIlSRJkiSNGv8DuG+NFaM1x3MAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1152x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_alpha_sort()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Attention"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The last part of this task is based on the algorithm of group # 24, we thank them for the help."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Find similar wines!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This k-means is implemented by reasoning in terms of products between matrices.\n",
    "\n",
    "Our goal is to create clusters, which will have deviance within (inside groups) and a deviance between (between groups).\n",
    "Therefore, we can minimize deviance within or maximize deviance between.\n",
    "We choose to minimize deviance within because we can set a threshold for which our objective function is satisfied, if it is satisfied it means that the position of the centroids varies little.\n",
    "\n",
    "The objective function will be:\n",
    "\\begin{equation*}\n",
    "minimize ||X-UX^{c}||^{2}\n",
    "\\end{equation*}\n",
    "\n",
    "Where:\n",
    "- X is our data matrix (NxJ)\n",
    "- U is our membership matrix (NxK) that defines where u_{ik} = 1 if the unit i belongs to the class k, u_{ik} = 0 otherwise.\n",
    "-  X^{c} is our centroid matrix (KxJ)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Read and standardize data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from numpy import genfromtxt\n",
    "import pandas as pd\n",
    "from random import randrange\n",
    "from scipy import stats\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.DataFrame(genfromtxt(r'C:\\Users\\aless\\Desktop\\data\\wine.data', delimiter=','))\n",
    "data = data.drop(data.columns[[0]], axis='columns')\n",
    "\n",
    "#### datacolumns\n",
    "data.columns = [\n",
    "\n",
    "    'Alcohol',\n",
    "    'Malic acid',\n",
    "    'Ash',\n",
    "    'Alcalinity_of_ash',\n",
    "    'Magnesium',\n",
    "    'Total_phenols',\n",
    "    'Flavanoids',\n",
    "    'Nonflavanoid_phenols',\n",
    "    'Proanthocyanins',\n",
    "    'Color_intensity',\n",
    "    'Hue',\n",
    "    'OD280/OD315_of_diluted_wines',\n",
    "    'Proline']\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Standardize dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def diagonalize(su):\n",
    "    '''\n",
    "    Create a matrix with weights on the diagonal.\n",
    "\n",
    "    :param su: diagonal of a matrix\n",
    "    :return: diagonal matrix\n",
    "    '''\n",
    "\n",
    "    Uzero = []\n",
    "    for j in range(0, len(su)):\n",
    "        Uzero.append([0] * len(su))\n",
    "\n",
    "    for j in range(0, len(su)):\n",
    "        Uzero[j][j] = su[j]\n",
    "    Uzero = pd.DataFrame(Uzero)\n",
    "    return Uzero"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       0       1       2       3       4       5       6       7       8   \\\n",
      "0  1.5679 -0.5555  0.2596 -1.1507  1.9358  0.8208  1.0413 -0.6503  1.2337   \n",
      "1  0.2973 -0.4927 -0.8000 -2.4717  0.0405  0.5804  0.7401 -0.8114 -0.5358   \n",
      "2  0.2479  0.0279  1.1366 -0.2500  0.1107  0.8208  1.2220 -0.4891  2.1447   \n",
      "3  1.7406 -0.3401  0.5154 -0.7904  0.9531  2.5032  1.4730 -0.9726  1.0410   \n",
      "4  0.3466  0.2344  1.8674  0.4706  1.3041  0.8208  0.6698  0.2361  0.4103   \n",
      "\n",
      "       9       10      11      12  \n",
      "0  0.2587  0.3754  1.8596  1.0206  \n",
      "1 -0.2863  0.4193  1.1252  0.9728  \n",
      "2  0.2760  0.3316  0.8004  1.4027  \n",
      "3  1.1930 -0.4139  1.1958  2.3421  \n",
      "4 -0.3123  0.3754  0.4614 -0.0303  \n"
     ]
    }
   ],
   "source": [
    "def standardizeDataFrame(data):\n",
    "    u = pd.DataFrame([1]*len(data))\n",
    "    _u = u.transpose()\n",
    "    _dot1 = pd.DataFrame(np.dot(u, _u))\n",
    "    mat = round(pd.DataFrame(np.dot(1/len(data), _dot1)), 4)\n",
    "    #centrature matrix\n",
    "    Jc = np.identity(len(data)) - mat\n",
    "    dot_data = pd.DataFrame(np.dot(Jc, data))\n",
    "    dot_data2 = np.dot(dot_data.transpose(), dot_data)\n",
    "    s_data = round(pd.DataFrame(np.dot(1/len(data), dot_data2)), 4)\n",
    "\n",
    "    diagonal = np.array(np.diag(s_data))\n",
    "    d2 = diagonalize(diagonal)**0.5\n",
    "    d2_inv = pd.DataFrame(np.linalg.pinv(d2.values)) #pseudo inverse\n",
    "\n",
    "    dot1 = pd.DataFrame(np.dot(data, d2_inv))\n",
    "    stand = round(pd.DataFrame(np.dot(Jc, dot1)), 4)\n",
    "    return stand\n",
    "\n",
    " # standardize DataFrame\n",
    "dataS = standardizeDataFrame(data)\n",
    "\n",
    "print(dataS.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Choose number of cluster and create them"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To choose the number of clusters we use Cost (SSE).\n",
    "The formula measures how the clusters are homogeneous internally.\n",
    "You select the number of clusters that (in the graph that will be created) corresponds to the point where there is an 'elbow': in that point the SSE will tend not to decrease more quickly.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we can then calculate the K-Means by different number of clusters and see their Cost.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Utils functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "def randPU(n, K):\n",
    "    '''\n",
    "    Membership of a unit in a cluster (initialization)\n",
    "\n",
    "\n",
    "    :param n: len(matrix)\n",
    "    :param K: number of cluster\n",
    "    :return: misleading matrix\n",
    "    '''\n",
    "    # all 0 matrix\n",
    "    Uzero = []\n",
    "    for k in range(0, n):\n",
    "        Uzero.append([0] * K)\n",
    "\n",
    "    # random matrix of appartenence\n",
    "    for row in range(0, n):\n",
    "        Uzero[row][randrange(K)] = 1\n",
    "\n",
    "    return Uzero\n",
    "\n",
    "\n",
    "def dwdb(data, U, Xm, K):\n",
    "    u = pd.DataFrame([1] * len(data))\n",
    "    _u = u.transpose()\n",
    "    _dot1 = pd.DataFrame(np.dot(u, _u))\n",
    "    mat = round(pd.DataFrame(np.dot(1 / len(data), _dot1)), 4)\n",
    "    # centrature matrix\n",
    "    Jc = pd.DataFrame(np.identity(len(data)) - mat)\n",
    "\n",
    "    data_c = round(pd.DataFrame(np.dot(Jc, data)), 4)\n",
    "    _Xm = round(pd.DataFrame(np.dot(pd.DataFrame(np.linalg.pinv(U.values)), data)), 4)\n",
    "\n",
    "    #WITHIN\n",
    "    p = data_c - np.dot(U, _Xm)\n",
    "    D_w = np.trace(np.dot(p.transpose(), p))\n",
    "\n",
    "    #BETWEEN\n",
    "    b = np.dot(U, _Xm)\n",
    "    D_b = np.trace(np.dot(b.transpose(), b))\n",
    "    \n",
    "   \n",
    "\n",
    "    return (D_b/(K-1))/(D_w/(len(data)-K))\n",
    "\n",
    "\n",
    "def diagSU(su):\n",
    "    '''\n",
    "    Diagonal matrix of sum_col calculate\n",
    "    \n",
    "    :param su: sum of U_0 columns\n",
    "    :return: diagonal matrix 1/sum_column\n",
    "    '''\n",
    "    Uzero = []\n",
    "    for j in range(0, len(su)):\n",
    "        Uzero.append([0] * len(su))\n",
    "\n",
    "    for j in range(0, len(su)):\n",
    "        Uzero[j][j] = 1 / su[j]\n",
    "\n",
    "    return Uzero\n",
    "\n",
    "def diagonalize(su):\n",
    "    '''\n",
    "\n",
    "    :param su: diagonal of a matrix\n",
    "    :return: diagonal matrix\n",
    "    '''\n",
    "\n",
    "    Uzero = []\n",
    "    for j in range(0, len(su)):\n",
    "        Uzero.append([0] * len(su))\n",
    "\n",
    "    for j in range(0, len(su)):\n",
    "        Uzero[j][j] = su[j]\n",
    "    Uzero = pd.DataFrame(Uzero)\n",
    "    return Uzero\n",
    "\n",
    "def mse(Xrow, Xmean0row1):\n",
    "    min_dif = 0\n",
    "    for r in range(0, len(Xrow)):\n",
    "\n",
    "        min_dif += (Xrow[r] - Xmean0row1[r])**2\n",
    "        \n",
    "    MSE = min_dif/len(Xrow)\n",
    "    return MSE\n",
    "\n",
    "def mse2(Xrow, Xmean0row1):\n",
    "    min_dif = 0\n",
    "    for r in range(0, len(Xrow)):\n",
    "        \n",
    "        min_dif += (Xrow[r] - Xmean0row1[r])**2\n",
    "        \n",
    "    MSE = min_dif/len(Xrow)\n",
    "    return MSE\n",
    "\n",
    "def costf(X, Xmean_ott, U):\n",
    "    cost = 0\n",
    "    for i in range(0, len(X)):\n",
    "        for j in range(0, len(U.iloc[0])):\n",
    "            if U.iloc[i, j] == 1:\n",
    "                cost += mse(X.iloc[i], Xmean_ott.iloc[j])\n",
    "    return cost            "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### K-Means Algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "def kmeans(X, K, Rndstart):\n",
    "    '''\n",
    "\n",
    "    :param X: data Matrix\n",
    "    :param K: Number of cluster of the partition\n",
    "    :param Rndstart: number of random start\n",
    "    :return:\n",
    "    '''\n",
    "\n",
    "    maxiter = 100\n",
    "    n = len(X)\n",
    "    j = len(X.iloc[0])\n",
    "    epsilon = 0.00001\n",
    "\n",
    "    # find the best solution in a fixed number of random start partitions\n",
    "    for loop in range(0, Rndstart):\n",
    "        \n",
    "        # initial partition U_0 is given\n",
    "        U_0 = pd.DataFrame(randPU(n, K))\n",
    "        \n",
    "        # column frequency = random cluster\n",
    "        sum_col = []\n",
    "        for r in range(0, K):\n",
    "            sum_col.append(sum(U_0[r]))\n",
    "        \n",
    "        # 1/su on diagonal of a NxN matrix\n",
    "        su_diag = diagSU(sum_col)\n",
    "\n",
    "        # given U, compute Xmean initial (centroids)\n",
    "        Ut = U_0.transpose()\n",
    "        dot1 = pd.DataFrame(np.dot(su_diag, Ut))\n",
    "\n",
    "        Xmean0 = round(pd.DataFrame(np.dot(dot1, X)), 4)\n",
    "        \n",
    "\n",
    "        U = []\n",
    "        for r in range(0, n):\n",
    "            U.append([0] * K)\n",
    "\n",
    "\n",
    "        for iter in range(1,maxiter):\n",
    "            #given Xmean, assign each units to the closest cluster\n",
    "            \n",
    "                    \n",
    "            \n",
    "            for r in range(0, n):\n",
    "                \n",
    "                min_dif = mse(X.iloc[r], Xmean0.iloc[0])\n",
    "                \n",
    "                posmin = 0\n",
    "                for j in range(1, K):\n",
    "                    dif = mse(X.iloc[r], Xmean0.iloc[j])\n",
    "                    if dif < min_dif:\n",
    "                        min_dif = dif\n",
    "                        posmin = j\n",
    "                U[r][posmin] = 1\n",
    "            \n",
    "            U = pd.DataFrame(U)\n",
    "            \n",
    "\n",
    "            # given a partition of units, so given U, compute Xmean uptaded (centroids)\n",
    "\n",
    "            # update sum_col\n",
    "            sum_col = []\n",
    "            for t in range(0, K):\n",
    "                sum_col.append(sum(U[t]))\n",
    "            \n",
    "            \n",
    "            ## RARE CASE (BUT POSSIBLE) #############################################################\n",
    "            #if there is some empty cluster we must split the cluster with max sum_col\n",
    "            while sum([sum_col[h] == 0 for h in range(0, len(sum_col))]) > 0: # some cluster is empty\n",
    "\n",
    "                p1 = min(sum_col)\n",
    "                p2 = max(sum_col)\n",
    "\n",
    "                # select min column (empty cluster)\n",
    "                for j in range(0, len(sum_col)):\n",
    "                    if p1 == sum_col[j]:\n",
    "                        c1 = j\n",
    "\n",
    "                # select max column (cluster) for split its points to empty cluster\n",
    "                for k in range(0, len(sum_col)):\n",
    "                    if p2 == sum_col[k]:\n",
    "                        c2 = k\n",
    "\n",
    "                # list of units in max column (cluster)\n",
    "                ind = []\n",
    "                for row in range(0, len(U)):\n",
    "                    if int(U.iloc[row, c2]) == 1:\n",
    "                        ind.append(row)\n",
    "\n",
    "                # split max cluster\n",
    "                ind2 = []\n",
    "                for row in range(0, p2//2):\n",
    "                    ind2.append(row)\n",
    "\n",
    "                for row in range(0, len(ind2)):\n",
    "                    U.iloc[row, c1] = 1\n",
    "                    U.iloc[row, c2] = 0\n",
    "\n",
    "                sum_col = []\n",
    "                for q in range(0, K):\n",
    "                    sum_col.append(sum(U[q]))\n",
    "            #################################################################################################\n",
    "            \n",
    "            # give U compute centroids\n",
    "            _U = U.transpose()\n",
    "            _dot1 = pd.DataFrame(np.dot(diagSU(sum_col), _U))\n",
    "            Xmean = round(pd.DataFrame(np.dot(dot1, X)), 4)\n",
    "            \n",
    "\n",
    "            #compute ojective function\n",
    "            BB = (np.dot(U, Xmean)) - X\n",
    "            f = round(np.trace(np.dot(BB.transpose(), BB)), 4)\n",
    "\n",
    "\n",
    "            #stopping rule\n",
    "            dif = 0\n",
    "            \n",
    "            for k in range(0, K):\n",
    "                dif += mse2(Xmean.iloc[k], Xmean0.iloc[k])\n",
    "                    \n",
    "            if dif > epsilon:\n",
    "                Xmean0 = Xmean\n",
    "            else:\n",
    "                break\n",
    "\n",
    "        if loop == 0:\n",
    "            U_ott = U\n",
    "            f_ott = f\n",
    "            Xmean_ott = Xmean\n",
    "\n",
    "        if f < f_ott:\n",
    "            \n",
    "            U_ott = U\n",
    "            f_ott = f\n",
    "            Xmean_ott = Xmean\n",
    "            \n",
    "\n",
    "    # calculate cost\n",
    "    cost = costf(X, Xmean_ott, U_ott)\n",
    "    print('Done')\n",
    "    return round(pd.DataFrame(U_ott), 4), f_ott, cost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Finally calculate cost for different K-means"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done\n",
      "Done\n",
      "Done\n",
      "Done\n",
      "Done\n"
     ]
    }
   ],
   "source": [
    "solutions = []\n",
    "for i in range(1, 6):\n",
    "    solutions.append(kmeans(dataS, i, 20))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[177.92262100461545, 158.59139555769232, 146.41675536538463, 146.09650042923076, 135.10381812615384]\n"
     ]
    }
   ],
   "source": [
    "cost=[]\n",
    "for i in range(0, len(solutions)):\n",
    "    cost.append(solutions[i][2])\n",
    "\n",
    "print(cost)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAENCAYAAADqsBXqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dd3RU5drG4d9MekMCJBB6b4EEpEYRFGlCEARUWmhHQUUBFaRKAE8siCKKYKGpiBpp0Ygglo8jvYihIy1AKCGhJoS0mfn+QCKRlglJZjK5r7Vca2bP7D3PvIdz7ydvdt5tsFgsFkRExCEZbV2AiIjkH4W8iIgDU8iLiDgwhbyIiANTyIuIODCFvIiIA8tRyCcnJxMaGkpcXBwAa9eu5dFHHyU0NJRXXnmF9PR0AE6ePEmfPn3o0KEDzz77LJcvX86/ykVE5I7uGPIxMTH06tWL2NjYrG3jx49n+vTpREdHk5qaSlRUFACTJ0+md+/erFy5knr16jFr1qx8K1xERO7sjiEfGRlJeHg4/v7+WdtMJhPJycmYTCbS0tJwc3MjIyODLVu20L59ewC6devGypUr869yERG5I+c7vSEiIuKGbZMmTSIsLAxvb2/Kly9Phw4dOH/+PN7e3jg7Xz2kn58f8fHxeV+xiIjkmNW/eE1ISGDatGlER0ezdu1agoODeeONN7BYLBgMhmzv/fdzEREpWHfs5P9t69at1KxZk4oVKwLwxBNPMGLECMaNG0dSUhImkwknJycSEhKyTfHk1PnzlzGbrV9Op2RJb86eTbZ6v6JMY2YdjZd1NF7Wye14GY0GfH29bvm61SFfs2ZN3nrrLRITEylVqhS//PIL9evXx8XFhcaNG7NixQo6d+7M8uXLadmypdUFm82WXIX8tX3FOhoz62i8rKPxsk5+jJfVIV+tWjWGDx9Ov379cHJyolKlSkyZMgWA8PBwxowZw+zZswkICODdd9/N84JFRCTnDPa21PDZs8m5Opv5+fmQkJCUDxU5Lo2ZdTRe1tF4WSe342U0GihZ0vvWr99NUSIiYt8U8iIiDkwhLyLiwBTyIiIOzCFC/mTiZf4TsZoDcRdsXYqIiF1xiJD39XHDyWjggyU7OXPhiq3LERGxGw4R8h5uzoQ/1RyLxcKMb2NISc2wdUkiInbBIUIeoJyfN893q8+Z81f4cNkuMk1mW5ckImJzDhPyALUq+jLgkdrsPXqehT/tx87+zktEpMBZvayBvbu/fgDx51OIXn+U0iU8eaRZJVuXJCJiMw4X8gBdH6jKmfNXWPzbIfyLe9ColvWrYYqIOAKHmq65xmgwMKhjHaqWLcan3+/hyKlLti5JRMQmHDLkAVxdnHi+exDFvFx5f/EOzl5MtXVJIiIFzmFDHuAeL1eGPx5MeqaJGYtjuJKWaeuSREQKlEOHPEC5Ul4817U+JxNT+ChqNyazLq0UkaLD4UMeILBKCfq2r8nOw2f5+ueDti5HRKTAOOTVNTfzYINyxJ9LYdXm45Qu4UGbxhVsXZKISL4rMiEP8PiD1Tlz/gpf/XIAv+IeBFcvZeuSRETyVZGYrrnGaDQwuHMgFf19+Oi73RyL163JRMSxFamQB3BzdWJYjyA83ZyZsXgH55PSbF2SiEi+KXIhD1eXJh7eI4iU1EzeX7KDtHSTrUsSEckXRTLkASqW9uGZLoEci0/ik+93Y9ZiZiLigIpsyAMEVy9Fz4drsP1AIov/75CtyxERyXNF6uqam2nTqDzx51JYuekYpX09aNWgnK1LEhHJM0U+5A0GA73a1CDhQipfrPqLUsU9CKxcwtZliYjkiSI9XXONk9HIM10CKVvKk1nLdnEi8bKtSxIRyRMK+b95uDkzvEcwLs5GZnwbw6XL6bYuSUTkrinkr1PyHneG9wji0uV0Pliyg/QMXVopIoWbQv5fqgQU46nQuhw6eYl5K/bq0koRKdQU8jfRuLY/jz9Yjc17zxD1+xFblyMikmtF/uqaW+nQrCKnz6Xw/fpY/H09uL9+gK1LEhGxmkL+FgwGA2Hta5F4MZUFP+6j1D3u1Kroa+uyRESsouma23B2MvLcY/Xw9/Vg5tKdxJ9LsXVJIiJWUcjfgZe7C8N7BGEwGHjv2xiSr2TYuiQRkRxTyOeAv68nL3Svz9lLqcxcupNMk+4TKyKFg0I+h2qUL86gjnX46/gFPvtxHxZdWikihYB+8WqF5oFlOHP+CsvXHqF0CU9C76ts65JERG4rxyGfnJxMz549+eijjzh06BDvvvtu1mvx8fEEBwfz8ccfs3fvXsaPH8/ly5dp3LgxkydPxtnZcc4lne+vTPz5FJb+7zD+vh40rVPa1iWJiNxSjqZrYmJi6NWrF7GxsQC0atWKqKgooqKimDNnDt7e3owdOxaAUaNGMXHiRFatWoXFYiEyMjLfircFg8HAgEfqUKP8PcyJ3svBExdtXZKIyC3lKOQjIyMJDw/H39//htemTp1Kz549qVy5MidOnCA1NZUGDRoA0K1bN1auXJm3FdsBF2cjz3erTwkfNz5YsoOEC1dsXZKIyE3lKOQjIiJo3LjxDdtjY2PZvHkz/fr1A+DMmTP4+fllve7n50d8fHwelWpffDxdGf54ECaThRmLd5CSmmnrkkREbnBXk+XffPMNvXv3xtXVFQCz2YzBYMh63WKxZHueEyVLeue6Hj8/n1zvm9vPGz+oKRM/3sCcFXsJf6o5zk6F64Klgh6zwk7jZR2Nl3XyY7zuKuR/+eUX5s6dm/W8TJkyJCQkZD1PTEy86RTP7Zw9m4zZbP3liX5+PiQkJFm9390KuMedfh1qMX/FPmYs2kZY+1pWn9hsxVZjVlhpvKyj8bJObsfLaDTctjnOddt57tw5UlNTqVChQta2cuXK4ebmxrZt2wCIioqiZcuWuf2IQuOBoLJ0CqnE//15kp+2HLd1OSIiWXLdycfFxVGmTJkbtk+bNo0JEyaQnJxMYGBg1ny9o3usZVXiz6UQ+etB/Ip7cG9NvzvvJCKSzwwWO/vTzcI2XXO99AwTby3azonEZMb0uZfKZYrZtJ47sYcxK0w0XtbReFnH7qZr5EauLk4M614fHw8XZizewblLqbYuSUSKOIV8HrvH243hjweTlm5ixuIdpKbr0koRsR2FfD4o7+fNc13rcSLhMh9H7c7V9JOISF5QyOeTelVL0qdtDWIOneXrXw/YuhwRKaIcZ+UwO/TQveU5fe4Kq7cep7SvJw83Km/rkkSkiFHI57MnW1cn4cIVFv38F37FPQiqVtLWJYlIEaLpmnxmNBoY/GhdKvh5MztqF8fPJNu6JBEpQhTyBcDd1ZlhPYLwcHVixuIYLian2bokESkiFPIFpEQxd4b3CCb5SgbvL9lBWobJ1iWJSBGgkC9Alcr4MOTRQGJPJTEneg9m+/pjYxFxQAr5Atawhh9Ptq7Otv0JLFlzyNbliIiD09U1NtC2SQVOn7/CjxuPUdrXk5bBZW1dkog4KIW8DRgMBvq0rUHChSt8sWo/fve4U6dyCVuXJSIOSNM1NuJkNPJsl3qUKeHJh8t2cersZVuXJCIOSCFvQ57uzgzvEYSzk4H3vo3hUkq6rUsSEQejkLexUsU9eKFHEBeS05m5dCcZmbq0UkTyjkLeDlQrew9PhdblYNxF5q/Yh53dx0VECjGFvJ1oUtuf7q2qsnFPPFFrj9i6HBFxELq6xo50bF6J0+dS+G5dLKVLeBISeOM9dEVErKGQtyMGg4H+HWpz9mIq81fspWQxd2pWKG7rskSkENN0jZ1xdjLy3GP1KXmPBzOX7iT+fIqtSxKRQkwhb4e8PVwY8XgQFouF977dweXUDFuXJCKFlELeTpX29eSF7kGcvXiFD5fuJNNktnVJIlIIKeTtWM0KxRn4SB32HbvA56v269JKEbGafvFq50LqlSH+/N9X3Ph60Cmksq1LEpFCRCFfCHRpUYX481dYsuYwpX09aVzb39YliUghoemaQsBgMDCoY22ql7uHT6P3cPjkJVuXJCKFhEK+kHBxduL57vW5x8uV95fsIPHiFVuXJCKFgEK+ECnm6cqIx4PJyDQz49sdpKRm2rokEbFzCvlCpmwpL4Y+Vo/T51L4KGoXJrMurRSRW1PIF0J1K5cgrH0tdh05x6LVB3RppYjckq6uKaRaBpcl/lwKP246RukSnrRrUsHWJYmIHVLIF2LdH6zGmfNX+OaXA/gX96BBjVK2LklE7Iymawoxo8HAU53rUqmMDx9/t5ujp5NsXZKI2BmFfCHn5uLEsB5BeHk4M2NxDOeT0mxdkojYEYW8Ayju7cbwHsFcSTcxY3EMqem6tFJErlLIO4gK/t482yWQ42eS+eS7PZjNuuJGRBTyDiWoWil6t6nJnwcTifztoK3LERE7kOOQT05OJjQ0lLi4OAC2b9/OE088QadOnXjppZdIT08HYO/evXTr1o327dszfvx4MjM1dVCQHm5UnjaNyvPTluP8tv2ErcsRERvLUcjHxMTQq1cvYmNjgauB/8ILLzBlyhR++OEHABYvXgzAqFGjmDhxIqtWrcJisRAZGZk/lcst9Xy4BkHVSvLlT3+x6/BZW5cjIjaUo5CPjIwkPDwcf/+rS9yuW7eOBg0aULt2bQAmTJhA27ZtOXHiBKmpqTRo0ACAbt26sXLlynwqXW7FaDQw5NFAypbyYnbULuISkm1dkojYSI7+GCoiIiLb86NHj+Lp6cmLL77I4cOHuffeexkzZgx79uzBz88v631+fn7Ex8dbVVDJkt5Wvf96fn4+ud7XEU0Zch8j31/DzKU7mTa8Jb4+7je8R2NmHY2XdTRe1smP8crVX7yaTCbWrl3LN998Q9myZRk/fjyffPIJ9913HwaDIet9Fosl2/OcOHs2OVdXhvj5+ZCQoD8G+rfnu9XnzS//YNInG3ilV0NcXZyyXtOYWUfjZR2Nl3VyO15Go+G2zXGurq4pVaoUwcHBVKhQAScnJx555BF27NhBmTJlSEhIyHpfYmJi1hSP2EblMsUY3DmQIycvMeeHvZi1mJlIkZKrkG/RogW7d+/m1KlTAPz2228EBgZSrlw53Nzc2LZtGwBRUVG0bNky76qVXLm3ph+PP1SdrfvOsOx/h21djogUoFxN1wQEBDBlyhSeeeYZ0tLSqFOnDqNHjwZg2rRpTJgwgeTkZAIDA+nXr1+eFiy5075pBeLPp/DDhqP4+3rwQFBZW5ckIgXAYLGzxcg1J59/Mk1m3vs2hv3HLvDSkw1o2biixswK+jdmHY2XdexqTl4KJ2cnI891rYe/rwcfLt3JniO6hl7E0SnkixhPdxdGPB6Mt4cLY2etI3p9rH4ZK+LAFPJFkF9xD8IHNqFFUFmW/u8w737zJxeTtUSxiCNSyBdRHm7OjOzbiAGP1OZg3EXC521m95Fzti5LRPKYQr4IMxgMtAwuy4T+jfH2dOXdb/5kyZpDmMxmW5cmInlEIS+U9/Pm1f6NaREUwA8bjvLWl9s5ezHV1mWJSB5QyAtw9TaCAzvWYfCjdYlLSGbS/M1s/yvhzjuKiF1TyEs2zeuWIXxgE0rd48EHS3fy5eq/yMjU9I1IYaWQlxuU9vVkXFgj2jQuzy/b4nj9i23En0uxdVkikgsKebkpF2cjvdvU5IXu9Um8eIVJC7awcfdpW5clIlZSyMttNazhx+RBTang780n3+9h3oq9pKWbbF2WiOSQQl7uqEQxd0b3bkjofZVYt+MUUz7bortNiRQSCnnJESejkW4tq/FSzwZcTs3ktc+28n9/nsDO1rcTkX9RyItVAiuXYPKgptQsfw+fr9zPR1G7SUnNtHVZInILCnmx2j1errz4ZAO6t6rKtv0JTF6wmSOnLtm6LBG5CYW85IrRYKBTSGVG92mIyWzh9S+28dPmY5q+EbEzCnm5KzXKF2fSwKYEVSvJ178e5P3FO0hKSbd1WSLyN4W83DVvDxee71af3m1qsDv2HJPmb+Gv4xdsXZaIoJCXPGIwGGjTuALjwxrj4mzkrUV/8N26I7m6laOI5B2FvOSpSmV8CB/QhGZ1S7P89yO8882fXNANSURsRiEvec7DzZmnQ+sysGNtDp28ekOSXYd1P1kRW1DIS74wGAw8EFSWif2bUMzLlXcjY/j2t4NkmrSipUhBUshLvipbyotX+zXmwQZl+XHTMd768g8SL1yxdVkiRYZCXvKdq4sT/TrU5pkugZw8e5lJ87ewbf8ZW5clUiQo5KXANK1TmvCBTfH39eDDZbtY+NN+MjK1oqVIflLIS4HyL+7BuLBGtGtSgV//OEHE59s4rRuSiOQbhbwUOGcnIz0frsHwHkGcS0pj8vwtrN91ytZliTgkhbzYTHD1Ukwa2IRKZXyYE72XudF7SE3XipYieUkhLzZVopg7o3o14NH7K7N+12mmLNjK8TO6IYlIXlHIi805GY10faAqI3s15Er61RuS/LZdNyQRyQsKebEbdSr5MnlgU2pXLM4Xq/Yze/kuUlIzbF2WSKGmkBe7UszLlRFPBPP4Q9XYfiCRSfO3cPikbkgiklsKebE7RoOBR5pVYkyfe7FY4I2F21i56RhmTd+IWE0hL3arWrl7mDSoCQ2qlyLyt4PM+HYHl3RDEhGrKOTFrnm5u/DcY/Xo264me4+eZ9K8zew7et7WZYkUGgp5sXsGg4HW95ZnQr9GuLk68/bX21n++2HdkEQkBxTyUmhULO1D+IDGhASW4bt1sbz91XbOJ+mGJCK3k+OQT05OJjQ0lLi4OADGjh1Lu3bt6NKlC126dGH16tUArF+/ns6dO9OuXTumT5+eP1VLkeXu6sxToXX5T6c6xJ5OInzeZnYc0g1JRG7FOSdviomJYcKECcTGxmZt27VrFwsXLsTf3z9rW2pqKuPGjeOLL74gICCAIUOGsGbNGlq1apXnhUvRdn/9AKqWLcbs5bt579sYOjStSLdWVXF20g+nItfL0f8jIiMjCQ8Pzwr0K1eucPLkScaNG0fnzp15//33MZvN7Nixg0qVKlGhQgWcnZ3p3LkzK1euzNcvIEVXQEkvXu3fiIfuLcfKzcd4Y+EfJOiGJCLZ5KiTj4iIyPY8MTGR5s2bEx4ejo+PD0OGDGHx4sV4enri5+eX9T5/f3/i4+PztmKR67g4OxHWrhZ1Kvoy/8d9TJq/hYGP1KZxbf877yxSBOQo5P+tQoUKfPjhh1nPw8LCWL58Oe3bt8dgMGRtt1gs2Z7nRMmS3rkpCQA/P59c71tUOcqYPeLnw72BAbz9xVZmLd/FIyGV+U+Xeri5OOXp5zjKeBUUjZd18mO8chXy+/fvJzY2lvbt2wNXw9zZ2ZkyZcqQkJCQ9b6EhIRsc/Y5cfZscq4ujfPz8yEhIcnq/YoyRxszI/Dyk8Es+99hftwQy86DiTzbNZCAkl55cnxHG6/8pvGyTm7Hy2g03LY5ztVvqSwWC6+//joXL14kIyODb775hrZt2xIcHMyRI0c4evQoJpOJ6OhoWrZsmZuPEMkVZycjjz9UnRGPB3MhOY3JC7awbqduSCJFV646+dq1azN48GB69epFZmYm7dq1IzQ0FIA333yTF154gbS0NFq1akWHDh3ytGCRnAiqVpLJg5ry6fe7mfvDXvbEnqdvu5p4uOXqn7xIoWWw2Nmi3ZquKThFYczMZgvR62OJWncEf19Pnu0SSMXSuZv3LArjlZc0Xtaxq+kakcLCaDTwaIsqvNKrIWnpmfz38238si1ONySRIkMhL0VCrYq+TB7UlLqVffly9V98uGwXl3VDEikCFPJSZPh4ujKsRxBPtq5OzMFEJs3bwsETF21dlki+UshLkWI0GGjftCJj+zbCYIA3F/7Bio1HC+yGJMuXL6Z//1707fs4ffs+wWuvvcrp06dztO+LLw7lwoULAIwcOYwjRw7nSU0REZNYtOiLPDlWTv3xx1bCwp4osM/bvHkj3buH8vTT/UhLS73j+0+dOknbtg/k+vPWr1/LnDkf5Xr/vKRLDaRIqlq2GJMGNmXByn0s/r9D7Dt6nqdC61LMyzXfPnPmzPc4ePAvpk6dTunSZTCbzaxatYJnnhnIJ58swN+/9G3337JlU9bjadPez7c6HdHPP6+ic+euDBjwVIF83t69u7l0yT5+SlTIS5Hl6e7Ms10CWVPZl69+PkD4vM0M7lyXOpVL5PlnnTkTT1TUEpYs+YFixYoBYDQaeeSRUPbv38cXXyzg5ZdH06NHZ9q0ac+WLZtITk6iZ8++PPZYD15/fTIAw4YN4e23ZzB06NP8979vkZKSwscff0jp0mU4duwoHh7u9OkzgMWLv+bYsaM8+GBrhg17GbPZzPvvv8vu3Tu5ciUFi8XC6NETCApqcMuaIyIm4eXlxaFDBzlzJp5q1WowYcJkPD09adGiMdHRP1O8eHGArOeHDx/MqufkyeO4uLjetB64ugbWhAmvEBcXh7e3N6+8Mp6KFSuRkZHB7Nnv8+eff2AymalZsxYjRozEy8ubHj06U7duPQ4dOsDgwUNp1eqhrHozMzP54IN32bZtC0ajkbp16zFs2EssX76E339fg5ubG5cvX2bo0OHZvue6db/z6aezsVjMuLt7MHLkWLy9/7laZe7cj7l48QIvvTT6hudr1vzKZ5/NxWAwYjQaGTp0OC4urkRFLcVsNuHl5c2QIUOJjl7O0qWLsVjMFCtWnJdeeoVKlSoTETGJS5cucuLECdq0aU3Dhs2ZOfNdTCYzBoOBsLABPPjgw3f1b0/TNVKkGQwGHmxQjlf7NcbT3ZlpX//Jsv8dxmQ25+nn7Nmzi0qVqmQF/PUaN27Kjh1/Zj2/dOkic+Z8zgcffMzcuR9x6NBBxo0LB+D99z+mdOky2fbft28Pffv2Z8GCRXh6erNw4QLefnsG8+Z9ydKl35KYmMCePbtITEzg44/ns3Dht3ToEMrChZ/dse79+/fyzjsf8OWXizl9+iS//fbzHfe5Vk9UVNQt64GrJ74nn+zDggWLaNu2A6+9NhGAhQsX4OTkzNy5C/nss68oVcqP2bNnZh2/atVqfPnl4mwBD/DZZ3NJTExgwYKvWLDgK8xmMx9+OIPevfvRokVLnnii9w0Bf+7cWV577VXGjZvIZ599Te/eYXz00Uxy6sMPZ/DSS2OYO/cLnn76WbZv30ZgYD26dOlG69ZtGTJkKNu3b+PHH39g1qw5zJ+/iD59+jFu3MisY6SmprFwYSSjRo1i3ryPefLJPsybt5CxYyeybdvWHNdyK+rkRYDy/t5M7N+EL1f/xffrY9l/7DyDHw2kRDH3PPuMzMzMm27PyEjPtsZTt25PYDAY8PcvTbNmIWzevJFq1arf8rgBAWWpWbM2AOXKlcPLyxsXFxeKFy+Ol5cXly5dpF69IAYPLkZU1FJOnIhj+/ZteHp63rHmZs3uw9X16hRW1arVuXTp0h33yUk9ANWqVad+/WAAOnbszDvvvElycjLr1/9OUlJy1vRUZmYGxYv7Zh3/Vj99bNy4jsGDn8PZ+Wqs9ejxJGPHjrzpe6/ZuTOGKlWqZdXbqlVrWrVqzalTJ+/4PQEefrgd48ePJCSkBU2aNKN37343vGfDhrXExR3nmWcGZW1LSkrKGoegoOCs7Q891IZ3353KunW/07hxU4YMGZqjOm5HIS/yNzdXJwZ1qkOdyr58vmo/k+ZvYVCnOlRN/IvEpUv46/w5nH1LUKpbd4o1v8+qYwcG1icu7hhnzyZSsmSpbK/98cc26tULynru5PTPompmswWnO6yR7+Liku35tZC73vr1a5kxYxo9e/blgQdaUalSZVatWnHHul1d3bIeGwyGbH9fcO1xRkb2S1FzUg+A0Zh98TiDwYCzszMmk5nhw18mJOR+AFJSUkhP/+cG7rc6OZlMZuCfk6XZbLnlifUaJyenGxZVPHToIF5e/6x3dPV7/7PP9cccMmQooaFd2LJlIytWRPP11wv59NPPb6irffuOPPfcsL/rMpOYmICPz9Wf6jw8/vk+Xbt2p0WLlmzevJFNm9Yzb94nLFq0BDc3N3JL0zUi/xISWIbwAU0oUcyN1fOWc2L+PDLPnQWLhcxzZ4n/fAGXNq636ph+fv706NGTSZPGk5BwJmv7Dz98x5o1v9K3b/+sbStX/gDA6dOn2bJlI83/PqE4OTndMbRuZcuWTdx//wM89lgPateuw++//x/mu5iSKl7cl3379gCwenXu7hlx8OBfHDiwH4CoqCXUrx+Mu7s7zZqFsHRpJBkZGZjNZt566798/PGdp1CaNQth+fLFZGZmYjabWbo0kiZNmt12n7p163H06BEOHz4EwO+/r+G1117N9p7ixX3Zv38vFouFlJTLrF//O3A17Hv06Exqaipdu/bg5ZdHc+jQQdLT07P9b9WsWQg//7yKxMREAJYvX8Lw4c/etJ5nnhnEX3/tp2PHzrzyyniSk5M4d+7u7nymTl7kJsqU8GR8WGP2vbQQJ1P2YLWkp3P8q2+ItZTFyWjEycmAs9GAk5MRJ6Ph6n/ZHhtwMhpp3akvrt4/8tLLI8jISCczM4MaNevw5tuzcfcuQVJKOhaLhRMnTjBoUF/S0tIYPnwkFStWBuDBBx/m+ecH8/rrU63+Pl27dmfSpHH06/ckJpOJJk2as2bNr7kO+hEjRvLuu1Px8fGmceNmN/x0khOVK1dh3rxPOXnyBL6+vkyYcPWXywMG/IeZM2cwcGAfzGYTNWrU5PnnR9zxeNf2GzCgNyZTJnXrBjJixCu33adEiZJMnPhfIiImYTKZ8PLyYtKk17O9p127R9i4cT09ez6Gn58/DRrcm7Xy7rBhLzN58nicnZ0xGIyMGTMRV1dXGjVqwuTJE5g+fSovvvgKffr058UXn8NoNOLp6UVExNs3XYb92WeHMWPGND79dBYGg4GBA58mIKCsFaN6I61dU4RpzO7sr6cG3HS7BXir+o3zr3fr8C9vULZRX9yLVwCuXtd/9SRxs5OH8erJ5boTyc0fG7KfjP5+fNv9bnfcmx3nX7W5ujhRpWIJ/fuyQn6tXaNOXuQ2nEuUvDpV8y8uJUry0cutMJktV/8zmTGZLWRe99hk+vs1s/mmjzOzbb+63/R1zjzcqDz+Zatct6PfBDwAAA8FSURBVP3v9/792HztONd9xvXHSsswYTJl3lDbjXVePU5+tXlN65bhyYeq4euT+/lkuXsKeZHbKNWtO/GfL8By3S/+DK6ulOrWHdc8vusUQJvvf8zzY96J2fJP4Gc7Od3sxGX++yRzk5OF6brHZy+l8dPmY+w6lEjvtjUICSxj9V3iJG8o5EVu49pVNIlLl5B5F1fX2DOjwYDR2YBLHl+H0emBqryzcBtzoveydV8C/TrUori3uvqCpjn5IkxjZh2Nl3X8/HyIj7/E6q3HWfq/w7g4GdXV34bWkxeRQsdovLog3KSBTShbyos50Xv5YMlOLiSn2bq0IkMhLyL5LqCkF2P63MuTrauzO/Ycr87ZxIZdp3XzlgKgkBeRAnF9Vx9Q0otPo/eoqy8ACnkRKVDq6guWQl5ECtz1XX2Zkp7q6vORQl5EbCagpBdj+zTiiYfU1ecXhbyI2JTRaKBDM3X1+UUhLyJ24aZd/W519XdLIS8iduOGrv57dfV3SyEvInZHXX3eUciLiF1SV583FPIiYtfU1d8dhbyI2L2bdfUzl+7korr6O1LIi0ihcX1Xv+vIOSaoq78jhbyIFCrq6q2jkBeRQkldfc4o5EWk0FJXf2cKeREp9NTV35pCXkQcgrr6m1PIi4hDub6r33n4ale/sQh39Qp5EXE417r6yYOaUKaEJ58U4a4+RyGfnJxMaGgocXFx2bYvXLiQsLCwrOcnT56kT58+dOjQgWeffZbLly/nbbUiIlYIKOnF2L5Fu6u/Y8jHxMTQq1cvYmNjs20/ePAgn3zySbZtkydPpnfv3qxcuZJ69eoxa9asPC1WRMRaRb2rv2PIR0ZGEh4ejr+/f9a29PR0Jk6cyLBhw7K2ZWRksGXLFtq3bw9At27dWLlyZT6ULCJivaLa1Tvf6Q0RERE3bHvnnXfo3r075cuXz9p2/vx5vL29cXa+ekg/Pz/i4+PzsFQRkbtzrasPrl6SeT/s5ZPv97Bl3xn6ta/FPd5uti4vX9wx5P9t3bp1nDp1irFjx7Jp06as7RaLBYPBkO29/36eEyVLelu9zzV+fj653reo0phZR+NlHXsdLz8/H96pWZqoNYdYuHIvE+dtZshjQbRsWC5XuZWXdeU1q0M+OjqaAwcO0KVLF1JSUkhMTGTEiBG8/fbbJCUlYTKZcHJyIiEhIdsUT06dPZuM2Wz9j09+fj4kJCRZvV9RpjGzjsbLOoVhvB6oV5rqAd7M+2Ev077cxi+bj9qsq8/teBmNhts2x1aH/BtvvJH1eNOmTcycOZP33nsPgMaNG7NixQo6d+7M8uXLadmypdUFi4gUpGtz9au2HGPZ/44wYc4m+rSrSbM6pW3a1eeVPL1OPjw8nMjISDp27MjWrVsZMWJEXh5eRCRfGI0GHmlW6Z8rcL5znCtwDBY7+9WypmsKjsbMOhov6xTW8TKbLVldvZuLscC6+vyartFfvIqIXOeWXf3ldFuXlisKeRGRm7g2V//4Q9WuXlf/6UY27il819Ur5EVEbuFmXf2Hy3YVqq5eIS8icgfXd/U7Dp0tVF29Ql5EJAeudfWTBjahdCHq6hXyIiJWKFvKi3GFqKtXyIuIWKkwdfUKeRGRXCoMXb1CXkTkLtysq59lR129Ql5EJA9c39XHHDrLq3M2sWlPvM27eoW8iEgeub6r9/f14OPvdtu8q1fIi4jksbKlvBjb914ef9D2Xb1CXkQkHzgZjTzS/GpX71fcdl29Ql5EJB+VLeXFuDDbdfUKeRGRfGbLrl4hLyJSQG7W1W/em79dvUJeRKQA/bur/yjqalefnJI/Xb1CXkTEBv7d1W/YeSpfPsfqG3mLiEjeuNbVt763PAFlinHu3OU8/wx18iIiNubm6oSTU/7EsUJeRMSBKeRFRByYQl5ExIEp5EVEHJhCXkTEgSnkRUQcmN1dJ280Gmyyb1GlMbOOxss6Gi/r5Ga87rSPwWLr25aIiEi+0XSNiIgDU8iLiDgwhbyIiANTyIuIODCFvIiIA1PIi4g4MIW8iIgDU8iLiDgwhbyIiANzmJBPTk4mNDSUuLg4W5di92bOnEmnTp3o1KkTU6dOtXU5dm/GjBl07NiRTp06MX/+fFuXU2i89dZbjBkzxtZl2L2wsDA6depEly5d6NKlCzExMXl6fLtbuyY3YmJimDBhArGxsbYuxe6tX7+etWvXsmzZMgwGA0899RSrV6+mbdu2ti7NLm3evJmNGzfy3XffkZmZSceOHWnVqhVVq1a1dWl2bcOGDSxbtowHH3zQ1qXYNYvFQmxsLL/99hvOzvkTxw7RyUdGRhIeHo6/v7+tS7F7fn5+jBkzBldXV1xcXKhWrRonT560dVl2q2nTpnz++ec4Oztz9uxZTCYTnp6eti7Lrl24cIHp06fzzDPP2LoUu3f48GEABg0axKOPPsrChQvz/DMcopOPiIiwdQmFRo0aNbIex8bG8uOPP/LVV1/ZsCL75+Liwvvvv8+8efPo0KEDpUuXtnVJdm3ixIm8+OKLnDp1ytal2L1Lly4REhLCq6++SkZGBv369aNKlSrcf//9efYZDtHJi/UOHDjAoEGDeOWVV6hcubKty7F7w4YNY8OGDZw6dYrIyEhbl2O3vv32WwICAggJCbF1KYVCw4YNmTp1Kj4+PpQoUYIePXqwZs2aPP0Mh+jkxTrbtm1j2LBhjBs3jk6dOtm6HLt26NAh0tPTqVOnDh4eHrRr1479+/fbuiy7tWLFChISEujSpQsXL14kJSWF119/nXHjxtm6NLu0detWMjIysk6KFoslz+fm1ckXMadOnWLo0KFMmzZNAZ8DcXFxTJgwgfT0dNLT0/nll19o1KiRrcuyW/Pnzyc6OpqoqCiGDRtG69atFfC3kZSUxNSpU0lLSyM5OZlly5bl+UUQ6uSLmLlz55KWlsabb76Zta1nz5706tXLhlXZr1atWrFjxw66du2Kk5MT7dq108lR8sxDDz1ETEwMXbt2xWw207t3bxo2bJinn6E7Q4mIODBN14iIODCFvIiIA1PIi4g4MIW8iIgDU8iLiDgwhbzkq7i4OGrVqsW3336bbfvcuXPzdIXC1q1bs3Pnzjw73u0kJyfTs2dPOnXqxE8//ZSjfcLCwli5cmWuPi8pKYl+/frlal8RXScv+c5oNPLWW2/RqFEjh1i9ce/evZw9e5bVq1cXyOddvHixwE5g4ngU8pLv3N3dGThwICNHjuTrr7/G1dU12+tjxoyhRo0a/Oc//7nheevWrQkNDWXjxo1cvHiRp556ij/++IPdu3fj7OzM7NmzsxYMW7RoEfv27SM9PZ2BAwfSo0cPAH799Vdmz55NRkYG7u7ujB49moYNG/LBBx/w559/cubMGWrVqsW0adOy1fXzzz8zc+ZMzGYzXl5ejB07Fm9vb8aNG0d8fDxdunThm2++wd3dPWufhIQEwsPDOXz4MEajkZ49e2brwuPi4ujcuTPbt2+/4XlCQgKjR4/m/PnzwNU/xBoxYgRjx44lNTWVLl26sHTpUmJjY4mIiODChQuYTCbCwsLo0aMHmzZtIiIiAk9PTy5fvsyiRYsYP348R48exWg0EhgYyJQpUzAa9QN8UaKQlwLx7LPPsmHDBqZPn87o0aOt2jctLY3IyEhWrFjByy+/zLJly6hduzZDhw5l2bJlWUvaurm5sWzZMuLj43nssccIDg7GxcWF6dOn8/nnn+Pr68uBAwcYOHBg1jTLiRMniI6OvmG9kEOHDhEeHs7XX39NhQoV2LBhA8899xwrV67kv//9L6+99hpRUVE31Dp58mQqV67MrFmzSEpKolevXrRq1SpH3zMyMpLy5cszb948UlJSGD9+PElJSbzxxht07tyZqKgoMjMzGTZsGFOnTiUwMJCkpCSefPJJqlevDlxdeO7nn3+mXLlyLF++nMuXLxMVFYXJZCI8PJzjx49TqVIlq8ZfCjeFvBQIo9HI22+/TdeuXWnRooVV+7Zr1w6AChUqUKpUKWrXrg1AxYoVuXjxYtb7evbsCUDp0qW5//772bBhA05OTpw5c4YBAwZkvc9gMHDs2DEAGjRocNMFoTZu3Ejz5s2pUKECACEhIZQoUYJdu3ZhMBhuWev69esZNWoUAD4+PkRHR+f4ez7wwAMMHjyYU6dOcd999/Hyyy/j4+OT7TvGxsZy7NixbOvBpKamsmfPHqpVq0ZAQADlypUDoFGjRkyfPp2wsDDuu+8++vfvr4AvghTyUmACAgKYPHkyo0ePpmvXrlnbDQYD16+ukZGRkW2/66d3XFxcbnn866chzGYzzs7OmEwmQkJCeO+997JeO3XqFP7+/qxevfqWNwAxm803hLnFYiEzM/O2NTg7O2fb7/jx4/j6+mY9v913DQoK4pdffmHDhg1s3LiRxx9/nE8//ZTixYtnvcdkMuHj45Ptp4jExER8fHz4888/s32fChUqsHr1ajZt2sTGjRsZOHAgU6ZMoXXr1resXxyPJuekQHXo0IGWLVvy2WefZW3z9fVl165dAMTHx7N58+ZcHXvZsmUAnDx5kg0bNhASEkJISAjr1q3j0KFDAKxZs4ZHH32U1NTU2x4rJCSEtWvXcvz4cYCsteSDg4PvuN+SJUuAq1fF9O/fP9ttKYsVK0ZGRgYHDx4E4Icffsh6bdq0acyaNYs2bdowfvx4qlevzoEDB7JOVhaLhSpVquDu7p4V8qdOnSI0NDRr/K63aNEixo4dS4sWLRg1ahQtWrRgz549t61fHI86eSlwEyZMYNu2bVnPw8LCGDlyJO3bt6d8+fI0b948V8dNS0vjscceIyMjgwkTJlClShUApkyZwksvvZS1Vvfs2bPx8vK67bGqV69OeHg4zz//PCaTCXd3dz766CN8fHxuu9/EiROZNGkSnTt3xmKxMGTIEOrVq5f1uo+PD6NGjeLpp5+mRIkSdOjQIeu1/v37M2bMGEJDQ3F1daVWrVp06tQJJycngoKC6NSpE19++SWzZs0iIiKCOXPmkJmZyfDhw2nUqBGbNm3KVkvXrl3ZvHkzHTt2xMPDg4CAAMLCwqwdVinktAqliIgD03SNiIgDU8iLiDgwhbyIiANTyIuIODCFvIiIA1PIi4g4MIW8iIgDU8iLiDiw/weF6dLQUfLZhQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(cost)\n",
    "plt.annotate('Optimal number of clusters',\n",
    "            xy=(2.2, cost[2]))\n",
    "plt.plot(2, cost[2], marker='o', color='r')\n",
    "plt.xlabel('Number of clusters')\n",
    "plt.xticks([0,1,2,3,4], [1,2,3,4,5])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So the optimal number of clusters is 3 and we can recalculate the clusters more thoroughly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {},
   "outputs": [],
   "source": [
    "cluster = solutions[2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see each cluster of what unit is formed.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cluster 1 \n",
      "     Alcohol  Malic acid   Ash  Alcalinity_of_ash  Magnesium  Total_phenols  \\\n",
      "59    12.37        0.94  1.36               10.6       88.0           1.98   \n",
      "60    12.33        1.10  2.28               16.0      101.0           2.05   \n",
      "62    13.67        1.25  1.92               18.0       94.0           2.10   \n",
      "64    12.17        1.45  2.53               19.0      104.0           1.89   \n",
      "65    12.37        1.21  2.56               18.1       98.0           2.42   \n",
      "\n",
      "    Flavanoids  Nonflavanoid_phenols  Proanthocyanins  Color_intensity   Hue  \\\n",
      "59        0.57                  0.28             0.42             1.95  1.05   \n",
      "60        1.09                  0.63             0.41             3.27  1.25   \n",
      "62        1.79                  0.32             0.73             3.80  1.23   \n",
      "64        1.75                  0.45             1.03             2.95  1.45   \n",
      "65        2.65                  0.37             2.08             4.60  1.19   \n",
      "\n",
      "    OD280/OD315_of_diluted_wines  Proline  \n",
      "59                          1.82    520.0  \n",
      "60                          1.67    680.0  \n",
      "62                          2.46    630.0  \n",
      "64                          2.23    355.0  \n",
      "65                          2.30    678.0   \n",
      "\n",
      "Cluster 2 \n",
      "    Alcohol  Malic acid   Ash  Alcalinity_of_ash  Magnesium  Total_phenols  \\\n",
      "0    14.23        1.71  2.43               15.6      127.0           2.80   \n",
      "1    13.20        1.78  2.14               11.2      100.0           2.65   \n",
      "2    13.16        2.36  2.67               18.6      101.0           2.80   \n",
      "3    14.37        1.95  2.50               16.8      113.0           3.85   \n",
      "4    13.24        2.59  2.87               21.0      118.0           2.80   \n",
      "\n",
      "   Flavanoids  Nonflavanoid_phenols  Proanthocyanins  Color_intensity   Hue  \\\n",
      "0        3.06                  0.28             2.29             5.64  1.04   \n",
      "1        2.76                  0.26             1.28             4.38  1.05   \n",
      "2        3.24                  0.30             2.81             5.68  1.03   \n",
      "3        3.49                  0.24             2.18             7.80  0.86   \n",
      "4        2.69                  0.39             1.82             4.32  1.04   \n",
      "\n",
      "   OD280/OD315_of_diluted_wines  Proline  \n",
      "0                          3.92   1065.0  \n",
      "1                          3.40   1050.0  \n",
      "2                          3.17   1185.0  \n",
      "3                          3.45   1480.0  \n",
      "4                          2.93    735.0   \n",
      "\n",
      "Cluster 3 \n",
      "     Alcohol  Malic acid   Ash  Alcalinity_of_ash  Magnesium  Total_phenols  \\\n",
      "61    12.64        1.36  2.02               16.8      100.0           2.02   \n",
      "70    12.29        1.61  2.21               20.4      103.0           1.10   \n",
      "77    11.84        2.89  2.23               18.0      112.0           1.72   \n",
      "83    13.05        3.86  2.32               22.5       85.0           1.65   \n",
      "92    12.69        1.53  2.26               20.7       80.0           1.38   \n",
      "\n",
      "    Flavanoids  Nonflavanoid_phenols  Proanthocyanins  Color_intensity    Hue  \\\n",
      "61        1.41                  0.53             0.62             5.75  0.980   \n",
      "70        1.02                  0.37             1.46             3.05  0.906   \n",
      "77        1.32                  0.43             0.95             2.65  0.960   \n",
      "83        1.59                  0.61             1.62             4.80  0.840   \n",
      "92        1.46                  0.58             1.62             3.05  0.960   \n",
      "\n",
      "    OD280/OD315_of_diluted_wines  Proline  \n",
      "61                          1.59    450.0  \n",
      "70                          1.82    870.0  \n",
      "77                          2.52    500.0  \n",
      "83                          2.01    515.0  \n",
      "92                          2.06    495.0   \n",
      "\n"
     ]
    }
   ],
   "source": [
    "def clusterization(cluster, data):\n",
    "    c1 = []\n",
    "    c2 = []\n",
    "    c3 = []\n",
    "    for i in range(0, len(data)):\n",
    "        if cluster[0].iloc[i][0] == 1:\n",
    "            c1.append(data.iloc[i])\n",
    "        if cluster[0].iloc[i][1] == 1:\n",
    "            c2.append(data.iloc[i])\n",
    "        if cluster[0].iloc[i][2] == 1:\n",
    "            c3.append(data.iloc[i])\n",
    "\n",
    "    c1 = pd.DataFrame(c1)\n",
    "    c2 = pd.DataFrame(c2)\n",
    "    c3 = pd.DataFrame(c3)\n",
    "    return c1, c2, c3\n",
    "\n",
    "division = clusterization(cluster, data) \n",
    "#output: pos0=cluster1, pos1=cluster2, pos2=cluster3\n",
    "\n",
    "for i in range(0, len(division)):\n",
    "    print('Cluster', i+1 ,'\\n',division[i].head(), '\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And how many units is formed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cluster 1 has 42 units\n",
      "Cluster 2 has 77 units\n",
      "Cluster 3 has 59 units\n"
     ]
    }
   ],
   "source": [
    "for i in range(0, len(division)):\n",
    "    print('Cluster', i+1, 'has', len(division[i]), 'units')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The optimal cost is..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "146.41675536538463\n"
     ]
    }
   ],
   "source": [
    "print(cluster[2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. See for which variables the clusters were formed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To see which variables contributed most to cluster creation, we can calculate the F-measure for each variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {},
   "outputs": [],
   "source": [
    "def f_variables(X, U):\n",
    "    U_inv = pd.DataFrame(np.linalg.pinv(U.values))\n",
    "\n",
    "    fm = []\n",
    "    for var in range(0, len(X.iloc[0])):\n",
    "        z_vars = stats.zscore(X.iloc[:, var])\n",
    "        Xm = np.dot(U_inv, z_vars)\n",
    "        Db = np.dot(np.dot(Xm.transpose(), U.transpose()), np.dot(U, Xm))\n",
    "        Dw = np.dot((z_vars - (np.dot(U, Xm))).transpose(), (z_vars - np.dot(U, Xm)))\n",
    "        fm.append( (Db/(len(U.iloc[0]) - 1)) / (Dw/(len(X) - len(U.iloc[0]))))\n",
    "\n",
    "    return fm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[37.535671756790244, 48.61906150085627, 10.975018843280521, 23.722095350076142, 23.189718437470326, 165.63064107501847, 274.4662471518977, 43.40307680974492, 49.52719042101561, 50.805800010795785, 112.06917479642435, 192.6395204288757, 83.2533169337406]\n"
     ]
    }
   ],
   "source": [
    "f_measure_variables = f_variables(data, cluster[0])\n",
    "print(f_measure_variables)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the variables with the highest F-measure are those corresponding to positions 5, 6, 11."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEJCAYAAACE39xMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3de3QUVZ4H8G8/qjvpdAMi4aEyuCIKOgzgiJARiC8i5rFB4Ggii8soC8wKCGf2RJKNZocRjO6sGVl0Zz2L7BGcGQIKMgi4HEFmMSiKnkQGHygkyiQb20RCOp10VVff/SPSpiHdXd2dpNLF9/NXuurWrV81l19ubtW9ZRJCCBARUdIz6x0AERH1DCZ0IiKDYEInIjIIJnQiIoNgQiciMggmdCIig2BCJyIyCKteJ/7uuzYEAt0/An/55U40NXn6OKKek8zxJ3PsQGf8egvXto3w3TJ+/Whp27ol9EBAhE3o5/cns2SOP5lj7w8ite1k/24Zf//GIRciIoNgQiciMggmdCIig2BCJyIyCCZ0IiKDYEInIjIIJnQiIoPQ7Tn03uQPAD7Fn1AddskKK3/dESUVG3yA0tHtPuVsO2yK3PlBSoEMex9G1jcMmdB9ih/vf9KYUB2Txw2D1W7Ir4fIuJQOeL74qNtdJqcdHo8PAOC8dhIgGS+hsw9KRGQQTOhERAbBhE5EZBBM6EREBsGETkRkEEzoREQGoem5vOeeew5vvvkmTCYT5s2bh5///OcoLi7GsWPHkJqaCgBYtmwZZs6c2avBEhFReFET+tGjR/Huu+9i165d8Pv9yM7ORmZmJo4fP44tW7Zg6NChfREnUa/weDwoKCjA7373O1x11VX46KOP8NRTT6GtrQ3XX389ysvLYbPZ9A6TSJOoQy633HILXn75ZVitVjQ1NUFVVaSkpKC+vh4lJSXIy8vD+vXrEQgE+iJeoh5TXV2NwsJC1NbWAuhM7suXL8eaNWvwxhtvAAC2b9+uY4REsdE05CJJEtavX4+XXnoJs2bNgt/vx9SpU1FWVgaXy4UlS5Zg+/btuO+++zSfONr78dLTXZrrupBo9sLlTIn7eABwOOxIH+yI+/hE4tdbMscei8rKSpSVlaGoqAgA8M4772DixIkYO3YsAKC0tBSqquoZIlFMTEIIzS/Za29vx9KlS5GdnY37778/uH3//v3YuXMnnn/+ec0nbmryhH2/X3q6C253q+a6LtTm65mp/2lxTv1PNH49JXPsQHy/jO644w68/PLL2LNnD7744gsoioJTp07hpptuwurVq2G3J9cUcY/chvYw65kAQKqUAqctrQ8j6hlquwcBnzdimYBfhu+rE1HrSr1mIqRBxhsujpqxvvzyS8iyjHHjxiE1NRVZWVnYs2cPBg0ahLvvvhsAIISA1cp1Tyi5qaqKw4cPY+vWrbjiiivwz//8z3jxxRexfPnymOoJ11npq1+WXuHBh/XHw+6/6Yofo90U+xCp3r/sbUpL2HVazht4zQ1o/X69lgu5nPbgPuGVISvJ1XHR0lmJOoZ+5swZlJaWQpZlyLKMt956C5MnT8a6devQ0tICRVGwdetWPuFCSW/IkCGYMGECRo4cCYvFgnvuuQc1NTV6h0WkWdRudWZmJmpqajB79mxYLBZkZWVh2bJluOyyy1BYWAi/34+srCzk5ub2RbxEvWbatGn493//dzQ0NGDEiBE4ePAgbrzxRr3DItJM0zjJ8uXLL/qzc/78+Zg/f36vBEWkhxEjRmDNmjVYunQpfD4fxo0bh8cee0zvsIg048B3EuALO3rXgQMHgj/fdtttuO222/QLhigBTOhJgC/sICIt2GcjIjIIJnQiIoNgQiciMggmdCIig+BdskuEyWxCmy/6kzKi2QtvhHJ8Woao/2JCv0T4FBXVn7ujlnM5U9DqCb8OCJ+WIeq/+D+TKImoZhk+VY5YJmDmCpGXKiZ0oiTiU+WIC28BwPgrruujaKi/4WgoEZFBMKETERkEEzoRkUEwoRMRGQQTOhGRQTChExEZBBM6XdI8Hg9yc3Nx5syZkO1btmzBggULdIqKKD5M6HTJqq6uRmFhIWpra0O2f/HFF3jxxRf1CYooAUzodMmqrKxEWVkZhg4dGtwmyzKeeOIJrFixQsfIiOKjKaE/99xzyM7ORk5ODjZt2gQAqKqqQl5eHrKyslBRUdGrQRL1hrVr1+Lmm28O2fZv//ZvmDt3LkaOHKlTVETxizr1/+jRo3j33Xexa9cu+P1+ZGdnIyMjAyUlJdi8eTNGjBiBJUuW4NChQ8jMzOyLmIl6xTvvvIOGhgYUFxfjvffei7ueyy93ht2Xnu6Ku14AcLfJcLlSIpaRJEvEMo40G9LT4osj0fi7o7Z7EPB5o5YLmACT0x6xjCRZ4IpQ5vy+VIcNAwf1/LXoLWpCv+WWW/Dyyy/DarWisbERqqri3LlzGDVqVLAXk5eXh3379jGhU1LbvXs3Tp48ifz8fHi9Xnz77bdYuXIlfvvb38ZUT1OTB4GAuGh7eroLbndrQjF6hYzW1vCrYQKA4lIjlvG6ZLi9scfRE/F3x6a0wPPFR1HLDbzmBrR6fBHLmBU1bBmX0x7cJ7wyZKXnr6U3afllqmlxLkmSsH79erz00kuYNWsWvvnmG6Snpwf3Dx06FI2Nsb3EOFIvBkisJyCavXA5I/dionE47Egf7Ij7+J7syfTE9UiSVXMdkcol+r30Z0899VTw5/feew8bNmyIOZkT6UnzaosrVqzAP/zDP2Dp0qWora2FyWQK7hNChHzWIlwvBki8J+D1+SOu6a2pDq8PbjW+ZUh7uifTE9ejKNrqiLYeeiLfS1/ojSEBomQRNaF/+eWXkGUZ48aNQ2pqKrKysrBv3z5YLJZgGbfbHfKkAFEyOXDgwEXbpkyZgilTpugQDVH8oj7lcubMGZSWlkKWZciyjLfeegsFBQU4ffo06urqoKoqdu/ejRkzZvRFvEREFEbUHnpmZiZqamowe/ZsWCwWZGVlIScnB4MHD8by5cvh8/mQmZmJWbNm9UW8REQUhqYx9OXLl2P58uUh2zIyMrBr165eCYqIiGLHV9CFYTKb0Obzx3WsaPbC+/2xdskKK+fjElEfYEIPw6eoqP7cHdexXZ8UmTxuGKx2fs1E1PvYdyQiMggmdCIig2BCJyIyCCZ0IiKDYEInIjIIJnQiIoNgQiciMggmdCIig2BCJyIyCCZ0IiKDYEKnS5rH40Fubi7OnDkDANi6dStyc3ORl5eH4uJiyLKsc4RE2jGh0yWruroahYWFqK2tBQCcPn0aGzduxB//+Efs2rULgUAAv//97/UNkigGTOh0yaqsrERZWVnwbVs2mw1lZWVwOp0wmUy47rrrUF9fr3OURNpxGUC6ZK1duzbk85VXXokrr7wSANDc3IxXXnkl5MXRRP0dEzrRBRobG7Fo0SLMnTs3rveKXn65M+y+RF9i7W6T4XKlRCwjSZaIZRxpNqSnxRdHb7yEWznbDpPTHrWcJFngilIuWpnz+1IdNgwcZLwXijOhE3Xx5ZdfYtGiRViwYAEeeuihuOpoavIgEBAXbU9Pd8Htbk0oPq+Q0draEbGM4lIjlvG6ZLi9scfRE/F3x6bI8Hh8UcuZFRWtUcpFKuNy2oP7hFeGrPT8tfQmLb9MmdCJvufxePDwww9j5cqVmD17tt7hEMVMU0LfsGED9u7dC6DzpdFFRUUoLi7GsWPHkJqaCgBYtmwZZs6c2XuREvWy7du349tvv8WmTZuwadMmAMAdd9yBRx99VOfIiLSJmtCrqqpw+PBh7NixAyaTCYsWLcL+/ftx/PhxbNmyJfiEAFGyOnDgAABg4cKFWLhwob7BECUg6mOL6enpWL16NWw2GyRJwujRo1FfX4/6+nqUlJQgLy8P69evRyAQ6It4iYgojKg99DFjxgR/rq2txd69e/HKK6/g6NGjKCsrg8vlwpIlS7B9+3bcd999mk8c6UkAILG76aLZC5cz8pMA0UiSNaE6zh/rcNiRPtiRUCx9fT2RyvXE9RBR79B8U/TkyZNYsmQJioqKcM011+D5558P7luwYAF27twZU0IP9yQAkPjddK/Pj1ZP5CcBolGU+OtwOVOCx3q9PrhVNaFY+vJ6usbebSw9cD29qTceqyNKFppmih47dgwLFy7EL3/5S9x777347LPP8Oabbwb3CyFgtfKBGSIiPUVN6A0NDXjkkUfwm9/8Bjk5OQA6E/i6devQ0tICRVGwdetWPuFCRKSzqN3qjRs3wufzoby8PLitoKAAixcvRmFhIfx+P7KyspCbm9urgRIRUWRRE3ppaSlKS0u73Td//vweD4iIiOLDge9eZjKb0ObzJ1RHmHvHREQhmNB7mU9RUf25O6E6JlyX3kPREJGRcT10IiKDYEInIjIIJnQiIoNgQiciMggmdCIig2BCJyIyCCZ0IiKDYEKnS5rH40Fubi7OnDkDoPOFLnl5ecjKykJFRYXO0RHFhgmdLlnV1dUoLCxEbW0tAKCjowMlJSV44YUXsGfPHhw/fhyHDh3SN0iiGDCh0yWrsrISZWVlwdco1tTUYNSoURg5ciSsVivy8vKwb98+naMk0o5T/+mStXbt2pDP33zzDdLTf1hmYejQoWhsbIy53khv40r0BRzuNhkuV+Q3T0mSJWIZR5oN6Wnh42j1ymjvuHj9oW+avUhNs8PlsGkPWAPlbDtMTnvUcpJkgStKuWhlzu9LddgwcJDxXobChE70vUAgAJPJFPwshAj5rFW4t3El+iYuAPAKGa2tkd88pbjUiGW8Lhlub/g42nx+vP/Jxb/IXM4UjB05EB1tPu0Ba2BTZHg80es0Kypao5SLVMbltAf3Ca8MWUns36KvaekMcMiF6HvDhw+H2/3DQmputzs4HEOUDJjQib43YcIEnD59GnV1dVBVFbt378aMGTP0DotIMw65EH3PbrejvLwcy5cvh8/nQ2ZmJmbNmqV3WESaMaHTJe/AgQPBnzMyMrBr1y4doyGKH4dciIgMQlNC37BhA3JycpCTk4NnnnkGAGfUERH1N1ETelVVFQ4fPowdO3Zg586d+Mtf/oLdu3dzRh0RUT8TNaGnp6dj9erVsNlskCQJo0ePRm1tLWfUERH1M1Fvio4ZMyb4c21tLfbu3Yu/+7u/S3hGXaTZdEBiM+pEsxcuZ+TZdNFIkjWhOs4fm2g9etQRqZzDYUf6YEdCsRBR79D8lMvJkyexZMkSFBUVwWKxBBc0AuKbURduNh2Q+Iw6r8+PVk/k2XTRKEr8dbicKcFjE6mnJ2KJtY6usXfH6/XBraoJxdKbEp1aT5TMNN0UPXbsGBYuXIhf/vKXuPfeezmjjoioH4raQ29oaMAjjzyCiooKZGRkAAidUXfVVVdh9+7dmDt3bq8HS0TUEyxmwKa0RC8opUBG9IXD+ouoCX3jxo3w+XwoLy8PbisoKOCMOiJKXn4fPKdORC3mvHYSIBkooZeWlqK0tLTbfZxRR0TUf3CmKBGRQTChExEZBBM6EZFBMKETERkEEzoRkUEwoRNd4PXXXw+uLvr000/rHQ6RZkzoRF20t7dj7dq12Lx5M15//XV88MEHqKqq0jssIk2Y0Im6UFUVgUAA7e3t8Pv98Pv9sNuTZ2IJXdr4CjqiLpxOJx599FHcc889SE1NxeTJk3HTTTfpHRaRJkzoRF18+umnePXVV3Hw4EG4XC780z/9EzZu3IhFixZpriPS0tCJrgbpbpPhckVeBlmSLBHLONJsSE8LH0ek5adjWT5Zbfcg4PNGLCP7VQRMAcAculqrZLUgxWYJ3SZZ4HJG/mspWpnz+7TUBQCpDhsGDkqeFTyZ0Im6OHz4MDIyMnD55ZcDAObMmYPf//73MSX0cEtDJ7osNAB4hYzW1sjLICsuNWIZr0uG2xs+jnDLT7ucKTEtn2xTWuD54qOIZWS/CtuV1+PzuuaQ7VePGABFDk3oZkVFq8cXsb5IZVxOe3CflroAQHhlyEpi/2Y9RUtngGPoRF2MHTsWVVVV8Hq9EELgwIEDGD9+vN5hEWnCHjpRF9OmTcOJEycwZ84cSJKE8ePHY/HixXqHRaQJEzrRBRYvXswkTkmJQy5ERAbBhE5EZBBM6EREBsGETkRkEEzoREQGoTmhezwe5Obm4syZMwCA4uJiZGVlIT8/H/n5+di/f3+vBUlERNFpemyxuroapaWlqK2tDW47fvw4tmzZgqFDh/ZWbEREFANNCb2yshJlZWUoKioC0LnEaH19PUpKStDY2IiZM2di2bJlMJs5gkPU35ksgFf1hN2vWAIYM9qBc97Q6f82WwCKxQuv6Px/brfYYAnYejVWio2mhL527dqQz99++y2mTp2KsrIyuFwuLFmyBNu3b8d9992n+cSRFjACElvEKNLiQlpJkjWhOs4fm2g9etQRqVwsizNR/ySrMj6u/zzsfp8/gDFDrsGR09Uh2+0pEoZflgq7tTOh33TFj+EwMaH3J3HNFB05ciSef/754OcFCxZg586dMSX0cAsYAYkvYhRucaFYKEr8dbicKcFjE6mnJ2KJtY6usXcnlsWZ9JDoaoZEySyuMZLPPvsMb775ZvCzEAJWK1cRICLSU1wJXQiBdevWoaWlBYqiYOvWrZg5c2ZPx0ZERDGIq1s9duxYLF68GIWFhfD7/cjKykJubm5Px0ZERDGIKaEfOHAg+PP8+fMxf/78Hg+IiIjiw+cMiYgMggmdiMggmNCJLnDgwAHMmTMH99xzD5588km9wyHSjAmdqIuvv/4aZWVleOGFF7Br1y6cOHEChw4d0jssIk348DhRF/v370d2djaGDx8OAKioqIDdbtc5KiJtmNCJuqirq4MkSVi6dCkaGhpw2223YeXKlTHVEWlZi0RnsrrbZLhckZdwkCRLxDLR9os2GRaLGfYU6aJ9NpsVrrTO6f6ONBvS0zqvp9Uro73DH1qPagbMJkhWC1Jslm7P1eqVYbGYkGIPPZfNZoXLEbqsgCRZ4HJG/uUarcz5fVrqAoBUhw0DByXP7GMmdKIuVFXFBx98gM2bN8PhcOAXv/gFduzYgTlz5miuI9yyFokuaQEAXiGjtTXyEg6KS41YJtp+2R+Aqgbg61BCtttTJMiyH62BQGcsLhlub+f1tPn8eP+TxpDy1w8RaKprxtUjBkCRu0/osl+FTRXo8IWeq/M8od+hWVHR6vGFjTtaGZfTHtynpS4AEF4ZspLYv1lP0dIZ4Bg6URdDhgxBRkYGBg8ejJSUFNx1112oqanROywiTZjQibq4/fbbcfjwYZw7dw6qquJ///d/ceONN+odFpEmHHIh6mLChAlYtGgRHnjgASiKgltvvRVz587VOywiTZjQiS4wb948zJs3T+8wiGLGhE4xMZlNaPP5oxeMwC5ZYeVgH1GPY0KnmPgUFdWfuxOqY/K4YbDa2fSIehr7SUREBsGETkRkEEzoREQGwYRORGQQvDNFRBSGxQzYlJbIhaQUyOgfC7gxoRMRheP3wXPqRMQizmsnAVL/SOiahlw8Hg9yc3Nx5swZAEBVVRXy8vKQlZWFioqKXg2QiIi0iZrQq6urUVhYiNraWgBAR0cHSkpK8MILL2DPnj04fvw4XwBARNQPRE3olZWVKCsrw9ChQwEANTU1GDVqFEaOHAmr1Yq8vDzs27ev1wMlIqLIoo6hr127NuTzN998g/T09ODnoUOHorGx8cLDoor0EgAgsRcBiGYvXM7ILwGIRpKsCdVx/thE69GjjkjleiIWh8OO9MGOhOogoovFfFM0EAjAZDIFPwshQj5rFe4lAEDiLwLw+vxo9UR+CUA0ihJ/HS5nSvDYROrpiVhiraNr7L0Vi9frg1tVE6ojnETfCESUzGJ+Dn348OFwu39Yy8PtdgeHY4iISD8xJ/QJEybg9OnTqKurg6qq2L17N2bMmNEbsRERUQxiHnKx2+0oLy/H8uXL4fP5kJmZiVmzZvVGbES6evrpp/Hdd9+hvLxc71CINNGc0A8cOBD8OSMjA7t27eqVgIj6gyNHjmDHjh247bbb9A6FSDOu5UJ0gbNnz6KiogJLly7VOxSimDChE13giSeewKpVqzBgwAC9QyGKCddyIepi27ZtGDFiBDIyMvDaa6/FVUekORaJPlbpbpPhckWeByBJlmCZDlmFrIQ+IipMgGSXkGKzdHu8aJNhsZhhT5Eu2me1WiDMnY8pC7MJwtxZh0W6eP6CzaYgxS7BZrPC5bB1e65WrwyLxYQUu3TBsRcfI0kWuJyR10yJVub8Pi11aS2X6rBh4KD+8bgsEzpRF3v27IHb7UZ+fj5aWlrg9Xqxbt06lJSUaK4j3ByLROdXAIBXyGhtjTwPQHGpwTI+fwB1DedC9o8aIMPT5oPi6/4PdNkfgKoG4OtQQrbbUyT4ZD/ONHZewxDzOTQ2nAUATLgu/aL5CXKKQIdPgSz70RpmzonsV2FTO8uFbO/mGLOiotXji3TpEcu4nPbgPi11aS0nvDJkJbF/Vy20dAaY0Im62LRpU/Dn1157DUePHo0pmRPpiWPoREQGwR46URhz5szBnDlz9A6DSDP20ImIDIIJnYjIIJjQiYgMggmdiMggmNCJiAyCCZ2IyCCY0ImIDILPoRP1AdUsw93WBK+Qw5aRJAsUJfKr+QLm3nl1XzwGOCVgROeU/YDUjmEjQvuH9tTv14oxmSD7u49bdL8iAMWJCZ2oD/hUGSf/74uI67CMv+I6fFz/ecR6xl9xXU+HFjcVfhw5XQ0AuMrrCq7xcl7uDRMAAH41cNG+864a1j8WtTIKDrkQERlEv+uh+wPAN81eeH3+uOsIs7AbEZGh9buE7lP8+PRU00VLccZiwnXpPRgREVFySCihL1iwAM3NzbBaO6tZs2YNJkyY0COBERFRbOJO6EII1NbW4uDBg8GETkRE+on7puipU6cAAA899BD+9m//Flu2bOmxoIiIKHZxd63PnTuHjIwMPP7441AUBQ8++CD+5m/+Brfeequm48O9d1E0ewFc/H7CWEiSNaHje6KO88f2h1hirSNSuZ6IxeGwI32wI6E6etOGDRuwd+9eAEBmZiaKiop0johIm7gT+qRJkzBp0qTg53nz5uHQoUOaE3q49y6ef7olkZuiiuJP6PhE63A5U4LH6h1LrHV0jb23YvF6fXCrvTNBJtGXMFdVVeHw4cPYsWMHTCYTFi1ahP3792PmzJk9FCFR74l7yOWDDz7AkSNHgp+FEBxLp6SXnp6O1atXw2azQZIkjB49GvX19XqHRaRJ3Am9tbUVzzzzDHw+HzweD3bs2MFeDCW9MWPGYOLEiQCA2tpa7N27F5mZmTpHRaRN3F3q22+/HdXV1Zg9ezYCgQAeeOCBkCEYomR28uRJLFmyBEVFRbj66qtjOra7+0PuNhloBVyuSPcnLBH3hyvTIauQu6wBI0yAMHf21cwWwJ4ihZS3WMywWi0QZtNF9VuFghSTjDQbcOWgC/t7KhxmGVcOMsMmWUPKnN/eVZrThkHXXw2RKmHQoMu7vx67BIvFhBR7aIw2mxUuhw0AIA8cjA4ItNpMEFcOh18VIcO1NgGYmtzfX68JLqe923MBCO6TJEvEcsH4NJRLSbHAYW6PWhfMFiAQfajRbHfAktr9PcZoEhojWblyJVauXJlIFUT9zrFjx7BixQqUlJQgJycn5uO7uz90flGuSGu5KC414v5wZXz+AOoazgU/jxog42RdM4DOtVJ8HUpIeVUNwCf7u11fZZgTkNr+D7LrKrR8fSpkn9VqgckhoeVcB4aOuhqypzVYxjQgBS3nQuOSR/8EVZ+/h0EDUnD2XPfXdddPboWkCnT4QmOUZT9av/8Ofal+HKs7htQhI9D+bQNUNYCmlh/qmzJ6Mlq/v96fjgqg1ePr9lwupz24z6yoYct1paWc2etFy6kTUesaeM0Nmso5r50E2XPx/UUt94c46E3URUNDAx555BFUVFQgIyND73CIYsKETtTFxo0b4fP5UF5eHtxWUFCAwsJCHaMi0oYJnaiL0tJSlJaW6h0GUVy4fC4RkUEwoRMRGQSHXKjPmcwmtCWw3j0A2CUrrOyOEIVgQqc+51NUVH/uTqiOyeOGwWpn8yXqin0cIiKDYEInIjIIJnQiIoPgICQR6cbqdMFntsB17Y9CtvuddvjROf3dlOKAqgYgROeyBXwHfHhM6ESkGwUBfPx1Nb6pqw3Z3nX9l+k334mmlg5YBqloaunAoAGJvWDFyDjkQkRkEEzoREQGwYRORGQQTOhERAbBhE5EZBBM6EREBsGETkRkEAkl9D/96U/Izs5GVlYWXnnllZ6KiUhXbNeUrOKeWNTY2IiKigq89tprsNlsKCgowJQpU3Dttdf2ZHxEfYrtmpJZ3Am9qqoKU6dOxaBBgwAAd999N/bt24dly5ZpOt5sNnUfkMWMVLsVql+KNzRYLWY4UuI/PtE6usavdyyx1hHtu+9P1xOuDSUi0XYNdN+2LcKCFKsdqi38cVazBQ5b5FmQ3ZWxmgMYkPrD+vI2qxUDUh0AgDRbKgakqiHlbVZrt9sBwJkCWAMuSFYbXGkDQq/BYkFaqhWqaoPD7ggpk5Zqh3rBxVktVrjSBnS77zzJKsFhd1x0rq7HnD/P+XIX1mezpSAlLQ0AYLJYYbGndnsuk80Gi90ctVzIMRrK9WRdneUscbdtkxAirqUR/vM//xNerxerVq0CAGzbtg01NTX49a9/HVcgRP0B2zUls7jH0AOBAEymH36LCCFCPhMlI7ZrSmZxJ/Thw4fD7f7hrTNutxtDhw7tkaCI9MJ2Tcks7oT+s5/9DEeOHEFzczPa29vxP//zP5gxY0ZPxkbU59iuKZnFfVN02LBhWLVqFR588EEoioJ58+bhJz/5SU/GRtTn2K4pmcV9U5SIiPoXzhQlIjIIJnQiIoNgQiciMggmdCIig+h3Cd3j8SA3NxdnzpzRO5SYbdiwATk5OcjJycEzzzyjdzgxe+6555CdnY2cnBxs2rRJ73Di8vTTT2P16tV6h9GtZG3bbNf9g5a23a8SenV1NQoLC1FbW6t3KDGrqqrC4cOHsWPHDuzcuRN/+ctfsH//fr3D0uzo0aN494PTyTwAAAc1SURBVN13sWvXLrz66qvYvHkzTp06pXdYMTly5Ah27NihdxjdSta2zXbdP2ht2/0qoVdWVqKsrCwpZ+alp6dj9erVsNlskCQJo0ePRn19vd5haXbLLbfg5ZdfhtVqRVNTE1RVhcPh0Dsszc6ePYuKigosXbpU71C6laxtm+1af7G07bgnFvWGtWvX6h1C3MaMGRP8uba2Fnv37sUf/vAHHSOKnSRJWL9+PV566SXMmjULw4YN0zskzZ544gmsWrUKDQ0NeofSrWRt22zX+oulbferHroRnDx5Eg899BCKiopw9dVX6x1OzFasWIEjR46goaEBlZWVeoejybZt2zBixAhkZGToHYphsV3rI9a23a966Mnu2LFjWLFiBUpKSpCTk6N3ODH58ssvIcsyxo0bh9TUVGRlZeGzzz7TOyxN9uzZA7fbjfz8fLS0tMDr9WLdunUoKSnROzRDYLvWT8xtW/RDt99+u/j666/1DiMm9fX1YsqUKaKqqkrvUOLy9ttvizlz5gifzyd8Pp/4+c9/Lnbv3q13WDF79dVXxWOPPaZ3GGElW9tmu+4/tLRt9tB7yMaNG+Hz+VBeXh7cVlBQgMLCQh2j0i4zMxM1NTWYPXs2LBYLsrKykq43Rj2P7Tq5cHEuIiKD4E1RIiKDYEInIjIIJnQiIoNgQiciMggmdCIig2BCv8CTTz6J/Px85Ofn48c//jHuvvvu4OeOjo5uj2lpacHChQuj1r1t2zb84z/+Y9yxVVRU9NkU8hkzZuCTTz7pk3NR72O77mT0ds3n0C9QWloa/PmOO+7Ab37zG4wfPz7iMWfPnsXx48d7OzSiuLFdXxqY0GNw9OhR/Ou//it8Ph8kScKqVaswbdo0FBcXo62tDfn5+Xj99ddRWVmJbdu2QVEUtLS0YOnSpbj//vs1naOurg4PP/wwpk6dio8//hgAUFZWhptuuglA51TmBQsWwO12Iz09HRUVFRgyZAgaGhqwZs0aNDY2QlEU5OXlYfHixairq8PixYuRkZGBjz/+GK2trXjsscdw5513QpZlPPXUUzh69CjMZjMmTpyI1atXIy0tLRiPx+NBcXExvvrqK5jNZowfPx6/+tWvYDKZev4LJl2wXRuoXffJnNUkdfvtt4uamhohhBBNTU0iIyMj+PnTTz8Vt9xyi/jrX/8qamtrxU9/+lMhhBCtra3ivvvuE999950QQoj3339f3HzzzUIIISorK8UvfvGLiOesra0V1113nXjjjTeEEEK89dZbYvr06cLv94tnn31W3HXXXaK5uVkIIcTixYvF7373OyGEEA888IB4++23hRBCtLe3i/nz54s333wzWN+hQ4eEEEK88cYb4s477xRCCPHss8+KRx99VCiKIvx+vygqKhK/+tWvhBBCTJ8+XZw4cUJs375dLF68WAghhKIoori4WHz11VeJfrWkI7Zr47ZrjqFr9NFHH+Gaa64J/pl6/fXXY8KECTh69GhIOafTif/4j//AwYMHUVFRgRdffBFerzemcw0ePBjZ2dkAOv88DgQCOHnyJABg2rRpuOyyywAAY8eORVNTEzweDz788EM8++yzyM/Px/3334/GxsbgWKHNZsP06dMBADfccANaWloAAH/+859RUFAAq9UKi8WC+fPn489//nNILJMnT8Ynn3yCBx98EP/1X/+Fhx9+GCNHjozpeqj/Yrs2VrvmkItGgUDgoj/HAoEA/H5/yLa//vWveOCBB1BQUICbb74ZWVlZOHz4cEznslgsF53HbO783Wu1hv6TCSGgqiqAzptTNpsNANDc3IyUlBS43W7Y7fZg7CaTCeL71R5UVQ25JiHERdfzox/9CPv378d7772Hd999F3//93+PtWvXIjMzM6Zrov6J7dpY7Zo9dI0mTZqEzz//PDj+99lnn+HDDz/ElClTYLFYoKoqhBD4+OOPkZ6ejqVLl2LatGk4ePAgAoFATOdyu9145513AAD79++Hw+HAtddeG7b8wIEDceONN+K///u/AXQ+nXD//ffj7bffjnie6dOn4w9/+AP8fj8CgQBeeeUV/OxnPwsps3nzZjz++OOYPn06ioqKMHXqVJw4cSKm66H+i+3aWO2aPXSNhgwZgt/+9rf4l3/5F8iyDLPZjGeeeQYjR46EoigYN24ccnNz8cc//hGvvfYaZs2aBZPJhClTpmDgwIH46quvNJ/L4XDg1VdfxdNPP43U1FRs2LAh2JMJp6KiAmvWrMGf/vQnyLKM2bNnIzs7G3V1dWGPWbZsGcrLy5Gfnw+/34+JEyeiuLg4pMy9996L999/Hzk5OUhJScGVV16J+fPna74W6t/Yro3VrrnaYj9TV1eHuXPn4oMPPtA7FKIew3bdN9hD18GTTz6J999/v9t9jz/+eB9HQ9Qz2K71xx46EZFB8KYoEZFBMKETERkEEzoRkUEwoRMRGQQTOhGRQTChExEZxP8DMFSDfIngbPIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAELCAYAAADECQ0AAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3df3AU9f0/8Of9Ti65iPC5AJZ8ESNWHDXiVEu0JW3nmyNyOYWKTFLEqqPEDgkj1WZCjOLomImUmRQK1E6HaatQC2ZIjJREqek3NpP4jaT9hsY6iMilZpKGMxGS42Bv78f3D/TIj7vcXXKXvXfyfMw4k7vdfee16/Jkee/u+63y+/1+EBGRsNRKF0BERFPDICciEhyDnIhIcAxyIiLBMciJiATHICciEpxW6QKIlLJnzx40NDQAAHJyclBaWopt27aho6MDycnJAIDi4mLk5uYqWSZRWAxympVaW1vR0tKC2tpaqFQqPPHEEzh+/Di6urpw4MABpKenK10iUcTYtUKzktlsRllZGfR6PXQ6HTIzM9Hb24ve3l6Ul5fDZrNh9+7d8Pl8SpdKFJZiV+RffXURPt/Me6l03rxUDAw4lS5jWiTivqrVKlx7bUrY9ZYuXRr42W63o6GhAQcPHkR7ezu2b98Ok8mEoqIi1NTUYP369VHVwHNbbIm6nxOd2yqlXtEfGHDOyJPdbDbB4RhWuoxpkYj7qlarMG9easTrnz59GkVFRSgpKcHatWtHLTt+/Djq6uqwd+/eWJdJFFPsI6dZq6OjA1u2bEF5eTmsVitOnToFu92OVatWAQD8fj+02uj/iPAiRWyJup8TXaSwj5xmpb6+PmzevBk7d+6E1WoFcCW4KysrceHCBciyjEOHDvGJFRICr8hpVtq/fz8kSUJVVVXgu4KCAmzatAmFhYXweDywWCzIz89XsEqiyLCPPMYS9Z9l8ZCI+xptH3k88NwWW6LuJ7tWiIhmMAY5EZHgGORERIJLuJudHh8gyZ4ptWHQaaFNkL+iYrE/Oq0WsmfmHBOisfSQAPlyZCvrkuCGIb4FCSbhglySPfjok/4ptXHXsvnQGhJj12KxP1k3mdH5qWNKbSTSMSEaR74M52f/jGjV1BuXAzoG+Ui8RiMiEhyDnIhIcAxyIiLBMciJiATHICciEhyDnIhIcAxyIiLBMciJiATHICciEhyDnIhIcAxyIiLBRTT4xq5du/Duu+9CpVJh3bp1eOyxx7Bt2zZ0dHQgOTkZAFBcXMxpsYiIFBA2yNvb2/Hhhx+ivr4eHo8Hq1evRk5ODrq6unDgwAGkp6dPR51ERBRC2K6Vu+++G6+//jq0Wi0GBgbg9XqRlJSE3t5elJeXw2azYffu3fD5fNNRLxERjRFRH7lOp8Pu3bthtVqRnZ0Nj8eDFStWoLKyEocPH8aJEydQU1MT71qJiCiIiAeo3rJlC5588kk89dRTaGtrw969ewPLNm7ciLq6Oqxfvz7iXxxqElH/oAum1KSI2wnGaDTAPNc4pTamwmw2BX6Oxf7odNqEPSYj95WIlBE2yM+cOQO3241ly5YhOTkZFosFx44dw5w5c7Bq1SoAgN/vh1Yb3aQFoWYad0keDDsjnCkkBJdLgsPrnVIbkzV2Bu5Y7I8sJ+YxScTZxieaaZxopgrbtdLT04OKigq43W643W68//77uOuuu1BZWYkLFy5AlmUcOnSIT6wQESkk7GV0Tk4OTp48iTVr1kCj0cBisaC4uBjXXnstCgsL4fF4YLFYkJ+fPx31EhHRGBH1h5SUlKCkpGTUdxs2bMCGDRviUhQREUWOb3YSEQmOQU5EJDgGORGR4BjkRESCi+7hbyKKOa/aDcnrDrrMoNFD49NPc0WxoYcEyJG9/6CBMu99zBQMciKFSV43/tHbFXTZndfdCqNKzCCHfBnOz/4Z0arX3HBLnIuZ2di1QkQkOAY5EZHgGORERIJjkNOstWfPHlitVlitVuzYsQMA0NraCpvNBovFgurqaoUrJIoMg5xmpdbWVrS0tKC2thZ1dXX4+OOPcfToUZSXl2Pfvn04duwYurq60NzcrHSpRGExyGlWMpvNKCsrg16vh06nQ2ZmJux2OxYvXoyMjAxotVrYbDY0NjYqXSpRWAxympWWLl2KO+64AwBgt9vR0NAAlUoFs9kcWCc9PR39/f1KlUgUMT5HTrPa6dOnUVRUhNLSUmg0Gtjt9sAyv98PlUoVdZvRTmzhuOiGyRR8Bihjih7mlMSZhSmaGaHk85egSjVEtK5Op4EpwnWTjXpcMye+x0S0ma8Y5DRrdXR0YMuWLSgvL4fVakV7ezscDkdgucPhQHp6etTthpr9KhSX343h4eBvQLpMbjhciTELU7QzQullN5xOKaJ11bIXwxGu63e54Zbjd0wSceYrYOLZr9i1QrNSX18fNm/ejJ07d8JqtQIAsrKycPbsWXR3d8Pr9eLo0aNYuXKlwpUShccrcpqV9u/fD0mSUFVVFfiuoKAAVVVVKCkpgSRJyMnJQV5enoJVEkWGQU6zUkVFBSoqKoIuq6+vn+ZqiKYmoq6VXbt2YfXq1bBarfj9738PgC9OEBElirBX5O3t7fjwww9RX18Pj8eD1atXIzs7G+Xl5XjjjTewcOFCFBUVobm5GTk5OdNRMxERjRD2ivzuu+/G66+/Dq1Wi4GBAXi9XgwNDfHFCSKiBBFR14pOp8Pu3bthtVqRnZ2Nc+fO8cUJIqIEEfHNzi1btuDJJ5/EU089BbvdPupFicm8OBHqeUj/oAum1OAvR0TKaDTAPNc4pTamYuTLBLHYH51Om7DHRLQXJ4hmorBBfubMGbjdbixbtgzJycmwWCxobGyERqMJrDOZFydCvTThkjwYdkY2PVQoLpcEh1eZqaPGvkwQi/2R5cQ8Jon44sREL00QzVRhu1Z6enpQUVEBt9sNt9uN999/HwUFBXxxgogoQYS9Is/JycHJkyexZs0aaDQaWCwWWK1WzJ07ly9OEBElgIj6yEtKSlBSUjLqu+zsbL44QUSUADjWChGR4BjkRESCY5ATEQmOQU5EJDgGORGR4BjkRESC43jkIXh8gCR7ot7OP+iCS7q6XRQzfhERTQqDPARJ9uCjT6IfCMyUmjTqdfqsm8wTrE1ENHXsWiEiEhyDnIhIcAxyIiLBMciJiATHICciEhyDnIhIcAxyIiLBMciJiATHICciEhyDnIhIcAxyIiLBRTTWyp49e9DQ0ADgymTMpaWl2LZtGzo6OpCcnAwAKC4uRm5ubvwqJSKioMIGeWtrK1paWlBbWwuVSoUnnngCx48fR1dXFw4cOID09PTpqJOIiEII27ViNptRVlYGvV4PnU6HzMxM9Pb2ore3F+Xl5bDZbNi9ezd8Pt901EtERGOEvSJfunRp4Ge73Y6GhgYcPHgQ7e3t2L59O0wmE4qKilBTU4P169dH/IvnzUsN+r1/0AVTalLE7QRjNBpgnmucUhtTqWPkdjqddsr7E4s2YnFMgjGbTTFvk4iiE/F45KdPn0ZRURFKS0txww03YO/evYFlGzduRF1dXVRBPjDghC/IrAsuyTNqPO/JcLkkOLzeqbUxyTrGjkcuy1Pfn1i0EYtjMpbZbILDMRzTNqdKrVaFvEggmqkiemqlo6MDjz76KJ555hmsXbsWp06dwrvvvhtY7vf7odVyjgoiIiWEDfK+vj5s3rwZO3fuhNVqBXAluCsrK3HhwgXIsoxDhw7xiRUSktPpRH5+Pnp6egAA27Ztg8ViwQMPPIAHHngAx48fV7hCovDCXkbv378fkiShqqoq8F1BQQE2bdqEwsJCeDweWCwW5Ofnx7VQoljr7OxERUUF7HZ74Ds+jUUiChvkFRUVqKioCLpsw4YNMS+IaLocPnwY27dvR2lpKQDg0qVLgaex+vv7kZubi+LiYqjVfG+OEhvPUJq1XnnlFXznO98JfP7yyy+xYsUKVFZW4vDhwzhx4gRqamoUrJAoMrxDSfS1jIyMKT+NBYR+tDYUx0U3TKbgj5caU/QwpyTOI57RPG4qn78EVaohonV1Og1MEa6bbNTjmjnxPSaiPVbLICf62qlTp2C327Fq1SoAk38aK9SjtaG4/G4MDwd/vNRlcsPhSoxHPKN93FQvu+F0ShGtq5a9GI5wXb/LDbccv2OSiI/VAhM/WsuuFaKv8WksEhWvyIm+dvPNN/NpLBISg5xmvaampsDPGzZs4NNYJBx2rRARCY5BTkQkOAY5EZHgGORERIKbkTc7VWoVLkqeKbURxWPARDSNNGpAL1+IbGVdEtyI7EUjkc3IIJdkLzo/dUypjaybzDGqhohiyiPB+fm/I1o19cblgG7mBzm7VoiIBMcgJyISHIOciEhwDHIiIsExyImIBMcgJyISXERBvmfPHlitVlitVuzYsQMA0NraCpvNBovFgurq6rgWSUREoYUN8tbWVrS0tKC2thZ1dXX4+OOPcfToUZSXl2Pfvn04duwYurq60NzcPB31EhHRGGGD3Gw2o6ysDHq9HjqdDpmZmbDb7Vi8eDEyMjKg1Wphs9nQ2Ng4HfUSEdEYYd/sXLp0aeBnu92OhoYGPPzwwzCbr775mJ6ejv7+/qh+cagpi/yDLphSg89fGCmdTqtoGyO3U7qWbxiNBpjnGqfURjCizW1INBNF/Ir+6dOnUVRUhNLSUmg0Gtjt9sAyv98PlUoV1S8ONa+hS/Jg2Bl8/sJIybJybZhSk0Ztp2QtI7lcEhxe75TaGCsR5zacaF5DopkqopudHR0dePTRR/HMM89g7dq1WLBgARyOq2OZOBwOpKenx61IIiIKLWyQ9/X1YfPmzdi5cyesVisAICsrC2fPnkV3dze8Xi+OHj2KlStXxr1YIiIaL2zXyv79+yFJEqqqqgLfFRQUoKqqCiUlJZAkCTk5OcjLy4troUREFFzYIK+oqEBFRUXQZfX19TEviIiIosM3O4mIBMcgJyISHIOciEhwDHIiIsExyImIBMcgJyISHIOciEhwDHIiIsFFPGgWEU2eV+2G5HUHXeZTx3YwM5p9GORE00DyuvGP3q6gy2677qZproZmGnatEBEJjkFORCQ4BjkRkeAY5EREgmOQ06zmdDqRn5+Pnp4eAEBraytsNhssFguqq6sVro4oMgxymrU6OztRWFgYmH/28uXLKC8vx759+3Ds2DF0dXWhublZ2SKJIsAgp1nr8OHD2L59e2C+2ZMnT2Lx4sXIyMiAVquFzWZDY2OjwlUShcfnyGnWeuWVV0Z9PnfuHMxmc+Bzeno6+vv7o2533rzUcd85LrphMiUFXV+n04RcZkzRw5xiirqGiQy73Lh02RNyeXKSFiajPuiypBRDyG3HbiefvwRVqiGimnQ6DUxxWDfZqMc1c6I/fmZzbI95vEUc5E6nEwUFBXjttdewaNEibNu2DR0dHUhOTgYAFBcXIzc3N26FEsWbz+eDSqUKfPb7/aM+R2pgwAmfzz/qO5ffjeHhy0HXl03ekMtcJjccruGoa5jIRcmDjz4J/RfUXcvm4/JFadz3ZrMJA4MXQ247dju97IbTOb6dYNSyF8NxWNfvcsMtR3f8zGYTHI7YHvNYUKtVQS8SgAiDvLOzExUVFYG+RADo6urCgQMHAv8sJRLdggUL4HA4Ap8dDgfPbxJCRH3kY/sSL126hN7eXpSXl8Nms2H37t3w+XxxLZQo3rKysnD27Fl0d3fD6/Xi6NGjWLlypdJlEYUV0RX52L7EL7/8EitWrMD27dthMplQVFSEmpoarF+/Pi5FEk0Hg8GAqqoqlJSUQJIk5OTkIC8vT+myiMKa1M3OjIwM7N27N/B548aNqKuriyrIQ/X1+AddMKUGv/ETKZ1Oq2gbI7dTupZvGI0GmOcap9RGMKLdFAqmqakp8HN2djbq6+sVrIYoepMK8lOnTsFut2PVqlUArtwU0mqjayrYDSEAcEkeDDuD3/iJlCwr14YpNWnUdkrWMpLLJcHhje1wqYl4U2iiG0JEM9WkniP3+/2orKzEhQsXIMsyDh06xCdWiIgUMqkr8ptvvhmbNm1CYWEhPB4PLBYL8vPzY10bERFFIKogH9mXuGHDBmzYsCHmBRFR4jJqZHz7f8Z3iQJAqn8YOvnqP/I14MxH04VvdhJRxNSeyxj490dBl5kWpkGv1QQ+X3PDLdNV1qzHsVaIiATHICciEhyDnIhIcAxyIiLBMciJiATHICciEhwfPyRKYCoN4PI6gy4zaPTQ+IJPAEGzC4OcKIG5vW78q/fToMvuvO5WGFUMcmLXChGR8BjkRESCY9fKLKFSq3BRCj3hbqQMOi20/OufKKEwyGcJSfai81NH+BXDuGvZfGgNPG2IEgmvrYiIBMcgJyISHIOciEhw7OwkmmG8ajckrzvoMoNGj3DXb6FujPsHXVDFokCKOQY50Qwjed34R29X0GV3XncrgKSJtw9xY9yUmoTl1zHKE1FEXStOpxP5+fno6ekBALS2tsJms8FisaC6ujquBRIR0cTCBnlnZycKCwtht9sBAJcvX0Z5eTn27duHY8eOoaurC83NzfGuk4iIQggb5IcPH8b27duRnp4OADh58iQWL16MjIwMaLVa2Gw2NDY2xr1QIiIKLmwf+SuvvDLq87lz52A2mwOf09PT0d/fH/vKiIgoIlHf7PT5fFCprt7w8Pv9oz5Hat681KDf+wddMKVOfDMmHJ1Oq2gbI7dTupZYtgEARqMB5rnGwGez2TTlNoloaqIO8gULFsDhuHpH2+FwBLpdojEw4ITP5x/3vUvyYNh5Oer2RpJl5dowpSaN2k7JWmLdBgC4XBIcXi+AKyHucAxPuc1YUqtVIS8SiGaqqF8IysrKwtmzZ9Hd3Q2v14ujR49i5cqV8aiNiIgiEPUVucFgQFVVFUpKSiBJEnJycpCXlxeP2ohoAqFmD/KpvQpUk5g0akAvX4h8A10SAPG6CyMO8qampsDP2dnZqK+vj0tBRBSZULMH3XbdTQpUk6A8Epyf/zvi1VNvXB7HYuKHY60QEQmOQU5EJDgGORGR4DhoFlEQGzduxODgILTaK39EXnrpJWRlZSlcFVFwDHKiMfx+P+x2O/72t78FgpwokbFrhWiMzz//HADw+OOP4/7778eBAwcUrohoYrzcIBpjaGgI2dnZeP755yHLMh555BEsWbIE9957b0TbB3uz1HHRDZMp+BAJOp0mpssm2saYoodfa5hwuIaJhnNQa9RIMuiCLtPrtTAZ9aPrSDWE/D3jalZ4XQBI/rp+0YaeYJATjbF8+XIsX371eeJ169ahubk54iAPNvyEy+/G8HDwIRJkkzemyybaxmVyw+9WTzhcQ6jhHEypSfB5fbgsyUG3c7s9GB6x32rZi2GnFPL3jJQI6wKA3+XGNXOQcENPABMPP8GuFaIxTpw4gba2tsBnv9/PvnJKaAxyojGGh4exY8cOSJIEp9OJ2tpa5ObmKl0WUUi8zCAa44c//CE6OzuxZs0a+Hw+/OQnPxnV1UKUaBjkREE8/fTTePrpp5Uugygi7FohIhIcg5yISHAMciIiwbGPnIhiQ6WC23N1Uguf3x/4rFFr4PUFn/BCq9FMS3kzGYOciGLC4/Whp//qizT6b8mw9w0BABbNN41aNtL1C9Ompb6ZjF0rRESCY5ATEQluSl0rHLOZiEh5kw5yjtlMRJQYJt21wjGbiYgSw6QvpeMxZjMA+AddE46VHImJxlOejjZGbqd0LbFsAwCMRgPMc42Bz6KN20w0E006yOMxZjMAuKTgYyFHI9R4ytPRhik1adR2StYS6zYAwOWS4PBeeR7YbDYl3LjNE43ZPJv4/IDs9Y373u31QTP+jx0JbtJBfuLECciyjOzsbAAcs5kokcheH7q/foZ7pOuSLmJBiNmDSFyT7iPnmM1ERIlh0pfQHLOZiCgxTKkvhGM2ExEpj292EhEJjkFORCQ4BjkRkeAY5EREguOD30SC8gOQPONf+vH6/fCHeOknLVUHn+4S5i8cfw2XZkzCkOty0OUqnxYXL8aiaooHBjmRoHx+f9CXfpb+jwxfiCT3woOOnlNBJ3n437fdgbaznVjkGj8JRPaSLAD6mNRNsceuFSIiwTHIiYgExyAnIhIcg5yISHAMciIiwTHIiYgExyAnIhIcnyMnUlioF3u+WTYZSRov5geZKClZCyRDGrXMkKSFdNmDZC0wP3X8cgBYeK0O3lQZbr0Gphv/V+B7tcePC/Yvgtdu1AfW9aXoYTJdG3w7lQo+vx9uj3dcG1qNBmpVhDudwPSQADnCGbp0SXDDEFX7DHIihYV6sQcAlmdMLspVXg8u9tnHfe/5ViYunfsPLg5dDZWUxddjsM8Oz7cycbHPDl1a0qjlAHDpW5lo+3//BwuvX4wvznwe+P67mXeFrMHt9+L/nvkIADAnLQnnR7Q5cjuP14dhlwx7kGNw/cI06LWaMHsrAPkynJ/9M6JVU29cDuiiC3J2rRARCY5X5BQVlVqFi5IHAOAfdMH19c/R0Gm1kD3RbzeSQaeFlpchRAAY5BQlSfai81MHAMCUmoRhZ4T9fiNk3WQOtDFZdy2bD62Bpy8RMMWulXfeeQerV6+GxWLBwYMHY1UTkaJ4XpNoJn1J09/fj+rqahw5cgR6vR4FBQX47ne/ixtvvDGW9RFNK57XJKJJB3lraytWrFiBOXPmAABWrVqFxsZGFBcXR7S9OsQzRVqNGsYk3WTLUryNZIMWXs/V7UTfn4naGbuv01mLVqMOeg6FOq8iNdXzOlQNGr8GRn1S0PW1ai3Sko0hl0W7nV6rhVGfDFNK2rhlOq0eKckmeL1Xh6Q1GowwpaRBp9XDlJKGlGTDqOXfbJdqTEOy3jiqXb0+CUkpKVfqSTIiKeXqY5RajTaw7tg2x26n0eoCn0ftY5IRGs2YsdE1WmgMyUGPyVjRrHtl/StPyEz1PArWbuQ1a6I+t1V+f6gh6Cf229/+Fi6XC1u3bgUAvPXWWzh58iRefvnlyTRHlBB4XpOIJt1H7vP5oFJd/RvC7/eP+kwkIp7XJKJJB/mCBQvgcFx98sDhcCA9PT0mRREphec1iWjSQX7PPfegra0Ng4ODuHTpEt577z2sXLkylrURTTue1ySiSd/snD9/PrZu3YpHHnkEsixj3bp1uP3222NZG9G043lNIpr0zU4iIkoMfMmZiEhwDHIiIsExyImIBMcgJyISHIM8hvbs2QOr1Qqr1YodO3YoXU7cvfrqqygrK1O6DIoznteJj0EeI62trWhpaUFtbS3q6urw8ccf4/jx40qXFTdtbW2ora1VugyKM57XYmCQx4jZbEZZWRn0ej10Oh0yMzPR29urdFlxcf78eVRXV+Opp55SuhSKM57XYuDI/DGydOnSwM92ux0NDQ148803Fawofl544QVs3boVfX19SpdCccbzWgy8Io+x06dP4/HHH0dpaSmuv/56pcuJubfeegsLFy5Edna20qXQNOJ5ndj4ZmcMdXR0YMuWLSgvL4fValW6nLh47LHH4HA4oNFocOHCBbhcLqxZswbl5eVKl0ZxwvM68THIY6Svrw9r165FdXW1sH+rR+vIkSNob29HVVWV0qVQnPC8FgP7yGNk//79kCRp1P/8goICFBYWKlgV0dTwvBYDr8iJiATHm51ERIJjkBMRCY5BTkQkOAY5EZHgGORERIJjkE/Ct7/9bdhsNjzwwAOB/5577rnAssHBQYUrvOLJJ5/EZ599Nu77xsZGbNy4UYGKKNHx3BYTnyOfpD/+8Y+YO3eu0mVM6He/+53SJZCAeG6Lh0EeJy6XCy+++CK6u7tx/vx5pKSkYOfOnVCpVCgoKMDf//536PV6eL1e/OAHP8Af/vAHDA8P45e//CXcbjccDgfuueceVFZWoqenB48++ihycnLQ2dmJoaEh/OIXv0Bubi5kWUZVVRXa2tqg0Whw++23Y9u2bUhNTcWPfvQj7Nq1C7fddht27dqFd955B3PmzMHixYsDdZ44cQJVVVXw+XwAgKKiIqxatUqpw0YC4LmdeNi1Mkk//elPR/3zc2BgYNTyDz74AGlpaTh06BDeffdd3HrrrTh48CCWLFmCpUuXoqmpCQDQ0tKCRYsWITMzE6+//jq2bNmCt956C3/5y1/Q1NSErq4uAMAXX3yB733ve6ipqcEzzzyDyspKAMBvfvMbnDt3Dm+//Tbefvtt+Hy+cYP///Wvf8V7772Huro6/PnPf4bT6Qws+/Wvf43HHnsMR44cQWVlJT788MN4HjYSAM9t8fCKfJLC/fMzLy8PGRkZeOONN9Dd3Y329nYsX74cALBu3TrU1tYiLy8PR44cwfr16wEAVVVV+OCDD/Daa6/h888/hyRJcLlcmDNnDnQ6HXJycgAAt9xyC86fPw/gyh+qrVu3QqfTAQA2btyIzZs3j6qlra0Nubm5SE1NBQA8+OCDeOONNwAA9913H1566SU0NTXhnnvuwc9//vMYHiUSEc9t8fCKPE7+9Kc/4bnnnkNSUhJsNhvy8/PxzWgI9913Hzo7O3HmzBl89NFHyMvLAwA8/PDDaG5uxg033IDNmzcjPT09sI1Op4NafeV/l0qlCvwen8837rMsy+PqGTkSg0ajCfxcUFCA+vp63HvvvWhpacH9998PSZJieCRopuG5nXgY5HHS0tKCtWvX4qGHHsKSJUvQ1NQEr9cLADAYDLBarSgrK4PFYkFycjKGhobwr3/9C88++ywsFgv++9//4j//+U+gfy+U73//+3jzzTchyzJ8Ph8OHjyIe++9d9Q6K1euRGNjI4aGhuDz+fD2228HlhUUFOCTTz7Bj3/8Y7z88ssYGhqCw+GI/QGhGYPnduJh10qcPP7443jhhRdQU1MDALjjjjvw6aefBpY/9NBDOHDgAF588UUAQFpaGjZt2oS1a9fCaDRi/vz5uPPOO9Hd3Y2MjIyQv+dnP/sZXn31VaxZswYejwe33347nn/++VHr5OTk4NSpU3jwwQeRlpaGm2++GV999RUA4Nlnn0VlZSV+9atfQaVSobi4GIsWLYrx0aCZhOd24uHoh0REgmPXChGR4BjkRESCY5ATEQmOQU5EJDgGORGR4BjkRESCY5ATEezkMksAAAANSURBVAmOQU5EJLj/D2zzTW8OxcIeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEJCAYAAACT/UyFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3df1xUdb4/8Nf8BmTMtAE1vXqjH5oZZrUsaOB2lUQgjNoWt3XXdk1wWyy2TZEI9uEuoj66kV1ve+vqdTdzK3+Ack3dxUw3w7TML2hqKgobF8IJU2ccOGfmzPn+oU6gzA9ghgHO6/nX/Hif93mf4cN7zpyZ8zkqWZZlEBGRYqiDXQAREfUsNn4iIoVh4yciUhg2fiIihWHjJyJSGDZ+IiKFYeMnIlIYbbBW/N13l+F09o1TCIYMCUdzszXYZfhFf9kWd9uhVqtw880DglDR9zi2e15/2Q6gZ8Z20Bq/0yn3mX8OAH2qVm/6y7b01u3g2A6O/rIdQOC3hYd6iIgUho2fiEhh2PiJiBSGjZ+ISGHY+ImIFIaNn4hIYdj4iYgUJmi/4yf/czgBwe7wGCOft8EmeI65xqDTQstdAwoyPQTA3uoxRmpR9VA1/QMbfz8i2B347HiTxxhjeAgsVs//RNc8ODYSWgOHCAWZvRXW04c9hoTdGwMgtGfq6Qe4P0dEpDBs/ERECsPGT0SkMGz8REQKw8ZPRKQwbPxERArDxk9EpDBs/KRoVqsVKSkpqK+vBwAcPnwYTz75JJKTk/Hb3/4WoigGuUIi/2PjJ8WqqqrCrFmzUFtbC+DKm0B2djaWLFmCDz74AACwadOmIFZIFBhs/KRYGzZsQGFhISIiIgAAn3zyCSZMmIAxY8YAAPLz8zFt2rRglkgUEDwfnxSrqKio3f26ujqEhYUhJycHZ86cwcSJE5Gbm9vpvEOGhHe5Jqt4GS1e5qUJ1YUgXO+/C8qbTEa/5QoE+4UWqMINXuN6+3Z0RqC3hY2f6CpJkrBv3z68//77GD58OF566SW89dZbyM7O7lSe5mZrly+WbZOt+KLhqMeYicPvQYvK2aX81zOZjDCbLX7JFSh6uwirVfAYEwr0+u3wlbu/iVqt6tZORbtcfslC1A/ccsstiI6OxsiRI6HRaJCUlITq6upgl0Xkd2z8RFdNnjwZX375JRobGwEAH330EcaNGxfkqoj8j4d6iK4aNmwYlixZgqysLAiCgLFjx2LRokXBLovI79j4SfF2797tuj1lyhRMmTIleMUQ9QAe6iEiUhg2fiIihWHjJyJSGDZ+IiKFYeMnIlIYNn4iIoXxufEvX77cNW/J8ePHkZ6ejkceeQQvvfQSHA5HwAokIiL/8qnx79+/H2VlZa77L774IgoKCvC3v/0Nsixjw4YNASuQiIj8y2vjv3DhAkpKSpCVlQUA+L//+z+0trZiwoQJAID09HTs3LkzsFUSEZHfeD1zt6CgADk5Oa75S86dOweTyeR63mQyoampqdMr9tcscz2lL0z5Kp+3wRge4jXOlxgACAszwDQ4rLtlBUxf+JsQ9UYeG//GjRsxbNgwxMbGorS0FADgdDqhUqlcMbIst7vvq+5MXdvT+sLUtQBgExywWD3P5W4MD/Ea48pnE2CWJH+U5nc9MXUtUX/lsfFv374dZrMZaWlpuHjxImw2G1QqFcxmsyvm22+/dV3BiIiIej+PjX/t2rWu26WlpTh48CCKi4uRkpKCQ4cO4f7778fWrVsRHx8f8EKJiMg/ujQ75yuvvIL8/HxYrVaMGzcOP//5z/1dFxERBYjPjT89PR3p6ekAgDFjxmDTpk0BK4qIiAKHZ+6SolmtVqSkpKC+vr7d4++88w5mz54dpKqIAouNnxSrqqoKs2bNQm1tbbvHT58+jbfeeis4RRH1ADZ+UqwNGzagsLCw3a/SRFFEQUEBFixYEMTKiAKLl14kxSoqKrrhsX//93/H448/jhEjRgShIqKewcZPdNUnn3yCxsZGLF68GAcOHOhynu6cQGa+LMJo9HxmtSFUA6hFr7lCdSEI1w/wGtfbz4C2X2iBKtzgNa63b0dnBHpb2PiJrtq2bRtOnTqFtLQ02Gw2fPvtt3j++efx2muvdSpPd85Kt8kiLBbPZ1ZbjS040nDSa66Jw+9Bi8rpMaYvnJWut4uwWgWPMaFAr98OX/XEWels/ERXFRcXu24fOHAAq1at6nTTJ+oL+OUuEZHCcI+fFG/37t03PBYTE4OYmJggVEMUeNzjJyJSGDZ+IiKFYeMnIlIYNn4iIoVh4yciUhg2fiIihWHjJyJSGP6On4gIgB4CYPcwXYYuBCK8zxnUF7DxExEBgL0V1tOH3T4dfvt9gK5/NH4e6iEiUhg2fiIihWHjJyJSGDZ+IiKFYeMnIlIYNn4iIoVh4ydFs1qtSElJQX19PQDg/fffR0pKClJTU7F48WKIovdr2xL1NWz8pFhVVVWYNWsWamtrAQBnz57FmjVr8N5776G8vBxOpxN//etfg1skUQCw8ZNibdiwAYWFhYiIiAAA6PV6FBYWIjw8HCqVCnfeeScaGhqCXCWR//HMXVKsoqKidvdvvfVW3HrrrQCA8+fPY/369e0uwE7UX7DxE12nqakJc+fOxeOPP96l6+4OGRLe5XWbL4swGkM8xuh0Gq8xABA2QA/TAKPXOJPJe0ww2S+0QBXufaqE7m6Ht/WEhulx06Ceea0C/Tdh4ydqo6amBnPnzsXs2bPxy1/+sks5mputcDrlLi1rk0VYLB4mCgNgN0peYwDAZhRhtlk8xphMRpjNnmOCTW8XYbUKHmNCgW5vh7f1yDYRoj3wr5W7v4larerWTkVbbPxEV1mtVvzqV7/C888/j5kzZwa7HKKA4Ze7RFdt2rQJ3377LdauXYu0tDSkpaVh5cqVwS6LyO+4x0+Kt3v3bgDAnDlzMGfOnOAWQ9QDuMdPRKQwbPxERArDQz3klkqtwmXB4bd8Bp0WWu5qEAWdT41/5cqV+Nvf/gaVSoUnnngCTz/9NCorK1FcXAxBEJCUlIScnJxA10o9TLBLqDpp9lu+B8dGQmvgvgZRsHn9Lzx48CA+/fRTlJeXw+FwYMaMGYiNjUVeXh7WrVuHYcOGITMzE3v37kVCQkJP1ExERN3g9YP3D37wA7z99tvQarVobm6GJEm4dOkSRo0ahZEjR0Kr1SI1NRU7d+7siXqJiKibfPrcrdPp8Prrr+N//ud/MH36dJw7dw4mk8n1fEREBJqamjq1Yn+dgdZTAnEKtcUmoqXVf8fQNTrAGO79VH5fYgBAp9P6HOuLsDADTIPD/Javt081QNRb+XzAdcGCBXjmmWeQlZWF2tpaqFQq13OyLLe774vunNbe0wJ1WvtlwYHPjnfuDdOT6DtNsFg9n8pvDA/xGnON3e7wOdYXNpsAsyT5JVdPnNZO1F95PdRTU1OD48ePAwBCQ0ORmJiIAwcOwGz+/ks/s9nsmtqWiIh6N6+Nv76+Hvn5+RBFEaIo4sMPP0RGRgbOnj2Luro6SJKEbdu2IT4+vifqJSKibvJ6qCchIQHV1dWYOXMmNBoNEhMTkZycjMGDByM7OxuCICAhIQHTp0/viXqJiKibfDrGn52djezs7HaPxcbGory8PCBFERFR4PA8SiIihWHjJyJSGDZ+IiKFYeMnIlIYNn5SNKvVipSUFNTX1wMAKisrkZqaisTERJSUlAS5OqLAYOMnxaqqqsKsWbNQW1sLAGhtbUVeXh7eeOMNbN++HUePHsXevXuDWyRRALDxk2Jt2LABhYWFrrPOq6urOfkgKQInRyfFKioqanffH5MPAt2bgNB8WYTR6HliPJ1O4zUGAMIG6GEa4H0iu94+2Z39QgtU4QavcZ62Q2qxwinYPC7vVMHjekJCNAhTt3jMoTaEQRPa/bmiAv03YeMnusrpdHZ78kGgexMQ2mQRFovnifHsRslrDADYjCLMNs+TCwZqAkJ/0ttFWK2Cx5hQwON26O0XYT192GOOm267GxYP61HbbLh45pjHHOG33wfR2r3JJ3tiAkIe6iG6aujQoZx8kBSBjZ/oqujoaE4+SIrAQz1EVxkMBixbtoyTD1K/x8ZPird7927XbU4+SErAQz1ERArDxk9EpDBs/ERECsPGT0SkML3uy12HExDsDr/lM+i00PLtjYjIpdc1fsHuwGfHO3+avDsPjo2E1tDrNpOIKGi4L0xEpDDcFSbqIZJahCCJHmOcaqmHqukd9BAAu+d5hzTw/prITgl6+8Vu5VASNn6iHiJIIr5oOOoxZvzwO3uoml7C3urT5GneyHYB1tNHupVDSXioh4hIYdj4iYgUho2fiEhh2PiJiBSm33+5q1KrcFno3glh8nkbbFdz8IQwIurr+n3jF+wSqk6avQd6YAwPgcV65SdnPCGMiPo67rsSXWfr1q1ITk5GcnIyli9fHuxyiPyOjZ+ojZaWFhQVFWHdunXYunUrPv/8c1RWVga7LCK/YuMnakOSJDidTrS0tMDhcMDhcMBgMAS7LCK/4sFqojbCw8Px3HPPISkpCaGhoXjwwQcxceLEYJdF5Fds/ERtnDhxAps3b8ZHH30Eo9GI3/3ud1izZg3mzp3rc44hQ8I7fNx8WYTRGOJxWZ1O45cYAAgboIdpgNFrnMnkPSYQLDYR0iU1oFZ5jFNrVDCGe//U5SlGp9N4zeEtxpccoWF63DSo+69noP8mbPxEbezbtw+xsbEYMmQIACA9PR1//etfO9X4m5utcDrlGx63ySIsFs8TktmNkl9iAMBmFGG2WTzGmExGmM2eYwLlsuDAJfNFNNed9xh3/ygnLFbBY4wB8Bijtktec3iL8SWHbBMh2rv3err7m6jVKrc7FZ3FY/xEbYwZMwaVlZWw2WyQZRm7d+/G+PHjg10WkV9xj5+ojcmTJ+PYsWNIT0+HTqfD+PHjMW/evGCXReRXPjX+VatWYceOHQCAhIQELFy4EJWVlSguLoYgCEhKSkJOTk5ACyXqKfPmzWOzp37N66GeyspK7Nu3D2VlZdiyZQu+/PJLbNu2DXl5eXjjjTewfft2HD16FHv37u2JeomIqJu8Nn6TyYTc3Fzo9XrodDpERUWhtrYWo0aNwsiRI6HVapGamoqdO3f2RL1ERNRNXg/13HHHHa7btbW12LFjB372s5/BZDK5Ho+IiEBTU+cukO7u22n5vA3GcO8/VfOVTqf1S75rOcLCDDANDut2PiB42+rrOv312l3jz9cOCN7PEIn6Op+/3D116hQyMzOxcOFCaDQa1NbWup6TZRkqleff4l7P7U/eBIdrQjR/sNu7n6/tJG02mwCz5J/rdwZjW9tuiz/ydYY/X7ue+MkbUX/l0885Dx06hDlz5uCFF17AY489hqFDh8Js/n7GS7PZjIiIiIAVSURE/uO18Tc2NuLZZ5/FK6+8guTkZABAdHQ0zp49i7q6OkiShG3btiE+Pj7gxRIRUfd5PdSzZs0aCIKAZcuWuR7LyMjAsmXLkJ2dDUEQkJCQgOnTpwe0UCIi8g+vjT8/Px/5+fkdPldeXu73goio51jFy7DJVo8xBo0eGqe+hyrqGsnphOhw//2RU5YhOiRo1BpIzo7jrsUAgFaj8TaFUJ/GM3eJFKzF3oovGo56jJk4/B6EqXp74wdqGy+5fV5/qx21jZcwItKI+qaO59K5FgMAo4cNhF6rCUitvQHn6iEiUhg2fiIihWHjJyJSGDZ+IiKFYeMnIlIYNn4iIoVh4yciUhg2fqLr7N69G+np6UhKSsIf//jHYJdD5Hds/ERtfP311ygsLMQbb7yB8vJyHDt2jBcZon6HZ+4StVFRUYEZM2Zg6NChAICSkhIYDIYgV0XkX2z8RG3U1dVBp9MhKysLjY2NmDJlCp5//vlO5XB3PQDzZRFGo+cL2+h0Gr/EAEDYAD1MAzxfrMZ8udlrLl/ydIV83oZWvRYhBp3HOJVaDWO49zdfT3k0GhVCDDpoNGq3cddiAECv18IY1n6aCp1O47WO0DA9bhrU/dcq0BcZYuPvJJVahcuCwy+5OrgODQWZJEn4/PPPsW7dOoSFhWH+/PkoKytDenq6zzncXmRIFmGxeL6wjd0o+SUGAGxGEWZbx/PSuITBay6f8nSBTXBAFB1oFewe42SnExar4DFmYCQ85pEkGa2CHZLkdBt3LQYARNEBy3V/Q7Vd8lqHbBMh2rv3WvXERYbY+DtJsEuoOmn2HuiD6DtN3oOoR91yyy2IjY3F4MGDAQBTp05FdXV1pxo/UW/HL3eJ2vjRj36Effv24dKlS5AkCR9//DHGjRsX7LKI/Ip7/ERtREdHY+7cufjpT38Ku92OSZMm4fHHHw92WUR+xcZPdJ0nnngCTzzxRLDLIAoYHuohIlIYNn4iIoVh4yciUhg2fiIihWHjJyJSGDZ+IiKF4c85ifoplQawSVaPMVpJ5ff1OpyAYPc+rQmnLAkeNn6ifkqURBxpOOkxZuKosX5fr2B34LPjTV7jOGVJ8PBQDxGRwrDxExEpDBs/EZHCsPETESkMGz8RkcKw8RMRKQwbPxGRwrDxE7mxfPly5ObmBrsMIr9j4yfqwP79+1FWVhbsMogCgo2f6DoXLlxASUkJsrKygl0KUUCw8RNdp6CgADk5ORg4cGCwSyEKCJ/n6rFarcjIyMB//dd/YcSIEaisrERxcTEEQUBSUhJycnICWSdRj9i4cSOGDRuG2NhYlJaWdinHkCHhHT5uvizCaAzxuKxOp/FLTGfivMWEDdDDNMAIALDYRLS0ep6ATaMDjOG+1KeFU69FiEHnMU6lUgFqz5PJybLsMY9Go0KIQQeNRu027loMAOj1WhjD9NfVq4Ex3OCxjtAwPW4aZPQY4wuTqfs5PPGp8VdVVSE/Px+1tbUAgNbWVuTl5WHdunUYNmwYMjMzsXfvXiQkJASyVqKA2759O8xmM9LS0nDx4kXYbDYsXboUeXl5PudobrbC2cHUkzZZhMXS6nFZu1HyS4zPcYPhNcZmFGG2WQAAlwXvE7BF32mCxepDfXYHRNGBVsHuMU6SnDhZd95jzMRR8JhHkmS0CnZIktNt3LUYABBFByzX/Q3VdgkWq+CxDtkmQrRbPMZ4YzIZYTbfmEOtVrndqegsnxr/hg0bUFhYiIULFwIAqqurMWrUKIwcORIAkJqaip07d7LxU5+3du1a1+3S0lIcPHiwU02fqC/wqfEXFRW1u3/u3DmYTN9PqRoREYGmJu/TsLbl7p1LPm/z6WOir3Q6rV/yXcvhr3z+ztWZfL6u09/1hYUZYBoc5rd8gf44TNRfdWk+fqfTeeW421WyLLe77wu3H4cFh08fE31lt3c/nzE8xJXDH/mu8WcuX/O13RZ/5OsMm02AWZL8kqsnPg6np6cjPT3dL7mIepMu/apn6NChMJvNrvtmsxkRERF+K4qIiAKnS40/OjoaZ8+eRV1dHSRJwrZt2xAfH+/v2oiIKAC6dKjHYDBg2bJlyM7OhiAISEhIwPTp0/1dGxERBUCnGv/u3btdt2NjY1FeXu73goiIKLB45i4RkcKw8RMRKQwbPxGRwnTpy10iousNjdRDVjvg1LUgctiVfcpBoWpo4HTFaKGBcEkEANwEK0LCNGgOSrWBoVEDevtF9wG6EIjwPN9PT2DjJyK/kNUO7D9bhRE2I+qbrpxcFxkOXG6sdcXERD0Iy+l/AgBCI40IGzkmCJUGkEOA9cwxt0+H334foAt+4+ehHiIihWHjJyJSGDZ+IiKF4TF+6jEqtQqXBc8X8fBViE30Sx4iJWLjpx4j2CVUnTR7D/RBwv3/gs7NB0tE1/BQDxGRwnCPn+g6q1atwo4dOwAACQkJrivPEfUX3OMnaqOyshL79u1DWVkZtmzZgi+//BIVFRXBLovIr7jHT9SGyWRCbm4u9Ho9ACAqKgoNDQ1BrorIv9j4idq44447XLdra2uxY8cOvPvuu0GsiMj/2PiJOnDq1ClkZmZi4cKFGD16dKeWdXfNX/NlEUaj54vX63Qan2NaRQmi3f01jGUVIKvV0Os0CNFr3MZ5zaVV4bLzMgBA0ssYOUrfYVioATCE6KDRqGEI0V1ZVCtBq/1+3RqNGiEGneu2RqNy3XdHpVZ3GBM+Yjgc2iu/7bJoZQy6a/SNpTtkWOsbXOtpu/7rta1Fo9UA6va/G5NVqnaP6bQ3vq46nQbGcPdTMoSEaBCmbnH7PABILSqYTEaPMd3Fxk90nUOHDmHBggXIy8tDcnJyp5dvbrbC6ZRveNwmi7BYPF+83m6UfI4RHE7UNV5yGzdqoIhTdecxathA2AU3X+cNhtdcowZasOvI/wMAjIj8fh6e600dPwFCqx2S5ITQagcAOLSAw/H9G4okOdEq2F23JUl23XdHdjo7jNGpZBw4eRAA8C9yFP55quaGmJioB9Eq2F3rabv+67WtRRAdN2znnSYBJ+vOu+6PHjYQdrF941fbJVisgtttUdtsuOhhLh8AiLg3BuetN44ftVrldqeis9j4idpobGzEs88+i5KSEsTGxga7HKKAYOMnamPNmjUQBAHLli1zPZaRkYFZs2YFsSoi/2LjJ2ojPz8f+fn5wS6DKKD4O34iIoVh4yciUhg2fiIihWHjJyJSGDZ+IiKFYeMnIlIYNn4iIoXh7/iJ+jmVSgXB4ezwOdEhQXA4Id84Q4BXQ8I10OL76RhCtUBkOBAKAZFXZxYI06twuc0yoYMGAbdfue0coIcQqoHx9n+5IbfaIeNi7dedL4p8wsZP1M85JKfb+XXGRDpQ13gJIyI7PymYFhIuN9Z+v55bo3C5sRa6gSG4fOnKfEMDRo1ut4ykcuJAzWcAgEEDQ6C/eSjO1dXiejFRD3a6HvIdD/UQESkMGz8RkcKw8RMRKQwbPxGRwrDxExEpDBs/EZHCsPETESlMtxr///7v/2LGjBlITEzE+vXr/VUTUdBxbFN/1uUTuJqamlBSUoLS0lLo9XpkZGQgJiYGt99+uz/rI+pxHNvU33W58VdWVuKHP/whBg0aBAB45JFHsHPnTvzmN7/xaXm1WtVxQRo1wkJ0XS0rIPlCDVpIDp3f8l0TjG1tuy3+yNcZ/synVqugkm8cQ+7GVWcEamxrZA3C9CEel9WqfY/Rqp0YGOpwG6fXajEwNAwD9KEYGCp1HKPxIeZqHgDt4sJDAPWAga44nVYP44CBGBBqgCTpAQBhhjAYO4gBgAGhBuiue961Tn0IQgYMAACoNVrX7etjri0bqvecR6PVIWTAAGhDwhAyoOPpK67FAOgwru3z12I0mvYHTVQaLTSG0A7z+/I8AECt7nAM+WNsu+qQ5a7M0gG8+eabsNlsyMnJAQBs3LgR1dXV+MMf/uC34oiCgWOb+rsuH+N3Op1Qqb5/B5Jlud19or6KY5v6uy43/qFDh8JsNrvum81mRERE+KUoomDi2Kb+rsuNPy4uDvv378f58+fR0tKCv//974iPj/dnbURBwbFN/V2Xv9yNjIxETk4Ofv7zn8Nut+OJJ57Avffe68/aiIKCY5v6uy5/uUtERH0Tz9wlIlIYNn4iIoVh4yciUhg2fiIihWHj92DVqlVITk5GcnIyVqxYEexyumXlypWYMWMGkpOTsXbt2mCX023Lly9Hbm5usMvoszi2e6+eGNts/G5UVlZi3759KCsrw5YtW/Dll1+ioqIi2GV1ycGDB/Hpp5+ivLwcmzdvxrp163DmzJlgl9Vl+/fvR1lZWbDL6LM4tnuvnhrbbPxumEwm5ObmQq/XQ6fTISoqCg0NDcEuq0t+8IMf4O2334ZWq0VzczMkSUJYWFiwy+qSCxcuoKSkBFlZWcEupc/i2O6denJss/G7cccdd2DChAkAgNraWuzYsQMJCQlBrqrrdDodXn/9dSQnJyM2NhaRkZHBLqlLCgoKkJOTg4EDb5yJkXzDsd079eTYZuP34tSpU/jlL3+JhQsXYvTo0cEup1sWLFiA/fv3o7GxERs2bAh2OZ22ceNGDBs2DLGxscEupV/g2O49enpsd3nKBiU4dOgQFixYgLy8PCQnJwe7nC6rqamBKIoYO3YsQkNDkZiYiK+++irYZXXa9u3bYTabkZaWhosXL8Jms2Hp0qXIy8sLdml9Dsd279LjY1umDjU0NMgxMTFyZWVlsEvptj179sjp6emyIAiyIAjy008/LW/bti3YZXXL5s2b5UWLFgW7jD6JY7t364mxzT1+N9asWQNBELBs2TLXYxkZGZg1a1YQq+qahIQEVFdXY+bMmdBoNEhMTOzTe3nUPRzbxEnaiIgUhl/uEhEpDBs/EZHCsPETESkMGz8RkcKw8RMRKQwbPxGRwvi98b/77rt49NFHXdOkvvjii64JoGbPno2HH34YaWlpSEtLc00Ja7VaXctv3boVjz76KNLS0pCRkYEjR464nvuP//gPJCUlISUlBYsWLYIgCACA8+fPY+7cuZgxYwZSUlLwxRdftKtpwYIFOHXqFADg2LFjyMzMxLRp05CWloZZs2Zh165drtjS0lLcf//9rhpTU1ORlZWFo0ePumJOnDiBjIwMPProo5g5cyb27t3bbn2yLGPRokVYs2ZNu8djYmJcedPS0lBeXt7l17mxsREpKSlIS0vD4cOHfV4uMzMTpaWlAIC0tDRcunQJpaWlyMzM9LrsqlWr2r1WvrrvvvtQX1/f6eWud63eYOHY5ti+Xp8d2/48G2zZsmXynDlz5IaGBlmWZVmSJLmsrEyePHmy3NjYKP/sZz+Td+zY4YoXRVEuKCiQMzMzZVmW5ZqaGnnSpElyU1OTLMtXzspLSEiQZVmWP/30U/mRRx6RW1paZKfTKf/617+W//u//1uWZVlesGCB/Kc//UmWZVk+duyYPHnyZNlms8myLMuCIMhpaWmyLMtyVVWV/NBDD8kfffSRq4bTp0/LycnJ8rvvvivL8pWz5ubNm9duuz755BM5JiZGrq+vl2VZllNSUuSKigpZlmX5q6++kidMmCALguDKN3v2bDk6OlpevXq1K0dNTY2cmJjYnZe3nbKyMvkXv/hFp5ebN2+evHnz5naPdbTNHbn+7+erCRMmyF9//XWnl+tNOLY5tjvSV8e2387c/eabb63LoUUAAAlUSURBVPDee+9hz549uOmmmwAAarUaM2fOxNGjR/Hmm2/esIxOp8PixYsxadIk1NTUwGAw4I9//CMiIiIAAPfccw++/fZbiKIIp9MJURTR2toKtVoNQRBgMBjgcDiwZ88eFBYWAgDGjh2L0aNH4+OPP0ZiYiIqKytdEx+tXLkSWVlZmDJliquGqKgorFixAnPmzEF6enqH2xYXF4dp06bh3Xffxe9+9zuUlZVBo9EAAP75z39i4MCBrvvr16/Hj3/8YwwfPrxdjsOHD0OtVuOnP/0pLBYLHnnkEcyfP9+1nDvvv/8+1q1bB7VajVtuuQUvv/wympqa8Nprr8FisWD27NlYt26d2+WbmpqQm5uLc+fOYfjw4WhubnY9d9ddd2H//v3t4mfPno2nnnoK06dPb3e/ubkZR48exYoVK6DRaJCQkIBXXnkFn332GSRJwt133438/HyEh4fj888/xx/+8AeoVCqMHz8eTqfT4zYeP34cWVlZrr3LX/3qV7jllluwfPlyiKKIhx56CLt27cIDDzyA/fv3Y8+ePaioqIBarUZdXR1CQkKwfPlyREVFwWKxoKioCCdPnoTdbkdsbCwWLlwIrVaL119/HRUVFdDpdLj55ptRXFzsGmuecGxzbPe3se23Qz1VVVW47bbbXP8YbcXFxeHQoUMdLhcSEoLRo0fj5MmTGDFihGvgyrKM4uJiPPzww9Dr9YiNjUVcXBx+9KMfYfLkybBYLPjJT36C7777Dk6nE4MHD3bljIyMxDfffAMA2LVrF/7t3/4NAPDFF1/gwQcfvKGGu+++GyqVCqdPn3a7fWPGjMHJkycBAFrtlffLqVOnIjs7G88884xrkBcUFCA1NfWG5SVJQlxcHFavXo3169dj3759Hgc1cOWiDKtXr8bbb7+N8vJypKSk4Nlnn0VMTAwWLFiABx54wGuOJUuWIDo6Gh988AHy8/Nx9uxZj/HuPPXUU7jnnnuwcOFCTJs2DW+99RY0Gg1KS0tRXl6OiIgIvPLKKxBFEc899xxyc3OxZcsWxMTEoLW11WPusWPHQqvV4uTJk2htbcWZM2fw6aeful6De++9F0ajsd0yn332GV5++WVs27YN0dHReOuttwAAS5cuxbhx41BaWootW7bgu+++w9q1a9HY2Ii//OUv2Lx5M0pLSzFp0iRUV1f7tO0c2xzb/W1s+3WuHofD0eHjoihCpVK5XU6lUiE0NNR132azITc3F9988w1Wr14NANi0aRPq6+vx8ccfQ6/XY/HixVi+fDnmzZt3Q25ZlqHRaCDLMqqqqrBkyZJu1whc+UduW/OuXbvw9ddf46mnnkJUVJTHKVWffPLJdveffvpprFu3DnPmzHG7zMcff4wZM2a4/vHT09NRVFTUqWOKlZWVWLRoEQBg1KhRiImJ8XlZT/bs2QOLxYLKykoAgN1ux5AhQ3Dy5ElotVrXa5GSkoKCggKv+aZNm4Z//OMfuOOOO/DDH/4QX331FU6dOoUPP/wQiYmJN8SPGzcOQ4cOBXCluV27gtSePXtw5MgRbNq0CQBc/5iRkZEYM2YMHnvsMcTHxyM+Pr5TU+BybHNs96ex7bc9/gkTJqCurg5ms/mG5w4cOID77ruvw+VaWlpQU1OD22+/HQDQ0NCAjIwMaDQavP32266LElRUVCA1NRXh4eHQ6/V48sknceDAAQwZMgSyLOPChQuunOfOnUNkZCQOHz6Me+65B2r1lc2cOHEiDhw4cEMN1dXV0Ol0uO2229xu39GjR3HnnXdCFEV88MEHro94I0eORFxcHI4fP+7x9dmyZQtOnDjhui/Lsmvvyp2OPkbKsuz2H7wjKpUKcpvpmLyt89o6rrHb7W5ry8vLw9atW7F161Zs3LgRK1euvGF5X9c5depU7N27F5988gkmTZqEuLg47Nu3D//4xz8wderUG+Kvb1TX1ul0OrFy5cp2dRUUFECtVuOdd95BcXExBg0ahKVLl/p8rVmObY7t/ja2/db4IyMjMXv2bPz2t79FU1OT6/HNmzfj73//O5555pkblmltbcXSpUsRHx+PESNGwGq1Yvbs2UhMTERJSUm7F+DaO5/D4YAsy6ioqEB0dDS0Wi2mTJniuvjCiRMnUFNTg5iYGHz44YftXtgXXngBq1evbvdLhZqaGixevBjPPfccDAZDh9u2d+9e7NmzBz/5yU+g1+vx2muv4YMPPgBw5TjjgQMHOvyY3dapU6fw+uuvQ5IktLa2Yv369ZgxY4bHZR566CFs374d58+fd72WgwYNwqhRozwud32O999/H8CVxtNRc2hr8ODBrl95nD59ut3c5hqNxvWPOXnyZKxfv951jPrll1/Gq6++irvuuguyLLte4w8//BAXL170WufEiRPx9ddfY8+ePYiLi8OkSZPwl7/8BaNHj8bNN9/s8/ZOnjwZf/7znyHLMkRRxPz58/HOO+/gxIkTSElJQVRUFDIzMzFnzpx2v6rxhGObY7u/jW2/Hup54YUXsHHjRsyfPx+iKEIURYwfPx7vvfcebr31VgDAihUr8Kc//QlqtRoOhwNxcXF46aWXAFz58qihoQEVFRXtLv785z//GVlZWSguLkZycjL0ej3uuusu15dehYWFyM/PR0pKClQqFVasWAGj0YjKykpkZ2e78tx9991Ys2YNVq5ciaVLl0Kj0WDgwIHIzs52feEDAJ9//jnS0tIAXHnHjYiIwJo1a2AymQBc+enXkiVLsHr1aqjVarz44osYP368x9fmN7/5DZYsWYLU1FQ4HA5Mnz4dP/7xjz0uM2nSJMyZMwe/+MUvXMd633zzTddeni8KCwuxePFiJCUlYejQoRgzZozH+Pnz5yM3Nxd79+7FbbfdhgceeMD13MMPP4xXX30Vdrsdv/71r7F8+XI89thjkCQJY8eORW5uLnQ6Hf7zP/8Tv//97/Hqq69i7NixGDJkiNc61Wo14uPjceTIEQwePBj3338/Ll682OFHYU9eeuklFBUVITU1FXa7HXFxcZg7dy50Oh2SkpLw+OOPIywsDCEhIcjPz/c5L8e2exzbnvXGsc1pmYmIFIYXYgmy559/3u2vEUpKSjwemwWAM2fOICcnp8Pn/vVf/xWvvfZat2v0h75SJ/kPx3bvqrMt7vETESkM5+ohIlIYNn4iIoVh4yciUhg2fiIihWHjJyJSmP8PgXKPYwx+/QsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# TOTAL PHENOLS\n",
    "sns.set(color_codes=True)\n",
    "plt.subplot(121)\n",
    "sns.distplot(data.iloc[:, 5], kde=False);\n",
    "plt.subplot(122)\n",
    "for i in range(0, len(division)):\n",
    "    sns.distplot(division[i].iloc[:, 5], kde=False, bins=10);\n",
    "\n",
    "plt.show()\n",
    "\n",
    "# Flavanoids\n",
    "plt.subplot(121)\n",
    "sns.distplot(data.iloc[:, 6], kde=False);\n",
    "plt.subplot(122)\n",
    "for i in range(0, len(division)):\n",
    "    sns.distplot(division[i].iloc[:, 6], kde=False, bins=10);\n",
    "plt.show()\n",
    "\n",
    "# OD280/OD315_of_diluted_wines\n",
    "plt.subplot(121)\n",
    "sns.distplot(data.iloc[:, 11], kde=False);\n",
    "plt.subplot(122)\n",
    "for i in range(0, len(division)):\n",
    "    sns.distplot(division[i].iloc[:, 11], kde=False, bins=10);\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. K-means can go wrong!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "K-means is able to find the optimal solution in a few steps, but sometimes it can take the wrong path.\n",
    "The choice of the initial partition can affect the final partition. In other words, the algorithm can be trapped between local minimal.\n",
    "\n",
    "In order not to make this thing happen it is possible to initialize the algorithm in different ways that guarantee us (more or less) an optimal solution or close to it.\n",
    "\n",
    "We can initialize K-means with:\n",
    "- Random Start: the algorithm is repeated with different initial classifications chosen randomly and it is chosen between the final ones, that of minimum deviance (cost);\n",
    "- Rational Choice: maximum distance centroids.\n",
    "\n",
    "We pay attention to rational choice. If we reason on the contrary it means that if we put the centroids all in the central point of the units (or in any case very close to each other), the algorithm could reach a NOT optimal solution.\n",
    "\n",
    "So we can implement a new algorithm to see if our objective function is arbitrarily larger from the cost of the optimal solution, which in our case is about '145'.\n",
    "\n",
    "Attention: there is no need to perform a random start because the centroids are chosen by us and the value of the objective function will always be the same."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### We choose the same number of clusters as in point 3, which is 3.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then implement K-means."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# WRONG INIZIALIZATION, this assures us that initially the centroids are all in the center.\n",
    "def randPU_wrong2(n, K):\n",
    "    # all 0 matrix\n",
    "    Uzero = []\n",
    "    for k in range(0, n):\n",
    "        Uzero.append([1] * K)\n",
    "\n",
    "    return Uzero\n",
    "\n",
    "# UTILS FUNCTIONS\n",
    "def mse(Xrow, Xmean0row1):\n",
    "    min_dif = 0\n",
    "    for r in range(0, len(Xrow)):\n",
    "\n",
    "        min_dif += (Xrow[r] - Xmean0row1[r])**2\n",
    "        \n",
    "    MSE = min_dif/len(Xrow)\n",
    "    return MSE\n",
    "\n",
    "def mse2(Xrow, Xmean0row1):\n",
    "    min_dif = 0\n",
    "    for r in range(0, len(Xrow)):\n",
    "        \n",
    "        min_dif += (Xrow[r] - Xmean0row1[r])**2\n",
    "        \n",
    "    MSE = min_dif/len(Xrow)\n",
    "    return MSE\n",
    "\n",
    "def costf(X, Xmean_ott, U):\n",
    "    cost = 0\n",
    "    for i in range(0, len(X)):\n",
    "        for j in range(0, len(U.iloc[0])):\n",
    "            if U.iloc[i, j] == 1:\n",
    "                cost += mse(X.iloc[i], Xmean_ott.iloc[j])\n",
    "    return cost            \n",
    "\n",
    "def diagSU(su):\n",
    "    '''\n",
    "\n",
    "    :param su: sum of U_0 columns\n",
    "    :return: diagonal matrix\n",
    "    '''\n",
    "    Uzero = []\n",
    "    for j in range(0, len(su)):\n",
    "        Uzero.append([0] * len(su))\n",
    "\n",
    "    for j in range(0, len(su)):\n",
    "        Uzero[j][j] = 1 / su[j]\n",
    "\n",
    "    return Uzero\n",
    "\n",
    "def diagonalize(su):\n",
    "    '''\n",
    "\n",
    "    :param su: diagonal of a matrix\n",
    "    :return: diagonal matrix\n",
    "    '''\n",
    "\n",
    "    Uzero = []\n",
    "    for j in range(0, len(su)):\n",
    "        Uzero.append([0] * len(su))\n",
    "\n",
    "    for j in range(0, len(su)):\n",
    "        Uzero[j][j] = su[j]\n",
    "    Uzero = pd.DataFrame(Uzero)\n",
    "    return Uzero"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def kmeans_wrong(X, K, Rndstart):\n",
    "\n",
    "    '''\n",
    "\n",
    "    :param X: data Matrix\n",
    "    :param K: Number of cluster of the partition\n",
    "    :param Rndstart: number of random start\n",
    "    :return:\n",
    "    '''\n",
    "\n",
    "    maxiter = 100\n",
    "    n = len(X)\n",
    "    j = len(X.iloc[0])\n",
    "    epsilon = 0.00001\n",
    "\n",
    "    # find the best solution in a fixed number of random start partitions\n",
    "    for loop in range(0, Rndstart):\n",
    "        \n",
    "        # initial WRONG partition U_0 is given\n",
    "         \n",
    "        U_0 = pd.DataFrame(randPU_wrong2(n, K))\n",
    "        \n",
    "        # column frequency = random cluster\n",
    "        sum_col = []\n",
    "        for r in range(0, K):\n",
    "            sum_col.append(sum(U_0[r]))\n",
    "        \n",
    "        # 1/su on diagonal of a NxN matrix\n",
    "        su_diag = diagSU(sum_col)\n",
    "\n",
    "        # given U compute Xmean (centroids)\n",
    "        Ut = U_0.transpose()\n",
    "        dot1 = pd.DataFrame(np.dot(su_diag, Ut))\n",
    "\n",
    "        Xmean0 = round(pd.DataFrame(np.dot(dot1, X)), 4)\n",
    "       \n",
    "\n",
    "\n",
    "\n",
    "\n",
    "        for iter in range(1,maxiter):\n",
    "            #given Xmean = assign each units to the closest cluster\n",
    "            U = []\n",
    "            for r in range(0, n):\n",
    "                 U.append([0] * K)\n",
    "                    \n",
    "            \n",
    "            for r in range(0, n):\n",
    "                \n",
    "                min_dif = mse(X.iloc[r], Xmean0.iloc[0])\n",
    "                \n",
    "                posmin = 0\n",
    "                for j in range(1, K):\n",
    "                    \n",
    "                    dif = mse(X.iloc[r], Xmean0.iloc[j])\n",
    "                    if dif < min_dif:\n",
    "                        min_dif = dif\n",
    "                        posmin = j\n",
    "                U[r][posmin] = 1\n",
    "            \n",
    "            U = pd.DataFrame(U)\n",
    "            \n",
    "\n",
    "            # given a partition of units, so given U compute Xmean (centroids)\n",
    "\n",
    "            # update sum_col\n",
    "            sum_col = []\n",
    "            for t in range(0, K):\n",
    "                sum_col.append(sum(U[t]))\n",
    "            \n",
    "            \n",
    "            ## RARE CASE (BUT POSSIBLE) #############################################################\n",
    "            #if there is some empty cluster we must split the cluster with max sum_col\n",
    "            while sum([sum_col[h] == 0 for h in range(0, len(sum_col))]) > 0: # some cluster is empty\n",
    "\n",
    "                p1 = min(sum_col)\n",
    "                p2 = max(sum_col)\n",
    "\n",
    "                # select min column (empty cluster)\n",
    "                for j in range(0, len(sum_col)):\n",
    "                    if p1 == sum_col[j]:\n",
    "                        c1 = j\n",
    "\n",
    "                # select max column (cluster) for split its points to empty cluster\n",
    "                for k in range(0, len(sum_col)):\n",
    "                    if p2 == sum_col[k]:\n",
    "                        c2 = k\n",
    "\n",
    "                # list of units in max column (cluster)\n",
    "                ind = []\n",
    "                for row in range(0, len(U)):\n",
    "                    if int(U.iloc[row, c2]) == 1:\n",
    "                        ind.append(row)\n",
    "\n",
    "                # split max cluster\n",
    "                ind2 = []\n",
    "                for row in range(0, p2//2):\n",
    "                    ind2.append(row)\n",
    "\n",
    "                for row in range(0, len(ind2)):\n",
    "                    U.iloc[row, c1] = 1\n",
    "                    U.iloc[row, c2] = 0\n",
    "\n",
    "                sum_col = []\n",
    "                for q in range(0, K):\n",
    "                    sum_col.append(sum(U[q]))\n",
    "            #################################################################################################\n",
    "            \n",
    "            # give U compute centroids\n",
    "            _U = U.transpose()\n",
    "            _dot1 = pd.DataFrame(np.dot(diagSU(sum_col), _U))\n",
    "            Xmean = round(pd.DataFrame(np.dot(dot1, X)), 4)\n",
    "            \n",
    "\n",
    "            #compute ojective function\n",
    "            BB = (np.dot(U, Xmean)) - X\n",
    "            f = round(np.trace(np.dot(BB.transpose(), BB)), 4)\n",
    "\n",
    "\n",
    "            #stopping rule\n",
    "            dif = 0\n",
    "            \n",
    "            for k in range(0, K):\n",
    "                dif += mse2(Xmean.iloc[k], Xmean0.iloc[k])\n",
    "                    \n",
    "            if dif > epsilon:\n",
    "                Xmean0 = Xmean\n",
    "            else:\n",
    "                break\n",
    "\n",
    "        if loop == 0:\n",
    "            U_ott = U\n",
    "            f_ott = f\n",
    "            Xmean_ott = Xmean\n",
    "\n",
    "        if f < f_ott:\n",
    "            \n",
    "            U_ott = U\n",
    "            f_ott = f\n",
    "            Xmean_ott = Xmean\n",
    "\n",
    "    # calculate cost\n",
    "    cost = costf(X, Xmean_ott, U_ott)\n",
    "    print('Done')\n",
    "    return round(pd.DataFrame(U_ott), 4), f_ott, cost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Done\n"
     ]
    }
   ],
   "source": [
    "kluster = kmeans_wrong(dataS, 3, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Our objective function has value:  177.92262100461545\n"
     ]
    }
   ],
   "source": [
    "print('Our objective function has value: ', kluster[2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The objective function in the case of incorrect initialization of the algorithm has a greater value than our optimal solution (about 30 more!).\n",
    "So we can say that K-means is conditioned by the choice of initial centroids."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}