Neural_Net.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## I. Import and Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training Data was pulled from two sources. \n",
    "\n",
    "Solar Irradience data was pulled from the [National Solar Radiation Database](https://nsrdb.nrel.gov/). Pointwise GHI, DHI, and DNI data is returned hourly for an entire calendar year. All irradience values are recorded in w/m^2. Location is for a latitude longitude coordinate pair. Coordinates were chosen based on the locations of [U.S. Climate Reference Network (USCRN)](https://www.ncdc.noaa.gov/crn/) monitoring stations. This is because the USCRN has real time measurements of GHI, which can be used to make real-time predictions of DHI. The NRSDB is not updated so frequently. \n",
    "\n",
    "The production prediction module in the [omf](https://github.com/dpinney/omf/wiki) uses GHI measuremeants read from USCRN. \n",
    "\n",
    "In addition to solar irradience data, this project uses other weather data such as cloud cover, air pressure, and season data to try and model the relationship between GHI and DHI. For that, this project pulls data from [darksky](https://darksky.net/dev). And that data is combined with the NRSDB data for testing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please NOTE! In this notebook, the directory \"Testing_Data\" does not mean it is solely Testing Data. Rather for this Neural network, it contains both testing and training data. This is because the neural network requires much more data to train than regression. So much of the same data used to test the regression model is now being used to train the neural network. The True training data for the neural network is in \"Proudction_Test\", where the data is from USCRN and combined with data combed from darksky."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import os\n",
    "import numpy as np\n",
    "from glob import glob\n",
    "from matplotlib import pyplot as plt\n",
    "import seaborn as sns\n",
    "from random import randint\n",
    "from glob import glob\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def featureHistograms(df, features=['GHI','DHI','Cloud Cover'],):\n",
    "    sns.set_style('darkgrid')\n",
    "    fig, axs = plt.subplots(ncols=len(features))\n",
    "    for i in range(len(features)):\n",
    "        sns.distplot(df[features[i]], ax=axs[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Lets try and make a neural network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data\n"
     ]
    }
   ],
   "source": [
    "#Get 2018 Data\n",
    "_Curr_Dir = os.getcwd()\n",
    "_Testing_Data_Dir = os.path.join(_Curr_Dir, 'Testing_Data')\n",
    "print(_Testing_Data_Dir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>Hour</th>\n",
       "      <th>Minute</th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Clearsky DHI</th>\n",
       "      <th>Clearsky DNI</th>\n",
       "      <th>Clearsky GHI</th>\n",
       "      <th>Cloud Type</th>\n",
       "      <th>...</th>\n",
       "      <th>GHI</th>\n",
       "      <th>Relative Humidity</th>\n",
       "      <th>Solar Zenith Angle</th>\n",
       "      <th>Surface Albedo</th>\n",
       "      <th>Pressure</th>\n",
       "      <th>Precipitable Water</th>\n",
       "      <th>Wind Direction</th>\n",
       "      <th>Wind Speed</th>\n",
       "      <th>timestamps</th>\n",
       "      <th>Cloud Cover</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>30</td>\n",
       "      <td>-22.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>62.75</td>\n",
       "      <td>103.76</td>\n",
       "      <td>0.69</td>\n",
       "      <td>978</td>\n",
       "      <td>0.1</td>\n",
       "      <td>296</td>\n",
       "      <td>2.1</td>\n",
       "      <td>1514739600</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "      <td>-23.1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>66.39</td>\n",
       "      <td>114.24</td>\n",
       "      <td>0.69</td>\n",
       "      <td>978</td>\n",
       "      <td>0.1</td>\n",
       "      <td>287</td>\n",
       "      <td>2.2</td>\n",
       "      <td>1514743200</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>30</td>\n",
       "      <td>-23.6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>69.34</td>\n",
       "      <td>125.01</td>\n",
       "      <td>0.69</td>\n",
       "      <td>978</td>\n",
       "      <td>0.1</td>\n",
       "      <td>283</td>\n",
       "      <td>2.3</td>\n",
       "      <td>1514746800</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>30</td>\n",
       "      <td>-23.9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>71.25</td>\n",
       "      <td>135.74</td>\n",
       "      <td>0.69</td>\n",
       "      <td>978</td>\n",
       "      <td>0.1</td>\n",
       "      <td>283</td>\n",
       "      <td>2.3</td>\n",
       "      <td>1514750400</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>30</td>\n",
       "      <td>-24.1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>72.87</td>\n",
       "      <td>145.93</td>\n",
       "      <td>0.69</td>\n",
       "      <td>979</td>\n",
       "      <td>0.1</td>\n",
       "      <td>284</td>\n",
       "      <td>2.3</td>\n",
       "      <td>1514754000</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Year  Month  Day  Hour  Minute  Temperature  Clearsky DHI  Clearsky DNI  \\\n",
       "0  2018      1    1     0      30        -22.5             0             0   \n",
       "1  2018      1    1     1      30        -23.1             0             0   \n",
       "2  2018      1    1     2      30        -23.6             0             0   \n",
       "3  2018      1    1     3      30        -23.9             0             0   \n",
       "4  2018      1    1     4      30        -24.1             0             0   \n",
       "\n",
       "   Clearsky GHI  Cloud Type  ...  GHI  Relative Humidity  Solar Zenith Angle  \\\n",
       "0             0           8  ...    0              62.75              103.76   \n",
       "1             0           0  ...    0              66.39              114.24   \n",
       "2             0           8  ...    0              69.34              125.01   \n",
       "3             0           8  ...    0              71.25              135.74   \n",
       "4             0           8  ...    0              72.87              145.93   \n",
       "\n",
       "   Surface Albedo  Pressure  Precipitable Water  Wind Direction  Wind Speed  \\\n",
       "0            0.69       978                 0.1             296         2.1   \n",
       "1            0.69       978                 0.1             287         2.2   \n",
       "2            0.69       978                 0.1             283         2.3   \n",
       "3            0.69       978                 0.1             283         2.3   \n",
       "4            0.69       979                 0.1             284         2.3   \n",
       "\n",
       "   timestamps  Cloud Cover  \n",
       "0  1514739600          0.0  \n",
       "1  1514743200          0.0  \n",
       "2  1514746800          0.0  \n",
       "3  1514750400          0.0  \n",
       "4  1514754000          0.0  \n",
       "\n",
       "[5 rows x 24 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_csv(os.path.join(_Testing_Data_Dir, 'psm_south_dakota', 'psm_testing_data2018.csv')).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEICAYAAABbOlNNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAIABJREFUeJzs3Xl4VOXZ+PHvmTXJZA9hN4EAUVkiBH51YStFpCqtBS1CXqGtK9SlVeBFsSoihqAFtSJYK6JvLCYp+LZ20b6lIrERFKMRggRkVdmSELLMJJn1/P5IZshAkplAJjMZ7891eWHmnJl5zpyZc59nux9FVVUVIYQQopkm2AUQQggRWiQwCCGE8CKBQQghhBcJDEIIIbxIYBBCCOFFAoMQQggvEhiEEEJ4kcAghBDCiwQGIYQQXnTBLsCFcLlcOJ3+T9jWapUO7d8VQrFM0LFy6fXaTn1vf89rqHx2oVCOQJQhWOfVl1D4vIOls47d33PbLQOD06lSXV3v9/7x8VEd2r8rhGKZoGPlSk6O6dT39ve8hspnFwrlCEQZgnVefQmFzztYOuvY/T230pQkhBDCiwQGIYQQXiQwCCGE8CKBQQghhBcJDEIIIbxIYBBCCOFFAoMQQggvEhiEEEJ4kcAghBDCS7ec+dxRtY12zticROm1GJVgl0ZcLDmf303u896SfAcC4zsRGCxWJ1vLypl0WU+Mhs7NAyO6npzP7yb3eW9JvgOBIU1JQgghvEhgEEII4UUCgxBCCC8++xhcLhdLly5l3759GAwGli9fTmpqqmd7QUEBeXl56HQ65s+fz6RJkzh+/DhLlizB6XSiqirLli0jLS2N999/n5deegmdTsfNN9/MzJkzaWxsZNGiRZw+fRqTycTKlStJTEwM6EELIYRom88aw5YtW7DZbOTn57NgwQJycnI82yoqKsjNzSUvL4/169ezevVqbDYbL7zwArfddhu5ubncc889rF69GrvdzooVK3jttdfIzc0lPz+fyspK3nrrLdLT09m4cSM/+clPWLt2bUAPWAghRPt8Bobi4mLGjx8PwMiRIyktLfVs27VrF6NGjcJgMBATE0NKSgplZWUsXryYiRMnAuB0OjEajRw8eJCUlBTi4uIwGAyMHj2anTt3er3+hAkT2L59eyCOUwghhJ98Bgaz2Ux0dLTnb61Wi8Ph8GyLiTm7IpDJZMJsNpOYmIher+fQoUOsXLmSe++9t819Wz5uMpmoq6vrtIMTQgjRcT77GKKjo7FYLJ6/XS4XOp2u1W0Wi8Vzkd+xYwdPPvkkzzzzDGlpadhstlb3bfkaFouF2NhYn4XWahXi46P8PERorLMSFWkgwqgnPi7C7+cFklar6dAxdJVQLZcQouv4DAyZmZls3bqVG264gZKSEtLT0z3bMjIyeP7557FardhsNg4ePEh6ejo7duzg6aef5tVXX6Vfv34ADBo0iKNHj1JdXU1UVBSffvopd9xxB8ePH2fbtm1kZGRQWFjI6NGjfRa6o2vIqoqG+gYbjVY71dUuv58XSKG6fm0w13wWQoQGn4FhypQpFBUVMWvWLFRVJTs7mw0bNpCSksLkyZOZM2cOWVlZqKrKgw8+iNFoJDs7G7vdzsMPPwzAwIEDWbZsGQ8//DB33HEHqqpy880306tXL2bPns3ixYuZPXs2er2eVatWBfyghRBCtE1RVVUNdiE6ym53duhuu0HR8O4Xx5h0WU8SQmT6vNQYzufveQ2V8xkK5zAQZQjWefXFfd5bCvZ3oKt01nn299zKBDchhBBeJDAIIYTwIoFBCCGEl+9E2m1xcVwuF6tW5XDgwFfo9Xoefvgx+ve/xLO9tbQo1dXVTJ061TOK7dprr+VnP/tZsA5BCNEBEhiETx9++AE2m43f/34DpaW7WbPmOXJyVgNn06Js3rwZq9VKVlYWY8eO5csvv2TatGk89thjQS69EKKjpClJ+LRrVwlXXnk1AMOHj6CsbG+Lba2nRSktLWXPnj3cdtttPPDAA5SXl7f18kKIECM1BuGTxWLBZDqbFkWj0fhMi5KWlsbw4cO55ppreOedd1i+fDm/+93v2n0ff2e0h8pM9lCYJR4KZRDhRwKD8MlkMlFff3YMtaqqPtOiZGRkEBkZCTRNkvQVFMD/Ge2hMpP9uzyPYfr06Z4cav3792fFihWdWgYRXBIYhE8jRlxBUdGHTJ48hdLS3aSlDfZsaystyuLFi7nuuuu44YYb2L59O8OGDQviEYjOZLVaUVWV3NzcYBdFBIgEBuHThAmT2LnzY+bNux1VVVmy5Any8t5k6ND0NtOiLFiwgCVLlvDWW28RGRnJ8uXLg30YopOUlZXR0NDA7bffjsPh4KGHHmLkyJHBLpboRJISI0hCoRmiNZISw3+hcA6D0ZS0b98+vvjiC376059y5MgR7rrrLt577z1P8+K5XC4XTufFX2ZO1ln5oMx7EMPE9GT6hEjG5EDSajU4nRffbKrX+/d7kRqDEKJDBg4cSGpqKoqiMHDgQOLj46moqKBPnz6t7t/RbMhtcfcttRTsfqauIrmShBAhbdOmTZ4lfk+dOoXZbCY5OTnIpRKdSWoMQogOueWWW3jkkUeYPXs2iqKQnZ3dZjOS6J7kbAohOsRgMMi6KWFOmpKEEEJ4kcAghBDCi8+mJJfLxdKlS9m3bx8Gg4Hly5eTmprq2d5aZk23119/ncrKShYuXEhFRQUPPfSQZ9vevXtZsGABs2bNYsKECQwYMACAkSNHsmDBgk48RCGEEB3hMzBs2bIFm81Gfn4+JSUl5OTksG7dOqDtzJoul4tHH32U3bt3c9111wGQnJzsmSn5+eef89xzzzFz5ky+/vprhg0bxssvvxzAwxRCCOEvn01JxcXFjB8/Hmi6my8tLfVsayuzptVqZfr06cybN++811NVlaeeeoqlS5ei1WrZs2cPp06dYs6cOdx1110cOnSoEw9PCCFER/msMZjNZk+yLACtVovD4UCn07WZWTMuLo5x48bx9ttvn/d677//PkOGDCEtLQ1oqkncfffdXH/99Xz66acsWrSIzZs3t1smf7NwuoVKNs6WQjUrZqiWSwjRdXwGhnOzZ7pcLp+ZNdvzzjvvMHfuXM/fw4cPR6ttmqY9ZswYysvLUVUVRVHafI2OzqQMlWycLYVCOoXWBDMlhhAiNPhsSsrMzKSwsBCAkpISz1KN0JRZs7i4GKvVSl1dnSezZntKS0vJzMz0/L1mzRreeOMNoCk5V58+fdoNCkIIIQLLZ41hypQpFBUVMWvWLFRVJTs7mw0bNpCSktJmZs22VFVVER0d7XXhv/vuu1m0aBHbtm1Dq9VKXnchhAgyya4aJNKUdD7JrhoaZQjWefXFfd5bCvZ3oKtIEj0hhBBBJYFBCCGEFwkMQgghvEhgEEII4UUCgxBCCC8SGIQQQniRwCCEEMKLBAYhhBBeJDAIIYTwIoFBCCGEFwkMQgghvEhgEEII4UUCgxBCCC8SGIQQQniRwCCEEMKLBAYhhBBeJDAIn1wuF88+m8099/yC++67m2+//cZre0FBATNmzGDmzJls3brVa9snn3zCxIkTu7K4QoiL5HNpT5fLxdKlS9m3bx8Gg4Hly5eTmprq2V5QUEBeXh46nY758+czadIkz7bXX3+dyspKFi5c6Pn7T3/6E4mJiQA8+eST9O3bl0WLFnH69GlMJhMrV670bBeh4cMPP8Bms/H732+gtHQ3a9Y8R07OagAqKirIzc1l8+bNWK1WsrKyGDt2LAaDgRMnTrBhwwYcDkeQj0AI0RE+awxbtmzBZrORn5/PggULyMnJ8WxzXxTy8vJYv349q1evxmaz0djYyIIFC9i4caPXa5WWlrJy5Upyc3PJzc0lLS2Nt956i/T0dDZu3MhPfvIT1q5d2/lHKS7Krl0lXHnl1QAMHz6CsrK9LbbtYtSoURgMBmJiYkhJSaGsrAyr1coTTzzB0qVLg1RqIcSF8lljKC4uZvz48QCMHDmS0tJSz7aWFwWDweC5KKSmpjJ9+nTGjh3LoUOHPPvv2bOHV155hYqKCr7//e9zzz33UFxczJ133gnAhAkTJDCEIIvFgskU7flbo9F4agFms5mYmLPryJpMJsxmM8uWLeP222+nV69efr+PVqsQHx/lc7/GOitRkQYijHri4yI6cCSdS6vV+FXecC3D6dOnmTFjBq+99hqDBg0KShlEYPgMDGazmejosxcFrVaLw+FAp9O1eVGIi4tj3LhxvP32216vdeONN5KVlUV0dDT33XcfW7du9XoNk8lEXV2dz0L7ewFxC5ULSUuhcFFpTWvlSkyMR1EcnscVBXr0iAUgOjoai8Xi2ddisaDX6/n000/5+uuveemll6ipqeHBBx/kueeea/e9nU7VrwXPVUVDfYONRqud6mpXRw+x03TWAu2hVgZ/Foy32+08/vjjRESExu9JdC6fgeHcH77L5UKn07W6zWKxeAWKllRV5Wc/+5ln+8SJE/nyyy+9XsNisRAbG+uz0P5eQDzvHSIXkpZC4aLSmtbKlZ4+lH//eytXXTWR0tLdDBgwiOrqepKTY8jIyOD555/HarVis9k4ePAgGRkZ/POf//Q8f+zYsT6DguheVq5cyaxZs3jllVeCXRQRAD4DQ2ZmJlu3buWGG26gpKSE9PR0z7bWLgott7dkNpuZNm0a//jHP4iKiuLjjz/m5ptvJjIykm3btpGRkUFhYSGjR4/uvKMTnWLChEns3Pkx8+bdjqqqLFnyBHl5bzJ0aDqTJ09mzpw5ZGVloaoqDz74IEajMdhFFgH09ttvk5iYyPjx4/0KDB2t4bfFXfNvKZRaAQKpq1sYfAaGKVOmUFRUxKxZs1BVlezsbDZs2EBKSkqHLgoxMTE8+OCDzJ07F4PBwNVXX83EiRP53ve+x+LFi5k9ezZ6vZ5Vq1Z1+kGKi6PRaFi0aInXY6mpAzxNDjNnzmTmzJltPr+oqCig5RNda/PmzSiKwvbt29m7dy+LFy9m3bp1JCcnt7p/R2v4bXHX/FsKpVaAQOqsFgZ/mgkBFFVV1Yt+ty5mtzs79CE1KBre/eIYky7rSYJBG8CS+a87NSW1xd8vmb/8Pa+hcj5D4RwGq4/Bbc6cOSxdurTdzueO/l7b4j7vLQX7O9BVujowyAQ3IYQQXnw2JQkhRFtyc3ODXQQRAFJjEEII4UUCgxBCCC8SGIQQQniRwCCEEMKLBAYhhBBeJDAIIYTwIoFBCCGEFwkMQgghvEhgEEII4UUCgxBCCC8SGIQQQniRwCCEEMKLBAYhhBBeJDAIIYTwIoFBCCGEF5/rMbhcLpYuXcq+ffswGAwsX76c1NRUz/aCggLy8vLQ6XTMnz+fSZMmeba9/vrrVFZWsnDhQgD+9re/8cYbb6DVaklPT2fp0qVoNBqmT59OdHQ0AP3792fFihWdfZxCCCH85DMwbNmyBZvNRn5+PiUlJeTk5LBu3ToAKioqyM3NZfPmzVitVrKyshg7diwul4tHH32U3bt3c9111wHQ2NjI888/z1//+lciIyN56KGH2Lp1K+PGjUNVVVnwQwghQoTPpqTi4mLGjx8PwMiRIyktLfVs27VrF6NGjcJgMBATE0NKSgplZWVYrVamT5/OvHnzPPsaDAby8vKIjIwEwOFwYDQaKSsro6Ghgdtvv525c+dSUlLS2ccohBCiA3zWGMxms6eZB0Cr1eJwONDpdJjNZmJizi4ubTKZMJvNxMXFMW7cON5++23PNo1GQ48ePYCm5QDr6+sZO3Ys+/fv54477uCnP/0pR44c4a677uK9995Dp2u7aFqtQnx8lN8H2VhnJSrSQIRRT3xchN/PCyStVtOhY+gqoVouIUTX8RkYoqOjsVgsnr9dLpfnon3uNovF4hUozuVyuXj22Wc5fPgwL774IoqiMHDgQFJTUz3/Hx8fT0VFBX369GnzdZxOlerqer8OEEBVNNQ32Gi02qmudvn9vECKj4/q0DF0lY6UKzm57XMthOi+fDYlZWZmUlhYCEBJSQnp6emebRkZGRQXF2O1Wqmrq+PgwYNe28/1+OOPY7VaWbt2radJadOmTeTk5ABw6tQpzGYzycnJF3VQQgghLpzPGsOUKVMoKipi1qxZqKpKdnY2GzZsICUlhcmTJzNnzhyysrJQVZUHH3wQo9HY6uvs2bOHTZs2MWbMGH72s58BMHfuXG655RYeeeQRZs+ejaIoZGdnt9uMJIQQIrAUVVXVYBeio+x2Z4eaYRoUDe9+cYxJl/UkwaANYMn8J01J5/P3vIbK+QyFcxiIMgTrvPriPu8tBfs70FU66zz7e25lgpsQQggvEhiEEEJ4kcAghBDCi/TyCp9cLherVuVw4MBX6PV6Hn74Mfr3v8SzvbW0KOXl5SxatAi73U5cXBzPPvus13wYIUTokhqD8OnDDz/AZrPx+99vYN68+1mz5jnPNndalLy8PNavX8/q1aux2Wz84Q9/YPr06WzcuJGhQ4eyadOmIB6BEKIjpMYgfNq1q4Qrr7wagOHDR1BWtrfFtrNpUQwGgyctypIlS1BVFZfLxYkTJ+jbt2+wii+E6CAJDMIni8WCyXS2GUij0eBwOADaTIuiKAoOh4ObbroJq9XKvffe6/N9/E11EiopTkIhfUgwyuB0OvnNb37D4cOHURSFJ598st2JraL7kcAgfDKZTNTXnx1DraqqX2lR9Ho9//jHP/joo49YvHgxb775Zrvv42+qk1BJcfJdncewdetWAPLy8vj444957rnnPBmXRXiQPgbh04gRV7BjRxEApaW7SUsb7NnWVlqUpUuXsmPHDqApsCiKEpSyi8537bXX8tRTTwFw/PhxYmNjg1wi0dmkxiB8mjBhEjt3fsy8ebejqipLljxBXt6bDB2a3mZalDlz5rB06VJeeuklNBoNS5cuDfZhiE6k0+lYvHgx//rXv/jd737X7r4dzYbcFncTYkvBbk7sKl3dZCgpMYIkFJohWiMpMfwXCucw2CkxKioqmDlzJn//+9+Jimr9wiUpMS5eV6fEkBqDEKJD/vznP3Pq1CnuueceIiMjURQFjUZapTuLVYV6u9PrMU2jvUvLIIFBCNEh1113HY888gj/9V//hcPhYMmSJUREhH9zTleptzvZWlbu9dj1V/QjsgvLIIFBCNEhUVFRvPDCC8Euhgggqf8JIYTwIoFBCCGEFwkMQgghvPgMDC6Xi8cff5xbb72VOXPmcPToUa/tBQUFzJgxg5kzZ3pmRLq9/vrr/Pa3v/X8/f7773PzzTdz6623UlBQAEBjYyP3338/WVlZ3HXXXVRVVXXGcQkhhLhAPgPDli1bsNls5Ofns2DBAnJycjzb2sqs2djYyIIFC9i4caNnX7vdzooVK3jttdfIzc0lPz+fyspK3nrrLdLT09m4cSM/+clPWLt2bWCOVAghhF98Bobi4mLGjx8PwMiRIyktLfVsa5lZMyYmxpNZ02q1Mn36dObNm+fZ9+DBg6SkpBAXF4fBYGD06NHs3LnT6/UnTJjA9u3bO/sYhRBCdIDP4apms9lrgRWtVovD4UCn07WZWTMuLo5x48bx9ttve71Oa/u2fNxkMlFXV+ez0B2dYh8q2ThbCoXMnK0J1XIJIbqOz8BwbvZMl8vlV2ZNX6/j3rfl4xaLxa+EXP5m4XQLlWycLYVCOoXWBDMlhhAiNPhsSsrMzKSwsBCAkpISr7zrbWXWbM2gQYM4evQo1dXV2Gw2Pv30U0aNGkVmZibbtm0DoLCwkNGjR3fGcQkhhLhAPmsMU6ZMoaioiFmzZqGqKtnZ2WzYsIGUlJQ2M2u2pmmt4Ie54447UFWVm2++mV69ejF79mwWL17M7Nmz0ev1rFq1qtMPUgghhP8ku2qQSFPS+SS7amiUIVjn1ZfvSnbVM7Y2ciWpF98M7u+5lQluQgghvEhgEEII4UUCgxAi5KmqytHTFrphy3e3JIFBCBHylv/ffuZs2EnxNzXBLsp3ggQGIURIq2mw87c9pwDYceQMLqk1BJwEBiFESNt+5AwuFWaM6ked1UGl2RbsIoU9CQxCiJBW/E01sRE6bh7VD4BjNY1BLlH4k8AghAhph07XM6iHif4JkUTqNRyXwBBwEhiEECFLVVUOn64nLSkKRVHoHRvBqTprsIsV9iQwCCFCVqXFRp3VQVpSU8bfpCgDZ+ptMmw1wCQwCCFC1uHTTak0BjYHhkSTHptTxWJzBrNYYU8CgxAiZJ2obepP6BcXCUBilB6AKouMTAokCQxCiJB1otaKRoGe0QYAEqOa/j1dbw9mscKeBAYhRMg6WdtIcrQRnbbpUhUToUOjNE16E4EjgUEIEbJO1FrpE3t2jReNohBj1FFrdQSxVOFPAoMQImSdrG2kd6z3Ou0xETrqGiUwBJLPFdxcLhdLly5l3759GAwGli9fTmpqqmd7QUEBeXl56HQ65s+fz6RJk6iqqmLhwoU0NjbSs2dPVqxYwZEjR8jOzvY8r6SkhJdeeomMjAymTp3qWRL02muv5Wc/+1kADlVcKJfLxapVORw48FXzSnyP0b//JZ7trX0Hjh8/zpIlS3A6naiqyrJly0hLSwviUYjuxqWqlJtt9IrxXhUyNkLH8RqZyxBIPgPDli1bsNls5OfnU1JSQk5ODuvWrQOgoqKC3NxcNm/ejNVqJSsri7Fjx7J27VqmTZvGjBkzeOWVV8jPz+fnP/85ubm5ALz77rv07NmTCRMm8NFHHzFt2jQee+yxwB6puGAffvgBNpuN3/9+A6Wlu1mz5jlyclYDbX8HXnjhBW677TauvfZaPvzwQ1avXs2aNWuCfCSiM9jtdpYsWcKxY8ew2WzMnz+fyZMnd/r71DY6cLhUkkwGr8djI/TsOyUpuAPJZ1NScXEx48ePB2DkyJGUlpZ6tu3atYtRo0ZhMBiIiYkhJSWFsrIyr+e4L/5u9fX1vPjiizz66KMAlJaWsmfPHm677TYeeOABysu9l7QTwbdrVwlXXnk1AMOHj6CsbG+Lba1/BxYvXszEiRMBcDqdba4FLrqfd955h/j4eDZu3Mirr77KU089FZD3qapvGpKa1DxE1S3GqMOpqtTLXIaA8VljMJvNREdHe/7WarU4HA50Oh1ms5mYmLNriJpMJsxms9fjJpOJuro6zz6bNm3ihz/8IYmJiQCkpaUxfPhwrrnmGt555x2WL1/O7373u3bLpNUqxMdH+X2QjXVWoiINRBj1xMdF+H5CF9BqNR06hq7SWrnsdiu9eiV5HtfptEQ3Dx9s6zvgPr+HDh1i5cqVvPTSS368t3/nNVTOZyicw2CU4Yc//CFTp04FmlJWaLWBWXP5TPOQVPcQVbfYiKbLlnRAB47PwBAdHY3FYvH87XK50Ol0rW6zWCzExMR4Ho+IiMBisRAbG+vZ569//avXhf+qq64iMrJp8sqUKVN8BgUAp1Pt0OLiqqKhvsFGo9VOdfXFL6jdGUJhIfnWtFYuvd5IefkZz+NOpwuz2UZkpLHN7wDAjh07ePLJJ3nmmWf86l/w97yGyvkMhXMYiDL4WjDeZDIBTTcFDzzwAL/+9a/b3b+jN3Jujc2L8qT2jiU+PspzQ5DcPNnN6iLoNweB0FDTSFSkdzBUNArxMV13A+AzMGRmZrJ161ZuuOEGSkpKPJ3EABkZGTz//PNYrVZsNhsHDx4kPT2dzMxMtm3bxowZMygsLGT06NEA1NXVYbPZ6NOnj+c1fvOb33Dddddxww03sH37doYNGxaAwxQXY8SIKygq+pDJk6dQWrqbtLTBnm1tfQd27NjB008/zauvvkq/fv2CWHoRCCdOnODee+8lKyuLH/3oR+3u29EbObdvKswA6J1OqqvrPTcEBqWpb6GytjHoNweB0GhzUt/gPbNbdV3YZ3guX0HfzWdgmDJlCkVFRcyaNQtVVcnOzmbDhg2kpKQwefJk5syZQ1ZWFqqq8uCDD2I0Gpk/fz6LFy+moKCAhIQEVq1aBcDhw4fPu0gsWLCAJUuW8NZbbxEZGcny5csv4HBFIE2YMImdOz9m3rzbUVWVJUueIC/vTYYOTW/zO5CdnY3dbufhhx8GYODAgSxbtizIRyI6Q2VlJbfffjuPP/44V199dcDep6rehlY523TkFqXXotUo1MqQ1YBR1G7YtW+3OzsUPRsUDe9+cYxJl/UkwRCY9tCOCoVmiNZ0pFz+3n34y9/zGirnMxTOYTCakpYvX867777r1Tz4hz/8gYiI1pt0Ovp79bzPP/dTdLiKd+ddBZw97wDr/nOEPrFGXph5Rcj8pjvLGZuTrWXeg3Cuv6IfkerF14w6rcYghBAt/eY3v+E3v/lNwN+nqt7mSZp3rtgIHXXS+RwwMvNZCBGSqurtJJ4zh8EtVmY/B5QEBiFESGqvxhBjbKoxuLpfS3i3IIFBdDsnaxpxueSCEM5UVW2qMUS1XmOINupwqZJlNVAkMIhu5fkPDjHzDzvYfaI22EURAWSxObE6XO3WGEAW7AmUsO983lduZl9VA3ZneI11/q4qOnwagNNyQQhr7lnP5+ZJcotuHsJaaZbvQSCEfWD477/s4XitlR9e3pPrhvUOdnHERXJfMOqskicnnLnzJLVdY2gaolopNwgBEdZNSfU2J8drm9LznmpeO1Z0XzaHi5rmkShmGaoY1txLdya00cdgMjTd00rNMTDCOjAcOn02h88pqXJ2e+67SECGKoY5d9/BuZlV3bQaBZNBy2n5XQdEWAeGAxVNgeHqtEQqzFYZ2tbNue8iL0mIpM7qkHz8Yayq3oYCxLdRY4CmDmhpSgqMsA4MX59pwKBVuCYtCbtTlY6qbs7dbHBprxgcLhWz9DOErap6O3GRenQapc19oo06aUoKkLAODNUNduIj9fRsXjNWvkTdm/v8De7ZtD7ImXo5n+GqaQ5D681IbjFGLafNssRnIHwnAkOP5kVlpD2yezvToikJmsa6i/BUZWl71rNbdISOmkYHNocMRe9sYR4YHE2BwdS0rKTUGLo3s9WBUachoXlsu4xMCl9N6TDa7l+As5PcpJ+h84V1YKhpbKoxxEXp0ShQaZFqZ3dmtjkwGbREN18QJDCEr/YS6Lm5vwcV0pzU6cJvsi5LAAAgAElEQVQ6MLibkjSKIh1VYcBidRJt1HkuCBYJDGGp0e7EYnP60cfQ9D0olybiThe2gcHhUqltbGpKAhnBEA7cNQb3BUH6GMLTmebEeEk+mpKkxhA4PlNiuFwuli5dyr59+zAYDCxfvpzU1FTP9oKCAvLy8tDpdMyfP59JkyZRVVXFwoULaWxspGfPnqxYscKzbOdnn33mWUx87dq12O32Vve9WLWNTV+uuMimQ4w2aqmySCbG7sxidWIy6jDoNGgVRZqSwpR7cluCjxpDpF6DXqtQITWGTuezxrBlyxZsNhv5+fksWLCAnJwcz7aKigpyc3PJy8tj/fr1rF69GpvNxtq1a5k2bRobN25k6NCh5OfnA7Bnzx5effVVcnNzyc3NJSYmps19L1Z1812Hu8Zg0mslRW83Z7Y5iDZoURQFo06DReYxhCX3REZffQyKotDDZJQaQwD4DAzFxcWMHz8egJEjR1JaWurZtmvXLkaNGoXBYCAmJoaUlBTKysq8njNhwgQ++ugjXC4XR48e5fHHH2fWrFls2rTpvNd379sZahqa7ibjmgNDlEFHbaMdp+Tx77bMzTUGAKNeIzWGMOUrHUZLSSaD1BgCwGdTktlsJjo62vO3VqvF4XCg0+kwm83ExJxdXNpkMmE2m70eN5lM1NXVUV9fz2233cYvfvELnE4nc+fOZfjw4a3u64tWqxAfH9XuPvbjTa9zSc8YFI1CfLQBlwoY9cT7uBPpClqtxucxBEOolgvA0lxjAIjQSWAIV1U+Eui1lBRt4FCFOdBF+s7xGRiio6OxWM4mo3O5XOh0ula3WSwWYmJiPI9HRERgsViIjY0lMjKSuXPnevoPrrrqKsrKylrd1xenU6W6ur7dfY5VNn1ZtA4nqkvFfe9x5EQN2h4mn+8RaPHxUT6PIRg6Uq7k5BjfO3USl6p6+hgAjDqtdD6Hqap6G9FGLUad77ExSSYDOw7bUFUVRWk7fYboGJ+ffGZmJoWFhQCUlJSQnp7u2ZaRkUFxcTFWq5W6ujoOHjxIeno6mZmZbNu2DYDCwkJGjx7NkSNHmD17Nk6nE7vdzmeffcawYcNa3bczuPsY4poX9IhqvtN0z54V3UuD3YkKZ2sM0pQUttpb0vNcPWOMWB0uT9NxOHG6VP6+5xRvf3Gc8rquXTbAZ41hypQpFBUVMWvWLFRVJTs7mw0bNpCSksLkyZOZM2cOWVlZqKrKgw8+iNFoZP78+SxevJiCggISEhJYtWoVUVFR3HTTTcycORO9Xs9NN93EkCFDWt23M1Q32InUa4jQa2ngbGCokvw63ZI7YZ67xhCh01BeJ52O4ahp1rPv/gWA3rFNWQ2O1zYS7+dzuouiw1XsOt60hO0L/z7A6puGdtl7+wwMGo2GZcuWeT02aNAgz//PnDmTmTNnem3v0aMH69evP++17rzzTu68806/9r1YNQ124iLOflFMUmPo1iy2pjtCd43BqNNKdtUwVWWxMzDJv36uXjFNCTKP1zQytHfXNW0GmsPp4rNvakhPNtEr1siHByo5UlXPgMSu6f8L2wlu7jxJbhF6LRrl7OQZ0b1Yzq0x6DXYnC5JoBaGLqTGcCLMVmg8eLqeBruTzEviyOjb1O+69avKLnv/sA0M7jxJbhpFITZCLzWGbur8GkPTV9dsC7+25e8yh7Np+VZfcxjcTEYdsRE6jteEWWCosGDUaUhJiCI2Qs9lvWMoPHi6y94/bANDdYPdM+vZLT5KL30M3ZR7BJJ7rd+I5sAgS3yGF3eN3t8aA0Cf2AhO1IZPf5Oqqhw6Xc/ApCi0zQsVjUlNYO8pM432rmk+DevA0LLGAE2zoKXG0D2dbUo628cAkmE13LjT1vg7KgmgT6wxrGoMFWYbdVYHKQlnUwON6BuH06Wy56TveV6dISwDg8Ppwmx1th4YpI+hw1wuF88+m8099/yC++67m2+//cZre0FBATNmzGDmzJls3brVa9vrr7/Ob3/724sug7vJyNRiuCpAnQSGsHK6uUbfkRpD37gIjtc2hs0a4GXNF/++zStPAgzv19TP4B6lFGhhGRiqm5sXzg0McZHSlHQhPvzwA2w2G7///QbmzbufNWue82xrK19WY2MjCxYsYOPGjZ1SBndTUtS5TUkyMilovvjiC+bMmdOpr+n+fSZ1IDtB39gIrA5X2Nz0lZ2qQ6tR6Blj9DwWE6GnX1wE+8u7Zpa3z+Gq3ZFnctu5NYYoPWarE5vDhcGPWZWiya5dJVx55dUADB8+grKyvS22nc2XZTAYPPmyUlNTmT59OmPHjuXQoUMXXQaL1YlRp0GnUbBztilJagzB8Yc//IF33nmnUzIht3RBTUlxTXfWJ2oaO/S8UFV20kzvGKOnf8FtSLKJryosbTyrc4VlYKjxZFY9p/O5OVBUN9i9orFon8ViwWQ6my9Lo9HgcDRdkNvKlxUXF8e4ceN4++23/X6f9nJgORSIidARHx9FY52VhObx6w7Fd96sQAmFvFLBKkNKSgovvvgi//3f/92pr3u63kakXkOk3v8bt77NgeFYTSPD+vhOqRPKHE4XX1WYPUNUWxqSbKLw4Gka7U4i9NqAliMsA8O5Kbfd3H+fqZfA0BEmk4n6+rP5k1RV9Zkv60K0lwOrqs5KlF5LdXU9qqLBbrOj1ShUVDcELedUKOS7CkQZ/MmBNXXqVL799lu/Xs+fpJduNVYnvWMjSEg4P59ZY52VqEjvGkGEUc/QlKZaS2WjM+iB+mLtOV6L1eFiQHK017EqGoWRA5Jwbf+acquLjADnKQvLwFDTVmBo7tCqapB+ho4YMeIKioo+ZPLkKZSW7iYtbbBnW0ZGBs8//zxWqxWbzebJl9XZLM2rt7kpikK0QStNSd2AP0kv3Y6fqScxSt/q/qqiof6c326j1U6k6qJ3jJG9x2uCHqgv1o6vygHoEanzOlbVpdI3quly/fnhSlKiLyz9h7+JL8MyMFS712KIaLvGIPw3YcIkdu78mHnzbkdVVZYseYK8vDcZOjS9zXxZna3e5vQKDNA0uUmGq4aXCouNYReQ2mJAYhRHq7p3UADYc6KOuAidJ/lnS33jIojSa7uknyFMA4Mdk0F7XgezOzBUSWDoEI1Gw6JFS7weS00d4Ln7aC1fltuMGTM6pQwWm9Nr+B40rfkr+ZLCh6qqVJhtJEd3/MYiNTGSd0prun367dKTdVzWO6bVY9AoCoN6dE0HdFgOzWma9Xx+VSvKoEWvVTgjQ1a7HYvV4Znc5mYySlNSMPXv35+CgoJOe706qwOrw0VydMdHFg1MiqLB7uJkN864a7Y6OHK6nkt7tV1jGpJs4kClJeBzNsI3MLRSFVMUhQSZ/dwtWWxOTzoMt2ijTgJDGHEv0dnjAlZYHJLcNGpuf3nXDOcMhC9P1qECl/WObnOftKQoahsdnnWxAyVsA8O5Hc9uiVGGsJkI812hqirmVvoYog3SxxBOKpsDw4U0JQ1JNqFAl00ACwR3uovL2qkxuNORH6oMbAAMy8BQ005gSIjSSx9DN2N1uHC61FY6n7WSRC+MVFiamoEupCkpUq8lJSGSfd04MJSeqCM1IZJoY9tdv2nNyxIfPh3YjvawDAznrsXQUmKUniqL9DF0J57Mqsbzm5IaHS7sTlmTIRxcTFMSwGW9otlzsi4kciZZVThjc3r9Z22nWC5V5YtjNYxoZWJbS0lRemIjdBwO8Agsn6OSXC4XS5cuZd++fRgMBpYvX05qaqpne0FBAXl5eeh0OubPn8+kSZOoqqpi4cKFNDY20rNnT1asWEFkZCSvv/46f//73wGYOHEi9913H6qqMmHCBAYMGADAyJEjWbBgwQUfkM3hot5+fgI9tySTkUqLDZeqounGoxe+S86m3D6nKak5UJitDhLCIBXCd12l2UZshO6CZ/Vm9I3jn2UVHK9tpF9cx1N1WFWoPyetdZRei/ECLhP1didby8q9Hpt0WU+MhtaP7UhVPTWNDkb1j2v3dRVFYWBiVMCbknwGhi1btmCz2cjPz6ekpIScnBzWrVsHnE2gtnnzZqxWK1lZWYwdO5a1a9cybdo0ZsyYwSuvvEJ+fj6TJ0/mnXfe4U9/+hMajYbZs2dz7bXXEhkZybBhw3j55Zc75YBqGltPh+HWK8aAw6Vypt7eoURdIngsnsyq3ufUPUqpzuokoXtPeBU0zWG40NoCwMjmDKRfHKu9oMDQ0Yt5Zyr5tgaAUf3aDwwAaT2ieH9/ZUCH5vpsSiouLmb8+PFA0918aWmpZ1vLBGoxMTGeBGotnzNhwgQ++ugjevfuzauvvopWq0VRFBwOB0ajkT179nDq1CnmzJnDXXfdddEJ19pKh+HWqzkVxqluPKztu8a9FkO08fzOZ5BEeuGi0my9oP4Ft0E9TMQYdXz6dXUnlqprfH6sliSTgf7xET73HZhkoqbREdC+Up81BrPZTHT02eFTWq0Wh8OBTqdrM4Fay8dNJhN1dXXo9XoSExNRVZVnnnmGoUOHMnDgQCorK7n77ru5/vrr+fTTT1m0aBGbN29ut0ztJluragCgX3K0Zx93jpUIo57BfZsissVFUPOqhEICttaEYrks56zF4BbdPCS5RkaZhYVys40xl/i+Y26LVqNwZWoCHx050+0mun3+bQ2j+sX5Vea05pFJh0/XB6zVw2dgODdJmsvl8plAzf14REQEFouF2NimKp7VamXJkiWYTCaeeOIJAIYPH45W2/SDHzNmDOXl5T5Panu5V76taBqVoHW6PPu4c6w0Wu1E0tQDdOhkLdV9A5uIqj2hkICtNR0pl795Vy7WuWsxuLlTLMsaG92fw+miwmyld6zvO+b2jEtLZMv+CvaeMjP0AlJrBMOxmgZO1VmZ+//8ywzrDgyHTlsYkxIfkDL5bErKzMyksLAQgJKSEq8EaRkZGRQXF2O1Wqmrq/MkUMvMzGTbtm0AFBYWMnr0aFRV5Ze//CWXXnopy5Yt8wSDNWvW8MYbbwBQVlZGnz59LirSu0cctbUCVEKkHoNWodwsTUndhTvtxbk1hoTmc+we/y66r5N1Vlwq56U96ajxgxLRaxX+eU5fQSj7z8EqAK5MTfBr/x4mA9FGLYcCOGTVZ41hypQpFBUVMWvWLFRVJTs7mw0bNpCSktJmArX58+ezePFiCgoKSEhIYNWqVWzZsoVPPvkEm83Ghx9+CMBDDz3E3XffzaJFi9i2bRtarZYVK1Zc1AFV1dvQKOcn0HNTFIXesRFhtUZsuGurKSlCr8Vk0AZ8FqgIvBO1Tb9H99oKFyo2Qs/VAxP5x5flZF2ZgkGrueCRRV1l28HTDEiMJDXRvyZcRVFISzIFdC6Dz8Cg0WhYtmyZ12ODBg3y/H9rCdR69OjB+vXrvR6bMmUKu3fvbvU9XnnlFb8L7MvpejsJUYbzVj9qKSUhkqNnGjrtPUVgWWxOtBoFYyur7iWZDJyWeSnd3omaphp8n7iLz8x7/bDeFB44zbpth8joG9tlI4suxKk6K8XfVPPz713SoecNTIpi24HTASpVGE5wq7LYfC4knpIQyddnGnCFwEQY4ZvF6iDaoG21iTHJZKBSAkO3d6y2Ea0CvWIursYAkHlJHL1ijPzn4OmQn/z4v7tO4FLhR8N7d+h5g3uYqG6wUxGgJvHwCwz1dpJ8THZKTYjE6nBRLkNWu4WaRker2XIBkqKkxhAOjlU30CvGiK6dmr6/FEXh2vQe1DQ62HHkTIee22B3svt4LZ8cPdMpzc2n6qxs/aqS9748xb/2llNvOzuBrtJs5a3iY3x/cBL94zs27+Kynk0jRctOBSYFSNitx3DaYiM1sf0P2d2Wd7Sq4aJHQYjAO9Ngb7PPqEe0gY8O27rd8ETh7WhVAyl+trG3pCgKZ2zes5WdKqQkRnF5r2i2HznDV+VmvudjRjHA+/sqWPefI1gdZ2sZX1VaePqGy9rNX9Qal6qy9atKdhw5g1ZR0OsUPj9Wy0vbDvHjEb0Z0SeGNz75Bpeqct/4gR16bYD0ntEoQFm5mfGDkjr8fF/CKjCoqkpVvc0zjLEtg5sTUZWVm7lygH8jAUTw1DTY6dNGAO8fF0G93clpi40eF5CVUwSfqqp8faaBK/r5N1yzpQaHi+1fVXg9dvWQZACmXJbMt9WNLPtHGW/elulZ2vdcLlXllY+Osn7H11wSH8EP0pOJi9Sx61gt/zlUxS//tIvf3TyizUmzrR3P8+8fYMeRM4zqF8v3h/TAqNPQMy6Sf315ij+VHCfvM5W4CB05Pxrqd6dzS1EGLamJkVJj8IfZ6sTmVH1O+oiL1JOSEEnpidouKpm4GGfq7Vzeq/Uc9QPck32q6iUwdFMVZhv1ducFXSDbYzLomHFFHzYWf8svN+1izS0jzrtpbLQ7efK9/WzZX8HUoT25ok+sZ+DK1QMTufbyniz7Rxn3/mkXv7/1Cr9qDi8WHua9L8sZOzCRCYPP3s0P7xvL+AEJLPzBIE7UWLkkIZKoi+gUv7RnNJ83p9LobGHVx3CyrqlNsHeM7wvEsN4x7D4RGpkYRdtUVW1eX6P1YD8w0T0LVEaZdVdHzzQNu0xN6Hh+I1/6xkWwbNpQvjnTwC/++Dkft+hz2HfKzNw/fs6/91dw//iBPPSDweeNZvzegESevWkYh07X8+D/ltJob38p2dyd35D76bf8eERvxg9KbHWf2Ag9l/aKvqigAHB5rxjKzbaATPAMqxrDidqmzuTesa0HBnd7ZJReS2b/ON7dW85XFRbSe7a9YpIILovNicOleiaznSs52oDJoOVIK2mI95Wb2X60mpO1jWgUhWiDhv5xkVw5IMGTM0sEn3sNY/daA51tdEo862Zm8MS7+7hv825SEyLRazUcqLSQGKXnxZtHcOWAhPP6KtyuGZjIshsu49G/7WXxX7/ktzcNQ689/57673tO8bvCw1ybnsz8CWkU7q9o5dU6z2W9znZAXzOw9SB0ocIqMJxsniTTVoeyuz1y0mU9mTg4iZwtX7Flf4UEhhB2Nili619VRVG4vFc0n35zNnHaaYuNZf/cx0eHm+4O9VoFvVbjWfBHp1G4eVRffnnNgIu+axMXb3+5mSST4aIyq/oyvE8sG+eO5n93neCTo2dwqTD1smSmZ/Rpc8RbS1MuTcZsdZD9r694YPNult94uafJWlVV/mfnt7z04WHGpMTz5PWXYnEFviXi0uYO6D0n6iQwtOdkrRWDVvE5j0FRFFSdljGpCfxl90l+cWUKkReYA14Elnt97oQ2mpIAJg1J5tn3D3Cg0kJNg51H/16G2erg7rEDuGZwD/Ydr+EHl/fC6nDxl8+/5Uh1IwXFxyj+uprfTh9OP6k9BNX+CgvpyYGpLbRk1GmYldmPWZn9Luj50zP6YNBqWLHlK259/VNuGNqL+Eg9hQdPs+dkHVMuTebxqekYdBpPfq9AijbqGJJs4rNjnd/PEFZ9DCdqm5Jw+VqAp8HhYmtZObPH9Keq3s6LhYelryFEuVMLtzWiBOAH6T2I1Gv4+R8/Z17BLkwGLb/7aQbTR/UjsfmursHhwgX0iDZy/6TBPHbj5RytqmdeXoknHYPoeo12J4dO13NpF9faW1thzenHJeDGYb14/b9GMbJfHJu+OM66oiNYbA4enTKEp2+87IIXGbpQmZfEs/t4badP5AurGsOxmgb6tNG/0JqhfWLJGt2PjcXHqG6wc881qZ0+MkJcnOPuHDrtnNceJgMvzBjBm59+y+AeUfx4ZF8+PlRFn3ZW7xmTmkDW6P5sKjnOvPwvePnWK9ocEisCp/REHU6XSoaPJS07W2uL8riHufoyuIeJ3/5kGE6Xit3puqhg0Nqqcf4EKLfR/ePI++wYXxyr7dRMq2ETGJwulUOn67n5ij5+P0dRFOZenYrRoOOPn3zDv/ZVMLR3DOPTEhmXlkh6z2hZ/jPIjtc0EqXXtjqGXFEUrCoYFRjaL45lfeOwOfy784OmESsrpw/n4T/vYV7+F6ybecVFJ3ETHfP5tzUowEg/Vi67UG1NgrvQ5xp0WmyOs4812JznPdba67dVjsJ9FxagAP5fajx6rcJ/DlVJYGjNt9UNWB0uBnVgZIO7M3r6qH7E6DWY7S62H67ilY+O8vuPjhIfqWdk/ziuGpjI2NR4mSUdBMeqG+gbF9HqrOaG5hmq9aqKUwW7S2X7VxUd+mGlJJlYOX04i/93N/cUfMFz04d7JkCKwNv59RnSe0YTExG4S1F7k+Au9Lm+Hmvt9S+mHG0xGXSM7h9P4cFKfjVxYKfN/g+bwHCweXHsC/1Rm4w6rh2eTK9oA5f98FL+tPMbzHYXO49U8cFXlUDTOOthfWLoHRtBUpSeKIOWKL2WSIMWo06DUashyqCjT6yxy9saw9WxmkYuaSePjPvHdjE/sEsSo/jpyL78efdJ5r75Gb+4KpWszL7nrTEtOtdpi42SY7XceXVKsIvSrU25LJmn/rmf0hN1jOikJrmw+ebvOWlGq1E8qxtdjIQoAyP6xnL1kGQ+2l9OWs8YPv36DF98W8On39RQaS7H12i0gUlRfC8lnjGXxJN5SRyxbeT6EW1zqSrHaxr9XsDkYvSOjeCFmVew/O97eaXoCH/c+Q3jByVxVWoCo/rH0SfWKLmYOtm/91eiApOG9Ah2Ubq1HwzpwTP/PsDbu05IYDjXjiNVXNE3ttPv1BVFoXd8JEkVZh694XIidRocThfVDXa27atgVGoC9TYHjQ4Xnx05Q0oPEx8fOk293cWfd58k//PjKMCQZBODk00kRhnQKGAw6qk1W7E6XFidLoxaDdFGHbEROhKi9CRGGehh0pNoMqDXanC61LP/NY+gio/UExehC9sL1qHKehodLtJ7dk3TTnyUgZtH9qVfYhR/232Sjw6d5r29Te2/ydEGRqfEc116MlcNSGh1gpPwn0tVyf/8GJf3ipamu4sUbdRx0/DebN51gruuTu2UfrKwCAzldVb2V1j45bgBAX+vlk0XCVF6+sRHev6utdi4ekgyBqWp7fDDfafoEx9Fybc1lB6vZefX1dQ0OICmTKAGrQajToNOq+BwqtRZHV6ZHf2h0yjER+pJiNITY9Sh1yroNBp0GgWdVmn6V9M0wSuxeRJR7xgjl/eKDvncQsXNk9Yy+wdmXdu2pCSZ+OX3BzGybwx9Ek3sOVFL4f4Kig5W8d6X5cQYdUwaksT00ZdwWWJkp6SK/q75864TfH2mgadvvCxsb2y60pz/1593Sk+Ss+Urnp8x/KIHzfgMDC6Xi6VLl7Jv3z4MBgPLly8nNTXVs72goIC8vDx0Oh3z589n0qRJVFVVsXDhQhobG+nZsycrVqwgMjKyQ/t2xFufHUOjNM1ODCU6jYbBvWKoqG3kltFD2f5VBT+4vFdTimidhvdLT3o6rdz/jh6YxNa9J7m8bxxVFltTsLA3jXpwp5bWaxVcqkp1g4Nd31QTG6WntsFBvc2B3ebC5nJgd7iaOmSdLpyqitXuorbR7tUENqiHiesuTebaS5NJaSdPjcvlYtWqHA4c+Aq9Xs/DDz9G//5nV5wK1HktOlxFn1hj0EYKKYrCwB4mBvYwkRShY0xaEntP1PLBvgq27K/kndJTGHUaesUYSYzSE6HXEqHTeP0bY9SSZDKQGGUgyWQgyaQnKcrQrfugfF0TfPn3/gpWf3CIMSnxIfeb7a56x0bwwMQ0nvn3AZa9t4/7xg8kyWS44KDrMzBs2bIFm81Gfn4+JSUl5OTksG7dOgAqKirIzc1l8+bNWK1WsrKyGDt2LGvXrmXatGnMmDGDV155hfz8fG688Ua/9/35z3/u9wF89m01bxV/y/WX9+zwYhfB4K5xjL+sV6vbDToNsRF6UpJMHKuqZ/w5gcPdybr9qwpuHNGHxAidZ5s76LiHwJ0bdL43qAfvf3mSfkkm/q/0JCfNVtYVHWFd0REu7xXNxMFJXDUkmViNQnK0AZ1Wg0aB/3v/31TW1rPu5df4ck8pa9Y8R07OaqBj34GOnNe9p+rYfuQM88b6f8EJNL1WQ0b/eCwNdh6cPIRdJ2rZebiKmgY71fU2ahodnLI7m5oHHS4a7c42Z8AadRq0ioJGQ9O/ioJGo2DQKkTotETovQOMu+ZnaNHkGBOho3diFFqni9gIXVPts7m2qPfUFjWemqNGUdBqFDRK8+x/VaXcbCPJZOhQrae9a4Ivp+qsPPzXvVzWM5oVN14utYVOdMsVfaiut/OH7Uf5+5flzMrsx4JJg3w/sRU+A0NxcTHjx48HYOTIkZSWlnq27dq1i1GjRmEwGDAYDKSkpFBWVkZxcTH33HMPABMmTGD16tVccsklfu/bkQtIbYODCYN7sGjy4I4cd1jyNUJHq1GINuoY2ieWGrOVq4ck88/dx7GrsHV/BS8XHeXloqPnP2/Xe6iJKWw7cJpJw0dQVrbXs60j34GOnNdInZYbh/Uia3T/jn0IXcQJXNE/nlqz1Ssgu4fM/uDyXjhcKlv3nuLy/vE0Wh2cbq4Bnqm3c6Y5I2bTjNWmGqDd6cKlNs0GbrA5sTldnGlw0FBrxelScbhc2JwuLFZnh5scz6Vtvh47VZg/dgC3X+X/yKD2rgm+9Iox8uZtmQxKNkkTXCdTFIW7rknlB+k9mtaC8GNxojapPixZskT94IMPPH9PnDhRtdvtqqqq6p///Gf1mWee8WxbtGiRWlRUpF577bVqQ0ODqqqq+vXXX6uzZs3q0L4itHTWd0CEh/a+DyI8+BxaER0djcVi8fztcrnQ6XStbrNYLMTExHg9brFYiI2N7dC+IrR01ndAhIf2vg8iPPgMDJmZmRQWFgJQUlJCenq6Z1tGRgbFxcVYrVbq6uo4ePAg6enpZGZmsm3bNgAKCwsZPXp0h/YVoaWzvgMiPLT3fRDhQVHV9tOKukcg7N+/H1VVyc7OprCwkJSUFCZPnkxBQQH5+fmoqso999zD1KlTqaysZCspNz4AAAkPSURBVPHixVgsFhISEli1ahVRUVEd2leEjs78Dojur7Xvw6BBF9bJKUKTz8AghBDiu0WmbwohhPAigUEIIYSXsB5KcLEzNDvD9OnTiY5uWp2qf//+3HrrrTz99NNotVrGjRvHfffd12Xl/OKLL/jtb39Lbm4uR48e5eGHH0ZRFIYMGcITTzyBRqNhzZo1fPDBB+h0OpYsWUJGRkab+3a1rj6fdrudJUuWcOzYMWw2G/Pnz2fw4MF+f26d6fTp08yYMYPXXnsNnU4XlDIESyj8joOp5e+2ywRrnGxX+Oc//6kuXrxYVVVV/fzzz9V58+Z16fs3NjaqN910k9djP/7xj9WjR4+qLpdLvfPOO9U9e/Z0STlfeeUVddq0aepPf/pTVVVV9Z577lF37NihqqqqPvbYY+r//d//qaWlpeqcOXNUl8ulHjt2TJ0xY0ab+wZDV5/PTZs2qcuXL1dVVVXPnDmjTpw4sUOfW2ex2WzqL3/5S/W6665TDxw4EJQyBFOwf8fBdO7vtquEdVPSxczQ7AxlZWU0NDRw++23M3fuXHbu3InNZiMlJQVFURg3bhwfffRRl5QzJSWFF1980fP3nj17+N73vgc0zUx2l2PcuHEoikLfvn1xOp1UVVW1um8wdPX5/OEPf8ivfvUrAFRVRavVduhz6ywrV65k1qxZ9OzZE+jYuQsHwf4dB9O5v9uuEtaBwWw2e5pxALRaLQ6Ho8vePyIigjvuuIP169fz5JNP8sgjj3glkjOZTNTV1XVJOadOneo1CUltTsjXXjncj7e2bzB09fk0mUxER0djNpt54IEH+PWvf92hz60zvP322yQmJnoujNCxcxcOgv07DqZzf7ddJaz7GII9Q3PgwIGkpqY2ZekcOJCYmBiqq6s9290zghsbG7u8nC37CHzNTm9t32AIxvk8ceIE9957L1lZWfzoRz/i2Wef9Wzz9bl1hs2bN6MoCtu3b2fv3r0sXrzYqybQFWUItmD/jr+LwrrGEOwZmps2bSInJweAU6dO0dDQQFRUFF9//TWqqvKf//yHMWPGBKWcQ4cO5eOPPwaaZia7y/Gf//wHl8vF8ePHcblcJCYmtrpvMHT151RZWcntt9/OokWLuOWWW4COfW6d4Y9//CNvvvkmubm5XH755axcuZIJEyZ0aRmCLdi/4++isA67U6ZMoaioiFmzZnlmaHalW265hUceeYTZs2ejKArZ2dloNBoWLlyI0+lk3LhxXHHFFYwYMaLLy7l48WIee+wxVq9eTVpaGlOnTkWr1TJmzBhuvfVWXC4Xjz/+eJv7BkNXn8+XX36Z2tpa1q5dy9q1awF49NFHWb58uV+fW6B05NyFg2D/jr+LZOazEEIIL2HdlCSEEKLjJDAIIYTwIoFBCCGEFwkMQgghvEhgEEII4SWsh6sG2zfffMOzzz7LyZMniYiIICIigkWLFvHee+/Ro0cPZs+e7dl35syZrF69mmPHjpGXl8dzzz0XxJKLtnz88cf8+te/ZvDgwaiqisPhYO7cuWRkZPDQQw9RUFDg2fett96isrKS+++/nx/84Ae8++67GI3GIJY+vHz11Vc8++yzNDQ0UF9fz8SJE7n//vv55JNPLvo31PLc+fOe7pno4UICQ4A0NDQwf/58nnrqKUaNGgXArl27WLZsmSfPjeierrrqKs9Fx2KxMGfOHJ5++ukgl+q7pba2loceeogXX3yRAQMG4HQ6+dWvfkVeXh5paWld/p4tb/LCgQSGANm6dStXXXWVJyhA0/rI//M//8OaNWuCWDLRmUwmE7feeivr168PdlG+U/79739z5ZVXMmDAAKApf9LKlSvR6/V8/vnnnv3eeecd3njjDQwGAwMGDGDZsmX89a9/5dChQyxcuBCr1cr111/P+++/z6effkp2djaxsbFotVpGjhzp93sC5OTkUFxcDMC0adPIysrihhtu4C9/+QtRUVGsX78erVbL1KlTeeyxx7BarRiNRp566imcTifz588nPj6eCRMmcNdddwX+Q2yHBIYA+fbbb0lJSfH8PX/+fMxmM+Xl5YwZM4a//e1v/OMf//BsP3DgQDCKKTpBUlISZ/5/e/fv0jgYx3H83Yo6JCC6KFJ/pLp16KBCNymCdHBwKWIh4j/QQRBBFGwcpENd6lCE4uIgKuLi5OKgUOmiIDgUM/gDKa3QwR8oaekNxXDhOPC48/TO72vLwwNPCCGf5HnC9ymVuLi4QNd1u71QKDAyMvKBZ/b/KhQKdHR0ONoURXEcl0olVlZW2N3dRVVVlpaW2Nzc/One44ZhkEwm0TSNhYWFXxrz4OCAm5sbtra2KJfLRCIRAoEAw8PD7O/vMzo6yt7eHmtraxiGga7rDA4OkslkSCQSTE1NUSwW2dnZoaGh4XcuzR8hwfBO2traHOWBU6kUUFtLqFQqTE5O/rDGIP5Nt7e39PX1cX9/79hM5XWeWvx57e3tnJ+fO9qur6/J5/OO497eXrsy68DAAEdHR/j9frvP94Uf7u7u0DQNqNVnurq6evOYpmnS39+Py+Wivr4ev9+PaZqEw2FisRherxdN02hubiaXy7G6uko6naZardoFAT0ez6cIBZC/kt7N0NAQmUyG09NTu+3y8pJ8Pv/fLVR9ZQ8PD2xvbxMKhT76VL6UYDDI4eGh/fC2LIt4PE4ul7P7eDweTNPk6ekJgGw2i6ZpNDY2UiwWgdreFq9aW1sxTROAs7OzXxqzp6fHnkayLIuTkxO6urro7u6mWq2STqcJh8MAeL1epqenWV9fxzAM+975iF0Rf0a+GN6JoiikUimWl5dJJBKUy2Xq6uqYnZ2VaaN/3PHxMbqu43a7qVQqRKPRT/Om91Woqko8Hmd+fp5qtcrj4yPBYJBIJEI2mwWgpaWFaDTKxMQEbrebzs5Oe11hY2OD8fFxfD6fPR20uLjIzMwMqqqiKApNTU1vHtPlcpHNZhkbG8OyLEKhED6fD6gV00wmkwQCAaBWBDEWi/Hy8sLz8zNzc3N/8cq9jRTRE0II4fB5vl2EEEJ8ChIMQgghHCQYhBBCOEgwCCGEcJBgEEII4SDBIIQQwkGCQQghhIMEgxBCCIdv3y8CLdhR46wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df = pd.read_csv(os.path.join(_Testing_Data_Dir, 'psm_south_dakota', 'psm_testing_data2018.csv'))\n",
    "featureHistograms(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## II. Features and Data Engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we have all the above variables....Which ones do we think would work best in a NN?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we want to predict DHI, at the very least. Maybe DNI as well. What can we use to predict? \n",
    "[More Information on GHI, DHI, and DNI here!](https://firstgreenconsulting.wordpress.com/2012/04/26/differentiate-between-the-dni-dhi-and-ghi/#:~:text=Global%20Horizontal%20Irradiance%20(GHI)%20is,Diffuse%20Horizontal%20Irradiance%20(DHI).)\n",
    "\n",
    "But a useful relationship to keep in mind is:\n",
    "\n",
    "*Global Horizontal (GHI) = Direct Normal (DNI) X cos(θ) + Diffuse Horizontal (DHI)*\n",
    "\n",
    "**GHI** is a must, as it is the primary metric our sensors will feed into such a model. \n",
    "\n",
    "**Cloud cover** is also essential, as cloud cover is one of the primary factors in how much direct normal solar irradience is diffused.\n",
    "\n",
    "**Solar Zenith Angle** is important as it dictates how much irradience is direct normal (DNI)\n",
    "\n",
    "Another useful relatinship to capture is **time**. We are given timestamps in addition to month, day, year, second, hour and minute variables. Timestamps and/or a combination between day + month would not be super useful, as it would overfit to the day. \n",
    "A more useful relationship would be to capture time of day. This is useful because in a non - featurized dataset, many GHI readings will read '0', simply because it is dark out. We are primarily concerned about the relationship between GHI and DHI, therefore factoring in readings when there is no sunlight to begin with could be confounding. Therefore adding in **minute** and **hour** variables in the input vector could adequately capture this relationship. \n",
    "\n",
    "Another solution would be to featurize this dataset, whereby we create a seperate feature which is a combination of day + time. OR, we could completly remove the nighttime entries entirely to ensure we only are concerned with readings that occur when the sun is out. However this occurs on a gradient, because sunlight is defracted even after sunset. So for now, let's just fit the model on the unedited dataset. \n",
    "\n",
    "Another good idea is to create a **season** variable, which is a dummy variable for each season (Fall, Winter, Spring, Summer), as different amounts of sunlight occur during each. The problem is that, for different locations on earth, seasonality could be inversely correlated with sunlight and thus GHI, DHI, and DNI. \n",
    "\n",
    "To keep the model as general as possible, we will exclude seasonality. \n",
    "\n",
    "Another useful metric is **Pressure**. According to the renowned [DISC model](https://www.nrel.gov/docs/legosti/old/3087.pdf), air pressure is included as a proxy for *air mass*, which is the amount of atmosphere solar rays have to travel through. The larger the solar mass, the more solar rays are scattered. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get training arrays and combine them into a useable vector\n",
    "\n",
    "-Useful params:  \n",
    "-Hours  \n",
    "-Cloud Cover  \n",
    "-GHI  \n",
    "-Solar Zenith  \n",
    "-Pressure  \n",
    "\n",
    "\n",
    "Training Locations with Lat/Lon:  \n",
    "-**Reliance, South Dakota** (43.85, -99.50) from 2010-2018  \n",
    "-**Everglades, FL** (26.004157 ,-81.119239) from 2010-2018  \n",
    "-**Spokane, WA** (47.41,-117.52) from 2010-2018  \n",
    "-**Austin, TX** (30.581736,-98.024098) from 2010-2018  \n",
    "-**Charlottesville, VA** (38.0086,-78.4532) from 2010-2018  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please NOTE! In this notebook, the directory \"Testing_Data\" does not mean it is solely Testing Data. Rather for this Neural network, it contains both testing and training data. \n",
    "This is because the neural network requires much more data to train than regression. So much of the same data used to test the regression model is now being used to train the neural network. The True training data for the neural network is in \"Proudction_Test\", where the data is from USCRN and combined with data combed from darksky. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "def combine_training_arrays(df, input_arry, output_arr):\n",
    "    ghi  = df['GHI'].values\n",
    "    cloud_cover = df['Cloud Cover'].values\n",
    "    hours = df['Hour'].values\n",
    "    minutes = df['Minute'].values\n",
    "    solar_zenith = df['Solar Zenith Angle'].values\n",
    "    pressure = df['Pressure'].values\n",
    "    dhi = df['DHI'].values\n",
    "    #Sanity check, should all be 8760\n",
    "    assert len(pressure)==len(solar_zenith)==len(minutes)==len(hours)==len(cloud_cover)==len(ghi)==8760, \"len of input array not 8760\"\n",
    "    ar = np.array([ghi, cloud_cover, hours, minutes, solar_zenith, pressure]).T\n",
    "    input_arry = np.concatenate((input_arry, ar))\n",
    "    output_arr = np.concatenate((output_arr, np.array(dhi).T))\n",
    "    return input_arry, output_arr\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[]\n",
      "[]\n"
     ]
    }
   ],
   "source": [
    "# test_files = ['Murphey_ID', 'OldTown_ME', 'Quinault_WA', 'Lincoln_NE', 'SanDiego_CA', 'Boulder_CO', 'Miami_FL', 'outer_banks']\n",
    "training_files = ['psm_south_dakota', 'Everglades_FL', 'Spokane_WA', 'Austin_TX', 'Charlottesville']\n",
    "input_size = 6\n",
    "input_arry = np.array([], dtype=np.float64).reshape(0,input_size)\n",
    "output_arry = np.array([], dtype=np.float64)\n",
    "print(input_arry)\n",
    "print(output_arry)\n",
    "for directory in training_files:\n",
    "    for file in glob(_Testing_Data_Dir+'/'+directory+'/psm_*'):\n",
    "        df = pd.read_csv(file)\n",
    "        input_arry, output_arry = combine_training_arrays(df, input_arry, output_arry)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "630720\n"
     ]
    }
   ],
   "source": [
    "#sanity check, make sure 8760*number of files we look at == 429240\n",
    "count = 0\n",
    "for path in glob(_Testing_Data_Dir+'/*'):\n",
    "    for file in glob(path+'/psm_'+'*'):\n",
    "        count+=1\n",
    "print(count*8760)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# III. Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.optimizers import Adam"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "keras.backend.clear_session()\n",
    "tf.keras.backend.clear_session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Epoch size of 50 seems reasonable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function tensorflow.python.framework.ops.reset_default_graph()>"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras.backend.clear_session()\n",
    "tf.keras.backend.clear_session()\n",
    "tf.compat.v1.reset_default_graph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "keras.backend.clear_session()\n",
    "tf.keras.backend.clear_session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "#make RMSE a custom metric"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "# def rmse(y_true, y_pred):\n",
    "#     return keras.backend.sqrt(keras.backend.mean(keras.backend.square(y_pred - y_true), axis=-1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = keras.Sequential()\n",
    "model.add(layers.Dense(300, activation=\"relu\"))\n",
    "model.add(layers.Dense(200, activation=\"relu\"))\n",
    "model.add(layers.Dense(100, activation=\"relu\"))\n",
    "model.add(layers.Dense(50, activation=\"relu\"))\n",
    "model.add(layers.Dense(25, activation=\"relu\"))\n",
    "model.add(layers.Dense(1, activation='relu'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 1107.8455 - mae: 15.3400 - root_mean_squared_error: 33.2843\n",
      "Epoch 2/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 456.9354 - mae: 9.5543 - root_mean_squared_error: 21.3760\n",
      "Epoch 3/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 405.9811 - mae: 8.8250 - root_mean_squared_error: 20.1490\n",
      "Epoch 4/100\n",
      "6160/6160 [==============================] - 14s 2ms/step - loss: 378.2072 - mae: 8.4276 - root_mean_squared_error: 19.4475 1s - loss:\n",
      "Epoch 5/100\n",
      "6160/6160 [==============================] - 13s 2ms/step - loss: 353.8577 - mae: 8.1382 - root_mean_squared_error: 18.8111\n",
      "Epoch 6/100\n",
      "6160/6160 [==============================] - 14s 2ms/step - loss: 326.4309 - mae: 7.7684 - root_mean_squared_error: 18.0674\n",
      "Epoch 7/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 301.3426 - mae: 7.4581 - root_mean_squared_error: 17.3592 0s - loss: 301.6481 - mae: 7.4614 - root_mea\n",
      "Epoch 8/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 273.6204 - mae: 7.0872 - root_mean_squared_error: 16.5415\n",
      "Epoch 9/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 248.3218 - mae: 6.7101 - root_mean_squared_error: 15.7582\n",
      "Epoch 10/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 224.7443 - mae: 6.3570 - root_mean_squared_error: 14.9915\n",
      "Epoch 11/100\n",
      "6160/6160 [==============================] - 13s 2ms/step - loss: 200.2121 - mae: 5.9722 - root_mean_squared_error: 14.1496- ETA: 0s - loss: 200.4169 - mae: 5.9764 - root_mean_square\n",
      "Epoch 12/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 187.2664 - mae: 5.7774 - root_mean_squared_error: 13.6845\n",
      "Epoch 13/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 175.7904 - mae: 5.5879 - root_mean_squared_error: 13.2586\n",
      "Epoch 14/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 173.4033 - mae: 5.5502 - root_mean_squared_error: 13.1683\n",
      "Epoch 15/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 169.8874 - mae: 5.4889 - root_mean_squared_error: 13.0341\n",
      "Epoch 16/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 164.9232 - mae: 5.4290 - root_mean_squared_error: 12.8422\n",
      "Epoch 17/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 166.0395 - mae: 5.4441 - root_mean_squared_error: 12.8856 0s - loss: 164.0255 - mae: 5.4102 - root_mean_\n",
      "Epoch 18/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 158.5362 - mae: 5.3050 - root_mean_squared_error: 12.5911\n",
      "Epoch 19/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 158.4719 - mae: 5.3008 - root_mean_squared_error: 12.5886 0s - loss: 158.1717 - mae: 5.2979 - root_mean_squared_error:\n",
      "Epoch 20/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 159.4291 - mae: 5.3170 - root_mean_squared_error: 12.6265\n",
      "Epoch 21/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 152.0271 - mae: 5.2084 - root_mean_squared_error: 12.3299\n",
      "Epoch 22/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 152.9687 - mae: 5.2261 - root_mean_squared_error: 12.3681\n",
      "Epoch 23/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 152.1685 - mae: 5.1950 - root_mean_squared_error: 12.3357\n",
      "Epoch 24/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 156.8861 - mae: 5.2504 - root_mean_squared_error: 12.5254\n",
      "Epoch 25/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 149.3286 - mae: 5.1380 - root_mean_squared_error: 12.2200\n",
      "Epoch 26/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 145.8792 - mae: 5.0867 - root_mean_squared_error: 12.0780\n",
      "Epoch 27/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 148.7688 - mae: 5.1256 - root_mean_squared_error: 12.1971\n",
      "Epoch 28/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 148.3398 - mae: 5.1357 - root_mean_squared_error: 12.1795\n",
      "Epoch 29/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 142.2547 - mae: 5.0023 - root_mean_squared_error: 11.9271\n",
      "Epoch 30/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 146.1351 - mae: 5.0666 - root_mean_squared_error: 12.0886\n",
      "Epoch 31/100\n",
      "6160/6160 [==============================] - 13s 2ms/step - loss: 142.7227 - mae: 5.0120 - root_mean_squared_error: 11.9467\n",
      "Epoch 32/100\n",
      "6160/6160 [==============================] - 16s 3ms/step - loss: 144.6910 - mae: 5.0419 - root_mean_squared_error: 12.0288 6s - loss: 145.3943 - mae: 5.0468 - root_mean_sq - ETA: 5s - loss: 144.6010 - mae: - ETA: 3s - loss: 143.9703 - mae: 5.0 - ETA: 2s -\n",
      "Epoch 33/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 142.0057 - mae: 4.9949 - root_mean_squared_error: 11.9166\n",
      "Epoch 34/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 139.5785 - mae: 4.9552 - root_mean_squared_error: 11.8143 1s - loss: 139.2480 - mae: 4.9434 - root_mea - ETA: 0s - loss: 139.4857 - mae: 4.955\n",
      "Epoch 35/100\n",
      "6160/6160 [==============================] - 14s 2ms/step - loss: 136.6903 - mae: 4.8847 - root_mean_squared_error: 11.6915\n",
      "Epoch 36/100\n",
      "6160/6160 [==============================] - 18s 3ms/step - loss: 135.3864 - mae: 4.8763 - root_mean_squared_error: 11.6356 2s\n",
      "Epoch 37/100\n",
      "6160/6160 [==============================] - 17s 3ms/step - loss: 135.6652 - mae: 4.8785 - root_mean_squared_error: 11.6475\n",
      "Epoch 38/100\n",
      "6160/6160 [==============================] - 14s 2ms/step - loss: 136.5887 - mae: 4.8776 - root_mean_squared_error: 11.6871 0s - loss: 136.3231 - mae: 4.8731 - root_mean_squared_error: 11.67\n",
      "Epoch 39/100\n",
      "6160/6160 [==============================] - 17s 3ms/step - loss: 134.6510 - mae: 4.8432 - root_mean_squared_error: 11.6039\n",
      "Epoch 40/100\n",
      "6160/6160 [==============================] - 13s 2ms/step - loss: 134.5308 - mae: 4.8464 - root_mean_squared_error: 11.5987 0s - loss: 134.8410 - mae: 4.8553 - root_mea\n",
      "Epoch 41/100\n",
      "6160/6160 [==============================] - 17s 3ms/step - loss: 131.2647 - mae: 4.7876 - root_mean_squared_error: 11.4571 1s - loss: 1\n",
      "Epoch 42/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 135.1526 - mae: 4.8702 - root_mean_squared_error: 11.6255 5s - loss: 136.2885 -  - ETA: 4s - loss: 135.9948 - mae: 4.8901 \n",
      "Epoch 43/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 132.5027 - mae: 4.8050 - root_mean_squared_error: 11.5110\n",
      "Epoch 44/100\n",
      "6160/6160 [==============================] - 13s 2ms/step - loss: 132.0560 - mae: 4.7717 - root_mean_squared_error: 11.4916\n",
      "Epoch 45/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 129.6895 - mae: 4.7454 - root_mean_squared_error: 11.3881\n",
      "Epoch 46/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 129.9437 - mae: 4.7264 - root_mean_squared_error: 11.3993\n",
      "Epoch 47/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 131.0351 - mae: 4.7651 - root_mean_squared_error: 11.4471\n",
      "Epoch 48/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 129.5023 - mae: 4.7208 - root_mean_squared_error: 11.3799\n",
      "Epoch 49/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 127.6799 - mae: 4.6932 - root_mean_squared_error: 11.2996\n",
      "Epoch 50/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 127.4406 - mae: 4.6887 - root_mean_squared_error: 11.2890\n",
      "Epoch 51/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 129.2890 - mae: 4.7122 - root_mean_squared_error: 11.3705\n",
      "Epoch 52/100\n",
      "6160/6160 [==============================] - 15s 2ms/step - loss: 124.0813 - mae: 4.6224 - root_mean_squared_error: 11.1392\n",
      "Epoch 53/100\n",
      "6160/6160 [==============================] - 13s 2ms/step - loss: 126.2866 - mae: 4.6560 - root_mean_squared_error: 11.2377\n",
      "Epoch 54/100\n",
      "6160/6160 [==============================] - 13s 2ms/step - loss: 124.1810 - mae: 4.6134 - root_mean_squared_error: 11.1437\n",
      "Epoch 55/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 124.6207 - mae: 4.6221 - root_mean_squared_error: 11.1634\n",
      "Epoch 56/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 124.0046 - mae: 4.6102 - root_mean_squared_error: 11.1357\n",
      "Epoch 57/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 123.2662 - mae: 4.5903 - root_mean_squared_error: 11.1025\n",
      "Epoch 58/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 124.3217 - mae: 4.6009 - root_mean_squared_error: 11.1500\n",
      "Epoch 59/100\n",
      "6160/6160 [==============================] - 11s 2ms/step - loss: 123.9193 - mae: 4.6005 - root_mean_squared_error: 11.1319\n",
      "Epoch 60/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 121.7785 - mae: 4.5517 - root_mean_squared_error: 11.0353 0s - loss: 121.9563 - mae: 4.5526 - root_mean_squared_error: 1\n",
      "Epoch 61/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 122.2367 - mae: 4.5519 - root_mean_squared_error: 11.0561\n",
      "Epoch 62/100\n",
      "6160/6160 [==============================] - 13s 2ms/step - loss: 121.0191 - mae: 4.5336 - root_mean_squared_error: 11.0009\n",
      "Epoch 63/100\n",
      "6160/6160 [==============================] - 14s 2ms/step - loss: 122.9096 - mae: 4.5782 - root_mean_squared_error: 11.0865\n",
      "Epoch 64/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 120.0609 - mae: 4.5206 - root_mean_squared_error: 10.9572\n",
      "Epoch 65/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 121.5449 - mae: 4.5384 - root_mean_squared_error: 11.0247 2s - loss: 119.3333 - ma - ETA: 1s - l\n",
      "Epoch 66/100\n",
      "6160/6160 [==============================] - 16s 3ms/step - loss: 121.3320 - mae: 4.5178 - root_mean_squared_error: 11.0151\n",
      "Epoch 67/100\n",
      "6160/6160 [==============================] - 17s 3ms/step - loss: 119.3786 - mae: 4.5145 - root_mean_squared_error: 10.9261\n",
      "Epoch 68/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 119.2348 - mae: 4.4894 - root_mean_squared_error: 10.9195 0s - loss: 119.4422 - mae: 4.4949 - root_mean_square\n",
      "Epoch 69/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 119.4841 - mae: 4.4941 - root_mean_squared_error: 10.9309\n",
      "Epoch 70/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 120.0175 - mae: 4.4955 - root_mean_squared_error: 10.9553\n",
      "Epoch 71/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 121.2103 - mae: 4.5364 - root_mean_squared_error: 11.0096\n",
      "Epoch 72/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 117.5511 - mae: 4.4595 - root_mean_squared_error: 10.8421 3s - loss: 117.3406 - mae: 4.4554 - root_mean_squared_error: 1 - ETA: 3s - loss: 117.204\n",
      "Epoch 73/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 117.6796 - mae: 4.4720 - root_mean_squared_error: 10.8480\n",
      "Epoch 74/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 118.2105 - mae: 4.4574 - root_mean_squared_error: 10.8725\n",
      "Epoch 75/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 118.8501 - mae: 4.4857 - root_mean_squared_error: 10.9018\n",
      "Epoch 76/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 117.7499 - mae: 4.4553 - root_mean_squared_error: 10.8513\n",
      "Epoch 77/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 117.8975 - mae: 4.4565 - root_mean_squared_error: 10.8581 0s - loss: 118.3074 - mae: 4.4640 - root_mean_sq - ETA: 0s - loss: 118.0807 - mae: 4.4604 - root_mean_squared_error:\n",
      "Epoch 78/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 117.6116 - mae: 4.4550 - root_mean_squared_error: 10.8449\n",
      "Epoch 79/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 115.8296 - mae: 4.4034 - root_mean_squared_error: 10.7624\n",
      "Epoch 80/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 116.1805 - mae: 4.4159 - root_mean_squared_error: 10.7787\n",
      "Epoch 81/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 117.0807 - mae: 4.4426 - root_mean_squared_error: 10.8204\n",
      "Epoch 82/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 115.2257 - mae: 4.4058 - root_mean_squared_error: 10.7343\n",
      "Epoch 83/100\n",
      "6160/6160 [==============================] - 13s 2ms/step - loss: 114.9078 - mae: 4.3963 - root_mean_squared_error: 10.7195\n",
      "Epoch 84/100\n",
      "6160/6160 [==============================] - 13s 2ms/step - loss: 115.6726 - mae: 4.4022 - root_mean_squared_error: 10.7551 0s - loss: 115.5675 - mae: 4.3973 - root_mean_squared_error: - ETA: 0s - loss: 115.7754 - mae: 4.4023 - root_m\n",
      "Epoch 85/100\n",
      "6160/6160 [==============================] - 14s 2ms/step - loss: 113.7772 - mae: 4.3596 - root_mean_squared_error: 10.6666 5s - loss: 115.1967 - mae: 4.3\n",
      "Epoch 86/100\n",
      "6160/6160 [==============================] - 14s 2ms/step - loss: 113.8578 - mae: 4.3739 - root_mean_squared_error: 10.6704 1s - loss: 113.416\n",
      "Epoch 87/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 116.0841 - mae: 4.4127 - root_mean_squared_error: 10.7742 1s - loss: 1 - ETA: 0s - loss: 116.0235 - mae: 4.4120 - root_mean_sq\n",
      "Epoch 88/100\n",
      "6160/6160 [==============================] - 13s 2ms/step - loss: 115.2686 - mae: 4.4084 - root_mean_squared_error: 10.7363 0s - loss: 115.5021 - mae: 4.4100 - \n",
      "Epoch 89/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 114.7726 - mae: 4.3860 - root_mean_squared_error: 10.7132 1s - loss: 115.4241 \n",
      "Epoch 90/100\n",
      "6160/6160 [==============================] - 13s 2ms/step - loss: 115.6289 - mae: 4.3892 - root_mean_squared_error: 10.7531\n",
      "Epoch 91/100\n",
      "6160/6160 [==============================] - 23s 4ms/step - loss: 114.1965 - mae: 4.3562 - root_mean_squared_error: 10.6863\n",
      "Epoch 92/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 113.8384 - mae: 4.3592 - root_mean_squared_error: 10.6695\n",
      "Epoch 93/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 114.0011 - mae: 4.3523 - root_mean_squared_error: 10.6771\n",
      "Epoch 94/100\n",
      "6160/6160 [==============================] - 14s 2ms/step - loss: 112.4321 - mae: 4.3177 - root_mean_squared_error: 10.6034 6s - loss: 112.6974 \n",
      "Epoch 95/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 112.6650 - mae: 4.3286 - root_mean_squared_error: 10.6144\n",
      "Epoch 96/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 111.7894 - mae: 4.3066 - root_mean_squared_error: 10.5730\n",
      "Epoch 97/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 115.3291 - mae: 4.3786 - root_mean_squared_error: 10.7391 1s - - ETA: 0s - loss: 115.4801 - mae: 4.3857 - root_mean_sq\n",
      "Epoch 98/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 112.1944 - mae: 4.3053 - root_mean_squared_error: 10.5922 0s - loss: 112.5637 - mae: 4.3111 - root_mean_squared_\n",
      "Epoch 99/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 114.1220 - mae: 4.3481 - root_mean_squared_error: 10.6828\n",
      "Epoch 100/100\n",
      "6160/6160 [==============================] - 12s 2ms/step - loss: 110.4760 - mae: 4.2719 - root_mean_squared_error: 10.5108\n"
     ]
    }
   ],
   "source": [
    "#Compile and train\n",
    "model.compile(optimizer=Adam(lr=0.0001), \n",
    "              loss='mse',\n",
    "              metrics=['mae',keras.metrics.RootMeanSquaredError()])\n",
    "\n",
    "history = model.fit(input_arry, output_arry, epochs=100, batch_size=64)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense (Dense)                multiple                  2100      \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              multiple                  60200     \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              multiple                  20100     \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              multiple                  5050      \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              multiple                  1275      \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              multiple                  26        \n",
      "=================================================================\n",
      "Total params: 88,751\n",
      "Trainable params: 88,751\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAESCAYAAAAVLtXjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAIABJREFUeJzt3Xl8VNX9//HXvXf2JJPJxi5hVQFBQcQFhLqiVqoWV9pIxa1KVepXyyKICkUFW62olNpW+kOtWpdK61IRC4gsVlEQVAQMq2zZk9nn3vP7Y0IECRAgk0lmPs/Hoy1JZu75nEn6vueee++5mlJKIYQQIq3oyS5ACCFE05PwF0KINCThL4QQaUjCXwgh0pCEvxBCpCEJfyGESEMS/iJlvfvuuxQVFSW7jJTw+uuvc+uttya7DNGIJPyFECIN2ZJdgGg5VqxYwe9//3tatWrF+vXrcbvd3HHHHcydO5fi4mIuvPBCJkyYgGVZTJs2jVWrVuH3+1FKMXXqVE499VQikQiPPfYY//vf/zBNk549ezJx4kQyMzP3a6ukpIT777+f0tJS9uzZQ/v27XniiSfIy8vj3HPP5YorrmDZsmXs2LGDiy++mN/85jcA/OEPf+Bf//oXPp+PwsLCg/bjt7/9LR6Ph0AgwL333stTTz112H75/X7Gjx/P5s2b0XWdXr168dBDD6HrOh988AGzZs0iGo3icrkYO3Ysffv23a/dbdu2UVRUxIABA/j6669RSnH//ffTv39/AGbNmsV7772HZVm0b9+eyZMn07p1a4qKisjOzubbb7/luuuuO+Bo5h//+Ad///vfsSwLn8/HpEmT6Nq1K+PGjUPTNDZu3EhZWRkDBw5k4sSJ2O12PvnkE6ZPn04wGMRutzNmzBgGDx4MwOzZs3njjTew2WwUFhbyyCOPALBnzx5uueUWduzYgWEY/O53v6Nr167H/oclkkMJ0UDLly9XPXr0UGvXrlVKKXXjjTeqa665RoXDYVVaWqp69eqldu7cqVauXKnuuOMOZZqmUkqp2bNnq1tvvVUppdTMmTPVI488oizLUkop9bvf/U5Nnjz5gLbmzJmjZs+erZRSyrIsddNNN6m//OUvSimlzjnnHPXII48opZTauXOn6t27t9qyZYuaP3++uuSSS1R1dbWKRqPqlltuUT//+c/r7ceJJ56otm3bdkT9euONN9SoUaOUUkrFYjF13333qU2bNqni4mJ16aWXqrKyMqWUUt98840aOHCg8vv9+7W7detWdfzxx6t58+YppZRauHChGjhwoIpEIuqNN95QY8aMUdFoVCml1EsvvaRuuukmpZRSP//5z9X48ePr/Z2sWLFCjRgxQgUCAaWUUh9++KG6+OKLlVJKjR07Vl1++eWqpqZGhcNh9bOf/UzNnTtXlZWVqTPPPFN9/vnndfUOGDBAbdmyRb3//vvqwgsvVBUVFUoppaZNm6aeeeYZ9dprr6n+/furTZs2KaWUmjJlykFrEi2DjPzFEenQoQM9e/YEoGPHjmRlZeFwOMjNzSUjI4PKykr69u1LdnY2L730Elu3bmXFihVkZGQAsHDhQqqrq1m6dCkA0WiUvLy8A9oZOXIkn3zyCc899xybNm1i/fr1nHzyyXU/P++88wBo3bo1eXl5VFZWsmzZMi644IK6o4jhw4czd+7cevvRtm1b2rdvf0T9OvXUU3n88ccpKirirLPOYuTIkRQWFvLCCy+we/dufvGLX9RtT9M0tmzZwoknnrhfu9nZ2QwbNgyAIUOGYBgG69at47///S9ffPEFw4cPB8CyLILBYN379h4d/NDChQvZvHkz1157bd33KisrqaioAOCKK66o++wvu+wyFixYwHHHHUfHjh3rPs/u3bvTr18/Pv74Y7766isuuugisrOzARg/fjwQn/Pv06dP3dFUjx49mD9/fr01iZZBwl8cEYfDsd/XNtuBf0ILFy7kt7/9LTfccAPnnXceXbp0Yd68eUA81CZMmMCQIUMA8Pv9hMPhA7YxY8YMVq9ezfDhwzn99NOJxWKofZahcjqddf/WNA2lVN3/7mUYxkH74fF4jrhfxx13HPPnz2fFihUsX76cG264gYkTJ2JZFmeeeSZPPPFE3Wt37NhBq1atDtjGD2uyLAvDMLAsi5tuuokRI0YAEIlEqKysPGi9+77/sssu49577637evfu3XXhvW97Sil0XceyrAO2o5QiFothGAaaptV9v6qqiqqqqgM+kx9+1qLlkRO+otF99NFHnHPOOYwYMYLevXvz/vvvY5omAIMGDeKFF14gEolgWRaTJk3i97///QHbWLJkCSNHjuTyyy8nLy+PpUuX1m3jYM4++2zeffddqqqqsCyLN998s1H79eKLLzJ+/HgGDRrEvffey6BBg1i/fj1nnHEGH330ERs3bgRg0aJF/OQnP6l3p1ZWVsbixYsB+OCDD7Db7Rx//PEMGjSIV199lZqaGiB+7mLveYxDGThwIG+99Ra7d+8G4O9//zsjR46s+/k777xDJBIhHA7zxhtvcM4553DyySdTXFzM6tWrAVi/fj3/+9//GDBgAGeddRbz58+vq2PmzJnMmTPn6D800WzJyF80umuvvZZ77rmHYcOGYRgG/fv3rzuRefvtt/Poo49yxRVXYJomPXr0YNy4cQdsY/To0UyfPp1nnnkGwzDo168fW7ZsOWS7Q4YMYd26dQwfPhyv18uJJ55IeXl5o/Xr8ssv5+OPP+aSSy7B7XbTrl07rr/+erKzs3nooYe4++67UUphs9mYNWtWvaN1p9PJm2++yWOPPYbL5eLpp5/GMAyuuuoqdu3axdVXX42mabRt27buROuhnH322dx8882MGjUKTdPIzMzkqaeeqhu9u1wuRowYQVVVFUOHDmX48OHous4f/vAHpkyZQigUQtM0Hn74YTp37kznzp3ZsGED1113HQDdunVjypQpvPfee432OYrmQVNy7CZEk9i2bRvDhg3js88+a5L2xo0bR/fu3bnxxhubpD3Rssi0jxBCpCEZ+QshRBqSkb8QQqQhCX8hhEhDzfZqH8uyMM2jn5EyDO2Y3t8SpWOfIT37LX1OH0fab7v94Pe37KvZhr9pKioqAkf9fp/Pc0zvb4nSsc+Qnv2WPqePI+13QUFWg14n0z5CCJGGJPyFECINSfgLIUQaarZz/vUxzRjl5XuIxSKHfe2uXS134SmbzUFOTgGG0aJ+PUKIFqRFpUt5+R5cLg8ZGW32W3mwPoahY5oHrl7Y3Cml8PurKC/fQ35+22SXI4RIUS1q2icWi5CR4T1s8LdkmqaRkeFt0NGNEEIcrRYV/kBKB/9e6dBHIURytbjwPyyl0IJl0ELn+4UQoimkXvjHQhjVWyFcnexKhBCi2Uq98N87ZaIO/dQnIYRIZy3qap99vbV2F/PW7DzwB8pCiwXA2IDSj6x7PzmpDT/u1fqQr3n77X/x0UeLCYfDlJaWcNVV1/Hhh4soLt7I6NF3sXv3LhYt+i/BYBCfz8e0aY+haRozZkxj27atWJbFzTffRr9+9T+QWwghmkKLDf+D23uyNHFz/oFAgMcff5r33/8PL7/8In/60xw+++xTXn75BU44oQdPPPEMuq5z992/4quv1rJx4ways32MH38/lZUVjB59C88//0rC6hNCiMNpseH/416t6x+lWya2kjWorHaY7oKEtN29+wkAZGZm0alTZzRNIysri2g0ht1u54EH7sPtdrN7925isRgbN25g9erP+PLLNUD8ZrWKigp8Pl9C6hNCiMNpseF/UFrtaYwEXu1zsEsxY7Eoixcv5Nln/0YoFOLGG38OQGFhJ1q1asX1148iHA7xt7/9Fa/Xm7D6hBDicFIw/DVAB9X0d/cahoHb7ea220YBkJeXT0nJHi677Kc8+uhUfvWrW/D7a7jiiqvQ9dQ71y6EaDma7TN8o1HzgDWsd+7cTJs2hYd9r7FnDXhyMTPaJaq8hGtoX/cl652nD+lz+pD1/I+ElpyRvxBCtBQS/kIIkYZSNPw1Wd5BCCEOITXDP0knfIUQoqVIzfCXaR8hhDiklAx/JeEvhBCHlJLhj6ahSfgLIcRBpWj4y8hfCCEOpcXe4ev8+lVcX71U7880MwyWibJ7jmiboR7XEj7xykO+5u23/8Vbb83DsixKSvbQp88pbN26hVNPPQ2/v4avvlpLx46FTJo0hUWLPuD55/+GzWYjP7+ABx+cRiAQ4JFHHqKyshKAMWPupWvXbkdUpxBCHKsWG/6HppHIVT2zsrJ45JHfM2TI6dx88+3k5+dz8cXn8qc/zeHXv/4NV199GdXV1cyf/x9GjCjinHPO5513/o3f72fu3Oc49dQBXHHFlWzduoVp0x5k1qy/JKxWIYSoT4sN//CJVx50lK7X7EAP7CFW0Pv7h7s0oo4d48sueL3ZtGnTBgC3203nzl0AyMjIJBIJc8cdv2bu3Dm89torFBZ2YvDgH/HttxtYufITFix4D4Dq6qpGr08IIQ6nxYb/IWk68ZG/4vv1/Rtx87Urhx7uQevz5r3BjTfeQk5OLtOn/5bFixdSWNiJCy/syYUXXkR5eRn/+tc/G70+IYQ4nISEv2maTJw4keLiYjRN48EHH8TpdDJu3Dg0TaN79+5Mnjw5YStbqn2XdW787G+wHj168ZvfjMHjycDtdnPWWYM466xBPPLIFObNe51AwM+oUbckr0AhRNpKyKqe77//PgsWLODhhx9mxYoVzJkzB6UUN9xwA6effjr3338/Z599NhdccMFBt3Esq3pqwVKM6m3E8nqCYT/m/iSDrOrZcOnYb+lz+mhRq3qef/75TJkyBYDvvvsOr9fL2rVrGTBgAACDBw9m6dKliWi61t6HuMvlnkIIUZ+EzfnbbDbGjh3L/PnzefLJJ/noo4/q5sgzMjKorq4+5PsNQ8Pn2/9SzV27NAyjAfsrI94tQ9/7Xy2Pph3Y/8MxDP2I35MK0rHf0uf0kah+J/SE76OPPso999zD1VdfTTgcrvu+3+8/7GMMTVMdcKijlCIWMw97olVTYBA/94De8kb/SimUOrD/hyOHxelD+pw+WtS0zz//+U9mz54NxC+B1DSNk046iRUrVgCwePFi+vfvf8Tbtdkc+P1VHPY0xd6rcVrgtI9SCr+/CpvNkexShBApLCEnfAOBAOPHj6ekpIRYLMbNN99M165dmTRpEtFolC5dujB16lQMwzjoNuo74WuaMcrL9xCLRQ5dgBlB9+/G8uSDzdUYXWpSNpuDnJwCDOPIDsxkZJQ+pM/pI1Ej/xb1DN+GMkq+JPflC6m8aDaRrj9u5MqaL/k/R/qQPqePFjXtk2zK5gZAiwWTXIkQQjRPKRn+2GvDPxpKciFCCNE8pWT41438TQl/IYSoT4qGf/wkrxaVaR8hhKhPSoY/uiO+vo/M+QshRL1SM/w1DeweOeErhBAHkZrhD2BzocVkzl8IIeqTuuFvd8vIXwghDkLCXwgh0lDqhr/NDTLtI4QQ9UrZ8Fcy8hdCiINK2fCXaR8hhDi41A1/m0eu9hFCiINI3fC3u+QmLyGEOIjUDX+bTPsIIcTBpGz4K7tM+wghxMGkbPhjlzt8hRDiYFI3/G1uNDMMlpnsSoQQotlJ3fCvfaCL3OglhBAHSt3wlwe6CCHEQaVs+Ku6RznKFT9CCPFDKRv+dc/xlcs9hRDiACkc/h5Awl8IIeqTuuFf+xxfOeErhBAHSt3wl2kfIYQ4qJQNfyXTPkIIcVC2xt5gNBplwoQJbN++nUgkwm233Ubbtm259dZb6dSpEwDXXXcdl1xySWM3vb/aaR+5y1cIIQ7U6OE/b948fD4fM2bMoKKigssvv5zRo0dzww03MGrUqMZu7uBk5C+EEAfV6OF/0UUXMXToUACUUhiGwZo1ayguLmbBggUUFhYyYcIEMjMzG7vp/dXd4SvhL4QQP6QppVQiNlxTU8Ntt93G1VdfTSQS4YQTTuCkk05i1qxZVFVVMXbs2EO+37IsTPPoSzPMIPoj7THPfQDrzDuPejstiWHomKaV7DKaXDr2W/qcPo6033a70aDXNfrIH2DHjh2MHj2aESNGMGzYMKqqqvB6vQBccMEFTJky5bDbME1FRUXgqGvwZbvQgVB1JYFj2E5L4vN5jukza6nSsd/S5/RxpP0uKMhq0Osa/WqfkpISRo0axb333suVV14JwI033sjq1asBWLZsGb169WrsZg+k6SjDKXP+QghRj0Yf+f/xj3+kqqqKZ555hmeeeQaAcePGMW3aNOx2O/n5+Q0a+TcGJU/zEkKIeiVszv9YRaPmsU37+DzoT/Qk0nEINef+rhEra77ksDh9SJ/TR4uZ9mlO4iN/uc5fCCF+KKXDH5tblnQWQoh6pHT4K5tLHuYihBD1SPHwlxO+QghRn9QOf7sbZNpHCCEOkNrhb8i0jxBC1Celwx+7nPAVQoj6pHT4y5y/EELUT8JfCCHSUIqHvyv+DN/meROzEEIkTYqHvxsNBWY42aUIIUSzktLh//2jHGXqRwgh9pXS4a/kOb5CCFGvFA//+KMcZeQvhBD7S4vwR0b+Qgixn5QOf5nzF0KI+qV0+H8/7SMjfyGE2FeahL+M/IUQYl/pEf6yvo8QQuwnpcPfcuUAoFdtTnIlQgjRvKR0+KuMVkTb9Me17nVZ4kEIIfaR0uEPEDrhSmzl32ArWZPsUoQQotlI+fAPd7sUpTtwfv1qsksRQohmI+XDX7l8RDpfgGv9P8GMJrscIYRoFlI+/CE+9aMHS3FsXZTsUoQQollIi/CPdPwRlisX19f/SHYpQgjRLNgae4PRaJQJEyawfft2IpEIt912G926dWPcuHFomkb37t2ZPHkyut6E+x3DTuj4y3GveR4tVIFy+ZqubSGEaIYaPYHnzZuHz+fjxRdf5M9//jNTpkzh4YcfZsyYMbz44osopViwYEFjN3tY4ROuRLMiuL56qcnbFkKI5qZB4b98+fIGb/Ciiy7irrvuAkAphWEYrF27lgEDBgAwePBgli5dehSlHptYQW/CHc8h4+Pfo1dta/L2hRCiOWnQtM/MmTM544wzGrTBjIwMAGpqarjzzjsZM2YMjz76KJqm1f28urr6sNsxDA2fz9OgNut/v37g+3/yBMw+i5yPJmBe+w+orSlV1NvnNJCO/ZY+p49E9btB4a9pGqNHj6Zz5851c/V33333QV+/Y8cORo8ezYgRIxg2bBgzZsyo+5nf78fr9R62TdNUVFQEGlJevXw+Tz3vz8N1xliyPryfmo+fJ3zC8KPefnNUf59TXzr2W/qcPo603wUFWQ16XYPCf/jwhodkSUkJo0aN4v777+fMM88EoGfPnqxYsYLTTz+dxYsXN/goIhFCJ43Etf5NMpc8QKTjj1DuvKTVIoQQydKgOf9hw4YRCARYvXo1VVVV/PjHPz7oa//4xz9SVVXFM888Q1FREUVFRYwZM4aZM2dyzTXXEI1GGTp0aKN14IjpBtXnzECL1JC1aLys+SOESEuaUodPvwkTJuD1eunfvz8ff/wxFRUVTJ8+PaGFRaNmAqZ9vude+QyZy6ZRde7vCPe45qjbaU7ksDh9SJ/TR1KnfTZv3swLL7wAwPnnn8+1117b4EKaq2DfX+LYspCsxZOIth2A5euc7JKEEKLJNGjaJxwOEwzGH4gSCoUwTTOhRTUJTaf6vCdQhh3v/Dtk3R8hRFpp0Mh/5MiRXHbZZXTv3p0NGzZw5513JrquJmFltaP6R4+S/Z9f4vl0JoEBB7+CSQghUkmDwr+goIBXXnmFrVu30qFDB3JychJdV5OJdLuU0LeX4Vn5NKHjr5DpHyFEWmjQtM/MmTPx+Xz07t07pYJ/L//A+1G6nawPJ8nVP0KItJCQm7xaGiujNYHT7yFzyQM4vn2HSNdLkl2SEEIkVIPC//LLL8cwjETXklTB3r/A9dUrZC6ZTNlxQ8CRkeyShBAiYRoU/m+//TZ//etfE11Lcuk2qodMI+f1y8n45HH8Z01MdkVCCJEwDZrz93q9vP/++2zcuJHi4mKKi4sTXVdSxNr2J9jjWtyfP4ttjzzwXQiRuho08i8tLeX//b//x96bgSORCC+//HJCC0sW/1n34dy0gMz//oaKK+eB3ujPuxFCiKQ75Mh/zJgxAMydO5chQ4Ywd+5c5s6di8PhaJLikkG5cqg5+yHse1bjXp3iU11CiLR1yPAvLS2t+/eiRd8//FxLsXXwfyjc7VLCnc4nY8UM9KotyS5HCCEaXYMf49iA9d9Sh6ZRM3gaStPJXDwp2dUIIUSjO2T47zvCT/XR/g9ZWe0InPZrnJsXYN+6JNnlCCFEozrk2cwNGzbwf//3fyil9vv3xo0bm6q+pAr2/gXuL+aQsXQqFVe/DVqjP+9eCCGS4pDh/8QTT9T9e99lnFNhSecGsbnwnzEW7/w7cH7zOuETrkx2RUII0SgOGf4DBgxoqjqarXD3y4h+/iwZy6cT7vpjsLmTXZIQQhwzmcc4HE3HP3AiRs13uFf9JdnVCCFEo5Dwb4Bo+7MId7oAz6dPoQX2JLscIYQ4ZhL+DeQfOAnNDJOx/NFklyKEEMdMwr+BTF8Xgn1G4frqZWx7vkh2OUIIcUwk/I9AoP9dKHcumR9Oloe+CCFaNAn/I6CcXvyn34t9x8c4N/w72eUIIcRRk/A/QqEe1xHL60nG0qkQDSa7HCGEOCoS/kdKN6gZPAWjZjuelU8nuxohhDgqKRn+O6pCCd1+tN3phLpfhuezWbLqpxCiRUpY+K9atYqioiIAvvzyS84++2yKioooKiri7bffTlSzbK8M8pNnP+aTzeUJawPiD31B08lc8mBC2xFCiERIyGOqnn32WebNm4fbHV8KYe3atdxwww2MGjUqEc3tZ+9FOFvKAnTLdiasHSuzHf7+d5G5/BHsWxYR7TgkYW0JIURjS8jIv2PHjsycObPu6zVr1rBw4UJ+9rOfMWHCBGpqahLRLAA+tx2A8kAkYW3sFTzlZkxvIZlLJoMVS3h7QgjRWBIy8h86dCjbtm2r+7pPnz5cddVVnHTSScyaNYunn36asWPHHnIbhqHh83mOuO1spbAbGhXB6FG9/8h4UBf+FturPydn8xuoviMT3N6hGYbeBH1uftKx39Ln9JGofjfJ08kvuOACvF5v3b+nTJly2PeYpqKiInBU7WW77JTVRI76/Uek1RB8bfqjL3yE8g6Xgj15q376fJ6m6XMzk479lj6njyPtd0FBVoNe1yRX+9x4442sXr0agGXLltGrV6+Etudz25tk2gcATcN/5niMwC7cq2XVTyFEy9AkI/8HHniAKVOmYLfbyc/Pb9DI/1j43DbKA9GEtrGvaLvTCReeh2flM4R6/QzlymmytoUQ4mgkLPw7dOjAK6+8AkCvXr146aWXEtXUAXxuO8XlTXv3rf/MceS8dCGelU/jP2tik7YthBBHKiVv8sp225t05A9g5vUgfMJw3Kv+in3bR03athBCHKmUDH+f205FIILVxCtv1gyajOnrTPZbv8D23cdN2rYQQhyJlA1/S0F1qGmvvVeuHCouewkzsx3Z/74e286VTdq+EEI0VMqGP0BFsGmnfgCUp4DKy15CufPI/tfPse38tMlrEEKIw0nR8I+fx05G+ANYmW2puPwVLHcuvjevw751SVLqEEKIg0nR8N878k/ekgtWVnsqrngd03sc2W+NxFH8XtJqEUKIH2qS6/yb2t7wr0zSyH8vldGKiiteJftfRWS/PQrT25FYQW+irfoQOe5HmPk9QdOSWqMQIj2ldPgna9pnX8qVQ+VlL+Fa+zy2XZ9j3/MFzo1vwbKHMTPaEOl0PqEe1xBr3TfZpQoh0khKhr/LbuCy680i/AGUI5Ng31/Wfa0F9uDY/F+cm9/Hte513GufJ9rqZIInjUQ5s9ADe9CDZYQ7Xxg/OhBCiEaWkuEPkONxNJvw/yHlKSDc42rCPa5Gi9Tg/PofuL+Yg/eDu/d7neeTJwj0vZ3AaXeBkbhnEwgh0o+Ef5IpRyahPjcQ6j0S2+5VoNuwPK1Quo3MpVPJ+PRJnN++Q7jLxXXnB7SoHz1Yih4sI1bQG3//uxq2mujem97kPIMQaS+Fw99ORRMv8XBMNP2Aef/q8x4n1O0nZC2eiGflU6AUGgpl82B58rEcXjwrn8Kx8d/UnPsY+M6tf9vRAK5vXsf9xd/Qa3bgP2sioR7XyE5AiDSWwuHvYGtZy1/7O1p4DmVFB18ryL7tI7L+ey++N67EWnYKuYFytGAJmhXDcvpQLh96zQ70SBWxvJ6Yud3J+u89OL59h5pzpmNltG7C3gghmosUDn97i5j2OVbRDgMpu/Z9Mv73OK6Kr4h6O2O580C3oYUr0UMVxFr1IdhzBLE2/QGFe/VzZCx/mJwXBhMpPI9Il6FEW/fDvutz7Ns+jE8/aQbK7kY5vIR6/ZxI4bn7HynEgmC45OhBiBYqhcPfQVUoRsxS2PQUDyi7B/9Z92H3eag+7BN/NIIn30ik8Bzcn/0RZ/F7uDbMq/up5fASa9MXpRlosSC2krVkvzWSSIdB+E+/F6NqC851r+PYuhgrow3hLkOJdB4KysRW8iW2snVYrlyi7c8k2vY0lDM7sX0XQhyV1A3/jPi1/lWhKLkeR5KraX5MXxdqzplOzZCHse1aiW3PF8RanUKsVR/Q9/mzMCO418zF87/HyXntsvi3MtsT7DMKo3IT7rUv4Fn91+9f7mmNHq7A8/lsFBqx/J7E2vQn2uZUou3OwMpq19RdFULUI3XDvzbwK4IS/oekG8Tankas7Wn1/9xwEDz5RkInDMe5/p+YeScSbTsAtNqVQSJ+HN8tQ9ncxPJ7xp9iFgth3/UZ9u3LsO/8BOe613Cv+RsAMV9XosedTbRNf0zvcZhZx8XfY0bQrAhaNIAeLEEPlIAVwcw9AdNbCLrRRB+IEOkhLcJfHDvl8hHq/YsDf+DIINLp/P2/Z3PFp33anxn/2jIxytbh2L4U+5ZFuL56CfcXcxrets1NzNcFDEftuQgPkQ6DiHS5CDO7M9r2T8n85HkcmxdgZnci2v4sIu3PxMpog7J7UHYP2BpwKawQaSSFwz/5i7vpwbf2AAAZIElEQVSJWrqBmd+TYH5PgiffBGYYo6IYo3obevU29FAFynCA4UDZPVjufCx3PugGRunX2Eq+xKgsRrNMUCZ6YDeZy6bBsmlYrlz0UBmG4SRy3BCM6q1krJhOxg9KMLOOI9q2P9E2/eNHGlYEzYygBcsw/DvR/btQhgPT1wUzuzNmbndiOd3B5qq3S1qwDM0MYWXKNJZomVI3/DNk5N9sGU7MvBMx80487EtjrU4mXM/39ertOIr/g33np9i6/4iKdkNRTi8QD2b7zk/QQuXxaaRINbaStdi3LcX1zRsHbMtyeLEy2qCZIZzr56ERvxlOaTqmrwtWVgeU4UQZDvRINUbpVxj+XQCEug0jcNrdmLndQSn0mu8wKjcRyz0B5ck/+s9IiARL3fBvJit7isSwstoT6jOKUJ9R+Hwe1D5XOSl3LpHOFx74ptpw1qIBlGEH3Y7l9IFjn+OEWAijcjNG+XpspV9hK/06fp+EFQUrijJcRDucTTCvB3qoHNcXz+Hc+BbRtgPiRzOBXXWbMjPbE8vvCcpCiwXQYuH4NJTTi+XwopzZWK4clNOHFqlEr9mJEdgNKJTNjbJ54ivBtulLtKAP2D3xu7TNEFg/OI9lmRiVxWBG4jtVLSVXaxeNKGXD32k38NgNGfmL72kaVlb7Q7/G5sLMOwEz7wQi3S497CYDp9yC57NncGxZRLTDQAKt+2L6OmMrXYdt9ypsZetQuh3sHpQjM34k4t+NLRK/B0Mzvz+usRxZtTfd6WixYHwZj1AZAEozUDZ3fCeiLJSmk5vRBjPrODQVw1byJVosGN+Ou4BI4Y+I5Z6AUbUFo+JbdP8u0A2UbkfZM4jl9SBW0Bszv0d8R6MZoOloVgzMcHxKLOpHi9SgRaowanbGp+j8O4nl9yJ84pWYvi6H/8xlSZFmS1OqiZ9y3kDRqEnFYa9ZPzifz8OQGf/llA7ZPHjx4acXUoHP5zmmz6ylatH9jgbRwxVYDu/+RyC1tGAp9l2fY9v1GVrUHz9ysLlx22JE92xCr94KaMTyexHL7wWahmPLQhxbFtVtNz511Q6UBWYUPVyBreQrtNiRfWaWKxfLU4BRvh5NWUTbnEospxuassCKocVCtTuLarRINXq4Ei1cBbqOmdkOK6MtypUNteduUApsTpTuQDmzieWdEL8L3dcZzYpCLBTfWQb2oAf3kKGHqHZ2IJZ/0sGn1JQ67I5GC1dBLIRy5+5/WXMzdaR/3wUFWQ16XfPv+THIdqfHXb6iBbO7sQ6xKJ9y5xHpdB6RTuft933nIW7oC594FVgmWqQK5fTVH4a100RG+Xo0MwJWLL5z0O21U2IOlCMD5cjCsmfGj0jsHgB0/06c617H9c0/cWxZBJoRP6qwuVGOTJQrG9N7HMqZHT8PY8bQ/Tswanagl60H3YbSbfG1qqxofKcRKscdqTrsx+XbW767AMuVjTJcYNjRIjXooTK0UAVWVntiBScRy++NVbuz0ZSJUbYe+85PsJWt+/5jcPqwXD6UPRPlyEBTCi2wBz1YCsqMn/z3dcHKah/f8Rqu+FVnygQrCpqB6etCLKc7lrdjfLrNisZ/vs8d8EbFtzjXvYaz+D+Yvi6Eu1wS/51aJrY9X2ArWYuZXUik8Lz49ptASoe/z22Xq31EetKN+FVNh/i5mdMNM6fbEW/aymhDsN/tBPvdfgwF/oBS6NXbsZV+iVG1BWW4UDYXyu6OB72nAG9+Pv5Nq7GVrMUoXRefljLDaGYYK7MtUVceyulFr9qCbc8XODe+vX/dDi+xNv3wd/8JltNXuzJuKVq4om6KS+k6ZkFvrNojC6OiGPvuVejfvotmRQ7dBU2PHwXt/dpwxlfotbmwla9HaTrRtgOw7fgU58a3UbotPs22b43uPELH/5RA39tQGa0a6cOtX8qH/+byYLLLEEIcjqZheTsQ8XY4+GsyPUQ7DCTaYWDDNhmpia9Bpemg6fGlRo7lRLhlxqeirEj8fhPdjmaGMSq+xShfj1G1Jb593YHSNPRQOXqwBC1UQajHNYSPvyJ+BKUsbDs/xbnpfSxHFrFWfYjl9cS+e1XdPTDK5iJwxtijr7UBEhb+q1at4rHHHmPu3Lls3ryZcePGoWka3bt3Z/Lkyeh64q9G8LntcrWPEGlKOTLBkdl4G9QNcGSg9rmLRNndxNr0I9amX8O3o+n13lVfN70X8R/0/pLGlJAEfvbZZ5k4cSLhcPxKhocffpgxY8bw4osvopRiwYIFiWj2AD63HX/EJBKzDv9iIYRoDhwZTbKcSULCv2PHjsycObPu67Vr1zJgwAAABg8ezNKlSxPR7AF87viBTWVIRv9CCLGvhEz7DB06lG3bttV9rZRCqz3rnZGRQXV19WG3YRgaPp/nqGswDJ12+fFDPtNmO6ZttRSGoadFP38oHfstfU4fiep3k5zw3Xd+3+/34/V6D/se01THfJ2/3YpP92zdXUUbV+qvCtmir3c/BunYb+lz+kjUdf5Ncg94z549WbFiBQCLFy+mf//+TdEsvtolHnZXH/oSLSGESDdNEv5jx45l5syZXHPNNUSjUYYOHdoUzXKcz007r5O/fbyVqCknfYUQYq+UXt6hoiLAkm9L+fUba7l9UCduOL1jI1bY/MhhcfqQPqePFj3tk0yDuuRxbvd8/rJ8C9sq5IYvIYSANAh/gLvP6Yqhacz4YAPN9EBHCCGaVFqEf+ssJ7cOLGRpcTn/+HxHsssRQoikS4vwB7i6b3sGds5lxgcbmPXRJjkCEEKktbQJf5uu8dhlPbnspDb8dfkWHnh3nVwBJIRIWym9qucP2Qyd+y7sThuvk9lLN7OxJMDEC7tzYuuGnR0XQohUkTYj/700TeOmMwuZ/pOelPgj/OKFz3hy0beEomaySxNCiCaTduG/1znd83nlF6cy7KQ2zP1kG6P+/jnfVYaSXZYQQjSJtA1/AK/Lzn0XHs8TPz2JHVUhRr7wGSu3VSS7LCGESLi0Dv+9BnbO5bkRfcl22bj9H1/w8srtcjWQECKlSfjX6pTr4bkRfTmzUw6P/Xcjd7z2Bbuqw8kuSwghEkLCfx9ZLhu/v7wX487vxqrtVVz3t095f92eZJclhBCNTsL/BzRNY/jJ7Xjx+lPplOtm/L+/4pklxVgyDSSESCES/gdxXI6b2deczOW92/Dciq3c88+11IRjyS5LCCEahYT/IdgNnQkXdOfec7uytLiMG//+OdsrZWVQIUTLJ+F/GJqmcXXf9jw5vDd7aiL84oXP+XxbZbLLEkKIYyLh30ADCnN4bsQpeF02bn91Nf9aszPZJQkhxFGT8D8ChbkenhtxCn3bZ/PQf77h0ffXy+JwQogWScL/CHlddv4wvDfXn9aBV1ft4NaXV8n9AEKIFkfC/yjYdI07Bnfh0WE92FgS4PrnV7JmR1WyyxJCiAaT8D8G5x5fwJyf9cVtN/jlK6vlhjAhRIsh4X+MOud5mDOiLye2ymT8v7/i2aWbZXloIUSzJ+HfCHweO09f1YeLerTiT8s2c+mfVjBzcTE7q2SJaCFE85RWT/JKJKdN56GLT+Cnfdry0srtPP/JVp7/ZCuDu+Zx1SntOK2jD03Tkl2mEEIAEv6NStM0+nbIpm+HbHZUhXj18x28+cUOFm4opX22i+Ny3OR67OS4HbTLdtHe56JDtou2XhcOmxyECSGajoR/grT1urhjcGduOauQ+et2s+CbEkr9ETaXBSgLRAnHvr8/QANaZzlpm+1CAyKmRSRmcXyrTIZ0zeP0Tjm47UbS+iKESD1NGv5XXHEFmZmZAHTo0IGHH364KZtPCqdN59Jebbi0V5u67ymlKPVH2FYRYltlkO8qQ2yvDLGjMoTSNDKdNnQXLNpQyr/X7sJp08nLcKBroGsaMdMiFLMIxyzyMhz0bR8/2ji+fTYlFQFCUQsN8LpteF12AHZWhfiuMkzMsji5nZcTWmdh02UaSoh01WThHw6HUUoxd+7cpmqy2dI0jfxMJ/mZTk7pkH3Q18VMi8+2V7Lk2zIqglEsBZalsBkaTpuOw9DZXhnig/UlvHmEy0147Aa92mbRzuuidZaTbLed6nCU8kCUimCUiKnq7l7u087L2V3y6JrvAaAsEOW7yhCZThttvU5ctUclllIEIiZOm47dkGksIZqzJgv/r7/+mmAwyKhRo4jFYtx9992ccsopTdV8i2QzdE7rmMNpHXMO+TpLKTaW+AlrOmY4istuoJSiKhSjOhzDtBRtvS7aep2gaXy+rZKV2ypZu7OaJcVllPojddvKctrIdttw2nRsuk7EtFjybRnPLNlEXoaDUNTEH9n/UtZslw0F1IRjWLWPPchx28nPdJCf4SDXYyfX46CN10lhrodOuR4KMh3ocgJciKTRVBM9rHbdunWsWrWKq666ik2bNnHzzTfz7rvvYrPVv/+xLAvTPPrSDEPHTLN1d462z5GYRWUwitdtx1nPieddVSEWfbOHFcVlZLvtFOZ56JDjwR+O8V1FfNrK0DW8Ljtet41AxGRXVYhdVWFK/WFKayKU+CNEYvvXZuha/D+ahkLxw79Em6Hhc9vJ8Tjqpq8spdA1jTa1J9DzM51sKQ/w5XdVbCr1k5fhpDDPQ6c8D3mZTnLcdnweB16XjSyXjUyXnQyHgdOmo2kalqWoDEUprYlg6BoFWU4ync3/VJj8faePI+23vYHnB5ss/CORCJZl4XK5ALjyyiuZOXMmbdu2rff10ahJRUXgqNvz+TzH9P6WqDn3WSlFaSDK5rIAxaUBygIRTEsRsxSmBZoWP/G978FAxFRUhaJUBuNHMBqgaxC1FDurwpTUHrE4bDpdcj0U5ropD0TZWhFkZ1WYQ/1ha4DLrhOJWfxwjOG26+R4HGQ5bWQ6Ddx2A13T0DUwLUUgahKImGiaRveCDHq0zqRjjht/2KQiGCUYs/C5beRnOPC57SgFMUthKYWha9iN+JSdy6bjcRi4bDoKiNZOte1bTqbDwFbPFFpz/l0nSjr2GY683wUFWQ16XZMNcV599VW++eYbHnjgAXbt2kVNTQ0FBQVN1bxIMk3TyM+ITwOdepyvUbYZipqUB6N0a+/DX73/DXVRM340UxGKURmMUhM28Udi1IRjBCImwZhFKBo/P5HjcZDjtmMpRUntUUpFMEp1OIY/HKOkJoKlFJaK75wyHAa5Hgdh02Lh+hLe/CJxy3trQF6Gg4JMBxmO70d0TocNuxa/oMC0FCX+CHtqIsQsRTtv/Moxn9tOJBa/OMC0FC67jttu4LIbGFr8d2JoGh6HQYbDwFO7fdOKH4UpqDsiqwhGKQtEKQ9EyMtw0CUvgy55HjwOg6ipiJgWugYOQ8dp04lZippwfKdtqfjUYJbLhttuHHCEp9W+L8NhyL0wTahJR/7jx4/nu+++Q9M07rnnHvr163fQ18vI/8ilY58huf1WSrGjKsz2yiBZThs+tx2X3aAiGKW0dieiaRq22imuWO3oPlJ7xVYoahKMmui1r7EZet1t9xZQFYyypybC7prwfsuGKF2jJhgjFDMxNK3u/Iqha+yoDPFdVZiKYBSXTY+Hva7VtRWKWoc8KjoYQ49Pw1UEIgccLTUGj92gdZaT3Aw7sdodSsS0qAmb1IRjhGIWmQ6DHI8dn9uOw4hfWGDTtbqds6kUXqeNvNpzTYauEY7Ft1MVilFeuwMLm6ru6jmHoeFx2HDb4zsuS8V/rzZdx+uyke2243HEd1pKKVTtZ2HTNexGfLrT57aT5bRhKkXMtIhZCpfNwO3QcdZemLG5LMjWiiA5Hjud8+LnvkwrfuVfaSCK12mjS76HXI9jv88lUSP/Jgv/IyXhf+TSsc+Qnv1ujD4rFZ92C0TiJ/EDERO0+NSajlY3BadrGlkuG16XDV3TiJoWm8uDfFviJ2oq7EZ8KkspRdi0CEet2nNA8dE+QHUoRlUoRjBq1U3xAXU7oVDUZHdNhJ1VISqCUWyGjsPQcBg6mU4bmU4bvkwneyqDVARrr0iLKWJWPGg14oGsaxpVoSil/iiBfXaWhgZZLjs5Hjs5bjsuu1539VzEtOr6HzWtuim+qKWorL3yrbEYGofdcfrcdu4+pwsX92gd/7qlT/sIIZoXTYuPXLPdOtlue4PfZzd0uuVn0C0/I4HVHehIQzAYNVEqfk7oaO9pUUoRilkEIiZ67VQZxC88MC1FOBY/oqgIRqkJx2qPCOLthWLxHUooZtEmy0nnPA9tvS6qQlGKywJsKgviNHTyMuIXNVQEomws9bOpLECrTOdR1XskJPyFECmpMe6K1zQNt91o1DvsczwOcjwO+nU48NzX6Z0OfVl3Y5I7cYQQIg1J+AshRBqS8BdCiDQk4S+EEGlIwl8IIdKQhL8QQqQhCX8hhEhDEv5CCJGGmu3yDkIIIRJHRv5CCJGGJPyFECINSfgLIUQakvAXQog0JOEvhBBpSMJfCCHSkIS/EEKkoZR6mItlWTzwwAOsW7cOh8PB1KlTKSwsTHZZCRGNRpkwYQLbt28nEolw22230a1bN8aNG4emaXTv3p3Jkyej66m3fy8tLeWnP/0pf/3rX7HZbGnR59mzZ/PBBx8QjUa57rrrGDBgQEr3OxqNMm7cOLZv346u60yZMiXlf9erVq3iscceY+7cuWzevLnevj711FMsXLgQm83GhAkT6NOnz9E3qFLIf/7zHzV27FillFKfffaZ+uUvf5nkihLn1VdfVVOnTlVKKVVeXq6GDBmibr31VrV8+XKllFKTJk1S7733XjJLTIhIJKJuv/12deGFF6oNGzakRZ+XL1+ubr31VmWapqqpqVFPPvlkyvd7/vz56s4771RKKbVkyRL1q1/9KqX7/Kc//Uldeuml6qqrrlJKqXr7umbNGlVUVKQsy1Lbt29XP/3pT4+pzdTZbQKffvopZ599NgCnnHIKa9asSXJFiXPRRRdx1113AfHnjBqGwdq1axkwYAAAgwcPZunSpcksMSEeffRRrr32Wlq1agWQFn1esmQJxx9/PKNHj+aXv/wlP/rRj1K+3507d8Y0TSzLoqamBpvNltJ97tixIzNnzqz7ur6+fvrppwwaNAhN02jXrh2maVJWVnbUbaZU+NfU1JCZmVn3tWEYxGKxJFaUOBkZGWRmZlJTU8Odd97JmDFjUErVPWA6IyOD6urqJFfZuF5//XVyc3PrdvBAyvcZoLy8nDVr1vCHP/yBBx98kHvuuSfl++3xeNi+fTsXX3wxkyZNoqioKKX7PHToUGy272fh6+vrD/PtWD+DlJrzz8zMxO/3131tWdZ+H2iq2bFjB6NHj2bEiBEMGzaMGTNm1P3M7/fj9XqTWF3je+2119A0jWXLlvHVV18xduzY/UY+qdhnAJ/PR5cuXXA4HHTp0gWn08nOnTvrfp6K/Z4zZw6DBg3i//7v/9ixYwcjR44kGo3W/TwV+7yvfc9l7O3rD/PN7/eTlZV19G0cU4XNTL9+/Vi8eDEAn3/+Occff3ySK0qckpISRo0axb333suVV14JQM+ePVmxYgUAixcvpn///skssdG98MILPP/888ydO5cePXrw6KOPMnjw4JTuM8Cpp57Khx9+iFKKXbt2EQwGOfPMM1O6316vty7YsrOzicViKf/3va/6+tqvXz+WLFmCZVl89913WJZFbm7uUbeRUqt67r3a55tvvkEpxbRp0+jatWuyy0qIqVOn8s4779ClS5e67913331MnTqVaDRKly5dmDp1KoZhJLHKxCkqKuKBBx5A13UmTZqU8n2ePn06K1asQCnFr3/9azp06JDS/fb7/UyYMIE9e/YQjUa5/vrrOemkk1K6z9u2bePuu+/mlVdeobi4uN6+zpw5k8WLF2NZFuPHjz+mHWBKhb8QQoiGSalpHyGEEA0j4S+EEGlIwl8IIdKQhL8QQqQhCX8hhEhDqXsHlBANsGLFCsaMGUO3bt3qvpeTk8OTTz55TNsdN24cl1xyCYMHDz7WEoVICAl/kfbOOOMMHn/88WSXIUSTkvAXoh5FRUV07tyZ4uJilFI8/vjjFBQU8Mgjj/Dpp58CcOmllzJy5Eg2bdrExIkTiUajuFyuuh3Jyy+/zJ///Gdqamp44IEHjm35XSEamYS/SHvLly+nqKio7ushQ4YA8eVCHnroIV544QVmz57NwIED2bZtG6+88gqxWIwRI0Zwxhln8MQTT3DLLbcwePBgFixYwJdffglAr169uP3223n99dd5/fXXJfxFsyLhL9JefdM+ixYt4owzzgDiO4EPPviANm3a0L9/fzRNw263c/LJJ7Nx40aKi4vp27cvAOeddx4A//73v+nVqxcA+fn5hEKhJuyREIcnV/sIcRB7nwexcuVKunXrRteuXeumfKLRKJ999hmFhYV07dqVL774AoB58+Yxd+5cgLoleYVojmTkL9LeD6d9AEKhEG+88QZz5szB7XYzffp0cnJy+Pjjj7nmmmuIRqNcdNFF9OrVi9/85jfcf//9zJo1C5fLxYwZM1i7dm2SeiNEw8jCbkLUY++qoam6KqwQMu0jhBBpSEb+QgiRhmTkL4QQaUjCXwgh0pCEvxBCpCEJfyGESEMS/kIIkYb+P9zufY11ZHgzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(history.history['mae'])\n",
    "plt.plot(history.history['root_mean_squared_error'])\n",
    "plt.title('mae and rmse per epoch')\n",
    "plt.ylabel('Error')\n",
    "plt.xlabel('Epoch')\n",
    "plt.legend(['mae', 'rmse'], loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "#mae: 4.2719\n",
    "#rmse 10.51"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "#MAE of around 4 w/m2. Considering DHI values go from 0 - 400, 4 represents a 1% error value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## IV. Model Validation - Outer Banks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>Hour</th>\n",
       "      <th>Minute</th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Clearsky DHI</th>\n",
       "      <th>Clearsky DNI</th>\n",
       "      <th>Clearsky GHI</th>\n",
       "      <th>Cloud Type</th>\n",
       "      <th>...</th>\n",
       "      <th>GHI</th>\n",
       "      <th>Relative Humidity</th>\n",
       "      <th>Solar Zenith Angle</th>\n",
       "      <th>Surface Albedo</th>\n",
       "      <th>Pressure</th>\n",
       "      <th>Precipitable Water</th>\n",
       "      <th>Wind Direction</th>\n",
       "      <th>Wind Speed</th>\n",
       "      <th>timestamps</th>\n",
       "      <th>Cloud Cover</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>30</td>\n",
       "      <td>1.8</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>85.84</td>\n",
       "      <td>119.89</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1026</td>\n",
       "      <td>1.2</td>\n",
       "      <td>350</td>\n",
       "      <td>8.6</td>\n",
       "      <td>1514782800</td>\n",
       "      <td>1.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "      <td>1.6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>87.76</td>\n",
       "      <td>132.05</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1026</td>\n",
       "      <td>1.2</td>\n",
       "      <td>348</td>\n",
       "      <td>8.7</td>\n",
       "      <td>1514786400</td>\n",
       "      <td>0.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>30</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>88.35</td>\n",
       "      <td>144.19</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1026</td>\n",
       "      <td>1.1</td>\n",
       "      <td>346</td>\n",
       "      <td>8.7</td>\n",
       "      <td>1514790000</td>\n",
       "      <td>1.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>30</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>89.16</td>\n",
       "      <td>155.85</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1026</td>\n",
       "      <td>1.1</td>\n",
       "      <td>346</td>\n",
       "      <td>8.7</td>\n",
       "      <td>1514793600</td>\n",
       "      <td>1.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>30</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>89.52</td>\n",
       "      <td>165.32</td>\n",
       "      <td>0.03</td>\n",
       "      <td>1026</td>\n",
       "      <td>1.1</td>\n",
       "      <td>347</td>\n",
       "      <td>8.9</td>\n",
       "      <td>1514797200</td>\n",
       "      <td>1.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Year  Month  Day  Hour  Minute  Temperature  Clearsky DHI  Clearsky DNI  \\\n",
       "0  2018      1    1     0      30          1.8             0             0   \n",
       "1  2018      1    1     1      30          1.6             0             0   \n",
       "2  2018      1    1     2      30          1.5             0             0   \n",
       "3  2018      1    1     3      30          1.4             0             0   \n",
       "4  2018      1    1     4      30          1.4             0             0   \n",
       "\n",
       "   Clearsky GHI  Cloud Type  ...  GHI  Relative Humidity  Solar Zenith Angle  \\\n",
       "0             0           4  ...    0              85.84              119.89   \n",
       "1             0           4  ...    0              87.76              132.05   \n",
       "2             0           4  ...    0              88.35              144.19   \n",
       "3             0           4  ...    0              89.16              155.85   \n",
       "4             0           4  ...    0              89.52              165.32   \n",
       "\n",
       "   Surface Albedo  Pressure  Precipitable Water  Wind Direction  Wind Speed  \\\n",
       "0            0.03      1026                 1.2             350         8.6   \n",
       "1            0.03      1026                 1.2             348         8.7   \n",
       "2            0.03      1026                 1.1             346         8.7   \n",
       "3            0.03      1026                 1.1             346         8.7   \n",
       "4            0.03      1026                 1.1             347         8.9   \n",
       "\n",
       "   timestamps  Cloud Cover  \n",
       "0  1514782800         1.00  \n",
       "1  1514786400         0.94  \n",
       "2  1514790000         1.00  \n",
       "3  1514793600         1.00  \n",
       "4  1514797200         1.00  \n",
       "\n",
       "[5 rows x 24 columns]"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "validation_df = pd.read_csv(os.path.join(_Testing_Data_Dir, 'outer_banks', 'r_banks.csv'))\n",
    "validation_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_size = 6\n",
    "val_input_arry = np.array([], dtype=np.float64).reshape(0,input_size)\n",
    "val_output_arry = np.array([], dtype=np.float64)\n",
    "val_input_arry, val_output_arry = combine_training_arrays(validation_df, val_input_arry, val_output_arry)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "#keras model.evaluate reveals a mae of 6.9433 w/m2 in the outer banks test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "274/274 [==============================] - 0s 945us/step - loss: 269.0166 - mae: 7.1228 - root_mean_squared_error: 16.4017\n"
     ]
    }
   ],
   "source": [
    "score = model.evaluate(val_input_arry, val_output_arry, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "preds = model.predict(val_input_arry)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Mape too high because of 0 values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Compare random induvidual resutls!!!!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "index is  3018\n",
      "Actual  265.0   Predicted  [265.45956]\n"
     ]
    }
   ],
   "source": [
    "i = randint(0,8760)\n",
    "print(\"index is \",i)\n",
    "print(\"Actual \",val_output_arry[i],\" \",\"Predicted \", preds[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: Neural_Net_National/assets\n"
     ]
    }
   ],
   "source": [
    "model.save('Neural_Net_National')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## V. Model Testing - Other Locations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets take the global model and test in new locations. Lets test on the following locations for the year 2018.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Outer Banks (35.5585, -75.4665)\n",
    "2. Miami Florida (25.7617, -80.1918)\n",
    "3. Boulder CO (40.035,-105.54)\n",
    "4. San Diego CA (32.7157, -117.1611)\n",
    "5. Lincoln Nebraska (40.848,-96.56)\n",
    "6. Quinault Washington (47.513,-123.81)\n",
    "7. Old Town Maine (44.928,-68.70)\n",
    "8. Murphey Idaho (43.204,-116.75)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Test General model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "National_model = keras.models.load_model('Neural_Nets/Neural_Net_National')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense (Dense)                multiple                  2100      \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              multiple                  60200     \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              multiple                  20100     \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              multiple                  5050      \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              multiple                  1275      \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              multiple                  26        \n",
      "=================================================================\n",
      "Total params: 88,751\n",
      "Trainable params: 88,751\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "National_model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Murphey_ID/psm_ID_Murphey2018.csv\n",
      "274/274 [==============================] - 0s 984us/step - loss: 256.5927 - mean_absolute_error: 6.7838 - root_mean_squared_error: 16.0185\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/OldTown_ME/psm_ME_OldTown2018.csv\n",
      "274/274 [==============================] - 0s 932us/step - loss: 229.4214 - mean_absolute_error: 6.8104 - root_mean_squared_error: 15.1467\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Quinault_WA/psm_WA_Quinault2018.csv\n",
      "274/274 [==============================] - 0s 1ms/step - loss: 104.5538 - mean_absolute_error: 4.2643 - root_mean_squared_error: 10.2252\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Lincoln_NE/psm_NE_Lincoln2018.csv\n",
      "274/274 [==============================] - 0s 1ms/step - loss: 161.9369 - mean_absolute_error: 5.2631 - root_mean_squared_error: 12.7254\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/SanDiego_CA/psm_CA_SanDiego2018.csv\n",
      "274/274 [==============================] - 0s 1ms/step - loss: 289.1909 - mean_absolute_error: 7.7851 - root_mean_squared_error: 17.0056\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Boulder_CO/psm_CO_Boulder2018.csv\n",
      "274/274 [==============================] - 0s 998us/step - loss: 1862.3188 - mean_absolute_error: 16.5438 - root_mean_squared_error: 43.1546\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Miami_FL/psm_FL_Miami2018.csv\n",
      "274/274 [==============================] - 0s 947us/step - loss: 675.2378 - mean_absolute_error: 10.5119 - root_mean_squared_error: 25.9853\n"
     ]
    }
   ],
   "source": [
    "test_files = ['Murphey_ID', 'OldTown_ME', 'Quinault_WA', 'Lincoln_NE', 'SanDiego_CA', 'Boulder_CO', 'Miami_FL', 'outer_banks']\n",
    "scores = {}\n",
    "for directory in test_files:\n",
    "    for file in glob(_Testing_Data_Dir+'/'+directory+'/psm_*2018.csv'):\n",
    "        test_df = pd.read_csv(str(file))\n",
    "        input_size = 6\n",
    "        test_input_arry = np.array([], dtype=np.float64).reshape(0,input_size)\n",
    "        test_output_arry = np.array([], dtype=np.float64)\n",
    "        test_input_arry, test_output_arry = combine_training_arrays(test_df, test_input_arry, test_output_arry)\n",
    "        print(file)\n",
    "        score = National_model.evaluate(test_input_arry, test_output_arry, verbose=1)\n",
    "        scores[directory] = score\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_items([('Murphey_ID', [256.5926818847656, 6.783843994140625, 16.018510818481445]), ('OldTown_ME', [229.42138671875, 6.810364246368408, 15.146662712097168]), ('Quinault_WA', [104.5538101196289, 4.2642903327941895, 10.2251558303833]), ('Lincoln_NE', [161.93692016601562, 5.263095855712891, 12.725443840026855]), ('SanDiego_CA', [289.1909484863281, 7.785069942474365, 17.005615234375]), ('Boulder_CO', [1862.31884765625, 16.543825149536133, 43.15459060668945]), ('Miami_FL', [675.23779296875, 10.51191234588623, 25.98533821105957])])"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scores.items()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Only Miami and Boulder_CO seem to be outliers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## V.I Model Testing, Boulder Specific"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets see if we can train a model specific for Boulder, CO and see if we can get the mae down.\n",
    "Let's fetch more data from NRSDB and combine it with our readings from Darksky. Lets get data from 2008-2018."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[]\n",
      "[]\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Boulder_CO/psm_CO_Boulder2018.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Boulder_CO/psm_CO_Boulder2008.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Boulder_CO/psm_CO_Boulder2009.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Boulder_CO/psm_CO_Boulder2010.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Boulder_CO/psm_CO_Boulder2011.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Boulder_CO/psm_CO_Boulder2013.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Boulder_CO/psm_CO_Boulder2012.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Boulder_CO/psm_CO_Boulder2016.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Boulder_CO/psm_CO_Boulder2017.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Boulder_CO/psm_CO_Boulder2015.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Boulder_CO/psm_CO_Boulder2014.csv\n"
     ]
    }
   ],
   "source": [
    "input_size = 6\n",
    "input_arry = np.array([], dtype=np.float64).reshape(0,input_size)\n",
    "output_arry = np.array([], dtype=np.float64)\n",
    "print(input_arry)\n",
    "print(output_arry)\n",
    "from glob import glob\n",
    "for path in glob(_Testing_Data_Dir+'/Boulder_CO'):\n",
    "    for file in glob(path+'/psm_'+'*'):\n",
    "        print(file)\n",
    "        df = pd.read_csv(file)\n",
    "        input_arry, output_arry = combine_training_arrays(df, input_arry, output_arry)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [],
   "source": [
    "keras.backend.clear_session()\n",
    "tf.keras.backend.clear_session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function tensorflow.python.framework.ops.reset_default_graph()>"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras.backend.clear_session()\n",
    "tf.keras.backend.clear_session()\n",
    "tf.compat.v1.reset_default_graph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_Boulder = keras.Sequential()\n",
    "model_Boulder.add(keras.layers.Dense(300, activation=\"relu\"))\n",
    "model_Boulder.add(layers.Dense(200, activation=\"relu\"))\n",
    "model_Boulder.add(layers.Dense(100, activation=\"relu\"))\n",
    "model_Boulder.add(layers.Dense(50, activation=\"relu\"))\n",
    "model_Boulder.add(layers.Dense(25, activation=\"relu\"))\n",
    "model_Boulder.add(layers.Dense(1, activation='relu'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      "3012/3012 [==============================] - 5s 1ms/step - loss: 3136.6509 - mae: 27.1907 - root_mean_squared_error: 56.0058\n",
      "Epoch 2/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 669.0169 - mae: 10.9336 - root_mean_squared_error: 25.8654\n",
      "Epoch 3/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 583.5402 - mae: 9.9415 - root_mean_squared_error: 24.1566\n",
      "Epoch 4/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 572.9196 - mae: 9.7745 - root_mean_squared_error: 23.9357\n",
      "Epoch 5/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 565.0255 - mae: 9.5376 - root_mean_squared_error: 23.7703\n",
      "Epoch 6/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 545.2877 - mae: 9.3087 - root_mean_squared_error: 23.3514: 2s - loss: 516.5451 - mae: 9.0326 - root_mean_sq\n",
      "Epoch 7/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 524.0295 - mae: 9.0873 - root_mean_squared_error: 22.8917\n",
      "Epoch 8/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 516.6038 - mae: 9.0893 - root_mean_squared_error: 22.7289\n",
      "Epoch 9/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 524.2489 - mae: 9.0986 - root_mean_squared_error: 22.8965\n",
      "Epoch 10/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 512.5381 - mae: 8.9168 - root_mean_squared_error: 22.6393: 0s - loss: 506.6469 - mae: 8.8751 - \n",
      "Epoch 11/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 492.6263 - mae: 8.7507 - root_mean_squared_error: 22.1952\n",
      "Epoch 12/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 494.3108 - mae: 8.7720 - root_mean_squared_error: 22.2331: 1s - loss: 492.8882 - mae: 8.7324 - root_mean_squared_error: 22.20 - ETA: 1s - los\n",
      "Epoch 13/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 512.3876 - mae: 8.9023 - root_mean_squared_error: 22.6360: 1s - loss:\n",
      "Epoch 14/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 496.9038 - mae: 8.7705 - root_mean_squared_error: 22.2913\n",
      "Epoch 15/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 494.8581 - mae: 8.7226 - root_mean_squared_error: 22.2454\n",
      "Epoch 16/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 487.8583 - mae: 8.6009 - root_mean_squared_error: 22.0875: \n",
      "Epoch 17/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 485.5327 - mae: 8.5918 - root_mean_squared_error: 22.0348\n",
      "Epoch 18/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 474.7318 - mae: 8.5533 - root_mean_squared_error: 21.7883\n",
      "Epoch 19/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 493.0307 - mae: 8.6855 - root_mean_squared_error: 22.2043\n",
      "Epoch 20/100\n",
      "3012/3012 [==============================] - 8s 3ms/step - loss: 476.4244 - mae: 8.4376 - root_mean_squared_error: 21.8271\n",
      "Epoch 21/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 478.3757 - mae: 8.4598 - root_mean_squared_error: 21.8718\n",
      "Epoch 22/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 476.8438 - mae: 8.4770 - root_mean_squared_error: 21.8368\n",
      "Epoch 23/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 473.7332 - mae: 8.4708 - root_mean_squared_error: 21.7654: 4s - loss: 468 - ETA: 2s - loss: 489.8315 - mae:\n",
      "Epoch 24/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 474.6563 - mae: 8.4040 - root_mean_squared_error: 21.7866\n",
      "Epoch 25/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 469.4034 - mae: 8.4124 - root_mean_squared_error: 21.6657\n",
      "Epoch 26/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 464.3542 - mae: 8.2993 - root_mean_squared_error: 21.5489: 0s - loss: 468.0563 - mae: 8.3313 - root_mea\n",
      "Epoch 27/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 462.8958 - mae: 8.2942 - root_mean_squared_error: 21.5150\n",
      "Epoch 28/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 458.3492 - mae: 8.2575 - root_mean_squared_error: 21.4091\n",
      "Epoch 29/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 459.2443 - mae: 8.2614 - root_mean_squared_error: 21.4300\n",
      "Epoch 30/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 456.1564 - mae: 8.1957 - root_mean_squared_error: 21.3578\n",
      "Epoch 31/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 457.0403 - mae: 8.2456 - root_mean_squared_error: 21.3785\n",
      "Epoch 32/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 452.7580 - mae: 8.1291 - root_mean_squared_error: 21.2781\n",
      "Epoch 33/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 455.0916 - mae: 8.1407 - root_mean_squared_error: 21.3329\n",
      "Epoch 34/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 449.8091 - mae: 8.1115 - root_mean_squared_error: 21.2087\n",
      "Epoch 35/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 448.6960 - mae: 8.1020 - root_mean_squared_error: 21.1824\n",
      "Epoch 36/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 448.0442 - mae: 8.1478 - root_mean_squared_error: 21.1671\n",
      "Epoch 37/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 455.3228 - mae: 8.1246 - root_mean_squared_error: 21.3383\n",
      "Epoch 38/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 451.6123 - mae: 8.1153 - root_mean_squared_error: 21.2512: 0s - loss: 457.7770 - mae: 8\n",
      "Epoch 39/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 444.6736 - mae: 7.9765 - root_mean_squared_error: 21.0873\n",
      "Epoch 40/100\n",
      "3012/3012 [==============================] - 8s 3ms/step - loss: 444.1586 - mae: 7.9963 - root_mean_squared_error: 21.0751\n",
      "Epoch 41/100\n",
      "3012/3012 [==============================] - 8s 3ms/step - loss: 443.6709 - mae: 8.0063 - root_mean_squared_error: 21.0635\n",
      "Epoch 42/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 439.5462 - mae: 7.9723 - root_mean_squared_error: 20.9654\n",
      "Epoch 43/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 443.2075 - mae: 8.0053 - root_mean_squared_error: 21.0525\n",
      "Epoch 44/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 438.3240 - mae: 7.9389 - root_mean_squared_error: 20.9362\n",
      "Epoch 45/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 439.9401 - mae: 7.9866 - root_mean_squared_error: 20.9747\n",
      "Epoch 46/100\n",
      "3012/3012 [==============================] - 10s 3ms/step - loss: 444.4282 - mae: 7.9454 - root_mean_squared_error: 21.0815 1s - loss:\n",
      "Epoch 47/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 436.3835 - mae: 7.9112 - root_mean_squared_error: 20.8898\n",
      "Epoch 48/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 442.0975 - mae: 7.9539 - root_mean_squared_error: 21.0261: 0s - loss: 440.5885 - mae: 7.9620 - root_mean_squared_erro\n",
      "Epoch 49/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 435.4044 - mae: 7.8778 - root_mean_squared_error: 20.8663: 1s\n",
      "Epoch 50/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 431.5168 - mae: 7.8407 - root_mean_squared_error: 20.7730\n",
      "Epoch 51/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 433.3185 - mae: 7.8882 - root_mean_squared_error: 20.8163\n",
      "Epoch 52/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 432.6559 - mae: 7.8289 - root_mean_squared_error: 20.8004\n",
      "Epoch 53/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 435.3001 - mae: 7.9093 - root_mean_squared_error: 20.8638\n",
      "Epoch 54/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 435.5841 - mae: 7.8813 - root_mean_squared_error: 20.8707\n",
      "Epoch 55/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 435.5659 - mae: 7.8412 - root_mean_squared_error: 20.8702\n",
      "Epoch 56/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 428.5637 - mae: 7.8220 - root_mean_squared_error: 20.7018\n",
      "Epoch 57/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 434.1787 - mae: 7.8164 - root_mean_squared_error: 20.8370\n",
      "Epoch 58/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 433.6161 - mae: 7.8048 - root_mean_squared_error: 20.8235\n",
      "Epoch 59/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 428.7990 - mae: 7.8011 - root_mean_squared_error: 20.7075\n",
      "Epoch 60/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 429.4618 - mae: 7.7510 - root_mean_squared_error: 20.7235\n",
      "Epoch 61/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 425.3711 - mae: 7.7080 - root_mean_squared_error: 20.6245: 0s - loss: 425.4912 - mae: 7\n",
      "Epoch 62/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 427.7862 - mae: 7.7421 - root_mean_squared_error: 20.6830\n",
      "Epoch 63/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 419.5352 - mae: 7.6860 - root_mean_squared_error: 20.4826: 1s - loss: 421.0826 - mae: 7.6814 - root_mea - ETA: 0s - loss: 419.1958 - mae: 7.6799 - root_mean_\n",
      "Epoch 64/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 420.7856 - mae: 7.6758 - root_mean_squared_error: 20.5131\n",
      "Epoch 65/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 420.8124 - mae: 7.6575 - root_mean_squared_error: 20.5137\n",
      "Epoch 66/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 426.4190 - mae: 7.7097 - root_mean_squared_error: 20.6499: 1s - loss: 424.1364 - mae:\n",
      "Epoch 67/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 418.0478 - mae: 7.6694 - root_mean_squared_error: 20.4462\n",
      "Epoch 68/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 420.3964 - mae: 7.6978 - root_mean_squared_error: 20.5036\n",
      "Epoch 69/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 418.7297 - mae: 7.6378 - root_mean_squared_error: 20.4629\n",
      "Epoch 70/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 418.7142 - mae: 7.6329 - root_mean_squared_error: 20.4625\n",
      "Epoch 71/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 407.5191 - mae: 7.4549 - root_mean_squared_error: 20.1871: 0s - loss: 409.2074 - mae: 7.4705 - root_mean_squared_\n",
      "Epoch 72/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 405.1358 - mae: 7.5055 - root_mean_squared_error: 20.1280\n",
      "Epoch 73/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 407.2210 - mae: 7.5052 - root_mean_squared_error: 20.1797\n",
      "Epoch 74/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 400.5823 - mae: 7.3957 - root_mean_squared_error: 20.0146\n",
      "Epoch 75/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 399.7678 - mae: 7.3596 - root_mean_squared_error: 19.9942\n",
      "Epoch 76/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 398.6359 - mae: 7.3066 - root_mean_squared_error: 19.9659\n",
      "Epoch 77/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 394.1367 - mae: 7.2512 - root_mean_squared_error: 19.8529: 1s - l\n",
      "Epoch 78/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 393.7986 - mae: 7.2313 - root_mean_squared_error: 19.8444: 1s - loss: 4\n",
      "Epoch 79/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 390.5683 - mae: 7.2321 - root_mean_squared_error: 19.7628\n",
      "Epoch 80/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 390.6879 - mae: 7.1833 - root_mean_squared_error: 19.7658\n",
      "Epoch 81/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 389.2727 - mae: 7.1799 - root_mean_squared_error: 19.7300\n",
      "Epoch 82/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 386.9371 - mae: 7.1489 - root_mean_squared_error: 19.6707\n",
      "Epoch 83/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 386.4530 - mae: 7.1966 - root_mean_squared_error: 19.6584\n",
      "Epoch 84/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 385.0334 - mae: 7.1100 - root_mean_squared_error: 19.6223\n",
      "Epoch 85/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 384.3309 - mae: 7.1422 - root_mean_squared_error: 19.6044\n",
      "Epoch 86/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 377.7376 - mae: 7.0068 - root_mean_squared_error: 19.4355: 0s - loss: 376.0996 - mae: 6.9479 - root_mean_\n",
      "Epoch 87/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 381.1904 - mae: 7.0501 - root_mean_squared_error: 19.5241\n",
      "Epoch 88/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 380.1472 - mae: 7.0325 - root_mean_squared_error: 19.4974\n",
      "Epoch 89/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 379.6038 - mae: 7.0006 - root_mean_squared_error: 19.4834\n",
      "Epoch 90/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 380.7220 - mae: 7.0699 - root_mean_squared_error: 19.5121: 2s - los - ETA: 1s - loss: 384.2755 - mae: 7.105\n",
      "Epoch 91/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 381.2592 - mae: 7.0650 - root_mean_squared_error: 19.5259\n",
      "Epoch 92/100\n",
      "3012/3012 [==============================] - 4s 1ms/step - loss: 383.7803 - mae: 7.0379 - root_mean_squared_error: 19.5903\n",
      "Epoch 93/100\n",
      "3012/3012 [==============================] - 5s 2ms/step - loss: 378.8813 - mae: 6.9901 - root_mean_squared_error: 19.4649: 0s - loss: 379.1909 - mae: 7.0070 \n",
      "Epoch 94/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 380.6503 - mae: 7.0289 - root_mean_squared_error: 19.5103: 2s - loss: 387.4841 - \n",
      "Epoch 95/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 372.7206 - mae: 6.8934 - root_mean_squared_error: 19.3060\n",
      "Epoch 96/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 374.2655 - mae: 6.9971 - root_mean_squared_error: 19.3459\n",
      "Epoch 97/100\n",
      "3012/3012 [==============================] - 7s 2ms/step - loss: 370.6898 - mae: 6.9395 - root_mean_squared_error: 19.2533: 0s - loss: 371.0768 - mae: 6.940\n",
      "Epoch 98/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 373.6794 - mae: 6.9509 - root_mean_squared_error: 19.3308: 1s - loss: 367\n",
      "Epoch 99/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 371.5406 - mae: 6.9311 - root_mean_squared_error: 19.2754\n",
      "Epoch 100/100\n",
      "3012/3012 [==============================] - 6s 2ms/step - loss: 371.0230 - mae: 6.9127 - root_mean_squared_error: 19.2620\n"
     ]
    }
   ],
   "source": [
    "#Compile and train\n",
    "model_Boulder.compile(optimizer=Adam(lr=0.0001), \n",
    "              loss='mse',\n",
    "              metrics=['mae',keras.metrics.RootMeanSquaredError()])\n",
    "\n",
    "history_Boulder = model_Boulder.fit(input_arry, output_arry, epochs=100, batch_size=32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "#MAE falls to around 6.99"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAESCAYAAAAVLtXjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAIABJREFUeJzt3Xl8VNX9//HXXWbPMiGEfZEAKuCKFFAQtC6olYpVK1ApLVZtRS21KossLrgg1g2rX+3P1ha1ahVbuosoIiBYFTdcQfadkIRkMpnl3vP74yYDIQsBMoTc+TwfDx9kmbn3nImP9zn3nHPP1ZRSCiGEEBlFb+4CCCGEOPIk/IUQIgNJ+AshRAaS8BdCiAwk4S+EEBlIwl8IITKQhL9wrf/85z+MGTOmuYvhCvPmzeO6665r7mKIJiThL4QQGchs7gKIlmPFihU89NBDtGnThm+++YZAIMCNN97I3LlzWbt2Leeffz5TpkzBtm3uvfdePv74YyKRCEopZs6cyWmnnUY8HufBBx/kf//7H5Zl0bt3b6ZOnUpWVlaNc+3atYvp06dTVFTEzp076dixI4888gj5+fl897vf5dJLL+Xdd99l69atXHjhhdx2220APProo/z9738nHA7TtWvXeutxzz33EAwGqaio4NZbb+Xxxx8/YL0ikQiTJ09m/fr16LpOnz59uOuuu9B1nTfffJMnn3ySRCKB3+9n4sSJnHrqqTXOu2nTJsaMGUP//v358ssvUUoxffp0+vXrB8CTTz7J66+/jm3bdOzYkRkzZtC2bVvGjBlDbm4u3377LaNGjap1NfOXv/yFP//5z9i2TTgcZtq0aXTv3p1JkyahaRpr1qxh9+7dDBo0iKlTp+LxeHj//fd54IEHiEajeDweJkyYwJAhQwB46qmneO211zBNk65du3L//fcDsHPnTq699lq2bt2KYRj85je/oXv37of/P5ZoHkqIRlq+fLnq1auXWrVqlVJKqauvvlpdeeWVKhaLqaKiItWnTx+1bds29eGHH6obb7xRWZallFLqqaeeUtddd51SSqk5c+ao+++/X9m2rZRS6je/+Y2aMWNGrXM9++yz6qmnnlJKKWXbtvrZz36mnnnmGaWUUmeffba6//77lVJKbdu2TZ144olqw4YNasGCBeqiiy5SZWVlKpFIqGuvvVZdddVVddbj+OOPV5s2bTqoer322mtq3LhxSimlksmkuv3229W6devU2rVr1cUXX6x2796tlFLq66+/VoMGDVKRSKTGeTdu3KiOPfZYNX/+fKWUUosWLVKDBg1S8Xhcvfbaa2rChAkqkUgopZR68cUX1c9+9jOllFJXXXWVmjx5cp1/kxUrVqjRo0eriooKpZRS77zzjrrwwguVUkpNnDhRjRgxQpWXl6tYLKZ+9KMfqblz56rdu3er008/XX300Uep8vbv319t2LBBvfHGG+r8889XJSUlSiml7r33XvXEE0+oV199VfXr10+tW7dOKaXU3XffXW+ZRMsgPX9xUDp16kTv3r0B6NKlC9nZ2Xi9Xlq1akUoFKK0tJRTTz2V3NxcXnzxRTZu3MiKFSsIhUIALFq0iLKyMpYtWwZAIpEgPz+/1nnGjh3L+++/zx/+8AfWrVvHN998w8knn5z6/TnnnANA27Ztyc/Pp7S0lHfffZfzzjsvdRVx2WWXMXfu3Drr0b59ezp27HhQ9TrttNN4+OGHGTNmDGeccQZjx46la9euPP/88+zYsYOf/OQnqeNpmsaGDRs4/vjja5w3NzeX4cOHAzB06FAMw+Crr77irbfe4tNPP+Wyyy4DwLZtotFo6n3VVwf7W7RoEevXr2fkyJGpn5WWllJSUgLApZdemvrsL7nkEhYuXEjnzp3p0qVL6vPs2bMnffv25b333uOLL77gggsuIDc3F4DJkycDzpj/SSedlLqa6tWrFwsWLKizTKJlkPAXB8Xr9db43jRr/y+0aNEi7rnnHn76059yzjnnUFhYyPz58wEn1KZMmcLQoUMBiEQixGKxWseYPXs2n3zyCZdddhkDBgwgmUyi9tmGyufzpb7WNA2lVOrfaoZh1FuPYDB40PXq3LkzCxYsYMWKFSxfvpyf/vSnTJ06Fdu2Of3003nkkUdSr926dStt2rSpdYz9y2TbNoZhYNs2P/vZzxg9ejQA8Xic0tLSesu77/svueQSbr311tT3O3bsSIX3vudTSqHrOrZt1zqOUopkMolhGGialvr5nj172LNnT63PZP/PWrQ8MuErmtzSpUs5++yzGT16NCeeeCJvvPEGlmUBMHjwYJ5//nni8Ti2bTNt2jQeeuihWsdYsmQJY8eOZcSIEeTn57Ns2bLUMepz5pln8p///Ic9e/Zg2zZ/+9vfmrReL7zwApMnT2bw4MHceuutDB48mG+++YaBAweydOlS1qxZA8Dbb7/N97///Tobtd27d7N48WIA3nzzTTweD8ceeyyDBw/mlVdeoby8HHDmLqrnMRoyaNAg/vnPf7Jjxw4A/vznPzN27NjU7//9738Tj8eJxWK89tprnH322Zx88smsXbuWTz75BIBvvvmG//3vf/Tv358zzjiDBQsWpMoxZ84cnn322UP/0MRRS3r+osmNHDmSW265heHDh2MYBv369UtNZF5//fXMmjWLSy+9FMuy6NWrF5MmTap1jPHjx/PAAw/wxBNPYBgGffv2ZcOGDQ2ed+jQoXz11Vdcdtll5OTkcPzxx1NcXNxk9RoxYgTvvfceF110EYFAgA4dOvDjH/+Y3Nxc7rrrLm6++WaUUpimyZNPPllnb93n8/G3v/2NBx98EL/fz29/+1sMw+CKK65g+/bt/PCHP0TTNNq3b5+aaG3ImWeeyTXXXMO4cePQNI2srCwef/zxVO/d7/czevRo9uzZw7Bhw7jsssvQdZ1HH32Uu+++m8rKSjRN47777qNbt25069aN1atXM2rUKAB69OjB3Xffzeuvv95kn6M4OmhKrt2EOCI2bdrE8OHDWbly5RE536RJk+jZsydXX331ETmfaFlk2EcIITKQ9PyFECIDSc9fCCEykIS/EEJkoKN2tY9t21jWoY9IGYZ2WO9viTKxzpCZ9ZY6Z46DrbfHU//9Lfs6asPfshQlJRWH/P5wOHhY72+JMrHOkJn1ljpnjoOtd0FBdqNeJ8M+QgiRgST8hRAiA0n4CyFEBjpqx/zrYllJiot3kkzGD/ja7dtb7sZTpuklL68Aw2hRfx4hRAvSotKluHgnfn+QUKhdjZ0H62IYOpZVe/fCo51SikhkD8XFO2ndun1zF0cI4VItatgnmYwTCuUcMPhbMk3TCIVyGnV1I4QQh6pFhT/g6uCvlgl1FEI0rxYX/gekFFp0N7TQ8X4hhDgS3Bf+yUqMso0QK2vukgghxFHLfeGf0vIme4UQ4khpUat99vXPVduZ/9m22r9QNlqyAsxvUNrBVe/7J7Tje33aNviaf/3r7yxduphYLEZR0S6uuGIU77zzNmvXrmH8+F+yY8d23n77LaLRKOFwmHvvfRBN05g9+142bdqIbdtcc80v6Nu37gdyCyHEkdBiw/+AFJCmedOKigoefvi3vPHGf3nppRd4+ulnWbnyA1566XmOO64XjzzyBLquc/PNN/DFF6tYs2Y1ublhJk+eTmlpCePHX8tzz72cnsIJIUQjtNjw/16ftnX30q0EZtHnqNzOWL5WaTl3z57HAZCVlc0xx3RD0zSys7NJJJJ4PB7uuON2AoEAO3bsIJlMsmbNaj75ZCWff/6ZU0QrSUlJCeFwOC3lE0KIA2mx4V+v6mWSaVztU99SzGQyweLFi/jd7/5IZWUlV199FQBdux5DmzZt+PGPxxGLVfLHP/6enJyctJVPCCEOxH3hT/rDvz6GYRAIBPjFL8YBkJ/fml27dnLJJT9g1qyZ3HDDtUQi5Vx66RXouovn2oUQR72j9hm+iYRVaw/rbdvW065d14bfqGzMnZ+isttjBdqksYTp1ai67kf2O88cUufMIfv5N1rz9fyFEKKlcF/4axpOAyDhL4QQ9XFf+ANK06XnL4QQDXBl+IMm4S+EEA1wZ/hrGrK9gxBC1M+d4S89fyGEaJA7w1+T8BdCiIa4M/xltY8QQjSoxd7h6/vyFfxfvFjn77RkFNBQpv+gjlnZaySx4y9v8DX/+tff+ec/52PbNrt27eSkk05h48YNnHbad4hEyvnii1V06dKVadPu5u233+S55/6IaZq0bl3AnXfeS0VFBffffxelpaUATJhwK9279ziocgohxOFqseHfnLKzs7n//ocYOnQA11xzPa1bt+bCC7/L008/y69+dRs//OEllJWVsWDBfxk9egxnn30u//73P4hEIsyd+wdOO60/l156ORs3buDee+/kySefae4qCSEyTIsN/9jxl9fbSzeK14AGVrh7Ws7dpYuz7UJOTi7t2rUDIBAI0K1bIQChUBbxeIwbb/wVc+c+y6uvvkzXrscwZMhZfPvtaj788H0WLnwdgLKyPWkpoxBCNKTFhn+DNA1U+pZ6appe9W/DDwyYP/81rr76WvLyWvHAA/ewePEiunY9hvPP7835519AcfFu/v73v6atnEIIUR9Xhr9CQz8KVvv06tWH226bQDAYIhAIcMYZgznjjMHcf//dzJ8/j4qKCOPGXdvcxRRCZCD37eoJ6KXr0K0YyVbHpat4aSe7ejZeJtZb6pw50rWrZ9p6/pdeeilZWVkAdOrUiSuvvJJ77rkHwzAYPHgwN9xwQ7pODcjePkII0ZC0hH8sFkMpxdy5c1M/u+SSS5gzZw6dO3fm2muv5fPPP6d3797pOH3V9g4S/kIIUZ+0hP+XX35JNBpl3LhxJJNJbrzxRuLxOF26dAFg8ODBLFu2rMHwNwyNcDhY42c7dujounbAiVZ0p+dvGC3zHjalFLqu16r/gRjGwb/HDTKx3lLnzJGueqcl/P1+P1dffTVXXHEF69at45prrqnxzNpQKMTGjRsbPIZlqVrjXLpusmdPCaFQToMNgK5AUzaW1fI2d1NKEYnsQdfNgx7flDHRzCF1zhwtasy/W7dudO3aFU3T6NatG9nZ2ZSUlKR+H4lEDukB5nl5BRQX76S8vKTB12mVpWiJCLa1/qDPcTQwTS95eQXNXQwhhIulJfxfeeUVvv76a+644w62b99ONBolGAyyYcMGOnfuzJIlSw5pwtcwTFq3bn/A1wWXzyL44RPsur5lhr8QQqRbWsL/8ssvZ/LkyYwaNQpN07j33nvRdZ1bbrkFy7IYPHgwJ598cjpO7dA9aMoC2wLdSN95hBCihUpL+Hu9Xn7zm9/U+vnLL7+cjtPVogyv84UdBz1wRM4phBAtSctcDnMgVeGvWYlmLogQQhydXBn+qZ6/FW/eggghxFHKleGP7gFAsyX8hRCiLq4Mf2X4nC+k5y+EEHVyZfhjVPX8ZcxfCCHq5MrwlzF/IYRomCvDH71qtY+M+QshRJ1cGf4qtdRTwl8IIeriyvCvHvNHxvyFEKJOrgx/VT3sY8WauSRCCHF0cmX4k9reQXr+QghRF1eGv5LtHYQQokHuDP+qO3yxZdhHCCHq4srwp+oOX+n5CyFE3VwZ/iq12keWegohRF1cGf6pLZ1lwlcIIerkyvCvXuqJLPUUQog6uTL85WEuQgjRMHeGv26gNF3G/IUQoh7uDH8AwycbuwkhRD1cHP4e6fkLIUQ93Bv+pk/G/IUQoh7uDX9dev5CCFEf94a/KWP+QghRH/eGv+GRh7kIIUQ9XBz+PnmYixBC1MO14a8MD5rs6imEEHVybfhLz18IIern4vD3ypi/EELUw9XhL49xFEKIurk6/OUB7kIIUTf3hr/plTF/IYSoR9rCv6ioiKFDh7JmzRrWr1/PqFGjGD16NDNmzMC27XSddi8Z8xdCiHqlJfwTiQTTp0/H7/cDcN999zFhwgReeOEFlFIsXLgwHaetyfCC3OErhBB1Skv4z5o1i5EjR9KmTRsAVq1aRf/+/QEYMmQIy5YtS8dpa1DS8xdCiHqZTX3AefPm0apVK84880yefvppAJRSaJoGQCgUoqys7IDHMQyNcDh4yOXQTWe1z+Eco6UxDD2j6lstE+stdc4c6ap3k4f/q6++iqZpvPvuu3zxxRdMnDiR3bt3p34fiUTIyck54HEsS1FSUnHI5Wile9GT8cM6RksTDgczqr7VMrHeUufMcbD1LijIbtTrmjz8n3/++dTXY8aM4Y477mD27NmsWLGCAQMGsHjxYgYOHNjUp63N8Di7eioFVVcdQgghHEdkqefEiROZM2cOV155JYlEgmHDhqX/pIbP+Vdu9BJCiFqavOe/r7lz56a+fu6559J5qtoMj/OvlXBW/gghhEhx8U1eTs9fHugihBC1uTf8dae3L8s9hRCiNteGvzKrhnok/IUQohbXhn/1OL8mE75CCFGL68Mf2dlTCCFqcX34a7KzpxBC1OL68JcxfyGEqM314S9LPYUQojb3hr+s9hFCiHq5N/x1GfMXQoj6uDb8VdUdvrLaRwghanNt+Ffv7SPr/IUQojYXh391z1/CXwgh9ufi8K/q+cuErxBC1OLi8K/ez1/CXwgh9ufi8JeevxBC1Me94V+9n7+M+QshRC3uDX+9+klestRTCCH2597w1zSU7pWlnkIIUQf3hj+gDI9s7yCEEHVoVPgvX7483eVID8MrG7sJIUQdGhX+c+bMSXc50kLpXun5CyFEHczGvEjTNMaPH0+3bt3Qdae9uPnmm9NasCZheGW1jxBC1KFR4X/ZZZeluxxpIWP+QghRt0YN+wwfPpyKigo++eQT9uzZw/e+9710l6tpyJi/EELUqVHhP336dDZu3MigQYPYvHkzU6dOTXe5moSM+QshRN0aNeyzfv16nn/+eQDOPfdcRo4cmdZCNRkZ8xdCiDo1qucfi8WIRqMAVFZWYllWWgvVVGTMXwgh6taonv/YsWO55JJL6NmzJ6tXr+amm25Kd7mahuFFS+5p7lIIIcRRp1HhX1BQwMsvv8zGjRvp1KkTeXl56S5Xk1C6V3b1FEKIOjT6Jq9wOMyJJ57YYoIfkDF/IYSoh6tv8lK6B0129RRCiFoaFf4jRozAMIxGH9SyLKZOncratWvRNI0777wTn8/HpEmT0DSNnj17MmPGjFRDki7K8IHs6imEELU0Kvz/9a9/8fvf/77RB33rrbcAePHFF1mxYgUPP/wwSikmTJjAgAEDmD59OgsXLuS88847tFI3luGRMX8hhKhDo8I/JyeHN954o8awT7du3ep9/bnnnstZZ50FwJYtW8jJyWHZsmX0798fgCFDhrB06dK0h7/SPdLzF0KIOjQq/IuKivjTn/6EUgqAeDzOSy+91PCBTZOJEyeyYMECHnvsMZYuXYqmaQCEQiHKysoafL9haITDwcYUr5736+ihEJoVP6zjtCSGoWdMXfeVifWWOmeOdNW7wfCfMGECjzzyCHPnzuWZZ57h6quvBmDMmDGNOvisWbO45ZZb+OEPf0gstnfiNRKJkJOT0+B7LUtRUlLRqPPUJRwOEk9oBK34YR2nJQmHgxlT131lYr2lzpnjYOtdUJDdqNc1OONaVFSU+vrtt99OfV3dg6/PX//6V5566ikAAoEAmqZxwgknsGLFCgAWL15Mv379GlXAw2J40ZQFdsu4I1kIIY6URg37AKkhn8Y4//zzmTx5Mj/60Y9IJpNMmTKF7t27M23aNB566CEKCwsZNmzYIRX4YKjqh7jbcdADaT+fEEK0FA2G/749/AP19vcVDAZ59NFHa/38ueeeO4iiNQHDB4BmxVGmhL8QQlRrMPxXr17Nr3/9a5RSNb5es2bNkSrfYVFGVc9f7vIVQogaGgz/Rx55JPX1vts4t6QtnaGq59/MRRFCiKNJg+FfvS6/pVK6E/7I07yEEKKG9O6v0Nz26fkLIYTYy9XhL2P+QghRN9eFv60U8z7eQjxpg17d85edPYUQYl+uC/81uyLc98ZqlqzZhaoe9pH9fYQQogbXhb/HcKpUXplMjfnLc3yFEKIm14V/0OM8d6AibqXu8JUJXyGEqMl94e+tDv+k8zAXkKWeQgixH9eFf6Cq5x+JW2BU9/xlzF8IIfbluvA3dA2fqTvDPp4QAHpkWzOXSgghji6uC39wxv0r4knsrA4k2pyC/7M/gbKbu1hCCHHUcGX4B7wGFTELNI3oKddhlq7Du3ZBcxdLCCGOGq4M/6DHcMb8gVj3C7GyOxH4+OlmLpUQQhw9XBn+gaphHwB0k+hJV+PdsgJzx8fNWzAhhDhKuDL8g16dSHX4A5W9R2J7swl8JL1/IYQAl4Z/wFM15l9FebOp7D0a3+p/YBR92YwlE0KIo0Ojn+HbkgS9BhXxmg9tj540jsBnf6TVi+eSbHUc8a5nk+hwOol2fVH+vGYqqRBCNA9Xhn/AY9QY9gGwszuye+RCfGv/i3f9mwQ+fobgyv8DIJl7DLGeI4ie+nOUN6s5iiyEEEeUK8PfWedv1fq5nduV6CnXEj3lWohH8Oz8GHP7Sryb3yX0/iMEVj1HpP/NVPYalbo7WAgh3MidY/5eg1jSJmk38OReb4hExzOI9h1P6fDnKL5sPsm87mS/PYW8l4bh2bT0yBVYCCGOMFeGf/XOnpWJ2r3/+iTb9aV0xCuUXvgMWjJK+G9Xkv36eIziNaD2aUSUQt+zEaPk26YuthBCHDGuHPYJePdu65zlO4gqahrxwmHs7jKE4Ae/JbjySfzf/A3bn0eizSlgeDG3r8So2AFAMu9YYj0uJnbsCKxwYf3HVTZ6ZDt2qB1o2uFUTQghmoQrwz+1p/9B9PxrMANUDLiFyl4j8W5chLl9JZ7tH4GdINH5TCranQbKxrf67wT/9zDB9x+hss8YIgNuqblySCk8G98mtHwWnp2fkmhzMtFTriPW/SLQXfnRCyFaCFcmUPW2ztFDDf8qdk4nKvtcBX2uqvP3lSf+BD2yjeAHj+P/bC6+b/5K9MSfgO5BS0Qwt3+Id8sKrOzORPr9Et8388l5/XqsUFus3GNQ3lyUP5dkq+NItjmJZMFJDa82Ujbe9W9i+3JJtv/OYdVNCJHZXBn+Qa8zlVHXip+mZofaUT5kJtE+V5H1zgxC7z8KgDJ82ME2lJ15N5V9fgSGl4r+v8a7dgH+r19FixZhlG1C2/kx/i//svd4vlyUL9cZamrbl3j3i0i0749n81JCy+7Bs2sVAPEOA6n4zq9IdDwDrEq0WBkYPrS4jjKDoBtpr7sQouVyZ/g3Uc//YFj5x1M64iW0eBnK8Ne9VFTTiRcOI144rOaPK3bh2fEx5q5VaBU70WOl6BU7CXz+AsFP/4DtyUJPlGNld2bPuY+iVxYT+PBJwn+7EqV7ajygvnXVv7YnCztYgB1sg/JmoVlxNCuG0k2scCFWXg+SeT2wwj2wszuA5sq5fyFEPVwZ/vtO+B5pypt98O8JtiZ+zDnEjzmn5i/iEbwb3sK7/i2s1r2InjAGqh5NGe1zFf4vX8Eo24jtzUb5cggG/UT3lKAlImiVJejRXegVO9ArdoDhQxleNCuGb/Xf0WOle89v+rGyu6B82SgziPJmYWV3xsrtip3dyZmktmJoVhLbH041KmgamhUDK4Hy58kNckK0IK4M/+bo+aeFN0S8x8XEe1xc+3emn8oTas5F+MNBoiUVBz6uUmjRIsyS1RjFa5z/yjagxSNoyQr0yDa8G95CS1YeVHFtfyusnM6g6WiVxeiVJShvNslWPbHyemJntUcZfpTpB8OL0k3QPejRXZjbPsCz7UP0yFbsUDvsrA5YOV1ItjmZRNtTsfJ6yFCWEE3IleEfSK32kad31UnTUMHWJIKtSXQYWPdrlEKv2I5ettn53vChdBM9VoIe2elcTeDMbSjDCXCjdANG2UZAw8rpgvLlosVKMXd/jXfTUucqoR62L5dEu9NIdDoDPbIdvXwLvm9WElg11zmP7kEZPqcB0E2UZoKug+5FC3ciO9gRK6s9emUJevkWZ2ltsDVWuDtWuJvTIMUjaIly9LItGHvWYZSuR/lbEe94OomOZ6BMH2bRl5hFXwCQLDiJRNtTSOb3Bk+gcZ+tstHiZc6T46r/S33uhrMaTJb7iqOAK8M/WDXsE22GYR/X0DSnBx5qV+PHh/yJ2hZavMxpAJKVaFYc7ASanUB5slIBXYOyMUrWYm5fiVn8NVgJsJNodhKU5RzTqsQb3YZn49v4I9uxfbnYWR2wQ20wyrfh3bys1hWMFWyDnXsMiU6D0SPbCKx6juAnz+z9fagdKIX/q1f3Fj/QGiu7I8oXRkuUo8XK0KzK1NULSqFX7karLEZT9X9KVrANyXZ9SRaciF6xA2PXl3vrphuge7Gy2jnzMuFClCdr7+fkzcbK6eJcXdlhzB1b0CuLUZ4Qibanguk/4J9Bq5pPssKFMs+T4VwZ/h5Dx2Noh77OXzQ93UD5wzSw4UZtmo6V1x0rrzv1XzNAOBykpKQCbKv20JCy0SPbnC89WShPqPZrrBjm9o/RVJJkfq/UvRp6+VbMHR9h7v4GvWwTRtlmtFgpypeDHWrrTOwry2nIgIT/O9iBfKeeulkVrlqqp68lY5i7PsPc9gG+b/+D7cnCyj+eWOGFKE/QadSsBEb5ZjzbV+L7Zj5aA5/YvnvRKsNHon1/kvnHo1kxtGQUqsoFoMXLMIu+wih3ruRsfx7xjoNItu8HyUr0ymJnsYLpR3mznc9JM1J/h0SHASQLTpSrFhdp8vBPJBJMmTKFzZs3E4/H+cUvfkGPHj2YNGkSmqbRs2dPZsyYga6nt9cR9JrS8880dc0JaDp2VoeG32f4SHboX+vHdlZ74lntiRde2EQF3Ec8Ap5gw2GarHR6/LrHuXckVoKxZwPGno0EfRBRWdj+PPSKXXg2LcG7aQmBbR+gPAGUGXDeV318w0+i/XeItv4xyt8Kz9b38Gx8B/+afwBVw3feHGdiP15WZ6OTzOtJ7NhLSRacgO3Pw/bnodkWWqwEPVaKHWhNMr+XbIrYQjR5+M+fP59wOMzs2bMpKSlhxIgRHH/88UyYMIEBAwYwffp0Fi5cyHnnndfUp64h6DWk5y+OXt6sjwcxAAAU9ElEQVTQgV9j+lHsHcpRgXySgXySbU8lEA4S32dyP97tPCIHcfrK3qOqJv53OUNL+85pKAXJKFr1fEUyim/t6/i+mkdoxQMNHlcZPpIFJzpzPoa3apWZD+UJgOHH9udhhbthhQtlu5NmpimlDupK/EAikQhKKbKysiguLubyyy8nHo+zePFiNE3jjTfeYOnSpcyYMaPB49i2jWUdetEufGwJPdqEmDPy1EM+RktjGDqWlXmT3JlY72arc9lWtD2boKIYokXONiX+MPhzYc9mtM3vo235AK1smzPsZMUhWQmJaJ1XE0oznCs20w/Z7VDZ7SFY4DQKSjmLEzxB8IbQfdnY/jxUMN85p+lzzq+bVa/Jcq6mTB8YXuc/FzQuB/u39ngatyquyXv+oZDToykvL+emm25iwoQJzJo1C63qjxAKhSgrKzvgcSxLOeO4hyjoNSiNxA/rGC1Nauw7w2RivZuvzrkQyoW6LlyyToAOw+r4BU6Q23H0iiKM0rUYJd86czHKRlMWWqLCWeUV2YZetG7fN6Ilos69K8kKDnaxr5XT1VlF1q6vs6y4fAtG2WanUTKdZcfK8DoNiO5xrnaSlWjJ6D6LEpKQjKJHi9Cju8GKOVcvecdi5XRyyhcvQ7MTJNr2JdF5cK2FEofjYP/WBQWNu9coLRO+W7duZfz48YwePZrhw4cze/bs1O8ikQg5OTnpOG0NQa9BdL+neQkhmommgeHDzu6And2BRKdBB32IcI6P0u1b966qsuJVE+4JZ5gqUeGsxLISTmgno5i7v8KzaQn+r+cBoDQ9NVmvWZXOSjDLWU2FsgDNmS8xAzXuRVGmDxXIJ5HXA6WbmCXf4ls9P3WzpF11c2fgsz8BztMBU8uLkxXYgXzs7E7YWR2c5caR7egVO50lzP48bH/YOadmgOEh2vtHzmR8GjV5+O/atYtx48Yxffp0Tj/9dAB69+7NihUrGDBgAIsXL2bgwHrWljehoNdgd3lDa0SEEC2KbqACrbACrQ7ufUqhl28BwA61bboddZVCS0Sc0NYNZ2nyri/wbnoHz7YPnGD3ZqFMP3rFToyyTXg2LkZ5s7FDbUm0Ow3sJHplMXr5VmcZtJ1Esy3iHU5veeH/f//3f+zZs4cnnniCJ554AoDbb7+dmTNn8tBDD1FYWMiwYfVcGjahoNds+Xf4CiEOn6ZhZ3dMy3FrbGmi6VgFfYgW9CHa9Gdrck0e/lOnTmXq1Km1fv7cc8819akaFPIZcoevEELUw7W3+Dlj/tLzF0KIurg4/J1hH7tpV7IKIYQruDj8DRQQS8rQjxBC7M+14R/yOtMZzbGnvxBCHO1cG/6pnT1lxY8QQtTi+vCXnr8QQtTm2vAP+ZxhH+n5CyFEbe4N/+qev4S/EELU4trwD1ZN+MpafyGEqM294e+Tnr8QQtTHveGfWuop6/yFEGJ/rg3/kCz1FEKIerk2/H2mjq7JsI8QQtTFteGvaRoBj2zuJoQQdXFt+EPVQ9wl/IUQohZXh3/AY8iwjxBC1MHV4R/0GDLhK4QQdXB1+Adk2EcIIerk6vCXnr8QQtTN1eEf8EjPXwgh6uLq8A96den5CyFEHVwd/rLaRwgh6ubq8A96nZu8lDzEXQghanB1+Ac8BpaCuCXhL4QQ+3J1+Ac9VZu7yaSvEELU4OrwD8jTvIQQok6uDv/qnr+EvxBC1OTq8K/u+cuwjxBC1OTq8JeevxBC1C0jwl96/kIIUZOrwz/L74T/1zvLm7kkQghxdHF1+HfI8XNWj3x+v3wDH20qbe7iCCHEUSNt4f/xxx8zZswYANavX8+oUaMYPXo0M2bMwLbtdJ22Bk3TmHHBcXTI9TP5H19QFIkfkfMKIcTRLi3h/7vf/Y6pU6cSi8UAuO+++5gwYQIvvPACSikWLlyYjtPWKctnMuv7vSmLJbn9n1+QtOVuXyGE0FQaNr7573//y3HHHcdtt93Gyy+/zJlnnsnixYvRNI033niDpUuXMmPGjAaPYds21mFsy2AYOpa19wpj3srNTJz3KVk+k5M75XJK5zBndM/ntC55GLp2yOc5muxf50yRifWWOmeOg623p2qhy4GYh1qghgwbNoxNmzalvldKoWlOwIZCIcrKyg54DMtSlJRUHHIZwuFgjfd/t1ses7/fm+Xri/l0yx6efHsNv120hnDAw+DCVgzpns+ArnkEvY374I5G+9c5U2RivaXOmeNg611QkN2o16Ul/Pen63tHlyKRCDk5OUfitLWc1bM1Z/VsDUB5LMnydcUsWr2LRat38Y9V2/EaGqd1DjO4MJ9BhXl0zA00SzmFECLdjkj49+7dmxUrVjBgwAAWL17MwIEDj8RpG5TlMzn3uALOPa6ApGXz0eY9vPNtEe+sKWL2m6uZ/SYc0ypA99YhqgfGbKWwbIWtINtvcnKHHE7plEthfhBdc8fQkRAiMxyR8J84cSLTpk3joYceorCwkGHDhh2J0zaaaej06xKmX5cwvzqrOxuKoyxdu5tl3+7m2yLncksDNA10TcPQNL7cUc5/vtgBQLbPpE/7bE5sn023/BC2rYhbNmWxJN8WVfDtrgq2l1XSKew0Jt3yg+QFPGT7TLL9Jp3DAbL9jftTWLZC10gNowkhxKFIy4RvU0gkrCYd829qSik2l1ayclMpn2zZw6ptZazZFWH/xUS5fpMeBSHaZvvYWFzJt0URInXccdwq6OGYVkGCXgND0zB0Db9HJ+AxCHgMdpbHWLOrgnW7K8gPeTm7Z2u+27M13VsHiVuKeNImEPJRVFJB0rKxFOga6Gh4TZ28oIccv+nKK5RMHAuWOmeOFj3m70aaptEpHKBTOMDwE9oBEIkn2VJaiUfX8ZgaIY9JbsCs0UtXSrGzPM6eWJLyyiQl0QQbiqOs3V3BhuIou8rjWFXDS5VJm4q4RUU8SX7IS/fWIU4/Jo/1xVHmfbyFFz/cfFBlNnSNLK+BrZwhLFN36tAlL0D7XD8eXUPXNHTNeRBOwOs0PBpQ3abl+Ezygh7CAQ9xy6Y8lqQsliQc8NA5HMDfyJUGQojmJeHfhEJek54FWQ2+RtM02mT7aJPtO6xzReJJlq0tZlckjtfQ8Bg6udl+krEEpq6ja05gK6WIJmyKowmKK+KUVSadgNc14kmbjSVRPtxUyvaqIazDoQHtcnyEvCaWrbCUwtCcKxifqeMxdAxNQ9chaSnK4xblsSQJy8Zr6HhTr3EaqoDHoDA/RM+CEF3yAthKkbAUCdvGtqm6Z0PRKhzEjiXwmwZxyyaasIglbUI+g1ZBL3kBD0lbURZLUh5LYugaIa9J0GsQ8hr4TT3VQCctm/KY5TSUPkOG14RrSfi3UCGvyXnHFdT42eFcFttKoZTTWCRtRWXCpiJhEU1YKJwhJKWgrDLJ7miCkoo4XlMn22eS5TMpisRZvzvK+uIKYkkbs+oqwlbOFUx1IFdPmpu6Rq7fpGOuH6+hpYau4lbVaxQUVyT4y8bNR+QxnAGPjq0glty7ntpn6rQOeckLegh6DIJeA13TUlc7CUuRGzAJBzz4TJ1d5XF2lscprUxg6hp+j9OwOI2M09hUN9Qew2mgdU1D00gN9em6ho4zv6RpGn5TJ8dvku33kO1zjpPlMym1YfuucqIJC01zPstcv4eQz0hdve3fcNlKEYlZVCSsVH0MXUMpRSxpk7BU6mfC/ST8BeCEEBqAhmmA32MQxtPcxSJpKzYWR9lcGsXQNbyGjqlrTuOia2iAx+9lZ3GEyoSNx9QJeHR8pkF5LElxRYLdFXG8xt6GylKKSDxJJOY0bhVxJxB1zentZ3lNErZiV3mcXZEYJdEE0YTN7ooEllJkVw19eXSdPZUJvt1VQTRh0TrLyzFVk/kJyyaWtKuG7pLsroizscQiYTkhm7Ds1PCbUqSG+ixb0VRNnaFreHSnsdE0Z3nz/nNSHkMjsU/jqgE5fpPcgAePoaHhNErd8oOc0jGHkzrk4DV0yqoawOQ+780NeOgSDtQa6hRHJwl/cVQzq4KnW36w3teEw0FKcg9vGO1oo5SzpDiasCiLJdlT6QxZlccsIvEkgYAXlbQIVl2xlFYmKI0mqYhbNRqShKVI2jZKQZbfJNdv4vcYVCYsInGLyoSF19DxewxMXaMs5sxDlUaTWEqhqoba/rehJLW67UCyfSats7wEPQYBj06Wz3SuXnwecgNOGXL8zmo3j6nh0XVMw2nIgapG2CTbZ6auZBr6nGJJG98+Q3eicST8hTgKaZqGoTn3o2T5TNrvd1/kkV75Ur267dOte1CKqjA38RjODZwK2B2Js7EkyobiKCXRBBVx58pqY0mUssokpZXJGsNqB0vXwG8a+D06XkMnmnDmjCzlDNG1zfbRJssLQDRhU5m0KAj56N46RI+CIAGPczUYiVvO0KOh49Gd1XCBqobKshVFFQmKInEq4lbVEm8NUwev6VxRBjx7ryL9Hp1owlmYkbRtuoQDFLYOEfAY2EqxoyzG9rIYftNwGr6AB429V3r7Xs0e6cZLwl8IcUD7rm47HLGkTWk0kbqSSdg2cUvVGD6ylKq6ynGG5mzlDIV5vSalkRiVCZtY0iLoNcn2OSvSSqJJtpfF2FkeQ9OcmzBbG162l8X4y0eHNm/kM6satqp5sMbuCakBBVleSqKJgzqvx9jbENxwZjdGnNT+oMt8MCT8hRBHjM/UD3m126Fe7SRtxaaSKElbkVU1+W7oWtX8i9P4RBPOEJiua7QKemkV9KSualLHsZw5nMqERVnMGY6LJiwC1ZPnmsa63RWs3hlhY0mU/JCXTmE/7XL8xJM2JVWNHjjzMbpGamguXjUXlLSdf3sWhA66ngdLwl8I4WqmrnFMq7rmjA7unhTT0MkynDmM1vWs6O6WH+Tsqv3DjnaufpKXEEKIukn4CyFEBpLwF0KIDCThL4QQGUjCXwghMpCEvxBCZCAJfyGEyEAS/kIIkYGO2id5CSGESB/p+QshRAaS8BdCiAwk4S+EEBlIwl8IITKQhL8QQmQgCX8hhMhAEv5CCJGBXPUwF9u2ueOOO/jqq6/wer3MnDmTrl27Nnex0iKRSDBlyhQ2b95MPB7nF7/4BT169GDSpElomkbPnj2ZMWMGuu6+9r2oqIgf/OAH/P73v8c0zYyo81NPPcWbb75JIpFg1KhR9O/f39X1TiQSTJo0ic2bN6PrOnfffbfr/9Yff/wxDz74IHPnzmX9+vV11vXxxx9n0aJFmKbJlClTOOmkkw79hMpF/vvf/6qJEycqpZRauXKl+vnPf97MJUqfV155Rc2cOVMppVRxcbEaOnSouu6669Ty5cuVUkpNmzZNvf76681ZxLSIx+Pq+uuvV+eff75avXp1RtR5+fLl6rrrrlOWZany8nL12GOPub7eCxYsUDfddJNSSqklS5aoG264wdV1fvrpp9XFF1+srrjiCqWUqrOun332mRozZoyybVtt3rxZ/eAHPzisc7qn2QQ++OADzjzzTABOOeUUPvvss2YuUfpccMEF/PKXvwScB0wbhsGqVavo378/AEOGDGHZsmXNWcS0mDVrFiNHjqRNmzYAGVHnJUuWcOyxxzJ+/Hh+/vOfc9ZZZ7m+3t26dcOyLGzbpry8HNM0XV3nLl26MGfOnNT3ddX1gw8+YPDgwWiaRocOHbAsi927dx/yOV0V/uXl5WRl7X24pmEYJJPJZixR+oRCIbKysigvL+emm25iwoQJKKXQNC31+7KysmYuZdOaN28erVq1SjXwgOvrDFBcXMxnn33Go48+yp133sktt9zi+noHg0E2b97MhRdeyLRp0xgzZoyr6zxs2DBMc+8ofF113T/fDvczcNWYf1ZWFpFIJPW9bds1PlC32bp1K+PHj2f06NEMHz6c2bNnp34XiUTIyclpxtI1vVdffRVN03j33Xf54osvmDhxYo2ejxvrDBAOhyksLMTr9VJYWIjP52Pbtm2p37ux3s8++yyDBw/m17/+NVu3bmXs2LEkEonU791Y533tO5dRXdf98y0SiZCdnX3o5zisEh5l+vbty+LFiwH46KOPOPbYY5u5ROmza9cuxo0bx6233srll18OQO/evVmxYgUAixcvpl+/fs1ZxCb3/PPP89xzzzF37lx69erFrFmzGDJkiKvrDHDaaafxzjvvoJRi+/btRKNRTj/9dFfXOycnJxVsubm5JJNJ1///va+66tq3b1+WLFmCbdts2bIF27Zp1arVIZ/DVbt6Vq/2+frrr1FKce+999K9e/fmLlZazJw5k3//+98UFhamfnb77bczc+ZMEokEhYWFzJw5E8MwmrGU6TNmzBjuuOMOdF1n2rRprq/zAw88wIoVK1BK8atf/YpOnTq5ut6RSIQpU6awc+dOEokEP/7xjznhhBNcXedNmzZx88038/LLL7N27do66zpnzhwWL16MbdtMnjz5sBpAV4W/EEKIxnHVsI8QQojGkfAXQogMJOEvhBAZSMJfCCEykIS/EEJkIPfeASVEI6xYsYIJEybQo0eP1M/y8vJ47LHHDuu4kyZN4qKLLmLIkCGHW0Qh0kLCX2S8gQMH8vDDDzd3MYQ4oiT8hajDmDFj6NatG2vXrkUpxcMPP0xBQQH3338/H3zwAQAXX3wxY8eOZd26dUydOpVEIoHf7081JC+99BL/7//9P8rLy7njjjsOb/tdIZqYhL/IeMuXL2fMmDGp74cOHQo424XcddddPP/88zz11FMMGjSITZs28fLLL5NMJhk9ejQDBw7kkUce4dprr2XIkCEsXLiQzz//HIA+ffpw/fXXM2/ePObNmyfhL44qEv4i49U17PP2228zcOBAwGkE3nzzTdq1a0e/fv3QNA2Px8PJJ5/MmjVrWLt2LaeeeioA55xzDgD/+Mc/6NOnDwCtW7emsrLyCNZIiAOT1T5C1KP6eRAffvghPXr0oHv37qkhn0QiwcqVK+natSvdu3fn008/BWD+/PnMnTsXILUlrxBHI+n5i4y3/7APQGVlJa+99hrPPvssgUCABx54gLy8PN577z2uvPJKEokEF1xwAX369OG2225j+vTpPPnkk/j9fmbPns2qVauaqTZCNI5s7CZEHap3DXXrrrBCyLCPEEJkIOn5CyFEBpKevxBCZCAJfyGEyEAS/kIIkYEk/IUQIgNJ+AshRAb6/8EswO3ITiSNAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(history_Boulder.history['mae'])\n",
    "plt.plot(history_Boulder.history['root_mean_squared_error'])\n",
    "plt.title('mae and rmse per epoch')\n",
    "plt.ylabel('Error')\n",
    "plt.xlabel('Epoch')\n",
    "plt.legend(['mae', 'rmse'], loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## V.II Model Testing,  Boulder Specific Model on Boulder 2018 data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test on 2018 data only"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>Hour</th>\n",
       "      <th>Minute</th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Clearsky DHI</th>\n",
       "      <th>Clearsky DNI</th>\n",
       "      <th>Clearsky GHI</th>\n",
       "      <th>Cloud Type</th>\n",
       "      <th>...</th>\n",
       "      <th>Solar Zenith Angle</th>\n",
       "      <th>Surface Albedo</th>\n",
       "      <th>Pressure</th>\n",
       "      <th>Precipitable Water</th>\n",
       "      <th>Wind Direction</th>\n",
       "      <th>Wind Speed</th>\n",
       "      <th>Global Horizontal UV Irradiance (280-400nm)</th>\n",
       "      <th>Global Horizontal UV Irradiance (285-385nm)</th>\n",
       "      <th>timestamps</th>\n",
       "      <th>Cloud Cover</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>30</td>\n",
       "      <td>-14.3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>98.21</td>\n",
       "      <td>0.67</td>\n",
       "      <td>711</td>\n",
       "      <td>0.2</td>\n",
       "      <td>261</td>\n",
       "      <td>0.9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1514782800</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "      <td>-15.6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>109.06</td>\n",
       "      <td>0.67</td>\n",
       "      <td>711</td>\n",
       "      <td>0.2</td>\n",
       "      <td>254</td>\n",
       "      <td>1.2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1514786400</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>30</td>\n",
       "      <td>-16.3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>120.35</td>\n",
       "      <td>0.67</td>\n",
       "      <td>711</td>\n",
       "      <td>0.1</td>\n",
       "      <td>252</td>\n",
       "      <td>1.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1514790000</td>\n",
       "      <td>0.64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>30</td>\n",
       "      <td>-16.7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>131.81</td>\n",
       "      <td>0.67</td>\n",
       "      <td>711</td>\n",
       "      <td>0.1</td>\n",
       "      <td>256</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1514793600</td>\n",
       "      <td>0.64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>30</td>\n",
       "      <td>-17.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>143.11</td>\n",
       "      <td>0.67</td>\n",
       "      <td>711</td>\n",
       "      <td>0.1</td>\n",
       "      <td>261</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1514797200</td>\n",
       "      <td>0.64</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Year  Month  Day  Hour  Minute  Temperature  Clearsky DHI  Clearsky DNI  \\\n",
       "0  2018      1    1     0      30        -14.3             0             0   \n",
       "1  2018      1    1     1      30        -15.6             0             0   \n",
       "2  2018      1    1     2      30        -16.3             0             0   \n",
       "3  2018      1    1     3      30        -16.7             0             0   \n",
       "4  2018      1    1     4      30        -17.0             0             0   \n",
       "\n",
       "   Clearsky GHI  Cloud Type  ...  Solar Zenith Angle  Surface Albedo  \\\n",
       "0             0           0  ...               98.21            0.67   \n",
       "1             0           0  ...              109.06            0.67   \n",
       "2             0           0  ...              120.35            0.67   \n",
       "3             0           0  ...              131.81            0.67   \n",
       "4             0           0  ...              143.11            0.67   \n",
       "\n",
       "   Pressure  Precipitable Water  Wind Direction  Wind Speed  \\\n",
       "0       711                 0.2             261         0.9   \n",
       "1       711                 0.2             254         1.2   \n",
       "2       711                 0.1             252         1.1   \n",
       "3       711                 0.1             256         1.0   \n",
       "4       711                 0.1             261         1.0   \n",
       "\n",
       "   Global Horizontal UV Irradiance (280-400nm)  \\\n",
       "0                                          NaN   \n",
       "1                                          NaN   \n",
       "2                                          NaN   \n",
       "3                                          NaN   \n",
       "4                                          NaN   \n",
       "\n",
       "   Global Horizontal UV Irradiance (285-385nm)  timestamps  Cloud Cover  \n",
       "0                                          NaN  1514782800         0.00  \n",
       "1                                          NaN  1514786400         0.00  \n",
       "2                                          NaN  1514790000         0.64  \n",
       "3                                          NaN  1514793600         0.64  \n",
       "4                                          NaN  1514797200         0.64  \n",
       "\n",
       "[5 rows x 26 columns]"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df = pd.read_csv(os.path.join(_Testing_Data_Dir, 'Boulder_CO', 'psm_CO_Boulder2018.csv'))\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_size = 6\n",
    "test_input_arry = np.array([], dtype=np.float64).reshape(0,input_size)\n",
    "test_output_arry = np.array([], dtype=np.float64)\n",
    "test_input_arry, test_output_arry = combine_training_arrays(test_df, test_input_arry, test_output_arry)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "274/274 [==============================] - 0s 956us/step - loss: 978.8859 - mae: 12.8149 - root_mean_squared_error: 31.2872\n"
     ]
    }
   ],
   "source": [
    "score = model_Boulder.evaluate(test_input_arry, test_output_arry, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [],
   "source": [
    "#12.8149 is better than 15"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## V.III Model Testing Miami Specific\n",
    "Lets train on NRSDB + Darksky data from 08-18"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[]\n",
      "[]\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Miami_FL/psm_FL_Miami2009.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Miami_FL/psm_FL_Miami2008.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Miami_FL/psm_FL_Miami2018.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Miami_FL/psm_FL_Miami2017.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Miami_FL/psm_FL_Miami2016.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Miami_FL/psm_FL_Miami2014.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Miami_FL/psm_FL_Miami2015.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Miami_FL/psm_FL_Miami2011.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Miami_FL/psm_FL_Miami2010.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Miami_FL/psm_FL_Miami2012.csv\n",
      "/Users/tuomastalvitie/Documents/GRIP/Diffuse:Direct/solarIrradiencePredictor/Testing_Data/Miami_FL/psm_FL_Miami2013.csv\n"
     ]
    }
   ],
   "source": [
    "input_size = 6\n",
    "input_arry = np.array([], dtype=np.float64).reshape(0,input_size)\n",
    "output_arry = np.array([], dtype=np.float64)\n",
    "print(input_arry)\n",
    "print(output_arry)\n",
    "from glob import glob\n",
    "for path in glob(_Testing_Data_Dir+'/Miami_FL'):\n",
    "    for file in glob(path+'/psm_'+'*'):\n",
    "        print(file)\n",
    "        df = pd.read_csv(file)\n",
    "        input_arry, output_arry = combine_training_arrays(df, input_arry, output_arry)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function tensorflow.python.framework.ops.reset_default_graph()>"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras.backend.clear_session()\n",
    "tf.keras.backend.clear_session()\n",
    "tf.compat.v1.reset_default_graph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_Miami = keras.Sequential()\n",
    "model_Miami.add(keras.layers.Dense(300, activation=\"relu\"))\n",
    "model_Miami.add(layers.Dense(200, activation=\"relu\"))\n",
    "model_Miami.add(layers.Dense(100, activation=\"relu\"))\n",
    "model_Miami.add(layers.Dense(50, activation=\"relu\"))\n",
    "model_Miami.add(layers.Dense(25, activation=\"relu\"))\n",
    "model_Miami.add(layers.Dense(1, activation='relu'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 2633.1914 - mae: 26.4663 - root_mean_squared_error: 51.3146\n",
      "Epoch 2/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 296.6196 - mae: 8.2286 - root_mean_squared_error: 17.2226\n",
      "Epoch 3/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 242.9046 - mae: 7.1133 - root_mean_squared_error: 15.5854\n",
      "Epoch 4/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 223.9117 - mae: 6.7212 - root_mean_squared_error: 14.9637: 0s - loss: 221.0543 - mae: 6.6691 - root_mean_squared_error: 1\n",
      "Epoch 5/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 214.0983 - mae: 6.5390 - root_mean_squared_error: 14.6321\n",
      "Epoch 6/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 208.1112 - mae: 6.3846 - root_mean_squared_error: 14.4261\n",
      "Epoch 7/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 205.5375 - mae: 6.3092 - root_mean_squared_error: 14.3366\n",
      "Epoch 8/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 200.0863 - mae: 6.1708 - root_mean_squared_error: 14.1452\n",
      "Epoch 9/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 203.2010 - mae: 6.2142 - root_mean_squared_error: 14.2549: 0s - loss: 204.2701 - mae: 6.2317 - root_mean_squared_erro\n",
      "Epoch 10/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 193.9835 - mae: 6.0385 - root_mean_squared_error: 13.9278\n",
      "Epoch 11/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 194.3957 - mae: 6.0348 - root_mean_squared_error: 13.9426\n",
      "Epoch 12/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 192.6377 - mae: 5.9820 - root_mean_squared_error: 13.8794\n",
      "Epoch 13/100\n",
      "1506/1506 [==============================] - 4s 2ms/step - loss: 202.9877 - mae: 6.1727 - root_mean_squared_error: 14.2474: 2s - loss: 2\n",
      "Epoch 14/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 192.4914 - mae: 5.9814 - root_mean_squared_error: 13.8741\n",
      "Epoch 15/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 190.6890 - mae: 5.8997 - root_mean_squared_error: 13.8090: 2s - loss: 183.5210 - mae: 5.6966 - root_mea\n",
      "Epoch 16/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 191.7949 - mae: 5.9418 - root_mean_squared_error: 13.8490\n",
      "Epoch 17/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 194.0050 - mae: 6.0089 - root_mean_squared_error: 13.9286: 1s - loss: 1\n",
      "Epoch 18/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 189.9124 - mae: 5.8930 - root_mean_squared_error: 13.7809\n",
      "Epoch 19/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 190.1159 - mae: 5.8793 - root_mean_squared_error: 13.7883: 1s - l\n",
      "Epoch 20/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 194.2146 - mae: 5.9737 - root_mean_squared_error: 13.9361: 1s - loss: 198.3057 - mae: 6.0728 - root_mean_squared_ - ETA: 0s - loss: 196.1225 - mae: 6\n",
      "Epoch 21/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 187.6270 - mae: 5.8716 - root_mean_squared_error: 13.6977\n",
      "Epoch 22/100\n",
      "1506/1506 [==============================] - 4s 3ms/step - loss: 190.0714 - mae: 5.9136 - root_mean_squared_error: 13.7866\n",
      "Epoch 23/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 189.1253 - mae: 5.8669 - root_mean_squared_error: 13.7523\n",
      "Epoch 24/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 185.8530 - mae: 5.8090 - root_mean_squared_error: 13.6328: 0s - loss: 182.7842 - mae: 5.7548 - root_mean_squa\n",
      "Epoch 25/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 188.1480 - mae: 5.9200 - root_mean_squared_error: 13.7167: 0s - loss: 187.9987 - mae: 5.8778 - root_mean_squa\n",
      "Epoch 26/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 188.0369 - mae: 5.8510 - root_mean_squared_error: 13.7127: 0s - loss: 186.5719 - mae: 5.8564 - root\n",
      "Epoch 27/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 185.3519 - mae: 5.7996 - root_mean_squared_error: 13.6144\n",
      "Epoch 28/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 188.3575 - mae: 5.8612 - root_mean_squared_error: 13.7243\n",
      "Epoch 29/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 186.1870 - mae: 5.8214 - root_mean_squared_error: 13.6450\n",
      "Epoch 30/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 183.7513 - mae: 5.7790 - root_mean_squared_error: 13.5555: 0s - loss: 185.1480 - mae: 5.8007 - root_mean_\n",
      "Epoch 31/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 183.8016 - mae: 5.7791 - root_mean_squared_error: 13.5573\n",
      "Epoch 32/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 187.1422 - mae: 5.8267 - root_mean_squared_error: 13.6800\n",
      "Epoch 33/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 182.6832 - mae: 5.7385 - root_mean_squared_error: 13.5160\n",
      "Epoch 34/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 187.5053 - mae: 5.8625 - root_mean_squared_error: 13.6933\n",
      "Epoch 35/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 186.2347 - mae: 5.8210 - root_mean_squared_error: 13.6468\n",
      "Epoch 36/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 184.1919 - mae: 5.7408 - root_mean_squared_error: 13.5717\n",
      "Epoch 37/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 182.7231 - mae: 5.7239 - root_mean_squared_error: 13.5175\n",
      "Epoch 38/100\n",
      "1506/1506 [==============================] - 4s 2ms/step - loss: 185.5048 - mae: 5.7876 - root_mean_squared_error: 13.6200\n",
      "Epoch 39/100\n",
      "1506/1506 [==============================] - 4s 2ms/step - loss: 181.1442 - mae: 5.7045 - root_mean_squared_error: 13.4590\n",
      "Epoch 40/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 184.3968 - mae: 5.7640 - root_mean_squared_error: 13.5793\n",
      "Epoch 41/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 186.1589 - mae: 5.7940 - root_mean_squared_error: 13.6440: 0s - loss: 185.0023 - mae: 5.7512 - root_mean_sq\n",
      "Epoch 42/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 181.3674 - mae: 5.7115 - root_mean_squared_error: 13.4673: 1s - loss: 173.7661 - mae: 5.5 - ETA: 0s - loss: 181.0788 - mae: 5.6931 - root_mean_sq\n",
      "Epoch 43/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 187.6123 - mae: 5.8369 - root_mean_squared_error: 13.6972\n",
      "Epoch 44/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 180.0558 - mae: 5.6486 - root_mean_squared_error: 13.4185\n",
      "Epoch 45/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 180.1421 - mae: 5.6424 - root_mean_squared_error: 13.4217\n",
      "Epoch 46/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 183.6418 - mae: 5.7437 - root_mean_squared_error: 13.5514: 1s - loss: 183.1800 - ma\n",
      "Epoch 47/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 181.6501 - mae: 5.7269 - root_mean_squared_error: 13.4778: 0s - loss: 180.5589 - mae: 5.7146 - root_mean_squared_error: - ETA: 0s - loss: 181.3714 - mae: 5.7345 - root_mean_squared_erro\n",
      "Epoch 48/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 179.8756 - mae: 5.6572 - root_mean_squared_error: 13.4118\n",
      "Epoch 49/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 178.7189 - mae: 5.6161 - root_mean_squared_error: 13.3686\n",
      "Epoch 50/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 178.3975 - mae: 5.6408 - root_mean_squared_error: 13.3566\n",
      "Epoch 51/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 180.5004 - mae: 5.6785 - root_mean_squared_error: 13.4350\n",
      "Epoch 52/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 177.5364 - mae: 5.5973 - root_mean_squared_error: 13.3243\n",
      "Epoch 53/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 180.1243 - mae: 5.6428 - root_mean_squared_error: 13.4210\n",
      "Epoch 54/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 179.7390 - mae: 5.6343 - root_mean_squared_error: 13.4067\n",
      "Epoch 55/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 180.6494 - mae: 5.6765 - root_mean_squared_error: 13.4406\n",
      "Epoch 56/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 176.8232 - mae: 5.5941 - root_mean_squared_error: 13.2975\n",
      "Epoch 57/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 176.9963 - mae: 5.5767 - root_mean_squared_error: 13.3040\n",
      "Epoch 58/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 178.6169 - mae: 5.6149 - root_mean_squared_error: 13.3648: 0s - loss: 175.8333 - mae: 5.5890 - root_mean_squared_error: 13.26 - ETA: 0s - loss: 176.0069 - mae: 5.5822 - root_m\n",
      "Epoch 59/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 177.4707 - mae: 5.5900 - root_mean_squared_error: 13.3218\n",
      "Epoch 60/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 176.9386 - mae: 5.5969 - root_mean_squared_error: 13.3018\n",
      "Epoch 61/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 176.7368 - mae: 5.5994 - root_mean_squared_error: 13.2942\n",
      "Epoch 62/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 177.7361 - mae: 5.5845 - root_mean_squared_error: 13.3318\n",
      "Epoch 63/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 178.0386 - mae: 5.6049 - root_mean_squared_error: 13.3431\n",
      "Epoch 64/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 178.8835 - mae: 5.6074 - root_mean_squared_error: 13.3747\n",
      "Epoch 65/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 175.3962 - mae: 5.5249 - root_mean_squared_error: 13.2437\n",
      "Epoch 66/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 174.8616 - mae: 5.5296 - root_mean_squared_error: 13.2235\n",
      "Epoch 67/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 176.3245 - mae: 5.5588 - root_mean_squared_error: 13.2787\n",
      "Epoch 68/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 175.4641 - mae: 5.5476 - root_mean_squared_error: 13.2463: 1s - loss: 180.7604 \n",
      "Epoch 69/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 176.2790 - mae: 5.5540 - root_mean_squared_error: 13.2770\n",
      "Epoch 70/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 175.5286 - mae: 5.5185 - root_mean_squared_error: 13.2487\n",
      "Epoch 71/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 174.2145 - mae: 5.5041 - root_mean_squared_error: 13.1990\n",
      "Epoch 72/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 173.3762 - mae: 5.5098 - root_mean_squared_error: 13.1672\n",
      "Epoch 73/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 173.3147 - mae: 5.4906 - root_mean_squared_error: 13.1649\n",
      "Epoch 74/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 174.0108 - mae: 5.4938 - root_mean_squared_error: 13.1913\n",
      "Epoch 75/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 175.4792 - mae: 5.5204 - root_mean_squared_error: 13.2469\n",
      "Epoch 76/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 172.0219 - mae: 5.4534 - root_mean_squared_error: 13.1157\n",
      "Epoch 77/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 173.4377 - mae: 5.4899 - root_mean_squared_error: 13.1696\n",
      "Epoch 78/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 171.3842 - mae: 5.4354 - root_mean_squared_error: 13.0914\n",
      "Epoch 79/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 177.4065 - mae: 5.5927 - root_mean_squared_error: 13.3194\n",
      "Epoch 80/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 172.3705 - mae: 5.4552 - root_mean_squared_error: 13.1290: 1s - loss: 171\n",
      "Epoch 81/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 175.6702 - mae: 5.5223 - root_mean_squared_error: 13.2541: 1s - loss: 168.5748 - ma - ETA: 0s - loss: 177.3295 - mae: 5.5417 - ro\n",
      "Epoch 82/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 173.7773 - mae: 5.5145 - root_mean_squared_error: 13.1825\n",
      "Epoch 83/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 174.4967 - mae: 5.5032 - root_mean_squared_error: 13.2097\n",
      "Epoch 84/100\n",
      "1506/1506 [==============================] - 2s 2ms/step - loss: 173.0802 - mae: 5.4772 - root_mean_squared_error: 13.1560\n",
      "Epoch 85/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 172.7567 - mae: 5.4835 - root_mean_squared_error: 13.1437\n",
      "Epoch 86/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 170.5160 - mae: 5.4219 - root_mean_squared_error: 13.0582\n",
      "Epoch 87/100\n",
      "1506/1506 [==============================] - 4s 2ms/step - loss: 173.3791 - mae: 5.4803 - root_mean_squared_error: 13.1674: 0s - loss: 170.9098 - mae: 5.4327 - root_mea\n",
      "Epoch 88/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 167.9872 - mae: 5.3636 - root_mean_squared_error: 12.9610: 0s - loss: 166.8847 - mae: 5.3514 - root_mea\n",
      "Epoch 89/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 172.4431 - mae: 5.4621 - root_mean_squared_error: 13.1318: 0s - loss: 171.8276 - mae: 5.4652 - root_mean_sq\n",
      "Epoch 90/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 172.3766 - mae: 5.4541 - root_mean_squared_error: 13.1292\n",
      "Epoch 91/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 173.8377 - mae: 5.4979 - root_mean_squared_error: 13.1848\n",
      "Epoch 92/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 171.4993 - mae: 5.4507 - root_mean_squared_error: 13.0958\n",
      "Epoch 93/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 170.8954 - mae: 5.4554 - root_mean_squared_error: 13.0727: 1s - loss:\n",
      "Epoch 94/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 170.5457 - mae: 5.4366 - root_mean_squared_error: 13.0593\n",
      "Epoch 95/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 171.7145 - mae: 5.4397 - root_mean_squared_error: 13.1040: 1s -\n",
      "Epoch 96/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 171.1251 - mae: 5.4294 - root_mean_squared_error: 13.0815: 1s - loss: 170.7934 - mae: 5.3 - ETA: 0s - loss: 173.1100 - mae: 5.4494 - root_m\n",
      "Epoch 97/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 173.2652 - mae: 5.4735 - root_mean_squared_error: 13.1630\n",
      "Epoch 98/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 168.6124 - mae: 5.3828 - root_mean_squared_error: 12.9851\n",
      "Epoch 99/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 172.9618 - mae: 5.4700 - root_mean_squared_error: 13.1515: 1s\n",
      "Epoch 100/100\n",
      "1506/1506 [==============================] - 3s 2ms/step - loss: 170.6965 - mae: 5.4439 - root_mean_squared_error: 13.0651\n"
     ]
    }
   ],
   "source": [
    "#Compile and train\n",
    "model_Miami.compile(optimizer=Adam(lr=0.0001), \n",
    "              loss='mse',\n",
    "              metrics=['mae', keras.metrics.RootMeanSquaredError()])\n",
    "\n",
    "history_Miami = model_Miami.fit(input_arry, output_arry, epochs=100, batch_size=64)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Mae down to 5.4439"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAESCAYAAAAVLtXjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAIABJREFUeJzt3Xl8VNX9//HXXWbPMiGERTZZy6JSleICwtfaitpatagoNdJK1VaqTa3KUgQVFxStC3612G+t/QJW/bm09tvailhFRXDfEC0gu6zZM+ude8/vj0kGQhbCEpLc+TwfDx8kmZl7z5nE9zn3c++cqymlFEIIIbKK3tYNEEIIceRJ+AshRBaS8BdCiCwk4S+EEFlIwl8IIbKQhL8QQmQhCX/hWv/85z8pLi5u62a4wvPPP8/VV1/d1s0Qh5GEvxBCZCGzrRsgOo6VK1fy29/+li5durBmzRoCgQDXXnstCxcuZP369Zx55pnMmDEDx3G48847+fjjj4lEIiiluP322znxxBNJJpPce++9vPvuu9i2zdChQ5k5cyY5OTn19rV7925mzZpFaWkpu3btokePHjzwwAMUFhby7W9/mwsuuIC3336bbdu2cfbZZ3PTTTcB8OCDD/K3v/2NcDhMnz59muzHHXfcQTAYJBqNcuONN/Lwww/vt1+RSITp06ezceNGdF1n2LBh3Hbbbei6zquvvsqjjz6KZVn4/X6mTp3K8ccfX2+/W7Zsobi4mJEjR/LFF1+glGLWrFmMGDECgEcffZSXX34Zx3Ho0aMHs2fPpmvXrhQXF5Ofn89XX33FpZde2uBo5v/9v//Hn//8ZxzHIRwOc/PNN9O/f3+mTZuGpmmsW7eOsrIyRo0axcyZM/F4PLz33nvcc889xGIxPB4PJSUljBkzBoAFCxbwwgsvYJomffr0Ye7cuQDs2rWLq666im3btmEYBvfddx/9+/c/9D8s0TaUEC20YsUKNWTIELVq1SqllFKTJ09WEyZMUIlEQpWWlqphw4ap7du3qw8++EBde+21yrZtpZRSCxYsUFdffbVSSqn58+eruXPnKsdxlFJK3XfffWr27NkN9vXEE0+oBQsWKKWUchxH/fSnP1V/+MMflFJKnX766Wru3LlKKaW2b9+ujj32WLVp0ya1ZMkSdc4556jq6mplWZa66qqr1GWXXdZoPwYPHqy2bNlyQP164YUX1BVXXKGUUiqVSqnf/OY3asOGDWr9+vXq+9//viorK1NKKfWf//xHjRo1SkUikXr73bx5sxo0aJB68cUXlVJKvfbaa2rUqFEqmUyqF154QZWUlCjLspRSSj311FPqpz/9qVJKqcsuu0xNnz690d/JypUr1cSJE1U0GlVKKfXGG2+os88+Wyml1NSpU9X555+vampqVCKRUD/60Y/UwoULVVlZmTrllFPURx99lGnvyJEj1aZNm9Qrr7yizjzzTFVRUaGUUurOO+9UjzzyiHruuefUiBEj1IYNG5RSSs2ZM6fJNomOQWb+4oD07NmToUOHAtC7d29yc3Pxer106tSJUChEZWUlxx9/PPn5+Tz11FNs3ryZlStXEgqFAHjttdeorq5m+fLlAFiWRWFhYYP9TJo0iffee48//vGPbNiwgTVr1jB8+PDM42eccQYAXbt2pbCwkMrKSt5++22++93vZo4ixo8fz8KFCxvtR/fu3enRo8cB9evEE0/k/vvvp7i4mFNPPZVJkybRp08fFi9ezM6dO/nxj3+c2Z6maWzatInBgwfX229+fj7nnnsuAGPHjsUwDL788kv+/e9/8+mnnzJ+/HgAHMchFotlXld3dLCv1157jY0bN3LJJZdkflZZWUlFRQUAF1xwQea9P++881i6dCm9evWid+/emfdz4MCBnHDCCbzzzjusXr2as846i/z8fACmT58OpGv+xx13XOZoasiQISxZsqTRNomOQcJfHBCv11vve9Ns+Cf02muvcccdd/CTn/yEM844g379+vHiiy8C6VCbMWMGY8eOBSASiZBIJBpsY968eXzyySeMHz+ek046iVQqhdprGSqfz5f5WtM0lFKZf+sYhtFkP4LB4AH3q1evXixZsoSVK1eyYsUKfvKTnzBz5kwcx+GUU07hgQceyDx327ZtdOnSpcE29m2T4zgYhoHjOPz0pz9l4sSJACSTSSorK5ts796vP++887jxxhsz3+/cuTMT3nvvTymFrus4jtNgO0opUqkUhmGgaVrm51VVVVRVVTV4T/Z9r0XHIyd8xWH31ltvcfrppzNx4kSOPfZYXnnlFWzbBmD06NEsXryYZDKJ4zjcfPPN/Pa3v22wjTfffJNJkyZx/vnnU1hYyPLlyzPbaMppp53GP//5T6qqqnAch7/+9a+HtV9PPvkk06dPZ/To0dx4442MHj2aNWvWcPLJJ/PWW2+xbt06AF5//XV+8IMfNDqolZWVsWzZMgBeffVVPB4PgwYNYvTo0Tz77LPU1NQA6XMXdecxmjNq1Cj+/ve/s3PnTgD+/Oc/M2nSpMzjL730EslkkkQiwQsvvMDpp5/O8OHDWb9+PZ988gkAa9as4d1332XkyJGceuqpLFmyJNOO+fPn88QTTxz8mybaLZn5i8Pukksu4YYbbuDcc8/FMAxGjBiROZF5zTXXcPfdd3PBBRdg2zZDhgxh2rRpDbYxZcoU7rnnHh555BEMw+CEE05g06ZNze537NixfPnll4wfP568vDwGDx5MeXn5YevX+eefzzvvvMM555xDIBDgqKOO4vLLLyc/P5/bbruN66+/HqUUpmny6KOPNjpb9/l8/PWvf+Xee+/F7/fz3//93xiGwUUXXcSOHTu4+OKL0TSN7t27Z060Nue0007jyiuv5IorrkDTNHJycnj44Yczs3e/38/EiROpqqpi3LhxjB8/Hl3XefDBB5kzZw7xeBxN07jrrrvo27cvffv2Ze3atVx66aUADBgwgDlz5vDyyy8ftvdRtA+akmM3IY6ILVu2cO655/Lhhx8ekf1NmzaNgQMHMnny5COyP9GxSNlHCCGykMz8hRAiC8nMXwghspCEvxBCZKF2e7WP4zjY9sFXpAxDO6TXd0TZ2GfIzn5Ln7PHgfbb42n68y17a7fhb9uKioroQb8+HA4e0us7omzsM2Rnv6XP2eNA+11UlNui50nZRwghspCEvxBCZCEJfyGEyELttubfGNtOUV6+i1Qqud/n7tjRcReeMk0vBQVFGEaH+vUIITqQDpUu5eW78PuDhELd6q082BjD0LHthqsXtndKKSKRKsrLd9G5c/e2bo4QwqU6VNknlUoSCuXtN/g7Mk3TCIXyWnR0I4QQB6tDhT/g6uCvkw19FEK0rQ4X/vulFFqsDFTHK/kIIcSR4r7wT8UxqjdDoqatWyKEEO2W+8I/Q2b+QgjRlA51tc/e/r5qBy9+tr3hA8pBS0XBXIPSDqx7PzimG98b1rXZ5/zjH3/jrbeWkUgkKC3dzUUXXcobb7zO+vXrmDLll+zcuYPXX/83sViMcDjMnXfei6ZpzJt3J1u2bMZxHK688ueccELjN+QWQogjodXC/4ILLiAnJweAnj17MmHCBO644w4Mw2D06NH84he/aK1dt7poNMr99/83r7zyL55++kkee+wJPvzwfZ5+ejHf+MYQHnjgEXRd5/rrf8Hq1atYt24t+flhpk+fRWVlBVOmXMWiRc+0dTeEEFmsVcI/kUiglGLhwoWZn5133nnMnz+fXr16cdVVV/H5558zdOjQg97H94Z1bXyWbicxS1ej8ntj+woOevvNGTjwGwDk5ORy9NF90TSN3NxcLCuFx+Phllt+QyAQYOfOnaRSKdatW8snn3zI559/lm6inaKiooJwONwq7RNCiP1plfD/4osviMViXHHFFaRSKa699lqSySS9e/cGYPTo0SxfvvyQwr9pdZdJtt6ne5u6FDOVsli27DV+//s/EY/HmTz5MgD69DmaLl26cPnlV5BIxPnTnx4nLy+v1donhBD70yrh7/f7mTx5MhdddBEbNmzgyiuvrBd2oVCIzZs3N7sNw9AIh4P1frZjh4Zh7OcctVa7lrVS+3/uQdB1DV1Pt0PXNTSt7msd0zQJBgP8/OfpG2Z37tyZsrLd/PCHF3LXXXP4xS+uIhKJMH78RXg8zb/1mtaw//tjGPoBv8YNsrHf0ufs0Vr9bpV7+CaTSRzHwe/3A+n6f2VlJa+++ioAf/rTn0ilUkyePLnJbViW3WAN6+3bN9KtW5/md+7YmLs/Q+X1wPZ3PrSOtKEW9XUfst559pA+Z48OtZ7/s88+y9y5cwHYsWMHsViMYDDIpk2bUErx5ptvMmJEa13tUluS6aCLugkhxJHQKmWfCy+8kOnTp3PppZeiaRp33nknuq5zww03YNs2o0ePZvjw4a2xa9Ak/IUQYn9aJfy9Xi/33Xdfg58/88yRvLxRwl8IIZrivk/4ahrpbkn4CyFEU9wX/pAu+0vZRwghmuTK8FdoEv5CCNEMV4Z/uvQj4S+EEE1xZ/jLzF8IIZrVYVf19H3xLP7VTzX6mGZFQTdQhu+AthkfcgmJwRc2+5x//ONv/P3vL+I4Drt37+K4477J5s2bOPHEbxGJ1LB69Sp69+7DzTfP4fXXX2XRoj9hmiadOxdx6613Eo1GmTv3NiorKwEoKbmR/v0HHFA7hRDiUHXY8G9Lubm5zJ37W8aOPYkrr7yGzp07c/bZ3+axx57gV7+6iYsvPo/q6mqWLPkXEycWc/rp3+Gll/6PSCTCwoV/5MQTR3LBBReyefMm7rzzVh599A9t3SUhRJbpsOGfGHxhk7N0o/RL8Pix8w5seYSW6t07vd28vHy6desGQCAQoG/ffgCEQjkkkwmuvfZXLFz4BM899wx9+hzNmDH/xVdfreWDD95j6dKXAaiurmqVNgohRHM6bPg3S2vdmr+m6bX/Nn+j9RdffIHJk6+ioKAT99xzB8uWvUafPkdz5plDOfPMsygvL+Nvf/tLq7VTCCGa4s7wp31c7TNkyDBuuqmEYDBEIBDg1FNHc+qpo5k7dw4vvvg80WiEK664qq2bKYTIQq2yqufhcNCregJG+VrQdOxwv9ZqXquTVT1bLhv7LX3OHh1qVc+21z5m/kII0V65M/xbueYvhBAdXYcL/5ZUqRQaWgee+bfTSpwQwkU6VPibppdIpGr/4diBZ/5KKSKRKkzT29ZNEUK4WIe62qegoIjy8l3U1FQ0+zwtVo7mWDjJjUeoZYeXaXopKChq62YIIVysQ4W/YZh07tx9v8/LXXIvvp0fsvtHbx6BVgkhRMfToco+LaV0LzhWWzdDCCHaLVeGP7oJtoS/EEI0xZ3hb5gy8xdCiGa4MvyV7pWZvxBCNMOV4S9lHyGEaJ4rw18ZcsJXCCGa48rwRzfRlAOO3dYtEUKIdsmV4a90T/oLmf0LIUSjXBn+1Ia/5qTauCFCCNE+uTT8az+4LDN/IYRolCvDXxm1ZR+54kcIIRrlyvDfU/aR8BdCiMa4MvzlhK8QQjTPleFfV/OXE75CCNE4V4b/npp/sm0bIoQQ7ZQrw18u9RRCiOa5Mvyl5i+EEM1zZfjL1T5CCNG8Vgv/0tJSxo4dy7p169i4cSOXXnopEydOZPbs2TiO01q7TTNqP+RlS9lHCCEa0yrhb1kWs2bNwu/3A3DXXXdRUlLCk08+iVKKpUuXtsZuM6TsI4QQzWuV8L/77ru55JJL6NKlCwCrVq1i5MiRAIwZM4bly5e3xm73kLKPEEI0yzzcG3z++efp1KkTp512Go899hgASik0TQMgFApRXV293+0YhkY4HDy4RiRy0/sK6AQPdhsdkGHoB/+edWDZ2G/pc/ZorX4f9vB/7rnn0DSNt99+m9WrVzN16lTKysoyj0ciEfLy8va7HdtWVFRED6oNRtSmExCtqiFxkNvoiMLh4EG/Zx1ZNvZb+pw9DrTfRUW5LXreYQ//xYsXZ74uLi7mlltuYd68eaxcuZKTTjqJZcuWcfLJJx/u3dajMqt6yglfIYRozBG51HPq1KnMnz+fCRMmYFkW48aNa90d6l5Aav5CCNGUwz7z39vChQszXy9atKg1d1WPkvX8hRCiWe78kFft2j6arOcvhBCNcmf4Z67zl5q/EEI0xpXhL2UfIYRonivDXz7kJYQQzXNp+BsoTZeyjxBCNMGd4Q+ge9DkZi5CCNEo94a/4ZGZvxBCNMHV4a85MvMXQojGuDf8dY+s5y+EEE1wb/hL2UcIIZrk3vDXpewjhBBNcW/4y8xfCCGa5Orwlw95CSFE49wb/roHZGE3IYRolGvDX+kmmpR9hBCiUa4NfwyvLOwmhBBNcHH4S81fCCGa4t7w102p+QshRBPcG/5S9hFCiCa5N/zlhK8QQjTJveEvM38hhGiSi8PfIzdwF0KIJrg3/HVTZv5CCNEE14a/MrxS8xdCiCa4NvzRPTLzF0KIJrg3/A1Tav5CCNEEF4e/V5Z0FkKIJrg3/HUzfTMXpdq6JUII0e64N/wNb/pfZbdtO4QQoh1yb/jrnvS/UvcXQogG3Bv+hgkgK3sKIUQj3Bv+em3ZR076CiFEA+4NfyNd9tGcZBs3RAgh2h/Xhr/S02UfbJn5CyHEvlwb/pmrfaTmL4QQDZitsVHbtpk5cybr169H0zRuvfVWfD4f06ZNQ9M0Bg4cyOzZs9H1Vhx75ISvEEI0qUXpu2LFigPa6L///W8AnnrqKUpKSrj//vu56667KCkp4cknn0QpxdKlSw+8tQei7lJPOeErhBANtCj858+ff0Ab/c53vsOcOXMA+Prrr8nLy2PVqlWMHDkSgDFjxrB8+fIDbOoBqi37yMxfCCEaalHZR9M0pkyZQt++fTOlmuuvv775DZsmU6dOZcmSJTz00EO89dZbaJoGQCgUorq6utnXG4ZGOBxsSfMapZenwz83qKMOYTsdiWHoh/SedVTZ2G/pc/ZorX63KPzHjx9/UBu/++67ueGGG7j44otJJBKZn0ciEfLy8pp9rW0rKiqiB7VfgALNRAdqKmuwcg5+Ox1JOBw8pPeso8rGfkufs8eB9ruoKLdFz2tR2efcc88lGo3yySefUFVVxfe+971mn/+Xv/yFBQsWABAIBNA0jWOOOYaVK1cCsGzZMkaMGNGiBh40o67mL2UfIYTYV4vCf9asWWzevJlRo0axdetWZs6c2ezzzzzzTD7//HN+9KMfMXnyZGbMmMGsWbOYP38+EyZMwLIsxo0bd1g60KTa6/xlTX8hhGioRWWfjRs3snjxYiB9MveSSy5p9vnBYJAHH3ywwc8XLVp0EE08OMqQ5R2EEKIpLZr5JxIJYrEYAPF4HNvuAMsk133CV8o+QgjRQItm/pMmTeK8885j4MCBrF27luuuu66123XoMmv7SPgLIcS+WhT+RUVFPPPMM2zevJmePXtSUFDQ2u06dFL2EUKIJrX4Q17hcJhjjz22YwQ/7HXCV1b1FEKIfbXah7zanCzvIIQQTWpR+J9//vkYhtHabTm8ZHkHIYRoUovC/x//+AePP/54a7fl8DLq1vOX8BdCiH21KPzz8vJ45ZVX6pV9+vbt26oNO2SZmb+UfYQQYl8tCv/S0lL+93//F6UUAMlkkqeffrpVG3bIMjV/OeErhBD7avZqn5KSEgAWLlzI2LFjWbhwIQsXLsTr9R6Rxh2MlKP4w4qNRC0bpRlywlcIIRrRbPiXlpZmvn799dczX9ctzdwerdsd4XdvbeTtdWVgeOSErxBCNKLF91GsK/m0dz4j3aWolULpHjnhK4QQjWg2/Pee4bfn2f7e/J50l+KWA7opJ3yFEKIRzZ7wXbt2Lb/+9a9RStX7et26dUeqfQfMb6Y/jxCzbJTulYXdhBCiEc2G/wMPPJD5eu9lnPe3pHNbqpv5x5J27cxfwl8IIfbVbPjX3XC9I/GadWWfdPhLzV8IIRpq8QnfjkLXNHymni77GF6Z+QshRCNcF/4AflPPnPCV6/yFEKIhd4a/x8jM/OWErxBCNOTO8Df1TM1fbuAuhBANuTP862b+uikzfyGEaIQ7wz8z8/fKh7yEEKIR7gx/jy4zfyGEaIY7w980iCcd0D1S8xdCiEa4M/xrZ/4YHrnUUwghGuHO8DcN4pmyj9zMRQgh9uXO8M/M/OWErxBCNMaV4e8z97rUU2r+QgjRgCvD3+/RsWyFo8mqnkII0Rh3hn/typ62Jmv7CCFEY9wZ/p70DV0sZaLJCV8hhGjAleEfqL2hSwpDZv5CCNEIV4Z/3a0cLWrv4dtBbj4vhBBHijvDv3bmnyQ9CMgSD0IIUZ87w79u5u/Uhb+UfoQQYm/N3sP3YFiWxYwZM9i6dSvJZJKf//znDBgwgGnTpqFpGgMHDmT27NnoeuuNO/vO/DXHQgo/Qgixx2EP/xdffJFwOMy8efOoqKjg/PPPZ/DgwZSUlHDSSScxa9Ysli5dyne/+93DveuMupl/UtXO/OWDXkIIUc9hn36fddZZ/PKXvwRAKYVhGKxatYqRI0cCMGbMGJYvX364d1tP3cw/ofbM/IUQQuxx2Gf+oVAIgJqaGq677jpKSkq4++670TQt83h1dfV+t2MYGuFw8KDakDTS4a9MLwB5OSYc5LY6EsPQD/o968iysd/S5+zRWv0+7OEPsG3bNqZMmcLEiRM599xzmTdvXuaxSCRCXl7efrdh24qKiuhB7T+ZSJ/grU6kK/3VFdXYHNy2OpJwOHjQ71lHlo39lj5njwPtd1FRboued9jLPrt37+aKK67gxhtv5MILLwRg6NChrFy5EoBly5YxYsSIw73beuqWd4g7tWOb1PyFEKKewx7+v/vd76iqquKRRx6huLiY4uJiSkpKmD9/PhMmTMCyLMaNG3e4d1uPaeiYukbcSZeapOYvhBD1Hfayz8yZM5k5c2aDny9atOhw76pZfo9B3JYPeQkhRGNc+SEvgIDHIGrXdk8+5CWEEPW4Nvz9Hp24k+6eZsvKnkIIsTfXhr/M/IUQommuDX+/1yBaN/OX8BdCiHpcG/4Bj0EsVTfzl7KPEELszbXh79+r7KPZMvMXQoi9uTb8Ax6dSN3MPxVr28YIIUQ74+LwN9hkd8IJFOL76qW2bo4QQrQrrg1/v8egOmUQG1aMd8Mr6BXr27pJQgjRbrg2/AMeg3jKJnbM5aCbBD79Y1s3SQgh2g3Xhr/fYxC3HJxgEYmBP8C/+mm0RFVbN0sIIdoF14Z/wKOjgKStiB03Gd2K4F/9dFs3Swgh2gXXhr/fm17ULW7ZpLoch9V9JIFPHgfHbuOWCSFE23Nt+Ac8teGfcgCIDp+MUb0Z/+eL27JZQgjRLrg2/P2ePTN/gGTfcSR7nEru6zMIvvcQKNWWzRNCiDbl2vDfd+aPblJ57kLigy4gtPIecl67Se7wJYTIWq1yD9/2IOCpvZWjtVeN3/BR/Z2HsPN6E3rvQbxblhM9/mfEB18IZqCNWiqEEEeea2f+/n1n/nU0jehJN1J5zh9x/GFyX59O4f+eQujN2/BseQtk7X8hRBZw8cy/rubvNPp4su93SR79HTxblxP4+PcEPn2C4MeP4XhCpIqOwcnthZ3bk1TnIVg9RqH84T0vti1AgeE9Aj0RQojDz7XhXzfzT6SaubRT07B6jsLqOQqSEbxbl+Pd+CpG2X/wbF2Or2YbGgql6aSKjsUJdsWoWIdRtRFlBogPu4zY8Mk4oW5N7sIoW0PwnfvwbXyV+DfGEx1xHU5O98PdXSGEOCCuDf+At/mZfwPeUPpooO939/zMTmDu/ATv5jfwbnkTo3IDdqdBJPudjV61kcBHCwh8/D9YPU9FaUb6pjGahuMvxAl0Ro/uwLf2bygzQLL3GPyrn8L/xTPEhl1GqstxOMEuOP5OaHYcLVmDnqhCj+5Ar9mGHtmOk9Mdq8s3SXUZnj7yUA44Nkblejw7PsLc9SmOL49k/+9hdf/WnnY7KbRUHOUJgubayp4Q4hC4N/wzNf9D+FCX4SPV/Vukun+L6MjrGzwcqdxI8OPf4/n6HZRugm6CsvGUf4Ue2wVoxL55FdHjr0EFOqFXbSL07v0EPv0jmmp6UFKGDyfUFX3dSwSbuRGNEyhCS1YR/ORx7FBXtML+dCrfhF6zDU2l+63MAE6gELtgAKlO38AJFqFHtmNUb0WPlaafgwaGFzu3B3b+0TjBLhjVW9JHOdVbcHxhnJyjcHK6Y+f2xM7rhZPTA2X6AQXKwajahFn2JUbZGpS/AKvbiVhdj0dLxfF8vQLP1yvQ4xUoXx6OLx/l74Sd0w0n1A0n2BXHXwCeIEC6fZXr0ZI16UFy7yMrK4ZRuR49uhM9uhstWYV29IkQGgyGDy1ZjXfDUjxb3iBVdCyJQRegfPmNv4G2lR4cdaPZPwMh3EhTqn1e8G5ZNhUV0YN+vS/o47g5S7j2tL5cPrLXYWxZCykFqEZn3lqypja8dqHFSsH043hzUd5cnFBXlC8MmpY+8ij9AnPnp2hWpDakNOzcHqS6DMcJdUOzIng3vIJv3d/xWOUkA0dh5/VCeXPRrAiaFUWPbMcsW4NRsQ7NTqDMIHZuD5xgZ9Ln/B20VByjajN6bHe6+Wg4tUGvJSoxarahx8v2323Dh2Yn0l9rxl6DkD8zWGmJKjQa/tkpwweahpaK1/u5ndsTO9wfo3IDetWmJl+bKhyMuXs1mpPE8YTQrQjK8JHo/z3scN/076RuoCpdjVG+FpSNEyyqHYS6pL8OFqH8BelBypuL8oRAN1C6By2VSA9ONdsgFcXO74td0H/PgFm5ET26A9BrJwQGWiqW/v2lEtgF/UkVHUuqcAiY/szfipaoRI+Xpf8eNAPly8fx5oInkH5fdE/6b6JWOD9ARWl5+necioNjoTk22Ik9+9MMUoWDUYHC/f7e2hUnlZ5I7SMcDh5SJnRUB9rvoqLcFj3PtTN/n1l7qeehzPwPhaYBWqMPKW8OtjcHO9yv+W0YPlJdhpPqMrzJpyhvDolB55MYdD7hcJDq5v5IHBvNqkF58+oFSb1m1w5Mdk73hpe/pmLpI4aqzRjVW8Gp/ZyEpuHk9CDVaRBOXi+0RBWe7e9j7vgADD/JHqeQ6nLcnhPkykGLl6MHl9iYAAAU9klEQVTXbMeIbEeP7kx/Hy8Hx8bO74Md7osyg3h2foS57T2Myg1YRcdif2M8dsFA7FBXnGAReALk16wmueZ1zF2fEjv2xyT6n02q24mYu1fh//zP+P7zF/TknkX97JzupAqHkOzzbZRupsO89mjIs+MjtHhps0dmmfd+r8Ftv8/VvZmBIP1aHTQj/V60YBsKLR2Imp7+z7EoauG9qe1QV+xwf5QZANOHQkdPVKDHytCS1SjTh/KEUIY/XYK0Ymh2HKV70q/xBHC8eSh/GMcXRnlzUJ4gygymByVI/6nbFloqnt5Gojo9mMXL0n/HhUNIdR6aPvKMV6DFy9GcVO3RX3c0O4F3/b/wffVPjPK1pLoMJ9nrNFJFx2JUbcIo/RIjuoU8I6d2oO6cKW9qdhzlycHxF6SPIE0/SjNA9+AECnFye+AEu0Aqjln5FUbFVzi+MNZRJ9UbgPWqTRg1W8l0yLHS71G8rLatR+Hk9cIJdUOZAZTpA91b//8lx05PcJIRMMzagdtEj+6qLefuwA73I1V0TGaA0xJVmDs/AjSc3B7YuT3A8LXod3soXDvzD4eDHHPry1z8zaO4bux+QtYlZGbUBOWkZ/11A3ITA19G3f/AiSr0ZBWaFU3PRpUNuokd6o6T0w10TzqYytehx3bVlsR64+QcBSiwU2jKTgeo4UkHTPUWzF2fYO7+HM1JpUNK01C+ME6gU/ockHJqA6S6NoiTYMfTM3tlg3LwBQPEHD/KmwOGH2WYoJkow4MygyhPCM2OY+5ejVn6OUblxvRRgZ0EZaf35++E8uVCKpE+grDjYPjSwW7400cTqRiaFU2/F4kKtEQlWrKm0aOvem95bbnR8XdCs2owKtbv/zWajnXUSaSKjsOz/T3MHR9lBkYnUASF/XCiFeiRHeiJisx+lOFNt7+ZwVDpZoPHlRkg2XMUygzg+fodjOiO5v8umtl2Osh1tFTL/v9zPDmkup2IHt2JUfpFg/em+rQ5xI/7CSAz/4PiN/WG1/mL7KPpTR2ENU43UP6CdOlnP0+1w/2aPoIzfPX/l9Y0nLxeJPN6kez/vQNoUEOecJBYCwLB6jXmkPbTKKUgFUdLRWsDVYFSKMObHjRMf8PzKFYUs/QL9Hj5nhm6bqBHdqRLaE6KZO+x9UpUWqIKo3wNdv7RqEBh/RC0rdojIS3TJs2qSR9R2Mn0UYFjoUd2otd8jVG9BeUJkSrojx3uh1H9Nd5N/8a78d/gJLF6nEz0qJOww/33tFk3MwMYmp7eTtVm9OgOtFRiT4nNsdNHwUqlj4p8eShPDqhU+nmOhRPojJPTHSdYhFn6BZ6tb+PZ/i5OsAuJ/udgdRuRfj+qt2LUfJ0+Kmll7g5/j1H/E75CiEOnaelzEZ7Afubye/EESXU7ocGPnbzeNDVfV748Ut1ObPxBw9OgTar2vFk9RY2/3C4cQvLoM5pv876v8YexOw89oNc0up1Og0gM/MEhb+dQufo6QJn5CyFE49wd/jLzF0KIRrk7/GXmL4QQjXJ3+Hv0ln/CVwghsoi7w9802u46fyGEaMfcHf4eXWr+QgjRCHeHv2lIzV8IIRrh7vCXmr8QQjSq1cL/448/pri4GICNGzdy6aWXMnHiRGbPno3jHJlA9knNXwghGtUq4f/73/+emTNnkkikV3e86667KCkp4cknn0QpxdKlS1tjtw34PTqWrUg57XL5IiGEaDOtEv69e/dm/vz5me9XrVrFyJEjARgzZgzLly9vjd024K9d2bPZu3kJIUQWapW1fcaNG8eWLVsy3yul0GoXYAqFQlRXV+93G4ahEQ4HD7oNhqHTKS+9JLEv6COc0/pLpLY1w9AP6T3rqLKx39Ln7NFa/T4iC7vp+p4DjEgkQl5e3n5fY9vqkJd0dqz0klE7Smsws2D2L0s6Zw/pc/ZorSWdj8jVPkOHDmXlypUALFu2jBEjRhyJ3WZu4i5X/AghRH1HJPynTp3K/PnzmTBhApZlMW7cuCOx20zNX671F0KI+lqt7NOzZ0+eeeYZAPr27cuiRYtaa1dN8ntqw18+5SuEEPW4+0NeZm3ZR2b+QghRj7vDv3bmn5CZvxBC1OPq8M/xpataX5Vm3xUCQgjRHFeHf7dcH6f168QT72xmS0WsrZsjhBDthqvDX9M0pn5nIKauMfeVNSglyzwIIQS4PPwBuub6mHJaX1ZurOAfn+9s6+YIIUS74PrwBxg/vDvHHZXH/a+tY2d1oq2bI4QQbS4rwl/XNGaeOYh4yuHiJ97jjys3ybX/QoislhXhD9C3MMiiy05gRK8wj7y5gfGPv8ujb23g3U3lMhAIIbLOEVnYrb04ujDIvecP48MtlSxYvoEnVm7i8RXgMTSGdM3lmO65HNM9j76FQbrkeMn1mZnVSIUQwk2yKvzrHN8zn99dPJyaRIqPtlby3qZKPt1WxXMfb+PJ97dmnuczdXqFA3yjaw6Du+RQGPJSk0hRk0jhM3W+2SOfAUUh9P0MELtrEqwrjTK4Sw75AU9rd08IIfYrK8O/To7PZHS/Qkb3KwTAsh3W7IqwpSLGrpokO2sSbCiL8vb6Mv6+akej28j1mQwsCmHoGroGpq6T5zfJD3hwHMV7mysyHzLTgCHdchnRK0zvAj/dcv10zvGStB1qEimqEzbl0SSlkSRlUYvCoJejC4P07RQkP2Bi6hqmrlMRs9haGWNrZRxD0+hVEKBXOIDh97K9Kk5N0kYpRa7PJM/vIeDR6x3BKKXYWZPeT0HQQ2HQi9fMmgqgEIIsD/99eQydod1yGdqt4XrYu2sSVMZT5PpMcnwmVXGLD7ZU8sHmSjaWR0mlFEqBZadYXxqhMp4i5Si+2SOP7w3tyoCiEKu2VbNyYzmL39uM3cxHDjQgz29SFU9xOD6ZYOoa4YCHgqAHXdPYVB4lts8y13l+k/zaQSvgMaiOpyiLJqmMp/CbOiGfSchrkOMzyan919DTA4oCokmb6rhFdcIm6DU4Ks9Htzw/0aTN2t0R1u2OELPs2tebhAMmXXJ9dM31URD04tE1PIaGraCsdvCLp2yKcnx0y/URDnioiluUxywqYymilk3csrFsxVGdghR4DQqCHr6ujLN2d4RN5TF6FwQY2aeAb/UK0znHi65paBrsrkmyuSLGlooYiZSDaeh4dI0cn0nnkJfOIS/hoIccr4Fp7BkUHaVIpBwiiVTtAAtFOd7MJ8khPbAmbYUG6LqGBkSSKWoSNvGUTaeAl/yAlBNF29NUO/3kk2XZh3wzl7a+8cPedzDbm2U77KpJsr06zu6aJD5TT4eiz6RT0ENBwINp6MQtm03lMTaURalJpAeTlKPI85scle/nqDw/tlJsKY+zqSIGpoHpOOT4THQNquLpElVFLEVFLEl51MJWit4FQfoUBCjK8VIRs9gdSVIasaiKp4M1krTJD5gUBDzk+T0kbScTYHVlr5qkjbPXvZEDXoM8n0mu3ySSSPF1VYJdNQm8hk6/ziH6FwZrH7OJJFOURS12VCfYWZPA2mck1ID8gAefqbM7ksTe5x7MIa9BwGMQ8OgYukZZ1KIqnr5xj65Bz3CA3gUB1u2OsK3q0C7t9Zk6HkMjkXIatHPv9oQDHiJJm+pEqkF7G9tmlxwvAY+B19Qx9fT2I0mbSNJG19LP8Zk6uqbhKIXjpAefui2bho5OemAHSNoOlu2QctKTEAV4DY2utQNsp6AXSP/cdhTxlE3cckjaDrqWPmo1dA2l0vupa2fQaxL06HiM9H91+3NUej9G7aDtMXR0jcy+LVuRSNnEUw4eXaNTyEthyIvf1IkmbWKWja0U+X4P+QEPtqNYtb2az76uYltVnL6FQb7RJYfeBQGqEzZlkSQxBfG4ha6lr+AzdA1TT//rM3X8poHP1NG0dB+VgpDPoCDopVPQQ8pWlMcsyqMWQa9O38IQhUEPmqYRSabYUhGnKm6lt61p2ErVHpGnsOz0kXSuPz0JquunUgqPoePd672p+x3VvaeaBrGkQ3UiRTRpEw6YdM/3UxDwkHIUO6oTbK9KYDsq3Q+PzoCinMz2WutmLhL+LtLe+pyyHXRda/aciKMU0aSdHthsB03TyA94Mn/4tqMoiyapiFnk+dMD474lqnA4yNadVZRGLLrkeDM38VFKsbUyzvubK6hJ2OkQVVAQ9NA7HKBnQYCQ18CyHZK2ojqeYnckwe5IkspYepCLJG0s28FnGvhr/8cMeQ2CXhMN2FmTYEd1goqYRY7PJNdnEvSm959y0gkR9Brk+Az8pkFZzGJ7VZyd1UkSKRurtt8+00gPal4DFMRTNona1Wi12nDWSAeJBhimQSxhpfcBtQGUDkO09BFHIuWwozrdvvJoEq3255mw9Bj4DB1HKWylcByV2RekV8ONJm2iSfuwHIG2RNdcHz3y/awvjVIes+o9pmmA4rC2Jc9vYmhag30dCT5TJ5lyGu3Pj0f2YsppfYHWC38p+4hWs3fJpCm6ptUrm+zL0DWKcnwU7ecezCGvSchbfzuaptEzHKBnONDsawO1g0XnkJe+hR3jHrFHcqBXSmGr9GBeN9jUldBsR2UGz7p5pKaljwbqZuKW7VAaTR9dxi17z+BZe3RaEbNQSjGkay5dcn2Zfe6sSbKlIkZ+wEPnoJfe3fKoqopl2mM7CttRpByHZMohXvtfun2go1FTe5RZHk1i6jrh2iPrmkSK9aVRviqNYitFz3w/vQoChAOezJGWrpMp85q6Rk3CpiphEUs6oJE50knVvgdWbbmvbq7j1LbRUap2AmAS9BiUxyy+royzrSpOyGvQPc9PtzwfHl0nUduHE3rmt/rvVcJfCNEsTdMwNTB146Beb+gGPfID9MhvfhDed591Jas6eu0hyZ721B1RHly7RvYpOKjXuYVc4iGEEFlIwl8IIbKQhL8QQmQhCX8hhMhCEv5CCJGFJPyFECILSfgLIUQWkvAXQogs1G6XdxBCCNF6ZOYvhBBZSMJfCCGykIS/EEJkIQl/IYTIQhL+QgiRhST8hRAiC0n4CyFEFnLVzVwcx+GWW27hyy+/xOv1cvvtt9OnT5+2blarsCyLGTNmsHXrVpLJJD//+c8ZMGAA06ZNQ9M0Bg4cyOzZs9F1943vpaWl/PCHP+Txxx/HNM2s6POCBQt49dVXsSyLSy+9lJEjR7q635ZlMW3aNLZu3Yqu68yZM8f1v+uPP/6Ye++9l4ULF7Jx48ZG+/rwww/z2muvYZomM2bM4Ljjjjv4HSoX+de//qWmTp2qlFLqww8/VD/72c/auEWt59lnn1W33367Ukqp8vJyNXbsWHX11VerFStWKKWUuvnmm9XLL7/clk1sFclkUl1zzTXqzDPPVGvXrs2KPq9YsUJdffXVyrZtVVNTox566CHX93vJkiXquuuuU0op9eabb6pf/OIXru7zY489pr7//e+riy66SCmlGu3rZ599poqLi5XjOGrr1q3qhz/84SHt0z3DJvD+++9z2mmnAfDNb36Tzz77rI1b1HrOOussfvnLXwLp+50ahsGqVasYOXIkAGPGjGH58uVt2cRWcffdd3PJJZfQpUsXgKzo85tvvsmgQYOYMmUKP/vZz/iv//ov1/e7b9++2LaN4zjU1NRgmqar+9y7d2/mz5+f+b6xvr7//vuMHj0aTdM46qijsG2bsrKyg96nq8K/pqaGnJyczPeGYZBKpdqwRa0nFAqRk5NDTU0N1113HSUlJSil0GrvHh0Khaiurm7jVh5ezz//PJ06dcoM8IDr+wxQXl7OZ599xoMPPsitt97KDTfc4Pp+B4NBtm7dytlnn83NN99McXGxq/s8btw4THNPFb6xvu6bb4f6Hriq5p+Tk0MkEsl87zhOvTfUbbZt28aUKVOYOHEi5557LvPmzcs8FolEyMvLa8PWHX7PPfccmqbx9ttvs3r1aqZOnVpv5uPGPgOEw2H69euH1+ulX79++Hw+tm/fnnncjf1+4oknGD16NL/+9a/Ztm0bkyZNwrKszONu7PPe9j6XUdfXffMtEomQm5t78Ps4pBa2MyeccALLli0D4KOPPmLQoEFt3KLWs3v3bq644gpuvPFGLrzwQgCGDh3KypUrAVi2bBkjRoxoyyYedosXL2bRokUsXLiQIUOGcPfddzNmzBhX9xngxBNP5I033kApxY4dO4jFYpxyyimu7ndeXl4m2PLz80mlUq7/+95bY3094YQTePPNN3Ech6+//hrHcejUqdNB78NVq3rWXe3zn//8B6UUd955J/3792/rZrWK22+/nZdeeol+/fplfvab3/yG22+/Hcuy6NevH7fffjuGYbRhK1tPcXExt9xyC7quc/PNN7u+z/fccw8rV65EKcWvfvUrevbs6ep+RyIRZsyYwa5du7Asi8svv5xjjjnG1X3esmUL119/Pc888wzr169vtK/z589n2bJlOI7D9OnTD2kAdFX4CyGEaBlXlX2EEEK0jIS/EEJkIQl/IYTIQhL+QgiRhST8hRAiC7n3E1BCtMDKlSspKSlhwIABmZ8VFBTw0EMPHdJ2p02bxjnnnMOYMWMOtYlCtAoJf5H1Tj75ZO6///62boYQR5SEvxCNKC4upm/fvqxfvx6lFPfffz9FRUXMnTuX999/H4Dvf//7TJo0iQ0bNjBz5kwsy8Lv92cGkqeffpr/+Z//oaamhltuueXQlt8V4jCT8BdZb8WKFRQXF2e+Hzt2LJBeLuS2225j8eLFLFiwgFGjRrFlyxaeeeYZUqkUEydO5OSTT+aBBx7gqquuYsyYMSxdupTPP/8cgGHDhnHNNdfw/PPP8/zzz0v4i3ZFwl9kvcbKPq+//jonn3wykB4EXn31Vbp168aIESPQNA2Px8Pw4cNZt24d69ev5/jjjwfgjDPOAOD//u//GDZsGACdO3cmHo8fwR4JsX9ytY8QTai7H8QHH3zAgAED6N+/f6bkY1kWH374IX369KF///58+umnALz44ossXLgQILMkrxDtkcz8Rdbbt+wDEI/HeeGFF3jiiScIBALcc889FBQU8M477zBhwgQsy+Kss85i2LBh3HTTTcyaNYtHH30Uv9/PvHnzWLVqVRv1RoiWkYXdhGhE3aqhbl0VVggp+wghRBaSmb8QQmQhmfkLIUQWkvAXQogsJOEvhBBZSMJfCCGykIS/EEJkof8P1m7mVnyFEzAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(history_Miami.history['mae'])\n",
    "plt.plot(history_Miami.history['root_mean_squared_error'])\n",
    "plt.title('mae and rmse per epoch')\n",
    "plt.ylabel('Error')\n",
    "plt.xlabel('Epoch')\n",
    "plt.legend(['mae', 'rmse'], loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test on Miami 2018 data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>Hour</th>\n",
       "      <th>Minute</th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Clearsky DHI</th>\n",
       "      <th>Clearsky DNI</th>\n",
       "      <th>Clearsky GHI</th>\n",
       "      <th>Cloud Type</th>\n",
       "      <th>...</th>\n",
       "      <th>Solar Zenith Angle</th>\n",
       "      <th>Surface Albedo</th>\n",
       "      <th>Pressure</th>\n",
       "      <th>Precipitable Water</th>\n",
       "      <th>Wind Direction</th>\n",
       "      <th>Wind Speed</th>\n",
       "      <th>Global Horizontal UV Irradiance (280-400nm)</th>\n",
       "      <th>Global Horizontal UV Irradiance (285-385nm)</th>\n",
       "      <th>timestamps</th>\n",
       "      <th>Cloud Cover</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>30</td>\n",
       "      <td>21.1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>113.89</td>\n",
       "      <td>0.08</td>\n",
       "      <td>1019</td>\n",
       "      <td>2.7</td>\n",
       "      <td>58</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1514782800</td>\n",
       "      <td>0.30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>30</td>\n",
       "      <td>21.2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>127.06</td>\n",
       "      <td>0.08</td>\n",
       "      <td>1019</td>\n",
       "      <td>2.8</td>\n",
       "      <td>63</td>\n",
       "      <td>1.8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1514786400</td>\n",
       "      <td>0.32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>30</td>\n",
       "      <td>21.2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>140.43</td>\n",
       "      <td>0.08</td>\n",
       "      <td>1019</td>\n",
       "      <td>2.8</td>\n",
       "      <td>65</td>\n",
       "      <td>1.6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1514790000</td>\n",
       "      <td>0.23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>30</td>\n",
       "      <td>21.2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>153.91</td>\n",
       "      <td>0.08</td>\n",
       "      <td>1018</td>\n",
       "      <td>2.9</td>\n",
       "      <td>57</td>\n",
       "      <td>1.3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1514793600</td>\n",
       "      <td>0.14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2018</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>30</td>\n",
       "      <td>21.1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>167.37</td>\n",
       "      <td>0.08</td>\n",
       "      <td>1018</td>\n",
       "      <td>2.9</td>\n",
       "      <td>41</td>\n",
       "      <td>1.1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1514797200</td>\n",
       "      <td>0.14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Year  Month  Day  Hour  Minute  Temperature  Clearsky DHI  Clearsky DNI  \\\n",
       "0  2018      1    1     0      30         21.1             0             0   \n",
       "1  2018      1    1     1      30         21.2             0             0   \n",
       "2  2018      1    1     2      30         21.2             0             0   \n",
       "3  2018      1    1     3      30         21.2             0             0   \n",
       "4  2018      1    1     4      30         21.1             0             0   \n",
       "\n",
       "   Clearsky GHI  Cloud Type  ...  Solar Zenith Angle  Surface Albedo  \\\n",
       "0             0           0  ...              113.89            0.08   \n",
       "1             0           0  ...              127.06            0.08   \n",
       "2             0           0  ...              140.43            0.08   \n",
       "3             0           0  ...              153.91            0.08   \n",
       "4             0           0  ...              167.37            0.08   \n",
       "\n",
       "   Pressure  Precipitable Water  Wind Direction  Wind Speed  \\\n",
       "0      1019                 2.7              58         2.0   \n",
       "1      1019                 2.8              63         1.8   \n",
       "2      1019                 2.8              65         1.6   \n",
       "3      1018                 2.9              57         1.3   \n",
       "4      1018                 2.9              41         1.1   \n",
       "\n",
       "   Global Horizontal UV Irradiance (280-400nm)  \\\n",
       "0                                          NaN   \n",
       "1                                          NaN   \n",
       "2                                          NaN   \n",
       "3                                          NaN   \n",
       "4                                          NaN   \n",
       "\n",
       "   Global Horizontal UV Irradiance (285-385nm)  timestamps  Cloud Cover  \n",
       "0                                          NaN  1514782800         0.30  \n",
       "1                                          NaN  1514786400         0.32  \n",
       "2                                          NaN  1514790000         0.23  \n",
       "3                                          NaN  1514793600         0.14  \n",
       "4                                          NaN  1514797200         0.14  \n",
       "\n",
       "[5 rows x 26 columns]"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df = pd.read_csv(os.path.join(_Testing_Data_Dir, 'Miami_FL', 'psm_FL_Miami2018.csv'))\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_size = 6\n",
    "test_input_arry = np.array([], dtype=np.float64).reshape(0,input_size)\n",
    "test_output_arry = np.array([], dtype=np.float64)\n",
    "test_input_arry, test_output_arry = combine_training_arrays(test_df, test_input_arry, test_output_arry)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "274/274 [==============================] - 0s 957us/step - loss: 639.4745 - mae: 10.1002 - root_mean_squared_error: 25.2878\n"
     ]
    }
   ],
   "source": [
    "score = model_Miami.evaluate(test_input_arry, test_output_arry, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Mae of 10.1002....not bad!~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Save models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: Neural_Nets/Neural_Net_Boulder_CO/assets\n"
     ]
    }
   ],
   "source": [
    "model_Boulder.save('Neural_Nets/Neural_Net_Boulder_CO')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: Neural_Nets/Neural_Net_Miami_FL/assets\n"
     ]
    }
   ],
   "source": [
    "model_Miami.save('Neural_Nets/Neural_Net_Miami_FL')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}