notebooks/assignments/Week3Notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "bf0a859d-14d0-452f-b0c6-c6926ffe69a0",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "# Scope of Notebook\n",
    "\n",
    "This notebook allows you to plug in your featuried dataset from the previous week into an ml model, in this case we use random forest.  You will then be able to store the trained model in mlflow and calculate performance characteristics around the model like AUC and accuracy.  The advanced section of this notebook outlines how to retrieve the best set of hyperparameters to certify a model for production use."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "22510862-b275-4ff0-8c04-f5a965a3eed7",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "![ml-model-train](../media/CMLE-Notebooks-Week3-Workflow.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "ad5416e3-ff10-4c54-a9b4-6ab4a0b5b788",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "# Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "ac6b4958-397f-4aad-9d81-afb1d45ea0b2",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "This notebook requires some configuration data to properly authenticate to your Adobe Experience Platform instance. You should be able to find all the values required above by following the Setup section of the **README**.\n",
    "\n",
    "The next cell will be looking for your configuration file under your **ADOBE_HOME** path to fetch the values used throughout this notebook. See more details in the Setup section of the **README** to understand how to create your configuration file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "c03b1588-2e38-44da-a7a6-d8aff3bf72eb",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from configparser import ConfigParser\n",
    "  \n",
    "config = ConfigParser()\n",
    "config_path = os.path.join(os.environ[\"ADOBE_HOME\"], \"conf\", \"config.ini\")\n",
    "if not os.path.exists(config_path):\n",
    "  raise Exception(f\"Looking for configuration under {config_path} but config not found, please verify path\")\n",
    "config.read(config_path)\n",
    "  \n",
    "ims_org_id = config.get(\"Platform\", \"ims_org_id\")\n",
    "sandbox_name = config.get(\"Platform\", \"sandbox_name\")\n",
    "environment = config.get(\"Platform\", \"environment\")\n",
    "client_id = config.get(\"Authentication\", \"client_id\")\n",
    "client_secret = config.get(\"Authentication\", \"client_secret\")\n",
    "scopes = config.get(\"Authentication\", \"scopes\")\n",
    "dataset_id = config.get(\"Platform\", \"dataset_id\")\n",
    "featurized_dataset_id = config.get(\"Platform\", \"featurized_dataset_id\")\n",
    "export_path = config.get(\"Cloud\", \"export_path\")\n",
    "import_path = config.get(\"Cloud\", \"import_path\")\n",
    "data_format = config.get(\"Cloud\", \"data_format\")\n",
    "compression_type = config.get(\"Cloud\", \"compression_type\")\n",
    "model_name = config.get(\"Cloud\", \"model_name\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "109006a1-8d31-4515-b9cc-143fe8ddd4d5",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Username: cmenguy@adobe.com\n",
      "Unique ID: cmenguy_adobe_com\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "username = dbutils.notebook.entry_point.getDbutils().notebook().getContext().userName().get()\n",
    "unique_id = s = re.sub(\"[^0-9a-zA-Z]+\", \"_\", username)\n",
    "\n",
    "print(f\"Username: {username}\")\n",
    "print(f\"Unique ID: {unique_id}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "8cf3303b-e086-4638-9aa2-7d62bdca24bd",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Before we run anything, make sure to install the following required libraries for this notebook. They are all publicly available libraries and the latest version should work fine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "44dd3429-850b-4679-82e3-dfea9688993b",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting aepp\r\n",
      "  Downloading aepp-0.2.9-py3-none-any.whl (120 kB)\r\n",
      "\u001b[?25l\r",
      "\u001b[K     |██▊                             | 10 kB 30.6 MB/s eta 0:00:01\r",
      "\u001b[K     |█████▍                          | 20 kB 14.9 MB/s eta 0:00:01\r",
      "\u001b[K     |████████▏                       | 30 kB 20.8 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████▉                     | 40 kB 10.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████▋                  | 51 kB 11.3 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████▎               | 61 kB 13.1 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████             | 71 kB 10.0 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████▊          | 81 kB 9.4 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████▍       | 92 kB 10.4 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████▏    | 102 kB 10.9 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████████▉  | 112 kB 10.9 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████████| 120 kB 10.9 MB/s \r\n",
      "\u001b[?25hCollecting pathlib2\r\n",
      "  Downloading pathlib2-2.3.7.post1-py2.py3-none-any.whl (18 kB)\r\n",
      "Collecting pathlib\r\n",
      "  Downloading pathlib-1.0.1-py3-none-any.whl (14 kB)\r\n",
      "Requirement already satisfied: PyJWT in /databricks/python3/lib/python3.9/site-packages (from aepp) (2.6.0)\r\n",
      "Requirement already satisfied: pandas in /databricks/python3/lib/python3.9/site-packages (from aepp) (1.4.2)\r\n",
      "Requirement already satisfied: requests in /databricks/python3/lib/python3.9/site-packages (from aepp) (2.27.1)\r\n",
      "Requirement already satisfied: python-dateutil>=2.8.1 in /databricks/python3/lib/python3.9/site-packages (from pandas->aepp) (2.8.2)\r\n",
      "Requirement already satisfied: pytz>=2020.1 in /databricks/python3/lib/python3.9/site-packages (from pandas->aepp) (2021.3)\r\n",
      "Requirement already satisfied: numpy>=1.18.5 in /databricks/python3/lib/python3.9/site-packages (from pandas->aepp) (1.21.5)\r\n",
      "Requirement already satisfied: six>=1.5 in /databricks/python3/lib/python3.9/site-packages (from python-dateutil>=2.8.1->pandas->aepp) (1.16.0)\r\n",
      "Requirement already satisfied: cryptography>=3.4.0 in /databricks/python3/lib/python3.9/site-packages (from PyJWT->aepp) (3.4.8)\r\n",
      "Requirement already satisfied: cffi>=1.12 in /databricks/python3/lib/python3.9/site-packages (from cryptography>=3.4.0->PyJWT->aepp) (1.15.0)\r\n",
      "Requirement already satisfied: pycparser in /databricks/python3/lib/python3.9/site-packages (from cffi>=1.12->cryptography>=3.4.0->PyJWT->aepp) (2.21)\r\n",
      "Requirement already satisfied: idna<4,>=2.5 in /databricks/python3/lib/python3.9/site-packages (from requests->aepp) (3.3)\r\n",
      "Requirement already satisfied: charset-normalizer~=2.0.0 in /databricks/python3/lib/python3.9/site-packages (from requests->aepp) (2.0.4)\r\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /databricks/python3/lib/python3.9/site-packages (from requests->aepp) (1.26.9)\r\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /databricks/python3/lib/python3.9/site-packages (from requests->aepp) (2021.10.8)\r\n",
      "Installing collected packages: pathlib2, pathlib, aepp\r\n",
      "Successfully installed aepp-0.2.9 pathlib-1.0.1 pathlib2-2.3.7.post1\r\n",
      "\u001b[33mWARNING: You are using pip version 21.2.4; however, version 23.0.1 is available.\r\n",
      "You should consider upgrading via the '/local_disk0/.ephemeral_nfs/envs/pythonEnv-ba66f562-f854-4b23-9191-cf0cc1b7ed77/bin/python -m pip install --upgrade pip' command.\u001b[0m\r\n",
      "Collecting adlfs\r\n",
      "  Downloading adlfs-2023.1.0-py3-none-any.whl (25 kB)\r\n",
      "Requirement already satisfied: fsspec>=2021.10.1 in /databricks/python3/lib/python3.9/site-packages (from adlfs) (2022.2.0)\r\n",
      "Collecting azure-datalake-store<0.1,>=0.0.46\r\n",
      "  Downloading azure_datalake_store-0.0.52-py2.py3-none-any.whl (61 kB)\r\n",
      "\u001b[?25l\r",
      "\u001b[K     |█████▎                          | 10 kB 35.0 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████▋                     | 20 kB 44.1 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████                | 30 kB 55.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████▎          | 40 kB 19.8 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████████▌     | 51 kB 12.3 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████████▉| 61 kB 14.4 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████████| 61 kB 333 kB/s \r\n",
      "\u001b[?25hCollecting azure-storage-blob>=12.12.0\r\n",
      "  Downloading azure_storage_blob-12.15.0-py3-none-any.whl (387 kB)\r\n",
      "\u001b[?25l\r",
      "\u001b[K     |▉                               | 10 kB 32.6 MB/s eta 0:00:01\r",
      "\u001b[K     |█▊                              | 20 kB 43.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██▌                             | 30 kB 54.7 MB/s eta 0:00:01\r",
      "\u001b[K     |███▍                            | 40 kB 56.3 MB/s eta 0:00:01\r",
      "\u001b[K     |████▎                           | 51 kB 60.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████                           | 61 kB 66.6 MB/s eta 0:00:01\r",
      "\u001b[K     |██████                          | 71 kB 71.0 MB/s eta 0:00:01\r",
      "\u001b[K     |██████▊                         | 81 kB 75.9 MB/s eta 0:00:01\r",
      "\u001b[K     |███████▋                        | 92 kB 21.8 MB/s eta 0:00:01\r",
      "\u001b[K     |████████▌                       | 102 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████▎                      | 112 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████▏                     | 122 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████                     | 133 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████▉                    | 143 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████▊                   | 153 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████▌                  | 163 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████▍                 | 174 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████▏                | 184 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████                | 194 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████               | 204 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████▊              | 215 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████▋             | 225 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████▍            | 235 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████▎           | 245 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████▏          | 256 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████          | 266 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████▉         | 276 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████▋        | 286 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████▌       | 296 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████▍      | 307 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████████▏     | 317 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████     | 327 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████▉    | 337 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████▊   | 348 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████████▋  | 358 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████████████▍ | 368 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████████▎| 378 kB 23.2 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████████| 387 kB 23.2 MB/s \r\n",
      "\u001b[?25hRequirement already satisfied: azure-core<2.0.0,>=1.23.1 in /databricks/python3/lib/python3.9/site-packages (from adlfs) (1.26.1)\r\n",
      "Collecting aiohttp>=3.7.0\r\n",
      "  Downloading aiohttp-3.8.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)\r\n",
      "\u001b[?25l\r",
      "\u001b[K     |▎                               | 10 kB 33.3 MB/s eta 0:00:01\r",
      "\u001b[K     |▋                               | 20 kB 38.4 MB/s eta 0:00:01\r",
      "\u001b[K     |█                               | 30 kB 47.9 MB/s eta 0:00:01\r",
      "\u001b[K     |█▎                              | 40 kB 51.0 MB/s eta 0:00:01\r",
      "\u001b[K     |█▋                              | 51 kB 47.1 MB/s eta 0:00:01\r",
      "\u001b[K     |██                              | 61 kB 51.6 MB/s eta 0:00:01\r",
      "\u001b[K     |██▎                             | 71 kB 55.6 MB/s eta 0:00:01\r",
      "\u001b[K     |██▋                             | 81 kB 59.4 MB/s eta 0:00:01\r",
      "\u001b[K     |██▉                             | 92 kB 63.4 MB/s eta 0:00:01\r",
      "\u001b[K     |███▏                            | 102 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███▌                            | 112 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███▉                            | 122 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████▏                           | 133 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████▌                           | 143 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████▉                           | 153 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████▏                          | 163 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████▌                          | 174 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████▊                          | 184 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████                          | 194 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████▍                         | 204 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████▊                         | 215 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████                         | 225 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████▍                        | 235 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████▊                        | 245 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████                        | 256 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████▍                       | 266 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████▋                       | 276 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████                       | 286 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████▎                      | 296 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████▋                      | 307 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████                      | 317 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████▎                     | 327 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████▋                     | 337 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████                     | 348 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████▏                    | 358 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████▌                    | 368 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████▉                    | 378 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████▏                   | 389 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████▌                   | 399 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████▉                   | 409 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████▏                  | 419 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████▌                  | 430 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████▉                  | 440 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████                  | 450 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████▍                 | 460 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████▊                 | 471 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████                 | 481 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████▍                | 491 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████▊                | 501 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████                | 512 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████▍               | 522 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████▊               | 532 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████               | 542 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████▎              | 552 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████▋              | 563 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████              | 573 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████▎             | 583 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████▋             | 593 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████             | 604 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████▎            | 614 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████▋            | 624 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████▉            | 634 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████▏           | 645 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████▌           | 655 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████▉           | 665 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████▏          | 675 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████▌          | 686 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████▉          | 696 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████▏         | 706 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████▍         | 716 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████▊         | 727 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████         | 737 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████▍        | 747 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████▊        | 757 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████        | 768 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████▍       | 778 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████▊       | 788 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████       | 798 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████▎      | 808 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████▋      | 819 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████████      | 829 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████████▎     | 839 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████████▋     | 849 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████     | 860 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████▎    | 870 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████▋    | 880 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████    | 890 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████▏   | 901 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████▌   | 911 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████▉   | 921 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████████▏  | 931 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████████▌  | 942 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████████▉  | 952 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████████████▏ | 962 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████████████▌ | 972 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████████████▉ | 983 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████████ | 993 kB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████████▍| 1.0 MB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████████▊| 1.0 MB 66.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████████| 1.0 MB 66.5 MB/s \r\n",
      "\u001b[?25hCollecting azure-identity\r\n",
      "  Downloading azure_identity-1.12.0-py3-none-any.whl (135 kB)\r\n",
      "\u001b[?25l\r",
      "\u001b[K     |██▍                             | 10 kB 39.9 MB/s eta 0:00:01\r",
      "\u001b[K     |████▉                           | 20 kB 49.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████▎                        | 30 kB 61.3 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████▊                      | 40 kB 68.5 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████                    | 51 kB 73.9 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████▌                 | 61 kB 80.9 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████               | 71 kB 85.1 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████▍            | 81 kB 88.3 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████▊          | 92 kB 92.1 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████▏       | 102 kB 94.6 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████████▋     | 112 kB 94.6 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████████   | 122 kB 94.6 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████████▌| 133 kB 94.6 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████████| 135 kB 94.6 MB/s \r\n",
      "\u001b[?25hCollecting frozenlist>=1.1.1\r\n",
      "  Downloading frozenlist-1.3.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (158 kB)\r\n",
      "\u001b[?25l\r",
      "\u001b[K     |██                              | 10 kB 37.8 MB/s eta 0:00:01\r",
      "\u001b[K     |████▏                           | 20 kB 47.3 MB/s eta 0:00:01\r",
      "\u001b[K     |██████▏                         | 30 kB 59.0 MB/s eta 0:00:01\r",
      "\u001b[K     |████████▎                       | 40 kB 66.9 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████▎                     | 51 kB 72.2 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████▍                   | 61 kB 78.6 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████▌                 | 71 kB 82.6 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████▌               | 81 kB 85.6 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████▋             | 92 kB 89.6 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████▋           | 102 kB 91.9 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████▊         | 112 kB 91.9 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████▊       | 122 kB 91.9 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████████▉     | 133 kB 91.9 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████████   | 143 kB 91.9 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████████ | 153 kB 91.9 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████████| 158 kB 91.9 MB/s \r\n",
      "\u001b[?25hRequirement already satisfied: charset-normalizer<4.0,>=2.0 in /databricks/python3/lib/python3.9/site-packages (from aiohttp>=3.7.0->adlfs) (2.0.4)\r\n",
      "Collecting yarl<2.0,>=1.0\r\n",
      "  Downloading yarl-1.8.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (264 kB)\r\n",
      "\u001b[?25l\r",
      "\u001b[K     |█▎                              | 10 kB 45.8 MB/s eta 0:00:01\r",
      "\u001b[K     |██▌                             | 20 kB 55.1 MB/s eta 0:00:01\r",
      "\u001b[K     |███▊                            | 30 kB 66.3 MB/s eta 0:00:01\r",
      "\u001b[K     |█████                           | 40 kB 72.9 MB/s eta 0:00:01\r",
      "\u001b[K     |██████▏                         | 51 kB 76.9 MB/s eta 0:00:01\r",
      "\u001b[K     |███████▍                        | 61 kB 83.1 MB/s eta 0:00:01\r",
      "\u001b[K     |████████▊                       | 71 kB 87.1 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████                      | 81 kB 90.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████▏                    | 92 kB 94.3 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████▍                   | 102 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████▋                  | 112 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████▉                 | 122 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████                | 133 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████▍              | 143 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████▋             | 153 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████▉            | 163 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████           | 174 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████▎         | 184 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████▌        | 194 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████▊       | 204 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████████      | 215 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████▎    | 225 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████▌   | 235 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████████▊  | 245 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████████ | 256 kB 97.2 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████████| 264 kB 97.2 MB/s \r\n",
      "\u001b[?25hCollecting async-timeout<5.0,>=4.0.0a3\r\n",
      "  Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)\r\n",
      "Collecting aiosignal>=1.1.2\r\n",
      "  Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)\r\n",
      "Collecting multidict<7.0,>=4.5\r\n",
      "  Downloading multidict-6.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)\r\n",
      "\u001b[?25l\r",
      "\u001b[K     |██▉                             | 10 kB 39.4 MB/s eta 0:00:01\r",
      "\u001b[K     |█████▊                          | 20 kB 49.2 MB/s eta 0:00:01\r",
      "\u001b[K     |████████▋                       | 30 kB 60.9 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████▌                    | 40 kB 68.2 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████▍                 | 51 kB 72.6 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████▏              | 61 kB 79.1 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████            | 71 kB 82.5 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████         | 81 kB 87.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████▉      | 92 kB 90.1 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████▊   | 102 kB 92.4 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████████████▋| 112 kB 92.4 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████████| 114 kB 92.4 MB/s \r\n",
      "\u001b[?25hRequirement already satisfied: attrs>=17.3.0 in /databricks/python3/lib/python3.9/site-packages (from aiohttp>=3.7.0->adlfs) (21.4.0)\r\n",
      "Requirement already satisfied: six>=1.11.0 in /databricks/python3/lib/python3.9/site-packages (from azure-core<2.0.0,>=1.23.1->adlfs) (1.16.0)\r\n",
      "Requirement already satisfied: requests>=2.18.4 in /databricks/python3/lib/python3.9/site-packages (from azure-core<2.0.0,>=1.23.1->adlfs) (2.27.1)\r\n",
      "Requirement already satisfied: typing-extensions>=4.0.1 in /databricks/python3/lib/python3.9/site-packages (from azure-core<2.0.0,>=1.23.1->adlfs) (4.1.1)\r\n",
      "Collecting adal>=0.4.2\r\n",
      "  Downloading adal-1.2.7-py2.py3-none-any.whl (55 kB)\r\n",
      "\u001b[?25l\r",
      "\u001b[K     |██████                          | 10 kB 39.1 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████▉                    | 20 kB 49.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████▊              | 30 kB 60.6 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████████████████▋        | 40 kB 68.5 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████████▌  | 51 kB 74.6 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████████| 55 kB 3.3 MB/s \r\n",
      "\u001b[?25hRequirement already satisfied: cffi in /databricks/python3/lib/python3.9/site-packages (from azure-datalake-store<0.1,>=0.0.46->adlfs) (1.15.0)\r\n",
      "Requirement already satisfied: python-dateutil<3,>=2.1.0 in /databricks/python3/lib/python3.9/site-packages (from adal>=0.4.2->azure-datalake-store<0.1,>=0.0.46->adlfs) (2.8.2)\r\n",
      "Requirement already satisfied: cryptography>=1.1.0 in /databricks/python3/lib/python3.9/site-packages (from adal>=0.4.2->azure-datalake-store<0.1,>=0.0.46->adlfs) (3.4.8)\r\n",
      "Requirement already satisfied: PyJWT<3,>=1.0.0 in /databricks/python3/lib/python3.9/site-packages (from adal>=0.4.2->azure-datalake-store<0.1,>=0.0.46->adlfs) (2.6.0)\r\n",
      "Requirement already satisfied: isodate>=0.6.1 in /databricks/python3/lib/python3.9/site-packages (from azure-storage-blob>=12.12.0->adlfs) (0.6.1)\r\n",
      "Requirement already satisfied: pycparser in /databricks/python3/lib/python3.9/site-packages (from cffi->azure-datalake-store<0.1,>=0.0.46->adlfs) (2.21)\r\n",
      "Requirement already satisfied: idna<4,>=2.5 in /databricks/python3/lib/python3.9/site-packages (from requests>=2.18.4->azure-core<2.0.0,>=1.23.1->adlfs) (3.3)\r\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /databricks/python3/lib/python3.9/site-packages (from requests>=2.18.4->azure-core<2.0.0,>=1.23.1->adlfs) (1.26.9)\r\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /databricks/python3/lib/python3.9/site-packages (from requests>=2.18.4->azure-core<2.0.0,>=1.23.1->adlfs) (2021.10.8)\r\n",
      "Collecting msal-extensions<2.0.0,>=0.3.0\r\n",
      "  Downloading msal_extensions-1.0.0-py2.py3-none-any.whl (19 kB)\r\n",
      "Collecting msal<2.0.0,>=1.12.0\r\n",
      "  Downloading msal-1.21.0-py2.py3-none-any.whl (89 kB)\r\n",
      "\u001b[?25l\r",
      "\u001b[K     |███▋                            | 10 kB 33.3 MB/s eta 0:00:01\r",
      "\u001b[K     |███████▎                        | 20 kB 42.7 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████                     | 30 kB 53.7 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████▋                 | 40 kB 60.8 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████▏             | 51 kB 65.3 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████▉          | 61 kB 70.7 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████▌      | 71 kB 74.6 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████████▏  | 81 kB 78.3 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████████| 89 kB 9.5 MB/s \r\n",
      "\u001b[?25hCollecting portalocker<3,>=1.0\r\n",
      "  Downloading portalocker-2.7.0-py2.py3-none-any.whl (15 kB)\r\n",
      "Installing collected packages: portalocker, multidict, msal, frozenlist, yarl, msal-extensions, async-timeout, aiosignal, adal, azure-storage-blob, azure-identity, azure-datalake-store, aiohttp, adlfs\r\n",
      "Successfully installed adal-1.2.7 adlfs-2023.1.0 aiohttp-3.8.4 aiosignal-1.3.1 async-timeout-4.0.2 azure-datalake-store-0.0.52 azure-identity-1.12.0 azure-storage-blob-12.15.0 frozenlist-1.3.3 msal-1.21.0 msal-extensions-1.0.0 multidict-6.0.4 portalocker-2.7.0 yarl-1.8.2\r\n",
      "\u001b[33mWARNING: You are using pip version 21.2.4; however, version 23.0.1 is available.\r\n",
      "You should consider upgrading via the '/local_disk0/.ephemeral_nfs/envs/pythonEnv-ba66f562-f854-4b23-9191-cf0cc1b7ed77/bin/python -m pip install --upgrade pip' command.\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!pip install aepp\n",
    "!pip install adlfs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "fe533d43-2e75-4d3b-9310-54e4e799d44f",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Before any calls can take place, we need to configure the library and setup authentication credentials. For this you'll need the following piece of information. For information about how you can get these, please refer to the `Setup` section of the **Readme**:\n",
    "- Client ID\n",
    "- Client secret"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "dc26cd22-8eb5-460b-9766-0e266b094d55",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "import aepp\n",
    "\n",
    "aepp.configure(\n",
    "  environment=environment,\n",
    "  sandbox=sandbox_name,\n",
    "  org_id=ims_org_id,\n",
    "  scopes=scopes, \n",
    "  secret=client_secret,\n",
    "  client_id=client_id\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "fa2b4524-29c9-4bce-a81e-cc1c4ef1c498",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "# 1. Running a model on AEP data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "54a12e05-2587-4ffb-a80b-a75a6f173edf",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "In the previous week we generated our featurized data in the Data Landing Zone under the `dlz-destination` container. We can now read it so we can use it to train our ML model. Because this data can be pretty big, we want to first read it via a Spark dataframe, so we can then use a sample of it for training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "147b3214-6fcc-4991-9b30-5377e34e3095",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "The featurized data exported into the Data Landing Zone is under the format **cmle/egress/$DATASETID/exportTime=$EXPORTTIME**. We know the dataset ID which is in your config under `featurized_dataset_id` so we're just missing the export time so we know what to read. To get that we can simply list files in the DLZ and find what the value is. The first step is to retrieve the credentials for the DLZ related to the destination container:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "af95a9a8-be52-4f6f-bcec-6ff4c84e597f",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "from aepp import connector\n",
    "\n",
    "connector = connector.AdobeRequest(\n",
    "  config_object=aepp.config.config_object,\n",
    "  header=aepp.config.header,\n",
    "  loggingEnabled=False,\n",
    "  logger=None,\n",
    ")\n",
    "\n",
    "endpoint = aepp.config.endpoints[\"global\"] + \"/data/foundation/connectors/landingzone/credentials\"\n",
    "\n",
    "dlz_credentials = connector.getData(endpoint=endpoint, params={\n",
    "  \"type\": \"dlz_destination\"\n",
    "})\n",
    "dlz_container = dlz_credentials[\"containerName\"]\n",
    "dlz_sas_token = dlz_credentials[\"SASToken\"]\n",
    "dlz_storage_account = dlz_credentials[\"storageAccountName\"]\n",
    "dlz_sas_uri = dlz_credentials[\"SASUri\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "7798bac9-4baf-45d4-bceb-60eed1f5add9",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Now we use some Python libraries to authenticate and issue listing commands so we can get the paths and extract the time from it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "abc50edb-5674-4572-8936-3724ea893127",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using featurized data export time of 20230401140556\n"
     ]
    }
   ],
   "source": [
    "from adlfs import AzureBlobFileSystem\n",
    "from fsspec import AbstractFileSystem\n",
    "\n",
    "fs = AzureBlobFileSystem(\n",
    "  account_name=dlz_storage_account,\n",
    "  sas_token=dlz_sas_token\n",
    ")\n",
    "\n",
    "def get_export_time(fs: AbstractFileSystem, container_name: str, base_path: str, dataset_id: str):\n",
    "  featurized_data_base_path = f\"{container_name}/{base_path}/{dataset_id}\"\n",
    "  featurized_data_export_paths = fs.ls(featurized_data_base_path)\n",
    "  \n",
    "  if len(featurized_data_export_paths) == 0:\n",
    "    raise Exception(f\"Found no exports for featurized data from dataset ID {dataset_id} under path {featurized_data_base_path}\")\n",
    "  elif len(featurized_data_export_paths) > 1:\n",
    "    print(f\"Found {len(featurized_data_export_paths)} exports from dataset dataset ID {dataset_id} under path {featurized_data_base_path}, using most recent one\")\n",
    "  \n",
    "  featurized_data_export_path = featurized_data_export_paths[-1]\n",
    "  featurized_data_export_time = featurized_data_export_path.strip().split(\"/\")[-1].split(\"=\")[-1]\n",
    "  return featurized_data_export_time\n",
    "\n",
    "export_time = get_export_time(fs, dlz_container, export_path, featurized_dataset_id)\n",
    "print(f\"Using featurized data export time of {export_time}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "344f7396-c122-47c7-83ae-2096b452054b",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "At that point we're ready to read this data. We're using Spark since it could be pretty large as we're not doing any sampling. Spark needs the following properties to be able to authenticate using SAS:\n",
    "- `fs.azure.account.auth.type.$ACCOUNT.dfs.core.windows.net` should be set to `SAS`.\n",
    "- `fs.azure.sas.token.provider.type.$ACCOUNT.dfs.core.windows.net` should be set to `org.apache.hadoop.fs.azurebfs.sas.FixedSASTokenProvider`.\n",
    "- `fs.azure.sas.fixed.token.$ACCOUNT.dfs.core.windows.net` should be set to the SAS token retrieved earlier.\n",
    "\n",
    "Let's put that in practice and create a Spark dataframe containing the entire featurized data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "bace8783-6773-42e7-86be-1f5134654453",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      " |-- userId: string (nullable = true)\n",
      " |-- eventType: string (nullable = true)\n",
      " |-- timestamp: timestamp (nullable = true)\n",
      " |-- subscriptionOccurred: long (nullable = true)\n",
      " |-- emailsReceived: long (nullable = true)\n",
      " |-- emailsOpened: long (nullable = true)\n",
      " |-- emailsClicked: long (nullable = true)\n",
      " |-- productsViewed: long (nullable = true)\n",
      " |-- propositionInteracts: long (nullable = true)\n",
      " |-- propositionDismissed: long (nullable = true)\n",
      " |-- webLinkClicks: long (nullable = true)\n",
      " |-- minutes_since_emailSent: integer (nullable = true)\n",
      " |-- minutes_since_emailOpened: integer (nullable = true)\n",
      " |-- minutes_since_emailClick: integer (nullable = true)\n",
      " |-- minutes_since_productView: integer (nullable = true)\n",
      " |-- minutes_since_propositionInteract: integer (nullable = true)\n",
      " |-- minutes_since_propositionDismiss: integer (nullable = true)\n",
      " |-- minutes_since_linkClick: integer (nullable = true)\n",
      " |-- random_row_number_for_user: integer (nullable = true)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "spark.conf.set(f\"fs.azure.account.auth.type.{dlz_storage_account}.dfs.core.windows.net\", \"SAS\")\n",
    "spark.conf.set(f\"fs.azure.sas.token.provider.type.{dlz_storage_account}.dfs.core.windows.net\", \"org.apache.hadoop.fs.azurebfs.sas.FixedSASTokenProvider\")\n",
    "spark.conf.set(f\"fs.azure.sas.fixed.token.{dlz_storage_account}.dfs.core.windows.net\", dlz_sas_token)\n",
    "\n",
    "protocol = \"abfss\"\n",
    "input_path = f\"{protocol}://{dlz_container}@{dlz_storage_account}.dfs.core.windows.net/{export_path}/{featurized_dataset_id}/exportTime={export_time}/\"\n",
    "\n",
    "df = spark.read.parquet(input_path)\n",
    "df.printSchema()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "153894ac-e08c-4cd7-b43c-e9db5a834384",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "We can then sample it to keep only a portion of the data for training before we bring the data in memory for use in the `scikit-learn` library. Here we're just going to use a sampling ratio of 50%, but you are welcome to use a bigger or smaller ratio. We use sampling **without** replacement to ensure the same profiles don't get picked up multiple times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "f90bca5c-16d6-4603-b85d-ce4b5fad185b",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "sampling_ratio = 0.5\n",
    "df = df.sample(withReplacement=False, fraction=sampling_ratio)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "6e40df83-de43-4588-bf61-abb0b9bb77ce",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "## 1.1 Creating baseline models as experiments in MLFlow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "c69d7233-f138-484d-b5f8-a768a6ed8541",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Before doing any ML we can look at summary statistics to understand the structure of the data, and what kind of algorithm(s) might be suited to solve the problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "2c2a5ca8-ab42-4ca3-a221-453eb31f59d9",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Out[10]: DataFrame[summary: string, userId: string, eventType: string, subscriptionOccurred: string, emailsReceived: string, emailsOpened: string, emailsClicked: string, productsViewed: string, propositionInteracts: string, propositionDismissed: string, webLinkClicks: string, minutes_since_emailSent: string, minutes_since_emailOpened: string, minutes_since_emailClick: string, minutes_since_productView: string, minutes_since_propositionInteract: string, minutes_since_propositionDismiss: string, minutes_since_linkClick: string, random_row_number_for_user: string]"
     ]
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "46f70ef9-2dcf-44a1-b811-1a7b646b598c",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "To keep the model name unique we append the username to the model name:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "35dd9c15-f90f-4f5d-8168-649a608eff42",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "model_name = f\"{model_name}_{unique_id}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "288890c1-9218-4a35-90e1-eb4e47540fc9",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "In order to feed data to our model, we need to do a few preparation steps:\n",
    "- Separate the target variable (which in our case is whether a subscription occured or not) from the other variables.\n",
    "- Split the data into a training and test set so we can evaluate our model performance down the line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "79327fc8-edc3-483b-bbc1-2614bc248cae",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import roc_auc_score\n",
    "\n",
    "df_train=df.toPandas()\n",
    "\n",
    "# Feature Selection\n",
    "features = ['emailsReceived', 'emailsOpened', 'emailsClicked', \n",
    "            'productsViewed', \n",
    "            'propositionInteracts', 'propositionDismissed', 'webLinkClicks', \n",
    "            'minutes_since_emailSent', 'minutes_since_emailOpened', 'minutes_since_emailClick', \n",
    "            'minutes_since_productView', 'minutes_since_propositionInteract', \n",
    "            'minutes_since_propositionDismiss', 'minutes_since_linkClick']\n",
    "X, y = df_train[features].fillna(0), df_train[\"subscriptionOccurred\"]\n",
    "\n",
    "# Train test split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.8, random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "7e5d7093-644e-4668-99fb-7c3915367df3",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "The predict method of sklearn's RandomForestClassifier returns a binary classification (0 or 1). The following code creates a wrapper function, `SklearnModelWrapper`, that uses the `predict_proba` method to return the probability that the observation belongs to each class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "69b82a53-98d2-4bf5-892c-dfacb39e56dd",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "import mlflow\n",
    "import mlflow.pyfunc\n",
    "\n",
    "class SklearnModelWrapper(mlflow.pyfunc.PythonModel):\n",
    "  def __init__(self, model):\n",
    "    self.model = model\n",
    "    \n",
    "  def predict(self, context, model_input):\n",
    "    return self.model.predict_proba(model_input)[:,1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "2e2e8f69-e419-44ba-a6ef-81093cf92f56",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "This task seems well suited to a random forest classifier, since the output is binary and there may be interactions between multiple variables.\n",
    "\n",
    "The following code builds a simple classifier using scikit-learn. It uses MLflow to keep track of the model accuracy, and to save the model for later use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "2ffdbba6-e384-4e40-adfa-307e60826fbb",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Random Forest (RF) Accuracy: 0.9283688337689784\n",
      "Random Forest (RF) AUC score: 0.9492314066678459\n",
      "/databricks/python/lib/python3.9/site-packages/mlflow/models/signature.py:130: UserWarning: Hint: Inferred schema contains integer column(s). Integer columns in Python cannot represent missing values. If your input data contains missing values at inference time, it will be encoded as floats and will cause a schema enforcement error. The best way to avoid this problem is to infer the model schema based on a realistic data sample (training dataset) that includes missing values. Alternatively, you can declare integer columns as doubles (float64) whenever these columns may have missing values. See `Handling Integers With Missing Values <https://www.mlflow.org/docs/latest/models.html#handling-integers-with-missing-values>`_ for more details.\n",
      "  inputs = _infer_schema(model_input)\n",
      "/databricks/python/lib/python3.9/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.\n",
      "  warnings.warn(\"Setuptools is replacing distutils.\")\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "from mlflow.models.signature import infer_signature\n",
    "from mlflow.utils.environment import _mlflow_conda_env\n",
    "\n",
    "import cloudpickle\n",
    "import sklearn\n",
    "\n",
    "\n",
    "run_name_untuned = f\"{model_name}_untuned\"\n",
    "\n",
    "# mlflow.start_run creates a new MLflow run to track the performance of this model. \n",
    "# Within the context, you call mlflow.log_param to keep track of the parameters used, and\n",
    "# mlflow.log_metric to record metrics like accuracy.\n",
    "with mlflow.start_run(run_name=run_name_untuned):\n",
    "  n_estimators = 10\n",
    "  rf_clf = RandomForestClassifier(n_estimators=n_estimators)\n",
    "  rf_clf.fit(X_train, y_train)\n",
    "  \n",
    "  mlflow.log_param('n_estimators', n_estimators)\n",
    "  \n",
    "  accuracy = rf_clf.score(X_test, y_test)\n",
    "  # Use the accuracy as a metric in MLFlow\n",
    "  mlflow.log_metric('accuracy', accuracy)\n",
    "  print(\"Random Forest (RF) Accuracy:\", accuracy)\n",
    "  \n",
    "  # predict_proba returns [prob_negative, prob_positive], so slice the output with [:, 1]\n",
    "  predictions_test = rf_clf.predict_proba(X_test)[:,1]\n",
    "  auc_score = roc_auc_score(y_test, predictions_test)\n",
    "  # Use the area under the ROC curve as a metric in MLFlow\n",
    "  mlflow.log_metric('auc', auc_score)\n",
    "  print(\"Random Forest (RF) AUC score:\", auc_score)\n",
    "  \n",
    "  # Wrap the model so it can be registered in MLFlow\n",
    "  rf_wrapped = SklearnModelWrapper(rf_clf)\n",
    "  \n",
    "  # Log the model with a signature that defines the schema of the model's inputs and outputs. \n",
    "  # When the model is deployed, this signature will be used to validate inputs.\n",
    "  signature = infer_signature(X_train, rf_wrapped.predict(None, X_train))\n",
    "  \n",
    "  # MLflow contains utilities to create a conda environment used to serve models.\n",
    "  # The necessary dependencies are added to a conda.yaml file which is logged along with the model.\n",
    "  conda_env =  _mlflow_conda_env(\n",
    "    additional_conda_deps=None,\n",
    "    additional_pip_deps=[\"cloudpickle=={}\".format(cloudpickle.__version__), \"scikit-learn=={}\".format(sklearn.__version__)],\n",
    "    additional_conda_channels=None,\n",
    "  )\n",
    "  mlflow.pyfunc.log_model(\"random_forest_model\", python_model=rf_wrapped, conda_env=conda_env, signature=signature)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "7f3ae656-e72f-4c12-8c73-3f52158036a1",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "## 1.2 Registering models in the Model Hub with MLFlow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "b3ba9f4f-2a13-419c-9da6-915fd45cf48f",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Now that we have a couple baseline models, we can already start registering them as full-fledged models in the Model Registry so we can easily fetch them again later to compare with future iterations of the models once we do more tuning. By registering this model in Model Registry, you can easily reference the model from anywhere within Databricks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "4db55dbe-cf53-4049-8dd5-31f7e3acfa97",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Let's start with the Random Forest model. First we need to find out the run ID where the experiment using that algorithm was stored. We can use the `mlflow` library again to find out by looking up our experiment run ID created earlier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "15556b32-ca57-4e51-b836-2ceb817cbc0c",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Out[15]: 'c68c1eb61259477d97bf97ab0a174a7e'"
     ]
    }
   ],
   "source": [
    "rf_run_id = mlflow.search_runs(filter_string=f'tags.mlflow.runName = \"{run_name_untuned}\"').iloc[0].run_id\n",
    "rf_run_id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "5e440c62-634e-4f61-92de-1a5ca0f9f4e5",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Now we're ready to register it. To register we need 2 pieces of information:\n",
    "- The path under which the model artifacts were stored.\n",
    "- The name of the model we'd like to register it under."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "7767acdd-3a32-4601-9891-0061cbaa45ed",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Successfully registered model 'cmle_propensity_model_cmenguy_adobe_com'.\n",
      "2023/04/05 16:31:11 INFO mlflow.tracking._model_registry.client: Waiting up to 300 seconds for model version to finish creation.                     Model name: cmle_propensity_model_cmenguy_adobe_com, version 1\n",
      "Created version '1' of model 'cmle_propensity_model_cmenguy_adobe_com'.\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "rf_model_name = model_name\n",
    "rf_model_version = mlflow.register_model(f\"runs:/{rf_run_id}/random_forest_model\", rf_model_name)\n",
    " \n",
    "# Registering the model takes a few seconds, so add a small delay\n",
    "time.sleep(15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "ab8c1cc2-b143-40de-9b70-b33377fa79e3",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "## 1.3 Tuning models at scale and selecting production model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "b9c1549b-8c86-41fb-93fb-8ee54aa93853",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Now that we have our baseline models computed and saved, we need to tune them and find the best performing model out of all of them. Each of these 2 algorithms has a lot of hyper-parameters, and different combinations of those can yield vastly different models with dramatically different performance characteristics. Because it wouldn't be possible to try all these combinations manually, we are leveraging Databricks' large-scale computing capabilities to distribute the hyper-parameter tuning process."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "108648d8-6d81-41fe-8997-5ff5ce71e241",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "The library `hyperopt` provides a good way to create efficient and scalable hyper-parameter tuning process, and integrates with Apache Spark via the use of `SparkTrials` to distribute the workload.\n",
    "\n",
    "The first thing we need to figure out is the degree of parallelism to use. There is a trade-off here:\n",
    "- Greater parallelism will lead to speedups, but a less optimal hyper-parameter sweep.\n",
    "- Lower parallelism will be slower but will do a better job trying the various combinations of hyper-parameters.\n",
    "\n",
    "A rule of thumb is to determine the number of trials you want to run, and then set the degree of parallelism to be the square root of that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "b0fe2c60-e81b-4cf6-8b90-2d2816a91594",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "from hyperopt import SparkTrials\n",
    "\n",
    "# Feel free to change max_evals if you want fewer/more trial runs\n",
    "max_evals = 100\n",
    "parallelism = int(math.sqrt(max_evals))\n",
    "spark_trials = SparkTrials(parallelism=parallelism)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "7ecda013-654c-46b3-bd13-bc90dd1aeba9",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Now we put everything together, and we'll be tracking each hyper-parameter tuning trial into a separate MLFlow experiment so we can easily refer to them later to find out the best-performing ones. Each parameter configuration will be saved in MLFlow, so we do not need to save anything manually.\n",
    "\n",
    "One thing we need to determine is the search space for the various hyper-parameters, which can greatly impact the quality of the distributed tuning. You can refer to the official API pages for [Random Forest Classifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html) to find out the range of acceptable values. We implement distributions following these definitions to define the search space below, and create a single search space that includes the choice of algorithm so we can run trials for both Random Forest in a distributed tuning job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "38ce1ad3-cd2c-4375-961d-60991286573f",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "from hyperopt import fmin, tpe, hp, Trials, STATUS_OK\n",
    "from hyperopt.pyll import scope\n",
    "\n",
    "rf_space = {\n",
    "  \"model\": \"random_forest\",\n",
    "  \"kwargs\": {\n",
    "    \"n_estimators\": scope.int(hp.quniform(\"rf_n_estimators\", 10, 200, 10)),\n",
    "    \"max_depth\": scope.int(hp.quniform(\"rf_max_depth\", 2, 12, 1)),\n",
    "    \"criterion\": hp.choice(\"rf_criterion\", [\"gini\", \"entropy\"]),\n",
    "    \"min_samples_leaf\": scope.int(hp.uniform(\"rf_min_samples_leaf\", 1, 5)),\n",
    "    \"min_samples_split\": scope.float(hp.uniform(\"rf_min_samples_split\", 0.01, 0.05))\n",
    "  }\n",
    "}\n",
    "\n",
    "search_space = {\n",
    "  \"model_choice\": hp.choice(\"model_choice\", [rf_space])\n",
    "}\n",
    "\n",
    "models = {\n",
    "  \"random_forest\": RandomForestClassifier\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "d8c49f64-9cbb-497c-9396-0fbb5c734c41",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "All that's left is to define the objective function that will be distributed in the cluster. `hyperopt` will be passing a sample which contains all the hyper-parameters chosen for a given trial (including the choice of algorithm), and we translate from that sample to an initialized model. Because in the search space we used keys matching the parameters for these algorithms, we can directly unpack the dictionary to get a model initialized with the requested hyper-parameters.\n",
    "\n",
    "Depending on your `max_evals` value earlier and the size of your Databricks cluster this could take some time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "b02a04cd-f7cb-4366-80b4-393c06d2b0af",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:hyperopt-spark:Hyperopt with SparkTrials will automatically track trials in MLflow. To view the MLflow experiment associated with the notebook, click the 'Runs' icon in the notebook context bar on the upper right. There, you can view all runs.\n",
      "INFO:hyperopt-spark:To view logs from trials, please check the Spark executor logs. To view executor logs, expand 'Spark Jobs' above until you see the (i) icon next to the stage from the trial job. Click it and find the list of tasks. Click the 'stderr' link for a task to view trial logs.\n",
      "\r",
      "  0%|          | 0/1000 [00:00<?, ?trial/s, best loss=?]\r",
      "  0%|          | 1/1000 [00:11<3:06:00, 11.17s/trial, best loss: -0.9626329836027753]\r",
      "  0%|          | 3/1000 [00:12<54:19,  3.27s/trial, best loss: -0.9626329836027753]  \r",
      "  0%|          | 4/1000 [00:14<47:01,  2.83s/trial, best loss: -0.9626329836027753]\r",
      "  1%|          | 6/1000 [00:15<28:05,  1.70s/trial, best loss: -0.9656749484632106]\r",
      "  1%|          | 7/1000 [00:16<25:03,  1.51s/trial, best loss: -0.9656749484632106]\r",
      "  1%|          | 8/1000 [00:17<22:44,  1.38s/trial, best loss: -0.9656749484632106]\r",
      "  1%|          | 9/1000 [00:18<21:00,  1.27s/trial, best loss: -0.9656749484632106]\r",
      "  1%|          | 11/1000 [00:19<15:19,  1.08trial/s, best loss: -0.9656749484632106]\r",
      "  1%|          | 12/1000 [00:20<15:36,  1.05trial/s, best loss: -0.9656749484632106]\r",
      "  2%|▏         | 15/1000 [00:23<16:07,  1.02trial/s, best loss: -0.9656749484632106]\r",
      "  2%|▏         | 16/1000 [00:24<16:28,  1.00s/trial, best loss: -0.967084334369667] \r",
      "  2%|▏         | 17/1000 [00:26<20:06,  1.23s/trial, best loss: -0.967084334369667]\r",
      "  2%|▏         | 20/1000 [00:27<13:02,  1.25trial/s, best loss: -0.967084334369667]\r",
      "  2%|▏         | 23/1000 [00:30<14:37,  1.11trial/s, best loss: -0.9683647510451154]\r",
      "  2%|▏         | 24/1000 [00:31<14:56,  1.09trial/s, best loss: -0.9683647510451154]\r",
      "  2%|▎         | 25/1000 [00:33<18:33,  1.14s/trial, best loss: -0.9684950532282324]\r",
      "  3%|▎         | 27/1000 [00:34<14:55,  1.09trial/s, best loss: -0.9684950532282324]\r",
      "  3%|▎         | 28/1000 [00:35<15:13,  1.06trial/s, best loss: -0.9684950532282324]\r",
      "  3%|▎         | 30/1000 [00:38<18:37,  1.15s/trial, best loss: -0.9684950532282324]\r",
      "  3%|▎         | 32/1000 [00:39<15:02,  1.07trial/s, best loss: -0.9684950532282324]\r",
      "  3%|▎         | 34/1000 [00:41<15:26,  1.04trial/s, best loss: -0.9684950532282324]\r",
      "  4%|▎         | 35/1000 [00:42<15:36,  1.03trial/s, best loss: -0.9684950532282324]\r",
      "  4%|▎         | 36/1000 [00:43<15:45,  1.02trial/s, best loss: -0.9684950532282324]\r",
      "  4%|▎         | 37/1000 [00:44<15:52,  1.01trial/s, best loss: -0.9684950532282324]\r",
      "  4%|▍         | 39/1000 [00:45<12:45,  1.25trial/s, best loss: -0.9684950532282324]\r",
      "  4%|▍         | 40/1000 [00:47<17:08,  1.07s/trial, best loss: -0.9684950532282324]\r",
      "  4%|▍         | 42/1000 [00:48<13:37,  1.17trial/s, best loss: -0.9684950532282324]\r",
      "  4%|▍         | 43/1000 [00:50<14:10,  1.13trial/s, best loss: -0.9684950532282324]\r",
      "  4%|▍         | 44/1000 [00:51<14:38,  1.09trial/s, best loss: -0.9684950532282324]\r",
      "  4%|▍         | 45/1000 [00:52<15:01,  1.06trial/s, best loss: -0.9684950532282324]\r",
      "  5%|▍         | 46/1000 [00:53<15:19,  1.04trial/s, best loss: -0.9684950532282324]\r",
      "  5%|▍         | 48/1000 [00:54<12:11,  1.30trial/s, best loss: -0.9684950532282324]\r",
      "  5%|▍         | 49/1000 [00:55<13:06,  1.21trial/s, best loss: -0.9684950532282324]\r",
      "  5%|▌         | 50/1000 [00:57<17:54,  1.13s/trial, best loss: -0.9684950532282324]\r",
      "  5%|▌         | 53/1000 [00:58<11:21,  1.39trial/s, best loss: -0.9684950532282324]\r",
      "  5%|▌         | 54/1000 [01:00<15:27,  1.02trial/s, best loss: -0.9684950532282324]\r",
      "  6%|▌         | 55/1000 [01:01<15:34,  1.01trial/s, best loss: -0.9684950532282324]\r",
      "  6%|▌         | 57/1000 [01:02<12:37,  1.24trial/s, best loss: -0.9684950532282324]\r",
      "  6%|▌         | 59/1000 [01:04<13:48,  1.14trial/s, best loss: -0.9684950532282324]\r",
      "  6%|▌         | 61/1000 [01:06<14:30,  1.08trial/s, best loss: -0.9684950532282324]\r",
      "  6%|▌         | 62/1000 [01:07<14:46,  1.06trial/s, best loss: -0.9684950532282324]\r",
      "  6%|▋         | 63/1000 [01:08<15:00,  1.04trial/s, best loss: -0.9684950532282324]\r",
      "  6%|▋         | 64/1000 [01:09<15:12,  1.03trial/s, best loss: -0.9684950532282324]\r",
      "  7%|▋         | 66/1000 [01:11<15:27,  1.01trial/s, best loss: -0.9684950532282324]\r",
      "  7%|▋         | 67/1000 [01:12<15:49,  1.02s/trial, best loss: -0.9685657516629143]\r",
      "  7%|▋         | 68/1000 [01:13<15:48,  1.02s/trial, best loss: -0.9685657516629143]\r",
      "  7%|▋         | 69/1000 [01:14<15:47,  1.02s/trial, best loss: -0.9685657516629143]\r",
      "  7%|▋         | 70/1000 [01:15<15:47,  1.02s/trial, best loss: -0.9685657516629143]\r",
      "  7%|▋         | 71/1000 [01:16<15:47,  1.02s/trial, best loss: -0.9685657516629143]\r",
      "  7%|▋         | 72/1000 [01:17<15:47,  1.02s/trial, best loss: -0.9685657516629143]\r",
      "  7%|▋         | 73/1000 [01:18<15:46,  1.02s/trial, best loss: -0.9685657516629143]\r",
      "  7%|▋         | 74/1000 [01:19<15:44,  1.02s/trial, best loss: -0.9685657516629143]\r",
      "  8%|▊         | 76/1000 [01:20<12:08,  1.27trial/s, best loss: -0.9685657516629143]\r",
      "  8%|▊         | 78/1000 [01:22<13:31,  1.14trial/s, best loss: -0.9685657516629143]\r",
      "  8%|▊         | 80/1000 [01:24<14:16,  1.07trial/s, best loss: -0.9685657516629143]\r",
      "  8%|▊         | 81/1000 [01:26<17:44,  1.16s/trial, best loss: -0.9685657516629143]\r",
      "  8%|▊         | 82/1000 [01:27<17:15,  1.13s/trial, best loss: -0.9685657516629143]\r",
      "  8%|▊         | 83/1000 [01:28<16:50,  1.10s/trial, best loss: -0.9685657516629143]\r",
      "  8%|▊         | 85/1000 [01:29<13:06,  1.16trial/s, best loss: -0.9685657516629143]\r",
      "  9%|▊         | 86/1000 [01:31<17:11,  1.13s/trial, best loss: -0.9685657516629143]\r",
      "  9%|▊         | 87/1000 [01:32<16:46,  1.10s/trial, best loss: -0.9685657516629143]\r",
      "  9%|▉         | 89/1000 [01:33<13:02,  1.16trial/s, best loss: -0.9685657516629143]\r",
      "  9%|▉         | 91/1000 [01:35<13:55,  1.09trial/s, best loss: -0.9685657516629143]\r",
      "  9%|▉         | 92/1000 [01:38<17:26,  1.15s/trial, best loss: -0.9685657516629143]\r",
      "  9%|▉         | 93/1000 [01:39<16:58,  1.12s/trial, best loss: -0.9685657516629143]\r",
      "  9%|▉         | 94/1000 [01:40<16:34,  1.10s/trial, best loss: -0.9685657516629143]\r",
      " 10%|▉         | 95/1000 [01:41<16:15,  1.08s/trial, best loss: -0.9685657516629143]\r",
      " 10%|▉         | 96/1000 [01:42<16:19,  1.08s/trial, best loss: -0.9689232365781918]\r",
      " 10%|▉         | 98/1000 [01:43<12:31,  1.20trial/s, best loss: -0.9689232365781918]\r",
      " 10%|▉         | 99/1000 [01:44<13:10,  1.14trial/s, best loss: -0.9689232365781918]\r",
      " 10%|█         | 100/1000 [01:46<17:35,  1.17s/trial, best loss: -0.9689232365781918]\r",
      " 10%|█         | 101/1000 [01:48<21:01,  1.40s/trial, best loss: -0.9689232365781918]\r",
      " 10%|█         | 102/1000 [01:50<23:38,  1.58s/trial, best loss: -0.9689232365781918]\r",
      " 10%|█         | 104/1000 [01:51<16:33,  1.11s/trial, best loss: -0.9689232365781918]\r",
      " 10%|█         | 105/1000 [01:52<16:13,  1.09s/trial, best loss: -0.9689232365781918]\r",
      " 11%|█         | 106/1000 [01:54<19:50,  1.33s/trial, best loss: -0.9689232365781918]\r",
      " 11%|█         | 109/1000 [01:55<12:06,  1.23trial/s, best loss: -0.9689232365781918]\r",
      " 11%|█         | 110/1000 [01:57<15:42,  1.06s/trial, best loss: -0.9689232365781918]\r",
      " 11%|█         | 112/1000 [01:59<15:28,  1.05s/trial, best loss: -0.9689232365781918]\r",
      " 11%|█▏        | 113/1000 [02:00<15:23,  1.04s/trial, best loss: -0.9689232365781918]\r",
      " 11%|█▏        | 114/1000 [02:01<15:18,  1.04s/trial, best loss: -0.9689232365781918]\r",
      " 12%|█▏        | 115/1000 [02:02<15:14,  1.03s/trial, best loss: -0.9689232365781918]\r",
      " 12%|█▏        | 116/1000 [02:03<15:28,  1.05s/trial, best loss: -0.9693741932315002]\r",
      " 12%|█▏        | 117/1000 [02:04<15:20,  1.04s/trial, best loss: -0.9693741932315002]\r",
      " 12%|█▏        | 119/1000 [02:05<11:54,  1.23trial/s, best loss: -0.9693741932315002]\r",
      " 12%|█▏        | 120/1000 [02:07<16:10,  1.10s/trial, best loss: -0.9693741932315002]\r",
      " 12%|█▏        | 121/1000 [02:08<15:51,  1.08s/trial, best loss: -0.9693741932315002]\r",
      " 12%|█▏        | 123/1000 [02:09<12:19,  1.19trial/s, best loss: -0.9693741932315002]\r",
      " 13%|█▎        | 126/1000 [02:12<13:31,  1.08trial/s, best loss: -0.9693741932315002]\r",
      " 13%|█▎        | 127/1000 [02:13<13:45,  1.06trial/s, best loss: -0.9693741932315002]\r",
      " 13%|█▎        | 129/1000 [02:17<16:39,  1.15s/trial, best loss: -0.9693741932315002]\r",
      " 13%|█▎        | 130/1000 [02:18<16:17,  1.12s/trial, best loss: -0.9693741932315002]\r",
      " 13%|█▎        | 131/1000 [02:19<16:00,  1.11s/trial, best loss: -0.9693741932315002]\r",
      " 13%|█▎        | 133/1000 [02:21<15:31,  1.07s/trial, best loss: -0.9693741932315002]\r",
      " 13%|█▎        | 134/1000 [02:22<15:22,  1.06s/trial, best loss: -0.9693741932315002]\r",
      " 14%|█▎        | 135/1000 [02:23<15:13,  1.06s/trial, best loss: -0.9693741932315002]\r",
      " 14%|█▎        | 136/1000 [02:24<15:05,  1.05s/trial, best loss: -0.9693741932315002]\r",
      " 14%|█▍        | 138/1000 [02:25<11:50,  1.21trial/s, best loss: -0.9693741932315002]\r",
      " 14%|█▍        | 139/1000 [02:27<15:53,  1.11s/trial, best loss: -0.9693741932315002]\r",
      " 14%|█▍        | 140/1000 [02:28<15:35,  1.09s/trial, best loss: -0.9693741932315002]\r",
      " 14%|█▍        | 142/1000 [02:29<12:08,  1.18trial/s, best loss: -0.9693741932315002]\r",
      " 14%|█▍        | 143/1000 [02:30<12:41,  1.13trial/s, best loss: -0.9693741932315002]\r",
      " 14%|█▍        | 145/1000 [02:32<13:25,  1.06trial/s, best loss: -0.9693741932315002]\r",
      " 15%|█▍        | 146/1000 [02:34<16:52,  1.19s/trial, best loss: -0.9693741932315002]\r",
      " 15%|█▍        | 147/1000 [02:35<16:18,  1.15s/trial, best loss: -0.9693741932315002]\r",
      " 15%|█▍        | 148/1000 [02:36<15:51,  1.12s/trial, best loss: -0.9693741932315002]\r",
      " 15%|█▍        | 149/1000 [02:37<15:29,  1.09s/trial, best loss: -0.9693741932315002]\r",
      " 15%|█▌        | 150/1000 [02:38<15:12,  1.07s/trial, best loss: -0.9693741932315002]\r",
      " 15%|█▌        | 151/1000 [02:39<15:00,  1.06s/trial, best loss: -0.9693741932315002]\r",
      " 15%|█▌        | 152/1000 [02:40<14:51,  1.05s/trial, best loss: -0.9693741932315002]\r",
      " 15%|█▌        | 153/1000 [02:41<14:44,  1.04s/trial, best loss: -0.9693741932315002]\r",
      " 16%|█▌        | 155/1000 [02:42<11:19,  1.24trial/s, best loss: -0.9693741932315002]\r",
      " 16%|█▌        | 156/1000 [02:43<12:04,  1.16trial/s, best loss: -0.9693741932315002]\r",
      " 16%|█▌        | 157/1000 [02:45<16:23,  1.17s/trial, best loss: -0.9693741932315002]\r",
      " 16%|█▌        | 159/1000 [02:46<12:27,  1.13trial/s, best loss: -0.9693741932315002]\r",
      " 16%|█▌        | 160/1000 [02:47<12:52,  1.09trial/s, best loss: -0.9693741932315002]\r",
      " 16%|█▌        | 162/1000 [02:50<14:07,  1.01s/trial, best loss: -0.9693741932315002]\r",
      " 16%|█▋        | 163/1000 [02:51<14:09,  1.01s/trial, best loss: -0.9693741932315002]\r",
      " 16%|█▋        | 164/1000 [02:52<14:10,  1.02s/trial, best loss: -0.9693741932315002]\r",
      " 17%|█▋        | 166/1000 [02:54<14:10,  1.02s/trial, best loss: -0.9693741932315002]\r",
      " 17%|█▋        | 167/1000 [02:55<14:10,  1.02s/trial, best loss: -0.9693741932315002]\r",
      " 17%|█▋        | 168/1000 [02:56<14:10,  1.02s/trial, best loss: -0.9693741932315002]\r",
      " 17%|█▋        | 169/1000 [02:58<17:49,  1.29s/trial, best loss: -0.9693741932315002]\r",
      " 17%|█▋        | 171/1000 [02:59<13:14,  1.04trial/s, best loss: -0.9693741932315002]\r",
      " 17%|█▋        | 172/1000 [03:00<13:26,  1.03trial/s, best loss: -0.9693741932315002]\r",
      " 17%|█▋        | 173/1000 [03:02<17:08,  1.24s/trial, best loss: -0.9693741932315002]\r",
      " 18%|█▊        | 175/1000 [03:03<12:55,  1.06trial/s, best loss: -0.9693741932315002]\r",
      " 18%|█▊        | 176/1000 [03:04<13:11,  1.04trial/s, best loss: -0.9693741932315002]\r",
      " 18%|█▊        | 178/1000 [03:06<13:31,  1.01trial/s, best loss: -0.9693741932315002]\r",
      " 18%|█▊        | 179/1000 [03:07<13:37,  1.00trial/s, best loss: -0.9693741932315002]\r",
      " 18%|█▊        | 180/1000 [03:09<17:04,  1.25s/trial, best loss: -0.9693741932315002]\r",
      " 18%|█▊        | 181/1000 [03:10<16:16,  1.19s/trial, best loss: -0.9693741932315002]\r",
      " 18%|█▊        | 183/1000 [03:11<12:20,  1.10trial/s, best loss: -0.9693741932315002]\r",
      " 18%|█▊        | 184/1000 [03:12<12:42,  1.07trial/s, best loss: -0.9693741932315002]\r",
      " 18%|█▊        | 185/1000 [03:14<16:27,  1.21s/trial, best loss: -0.9693741932315002]\r",
      " 19%|█▊        | 187/1000 [03:15<12:29,  1.08trial/s, best loss: -0.9693741932315002]\r",
      " 19%|█▉        | 188/1000 [03:16<12:48,  1.06trial/s, best loss: -0.9693741932315002]\r",
      " 19%|█▉        | 189/1000 [03:17<13:03,  1.03trial/s, best loss: -0.9693741932315002]\r",
      " 19%|█▉        | 191/1000 [03:19<13:21,  1.01trial/s, best loss: -0.9693741932315002]\r",
      " 19%|█▉        | 193/1000 [03:21<13:30,  1.00s/trial, best loss: -0.9693741932315002]\r",
      " 19%|█▉        | 194/1000 [03:23<16:27,  1.22s/trial, best loss: -0.9693741932315002]\r",
      " 20%|█▉        | 195/1000 [03:25<15:49,  1.18s/trial, best loss: -0.9693741932315002]\r",
      " 20%|█▉        | 197/1000 [03:27<14:58,  1.12s/trial, best loss: -0.9693741932315002]\r",
      " 20%|█▉        | 199/1000 [03:28<12:01,  1.11trial/s, best loss: -0.9693741932315002]\r",
      " 20%|██        | 200/1000 [03:30<15:09,  1.14s/trial, best loss: -0.9693741932315002]\r",
      " 20%|██        | 201/1000 [03:31<14:48,  1.11s/trial, best loss: -0.9693741932315002]\r",
      " 20%|██        | 203/1000 [03:32<11:39,  1.14trial/s, best loss: -0.9693741932315002]\r",
      " 20%|██        | 204/1000 [03:33<12:04,  1.10trial/s, best loss: -0.9693741932315002]\r",
      " 21%|██        | 206/1000 [03:35<12:38,  1.05trial/s, best loss: -0.9693741932315002]\r",
      " 21%|██        | 207/1000 [03:36<12:50,  1.03trial/s, best loss: -0.9693741932315002]\r",
      " 21%|██        | 208/1000 [03:37<13:00,  1.02trial/s, best loss: -0.9693741932315002]\r",
      " 21%|██        | 211/1000 [03:39<11:08,  1.18trial/s, best loss: -0.9694597956605205]\r",
      " 21%|██▏       | 214/1000 [03:43<13:58,  1.07s/trial, best loss: -0.9694597956605205]\r",
      " 22%|██▏       | 215/1000 [03:45<16:07,  1.23s/trial, best loss: -0.9694597956605205]\r",
      " 22%|██▏       | 217/1000 [03:47<15:14,  1.17s/trial, best loss: -0.9694597956605205]\r",
      " 22%|██▏       | 219/1000 [03:49<14:38,  1.12s/trial, best loss: -0.9694597956605205]\r",
      " 22%|██▏       | 220/1000 [03:50<14:24,  1.11s/trial, best loss: -0.9694597956605205]\r",
      " 22%|██▏       | 221/1000 [03:52<17:00,  1.31s/trial, best loss: -0.9694597956605205]\r",
      " 22%|██▏       | 223/1000 [03:53<13:07,  1.01s/trial, best loss: -0.9694597956605205]\r",
      " 22%|██▏       | 224/1000 [03:55<15:57,  1.23s/trial, best loss: -0.9694597956605205]\r",
      " 22%|██▎       | 225/1000 [03:57<15:19,  1.19s/trial, best loss: -0.9694597956605205]\r",
      " 23%|██▎       | 226/1000 [03:59<18:05,  1.40s/trial, best loss: -0.9694597956605205]\r",
      " 23%|██▎       | 227/1000 [04:00<16:49,  1.31s/trial, best loss: -0.9694597956605205]\r",
      " 23%|██▎       | 229/1000 [04:01<12:25,  1.03trial/s, best loss: -0.9694597956605205]\r",
      " 23%|██▎       | 230/1000 [04:03<15:41,  1.22s/trial, best loss: -0.9694597956605205]\r",
      " 23%|██▎       | 231/1000 [04:04<15:03,  1.18s/trial, best loss: -0.9694597956605205]\r",
      " 23%|██▎       | 233/1000 [04:05<11:30,  1.11trial/s, best loss: -0.9694597956605205]\r",
      " 24%|██▎       | 235/1000 [04:07<12:05,  1.05trial/s, best loss: -0.9694597956605205]\r",
      " 24%|██▎       | 236/1000 [04:08<12:18,  1.04trial/s, best loss: -0.9694597956605205]\r",
      " 24%|██▎       | 237/1000 [04:09<12:28,  1.02trial/s, best loss: -0.9694597956605205]\r",
      " 24%|██▍       | 238/1000 [04:11<15:51,  1.25s/trial, best loss: -0.9694597956605205]\r",
      " 24%|██▍       | 239/1000 [04:12<15:07,  1.19s/trial, best loss: -0.9694597956605205]\r",
      " 24%|██▍       | 240/1000 [04:13<14:33,  1.15s/trial, best loss: -0.9694597956605205]\r",
      " 24%|██▍       | 241/1000 [04:14<14:06,  1.12s/trial, best loss: -0.9694597956605205]\r",
      " 24%|██▍       | 242/1000 [04:16<17:30,  1.39s/trial, best loss: -0.9694597956605205]\r",
      " 24%|██▍       | 244/1000 [04:17<12:31,  1.01trial/s, best loss: -0.9694597956605205]\r",
      " 24%|██▍       | 245/1000 [04:18<12:37,  1.00s/trial, best loss: -0.9694597956605205]\r",
      " 25%|██▍       | 246/1000 [04:19<12:42,  1.01s/trial, best loss: -0.9694597956605205]\r",
      " 25%|██▍       | 249/1000 [04:21<10:32,  1.19trial/s, best loss: -0.9694597956605205]\r",
      " 25%|██▌       | 252/1000 [04:25<13:22,  1.07s/trial, best loss: -0.9694597956605205]\r",
      " 25%|██▌       | 253/1000 [04:26<13:16,  1.07s/trial, best loss: -0.9694597956605205]\r",
      " 25%|██▌       | 254/1000 [04:28<15:40,  1.26s/trial, best loss: -0.9694597956605205]\r",
      " 26%|██▌       | 257/1000 [04:30<10:26,  1.19trial/s, best loss: -0.9694597956605205]\r",
      " 26%|██▌       | 258/1000 [04:32<13:17,  1.08s/trial, best loss: -0.9697555214804895]\r",
      " 26%|██▌       | 260/1000 [04:33<11:06,  1.11trial/s, best loss: -0.969868006129452] \r",
      " 26%|██▌       | 261/1000 [04:34<11:24,  1.08trial/s, best loss: -0.969868006129452]\r",
      " 26%|██▌       | 262/1000 [04:35<11:40,  1.05trial/s, best loss: -0.969868006129452]\r",
      " 26%|██▋       | 263/1000 [04:36<11:53,  1.03trial/s, best loss: -0.969868006129452]\r",
      " 26%|██▋       | 265/1000 [04:38<12:10,  1.01trial/s, best loss: -0.969868006129452]\r",
      " 27%|██▋       | 266/1000 [04:39<12:16,  1.00s/trial, best loss: -0.969868006129452]\r",
      " 27%|██▋       | 267/1000 [04:40<12:20,  1.01s/trial, best loss: -0.969868006129452]\r",
      " 27%|██▋       | 268/1000 [04:41<12:23,  1.02s/trial, best loss: -0.969868006129452]\r",
      " 27%|██▋       | 270/1000 [04:42<09:47,  1.24trial/s, best loss: -0.969868006129452]\r",
      " 27%|██▋       | 271/1000 [04:43<10:26,  1.16trial/s, best loss: -0.969868006129452]\r",
      " 27%|██▋       | 272/1000 [04:44<10:56,  1.11trial/s, best loss: -0.969868006129452]\r",
      " 27%|██▋       | 274/1000 [04:45<08:58,  1.35trial/s, best loss: -0.969868006129452]\r",
      " 28%|██▊       | 275/1000 [04:46<09:46,  1.24trial/s, best loss: -0.969868006129452]\r",
      " 28%|██▊       | 277/1000 [04:48<10:49,  1.11trial/s, best loss: -0.969868006129452]\r",
      " 28%|██▊       | 278/1000 [04:49<11:09,  1.08trial/s, best loss: -0.969868006129452]\r",
      " 28%|██▊       | 279/1000 [04:50<11:28,  1.05trial/s, best loss: -0.969868006129452]\r",
      " 28%|██▊       | 280/1000 [04:51<11:43,  1.02trial/s, best loss: -0.969868006129452]\r",
      " 28%|██▊       | 281/1000 [04:52<11:54,  1.01trial/s, best loss: -0.969868006129452]\r",
      " 28%|██▊       | 282/1000 [04:53<12:02,  1.01s/trial, best loss: -0.969868006129452]\r",
      " 28%|██▊       | 283/1000 [04:55<12:07,  1.01s/trial, best loss: -0.969868006129452]\r",
      " 28%|██▊       | 285/1000 [04:56<09:27,  1.26trial/s, best loss: -0.969868006129452]\r",
      " 29%|██▊       | 287/1000 [04:58<10:33,  1.13trial/s, best loss: -0.969868006129452]\r",
      " 29%|██▉       | 288/1000 [05:00<13:35,  1.15s/trial, best loss: -0.969868006129452]\r",
      " 29%|██▉       | 290/1000 [05:01<10:42,  1.11trial/s, best loss: -0.969868006129452]\r",
      " 29%|██▉       | 291/1000 [05:02<11:00,  1.07trial/s, best loss: -0.969868006129452]\r",
      " 29%|██▉       | 292/1000 [05:03<11:16,  1.05trial/s, best loss: -0.969868006129452]\r",
      " 29%|██▉       | 293/1000 [05:11<32:53,  2.79s/trial, best loss: -0.969868006129452]\r",
      " 30%|██▉       | 297/1000 [05:12<15:12,  1.30s/trial, best loss: -0.969868006129452]\r",
      " 30%|██▉       | 298/1000 [05:13<14:40,  1.25s/trial, best loss: -0.969868006129452]\r",
      " 30%|██▉       | 299/1000 [05:14<14:08,  1.21s/trial, best loss: -0.969868006129452]\r",
      " 30%|███       | 301/1000 [05:15<11:10,  1.04trial/s, best loss: -0.969868006129452]\r",
      " 30%|███       | 303/1000 [05:16<09:23,  1.24trial/s, best loss: -0.969868006129452]\r",
      " 30%|███       | 305/1000 [05:17<08:15,  1.40trial/s, best loss: -0.969868006129452]\r",
      " 31%|███       | 307/1000 [05:18<07:31,  1.54trial/s, best loss: -0.969868006129452]\r",
      " 31%|███       | 308/1000 [05:19<08:19,  1.39trial/s, best loss: -0.969868006129452]\r",
      " 31%|███       | 309/1000 [05:21<11:35,  1.01s/trial, best loss: -0.969868006129452]\r",
      " 31%|███       | 310/1000 [05:22<11:39,  1.01s/trial, best loss: -0.969868006129452]\r",
      " 31%|███       | 311/1000 [05:24<11:43,  1.02s/trial, best loss: -0.969868006129452]\r",
      " 31%|███▏      | 313/1000 [05:25<09:18,  1.23trial/s, best loss: -0.969868006129452]\r",
      " 31%|███▏      | 314/1000 [05:26<09:52,  1.16trial/s, best loss: -0.969868006129452]\r",
      " 32%|███▏      | 315/1000 [05:30<19:05,  1.67s/trial, best loss: -0.969868006129452]\r",
      " 32%|███▏      | 316/1000 [05:31<17:11,  1.51s/trial, best loss: -0.969868006129452]\r",
      " 32%|███▏      | 318/1000 [05:33<14:52,  1.31s/trial, best loss: -0.969868006129452]\r",
      " 32%|███▏      | 321/1000 [05:34<09:37,  1.18trial/s, best loss: -0.969868006129452]\r",
      " 32%|███▏      | 322/1000 [05:35<10:00,  1.13trial/s, best loss: -0.969868006129452]\r",
      " 32%|███▏      | 324/1000 [05:36<08:32,  1.32trial/s, best loss: -0.969868006129452]\r",
      " 32%|███▎      | 325/1000 [05:37<09:09,  1.23trial/s, best loss: -0.969868006129452]\r",
      " 33%|███▎      | 326/1000 [05:38<09:41,  1.16trial/s, best loss: -0.969868006129452]\r",
      " 33%|███▎      | 327/1000 [05:39<10:08,  1.11trial/s, best loss: -0.969868006129452]\r",
      " 33%|███▎      | 328/1000 [05:40<10:30,  1.07trial/s, best loss: -0.969868006129452]\r",
      " 33%|███▎      | 329/1000 [05:41<10:47,  1.04trial/s, best loss: -0.969868006129452]\r",
      " 33%|███▎      | 330/1000 [05:42<11:00,  1.01trial/s, best loss: -0.969868006129452]\r",
      " 33%|███▎      | 331/1000 [05:43<11:09,  1.00s/trial, best loss: -0.969868006129452]\r",
      " 33%|███▎      | 332/1000 [05:45<14:34,  1.31s/trial, best loss: -0.969868006129452]\r",
      " 33%|███▎      | 333/1000 [05:46<13:40,  1.23s/trial, best loss: -0.969868006129452]\r",
      " 33%|███▎      | 334/1000 [05:47<13:01,  1.17s/trial, best loss: -0.969868006129452]\r",
      " 34%|███▎      | 335/1000 [05:48<12:33,  1.13s/trial, best loss: -0.969868006129452]\r",
      " 34%|███▎      | 336/1000 [05:49<12:14,  1.11s/trial, best loss: -0.969868006129452]\r",
      " 34%|███▎      | 337/1000 [05:51<12:00,  1.09s/trial, best loss: -0.969868006129452]\r",
      " 34%|███▍      | 339/1000 [05:52<09:05,  1.21trial/s, best loss: -0.969868006129452]\r",
      " 34%|███▍      | 340/1000 [05:54<12:28,  1.13s/trial, best loss: -0.969868006129452]\r",
      " 34%|███▍      | 341/1000 [05:55<12:10,  1.11s/trial, best loss: -0.969868006129452]\r",
      " 34%|███▍      | 342/1000 [05:56<11:57,  1.09s/trial, best loss: -0.969868006129452]\r",
      " 34%|███▍      | 344/1000 [05:57<09:11,  1.19trial/s, best loss: -0.969868006129452]\r",
      " 35%|███▍      | 346/1000 [05:59<09:59,  1.09trial/s, best loss: -0.969868006129452]\r",
      " 35%|███▍      | 347/1000 [06:01<12:41,  1.17s/trial, best loss: -0.969868006129452]\r",
      " 35%|███▍      | 349/1000 [06:02<09:58,  1.09trial/s, best loss: -0.969868006129452]\r",
      " 35%|███▌      | 350/1000 [06:03<10:14,  1.06trial/s, best loss: -0.969868006129452]\r",
      " 35%|███▌      | 351/1000 [06:05<13:06,  1.21s/trial, best loss: -0.969868006129452]\r",
      " 35%|███▌      | 352/1000 [06:06<12:36,  1.17s/trial, best loss: -0.969868006129452]\r",
      " 35%|███▌      | 354/1000 [06:07<09:39,  1.11trial/s, best loss: -0.969868006129452]\r",
      " 36%|███▌      | 355/1000 [06:08<09:59,  1.08trial/s, best loss: -0.969868006129452]\r",
      " 36%|███▌      | 357/1000 [06:10<10:25,  1.03trial/s, best loss: -0.969868006129452]\r",
      " 36%|███▌      | 358/1000 [06:11<10:34,  1.01trial/s, best loss: -0.969868006129452]\r",
      " 36%|███▌      | 359/1000 [06:13<13:20,  1.25s/trial, best loss: -0.969868006129452]\r",
      " 36%|███▌      | 360/1000 [06:14<12:45,  1.20s/trial, best loss: -0.969868006129452]\r",
      " 36%|███▌      | 361/1000 [06:15<12:19,  1.16s/trial, best loss: -0.969868006129452]\r",
      " 36%|███▌      | 362/1000 [06:18<14:59,  1.41s/trial, best loss: -0.969868006129452]\r",
      " 36%|███▋      | 363/1000 [06:19<13:51,  1.31s/trial, best loss: -0.969868006129452]\r",
      " 36%|███▋      | 365/1000 [06:20<10:06,  1.05trial/s, best loss: -0.969868006129452]\r",
      " 37%|███▋      | 367/1000 [06:21<08:17,  1.27trial/s, best loss: -0.969868006129452]\r",
      " 37%|███▋      | 368/1000 [06:23<11:15,  1.07s/trial, best loss: -0.969868006129452]\r",
      " 37%|███▋      | 369/1000 [06:24<11:10,  1.06s/trial, best loss: -0.969868006129452]\r",
      " 37%|███▋      | 370/1000 [06:25<11:05,  1.06s/trial, best loss: -0.969868006129452]\r",
      " 37%|███▋      | 371/1000 [06:26<11:02,  1.05s/trial, best loss: -0.969868006129452]\r",
      " 37%|███▋      | 372/1000 [06:27<10:59,  1.05s/trial, best loss: -0.969868006129452]\r",
      " 37%|███▋      | 373/1000 [06:28<10:56,  1.05s/trial, best loss: -0.969868006129452]\r",
      " 37%|███▋      | 374/1000 [06:29<10:54,  1.05s/trial, best loss: -0.969868006129452]\r",
      " 38%|███▊      | 376/1000 [06:30<08:24,  1.24trial/s, best loss: -0.969868006129452]\r",
      " 38%|███▊      | 377/1000 [06:32<11:37,  1.12s/trial, best loss: -0.969868006129452]\r",
      " 38%|███▊      | 378/1000 [06:33<11:23,  1.10s/trial, best loss: -0.969868006129452]\r",
      " 38%|███▊      | 380/1000 [06:34<08:49,  1.17trial/s, best loss: -0.969868006129452]\r",
      " 38%|███▊      | 381/1000 [06:35<09:15,  1.12trial/s, best loss: -0.969868006129452]\r",
      " 38%|███▊      | 382/1000 [06:36<09:36,  1.07trial/s, best loss: -0.969868006129452]\r",
      " 38%|███▊      | 383/1000 [06:37<09:52,  1.04trial/s, best loss: -0.969868006129452]\r",
      " 38%|███▊      | 385/1000 [06:39<10:11,  1.01trial/s, best loss: -0.969868006129452]\r",
      " 39%|███▊      | 386/1000 [06:41<12:46,  1.25s/trial, best loss: -0.969868006129452]\r",
      " 39%|███▊      | 387/1000 [06:43<12:14,  1.20s/trial, best loss: -0.969868006129452]\r",
      " 39%|███▉      | 388/1000 [06:44<11:47,  1.16s/trial, best loss: -0.969868006129452]\r",
      " 39%|███▉      | 389/1000 [06:45<11:27,  1.12s/trial, best loss: -0.969868006129452]\r",
      " 39%|███▉      | 390/1000 [06:46<11:13,  1.10s/trial, best loss: -0.969868006129452]\r",
      " 39%|███▉      | 391/1000 [06:47<11:01,  1.09s/trial, best loss: -0.\n",
      "\n",
      "*** WARNING: max output size exceeded, skipping output. ***\n",
      "\n",
      "███    | 608/1000 [10:34<06:21,  1.03trial/s, best loss: -0.969868006129452]\r",
      " 61%|██████    | 609/1000 [10:35<06:29,  1.00trial/s, best loss: -0.969868006129452]\r",
      " 61%|██████    | 610/1000 [10:36<06:34,  1.01s/trial, best loss: -0.969868006129452]\r",
      " 61%|██████    | 611/1000 [10:37<06:39,  1.03s/trial, best loss: -0.969868006129452]\r",
      " 61%|██████    | 612/1000 [10:38<06:41,  1.04s/trial, best loss: -0.969868006129452]\r",
      " 61%|██████▏   | 613/1000 [10:40<08:42,  1.35s/trial, best loss: -0.969868006129452]\r",
      " 61%|██████▏   | 614/1000 [10:41<08:09,  1.27s/trial, best loss: -0.969868006129452]\r",
      " 62%|██████▏   | 615/1000 [10:42<07:44,  1.21s/trial, best loss: -0.969868006129452]\r",
      " 62%|██████▏   | 616/1000 [10:43<07:26,  1.16s/trial, best loss: -0.969868006129452]\r",
      " 62%|██████▏   | 618/1000 [10:44<05:33,  1.15trial/s, best loss: -0.969868006129452]\r",
      " 62%|██████▏   | 619/1000 [10:47<09:08,  1.44s/trial, best loss: -0.969868006129452]\r",
      " 62%|██████▏   | 621/1000 [10:49<06:42,  1.06s/trial, best loss: -0.969868006129452]\r",
      " 62%|██████▏   | 622/1000 [10:50<06:41,  1.06s/trial, best loss: -0.969868006129452]\r",
      " 62%|██████▏   | 624/1000 [10:51<05:19,  1.18trial/s, best loss: -0.969868006129452]\r",
      " 62%|██████▎   | 625/1000 [10:52<05:36,  1.12trial/s, best loss: -0.969868006129452]\r",
      " 63%|██████▎   | 626/1000 [10:54<07:23,  1.19s/trial, best loss: -0.969868006129452]\r",
      " 63%|██████▎   | 627/1000 [10:55<07:17,  1.17s/trial, best loss: -0.9699429785711257]\r",
      " 63%|██████▎   | 629/1000 [10:57<06:57,  1.13s/trial, best loss: -0.9699429785711257]\r",
      " 63%|██████▎   | 630/1000 [10:59<08:21,  1.35s/trial, best loss: -0.9699429785711257]\r",
      " 63%|██████▎   | 632/1000 [11:00<06:17,  1.03s/trial, best loss: -0.9699429785711257]\r",
      " 63%|██████▎   | 633/1000 [11:01<06:19,  1.03s/trial, best loss: -0.9699429785711257]\r",
      " 64%|██████▎   | 635/1000 [11:03<06:21,  1.05s/trial, best loss: -0.9699429785711257]\r",
      " 64%|██████▎   | 636/1000 [11:05<06:21,  1.05s/trial, best loss: -0.9699429785711257]\r",
      " 64%|██████▎   | 637/1000 [11:07<07:52,  1.30s/trial, best loss: -0.9699429785711257]\r",
      " 64%|██████▍   | 638/1000 [11:08<07:28,  1.24s/trial, best loss: -0.9699429785711257]\r",
      " 64%|██████▍   | 641/1000 [11:10<05:45,  1.04trial/s, best loss: -0.9699429785711257]\r",
      " 64%|██████▍   | 642/1000 [11:11<05:51,  1.02trial/s, best loss: -0.9699429785711257]\r",
      " 64%|██████▍   | 644/1000 [11:12<04:51,  1.22trial/s, best loss: -0.9699429785711257]\r",
      " 64%|██████▍   | 645/1000 [11:13<05:08,  1.15trial/s, best loss: -0.9699429785711257]\r",
      " 65%|██████▍   | 646/1000 [11:14<05:22,  1.10trial/s, best loss: -0.9699429785711257]\r",
      " 65%|██████▍   | 647/1000 [11:16<07:06,  1.21s/trial, best loss: -0.9699429785711257]\r",
      " 65%|██████▍   | 649/1000 [11:18<06:42,  1.15s/trial, best loss: -0.9699429785711257]\r",
      " 65%|██████▌   | 650/1000 [11:19<06:34,  1.13s/trial, best loss: -0.9699429785711257]\r",
      " 65%|██████▌   | 651/1000 [11:20<06:27,  1.11s/trial, best loss: -0.9699429785711257]\r",
      " 65%|██████▌   | 652/1000 [11:21<06:21,  1.10s/trial, best loss: -0.9699429785711257]\r",
      " 65%|██████▌   | 654/1000 [11:23<04:56,  1.17trial/s, best loss: -0.9699429785711257]\r",
      " 66%|██████▌   | 656/1000 [11:25<05:20,  1.07trial/s, best loss: -0.9699429785711257]\r",
      " 66%|██████▌   | 657/1000 [11:26<05:29,  1.04trial/s, best loss: -0.9699429785711257]\r",
      " 66%|██████▌   | 658/1000 [11:27<05:36,  1.02trial/s, best loss: -0.9699429785711257]\r",
      " 66%|██████▌   | 659/1000 [11:28<05:41,  1.00s/trial, best loss: -0.9699429785711257]\r",
      " 66%|██████▌   | 660/1000 [11:30<07:20,  1.29s/trial, best loss: -0.9699429785711257]\r",
      " 66%|██████▌   | 661/1000 [11:31<06:57,  1.23s/trial, best loss: -0.9699429785711257]\r",
      " 66%|██████▌   | 662/1000 [11:32<06:40,  1.18s/trial, best loss: -0.9699429785711257]\r",
      " 66%|██████▋   | 663/1000 [11:34<08:09,  1.45s/trial, best loss: -0.9699429785711257]\r",
      " 66%|██████▋   | 664/1000 [11:35<07:29,  1.34s/trial, best loss: -0.9699429785711257]\r",
      " 66%|██████▋   | 665/1000 [11:36<07:01,  1.26s/trial, best loss: -0.9699429785711257]\r",
      " 67%|██████▋   | 668/1000 [11:37<04:10,  1.32trial/s, best loss: -0.9699429785711257]\r",
      " 67%|██████▋   | 669/1000 [11:39<05:44,  1.04s/trial, best loss: -0.9699429785711257]\r",
      " 67%|██████▋   | 671/1000 [11:41<04:39,  1.18trial/s, best loss: -0.9699429785711257]\r",
      " 67%|██████▋   | 672/1000 [11:42<04:53,  1.12trial/s, best loss: -0.9699429785711257]\r",
      " 68%|██████▊   | 675/1000 [11:45<05:16,  1.03trial/s, best loss: -0.9699429785711257]\r",
      " 68%|██████▊   | 677/1000 [11:47<05:23,  1.00s/trial, best loss: -0.9699429785711257]\r",
      " 68%|██████▊   | 678/1000 [11:48<05:25,  1.01s/trial, best loss: -0.9699429785711257]\r",
      " 68%|██████▊   | 679/1000 [11:49<05:28,  1.02s/trial, best loss: -0.9699429785711257]\r",
      " 68%|██████▊   | 680/1000 [11:52<08:04,  1.52s/trial, best loss: -0.9699429785711257]\r",
      " 68%|██████▊   | 683/1000 [11:53<04:59,  1.06trial/s, best loss: -0.9699429785711257]\r",
      " 68%|██████▊   | 684/1000 [11:55<06:09,  1.17s/trial, best loss: -0.9699429785711257]\r",
      " 68%|██████▊   | 685/1000 [11:57<06:01,  1.15s/trial, best loss: -0.9699429785711257]\r",
      " 69%|██████▊   | 687/1000 [11:58<04:45,  1.10trial/s, best loss: -0.9699429785711257]\r",
      " 69%|██████▉   | 689/1000 [12:00<05:00,  1.04trial/s, best loss: -0.9699429785711257]\r",
      " 69%|██████▉   | 690/1000 [12:01<05:05,  1.02trial/s, best loss: -0.9699429785711257]\r",
      " 69%|██████▉   | 691/1000 [12:02<05:09,  1.00s/trial, best loss: -0.9699429785711257]\r",
      " 69%|██████▉   | 692/1000 [12:04<06:32,  1.27s/trial, best loss: -0.9699429785711257]\r",
      " 69%|██████▉   | 694/1000 [12:05<04:56,  1.03trial/s, best loss: -0.9699429785711257]\r",
      " 70%|██████▉   | 695/1000 [12:07<06:15,  1.23s/trial, best loss: -0.9699429785711257]\r",
      " 70%|██████▉   | 697/1000 [12:10<06:56,  1.37s/trial, best loss: -0.9699429785711257]\r",
      " 70%|███████   | 700/1000 [12:11<04:32,  1.10trial/s, best loss: -0.9699429785711257]\r",
      " 70%|███████   | 702/1000 [12:15<05:34,  1.12s/trial, best loss: -0.9699429785711257]\r",
      " 70%|███████   | 704/1000 [12:16<04:39,  1.06trial/s, best loss: -0.9699429785711257]\r",
      " 71%|███████   | 706/1000 [12:18<04:48,  1.02trial/s, best loss: -0.9699429785711257]\r",
      " 71%|███████   | 707/1000 [12:19<04:52,  1.00trial/s, best loss: -0.9699429785711257]\r",
      " 71%|███████   | 708/1000 [12:21<05:58,  1.23s/trial, best loss: -0.9699429785711257]\r",
      " 71%|███████   | 709/1000 [12:22<05:46,  1.19s/trial, best loss: -0.9699429785711257]\r",
      " 71%|███████   | 711/1000 [12:23<04:29,  1.07trial/s, best loss: -0.9699429785711257]\r",
      " 71%|███████   | 712/1000 [12:24<04:37,  1.04trial/s, best loss: -0.9699429785711257]\r",
      " 71%|███████▏  | 713/1000 [12:26<05:56,  1.24s/trial, best loss: -0.9699429785711257]\r",
      " 71%|███████▏  | 714/1000 [12:27<05:42,  1.20s/trial, best loss: -0.9699429785711257]\r",
      " 72%|███████▏  | 716/1000 [12:30<05:23,  1.14s/trial, best loss: -0.9699429785711257]\r",
      " 72%|███████▏  | 717/1000 [12:31<05:17,  1.12s/trial, best loss: -0.9699429785711257]\r",
      " 72%|███████▏  | 718/1000 [12:32<05:12,  1.11s/trial, best loss: -0.9699429785711257]\r",
      " 72%|███████▏  | 719/1000 [12:33<05:08,  1.10s/trial, best loss: -0.9699429785711257]\r",
      " 72%|███████▏  | 721/1000 [12:34<03:58,  1.17trial/s, best loss: -0.9699429785711257]\r",
      " 72%|███████▏  | 722/1000 [12:36<05:20,  1.15s/trial, best loss: -0.9699429785711257]\r",
      " 72%|███████▏  | 724/1000 [12:37<04:09,  1.11trial/s, best loss: -0.9699429785711257]\r",
      " 73%|███████▎  | 726/1000 [12:39<04:23,  1.04trial/s, best loss: -0.9699429785711257]\r",
      " 73%|███████▎  | 727/1000 [12:40<04:28,  1.02trial/s, best loss: -0.9699429785711257]\r",
      " 73%|███████▎  | 728/1000 [12:42<05:38,  1.24s/trial, best loss: -0.9699429785711257]\r",
      " 73%|███████▎  | 729/1000 [12:43<05:25,  1.20s/trial, best loss: -0.9699429785711257]\r",
      " 73%|███████▎  | 730/1000 [12:44<05:14,  1.16s/trial, best loss: -0.9699429785711257]\r",
      " 73%|███████▎  | 731/1000 [12:46<05:06,  1.14s/trial, best loss: -0.9699429785711257]\r",
      " 73%|███████▎  | 732/1000 [12:47<04:59,  1.12s/trial, best loss: -0.9699429785711257]\r",
      " 73%|███████▎  | 733/1000 [12:48<04:54,  1.10s/trial, best loss: -0.9699429785711257]\r",
      " 73%|███████▎  | 734/1000 [12:49<04:50,  1.09s/trial, best loss: -0.9699429785711257]\r",
      " 74%|███████▎  | 736/1000 [12:50<03:41,  1.19trial/s, best loss: -0.9699429785711257]\r",
      " 74%|███████▍  | 738/1000 [12:53<04:57,  1.14s/trial, best loss: -0.9699429785711257]\r",
      " 74%|███████▍  | 739/1000 [12:54<04:52,  1.12s/trial, best loss: -0.9699429785711257]\r",
      " 74%|███████▍  | 740/1000 [12:55<04:47,  1.11s/trial, best loss: -0.9699429785711257]\r",
      " 74%|███████▍  | 742/1000 [12:56<03:45,  1.14trial/s, best loss: -0.9699429785711257]\r",
      " 74%|███████▍  | 743/1000 [12:57<03:55,  1.09trial/s, best loss: -0.9699429785711257]\r",
      " 74%|███████▍  | 744/1000 [12:58<04:03,  1.05trial/s, best loss: -0.9699429785711257]\r",
      " 74%|███████▍  | 745/1000 [13:01<06:30,  1.53s/trial, best loss: -0.9699429785711257]\r",
      " 75%|███████▍  | 747/1000 [13:03<04:40,  1.11s/trial, best loss: -0.9699429785711257]\r",
      " 75%|███████▍  | 748/1000 [13:04<04:37,  1.10s/trial, best loss: -0.9699429785711257]\r",
      " 75%|███████▍  | 749/1000 [13:05<04:34,  1.09s/trial, best loss: -0.9699429785711257]\r",
      " 75%|███████▌  | 750/1000 [13:06<04:31,  1.08s/trial, best loss: -0.9699429785711257]\r",
      " 75%|███████▌  | 751/1000 [13:07<04:28,  1.08s/trial, best loss: -0.9699429785711257]\r",
      " 75%|███████▌  | 753/1000 [13:08<03:27,  1.19trial/s, best loss: -0.9699429785711257]\r",
      " 76%|███████▌  | 755/1000 [13:10<03:47,  1.08trial/s, best loss: -0.9699429785711257]\r",
      " 76%|███████▌  | 757/1000 [13:12<03:57,  1.02trial/s, best loss: -0.9699429785711257]\r",
      " 76%|███████▌  | 758/1000 [13:13<04:00,  1.01trial/s, best loss: -0.9699429785711257]\r",
      " 76%|███████▌  | 759/1000 [13:15<05:01,  1.25s/trial, best loss: -0.9699429785711257]\r",
      " 76%|███████▌  | 760/1000 [13:16<04:49,  1.21s/trial, best loss: -0.9699429785711257]\r",
      " 76%|███████▌  | 761/1000 [13:17<04:39,  1.17s/trial, best loss: -0.9699429785711257]\r",
      " 76%|███████▌  | 762/1000 [13:19<04:32,  1.14s/trial, best loss: -0.9699429785711257]\r",
      " 76%|███████▋  | 763/1000 [13:20<04:25,  1.12s/trial, best loss: -0.9699429785711257]\r",
      " 76%|███████▋  | 765/1000 [13:22<04:17,  1.10s/trial, best loss: -0.9699429785711257]\r",
      " 77%|███████▋  | 767/1000 [13:24<04:12,  1.09s/trial, best loss: -0.9699429785711257]\r",
      " 77%|███████▋  | 768/1000 [13:25<04:10,  1.08s/trial, best loss: -0.9699429785711257]\r",
      " 77%|███████▋  | 769/1000 [13:26<04:09,  1.08s/trial, best loss: -0.9699429785711257]\r",
      " 77%|███████▋  | 770/1000 [13:27<04:07,  1.08s/trial, best loss: -0.9699429785711257]\r",
      " 77%|███████▋  | 772/1000 [13:28<03:13,  1.18trial/s, best loss: -0.9699429785711257]\r",
      " 77%|███████▋  | 773/1000 [13:29<03:24,  1.11trial/s, best loss: -0.9699429785711257]\r",
      " 77%|███████▋  | 774/1000 [13:30<03:32,  1.06trial/s, best loss: -0.9699429785711257]\r",
      " 78%|███████▊  | 775/1000 [13:32<04:42,  1.25s/trial, best loss: -0.9699429785711257]\r",
      " 78%|███████▊  | 776/1000 [13:34<04:29,  1.20s/trial, best loss: -0.9699429785711257]\r",
      " 78%|███████▊  | 778/1000 [13:35<03:22,  1.10trial/s, best loss: -0.9699429785711257]\r",
      " 78%|███████▊  | 779/1000 [13:38<05:21,  1.46s/trial, best loss: -0.9699429785711257]\r",
      " 78%|███████▊  | 781/1000 [13:39<03:57,  1.08s/trial, best loss: -0.9699429785711257]\r",
      " 78%|███████▊  | 783/1000 [13:40<03:11,  1.13trial/s, best loss: -0.9699429785711257]\r",
      " 78%|███████▊  | 784/1000 [13:41<03:18,  1.09trial/s, best loss: -0.9699429785711257]\r",
      " 78%|███████▊  | 785/1000 [13:43<04:17,  1.20s/trial, best loss: -0.9699429785711257]\r",
      " 79%|███████▊  | 787/1000 [13:44<03:23,  1.05trial/s, best loss: -0.9700942778071894]\r",
      " 79%|███████▉  | 789/1000 [13:46<03:29,  1.01trial/s, best loss: -0.9700942778071894]\r",
      " 79%|███████▉  | 790/1000 [13:49<04:18,  1.23s/trial, best loss: -0.9700942778071894]\r",
      " 79%|███████▉  | 791/1000 [13:50<04:09,  1.20s/trial, best loss: -0.9700942778071894]\r",
      " 79%|███████▉  | 792/1000 [13:51<04:02,  1.17s/trial, best loss: -0.9700942778071894]\r",
      " 79%|███████▉  | 794/1000 [13:52<03:06,  1.10trial/s, best loss: -0.9700942778071894]\r",
      " 80%|███████▉  | 795/1000 [13:53<03:13,  1.06trial/s, best loss: -0.9700942778071894]\r",
      " 80%|███████▉  | 796/1000 [13:54<03:18,  1.03trial/s, best loss: -0.9700942778071894]\r",
      " 80%|███████▉  | 797/1000 [13:55<03:22,  1.00trial/s, best loss: -0.9700942778071894]\r",
      " 80%|███████▉  | 798/1000 [13:57<04:23,  1.31s/trial, best loss: -0.9700942778071894]\r",
      " 80%|███████▉  | 799/1000 [13:58<04:09,  1.24s/trial, best loss: -0.9700942778071894]\r",
      " 80%|████████  | 801/1000 [13:59<03:04,  1.08trial/s, best loss: -0.9700942778071894]\r",
      " 80%|████████  | 803/1000 [14:01<03:13,  1.02trial/s, best loss: -0.9700942778071894]\r",
      " 80%|████████  | 804/1000 [14:02<03:16,  1.00s/trial, best loss: -0.9700942778071894]\r",
      " 80%|████████  | 805/1000 [14:06<04:57,  1.52s/trial, best loss: -0.9700942778071894]\r",
      " 81%|████████  | 808/1000 [14:07<02:59,  1.07trial/s, best loss: -0.9700942778071894]\r",
      " 81%|████████  | 810/1000 [14:09<03:06,  1.02trial/s, best loss: -0.9700942778071894]\r",
      " 81%|████████  | 812/1000 [14:12<03:41,  1.18s/trial, best loss: -0.9700942778071894]\r",
      " 81%|████████▏ | 814/1000 [14:13<03:01,  1.02trial/s, best loss: -0.9700942778071894]\r",
      " 82%|████████▏ | 815/1000 [14:14<03:03,  1.01trial/s, best loss: -0.9700942778071894]\r",
      " 82%|████████▏ | 817/1000 [14:16<03:06,  1.02s/trial, best loss: -0.9700942778071894]\r",
      " 82%|████████▏ | 818/1000 [14:17<03:07,  1.03s/trial, best loss: -0.9700942778071894]\r",
      " 82%|████████▏ | 819/1000 [14:19<03:07,  1.04s/trial, best loss: -0.9700942778071894]\r",
      " 82%|████████▏ | 820/1000 [14:21<03:54,  1.30s/trial, best loss: -0.9700942778071894]\r",
      " 82%|████████▏ | 821/1000 [14:22<03:42,  1.24s/trial, best loss: -0.9700942778071894]\r",
      " 82%|████████▏ | 822/1000 [14:23<03:33,  1.20s/trial, best loss: -0.9700942778071894]\r",
      " 82%|████████▏ | 823/1000 [14:24<03:25,  1.16s/trial, best loss: -0.9700942778071894]\r",
      " 82%|████████▎ | 825/1000 [14:25<02:35,  1.13trial/s, best loss: -0.9700942778071894]\r",
      " 83%|████████▎ | 827/1000 [14:27<02:45,  1.04trial/s, best loss: -0.9700942778071894]\r",
      " 83%|████████▎ | 828/1000 [14:30<04:08,  1.44s/trial, best loss: -0.9700942778071894]\r",
      " 83%|████████▎ | 830/1000 [14:31<03:06,  1.10s/trial, best loss: -0.9700942778071894]\r",
      " 83%|████████▎ | 831/1000 [14:32<03:04,  1.09s/trial, best loss: -0.9700942778071894]\r",
      " 83%|████████▎ | 833/1000 [14:34<02:27,  1.14trial/s, best loss: -0.9700942778071894]\r",
      " 83%|████████▎ | 834/1000 [14:36<03:11,  1.15s/trial, best loss: -0.9700942778071894]\r",
      " 84%|████████▎ | 835/1000 [14:37<03:06,  1.13s/trial, best loss: -0.9700942778071894]\r",
      " 84%|████████▎ | 836/1000 [14:38<03:03,  1.12s/trial, best loss: -0.9700942778071894]\r",
      " 84%|████████▎ | 837/1000 [14:39<03:00,  1.10s/trial, best loss: -0.9700942778071894]\r",
      " 84%|████████▍ | 840/1000 [14:41<02:23,  1.11trial/s, best loss: -0.9700942778071894]\r",
      " 84%|████████▍ | 841/1000 [14:43<03:02,  1.15s/trial, best loss: -0.9700942778071894]\r",
      " 84%|████████▍ | 843/1000 [14:44<02:25,  1.08trial/s, best loss: -0.9700942778071894]\r",
      " 84%|████████▍ | 844/1000 [14:45<02:29,  1.05trial/s, best loss: -0.9700942778071894]\r",
      " 84%|████████▍ | 845/1000 [14:46<02:32,  1.02trial/s, best loss: -0.9700942778071894]\r",
      " 85%|████████▍ | 846/1000 [14:47<02:34,  1.01s/trial, best loss: -0.9700942778071894]\r",
      " 85%|████████▍ | 848/1000 [14:49<02:03,  1.23trial/s, best loss: -0.9700942778071894]\r",
      " 85%|████████▌ | 851/1000 [14:53<02:44,  1.10s/trial, best loss: -0.9700942778071894]\r",
      " 85%|████████▌ | 852/1000 [14:55<03:17,  1.34s/trial, best loss: -0.9700942778071894]\r",
      " 85%|████████▌ | 853/1000 [14:56<03:08,  1.28s/trial, best loss: -0.9700942778071894]\r",
      " 86%|████████▌ | 855/1000 [14:57<02:25,  1.00s/trial, best loss: -0.9700942778071894]\r",
      " 86%|████████▌ | 856/1000 [14:58<02:26,  1.02s/trial, best loss: -0.9700942778071894]\r",
      " 86%|████████▌ | 857/1000 [15:01<03:03,  1.28s/trial, best loss: -0.9700942778071894]\r",
      " 86%|████████▌ | 858/1000 [15:03<03:32,  1.50s/trial, best loss: -0.9700942778071894]\r",
      " 86%|████████▌ | 859/1000 [15:04<03:15,  1.39s/trial, best loss: -0.9700942778071894]\r",
      " 86%|████████▌ | 861/1000 [15:05<02:22,  1.03s/trial, best loss: -0.9700942778071894]\r",
      " 86%|████████▋ | 863/1000 [15:06<01:55,  1.19trial/s, best loss: -0.9700942778071894]\r",
      " 86%|████████▋ | 864/1000 [15:07<02:01,  1.12trial/s, best loss: -0.9700942778071894]\r",
      " 86%|████████▋ | 865/1000 [15:09<02:39,  1.18s/trial, best loss: -0.9700942778071894]\r",
      " 87%|████████▋ | 867/1000 [15:10<02:03,  1.08trial/s, best loss: -0.9700942778071894]\r",
      " 87%|████████▋ | 868/1000 [15:11<02:06,  1.04trial/s, best loss: -0.9700942778071894]\r",
      " 87%|████████▋ | 869/1000 [15:12<02:09,  1.01trial/s, best loss: -0.9700942778071894]\r",
      " 87%|████████▋ | 870/1000 [15:13<02:11,  1.01s/trial, best loss: -0.9700942778071894]\r",
      " 87%|████████▋ | 871/1000 [15:15<02:12,  1.03s/trial, best loss: -0.9700942778071894]\r",
      " 87%|████████▋ | 872/1000 [15:16<02:13,  1.04s/trial, best loss: -0.9700942778071894]\r",
      " 87%|████████▋ | 874/1000 [15:18<02:13,  1.06s/trial, best loss: -0.9700942778071894]\r",
      " 88%|████████▊ | 876/1000 [15:20<02:11,  1.06s/trial, best loss: -0.9700942778071894]\r",
      " 88%|████████▊ | 877/1000 [15:21<02:11,  1.07s/trial, best loss: -0.9700942778071894]\r",
      " 88%|████████▊ | 878/1000 [15:22<02:10,  1.07s/trial, best loss: -0.9700942778071894]\r",
      " 88%|████████▊ | 879/1000 [15:23<02:09,  1.07s/trial, best loss: -0.9700942778071894]\r",
      " 88%|████████▊ | 881/1000 [15:25<02:07,  1.07s/trial, best loss: -0.9700942778071894]\r",
      " 88%|████████▊ | 882/1000 [15:26<02:06,  1.07s/trial, best loss: -0.9700942778071894]\r",
      " 88%|████████▊ | 883/1000 [15:27<02:05,  1.07s/trial, best loss: -0.9700942778071894]\r",
      " 88%|████████▊ | 884/1000 [15:30<02:36,  1.35s/trial, best loss: -0.9700942778071894]\r",
      " 88%|████████▊ | 885/1000 [15:31<02:27,  1.28s/trial, best loss: -0.9700942778071894]\r",
      " 89%|████████▉ | 888/1000 [15:33<01:49,  1.02trial/s, best loss: -0.9700942778071894]\r",
      " 89%|████████▉ | 889/1000 [15:35<02:15,  1.22s/trial, best loss: -0.9700942778071894]\r",
      " 89%|████████▉ | 890/1000 [15:36<02:11,  1.19s/trial, best loss: -0.9700942778071894]\r",
      " 89%|████████▉ | 891/1000 [15:37<02:06,  1.16s/trial, best loss: -0.9700942778071894]\r",
      " 89%|████████▉ | 892/1000 [15:38<02:03,  1.14s/trial, best loss: -0.9700942778071894]\r",
      " 89%|████████▉ | 893/1000 [15:39<02:00,  1.13s/trial, best loss: -0.9700942778071894]\r",
      " 90%|████████▉ | 895/1000 [15:40<01:31,  1.15trial/s, best loss: -0.9700942778071894]\r",
      " 90%|████████▉ | 897/1000 [15:43<01:37,  1.05trial/s, best loss: -0.9700942778071894]\r",
      " 90%|████████▉ | 898/1000 [15:44<01:39,  1.02trial/s, best loss: -0.9700942778071894]\r",
      " 90%|████████▉ | 899/1000 [15:45<01:40,  1.00trial/s, best loss: -0.9700942778071894]\r",
      " 90%|█████████ | 900/1000 [15:47<02:09,  1.29s/trial, best loss: -0.9700942778071894]\r",
      " 90%|█████████ | 901/1000 [15:48<02:02,  1.24s/trial, best loss: -0.9700942778071894]\r",
      " 90%|█████████ | 902/1000 [15:49<01:56,  1.19s/trial, best loss: -0.9700942778071894]\r",
      " 90%|█████████ | 903/1000 [15:50<01:52,  1.16s/trial, best loss: -0.9700942778071894]\r",
      " 90%|█████████ | 905/1000 [15:51<01:23,  1.13trial/s, best loss: -0.9700942778071894]\r",
      " 91%|█████████ | 906/1000 [15:53<01:51,  1.19s/trial, best loss: -0.9700942778071894]\r",
      " 91%|█████████ | 908/1000 [15:54<01:24,  1.08trial/s, best loss: -0.9700942778071894]\r",
      " 91%|█████████ | 909/1000 [15:55<01:27,  1.04trial/s, best loss: -0.9700942778071894]\r",
      " 91%|█████████ | 911/1000 [15:58<01:29,  1.01s/trial, best loss: -0.9700942778071894]\r",
      " 91%|█████████ | 912/1000 [15:59<01:29,  1.02s/trial, best loss: -0.9700942778071894]\r",
      " 91%|█████████▏| 914/1000 [16:02<01:47,  1.25s/trial, best loss: -0.9700942778071894]\r",
      " 92%|█████████▏| 915/1000 [16:03<01:43,  1.21s/trial, best loss: -0.9700942778071894]\r",
      " 92%|█████████▏| 917/1000 [16:04<01:19,  1.05trial/s, best loss: -0.9700942778071894]\r",
      " 92%|█████████▏| 918/1000 [16:05<01:20,  1.02trial/s, best loss: -0.9700942778071894]\r",
      " 92%|█████████▏| 919/1000 [16:06<01:21,  1.01s/trial, best loss: -0.9700942778071894]\r",
      " 92%|█████████▏| 920/1000 [16:07<01:21,  1.02s/trial, best loss: -0.9700942778071894]\r",
      " 92%|█████████▏| 921/1000 [16:08<01:22,  1.04s/trial, best loss: -0.9700942778071894]\r",
      " 92%|█████████▏| 922/1000 [16:10<01:21,  1.05s/trial, best loss: -0.9700942778071894]\r",
      " 92%|█████████▏| 923/1000 [16:12<01:44,  1.36s/trial, best loss: -0.9700942778071894]\r",
      " 92%|█████████▏| 924/1000 [16:13<01:37,  1.28s/trial, best loss: -0.9700942778071894]\r",
      " 93%|█████████▎| 926/1000 [16:15<01:28,  1.19s/trial, best loss: -0.9700942778071894]\r",
      " 93%|█████████▎| 928/1000 [16:16<01:07,  1.07trial/s, best loss: -0.9700942778071894]\r",
      " 93%|█████████▎| 930/1000 [16:18<01:09,  1.01trial/s, best loss: -0.9700942778071894]\r",
      " 93%|█████████▎| 932/1000 [16:21<01:21,  1.20s/trial, best loss: -0.9700942778071894]\r",
      " 93%|█████████▎| 934/1000 [16:24<01:16,  1.16s/trial, best loss: -0.9700942778071894]\r",
      " 94%|█████████▎| 935/1000 [16:25<01:14,  1.14s/trial, best loss: -0.9700942778071894]\r",
      " 94%|█████████▎| 936/1000 [16:26<01:12,  1.13s/trial, best loss: -0.9700942778071894]\r",
      " 94%|█████████▎| 937/1000 [16:27<01:10,  1.12s/trial, best loss: -0.9700942778071894]\r",
      " 94%|█████████▍| 938/1000 [16:28<01:08,  1.11s/trial, best loss: -0.9700942778071894]\r",
      " 94%|█████████▍| 939/1000 [16:29<01:07,  1.10s/trial, best loss: -0.9700942778071894]\r",
      " 94%|█████████▍| 941/1000 [16:31<01:04,  1.09s/trial, best loss: -0.9700942778071894]\r",
      " 94%|█████████▍| 942/1000 [16:32<01:03,  1.09s/trial, best loss: -0.9700942778071894]\r",
      " 94%|█████████▍| 944/1000 [16:33<00:48,  1.15trial/s, best loss: -0.9700942778071894]\r",
      " 94%|█████████▍| 945/1000 [16:34<00:50,  1.09trial/s, best loss: -0.9700942778071894]\r",
      " 95%|█████████▍| 946/1000 [16:35<00:51,  1.05trial/s, best loss: -0.9700942778071894]\r",
      " 95%|█████████▍| 947/1000 [16:36<00:52,  1.01trial/s, best loss: -0.9700942778071894]\r",
      " 95%|█████████▍| 948/1000 [16:38<00:52,  1.01s/trial, best loss: -0.9700942778071894]\r",
      " 95%|█████████▌| 950/1000 [16:41<01:03,  1.28s/trial, best loss: -0.9700942778071894]\r",
      " 95%|█████████▌| 951/1000 [16:42<01:00,  1.23s/trial, best loss: -0.9700942778071894]\r",
      " 95%|█████████▌| 952/1000 [16:44<01:10,  1.47s/trial, best loss: -0.9700942778071894]\r",
      " 95%|█████████▌| 954/1000 [16:45<00:49,  1.08s/trial, best loss: -0.9700942778071894]\r",
      " 96%|█████████▌| 955/1000 [16:46<00:48,  1.08s/trial, best loss: -0.9700942778071894]\r",
      " 96%|█████████▌| 956/1000 [16:47<00:47,  1.08s/trial, best loss: -0.9700942778071894]\r",
      " 96%|█████████▌| 957/1000 [16:48<00:46,  1.08s/trial, best loss: -0.9700942778071894]\r",
      " 96%|█████████▌| 958/1000 [16:49<00:45,  1.08s/trial, best loss: -0.9700942778071894]\r",
      " 96%|█████████▌| 959/1000 [16:51<00:44,  1.08s/trial, best loss: -0.9700942778071894]\r",
      " 96%|█████████▌| 960/1000 [16:52<00:43,  1.08s/trial, best loss: -0.9700942778071894]\r",
      " 96%|█████████▌| 962/1000 [16:53<00:31,  1.19trial/s, best loss: -0.9700942778071894]\r",
      " 96%|█████████▋| 963/1000 [16:54<00:33,  1.12trial/s, best loss: -0.9700942778071894]\r",
      " 96%|█████████▋| 964/1000 [16:55<00:33,  1.06trial/s, best loss: -0.9700942778071894]\r",
      " 96%|█████████▋| 965/1000 [16:57<00:44,  1.28s/trial, best loss: -0.9700942778071894]\r",
      " 97%|█████████▋| 967/1000 [16:58<00:31,  1.05trial/s, best loss: -0.9700942778071894]\r",
      " 97%|█████████▋| 969/1000 [17:00<00:31,  1.00s/trial, best loss: -0.9700942778071894]\r",
      " 97%|█████████▋| 970/1000 [17:01<00:30,  1.02s/trial, best loss: -0.9700942778071894]\r",
      " 97%|█████████▋| 971/1000 [17:02<00:30,  1.03s/trial, best loss: -0.9700942778071894]\r",
      " 97%|█████████▋| 972/1000 [17:05<00:36,  1.32s/trial, best loss: -0.9700942778071894]\r",
      " 97%|█████████▋| 973/1000 [17:06<00:33,  1.26s/trial, best loss: -0.9700942778071894]\r",
      " 98%|█████████▊| 975/1000 [17:07<00:23,  1.05trial/s, best loss: -0.9700942778071894]\r",
      " 98%|█████████▊| 977/1000 [17:09<00:22,  1.00trial/s, best loss: -0.9700942778071894]\r",
      " 98%|█████████▊| 979/1000 [17:12<00:25,  1.22s/trial, best loss: -0.9700942778071894]\r",
      " 98%|█████████▊| 980/1000 [17:13<00:23,  1.19s/trial, best loss: -0.9700942778071894]\r",
      " 98%|█████████▊| 981/1000 [17:14<00:22,  1.17s/trial, best loss: -0.9700942778071894]\r",
      " 98%|█████████▊| 982/1000 [17:15<00:20,  1.15s/trial, best loss: -0.9700942778071894]\r",
      " 98%|█████████▊| 983/1000 [17:17<00:19,  1.13s/trial, best loss: -0.9700942778071894]\r",
      " 98%|█████████▊| 985/1000 [17:18<00:13,  1.13trial/s, best loss: -0.9700942778071894]\r",
      " 99%|█████████▊| 987/1000 [17:20<00:12,  1.04trial/s, best loss: -0.9700942778071894]\r",
      " 99%|█████████▉| 990/1000 [17:23<00:10,  1.01s/trial, best loss: -0.9700942778071894]\r",
      " 99%|█████████▉| 991/1000 [17:25<00:10,  1.22s/trial, best loss: -0.9700942778071894]\r",
      " 99%|█████████▉| 992/1000 [17:26<00:09,  1.19s/trial, best loss: -0.9700942778071894]\r",
      " 99%|█████████▉| 994/1000 [17:27<00:05,  1.05trial/s, best loss: -0.9700942778071894]\r",
      "100%|█████████▉| 995/1000 [17:28<00:04,  1.02trial/s, best loss: -0.9700942778071894]\r",
      "100%|█████████▉| 996/1000 [17:30<00:04,  1.00s/trial, best loss: -0.9700942778071894]\r",
      "100%|█████████▉| 998/1000 [17:32<00:02,  1.02s/trial, best loss: -0.9700942778071894]\r",
      "100%|██████████| 1000/1000 [17:34<00:00,  1.01s/trial, best loss: -0.9700942778071894]\r",
      "100%|██████████| 1000/1000 [17:34<00:00,  1.05s/trial, best loss: -0.9700942778071894]\n",
      "INFO:hyperopt-spark:Total Trials: 1000: 1000 succeeded, 0 failed, 0 cancelled.\n",
      "{'model_choice': 0, 'rf_criterion': 1, 'rf_max_depth': 12.0, 'rf_min_samples_leaf': 1.7017684943569396, 'rf_min_samples_split': 0.010046818415470619, 'rf_n_estimators': 50.0}\n"
     ]
    }
   ],
   "source": [
    "def sample_to_model(sample):\n",
    "    kwargs = sample[\"model_choice\"][\"kwargs\"]\n",
    "    return models[sample[\"model_choice\"][\"model\"]](**kwargs)\n",
    "\n",
    "def objective_fn(sample):  \n",
    "  with mlflow.start_run(nested=True):\n",
    "    model_name = sample[\"model_choice\"][\"model\"]\n",
    "    mlflow.set_tag(\"model_choice\", model_name)\n",
    "    \n",
    "    model = sample_to_model(sample)\n",
    "    model.fit(X_train, y_train)\n",
    "        \n",
    "    predictions_test = model.predict_proba(X_test)[:,1]\n",
    "    auc_score = roc_auc_score(y_test, predictions_test)\n",
    "    mlflow.log_metric(\"auc\", auc_score)\n",
    "    \n",
    "    model_wrap = SklearnModelWrapper(model)\n",
    "    signature = infer_signature(X_train, model_wrap.predict(None, X_train))\n",
    "  \n",
    "    conda_env =  _mlflow_conda_env(\n",
    "      additional_conda_deps=None,\n",
    "      additional_pip_deps=[\"cloudpickle=={}\".format(cloudpickle.__version__), \"scikit-learn=={}\".format(sklearn.__version__)],\n",
    "      additional_conda_channels=None,\n",
    "    )\n",
    "    mlflow.pyfunc.log_model(\"hyperopt_model\", python_model=model_wrap, conda_env=conda_env, signature=signature)\n",
    "    \n",
    "    # Set the loss to -1*auc_score so fmin maximizes the auc_score\n",
    "    return {'status': STATUS_OK, 'loss': -1*auc_score}\n",
    "\n",
    "with mlflow.start_run(run_name=f'{model_name}_hyperopt_tuning'):\n",
    "  best_params = fmin(\n",
    "    fn=objective_fn, \n",
    "    space=search_space, \n",
    "    algo=tpe.suggest, \n",
    "    max_evals=max_evals,\n",
    "    trials=spark_trials,\n",
    "  )\n",
    "  print(best_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "1a845d01-01d7-4d2b-a29e-166cadbd0356",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "After the computations have completed, you'll have access to a lot of information:\n",
    "- What is your best-performing model, what are its hyper-parameters, and what is its AUC.\n",
    "- How do the various hyper-parameters interact with each other.\n",
    "- Is there any of the models that performs better on aggregate.\n",
    "\n",
    "A lot of this information can be found in MLFlow by going into the Databricks **Experiments**, selecting the experiment group (typically the name of this notebook), and then expanding the experiment container that we started with `hyperopt` and selecting the individual trials here. After clicking on `Compare` you will see a few visualizations to understand the results of the tuning better."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "4d396a71-15cf-4fe8-ba78-fdcdd0167c2b",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "To complete our understanding of the tuning, we'll do a scatter plot to represent the AUC (or loss) for each of the trials. This can help identify whether there are some outliers, and if there is a general trend emerging that could be caught by visual inspection. We use `plotly` to draw the graph, and also add additional hover data to each scatter point representing the hyper-parameters used for that trial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "d4a26b5d-a3c4-47cb-8acf-877b7ccdfead",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.databricks.v1+bamboolib_hint": "{\"pd.DataFrames\": [\"df_train\", \"X\", \"X_train\", \"X_test\"], \"version\": \"0.0.1\"}",
      "text/plain": []
     },
     "metadata": {
      "application/vnd.databricks.v1+output": {
       "addedWidgets": {},
       "arguments": {},
       "data": {
        "application/vnd.databricks.v1+bamboolib_hint": "{\"pd.DataFrames\": [\"df_train\", \"X\", \"X_train\", \"X_test\"], \"version\": \"0.0.1\"}",
        "text/plain": ""
       },
       "datasetInfos": [],
       "executionCount": null,
       "metadata": {
        "kernelSessionId": "d2aa73c1-52c3ecfabfcd81e765b2262d"
       },
       "removedWidgets": [],
       "type": "mimeBundle"
      }
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.9.0.min.js\"></script>                <div id=\"ba448efd-2045-4980-8572-4a5f9e6da966\" class=\"plotly-graph-div\" style=\"height:100%; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"ba448efd-2045-4980-8572-4a5f9e6da966\")) {                    Plotly.newPlot(                        \"ba448efd-2045-4980-8572-4a5f9e6da966\",                        [{\"hovertemplate\":\"rf_criterion: %{customdata[1]}<br>rf_max_depth: %{customdata[2]}<br>rf_min_samples_leaf: %{customdata[3]}<br>rf_min_samples_split: %{customdata[4]}<br>rf_n_estimators: %{customdata[5]}\",\"legendgroup\":\"random_forest\",\"marker\":{\"color\":\"#636efa\",\"symbol\":\"circle\"},\"mode\":\"markers\",\"name\":\"random_forest\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512,513,514,515,516,517,518,519,520,521,522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542,543,544,545,546,547,548,549,550,551,552,553,554,555,556,557,558,559,560,561,562,563,564,565,566,567,568,569,570,571,572,573,574,575,576,577,578,579,580,581,582,583,584,585,586,587,588,589,590,591,592,593,594,595,596,597,598,599,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,620,621,622,623,624,625,626,627,628,629,630,631,632,633,634,635,636,637,638,639,640,641,642,643,644,645,646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664,665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684,685,686,687,688,689,690,691,692,693,694,695,696,697,698,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,732,733,734,735,736,737,738,739,740,741,742,743,744,745,746,747,748,749,750,751,752,753,754,755,756,757,758,759,760,761,762,763,764,765,766,767,768,769,770,771,772,773,774,775,776,777,778,779,780,781,782,783,784,785,786,787,788,789,790,791,792,793,794,795,796,797,798,799,800,801,802,803,804,805,806,807,808,809,810,811,812,813,814,815,816,817,818,819,820,821,822,823,824,825,826,827,828,829,830,831,832,833,834,835,836,837,838,839,840,841,842,843,844,845,846,847,848,849,850,851,852,853,854,855,856,857,858,859,860,861,862,863,864,865,866,867,868,869,870,871,872,873,874,875,876,877,878,879,880,881,882,883,884,885,886,887,888,889,890,891,892,893,894,895,896,897,898,899,900,901,902,903,904,905,906,907,908,909,910,911,912,913,914,915,916,917,918,919,920,921,922,923,924,925,926,927,928,929,930,931,932,933,934,935,936,937,938,939,940,941,942,943,944,945,946,947,948,949,950,951,952,953,954,955,956,957,958,959,960,961,962,963,964,965,966,967,968,969,970,971,972,973,974,975,976,977,978,979,980,981,982,983,984,985,986,987,988,989,990,991,992,993,994,995,996,997,998,999],\"xaxis\":\"x\",\"y\":[-0.9626329836027753,-0.9562822708546335,-0.9624956192192642,-0.945237044468279,-0.9656749484632106,-0.9586555859703025,-0.9633421966699789,-0.9580469526999682,-0.9505051218959941,-0.9619859576430254,-0.9618390852754338,-0.9656506465450575,-0.9488812774470049,-0.967084334369667,-0.9374047554509501,-0.9587348507623457,-0.9634928141438399,-0.9660973781070481,-0.9470437723924107,-0.9212283329781137,-0.9683647510451154,-0.9675297360367779,-0.960723908803061,-0.9675501461016399,-0.9684950532282324,-0.9639596439332724,-0.9661501917485239,-0.9666057953908362,-0.9644503476287128,-0.9659897116519477,-0.9677541091749289,-0.9671651277765141,-0.9670907881813691,-0.9625594083150323,-0.9566308470028337,-0.9682401383662989,-0.9529344362863903,-0.9658821460854244,-0.9643388565793731,-0.9654529385635462,-0.9629872156257498,-0.9552149858840677,-0.9641748728976662,-0.9634675033399163,-0.9656062708572993,-0.9673041522357023,-0.964779736603894,-0.9611787450807417,-0.9233628937475564,-0.9648019137474695,-0.9663002528065964,-0.9369208968926019,-0.9630745943051725,-0.9508608550187063,-0.9576589382898265,-0.9610656795581541,-0.9633246145425038,-0.9626890929377685,-0.9652753991257583,-0.9632069112026009,-0.9618518155795212,-0.9625740432731651,-0.9369199705520306,-0.96815187309029,-0.9682601295907045,-0.9666819173508425,-0.9685657516629143,-0.9655268990622199,-0.9663511373361898,-0.9674739691115007,-0.9668579037159899,-0.9672645030249177,-0.9682469070726505,-0.9659472192176279,-0.9648739237336531,-0.9651323299517989,-0.9644082801493084,-0.9656319699293833,-0.9621477768061699,-0.963675725134049,-0.9652765425296317,-0.9488435604053326,-0.9635082745539669,-0.9604443434987989,-0.967984009784211,-0.9654042277242042,-0.9655790339987229,-0.961942606127186,-0.9590642917104301,-0.9631354698611243,-0.9573319033814434,-0.9656298818415607,-0.9650419123861462,-0.9625710716459864,-0.9146365607326213,-0.9689232365781918,-0.9581106164493225,-0.9609286300692926,-0.9647325207498301,-0.9623183274170743,-0.9668207308611878,-0.9688700866414591,-0.9661846834129597,-0.9682822425324583,-0.9654526175544375,-0.9616896662355789,-0.9662189396707159,-0.9674226871420585,-0.9630372043870682,-0.9687808736242718,-0.9675697001422119,-0.9656123364008413,-0.9683667932744935,-0.9673747528390366,-0.949407426662519,-0.9693741932315002,-0.9686894655162263,-0.9557833187537109,-0.936386279151074,-0.9681978538235925,-0.9682305692376263,-0.9665499612065077,-0.9664636800725145,-0.96934113846498,-0.967575203155506,-0.9669742160164212,-0.9660269548230307,-0.9659529301225354,-0.9630913846101787,-0.9671429781480052,-0.9671431432384038,-0.9674184589935106,-0.9678378772661845,-0.9620385572784287,-0.9638508982761207,-0.9215895813430386,-0.9668926552449424,-0.9684408791195809,-0.9627423284769303,-0.9680765337810633,-0.9638720818200734,-0.9622389953659797,-0.9638861450762696,-0.9626306692799621,-0.9374083935541836,-0.9688700866414591,-0.9687817051907253,-0.9670809500164911,-0.968057591186413,-0.969253279800509,-0.9693086401142484,-0.9676951474447115,-0.9684512308990332,-0.9686964665720282,-0.9677368113698076,-0.9682189242500497,-0.9686358050221491,-0.967246483713609,-0.9686434389244798,-0.9672374312567401,-0.968734764487326,-0.9682407253543834,-0.9682886504857164,-0.9686413600083464,-0.9683928959008855,-0.9676248984227821,-0.9549889496130102,-0.9669017352168778,-0.9665485640525769,-0.9684274762249802,-0.9654107824244834,-0.9670577273003895,-0.9657223324649029,-0.9611665314484584,-0.9497717842299549,-0.9628851072140768,-0.9663152148882974,-0.9682951226407963,-0.964086861371715,-0.9678818891436193,-0.963521940370314,-0.969260773070278,-0.9685741590443359,-0.9607952186836644,-0.9687648476266674,-0.9676757676662272,-0.9671339868357283,-0.9677879557639175,-0.9672487307773708,-0.9665786624780663,-0.9688044020633342,-0.9683169848897223,-0.968778253578498,-0.9639458221982152,-0.9686192745816593,-0.9684864165545901,-0.9593628485250161,-0.9678945735892622,-0.9507725438842534,-0.9648038214587447,-0.9624908407693871,-0.9665021063914558,-0.9688752411305779,-0.9670362777774608,-0.9568500167361921,-0.9685735078544295,-0.9662170869895736,-0.966559080922428,-0.9694597956605205,-0.9673544161476851,-0.9677629353968068,-0.9631918207172566,-0.966937238824314,-0.9623634215537902,-0.9658879059060056,-0.9675791408672411,-0.9189801502702948,-0.9666789640670415,-0.9598055506583831,-0.9368604371198765,-0.967861503536594,-0.9647142446312342,-0.9677968614737653,-0.9679656450059565,-0.9634196699254718,-0.9680982676263459,-0.96108403210749,-0.966411517620945,-0.9649561204088903,-0.9684967897346497,-0.9658957599288683,-0.9657574814337047,-0.9687617903970596,-0.966797153506452,-0.9682485121181946,-0.9440580422133826,-0.9524498256495584,-0.9686434389244798,-0.9627558139167305,-0.9669078221610272,-0.9671410429216633,-0.9678298948396784,-0.9683465788723262,-0.96721380804356,-0.9685733244206529,-0.9693327494269361,-0.9651488756784367,-0.9659959025419036,-0.9673623649446655,-0.9668249834675723,-0.9690074296243629,-0.9678912809529747,-0.9686670193364454,-0.9567239274154747,-0.9633268707779543,-0.9372913689192537,-0.9584315063261938,-0.9697555214804895,-0.969868006129452,-0.9688026044123246,-0.9688481999346964,-0.9688526971194495,-0.9670827843542559,-0.9696022167018031,-0.9684125019143608,-0.9690926498996824,-0.9695863099361534,-0.9681676178227705,-0.9681884039268746,-0.9689105460180895,-0.9686680404511343,-0.9696699801960615,-0.9685011096000856,-0.9680393609262613,-0.9649045571743238,-0.9681894739572373,-0.9677743174626368,-0.9667024833344147,-0.9680685849840829,-0.9641423378601794,-0.9683534117804999,-0.9680928379865621,-0.96778130934675,-0.9645058027165699,-0.9680569889121804,-0.9676866728042385,-0.9658397147956967,-0.9666988758034775,-0.9677680501419409,-0.9665453142175039,-0.9616707511559951,-0.9670927570372365,-0.9673898616677588,-0.9672751605273308,-0.9692554962919749,-0.9666264561485263,-0.9640978245970888,-0.9646037471815171,-0.9679175242119286,-0.9661959584757536,-0.9674509848593089,-0.9677816609281551,-0.9637531403021794,-0.9668268116908778,-0.962779238409986,-0.9661066506844486,-0.9652692602087056,-0.9663758947815542,-0.9642556815906613,-0.969587927210616,-0.9636899320800367,-0.9680575392135098,-0.9687362136141602,-0.9242872287761174,-0.9665582921571891,-0.9678829683456709,-0.9696522115775806,-0.9627747412252329,-0.9687226425719309,-0.9666827428028366,-0.9651150260322185,-0.9614771796061434,-0.9687081513035897,-0.9677038850069307,-0.9610118172869225,-0.9693421565224394,-0.9658215059361525,-0.948815779359886,-0.9690903539202467,-0.9684220007267523,-0.9664832830287602,-0.9689190359447106,-0.9687922190033468,-0.9655590580604652,-0.9629066331677459,-0.9651195446175789,-0.9665022347950993,-0.9599157760146654,-0.9671932940328912,-0.9689351750598105,-0.9687159350101712,-0.937493152187832,-0.9681693023562846,-0.9682908425193453,-0.9659806011077163,-0.9647152657459233,-0.957751829154232,-0.9679961592146725,-0.9664924791754207,-0.9688789036916482,-0.9660094246684592,-0.96516849697806,-0.9665345833415802,-0.9659101747664695,-0.9650261615392065,-0.9689919722714655,-0.9672739498644061,-0.9605780575501583,-0.9598366610268729,-0.9679412850004409,-0.9676333210903515,-0.9686662550290434,-0.9658808161905452,-0.9648658495902588,-0.9648544155515254,-0.9679239260507274,-0.968376441891136,-0.9679786382317899,-0.969861589004505,-0.9679965505400624,-0.9667849521030869,-0.9696912493424434,-0.9656319301853982,-0.9662663114434897,-0.9674751247442925,-0.9653639945825645,-0.954707708946923,-0.9642175854525178,-0.9566025798578793,-0.9659751592390143,-0.9625173469500873,-0.9631023111487973,-0.9616393289500854,-0.9687144522538115,-0.9665359438087558,-0.9694969929731596,-0.9679049620554698,-0.9631333206287099,-0.9671740029140657,-0.9672849344903872,-0.9635474254363251,-0.9636620532032426,-0.946103310477668,-0.9653341988228064,-0.968152239957843,-0.9657927312910832,-0.9667126363939424,-0.9220654055019775,-0.9686934704870126,-0.967531854696896,-0.9683826480672401,-0.96889573068341,-0.9696924386047608,-0.9692886336036944,-0.9698118142492593,-0.9690616740492952,-0.9691820066066608,-0.9688924686194182,-0.9689662028831008,-0.9685690871004164,-0.968426721089267,-0.969342767968361,-0.9688166554396024,-0.9690676448187195,-0.9697220662168907,-0.969287765350486,-0.9682267293572389,-0.969315029724129,-0.9688645163691135,-0.9683723299173133,-0.9687162927060353,-0.9679529789036909,-0.968396494260134,-0.9682971342978782,-0.9693767001597788,-0.9687398853469192,-0.9685418380129216,-0.9681752547823311,-0.9670655966094004,-0.9680463925543594,-0.9686059511750281,-0.9679499247313128,-0.9687055801734893,-0.9691952841548479,-0.9691155454922157,-0.9689531637988233,-0.9671815420422785,-0.9678256024893089,-0.9674240904104483,-0.9689496877287591,-0.9685124488647011,-0.9692142787224016,-0.9690502400105618,-0.9673880793028973,-0.9393017969949279,-0.9681861201763574,-0.9684508823748578,-0.9689897924667552,-0.9668919367959846,-0.9691466986619197,-0.9686955127163906,-0.9649026005473749,-0.9659254150560646,-0.9676157511917951,-0.9681859612004178,-0.9686470281120395,-0.9686820547916568,-0.9675205704624136,-0.9691714866795803,-0.9669702935908342,-0.9674662220916745,-0.9692147067345467,-0.9665472219287792,-0.9479917612064478,-0.9679596772937618,-0.9611150660452394,-0.9571367237288178,-0.9692835586025453,-0.9686633445464566,-0.9688323787714757,-0.9697028484715756,-0.9687502738131267,-0.9619866883209017,-0.9675096989539279,-0.9670993973399449,-0.9669599846125965,-0.968249679979905,-0.9675658358039875,-0.9686655518662335,-0.9603213664378227,-0.9617440971515174,-0.9670302642068221,-0.9687243729638889,-0.9651170621471372,-0.9696874430915816,-0.9684319856386517,-0.9622578248431347,-0.962503720877725,-0.9672436863485179,-0.9684195732864438,-0.9636437006539066,-0.9267359535172941,-0.9694488446640652,-0.9681720263478651,-0.967917802419823,-0.9632309746568444,-0.966905507838214,-0.9677749472519361,-0.9697045360623193,-0.9600397161029682,-0.9621646496563756,-0.9687274240790374,-0.9669106653845624,-0.9688864305909427,-0.9639918243321248,-0.963718293999109,-0.9653004867519205,-0.9671660021441819,-0.9661510141432884,-0.9691458701526958,-0.9685985282215402,-0.9679763575385023,-0.9666226040392203,-0.9660564142875321,-0.9676707904964257,-0.966836640684067,-0.9580104310350726,-0.9690710444580435,-0.9692994867688024,-0.962648126061023,-0.9601559886594146,-0.9673247090475855,-0.9690730622295847,-0.9676021526344994,-0.9689102097228327,-0.9629143863020314,-0.9637988489420466,-0.9380922285584121,-0.9638753530557539,-0.9666346709244825,-0.9684298944936001,-0.967258825749536,-0.9682681762190325,-0.9687233059907558,-0.9660870844149582,-0.9691369032982562,-0.9652381223251496,-0.9667944967739225,-0.9667070049770048,-0.9680575392135098,-0.9694962592380537,-0.9479252695197063,-0.967225229853375,-0.9683282385519086,-0.9636918764780674,-0.9669984445610635,-0.9674845990988473,-0.9631970577515748,-0.9674133564772953,-0.9693871405988896,-0.9664312336946862,-0.965748254714748,-0.9685024578383427,-0.9647969060053717,-0.9575989187581649,-0.9680825259510948,-0.9652909787678399,-0.9656563910794906,-0.9697249614133294,-0.9670003217000428,-0.9682404379748004,-0.9668665709619281,-0.9686391129445847,-0.9691711840138493,-0.9686047894277773,-0.968799700044197,-0.965610624352261,-0.966596363837496,-0.9591561920324427,-0.9675684313919246,-0.9689965092002036,-0.9667562172020024,-0.9669427999249706,-0.9676056684485485,-0.9677614373542988,-0.961546040645831,-0.9694947183943312,-0.9696202604709487,-0.9691475210566841,-0.9691242983405827,-0.9692140677735586,-0.9683235365327718,-0.9695771321328706,-0.9688467110638773,-0.9684304142226331,-0.9690138406348506,-0.9691607313458198,-0.9680260038901046,-0.9692492778869525,-0.967677733464865,-0.9684772173507001,-0.9691871580385503,-0.9681715157905205,-0.9684328600063195,-0.9695390237658085,-0.9686260371735519,-0.9688754795944873,-0.9666451511075783,-0.9693185027369633,-0.9696621445165765,-0.9698499745892246,-0.9678027527552197,-0.9683404796992587,-0.968975768954544,-0.967634696843675,-0.968503289404796,-0.9685001893739736,-0.9682990939820568,-0.9684164793700807,-0.9671735107000987,-0.969420708979984,-0.9683830118775634,-0.9690019052104615,-0.9679064356401409,-0.9685128401900911,-0.9674720369423887,-0.9691447114626746,-0.9690245990258407,-0.9689372295181071,-0.968266503914437,-0.9682744741120246,-0.9672192499122622,-0.9691339347283068,-0.9684156875476121,-0.9677661485451248,-0.9682443695720762,-0.9699429785711257,-0.9687972940044959,-0.9673936679186206,-0.9683568114198239,-0.9671762683212051,-0.9689330502852331,-0.9681618580021893,-0.9366612494392353,-0.9684478679464648,-0.9646469336069579,-0.9658315214203478,-0.9688752227872003,-0.9670996663761504,-0.9677026498861692,-0.9681151343620923,-0.9678272381071489,-0.9673159623136773,-0.9612073852077083,-0.9684968722798492,-0.9685135341812121,-0.9690094749109707,-0.9679029381694694,-0.9690559814877654,-0.9685683717086884,-0.9670941419622491,-0.9686750812509212,-0.967031263920904,-0.9690263997340797,-0.9638000382043641,-0.9659511691582813,-0.9678287697791825,-0.9676820380441531,-0.9662831048057259,-0.9694697347139756,-0.9677908784754228,-0.9691619236653668,-0.9681153177958689,-0.967688727262535,-0.9663372238842445,-0.9683582850044947,-0.9679967828895125,-0.9546946637481863,-0.9679567423533383,-0.9653806625983868,-0.9604675448142932,-0.9683571202000142,-0.9690526521647224,-0.9595650384051322,-0.967811918329584,-0.9647865511686899,-0.9667437131329065,-0.9684389897516833,-0.9684990979430036,-0.9596102150870478,-0.9624546370563708,-0.967845419451627,-0.9514851688196991,-0.9669038080185519,-0.9641688470981092,-0.9682773295644784,-0.9109632963015393,-0.9617204250226636,-0.9675961574072385,-0.9690718026509861,-0.966711804827489,-0.9666917952597055,-0.9678390695857315,-0.9683582850044947,-0.9685457054083756,-0.966732792708747,-0.9669868087451758,-0.9684363269046949,-0.9649431057824496,-0.9665733856997631,-0.9684572444696719,-0.9638640076766791,-0.9682002201193088,-0.9698153881506708,-0.9652615682190122,-0.95919714362304,-0.9667181149493997,-0.9687616742223345,-0.9679203827216121,-0.9678547776314567,-0.9672325794333525,-0.9683003871901812,-0.9673320525131035,-0.9671075570857683,-0.9679496495806481,-0.9638736532360919,-0.9686184797019611,-0.9675027437565701,-0.9679782744214663,-0.9663928960354036,-0.9697558027456137,-0.9675402712500066,-0.9658799234794996,-0.961504743588288,-0.9649393729050985,-0.9685707563477823,-0.9689654752624541,-0.9688335558048747,-0.9675110502494144,-0.9207098940956708,-0.9681485223666397,-0.9682124643238884,-0.9648607990469465,-0.9688552713067793,-0.9662846884506627,-0.9680001825288365,-0.9682419941046708,-0.968882046523685,-0.9628990451238592,-0.9690911487999448,-0.9484820460614319,-0.9682656020317028,-0.9671057227480035,-0.9647609499279536,-0.9688796955141166,-0.9687338778907397,-0.9678462601897693,-0.9681034221154645,-0.960767303120115,-0.9676602002530639,-0.9409223121208741,-0.9675605223389291,-0.9682890846123209,-0.9627494732225238,-0.9685388694429723,-0.965652062042366,-0.958279934996695,-0.9634702548465635,-0.9614151117306444,-0.9672192223971956,-0.9696229141462485,-0.9613388827676019,-0.9684787245648969,-0.9685741590443357,-0.9686868271270747,-0.9680739106780597,-0.9676895832868252,-0.968733345932788,-0.9640935261322602,-0.9685849663509997,-0.9689603360594832,-0.9676373230039084,-0.9682980575812199,-0.9677413727563825,-0.9666637604642014,-0.967213178254261,-0.9686818774723396,-0.9649759373712086,-0.9686921864505772,-0.9679935085966026,-0.9657050530031591,-0.9681370027254772,-0.9615712933623919,-0.969070408554285,-0.9668798699107224,-0.9700942778071894,-0.9688376188630236,-0.9683654052922517,-0.9694100423058821,-0.9691749505207259,-0.9679238526772169,-0.9694186117204731,-0.9682152647462092,-0.9691492973070865,-0.9690298758041439,-0.9693440642337147,-0.9693464977884827,-0.968637948140104,-0.967974645489922,-0.9689598683033532,-0.9688119962216798,-0.9694007422134151,-0.9691177436403036,-0.9696621445165765,-0.9693362866415925,-0.9682873695065108,-0.9681723565286628,-0.9682293035445686,-0.9672604583101465,-0.9671828107925659,-0.9695382319433401,-0.9685116998434471,-0.9696603132360413,-0.9683523203495299,-0.9670312608636743,-0.9696652139751029,-0.9683671356842096,-0.9679434434045441,-0.9681544289342422,-0.968637786106935,-0.9681409924101156,-0.9604287302271918,-0.9683771542256345,-0.9692886336036944,-0.9693480019454497,-0.9690024555117909,-0.9686806545804965,-0.9677075536824602,-0.9685891272404961,-0.9686331513468494,-0.9669198034438603,-0.968549474972482,-0.9682868008618037,-0.9693421565224394,-0.9609909302942415,-0.9696574761269653,-0.9681874745290736,-0.9693337093970329,-0.9680425710173495,-0.9677035059104594,-0.9684914915557392,-0.9680474595274925,-0.9676667763539506,-0.9693417132241464,-0.9674808173058222,-0.9687016149466879,-0.9694054656331592,-0.9677642744633751,-0.9685901361262667,-0.9684278308636147,-0.9685966419108722,-0.9670196953640677,-0.9692624820616289,-0.9686980502169651,-0.9680518069079949,-0.969010171959321,-0.9676152528633691,-0.96818204488929,-0.9682742601059523,-0.9682780632995844,-0.9662128924705515,-0.9688980786357488,-0.9653416309479831,-0.9552503305155642,-0.9681577337994484,-0.9682017701347202,-0.9634212474559494,-0.9684887522780106,-0.9688115620950756,-0.9627449790950002,-0.9690810996862238,-0.9688516362607755,-0.9686693000297326,-0.9497142746838011,-0.9686932809387768,-0.9685567083777342,-0.9645854802346101,-0.9651086639374042,-0.9594493497795411,-0.9684666882519305,-0.964452209481544,-0.968682699867104,-0.9235552148904986,-0.9685214738065034,-0.9690412334121371,-0.9656847682847108,-0.9677024756240816,-0.9682743457083812,-0.9692988386361256,-0.9680827644150042,-0.9663136709873454,-0.9678360337567308,-0.9692182011479885,-0.96884232393939,-0.9693155188808662,-0.9692845552593976,-0.9604824029501875,-0.9673308479646381,-0.968562293936228,-0.9686478688501817,-0.9685509699577601,-0.9372578158243073,-0.9674656137029823,-0.9688302723402757,-0.9677353775291215,-0.9697435860561004,-0.9673768439840885,-0.9666088159336889,-0.9686805445202304,-0.9684747287657992,-0.9639210616956212,-0.9598633353552015,-0.9666243558317856,-0.9681229272403629,-0.9670753583435382,-0.9685261269099669,-0.9654611991979469,-0.9688293093129494,-0.9623365974212108,-0.9687016149466879,-0.9691003143743091,-0.9644732921369199,-0.9694552403884048,-0.9672218393857399,-0.9681754412733371,-0.9693853123755842,-0.9683136219371535,-0.9680163583306917,-0.9623921778554819,-0.9655459425454473,-0.9664124378470571,-0.9686326071599791,-0.9692586819252262,-0.9682425016047856,-0.9668969659386896,-0.9673088359114616,-0.9668506978258041,-0.9655154436228792,-0.9690234586791968,-0.9685104463793079,-0.9598601527791798,-0.9678205458315375,-0.9693624809848725,-0.9676684792308421,-0.9676831753335671,-0.9633626189637592,-0.9676454246623691,-0.9589284681706415,-0.9691728012883116,-0.9610051739269847,-0.967420663256058,-0.9691592241316231,-0.968630931798154,-0.9678635488232018,-0.9682614594855841,-0.9673640280775724,-0.9607921461679084,-0.9655051346446413,-0.9678603540182614,-0.9690718026509861,-0.9675308855551106,-0.9688207307266696,-0.9684633650433468,-0.968637465097826,-0.9633318999206594,-0.966173594841172,-0.9659503651068944,-0.9618031108546381,-0.9689424145795219,-0.9677863018026998,-0.9667409371684225,-0.9676644834317446,-0.9684917881070112,-0.9358518601294709,-0.9676236724737093,-0.9662005045761806,-0.9687298423476574,-0.9684489318623682,-0.9681078336977887,-0.966194341201291,-0.968089572865341,-0.9681676178227705,-0.9692478012450518,-0.965561265380242,-0.9672873344156293,-0.9658368776866207,-0.9502073079882027,-0.9675876216221733,-0.9662801148351694,-0.96247210606635,-0.9685122256869398,-0.968555595546157,-0.9686794072308164,-0.9583189574754097,-0.9683340809176895,-0.9559739492486801,-0.9644778932674797,-0.967677733464865,-0.9685262430846919,-0.9686531150561887],\"yaxis\":\"y\",\"type\":\"scatter\",\"customdata\":[[\"random_forest\",0.0,8.0,3.087288899749003,0.023329650860474123,20.0,-0.9626329836027753,0],[\"random_forest\",1.0,5.0,3.1491846195925843,0.017120865520531006,190.0,-0.9562822708546335,1],[\"random_forest\",1.0,11.0,4.0330545960467505,0.041616714812500316,170.0,-0.9624956192192642,2],[\"random_forest\",1.0,4.0,1.2230890863844883,0.040745334039664696,200.0,-0.945237044468279,3],[\"random_forest\",0.0,8.0,1.7886630602058156,0.01227044033913774,150.0,-0.9656749484632106,4],[\"random_forest\",0.0,8.0,4.125064868782711,0.04638893503104118,140.0,-0.9586555859703025,5],[\"random_forest\",0.0,9.0,4.2022433072116625,0.028145601504114375,100.0,-0.9633421966699789,6],[\"random_forest\",0.0,7.0,2.029945542955745,0.037221326421165345,50.0,-0.9580469526999682,7],[\"random_forest\",0.0,4.0,3.275985213017331,0.026657172413461147,120.0,-0.9505051218959941,8],[\"random_forest\",0.0,6.0,4.550678081409195,0.012372126103673269,180.0,-0.9619859576430254,9],[\"random_forest\",0.0,11.0,2.5515338534410845,0.04102517709528973,120.0,-0.9618390852754338,10],[\"random_forest\",1.0,8.0,2.9749118127225387,0.01627475895489639,100.0,-0.9656506465450575,11],[\"random_forest\",1.0,4.0,4.9487852205229235,0.02593866527459223,160.0,-0.9488812774470049,12],[\"random_forest\",1.0,12.0,3.3955805601426747,0.018936863669885007,170.0,-0.967084334369667,13],[\"random_forest\",1.0,3.0,4.442881994787241,0.021141602348926192,30.0,-0.9374047554509501,14],[\"random_forest\",0.0,6.0,2.7892406588690504,0.03956386158518332,70.0,-0.9587348507623457,15],[\"random_forest\",0.0,11.0,4.647622356118037,0.029298032384519807,190.0,-0.9634928141438399,16],[\"random_forest\",0.0,11.0,2.4352816865187217,0.018015129401768856,180.0,-0.9660973781070481,17],[\"random_forest\",0.0,4.0,1.8251679924594026,0.020348207130603764,30.0,-0.9470437723924107,18],[\"random_forest\",1.0,2.0,1.262590898484663,0.041911998789381136,130.0,-0.9212283329781137,19],[\"random_forest\",1.0,9.0,1.685161135985639,0.01097254098623826,90.0,-0.9683647510451154,20],[\"random_forest\",1.0,9.0,3.6627843368646205,0.01414459020356588,150.0,-0.9675297360367779,21],[\"random_forest\",1.0,7.0,2.189424121875658,0.0331691008458295,80.0,-0.960723908803061,22],[\"random_forest\",1.0,10.0,1.5579709178226173,0.015002675422565283,110.0,-0.9675501461016399,23],[\"random_forest\",0.0,12.0,3.656062331458171,0.010423964074142832,160.0,-0.9684950532282324,24],[\"random_forest\",1.0,12.0,3.5313475201246063,0.032980781171731394,140.0,-0.9639596439332724,25],[\"random_forest\",1.0,10.0,1.0052561760217673,0.021862450083837987,160.0,-0.9661501917485239,26],[\"random_forest\",0.0,12.0,2.5227508492604755,0.017917169988287927,200.0,-0.9666057953908362,27],[\"random_forest\",0.0,10.0,2.331380057571953,0.025066367833446825,180.0,-0.9644503476287128,28],[\"random_forest\",1.0,11.0,2.8099177020002135,0.024089561150875065,180.0,-0.9659897116519477,29],[\"random_forest\",1.0,9.0,3.821177687444521,0.010255750551421618,80.0,-0.9677541091749289,30],[\"random_forest\",1.0,10.0,1.4343814753287367,0.014453376873089356,60.0,-0.9671651277765141,31],[\"random_forest\",1.0,9.0,1.6039837272660382,0.015309217489745971,110.0,-0.9670907881813691,32],[\"random_forest\",0.0,6.0,4.979700850496388,0.010169219002251834,90.0,-0.9625594083150323,33],[\"random_forest\",0.0,7.0,3.7176259580529774,0.04793643158654716,40.0,-0.9566308470028337,34],[\"random_forest\",0.0,12.0,4.023020989070647,0.011910669711395542,90.0,-0.9682401383662989,35],[\"random_forest\",1.0,5.0,3.043726402267325,0.03183288257646312,10.0,-0.9529344362863903,36],[\"random_forest\",0.0,8.0,1.0975698668823128,0.010349469895533895,130.0,-0.9658821460854244,37],[\"random_forest\",0.0,9.0,3.232992123480172,0.023028728563479244,60.0,-0.9643388565793731,38],[\"random_forest\",0.0,8.0,1.9375157462036254,0.01263995783589422,120.0,-0.9654529385635462,39],[\"random_forest\",1.0,11.0,4.352547516324519,0.036406649139798755,140.0,-0.9629872156257498,40],[\"random_forest\",0.0,5.0,4.7423246662661285,0.04474886015759178,90.0,-0.9552149858840677,41],[\"random_forest\",1.0,10.0,3.9255761071065423,0.027683767267073785,160.0,-0.9641748728976662,42],[\"random_forest\",0.0,7.0,2.728564489484436,0.020166863380509797,50.0,-0.9634675033399163,43],[\"random_forest\",1.0,8.0,4.285067506134287,0.016398308824169414,100.0,-0.9656062708572993,44],[\"random_forest\",0.0,12.0,3.3895861299985777,0.012614531745646716,130.0,-0.9673041522357023,45],[\"random_forest\",1.0,11.0,2.170587590348667,0.030030121649453536,200.0,-0.964779736603894,46],[\"random_forest\",0.0,6.0,3.1206377883657552,0.018807898199609333,150.0,-0.9611787450807417,47],[\"random_forest\",1.0,2.0,2.8707091292018765,0.010001736559353892,70.0,-0.9233628937475564,48],[\"random_forest\",0.0,10.0,3.484253265209595,0.023100187634693577,170.0,-0.9648019137474695,49],[\"random_forest\",0.0,9.0,2.6222642779704604,0.013500971168462725,110.0,-0.9663002528065964,50],[\"random_forest\",1.0,3.0,4.132991794059827,0.01631288876647256,190.0,-0.9369208968926019,51],[\"random_forest\",1.0,12.0,1.2381121231837118,0.038395323611431914,80.0,-0.9630745943051725,52],[\"random_forest\",0.0,8.0,1.7671441975181763,0.04975731156902563,10.0,-0.9508608550187063,53],[\"random_forest\",1.0,5.0,2.3228710047071344,0.011683851449563019,20.0,-0.9576589382898265,54],[\"random_forest\",0.0,6.0,4.49880287924134,0.026230788408666134,120.0,-0.9610656795581541,55],[\"random_forest\",1.0,11.0,4.726888606775727,0.03531286911856843,70.0,-0.9633246145425038,56],[\"random_forest\",0.0,7.0,3.6238264233385515,0.01986774983639439,170.0,-0.9626890929377685,57],[\"random_forest\",1.0,10.0,3.854560317595624,0.021751575428758572,40.0,-0.9652753991257583,58],[\"random_forest\",0.0,9.0,1.3929310493524654,0.029217678839885245,150.0,-0.9632069112026009,59],[\"random_forest\",1.0,11.0,2.021660169595431,0.04388967711842039,100.0,-0.9618518155795212,60],[\"random_forest\",0.0,8.0,2.919449436450673,0.02462028685457974,130.0,-0.9625740432731651,61],[\"random_forest\",1.0,3.0,3.367457070373632,0.017313791946854316,190.0,-0.9369199705520306,62],[\"random_forest\",0.0,11.0,2.6988437476161806,0.011348109313947974,140.0,-0.96815187309029,63],[\"random_forest\",1.0,12.0,3.218922451300524,0.013539916241415097,160.0,-0.9682601295907045,64],[\"random_forest\",0.0,12.0,4.040869710552425,0.015309502803823538,60.0,-0.9666819173508425,65],[\"random_forest\",0.0,12.0,3.973151141623964,0.010967986258700291,90.0,-0.9685657516629143,66],[\"random_forest\",0.0,11.0,4.851584204603317,0.018701510914611138,80.0,-0.9655268990622199,67],[\"random_forest\",0.0,10.0,4.223117153170825,0.013761994358552854,100.0,-0.9663511373361898,68],[\"random_forest\",0.0,12.0,3.5706494736211947,0.01216480979947925,90.0,-0.9674739691115007,69],[\"random_forest\",1.0,9.0,3.7413561998284672,0.015432591619255727,160.0,-0.9668579037159899,70],[\"random_forest\",0.0,10.0,4.425988402905299,0.011192231578097987,50.0,-0.9672645030249177,71],[\"random_forest\",0.0,12.0,4.596092326527463,0.01084343720868759,110.0,-0.9682469070726505,72],[\"random_forest\",0.0,11.0,3.96931181624806,0.017185536065239183,70.0,-0.9659472192176279,73],[\"random_forest\",0.0,10.0,3.0098176429923416,0.021371073521401936,120.0,-0.9648739237336531,74],[\"random_forest\",0.0,9.0,4.12807059382752,0.01447440254709473,40.0,-0.9651323299517989,75],[\"random_forest\",0.0,11.0,3.79319225649346,0.027375685378309644,90.0,-0.9644082801493084,76],[\"random_forest\",0.0,12.0,2.366520130730358,0.022523479410689055,60.0,-0.9656319699293833,77],[\"random_forest\",0.0,6.0,3.5032962257793994,0.01293074010115412,120.0,-0.9621477768061699,78],[\"random_forest\",1.0,7.0,3.1569305905356977,0.02056690347865377,80.0,-0.963675725134049,79],[\"random_forest\",0.0,10.0,4.345863592226795,0.01823871351054398,140.0,-0.9652765425296317,80],[\"random_forest\",1.0,4.0,3.2873427801565605,0.01965655857138175,100.0,-0.9488435604053326,81],[\"random_forest\",0.0,11.0,4.883068810709629,0.031197963649425155,110.0,-0.9635082745539669,82],[\"random_forest\",0.0,8.0,1.6393418206755617,0.0347110517556167,50.0,-0.9604443434987989,83],[\"random_forest\",1.0,12.0,3.3854184985346407,0.010003776632744953,20.0,-0.967984009784211,84],[\"random_forest\",0.0,9.0,1.0815887849821142,0.01616003541138538,130.0,-0.9654042277242042,85],[\"random_forest\",1.0,12.0,2.200763339901485,0.02506649410951746,70.0,-0.9655790339987229,86],[\"random_forest\",0.0,10.0,3.6862690657691677,0.040047340548139104,80.0,-0.961942606127186,87],[\"random_forest\",1.0,7.0,3.9122223068961888,0.04229326163690983,90.0,-0.9590642917104301,88],[\"random_forest\",0.0,8.0,4.217083917116031,0.019332082768225463,60.0,-0.9631354698611243,89],[\"random_forest\",1.0,5.0,2.481566407392046,0.01739888119277392,100.0,-0.9573319033814434,90],[\"random_forest\",0.0,9.0,2.6157575438485217,0.014568980860414424,30.0,-0.9656298818415607,91],[\"random_forest\",0.0,11.0,1.367384828236055,0.02394438562687731,190.0,-0.9650419123861462,92],[\"random_forest\",1.0,6.0,4.729550045175682,0.016312262350122633,170.0,-0.9625710716459864,93],[\"random_forest\",0.0,2.0,4.046717691772707,0.012786677258068419,180.0,-0.9146365607326213,94],[\"random_forest\",1.0,11.0,3.08223804262636,0.010748880363931254,150.0,-0.9689232365781918,95],[\"random_forest\",0.0,9.0,1.9011213954175954,0.049032739129249533,140.0,-0.9581106164493225,96],[\"random_forest\",1.0,12.0,2.948890303478607,0.04639157174310765,110.0,-0.9609286300692926,97],[\"random_forest\",0.0,10.0,3.591028895425033,0.02225164618393378,90.0,-0.9647325207498301,98],[\"random_forest\",0.0,8.0,2.8166379642626973,0.02865238909195381,120.0,-0.9623183274170743,99],[\"random_forest\",1.0,11.0,3.29875213678646,0.020651539767993707,200.0,-0.9668207308611878,100],[\"random_forest\",1.0,12.0,3.479414197242027,0.010537993808780108,150.0,-0.9688700866414591,101],[\"random_forest\",0.0,11.0,3.0542102146536068,0.01542187941837511,160.0,-0.9661846834129597,102],[\"random_forest\",0.0,12.0,3.1751979362767546,0.011988463171035838,170.0,-0.9682822425324583,103],[\"random_forest\",1.0,11.0,4.487963268015834,0.02558604062660696,180.0,-0.9654526175544375,104],[\"random_forest\",0.0,12.0,3.8528824780620785,0.038016722304316464,130.0,-0.9616896662355789,105],[\"random_forest\",1.0,10.0,3.743369726873844,0.018142270255662372,140.0,-0.9662189396707159,106],[\"random_forest\",0.0,11.0,4.091223064711909,0.013201714884381984,180.0,-0.9674226871420585,107],[\"random_forest\",0.0,12.0,4.638145736659773,0.033285979525423066,150.0,-0.9630372043870682,108],[\"random_forest\",1.0,12.0,3.4326744698230063,0.010407786049931901,150.0,-0.9687808736242718,109],[\"random_forest\",1.0,10.0,2.702777934952766,0.01419004852408351,150.0,-0.9675697001422119,110],[\"random_forest\",1.0,11.0,3.4652336135063075,0.02680474118747872,130.0,-0.9656123364008413,111],[\"random_forest\",1.0,10.0,2.59627020537494,0.011248470098400446,190.0,-0.9683667932744935,112],[\"random_forest\",1.0,12.0,3.096137571112858,0.016375655062103135,170.0,-0.9673747528390366,113],[\"random_forest\",1.0,4.0,2.8921788591835904,0.021292012646190444,120.0,-0.949407426662519,114],[\"random_forest\",1.0,12.0,2.7609495299219744,0.010059561062337234,160.0,-0.9693741932315002,115],[\"random_forest\",1.0,11.0,2.228228355938489,0.012187056256452307,150.0,-0.9686894655162263,116],[\"random_forest\",1.0,5.0,2.4358719112101634,0.023875534288755974,150.0,-0.9557833187537109,117],[\"random_forest\",1.0,3.0,3.466848525261748,0.030715470878655067,200.0,-0.936386279151074,118],[\"random_forest\",1.0,12.0,3.333408939760092,0.013708598617024873,160.0,-0.9681978538235925,119],[\"random_forest\",1.0,11.0,2.037735297145532,0.01501846288144416,170.0,-0.9682305692376263,120],[\"random_forest\",1.0,10.0,2.792178425038453,0.018683077643344627,140.0,-0.9665499612065077,121],[\"random_forest\",1.0,9.0,2.1023450490805975,0.017757044606250383,180.0,-0.9664636800725145,122],[\"random_forest\",1.0,12.0,2.3559026732285124,0.010313501070363496,160.0,-0.96934113846498,123],[\"random_forest\",1.0,11.0,2.976805414877771,0.016747455296058675,190.0,-0.967575203155506,124],[\"random_forest\",1.0,9.0,1.5149109605199222,0.015851102712331068,130.0,-0.9669742160164212,125],[\"random_forest\",1.0,7.0,2.2923644155538057,0.011568726321506085,140.0,-0.9660269548230307,126],[\"random_forest\",1.0,10.0,3.233116031793643,0.02287693322503756,190.0,-0.9659529301225354,127],[\"random_forest\",1.0,12.0,1.8314851239877128,0.03615576557001173,170.0,-0.9630913846101787,128],[\"random_forest\",1.0,11.0,2.496535818375244,0.01929148766037446,180.0,-0.9671429781480052,129],[\"random_forest\",1.0,8.0,1.7051329335842178,0.010034910174195917,160.0,-0.9671431432384038,130],[\"random_forest\",1.0,9.0,2.4040405121096784,0.01296014730930345,200.0,-0.9674184589935106,131],[\"random_forest\",1.0,10.0,2.098925472043847,0.014624056925940898,160.0,-0.9678378772661845,132],[\"random_forest\",1.0,11.0,2.6712721500617107,0.04502129396300464,190.0,-0.9620385572784287,133],[\"random_forest\",1.0,12.0,2.555891302598985,0.032900350465069346,200.0,-0.9638508982761207,134],[\"random_forest\",1.0,2.0,1.3184025991877315,0.028103635134021455,160.0,-0.9215895813430386,135],[\"random_forest\",1.0,11.0,2.291053287560139,0.020454489349688576,170.0,-0.9668926552449424,136],[\"random_forest\",1.0,12.0,1.9988538561022406,0.013751814170157478,130.0,-0.9684408791195809,137],[\"random_forest\",1.0,9.0,1.932178024078161,0.04105804145428079,180.0,-0.9627423284769303,138],[\"random_forest\",1.0,10.0,1.868353276366384,0.012134732120828585,110.0,-0.9680765337810633,139],[\"random_forest\",1.0,8.0,1.521973965480188,0.029718894373261622,150.0,-0.9638720818200734,140],[\"random_forest\",1.0,12.0,1.7466216475507823,0.04307283841624751,120.0,-0.9622389953659797,141],[\"random_forest\",1.0,6.0,2.844069691017523,0.010199980026016516,140.0,-0.9638861450762696,142],[\"random_forest\",1.0,10.0,2.7058848636847364,0.03887584239524678,160.0,-0.9626306692799621,143],[\"random_forest\",1.0,3.0,2.1259857361590595,0.011151414085086232,140.0,-0.9374083935541836,144],[\"random_forest\",1.0,12.0,3.041899662203752,0.010530223298598903,150.0,-0.9688700866414591,145],[\"random_forest\",1.0,12.0,3.1373694798762526,0.012834672689304882,170.0,-0.9687817051907253,146],[\"random_forest\",1.0,11.0,3.5445154974015773,0.017424987808288418,160.0,-0.9670809500164911,147],[\"random_forest\",1.0,12.0,3.618467200989449,0.014753167477870704,150.0,-0.968057591186413,148],[\"random_forest\",1.0,11.0,3.2092243411849646,0.010070322516070002,130.0,-0.969253279800509,149],[\"random_forest\",1.0,12.0,2.953131220173957,0.01184723019045927,180.0,-0.9693086401142484,150],[\"random_forest\",1.0,11.0,3.0597001795397354,0.014008537104703676,140.0,-0.9676951474447115,151],[\"random_forest\",1.0,12.0,2.7865936618017058,0.01563253689256233,130.0,-0.9684512308990332,152],[\"random_forest\",1.0,12.0,2.38101566834899,0.012989078550454742,170.0,-0.9686964665720282,153],[\"random_forest\",1.0,11.0,2.4756155656961667,0.01677868193311637,110.0,-0.9677368113698076,154],[\"random_forest\",1.0,10.0,2.606746604358728,0.011309656930239498,130.0,-0.9682189242500497,155],[\"random_forest\",1.0,11.0,2.943986665714902,0.01204585381204137,180.0,-0.9686358050221491,156],[\"random_forest\",1.0,12.0,3.3585116523553262,0.018131141321778906,180.0,-0.967246483713609,157],[\"random_forest\",1.0,12.0,2.2530874592134853,0.015287566351752317,190.0,-0.9686434389244798,158],[\"random_forest\",1.0,11.0,2.9121749187104204,0.01935726461641577,120.0,-0.9672374312567401,159],[\"random_forest\",1.0,12.0,3.2325815783847403,0.010104856377267222,200.0,-0.968734764487326,160],[\"random_forest\",1.0,11.0,2.3498124078136873,0.013547902273326378,170.0,-0.9682407253543834,161],[\"random_forest\",1.0,10.0,2.7927161269397636,0.012355176332144996,160.0,-0.9682886504857164,162],[\"random_forest\",1.0,12.0,2.512655031158346,0.014295937386208792,100.0,-0.9686413600083464,163],[\"random_forest\",1.0,10.0,2.7428787018512293,0.010102185767611845,190.0,-0.9683928959008855,164],[\"random_forest\",1.0,11.0,2.6380610418736197,0.016833385215480508,120.0,-0.9676248984227821,165],[\"random_forest\",1.0,5.0,2.1710699525014583,0.02102280735472386,200.0,-0.9549889496130102,166],[\"random_forest\",1.0,12.0,3.2900733551963066,0.0200980221051522,140.0,-0.9669017352168778,167],[\"random_forest\",1.0,11.0,1.6258526029919342,0.022262763459580878,170.0,-0.9665485640525769,168],[\"random_forest\",1.0,9.0,1.9931930340443706,0.01167220395481499,180.0,-0.9684274762249802,169],[\"random_forest\",1.0,12.0,2.9838774423775907,0.027089734535238573,130.0,-0.9654107824244834,170],[\"random_forest\",1.0,11.0,2.5768787336688104,0.018728714143518642,160.0,-0.9670577273003895,171],[\"random_forest\",1.0,12.0,3.1456590829022226,0.025970545642391374,150.0,-0.9657223324649029,172],[\"random_forest\",1.0,10.0,3.693549865317406,0.047895845467133394,190.0,-0.9611665314484584,173],[\"random_forest\",1.0,4.0,1.1249077275896824,0.015885558658733753,170.0,-0.9497717842299549,174],[\"random_forest\",1.0,7.0,2.8344617393977987,0.024860239380541542,160.0,-0.9628851072140768,175],[\"random_forest\",1.0,10.0,2.435269045898203,0.023725945308559485,140.0,-0.9663152148882974,176],[\"random_forest\",1.0,12.0,3.2032670108863646,0.011159385925418812,110.0,-0.9682951226407963,177],[\"random_forest\",1.0,11.0,2.894863797142421,0.03223178614248129,150.0,-0.964086861371715,178],[\"random_forest\",1.0,9.0,3.7807599037396504,0.013451150632249676,100.0,-0.9678818891436193,179],[\"random_forest\",1.0,12.0,3.422793598200784,0.033965638960301275,200.0,-0.963521940370314,180],[\"random_forest\",1.0,11.0,2.6732445401813023,0.010030521682825106,120.0,-0.969260773070278,181],[\"random_forest\",1.0,11.0,2.2246576698478897,0.012418057282821098,180.0,-0.9685741590443359,182],[\"random_forest\",1.0,8.0,3.8718236095914844,0.03738233900484618,130.0,-0.9607952186836644,183],[\"random_forest\",1.0,12.0,2.0693761715157173,0.01448657836584671,140.0,-0.9687648476266674,184],[\"random_forest\",1.0,10.0,2.9835574168421517,0.015074086014274995,160.0,-0.9676757676662272,185],[\"random_forest\",1.0,11.0,2.719983266875193,0.01730247354849615,70.0,-0.9671339868357283,186],[\"random_forest\",1.0,12.0,1.7861965304060006,0.018089777540204327,120.0,-0.9677879557639175,187],[\"random_forest\",1.0,10.0,1.9381310227198245,0.01625605423732903,80.0,-0.9672487307773708,188],[\"random_forest\",1.0,12.0,2.3341523139273415,0.021905639995219522,180.0,-0.9665786624780663,189],[\"random_forest\",1.0,11.0,2.5155297663586818,0.01164087456196693,190.0,-0.9688044020633342,190],[\"random_forest\",1.0,9.0,1.472115518766536,0.010834469529850663,110.0,-0.9683169848897223,191],[\"random_forest\",1.0,12.0,2.2612435465633007,0.0130273991528466,170.0,-0.968778253578498,192],[\"random_forest\",1.0,7.0,2.1448313934550978,0.019660032290975214,120.0,-0.9639458221982152,193],[\"random_forest\",1.0,11.0,2.670833118317452,0.014019325149352498,90.0,-0.9686192745816593,194],[\"random_forest\",1.0,10.0,2.370560042606379,0.01002568281003937,200.0,-0.9684864165545901,195],[\"random_forest\",1.0,6.0,1.676752115958316,0.040113718849178914,190.0,-0.9593628485250161,196],[\"random_forest\",1.0,12.0,3.0788470152562906,0.015399012569993565,150.0,-0.9678945735892622,197],[\"random_forest\",1.0,4.0,1.5810438415940384,0.012196957417720608,50.0,-0.9507725438842534,198],[\"random_forest\",1.0,11.0,2.5417346050846428,0.03028609210617013,180.0,-0.9648038214587447,199],[\"random_forest\",1.0,8.0,2.4336477760403854,0.028979084079513703,40.0,-0.9624908407693871,200],[\"random_forest\",1.0,12.0,1.8395663663459916,0.02331012631075885,160.0,-0.9665021063914558,201],[\"random_forest\",1.0,11.0,3.3538028844735663,0.010848028545557904,170.0,-0.9688752411305779,202],[\"random_forest\",1.0,10.0,2.8596872501830255,0.017207759745455013,110.0,-0.9670362777774608,203],[\"random_forest\",1.0,5.0,2.776982017710807,0.013522822694428161,200.0,-0.9568500167361921,204],[\"random_forest\",1.0,12.0,2.6275582123935406,0.010044183426949595,30.0,-0.9685735078544295,205],[\"random_forest\",1.0,10.0,3.5384657656930574,0.018487655933024195,100.0,-0.9662170869895736,206],[\"random_forest\",1.0,9.0,1.2155196876424474,0.01603432016729971,140.0,-0.966559080922428,207],[\"random_forest\",1.0,12.0,1.9892615317586329,0.011640787936252395,190.0,-0.9694597956605205,208],[\"random_forest\",1.0,11.0,3.275090367759808,0.014685334459854501,150.0,-0.9673544161476851,209],[\"random_forest\",1.0,10.0,3.016644843925015,0.012759467287184481,130.0,-0.9677629353968068,210],[\"random_forest\",1.0,12.0,2.1983653715795226,0.03568090682817669,180.0,-0.9631918207172566,211],[\"random_forest\",1.0,11.0,2.307420889014852,0.020701345885999492,170.0,-0.966937238824314,212],[\"random_forest\",1.0,6.0,3.1305522448940524,0.02133192663569329,160.0,-0.9623634215537902,213],[\"random_forest\",1.0,12.0,1.3422306562071555,0.024431804263958398,200.0,-0.9658879059060056,214],[\"random_forest\",1.0,12.0,2.0073188880644577,0.019100090512188424,190.0,-0.9675791408672411,215],[\"random_forest\",1.0,2.0,1.8571412067227389,0.011569005600504667,200.0,-0.9189801502702948,216],[\"random_forest\",1.0,9.0,1.9440550621764205,0.016875079544115306,190.0,-0.9666789640670415,217],[\"random_forest\",0.0,8.0,1.7196634438524934,0.045338099241880814,180.0,-0.9598055506583831,218],[\"random_forest\",1.0,3.0,1.469684791745935,0.019920474067725895,190.0,-0.9368604371198765,219],[\"random_forest\",1.0,10.0,1.5813901272053874,0.014253520745261965,180.0,-0.967861503536594,220],[\"random_forest\",1.0,11.0,1.1790213613498075,0.031647095360565085,200.0,-0.9647142446312342,221],[\"random_forest\",1.0,12.0,2.1409911739074596,0.01800406953344565,170.0,-0.9677968614737653,222],[\"random_forest\",0.0,12.0,2.0654733197161548,0.012535665414252752,190.0,-0.9679656450059565,223],[\"random_forest\",1.0,12.0,1.7655701868588825,0.03447617623475212,170.0,-0.9634196699254718,224],[\"random_forest\",1.0,11.0,2.424692602234172,0.015603477910120036,160.0,-0.9680982676263459,225],[\"random_forest\",1.0,9.0,1.059067500351181,0.04746205982807114,180.0,-0.96108403210749,226],[\"random_forest\",1.0,11.0,1.0184507001188234,0.0226255897577156,200.0,-0.966411517620945,227],[\"random_forest\",1.0,10.0,2.5469518442366335,0.02809567923355,170.0,-0.9649561204088903,228],[\"random_forest\",0.0,12.0,2.2540489109259276,0.010771575536982866,190.0,-0.9684967897346497,229],[\"random_forest\",1.0,11.0,1.424693760451514,0.02548213924306561,150.0,-0.9658957599288683,230],[\"random_forest\",1.0,11.0,1.2815806590857979,0.026641478358946747,180.0,-0.9657574814337047,231],[\"random_forest\",1.0,12.0,1.9331832945815504,0.013665311302341424,200.0,-0.9687617903970596,232],[\"random_forest\",1.0,9.0,2.930927706108702,0.016268430770328333,150.0,-0.966797153506452,233],[\"random_forest\",1.0,10.0,2.0594487597443623,0.011765043279413977,160.0,-0.9682485121181946,234],[\"random_forest\",0.0,4.0,2.7430287409957526,0.04928177478824691,160.0,-0.9440580422133826,235],[\"random_forest\",1.0,7.0,1.6820269248783917,0.04362508597157272,10.0,-0.9524498256495584,236],[\"random_forest\",1.0,12.0,2.4856171922212917,0.015240737165513916,190.0,-0.9686434389244798,237],[\"random_forest\",1.0,11.0,3.6535691691449284,0.04216399974924506,170.0,-0.9627558139167305,238],[\"random_forest\",1.0,12.0,2.368714265674577,0.02160103976497919,180.0,-0.9669078221610272,239],[\"random_forest\",1.0,10.0,1.8213352319800882,0.0174936700146313,140.0,-0.9671410429216633,240],[\"random_forest\",0.0,12.0,2.848813599252701,0.01309485240768132,200.0,-0.9678298948396784,241],[\"random_forest\",1.0,10.0,2.125813273691965,0.011188278797424964,180.0,-0.9683465788723262,242],[\"random_forest\",1.0,11.0,3.964422940070781,0.014794336111125232,160.0,-0.96721380804356,243],[\"random_forest\",1.0,11.0,2.190977779952064,0.012341079561504962,190.0,-0.9685733244206529,244],[\"random_forest\",1.0,12.0,1.5310476400513882,0.010061353154474158,20.0,-0.9693327494269361,245],[\"random_forest\",1.0,8.0,2.609166709685901,0.02033464694766336,150.0,-0.9651488756784367,246],[\"random_forest\",0.0,12.0,3.394631623253176,0.019018938351235637,170.0,-0.9659959025419036,247],[\"random_forest\",1.0,10.0,3.0430027920781986,0.013636653642502642,140.0,-0.9673623649446655,248],[\"random_forest\",1.0,9.0,2.2771391306800686,0.016558453021740024,190.0,-0.9668249834675723,249],[\"random_forest\",1.0,11.0,2.721843549767576,0.01061779459866689,160.0,-0.9690074296243629,250],[\"random_forest\",1.0,12.0,2.011404325948009,0.017779373090514576,180.0,-0.9678912809529747,251],[\"random_forest\",1.0,11.0,3.154538158363285,0.011903519919969671,170.0,-0.9686670193364454,252],[\"random_forest\",0.0,6.0,1.3675719016665324,0.03926648173151104,30.0,-0.9567239274154747,253],[\"random_forest\",1.0,12.0,1.5788263736333499,0.036960223447337494,20.0,-0.9633268707779543,254],[\"random_forest\",1.0,3.0,1.1887228638100158,0.014201625487400352,20.0,-0.9372913689192537,255],[\"random_forest\",1.0,5.0,1.4630058096029315,0.012828494135760144,80.0,-0.9584315063261938,256],[\"random_forest\",1.0,12.0,1.6509684710685046,0.010216625355476787,60.0,-0.9697555214804895,257],[\"random_forest\",1.0,12.0,1.9033857457605352,0.010071798396850585,50.0,-0.969868006129452,258],[\"random_forest\",1.0,12.0,1.8054527958944993,0.010932427357017072,40.0,-0.9688026044123246,259],[\"random_forest\",1.0,11.0,1.5194671955948649,0.011640400939172865,70.0,-0.9688481999346964,260],[\"random_forest\",1.0,12.0,1.760124940828503,0.013114032334196533,170.0,-0.9688526971194495,261],[\"random_forest\",1.0,12.0,1.3281260533885733,0.015037452256851267,10.0,-0.9670827843542559,262],[\"random_forest\",1.0,11.0,1.24988812658379,0.010020652995211552,150.0,-0.9696022167018031,263],[\"random_forest\",1.0,12.0,1.1362136728213368,0.01600367525491511,180.0,-0.9684125019143608,264],[\"random_forest\",1.0,12.0,1.0070287615913323,0.010815325865282334,60.0,-0.9690926498996824,265],[\"random_forest\",1.0,11.0,1.9188151709181867,0.0100651901952459,50.0,-0.9695863099361534,266],[\"random_forest\",1.0,12.0,1.6408594199277409,0.013999663839397138,60.0,-0.9681676178227705,267],[\"random_forest\",1.0,11.0,1.6762783228592493,0.012152226812942138,50.0,-0.9681884039268746,268],[\"random_forest\",1.0,12.0,1.9082967408416462,0.011215645150690555,30.0,-0.9689105460180895,269],[\"random_forest\",1.0,11.0,2.0539910441829354,0.013236182427339479,40.0,-0.9686680404511343,270],[\"random_forest\",1.0,11.0,1.2532196251590957,0.010070139323044808,40.0,-0.9696699801960615,271],[\"random_forest\",1.0,10.0,1.2962464500819868,0.010010852714371011,70.0,-0.9685011096000856,272],[\"random_forest\",1.0,10.0,1.0863156692922784,0.010086630899422773,50.0,-0.9680393609262613,273],[\"random_forest\",1.0,10.0,1.1855155149086087,0.014860439348622172,10.0,-0.9649045571743238,274],[\"random_forest\",1.0,11.0,1.4615143900233376,0.012550585053917922,50.0,-0.9681894739572373,275],[\"random_forest\",1.0,11.0,1.3811221703342882,0.01560640870009005,60.0,-0.9677743174626368,276],[\"random_forest\",1.0,9.0,1.0054981707827348,0.014265587551152658,40.0,-0.9667024833344147,277],[\"random_forest\",1.0,10.0,1.255827717391834,0.010054954060501184,40.0,-0.9680685849840829,278],[\"random_forest\",1.0,9.0,1.6284766598145541,0.017225169624463802,30.0,-0.9641423378601794,279],[\"random_forest\",1.0,10.0,1.4038121339934118,0.010963459572053698,60.0,-0.9683534117804999,280],[\"random_forest\",1.0,10.0,1.1133974577579009,0.010002908469276582,50.0,-0.9680928379865621,281],[\"random_forest\",1.0,11.0,1.237479928406387,0.013642920308933276,40.0,-0.96778130934675,282],[\"random_forest\",0.0,9.0,1.2433988904593483,0.016615311074695452,70.0,-0.9645058027165699,283],[\"random_forest\",1.0,11.0,1.434508834364331,0.012987030437564804,60.0,-0.9680569889121804,284],[\"random_forest\",1.0,10.0,1.553505789865541,0.012224074201158409,30.0,-0.9676866728042385,285],[\"random_forest\",1.0,11.0,1.0578242300703806,0.018867145251127336,20.0,-0.9658397147956967,286],[\"random_forest\",1.0,8.0,1.726608015256408,0.011179738787991474,80.0,-0.9666988758034775,287],[\"random_forest\",1.0,11.0,1.1233838912244893,0.015502629723650307,50.0,-0.9677680501419409,288],[\"random_forest\",0.0,9.0,1.306605479533704,0.010090671540235352,70.0,-0.9665453142175039,289],[\"random_forest\",1.0,10.0,1.8952243230755708,0.041321264761541354,30.0,-0.9616707511559951,290],[\"random_forest\",1.0,11.0,1.6350132736469067,0.01795295593239894,80.0,-0.9670927570372365,291],[\"random_forest\",1.0,8.0,1.024522259086666,0.010028693957962277,90.0,-0.9673898616677588,292],[\"random_forest\",1.0,10.0,1.5247670131130406,0.014399227859819903,50.0,-0.9672751605273308,293],[\"random_forest\",1.0,11.0,1.362381700312631,0.012301312177615913,20.0,-0.9692554962919749,294],[\"random_forest\",1.0,9.0,1.0056764310418351,0.011553348344339819,40.0,-0.9666264561485263,295],[\"random_forest\",0.0,10.0,1.7637408763312585,0.013475527994285827,10.0,-0.9640978245970888,296],[\"random_forest\",1.0,7.0,1.868886396708502,0.01600886348447256,60.0,-0.9646037471815171,297],[\"random_forest\",1.0,11.0,1.147002634552254,0.010846447293188963,30.0,-0.9679175242119286,298],[\"random_forest\",1.0,10.0,1.2501710902803609,0.019964572808687103,80.0,-0.9661959584757536,299],[\"random_forest\",1.0,11.0,1.4148756764032897,0.014863713916912108,70.0,-0.9674509848593089,300],[\"random_forest\",1.0,11.0,1.604156081685442,0.016921842013707396,40.0,-0.9677816609281551,301],[\"random_forest\",1.0,12.0,1.8253739990294278,0.03260353775371973,50.0,-0.9637531403021794,302],[\"random_forest\",0.0,12.0,1.6905510023889847,0.0129270806677605,60.0,-0.9668268116908778,303],[\"random_forest\",1.0,9.0,1.482218603707368,0.03811239205125892,70.0,-0.962779238409986,304],[\"random_forest\",1.0,10.0,1.9148571918892028,0.02306183873800355,90.0,-0.9661066506844486,305],[\"random_forest\",1.0,11.0,1.5720896804471711,0.011751576445291383,10.0,-0.9652692602087056,306],[\"random_forest\",1.0,12.0,1.2987853019387883,0.018607798728394607,30.0,-0.9663758947815542,307],[\"random_forest\",1.0,8.0,1.3727048620017093,0.013935915571518609,20.0,-0.9642556815906613,308],[\"random_forest\",1.0,12.0,1.0008367952770854,0.01077883952289076,60.0,-0.969587927210616,309],[\"random_forest\",0.0,10.0,1.2069710185350435,0.0210392393229879,40.0,-0.9636899320800367,310],[\"random_forest\",1.0,11.0,1.7369297607027598,0.010088429158059948,20.0,-0.9680575392135098,311],[\"random_forest\",1.0,11.0,2.1469645070047396,0.01232479573383642,50.0,-0.9687362136141602,312],[\"random_forest\",1.0,2.0,1.5086916804932535,0.046764224159042425,30.0,-0.9242872287761174,313],[\"random_forest\",1.0,12.0,1.0763056152532886,0.01621366762560467,60.0,-0.9665582921571891,314],[\"random_forest\",1.0,12.0,1.0997271705367586,0.015047610469291836,70.0,-0.9678829683456709,315],[\"random_forest\",1.0,12.0,1.00782306712916,0.010733715083586146,80.0,-0.9696522115775806,316],[\"random_forest\",0.0,12.0,1.1932519933302703,0.03097508937879278,90.0,-0.9627747412252329,317],[\"random_forest\",1.0,12.0,1.3235567604986334,0.013438125666118971,100.0,-0.9687226425719309,318],[\"random_forest\",1.0,12.0,1.4324771622314922,0.019715405228533676,60.0,-0.9666827428028366,319],[\"random_forest\",1.0,12.0,1.152368967761667,0.029375741312875306,70.0,-0.9651150260322185,320],[\"random_forest\",1.0,6.0,1.2873374840041876,0.024093237955045428,100.0,-0.9614771796061434,321],[\"random_forest\",1.0,12.0,1.5885557446451413,0.014449930128363472,80.0,-0.9687081513035897,322],[\"random_forest\",1.0,11.0,1.0261172501681401,0.01760026815272455,80.0,-0.9677038850069307,323],[\"random_forest\",0.0,11.0,1.1926062411023786,0.03380730290941413,90.0,-0.9610118172869225,324],[\"random_forest\",1.0,12.0,1.6828883637367165,0.011697424884444394,80.0,-0.9693421565224394,325],[\"random_forest\",1.0,12.0,1.3879213076131642,0.021868266737395177,40.0,-0.9658215059361525,326],[\"random_forest\",1.0,4.0,1.0082870660018635,0.012516164683490653,50.0,-0.948815779359886,327],[\"random_forest\",1.0,11.0,1.0760109305676449,0.010010253127104789,70.0,-0.9690903539202467,328],[\"random_forest\",1.0,11.0,1.5075761760671775,0.015517741720243665,100.0,-0.9684220007267523,329],[\"random_forest\",1.0,7.0,1.245015298989562,0.0115182836867601,90.0,-0.9664832830287602,330],[\"random_forest\",0.0,12.0,1.8009975902150734,0.010730491257025767,60.0,-0.9689190359447106,331],[\"random_forest\",1.0,10.0,1.3458976938768505,0.013256723969759764,80.0,-0.9687922190033468,332],[\"random_forest\",1.0,9.0,4.986381456783599,0.018204365688678677,110.0,-0.9655590580604652,333],[\"random_forest\",1.0,11.0,1.9760395996175524,0.03522939341787845,70.0,-0.9629066331677459,334],[\"random_forest\",1.0,12.0,1.0051608577624298,0.026280165347915538,50.0,-0.9651195446175789,335],[\"random_forest\",1.0,10.0,1.146585316444711,0.016232812353155802,60.0,-0.9665022347950993,336],[\"random_forest\",1.0,5.0,1.6253574086685272,0.01691477557768748,80.0,-0.9599157760146654,337],[\"random_forest\",0.0,12.0,1.000038809041524,0.014775054446625111,40.0,-0.9671932940328912,338],[\"random_forest\",1.0,11.0,1.4217948127194047,0.010002447304991285,50.0,-0.9689351750598105,339],[\"random_forest\",1.0,10.0,1.7313054414425904,0.013893087377597544,30.0,-0.9687159350101712,340],[\"random_forest\",1.0,3.0,1.526938339632309,0.049904204948850034,70.0,-0.937493152187832,341],[\"random_forest\",1.0,11.0,1.242413864453863,0.012646051388673946,60.0,-0.9681693023562846,342],[\"random_forest\",1.0,12.0,1.8217651050034407,0.010994035774571427,40.0,-0.9682908425193453,343],[\"random_forest\",1.0,12.0,1.0959245234079564,0.027493887449677634,90.0,-0.9659806011077163,344],[\"random_forest\",0.0,10.0,2.064391529280183,0.019518954684153177,80.0,-0.9647152657459233,345],[\"random_forest\",1.0,11.0,1.3195547118879751,0.04420101708618872,40.0,-0.957751829154232,346],[\"random_forest\",1.0,12.0,1.4877837772160118,0.015588129344936242,50.0,-0.9679961592146725,347],[\"random_forest\",1.0,11.0,1.6593733226894996,0.020615741724359676,90.0,-0.9664924791754207,348],[\"random_forest\",1.0,12.0,1.990635745438286,0.011643502853432306,100.0,-0.9688789036916482,349],[\"random_forest\",1.0,9.0,1.5837657376404177,0.013750008313061241,20.0,-0.9660094246684592,350],[\"random_forest\",1.0,12.0,1.1713813460999145,0.016864832075848962,10.0,-0.96516849697806,351],[\"random_forest\",0.0,8.0,1.4203919198231294,0.010712285195541679,60.0,-0.9665345833415802,352],[\"random_forest\",1.0,9.0,1.8378454994301858,0.012744102886898324,30.0,-0.9659101747664695,353],[\"random_forest\",1.0,11.0,4.352480320814015,0.02526612719989133,70.0,-0.9650261615392065,354],[\"random_forest\",1.0,10.0,1.2649310561254639,0.010015908365167041,80.0,-0.9689919722714655,355],[\"random_forest\",1.0,11.0,2.1779019389478953,0.018417362612271962,100.0,-0.9672739498644061,356],[\"random_forest\",1.0,12.0,4.810613431240105,0.039996514371312054,50.0,-0.9605780575501583,357],[\"random_forest\",1.0,10.0,1.716006112478271,0.04862373956113482,110.0,-0.9598366610268729,358],[\"random_forest\",0.0,11.0,1.4475423403064467,0.011907854131647501,70.0,-0.9679412850004409,359],[\"random_forest\",1.0,12.0,1.0778795206045615,0.014469908364705145,40.0,-0.9676333210903515,360],[\"random_forest\",1.0,12.0,1.9043403233957856,0.015769188850084177,90.0,-0.9686662550290434,361],[\"random_forest\",1.0,11.0,1.3396095446946663,0.022298859021105896,60.0,-0.9658808161905452,362],[\"random_forest\",1.0,7.0,1.561000724137926,0.017335067716133377,70.0,-0.9648658495902588,363],[\"random_forest\",1.0,6.0,1.1638596565753707,0.013072772885980882,20.0,-0.9648544155515254,364],[\"random_forest\",1.0,10.0,2.105360156457766,0.012114017504267317,30.0,-0.9679239260507274,365],[\"random_forest\",0.0,12.0,1.7734378226857614,0.010858601959881399,40.0,-0.968376441891136,366],[\"random_forest\",1.0,11.0,1.0009050120145342,0.014134909811226348,50.0,-0.9679786382317899,367],[\"random_forest\",1.0,12.0,1.645928202268336,0.010006062478482047,60.0,-0.969861589004505,368],[\"random_forest\",1.0,10.0,1.2869896080821617,0.011348908030949208,80.0,-0.9679965505400624,369],[\"random_forest\",1.0,9.0,2.2913606275551857,0.014852739267099879,50.0,-0.9667849521030869,370],[\"random_forest\",1.0,11.0,1.3796384165087656,0.010024111693673862,100.0,-0.9696912493424434,371],[\"random_forest\",1.0,12.0,1.4833228040548108,0.0233964194493416,90.0,-0.9656319301853982,372],[\"random_forest\",1.0,12.0,2.012123539815878,0.019247066176403753,60.0,-0.9662663114434897,373],[\"random_forest\",1.0,12.0,1.6555044207321128,0.013426195157288967,60.0,-0.9674751247442925,374],[\"random_forest\",0.0,12.0,1.8785342121475443,0.017926685305969345,30.0,-0.9653639945825645,375],[\"random_forest\",1.0,5.0,2.1789662294452805,0.010000189785230273,10.0,-0.954707708946923,376],[\"random_forest\",1.0,8.0,2.4057377939496587,0.016432324948800623,30.0,-0.9642175854525178,377],[\"random_forest\",1.0,11.0,1.9586885331225905,0.045544749835531806,10.0,-0.9566025798578793,378],[\"random_forest\",1.0,10.0,2.0412483822731904,0.012525563563074656,20.0,-0.9659751592390143,379],[\"random_forest\",1.0,11.0,2.273531122059759,0.04317366113758979,40.0,-0.9625173469500873,380],[\"random_forest\",0.0,10.0,1.7497806807086862,0.030268686515889466,110.0,-0.9631023111487973,381],[\"random_forest\",1.0,9.0,1.8416279674665064,0.028288472264865836,20.0,-0.9616393289500854,382],[\"random_forest\",1.0,11.0,1.6299213745875614,0.01523362919686219,50.0,-0.9687144522538115,383],[\"random_forest\",1.0,12.0,2.2402978285285617,0.02133305796247447,70.0,-0.9665359438087558,384],[\"random_forest\",1.0,11.0,1.5751076092241223,0.01151826600639978,40.0,-0.9694969929731596,385],[\"random_forest\",1.0,10.0,2.1331578931770667,0.014007387341606057,100.0,-0.9679049620554698,386],[\"random_forest\",1.0,11.0,1.6999993495267052,0.03711584012998052,120.0,-0.9631333206287099,387],[\"random_forest\",0.0,12.0,1.38637622349386,0.012411755254864812,30.0,-0.9671740029140657,388],[\"random_forest\",1.0,10.0,1.9325114774093173,0.013169303959682247,50.0,-0.9672849344903872,389],[\"random_forest\",1.0,11.0,1.7857603463418297,0.031948282454511624,60.0,-0.9635474254363251,390],[\"random_forest\",1.0,7.0,2.359438686344292,0.020455749857187636,40.0,-0.9636620532032426,391],[\"random_forest\",1.0,4.0,2.0730191041069066,0.024451329965602724,20.0,-0.946103310477668,392],[\"random_forest\",1.0,8.0,1.431897993684276,0.017273573077014562,70.0,-0.9653341988228064,393],[\"random_forest\",1.0,9.0,1.5117414019776365,0.01068726463227556,50.0,-0.968152239957843,394],[\"random_forest\",0.0,12.0,2.4838295074231747,0.014889339865554158,10.0,-0.9657927312910832,395],[\"random_forest\",1.0,12.0,1.636932914279144,0.01876626446442147,60.0,-0.9667126363939424,396],[\"random_forest\",1.0,2.0,1.969643551281325,0.01587044765232727,90.0,-0.9220654055019775,397],[\"random_forest\",1.0,11.0,1.8904554491719894,0.01149321785262893,70.0,-0.9686934704870126,398],[\"random_forest\",1.0,12.0,1.7198748664714774,0.013910185513121358,30.0,-0.967531854696896,399],[\"random_forest\",1.0,10.0,1.3570682256447193,0.012284889888694285,80.0,-0.9683826480672401,400],[\"random_forest\",1.0,12.0,1.1032182476076202,0.010804148370234835,80.0,-0.96889573068341,401],[\"random_forest\",1.0,12.0,1.21297919282991,0.010235040470867134,100.0,-0.9696924386047608,402],[\"random_forest\",1.0,12.0,1.0007556497360481,0.01011175188567018,80.0,-0.9692886336036944,403],[\"random_forest\",1.0,12.0,1.5613247163054051,0.010001185441808296,90.0,-0.9698118142492593,404],[\"random_forest\",1.0,12.0,1.3472535790531168,0.011235022942017664,100.0,-0.9690616740492952,405],[\"random_forest\",1.0,11.0,1.153630639315736,0.012666410278473327,70.0,-0.9691820066066608,406],[\"random_forest\",1.0,12.0,1.2152309209184593,0.011726808510814594,120.0,-0.9688924686194182,407],[\"random_forest\",1.0,11.0,1.486077414389114,0.010002542025816227,100.0,-0.9689662028831008,408],[\"random_forest\",1.0,12.0,1.248017597342105,0.013298573697437217,110.0,-0.9685690871004164,409],[\"random_forest\",1.0,11.0,1.3822847063825936,0.012142526241426449,110.0,-0.968426721089267,410],[\"random_forest\",1.0,12.0,1.5457085317977477,0.010677721527119804,90.0,-0.969342767968361,411],[\"random_forest\",1.0,12.0,1.8009768135553967,0.014324805147379037,100.0,-0.9688166554396024,412],[\"random_forest\",1.0,12.0,1.6179484520987484,0.01086364458439627,110.0,-0.9690676448187195,413],[\"random_forest\",1.0,12.0,1.6973648257557685,0.010022509071185372,120.0,-0.9697220662168907,414],[\"random_forest\",1.0,12.0,1.5611809643233456,0.010029390197415906,90.0,-0.969287765350486,415],[\"random_forest\",1.0,12.0,1.8409336462175978,0.013192557680666683,100.0,-0.9682267293572389,416],[\"random_forest\",1.0,12.0,2.044648520005733,0.010008212877278077,110.0,-0.969315029724129,417],[\"random_forest\",1.0,12.0,1.4723050676686316,0.011397184097211173,90.0,-0.9688645163691135,418],[\"random_forest\",1.0,12.0,2.1302002139510714,0.015129401911661138,110.0,-0.9683723299173133,419],[\"random_forest\",1.0,12.0,1.72209240338217,0.012230985754172708,130.0,-0.9687162927060353,420],[\"random_forest\",1.0,12.0,2.202823315087521,0.013880412940759005,120.0,-0.9679529789036909,421],[\"random_forest\",1.0,12.0,1.8900972812384447,0.012813536674763008,120.0,-0.968396494260134,422],[\"random_forest\",1.0,11.0,1.9969409518139767,0.01625374413141846,120.0,-0.9682971342978782,423],[\"random_forest\",1.0,12.0,1.6540086003587093,0.011665970154443577,130.0,-0.9693767001597788,424],[\"random_forest\",1.0,11.0,1.7706951348813276,0.014783630487391447,130.0,-0.9687398853469192,425],[\"random_forest\",1.0,12.0,2.0876869045370796,0.013412460363793762,120.0,-0.9685418380129216,426],[\"random_forest\",1.0,12.0,4.176481267934136,0.010896696708034044,100.0,-0.9681752547823311,427],[\"random_forest\",1.0,11.0,2.3217393662605756,0.016674000083635777,130.0,-0.9670655966094004,428],[\"random_forest\",1.0,12.0,1.965916899804948,0.015463796221556747,110.0,-0.9680463925543594,429],[\"random_forest\",1.0,12.0,1.6801282648308373,0.012009897792828931,90.0,-0.9686059511750281,430],[\"random_forest\",1.0,11.0,1.5909701791584134,0.014420356389560251,110.0,-0.9679499247313128,431],[\"random_forest\",0.0,11.0,2.2057892803609396,0.010040940518903956,120.0,-0.9687055801734893,432],[\"random_forest\",1.0,12.0,1.879336279959513,0.01305024854724842,80.0,-0.9691952841548479,433],[\"random_forest\",1.0,12.0,1.4701729703475213,0.010001645357050021,90.0,-0.9691155454922157,434],[\"random_forest\",1.0,11.0,1.7720712041703677,0.011078509945363257,120.0,-0.9689531637988233,435],[\"random_forest\",1.0,11.0,1.8534594918801637,0.017888403048545408,100.0,-0.9671815420422785,436],[\"random_forest\",1.0,12.0,2.435786807299511,0.013959045411707432,70.0,-0.9678256024893089,437],[\"random_forest\",1.0,12.0,1.672954192669179,0.0156614643532094,90.0,-0.9674240904104483,438],[\"random_forest\",1.0,11.0,1.5388186057284523,0.012475742641197397,110.0,-0.9689496877287591,439],[\"random_forest\",0.0,12.0,1.990791819008389,0.011563124538182694,80.0,-0.9685124488647011,440],[\"random_forest\",1.0,11.0,1.7838537512205996,0.010036391878797403,60.0,-0.9692142787224016,441],[\"random_forest\",1.0,12.0,1.2978582776735872,0.010699179487733291,140.0,-0.9690502400105618,442],[\"random_forest\",1.0,12.0,1.4181941248781667,0.017066066189009233,130.0,-0.9673880793028973,443],[\"random_forest\",1.0,3.0,2.1124791034492554,0.013508433905032011,90.0,-0.9393017969949279,444],[\"random_forest\",1.0,11.0,2.5724095076082163,0.014669400476468662,100.0,-0.9681861201763574,445],[\"random_forest\",1.0,12.0,2.261007721919608,0.011943579160601087,80.0,-0.9684508823748578,446],[\"random_forest\",1.0,12.0,1.5717122861383115,0.012920325972750349,120.0,-0.9689897924667552,447],[\"random_forest\",0.0,11.0,1.30829397464768,0.01591099403012044,60.0,-0.9668919367959846,448],[\"random_forest\",1.0,11.0,1.1622832602939117,0.010723297825496587,70.0,-0.9691466986619197,449],[\"random_forest\",1.0,12.0,2.0502945915927886,0.013888572839416292,100.0,-0.9686955127163906,450],[\"random_forest\",1.0,6.0,1.911533845873905,0.011509245565854245,110.0,-0.9649026005473749,451],[\"random_forest\",1.0,11.0,4.581334931478775,0.018907034446245384,70.0,-0.9659254150560646,452],[\"random_forest\",1.0,12.0,1.7021790581104097,0.015309276788145159,80.0,-0.9676157511917951,453],[\"random_forest\",1.0,12.0,1.4895971375511765,0.01642005308362743,60.0,-0.9681859612004178,454],[\"random_forest\",1.0,11.0,2.3596906580757446,0.01004466037587829,100.0,-0.9686470281120395,455],[\"random_forest\",1.0,12.0,1.6251624470686408,0.012159104561384415,90.0,-0.9686820547916568,456],[\"random_forest\",0.0,12.0,1.397442145454339,0.01265790597620344,50.0,-0.9675205704624136,457],[\"random_forest\",1.0,11.0,1.0852898440032526,0.010000112075556969,130.0,-0.9691714866795803,458],[\"random_forest\",1.0,11.0,1.8291398886583181,0.017767135536144927,120.0,-0.9669702935908342,459],[\"random_forest\",1.0,12.0,2.144924459685851,0.014384287613101145,70.0,-0.9674662220916745,460],[\"random_forest\",1.0,12.0,1.2126682245714662,0.010638514808167096,80.0,-0.9692147067345467,461],[\"random_forest\",1.0,10.0,1.9608585857331642,0.019947037875065757,110.0,-0.9665472219287792,462],[\"random_forest\",1.0,4.0,1.7506227229696967,0.011133043903539377,100.0,-0.9479917612064478,463],[\"random_forest\",0.0,12.0,1.2995183408715851,0.013371811613975375,50.0,-0.9679596772937618,464],[\"random_forest\",1.0,11.0,1.5280825035122843,0.035689242729786144,60.0,-0.9611150660452394,465],[\"random_forest\",1.0,5.0,2.2246120308624198,0.014985160560760576,90.0,-0.9571367237288178,466],[\"random_forest\",1.0,11.0,1.689503309357002,0.010032423976596292,70.0,-0.9692835586025453,467],[\"random_forest\",1.0,12.0,1.4405425176575812,0.012023331865854157,110.0,-0.9686633445464566,468],[\"random_forest\",1.0,12.0,2.504648509697417,0.01264144418686167,80.0,-0.9688323787714757,469],[\"random_forest\",1.0,11.0,1.5934053121097989,0.010018799005959627,140.0,-0.9697028484715756,470],[\"random_forest\",1.0,12.0,2.0451714870610727,0.011232967209738109,60.0,-0.9687502738131267,471],[\"random_forest\",0.0,11.0,1.0985328833706163,0.033735785794044286,120.0,-0.9619866883209017,472],[\"random_forest\",1.0,10.0,1.1859085091613781,0.016612982403644047,70.0,-0.9675096989539279,473],[\"random_forest\",1.0,12.0,1.929570172897458,0.013960952469030757,50.0,-0.9670993973399449,474],[\"random_forest\",1.0,11.0,2.330870682249331,0.01740203585267469,140.0,-0.9669599846125965,475],[\"random_forest\",1.0,11.0,1.8343703830381692,0.013321196229026578,140.0,-0.968249679979905,476],[\"random_forest\",1.0,10.0,1.5844484221017483,0.015884061419758896,140.0,-0.9675658358039875,477],[\"random_forest\",1.0,11.0,2.428754697641278,0.011603892677148456,140.0,-0.9686655518662335,478],[\"random_forest\",1.0,7.0,2.2010516027195512,0.034719748813046764,130.0,-0.9603213664378227,479],[\"random_forest\",0.0,10.0,2.1090925091337662,0.03887035362399737,150.0,-0.9617440971515174,480],[\"random_forest\",1.0,12.0,1.721799193766233,0.0184338775215525,150.0,-0.9670302642068221,481],[\"random_forest\",1.0,12.0,2.651543492516266,0.01420943962140658,140.0,-0.9687243729638889,482],[\"random_forest\",1.0,11.0,2.0198807373819836,0.027225143703742113,130.0,-0.9651170621471372,483],[\"random_forest\",1.0,12.0,1.8114237703709573,0.01004515582816285,130.0,-0.9696874430915816,484],[\"random_forest\",1.0,11.0,1.6556139467085171,0.012706236440105143,130.0,-0.9684319856386517,485],[\"random_forest\",1.0,12.0,1.9096705351998216,0.04270565155421659,40.0,-0.9622578248431347,486],[\"random_forest\",1.0,11.0,1.5126315886541448,0.04103112895422084,120.0,-0.962503720877725,487],[\"random_forest\",0.0,10.0,1.6201198113546367,0.015108009679677577,150.0,-0.9672436863485179,488],[\"random_forest\",1.0,12.0,1.3546975370290306,0.011074287454744713,40.0,-0.9684195732864438,489],[\"random_forest\",1.0,6.0,2.2894074537642943,0.012131118013193498,140.0,-0.9636437006539066,490],[\"random_forest\",1.0,2.0,1.8933323901146286,0.022380528309656447,50.0,-0.9267359535172941,491],[\"random_forest\",1.0,11.0,1.7712039235399915,0.01086099913230523,60.0,-0.9694488446640652,492],[\"random_forest\",1.0,12.0,4.450159166362431,0.013614777718068561,130.0,-0.9681720263478651,493],[\"random_forest\",1.0,12.0,2.0389534781719507,0.01660888461104716,120.0,-0.967917802419823,494],[\"random_forest\",1.0,11.0,1.4452670419667675,0.031203726490187027,50.0,-0.9632309746568444,495],[\"random_forest\",0.0,10.0,1.5590724744325024,0.01972792643921481,60.0,-0.966905507838214,496],[\"random_forest\",1.0,12.0,1.7252942826522846,0.014438978400092183,40.0,-0.9677749472519361,497],[\"random_forest\",1.0,11.0,2.1250039232808913,0.010030465262644263,50.0,-0.9697045360623193,498],[\"random_forest\",1.0,12.0,2.5649541371614326,0.046925433402232444,150.0,-0.9600397161029682,499],[\"random_forest\",1.0,12.0,1.9680696802316848,0.045835085552282225,120.0,-0.9621646496563756,500],[\"random_forest\",1.0,11.0,1.841285659832303,0.011866245025258834,140.0,-0.9687274240790374,501],[\"random_forest\",1.0,8.0,1.3865841099349205,0.013028755920915866,130.0,-0.9669106653845624,502],[\"random_forest\",1.0,10.0,2.1996653911585686,0.010032856858131203,30.0,-0.9688864305909427,503],[\"random_forest\",0.0,12.0,2.3222072522257324,0.029056856417020147,50.0,-0.9639918243321248,504],[\"random_forest\",1.0,7.0,2.107689701615625,0.0212440440103224,40.0,-0.963718293999109,505],[\"random_forest\",1.0,11.0,2.51545294141474,0.025670817067385658,50.0,-0.9653004867519205,506],[\"random_forest\",1.0,12.0,2.3660073845549765,0.015531060707799797,30.0,-0.9671660021441819,507],[\"random_forest\",1.0,10.0,2.4459964596526076,0.017458334674234535,60.0,-0.9661510141432884,508],[\"random_forest\",1.0,11.0,2.1749333045157275,0.010948442734301275,70.0,-0.9691458701526958,509],[\"random_forest\",1.0,12.0,2.622148417897158,0.012524395997727235,60.0,-0.9685985282215402,510],[\"random_forest\",1.0,11.0,1.9421391081820316,0.011584027127649875,40.0,-0.9679763575385023,511],[\"random_forest\",0.0,12.0,2.8346422153234094,0.013520537289441092,20.0,-0.9666226040392203,512],[\"random_forest\",1.0,11.0,2.2443207391660724,0.01909170128327261,30.0,-0.9660564142875321,513],[\"random_forest\",1.0,10.0,2.0895925193210196,0.0149838693029507,50.0,-0.9676707904964257,514],[\"random_forest\",1.0,12.0,2.7040116519327393,0.016203940616607716,40.0,-0.966836640684067,515],[\"random_forest\",1.0,5.0,2.41782085767972,0.018166174360204124,60.0,-0.9580104310350726,516],[\"random_forest\",1.0,11.0,1.8318622084273326,0.010664376700081376,70.0,-0.9690710444580435,517],[\"random_forest\",1.0,12.0,2.130562586808751,0.010065052829999557,50.0,-0.9692994867688024,518],[\"random_forest\",1.0,12.0,1.669140745863431,0.044440407647308146,70.0,-0.962648126061023,519],[\"random_forest\",0.0,11.0,1.9767106596054225,0.03652599303633723,30.0,-0.9601559886594146,520],[\"random_forest\",1.0,10.0,2.252176833354064,0.014219309277853035,80.0,-0.9673247090475855,521],[\"random_forest\",1.0,12.0,2.0396264945196454,0.012312814253874392,50.0,-0.9690730622295847,522],[\"random_forest\",1.0,11.0,1.7390912670128653,0.011586825311203659,20.0,-0.9676021526344994,523],[\"random_forest\",1.0,12.0,2.522962021116643,0.013009841876238538,80.0,-0.9689102097228327,524],[\"random_forest\",1.0,9.0,1.8710097400104904,0.03293055124193338,40.0,-0.9629143863020314,525],[\"random_forest\",1.0,6.0,1.448925844081153,0.010030966919177702,60.0,-0.9637988489420466,526],[\"random_forest\",1.0,3.0,2.7587544165012066,0.010913730300257544,70.0,-0.9380922285584121,527],[\"random_forest\",0.0,12.0,1.7973199707019074,0.023431696175398836,60.0,-0.9638753530557539,528],[\"random_forest\",1.0,10.0,1.5311251683869371,0.015661125978039894,40.0,-0.9666346709244825,529],[\"random_forest\",1.0,11.0,2.0078738345098053,0.014841635738351532,50.0,-0.9684298944936001,530],[\"random_forest\",1.0,11.0,2.33663170503917,0.01729449796822933,30.0,-0.967258825749536,531],[\"random_forest\",1.0,12.0,1.9049646481170026,0.01347838889462111,80.0,-0.9682681762190325,532],[\"random_forest\",1.0,12.0,1.6382763553504156,0.012453727716237593,90.0,-0.9687233059907558,533],[\"random_forest\",1.0,11.0,2.1430778680666904,0.020753777457941235,70.0,-0.9660870844149582,534],[\"random_forest\",1.0,12.0,1.2729968209736633,0.01151155955480321,60.0,-0.9691369032982562,535],[\"random_forest\",0.0,10.0,1.746390335208642,0.016568563341338327,10.0,-0.9652381223251496,536],[\"random_forest\",1.0,9.0,2.4112736095874974,0.013979855861363823,40.0,-0.9667944967739225,537],[\"random_forest\",1.0,8.0,2.2601235340023464,0.010871380102568171,50.0,-0.9667070049770048,538],[\"random_forest\",1.0,11.0,1.3090552510590852,0.010073701982047528,20.0,-0.9680575392135098,539],[\"random_forest\",1.0,12.0,1.4929063110216427,0.01199726205391528,70.0,-0.9694962592380537,540],[\"random_forest\",1.0,4.0,2.0782751059400177,0.04022326677661099,80.0,-0.9479252695197063,541],[\"random_forest\",1.0,12.0,2.598443962199427,0.014603007932401815,40.0,-0.967225229853375,542],[\"random_forest\",1.0,10.0,1.6713814322242957,0.013008854866398837,60.0,-0.9683282385519086,543],[\"random_forest\",0.0,11.0,1.971742698348187,0.026499788078498496,30.0,-0.9636918764780674,544],[\"random_forest\",1.0,12.0,1.6022570902890858,0.01846049239879507,50.0,-0.9669984445610635,545],[\"random_forest\",1.0,11.0,1.8443755330143772,0.015559264349445514,60.0,-0.9674845990988473,546],[\"random_forest\",1.0,12.0,1.3960347444200956,0.0380413746321161,90.0,-0.9631970577515748,547],[\"random_forest\",1.0,9.0,2.1968326295762592,0.010072111606144225,40.0,-0.9674133564772953,548],[\"random_forest\",1.0,12.0,1.7830392524666618,0.011405547915479703,90.0,-0.9693871405988896,549],[\"random_forest\",1.0,11.0,2.4754501741027903,0.019221522115926058,70.0,-0.9664312336946862,550],[\"random_forest\",1.0,12.0,1.9400743478349092,0.012438328716656216,10.0,-0.965748254714748,551],[\"random_forest\",1.0,10.0,1.3363598321616879,0.010736619961525196,30.0,-0.9685024578383427,552],[\"random_forest\",0.0,11.0,1.5120388742773547,0.021839614033170637,60.0,-0.9647969060053717,553],[\"random_forest\",1.0,11.0,2.310491995463809,0.04867001181379607,20.0,-0.9575989187581649,554],[\"random_forest\",1.0,12.0,2.0636463608976383,0.013628795842831542,50.0,-0.9680825259510948,555],[\"random_forest\",1.0,12.0,2.9040144088139224,0.02991434412589512,80.0,-0.9652909787678399,556],[\"random_forest\",1.0,10.0,1.6857537274398797,0.024599332960353312,50.0,-0.9656563910794906,557],[\"random_forest\",1.0,11.0,1.1362732832667297,0.010013839920542191,70.0,-0.9697249614133294,558],[\"random_forest\",1.0,12.0,1.5713417990723788,0.016998064798251483,80.0,-0.9670003217000428,559],[\"random_forest\",1.0,11.0,1.4277948611531075,0.014293317522013817,60.0,-0.9682404379748004,560],[\"random_forest\",0.0,12.0,1.8966970495488704,0.01582269088530671,40.0,-0.9668665709619281,561],[\"random_forest\",1.0,12.0,1.2442522261130418,0.014873844957980269,90.0,-0.9686391129445847,562],[\"random_forest\",1.0,11.0,1.2360106828995252,0.010002343006886544,70.0,-0.9691711840138493,563],[\"random_forest\",1.0,12.0,1.0798471472812492,0.011861007656426937,80.0,-0.9686047894277773,564],[\"random_forest\",1.0,11.0,1.1617301490236864,0.013108154965959193,90.0,-0.968799700044197,565],[\"random_forest\",1.0,7.0,1.019136631711371,0.010644320381465703,70.0,-0.965610624352261,566],[\"random_forest\",1.0,12.0,1.1513295302869704,0.022825777113286636,80.0,-0.966596363837496,567],[\"random_forest\",1.0,5.0,1.0005884967563827,0.017805831246465603,100.0,-0.9591561920324427,568],[\"random_forest\",0.0,10.0,1.0673494731146218,0.011561429749241717,110.0,-0.9675684313919246,569],[\"random_forest\",1.0,12.0,1.3263760883037807,0.012599928160974678,90.0,-0.9689965092002036,570],[\"random_forest\",1.0,11.0,1.1363758848799435,0.019850980413924302,100.0,-0.9667562172020024,571],[\"random_forest\",1.0,10.0,4.019772219996794,0.016674481018844292,70.0,-0.9669427999249706,572],[\"random_forest\",1.0,9.0,1.0048533618222337,0.013944453787967919,80.0,-0.9676056684485485,573],[\"random_forest\",1.0,12.0,1.2364765386889018,0.015392789184217947,90.0,-0.9677614373542988,574],[\"random_forest\",1.0,11.0,1.3995260921905126,0.04198537298257323,70.0,-0.961546040645831,575],[\"random_forest\",1.0,12.0,1.4761634242340087,0.010717631963376134,100.0,-0.9694947183943312,576],[\"random_forest\",1.0,11.0,1.57695243443068,0.010068860294124425,60.0,-0.9696202604709487,577],[\"random_forest\",1.0,11.0,1.7191077716673437,0.01003568324205197,50.0,-0.9691475210566841,578],[\"random_forest\",1.0,11.0,1.6483900300062413,0.011641313425511243,60.0,-0.9691242983405827,579],[\"random_forest\",1.0,11.0,1.512934392000885,0.011181841928815954,70.0,-0.9692140677735586,580],[\"random_forest\",1.0,12.0,1.7832795946100508,0.012038075929995378,60.0,-0.9683235365327718,581],[\"random_forest\",1.0,12.0,1.3407762821996831,0.010065662662732273,50.0,-0.9695771321328706,582],[\"random_forest\",1.0,10.0,1.4588310690739608,0.010019052445232273,60.0,-0.9688467110638773,583],[\"random_forest\",1.0,11.0,1.8443406516156777,0.01320478170453917,50.0,-0.9684304142226331,584],[\"random_forest\",1.0,12.0,1.6703501941746217,0.010805306704154466,80.0,-0.9690138406348506,585],[\"random_forest\",1.0,12.0,1.60003970063389,0.01247759798214653,70.0,-0.9691607313458198,586],[\"random_forest\",1.0,11.0,1.3887643257145108,0.01136060758769236,60.0,-0.9680260038901046,587],[\"random_forest\",1.0,12.0,1.7556419775875463,0.010039680614252604,70.0,-0.9692492778869525,588],[\"random_forest\",1.0,11.0,2.0062823952941367,0.013030713890618075,50.0,-0.967677733464865,589],[\"random_forest\",1.0,12.0,1.5492733707230293,0.012189815024690239,40.0,-0.9684772173507001,590],[\"random_forest\",1.0,11.0,1.3178408002003135,0.01061026273195172,60.0,-0.9691871580385503,591],[\"random_forest\",1.0,10.0,1.9058116033137755,0.01400293384660051,80.0,-0.9681715157905205,592],[\"random_forest\",1.0,11.0,1.1940350488112157,0.010834499186987617,70.0,-0.9684328600063195,593],[\"random_forest\",1.0,12.0,1.4579602073377778,0.01159325207024612,50.0,-0.9695390237658085,594],[\"random_forest\",1.0,12.0,1.792891017516519,0.012705998774107805,50.0,-0.9686260371735519,595],[\"random_forest\",1.0,10.0,1.6462271207286914,0.013757105869253201,70.0,-0.9688754795944873,596],[\"random_forest\",1.0,11.0,4.293993839820763,0.01482652772216762,80.0,-0.9666451511075783,597],[\"random_forest\",1.0,12.0,2.1241218867360674,0.011413440741267981,40.0,-0.9693185027369633,598],[\"random_forest\",1.0,12.0,1.2709634874430116,0.010066103733599846,60.0,-0.9696621445165765,599],[\"random_forest\",1.0,11.0,1.8654882884710762,0.01000671723332036,40.0,-0.9698499745892246,600],[\"random_forest\",1.0,10.0,1.7356340757271975,0.01223636026720808,70.0,-0.9678027527552197,601],[\"random_forest\",1.0,12.0,1.0872717159947074,0.013063078815027551,60.0,-0.9683404796992587,602],[\"random_forest\",1.0,11.0,1.9923193256429463,0.010841978672227972,80.0,-0.968975768954544,603],[\"random_forest\",1.0,11.0,1.5605974319222633,0.014215097936125153,40.0,-0.967634696843675,604],[\"random_forest\",1.0,12.0,1.3887677235574791,0.011942712043351664,30.0,-0.968503289404796,605],[\"random_forest\",1.0,12.0,1.1412399572665533,0.014981183093521554,40.0,-0.9685001893739736,606],[\"random_forest\",1.0,10.0,4.672289135118871,0.010022022875267036,30.0,-0.9682990939820568,607],[\"random_forest\",1.0,12.0,1.505630440001214,0.0131721203457057,110.0,-0.9684164793700807,608],[\"random_forest\",1.0,11.0,1.283234099897519,0.01594802756835141,90.0,-0.9671735107000987,609],[\"random_forest\",1.0,12.0,1.8571857491900379,0.012142553528014294,30.0,-0.969420708979984,610],[\"random_forest\",1.0,11.0,1.646905463865192,0.01001551903079917,20.0,-0.9683830118775634,611],[\"random_forest\",1.0,10.0,1.445766947759111,0.010858114451200638,50.0,-0.9690019052104615,612],[\"random_forest\",1.0,11.0,1.7007422406441737,0.011318999985015857,40.0,-0.9679064356401409,613],[\"random_forest\",0.0,12.0,1.0060503529971307,0.013596320353231831,60.0,-0.9685128401900911,614],[\"random_forest\",1.0,12.0,3.7800336545661097,0.014429369566271586,70.0,-0.9674720369423887,615],[\"random_forest\",1.0,11.0,1.8621141759587767,0.01003578735896058,80.0,-0.9691447114626746,616],[\"random_forest\",1.0,12.0,1.2221859848763414,0.012646047651242661,30.0,-0.9690245990258407,617],[\"random_forest\",1.0,11.0,1.3526686252311748,0.011319716490174388,100.0,-0.9689372295181071,618],[\"random_forest\",1.0,10.0,1.597661497489947,0.01610119064135795,60.0,-0.968266503914437,619],[\"random_forest\",1.0,12.0,1.9686499411235343,0.013594724900062229,90.0,-0.9682744741120246,620],[\"random_forest\",1.0,9.0,1.7909533035067977,0.012324804179083474,50.0,-0.9672192499122622,621],[\"random_forest\",0.0,12.0,1.1285536519079664,0.010011871814457694,110.0,-0.9691339347283068,622],[\"random_forest\",1.0,11.0,1.5022671266299343,0.010839980328119586,40.0,-0.9684156875476121,623],[\"random_forest\",1.0,12.0,1.722534854425015,0.014984759928096806,50.0,-0.9677661485451248,624],[\"random_forest\",1.0,11.0,2.0341824023352713,0.011581005533346634,70.0,-0.9682443695720762,625],[\"random_forest\",1.0,12.0,1.301295257049765,0.010001235764334687,90.0,-0.9699429785711257,626],[\"random_forest\",1.0,10.0,1.4284534394306778,0.01280015074541178,80.0,-0.9687972940044959,627],[\"random_forest\",1.0,11.0,1.5955181875673246,0.01384766892038933,60.0,-0.9673936679186206,628],[\"random_forest\",1.0,12.0,2.1965042305525406,0.01088422155505655,40.0,-0.9683568114198239,629],[\"random_forest\",1.0,12.0,4.87422648575073,0.017048066181642917,70.0,-0.9671762683212051,630],[\"random_forest\",0.0,11.0,1.1710721718905202,0.01000561771680915,90.0,-0.9689330502852331,631],[\"random_forest\",1.0,12.0,1.3678195389770278,0.01552274814398262,90.0,-0.9681618580021893,632],[\"random_forest\",1.0,3.0,1.010035490980406,0.0118160424705487,80.0,-0.9366612494392353,633],[\"random_forest\",1.0,11.0,1.0630546864949095,0.012992238390594376,80.0,-0.9684478679464648,634],[\"random_forest\",1.0,12.0,1.2709055522136417,0.027911564383930006,80.0,-0.9646469336069579,635],[\"random_forest\",1.0,8.0,1.1996442539642511,0.014429657934775806,90.0,-0.9658315214203478,636],[\"random_forest\",1.0,10.0,1.003276877860472,0.010828348962742252,100.0,-0.9688752227872003,637],[\"random_forest\",1.0,11.0,1.105229523100596,0.016251115638427495,70.0,-0.9670996663761504,638],[\"random_forest\",1.0,10.0,1.3466588602940526,0.013490484074175118,90.0,-0.9677026498861692,639],[\"random_forest\",0.0,12.0,1.2496991159210598,0.011967910173163652,70.0,-0.9681151343620923,640],[\"random_forest\",1.0,11.0,1.0824713318341304,0.01776368775707592,100.0,-0.9678272381071489,641],[\"random_forest\",1.0,9.0,1.5332087564546883,0.012515886768798995,70.0,-0.9673159623136773,642],[\"random_forest\",1.0,6.0,1.4502383592529784,0.010022216018262609,80.0,-0.9612073852077083,643],[\"random_forest\",1.0,12.0,1.2920659893033997,0.014908179592891407,60.0,-0.9684968722798492,644],[\"random_forest\",1.0,12.0,1.2086944525493724,0.011341563145596877,80.0,-0.9685135341812121,645],[\"random_forest\",1.0,11.0,1.398032703547701,0.010547847856464293,90.0,-0.9690094749109707,646],[\"random_forest\",1.0,12.0,1.5321813844326224,0.014191894624199035,50.0,-0.9679029381694694,647],[\"random_forest\",1.0,11.0,1.0801166482933748,0.01003718329943107,60.0,-0.9690559814877654,648],[\"random_forest\",1.0,12.0,1.3246306889867019,0.013139679490627651,20.0,-0.9685683717086884,649],[\"random_forest\",0.0,12.0,1.0004937130298877,0.01554306756191979,70.0,-0.9670941419622491,650],[\"random_forest\",1.0,10.0,1.1722508897742252,0.012112237498721765,50.0,-0.9686750812509212,651],[\"random_forest\",1.0,11.0,1.6563141818817573,0.01660244609621568,100.0,-0.967031263920904,652],[\"random_forest\",1.0,12.0,1.457329113237979,0.010770166456419272,60.0,-0.9690263997340797,653],[\"random_forest\",1.0,11.0,1.9134330613949806,0.032040087870672695,90.0,-0.9638000382043641,654],[\"random_forest\",1.0,10.0,1.8202464687324424,0.018513248925617398,80.0,-0.9659511691582813,655],[\"random_forest\",1.0,11.0,1.6064137231249793,0.011689428283653543,30.0,-0.9678287697791825,656],[\"random_forest\",1.0,12.0,1.1484083059956114,0.013732185839868908,50.0,-0.9676820380441531,657],[\"random_forest\",1.0,9.0,1.3358813843325974,0.012697288067212702,40.0,-0.9662831048057259,658],[\"random_forest\",1.0,12.0,1.524942066998409,0.01002011362697564,60.0,-0.9694697347139756,659],[\"random_forest\",0.0,12.0,3.895407162680225,0.014678927877801201,70.0,-0.9677908784754228,660],[\"random_forest\",1.0,11.0,1.2571796817653613,0.010025730596320807,90.0,-0.9691619236653668,661],[\"random_forest\",1.0,10.0,1.7631047922060505,0.011344150359079144,80.0,-0.9681153177958689,662],[\"random_forest\",1.0,12.0,1.4220529274929383,0.015599304339244894,50.0,-0.967688727262535,663],[\"random_forest\",1.0,11.0,1.9164887330194784,0.017242196486037566,60.0,-0.9663372238842445,664],[\"random_forest\",1.0,12.0,1.6405012051759247,0.012315063001444585,40.0,-0.9683582850044947,665],[\"random_forest\",1.0,10.0,2.0527796673067638,0.013594730151405872,70.0,-0.9679967828895125,666],[\"random_forest\",1.0,5.0,1.0689681360946544,0.02872674540219408,20.0,-0.9546946637481863,667],[\"random_forest\",0.0,11.0,1.73802576021855,0.010654798810498483,100.0,-0.9679567423533383,668],[\"random_forest\",1.0,8.0,1.8351914434258962,0.014251662071142467,30.0,-0.9653806625983868,669],[\"random_forest\",1.0,12.0,1.5506846771863234,0.034355286072182474,10.0,-0.9604675448142932,670],[\"random_forest\",1.0,11.0,1.3891610171344277,0.013011480163845368,60.0,-0.9683571202000142,671],[\"random_forest\",1.0,12.0,1.005100951638413,0.011357680655891877,80.0,-0.9690526521647224,672],[\"random_forest\",1.0,12.0,1.301491720486779,0.0499613610281003,40.0,-0.9595650384051322,673],[\"random_forest\",1.0,11.0,1.2083115491818757,0.016314721302628198,70.0,-0.967811918329584,674],[\"random_forest\",1.0,7.0,1.4967074884580784,0.012014287454623856,90.0,-0.9647865511686899,675],[\"random_forest\",1.0,12.0,3.5109318233289537,0.01500298709292949,50.0,-0.9667437131329065,676],[\"random_forest\",0.0,11.0,1.9650333546729697,0.010999768739274805,100.0,-0.9684389897516833,677],[\"random_forest\",1.0,12.0,2.261288083238961,0.010032074780767973,80.0,-0.9684990979430036,678],[\"random_forest\",1.0,10.0,1.6170270923167858,0.047572565832173515,70.0,-0.9596102150870478,679],[\"random_forest\",1.0,12.0,1.7235947603348067,0.039200493674969056,90.0,-0.9624546370563708,680],[\"random_forest\",1.0,11.0,1.1628491475716964,0.012889413803990557,30.0,-0.967845419451627,681],[\"random_forest\",1.0,4.0,2.3678905400424624,0.010672412362491284,50.0,-0.9514851688196991,682],[\"random_forest\",1.0,10.0,1.466355276459346,0.018103631350491874,60.0,-0.9669038080185519,683],[\"random_forest\",1.0,9.0,2.1522683206348248,0.030762979690369365,40.0,-0.9641688470981092,684],[\"random_forest\",1.0,11.0,1.2955800861129632,0.013726319452521592,60.0,-0.9682773295644784,685],[\"random_forest\",0.0,2.0,3.284584900184332,0.010006198576596983,80.0,-0.9109632963015393,686],[\"random_forest\",1.0,12.0,1.8732289060327534,0.04356140357161387,70.0,-0.9617204250226636,687],[\"random_forest\",1.0,11.0,1.003096193989466,0.01193627133022006,50.0,-0.9675961574072385,688],[\"random_forest\",1.0,12.0,2.050532634193711,0.010018149735119126,60.0,-0.9690718026509861,689],[\"random_forest\",1.0,12.0,1.6683992766945768,0.014540688844383212,20.0,-0.966711804827489,690],[\"random_forest\",1.0,11.0,1.3833556379011096,0.019396578224290417,90.0,-0.9666917952597055,691],[\"random_forest\",1.0,12.0,1.1081661193085783,0.015287956943726663,70.0,-0.9678390695857315,692],[\"random_forest\",1.0,12.0,1.7766770427054337,0.012272356205625484,40.0,-0.9683582850044947,693],[\"random_forest\",1.0,11.0,1.5451521716075243,0.011421160883494394,80.0,-0.9685457054083756,694],[\"random_forest\",0.0,10.0,1.933648428098573,0.017119755969489392,110.0,-0.966732792708747,695],[\"random_forest\",1.0,12.0,1.604545598881953,0.02029479997684171,100.0,-0.9669868087451758,696],[\"random_forest\",1.0,11.0,2.1827184827714663,0.013283520253683123,70.0,-0.9684363269046949,697],[\"random_forest\",1.0,10.0,1.2461955842983286,0.025311879269445337,30.0,-0.9649431057824496,698],[\"random_forest\",1.0,12.0,1.817561921351,0.01589203454905969,40.0,-0.9665733856997631,699],[\"random_forest\",1.0,11.0,1.4594783612133981,0.010036887659929676,50.0,-0.9684572444696719,700],[\"random_forest\",1.0,12.0,1.3601176712090106,0.037460541147242864,60.0,-0.9638640076766791,701],[\"random_forest\",1.0,11.0,2.08901200613213,0.010835754431363136,90.0,-0.9682002201193088,702],[\"random_forest\",1.0,12.0,1.9940898294103282,0.010001788689962066,80.0,-0.9698153881506708,703],[\"random_forest\",0.0,8.0,1.0015873630695753,0.014038127520309734,60.0,-0.9652615682190122,704],[\"random_forest\",1.0,6.0,1.7080520253511318,0.011678762776682804,10.0,-0.95919714362304,705],[\"random_forest\",1.0,9.0,2.303464483194456,0.010843010415357684,30.0,-0.9667181149493997,706],[\"random_forest\",1.0,12.0,2.5377332745539727,0.013095701015534998,100.0,-0.9687616742223345,707],[\"random_forest\",1.0,12.0,2.388455587706254,0.012458425266385652,110.0,-0.9679203827216121,708],[\"random_forest\",1.0,12.0,2.637689563124622,0.014407134509858852,100.0,-0.9678547776314567,709],[\"random_forest\",1.0,12.0,2.2584268601926136,0.016300136572492987,90.0,-0.9672325794333525,710],[\"random_forest\",1.0,12.0,2.452867967660553,0.011399340865486173,80.0,-0.9683003871901812,711],[\"random_forest\",1.0,12.0,2.0002048201386033,0.01880765101891091,80.0,-0.9673320525131035,712],[\"random_forest\",0.0,12.0,2.123316060665655,0.012475419617741885,90.0,-0.9671075570857683,713],[\"random_forest\",1.0,12.0,2.1699844166375737,0.01359297982565903,100.0,-0.9679496495806481,714],[\"random_forest\",1.0,12.0,1.9590830272405189,0.0332220700929097,110.0,-0.9638736532360919,715],[\"random_forest\",1.0,12.0,2.805889697344976,0.010756531924200112,90.0,-0.9686184797019611,716],[\"random_forest\",1.0,12.0,2.026533597539889,0.015272530081382019,80.0,-0.9675027437565701,717],[\"random_forest\",1.0,12.0,2.3320026536745426,0.011849512246245426,70.0,-0.9679782744214663,718],[\"random_forest\",1.0,12.0,1.825494414958315,0.02125862863710218,100.0,-0.9663928960354036,719],[\"random_forest\",1.0,12.0,2.6805305655321607,0.010014524820994797,80.0,-0.9697558027456137,720],[\"random_forest\",1.0,12.0,1.8701142548036709,0.017250343236416766,90.0,-0.9675402712500066,721],[\"random_forest\",0.0,12.0,2.236503507511566,0.010011702096651132,20.0,-0.9658799234794996,722],[\"random_forest\",1.0,11.0,2.0887333081703354,0.035429157649198496,50.0,-0.961504743588288,723],[\"random_forest\",1.0,7.0,2.9857502295252187,0.012806087558731976,110.0,-0.9649393729050985,724],[\"random_forest\",1.0,12.0,2.494279892687225,0.014313264176809153,90.0,-0.9685707563477823,725],[\"random_forest\",1.0,12.0,3.0975969834446255,0.01140396729554668,100.0,-0.9689654752624541,726],[\"random_forest\",1.0,12.0,3.289578381396389,0.010046486584332687,90.0,-0.9688335558048747,727],[\"random_forest\",1.0,11.0,3.351548792646504,0.017844017589547234,110.0,-0.9675110502494144,728],[\"random_forest\",1.0,2.0,3.1711359113002553,0.015503791255143212,100.0,-0.9207098940956708,729],[\"random_forest\",1.0,12.0,3.631678473245898,0.010817678612875144,80.0,-0.9681485223666397,730],[\"random_forest\",1.0,11.0,2.9677589694997093,0.013334470054360337,80.0,-0.9682124643238884,731],[\"random_forest\",0.0,12.0,2.906329295449759,0.023604299898783054,110.0,-0.9648607990469465,732],[\"random_forest\",1.0,12.0,2.562496183502969,0.0121695837042564,90.0,-0.9688552713067793,733],[\"random_forest\",1.0,11.0,3.430799011085899,0.016557825097095594,100.0,-0.9662846884506627,734],[\"random_forest\",1.0,11.0,2.4848005907834345,0.014829776855198656,90.0,-0.9680001825288365,735],[\"random_forest\",1.0,12.0,2.6531156242464293,0.013782582612090505,80.0,-0.9682419941046708,736],[\"random_forest\",1.0,12.0,2.823851119303428,0.01002430348285566,100.0,-0.968882046523685,737],[\"random_forest\",1.0,6.0,2.5467109062230486,0.012768356495378021,80.0,-0.9628990451238592,738],[\"random_forest\",1.0,11.0,2.6863358666477613,0.010019739750584738,70.0,-0.9690911487999448,739],[\"random_forest\",1.0,4.0,2.723912029084614,0.011335143702054087,80.0,-0.9484820460614319,740],[\"random_forest\",0.0,12.0,2.4103978041929235,0.012119606048381016,90.0,-0.9682656020317028,741],[\"random_forest\",1.0,11.0,3.715521772601103,0.016075157390438823,70.0,-0.9671057227480035,742],[\"random_forest\",1.0,12.0,3.199499588364721,0.02615009525884304,90.0,-0.9647609499279536,743],[\"random_forest\",1.0,12.0,2.774769151512963,0.010006608546286449,100.0,-0.9688796955141166,744],[\"random_forest\",1.0,11.0,3.5390650157835117,0.010805682578296944,110.0,-0.9687338778907397,745],[\"random_forest\",1.0,12.0,2.6025319434858964,0.014048350137431827,80.0,-0.9678462601897693,746],[\"random_forest\",1.0,11.0,2.368847176725488,0.015023868151366668,100.0,-0.9681034221154645,747],[\"random_forest\",1.0,12.0,2.290835653259685,0.04478544076846481,70.0,-0.960767303120115,748],[\"random_forest\",1.0,12.0,2.844215179945185,0.013145078165414359,80.0,-0.9676602002530639,749],[\"random_forest\",0.0,3.0,2.7243598989359,0.011669018640096583,90.0,-0.9409223121208741,750],[\"random_forest\",1.0,11.0,2.9170520949784082,0.018646203729351593,70.0,-0.9675605223389291,751],[\"random_forest\",1.0,12.0,3.0376736088880345,0.011063854987876046,80.0,-0.9682890846123209,752],[\"random_forest\",1.0,12.0,2.2025826282055427,0.03655293117222494,80.0,-0.9627494732225238,753],[\"random_forest\",1.0,11.0,2.4266167028578978,0.01223468662605892,60.0,-0.9685388694429723,754],[\"random_forest\",1.0,11.0,3.119189157385215,0.022296480545275537,100.0,-0.965652062042366,755],[\"random_forest\",1.0,5.0,2.1291674131923592,0.016894573388201037,110.0,-0.958279934996695,756],[\"random_forest\",1.0,7.0,2.319245876906381,0.020629753656049236,70.0,-0.9634702548465635,757],[\"random_forest\",1.0,12.0,2.660125303191189,0.0416426644984162,90.0,-0.9614151117306444,758],[\"random_forest\",0.0,12.0,4.109919914526648,0.013492392686099787,120.0,-0.9672192223971956,759],[\"random_forest\",1.0,12.0,2.0422833328915,0.01003770716377194,90.0,-0.9696229141462485,760],[\"random_forest\",1.0,8.0,2.192005634338738,0.029926306436174124,60.0,-0.9613388827676019,761],[\"random_forest\",1.0,11.0,3.028447238519247,0.014881784388208396,70.0,-0.9684787245648969,762],[\"random_forest\",1.0,12.0,1.9543569379742525,0.012686552743547906,100.0,-0.9685741590443357,763],[\"random_forest\",1.0,11.0,2.4795821761992087,0.010610562078426056,80.0,-0.9686868271270747,764],[\"random_forest\",1.0,11.0,1.8923521149132982,0.015600256123657504,50.0,-0.9680739106780597,765],[\"random_forest\",1.0,12.0,3.2380209381881073,0.017770717722892376,60.0,-0.9676895832868252,766],[\"random_forest\",1.0,11.0,2.307457332769991,0.011565682564729306,110.0,-0.968733345932788,767],[\"random_forest\",0.0,12.0,2.5772779351631216,0.027214583922411877,90.0,-0.9640935261322602,768],[\"random_forest\",1.0,12.0,2.2341216607290453,0.014157369416030569,70.0,-0.9685849663509997,769],[\"random_forest\",1.0,12.0,2.0229498033182765,0.010759762750248562,40.0,-0.9689603360594832,770],[\"random_forest\",1.0,11.0,2.106944763092769,0.010024836929648239,30.0,-0.9676373230039084,771],[\"random_forest\",1.0,12.0,2.383313009816546,0.012579783923007565,80.0,-0.9682980575812199,772],[\"random_forest\",1.0,11.0,1.7721418344850204,0.011852145848606256,50.0,-0.9677413727563825,773],[\"random_forest\",1.0,12.0,1.9638197220741413,0.019874721241400092,70.0,-0.9666637604642014,774],[\"random_forest\",1.0,11.0,1.866955307982335,0.01629451467414019,90.0,-0.967213178254261,775],[\"random_forest\",1.0,12.0,2.775335629347603,0.013414227612453252,100.0,-0.9686818774723396,776],[\"random_forest\",0.0,12.0,3.818608588688898,0.014392468706403302,10.0,-0.9649759373712086,777],[\"random_forest\",1.0,11.0,3.4153556920040593,0.010676758660044977,80.0,-0.9686921864505772,778],[\"random_forest\",1.0,10.0,1.7390459541495629,0.011365818248448836,60.0,-0.9679935085966026,779],[\"random_forest\",1.0,12.0,1.9468543817842647,0.024655606678286538,70.0,-0.9657050530031591,780],[\"random_forest\",1.0,12.0,2.88447611506363,0.012853761165026779,40.0,-0.9681370027254772,781],[\"random_forest\",1.0,11.0,2.0943456420232764,0.031542391840747655,20.0,-0.9615712933623919,782],[\"random_forest\",1.0,12.0,2.2061850664693807,0.010040217027411417,100.0,-0.969070408554285,783],[\"random_forest\",1.0,11.0,2.4666153610535346,0.01571810313649758,80.0,-0.9668798699107224,784],[\"random_forest\",1.0,12.0,1.7017684943569396,0.010046818415470619,50.0,-0.9700942778071894,785],[\"random_forest\",1.0,12.0,1.6473324166169674,0.011324835471012,60.0,-0.9688376188630236,786],[\"random_forest\",1.0,12.0,1.5812395246551838,0.01226360083790657,50.0,-0.9683654052922517,787],[\"random_forest\",1.0,12.0,1.781491041826401,0.01100201470805939,60.0,-0.9694100423058821,788],[\"random_forest\",1.0,12.0,1.863221447253716,0.010926517868138502,70.0,-0.9691749505207259,789],[\"random_forest\",1.0,12.0,1.7044719375102348,0.010619522828969557,40.0,-0.9679238526772169,790],[\"random_forest\",1.0,12.0,1.7850947248975275,0.010087757173877032,30.0,-0.9694186117204731,791],[\"random_forest\",1.0,12.0,1.897531413556779,0.01198672469908608,50.0,-0.9682152647462092,792],[\"random_forest\",1.0,12.0,1.6478924078091768,0.010078068444710615,30.0,-0.9691492973070865,793],[\"random_forest\",1.0,12.0,2.0108890033950884,0.011605008972505155,40.0,-0.9690298758041439,794],[\"random_forest\",1.0,12.0,1.5565982137226382,0.012868066312002765,50.0,-0.9693440642337147,795],[\"random_forest\",1.0,12.0,1.7135195971039632,0.010035315731685484,40.0,-0.9693464977884827,796],[\"random_forest\",1.0,12.0,1.8176282786281526,0.013589947322642987,50.0,-0.968637948140104,797],[\"random_forest\",1.0,12.0,1.9195292316708406,0.011912803514857156,40.0,-0.967974645489922,798],[\"random_forest\",1.0,12.0,1.5185741773776136,0.010864529649888386,60.0,-0.9689598683033532,799],[\"random_forest\",1.0,12.0,2.0624907915604664,0.012697889087137357,50.0,-0.9688119962216798,800],[\"random_forest\",1.0,12.0,1.6699411601523055,0.010002000659486478,40.0,-0.9694007422134151,801],[\"random_forest\",1.0,11.0,1.7799216709269383,0.01002001173585717,30.0,-0.9691177436403036,802],[\"random_forest\",1.0,12.0,1.4674974793661584,0.010013709115204559,60.0,-0.9696621445165765,803],[\"random_forest\",1.0,11.0,1.6090068884125284,0.011459253934308059,50.0,-0.9693362866415925,804],[\"random_forest\",1.0,12.0,2.150917513442052,0.012167539274551937,60.0,-0.9682873695065108,805],[\"random_forest\",1.0,12.0,1.92117802550538,0.013492603711798022,50.0,-0.9681723565286628,806],[\"random_forest\",1.0,11.0,1.6961581527204226,0.010700392249541009,30.0,-0.9682293035445686,807],[\"random_forest\",1.0,12.0,1.9850029363353523,0.010005027931762948,20.0,-0.9672604583101465,808],[\"random_forest\",1.0,11.0,1.528206645745706,0.01435822421366976,40.0,-0.9671828107925659,809],[\"random_forest\",1.0,12.0,1.834693486185519,0.011422450861533897,80.0,-0.9695382319433401,810],[\"random_forest\",1.0,11.0,2.2555932321098657,0.01274816118694452,70.0,-0.9685116998434471,811],[\"random_forest\",1.0,12.0,1.5970917208842428,0.010011907441737269,60.0,-0.9696603132360413,812],[\"random_forest\",1.0,12.0,1.4263963772277177,0.012009515223285002,70.0,-0.9683523203495299,813],[\"random_forest\",1.0,11.0,2.02850066452563,0.01327767624777379,50.0,-0.9670312608636743,814],[\"random_forest\",1.0,12.0,1.7233668305104628,0.011333505308317528,90.0,-0.9696652139751029,815],[\"random_forest\",1.0,11.0,1.8434112120876731,0.010606053906464066,30.0,-0.9683671356842096,816],[\"random_forest\",1.0,12.0,2.132579976116133,0.014162603840806604,60.0,-0.9679434434045441,817],[\"random_forest\",1.0,12.0,1.6068747503201468,0.012464833237360029,40.0,-0.9681544289342422,818],[\"random_forest\",1.0,12.0,1.4621511587071145,0.014706502131639893,70.0,-0.968637786106935,819],[\"random_forest\",1.0,11.0,1.7795696838680317,0.01100499024411095,50.0,-0.9681409924101156,820],[\"random_forest\",1.0,12.0,1.9488887744412986,0.04594328306417436,80.0,-0.9604287302271918,821],[\"random_forest\",1.0,11.0,2.063637904199271,0.013296360222544375,90.0,-0.9683771542256345,822],[\"random_forest\",1.0,12.0,1.3870982274832064,0.010022593618037763,80.0,-0.9692886336036944,823],[\"random_forest\",1.0,12.0,1.655211533000327,0.011915711714670468,60.0,-0.9693480019454497,824],[\"random_forest\",1.0,11.0,1.5279164092738735,0.01072346099848429,70.0,-0.9690024555117909,825],[\"random_forest\",1.0,12.0,2.177494561344483,0.010009162764357031,40.0,-0.9686806545804965,826],[\"random_forest\",1.0,11.0,1.8530642799491144,0.013919833580703703,50.0,-0.9677075536824602,827],[\"random_forest\",1.0,12.0,1.7470290771274795,0.012375415869049233,60.0,-0.9685891272404961,828],[\"random_forest\",1.0,11.0,2.371044020154414,0.011332575890115264,90.0,-0.9686331513468494,829],[\"random_forest\",0.0,12.0,2.266254713916741,0.015283574375849065,70.0,-0.9669198034438603,830],[\"random_forest\",1.0,12.0,1.9691259451880911,0.01305535374830783,90.0,-0.968549474972482,831],[\"random_forest\",1.0,11.0,1.8936097859800431,0.01073893268329734,40.0,-0.9682868008618037,832],[\"random_forest\",1.0,12.0,1.3372683557117184,0.01169914814541255,80.0,-0.9693421565224394,833],[\"random_forest\",1.0,11.0,2.0899047072961774,0.04071339142485943,30.0,-0.9609909302942415,834],[\"random_forest\",1.0,12.0,1.5796167990275867,0.010028010560651394,100.0,-0.9696574761269653,835],[\"random_forest\",1.0,12.0,1.4694341448758026,0.01393101082529517,20.0,-0.9681874745290736,836],[\"random_forest\",1.0,12.0,1.404877228896105,0.010000195693345534,50.0,-0.9693337093970329,837],[\"random_forest\",1.0,11.0,1.7008797894261845,0.012730110097539528,60.0,-0.9680425710173495,838],[\"random_forest\",1.0,11.0,1.8186014439887772,0.014521045486786163,80.0,-0.9677035059104594,839],[\"random_forest\",0.0,12.0,1.6543520192716707,0.011978847388171503,70.0,-0.9684914915557392,840],[\"random_forest\",1.0,12.0,2.1838954405701254,0.011357085463900823,50.0,-0.9680474595274925,841],[\"random_forest\",1.0,11.0,1.9872406341169897,0.015067162760920974,90.0,-0.9676667763539506,842],[\"random_forest\",1.0,12.0,1.2938884755344349,0.013327604295046994,40.0,-0.9693417132241464,843],[\"random_forest\",1.0,11.0,4.510966767874888,0.01104436362599939,30.0,-0.9674808173058222,844],[\"random_forest\",1.0,12.0,2.2975908224975052,0.012622533594883441,60.0,-0.9687016149466879,845],[\"random_forest\",1.0,12.0,1.50452544497056,0.010623730395263428,70.0,-0.9694054656331592,846],[\"random_forest\",1.0,11.0,1.774290811941292,0.013740816736197325,10.0,-0.9677642744633751,847],[\"random_forest\",1.0,12.0,1.8773229965624438,0.016008433685270607,100.0,-0.9685901361262667,848],[\"random_forest\",1.0,12.0,2.0806704052603875,0.012144119526635215,80.0,-0.9684278308636147,849],[\"random_forest\",0.0,11.0,1.5527754882321092,0.010018722663859269,40.0,-0.9685966419108722,850],[\"random_forest\",1.0,12.0,2.4331858646057833,0.014870781861803975,50.0,-0.9670196953640677,851],[\"random_forest\",1.0,11.0,1.6971744570602438,0.01065944163481675,110.0,-0.9692624820616289,852],[\"random_forest\",1.0,12.0,2.5355473033508438,0.011616192721758038,70.0,-0.9686980502169651,853],[\"random_forest\",1.0,11.0,1.9778008307906187,0.01309150367449185,60.0,-0.9680518069079949,854],[\"random_forest\",1.0,12.0,2.6530230122568783,0.010014427675728299,90.0,-0.969010171959321,855],[\"random_forest\",1.0,12.0,1.620594476462196,0.016833584664612773,80.0,-0.9676152528633691,856],[\"random_forest\",1.0,11.0,1.315261867453921,0.01242522177651468,60.0,-0.96818204488929,857],[\"random_forest\",1.0,12.0,1.2217064973923364,0.013794890777948137,100.0,-0.9682742601059523,858],[\"random_forest\",1.0,12.0,3.0617281134324568,0.01004247645358777,70.0,-0.9682780632995844,859],[\"random_forest\",0.0,11.0,1.9178104467016297,0.015585687029690577,90.0,-0.9662128924705515,860],[\"random_forest\",1.0,11.0,2.1503903223217566,0.011158518239147352,80.0,-0.9688980786357488,861],[\"random_forest\",1.0,12.0,3.5720443817963385,0.01442729451225595,20.0,-0.9653416309479831,862],[\"random_forest\",1.0,5.0,1.4179001202444415,0.011876726526232149,40.0,-0.9552503305155642,863],[\"random_forest\",1.0,12.0,4.774543368504096,0.010022619323427096,50.0,-0.9681577337994484,864],[\"random_forest\",1.0,12.0,4.245975365157967,0.010755337316793775,30.0,-0.9682017701347202,865],[\"random_forest\",1.0,6.0,2.0369986294856472,0.012985358057124925,60.0,-0.9634212474559494,866],[\"random_forest\",1.0,11.0,2.352287833736751,0.01153491888830209,70.0,-0.9684887522780106,867],[\"random_forest\",1.0,12.0,2.2600301282316986,0.013750712613252634,90.0,-0.9688115620950756,868],[\"random_forest\",1.0,11.0,1.7987180055770489,0.03841437531352716,100.0,-0.9627449790950002,869],[\"random_forest\",0.0,12.0,1.5660513114527956,0.010025480910498485,50.0,-0.9690810996862238,870],[\"random_forest\",1.0,12.0,1.3731877528982492,0.012486283865589105,110.0,-0.9688516362607755,871],[\"random_forest\",1.0,11.0,1.6791012855170675,0.015158186003726908,80.0,-0.9686693000297326,872],[\"random_forest\",1.0,4.0,2.9571413910018443,0.011024740917830315,60.0,-0.9497142746838011,873],[\"random_forest\",1.0,10.0,1.4752690035783973,0.011866652585458179,70.0,-0.9686932809387768,874],[\"random_forest\",1.0,12.0,1.8849120941095776,0.010004493761459076,30.0,-0.9685567083777342,875],[\"random_forest\",1.0,7.0,1.7982876318475856,0.0162783306480403,40.0,-0.9645854802346101,876],[\"random_forest\",1.0,10.0,1.7411122274136215,0.014439475352824216,20.0,-0.9651086639374042,877],[\"random_forest\",1.0,12.0,3.9407874388898527,0.04918681629842258,80.0,-0.9594493497795411,878],[\"random_forest\",1.0,11.0,2.107788092624762,0.0130620144474764,90.0,-0.9684666882519305,879],[\"random_forest\",0.0,8.0,1.6253538292850864,0.017117046221581986,50.0,-0.964452209481544,880],[\"random_forest\",1.0,11.0,2.2157845492932107,0.011047828012164262,60.0,-0.968682699867104,881],[\"random_forest\",1.0,2.0,2.0123763682930678,0.012300332747996565,100.0,-0.9235552148904986,882],[\"random_forest\",1.0,12.0,2.7285945788047887,0.013851176413254075,80.0,-0.9685214738065034,883],[\"random_forest\",1.0,12.0,3.154908591138674,0.010720265944271805,70.0,-0.9690412334121371,884],[\"random_forest\",1.0,12.0,1.2618351468828413,0.028427390028130467,110.0,-0.9656847682847108,885],[\"random_forest\",1.0,11.0,2.5951244681792125,0.015363110816036704,50.0,-0.9677024756240816,886],[\"random_forest\",1.0,12.0,3.3636074126560636,0.010044946589795219,40.0,-0.9682743457083812,887],[\"random_forest\",1.0,12.0,1.5331672361816333,0.01174889973841164,70.0,-0.9692988386361256,888],[\"random_forest\",1.0,11.0,1.9226802110569121,0.012698010859803367,90.0,-0.9680827644150042,889],[\"random_forest\",0.0,12.0,1.4322548149150376,0.018237693711382084,60.0,-0.9663136709873454,890],[\"random_forest\",1.0,10.0,2.384548860390496,0.013420331580364734,30.0,-0.9678360337567308,891],[\"random_forest\",1.0,11.0,1.6876922523571134,0.010687191385244198,80.0,-0.9692182011479885,892],[\"random_forest\",1.0,12.0,1.3428668477469898,0.014384577210702052,50.0,-0.96884232393939,893],[\"random_forest\",1.0,11.0,1.861106437793445,0.010031813181611306,60.0,-0.9693155188808662,894],[\"random_forest\",1.0,12.0,1.2105544629411988,0.011393463363633373,100.0,-0.9692845552593976,895],[\"random_forest\",1.0,12.0,3.469613905452045,0.048114561927212474,40.0,-0.9604824029501875,896],[\"random_forest\",1.0,11.0,1.7484168893282004,0.016232232395504684,70.0,-0.9673308479646381,897],[\"random_forest\",1.0,12.0,1.6008531543417728,0.012109905032897089,90.0,-0.968562293936228,898],[\"random_forest\",1.0,11.0,2.0341843220898053,0.013312179623107339,80.0,-0.9686478688501817,899],[\"random_forest\",0.0,12.0,2.121096729068577,0.010018233232093107,70.0,-0.9685509699577601,900],[\"random_forest\",1.0,3.0,1.5090830763191434,0.010012225818850851,90.0,-0.9372578158243073,901],[\"random_forest\",1.0,12.0,2.5224947080213815,0.015197431690216465,50.0,-0.9674656137029823,902],[\"random_forest\",1.0,11.0,2.2772745749309338,0.011433179570447292,60.0,-0.9688302723402757,903],[\"random_forest\",1.0,10.0,1.9513497753725635,0.012501006057605665,10.0,-0.9677353775291215,904],[\"random_forest\",1.0,12.0,1.8013903347876603,0.010016215206828312,120.0,-0.9697435860561004,905],[\"random_forest\",1.0,11.0,3.6575716993868914,0.01743308247075767,70.0,-0.9673768439840885,906],[\"random_forest\",1.0,12.0,4.974558766931807,0.014212883690555393,40.0,-0.9666088159336889,907],[\"random_forest\",1.0,12.0,1.4138895769771231,0.010938955578794419,110.0,-0.9686805445202304,908],[\"random_forest\",1.0,11.0,2.209867900616781,0.013050995076980737,30.0,-0.9684747287657992,909],[\"random_forest\",0.0,6.0,1.279154620740106,0.01207813067894748,80.0,-0.9639210616956212,910],[\"random_forest\",1.0,10.0,1.6109101691261518,0.0424078354873403,50.0,-0.9598633353552015,911],[\"random_forest\",1.0,12.0,1.1558684486103035,0.01928561617732865,100.0,-0.9666243558317856,912],[\"random_forest\",1.0,12.0,1.7013945078090615,0.015973006155690507,60.0,-0.9681229272403629,913],[\"random_forest\",1.0,11.0,2.460139453737798,0.014810046161605549,90.0,-0.9670753583435382,914],[\"random_forest\",1.0,11.0,2.872023436565987,0.013742590308172964,40.0,-0.9685261269099669,915],[\"random_forest\",1.0,7.0,1.8690483730401684,0.010788106548663808,20.0,-0.9654611991979469,916],[\"random_forest\",1.0,12.0,1.5333336623010958,0.011430056484878652,80.0,-0.9688293093129494,917],[\"random_forest\",1.0,12.0,2.0539626804099513,0.03607954970445592,50.0,-0.9623365974212108,918],[\"random_forest\",1.0,12.0,2.3235127210665123,0.012618306339539231,60.0,-0.9687016149466879,919],[\"random_forest\",0.0,11.0,1.46726262768798,0.010024619304878819,70.0,-0.9691003143743091,920],[\"random_forest\",1.0,8.0,1.3705697923653843,0.029441764851976374,30.0,-0.9644732921369199,921],[\"random_forest\",1.0,12.0,1.9517797638663663,0.010735998694523867,100.0,-0.9694552403884048,922],[\"random_forest\",1.0,11.0,1.8194708141863158,0.016691122629216112,80.0,-0.9672218393857399,923],[\"random_forest\",1.0,10.0,2.1639416271916527,0.011521272498431694,70.0,-0.9681754412733371,924],[\"random_forest\",1.0,12.0,1.6541764514489523,0.010001952782221631,110.0,-0.9693853123755842,925],[\"random_forest\",1.0,11.0,3.2336967423867837,0.01329399636108915,90.0,-0.9683136219371535,926],[\"random_forest\",1.0,12.0,1.7282892353099755,0.014861933071036573,50.0,-0.9680163583306917,927],[\"random_forest\",1.0,12.0,2.8085062051996035,0.03465761598707075,60.0,-0.9623921778554819,928],[\"random_forest\",1.0,11.0,2.013869907906401,0.02397344178012514,40.0,-0.9655459425454473,929],[\"random_forest\",0.0,12.0,1.3019147358361935,0.012074477449045374,20.0,-0.9664124378470571,930],[\"random_forest\",1.0,11.0,1.5575314082055423,0.013893751464106052,90.0,-0.9686326071599791,931],[\"random_forest\",1.0,12.0,1.1063622246131692,0.010822404097571927,80.0,-0.9692586819252262,932],[\"random_forest\",1.0,12.0,1.2072398613439583,0.012884398595077111,100.0,-0.9682425016047856,933],[\"random_forest\",1.0,10.0,1.4621474540891748,0.015573835899942204,70.0,-0.9668969659386896,934],[\"random_forest\",1.0,9.0,1.9001215201248889,0.011938808905577589,60.0,-0.9673088359114616,935],[\"random_forest\",1.0,11.0,2.6531565771561003,0.01422245758830681,50.0,-0.9668506978258041,936],[\"random_forest\",1.0,12.0,1.7490618513844285,0.0267729005125453,80.0,-0.9655154436228792,937],[\"random_forest\",1.0,11.0,2.4083580728877463,0.010703897744676174,120.0,-0.9690234586791968,938],[\"random_forest\",1.0,12.0,1.6218652338281925,0.012536232840307887,40.0,-0.9685104463793079,939],[\"random_forest\",0.0,11.0,2.114216312328839,0.03288630003485144,70.0,-0.9598601527791798,940],[\"random_forest\",1.0,12.0,2.240007218046965,0.017433585468937036,90.0,-0.9678205458315375,941],[\"random_forest\",1.0,12.0,1.8219047805030013,0.011245793110302693,100.0,-0.9693624809848725,942],[\"random_forest\",1.0,11.0,1.3328981991001643,0.015769529237423793,30.0,-0.9676684792308421,943],[\"random_forest\",1.0,12.0,1.4172666521540966,0.013447288990717567,50.0,-0.9676831753335671,944],[\"random_forest\",1.0,8.0,1.970601278484792,0.010019823538853261,10.0,-0.9633626189637592,945],[\"random_forest\",1.0,11.0,3.0022320935291313,0.018295636826168908,60.0,-0.9676454246623691,946],[\"random_forest\",1.0,5.0,1.5058263128083689,0.01177098821767844,70.0,-0.9589284681706415,947],[\"random_forest\",1.0,10.0,1.6758146658572641,0.010721910913228409,90.0,-0.9691728012883116,948],[\"random_forest\",1.0,12.0,1.5840090708074348,0.046567863373678055,80.0,-0.9610051739269847,949],[\"random_forest\",0.0,12.0,2.5749464920286225,0.01464612446610357,50.0,-0.967420663256058,950],[\"random_forest\",1.0,12.0,2.3370652182310403,0.010020239729466583,60.0,-0.9691592241316231,951],[\"random_forest\",1.0,11.0,2.0857371695134943,0.01280455127422387,100.0,-0.968630931798154,952],[\"random_forest\",1.0,12.0,1.0592890513036706,0.016620284716650315,110.0,-0.9678635488232018,953],[\"random_forest\",1.0,11.0,1.8728333846376024,0.011565355012743576,40.0,-0.9682614594855841,954],[\"random_forest\",1.0,10.0,1.729430760103601,0.013865895682682546,80.0,-0.9673640280775724,955],[\"random_forest\",1.0,12.0,1.2600700599919743,0.039615608336381475,70.0,-0.9607921461679084,956],[\"random_forest\",1.0,11.0,2.0017667669023917,0.02295251085856518,40.0,-0.9655051346446413,957],[\"random_forest\",1.0,12.0,2.1697638681539924,0.01076382719791267,50.0,-0.9678603540182614,958],[\"random_forest\",1.0,12.0,2.7393538179453207,0.010015158373000274,60.0,-0.9690718026509861,959],[\"random_forest\",0.0,11.0,1.8123784038935007,0.01264770364451151,80.0,-0.9675308855551106,960],[\"random_forest\",1.0,12.0,1.1797142519061716,0.011996546808757336,30.0,-0.9688207307266696,961],[\"random_forest\",1.0,12.0,1.3685783457074405,0.014905454100469874,90.0,-0.9684633650433468,962],[\"random_forest\",1.0,11.0,2.482610915379292,0.010039405385124676,70.0,-0.968637465097826,963],[\"random_forest\",1.0,12.0,1.9186809251588237,0.030665996978995844,60.0,-0.9633318999206594,964],[\"random_forest\",1.0,12.0,3.1151524098310954,0.02189507200046751,100.0,-0.966173594841172,965],[\"random_forest\",1.0,10.0,1.5013128279512091,0.013237272629424233,20.0,-0.9659503651068944,966],[\"random_forest\",1.0,11.0,1.6535421083764554,0.03729598909221474,50.0,-0.9618031108546381,967],[\"random_forest\",1.0,12.0,1.5815968551476396,0.011485481792956118,30.0,-0.9689424145795219,968],[\"random_forest\",1.0,9.0,1.7601843560606767,0.010779773239977895,40.0,-0.9677863018026998,969],[\"random_forest\",0.0,11.0,4.024086733898119,0.01561948609368153,90.0,-0.9667409371684225,970],[\"random_forest\",1.0,10.0,2.2778294799426826,0.014339701196935463,110.0,-0.9676644834317446,971],[\"random_forest\",1.0,12.0,4.408819129714874,0.010040041779133951,80.0,-0.9684917881070112,972],[\"random_forest\",1.0,3.0,2.0702388904442173,0.012414012513479532,70.0,-0.9358518601294709,973],[\"random_forest\",1.0,11.0,1.4296960419178293,0.013353542001026614,60.0,-0.9676236724737093,974],[\"random_forest\",1.0,12.0,3.3019586549871134,0.017719894178893896,50.0,-0.9662005045761806,975],[\"random_forest\",1.0,12.0,2.2006546114918093,0.011276328624796066,80.0,-0.9687298423476574,976],[\"random_forest\",1.0,12.0,1.9430227907772213,0.012215383457112062,70.0,-0.9684489318623682,977],[\"random_forest\",1.0,11.0,1.1213340724914507,0.016790883269533492,90.0,-0.9681078336977887,978],[\"random_forest\",1.0,12.0,1.6890255174659066,0.018997108701714317,40.0,-0.966194341201291,979],[\"random_forest\",0.0,11.0,1.3231209726399402,0.010745214341044272,100.0,-0.968089572865341,980],[\"random_forest\",1.0,12.0,1.2462803680800465,0.013920479696257686,60.0,-0.9681676178227705,981],[\"random_forest\",1.0,11.0,1.8407865590495855,0.010042482572985668,50.0,-0.9692478012450518,982],[\"random_forest\",1.0,12.0,1.553191843701727,0.02523166604686803,30.0,-0.965561265380242,983],[\"random_forest\",1.0,10.0,2.131913516324226,0.015174217162062282,90.0,-0.9672873344156293,984],[\"random_forest\",1.0,7.0,2.4000528708587257,0.011508073989221258,70.0,-0.9658368776866207,985],[\"random_forest\",1.0,4.0,1.7546061903550896,0.013143400847057223,80.0,-0.9502073079882027,986],[\"random_forest\",1.0,12.0,2.0157736410678226,0.015979754351599974,120.0,-0.9675876216221733,987],[\"random_forest\",1.0,11.0,1.44193745911186,0.020684591384226,110.0,-0.9662801148351694,988],[\"random_forest\",1.0,12.0,1.6295834398633446,0.03384627985799891,20.0,-0.96247210606635,989],[\"random_forest\",0.0,11.0,1.8849036693025485,0.010031251636003458,60.0,-0.9685122256869398,990],[\"random_forest\",1.0,12.0,1.0681072901690438,0.012456359561560965,80.0,-0.968555595546157,991],[\"random_forest\",1.0,12.0,2.680614593060377,0.010033760745050016,40.0,-0.9686794072308164,992],[\"random_forest\",1.0,5.0,1.5161218491405273,0.014029487942422545,90.0,-0.9583189574754097,993],[\"random_forest\",1.0,11.0,1.7993453750655413,0.01163394796425894,70.0,-0.9683340809176895,994],[\"random_forest\",1.0,6.0,1.190128579483869,0.010769614239335833,10.0,-0.9559739492486801,995],[\"random_forest\",1.0,12.0,1.3883771544797878,0.027711361853029748,100.0,-0.9644778932674797,996],[\"random_forest\",1.0,11.0,2.3016043954311565,0.013089015132369663,50.0,-0.967677733464865,997],[\"random_forest\",1.0,12.0,2.047157455155254,0.014601292038352193,60.0,-0.9685262430846919,998],[\"random_forest\",1.0,12.0,2.5013415139734616,0.010004104466533824,80.0,-0.9686531150561887,999]]}],                        {\"template\":{\"data\":{\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"geo\":{\"bgcolor\":\"white\",\"lakecolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"white\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"light\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"ternary\":{\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"trial_number\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"loss\"}},\"legend\":{\"title\":{\"text\":\"model_choice\"},\"tracegroupgap\":0},\"margin\":{\"t\":60}},                        {\"responsive\": true}                    )                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {
      "application/vnd.databricks.v1+output": {
       "addedWidgets": {},
       "arguments": {},
       "data": "<html>\n<head><meta charset=\"utf-8\" /></head>\n<body>\n    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n        <script src=\"https://cdn.plot.ly/plotly-2.9.0.min.js\"></script>                <div id=\"ba448efd-2045-4980-8572-4a5f9e6da966\" class=\"plotly-graph-div\" style=\"height:100%; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"ba448efd-2045-4980-8572-4a5f9e6da966\")) {                    Plotly.newPlot(                        \"ba448efd-2045-4980-8572-4a5f9e6da966\",                        [{\"hovertemplate\":\"rf_criterion: %{customdata[1]}<br>rf_max_depth: %{customdata[2]}<br>rf_min_samples_leaf: %{customdata[3]}<br>rf_min_samples_split: %{customdata[4]}<br>rf_n_estimators: %{customdata[5]}\",\"legendgroup\":\"random_forest\",\"marker\":{\"color\":\"#636efa\",\"symbol\":\"circle\"},\"mode\":\"markers\",\"name\":\"random_forest\",\"orientation\":\"v\",\"showlegend\":true,\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512,513,514,515,516,517,518,519,520,521,522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542,543,544,545,546,547,548,549,550,551,552,553,554,555,556,557,558,559,560,561,562,563,564,565,566,567,568,569,570,571,572,573,574,575,576,577,578,579,580,581,582,583,584,585,586,587,588,589,590,591,592,593,594,595,596,597,598,599,600,601,602,603,604,605,606,607,608,609,610,611,612,613,614,615,616,617,618,619,620,621,622,623,624,625,626,627,628,629,630,631,632,633,634,635,636,637,638,639,640,641,642,643,644,645,646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,664,665,666,667,668,669,670,671,672,673,674,675,676,677,678,679,680,681,682,683,684,685,686,687,688,689,690,691,692,693,694,695,696,697,698,699,700,701,702,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,724,725,726,727,728,729,730,731,732,733,734,735,736,737,738,739,740,741,742,743,744,745,746,747,748,749,750,751,752,753,754,755,756,757,758,759,760,761,762,763,764,765,766,767,768,769,770,771,772,773,774,775,776,777,778,779,780,781,782,783,784,785,786,787,788,789,790,791,792,793,794,795,796,797,798,799,800,801,802,803,804,805,806,807,808,809,810,811,812,813,814,815,816,817,818,819,820,821,822,823,824,825,826,827,828,829,830,831,832,833,834,835,836,837,838,839,840,841,842,843,844,845,846,847,848,849,850,851,852,853,854,855,856,857,858,859,860,861,862,863,864,865,866,867,868,869,870,871,872,873,874,875,876,877,878,879,880,881,882,883,884,885,886,887,888,889,890,891,892,893,894,895,896,897,898,899,900,901,902,903,904,905,906,907,908,909,910,911,912,913,914,915,916,917,918,919,920,921,922,923,924,925,926,927,928,929,930,931,932,933,934,935,936,937,938,939,940,941,942,943,944,945,946,947,948,949,950,951,952,953,954,955,956,957,958,959,960,961,962,963,964,965,966,967,968,969,970,971,972,973,974,975,976,977,978,979,980,981,982,983,984,985,986,987,988,989,990,991,992,993,994,995,996,997,998,999],\"xaxis\":\"x\",\"y\":[-0.9626329836027753,-0.9562822708546335,-0.9624956192192642,-0.945237044468279,-0.9656749484632106,-0.9586555859703025,-0.9633421966699789,-0.9580469526999682,-0.9505051218959941,-0.9619859576430254,-0.9618390852754338,-0.9656506465450575,-0.9488812774470049,-0.967084334369667,-0.9374047554509501,-0.9587348507623457,-0.9634928141438399,-0.9660973781070481,-0.9470437723924107,-0.9212283329781137,-0.9683647510451154,-0.9675297360367779,-0.960723908803061,-0.9675501461016399,-0.9684950532282324,-0.9639596439332724,-0.9661501917485239,-0.9666057953908362,-0.9644503476287128,-0.9659897116519477,-0.9677541091749289,-0.9671651277765141,-0.9670907881813691,-0.9625594083150323,-0.9566308470028337,-0.9682401383662989,-0.9529344362863903,-0.9658821460854244,-0.9643388565793731,-0.9654529385635462,-0.9629872156257498,-0.9552149858840677,-0.9641748728976662,-0.9634675033399163,-0.9656062708572993,-0.9673041522357023,-0.964779736603894,-0.9611787450807417,-0.9233628937475564,-0.9648019137474695,-0.9663002528065964,-0.9369208968926019,-0.9630745943051725,-0.9508608550187063,-0.9576589382898265,-0.9610656795581541,-0.9633246145425038,-0.9626890929377685,-0.9652753991257583,-0.9632069112026009,-0.9618518155795212,-0.9625740432731651,-0.9369199705520306,-0.96815187309029,-0.9682601295907045,-0.9666819173508425,-0.9685657516629143,-0.9655268990622199,-0.9663511373361898,-0.9674739691115007,-0.9668579037159899,-0.9672645030249177,-0.9682469070726505,-0.9659472192176279,-0.9648739237336531,-0.9651323299517989,-0.9644082801493084,-0.9656319699293833,-0.9621477768061699,-0.963675725134049,-0.9652765425296317,-0.9488435604053326,-0.9635082745539669,-0.9604443434987989,-0.967984009784211,-0.9654042277242042,-0.9655790339987229,-0.961942606127186,-0.9590642917104301,-0.9631354698611243,-0.9573319033814434,-0.9656298818415607,-0.9650419123861462,-0.9625710716459864,-0.9146365607326213,-0.9689232365781918,-0.9581106164493225,-0.9609286300692926,-0.9647325207498301,-0.9623183274170743,-0.9668207308611878,-0.9688700866414591,-0.9661846834129597,-0.9682822425324583,-0.9654526175544375,-0.9616896662355789,-0.9662189396707159,-0.9674226871420585,-0.9630372043870682,-0.9687808736242718,-0.9675697001422119,-0.9656123364008413,-0.9683667932744935,-0.9673747528390366,-0.949407426662519,-0.9693741932315002,-0.9686894655162263,-0.9557833187537109,-0.936386279151074,-0.9681978538235925,-0.9682305692376263,-0.9665499612065077,-0.9664636800725145,-0.96934113846498,-0.967575203155506,-0.9669742160164212,-0.9660269548230307,-0.9659529301225354,-0.9630913846101787,-0.9671429781480052,-0.9671431432384038,-0.9674184589935106,-0.9678378772661845,-0.9620385572784287,-0.9638508982761207,-0.9215895813430386,-0.9668926552449424,-0.9684408791195809,-0.9627423284769303,-0.9680765337810633,-0.9638720818200734,-0.9622389953659797,-0.9638861450762696,-0.9626306692799621,-0.9374083935541836,-0.9688700866414591,-0.9687817051907253,-0.9670809500164911,-0.968057591186413,-0.969253279800509,-0.9693086401142484,-0.9676951474447115,-0.9684512308990332,-0.9686964665720282,-0.9677368113698076,-0.9682189242500497,-0.9686358050221491,-0.967246483713609,-0.9686434389244798,-0.9672374312567401,-0.968734764487326,-0.9682407253543834,-0.9682886504857164,-0.9686413600083464,-0.9683928959008855,-0.9676248984227821,-0.9549889496130102,-0.9669017352168778,-0.9665485640525769,-0.9684274762249802,-0.9654107824244834,-0.9670577273003895,-0.9657223324649029,-0.9611665314484584,-0.9497717842299549,-0.9628851072140768,-0.9663152148882974,-0.9682951226407963,-0.964086861371715,-0.9678818891436193,-0.963521940370314,-0.969260773070278,-0.9685741590443359,-0.9607952186836644,-0.9687648476266674,-0.9676757676662272,-0.9671339868357283,-0.9677879557639175,-0.9672487307773708,-0.9665786624780663,-0.9688044020633342,-0.9683169848897223,-0.968778253578498,-0.9639458221982152,-0.9686192745816593,-0.9684864165545901,-0.9593628485250161,-0.9678945735892622,-0.9507725438842534,-0.9648038214587447,-0.9624908407693871,-0.9665021063914558,-0.9688752411305779,-0.9670362777774608,-0.9568500167361921,-0.9685735078544295,-0.9662170869895736,-0.966559080922428,-0.9694597956605205,-0.9673544161476851,-0.9677629353968068,-0.9631918207172566,-0.966937238824314,-0.9623634215537902,-0.9658879059060056,-0.9675791408672411,-0.9189801502702948,-0.9666789640670415,-0.9598055506583831,-0.9368604371198765,-0.967861503536594,-0.9647142446312342,-0.9677968614737653,-0.9679656450059565,-0.9634196699254718,-0.9680982676263459,-0.96108403210749,-0.966411517620945,-0.9649561204088903,-0.9684967897346497,-0.9658957599288683,-0.9657574814337047,-0.9687617903970596,-0.966797153506452,-0.9682485121181946,-0.9440580422133826,-0.9524498256495584,-0.9686434389244798,-0.9627558139167305,-0.9669078221610272,-0.9671410429216633,-0.9678298948396784,-0.9683465788723262,-0.96721380804356,-0.9685733244206529,-0.9693327494269361,-0.9651488756784367,-0.9659959025419036,-0.9673623649446655,-0.9668249834675723,-0.9690074296243629,-0.9678912809529747,-0.9686670193364454,-0.9567239274154747,-0.9633268707779543,-0.9372913689192537,-0.9584315063261938,-0.9697555214804895,-0.969868006129452,-0.9688026044123246,-0.9688481999346964,-0.9688526971194495,-0.9670827843542559,-0.9696022167018031,-0.9684125019143608,-0.9690926498996824,-0.9695863099361534,-0.9681676178227705,-0.9681884039268746,-0.9689105460180895,-0.9686680404511343,-0.9696699801960615,-0.9685011096000856,-0.9680393609262613,-0.9649045571743238,-0.9681894739572373,-0.9677743174626368,-0.9667024833344147,-0.9680685849840829,-0.9641423378601794,-0.9683534117804999,-0.9680928379865621,-0.96778130934675,-0.9645058027165699,-0.9680569889121804,-0.9676866728042385,-0.9658397147956967,-0.9666988758034775,-0.9677680501419409,-0.9665453142175039,-0.9616707511559951,-0.9670927570372365,-0.9673898616677588,-0.9672751605273308,-0.9692554962919749,-0.9666264561485263,-0.9640978245970888,-0.9646037471815171,-0.9679175242119286,-0.9661959584757536,-0.9674509848593089,-0.9677816609281551,-0.9637531403021794,-0.9668268116908778,-0.962779238409986,-0.9661066506844486,-0.9652692602087056,-0.9663758947815542,-0.9642556815906613,-0.969587927210616,-0.9636899320800367,-0.9680575392135098,-0.9687362136141602,-0.9242872287761174,-0.9665582921571891,-0.9678829683456709,-0.9696522115775806,-0.9627747412252329,-0.9687226425719309,-0.9666827428028366,-0.9651150260322185,-0.9614771796061434,-0.9687081513035897,-0.9677038850069307,-0.9610118172869225,-0.9693421565224394,-0.9658215059361525,-0.948815779359886,-0.9690903539202467,-0.9684220007267523,-0.9664832830287602,-0.9689190359447106,-0.9687922190033468,-0.9655590580604652,-0.9629066331677459,-0.9651195446175789,-0.9665022347950993,-0.9599157760146654,-0.9671932940328912,-0.9689351750598105,-0.9687159350101712,-0.937493152187832,-0.9681693023562846,-0.9682908425193453,-0.9659806011077163,-0.9647152657459233,-0.957751829154232,-0.9679961592146725,-0.9664924791754207,-0.9688789036916482,-0.9660094246684592,-0.96516849697806,-0.9665345833415802,-0.9659101747664695,-0.9650261615392065,-0.9689919722714655,-0.9672739498644061,-0.9605780575501583,-0.9598366610268729,-0.9679412850004409,-0.9676333210903515,-0.9686662550290434,-0.9658808161905452,-0.9648658495902588,-0.9648544155515254,-0.9679239260507274,-0.968376441891136,-0.9679786382317899,-0.969861589004505,-0.9679965505400624,-0.9667849521030869,-0.9696912493424434,-0.9656319301853982,-0.9662663114434897,-0.9674751247442925,-0.9653639945825645,-0.954707708946923,-0.9642175854525178,-0.9566025798578793,-0.9659751592390143,-0.9625173469500873,-0.9631023111487973,-0.9616393289500854,-0.9687144522538115,-0.9665359438087558,-0.9694969929731596,-0.9679049620554698,-0.9631333206287099,-0.9671740029140657,-0.9672849344903872,-0.9635474254363251,-0.9636620532032426,-0.946103310477668,-0.9653341988228064,-0.968152239957843,-0.9657927312910832,-0.9667126363939424,-0.9220654055019775,-0.9686934704870126,-0.967531854696896,-0.9683826480672401,-0.96889573068341,-0.9696924386047608,-0.9692886336036944,-0.9698118142492593,-0.9690616740492952,-0.9691820066066608,-0.9688924686194182,-0.9689662028831008,-0.9685690871004164,-0.968426721089267,-0.969342767968361,-0.9688166554396024,-0.9690676448187195,-0.9697220662168907,-0.969287765350486,-0.9682267293572389,-0.969315029724129,-0.9688645163691135,-0.9683723299173133,-0.9687162927060353,-0.9679529789036909,-0.968396494260134,-0.9682971342978782,-0.9693767001597788,-0.9687398853469192,-0.9685418380129216,-0.9681752547823311,-0.9670655966094004,-0.9680463925543594,-0.9686059511750281,-0.9679499247313128,-0.9687055801734893,-0.9691952841548479,-0.9691155454922157,-0.9689531637988233,-0.9671815420422785,-0.9678256024893089,-0.9674240904104483,-0.9689496877287591,-0.9685124488647011,-0.9692142787224016,-0.9690502400105618,-0.9673880793028973,-0.9393017969949279,-0.9681861201763574,-0.9684508823748578,-0.9689897924667552,-0.9668919367959846,-0.9691466986619197,-0.9686955127163906,-0.9649026005473749,-0.9659254150560646,-0.9676157511917951,-0.9681859612004178,-0.9686470281120395,-0.9686820547916568,-0.9675205704624136,-0.9691714866795803,-0.9669702935908342,-0.9674662220916745,-0.9692147067345467,-0.9665472219287792,-0.9479917612064478,-0.9679596772937618,-0.9611150660452394,-0.9571367237288178,-0.9692835586025453,-0.9686633445464566,-0.9688323787714757,-0.9697028484715756,-0.9687502738131267,-0.9619866883209017,-0.9675096989539279,-0.9670993973399449,-0.9669599846125965,-0.968249679979905,-0.9675658358039875,-0.9686655518662335,-0.9603213664378227,-0.9617440971515174,-0.9670302642068221,-0.9687243729638889,-0.9651170621471372,-0.9696874430915816,-0.9684319856386517,-0.9622578248431347,-0.962503720877725,-0.9672436863485179,-0.9684195732864438,-0.9636437006539066,-0.9267359535172941,-0.9694488446640652,-0.9681720263478651,-0.967917802419823,-0.9632309746568444,-0.966905507838214,-0.9677749472519361,-0.9697045360623193,-0.9600397161029682,-0.9621646496563756,-0.9687274240790374,-0.9669106653845624,-0.9688864305909427,-0.9639918243321248,-0.963718293999109,-0.9653004867519205,-0.9671660021441819,-0.9661510141432884,-0.9691458701526958,-0.9685985282215402,-0.9679763575385023,-0.9666226040392203,-0.9660564142875321,-0.9676707904964257,-0.966836640684067,-0.9580104310350726,-0.9690710444580435,-0.9692994867688024,-0.962648126061023,-0.9601559886594146,-0.9673247090475855,-0.9690730622295847,-0.9676021526344994,-0.9689102097228327,-0.9629143863020314,-0.9637988489420466,-0.9380922285584121,-0.9638753530557539,-0.9666346709244825,-0.9684298944936001,-0.967258825749536,-0.9682681762190325,-0.9687233059907558,-0.9660870844149582,-0.9691369032982562,-0.9652381223251496,-0.9667944967739225,-0.9667070049770048,-0.9680575392135098,-0.9694962592380537,-0.9479252695197063,-0.967225229853375,-0.9683282385519086,-0.9636918764780674,-0.9669984445610635,-0.9674845990988473,-0.9631970577515748,-0.9674133564772953,-0.9693871405988896,-0.9664312336946862,-0.965748254714748,-0.9685024578383427,-0.9647969060053717,-0.9575989187581649,-0.9680825259510948,-0.9652909787678399,-0.9656563910794906,-0.9697249614133294,-0.9670003217000428,-0.9682404379748004,-0.9668665709619281,-0.9686391129445847,-0.9691711840138493,-0.9686047894277773,-0.968799700044197,-0.965610624352261,-0.966596363837496,-0.9591561920324427,-0.9675684313919246,-0.9689965092002036,-0.9667562172020024,-0.9669427999249706,-0.9676056684485485,-0.9677614373542988,-0.961546040645831,-0.9694947183943312,-0.9696202604709487,-0.9691475210566841,-0.9691242983405827,-0.9692140677735586,-0.9683235365327718,-0.9695771321328706,-0.9688467110638773,-0.9684304142226331,-0.9690138406348506,-0.9691607313458198,-0.9680260038901046,-0.9692492778869525,-0.967677733464865,-0.9684772173507001,-0.9691871580385503,-0.9681715157905205,-0.9684328600063195,-0.9695390237658085,-0.9686260371735519,-0.9688754795944873,-0.9666451511075783,-0.9693185027369633,-0.9696621445165765,-0.9698499745892246,-0.9678027527552197,-0.9683404796992587,-0.968975768954544,-0.967634696843675,-0.968503289404796,-0.9685001893739736,-0.9682990939820568,-0.9684164793700807,-0.9671735107000987,-0.969420708979984,-0.9683830118775634,-0.9690019052104615,-0.9679064356401409,-0.9685128401900911,-0.9674720369423887,-0.9691447114626746,-0.9690245990258407,-0.9689372295181071,-0.968266503914437,-0.9682744741120246,-0.9672192499122622,-0.9691339347283068,-0.9684156875476121,-0.9677661485451248,-0.9682443695720762,-0.9699429785711257,-0.9687972940044959,-0.9673936679186206,-0.9683568114198239,-0.9671762683212051,-0.9689330502852331,-0.9681618580021893,-0.9366612494392353,-0.9684478679464648,-0.9646469336069579,-0.9658315214203478,-0.9688752227872003,-0.9670996663761504,-0.9677026498861692,-0.9681151343620923,-0.9678272381071489,-0.9673159623136773,-0.9612073852077083,-0.9684968722798492,-0.9685135341812121,-0.9690094749109707,-0.9679029381694694,-0.9690559814877654,-0.9685683717086884,-0.9670941419622491,-0.9686750812509212,-0.967031263920904,-0.9690263997340797,-0.9638000382043641,-0.9659511691582813,-0.9678287697791825,-0.9676820380441531,-0.9662831048057259,-0.9694697347139756,-0.9677908784754228,-0.9691619236653668,-0.9681153177958689,-0.967688727262535,-0.9663372238842445,-0.9683582850044947,-0.9679967828895125,-0.9546946637481863,-0.9679567423533383,-0.9653806625983868,-0.9604675448142932,-0.9683571202000142,-0.9690526521647224,-0.9595650384051322,-0.967811918329584,-0.9647865511686899,-0.9667437131329065,-0.9684389897516833,-0.9684990979430036,-0.9596102150870478,-0.9624546370563708,-0.967845419451627,-0.9514851688196991,-0.9669038080185519,-0.9641688470981092,-0.9682773295644784,-0.9109632963015393,-0.9617204250226636,-0.9675961574072385,-0.9690718026509861,-0.966711804827489,-0.9666917952597055,-0.9678390695857315,-0.9683582850044947,-0.9685457054083756,-0.966732792708747,-0.9669868087451758,-0.9684363269046949,-0.9649431057824496,-0.9665733856997631,-0.9684572444696719,-0.9638640076766791,-0.9682002201193088,-0.9698153881506708,-0.9652615682190122,-0.95919714362304,-0.9667181149493997,-0.9687616742223345,-0.9679203827216121,-0.9678547776314567,-0.9672325794333525,-0.9683003871901812,-0.9673320525131035,-0.9671075570857683,-0.9679496495806481,-0.9638736532360919,-0.9686184797019611,-0.9675027437565701,-0.9679782744214663,-0.9663928960354036,-0.9697558027456137,-0.9675402712500066,-0.9658799234794996,-0.961504743588288,-0.9649393729050985,-0.9685707563477823,-0.9689654752624541,-0.9688335558048747,-0.9675110502494144,-0.9207098940956708,-0.9681485223666397,-0.9682124643238884,-0.9648607990469465,-0.9688552713067793,-0.9662846884506627,-0.9680001825288365,-0.9682419941046708,-0.968882046523685,-0.9628990451238592,-0.9690911487999448,-0.9484820460614319,-0.9682656020317028,-0.9671057227480035,-0.9647609499279536,-0.9688796955141166,-0.9687338778907397,-0.9678462601897693,-0.9681034221154645,-0.960767303120115,-0.9676602002530639,-0.9409223121208741,-0.9675605223389291,-0.9682890846123209,-0.9627494732225238,-0.9685388694429723,-0.965652062042366,-0.958279934996695,-0.9634702548465635,-0.9614151117306444,-0.9672192223971956,-0.9696229141462485,-0.9613388827676019,-0.9684787245648969,-0.9685741590443357,-0.9686868271270747,-0.9680739106780597,-0.9676895832868252,-0.968733345932788,-0.9640935261322602,-0.9685849663509997,-0.9689603360594832,-0.9676373230039084,-0.9682980575812199,-0.9677413727563825,-0.9666637604642014,-0.967213178254261,-0.9686818774723396,-0.9649759373712086,-0.9686921864505772,-0.9679935085966026,-0.9657050530031591,-0.9681370027254772,-0.9615712933623919,-0.969070408554285,-0.9668798699107224,-0.9700942778071894,-0.9688376188630236,-0.9683654052922517,-0.9694100423058821,-0.9691749505207259,-0.9679238526772169,-0.9694186117204731,-0.9682152647462092,-0.9691492973070865,-0.9690298758041439,-0.9693440642337147,-0.9693464977884827,-0.968637948140104,-0.967974645489922,-0.9689598683033532,-0.9688119962216798,-0.9694007422134151,-0.9691177436403036,-0.9696621445165765,-0.9693362866415925,-0.9682873695065108,-0.9681723565286628,-0.9682293035445686,-0.9672604583101465,-0.9671828107925659,-0.9695382319433401,-0.9685116998434471,-0.9696603132360413,-0.9683523203495299,-0.9670312608636743,-0.9696652139751029,-0.9683671356842096,-0.9679434434045441,-0.9681544289342422,-0.968637786106935,-0.9681409924101156,-0.9604287302271918,-0.9683771542256345,-0.9692886336036944,-0.9693480019454497,-0.9690024555117909,-0.9686806545804965,-0.9677075536824602,-0.9685891272404961,-0.9686331513468494,-0.9669198034438603,-0.968549474972482,-0.9682868008618037,-0.9693421565224394,-0.9609909302942415,-0.9696574761269653,-0.9681874745290736,-0.9693337093970329,-0.9680425710173495,-0.9677035059104594,-0.9684914915557392,-0.9680474595274925,-0.9676667763539506,-0.9693417132241464,-0.9674808173058222,-0.9687016149466879,-0.9694054656331592,-0.9677642744633751,-0.9685901361262667,-0.9684278308636147,-0.9685966419108722,-0.9670196953640677,-0.9692624820616289,-0.9686980502169651,-0.9680518069079949,-0.969010171959321,-0.9676152528633691,-0.96818204488929,-0.9682742601059523,-0.9682780632995844,-0.9662128924705515,-0.9688980786357488,-0.9653416309479831,-0.9552503305155642,-0.9681577337994484,-0.9682017701347202,-0.9634212474559494,-0.9684887522780106,-0.9688115620950756,-0.9627449790950002,-0.9690810996862238,-0.9688516362607755,-0.9686693000297326,-0.9497142746838011,-0.9686932809387768,-0.9685567083777342,-0.9645854802346101,-0.9651086639374042,-0.9594493497795411,-0.9684666882519305,-0.964452209481544,-0.968682699867104,-0.9235552148904986,-0.9685214738065034,-0.9690412334121371,-0.9656847682847108,-0.9677024756240816,-0.9682743457083812,-0.9692988386361256,-0.9680827644150042,-0.9663136709873454,-0.9678360337567308,-0.9692182011479885,-0.96884232393939,-0.9693155188808662,-0.9692845552593976,-0.9604824029501875,-0.9673308479646381,-0.968562293936228,-0.9686478688501817,-0.9685509699577601,-0.9372578158243073,-0.9674656137029823,-0.9688302723402757,-0.9677353775291215,-0.9697435860561004,-0.9673768439840885,-0.9666088159336889,-0.9686805445202304,-0.9684747287657992,-0.9639210616956212,-0.9598633353552015,-0.9666243558317856,-0.9681229272403629,-0.9670753583435382,-0.9685261269099669,-0.9654611991979469,-0.9688293093129494,-0.9623365974212108,-0.9687016149466879,-0.9691003143743091,-0.9644732921369199,-0.9694552403884048,-0.9672218393857399,-0.9681754412733371,-0.9693853123755842,-0.9683136219371535,-0.9680163583306917,-0.9623921778554819,-0.9655459425454473,-0.9664124378470571,-0.9686326071599791,-0.9692586819252262,-0.9682425016047856,-0.9668969659386896,-0.9673088359114616,-0.9668506978258041,-0.9655154436228792,-0.9690234586791968,-0.9685104463793079,-0.9598601527791798,-0.9678205458315375,-0.9693624809848725,-0.9676684792308421,-0.9676831753335671,-0.9633626189637592,-0.9676454246623691,-0.9589284681706415,-0.9691728012883116,-0.9610051739269847,-0.967420663256058,-0.9691592241316231,-0.968630931798154,-0.9678635488232018,-0.9682614594855841,-0.9673640280775724,-0.9607921461679084,-0.9655051346446413,-0.9678603540182614,-0.9690718026509861,-0.9675308855551106,-0.9688207307266696,-0.9684633650433468,-0.968637465097826,-0.9633318999206594,-0.966173594841172,-0.9659503651068944,-0.9618031108546381,-0.9689424145795219,-0.9677863018026998,-0.9667409371684225,-0.9676644834317446,-0.9684917881070112,-0.9358518601294709,-0.9676236724737093,-0.9662005045761806,-0.9687298423476574,-0.9684489318623682,-0.9681078336977887,-0.966194341201291,-0.968089572865341,-0.9681676178227705,-0.9692478012450518,-0.965561265380242,-0.9672873344156293,-0.9658368776866207,-0.9502073079882027,-0.9675876216221733,-0.9662801148351694,-0.96247210606635,-0.9685122256869398,-0.968555595546157,-0.9686794072308164,-0.9583189574754097,-0.9683340809176895,-0.9559739492486801,-0.9644778932674797,-0.967677733464865,-0.9685262430846919,-0.9686531150561887],\"yaxis\":\"y\",\"type\":\"scatter\",\"customdata\":[[\"random_forest\",0.0,8.0,3.087288899749003,0.023329650860474123,20.0,-0.9626329836027753,0],[\"random_forest\",1.0,5.0,3.1491846195925843,0.017120865520531006,190.0,-0.9562822708546335,1],[\"random_forest\",1.0,11.0,4.0330545960467505,0.041616714812500316,170.0,-0.9624956192192642,2],[\"random_forest\",1.0,4.0,1.2230890863844883,0.040745334039664696,200.0,-0.945237044468279,3],[\"random_forest\",0.0,8.0,1.7886630602058156,0.01227044033913774,150.0,-0.9656749484632106,4],[\"random_forest\",0.0,8.0,4.125064868782711,0.04638893503104118,140.0,-0.9586555859703025,5],[\"random_forest\",0.0,9.0,4.2022433072116625,0.028145601504114375,100.0,-0.9633421966699789,6],[\"random_forest\",0.0,7.0,2.029945542955745,0.037221326421165345,50.0,-0.9580469526999682,7],[\"random_forest\",0.0,4.0,3.275985213017331,0.026657172413461147,120.0,-0.9505051218959941,8],[\"random_forest\",0.0,6.0,4.550678081409195,0.012372126103673269,180.0,-0.9619859576430254,9],[\"random_forest\",0.0,11.0,2.5515338534410845,0.04102517709528973,120.0,-0.9618390852754338,10],[\"random_forest\",1.0,8.0,2.9749118127225387,0.01627475895489639,100.0,-0.9656506465450575,11],[\"random_forest\",1.0,4.0,4.9487852205229235,0.02593866527459223,160.0,-0.9488812774470049,12],[\"random_forest\",1.0,12.0,3.3955805601426747,0.018936863669885007,170.0,-0.967084334369667,13],[\"random_forest\",1.0,3.0,4.442881994787241,0.021141602348926192,30.0,-0.9374047554509501,14],[\"random_forest\",0.0,6.0,2.7892406588690504,0.03956386158518332,70.0,-0.9587348507623457,15],[\"random_forest\",0.0,11.0,4.647622356118037,0.029298032384519807,190.0,-0.9634928141438399,16],[\"random_forest\",0.0,11.0,2.4352816865187217,0.018015129401768856,180.0,-0.9660973781070481,17],[\"random_forest\",0.0,4.0,1.8251679924594026,0.020348207130603764,30.0,-0.9470437723924107,18],[\"random_forest\",1.0,2.0,1.262590898484663,0.041911998789381136,130.0,-0.9212283329781137,19],[\"random_forest\",1.0,9.0,1.685161135985639,0.01097254098623826,90.0,-0.9683647510451154,20],[\"random_forest\",1.0,9.0,3.6627843368646205,0.01414459020356588,150.0,-0.9675297360367779,21],[\"random_forest\",1.0,7.0,2.189424121875658,0.0331691008458295,80.0,-0.960723908803061,22],[\"random_forest\",1.0,10.0,1.5579709178226173,0.015002675422565283,110.0,-0.9675501461016399,23],[\"random_forest\",0.0,12.0,3.656062331458171,0.010423964074142832,160.0,-0.9684950532282324,24],[\"random_forest\",1.0,12.0,3.5313475201246063,0.032980781171731394,140.0,-0.9639596439332724,25],[\"random_forest\",1.0,10.0,1.0052561760217673,0.021862450083837987,160.0,-0.9661501917485239,26],[\"random_forest\",0.0,12.0,2.5227508492604755,0.017917169988287927,200.0,-0.9666057953908362,27],[\"random_forest\",0.0,10.0,2.331380057571953,0.025066367833446825,180.0,-0.9644503476287128,28],[\"random_forest\",1.0,11.0,2.8099177020002135,0.024089561150875065,180.0,-0.9659897116519477,29],[\"random_forest\",1.0,9.0,3.821177687444521,0.010255750551421618,80.0,-0.9677541091749289,30],[\"random_forest\",1.0,10.0,1.4343814753287367,0.014453376873089356,60.0,-0.9671651277765141,31],[\"random_forest\",1.0,9.0,1.6039837272660382,0.015309217489745971,110.0,-0.9670907881813691,32],[\"random_forest\",0.0,6.0,4.979700850496388,0.010169219002251834,90.0,-0.9625594083150323,33],[\"random_forest\",0.0,7.0,3.7176259580529774,0.04793643158654716,40.0,-0.9566308470028337,34],[\"random_forest\",0.0,12.0,4.023020989070647,0.011910669711395542,90.0,-0.9682401383662989,35],[\"random_forest\",1.0,5.0,3.043726402267325,0.03183288257646312,10.0,-0.9529344362863903,36],[\"random_forest\",0.0,8.0,1.0975698668823128,0.010349469895533895,130.0,-0.9658821460854244,37],[\"random_forest\",0.0,9.0,3.232992123480172,0.023028728563479244,60.0,-0.9643388565793731,38],[\"random_forest\",0.0,8.0,1.9375157462036254,0.01263995783589422,120.0,-0.9654529385635462,39],[\"random_forest\",1.0,11.0,4.352547516324519,0.036406649139798755,140.0,-0.9629872156257498,40],[\"random_forest\",0.0,5.0,4.7423246662661285,0.04474886015759178,90.0,-0.9552149858840677,41],[\"random_forest\",1.0,10.0,3.9255761071065423,0.027683767267073785,160.0,-0.9641748728976662,42],[\"random_forest\",0.0,7.0,2.728564489484436,0.020166863380509797,50.0,-0.9634675033399163,43],[\"random_forest\",1.0,8.0,4.285067506134287,0.016398308824169414,100.0,-0.9656062708572993,44],[\"random_forest\",0.0,12.0,3.3895861299985777,0.012614531745646716,130.0,-0.9673041522357023,45],[\"random_forest\",1.0,11.0,2.170587590348667,0.030030121649453536,200.0,-0.964779736603894,46],[\"random_forest\",0.0,6.0,3.1206377883657552,0.018807898199609333,150.0,-0.9611787450807417,47],[\"random_forest\",1.0,2.0,2.8707091292018765,0.010001736559353892,70.0,-0.9233628937475564,48],[\"random_forest\",0.0,10.0,3.484253265209595,0.023100187634693577,170.0,-0.9648019137474695,49],[\"random_forest\",0.0,9.0,2.6222642779704604,0.013500971168462725,110.0,-0.9663002528065964,50],[\"random_forest\",1.0,3.0,4.132991794059827,0.01631288876647256,190.0,-0.9369208968926019,51],[\"random_forest\",1.0,12.0,1.2381121231837118,0.038395323611431914,80.0,-0.9630745943051725,52],[\"random_forest\",0.0,8.0,1.7671441975181763,0.04975731156902563,10.0,-0.9508608550187063,53],[\"random_forest\",1.0,5.0,2.3228710047071344,0.011683851449563019,20.0,-0.9576589382898265,54],[\"random_forest\",0.0,6.0,4.49880287924134,0.026230788408666134,120.0,-0.9610656795581541,55],[\"random_forest\",1.0,11.0,4.726888606775727,0.03531286911856843,70.0,-0.9633246145425038,56],[\"random_forest\",0.0,7.0,3.6238264233385515,0.01986774983639439,170.0,-0.9626890929377685,57],[\"random_forest\",1.0,10.0,3.854560317595624,0.021751575428758572,40.0,-0.9652753991257583,58],[\"random_forest\",0.0,9.0,1.3929310493524654,0.029217678839885245,150.0,-0.9632069112026009,59],[\"random_forest\",1.0,11.0,2.021660169595431,0.04388967711842039,100.0,-0.9618518155795212,60],[\"random_forest\",0.0,8.0,2.919449436450673,0.02462028685457974,130.0,-0.9625740432731651,61],[\"random_forest\",1.0,3.0,3.367457070373632,0.017313791946854316,190.0,-0.9369199705520306,62],[\"random_forest\",0.0,11.0,2.6988437476161806,0.011348109313947974,140.0,-0.96815187309029,63],[\"random_forest\",1.0,12.0,3.218922451300524,0.013539916241415097,160.0,-0.9682601295907045,64],[\"random_forest\",0.0,12.0,4.040869710552425,0.015309502803823538,60.0,-0.9666819173508425,65],[\"random_forest\",0.0,12.0,3.973151141623964,0.010967986258700291,90.0,-0.9685657516629143,66],[\"random_forest\",0.0,11.0,4.851584204603317,0.018701510914611138,80.0,-0.9655268990622199,67],[\"random_forest\",0.0,10.0,4.223117153170825,0.013761994358552854,100.0,-0.9663511373361898,68],[\"random_forest\",0.0,12.0,3.5706494736211947,0.01216480979947925,90.0,-0.9674739691115007,69],[\"random_forest\",1.0,9.0,3.7413561998284672,0.015432591619255727,160.0,-0.9668579037159899,70],[\"random_forest\",0.0,10.0,4.425988402905299,0.011192231578097987,50.0,-0.9672645030249177,71],[\"random_forest\",0.0,12.0,4.596092326527463,0.01084343720868759,110.0,-0.9682469070726505,72],[\"random_forest\",0.0,11.0,3.96931181624806,0.017185536065239183,70.0,-0.9659472192176279,73],[\"random_forest\",0.0,10.0,3.0098176429923416,0.021371073521401936,120.0,-0.9648739237336531,74],[\"random_forest\",0.0,9.0,4.12807059382752,0.01447440254709473,40.0,-0.9651323299517989,75],[\"random_forest\",0.0,11.0,3.79319225649346,0.027375685378309644,90.0,-0.9644082801493084,76],[\"random_forest\",0.0,12.0,2.366520130730358,0.022523479410689055,60.0,-0.9656319699293833,77],[\"random_forest\",0.0,6.0,3.5032962257793994,0.01293074010115412,120.0,-0.9621477768061699,78],[\"random_forest\",1.0,7.0,3.1569305905356977,0.02056690347865377,80.0,-0.963675725134049,79],[\"random_forest\",0.0,10.0,4.345863592226795,0.01823871351054398,140.0,-0.9652765425296317,80],[\"random_forest\",1.0,4.0,3.2873427801565605,0.01965655857138175,100.0,-0.9488435604053326,81],[\"random_forest\",0.0,11.0,4.883068810709629,0.031197963649425155,110.0,-0.9635082745539669,82],[\"random_forest\",0.0,8.0,1.6393418206755617,0.0347110517556167,50.0,-0.9604443434987989,83],[\"random_forest\",1.0,12.0,3.3854184985346407,0.010003776632744953,20.0,-0.967984009784211,84],[\"random_forest\",0.0,9.0,1.0815887849821142,0.01616003541138538,130.0,-0.9654042277242042,85],[\"random_forest\",1.0,12.0,2.200763339901485,0.02506649410951746,70.0,-0.9655790339987229,86],[\"random_forest\",0.0,10.0,3.6862690657691677,0.040047340548139104,80.0,-0.961942606127186,87],[\"random_forest\",1.0,7.0,3.9122223068961888,0.04229326163690983,90.0,-0.9590642917104301,88],[\"random_forest\",0.0,8.0,4.217083917116031,0.019332082768225463,60.0,-0.9631354698611243,89],[\"random_forest\",1.0,5.0,2.481566407392046,0.01739888119277392,100.0,-0.9573319033814434,90],[\"random_forest\",0.0,9.0,2.6157575438485217,0.014568980860414424,30.0,-0.9656298818415607,91],[\"random_forest\",0.0,11.0,1.367384828236055,0.02394438562687731,190.0,-0.9650419123861462,92],[\"random_forest\",1.0,6.0,4.729550045175682,0.016312262350122633,170.0,-0.9625710716459864,93],[\"random_forest\",0.0,2.0,4.046717691772707,0.012786677258068419,180.0,-0.9146365607326213,94],[\"random_forest\",1.0,11.0,3.08223804262636,0.010748880363931254,150.0,-0.9689232365781918,95],[\"random_forest\",0.0,9.0,1.9011213954175954,0.049032739129249533,140.0,-0.9581106164493225,96],[\"random_forest\",1.0,12.0,2.948890303478607,0.04639157174310765,110.0,-0.9609286300692926,97],[\"random_forest\",0.0,10.0,3.591028895425033,0.02225164618393378,90.0,-0.9647325207498301,98],[\"random_forest\",0.0,8.0,2.8166379642626973,0.02865238909195381,120.0,-0.9623183274170743,99],[\"random_forest\",1.0,11.0,3.29875213678646,0.020651539767993707,200.0,-0.9668207308611878,100],[\"random_forest\",1.0,12.0,3.479414197242027,0.010537993808780108,150.0,-0.9688700866414591,101],[\"random_forest\",0.0,11.0,3.0542102146536068,0.01542187941837511,160.0,-0.9661846834129597,102],[\"random_forest\",0.0,12.0,3.1751979362767546,0.011988463171035838,170.0,-0.9682822425324583,103],[\"random_forest\",1.0,11.0,4.487963268015834,0.02558604062660696,180.0,-0.9654526175544375,104],[\"random_forest\",0.0,12.0,3.8528824780620785,0.038016722304316464,130.0,-0.9616896662355789,105],[\"random_forest\",1.0,10.0,3.743369726873844,0.018142270255662372,140.0,-0.9662189396707159,106],[\"random_forest\",0.0,11.0,4.091223064711909,0.013201714884381984,180.0,-0.9674226871420585,107],[\"random_forest\",0.0,12.0,4.638145736659773,0.033285979525423066,150.0,-0.9630372043870682,108],[\"random_forest\",1.0,12.0,3.4326744698230063,0.010407786049931901,150.0,-0.9687808736242718,109],[\"random_forest\",1.0,10.0,2.702777934952766,0.01419004852408351,150.0,-0.9675697001422119,110],[\"random_forest\",1.0,11.0,3.4652336135063075,0.02680474118747872,130.0,-0.9656123364008413,111],[\"random_forest\",1.0,10.0,2.59627020537494,0.011248470098400446,190.0,-0.9683667932744935,112],[\"random_forest\",1.0,12.0,3.096137571112858,0.016375655062103135,170.0,-0.9673747528390366,113],[\"random_forest\",1.0,4.0,2.8921788591835904,0.021292012646190444,120.0,-0.949407426662519,114],[\"random_forest\",1.0,12.0,2.7609495299219744,0.010059561062337234,160.0,-0.9693741932315002,115],[\"random_forest\",1.0,11.0,2.228228355938489,0.012187056256452307,150.0,-0.9686894655162263,116],[\"random_forest\",1.0,5.0,2.4358719112101634,0.023875534288755974,150.0,-0.9557833187537109,117],[\"random_forest\",1.0,3.0,3.466848525261748,0.030715470878655067,200.0,-0.936386279151074,118],[\"random_forest\",1.0,12.0,3.333408939760092,0.013708598617024873,160.0,-0.9681978538235925,119],[\"random_forest\",1.0,11.0,2.037735297145532,0.01501846288144416,170.0,-0.9682305692376263,120],[\"random_forest\",1.0,10.0,2.792178425038453,0.018683077643344627,140.0,-0.9665499612065077,121],[\"random_forest\",1.0,9.0,2.1023450490805975,0.017757044606250383,180.0,-0.9664636800725145,122],[\"random_forest\",1.0,12.0,2.3559026732285124,0.010313501070363496,160.0,-0.96934113846498,123],[\"random_forest\",1.0,11.0,2.976805414877771,0.016747455296058675,190.0,-0.967575203155506,124],[\"random_forest\",1.0,9.0,1.5149109605199222,0.015851102712331068,130.0,-0.9669742160164212,125],[\"random_forest\",1.0,7.0,2.2923644155538057,0.011568726321506085,140.0,-0.9660269548230307,126],[\"random_forest\",1.0,10.0,3.233116031793643,0.02287693322503756,190.0,-0.9659529301225354,127],[\"random_forest\",1.0,12.0,1.8314851239877128,0.03615576557001173,170.0,-0.9630913846101787,128],[\"random_forest\",1.0,11.0,2.496535818375244,0.01929148766037446,180.0,-0.9671429781480052,129],[\"random_forest\",1.0,8.0,1.7051329335842178,0.010034910174195917,160.0,-0.9671431432384038,130],[\"random_forest\",1.0,9.0,2.4040405121096784,0.01296014730930345,200.0,-0.9674184589935106,131],[\"random_forest\",1.0,10.0,2.098925472043847,0.014624056925940898,160.0,-0.9678378772661845,132],[\"random_forest\",1.0,11.0,2.6712721500617107,0.04502129396300464,190.0,-0.9620385572784287,133],[\"random_forest\",1.0,12.0,2.555891302598985,0.032900350465069346,200.0,-0.9638508982761207,134],[\"random_forest\",1.0,2.0,1.3184025991877315,0.028103635134021455,160.0,-0.9215895813430386,135],[\"random_forest\",1.0,11.0,2.291053287560139,0.020454489349688576,170.0,-0.9668926552449424,136],[\"random_forest\",1.0,12.0,1.9988538561022406,0.013751814170157478,130.0,-0.9684408791195809,137],[\"random_forest\",1.0,9.0,1.932178024078161,0.04105804145428079,180.0,-0.9627423284769303,138],[\"random_forest\",1.0,10.0,1.868353276366384,0.012134732120828585,110.0,-0.9680765337810633,139],[\"random_forest\",1.0,8.0,1.521973965480188,0.029718894373261622,150.0,-0.9638720818200734,140],[\"random_forest\",1.0,12.0,1.7466216475507823,0.04307283841624751,120.0,-0.9622389953659797,141],[\"random_forest\",1.0,6.0,2.844069691017523,0.010199980026016516,140.0,-0.9638861450762696,142],[\"random_forest\",1.0,10.0,2.7058848636847364,0.03887584239524678,160.0,-0.9626306692799621,143],[\"random_forest\",1.0,3.0,2.1259857361590595,0.011151414085086232,140.0,-0.9374083935541836,144],[\"random_forest\",1.0,12.0,3.041899662203752,0.010530223298598903,150.0,-0.9688700866414591,145],[\"random_forest\",1.0,12.0,3.1373694798762526,0.012834672689304882,170.0,-0.9687817051907253,146],[\"random_forest\",1.0,11.0,3.5445154974015773,0.017424987808288418,160.0,-0.9670809500164911,147],[\"random_forest\",1.0,12.0,3.618467200989449,0.014753167477870704,150.0,-0.968057591186413,148],[\"random_forest\",1.0,11.0,3.2092243411849646,0.010070322516070002,130.0,-0.969253279800509,149],[\"random_forest\",1.0,12.0,2.953131220173957,0.01184723019045927,180.0,-0.9693086401142484,150],[\"random_forest\",1.0,11.0,3.0597001795397354,0.014008537104703676,140.0,-0.9676951474447115,151],[\"random_forest\",1.0,12.0,2.7865936618017058,0.01563253689256233,130.0,-0.9684512308990332,152],[\"random_forest\",1.0,12.0,2.38101566834899,0.012989078550454742,170.0,-0.9686964665720282,153],[\"random_forest\",1.0,11.0,2.4756155656961667,0.01677868193311637,110.0,-0.9677368113698076,154],[\"random_forest\",1.0,10.0,2.606746604358728,0.011309656930239498,130.0,-0.9682189242500497,155],[\"random_forest\",1.0,11.0,2.943986665714902,0.01204585381204137,180.0,-0.9686358050221491,156],[\"random_forest\",1.0,12.0,3.3585116523553262,0.018131141321778906,180.0,-0.967246483713609,157],[\"random_forest\",1.0,12.0,2.2530874592134853,0.015287566351752317,190.0,-0.9686434389244798,158],[\"random_forest\",1.0,11.0,2.9121749187104204,0.01935726461641577,120.0,-0.9672374312567401,159],[\"random_forest\",1.0,12.0,3.2325815783847403,0.010104856377267222,200.0,-0.968734764487326,160],[\"random_forest\",1.0,11.0,2.3498124078136873,0.013547902273326378,170.0,-0.9682407253543834,161],[\"random_forest\",1.0,10.0,2.7927161269397636,0.012355176332144996,160.0,-0.9682886504857164,162],[\"random_forest\",1.0,12.0,2.512655031158346,0.014295937386208792,100.0,-0.9686413600083464,163],[\"random_forest\",1.0,10.0,2.7428787018512293,0.010102185767611845,190.0,-0.9683928959008855,164],[\"random_forest\",1.0,11.0,2.6380610418736197,0.016833385215480508,120.0,-0.9676248984227821,165],[\"random_forest\",1.0,5.0,2.1710699525014583,0.02102280735472386,200.0,-0.9549889496130102,166],[\"random_forest\",1.0,12.0,3.2900733551963066,0.0200980221051522,140.0,-0.9669017352168778,167],[\"random_forest\",1.0,11.0,1.6258526029919342,0.022262763459580878,170.0,-0.9665485640525769,168],[\"random_forest\",1.0,9.0,1.9931930340443706,0.01167220395481499,180.0,-0.9684274762249802,169],[\"random_forest\",1.0,12.0,2.9838774423775907,0.027089734535238573,130.0,-0.9654107824244834,170],[\"random_forest\",1.0,11.0,2.5768787336688104,0.018728714143518642,160.0,-0.9670577273003895,171],[\"random_forest\",1.0,12.0,3.1456590829022226,0.025970545642391374,150.0,-0.9657223324649029,172],[\"random_forest\",1.0,10.0,3.693549865317406,0.047895845467133394,190.0,-0.9611665314484584,173],[\"random_forest\",1.0,4.0,1.1249077275896824,0.015885558658733753,170.0,-0.9497717842299549,174],[\"random_forest\",1.0,7.0,2.8344617393977987,0.024860239380541542,160.0,-0.9628851072140768,175],[\"random_forest\",1.0,10.0,2.435269045898203,0.023725945308559485,140.0,-0.9663152148882974,176],[\"random_forest\",1.0,12.0,3.2032670108863646,0.011159385925418812,110.0,-0.9682951226407963,177],[\"random_forest\",1.0,11.0,2.894863797142421,0.03223178614248129,150.0,-0.964086861371715,178],[\"random_forest\",1.0,9.0,3.7807599037396504,0.013451150632249676,100.0,-0.9678818891436193,179],[\"random_forest\",1.0,12.0,3.422793598200784,0.033965638960301275,200.0,-0.963521940370314,180],[\"random_forest\",1.0,11.0,2.6732445401813023,0.010030521682825106,120.0,-0.969260773070278,181],[\"random_forest\",1.0,11.0,2.2246576698478897,0.012418057282821098,180.0,-0.9685741590443359,182],[\"random_forest\",1.0,8.0,3.8718236095914844,0.03738233900484618,130.0,-0.9607952186836644,183],[\"random_forest\",1.0,12.0,2.0693761715157173,0.01448657836584671,140.0,-0.9687648476266674,184],[\"random_forest\",1.0,10.0,2.9835574168421517,0.015074086014274995,160.0,-0.9676757676662272,185],[\"random_forest\",1.0,11.0,2.719983266875193,0.01730247354849615,70.0,-0.9671339868357283,186],[\"random_forest\",1.0,12.0,1.7861965304060006,0.018089777540204327,120.0,-0.9677879557639175,187],[\"random_forest\",1.0,10.0,1.9381310227198245,0.01625605423732903,80.0,-0.9672487307773708,188],[\"random_forest\",1.0,12.0,2.3341523139273415,0.021905639995219522,180.0,-0.9665786624780663,189],[\"random_forest\",1.0,11.0,2.5155297663586818,0.01164087456196693,190.0,-0.9688044020633342,190],[\"random_forest\",1.0,9.0,1.472115518766536,0.010834469529850663,110.0,-0.9683169848897223,191],[\"random_forest\",1.0,12.0,2.2612435465633007,0.0130273991528466,170.0,-0.968778253578498,192],[\"random_forest\",1.0,7.0,2.1448313934550978,0.019660032290975214,120.0,-0.9639458221982152,193],[\"random_forest\",1.0,11.0,2.670833118317452,0.014019325149352498,90.0,-0.9686192745816593,194],[\"random_forest\",1.0,10.0,2.370560042606379,0.01002568281003937,200.0,-0.9684864165545901,195],[\"random_forest\",1.0,6.0,1.676752115958316,0.040113718849178914,190.0,-0.9593628485250161,196],[\"random_forest\",1.0,12.0,3.0788470152562906,0.015399012569993565,150.0,-0.9678945735892622,197],[\"random_forest\",1.0,4.0,1.5810438415940384,0.012196957417720608,50.0,-0.9507725438842534,198],[\"random_forest\",1.0,11.0,2.5417346050846428,0.03028609210617013,180.0,-0.9648038214587447,199],[\"random_forest\",1.0,8.0,2.4336477760403854,0.028979084079513703,40.0,-0.9624908407693871,200],[\"random_forest\",1.0,12.0,1.8395663663459916,0.02331012631075885,160.0,-0.9665021063914558,201],[\"random_forest\",1.0,11.0,3.3538028844735663,0.010848028545557904,170.0,-0.9688752411305779,202],[\"random_forest\",1.0,10.0,2.8596872501830255,0.017207759745455013,110.0,-0.9670362777774608,203],[\"random_forest\",1.0,5.0,2.776982017710807,0.013522822694428161,200.0,-0.9568500167361921,204],[\"random_forest\",1.0,12.0,2.6275582123935406,0.010044183426949595,30.0,-0.9685735078544295,205],[\"random_forest\",1.0,10.0,3.5384657656930574,0.018487655933024195,100.0,-0.9662170869895736,206],[\"random_forest\",1.0,9.0,1.2155196876424474,0.01603432016729971,140.0,-0.966559080922428,207],[\"random_forest\",1.0,12.0,1.9892615317586329,0.011640787936252395,190.0,-0.9694597956605205,208],[\"random_forest\",1.0,11.0,3.275090367759808,0.014685334459854501,150.0,-0.9673544161476851,209],[\"random_forest\",1.0,10.0,3.016644843925015,0.012759467287184481,130.0,-0.9677629353968068,210],[\"random_forest\",1.0,12.0,2.1983653715795226,0.03568090682817669,180.0,-0.9631918207172566,211],[\"random_forest\",1.0,11.0,2.307420889014852,0.020701345885999492,170.0,-0.966937238824314,212],[\"random_forest\",1.0,6.0,3.1305522448940524,0.02133192663569329,160.0,-0.9623634215537902,213],[\"random_forest\",1.0,12.0,1.3422306562071555,0.024431804263958398,200.0,-0.9658879059060056,214],[\"random_forest\",1.0,12.0,2.0073188880644577,0.019100090512188424,190.0,-0.9675791408672411,215],[\"random_forest\",1.0,2.0,1.8571412067227389,0.011569005600504667,200.0,-0.9189801502702948,216],[\"random_forest\",1.0,9.0,1.9440550621764205,0.016875079544115306,190.0,-0.9666789640670415,217],[\"random_forest\",0.0,8.0,1.7196634438524934,0.045338099241880814,180.0,-0.9598055506583831,218],[\"random_forest\",1.0,3.0,1.469684791745935,0.019920474067725895,190.0,-0.9368604371198765,219],[\"random_forest\",1.0,10.0,1.5813901272053874,0.014253520745261965,180.0,-0.967861503536594,220],[\"random_forest\",1.0,11.0,1.1790213613498075,0.031647095360565085,200.0,-0.9647142446312342,221],[\"random_forest\",1.0,12.0,2.1409911739074596,0.01800406953344565,170.0,-0.9677968614737653,222],[\"random_forest\",0.0,12.0,2.0654733197161548,0.012535665414252752,190.0,-0.9679656450059565,223],[\"random_forest\",1.0,12.0,1.7655701868588825,0.03447617623475212,170.0,-0.9634196699254718,224],[\"random_forest\",1.0,11.0,2.424692602234172,0.015603477910120036,160.0,-0.9680982676263459,225],[\"random_forest\",1.0,9.0,1.059067500351181,0.04746205982807114,180.0,-0.96108403210749,226],[\"random_forest\",1.0,11.0,1.0184507001188234,0.0226255897577156,200.0,-0.966411517620945,227],[\"random_forest\",1.0,10.0,2.5469518442366335,0.02809567923355,170.0,-0.9649561204088903,228],[\"random_forest\",0.0,12.0,2.2540489109259276,0.010771575536982866,190.0,-0.9684967897346497,229],[\"random_forest\",1.0,11.0,1.424693760451514,0.02548213924306561,150.0,-0.9658957599288683,230],[\"random_forest\",1.0,11.0,1.2815806590857979,0.026641478358946747,180.0,-0.9657574814337047,231],[\"random_forest\",1.0,12.0,1.9331832945815504,0.013665311302341424,200.0,-0.9687617903970596,232],[\"random_forest\",1.0,9.0,2.930927706108702,0.016268430770328333,150.0,-0.966797153506452,233],[\"random_forest\",1.0,10.0,2.0594487597443623,0.011765043279413977,160.0,-0.9682485121181946,234],[\"random_forest\",0.0,4.0,2.7430287409957526,0.04928177478824691,160.0,-0.9440580422133826,235],[\"random_forest\",1.0,7.0,1.6820269248783917,0.04362508597157272,10.0,-0.9524498256495584,236],[\"random_forest\",1.0,12.0,2.4856171922212917,0.015240737165513916,190.0,-0.9686434389244798,237],[\"random_forest\",1.0,11.0,3.6535691691449284,0.04216399974924506,170.0,-0.9627558139167305,238],[\"random_forest\",1.0,12.0,2.368714265674577,0.02160103976497919,180.0,-0.9669078221610272,239],[\"random_forest\",1.0,10.0,1.8213352319800882,0.0174936700146313,140.0,-0.9671410429216633,240],[\"random_forest\",0.0,12.0,2.848813599252701,0.01309485240768132,200.0,-0.9678298948396784,241],[\"random_forest\",1.0,10.0,2.125813273691965,0.011188278797424964,180.0,-0.9683465788723262,242],[\"random_forest\",1.0,11.0,3.964422940070781,0.014794336111125232,160.0,-0.96721380804356,243],[\"random_forest\",1.0,11.0,2.190977779952064,0.012341079561504962,190.0,-0.9685733244206529,244],[\"random_forest\",1.0,12.0,1.5310476400513882,0.010061353154474158,20.0,-0.9693327494269361,245],[\"random_forest\",1.0,8.0,2.609166709685901,0.02033464694766336,150.0,-0.9651488756784367,246],[\"random_forest\",0.0,12.0,3.394631623253176,0.019018938351235637,170.0,-0.9659959025419036,247],[\"random_forest\",1.0,10.0,3.0430027920781986,0.013636653642502642,140.0,-0.9673623649446655,248],[\"random_forest\",1.0,9.0,2.2771391306800686,0.016558453021740024,190.0,-0.9668249834675723,249],[\"random_forest\",1.0,11.0,2.721843549767576,0.01061779459866689,160.0,-0.9690074296243629,250],[\"random_forest\",1.0,12.0,2.011404325948009,0.017779373090514576,180.0,-0.9678912809529747,251],[\"random_forest\",1.0,11.0,3.154538158363285,0.011903519919969671,170.0,-0.9686670193364454,252],[\"random_forest\",0.0,6.0,1.3675719016665324,0.03926648173151104,30.0,-0.9567239274154747,253],[\"random_forest\",1.0,12.0,1.5788263736333499,0.036960223447337494,20.0,-0.9633268707779543,254],[\"random_forest\",1.0,3.0,1.1887228638100158,0.014201625487400352,20.0,-0.9372913689192537,255],[\"random_forest\",1.0,5.0,1.4630058096029315,0.012828494135760144,80.0,-0.9584315063261938,256],[\"random_forest\",1.0,12.0,1.6509684710685046,0.010216625355476787,60.0,-0.9697555214804895,257],[\"random_forest\",1.0,12.0,1.9033857457605352,0.010071798396850585,50.0,-0.969868006129452,258],[\"random_forest\",1.0,12.0,1.8054527958944993,0.010932427357017072,40.0,-0.9688026044123246,259],[\"random_forest\",1.0,11.0,1.5194671955948649,0.011640400939172865,70.0,-0.9688481999346964,260],[\"random_forest\",1.0,12.0,1.760124940828503,0.013114032334196533,170.0,-0.9688526971194495,261],[\"random_forest\",1.0,12.0,1.3281260533885733,0.015037452256851267,10.0,-0.9670827843542559,262],[\"random_forest\",1.0,11.0,1.24988812658379,0.010020652995211552,150.0,-0.9696022167018031,263],[\"random_forest\",1.0,12.0,1.1362136728213368,0.01600367525491511,180.0,-0.9684125019143608,264],[\"random_forest\",1.0,12.0,1.0070287615913323,0.010815325865282334,60.0,-0.9690926498996824,265],[\"random_forest\",1.0,11.0,1.9188151709181867,0.0100651901952459,50.0,-0.9695863099361534,266],[\"random_forest\",1.0,12.0,1.6408594199277409,0.013999663839397138,60.0,-0.9681676178227705,267],[\"random_forest\",1.0,11.0,1.6762783228592493,0.012152226812942138,50.0,-0.9681884039268746,268],[\"random_forest\",1.0,12.0,1.9082967408416462,0.011215645150690555,30.0,-0.9689105460180895,269],[\"random_forest\",1.0,11.0,2.0539910441829354,0.013236182427339479,40.0,-0.9686680404511343,270],[\"random_forest\",1.0,11.0,1.2532196251590957,0.010070139323044808,40.0,-0.9696699801960615,271],[\"random_forest\",1.0,10.0,1.2962464500819868,0.010010852714371011,70.0,-0.9685011096000856,272],[\"random_forest\",1.0,10.0,1.0863156692922784,0.010086630899422773,50.0,-0.9680393609262613,273],[\"random_forest\",1.0,10.0,1.1855155149086087,0.014860439348622172,10.0,-0.9649045571743238,274],[\"random_forest\",1.0,11.0,1.4615143900233376,0.012550585053917922,50.0,-0.9681894739572373,275],[\"random_forest\",1.0,11.0,1.3811221703342882,0.01560640870009005,60.0,-0.9677743174626368,276],[\"random_forest\",1.0,9.0,1.0054981707827348,0.014265587551152658,40.0,-0.9667024833344147,277],[\"random_forest\",1.0,10.0,1.255827717391834,0.010054954060501184,40.0,-0.9680685849840829,278],[\"random_forest\",1.0,9.0,1.6284766598145541,0.017225169624463802,30.0,-0.9641423378601794,279],[\"random_forest\",1.0,10.0,1.4038121339934118,0.010963459572053698,60.0,-0.9683534117804999,280],[\"random_forest\",1.0,10.0,1.1133974577579009,0.010002908469276582,50.0,-0.9680928379865621,281],[\"random_forest\",1.0,11.0,1.237479928406387,0.013642920308933276,40.0,-0.96778130934675,282],[\"random_forest\",0.0,9.0,1.2433988904593483,0.016615311074695452,70.0,-0.9645058027165699,283],[\"random_forest\",1.0,11.0,1.434508834364331,0.012987030437564804,60.0,-0.9680569889121804,284],[\"random_forest\",1.0,10.0,1.553505789865541,0.012224074201158409,30.0,-0.9676866728042385,285],[\"random_forest\",1.0,11.0,1.0578242300703806,0.018867145251127336,20.0,-0.9658397147956967,286],[\"random_forest\",1.0,8.0,1.726608015256408,0.011179738787991474,80.0,-0.9666988758034775,287],[\"random_forest\",1.0,11.0,1.1233838912244893,0.015502629723650307,50.0,-0.9677680501419409,288],[\"random_forest\",0.0,9.0,1.306605479533704,0.010090671540235352,70.0,-0.9665453142175039,289],[\"random_forest\",1.0,10.0,1.8952243230755708,0.041321264761541354,30.0,-0.9616707511559951,290],[\"random_forest\",1.0,11.0,1.6350132736469067,0.01795295593239894,80.0,-0.9670927570372365,291],[\"random_forest\",1.0,8.0,1.024522259086666,0.010028693957962277,90.0,-0.9673898616677588,292],[\"random_forest\",1.0,10.0,1.5247670131130406,0.014399227859819903,50.0,-0.9672751605273308,293],[\"random_forest\",1.0,11.0,1.362381700312631,0.012301312177615913,20.0,-0.9692554962919749,294],[\"random_forest\",1.0,9.0,1.0056764310418351,0.011553348344339819,40.0,-0.9666264561485263,295],[\"random_forest\",0.0,10.0,1.7637408763312585,0.013475527994285827,10.0,-0.9640978245970888,296],[\"random_forest\",1.0,7.0,1.868886396708502,0.01600886348447256,60.0,-0.9646037471815171,297],[\"random_forest\",1.0,11.0,1.147002634552254,0.010846447293188963,30.0,-0.9679175242119286,298],[\"random_forest\",1.0,10.0,1.2501710902803609,0.019964572808687103,80.0,-0.9661959584757536,299],[\"random_forest\",1.0,11.0,1.4148756764032897,0.014863713916912108,70.0,-0.9674509848593089,300],[\"random_forest\",1.0,11.0,1.604156081685442,0.016921842013707396,40.0,-0.9677816609281551,301],[\"random_forest\",1.0,12.0,1.8253739990294278,0.03260353775371973,50.0,-0.9637531403021794,302],[\"random_forest\",0.0,12.0,1.6905510023889847,0.0129270806677605,60.0,-0.9668268116908778,303],[\"random_forest\",1.0,9.0,1.482218603707368,0.03811239205125892,70.0,-0.962779238409986,304],[\"random_forest\",1.0,10.0,1.9148571918892028,0.02306183873800355,90.0,-0.9661066506844486,305],[\"random_forest\",1.0,11.0,1.5720896804471711,0.011751576445291383,10.0,-0.9652692602087056,306],[\"random_forest\",1.0,12.0,1.2987853019387883,0.018607798728394607,30.0,-0.9663758947815542,307],[\"random_forest\",1.0,8.0,1.3727048620017093,0.013935915571518609,20.0,-0.9642556815906613,308],[\"random_forest\",1.0,12.0,1.0008367952770854,0.01077883952289076,60.0,-0.969587927210616,309],[\"random_forest\",0.0,10.0,1.2069710185350435,0.0210392393229879,40.0,-0.9636899320800367,310],[\"random_forest\",1.0,11.0,1.7369297607027598,0.010088429158059948,20.0,-0.9680575392135098,311],[\"random_forest\",1.0,11.0,2.1469645070047396,0.01232479573383642,50.0,-0.9687362136141602,312],[\"random_forest\",1.0,2.0,1.5086916804932535,0.046764224159042425,30.0,-0.9242872287761174,313],[\"random_forest\",1.0,12.0,1.0763056152532886,0.01621366762560467,60.0,-0.9665582921571891,314],[\"random_forest\",1.0,12.0,1.0997271705367586,0.015047610469291836,70.0,-0.9678829683456709,315],[\"random_forest\",1.0,12.0,1.00782306712916,0.010733715083586146,80.0,-0.9696522115775806,316],[\"random_forest\",0.0,12.0,1.1932519933302703,0.03097508937879278,90.0,-0.9627747412252329,317],[\"random_forest\",1.0,12.0,1.3235567604986334,0.013438125666118971,100.0,-0.9687226425719309,318],[\"random_forest\",1.0,12.0,1.4324771622314922,0.019715405228533676,60.0,-0.9666827428028366,319],[\"random_forest\",1.0,12.0,1.152368967761667,0.029375741312875306,70.0,-0.9651150260322185,320],[\"random_forest\",1.0,6.0,1.2873374840041876,0.024093237955045428,100.0,-0.9614771796061434,321],[\"random_forest\",1.0,12.0,1.5885557446451413,0.014449930128363472,80.0,-0.9687081513035897,322],[\"random_forest\",1.0,11.0,1.0261172501681401,0.01760026815272455,80.0,-0.9677038850069307,323],[\"random_forest\",0.0,11.0,1.1926062411023786,0.03380730290941413,90.0,-0.9610118172869225,324],[\"random_forest\",1.0,12.0,1.6828883637367165,0.011697424884444394,80.0,-0.9693421565224394,325],[\"random_forest\",1.0,12.0,1.3879213076131642,0.021868266737395177,40.0,-0.9658215059361525,326],[\"random_forest\",1.0,4.0,1.0082870660018635,0.012516164683490653,50.0,-0.948815779359886,327],[\"random_forest\",1.0,11.0,1.0760109305676449,0.010010253127104789,70.0,-0.9690903539202467,328],[\"random_forest\",1.0,11.0,1.5075761760671775,0.015517741720243665,100.0,-0.9684220007267523,329],[\"random_forest\",1.0,7.0,1.245015298989562,0.0115182836867601,90.0,-0.9664832830287602,330],[\"random_forest\",0.0,12.0,1.8009975902150734,0.010730491257025767,60.0,-0.9689190359447106,331],[\"random_forest\",1.0,10.0,1.3458976938768505,0.013256723969759764,80.0,-0.9687922190033468,332],[\"random_forest\",1.0,9.0,4.986381456783599,0.018204365688678677,110.0,-0.9655590580604652,333],[\"random_forest\",1.0,11.0,1.9760395996175524,0.03522939341787845,70.0,-0.9629066331677459,334],[\"random_forest\",1.0,12.0,1.0051608577624298,0.026280165347915538,50.0,-0.9651195446175789,335],[\"random_forest\",1.0,10.0,1.146585316444711,0.016232812353155802,60.0,-0.9665022347950993,336],[\"random_forest\",1.0,5.0,1.6253574086685272,0.01691477557768748,80.0,-0.9599157760146654,337],[\"random_forest\",0.0,12.0,1.000038809041524,0.014775054446625111,40.0,-0.9671932940328912,338],[\"random_forest\",1.0,11.0,1.4217948127194047,0.010002447304991285,50.0,-0.9689351750598105,339],[\"random_forest\",1.0,10.0,1.7313054414425904,0.013893087377597544,30.0,-0.9687159350101712,340],[\"random_forest\",1.0,3.0,1.526938339632309,0.049904204948850034,70.0,-0.937493152187832,341],[\"random_forest\",1.0,11.0,1.242413864453863,0.012646051388673946,60.0,-0.9681693023562846,342],[\"random_forest\",1.0,12.0,1.8217651050034407,0.010994035774571427,40.0,-0.9682908425193453,343],[\"random_forest\",1.0,12.0,1.0959245234079564,0.027493887449677634,90.0,-0.9659806011077163,344],[\"random_forest\",0.0,10.0,2.064391529280183,0.019518954684153177,80.0,-0.9647152657459233,345],[\"random_forest\",1.0,11.0,1.3195547118879751,0.04420101708618872,40.0,-0.957751829154232,346],[\"random_forest\",1.0,12.0,1.4877837772160118,0.015588129344936242,50.0,-0.9679961592146725,347],[\"random_forest\",1.0,11.0,1.6593733226894996,0.020615741724359676,90.0,-0.9664924791754207,348],[\"random_forest\",1.0,12.0,1.990635745438286,0.011643502853432306,100.0,-0.9688789036916482,349],[\"random_forest\",1.0,9.0,1.5837657376404177,0.013750008313061241,20.0,-0.9660094246684592,350],[\"random_forest\",1.0,12.0,1.1713813460999145,0.016864832075848962,10.0,-0.96516849697806,351],[\"random_forest\",0.0,8.0,1.4203919198231294,0.010712285195541679,60.0,-0.9665345833415802,352],[\"random_forest\",1.0,9.0,1.8378454994301858,0.012744102886898324,30.0,-0.9659101747664695,353],[\"random_forest\",1.0,11.0,4.352480320814015,0.02526612719989133,70.0,-0.9650261615392065,354],[\"random_forest\",1.0,10.0,1.2649310561254639,0.010015908365167041,80.0,-0.9689919722714655,355],[\"random_forest\",1.0,11.0,2.1779019389478953,0.018417362612271962,100.0,-0.9672739498644061,356],[\"random_forest\",1.0,12.0,4.810613431240105,0.039996514371312054,50.0,-0.9605780575501583,357],[\"random_forest\",1.0,10.0,1.716006112478271,0.04862373956113482,110.0,-0.9598366610268729,358],[\"random_forest\",0.0,11.0,1.4475423403064467,0.011907854131647501,70.0,-0.9679412850004409,359],[\"random_forest\",1.0,12.0,1.0778795206045615,0.014469908364705145,40.0,-0.9676333210903515,360],[\"random_forest\",1.0,12.0,1.9043403233957856,0.015769188850084177,90.0,-0.9686662550290434,361],[\"random_forest\",1.0,11.0,1.3396095446946663,0.022298859021105896,60.0,-0.9658808161905452,362],[\"random_forest\",1.0,7.0,1.561000724137926,0.017335067716133377,70.0,-0.9648658495902588,363],[\"random_forest\",1.0,6.0,1.1638596565753707,0.013072772885980882,20.0,-0.9648544155515254,364],[\"random_forest\",1.0,10.0,2.105360156457766,0.012114017504267317,30.0,-0.9679239260507274,365],[\"random_forest\",0.0,12.0,1.7734378226857614,0.010858601959881399,40.0,-0.968376441891136,366],[\"random_forest\",1.0,11.0,1.0009050120145342,0.014134909811226348,50.0,-0.9679786382317899,367],[\"random_forest\",1.0,12.0,1.645928202268336,0.010006062478482047,60.0,-0.969861589004505,368],[\"random_forest\",1.0,10.0,1.2869896080821617,0.011348908030949208,80.0,-0.9679965505400624,369],[\"random_forest\",1.0,9.0,2.2913606275551857,0.014852739267099879,50.0,-0.9667849521030869,370],[\"random_forest\",1.0,11.0,1.3796384165087656,0.010024111693673862,100.0,-0.9696912493424434,371],[\"random_forest\",1.0,12.0,1.4833228040548108,0.0233964194493416,90.0,-0.9656319301853982,372],[\"random_forest\",1.0,12.0,2.012123539815878,0.019247066176403753,60.0,-0.9662663114434897,373],[\"random_forest\",1.0,12.0,1.6555044207321128,0.013426195157288967,60.0,-0.9674751247442925,374],[\"random_forest\",0.0,12.0,1.8785342121475443,0.017926685305969345,30.0,-0.9653639945825645,375],[\"random_forest\",1.0,5.0,2.1789662294452805,0.010000189785230273,10.0,-0.954707708946923,376],[\"random_forest\",1.0,8.0,2.4057377939496587,0.016432324948800623,30.0,-0.9642175854525178,377],[\"random_forest\",1.0,11.0,1.9586885331225905,0.045544749835531806,10.0,-0.9566025798578793,378],[\"random_forest\",1.0,10.0,2.0412483822731904,0.012525563563074656,20.0,-0.9659751592390143,379],[\"random_forest\",1.0,11.0,2.273531122059759,0.04317366113758979,40.0,-0.9625173469500873,380],[\"random_forest\",0.0,10.0,1.7497806807086862,0.030268686515889466,110.0,-0.9631023111487973,381],[\"random_forest\",1.0,9.0,1.8416279674665064,0.028288472264865836,20.0,-0.9616393289500854,382],[\"random_forest\",1.0,11.0,1.6299213745875614,0.01523362919686219,50.0,-0.9687144522538115,383],[\"random_forest\",1.0,12.0,2.2402978285285617,0.02133305796247447,70.0,-0.9665359438087558,384],[\"random_forest\",1.0,11.0,1.5751076092241223,0.01151826600639978,40.0,-0.9694969929731596,385],[\"random_forest\",1.0,10.0,2.1331578931770667,0.014007387341606057,100.0,-0.9679049620554698,386],[\"random_forest\",1.0,11.0,1.6999993495267052,0.03711584012998052,120.0,-0.9631333206287099,387],[\"random_forest\",0.0,12.0,1.38637622349386,0.012411755254864812,30.0,-0.9671740029140657,388],[\"random_forest\",1.0,10.0,1.9325114774093173,0.013169303959682247,50.0,-0.9672849344903872,389],[\"random_forest\",1.0,11.0,1.7857603463418297,0.031948282454511624,60.0,-0.9635474254363251,390],[\"random_forest\",1.0,7.0,2.359438686344292,0.020455749857187636,40.0,-0.9636620532032426,391],[\"random_forest\",1.0,4.0,2.0730191041069066,0.024451329965602724,20.0,-0.946103310477668,392],[\"random_forest\",1.0,8.0,1.431897993684276,0.017273573077014562,70.0,-0.9653341988228064,393],[\"random_forest\",1.0,9.0,1.5117414019776365,0.01068726463227556,50.0,-0.968152239957843,394],[\"random_forest\",0.0,12.0,2.4838295074231747,0.014889339865554158,10.0,-0.9657927312910832,395],[\"random_forest\",1.0,12.0,1.636932914279144,0.01876626446442147,60.0,-0.9667126363939424,396],[\"random_forest\",1.0,2.0,1.969643551281325,0.01587044765232727,90.0,-0.9220654055019775,397],[\"random_forest\",1.0,11.0,1.8904554491719894,0.01149321785262893,70.0,-0.9686934704870126,398],[\"random_forest\",1.0,12.0,1.7198748664714774,0.013910185513121358,30.0,-0.967531854696896,399],[\"random_forest\",1.0,10.0,1.3570682256447193,0.012284889888694285,80.0,-0.9683826480672401,400],[\"random_forest\",1.0,12.0,1.1032182476076202,0.010804148370234835,80.0,-0.96889573068341,401],[\"random_forest\",1.0,12.0,1.21297919282991,0.010235040470867134,100.0,-0.9696924386047608,402],[\"random_forest\",1.0,12.0,1.0007556497360481,0.01011175188567018,80.0,-0.9692886336036944,403],[\"random_forest\",1.0,12.0,1.5613247163054051,0.010001185441808296,90.0,-0.9698118142492593,404],[\"random_forest\",1.0,12.0,1.3472535790531168,0.011235022942017664,100.0,-0.9690616740492952,405],[\"random_forest\",1.0,11.0,1.153630639315736,0.012666410278473327,70.0,-0.9691820066066608,406],[\"random_forest\",1.0,12.0,1.2152309209184593,0.011726808510814594,120.0,-0.9688924686194182,407],[\"random_forest\",1.0,11.0,1.486077414389114,0.010002542025816227,100.0,-0.9689662028831008,408],[\"random_forest\",1.0,12.0,1.248017597342105,0.013298573697437217,110.0,-0.9685690871004164,409],[\"random_forest\",1.0,11.0,1.3822847063825936,0.012142526241426449,110.0,-0.968426721089267,410],[\"random_forest\",1.0,12.0,1.5457085317977477,0.010677721527119804,90.0,-0.969342767968361,411],[\"random_forest\",1.0,12.0,1.8009768135553967,0.014324805147379037,100.0,-0.9688166554396024,412],[\"random_forest\",1.0,12.0,1.6179484520987484,0.01086364458439627,110.0,-0.9690676448187195,413],[\"random_forest\",1.0,12.0,1.6973648257557685,0.010022509071185372,120.0,-0.9697220662168907,414],[\"random_forest\",1.0,12.0,1.5611809643233456,0.010029390197415906,90.0,-0.969287765350486,415],[\"random_forest\",1.0,12.0,1.8409336462175978,0.013192557680666683,100.0,-0.9682267293572389,416],[\"random_forest\",1.0,12.0,2.044648520005733,0.010008212877278077,110.0,-0.969315029724129,417],[\"random_forest\",1.0,12.0,1.4723050676686316,0.011397184097211173,90.0,-0.9688645163691135,418],[\"random_forest\",1.0,12.0,2.1302002139510714,0.015129401911661138,110.0,-0.9683723299173133,419],[\"random_forest\",1.0,12.0,1.72209240338217,0.012230985754172708,130.0,-0.9687162927060353,420],[\"random_forest\",1.0,12.0,2.202823315087521,0.013880412940759005,120.0,-0.9679529789036909,421],[\"random_forest\",1.0,12.0,1.8900972812384447,0.012813536674763008,120.0,-0.968396494260134,422],[\"random_forest\",1.0,11.0,1.9969409518139767,0.01625374413141846,120.0,-0.9682971342978782,423],[\"random_forest\",1.0,12.0,1.6540086003587093,0.011665970154443577,130.0,-0.9693767001597788,424],[\"random_forest\",1.0,11.0,1.7706951348813276,0.014783630487391447,130.0,-0.9687398853469192,425],[\"random_forest\",1.0,12.0,2.0876869045370796,0.013412460363793762,120.0,-0.9685418380129216,426],[\"random_forest\",1.0,12.0,4.176481267934136,0.010896696708034044,100.0,-0.9681752547823311,427],[\"random_forest\",1.0,11.0,2.3217393662605756,0.016674000083635777,130.0,-0.9670655966094004,428],[\"random_forest\",1.0,12.0,1.965916899804948,0.015463796221556747,110.0,-0.9680463925543594,429],[\"random_forest\",1.0,12.0,1.6801282648308373,0.012009897792828931,90.0,-0.9686059511750281,430],[\"random_forest\",1.0,11.0,1.5909701791584134,0.014420356389560251,110.0,-0.9679499247313128,431],[\"random_forest\",0.0,11.0,2.2057892803609396,0.010040940518903956,120.0,-0.9687055801734893,432],[\"random_forest\",1.0,12.0,1.879336279959513,0.01305024854724842,80.0,-0.9691952841548479,433],[\"random_forest\",1.0,12.0,1.4701729703475213,0.010001645357050021,90.0,-0.9691155454922157,434],[\"random_forest\",1.0,11.0,1.7720712041703677,0.011078509945363257,120.0,-0.9689531637988233,435],[\"random_forest\",1.0,11.0,1.8534594918801637,0.017888403048545408,100.0,-0.9671815420422785,436],[\"random_forest\",1.0,12.0,2.435786807299511,0.013959045411707432,70.0,-0.9678256024893089,437],[\"random_forest\",1.0,12.0,1.672954192669179,0.0156614643532094,90.0,-0.9674240904104483,438],[\"random_forest\",1.0,11.0,1.5388186057284523,0.012475742641197397,110.0,-0.9689496877287591,439],[\"random_forest\",0.0,12.0,1.990791819008389,0.011563124538182694,80.0,-0.9685124488647011,440],[\"random_forest\",1.0,11.0,1.7838537512205996,0.010036391878797403,60.0,-0.9692142787224016,441],[\"random_forest\",1.0,12.0,1.2978582776735872,0.010699179487733291,140.0,-0.9690502400105618,442],[\"random_forest\",1.0,12.0,1.4181941248781667,0.017066066189009233,130.0,-0.9673880793028973,443],[\"random_forest\",1.0,3.0,2.1124791034492554,0.013508433905032011,90.0,-0.9393017969949279,444],[\"random_forest\",1.0,11.0,2.5724095076082163,0.014669400476468662,100.0,-0.9681861201763574,445],[\"random_forest\",1.0,12.0,2.261007721919608,0.011943579160601087,80.0,-0.9684508823748578,446],[\"random_forest\",1.0,12.0,1.5717122861383115,0.012920325972750349,120.0,-0.9689897924667552,447],[\"random_forest\",0.0,11.0,1.30829397464768,0.01591099403012044,60.0,-0.9668919367959846,448],[\"random_forest\",1.0,11.0,1.1622832602939117,0.010723297825496587,70.0,-0.9691466986619197,449],[\"random_forest\",1.0,12.0,2.0502945915927886,0.013888572839416292,100.0,-0.9686955127163906,450],[\"random_forest\",1.0,6.0,1.911533845873905,0.011509245565854245,110.0,-0.9649026005473749,451],[\"random_forest\",1.0,11.0,4.581334931478775,0.018907034446245384,70.0,-0.9659254150560646,452],[\"random_forest\",1.0,12.0,1.7021790581104097,0.015309276788145159,80.0,-0.9676157511917951,453],[\"random_forest\",1.0,12.0,1.4895971375511765,0.01642005308362743,60.0,-0.9681859612004178,454],[\"random_forest\",1.0,11.0,2.3596906580757446,0.01004466037587829,100.0,-0.9686470281120395,455],[\"random_forest\",1.0,12.0,1.6251624470686408,0.012159104561384415,90.0,-0.9686820547916568,456],[\"random_forest\",0.0,12.0,1.397442145454339,0.01265790597620344,50.0,-0.9675205704624136,457],[\"random_forest\",1.0,11.0,1.0852898440032526,0.010000112075556969,130.0,-0.9691714866795803,458],[\"random_forest\",1.0,11.0,1.8291398886583181,0.017767135536144927,120.0,-0.9669702935908342,459],[\"random_forest\",1.0,12.0,2.144924459685851,0.014384287613101145,70.0,-0.9674662220916745,460],[\"random_forest\",1.0,12.0,1.2126682245714662,0.010638514808167096,80.0,-0.9692147067345467,461],[\"random_forest\",1.0,10.0,1.9608585857331642,0.019947037875065757,110.0,-0.9665472219287792,462],[\"random_forest\",1.0,4.0,1.7506227229696967,0.011133043903539377,100.0,-0.9479917612064478,463],[\"random_forest\",0.0,12.0,1.2995183408715851,0.013371811613975375,50.0,-0.9679596772937618,464],[\"random_forest\",1.0,11.0,1.5280825035122843,0.035689242729786144,60.0,-0.9611150660452394,465],[\"random_forest\",1.0,5.0,2.2246120308624198,0.014985160560760576,90.0,-0.9571367237288178,466],[\"random_forest\",1.0,11.0,1.689503309357002,0.010032423976596292,70.0,-0.9692835586025453,467],[\"random_forest\",1.0,12.0,1.4405425176575812,0.012023331865854157,110.0,-0.9686633445464566,468],[\"random_forest\",1.0,12.0,2.504648509697417,0.01264144418686167,80.0,-0.9688323787714757,469],[\"random_forest\",1.0,11.0,1.5934053121097989,0.010018799005959627,140.0,-0.9697028484715756,470],[\"random_forest\",1.0,12.0,2.0451714870610727,0.011232967209738109,60.0,-0.9687502738131267,471],[\"random_forest\",0.0,11.0,1.0985328833706163,0.033735785794044286,120.0,-0.9619866883209017,472],[\"random_forest\",1.0,10.0,1.1859085091613781,0.016612982403644047,70.0,-0.9675096989539279,473],[\"random_forest\",1.0,12.0,1.929570172897458,0.013960952469030757,50.0,-0.9670993973399449,474],[\"random_forest\",1.0,11.0,2.330870682249331,0.01740203585267469,140.0,-0.9669599846125965,475],[\"random_forest\",1.0,11.0,1.8343703830381692,0.013321196229026578,140.0,-0.968249679979905,476],[\"random_forest\",1.0,10.0,1.5844484221017483,0.015884061419758896,140.0,-0.9675658358039875,477],[\"random_forest\",1.0,11.0,2.428754697641278,0.011603892677148456,140.0,-0.9686655518662335,478],[\"random_forest\",1.0,7.0,2.2010516027195512,0.034719748813046764,130.0,-0.9603213664378227,479],[\"random_forest\",0.0,10.0,2.1090925091337662,0.03887035362399737,150.0,-0.9617440971515174,480],[\"random_forest\",1.0,12.0,1.721799193766233,0.0184338775215525,150.0,-0.9670302642068221,481],[\"random_forest\",1.0,12.0,2.651543492516266,0.01420943962140658,140.0,-0.9687243729638889,482],[\"random_forest\",1.0,11.0,2.0198807373819836,0.027225143703742113,130.0,-0.9651170621471372,483],[\"random_forest\",1.0,12.0,1.8114237703709573,0.01004515582816285,130.0,-0.9696874430915816,484],[\"random_forest\",1.0,11.0,1.6556139467085171,0.012706236440105143,130.0,-0.9684319856386517,485],[\"random_forest\",1.0,12.0,1.9096705351998216,0.04270565155421659,40.0,-0.9622578248431347,486],[\"random_forest\",1.0,11.0,1.5126315886541448,0.04103112895422084,120.0,-0.962503720877725,487],[\"random_forest\",0.0,10.0,1.6201198113546367,0.015108009679677577,150.0,-0.9672436863485179,488],[\"random_forest\",1.0,12.0,1.3546975370290306,0.011074287454744713,40.0,-0.9684195732864438,489],[\"random_forest\",1.0,6.0,2.2894074537642943,0.012131118013193498,140.0,-0.9636437006539066,490],[\"random_forest\",1.0,2.0,1.8933323901146286,0.022380528309656447,50.0,-0.9267359535172941,491],[\"random_forest\",1.0,11.0,1.7712039235399915,0.01086099913230523,60.0,-0.9694488446640652,492],[\"random_forest\",1.0,12.0,4.450159166362431,0.013614777718068561,130.0,-0.9681720263478651,493],[\"random_forest\",1.0,12.0,2.0389534781719507,0.01660888461104716,120.0,-0.967917802419823,494],[\"random_forest\",1.0,11.0,1.4452670419667675,0.031203726490187027,50.0,-0.9632309746568444,495],[\"random_forest\",0.0,10.0,1.5590724744325024,0.01972792643921481,60.0,-0.966905507838214,496],[\"random_forest\",1.0,12.0,1.7252942826522846,0.014438978400092183,40.0,-0.9677749472519361,497],[\"random_forest\",1.0,11.0,2.1250039232808913,0.010030465262644263,50.0,-0.9697045360623193,498],[\"random_forest\",1.0,12.0,2.5649541371614326,0.046925433402232444,150.0,-0.9600397161029682,499],[\"random_forest\",1.0,12.0,1.9680696802316848,0.045835085552282225,120.0,-0.9621646496563756,500],[\"random_forest\",1.0,11.0,1.841285659832303,0.011866245025258834,140.0,-0.9687274240790374,501],[\"random_forest\",1.0,8.0,1.3865841099349205,0.013028755920915866,130.0,-0.9669106653845624,502],[\"random_forest\",1.0,10.0,2.1996653911585686,0.010032856858131203,30.0,-0.9688864305909427,503],[\"random_forest\",0.0,12.0,2.3222072522257324,0.029056856417020147,50.0,-0.9639918243321248,504],[\"random_forest\",1.0,7.0,2.107689701615625,0.0212440440103224,40.0,-0.963718293999109,505],[\"random_forest\",1.0,11.0,2.51545294141474,0.025670817067385658,50.0,-0.9653004867519205,506],[\"random_forest\",1.0,12.0,2.3660073845549765,0.015531060707799797,30.0,-0.9671660021441819,507],[\"random_forest\",1.0,10.0,2.4459964596526076,0.017458334674234535,60.0,-0.9661510141432884,508],[\"random_forest\",1.0,11.0,2.1749333045157275,0.010948442734301275,70.0,-0.9691458701526958,509],[\"random_forest\",1.0,12.0,2.622148417897158,0.012524395997727235,60.0,-0.9685985282215402,510],[\"random_forest\",1.0,11.0,1.9421391081820316,0.011584027127649875,40.0,-0.9679763575385023,511],[\"random_forest\",0.0,12.0,2.8346422153234094,0.013520537289441092,20.0,-0.9666226040392203,512],[\"random_forest\",1.0,11.0,2.2443207391660724,0.01909170128327261,30.0,-0.9660564142875321,513],[\"random_forest\",1.0,10.0,2.0895925193210196,0.0149838693029507,50.0,-0.9676707904964257,514],[\"random_forest\",1.0,12.0,2.7040116519327393,0.016203940616607716,40.0,-0.966836640684067,515],[\"random_forest\",1.0,5.0,2.41782085767972,0.018166174360204124,60.0,-0.9580104310350726,516],[\"random_forest\",1.0,11.0,1.8318622084273326,0.010664376700081376,70.0,-0.9690710444580435,517],[\"random_forest\",1.0,12.0,2.130562586808751,0.010065052829999557,50.0,-0.9692994867688024,518],[\"random_forest\",1.0,12.0,1.669140745863431,0.044440407647308146,70.0,-0.962648126061023,519],[\"random_forest\",0.0,11.0,1.9767106596054225,0.03652599303633723,30.0,-0.9601559886594146,520],[\"random_forest\",1.0,10.0,2.252176833354064,0.014219309277853035,80.0,-0.9673247090475855,521],[\"random_forest\",1.0,12.0,2.0396264945196454,0.012312814253874392,50.0,-0.9690730622295847,522],[\"random_forest\",1.0,11.0,1.7390912670128653,0.011586825311203659,20.0,-0.9676021526344994,523],[\"random_forest\",1.0,12.0,2.522962021116643,0.013009841876238538,80.0,-0.9689102097228327,524],[\"random_forest\",1.0,9.0,1.8710097400104904,0.03293055124193338,40.0,-0.9629143863020314,525],[\"random_forest\",1.0,6.0,1.448925844081153,0.010030966919177702,60.0,-0.9637988489420466,526],[\"random_forest\",1.0,3.0,2.7587544165012066,0.010913730300257544,70.0,-0.9380922285584121,527],[\"random_forest\",0.0,12.0,1.7973199707019074,0.023431696175398836,60.0,-0.9638753530557539,528],[\"random_forest\",1.0,10.0,1.5311251683869371,0.015661125978039894,40.0,-0.9666346709244825,529],[\"random_forest\",1.0,11.0,2.0078738345098053,0.014841635738351532,50.0,-0.9684298944936001,530],[\"random_forest\",1.0,11.0,2.33663170503917,0.01729449796822933,30.0,-0.967258825749536,531],[\"random_forest\",1.0,12.0,1.9049646481170026,0.01347838889462111,80.0,-0.9682681762190325,532],[\"random_forest\",1.0,12.0,1.6382763553504156,0.012453727716237593,90.0,-0.9687233059907558,533],[\"random_forest\",1.0,11.0,2.1430778680666904,0.020753777457941235,70.0,-0.9660870844149582,534],[\"random_forest\",1.0,12.0,1.2729968209736633,0.01151155955480321,60.0,-0.9691369032982562,535],[\"random_forest\",0.0,10.0,1.746390335208642,0.016568563341338327,10.0,-0.9652381223251496,536],[\"random_forest\",1.0,9.0,2.4112736095874974,0.013979855861363823,40.0,-0.9667944967739225,537],[\"random_forest\",1.0,8.0,2.2601235340023464,0.010871380102568171,50.0,-0.9667070049770048,538],[\"random_forest\",1.0,11.0,1.3090552510590852,0.010073701982047528,20.0,-0.9680575392135098,539],[\"random_forest\",1.0,12.0,1.4929063110216427,0.01199726205391528,70.0,-0.9694962592380537,540],[\"random_forest\",1.0,4.0,2.0782751059400177,0.04022326677661099,80.0,-0.9479252695197063,541],[\"random_forest\",1.0,12.0,2.598443962199427,0.014603007932401815,40.0,-0.967225229853375,542],[\"random_forest\",1.0,10.0,1.6713814322242957,0.013008854866398837,60.0,-0.9683282385519086,543],[\"random_forest\",0.0,11.0,1.971742698348187,0.026499788078498496,30.0,-0.9636918764780674,544],[\"random_forest\",1.0,12.0,1.6022570902890858,0.01846049239879507,50.0,-0.9669984445610635,545],[\"random_forest\",1.0,11.0,1.8443755330143772,0.015559264349445514,60.0,-0.9674845990988473,546],[\"random_forest\",1.0,12.0,1.3960347444200956,0.0380413746321161,90.0,-0.9631970577515748,547],[\"random_forest\",1.0,9.0,2.1968326295762592,0.010072111606144225,40.0,-0.9674133564772953,548],[\"random_forest\",1.0,12.0,1.7830392524666618,0.011405547915479703,90.0,-0.9693871405988896,549],[\"random_forest\",1.0,11.0,2.4754501741027903,0.019221522115926058,70.0,-0.9664312336946862,550],[\"random_forest\",1.0,12.0,1.9400743478349092,0.012438328716656216,10.0,-0.965748254714748,551],[\"random_forest\",1.0,10.0,1.3363598321616879,0.010736619961525196,30.0,-0.9685024578383427,552],[\"random_forest\",0.0,11.0,1.5120388742773547,0.021839614033170637,60.0,-0.9647969060053717,553],[\"random_forest\",1.0,11.0,2.310491995463809,0.04867001181379607,20.0,-0.9575989187581649,554],[\"random_forest\",1.0,12.0,2.0636463608976383,0.013628795842831542,50.0,-0.9680825259510948,555],[\"random_forest\",1.0,12.0,2.9040144088139224,0.02991434412589512,80.0,-0.9652909787678399,556],[\"random_forest\",1.0,10.0,1.6857537274398797,0.024599332960353312,50.0,-0.9656563910794906,557],[\"random_forest\",1.0,11.0,1.1362732832667297,0.010013839920542191,70.0,-0.9697249614133294,558],[\"random_forest\",1.0,12.0,1.5713417990723788,0.016998064798251483,80.0,-0.9670003217000428,559],[\"random_forest\",1.0,11.0,1.4277948611531075,0.014293317522013817,60.0,-0.9682404379748004,560],[\"random_forest\",0.0,12.0,1.8966970495488704,0.01582269088530671,40.0,-0.9668665709619281,561],[\"random_forest\",1.0,12.0,1.2442522261130418,0.014873844957980269,90.0,-0.9686391129445847,562],[\"random_forest\",1.0,11.0,1.2360106828995252,0.010002343006886544,70.0,-0.9691711840138493,563],[\"random_forest\",1.0,12.0,1.0798471472812492,0.011861007656426937,80.0,-0.9686047894277773,564],[\"random_forest\",1.0,11.0,1.1617301490236864,0.013108154965959193,90.0,-0.968799700044197,565],[\"random_forest\",1.0,7.0,1.019136631711371,0.010644320381465703,70.0,-0.965610624352261,566],[\"random_forest\",1.0,12.0,1.1513295302869704,0.022825777113286636,80.0,-0.966596363837496,567],[\"random_forest\",1.0,5.0,1.0005884967563827,0.017805831246465603,100.0,-0.9591561920324427,568],[\"random_forest\",0.0,10.0,1.0673494731146218,0.011561429749241717,110.0,-0.9675684313919246,569],[\"random_forest\",1.0,12.0,1.3263760883037807,0.012599928160974678,90.0,-0.9689965092002036,570],[\"random_forest\",1.0,11.0,1.1363758848799435,0.019850980413924302,100.0,-0.9667562172020024,571],[\"random_forest\",1.0,10.0,4.019772219996794,0.016674481018844292,70.0,-0.9669427999249706,572],[\"random_forest\",1.0,9.0,1.0048533618222337,0.013944453787967919,80.0,-0.9676056684485485,573],[\"random_forest\",1.0,12.0,1.2364765386889018,0.015392789184217947,90.0,-0.9677614373542988,574],[\"random_forest\",1.0,11.0,1.3995260921905126,0.04198537298257323,70.0,-0.961546040645831,575],[\"random_forest\",1.0,12.0,1.4761634242340087,0.010717631963376134,100.0,-0.9694947183943312,576],[\"random_forest\",1.0,11.0,1.57695243443068,0.010068860294124425,60.0,-0.9696202604709487,577],[\"random_forest\",1.0,11.0,1.7191077716673437,0.01003568324205197,50.0,-0.9691475210566841,578],[\"random_forest\",1.0,11.0,1.6483900300062413,0.011641313425511243,60.0,-0.9691242983405827,579],[\"random_forest\",1.0,11.0,1.512934392000885,0.011181841928815954,70.0,-0.9692140677735586,580],[\"random_forest\",1.0,12.0,1.7832795946100508,0.012038075929995378,60.0,-0.9683235365327718,581],[\"random_forest\",1.0,12.0,1.3407762821996831,0.010065662662732273,50.0,-0.9695771321328706,582],[\"random_forest\",1.0,10.0,1.4588310690739608,0.010019052445232273,60.0,-0.9688467110638773,583],[\"random_forest\",1.0,11.0,1.8443406516156777,0.01320478170453917,50.0,-0.9684304142226331,584],[\"random_forest\",1.0,12.0,1.6703501941746217,0.010805306704154466,80.0,-0.9690138406348506,585],[\"random_forest\",1.0,12.0,1.60003970063389,0.01247759798214653,70.0,-0.9691607313458198,586],[\"random_forest\",1.0,11.0,1.3887643257145108,0.01136060758769236,60.0,-0.9680260038901046,587],[\"random_forest\",1.0,12.0,1.7556419775875463,0.010039680614252604,70.0,-0.9692492778869525,588],[\"random_forest\",1.0,11.0,2.0062823952941367,0.013030713890618075,50.0,-0.967677733464865,589],[\"random_forest\",1.0,12.0,1.5492733707230293,0.012189815024690239,40.0,-0.9684772173507001,590],[\"random_forest\",1.0,11.0,1.3178408002003135,0.01061026273195172,60.0,-0.9691871580385503,591],[\"random_forest\",1.0,10.0,1.9058116033137755,0.01400293384660051,80.0,-0.9681715157905205,592],[\"random_forest\",1.0,11.0,1.1940350488112157,0.010834499186987617,70.0,-0.9684328600063195,593],[\"random_forest\",1.0,12.0,1.4579602073377778,0.01159325207024612,50.0,-0.9695390237658085,594],[\"random_forest\",1.0,12.0,1.792891017516519,0.012705998774107805,50.0,-0.9686260371735519,595],[\"random_forest\",1.0,10.0,1.6462271207286914,0.013757105869253201,70.0,-0.9688754795944873,596],[\"random_forest\",1.0,11.0,4.293993839820763,0.01482652772216762,80.0,-0.9666451511075783,597],[\"random_forest\",1.0,12.0,2.1241218867360674,0.011413440741267981,40.0,-0.9693185027369633,598],[\"random_forest\",1.0,12.0,1.2709634874430116,0.010066103733599846,60.0,-0.9696621445165765,599],[\"random_forest\",1.0,11.0,1.8654882884710762,0.01000671723332036,40.0,-0.9698499745892246,600],[\"random_forest\",1.0,10.0,1.7356340757271975,0.01223636026720808,70.0,-0.9678027527552197,601],[\"random_forest\",1.0,12.0,1.0872717159947074,0.013063078815027551,60.0,-0.9683404796992587,602],[\"random_forest\",1.0,11.0,1.9923193256429463,0.010841978672227972,80.0,-0.968975768954544,603],[\"random_forest\",1.0,11.0,1.5605974319222633,0.014215097936125153,40.0,-0.967634696843675,604],[\"random_forest\",1.0,12.0,1.3887677235574791,0.011942712043351664,30.0,-0.968503289404796,605],[\"random_forest\",1.0,12.0,1.1412399572665533,0.014981183093521554,40.0,-0.9685001893739736,606],[\"random_forest\",1.0,10.0,4.672289135118871,0.010022022875267036,30.0,-0.9682990939820568,607],[\"random_forest\",1.0,12.0,1.505630440001214,0.0131721203457057,110.0,-0.9684164793700807,608],[\"random_forest\",1.0,11.0,1.283234099897519,0.01594802756835141,90.0,-0.9671735107000987,609],[\"random_forest\",1.0,12.0,1.8571857491900379,0.012142553528014294,30.0,-0.969420708979984,610],[\"random_forest\",1.0,11.0,1.646905463865192,0.01001551903079917,20.0,-0.9683830118775634,611],[\"random_forest\",1.0,10.0,1.445766947759111,0.010858114451200638,50.0,-0.9690019052104615,612],[\"random_forest\",1.0,11.0,1.7007422406441737,0.011318999985015857,40.0,-0.9679064356401409,613],[\"random_forest\",0.0,12.0,1.0060503529971307,0.013596320353231831,60.0,-0.9685128401900911,614],[\"random_forest\",1.0,12.0,3.7800336545661097,0.014429369566271586,70.0,-0.9674720369423887,615],[\"random_forest\",1.0,11.0,1.8621141759587767,0.01003578735896058,80.0,-0.9691447114626746,616],[\"random_forest\",1.0,12.0,1.2221859848763414,0.012646047651242661,30.0,-0.9690245990258407,617],[\"random_forest\",1.0,11.0,1.3526686252311748,0.011319716490174388,100.0,-0.9689372295181071,618],[\"random_forest\",1.0,10.0,1.597661497489947,0.01610119064135795,60.0,-0.968266503914437,619],[\"random_forest\",1.0,12.0,1.9686499411235343,0.013594724900062229,90.0,-0.9682744741120246,620],[\"random_forest\",1.0,9.0,1.7909533035067977,0.012324804179083474,50.0,-0.9672192499122622,621],[\"random_forest\",0.0,12.0,1.1285536519079664,0.010011871814457694,110.0,-0.9691339347283068,622],[\"random_forest\",1.0,11.0,1.5022671266299343,0.010839980328119586,40.0,-0.9684156875476121,623],[\"random_forest\",1.0,12.0,1.722534854425015,0.014984759928096806,50.0,-0.9677661485451248,624],[\"random_forest\",1.0,11.0,2.0341824023352713,0.011581005533346634,70.0,-0.9682443695720762,625],[\"random_forest\",1.0,12.0,1.301295257049765,0.010001235764334687,90.0,-0.9699429785711257,626],[\"random_forest\",1.0,10.0,1.4284534394306778,0.01280015074541178,80.0,-0.9687972940044959,627],[\"random_forest\",1.0,11.0,1.5955181875673246,0.01384766892038933,60.0,-0.9673936679186206,628],[\"random_forest\",1.0,12.0,2.1965042305525406,0.01088422155505655,40.0,-0.9683568114198239,629],[\"random_forest\",1.0,12.0,4.87422648575073,0.017048066181642917,70.0,-0.9671762683212051,630],[\"random_forest\",0.0,11.0,1.1710721718905202,0.01000561771680915,90.0,-0.9689330502852331,631],[\"random_forest\",1.0,12.0,1.3678195389770278,0.01552274814398262,90.0,-0.9681618580021893,632],[\"random_forest\",1.0,3.0,1.010035490980406,0.0118160424705487,80.0,-0.9366612494392353,633],[\"random_forest\",1.0,11.0,1.0630546864949095,0.012992238390594376,80.0,-0.9684478679464648,634],[\"random_forest\",1.0,12.0,1.2709055522136417,0.027911564383930006,80.0,-0.9646469336069579,635],[\"random_forest\",1.0,8.0,1.1996442539642511,0.014429657934775806,90.0,-0.9658315214203478,636],[\"random_forest\",1.0,10.0,1.003276877860472,0.010828348962742252,100.0,-0.9688752227872003,637],[\"random_forest\",1.0,11.0,1.105229523100596,0.016251115638427495,70.0,-0.9670996663761504,638],[\"random_forest\",1.0,10.0,1.3466588602940526,0.013490484074175118,90.0,-0.9677026498861692,639],[\"random_forest\",0.0,12.0,1.2496991159210598,0.011967910173163652,70.0,-0.9681151343620923,640],[\"random_forest\",1.0,11.0,1.0824713318341304,0.01776368775707592,100.0,-0.9678272381071489,641],[\"random_forest\",1.0,9.0,1.5332087564546883,0.012515886768798995,70.0,-0.9673159623136773,642],[\"random_forest\",1.0,6.0,1.4502383592529784,0.010022216018262609,80.0,-0.9612073852077083,643],[\"random_forest\",1.0,12.0,1.2920659893033997,0.014908179592891407,60.0,-0.9684968722798492,644],[\"random_forest\",1.0,12.0,1.2086944525493724,0.011341563145596877,80.0,-0.9685135341812121,645],[\"random_forest\",1.0,11.0,1.398032703547701,0.010547847856464293,90.0,-0.9690094749109707,646],[\"random_forest\",1.0,12.0,1.5321813844326224,0.014191894624199035,50.0,-0.9679029381694694,647],[\"random_forest\",1.0,11.0,1.0801166482933748,0.01003718329943107,60.0,-0.9690559814877654,648],[\"random_forest\",1.0,12.0,1.3246306889867019,0.013139679490627651,20.0,-0.9685683717086884,649],[\"random_forest\",0.0,12.0,1.0004937130298877,0.01554306756191979,70.0,-0.9670941419622491,650],[\"random_forest\",1.0,10.0,1.1722508897742252,0.012112237498721765,50.0,-0.9686750812509212,651],[\"random_forest\",1.0,11.0,1.6563141818817573,0.01660244609621568,100.0,-0.967031263920904,652],[\"random_forest\",1.0,12.0,1.457329113237979,0.010770166456419272,60.0,-0.9690263997340797,653],[\"random_forest\",1.0,11.0,1.9134330613949806,0.032040087870672695,90.0,-0.9638000382043641,654],[\"random_forest\",1.0,10.0,1.8202464687324424,0.018513248925617398,80.0,-0.9659511691582813,655],[\"random_forest\",1.0,11.0,1.6064137231249793,0.011689428283653543,30.0,-0.9678287697791825,656],[\"random_forest\",1.0,12.0,1.1484083059956114,0.013732185839868908,50.0,-0.9676820380441531,657],[\"random_forest\",1.0,9.0,1.3358813843325974,0.012697288067212702,40.0,-0.9662831048057259,658],[\"random_forest\",1.0,12.0,1.524942066998409,0.01002011362697564,60.0,-0.9694697347139756,659],[\"random_forest\",0.0,12.0,3.895407162680225,0.014678927877801201,70.0,-0.9677908784754228,660],[\"random_forest\",1.0,11.0,1.2571796817653613,0.010025730596320807,90.0,-0.9691619236653668,661],[\"random_forest\",1.0,10.0,1.7631047922060505,0.011344150359079144,80.0,-0.9681153177958689,662],[\"random_forest\",1.0,12.0,1.4220529274929383,0.015599304339244894,50.0,-0.967688727262535,663],[\"random_forest\",1.0,11.0,1.9164887330194784,0.017242196486037566,60.0,-0.9663372238842445,664],[\"random_forest\",1.0,12.0,1.6405012051759247,0.012315063001444585,40.0,-0.9683582850044947,665],[\"random_forest\",1.0,10.0,2.0527796673067638,0.013594730151405872,70.0,-0.9679967828895125,666],[\"random_forest\",1.0,5.0,1.0689681360946544,0.02872674540219408,20.0,-0.9546946637481863,667],[\"random_forest\",0.0,11.0,1.73802576021855,0.010654798810498483,100.0,-0.9679567423533383,668],[\"random_forest\",1.0,8.0,1.8351914434258962,0.014251662071142467,30.0,-0.9653806625983868,669],[\"random_forest\",1.0,12.0,1.5506846771863234,0.034355286072182474,10.0,-0.9604675448142932,670],[\"random_forest\",1.0,11.0,1.3891610171344277,0.013011480163845368,60.0,-0.9683571202000142,671],[\"random_forest\",1.0,12.0,1.005100951638413,0.011357680655891877,80.0,-0.9690526521647224,672],[\"random_forest\",1.0,12.0,1.301491720486779,0.0499613610281003,40.0,-0.9595650384051322,673],[\"random_forest\",1.0,11.0,1.2083115491818757,0.016314721302628198,70.0,-0.967811918329584,674],[\"random_forest\",1.0,7.0,1.4967074884580784,0.012014287454623856,90.0,-0.9647865511686899,675],[\"random_forest\",1.0,12.0,3.5109318233289537,0.01500298709292949,50.0,-0.9667437131329065,676],[\"random_forest\",0.0,11.0,1.9650333546729697,0.010999768739274805,100.0,-0.9684389897516833,677],[\"random_forest\",1.0,12.0,2.261288083238961,0.010032074780767973,80.0,-0.9684990979430036,678],[\"random_forest\",1.0,10.0,1.6170270923167858,0.047572565832173515,70.0,-0.9596102150870478,679],[\"random_forest\",1.0,12.0,1.7235947603348067,0.039200493674969056,90.0,-0.9624546370563708,680],[\"random_forest\",1.0,11.0,1.1628491475716964,0.012889413803990557,30.0,-0.967845419451627,681],[\"random_forest\",1.0,4.0,2.3678905400424624,0.010672412362491284,50.0,-0.9514851688196991,682],[\"random_forest\",1.0,10.0,1.466355276459346,0.018103631350491874,60.0,-0.9669038080185519,683],[\"random_forest\",1.0,9.0,2.1522683206348248,0.030762979690369365,40.0,-0.9641688470981092,684],[\"random_forest\",1.0,11.0,1.2955800861129632,0.013726319452521592,60.0,-0.9682773295644784,685],[\"random_forest\",0.0,2.0,3.284584900184332,0.010006198576596983,80.0,-0.9109632963015393,686],[\"random_forest\",1.0,12.0,1.8732289060327534,0.04356140357161387,70.0,-0.9617204250226636,687],[\"random_forest\",1.0,11.0,1.003096193989466,0.01193627133022006,50.0,-0.9675961574072385,688],[\"random_forest\",1.0,12.0,2.050532634193711,0.010018149735119126,60.0,-0.9690718026509861,689],[\"random_forest\",1.0,12.0,1.6683992766945768,0.014540688844383212,20.0,-0.966711804827489,690],[\"random_forest\",1.0,11.0,1.3833556379011096,0.019396578224290417,90.0,-0.9666917952597055,691],[\"random_forest\",1.0,12.0,1.1081661193085783,0.015287956943726663,70.0,-0.9678390695857315,692],[\"random_forest\",1.0,12.0,1.7766770427054337,0.012272356205625484,40.0,-0.9683582850044947,693],[\"random_forest\",1.0,11.0,1.5451521716075243,0.011421160883494394,80.0,-0.9685457054083756,694],[\"random_forest\",0.0,10.0,1.933648428098573,0.017119755969489392,110.0,-0.966732792708747,695],[\"random_forest\",1.0,12.0,1.604545598881953,0.02029479997684171,100.0,-0.9669868087451758,696],[\"random_forest\",1.0,11.0,2.1827184827714663,0.013283520253683123,70.0,-0.9684363269046949,697],[\"random_forest\",1.0,10.0,1.2461955842983286,0.025311879269445337,30.0,-0.9649431057824496,698],[\"random_forest\",1.0,12.0,1.817561921351,0.01589203454905969,40.0,-0.9665733856997631,699],[\"random_forest\",1.0,11.0,1.4594783612133981,0.010036887659929676,50.0,-0.9684572444696719,700],[\"random_forest\",1.0,12.0,1.3601176712090106,0.037460541147242864,60.0,-0.9638640076766791,701],[\"random_forest\",1.0,11.0,2.08901200613213,0.010835754431363136,90.0,-0.9682002201193088,702],[\"random_forest\",1.0,12.0,1.9940898294103282,0.010001788689962066,80.0,-0.9698153881506708,703],[\"random_forest\",0.0,8.0,1.0015873630695753,0.014038127520309734,60.0,-0.9652615682190122,704],[\"random_forest\",1.0,6.0,1.7080520253511318,0.011678762776682804,10.0,-0.95919714362304,705],[\"random_forest\",1.0,9.0,2.303464483194456,0.010843010415357684,30.0,-0.9667181149493997,706],[\"random_forest\",1.0,12.0,2.5377332745539727,0.013095701015534998,100.0,-0.9687616742223345,707],[\"random_forest\",1.0,12.0,2.388455587706254,0.012458425266385652,110.0,-0.9679203827216121,708],[\"random_forest\",1.0,12.0,2.637689563124622,0.014407134509858852,100.0,-0.9678547776314567,709],[\"random_forest\",1.0,12.0,2.2584268601926136,0.016300136572492987,90.0,-0.9672325794333525,710],[\"random_forest\",1.0,12.0,2.452867967660553,0.011399340865486173,80.0,-0.9683003871901812,711],[\"random_forest\",1.0,12.0,2.0002048201386033,0.01880765101891091,80.0,-0.9673320525131035,712],[\"random_forest\",0.0,12.0,2.123316060665655,0.012475419617741885,90.0,-0.9671075570857683,713],[\"random_forest\",1.0,12.0,2.1699844166375737,0.01359297982565903,100.0,-0.9679496495806481,714],[\"random_forest\",1.0,12.0,1.9590830272405189,0.0332220700929097,110.0,-0.9638736532360919,715],[\"random_forest\",1.0,12.0,2.805889697344976,0.010756531924200112,90.0,-0.9686184797019611,716],[\"random_forest\",1.0,12.0,2.026533597539889,0.015272530081382019,80.0,-0.9675027437565701,717],[\"random_forest\",1.0,12.0,2.3320026536745426,0.011849512246245426,70.0,-0.9679782744214663,718],[\"random_forest\",1.0,12.0,1.825494414958315,0.02125862863710218,100.0,-0.9663928960354036,719],[\"random_forest\",1.0,12.0,2.6805305655321607,0.010014524820994797,80.0,-0.9697558027456137,720],[\"random_forest\",1.0,12.0,1.8701142548036709,0.017250343236416766,90.0,-0.9675402712500066,721],[\"random_forest\",0.0,12.0,2.236503507511566,0.010011702096651132,20.0,-0.9658799234794996,722],[\"random_forest\",1.0,11.0,2.0887333081703354,0.035429157649198496,50.0,-0.961504743588288,723],[\"random_forest\",1.0,7.0,2.9857502295252187,0.012806087558731976,110.0,-0.9649393729050985,724],[\"random_forest\",1.0,12.0,2.494279892687225,0.014313264176809153,90.0,-0.9685707563477823,725],[\"random_forest\",1.0,12.0,3.0975969834446255,0.01140396729554668,100.0,-0.9689654752624541,726],[\"random_forest\",1.0,12.0,3.289578381396389,0.010046486584332687,90.0,-0.9688335558048747,727],[\"random_forest\",1.0,11.0,3.351548792646504,0.017844017589547234,110.0,-0.9675110502494144,728],[\"random_forest\",1.0,2.0,3.1711359113002553,0.015503791255143212,100.0,-0.9207098940956708,729],[\"random_forest\",1.0,12.0,3.631678473245898,0.010817678612875144,80.0,-0.9681485223666397,730],[\"random_forest\",1.0,11.0,2.9677589694997093,0.013334470054360337,80.0,-0.9682124643238884,731],[\"random_forest\",0.0,12.0,2.906329295449759,0.023604299898783054,110.0,-0.9648607990469465,732],[\"random_forest\",1.0,12.0,2.562496183502969,0.0121695837042564,90.0,-0.9688552713067793,733],[\"random_forest\",1.0,11.0,3.430799011085899,0.016557825097095594,100.0,-0.9662846884506627,734],[\"random_forest\",1.0,11.0,2.4848005907834345,0.014829776855198656,90.0,-0.9680001825288365,735],[\"random_forest\",1.0,12.0,2.6531156242464293,0.013782582612090505,80.0,-0.9682419941046708,736],[\"random_forest\",1.0,12.0,2.823851119303428,0.01002430348285566,100.0,-0.968882046523685,737],[\"random_forest\",1.0,6.0,2.5467109062230486,0.012768356495378021,80.0,-0.9628990451238592,738],[\"random_forest\",1.0,11.0,2.6863358666477613,0.010019739750584738,70.0,-0.9690911487999448,739],[\"random_forest\",1.0,4.0,2.723912029084614,0.011335143702054087,80.0,-0.9484820460614319,740],[\"random_forest\",0.0,12.0,2.4103978041929235,0.012119606048381016,90.0,-0.9682656020317028,741],[\"random_forest\",1.0,11.0,3.715521772601103,0.016075157390438823,70.0,-0.9671057227480035,742],[\"random_forest\",1.0,12.0,3.199499588364721,0.02615009525884304,90.0,-0.9647609499279536,743],[\"random_forest\",1.0,12.0,2.774769151512963,0.010006608546286449,100.0,-0.9688796955141166,744],[\"random_forest\",1.0,11.0,3.5390650157835117,0.010805682578296944,110.0,-0.9687338778907397,745],[\"random_forest\",1.0,12.0,2.6025319434858964,0.014048350137431827,80.0,-0.9678462601897693,746],[\"random_forest\",1.0,11.0,2.368847176725488,0.015023868151366668,100.0,-0.9681034221154645,747],[\"random_forest\",1.0,12.0,2.290835653259685,0.04478544076846481,70.0,-0.960767303120115,748],[\"random_forest\",1.0,12.0,2.844215179945185,0.013145078165414359,80.0,-0.9676602002530639,749],[\"random_forest\",0.0,3.0,2.7243598989359,0.011669018640096583,90.0,-0.9409223121208741,750],[\"random_forest\",1.0,11.0,2.9170520949784082,0.018646203729351593,70.0,-0.9675605223389291,751],[\"random_forest\",1.0,12.0,3.0376736088880345,0.011063854987876046,80.0,-0.9682890846123209,752],[\"random_forest\",1.0,12.0,2.2025826282055427,0.03655293117222494,80.0,-0.9627494732225238,753],[\"random_forest\",1.0,11.0,2.4266167028578978,0.01223468662605892,60.0,-0.9685388694429723,754],[\"random_forest\",1.0,11.0,3.119189157385215,0.022296480545275537,100.0,-0.965652062042366,755],[\"random_forest\",1.0,5.0,2.1291674131923592,0.016894573388201037,110.0,-0.958279934996695,756],[\"random_forest\",1.0,7.0,2.319245876906381,0.020629753656049236,70.0,-0.9634702548465635,757],[\"random_forest\",1.0,12.0,2.660125303191189,0.0416426644984162,90.0,-0.9614151117306444,758],[\"random_forest\",0.0,12.0,4.109919914526648,0.013492392686099787,120.0,-0.9672192223971956,759],[\"random_forest\",1.0,12.0,2.0422833328915,0.01003770716377194,90.0,-0.9696229141462485,760],[\"random_forest\",1.0,8.0,2.192005634338738,0.029926306436174124,60.0,-0.9613388827676019,761],[\"random_forest\",1.0,11.0,3.028447238519247,0.014881784388208396,70.0,-0.9684787245648969,762],[\"random_forest\",1.0,12.0,1.9543569379742525,0.012686552743547906,100.0,-0.9685741590443357,763],[\"random_forest\",1.0,11.0,2.4795821761992087,0.010610562078426056,80.0,-0.9686868271270747,764],[\"random_forest\",1.0,11.0,1.8923521149132982,0.015600256123657504,50.0,-0.9680739106780597,765],[\"random_forest\",1.0,12.0,3.2380209381881073,0.017770717722892376,60.0,-0.9676895832868252,766],[\"random_forest\",1.0,11.0,2.307457332769991,0.011565682564729306,110.0,-0.968733345932788,767],[\"random_forest\",0.0,12.0,2.5772779351631216,0.027214583922411877,90.0,-0.9640935261322602,768],[\"random_forest\",1.0,12.0,2.2341216607290453,0.014157369416030569,70.0,-0.9685849663509997,769],[\"random_forest\",1.0,12.0,2.0229498033182765,0.010759762750248562,40.0,-0.9689603360594832,770],[\"random_forest\",1.0,11.0,2.106944763092769,0.010024836929648239,30.0,-0.9676373230039084,771],[\"random_forest\",1.0,12.0,2.383313009816546,0.012579783923007565,80.0,-0.9682980575812199,772],[\"random_forest\",1.0,11.0,1.7721418344850204,0.011852145848606256,50.0,-0.9677413727563825,773],[\"random_forest\",1.0,12.0,1.9638197220741413,0.019874721241400092,70.0,-0.9666637604642014,774],[\"random_forest\",1.0,11.0,1.866955307982335,0.01629451467414019,90.0,-0.967213178254261,775],[\"random_forest\",1.0,12.0,2.775335629347603,0.013414227612453252,100.0,-0.9686818774723396,776],[\"random_forest\",0.0,12.0,3.818608588688898,0.014392468706403302,10.0,-0.9649759373712086,777],[\"random_forest\",1.0,11.0,3.4153556920040593,0.010676758660044977,80.0,-0.9686921864505772,778],[\"random_forest\",1.0,10.0,1.7390459541495629,0.011365818248448836,60.0,-0.9679935085966026,779],[\"random_forest\",1.0,12.0,1.9468543817842647,0.024655606678286538,70.0,-0.9657050530031591,780],[\"random_forest\",1.0,12.0,2.88447611506363,0.012853761165026779,40.0,-0.9681370027254772,781],[\"random_forest\",1.0,11.0,2.0943456420232764,0.031542391840747655,20.0,-0.9615712933623919,782],[\"random_forest\",1.0,12.0,2.2061850664693807,0.010040217027411417,100.0,-0.969070408554285,783],[\"random_forest\",1.0,11.0,2.4666153610535346,0.01571810313649758,80.0,-0.9668798699107224,784],[\"random_forest\",1.0,12.0,1.7017684943569396,0.010046818415470619,50.0,-0.9700942778071894,785],[\"random_forest\",1.0,12.0,1.6473324166169674,0.011324835471012,60.0,-0.9688376188630236,786],[\"random_forest\",1.0,12.0,1.5812395246551838,0.01226360083790657,50.0,-0.9683654052922517,787],[\"random_forest\",1.0,12.0,1.781491041826401,0.01100201470805939,60.0,-0.9694100423058821,788],[\"random_forest\",1.0,12.0,1.863221447253716,0.010926517868138502,70.0,-0.9691749505207259,789],[\"random_forest\",1.0,12.0,1.7044719375102348,0.010619522828969557,40.0,-0.9679238526772169,790],[\"random_forest\",1.0,12.0,1.7850947248975275,0.010087757173877032,30.0,-0.9694186117204731,791],[\"random_forest\",1.0,12.0,1.897531413556779,0.01198672469908608,50.0,-0.9682152647462092,792],[\"random_forest\",1.0,12.0,1.6478924078091768,0.010078068444710615,30.0,-0.9691492973070865,793],[\"random_forest\",1.0,12.0,2.0108890033950884,0.011605008972505155,40.0,-0.9690298758041439,794],[\"random_forest\",1.0,12.0,1.5565982137226382,0.012868066312002765,50.0,-0.9693440642337147,795],[\"random_forest\",1.0,12.0,1.7135195971039632,0.010035315731685484,40.0,-0.9693464977884827,796],[\"random_forest\",1.0,12.0,1.8176282786281526,0.013589947322642987,50.0,-0.968637948140104,797],[\"random_forest\",1.0,12.0,1.9195292316708406,0.011912803514857156,40.0,-0.967974645489922,798],[\"random_forest\",1.0,12.0,1.5185741773776136,0.010864529649888386,60.0,-0.9689598683033532,799],[\"random_forest\",1.0,12.0,2.0624907915604664,0.012697889087137357,50.0,-0.9688119962216798,800],[\"random_forest\",1.0,12.0,1.6699411601523055,0.010002000659486478,40.0,-0.9694007422134151,801],[\"random_forest\",1.0,11.0,1.7799216709269383,0.01002001173585717,30.0,-0.9691177436403036,802],[\"random_forest\",1.0,12.0,1.4674974793661584,0.010013709115204559,60.0,-0.9696621445165765,803],[\"random_forest\",1.0,11.0,1.6090068884125284,0.011459253934308059,50.0,-0.9693362866415925,804],[\"random_forest\",1.0,12.0,2.150917513442052,0.012167539274551937,60.0,-0.9682873695065108,805],[\"random_forest\",1.0,12.0,1.92117802550538,0.013492603711798022,50.0,-0.9681723565286628,806],[\"random_forest\",1.0,11.0,1.6961581527204226,0.010700392249541009,30.0,-0.9682293035445686,807],[\"random_forest\",1.0,12.0,1.9850029363353523,0.010005027931762948,20.0,-0.9672604583101465,808],[\"random_forest\",1.0,11.0,1.528206645745706,0.01435822421366976,40.0,-0.9671828107925659,809],[\"random_forest\",1.0,12.0,1.834693486185519,0.011422450861533897,80.0,-0.9695382319433401,810],[\"random_forest\",1.0,11.0,2.2555932321098657,0.01274816118694452,70.0,-0.9685116998434471,811],[\"random_forest\",1.0,12.0,1.5970917208842428,0.010011907441737269,60.0,-0.9696603132360413,812],[\"random_forest\",1.0,12.0,1.4263963772277177,0.012009515223285002,70.0,-0.9683523203495299,813],[\"random_forest\",1.0,11.0,2.02850066452563,0.01327767624777379,50.0,-0.9670312608636743,814],[\"random_forest\",1.0,12.0,1.7233668305104628,0.011333505308317528,90.0,-0.9696652139751029,815],[\"random_forest\",1.0,11.0,1.8434112120876731,0.010606053906464066,30.0,-0.9683671356842096,816],[\"random_forest\",1.0,12.0,2.132579976116133,0.014162603840806604,60.0,-0.9679434434045441,817],[\"random_forest\",1.0,12.0,1.6068747503201468,0.012464833237360029,40.0,-0.9681544289342422,818],[\"random_forest\",1.0,12.0,1.4621511587071145,0.014706502131639893,70.0,-0.968637786106935,819],[\"random_forest\",1.0,11.0,1.7795696838680317,0.01100499024411095,50.0,-0.9681409924101156,820],[\"random_forest\",1.0,12.0,1.9488887744412986,0.04594328306417436,80.0,-0.9604287302271918,821],[\"random_forest\",1.0,11.0,2.063637904199271,0.013296360222544375,90.0,-0.9683771542256345,822],[\"random_forest\",1.0,12.0,1.3870982274832064,0.010022593618037763,80.0,-0.9692886336036944,823],[\"random_forest\",1.0,12.0,1.655211533000327,0.011915711714670468,60.0,-0.9693480019454497,824],[\"random_forest\",1.0,11.0,1.5279164092738735,0.01072346099848429,70.0,-0.9690024555117909,825],[\"random_forest\",1.0,12.0,2.177494561344483,0.010009162764357031,40.0,-0.9686806545804965,826],[\"random_forest\",1.0,11.0,1.8530642799491144,0.013919833580703703,50.0,-0.9677075536824602,827],[\"random_forest\",1.0,12.0,1.7470290771274795,0.012375415869049233,60.0,-0.9685891272404961,828],[\"random_forest\",1.0,11.0,2.371044020154414,0.011332575890115264,90.0,-0.9686331513468494,829],[\"random_forest\",0.0,12.0,2.266254713916741,0.015283574375849065,70.0,-0.9669198034438603,830],[\"random_forest\",1.0,12.0,1.9691259451880911,0.01305535374830783,90.0,-0.968549474972482,831],[\"random_forest\",1.0,11.0,1.8936097859800431,0.01073893268329734,40.0,-0.9682868008618037,832],[\"random_forest\",1.0,12.0,1.3372683557117184,0.01169914814541255,80.0,-0.9693421565224394,833],[\"random_forest\",1.0,11.0,2.0899047072961774,0.04071339142485943,30.0,-0.9609909302942415,834],[\"random_forest\",1.0,12.0,1.5796167990275867,0.010028010560651394,100.0,-0.9696574761269653,835],[\"random_forest\",1.0,12.0,1.4694341448758026,0.01393101082529517,20.0,-0.9681874745290736,836],[\"random_forest\",1.0,12.0,1.404877228896105,0.010000195693345534,50.0,-0.9693337093970329,837],[\"random_forest\",1.0,11.0,1.7008797894261845,0.012730110097539528,60.0,-0.9680425710173495,838],[\"random_forest\",1.0,11.0,1.8186014439887772,0.014521045486786163,80.0,-0.9677035059104594,839],[\"random_forest\",0.0,12.0,1.6543520192716707,0.011978847388171503,70.0,-0.9684914915557392,840],[\"random_forest\",1.0,12.0,2.1838954405701254,0.011357085463900823,50.0,-0.9680474595274925,841],[\"random_forest\",1.0,11.0,1.9872406341169897,0.015067162760920974,90.0,-0.9676667763539506,842],[\"random_forest\",1.0,12.0,1.2938884755344349,0.013327604295046994,40.0,-0.9693417132241464,843],[\"random_forest\",1.0,11.0,4.510966767874888,0.01104436362599939,30.0,-0.9674808173058222,844],[\"random_forest\",1.0,12.0,2.2975908224975052,0.012622533594883441,60.0,-0.9687016149466879,845],[\"random_forest\",1.0,12.0,1.50452544497056,0.010623730395263428,70.0,-0.9694054656331592,846],[\"random_forest\",1.0,11.0,1.774290811941292,0.013740816736197325,10.0,-0.9677642744633751,847],[\"random_forest\",1.0,12.0,1.8773229965624438,0.016008433685270607,100.0,-0.9685901361262667,848],[\"random_forest\",1.0,12.0,2.0806704052603875,0.012144119526635215,80.0,-0.9684278308636147,849],[\"random_forest\",0.0,11.0,1.5527754882321092,0.010018722663859269,40.0,-0.9685966419108722,850],[\"random_forest\",1.0,12.0,2.4331858646057833,0.014870781861803975,50.0,-0.9670196953640677,851],[\"random_forest\",1.0,11.0,1.6971744570602438,0.01065944163481675,110.0,-0.9692624820616289,852],[\"random_forest\",1.0,12.0,2.5355473033508438,0.011616192721758038,70.0,-0.9686980502169651,853],[\"random_forest\",1.0,11.0,1.9778008307906187,0.01309150367449185,60.0,-0.9680518069079949,854],[\"random_forest\",1.0,12.0,2.6530230122568783,0.010014427675728299,90.0,-0.969010171959321,855],[\"random_forest\",1.0,12.0,1.620594476462196,0.016833584664612773,80.0,-0.9676152528633691,856],[\"random_forest\",1.0,11.0,1.315261867453921,0.01242522177651468,60.0,-0.96818204488929,857],[\"random_forest\",1.0,12.0,1.2217064973923364,0.013794890777948137,100.0,-0.9682742601059523,858],[\"random_forest\",1.0,12.0,3.0617281134324568,0.01004247645358777,70.0,-0.9682780632995844,859],[\"random_forest\",0.0,11.0,1.9178104467016297,0.015585687029690577,90.0,-0.9662128924705515,860],[\"random_forest\",1.0,11.0,2.1503903223217566,0.011158518239147352,80.0,-0.9688980786357488,861],[\"random_forest\",1.0,12.0,3.5720443817963385,0.01442729451225595,20.0,-0.9653416309479831,862],[\"random_forest\",1.0,5.0,1.4179001202444415,0.011876726526232149,40.0,-0.9552503305155642,863],[\"random_forest\",1.0,12.0,4.774543368504096,0.010022619323427096,50.0,-0.9681577337994484,864],[\"random_forest\",1.0,12.0,4.245975365157967,0.010755337316793775,30.0,-0.9682017701347202,865],[\"random_forest\",1.0,6.0,2.0369986294856472,0.012985358057124925,60.0,-0.9634212474559494,866],[\"random_forest\",1.0,11.0,2.352287833736751,0.01153491888830209,70.0,-0.9684887522780106,867],[\"random_forest\",1.0,12.0,2.2600301282316986,0.013750712613252634,90.0,-0.9688115620950756,868],[\"random_forest\",1.0,11.0,1.7987180055770489,0.03841437531352716,100.0,-0.9627449790950002,869],[\"random_forest\",0.0,12.0,1.5660513114527956,0.010025480910498485,50.0,-0.9690810996862238,870],[\"random_forest\",1.0,12.0,1.3731877528982492,0.012486283865589105,110.0,-0.9688516362607755,871],[\"random_forest\",1.0,11.0,1.6791012855170675,0.015158186003726908,80.0,-0.9686693000297326,872],[\"random_forest\",1.0,4.0,2.9571413910018443,0.011024740917830315,60.0,-0.9497142746838011,873],[\"random_forest\",1.0,10.0,1.4752690035783973,0.011866652585458179,70.0,-0.9686932809387768,874],[\"random_forest\",1.0,12.0,1.8849120941095776,0.010004493761459076,30.0,-0.9685567083777342,875],[\"random_forest\",1.0,7.0,1.7982876318475856,0.0162783306480403,40.0,-0.9645854802346101,876],[\"random_forest\",1.0,10.0,1.7411122274136215,0.014439475352824216,20.0,-0.9651086639374042,877],[\"random_forest\",1.0,12.0,3.9407874388898527,0.04918681629842258,80.0,-0.9594493497795411,878],[\"random_forest\",1.0,11.0,2.107788092624762,0.0130620144474764,90.0,-0.9684666882519305,879],[\"random_forest\",0.0,8.0,1.6253538292850864,0.017117046221581986,50.0,-0.964452209481544,880],[\"random_forest\",1.0,11.0,2.2157845492932107,0.011047828012164262,60.0,-0.968682699867104,881],[\"random_forest\",1.0,2.0,2.0123763682930678,0.012300332747996565,100.0,-0.9235552148904986,882],[\"random_forest\",1.0,12.0,2.7285945788047887,0.013851176413254075,80.0,-0.9685214738065034,883],[\"random_forest\",1.0,12.0,3.154908591138674,0.010720265944271805,70.0,-0.9690412334121371,884],[\"random_forest\",1.0,12.0,1.2618351468828413,0.028427390028130467,110.0,-0.9656847682847108,885],[\"random_forest\",1.0,11.0,2.5951244681792125,0.015363110816036704,50.0,-0.9677024756240816,886],[\"random_forest\",1.0,12.0,3.3636074126560636,0.010044946589795219,40.0,-0.9682743457083812,887],[\"random_forest\",1.0,12.0,1.5331672361816333,0.01174889973841164,70.0,-0.9692988386361256,888],[\"random_forest\",1.0,11.0,1.9226802110569121,0.012698010859803367,90.0,-0.9680827644150042,889],[\"random_forest\",0.0,12.0,1.4322548149150376,0.018237693711382084,60.0,-0.9663136709873454,890],[\"random_forest\",1.0,10.0,2.384548860390496,0.013420331580364734,30.0,-0.9678360337567308,891],[\"random_forest\",1.0,11.0,1.6876922523571134,0.010687191385244198,80.0,-0.9692182011479885,892],[\"random_forest\",1.0,12.0,1.3428668477469898,0.014384577210702052,50.0,-0.96884232393939,893],[\"random_forest\",1.0,11.0,1.861106437793445,0.010031813181611306,60.0,-0.9693155188808662,894],[\"random_forest\",1.0,12.0,1.2105544629411988,0.011393463363633373,100.0,-0.9692845552593976,895],[\"random_forest\",1.0,12.0,3.469613905452045,0.048114561927212474,40.0,-0.9604824029501875,896],[\"random_forest\",1.0,11.0,1.7484168893282004,0.016232232395504684,70.0,-0.9673308479646381,897],[\"random_forest\",1.0,12.0,1.6008531543417728,0.012109905032897089,90.0,-0.968562293936228,898],[\"random_forest\",1.0,11.0,2.0341843220898053,0.013312179623107339,80.0,-0.9686478688501817,899],[\"random_forest\",0.0,12.0,2.121096729068577,0.010018233232093107,70.0,-0.9685509699577601,900],[\"random_forest\",1.0,3.0,1.5090830763191434,0.010012225818850851,90.0,-0.9372578158243073,901],[\"random_forest\",1.0,12.0,2.5224947080213815,0.015197431690216465,50.0,-0.9674656137029823,902],[\"random_forest\",1.0,11.0,2.2772745749309338,0.011433179570447292,60.0,-0.9688302723402757,903],[\"random_forest\",1.0,10.0,1.9513497753725635,0.012501006057605665,10.0,-0.9677353775291215,904],[\"random_forest\",1.0,12.0,1.8013903347876603,0.010016215206828312,120.0,-0.9697435860561004,905],[\"random_forest\",1.0,11.0,3.6575716993868914,0.01743308247075767,70.0,-0.9673768439840885,906],[\"random_forest\",1.0,12.0,4.974558766931807,0.014212883690555393,40.0,-0.9666088159336889,907],[\"random_forest\",1.0,12.0,1.4138895769771231,0.010938955578794419,110.0,-0.9686805445202304,908],[\"random_forest\",1.0,11.0,2.209867900616781,0.013050995076980737,30.0,-0.9684747287657992,909],[\"random_forest\",0.0,6.0,1.279154620740106,0.01207813067894748,80.0,-0.9639210616956212,910],[\"random_forest\",1.0,10.0,1.6109101691261518,0.0424078354873403,50.0,-0.9598633353552015,911],[\"random_forest\",1.0,12.0,1.1558684486103035,0.01928561617732865,100.0,-0.9666243558317856,912],[\"random_forest\",1.0,12.0,1.7013945078090615,0.015973006155690507,60.0,-0.9681229272403629,913],[\"random_forest\",1.0,11.0,2.460139453737798,0.014810046161605549,90.0,-0.9670753583435382,914],[\"random_forest\",1.0,11.0,2.872023436565987,0.013742590308172964,40.0,-0.9685261269099669,915],[\"random_forest\",1.0,7.0,1.8690483730401684,0.010788106548663808,20.0,-0.9654611991979469,916],[\"random_forest\",1.0,12.0,1.5333336623010958,0.011430056484878652,80.0,-0.9688293093129494,917],[\"random_forest\",1.0,12.0,2.0539626804099513,0.03607954970445592,50.0,-0.9623365974212108,918],[\"random_forest\",1.0,12.0,2.3235127210665123,0.012618306339539231,60.0,-0.9687016149466879,919],[\"random_forest\",0.0,11.0,1.46726262768798,0.010024619304878819,70.0,-0.9691003143743091,920],[\"random_forest\",1.0,8.0,1.3705697923653843,0.029441764851976374,30.0,-0.9644732921369199,921],[\"random_forest\",1.0,12.0,1.9517797638663663,0.010735998694523867,100.0,-0.9694552403884048,922],[\"random_forest\",1.0,11.0,1.8194708141863158,0.016691122629216112,80.0,-0.9672218393857399,923],[\"random_forest\",1.0,10.0,2.1639416271916527,0.011521272498431694,70.0,-0.9681754412733371,924],[\"random_forest\",1.0,12.0,1.6541764514489523,0.010001952782221631,110.0,-0.9693853123755842,925],[\"random_forest\",1.0,11.0,3.2336967423867837,0.01329399636108915,90.0,-0.9683136219371535,926],[\"random_forest\",1.0,12.0,1.7282892353099755,0.014861933071036573,50.0,-0.9680163583306917,927],[\"random_forest\",1.0,12.0,2.8085062051996035,0.03465761598707075,60.0,-0.9623921778554819,928],[\"random_forest\",1.0,11.0,2.013869907906401,0.02397344178012514,40.0,-0.9655459425454473,929],[\"random_forest\",0.0,12.0,1.3019147358361935,0.012074477449045374,20.0,-0.9664124378470571,930],[\"random_forest\",1.0,11.0,1.5575314082055423,0.013893751464106052,90.0,-0.9686326071599791,931],[\"random_forest\",1.0,12.0,1.1063622246131692,0.010822404097571927,80.0,-0.9692586819252262,932],[\"random_forest\",1.0,12.0,1.2072398613439583,0.012884398595077111,100.0,-0.9682425016047856,933],[\"random_forest\",1.0,10.0,1.4621474540891748,0.015573835899942204,70.0,-0.9668969659386896,934],[\"random_forest\",1.0,9.0,1.9001215201248889,0.011938808905577589,60.0,-0.9673088359114616,935],[\"random_forest\",1.0,11.0,2.6531565771561003,0.01422245758830681,50.0,-0.9668506978258041,936],[\"random_forest\",1.0,12.0,1.7490618513844285,0.0267729005125453,80.0,-0.9655154436228792,937],[\"random_forest\",1.0,11.0,2.4083580728877463,0.010703897744676174,120.0,-0.9690234586791968,938],[\"random_forest\",1.0,12.0,1.6218652338281925,0.012536232840307887,40.0,-0.9685104463793079,939],[\"random_forest\",0.0,11.0,2.114216312328839,0.03288630003485144,70.0,-0.9598601527791798,940],[\"random_forest\",1.0,12.0,2.240007218046965,0.017433585468937036,90.0,-0.9678205458315375,941],[\"random_forest\",1.0,12.0,1.8219047805030013,0.011245793110302693,100.0,-0.9693624809848725,942],[\"random_forest\",1.0,11.0,1.3328981991001643,0.015769529237423793,30.0,-0.9676684792308421,943],[\"random_forest\",1.0,12.0,1.4172666521540966,0.013447288990717567,50.0,-0.9676831753335671,944],[\"random_forest\",1.0,8.0,1.970601278484792,0.010019823538853261,10.0,-0.9633626189637592,945],[\"random_forest\",1.0,11.0,3.0022320935291313,0.018295636826168908,60.0,-0.9676454246623691,946],[\"random_forest\",1.0,5.0,1.5058263128083689,0.01177098821767844,70.0,-0.9589284681706415,947],[\"random_forest\",1.0,10.0,1.6758146658572641,0.010721910913228409,90.0,-0.9691728012883116,948],[\"random_forest\",1.0,12.0,1.5840090708074348,0.046567863373678055,80.0,-0.9610051739269847,949],[\"random_forest\",0.0,12.0,2.5749464920286225,0.01464612446610357,50.0,-0.967420663256058,950],[\"random_forest\",1.0,12.0,2.3370652182310403,0.010020239729466583,60.0,-0.9691592241316231,951],[\"random_forest\",1.0,11.0,2.0857371695134943,0.01280455127422387,100.0,-0.968630931798154,952],[\"random_forest\",1.0,12.0,1.0592890513036706,0.016620284716650315,110.0,-0.9678635488232018,953],[\"random_forest\",1.0,11.0,1.8728333846376024,0.011565355012743576,40.0,-0.9682614594855841,954],[\"random_forest\",1.0,10.0,1.729430760103601,0.013865895682682546,80.0,-0.9673640280775724,955],[\"random_forest\",1.0,12.0,1.2600700599919743,0.039615608336381475,70.0,-0.9607921461679084,956],[\"random_forest\",1.0,11.0,2.0017667669023917,0.02295251085856518,40.0,-0.9655051346446413,957],[\"random_forest\",1.0,12.0,2.1697638681539924,0.01076382719791267,50.0,-0.9678603540182614,958],[\"random_forest\",1.0,12.0,2.7393538179453207,0.010015158373000274,60.0,-0.9690718026509861,959],[\"random_forest\",0.0,11.0,1.8123784038935007,0.01264770364451151,80.0,-0.9675308855551106,960],[\"random_forest\",1.0,12.0,1.1797142519061716,0.011996546808757336,30.0,-0.9688207307266696,961],[\"random_forest\",1.0,12.0,1.3685783457074405,0.014905454100469874,90.0,-0.9684633650433468,962],[\"random_forest\",1.0,11.0,2.482610915379292,0.010039405385124676,70.0,-0.968637465097826,963],[\"random_forest\",1.0,12.0,1.9186809251588237,0.030665996978995844,60.0,-0.9633318999206594,964],[\"random_forest\",1.0,12.0,3.1151524098310954,0.02189507200046751,100.0,-0.966173594841172,965],[\"random_forest\",1.0,10.0,1.5013128279512091,0.013237272629424233,20.0,-0.9659503651068944,966],[\"random_forest\",1.0,11.0,1.6535421083764554,0.03729598909221474,50.0,-0.9618031108546381,967],[\"random_forest\",1.0,12.0,1.5815968551476396,0.011485481792956118,30.0,-0.9689424145795219,968],[\"random_forest\",1.0,9.0,1.7601843560606767,0.010779773239977895,40.0,-0.9677863018026998,969],[\"random_forest\",0.0,11.0,4.024086733898119,0.01561948609368153,90.0,-0.9667409371684225,970],[\"random_forest\",1.0,10.0,2.2778294799426826,0.014339701196935463,110.0,-0.9676644834317446,971],[\"random_forest\",1.0,12.0,4.408819129714874,0.010040041779133951,80.0,-0.9684917881070112,972],[\"random_forest\",1.0,3.0,2.0702388904442173,0.012414012513479532,70.0,-0.9358518601294709,973],[\"random_forest\",1.0,11.0,1.4296960419178293,0.013353542001026614,60.0,-0.9676236724737093,974],[\"random_forest\",1.0,12.0,3.3019586549871134,0.017719894178893896,50.0,-0.9662005045761806,975],[\"random_forest\",1.0,12.0,2.2006546114918093,0.011276328624796066,80.0,-0.9687298423476574,976],[\"random_forest\",1.0,12.0,1.9430227907772213,0.012215383457112062,70.0,-0.9684489318623682,977],[\"random_forest\",1.0,11.0,1.1213340724914507,0.016790883269533492,90.0,-0.9681078336977887,978],[\"random_forest\",1.0,12.0,1.6890255174659066,0.018997108701714317,40.0,-0.966194341201291,979],[\"random_forest\",0.0,11.0,1.3231209726399402,0.010745214341044272,100.0,-0.968089572865341,980],[\"random_forest\",1.0,12.0,1.2462803680800465,0.013920479696257686,60.0,-0.9681676178227705,981],[\"random_forest\",1.0,11.0,1.8407865590495855,0.010042482572985668,50.0,-0.9692478012450518,982],[\"random_forest\",1.0,12.0,1.553191843701727,0.02523166604686803,30.0,-0.965561265380242,983],[\"random_forest\",1.0,10.0,2.131913516324226,0.015174217162062282,90.0,-0.9672873344156293,984],[\"random_forest\",1.0,7.0,2.4000528708587257,0.011508073989221258,70.0,-0.9658368776866207,985],[\"random_forest\",1.0,4.0,1.7546061903550896,0.013143400847057223,80.0,-0.9502073079882027,986],[\"random_forest\",1.0,12.0,2.0157736410678226,0.015979754351599974,120.0,-0.9675876216221733,987],[\"random_forest\",1.0,11.0,1.44193745911186,0.020684591384226,110.0,-0.9662801148351694,988],[\"random_forest\",1.0,12.0,1.6295834398633446,0.03384627985799891,20.0,-0.96247210606635,989],[\"random_forest\",0.0,11.0,1.8849036693025485,0.010031251636003458,60.0,-0.9685122256869398,990],[\"random_forest\",1.0,12.0,1.0681072901690438,0.012456359561560965,80.0,-0.968555595546157,991],[\"random_forest\",1.0,12.0,2.680614593060377,0.010033760745050016,40.0,-0.9686794072308164,992],[\"random_forest\",1.0,5.0,1.5161218491405273,0.014029487942422545,90.0,-0.9583189574754097,993],[\"random_forest\",1.0,11.0,1.7993453750655413,0.01163394796425894,70.0,-0.9683340809176895,994],[\"random_forest\",1.0,6.0,1.190128579483869,0.010769614239335833,10.0,-0.9559739492486801,995],[\"random_forest\",1.0,12.0,1.3883771544797878,0.027711361853029748,100.0,-0.9644778932674797,996],[\"random_forest\",1.0,11.0,2.3016043954311565,0.013089015132369663,50.0,-0.967677733464865,997],[\"random_forest\",1.0,12.0,2.047157455155254,0.014601292038352193,60.0,-0.9685262430846919,998],[\"random_forest\",1.0,12.0,2.5013415139734616,0.010004104466533824,80.0,-0.9686531150561887,999]]}],                        {\"template\":{\"data\":{\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"choropleth\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"contour\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"contourcarpet\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmap\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"heatmapgl\"}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2d\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"histogram2dcontour\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scattermapbox\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolar\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterpolargl\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]],\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"geo\":{\"bgcolor\":\"white\",\"lakecolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"white\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"light\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"gridwidth\":2,\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\"}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"ternary\":{\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"bgcolor\":\"#E5ECF6\",\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"title\":{\"x\":0.05},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"zerolinewidth\":2}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"trial_number\"}},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"loss\"}},\"legend\":{\"title\":{\"text\":\"model_choice\"},\"tracegroupgap\":0},\"margin\":{\"t\":60}},                        {\"responsive\": true}                    )                };                            </script>        </div>\n</body>\n</html>",
       "datasetInfos": [],
       "metadata": {},
       "removedWidgets": [],
       "textData": null,
       "type": "htmlSandbox"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "import plotly.express as px\n",
    "\n",
    "def unpack(x):\n",
    "    if x:\n",
    "        return x[0]\n",
    "    return np.nan\n",
    "  \n",
    "def add_hover_data(fig, df, model_choice, ignore_cols=[\"loss\", \"trial_number\", \"model_choice\"]):\n",
    "  fig.update_traces(\n",
    "      customdata = df.loc[df[\"model_choice\"] == model_choice],\n",
    "      hovertemplate = \"<br>\".join(\n",
    "          [\n",
    "              f\"{col}: %{{customdata[{i}]}}\"\n",
    "              for i, col in enumerate(df.columns) if not trials_df.loc[trials_df[\"model_choice\"] == model_choice][col].isnull().any() \n",
    "            and col not in ignore_cols\n",
    "          ]\n",
    "      ),\n",
    "      selector = {\"name\": model_choice},\n",
    "  )\n",
    "  return fig\n",
    "\n",
    "\n",
    "trials_df = pd.DataFrame([pd.Series(t[\"misc\"][\"vals\"]).apply(unpack) for t in spark_trials])\n",
    "trials_df[\"loss\"] = [t[\"result\"][\"loss\"] for t in spark_trials]\n",
    "trials_df[\"trial_number\"] = trials_df.index\n",
    "trials_df[\"model_choice\"] = trials_df[\"model_choice\"].apply(\n",
    "    lambda x: \"random_forest\" if x == 0 else \"gradient_boosting\"\n",
    ")\n",
    "fig = px.scatter(trials_df, x=\"trial_number\", y=\"loss\", color=\"model_choice\")\n",
    "fig = add_hover_data(fig, trials_df, \"random_forest\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "b2dfb25b-3404-42fa-afed-5a9f0302d012",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Because we have lots of experiments in MLFlow now and they are all nested under our experiment container `cmle_hyperopt_tuning`, we need to first find out the experiment run ID from the parent so we can then filter based on that to find the individual trials."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "00e6a8e2-c591-4b56-b6bf-20294e4d8631",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "parent_run_id = mlflow.search_runs(filter_string=f'tags.mlflow.runName = \"{model_name}_hyperopt_tuning\"').iloc[0].run_id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "ca3852fc-555f-4a77-8969-8427c6890cbf",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "And finally we find and record the best overall performing model. Because this one is the best overall, we create a brand new model in the Model Hub and this is the model we'll be using going forward for scoring."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "876a1485-bf08-4fd2-a0a8-100ec90c67a0",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best global run ID: 8caa4810edd34291bae827e2998131cd\n",
      "Best global AUC: 0.9700942778071894\n",
      "Registered model 'cmle_propensity_model_cmenguy_adobe_com' already exists. Creating a new version of this model...\n",
      "2023/04/05 16:56:52 INFO mlflow.tracking._model_registry.client: Waiting up to 300 seconds for model version to finish creation.                     Model name: cmle_propensity_model_cmenguy_adobe_com, version 3\n",
      "Created version '3' of model 'cmle_propensity_model_cmenguy_adobe_com'.\n"
     ]
    }
   ],
   "source": [
    "best_run_global = mlflow.search_runs(filter_string=f\"tags.mlflow.parentRunId = '{parent_run_id}'\", order_by=['metrics.auc DESC']).iloc[0]\n",
    "best_run_id_global = best_run_global.run_id\n",
    "best_auc_global = best_run_global[\"metrics.auc\"]\n",
    "print(f\"Best global run ID: {best_run_id_global}\")\n",
    "print(f\"Best global AUC: {best_auc_global}\")\n",
    "\n",
    "top_model_name = model_name\n",
    "top_model_version = mlflow.register_model(f\"runs:/{best_run_id_global}/hyperopt_model\", top_model_name)\n",
    "time.sleep(15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "e801c288-4c56-4f1b-bf7a-f86a2b83669c",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Because this model is our top model after tuning, we are ready to promote it to production. Promoting it will help refer to it when calling MLFlow, and also will show its status as **Production** in the Model Hub UI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "0e6f5a81-eb04-49b1-90c1-c955503133cf",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Out[25]: <ModelVersion: creation_timestamp=1680713812147, current_stage='Production', description='', last_updated_timestamp=1680713851786, name='cmle_propensity_model_cmenguy_adobe_com', run_id='8caa4810edd34291bae827e2998131cd', run_link='', source='dbfs:/databricks/mlflow-tracking/3680348664249912/8caa4810edd34291bae827e2998131cd/artifacts/hyperopt_model', status='READY', status_message='', tags={}, user_id='8428177847461467', version='3'>"
     ]
    }
   ],
   "source": [
    "from mlflow.tracking import MlflowClient\n",
    " \n",
    "client = MlflowClient()\n",
    "client.transition_model_version_stage(\n",
    "  name=top_model_name,\n",
    "  version=top_model_version.version,\n",
    "  stage=\"Production\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "b0191008-63a5-414c-81ba-097456576ed5",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "The Models page now shows the model version in stage **Production**\n",
    "\n",
    "You can now refer to the model using the path **models:/$MODELNAME/production**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "020006d4-9c22-4159-8302-70d6c7e98969",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AUC: 0.9700942778071894\n"
     ]
    }
   ],
   "source": [
    "prod_model = mlflow.pyfunc.load_model(f\"models:/{top_model_name}/production\")\n",
    " \n",
    "# Sanity-check: This should match the AUC logged by MLflow\n",
    "print(f'AUC: {roc_auc_score(y_test, prod_model.predict(X_test))}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "51a39250-57a3-416f-b6ac-7457b2dfdbc9",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "We can do one last validation on our best model to make sure it doesn't overfit. For that we are computing the confusion matrix to see the share of predictions who are false positives or false negatives which we'd like to minimize."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "71ba1edd-6af7-4758-bcda-fdc157a47dde",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Out[27]: <sklearn.metrics._plot.confusion_matrix.ConfusionMatrixDisplay at 0x7f703dd82cd0>"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEGCAYAAABhMDI9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAr1klEQVR4nO3debxVVf3/8debeRYQJARUFExxgJBQM/2aI1p9tSyHbyWappaaX0tNq5+aZpOVOZRT4vxVMSdSE9E0hxwARSYFCVQmZUaQ+d7P74+97uXcyx3Oud7Lnd7Px2M/7jmfvffa65wD53PWWnuvrYjAzMysRIv6roCZmTUsTgxmZlaGE4OZmZXhxGBmZmU4MZiZWRmt6rsCVlaP7i1jp36t67saVoCZUzrWdxWsAOviEzbEOn2aMo78UsdYuqwor20nTl4/NiJGfJrjbW1ODA3MTv1a8/rYfvVdDSvAiP771ncVrACvrv/Hpy5jybIiXhvbN69tW/f+T49PfcCtzInBzKxgQVEU13cl6owTg5lZgQIopuleHOzEYGZWA8W4xWBmZkkQbHRXkpmZlQigyF1JZmaWy2MMZmZWKoCiJjwztRODmVkNNN0RBicGM7OCBdGkxxg8V5KZWYEiYGOeS3UktZP0uqS3JE2T9IsUv0PSHEmT0jIkxSXpOkmzJE2WNDSnrJGS3k3LyJz4PpKmpH2uk1TllCBuMZiZFUwU8ammW8q1HjgkIlZLag28JKlk3o4LI+Jv5bY/ChiYln2BG4F9JXUHLgOGkQ2DTJQ0JiKWp22+B7wGPAmMACqdG8QtBjOzAgVQHPkt1ZaVWZ2etk5LVXseA9yV9nsV6CqpN3AkMC4ilqVkMA4YkdZ1iYhXI7uX813AsVXVyYnBzKwGilKroboF6CFpQs5yRvmyJLWUNAlYRPbl/lpadVXqLrpGUtsU6wPMzdl9XopVFZ9XQbxS7koyMytQdoFb3l1JSyJiWJXlRRQBQyR1BR6RtCdwCfAh0Aa4BfgJcEVN61wItxjMzAoUwMZokddSULkRK4DngBERsTB1F60HbgeGp83mA7lz8/dNsarifSuIV8qJwcysQIEookVeS3Uk9UwtBSS1Bw4H3kljA6QziI4FpqZdxgAnp7OT9gNWRsRCYCxwhKRukroBRwBj07qPJe2XyjoZeKyqOrkrycysBoqj1s5K6g3cKakl2Y/10RHxuKR/SuoJCJgEnJW2fxI4GpgFrAFOBYiIZZKuBMan7a6IiGXp8Q+AO4D2ZGcjVXm3IicGM7MCFTjGUHVZEZOBz1UQP6SS7QM4u5J1o4BRFcQnAHvmWycnBjOzgomiAscPGhMnBjOzAmV3cHNiMDOzJEJsiJb1XY0648RgZlYDxbU3JUaD48RgZlagbPDZXUlmZlbKg89mZpbDg89mZraFotq7wK3BcWIwMytQIDZG0/36bLqvzMysjnjw2czMygjkriQzMyvLg89mZlYqAp+uamZmm2WDz54Sw8zMcnjw2czMSgWqzRv1NDhODGZmNeAWg5mZlQqg2IPPZma2mWrt1p4NkRODmVmBAnxWkpmZbRahJt2V1HRfmZlZHSqKFnkt1ZHUTtLrkt6SNE3SL1K8v6TXJM2S9ICkNineNj2fldbvlFPWJSk+Q9KROfERKTZL0sXV1cmJwcysQNn9GJTXkof1wCERMRgYAoyQtB/wW+CaiBgALAdOS9ufBixP8WvSdkgaBJwI7AGMAP4iqaWklsCfgaOAQcBJadtKOTGYmRVMtdZiiMzq9LR1WgI4BPhbit8JHJseH5Oek9YfKkkpfn9ErI+IOcAsYHhaZkXE7IjYANyftq2UE4OZWYGy01WV1wL0kDQhZzmjfHnpl/0kYBEwDvgPsCIiNqVN5gF90uM+wFyAtH4lsG1uvNw+lcUr5cFnM7MCFThX0pKIGFZleRFFwBBJXYFHgN0+XQ0/HScGM7MaqItptyNihaTngP2BrpJapVZBX2B+2mw+0A+YJ6kVsA2wNCdeInefyuIVcleSmVmBsmm3lddSHUk9U0sBSe2Bw4G3geeAb6TNRgKPpcdj0nPS+n9GRKT4iemspf7AQOB1YDwwMJ3l1IZsgHpMVXVyi8HMrAZqcRK93sCd6eyhFsDoiHhc0nTgfkm/BN4Ebkvb3wbcLWkWsIzsi56ImCZpNDAd2AScnbqokHQOMBZoCYyKiGlVVciJwcysQNnsqrXT4RIRk4HPVRCfTXZGUfn4OuCblZR1FXBVBfEngSfzrZMTg5lZgbIpMZpuT7wTg+Vtwzrx468PYOOGFhRtggO/vJKTL/yQHx07gLWrszM0VixtxWeHrOHy2+fwz4e7MfrP2xEB7TsWc+5v5rLLHutYNL81V5+3AysWtwYFR397KV87fUmZY/3tpp7cekUfRk+ZwjbbFtXHy21yzv/tbPY9ZAUrlrbmrBF7AbDz7p9w7lXv0aZtULQJbrh0J2a+1YlOXTZx/u/msP2O69iwvgV/vKg/78/sUGk5zY+nxNgqJJ0iafuc51+R9Ga6THy6pDOr2f9gSY/XQb3+XRvl11X9tqbWbYPfPfgfbnpmBjeOm8GE5zvz9sQO/PHRWdz4zAxufGYGu+/zCQccvQKAXv3Wc/VDs7j5nzP41vkfcu1F2YkRLVsFZ1y6gFv/9Q7XPv4uf7+jB+/PbFt6nEXzW/PGvzqzXZ8N9fEym6xxD/Xg56d8tkzstEvmcu+1fTj7y3ty9zV9Of3i7HT3E89ewOzpHfj+UXtx9Y925qxLP6iynOaoFq98bnAaTGIATgG2B5DUGrgF+Gq6TPxzwPNbszLpNDAi4gtb87gNmZT98gfYtFEUbRTK+Xf/yaoWvPVyJ74wYiUAe3x+DZ27Zr/2dxu6hiULWwOwba9NDNx7LQAdOhXTb8D60nUAN1/eh9N+vqBM2fbpTX29C6tWlOskCOjQKfuMOnYuYulH2eeww4C1THqlCwDzZrenV9/1dO2xsfJympnaPCupIaqzxCBpJ0lvS7o1TQz1tKT2koZIelXSZEmPSOom6RvAMODedPXfdmTdXEsB0iXeM1K5d6TtS46zOuewXSQ9kSaLuklSi3RF4R2SpkqaIun8tN8ASc+kFskbknZJv+pflDSGbGS/2vLTNkdIeiWV86CkTik+QtI7kt4Avl437/TWVVQE3z/ss5yw95587qBV7DZ0Tem6fz+1DUO+uJqOnYu32O+p+7rz+S+t2iL+4dw2/Gdq+9Jy/v1UF3p8ZiO77LGu7l6Elbrpih05/ZK53P3yJE7/6QfcfnXWqpv9dgcOOHIZALsOXk2vPuvp8Rm34HIVR4u8lsaorms9EPhzROwBrACOA+4CfhIRewNTgMsi4m/ABOBbETEkIuaTnWf7vqT7JH2r5Eu4GsOBc8kmitqF7Mt4CNAnIvaMiL2A29O296a6DQa+ACxM8aHAeRGxaz7lS+oB/Bw4LCKGptfxI0ntgFuBrwL7AJ+prNKSzii5XH7x0obdn96yJdz4zAzunTidGZM68N477UrXPf9oNw4+dvkW+0x6uRNj79uW0362oEx87SctuPL0nTjrivl07FzMujXi/ut7cfKFC7cow+rGV769iJt/uQPfOWAIN/9yB87/zRwARt+0PZ26FPHnJ6ZyzMiP+M+0jhQ37H+aW1XJPZ/znBKj0anrxDAnIialxxPJvky7RsS/UuxO4KCKdoyI04FDyS7QuAAYlcfxXk8TRRUB9wFfBGYDO0u6XtII4GNJncmSxSPpWOsiYk1OGXMKKH8/skTxcmrtjAR2JLukfU5EvJsuPrmnskpHxC0RMSwihvXctnHc/KPTNkUM/sJqxj/XGYCVS1syY1IH9j304zLbzZ7ejj9d0I/Lb59Dl+6bv1k2bYQrT9+JQ76+nC8enXU9LXy/LR9+0IbvH7YbJw8fxOKFrTn7yM+ybFHz7raoS4d9fQkvP9UNgBef6M6ug7MG8prVLfnjRTtz9pf35Oof7cw2227kw7ntqiqqWQlgU7TIa2mM6vp/3Pqcx0VA10J2jogpwBRJdwNzyMYhNpESWmpFtMndZcsiYrmkwcCRwFnA8cB5VRz2k6qqVMFzAeMi4qTcFZKGVFFOo7RiaUtatcqSwvq14o0XOnP82YsAePGJrux72Me0abf5LVo0rzVXnN6fC697n767bP6nEAF//PEO9Bu4nuPOXFwa77/7OkZP2XzdzcnDB3H9P2b4rKQ6tHRRa/bedxWTX+vCkC98zIL3si//jp03sX5dCzZtbMGIExcz5fXOrFndOH60bC2NtZsoH1v7p9hKYLmkAyPiReA7QEnrYRXQGSD10Q+LiOfTuiHA++nxe2RdM6OB/yaborbE8HQp+PvACcAtqatnQ0Q8JGkGcE9ErJI0T9KxEfGopLZkVwRWZ4vygVeBP0saEBGzJHUkm7nwHWAnSbtExH+AkyottZFY9lFrfn/eDhQXi+JiOOirK9jv8KyF8K/HunH8OR+V2f7eaz7DquUtueGSzWcj3fDUTKa93pFn/9ad/ruv5fuHZWe3nHrJAoYfuuUYhNWei6+dxd77raJLt03c/e83uedPfbn2kv6cden7tGwVbFjfgmt/2h/IBp9//IfZEOL9me255if9qyxn7Oie9fWy6kcj7ibKh7JejjooOLur0OMRsWd6fgHQCXgUuAnoQNbNc2r6VX8c8CtgLXAg8ABZ19Nasl/x50XEBEm9yOYMaQ88RXbZdydJBwNXkCWYAWTzjPwAKBlXKEnvl0TEPyQNBG4GegAbya4k3AG4ICK+kvM6VldVfkQUSzqE7GYZJedc/jwixqSuqz8Ba4AXgV1yy67IsMHt4vWx/araxBqYEf33re8qWAFeXf8PPi5e+qm+1bvttl0cMuob1W8IPHzAjROrm121oamzxGA148TQ+DgxNC61lRgOvq3CWSm28OgX/9LoEoNH9czMClRyo56myonBzKxAgdhU7MFnMzPL0Vinu8iHE4OZWaHCXUlmZpbDYwxmZrYFJwYzMysViCIPPpuZWS4PPpuZWanw4LOZmZUXTTgxNN1OMjOzOlN792OQ1E/Sc+kWxtMknZfil0uaL2lSWo7O2ecSSbPSTcOOzImPSLFZki7OifeX9FqKPyCpDVVwYjAzq4EI5bXkYRPw44gYRHZ/l7MlDUrrrkk3LxsSEU8CpHUnAnsAI4C/pDtVtgT+DBxFdo+Yk3LK+W0qawCwHDitqgo5MZiZFSgCioqV11J9WbEwIt5Ij1cBb5NN3V+ZY4D70y2P5wCzyO4uORyYlW4mtgG4HzhGkoBDgL+l/e8Ejq2qTk4MZmY1UIzyWoAeJbfuTcsZlZWZblfwOeC1FDpH0mRJoyR1S7E+wNyc3ealWGXxbYEVEbGpXLxSTgxmZgUKCupKWlJy69603FJRmekGZQ8B/xsRHwM3kt2TZgjZPen/sFVeHD4rycysBmr3Dm6SWpMlhXsj4mGAiPgoZ/2twOPp6Xwg96YtfVOMSuJLga6SWqVWQ+72FXKLwcysBiLyW6qTxgBuA96OiD/mxHvnbPY1YGp6PAY4UVLbdKvhgcDrwHhgYDoDqQ3ZAPWYyO7G9hxQcsu5kWR3wayUWwxmZjVQi9cxHAB8B5giaVKK/ZTsrKIhZD1X7wFnZseNaZJGA9PJzmg6OyKKACSdA4wlu4f9qIiYlsr7CXC/pF8Cb5Iloko5MZiZFSg7K6l2Olwi4iWocH6NJ6vY5yrgqgriT1a0X0TMJjtrKS9ODGZmNZBPN1Fj5cRgZlYDTXlKDCcGM7MCBXlf1dwoOTGYmdVAE+5JcmIwMytYQOQx3UVj5cRgZlYD7koyM7MymuVZSZKup4putIj4YZ3UyMysgSuZK6mpqqrFMGGr1cLMrDEJoDkmhoi4M/e5pA4Rsabuq2Rm1vA15a6kaq/plrS/pOnAO+n5YEl/qfOamZk1WCKK81sao3wm+/gTcCTZ1K1ExFvAQXVYJzOzhi/yXBqhvM5Kioi52cywpYrqpjpmZo1ANN/B5xJzJX0BiHQzifPI7klqZtZ8NdLWQD7y6Uo6Czib7B6hC8huM3d2HdbJzKwRUJ5L41NtiyEilgDf2gp1MTNrPIrruwJ1J5+zknaW9HdJiyUtkvSYpJ23RuXMzBqkkusY8lkaoXy6kv4PGA30BrYHHgTuq8tKmZk1dLV1z+eGKJ/E0CEi7o6ITWm5B2hX1xUzM2vQmuPpqpK6p4f/kHQxcD/ZyzyBKu5FambWLDTSbqJ8VDX4PJEsEZS8+jNz1gVwSV1VysysoVMjbQ3ko9KupIjoHxE7p7/lFw8+m1nzFYLiPJdqSOon6TlJ0yVNk3ReineXNE7Su+lvtxSXpOskzZI0WdLQnLJGpu3flTQyJ76PpClpn+tU7orl8vIZY0DSnpKOl3RyyZLPfmZmTVbtjTFsAn4cEYOA/YCzJQ0CLgaejYiBwLPpOcBRwMC0nAHcCKXd/5cB+wLDgctKkkna5ns5+42oqkL5nK56GXB9Wr4E/A7477xerplZU1VLiSEiFkbEG+nxKrKZJfoAxwAls1zfCRybHh8D3BWZV4GuknqTzWk3LiKWRcRyYBwwIq3rEhGvRkQAd+WUVaF8WgzfAA4FPoyIU4HBwDZ57Gdm1nTlnxh6SJqQs5xRWZGSdgI+B7wG9IqIhWnVh0Cv9LgPMDdnt3kpVlV8XgXxSuUzV9LaiCiWtElSF2AR0C+P/czMmqbCbtSzJCKGVbeRpE7AQ8D/RsTHucMAERHS1hvuzqfFMEFSV+BWsjOV3gBeqctKmZk1dIr8lrzKyiYofQi4NyIeTuGPUjcQ6e+iFJ9P2R/nfVOsqnjfCuKVqjYxRMQPImJFRNwEHA6MTF1KZmbNVy2NMaQzhG4D3o6IP+asGgOUnFk0EngsJ35yOjtpP2Bl6nIaCxwhqVsadD4CGJvWfSxpv3Ssk3PKqlBVF7gNrWpdyWCJmVlzVIsdOwcA3wGmSJqUYj8FfgOMlnQa8D5wfFr3JHA0MAtYA5wKEBHLJF0JjE/bXRERy9LjHwB3AO2Bf6SlUlWNMfyhinUBHFJVwVYzMyd34Mjth9R3NawALQdsX99VsALog9a1U1AtXfkcES9R+fzch1awfVDJrQ8iYhQwqoL4BGDPfOtUaWKIiC/lW4iZWbPSiOdBykdet/Y0M7NynBjMzCyXmvCNepwYzMxqogm3GPKZEkOSvi3p0vR8B0nD675qZmYNU77XMDTWGVjzucDtL8D+wEnp+Srgz3VWIzOzxqAJ39ozn66kfSNiqKQ3ASJiuaQ2dVwvM7OGrZG2BvKRT2LYKKkl6W2Q1BNowsMuZmbVa6zdRPnIJzFcBzwCbCfpKrLZVn9ep7UyM2vIopmflRQR90qaSHYFnoBjI+LtOq+ZmVlD1pxbDJJ2IJuP4++5sYj4oC4rZmbWoDXnxAA8QfYWCGgH9AdmAHvUYb3MzBq0Zj3GEBF75T5Ps67+oM5qZGZm9argK58j4g1J+9ZFZczMGo3m3GKQ9KOcpy2AocCCOquRmVlD19zPSgI65zzeRDbm8FDdVMfMrJFori2GdGFb54i4YCvVx8yswRPNdPBZUquI2CTpgK1ZITOzRqE5JgbgdbLxhEmSxgAPAp+UrIyIh+u4bmZmDVMjnjk1H/mMMbQDlpLd47nkeoYAnBjMrPlqpoPP26UzkqayOSGUaMK50sysek25xVDV/RhaAp3S0jnnccliZtZ8RZ5LNSSNkrRI0tSc2OWS5kualJajc9ZdImmWpBmSjsyJj0ixWZIuzon3l/Raij+Qz20TqmoxLIyIK6p/WWZmzUyeX/p5ugO4AbirXPyaiPh9bkDSIOBEsimJtgeekbRrWv1n4HBgHjBe0piImA78NpV1v6SbgNOAG6uqUFUthsZ56yEzs62gtm7tGREvAMvyPOwxwP0RsT4i5gCzgOFpmRURsyNiA3A/cIwkkY0P/y3tfydwbHUHqSoxHJpnRc3Mmp/8u5J6SJqQs5yR5xHOkTQ5dTV1S7E+wNycbealWGXxbYEVEbGpXLxKlXYlRUS+GczMrNkpYEqMJRExrMDibwSuJEstVwJ/AL5bYBk1VvAkemZmzV7tjjFsWXzERyWPJd0KPJ6ezgf65WzaN8WoJL4U6FpywXK57StVVVeSmZlVQAUsNSpf6p3z9Gtklw0AjAFOlNRWUn9gINnFyOOBgekMpDZkA9RjIiKA58huyQwwEnisuuO7xWBmVhO11GKQdB9wMNlYxDzgMuBgSUPSUd4DzgSIiGmSRgPTySY1PTsiilI55wBjyS41GBUR09IhfgLcL+mXwJvAbdXVyYnBzKwGausCt4g4qYJwpV/eEXEVcFUF8SeBJyuIzyY7aylvTgxmZjXRhK98dmIwMyuUb9RjZmZbcIvBzMxyNeVJ9JwYzMxqwonBzMxyucVgZmabBc32Rj1mZlYB4RaDmZmV58RgZma5FE03MzgxmJkVqo5nV61vTgxmZjXgMQYzMyvDU2KYmVlZbjGYmVmpcFeSmZmV58RgZmYlfIGbmZltQcVNNzM4MZiZFcrXMZhV7Ed//IB9D1vFiiWtOPOQzwLQuesmfnrT+/Tqu4GP5rXhqjN3ZPXKVnToXMRPbviA7bbfQMtWwd9u2o6nH+gOwFX3zma3oZ8w7fWOXDpy5/p8Sc1CixbBtbc8z9Il7bj84v353fUv0r7DRgC6dtvAzLe7cuXP9mOvIYu59Fev8eHCDgD8+4Xtue/O3QDo2GkD5100iR37f0wAf/rNUN6Z1r2+XlK98OmqjYCky4HVEfH7WixzGHByRPzw05ZfF/Wrb08/0J0xt/fgwmvnlsaOP2cRb77UidE39OL4cz7ihHMWcdtV2/Pfpyzhg5ltuWxkf7bpvonbXnyHfz7clU0bW/DgjT1p235bvvztpfX4apqPY77xH+a+35kOHbNkcNG5B5au+9mVr/HKS71Ln0+bvC2XX7z/FmWc+cMpTHxtO3516XBatSqmbbtNdV/xhqYJtxha1HcFGipJrSJiQkT8sL7r0lBNfa0Tq5aX/W2x/5Ef88zo7JfjM6O7s/+IjwGIgPYdi4GgXcciVq1oSdEmATDppc6sXd1yq9a9udq251o+v/+HjH1ixy3Wte+wkb2HLuGVF3tXsOdmHTpuZM/BS0vL2LSpBZ+sblMn9W3IFPkt1ZYjjZK0SNLUnFh3SeMkvZv+dktxSbpO0ixJkyUNzdlnZNr+XUkjc+L7SJqS9rlOkqqrU4NODJI6SnpC0luSpko6QdJ7knqk9cMkPZ+zy2BJr6Q35ntpm96SXpA0KZVxYIqPkPRGKvvZFLtc0t2SXgbulnSwpMerKj/td6Gk8emD+kVO/GeSZkp6Cfhsnb1RDUi3HhtZtqg1AMsWtaJbj+xX6Zjbe7DDwHX835vTufmfM7nx0j5EVPvv02rZmedOYdSNe1JcQTfI/gcu5K2JPVm7pnVpbLc9lnHDqH9yxe/+zQ47ZUn+M70/YeWKNpx/yRtc/9fnOO+iN5tfiyHIfu3ks1TvDmBEudjFwLMRMRB4Nj0HOAoYmJYzgBshSyTAZcC+wHDgspJkkrb5Xs5+5Y+1hQadGMhewIKIGBwRewJPVbP93sAhwP7ApZK2B/4HGBsRQ4DBwCRJPYFbgeMiYjDwzZwyBgGHRcRJ+ZQv6QiyN3s4MATYR9JBkvYBTkyxo4HPV1ZpSWdImiBpwkbWV/MSGxOVfvnvc/Aq/jOtPf/zuUH84PBdOfuq+XToVFTP9Wtehu//ISuWt2XWzK4Vrj/40Hn869k+pc9nzezKKccfyTnfPYQxD+/M//vVawC0bBkMGLiSJx/tz7mnf4l161py/Ldmbo2X0KCoOL+lOhHxArCsXPgY4M70+E7g2Jz4XZF5FegqqTdwJDAuIpZFxHJgHDAiresSEa9GRAB35ZRVqYaeGKYAh0v6raQDI2JlNds/FhFrI2IJ8BzZl/V44NTUx79XRKwC9gNeiIg5ABGR+6GMiYi1BZR/RFreBN4AdiNLFAcCj0TEmoj4GBhTWaUj4paIGBYRw1rTtpqX2LAtX9Ka7ttlrYTu221kxdKsq+mIE5bx8pPbAGLBe2358IM29BvQlJJgwzdor6Xsd8BCbn9gLD+5bAJ7D13CBT+fAECXbdaz6+7Lef2Vz5Ruv3ZNa9atzT6/Ca9+hlYti+myzXqWLG7PksXtmPF21mX40vPbs8uu1f3XbFpKrmPIsyupR8kPv7SckcchekXEwvT4Q6BXetwHmJuz3bwUqyo+r4J4lRr04HNEzEx9aEcDv0xdPpvYnNDald9lyyLiBUkHAV8G7pD0R2B5FYf9pKoqVfBcwK8j4ubcFZL+t4pymqxXn+7CYccvY/QNvTjs+GW8MrYLAIvnt2HIgauZ+nonuvbYSN9d1rHwg+bXL12f7rhlD+64ZQ8A9hqymONOnMXvfzkMgC/+1wJef+UzbNyweaynW/d1LF/WFhC77r4ctYCPV7YBxOJFHejTbxXz53ZmyD6L+eC9zvXwiupR/t1EAEsiYljNDxUhbd3L6Rp0YkhdQcsi4h5JK4DTgfeAfYB/AMeV2+UYSb8GOgIHAxdL2hGYFxG3SmoLDAWuAv4iqX9EzJHUvVyroTJblA+sBa6UdG9ErJbUB9gIvECWiH5N9j5/Fbi54mIbp4v/8j5777+abbpv4p4J07n7D7144Ibt+NlN7zPixGUsmp+drgpw7596ccGfPuCmZ2cgwW1Xbc/Hy7J/fn94ZBZ9B6yjfYdi7pkwnWt+3JeJ/+pSny+t2Tno0Hk8eO+uZWIHHLyALx8zh6IisWF9S377i2Fkv4Pgpmv34qL/N5FWrYv5cEEHrvn10ApKbdrq+Kv6I0m9I2Jh6g5alOLzgX452/VNsflk30m58edTvG8F21epQScGYC/gaknFZF+23wfaA7dJupLsheeaTNbF0wO4MiIWpNH5CyVtBFaTnX66ODXnHpbUguxNPzyP+mxRPrBA0u7AK2mwfzXw7Yh4Q9IDwFup/PE1fhcaqN/8YMszWwAuPmGXLWLLPmrNT0/aMg7w468NqNV6WfWmTOrJlEk9S59ffN6BW2zz+MM78/jDFV9XMntWV8474+C6ql7jULeJYQwwEvhN+vtYTvwcSfeTDTSvTMljLPCrnAHnI4BLImKZpI8l7Qe8BpwMXF/dwRVN+PZ0jVEXdY99dWh9V8MK0HJA//qughXglQ/uYuW6Dz/VKXGdu/aNoQeel9e2Lzx+0cSqupIk3Uf2a78H8BHZ2UWPAqOBHYD3gePTl7yAG8hOzFkDnBoRE1I53wV+moq9KiJuT/FhZGc+tSfraTk3qvnib+gtBjOzhieAotr5UV3JGZAAW/xCTF/oZ1dSzihgVAXxCcCehdTJicHMrAY8u6qZmZXVhLvhnRjMzGrALQYzM9vM026bmVkuAaqlweeGyInBzKwG5DEGMzMr5a4kMzMrq6C5khodJwYzsxrwWUlmZlaWWwxmZlYqfFaSmZmV13TzghODmVlN+HRVMzMry4nBzMxKBVBc35WoO04MZmYFEuGuJDMzK6e46TYZnBjMzArlriQzMyvPXUlmZlaWE4OZmW3WtCfRa1HfFTAza3QCKIr8ljxIek/SFEmTJE1Ise6Sxkl6N/3tluKSdJ2kWZImSxqaU87ItP27kkbW9OU5MZiZ1YAi8loK8KWIGBIRw9Lzi4FnI2Ig8Gx6DnAUMDAtZwA3QpZIgMuAfYHhwGUlyaRQTgxmZjURkd9Sc8cAd6bHdwLH5sTvisyrQFdJvYEjgXERsSwilgPjgBE1ObATg5lZoQIojvwW6CFpQs5yRiUlPi1pYs76XhGxMD3+EOiVHvcB5ubsOy/FKosXzIPPZmYFK6g1sCSne6gyX4yI+ZK2A8ZJeqfM0SJC2nq3BnKLwcysJmqxKyki5qe/i4BHyMYIPkpdRKS/i9Lm84F+Obv3TbHK4gVzYjAzK1QARcX5LdWQ1FFS55LHwBHAVGAMUHJm0UjgsfR4DHByOjtpP2Bl6nIaCxwhqVsadD4ixQrmriQzs4IFRK3NidELeEQSZN/J/xcRT0kaD4yWdBrwPnB82v5J4GhgFrAGOBUgIpZJuhIYn7a7IiKW1aRCTgxmZjVRSxe4RcRsYHAF8aXAoRXEAzi7krJGAaM+bZ2cGMzMClVyVlIT5cRgZlYTTXhKDCcGM7OacGIwM7NSEVBUVN+1qDNODGZmNeEWg5mZleHEYGZmm4XPSjIzsxwBUXsXuDU4TgxmZjWRx3QXjZUTg5lZoSKg2InBzMxyefDZzMxyhVsMZma22ae+bWeD5sRgZlYoT6JnZma5AghPiWFmZqWiVm/U0+A4MZiZ1UC4K8nMzMpowi0GRRMeWW+MJC0mu79rU9MDWFLflbCCNNXPbMeI6PlpCpD0FNn7k48lETHi0xxva3NisK1C0oSIGFbf9bD8+TNrvlrUdwXMzKxhcWIwM7MynBhsa7mlvitgBfNn1kx5jMHMzMpwi8HMzMpwYjAzszKcGKyUpFMkbZ/z/CuS3pT0lqTpks6sZv+DJT1eB/X6d22UX1f1a6gkXS7pglouc5ik62qj/Lqon9UOX/lsuU4BpgILJLUmG3wcHhHzJLUFdtqalZHUKiI2RcQXtuZxrWLp85gATKjvuljdcouhCZO0k6S3Jd0qaZqkpyW1lzRE0quSJkt6RFI3Sd8AhgH3SpoEbEf2w2EpQESsj4gZqdw70vYlx1mdc9gukp6QNEPSTZJaSGqZ9pkqaYqk89N+AyQ9k1okb0jaJf2qf1HSGGB6PuWnbY6Q9Eoq50FJnVJ8hKR3JL0BfL1u3umtR1LH9PrfSu/nCZLek9QjrR8m6fmcXQan9+VdSd9L2/SW9IKkSamMA1N8RHr/3pL0bIpdLuluSS8Dd1fQ6tqi/LTfhZLGp39jv8iJ/0zSTEkvAZ+tszfKPp2I8NJEF7Jf+JuAIen5aODbwGTgv1LsCuBP6fHzwLCc/f8KLALuA74FtEjxO4Bv5Gy3Ov09GFgH7Ay0BMYB3wD2AcblbN81/X0N+Fp63A7okMr4BOhfQPk9gBeAjmm7nwCXpjLnAgMBpdf/eH1/Lp/yMz0OuDXn+TbAe0CP9HwY8Hx6fDnwFtA+vUdzge2BHwM/S9u0BDoDPdP6/inePaeMiUD7nM/g8WrKP4KstSmyH5+PAwelfwdT0ufcBZgFXFDf76mXLRd3JTV9cyJiUno8EdiF7Iv5Xyl2J/BgRTtGxOmS9gIOAy4ADifrbqrK6xExG0DSfcAXgWeBnSVdDzwBPC2pM9AnIh5Jx1qX9ikpY04B5a8DBgEvp/3bAK8Au6XX/27a/h7gjGrq39BNAf4g6bdkX9AvptdcmcciYi2wVtJzwHBgPDAqdRc+GhGTJB0MvFDyvkfEspwyxqQy8i3/i2TJ4c20TSey5NwZeCQi1gCkVqE1QO5KavrW5zwuAroWsnNETImIa8iSwnEpvIn0byd15bTJ3WXLImI5MJisRXIWWUukKp9UVaUKnousRTIkLYMi4rRqjtEoRcRMYChZgvilpEvJ+TzIWklldtmyiHiB7Bf8fOAOSSdXc9iafB6/zvk8BkTEbdUcwxoQJ4bmZyWwvKRfGfgOUNJ6WEX2qw5JndKvyBJD2Dzr63tk3QIA/w20ztluuKT+KWGcALyU+r9bRMRDwM+BoRGxCpgn6dh0vLaSOuRR/y3KB14FDpA0IJXVUdKuwDvATpJ2SfuelEf5DZqys8bWRMQ9wNVkSeI9Nn8ex5Xb5RhJ7SRtS9YNNF7SjsBHEXErWZIeSvYeHiSpfzpO9zyrtEX5wFjguznjPH0kbUfW3XdsGufqDHy14DfAtgp3JTVPI4Gb0hfxbODUFL8jxdcCBwIXSboZWEv2q/GUtN2twGOS3gKeouwvyvHADcAA4DngEWAv4PaSgWLgkvT3O8DNkq4ANgLfzKPuW5QfEcWSTgHuU3b2FMDPI2KmpDOAJyStAV4kJb5GbC/gaknFZO/Z98n6+G+TdCVZqyzXZLL3qQdwZUQskDQSuFDSRmA1cHJELE7v1cPpc1pE1kqszhblk53VtjvwSurmWg18OyLekPQA2bjEIrLP0hogT4lhZmZluCvJzMzKcGIwM7MynBjMzKwMJwYzMyvDicHMzMpwYrBGRVJRzhw/D+Z57UNlZZXO+STpr5IGVbHtwZIKnswvdx6jfOLltlld1foKtvdspVYrnBissVmbrqbdE9hAdiV1KUk1ujYnIk6PiOlVbHIw4FlerVlwYrDG7EVggMrNyKpsNterc2b3PBNAmRuUzcz6DNkMsqR1z0salh6XmWVU0k5kCej81Fo5UFJPSQ+lY4yXdEDad1tls9hOk/RXsukhqiTpUUkT0z5nlFt3TYo/K6lniu0i6am0z4uSdquVd9Ms8ZXP1iillsFRZFdeQzatw54RMSd9ua6MiM+nK6FflvQ08DmyqZ4HAb3IpvUeVa7cnmRXdh+UyuoeEcsk3UQ2y+vv03b/B1wTES9J2oFsGojdgcuAlyLiCklfBvKZs+m76RjtyaaseCgilgIdgQkRcb6yOZEuA84hm7n0rIh4V9K+wF+AQ2rwNppVyInBGpv2yu4XAVmL4TayLp7cGVmPAPbW5ntGbEM2u+dBwH0RUUQ2bcM/Kyh/PyqfZTTXYcAgbZ7ZtEuaG+gg0n0fIuIJScvzeE0/lPS19LhfqutSoBh4IMXvIZuuolN6vQ/mHLstZrXIicEam7URMSQ3kL4gc+drEnBuRIwtt93RtViPFsB+JdOFl6tL3pRNVHgYsH9ErFF2k53yM6SWiHTcFeXfA7Pa5DEGa4rGAt9Xdr8BJO0qqSPZ7J4npDGI3sCXKti3sllGS2eeTZ4Gzi15ImlIevgC8D8pdhTQrZq6bgMsT0lhN7IWS4kWZDciIpX5UkR8DMyR9M10DEkaXM0xzArixGBN0V/Jxg/ekDQVuJmsdfwI8G5adxfZzXzKiIjFZDfzeVjZ7LElXTl/B75WMvgM/BAYlga3p7P57KhfkCWWaWRdSh9UU9engFaS3gZ+Q5aYSnxCNs34VLIxhCtS/FvAaal+04Bj8nhPzPLm2VXNzKwMtxjMzKwMJwYzMyvDicHMzMpwYjAzszKcGMzMrAwnBjMzK8OJwczMyvj/tEA8U6qmGz4AAAAASUVORK5CYII=\n"
     },
     "metadata": {
      "application/vnd.databricks.v1+output": {
       "addedWidgets": {},
       "arguments": {},
       "data": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAYYAAAEGCAYAAABhMDI9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAr1klEQVR4nO3debxVVf3/8debeRYQJARUFExxgJBQM/2aI1p9tSyHbyWappaaX0tNq5+aZpOVOZRT4vxVMSdSE9E0hxwARSYFCVQmZUaQ+d7P74+97uXcyx3Oud7Lnd7Px2M/7jmfvffa65wD53PWWnuvrYjAzMysRIv6roCZmTUsTgxmZlaGE4OZmZXhxGBmZmU4MZiZWRmt6rsCVlaP7i1jp36t67saVoCZUzrWdxWsAOviEzbEOn2aMo78UsdYuqwor20nTl4/NiJGfJrjbW1ODA3MTv1a8/rYfvVdDSvAiP771ncVrACvrv/Hpy5jybIiXhvbN69tW/f+T49PfcCtzInBzKxgQVEU13cl6owTg5lZgQIopuleHOzEYGZWA8W4xWBmZkkQbHRXkpmZlQigyF1JZmaWy2MMZmZWKoCiJjwztRODmVkNNN0RBicGM7OCBdGkxxg8V5KZWYEiYGOeS3UktZP0uqS3JE2T9IsUv0PSHEmT0jIkxSXpOkmzJE2WNDSnrJGS3k3LyJz4PpKmpH2uk1TllCBuMZiZFUwU8ammW8q1HjgkIlZLag28JKlk3o4LI+Jv5bY/ChiYln2BG4F9JXUHLgOGkQ2DTJQ0JiKWp22+B7wGPAmMACqdG8QtBjOzAgVQHPkt1ZaVWZ2etk5LVXseA9yV9nsV6CqpN3AkMC4ilqVkMA4YkdZ1iYhXI7uX813AsVXVyYnBzKwGilKroboF6CFpQs5yRvmyJLWUNAlYRPbl/lpadVXqLrpGUtsU6wPMzdl9XopVFZ9XQbxS7koyMytQdoFb3l1JSyJiWJXlRRQBQyR1BR6RtCdwCfAh0Aa4BfgJcEVN61wItxjMzAoUwMZokddSULkRK4DngBERsTB1F60HbgeGp83mA7lz8/dNsarifSuIV8qJwcysQIEookVeS3Uk9UwtBSS1Bw4H3kljA6QziI4FpqZdxgAnp7OT9gNWRsRCYCxwhKRukroBRwBj07qPJe2XyjoZeKyqOrkrycysBoqj1s5K6g3cKakl2Y/10RHxuKR/SuoJCJgEnJW2fxI4GpgFrAFOBYiIZZKuBMan7a6IiGXp8Q+AO4D2ZGcjVXm3IicGM7MCFTjGUHVZEZOBz1UQP6SS7QM4u5J1o4BRFcQnAHvmWycnBjOzgomiAscPGhMnBjOzAmV3cHNiMDOzJEJsiJb1XY0648RgZlYDxbU3JUaD48RgZlagbPDZXUlmZlbKg89mZpbDg89mZraFotq7wK3BcWIwMytQIDZG0/36bLqvzMysjnjw2czMygjkriQzMyvLg89mZlYqAp+uamZmm2WDz54Sw8zMcnjw2czMSgWqzRv1NDhODGZmNeAWg5mZlQqg2IPPZma2mWrt1p4NkRODmVmBAnxWkpmZbRahJt2V1HRfmZlZHSqKFnkt1ZHUTtLrkt6SNE3SL1K8v6TXJM2S9ICkNineNj2fldbvlFPWJSk+Q9KROfERKTZL0sXV1cmJwcysQNn9GJTXkof1wCERMRgYAoyQtB/wW+CaiBgALAdOS9ufBixP8WvSdkgaBJwI7AGMAP4iqaWklsCfgaOAQcBJadtKOTGYmRVMtdZiiMzq9LR1WgI4BPhbit8JHJseH5Oek9YfKkkpfn9ErI+IOcAsYHhaZkXE7IjYANyftq2UE4OZWYGy01WV1wL0kDQhZzmjfHnpl/0kYBEwDvgPsCIiNqVN5gF90uM+wFyAtH4lsG1uvNw+lcUr5cFnM7MCFThX0pKIGFZleRFFwBBJXYFHgN0+XQ0/HScGM7MaqItptyNihaTngP2BrpJapVZBX2B+2mw+0A+YJ6kVsA2wNCdeInefyuIVcleSmVmBsmm3lddSHUk9U0sBSe2Bw4G3geeAb6TNRgKPpcdj0nPS+n9GRKT4iemspf7AQOB1YDwwMJ3l1IZsgHpMVXVyi8HMrAZqcRK93sCd6eyhFsDoiHhc0nTgfkm/BN4Ebkvb3wbcLWkWsIzsi56ImCZpNDAd2AScnbqokHQOMBZoCYyKiGlVVciJwcysQNnsqrXT4RIRk4HPVRCfTXZGUfn4OuCblZR1FXBVBfEngSfzrZMTg5lZgbIpMZpuT7wTg+Vtwzrx468PYOOGFhRtggO/vJKTL/yQHx07gLWrszM0VixtxWeHrOHy2+fwz4e7MfrP2xEB7TsWc+5v5rLLHutYNL81V5+3AysWtwYFR397KV87fUmZY/3tpp7cekUfRk+ZwjbbFtXHy21yzv/tbPY9ZAUrlrbmrBF7AbDz7p9w7lXv0aZtULQJbrh0J2a+1YlOXTZx/u/msP2O69iwvgV/vKg/78/sUGk5zY+nxNgqJJ0iafuc51+R9Ga6THy6pDOr2f9gSY/XQb3+XRvl11X9tqbWbYPfPfgfbnpmBjeOm8GE5zvz9sQO/PHRWdz4zAxufGYGu+/zCQccvQKAXv3Wc/VDs7j5nzP41vkfcu1F2YkRLVsFZ1y6gFv/9Q7XPv4uf7+jB+/PbFt6nEXzW/PGvzqzXZ8N9fEym6xxD/Xg56d8tkzstEvmcu+1fTj7y3ty9zV9Of3i7HT3E89ewOzpHfj+UXtx9Y925qxLP6iynOaoFq98bnAaTGIATgG2B5DUGrgF+Gq6TPxzwPNbszLpNDAi4gtb87gNmZT98gfYtFEUbRTK+Xf/yaoWvPVyJ74wYiUAe3x+DZ27Zr/2dxu6hiULWwOwba9NDNx7LQAdOhXTb8D60nUAN1/eh9N+vqBM2fbpTX29C6tWlOskCOjQKfuMOnYuYulH2eeww4C1THqlCwDzZrenV9/1dO2xsfJympnaPCupIaqzxCBpJ0lvS7o1TQz1tKT2koZIelXSZEmPSOom6RvAMODedPXfdmTdXEsB0iXeM1K5d6TtS46zOuewXSQ9kSaLuklSi3RF4R2SpkqaIun8tN8ASc+kFskbknZJv+pflDSGbGS/2vLTNkdIeiWV86CkTik+QtI7kt4Avl437/TWVVQE3z/ss5yw95587qBV7DZ0Tem6fz+1DUO+uJqOnYu32O+p+7rz+S+t2iL+4dw2/Gdq+9Jy/v1UF3p8ZiO77LGu7l6Elbrpih05/ZK53P3yJE7/6QfcfnXWqpv9dgcOOHIZALsOXk2vPuvp8Rm34HIVR4u8lsaorms9EPhzROwBrACOA+4CfhIRewNTgMsi4m/ABOBbETEkIuaTnWf7vqT7JH2r5Eu4GsOBc8kmitqF7Mt4CNAnIvaMiL2A29O296a6DQa+ACxM8aHAeRGxaz7lS+oB/Bw4LCKGptfxI0ntgFuBrwL7AJ+prNKSzii5XH7x0obdn96yJdz4zAzunTidGZM68N477UrXPf9oNw4+dvkW+0x6uRNj79uW0362oEx87SctuPL0nTjrivl07FzMujXi/ut7cfKFC7cow+rGV769iJt/uQPfOWAIN/9yB87/zRwARt+0PZ26FPHnJ6ZyzMiP+M+0jhQ37H+aW1XJPZ/znBKj0anrxDAnIialxxPJvky7RsS/UuxO4KCKdoyI04FDyS7QuAAYlcfxXk8TRRUB9wFfBGYDO0u6XtII4GNJncmSxSPpWOsiYk1OGXMKKH8/skTxcmrtjAR2JLukfU5EvJsuPrmnskpHxC0RMSwihvXctnHc/KPTNkUM/sJqxj/XGYCVS1syY1IH9j304zLbzZ7ejj9d0I/Lb59Dl+6bv1k2bYQrT9+JQ76+nC8enXU9LXy/LR9+0IbvH7YbJw8fxOKFrTn7yM+ybFHz7raoS4d9fQkvP9UNgBef6M6ug7MG8prVLfnjRTtz9pf35Oof7cw2227kw7ntqiqqWQlgU7TIa2mM6vp/3Pqcx0VA10J2jogpwBRJdwNzyMYhNpESWmpFtMndZcsiYrmkwcCRwFnA8cB5VRz2k6qqVMFzAeMi4qTcFZKGVFFOo7RiaUtatcqSwvq14o0XOnP82YsAePGJrux72Me0abf5LVo0rzVXnN6fC697n767bP6nEAF//PEO9Bu4nuPOXFwa77/7OkZP2XzdzcnDB3H9P2b4rKQ6tHRRa/bedxWTX+vCkC98zIL3si//jp03sX5dCzZtbMGIExcz5fXOrFndOH60bC2NtZsoH1v7p9hKYLmkAyPiReA7QEnrYRXQGSD10Q+LiOfTuiHA++nxe2RdM6OB/yaborbE8HQp+PvACcAtqatnQ0Q8JGkGcE9ErJI0T9KxEfGopLZkVwRWZ4vygVeBP0saEBGzJHUkm7nwHWAnSbtExH+AkyottZFY9lFrfn/eDhQXi+JiOOirK9jv8KyF8K/HunH8OR+V2f7eaz7DquUtueGSzWcj3fDUTKa93pFn/9ad/ruv5fuHZWe3nHrJAoYfuuUYhNWei6+dxd77raJLt03c/e83uedPfbn2kv6cden7tGwVbFjfgmt/2h/IBp9//IfZEOL9me255if9qyxn7Oie9fWy6kcj7ibKh7JejjooOLur0OMRsWd6fgHQCXgUuAnoQNbNc2r6VX8c8CtgLXAg8ABZ19Nasl/x50XEBEm9yOYMaQ88RXbZdydJBwNXkCWYAWTzjPwAKBlXKEnvl0TEPyQNBG4GegAbya4k3AG4ICK+kvM6VldVfkQUSzqE7GYZJedc/jwixqSuqz8Ba4AXgV1yy67IsMHt4vWx/araxBqYEf33re8qWAFeXf8PPi5e+qm+1bvttl0cMuob1W8IPHzAjROrm121oamzxGA148TQ+DgxNC61lRgOvq3CWSm28OgX/9LoEoNH9czMClRyo56myonBzKxAgdhU7MFnMzPL0Vinu8iHE4OZWaHCXUlmZpbDYwxmZrYFJwYzMysViCIPPpuZWS4PPpuZWanw4LOZmZUXTTgxNN1OMjOzOlN792OQ1E/Sc+kWxtMknZfil0uaL2lSWo7O2ecSSbPSTcOOzImPSLFZki7OifeX9FqKPyCpDVVwYjAzq4EI5bXkYRPw44gYRHZ/l7MlDUrrrkk3LxsSEU8CpHUnAnsAI4C/pDtVtgT+DBxFdo+Yk3LK+W0qawCwHDitqgo5MZiZFSgCioqV11J9WbEwIt5Ij1cBb5NN3V+ZY4D70y2P5wCzyO4uORyYlW4mtgG4HzhGkoBDgL+l/e8Ejq2qTk4MZmY1UIzyWoAeJbfuTcsZlZWZblfwOeC1FDpH0mRJoyR1S7E+wNyc3ealWGXxbYEVEbGpXLxSTgxmZgUKCupKWlJy69603FJRmekGZQ8B/xsRHwM3kt2TZgjZPen/sFVeHD4rycysBmr3Dm6SWpMlhXsj4mGAiPgoZ/2twOPp6Xwg96YtfVOMSuJLga6SWqVWQ+72FXKLwcysBiLyW6qTxgBuA96OiD/mxHvnbPY1YGp6PAY4UVLbdKvhgcDrwHhgYDoDqQ3ZAPWYyO7G9hxQcsu5kWR3wayUWwxmZjVQi9cxHAB8B5giaVKK/ZTsrKIhZD1X7wFnZseNaZJGA9PJzmg6OyKKACSdA4wlu4f9qIiYlsr7CXC/pF8Cb5Iloko5MZiZFSg7K6l2Olwi4iWocH6NJ6vY5yrgqgriT1a0X0TMJjtrKS9ODGZmNZBPN1Fj5cRgZlYDTXlKDCcGM7MCBXlf1dwoOTGYmdVAE+5JcmIwMytYQOQx3UVj5cRgZlYD7koyM7MymuVZSZKup4putIj4YZ3UyMysgSuZK6mpqqrFMGGr1cLMrDEJoDkmhoi4M/e5pA4Rsabuq2Rm1vA15a6kaq/plrS/pOnAO+n5YEl/qfOamZk1WCKK81sao3wm+/gTcCTZ1K1ExFvAQXVYJzOzhi/yXBqhvM5Kioi52cywpYrqpjpmZo1ANN/B5xJzJX0BiHQzifPI7klqZtZ8NdLWQD7y6Uo6Czib7B6hC8huM3d2HdbJzKwRUJ5L41NtiyEilgDf2gp1MTNrPIrruwJ1J5+zknaW9HdJiyUtkvSYpJ23RuXMzBqkkusY8lkaoXy6kv4PGA30BrYHHgTuq8tKmZk1dLV1z+eGKJ/E0CEi7o6ITWm5B2hX1xUzM2vQmuPpqpK6p4f/kHQxcD/ZyzyBKu5FambWLDTSbqJ8VDX4PJEsEZS8+jNz1gVwSV1VysysoVMjbQ3ko9KupIjoHxE7p7/lFw8+m1nzFYLiPJdqSOon6TlJ0yVNk3ReineXNE7Su+lvtxSXpOskzZI0WdLQnLJGpu3flTQyJ76PpClpn+tU7orl8vIZY0DSnpKOl3RyyZLPfmZmTVbtjTFsAn4cEYOA/YCzJQ0CLgaejYiBwLPpOcBRwMC0nAHcCKXd/5cB+wLDgctKkkna5ns5+42oqkL5nK56GXB9Wr4E/A7477xerplZU1VLiSEiFkbEG+nxKrKZJfoAxwAls1zfCRybHh8D3BWZV4GuknqTzWk3LiKWRcRyYBwwIq3rEhGvRkQAd+WUVaF8WgzfAA4FPoyIU4HBwDZ57Gdm1nTlnxh6SJqQs5xRWZGSdgI+B7wG9IqIhWnVh0Cv9LgPMDdnt3kpVlV8XgXxSuUzV9LaiCiWtElSF2AR0C+P/czMmqbCbtSzJCKGVbeRpE7AQ8D/RsTHucMAERHS1hvuzqfFMEFSV+BWsjOV3gBeqctKmZk1dIr8lrzKyiYofQi4NyIeTuGPUjcQ6e+iFJ9P2R/nfVOsqnjfCuKVqjYxRMQPImJFRNwEHA6MTF1KZmbNVy2NMaQzhG4D3o6IP+asGgOUnFk0EngsJ35yOjtpP2Bl6nIaCxwhqVsadD4CGJvWfSxpv3Ssk3PKqlBVF7gNrWpdyWCJmVlzVIsdOwcA3wGmSJqUYj8FfgOMlnQa8D5wfFr3JHA0MAtYA5wKEBHLJF0JjE/bXRERy9LjHwB3AO2Bf6SlUlWNMfyhinUBHFJVwVYzMyd34Mjth9R3NawALQdsX99VsALog9a1U1AtXfkcES9R+fzch1awfVDJrQ8iYhQwqoL4BGDPfOtUaWKIiC/lW4iZWbPSiOdBykdet/Y0M7NynBjMzCyXmvCNepwYzMxqogm3GPKZEkOSvi3p0vR8B0nD675qZmYNU77XMDTWGVjzucDtL8D+wEnp+Srgz3VWIzOzxqAJ39ozn66kfSNiqKQ3ASJiuaQ2dVwvM7OGrZG2BvKRT2LYKKkl6W2Q1BNowsMuZmbVa6zdRPnIJzFcBzwCbCfpKrLZVn9ep7UyM2vIopmflRQR90qaSHYFnoBjI+LtOq+ZmVlD1pxbDJJ2IJuP4++5sYj4oC4rZmbWoDXnxAA8QfYWCGgH9AdmAHvUYb3MzBq0Zj3GEBF75T5Ps67+oM5qZGZm9argK58j4g1J+9ZFZczMGo3m3GKQ9KOcpy2AocCCOquRmVlD19zPSgI65zzeRDbm8FDdVMfMrJFori2GdGFb54i4YCvVx8yswRPNdPBZUquI2CTpgK1ZITOzRqE5JgbgdbLxhEmSxgAPAp+UrIyIh+u4bmZmDVMjnjk1H/mMMbQDlpLd47nkeoYAnBjMrPlqpoPP26UzkqayOSGUaMK50sysek25xVDV/RhaAp3S0jnnccliZtZ8RZ5LNSSNkrRI0tSc2OWS5kualJajc9ZdImmWpBmSjsyJj0ixWZIuzon3l/Raij+Qz20TqmoxLIyIK6p/WWZmzUyeX/p5ugO4AbirXPyaiPh9bkDSIOBEsimJtgeekbRrWv1n4HBgHjBe0piImA78NpV1v6SbgNOAG6uqUFUthsZ56yEzs62gtm7tGREvAMvyPOwxwP0RsT4i5gCzgOFpmRURsyNiA3A/cIwkkY0P/y3tfydwbHUHqSoxHJpnRc3Mmp/8u5J6SJqQs5yR5xHOkTQ5dTV1S7E+wNycbealWGXxbYEVEbGpXLxKlXYlRUS+GczMrNkpYEqMJRExrMDibwSuJEstVwJ/AL5bYBk1VvAkemZmzV7tjjFsWXzERyWPJd0KPJ6ezgf65WzaN8WoJL4U6FpywXK57StVVVeSmZlVQAUsNSpf6p3z9Gtklw0AjAFOlNRWUn9gINnFyOOBgekMpDZkA9RjIiKA58huyQwwEnisuuO7xWBmVhO11GKQdB9wMNlYxDzgMuBgSUPSUd4DzgSIiGmSRgPTySY1PTsiilI55wBjyS41GBUR09IhfgLcL+mXwJvAbdXVyYnBzKwGausCt4g4qYJwpV/eEXEVcFUF8SeBJyuIzyY7aylvTgxmZjXRhK98dmIwMyuUb9RjZmZbcIvBzMxyNeVJ9JwYzMxqwonBzMxyucVgZmabBc32Rj1mZlYB4RaDmZmV58RgZma5FE03MzgxmJkVqo5nV61vTgxmZjXgMQYzMyvDU2KYmVlZbjGYmVmpcFeSmZmV58RgZmYlfIGbmZltQcVNNzM4MZiZFcrXMZhV7Ed//IB9D1vFiiWtOPOQzwLQuesmfnrT+/Tqu4GP5rXhqjN3ZPXKVnToXMRPbviA7bbfQMtWwd9u2o6nH+gOwFX3zma3oZ8w7fWOXDpy5/p8Sc1CixbBtbc8z9Il7bj84v353fUv0r7DRgC6dtvAzLe7cuXP9mOvIYu59Fev8eHCDgD8+4Xtue/O3QDo2GkD5100iR37f0wAf/rNUN6Z1r2+XlK98OmqjYCky4HVEfH7WixzGHByRPzw05ZfF/Wrb08/0J0xt/fgwmvnlsaOP2cRb77UidE39OL4cz7ihHMWcdtV2/Pfpyzhg5ltuWxkf7bpvonbXnyHfz7clU0bW/DgjT1p235bvvztpfX4apqPY77xH+a+35kOHbNkcNG5B5au+9mVr/HKS71Ln0+bvC2XX7z/FmWc+cMpTHxtO3516XBatSqmbbtNdV/xhqYJtxha1HcFGipJrSJiQkT8sL7r0lBNfa0Tq5aX/W2x/5Ef88zo7JfjM6O7s/+IjwGIgPYdi4GgXcciVq1oSdEmATDppc6sXd1yq9a9udq251o+v/+HjH1ixy3Wte+wkb2HLuGVF3tXsOdmHTpuZM/BS0vL2LSpBZ+sblMn9W3IFPkt1ZYjjZK0SNLUnFh3SeMkvZv+dktxSbpO0ixJkyUNzdlnZNr+XUkjc+L7SJqS9rlOkqqrU4NODJI6SnpC0luSpko6QdJ7knqk9cMkPZ+zy2BJr6Q35ntpm96SXpA0KZVxYIqPkPRGKvvZFLtc0t2SXgbulnSwpMerKj/td6Gk8emD+kVO/GeSZkp6Cfhsnb1RDUi3HhtZtqg1AMsWtaJbj+xX6Zjbe7DDwHX835vTufmfM7nx0j5EVPvv02rZmedOYdSNe1JcQTfI/gcu5K2JPVm7pnVpbLc9lnHDqH9yxe/+zQ47ZUn+M70/YeWKNpx/yRtc/9fnOO+iN5tfiyHIfu3ks1TvDmBEudjFwLMRMRB4Nj0HOAoYmJYzgBshSyTAZcC+wHDgspJkkrb5Xs5+5Y+1hQadGMhewIKIGBwRewJPVbP93sAhwP7ApZK2B/4HGBsRQ4DBwCRJPYFbgeMiYjDwzZwyBgGHRcRJ+ZQv6QiyN3s4MATYR9JBkvYBTkyxo4HPV1ZpSWdImiBpwkbWV/MSGxOVfvnvc/Aq/jOtPf/zuUH84PBdOfuq+XToVFTP9Wtehu//ISuWt2XWzK4Vrj/40Hn869k+pc9nzezKKccfyTnfPYQxD+/M//vVawC0bBkMGLiSJx/tz7mnf4l161py/Ldmbo2X0KCoOL+lOhHxArCsXPgY4M70+E7g2Jz4XZF5FegqqTdwJDAuIpZFxHJgHDAiresSEa9GRAB35ZRVqYaeGKYAh0v6raQDI2JlNds/FhFrI2IJ8BzZl/V44NTUx79XRKwC9gNeiIg5ABGR+6GMiYi1BZR/RFreBN4AdiNLFAcCj0TEmoj4GBhTWaUj4paIGBYRw1rTtpqX2LAtX9Ka7ttlrYTu221kxdKsq+mIE5bx8pPbAGLBe2358IM29BvQlJJgwzdor6Xsd8BCbn9gLD+5bAJ7D13CBT+fAECXbdaz6+7Lef2Vz5Ruv3ZNa9atzT6/Ca9+hlYti+myzXqWLG7PksXtmPF21mX40vPbs8uu1f3XbFpKrmPIsyupR8kPv7SckcchekXEwvT4Q6BXetwHmJuz3bwUqyo+r4J4lRr04HNEzEx9aEcDv0xdPpvYnNDald9lyyLiBUkHAV8G7pD0R2B5FYf9pKoqVfBcwK8j4ubcFZL+t4pymqxXn+7CYccvY/QNvTjs+GW8MrYLAIvnt2HIgauZ+nonuvbYSN9d1rHwg+bXL12f7rhlD+64ZQ8A9hqymONOnMXvfzkMgC/+1wJef+UzbNyweaynW/d1LF/WFhC77r4ctYCPV7YBxOJFHejTbxXz53ZmyD6L+eC9zvXwiupR/t1EAEsiYljNDxUhbd3L6Rp0YkhdQcsi4h5JK4DTgfeAfYB/AMeV2+UYSb8GOgIHAxdL2hGYFxG3SmoLDAWuAv4iqX9EzJHUvVyroTJblA+sBa6UdG9ErJbUB9gIvECWiH5N9j5/Fbi54mIbp4v/8j5777+abbpv4p4J07n7D7144Ibt+NlN7zPixGUsmp+drgpw7596ccGfPuCmZ2cgwW1Xbc/Hy7J/fn94ZBZ9B6yjfYdi7pkwnWt+3JeJ/+pSny+t2Tno0Hk8eO+uZWIHHLyALx8zh6IisWF9S377i2Fkv4Pgpmv34qL/N5FWrYv5cEEHrvn10ApKbdrq+Kv6I0m9I2Jh6g5alOLzgX452/VNsflk30m58edTvG8F21epQScGYC/gaknFZF+23wfaA7dJupLsheeaTNbF0wO4MiIWpNH5CyVtBFaTnX66ODXnHpbUguxNPzyP+mxRPrBA0u7AK2mwfzXw7Yh4Q9IDwFup/PE1fhcaqN/8YMszWwAuPmGXLWLLPmrNT0/aMg7w468NqNV6WfWmTOrJlEk9S59ffN6BW2zz+MM78/jDFV9XMntWV8474+C6ql7jULeJYQwwEvhN+vtYTvwcSfeTDTSvTMljLPCrnAHnI4BLImKZpI8l7Qe8BpwMXF/dwRVN+PZ0jVEXdY99dWh9V8MK0HJA//qughXglQ/uYuW6Dz/VKXGdu/aNoQeel9e2Lzx+0cSqupIk3Uf2a78H8BHZ2UWPAqOBHYD3gePTl7yAG8hOzFkDnBoRE1I53wV+moq9KiJuT/FhZGc+tSfraTk3qvnib+gtBjOzhieAotr5UV3JGZAAW/xCTF/oZ1dSzihgVAXxCcCehdTJicHMrAY8u6qZmZXVhLvhnRjMzGrALQYzM9vM026bmVkuAaqlweeGyInBzKwG5DEGMzMr5a4kMzMrq6C5khodJwYzsxrwWUlmZlaWWwxmZlYqfFaSmZmV13TzghODmVlN+HRVMzMry4nBzMxKBVBc35WoO04MZmYFEuGuJDMzK6e46TYZnBjMzArlriQzMyvPXUlmZlaWE4OZmW3WtCfRa1HfFTAza3QCKIr8ljxIek/SFEmTJE1Ise6Sxkl6N/3tluKSdJ2kWZImSxqaU87ItP27kkbW9OU5MZiZ1YAi8loK8KWIGBIRw9Lzi4FnI2Ig8Gx6DnAUMDAtZwA3QpZIgMuAfYHhwGUlyaRQTgxmZjURkd9Sc8cAd6bHdwLH5sTvisyrQFdJvYEjgXERsSwilgPjgBE1ObATg5lZoQIojvwW6CFpQs5yRiUlPi1pYs76XhGxMD3+EOiVHvcB5ubsOy/FKosXzIPPZmYFK6g1sCSne6gyX4yI+ZK2A8ZJeqfM0SJC2nq3BnKLwcysJmqxKyki5qe/i4BHyMYIPkpdRKS/i9Lm84F+Obv3TbHK4gVzYjAzK1QARcX5LdWQ1FFS55LHwBHAVGAMUHJm0UjgsfR4DHByOjtpP2Bl6nIaCxwhqVsadD4ixQrmriQzs4IFRK3NidELeEQSZN/J/xcRT0kaD4yWdBrwPnB82v5J4GhgFrAGOBUgIpZJuhIYn7a7IiKW1aRCTgxmZjVRSxe4RcRsYHAF8aXAoRXEAzi7krJGAaM+bZ2cGMzMClVyVlIT5cRgZlYTTXhKDCcGM7OacGIwM7NSEVBUVN+1qDNODGZmNeEWg5mZleHEYGZmm4XPSjIzsxwBUXsXuDU4TgxmZjWRx3QXjZUTg5lZoSKg2InBzMxyefDZzMxyhVsMZma22ae+bWeD5sRgZlYoT6JnZma5AghPiWFmZqWiVm/U0+A4MZiZ1UC4K8nMzMpowi0GRRMeWW+MJC0mu79rU9MDWFLflbCCNNXPbMeI6PlpCpD0FNn7k48lETHi0xxva3NisK1C0oSIGFbf9bD8+TNrvlrUdwXMzKxhcWIwM7MynBhsa7mlvitgBfNn1kx5jMHMzMpwi8HMzMpwYjAzszKcGKyUpFMkbZ/z/CuS3pT0lqTpks6sZv+DJT1eB/X6d22UX1f1a6gkXS7pglouc5ik62qj/Lqon9UOX/lsuU4BpgILJLUmG3wcHhHzJLUFdtqalZHUKiI2RcQXtuZxrWLp85gATKjvuljdcouhCZO0k6S3Jd0qaZqkpyW1lzRE0quSJkt6RFI3Sd8AhgH3SpoEbEf2w2EpQESsj4gZqdw70vYlx1mdc9gukp6QNEPSTZJaSGqZ9pkqaYqk89N+AyQ9k1okb0jaJf2qf1HSGGB6PuWnbY6Q9Eoq50FJnVJ8hKR3JL0BfL1u3umtR1LH9PrfSu/nCZLek9QjrR8m6fmcXQan9+VdSd9L2/SW9IKkSamMA1N8RHr/3pL0bIpdLuluSS8Dd1fQ6tqi/LTfhZLGp39jv8iJ/0zSTEkvAZ+tszfKPp2I8NJEF7Jf+JuAIen5aODbwGTgv1LsCuBP6fHzwLCc/f8KLALuA74FtEjxO4Bv5Gy3Ov09GFgH7Ay0BMYB3wD2AcblbN81/X0N+Fp63A7okMr4BOhfQPk9gBeAjmm7nwCXpjLnAgMBpdf/eH1/Lp/yMz0OuDXn+TbAe0CP9HwY8Hx6fDnwFtA+vUdzge2BHwM/S9u0BDoDPdP6/inePaeMiUD7nM/g8WrKP4KstSmyH5+PAwelfwdT0ufcBZgFXFDf76mXLRd3JTV9cyJiUno8EdiF7Iv5Xyl2J/BgRTtGxOmS9gIOAy4ADifrbqrK6xExG0DSfcAXgWeBnSVdDzwBPC2pM9AnIh5Jx1qX9ikpY04B5a8DBgEvp/3bAK8Au6XX/27a/h7gjGrq39BNAf4g6bdkX9AvptdcmcciYi2wVtJzwHBgPDAqdRc+GhGTJB0MvFDyvkfEspwyxqQy8i3/i2TJ4c20TSey5NwZeCQi1gCkVqE1QO5KavrW5zwuAroWsnNETImIa8iSwnEpvIn0byd15bTJ3WXLImI5MJisRXIWWUukKp9UVaUKnousRTIkLYMi4rRqjtEoRcRMYChZgvilpEvJ+TzIWklldtmyiHiB7Bf8fOAOSSdXc9iafB6/zvk8BkTEbdUcwxoQJ4bmZyWwvKRfGfgOUNJ6WEX2qw5JndKvyBJD2Dzr63tk3QIA/w20ztluuKT+KWGcALyU+r9bRMRDwM+BoRGxCpgn6dh0vLaSOuRR/y3KB14FDpA0IJXVUdKuwDvATpJ2SfuelEf5DZqys8bWRMQ9wNVkSeI9Nn8ex5Xb5RhJ7SRtS9YNNF7SjsBHEXErWZIeSvYeHiSpfzpO9zyrtEX5wFjguznjPH0kbUfW3XdsGufqDHy14DfAtgp3JTVPI4Gb0hfxbODUFL8jxdcCBwIXSboZWEv2q/GUtN2twGOS3gKeouwvyvHADcAA4DngEWAv4PaSgWLgkvT3O8DNkq4ANgLfzKPuW5QfEcWSTgHuU3b2FMDPI2KmpDOAJyStAV4kJb5GbC/gaknFZO/Z98n6+G+TdCVZqyzXZLL3qQdwZUQskDQSuFDSRmA1cHJELE7v1cPpc1pE1kqszhblk53VtjvwSurmWg18OyLekPQA2bjEIrLP0hogT4lhZmZluCvJzMzKcGIwM7MynBjMzKwMJwYzMyvDicHMzMpwYrBGRVJRzhw/D+Z57UNlZZXO+STpr5IGVbHtwZIKnswvdx6jfOLltlld1foKtvdspVYrnBissVmbrqbdE9hAdiV1KUk1ujYnIk6PiOlVbHIw4FlerVlwYrDG7EVggMrNyKpsNterc2b3PBNAmRuUzcz6DNkMsqR1z0salh6XmWVU0k5kCej81Fo5UFJPSQ+lY4yXdEDad1tls9hOk/RXsukhqiTpUUkT0z5nlFt3TYo/K6lniu0i6am0z4uSdquVd9Ms8ZXP1iillsFRZFdeQzatw54RMSd9ua6MiM+nK6FflvQ08DmyqZ4HAb3IpvUeVa7cnmRXdh+UyuoeEcsk3UQ2y+vv03b/B1wTES9J2oFsGojdgcuAlyLiCklfBvKZs+m76RjtyaaseCgilgIdgQkRcb6yOZEuA84hm7n0rIh4V9K+wF+AQ2rwNppVyInBGpv2yu4XAVmL4TayLp7cGVmPAPbW5ntGbEM2u+dBwH0RUUQ2bcM/Kyh/PyqfZTTXYcAgbZ7ZtEuaG+gg0n0fIuIJScvzeE0/lPS19LhfqutSoBh4IMXvIZuuolN6vQ/mHLstZrXIicEam7URMSQ3kL4gc+drEnBuRIwtt93RtViPFsB+JdOFl6tL3pRNVHgYsH9ErFF2k53yM6SWiHTcFeXfA7Pa5DEGa4rGAt9Xdr8BJO0qqSPZ7J4npDGI3sCXKti3sllGS2eeTZ4Gzi15ImlIevgC8D8pdhTQrZq6bgMsT0lhN7IWS4kWZDciIpX5UkR8DMyR9M10DEkaXM0xzArixGBN0V/Jxg/ekDQVuJmsdfwI8G5adxfZzXzKiIjFZDfzeVjZ7LElXTl/B75WMvgM/BAYlga3p7P57KhfkCWWaWRdSh9UU9engFaS3gZ+Q5aYSnxCNs34VLIxhCtS/FvAaal+04Bj8nhPzPLm2VXNzKwMtxjMzKwMJwYzMyvDicHMzMpwYjAzszKcGMzMrAwnBjMzK8OJwczMyvj/tEA8U6qmGz4AAAAASUVORK5CYII=\n",
       "datasetInfos": [],
       "metadata": {},
       "removedWidgets": [],
       "type": "image"
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import ConfusionMatrixDisplay, confusion_matrix\n",
    "\n",
    "predicted_probas = prod_model.predict(X)\n",
    "predicted_probas[predicted_probas > 0.5] = 1\n",
    "predicted_probas[predicted_probas <= 0.5] = 0\n",
    "matrix = confusion_matrix(predicted_probas, y)\n",
    "fig = ConfusionMatrixDisplay(matrix, display_labels=[\"notSubscribed\", \"subscribed\"])\n",
    "fig.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "786ee31c-9b36-4f94-a8b8-1c5d8aca6ca0",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "## 1.4 Saving the final model name to configuration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "3b90e437-2bce-439c-9319-8dd1da72a130",
     "showTitle": false,
     "title": ""
    }
   },
   "source": [
    "Now that we got everything working, we just need to save the updated `model_name` variable in the original configuration file, so we can refer to it in the following weekly assignments. To do that, execute the code below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {
      "byteLimit": 2048000,
      "rowLimit": 10000
     },
     "inputWidgets": {},
     "nuid": "874d109e-a06b-4b4b-bd00-a28e3f5f3be2",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": [
    "config.set(\"Cloud\", \"model_name\", model_name)\n",
    "\n",
    "with open(config_path, \"w\") as configfile:\n",
    "    config.write(configfile)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "application/vnd.databricks.v1+cell": {
     "cellMetadata": {},
     "inputWidgets": {},
     "nuid": "b43c0beb-a10e-495f-a71f-b58b5133553f",
     "showTitle": false,
     "title": ""
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "application/vnd.databricks.v1+notebook": {
   "dashboards": [],
   "language": "python",
   "notebookMetadata": {
    "pythonIndentUnit": 2,
    "widgetLayout": []
   },
   "notebookName": "Week3Notebook",
   "notebookOrigID": 3680348664249912,
   "widgets": {}
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}