restaurants in chicago.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Way to a City's Heart...\n",
    "\n",
    "How much can we tell about an area in a city based on the restaurants it has? That is the goal of this project.\n",
    "\n",
    "## Introduction\n",
    "\n",
    "People of varying backgrounds, economic statuses, and tastes will shape the restaurant options located near them. So, it would seem likely that the different neighborhoods in a city would be composed of different sorts of food establishments, as well as different cities. This project investigates how closely established city neighborhoods align with clusters determined by restaurants and what sorts of restaurants are present in the various clusters. It also compares the restaurant compositions of different cities.\n",
    "\n",
    "## Using the Notebook\n",
    "\n",
    "The first few cells in this notebook gather the dataset for analysis. They should just be run once to prevent needing to constantly make API calls and the data should be saved once gathered. To run the code, you will need your own Foursquare API credentials, and you will have to edit the codecell that loads mine from a file. \n",
    "\n",
    "The second half of the notebook is dedicated to analyzing the dataset once it is saved. Since I'm focused on Chicago, I place a few markers on the airports in that city for all my maps, but obviously they should be removed for other cities.\n",
    "\n",
    "## Data\n",
    "\n",
    "Foursquare's API has a search features that allows the user to search for venues in a city which belong to certain categories. Among these categories are Food and Bars. Each of these categories is subdivided into smaller subcategories such as Afghan Restaurant, Burger Joint, etc. Given a city, we can consider all of its restaurants and the corresponding list of which categories it belongs to. Using this data, we can hope to understand the \"culinary neighborhoods\" of a city. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import requests\n",
    "import json\n",
    "import folium\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from matplotlib import cm\n",
    "from matplotlib.colors import to_hex\n",
    "\n",
    "from math import sqrt\n",
    "\n",
    "from collections import defaultdict\n",
    "\n",
    "from sklearn.cluster import DBSCAN\n",
    "from sklearn.cluster import KMeans\n",
    "from sklearn.preprocessing import OneHotEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get foursquare creds\n",
    "\n",
    "with open('foursquare_api_key.json', 'r') as f:\n",
    "    api_info = json.loads(f.read())\n",
    "    client_id = api_info['client_id']\n",
    "    client_secret = api_info['client_secret']\n",
    "    v = api_info['v']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set up variables for loading in data\n",
    "\n",
    "loc_list = []\n",
    "limit = 50\n",
    "offset = 0\n",
    "radius = 10000\n",
    "cats = '4d4b7105d754a06374d81259,4bf58dd8d48988d116941735'\n",
    "city = 'Chicago'\n",
    "restaurants_id_set = set()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# helper function to parse the venue data\n",
    "\n",
    "def get_restaurant_info(group_item):\n",
    "    rest_id = group_item['id']\n",
    "    name = group_item['name']\n",
    "    lat = group_item['location']['lat']\n",
    "    lng = group_item['location']['lng']\n",
    "    cat = group_item['categories'][0]['name']\n",
    "    return (rest_id, name, lat, lng, cat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# populate location list\n",
    "\n",
    "while len(loc_list) == offset:\n",
    "    # pull api data for locations\n",
    "    url = 'https://api.foursquare.com/v2/venues/explore?client_id={}&client_secret={}&v={}&near={}&radius={}&categoryId={}&offset={}&limit={}'.format(\n",
    "        client_id,\n",
    "        client_secret,\n",
    "        v,\n",
    "        city,\n",
    "        radius,\n",
    "        cats,\n",
    "        offset,\n",
    "        limit\n",
    "    )\n",
    "    response = requests.get(url)\n",
    "    result = json.loads(response.content)['response']\n",
    "    total_results = int(result['totalResults'])\n",
    "    for item in result['groups'][0]['items']:\n",
    "        info = get_restaurant_info(item['venue'])\n",
    "        loc_list.append(info)\n",
    "        restaurants_id_set.add(info[0])\n",
    "    offset += limit\n",
    "print('Found {} locations/ {} unique locations.'.format(len(loc_list), len(restaurants_id_set)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use the locations found to come up with a place to center the map\n",
    "\n",
    "def getAvgCoords(df):\n",
    "    return df['Lat'].mean(), df['Lng'].mean()\n",
    "\n",
    "def getAvgCoords_list(loc_list):\n",
    "    lat = 0\n",
    "    lng = 0\n",
    "    for item in loc_list:\n",
    "        lat += item[2]\n",
    "        lng += item[3]\n",
    "    return lat/len(loc_list), lng/len(loc_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create map\n",
    "\n",
    "my_map = folium.Map(location = getAvgCoords_list(loc_list), zoom_start = 11, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "\n",
    "for entry in loc_list:\n",
    "    folium.CircleMarker(\n",
    "        location=[entry[2],entry[3]], \n",
    "        popup = entry[1], \n",
    "        fill = True, \n",
    "        radius=3, \n",
    "        weight=2,\n",
    "        color='green', \n",
    "        fill_color='white',\n",
    "        fill_opacity=0.8).add_to(my_map)\n",
    "my_map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need more data! To get it, I propose the following: Use DBScan to locate isolated points and boundary points. Then pick a random isolated point and boundary point at random and query around around it. If no isolate points or boundary points are found, decrease the DBScan radius slightly and continue. Stop when either \n",
    "1. The DBScan radius hits a given threshhold\n",
    "2. Some maximum number of points has been added."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set up initial search parameters\n",
    "\n",
    "search_radius = 1000\n",
    "eps = 0.003"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# form a dataframe from the initial location data collected\n",
    "\n",
    "def create_df(loc_list, step):\n",
    "    new_restaurant_data = pd.DataFrame(loc_list)\n",
    "    new_restaurant_data.set_index(0)\n",
    "    new_restaurant_data.rename( columns = {1:'Name', 2:'Lat', 3:'Lng', 4:'Category'},inplace=True)\n",
    "    new_restaurant_data['step'] = step\n",
    "    new_restaurant_data.index = new_restaurant_data[0]\n",
    "    new_restaurant_data.index.rename('id', inplace=True)\n",
    "    new_restaurant_data.drop(0, axis=1,inplace=True)\n",
    "    return new_restaurant_data\n",
    "\n",
    "restaurant_data = create_df(loc_list,0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# run dbscan\n",
    "\n",
    "def run_dbscan(eps, data):\n",
    "    scanner = DBSCAN(eps=eps)\n",
    "    data['label'] = scanner.fit_predict(data[['Lat','Lng']])\n",
    "    core_indices = data.index[scanner.core_sample_indices_]\n",
    "    data.loc[core_indices,'isCore'] = True\n",
    "    data['isCore'].fillna(False, inplace=True)\n",
    "    return data\n",
    "\n",
    "restaurant_data = run_dbscan(0.003, restaurant_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# visualize results...\n",
    "# first set up functions to color the points\n",
    "\n",
    "\n",
    "def get_fill_color(cmap, label):\n",
    "    if label == -1:\n",
    "        return '#444444'\n",
    "    else:\n",
    "        return to_hex(cmap(label))\n",
    "    \n",
    "def get_boundary_color(cmap, is_core):\n",
    "    if is_core:\n",
    "        return '#000000'\n",
    "    else:\n",
    "        return '#666666'\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create map\n",
    "\n",
    "cmap = cm.get_cmap(name='cool', lut=restaurant_data['label'].max())\n",
    "my_map = folium.Map(location = getAvgCoords(restaurant_data), zoom_start = 11, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference but delete them if you use a new city\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "\n",
    "for i in restaurant_data.index:\n",
    "    entry = restaurant_data.loc[i,:]\n",
    "    folium.CircleMarker(\n",
    "        location=[entry['Lat'],entry['Lng']], \n",
    "        popup = entry['Name'], \n",
    "        fill = True, \n",
    "        radius=3, \n",
    "        weight=2,\n",
    "        color=get_boundary_color(cmap, entry['isCore']), \n",
    "        fill_color=get_fill_color(cmap, entry['label']),\n",
    "        fill_opacity=0.8).add_to(my_map)\n",
    "my_map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# this function will take the venues in data and add them to loc_list if they do not appear in restaurants_id_set\n",
    "\n",
    "def make_queries(data, seach_radius):\n",
    "    for venue_id in data.index:\n",
    "    \n",
    "        venue = data.loc[venue_id]\n",
    "        url = 'https://api.foursquare.com/v2/venues/search?client_id={}&client_secret={}&v={}&ll={},{}&radius={}&categoryId={}&limit={}'.format(\n",
    "            client_id,\n",
    "            client_secret,\n",
    "            v,\n",
    "            venue['Lat'],\n",
    "            venue['Lng'],\n",
    "            str(search_radius),\n",
    "            cats,\n",
    "            limit\n",
    "        )\n",
    "        try:\n",
    "            response = requests.get(url)\n",
    "            result = json.loads(response.content)['response']\n",
    "        except:\n",
    "            print('At the time of failure...')\n",
    "            print(response)\n",
    "            print(response.content)\n",
    "        if 'venues' in result.keys():\n",
    "            for item in result['venues']:\n",
    "                info = get_restaurant_info(item)\n",
    "                if not info[0] in restaurants_id_set:\n",
    "                    loc_list.append(info)\n",
    "                    restaurants_id_set.add(info[0])\n",
    "        elif 'venue' in result.keys():\n",
    "            into = get_restaurant_info(result['venue'])\n",
    "            if not info[0] in restaurants_id_set:\n",
    "                    loc_list.append(info)\n",
    "                    restaurants_id_set.add(info[0])\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# restaurants_id_set and restaurant_data should be initialized with first data\n",
    "# you then run this code cell repeatedly until you get enough restaurants and bars\n",
    "# up the number in the loop if you want to run more at a time\n",
    "# sometimes a 502 error arises, but that's probably ok\n",
    "\n",
    "for step in range(1):\n",
    "    # 1 query around the boundary points and outliers\n",
    "    # 2 adjust parameters as follows:\n",
    "    # 2a if there are few boundary points and outliers, decrease eps by about 10%\n",
    "    # 2b if there are no new restaurants, increase search radius by about 10%\n",
    "    # 3 make a new df with results for analysis\n",
    "    # 4 run DBscan to get outliers and boundary points\n",
    "    \n",
    "    \n",
    "    outliers = restaurant_data[restaurant_data['label'] == -1]\n",
    "    boundaries = restaurant_data[(restaurant_data['label'] != -1) & (restaurant_data['isCore'] == False)]\n",
    "    \n",
    "    loc_list = []\n",
    "    \n",
    "    make_queries(outliers, search_radius)\n",
    "    make_queries(boundaries, search_radius)\n",
    "    \n",
    "    \n",
    "    \n",
    "    if len(loc_list) < 10:\n",
    "        seach_radius = int(1.1*search_radius)\n",
    "    \n",
    "    if len(outliers.index) + len(boundaries.index) < 20:\n",
    "        eps /= 1.1\n",
    "    if len(loc_list) > 0:  \n",
    "        new_restaurant_data = create_df(loc_list, step)\n",
    "\n",
    "        restaurant_data = pd.concat([restaurant_data[['Name','Lat','Lng','Category','step']],new_restaurant_data])\n",
    "\n",
    "        restaurant_data = run_dbscan(eps, restaurant_data)\n",
    "\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create map\n",
    "\n",
    "cmap = cm.get_cmap(name='cool', lut=restaurant_data['label'].max())\n",
    "my_map = folium.Map(location = getAvgCoords(restaurant_data), zoom_start = 11, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "\n",
    "for i in restaurant_data.index:\n",
    "    entry = restaurant_data.loc[i,:]\n",
    "    folium.CircleMarker(\n",
    "        location=[entry['Lat'],entry['Lng']], \n",
    "        popup = entry['Name'], \n",
    "        fill = True, \n",
    "        radius=3, \n",
    "        weight=2,\n",
    "        color=get_boundary_color(cmap, entry['isCore']), \n",
    "        fill_color=get_fill_color(cmap, entry['label']),\n",
    "        fill_opacity=0.8).add_to(my_map)\n",
    "my_map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# you've worked hard for your data, so save it\n",
    "\n",
    "restaurant_data.to_csv('restaurant_data')\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analysis\n",
    "\n",
    "Now it's time to cluster and explore the data we've gathered. I'll start by examining how k-means clustering on the lat-long coordinates of the venues along with a one-hot encoding of the venue category. This will result in a 2D spatial vector $X$ and an ND \"category vector\" $V$. I'll normalize $X$ and then run k-means clustering on the combination $\\alpha X + (1-\\alpha)V$. The initial exploration will be carried out for various choices of $k$ and $\\alpha$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# read data from the csv file and clean\n",
    "\n",
    "def make_clusters(k = 20, alpha = 1):\n",
    "    restaurant_data = pd.read_csv('restaurant_data')\n",
    "    restaurant_data.set_index('id', inplace=True)\n",
    "    restaurant_data.drop(['Name', 'step','label','isCore'], axis = 1, inplace=True)\n",
    "    coord_data = restaurant_data[['Lat','Lng']].copy()\n",
    "\n",
    "    # encode category column\n",
    "\n",
    "    restaurant_data = pd.get_dummies(restaurant_data, columns = ['Category'], prefix = '', prefix_sep = '')\n",
    "\n",
    "    # standardize\n",
    "\n",
    "    for col in ['Lat','Lng']:\n",
    "        restaurant_data[col] = (restaurant_data[col] - restaurant_data[col].mean())/restaurant_data[col].std()\n",
    "\n",
    "    # weight\n",
    "\n",
    "    dummy_cols = [col for col in restaurant_data.columns if col not in ['Lat','Lng']]\n",
    "    restaurant_data[['Lat','Lng']] *= alpha\n",
    "    restaurant_data[dummy_cols] *= 1-alpha\n",
    "\n",
    "    # predict, setting seed for comparability\n",
    "    np.random.seed(0)\n",
    "    clusterer = KMeans(n_clusters = k)\n",
    "    restaurant_data['label'] = clusterer.fit_predict(restaurant_data)\n",
    "    return restaurant_data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create map\n",
    "# make sure to load getAvgCoords and get_fill_color from above\n",
    "\n",
    "cmap = cm.get_cmap(name='cool', lut=restaurant_data['label'].max())\n",
    "my_map = folium.Map(location = getAvgCoords(coord_data), zoom_start = 10, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "\n",
    "for i in restaurant_data.index:\n",
    "    if np.random.rand() < 0.2:\n",
    "        entry = coord_data.loc[i,:]\n",
    "        folium.CircleMarker(\n",
    "            location=[entry['Lat'],entry['Lng']],\n",
    "            popup = \"group\" + str(restaurant_data.loc[i,'label']),\n",
    "            fill = True, \n",
    "            radius=3, \n",
    "            weight=2,\n",
    "            color=get_fill_color(cmap, restaurant_data.loc[i,'label']),\n",
    "            fill_color=get_fill_color(cmap, restaurant_data.loc[i,'label']),\n",
    "            fill_opacity=1).add_to(my_map)\n",
    "my_map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "scrolled": false
   },
   "source": [
    "We also need to examine what sorts of values we got for our labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "onehotcats = make_clusters(20,0)\n",
    "\n",
    "cat_counts = onehotcats.drop(['Lat','Lng','label'], axis=1).sum().sort_values(ascending=False)\n",
    "print(cat_counts.head)\n",
    "\n",
    "cat_counts[20::-1].plot(kind='barh')\n",
    "#plt.xticks(rotation=45)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some things to note:\n",
    "1. Many restaurant categories don't have but 1 entry, so we are greatly increasing the dimensionality of the dataspace by keeping them. Perhaps they should be dropped.\n",
    "2. Some non-restaurant categories have crept in to the dataset! It's unclear why those have appeared with the search critera. Perhaps the first category isn't always the one we need.\n",
    "\n",
    "The next step is to understand whether or not the groupings are actually picking out neighborhoods based on the restaurants in them or not. We start by clustering locations based only on their geographical coordindates and finding the proportions of different kinds of restaurants in that area. To measure how well the neighborhoods have concentrated restaurant types, we'll use the quantity $\\sum_i p_i^2$, where $p_i$ is the proportion of restaurant type $i$ in that neighborhood. This can be averaged across all the neighborhoods to see how concentrated the restaurants in a neighborhood for a given setting of $\\alpha$ are."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create dataframe which stores the onehot info and lat/long info without any scaling at all\n",
    "\n",
    "restaurant_data = pd.read_csv('restaurant_data')\n",
    "restaurant_data.set_index('id', inplace=True)\n",
    "restaurant_data.drop(['Name', 'step','label','isCore'], axis = 1, inplace=True)\n",
    "\n",
    "restaurant_data = pd.get_dummies(restaurant_data, columns = ['Category'], prefix = '', prefix_sep = '')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check how clustered the restaurant types are with geographical clustering only\n",
    "\n",
    "# form df with labels and tally \n",
    "loc_only = pd.concat([restaurant_data,make_clusters(20,1)['label']],axis=1).drop(['Lat','Lng'],axis=1)\n",
    "cluster_info = loc_only.groupby('label').sum()\n",
    "cluster_info['total'] =  cluster_info.sum(axis=1)\n",
    "\n",
    "# walk through each group and compute concentration \n",
    "base_result_table = []\n",
    "base_prop_measure = 0\n",
    "for i in cluster_info.index:\n",
    "    cluster_info.loc[i,:] /= cluster_info.loc[i,'total']\n",
    "    tmp = 100*cluster_info.loc[i,:].drop(['total']).sort_values(ascending=False).transpose()\n",
    "    base_prop_measure += (tmp*tmp).sum()/10000\n",
    "                                \n",
    "print('Geographically clustered concentration measure:', base_prop_measure/len(cluster_info.index))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# try again with alpha = 0.5\n",
    "\n",
    "# form df with labels and tally \n",
    "alpha8 = pd.concat([restaurant_data,make_clusters(20,0.5)['label']],axis=1)\n",
    "cluster_info = alpha8.groupby('label').sum()\n",
    "cluster_info['total'] =  cluster_info.sum(axis=1)\n",
    "\n",
    "# walk through each group and compute concentration \n",
    "alpha8_result_table = []\n",
    "alpha8_prop_measure = 0\n",
    "for i in cluster_info.index:\n",
    "    cluster_info.loc[i,:] /= cluster_info.loc[i,'total']\n",
    "    tmp = 100*cluster_info.loc[i,:].drop(['Lat','Lng','total']).sort_values(ascending=False).transpose()\n",
    "    alpha8_prop_measure += (tmp*tmp).sum()/10000\n",
    "\n",
    "\n",
    "print('Alpha = 0.5 concentration measure:', alpha8_prop_measure/len(cluster_info.index))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also get a quantitaive measure for how clustered the neighborhoods are by computing variances of the square distances from the mean."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# compute variances of sqdists from mean with geographically clustered\n",
    "\n",
    "loc_only = make_clusters(20,1)\n",
    "\n",
    "# find centroids\n",
    "centroids = loc_only[['Lat','Lng','label']].groupby('label').mean().reset_index()\n",
    "loc_only = loc_only.merge(centroids, on='label')\n",
    "\n",
    "# compute sqdist\n",
    "loc_only['distsq'] = (loc_only['Lat_x']-loc_only['Lat_y'])**2 + (loc_only['Lng_x']-loc_only['Lng_y'])**2\n",
    "                                \n",
    "print('Mean variance of sqdist for geographically clustered:', loc_only.groupby('label').std()['distsq'].mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# same, but with alpha = 0.5\n",
    "\n",
    "alpha5 = pd.concat([restaurant_data,make_clusters(20,0.5)['label']],axis=1)\n",
    "\n",
    "# find centroids\n",
    "centoids = alpha5[['Lat','Lng','label']].groupby('label').mean().reset_index()\n",
    "alpha5 = alpha5.merge( centroids, on='label')\n",
    "\n",
    "# compute sqdist\n",
    "alpha5['distsq'] = (alpha5['Lat_x']-alpha5['Lat_y'])**2 + (alpha5['Lng_x']-alpha5['Lng_y'])**2\n",
    "\n",
    "print('Mean variance of sqdist for alpha 0.5:', alpha5.groupby('label').std()['distsq'].mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compute concentrations and mean sqdist variance for varying alpha\n",
    "\n",
    "Let's plot the above processes for several values of $\\alpha$ to compare"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alphas = []\n",
    "concentrations = []\n",
    "sqdistvars = []\n",
    "\n",
    "for i in range(11):\n",
    "    alpha = i/10\n",
    "    alphas.append(alpha)\n",
    "    clusters = make_clusters(20,alpha)\n",
    "    results = pd.concat([restaurant_data,clusters['label']],axis=1)\n",
    "    \n",
    "    \n",
    "    # add concentration info\n",
    "    cluster_info = results.groupby('label').sum().drop(['Lat','Lng'], axis=1)\n",
    "    cluster_info['total'] =  cluster_info.sum(axis=1)\n",
    "\n",
    "    prop_measure = 0\n",
    "    for i in cluster_info.index:\n",
    "        cluster_info.loc[i,:] /= cluster_info.loc[i,'total']\n",
    "        tmp = 100*cluster_info.loc[i,:].drop('total').sort_values(ascending=False).transpose()\n",
    "        prop_measure += (tmp*tmp).sum()/10000\n",
    "    concentrations.append(prop_measure/len(cluster_info.index))\n",
    "    \n",
    "    # add geographical dispersion info\n",
    "    centoids = results[['Lat','Lng','label']].groupby('label').mean().reset_index()\n",
    "    results = results.merge( centroids, on='label')\n",
    "    results['distsq'] = (results['Lat_x']-results['Lat_y'])**2 + (results['Lng_x']-results['Lng_y'])**2\n",
    "    sqdistvars.append(results.groupby('label').std()['distsq'].mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(alphas, concentrations)\n",
    "plt.plot(alphas, sqdistvars)\n",
    "plt.title(\"Comparing clustering\")\n",
    "plt.xlabel(\"Alpha\")\n",
    "plt.legend(['Venue Concentrations', 'Sq Distance Variance'])\n",
    "#pd.DataFrame(alpha8_result_table)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we want to choose $\\alpha$ to make venue concentration high and distance variance small, let's choose $\\alpha$ that maximizes $$\\frac{\\mbox{venue concentration}}{\\mbox{square distance variance}}.$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ratios = []\n",
    "for i in range(len(alphas)):\n",
    "    ratios.append(concentrations[i]/sqdistvars[i])\n",
    "plt.plot(alphas, ratios)\n",
    "plt.title(\"Ratio of interest\")\n",
    "plt.xlabel(\"Alpha\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we'll form our final results using $\\alpha = 0.4$, which has a good mix of geographic and cuisine-type clustering features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Final Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_cluster = make_clusters(20,0.4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create map\n",
    "\n",
    "cmap = cm.get_cmap(name='cool', lut=final_cluster['label'].max())\n",
    "my_map = folium.Map(location = getAvgCoords(coord_data), zoom_start = 10, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "\n",
    "for i in restaurant_data.index:\n",
    "    if np.random.rand() < 0.2:\n",
    "        entry = coord_data.loc[i,:]\n",
    "        folium.CircleMarker(\n",
    "            location=[entry['Lat'],entry['Lng']],\n",
    "            popup = \"group\" + str(final_cluster.loc[i,'label']),\n",
    "            fill = True, \n",
    "            radius=3, \n",
    "            weight=2,\n",
    "            color=get_fill_color(cmap, final_cluster.loc[i,'label']),#'#555555',#get_boundary_color(cmap, label[i]), \n",
    "            fill_color=get_fill_color(cmap, final_cluster.loc[i,'label']),\n",
    "            fill_opacity=1).add_to(my_map)\n",
    "# centroids = pd.concat([restaurant_data, final_cluster['label']],axis=1).groupby('label').mean()[['Lat','Lng']]\n",
    "# for i in centroids.index:\n",
    "#     entry = centroids.loc[i,:]\n",
    "#     folium.CircleMarker(\n",
    "#             location=[entry['Lat'],entry['Lng']],\n",
    "#             popup = \"group\" + str(i),\n",
    "#             fill = True, \n",
    "#             radius=3, \n",
    "#             weight=5,\n",
    "#             color=get_fill_color(cmap, i),#'#555555',#get_boundary_color(cmap, label[i]), \n",
    "#             fill_color=get_fill_color(cmap, i),\n",
    "#             fill_opacity=1).add_to(my_map)\n",
    "\n",
    "print('Locations of Final Groups')\n",
    "my_map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# print out the information about restaurant types in each cluster\n",
    "\n",
    "def process_row(row):\n",
    "    sorted_row = row.sort_values(ascending=False)\n",
    "    results = []\n",
    "    for i in sorted_row.index:\n",
    "        if sorted_row[i] > 1:\n",
    "            results.append(str(i) + ', ' + '{:.2f}'.format(sorted_row[i]) + '%')\n",
    "        else:\n",
    "            break\n",
    "    print('Group {}\\n{}\\n'.format(row.name, '\\n'.join(results[:5])))\n",
    "\n",
    "final_info = pd.concat([restaurant_data, final_cluster['label']], axis=1)\n",
    "tallies = final_info.groupby('label').sum().drop(['Lat','Lng'],axis=1)\n",
    "tallies['total'] = tallies.sum(axis=1)\n",
    "\n",
    "for i in tallies.index:\n",
    "    tallies.loc[i,:] /= tallies.loc[i,'total']\n",
    "tallies.drop('total', axis = 1, inplace=True)\n",
    "tallies = tallies*100\n",
    "for i in tallies.index:\n",
    "    process_row(tallies.loc[i,:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# manually sort categories into lists for analysis below\n",
    "\n",
    "mexican_restaurants = [0,6]\n",
    "sandwich_places = [1,13]\n",
    "fast_food = [2,3]\n",
    "bars = [4,10,16]\n",
    "american_and_food = [5,7,9,14,19]\n",
    "restaurant = [8]\n",
    "chinese_restaurants = [11]\n",
    "coffee = [12]\n",
    "pizza = [15,18]\n",
    "donuts = [17]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What's happened? A few very popular sorts of restaurants like Mexican, Chinese, American have formed a cluster that contains only them. If we examine the distribution of one of these mono-category clusters, we get:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# only mexican clusters\n",
    "\n",
    "cmap = cm.get_cmap(name='cool', lut=final_cluster['label'].max())\n",
    "my_map = folium.Map(location = getAvgCoords(coord_data), zoom_start = 10, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "LABELS = mexican_restaurants\n",
    "for i in final_info.index:\n",
    "    if np.random.rand() < 1 and final_info.loc[i,'label'] in LABELS:\n",
    "        entry = final_info.loc[i,:]\n",
    "        folium.CircleMarker(\n",
    "            location=[entry['Lat'],entry['Lng']],\n",
    "            popup = \"group\" + str(final_cluster.loc[i,'label']),\n",
    "            fill = True, \n",
    "            radius=3, \n",
    "            weight=2,\n",
    "            color=get_fill_color(cmap, final_cluster.loc[i,'label']),#'#555555',#get_boundary_color(cmap, label[i]), \n",
    "            fill_color=get_fill_color(cmap, final_cluster.loc[i,'label']),\n",
    "            fill_opacity=1).add_to(my_map)\n",
    "my_map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare this to another mono-category cluster of the same kind:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# only fast food clusters\n",
    "\n",
    "# since it's hard to see the colors, let's make a custom function\n",
    "\n",
    "def get_fastfood_color(label):\n",
    "    if label == 2:\n",
    "        return '#FF0000'\n",
    "    else:\n",
    "        return '#0000FF'\n",
    "cmap = cm.get_cmap(name='cool', lut=final_cluster['label'].max())\n",
    "my_map = folium.Map(location = getAvgCoords(coord_data), zoom_start = 10, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "LABELS = fast_food\n",
    "for i in final_info.index:\n",
    "    if np.random.rand() < 1 and final_info.loc[i,'label'] in LABELS:\n",
    "        entry = final_info.loc[i,:]\n",
    "        folium.CircleMarker(\n",
    "            location=[entry['Lat'],entry['Lng']],\n",
    "            popup = \"group\" + str(final_cluster.loc[i,'label']),\n",
    "            fill = True, \n",
    "            radius=3, \n",
    "            weight=2,\n",
    "            color=get_fastfood_color(final_cluster.loc[i,'label']),#'#555555',#get_boundary_color(cmap, label[i]), \n",
    "            fill_color=get_fastfood_color(final_cluster.loc[i,'label']),\n",
    "            fill_opacity=1).add_to(my_map)\n",
    "my_map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# bars\n",
    "\n",
    "cmap = cm.get_cmap(name='cool', lut=final_cluster['label'].max())\n",
    "my_map = folium.Map(location = getAvgCoords(coord_data), zoom_start = 10, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "LABELS = bars\n",
    "for i in final_info.index:\n",
    "    if np.random.rand() < 1 and final_info.loc[i,'label'] in LABELS:\n",
    "        entry = final_info.loc[i,:]\n",
    "        folium.CircleMarker(\n",
    "            location=[entry['Lat'],entry['Lng']],\n",
    "            popup = \"group\" + str(final_cluster.loc[i,'label']),\n",
    "            fill = True, \n",
    "            radius=3, \n",
    "            weight=2,\n",
    "            color=get_fill_color(cmap, final_cluster.loc[i,'label']),#'#555555',#get_boundary_color(cmap, label[i]), \n",
    "            fill_color=get_fill_color(cmap, final_cluster.loc[i,'label']),\n",
    "            fill_opacity=1).add_to(my_map)\n",
    "my_map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# sandwich places\n",
    "\n",
    "cmap = cm.get_cmap(name='cool', lut=final_cluster['label'].max())\n",
    "my_map = folium.Map(location = getAvgCoords(coord_data), zoom_start = 10, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "LABELS = sandwich_places\n",
    "for i in final_info.index:\n",
    "    if np.random.rand() < 1 and final_info.loc[i,'label'] in LABELS:\n",
    "        entry = final_info.loc[i,:]\n",
    "        folium.CircleMarker(\n",
    "            location=[entry['Lat'],entry['Lng']],\n",
    "            popup = \"group\" + str(final_cluster.loc[i,'label']),\n",
    "            fill = True, \n",
    "            radius=3, \n",
    "            weight=2,\n",
    "            color=get_fill_color(cmap, final_cluster.loc[i,'label']),#'#555555',#get_boundary_color(cmap, label[i]), \n",
    "            fill_color=get_fill_color(cmap, final_cluster.loc[i,'label']),\n",
    "            fill_opacity=1).add_to(my_map)\n",
    "my_map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# \"food\" and \"american\" restaurants\n",
    "\n",
    "cmap = cm.get_cmap(name='cool', lut=final_cluster['label'].max())\n",
    "my_map = folium.Map(location = getAvgCoords(coord_data), zoom_start = 10, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "LABELS = american_and_food \n",
    "for i in final_info.index:\n",
    "    if np.random.rand() < 0.5 and final_info.loc[i,'label'] in LABELS:\n",
    "        entry = final_info.loc[i,:]\n",
    "        folium.CircleMarker(\n",
    "            location=[entry['Lat'],entry['Lng']],\n",
    "            popup = \"group\" + str(final_cluster.loc[i,'label']),\n",
    "            fill = True, \n",
    "            radius=3, \n",
    "            weight=2,\n",
    "            color=get_fill_color(cmap, final_cluster.loc[i,'label']),#'#555555',#get_boundary_color(cmap, label[i]), \n",
    "            fill_color=get_fill_color(cmap, final_cluster.loc[i,'label']),\n",
    "            fill_opacity=1).add_to(my_map)\n",
    "my_map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# pizza restaurants\n",
    "\n",
    "cmap = cm.get_cmap(name='cool', lut=final_cluster['label'].max())\n",
    "my_map = folium.Map(location = getAvgCoords(coord_data), zoom_start = 10, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "LABELS = coffee+donuts\n",
    "for i in final_info.index:\n",
    "    if np.random.rand() < 1 and final_info.loc[i,'label'] in LABELS:\n",
    "        entry = final_info.loc[i,:]\n",
    "        folium.CircleMarker(\n",
    "            location=[entry['Lat'],entry['Lng']],\n",
    "            popup = \"group\" + str(final_cluster.loc[i,'label']),\n",
    "            fill = True, \n",
    "            radius=3, \n",
    "            weight=2,\n",
    "            color=get_fill_color(cmap, final_cluster.loc[i,'label']),#'#555555',#get_boundary_color(cmap, label[i]), \n",
    "            fill_color=get_fill_color(cmap, final_cluster.loc[i,'label']),\n",
    "            fill_opacity=1).add_to(my_map)\n",
    "my_map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# view distribution of clusters with multiple venue categories\n",
    "\n",
    "cmap = cm.get_cmap(name='cool', lut=final_cluster['label'].max())\n",
    "my_map = folium.Map(location = getAvgCoords(coord_data), zoom_start = 10, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "mono_cats = [0, 2, 3, 6, 8, 9, 10, 11, 12, 15, 16,17]\n",
    "LABELS = [i for i in range(20) if i not in mono_cats]\n",
    "for i in final_info.index:\n",
    "    if np.random.rand() < 1 and final_info.loc[i,'label'] in LABELS:\n",
    "        entry = final_info.loc[i,:]\n",
    "        folium.CircleMarker(\n",
    "            location=[entry['Lat'],entry['Lng']],\n",
    "            popup = \"group\" + str(final_cluster.loc[i,'label']),\n",
    "            fill = True, \n",
    "            radius=3, \n",
    "            weight=2,\n",
    "            color=get_fill_color(cmap, final_cluster.loc[i,'label']),#'#555555',#get_boundary_color(cmap, label[i]), \n",
    "            fill_color=get_fill_color(cmap, final_cluster.loc[i,'label']),\n",
    "            fill_opacity=1).add_to(my_map)\n",
    "        \n",
    "centroids = final_info[final_info['label'].isin(LABELS)].groupby('label').mean()[['Lat','Lng']]\n",
    "\n",
    "for i in centroids.index:\n",
    "    text = 'Group '+ str(i)\n",
    "    iframe = folium.IFrame(text, width=30, height=30)\n",
    "    popup = folium.Popup(iframe, max_width=3000)\n",
    "\n",
    "    divIcon = folium.DivIcon(icon_size=(50,50), html='<div style=\"text-align:center;color:{}; background:#FFFFFF;\"> {} </div>'.format(get_fill_color(cmap,i),i))\n",
    "    Text = folium.Marker(location=[centroids.loc[i,'Lat'], centroids.loc[i,'Lng']], popup=popup,\n",
    "                     icon=divIcon)\n",
    "\n",
    "    my_map.add_child(Text)\n",
    "centroids\n",
    "my_map.save('clusters.html')\n",
    "my_map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for one last comparison, we do 8-means on the venues in the multilabel categories to see how they compare\n",
    "\n",
    "multilabel_venues = restaurant_data[['Lat','Lng']]\n",
    "#multilabel_venues = final_info[final_info['label'].isin(LABELS)][['Lat','Lng']]\n",
    "\n",
    "multilabel_clusterer = KMeans(n_clusters = 8)\n",
    "multilabel_venues['new_label'] = multilabel_clusterer.fit_predict(multilabel_venues)\n",
    "\n",
    "cmap = cm.get_cmap(name='cool', lut=multilabel_venues['new_label'].max())\n",
    "my_map = folium.Map(location = getAvgCoords(coord_data), zoom_start = 10, width='50%')\n",
    "\n",
    "# these are the airports in Chicago for reference\n",
    "folium.CircleMarker(location = [41.9742, -87.9073], popup=\"O'Hare\", color='blue').add_to(my_map) \n",
    "folium.CircleMarker(location = [41.7868, -87.7522], popup=\"Midway\", color='blue').add_to(my_map) \n",
    "\n",
    "\n",
    "for i in multilabel_venues.index:\n",
    "    if np.random.rand() < 1:\n",
    "        entry = multilabel_venues.loc[i,:]\n",
    "        folium.CircleMarker(\n",
    "            location=[entry['Lat'],entry['Lng']],\n",
    "            popup = \"group\" + str(multilabel_venues.loc[i,'new_label']),\n",
    "            fill = True, \n",
    "            radius=3, \n",
    "            weight=2,\n",
    "            color=get_fill_color(cmap,multilabel_venues.loc[i,'new_label']),#'#555555',#get_boundary_color(cmap, label[i]), \n",
    "            fill_color=get_fill_color(cmap, multilabel_venues.loc[i,'new_label']),\n",
    "            fill_opacity=1).add_to(my_map)\n",
    "        \n",
    "centroids = multilabel_venues.groupby('new_label').mean()[['Lat','Lng']]\n",
    "\n",
    "for i in centroids.index:\n",
    "    text = 'Group '+ str(i)\n",
    "    iframe = folium.IFrame(text, width=30, height=30)\n",
    "    popup = folium.Popup(iframe, max_width=3000)\n",
    "\n",
    "    divIcon = folium.DivIcon(icon_size=(50,50), html='<div style=\"text-align:center;color:{}; background:#FFFFFF;\"> {} </div>'.format(get_fill_color(cmap,i),i))\n",
    "    Text = folium.Marker(location=[centroids.loc[i,'Lat'], centroids.loc[i,'Lng']], popup=popup,\n",
    "                     icon=divIcon)\n",
    "\n",
    "    my_map.add_child(Text)\n",
    "centroids\n",
    "my_map.save('k_means_8_all.html')\n",
    "my_map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# a scratchpad to plot lat/long data if Chrome won't load the map\n",
    "\n",
    "\n",
    "raw_data = pd.read_csv('restaurant_data')\n",
    "#print(raw_data['Category'].value_counts())\n",
    "X = raw_data[raw_data['Category'].isin(['Sandwich Place'])][['Lat','Lng']].reset_index()\n",
    "ff_rest_clusterer = KMeans(n_clusters=1)\n",
    "labels = ff_rest_clusterer.fit_predict(X[['Lat','Lng']])\n",
    "\n",
    "for i in X.index:\n",
    "    if labels[i] == 0:\n",
    "        plt.plot(X.loc[i,'Lng'], X.loc[i,'Lat'], 'bo',alpha=.5)\n",
    "    elif labels[i] == 1:\n",
    "        plt.plot(X.loc[i,'Lng'], X.loc[i,'Lat'], 'bo')\n",
    "    else:\n",
    "        plt.plot(X.loc[i,'Lng'], X.loc[i,'Lat'], 'bo')\n",
    "        \n",
    "for i in final_info.index:\n",
    "    if final_info.loc[i,'label'] in [13]:\n",
    "        if final_info.loc[i,'label'] == 13:\n",
    "            plt.plot(final_info.loc[i,'Lng'], final_info.loc[i,'Lat'], 'ro',alpha=.5)\n",
    "        elif final_info.loc[i,'label'] == 7:\n",
    "            plt.plot(final_info.loc[i,'Lng'], final_info.loc[i,'Lat'], 'bo')\n",
    "        else:\n",
    "            plt.plot(final_info.loc[i,'Lng'], final_info.loc[i,'Lat'], 'go')\n",
    "plt.title('Sandwich Shops')\n",
    "plt.legend([plt.Line2D([0], [0], marker='o', color='w', label='Scatter',\n",
    "                          markerfacecolor='b', markersize=10),plt.Line2D([0], [0], marker='o', color='w', label='Scatter',\n",
    "                          markerfacecolor='r', markersize=10)],['Original Dataset','Sandwich Place Cluster'],bbox_to_anchor=(1.05, 1), loc='upper left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}