WriteUp.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Finding Lane Lines on the Road\n",
    "\n",
    "Author: **Lyuboslav Petrov**\n",
    "\n",
    "[//]: # (Image References)\n",
    "\n",
    "[grayscale]: ./test_images_output/grayscale_example.jpg \"Grayscale\"\n",
    "[blur_thresh]: ./test_images_output/step_2.jpg \"Blur and Threshold\"\n",
    "[roi]: ./test_images_output/step_3.jpg \"Region of Interest Mask applied\"\n",
    "[edges_lines]: ./test_images_output/step_4.jpg \"Edges and Hough Lines\"\n",
    "[result]: ./test_images_output/step_5.jpg \"Final\"\n",
    "[RGB]: ./test_images_output/RGB.jpg \"RGB Colorspace\"\n",
    "[YCBCR]: ./test_images_output/YCBCR.jpg \"YCbCr\"\n",
    "[Angle]: ./test_images_output/AngleA.jpg \"Angle\"\n",
    "\n",
    "---\n",
    "\n",
    "### Introduction\n",
    "\n",
    "One of the most distinctive features of any road is the markings used for guidance of drivers. Naturally, one of the first features to be discerned in a computer vision based self-driving car are **lane-markings**.\n",
    "\n",
    "---\n",
    "\n",
    "### Method\n",
    "\n",
    "The markings in this project where infered using the following pipeline:\n",
    "\n",
    "#### Step 1: Aquire image, convert it to grayscale and equalize-out the image using the opencv built-in histogram equalization routine\n",
    "   \n",
    "![][grayscale]\n",
    "\n",
    "#### Step 2: Apply a Gaussian blur for noise removal and perform a binary threshold\n",
    "   \n",
    "![][blur_thresh]\n",
    "\n",
    "#### Step 3: Extract the Region of interest using a trapezoid polygon\n",
    "* Top threshold is set to 52% of image height\n",
    "* Bottom threshold is the bottom border\n",
    "* Left-Top edge is set to 48% of image width\n",
    "* Rigth-Top edge is set to 52% of image width\n",
    "* Left Bottom is set to 5% image width\n",
    "* Right-Bottom is set to 95% image width\n",
    "\n",
    "![][roi]\n",
    "\n",
    "#### Step4: Convert ROI to edge image and apply a hough transform to identify lines\n",
    "\n",
    "![][edges_lines]\n",
    "\n",
    "#### Step5:\n",
    "The lines detected through the Hough transform are now iterated through to:\n",
    "* Filter line candidates by a slope threshold (abs(dy/dx) >= 0.44)\n",
    "* Segment left from right lines by positive and negative slopes respectively\n",
    "* Filter out noise by extracting all slopes that fall within the median +- one standard deviation\n",
    "* Using the gathered information get good line candidates and make a first degree polynomial fit\n",
    "* Finally, find the crossection of the lines found and draw them to the image\n",
    "\n",
    "![][result]\n",
    "\n",
    "---\n",
    "\n",
    "### Shortcommings\n",
    "\n",
    "The algorithm detailed is very unstable and assumes:\n",
    "\n",
    "1. A given image color distribution - all images used for tests have a very similar spectrum\n",
    "2. The camera is mounted at the centre of the car. Otherwise the chosen hard-coded ROI would be useless\n",
    "3. The road is of a certain color, as well as the lane markings - dark and bright, respectively\n",
    "    It can be seen that the algorithm brakes at the '*challange*' when a brighter section of the tarmac is entered. \n",
    "4. The road is straight - The results from the '*challange*' task prove convinsingly that the straight-road assumption is flawed.\n",
    "\n",
    "---\n",
    "\n",
    "### Potential Improvements\n",
    "\n",
    "Every of the mentioned shortcommings is a good candidate for improvements. \n",
    "\n",
    "---\n",
    "\n",
    "It has been attempted in this excercise to enable parametric polynomial inference of line-markings, however tests showed that the problem is easily overfitted and results are worst than a 1D polynomial fit.\n",
    "\n",
    "---\n",
    "\n",
    "Multiple colorspaces where tested. One promising candidate for detection of obstacles (i.e. pedestrians) on the road is the Cr-AngleA colorspace, that is derived from a YCbCr colorspace, which attempts a quasi 3D representation of the 2D image.\n",
    "\n",
    "The RGB colorspace does not convey information on the orientation of surfaces in the image, whereas the Cb and Cr channels of the YCbCr colorspace carry this information, which can further be reduced to an Angle image in Cr-AngleA colorspace using:\n",
    "\n",
    "        Cr = 0.5R - 0.419G - 0.08B + 128\n",
    "        L = sqrt(R^2+G^2+B^2)\n",
    "        AngleA = acos(B/L)\n",
    "        \n",
    "As can be seen from the results below, the Cr-AngleA colorspace does make perpendicular, with respect to the camera's frame of reference, objects more 'visible' than the plain r-g-b channels. \n",
    "\n",
    "#### R-G-B colorspace\n",
    "\n",
    "\n",
    "    \n",
    "![][RGB]\n",
    "\n",
    "#### Y-Cb-Cr colorspace\n",
    "\n",
    "\n",
    "![][YCBCR]\n",
    "\n",
    "#### Cr-AngleA colorspace\n",
    "\n",
    "![][Angle]\n",
    "---\n",
    "\n",
    "---\n",
    "\n",
    "### Conclusions\n",
    "\n",
    "The attempted lane segmentation excercise showed the difficulties and peculiarities in the topic, while scratching the surface of Computer Vision in the Self Driving Car's domain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}