TDD test Types.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Brief introduction to TDD Test Types - In Python! \n",
    "\n",
    "This document will have some code and exlanations of the following:\n",
    "\n",
    "* Dummy\n",
    "* Fake\n",
    "* Stub\n",
    "* Mock\n",
    "* Spy\n",
    "\n",
    "For these examples we will be using the _unittest_ library in python, but most concepts apply to other languages or libraries. \n",
    "\n",
    "##### Disclaimer: I mainly made this for self understanding. Any comment, correction, clarification or sugestion is well received, just add an issue over the repo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import unittest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now Let's say we are building an Insect identification system. The unit that we want to test consists of two methods, one to discern arachnids from insects, and the other one for retrieving the insect type, which calls the _isArachnid_ function  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def isArachnid(legs):\n",
    "    if legs ==8:\n",
    "        return True\n",
    "    else:\n",
    "        return False\n",
    "\n",
    "def insectType(legs, color):\n",
    "    if (isArachnid(legs)):\n",
    "        return \"Arachnid\"\n",
    "    else:\n",
    "        if color == \"black\":\n",
    "            return \"Cockroach\"\n",
    "        else:\n",
    "            return \"Other insect\"\n",
    "    # ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dummy\n",
    "\n",
    "A Dummy is a placeholder required to pass the unit test. Unit in the context (SUT) doesn’t exercise this placeholder. A dummy can be something as simple as passing ‘null’ or a void implementation with exceptions to ensure it’s never leveraged. Dummies should always try to be explicit so it doesn't confuse the developer reading the tests, into thinking that the dummy value holds additional meaning. Defaults and empty values should be used."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's say we write a test to check whether we can identify arachnids from other insects. Siince the _color_ attribute is not needed, we will send an empty string instead. This is what we call a dummy. Dummies could also be more complex objects, as long as proper constructors are provided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class InsectTests(unittest.TestCase):\n",
    "\n",
    "    def test_it_identifies_arachnids(self):\n",
    "        arachnid_legs= 8\n",
    "        dummy_arachnid_color = str() # <---  empty string as a dummy value\n",
    "        insect_type = insectType(arachnid_legs, dummy_arachnid_color)\n",
    "        self.assertEqual(insect_type, \"Arachnid\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we run the tests:...."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      ".\n",
      "----------------------------------------------------------------------\n",
      "Ran 1 test in 0.001s\n",
      "\n",
      "OK\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<unittest.main.TestProgram at 0x20b6f92ce80>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unittest.main(argv=['first-arg-is-ignored'], exit=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fake\n",
    "\n",
    "A Fake is used to simplify a dependecy, in order to make our tests pass easily. Fakes usually are the working implementation of an interface, but they are not prduction ready, most of the time they will return just pre-defined static values. Fakes are frequently used to mimic database functionalities using in-memory databases or pre-defined files."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, let's pretend we have a _ForestAPI_ that returns insects in real time that are spotted by cameras inside a forest. APIs usually have a limited quota, so we will write a fake object that mimics the values that the real API would return.\n",
    "\n",
    "Our Fake API will  return a random insect fro a pre-defined list when we call the __getInsect()__ function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import random\n",
    "class ForestAPI_fake():\n",
    "    def __init__(self):\n",
    "        #fake insect list:\n",
    "\n",
    "        self.insects = [\n",
    "            {\n",
    "                \"name\": \"Cockroach\",\n",
    "                \"legs\": 6,\n",
    "                \"color\": \"black\"\n",
    "            },\n",
    "            {\n",
    "                \"name\": \"Spider\",\n",
    "                \"legs\":8,\n",
    "                \"color\": \"green\"\n",
    "            },\n",
    "            {\n",
    "                \"name\": \"Ladybug\",\n",
    "                \"legs\": 6,\n",
    "                \"color\": \"red\"\n",
    "            }]\n",
    "    def getInsect(self):\n",
    "        #We will return a random insect:\n",
    "        return self.insects[random.randint(0,2)]\n",
    "    def acknowledgeRelease(self):\n",
    "        return True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check that running the fake api, we will be returned with a simple object from our fake list.\n",
    "\n",
    "We expect the real APi to return more diverse data, and not do it in a random manner. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'color': 'red', 'legs': 6, 'name': 'Ladybug'}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "api = ForestAPI_fake()\n",
    "api.getInsect()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, lets re-write our tests and add a method to check wether our calls to the API work or not."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class InsectTests(unittest.TestCase):\n",
    "\n",
    "    def test_it_identifies_arachnids(self):\n",
    "        arachnid_legs= 8\n",
    "        dummy_arachnid_color = str() # <---  empty string as a dummy value\n",
    "        insect_type = insectType(arachnid_legs, dummy_arachnid_color)\n",
    "        self.assertEqual(insect_type, \"Arachnid\")\n",
    "        \n",
    "    def test_it_retrieves_insects_from_api(self):\n",
    "        api = ForestAPI_fake()\n",
    "        insect = api.getInsect()\n",
    "        self.assertIsNotNone(insect) # For now, we will just check that the object returned contains something."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "..\n",
      "----------------------------------------------------------------------\n",
      "Ran 2 tests in 0.002s\n",
      "\n",
      "OK\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<unittest.main.TestProgram at 0x20b6f94f550>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unittest.main(argv=['first-arg-is-ignored'], exit=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many libraries to create \"Fake\" objects from API calls. These are very useful to speed up your tests since they save the API response inside a file. The next time you call that API method, instead of calling the actual API over the internet, you are returned the stored response from your filesystem. You can also choose to re-create the _cassetes_ if the API changes, or you want to have more updated data for your tests.\n",
    "\n",
    "For python, there is [vcr.py](https://github.com/kevin1024/vcrpy), as well as vcr for [ruby](https://github.com/vcr/vcr) and [PHP](https://github.com/php-vcr/php-vcr). Just search for _\"vcr + language\"_ to check if there is a similar implementation for that language."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stub\n",
    "\n",
    "A Stub is an object that holds predefined data and uses it to answer calls during tests. It is used when we cannot or don’t want to involve objects that would answer with real data or have undesirable side effects. \n",
    "\n",
    "Martin Fowler defines stubs as objects that \"provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test\".\n",
    "\n",
    "The main difference with fakes, is that a stub doesn't need to be a full implementation, as long as it returns the expected values and receive the expected parameters. \n",
    "Also stubs are often automatically created by external libraries. Because Python lacks the concept of interfaces, and instead uses duck-typing, they way of doing stubbing is less standard than in other languages like Java. \n",
    "\n",
    "We will use the included _mock_ library in Python 3.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import unittest.mock\n",
    "from unittest.mock import MagicMock"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our ForestAPI will ocasionally return an insect with a legcount of 0. We have to able to identify them to remove them from further processing. We will add a method to identify if an insect is dead:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def isInsectDead(insect):\n",
    "    legcount =  insect['legs']\n",
    "    if (legcount > 0):\n",
    "        return False\n",
    "    else:\n",
    "        return True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will add a new test case to see if it works properly.\n",
    "\n",
    "We cannot use the fake _ForestAPI_ to test this, because we don't have any value for a dead insect. Instead of adding one, we will define a use a stub that returns a dead insect at all times me for this test:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class InsectTests(unittest.TestCase):\n",
    "\n",
    "    def test_it_identifies_arachnids(self):\n",
    "        arachnid_legs= 8\n",
    "        dummy_arachnid_color = str() # <---  empty string as a dummy value\n",
    "        insect_type = insectType(arachnid_legs, dummy_arachnid_color)\n",
    "        self.assertEqual(insect_type, \"Arachnid\")\n",
    "        \n",
    "    def test_it_retrieves_insects_from_api(self):\n",
    "        api = ForestAPI_fake()\n",
    "        insect = api.getInsect()\n",
    "        self.assertIsNotNone(insect) # For now, we will just check that the object returned contains something.\n",
    "        \n",
    "    def test_it_identifies_dead_insects(self):\n",
    "        api = ForestAPI_fake() \n",
    "        api.getInsect =  MagicMock(return_value= {\"name\": \"dead ant\", \"legs\": 0, \"color\": \"black\"}) #<- Our dead insect stub\n",
    "        insect = api.getInsect()\n",
    "        self.assertTrue(isInsectDead(insect)) # For now, we will just check that the object returned contains something.\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the tests passing. If we comment the stub line, we can see that the test will fail every time. And t would also fail sometimes even if we had a dead insect to the fake API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "...\n",
      "----------------------------------------------------------------------\n",
      "Ran 3 tests in 0.007s\n",
      "\n",
      "OK\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<unittest.main.TestProgram at 0x20b6f96d710>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unittest.main(argv=['first-arg-is-ignored'], exit=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mocks ##\n",
    "\n",
    "Mocks are similar to stubs, but they aso specify behavior in them.\n",
    "Martin Fowler defines mocks as: _\"objects pre-programmed with expectations which form a specification of the calls they are expected to receive.\"_.\n",
    "\n",
    "While with stubs we test states, mocks are intended to test behaviour.\n",
    "\n",
    "### Stub vs Mock\n",
    "\n",
    "##### Stub:\n",
    "* Setup\n",
    "* Test\n",
    "* Verify State\n",
    "* Teardown\n",
    "\n",
    "##### Mock\n",
    "* Setup\n",
    "* __Setup Expectations__\n",
    "* Test\n",
    "* __Verify Expectations__\n",
    "* Verify State\n",
    "* Teardown"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To test with mocks this, let's make an \"Insect Lab\" class, which contains some of the methods described before. We will also have a method to releasean insect if it is an arachnid. (I know... the logic is very stupid, but the redundant calls will show how the mocks can record that information)\n",
    "\n",
    "The basic idea, is that we will get one insect from our _ForestAPI_ , then check whether it is an Arachnid or not to release it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "class InsectLab(object):\n",
    "    def shouldReleaseInsect(self,insect):\n",
    "        itype =  self.insectType(insect)\n",
    "        if (itype == \"Arachnid\"):\n",
    "            \n",
    "            return True\n",
    "        else:\n",
    "           \n",
    "            return False            \n",
    "        \n",
    "    def isArachnid(self,legs):\n",
    "        if legs ==8:\n",
    "            return True\n",
    "        else:\n",
    "            return False\n",
    "\n",
    "    def insectType(self,insect):\n",
    "        if (self.isArachnid(insect['legs'])):\n",
    "            return \"Arachnid\"\n",
    "        else:\n",
    "            if insect['color'] == \"black\":\n",
    "                return \"Grounder\"\n",
    "            else:\n",
    "                return \"Other insect\"\n",
    "        # ...\n",
    "    def getAndProcessInsect(self,api):\n",
    "        insect =  api.getInsect()\n",
    "        was_released = self.shouldReleaseInsect(insect)\n",
    "        if (was_released):\n",
    "            api.acknowledgeRelease()\n",
    "        return was_released\n",
    "\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time lets make a test that fails. The mock will be the API (We built it over the fake api used before using the _(spec=...)_ command.\n",
    "\n",
    "\n",
    "In our test, we will check if our mock api calls the __AcknowledgeRelease()__ function. If it is called, it means that the whole process from getting an insect, analyzing it and then releasing the arachnids works. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "..F.\n",
      "======================================================================\n",
      "FAIL: test_it_releases_arachnids (__main__.InsectTests)\n",
      "----------------------------------------------------------------------\n",
      "Traceback (most recent call last):\n",
      "  File \"<ipython-input-14-55e0b60ed041>\", line 28, in test_it_releases_arachnids\n",
      "    mock_api.acknowledgeRelease.assert_called_with() #<- We want to check if the release method was called\n",
      "  File \"C:\\Users\\isaac\\Anaconda3\\lib\\unittest\\mock.py\", line 783, in assert_called_with\n",
      "    raise AssertionError('Expected call: %s\\nNot called' % (expected,))\n",
      "AssertionError: Expected call: acknowledgeRelease()\n",
      "Not called\n",
      "\n",
      "----------------------------------------------------------------------\n",
      "Ran 4 tests in 0.015s\n",
      "\n",
      "FAILED (failures=1)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<unittest.main.TestProgram at 0x20b6f93d7f0>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from unittest.mock import Mock\n",
    "class InsectTests(unittest.TestCase):\n",
    "\n",
    "    def test_it_identifies_arachnids(self):\n",
    "        arachnid_legs= 8\n",
    "        dummy_arachnid_color = str() # <---  empty string as a dummy value\n",
    "        insect_type = insectType(arachnid_legs, dummy_arachnid_color)\n",
    "        self.assertEqual(insect_type, \"Arachnid\")\n",
    "        \n",
    "    def test_it_retrieves_insects_from_api(self):\n",
    "        api = ForestAPI_fake()\n",
    "        insect = api.getInsect()\n",
    "        self.assertIsNotNone(insect) # For now, we will just check that the object returned contains something.\n",
    "        \n",
    "    def test_it_identifies_dead_insects(self):\n",
    "        api = ForestAPI_fake() \n",
    "        api.getInsect =  MagicMock(return_value= {\"name\": \"dead ant\", \"legs\": 0, \"color\": \"black\"}) #<- Our dead insect stub\n",
    "        insect = api.getInsect()\n",
    "        self.assertTrue(isInsectDead(insect)) # For now, we will just check that the object returned contains something.\n",
    "        \n",
    "    def test_it_releases_arachnids(self):\n",
    "        mock_api =  Mock(spec=ForestAPI_fake) \n",
    "        mock_api.getInsect.return_value = {\"name\": \"Ant\", \"legs\": 6, \"color\": \"black\"} #<- Our non arachnid stub\n",
    "        mock_api.acknowledgeRelease = MagicMock(return_value = True)\n",
    "        lab = InsectLab()\n",
    "        \n",
    "        lab.getAndProcessInsect(mock_api)        \n",
    "        mock_api.acknowledgeRelease.assert_called_with() #<- We want to check if the release method was called\n",
    "\n",
    "        \n",
    "unittest.main(argv=['first-arg-is-ignored'], exit=False)     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can observe that the method was not called. If we switch the insect stub to a __\"legs\":8__, the test passes, even tho we didn't call the real api, or any of their methods.\n",
    "\n",
    "With mocks, we can check if a method is called, how many times it is called and more things that with stubs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Spies\n",
    "\n",
    "According to Martin Fowler \"Spies are stubs that also record some information based on how they were called. One form of this might be an email service that records how many messages it was sent.\".\n",
    "\n",
    "Spies should act as normal objects, with the added mock functionalities. \n",
    "\n",
    "On this specific python library, we can create a Mock with the parameter __(wraps=...)__ where the wrapped object will be the one we spy upon.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Calls: [call.getInsect(), call.acknowledgeRelease()] \n",
      "\n",
      "Calls: [call.getInsect(), call.acknowledgeRelease(), call.getInsect()] \n",
      "\n",
      "Calls: [call.getInsect(),\n",
      " call.acknowledgeRelease(),\n",
      " call.getInsect(),\n",
      " call.getInsect()] \n",
      "\n"
     ]
    }
   ],
   "source": [
    "import unittest.mock  as mock\n",
    "\n",
    "lab = InsectLab()\n",
    "mock_api =  ForestAPI_fake()\n",
    "spy_api = mock.Mock(wraps=mock_api)\n",
    "\n",
    "for i in range(3):\n",
    "    lab.getAndProcessInsect(spy_api)\n",
    "     \n",
    "    print (\"Calls: %s \\n\" % spy_api.mock_calls)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We ran the  _getAndProcessInsect()_ function three times. We can see the historical calls made by the fake _ForestAPI_. In some cases, the _acknowledgeRelease()_ was called, which means the insect retrieved was an arachnid.\n",
    "\n",
    "With Spies we can check how many times a method is being called, what are the values of the parameters it was called with, and more. Check the documentation for the specific mocking library that you use.\n",
    "\n",
    "\n",
    "Happy testing!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sources:\n",
    "\n",
    "* https://martinfowler.com/articles/mocksArentStubs.html\n",
    "* http://www.telerik.com/blogs/fakes-stubs-and-mocks\n",
    "* https://nirajrules.wordpress.com/2011/08/27/dummy-vs-stub-vs-spy-vs-fake-vs-mock/\n",
    "* http://igorsobreira.com/2010/12/15/mocks-stubs-fakes-spies.html\n",
    "* https://dev.to/milipski/test-doubles---fakes-mocks-and-stubs\n",
    "* https://semaphoreci.com/community/tutorials/getting-started-with-mocking-in-python\n",
    "* https://docs.python.org/3/library/unittest.mock-examples.html\n",
    "* http://blog.thedigitalcatonline.com/blog/2016/03/06/python-mocks-a-gentle-introduction-part-1/#.WdwJVmhSyUl\n",
    "\n",
    "\n",
    "...And of course, StackOverflow.\n"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}