7_BestPractices_Testing_Performance.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 7 Best Practices, Testing and Performance\n",
    "\n",
    "Advanced bits marked with ⚠ might be more for reference later."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## PEP8 -- Style Guide for Python Code\n",
    "\n",
    "Intendation is significant in Python, preventing the worst mess. The remaining stylistic questions, like naming conventions, used in the standard library are summarized in the [Python Enhancement Proposal 8](https://www.python.org/dev/peps/pep-0008/). Most Python programmers adhere to this style guide (Google has their [own](https://google.github.io/styleguide/pyguide.html) which is similar and also popular), making it easy to read someone elses code. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "⚠ Some editors and tools like `autopep8` help check for violations. Other tools, like [yapf](https://github.com/google/yapf), go further and reformat your code, e.g.\n",
    "\n",
    "```python\n",
    "def forces(t,\n",
    "                X,      N):\n",
    "    \"\"\"Calculate forces from neighbouring cells\"\"\"\n",
    "    for i, x in enumerate(X):\n",
    "        r = X[N[i]] - x\n",
    "        norm_r = np.minimum             (np.linalg.norm(r,\n",
    "\n",
    "\n",
    "        \n",
    "        axis=1), r_max)\n",
    "        norm_F = 2*(r_min -       norm_r)*(norm_r - r_max) - (norm_r - r_max)**2\n",
    "        F[i] = np.sum(r*(\n",
    "                                norm_F/\n",
    "                    norm_r)[:, None], axis=0)\n",
    "    return F.ravel()\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "becomes:\n",
    "\n",
    "```python\n",
    "def forces(t, X, N):\n",
    "    \"\"\"Calculate forces from neighbouring cells\"\"\"\n",
    "    for i, x in enumerate(X):\n",
    "        r = X[N[i]] - x\n",
    "        norm_r = np.minimum(np.linalg.norm(r, axis=1), r_max)\n",
    "        norm_F = 2 * (r_min - norm_r) * (norm_r - r_max) - (norm_r - r_max)**2\n",
    "        F[i] = np.sum(r * (norm_F / norm_r)[:, None], axis=0)\n",
    "    return F.ravel()\n",
    "\n",
    "```\n",
    "\n",
    "As of 2021, [black](https://black.readthedocs.io/en/latest/) is the de facto standard for Python formatting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "⚠ Compilation would prevent many erros that are only detected at run-time in uncompiled languages like Python. But some Editors and tools like [pyflakes](https://pypi.python.org/pypi/pyflakes) and [pylint](https://www.pylint.org/) help with static code analysis to prevent the worst errors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## PEP 20 -- The Zen of Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Zen of Python, by Tim Peters\n",
      "\n",
      "Beautiful is better than ugly.\n",
      "Explicit is better than implicit.\n",
      "Simple is better than complex.\n",
      "Complex is better than complicated.\n",
      "Flat is better than nested.\n",
      "Sparse is better than dense.\n",
      "Readability counts.\n",
      "Special cases aren't special enough to break the rules.\n",
      "Although practicality beats purity.\n",
      "Errors should never pass silently.\n",
      "Unless explicitly silenced.\n",
      "In the face of ambiguity, refuse the temptation to guess.\n",
      "There should be one-- and preferably only one --obvious way to do it.\n",
      "Although that way may not be obvious at first unless you're Dutch.\n",
      "Now is better than never.\n",
      "Although never is often better than *right* now.\n",
      "If the implementation is hard to explain, it's a bad idea.\n",
      "If the implementation is easy to explain, it may be a good idea.\n",
      "Namespaces are one honking great idea -- let's do more of those!\n"
     ]
    }
   ],
   "source": [
    "import this"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Structuring Code\n",
    "\n",
    "* Don't repeat yourself (and others) to keep your programs small"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Structure code into functions and files. You can import functions and data from files (so-called modules):\n",
    "\n",
    "```python\n",
    "# File module/example.py\n",
    "x = 'Example data'\n",
    "\n",
    "def example_function():\n",
    "    pass\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    whatever()\n",
    "    \n",
    "# File in .\n",
    "from module import example\n",
    "\n",
    "example.x\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Documentation\n",
    "\n",
    "* Document why and not what (what states the code)\n",
    "\n",
    "* Commit early and often to keep track of why you change things\n",
    "\n",
    "* False (usually out-dated) documentation is worse than none\n",
    "\n",
    "* Turn questions (yours or others) into documentation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Write [self-documenting code](https://testing.googleblog.com/2017/07/code-health-to-comment-or-not-to-comment.html), e.g. (from real-life C++) change\n",
    "\n",
    "```C++\n",
    "if ((j != k) and (d_type[d_lattice->d_cell_id[j]] == mesenchyme)\n",
    "        and (d_type[d_lattice->d_cell_id[k]] == mesenchyme)\n",
    "        and (dist < r_link)\n",
    "        and (fabs(r.w/(d_X[d_lattice->d_cell_id[j]].w + \n",
    "             d_X[d_lattice->d_cell_id[k]].w)) > 0.2)) {\n",
    "    d_link[i].a = d_lattice->d_cell_id[j];\n",
    "    d_link[i].b = d_lattice->d_cell_id[k];\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "to\n",
    "```C++\n",
    "auto both_mesenchyme = (d_type[d_lattice->d_cell_id[j]] == mesenchyme)\n",
    "    and (d_type[d_lattice->d_cell_id[k]] == mesenchyme);\n",
    "auto along_w = fabs(r.w/(d_X[d_lattice->d_cell_id[j]].w + \n",
    "                    d_X[d_lattice->d_cell_id[k]].w)) > 0.2;\n",
    "if (both_mesenchyme and (dist < r_link) and along_w) {\n",
    "    d_link[i].a = d_lattice->d_cell_id[j];\n",
    "    d_link[i].b = d_lattice->d_cell_id[k];\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Use `docstrings`: \n",
    "\n",
    "```python\n",
    "def documented_function():\n",
    "    \"\"\"This is a docstring\"\"\"\n",
    "    pass\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* ⚠ Turn documentation into code using `docopt`:\n",
    "\n",
    "```python\n",
    "\"\"\"A Docopt Example.\n",
    "\n",
    "Usage:\n",
    "  docoptest.py [flag] [--parameter <x>]\n",
    "  docoptest.py (-h | --help)\n",
    "\n",
    "Options:\n",
    "  -h --help        Show this screen.\n",
    "  --parameter <x>  Pass parameter.\n",
    "\"\"\"\n",
    "if __name__ == '__main__':\n",
    "    from docopt import docopt\n",
    "\n",
    "    args = docopt(__doc__)\n",
    "    print(args)\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Results in\n",
    "```bash\n",
    "$ python docoptest.py wrong usage\n",
    "Usage:\n",
    "  docoptest.py [flag] [--parameter <x>]\n",
    "  docoptest.py (-h | --help)\n",
    "\n",
    "$ python docoptest.py -h\n",
    "A Docopt Example.\n",
    "\n",
    "Usage:\n",
    "  docoptest.py [flag] [--parameter <x>]\n",
    "  docoptest.py (-h | --help)\n",
    "\n",
    "Options:\n",
    "  -h --help        Show this screen.\n",
    "  --parameter <x>  Pass parameter.\n",
    "\n",
    "$ python docoptest.py flag\n",
    "{'--help': False,\n",
    " '--parameter': None,\n",
    " 'flag': True}\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Testing\n",
    "\n",
    "* Turn debugging `print`-statements into `assert`-statements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "ename": "AssertionError",
     "evalue": "Mass cannot be negative!",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mAssertionError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-2-63c5e1a96e4e>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[0mmass\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m-\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m \u001b[1;32massert\u001b[0m \u001b[0mmass\u001b[0m \u001b[1;33m>\u001b[0m \u001b[1;36m0\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'Mass cannot be negative!'\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mAssertionError\u001b[0m: Mass cannot be negative!"
     ]
    }
   ],
   "source": [
    "mass = -1\n",
    "assert mass > 0, \"Mass cannot be negative!\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Make sure your code does what it is supposed to do by testing it with simple examples where you know what to expect\n",
    "\n",
    "* Save those tests (e.g. in `if __name__ == __main__` of each module) to keep your code correct while changing parts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* ⚠ Automate testing\n",
    "\n",
    "```python\n",
    "import unittest\n",
    "\n",
    "def square(x):\n",
    "    return x*x\n",
    "\n",
    "class TestSquare(unittest.TestCase):\n",
    "\n",
    "    def test_square(self):\n",
    "        self.assertEqual(square(2), 4)\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    unittest.main()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* ⚠ Generate fuzzy tests using `hypothesis`:\n",
    "\n",
    "```python\n",
    "from hypothesis import given\n",
    "from hypothesis.strategies import text, floats\n",
    "\n",
    "@given(text(min_size=10), text(min_size=10), text(min_size=10))\n",
    "def test_triangle_inequality(self, a, b, c):\n",
    "    self.assertTrue(zipstance(a, c) <= zipstance(a, b) + zipstance(b, c))\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* ⚠ Turn documentation into tests using `doctest`:\n",
    "\n",
    "```python\n",
    "def square(x):\n",
    "    \"\"\"Return the square of x.\n",
    "\n",
    "    >>> square(2)\n",
    "    0\n",
    "    \"\"\"\n",
    "    return x*x\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    import doctest\n",
    "    doctest.testmod()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Will give\n",
    "\n",
    "```bash\n",
    "$ python doctestest.py \n",
    "**********************************************************************\n",
    "File \"doctestest.py\", line 4, in __main__.square\n",
    "Failed example:\n",
    "    square(2)\n",
    "Expected:\n",
    "    0\n",
    "Got:\n",
    "    4\n",
    "**********************************************************************\n",
    "1 items had failures:\n",
    "   1 of   1 in __main__.square\n",
    "***Test Failed*** 1 failures.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* ⚠ Use [mutation tests](http://cosmic-ray.readthedocs.io/en/latest/) if you need to be sure that your code is correct. They replace pieces of code, e.g. `<` with `>` to find out whether your tests cover everything."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Optimizing Performance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Quick Tests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20.1 ms ± 173 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit [i**2 for i in range(100000)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "106 µs ± 1.46 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "%timeit np.arange(100000)**2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Real-life Example\n",
    "\n",
    "We tried to speed-up the simulation of an N-body problem describing cells interacting in pairs via the spherical potential $U(r) = -(r - r_{min})(r - r_{max})^2$, where $r = |\\vec x_j - \\vec x_i|$. The resulting forces can be calculated as\n",
    "\n",
    "```python\n",
    "def forces(t, X, N):\n",
    "    \"\"\"Calculate forces from neighbouring cells\"\"\"\n",
    "    for i, x in enumerate(X):\n",
    "        r = X[N[i]] - x\n",
    "        norm_r = np.minimum(np.linalg.norm(r, axis=1), r_max)\n",
    "        norm_F = 2*(r_min - norm_r)*(norm_r - r_max) - (norm_r - r_max)**2\n",
    "        F[i] = np.sum(r*(norm_F/norm_r)[:, None], axis=0)\n",
    "    return F.ravel()\n",
    "```\n",
    "\n",
    "where `N` is a matrix giving the $k$ nearest neighbours."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Profiling\n",
    "\n",
    "The `cProfile` module help identify bottlenecks when running `'command'`:\n",
    "\n",
    "\n",
    "```python\n",
    "import cProfile\n",
    "\n",
    "cProfile.run('command', 'nbodystats')\n",
    "```\n",
    "\n",
    "and `pstat` allows you then to analyse the data saved in `nbodystats` (in the actual code `forces` was wrapped into a class with `__call__`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import pstats\n",
    "\n",
    "p = pstats.Stats(\"nbodystats\")\n",
    "p.strip_dirs().sort_stats(\"cumulative\").print_stats(10)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "Wed Aug  9 14:52:36 2017    nbodystats\n",
    "\n",
    "         20147135 function calls (20133338 primitive calls) in 54.027 seconds\n",
    "\n",
    "   Ordered by: cumulative time\n",
    "   List reduced from 341 to 10 due to restriction <10>\n",
    "\n",
    "   ncalls  tottime  percall  cumtime  percall filename:lineno(function)\n",
    "        1    0.000    0.000   54.027   54.027 {built-in method builtins.exec}\n",
    "        1    0.000    0.000   54.027   54.027 <string>:1(<module>)\n",
    "        1    0.012    0.012   54.027   54.027 solve.py:36(scipy_ode)\n",
    "      300    0.002    0.000   53.165    0.177 _ode.py:396(integrate)\n",
    "      300    0.047    0.000   53.163    0.177 _ode.py:1030(run)\n",
    "    15811    0.060    0.000   53.116    0.003 round_up.py:93(squeezed_cubic_knn)\n",
    "    15811   29.676    0.002   52.630    0.003 round_up.py:58(__call__)\n",
    "  1581100    6.794    0.000   14.802    0.000 linalg.py:1976(norm)\n",
    "  3166399   11.091    0.000   11.091    0.000 {method 'reduce' of 'numpy.ufunc' objects}\n",
    "  1581100    2.429    0.000    8.121    0.000 fromnumeric.py:1743(sum)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "⚠ Note that these tools generate a lot of overhead, the amount depending on the architecture. Therefore, these tools cannot be used to compare the performance across major changes in the code. Then simple time measurements are more reliable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Vectorize, 7x\n",
    "\n",
    "While the initial function was already using `numpy` and vectors it still involves a `for`-loop that can be vectorized ... well, actually tensorized:\n",
    "\n",
    "```python\n",
    "def forces(t, X, N):\n",
    "    r = X[N] - np.tile(X, (k, 1, 1)).transpose(1, 0, 2)\n",
    "    norm_r = np.minimum(np.linalg.norm(r, axis=2), r_max)\n",
    "    norm_F = 2*(r_min - norm_r)*(norm_r - r_max) - (norm_r - r_max)**2\n",
    "    F = np.sum(r*(norm_F/norm_r)[:, None].transpose(0, 2, 1), axis=1)\n",
    "    return F.ravel()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Re-use resources, 1.5x\n",
    "\n",
    "Broadcasting gives just a \"view\" instead of a copy returned by `np.tile`:\n",
    "\n",
    "```python\n",
    "def forces(t, X, N):\n",
    "    r = X[N] - X[:, None, :]\n",
    "    norm_r = np.minimum(np.linalg.norm(r, axis=2), r_max)\n",
    "    norm_F = 2*(r_min - norm_r)*(norm_r - r_max) - (norm_r - r_max)**2\n",
    "    F = np.sum(r*(norm_F/norm_r)[:, None].transpose(0, 2, 1), axis=1)\n",
    "    return F.ravel()\n",
    "```\n",
    "\n",
    "The standard inplace operators `+=`, `-=`, `*=`, and `/=` can speed up large numpy calculations a little, Pandas operations often have an `inplace` flag, e.g. `df.reset_index(inplace=True)`. Sometimes allocating memory with `np.empty` or `pd.DataFrame` pays off. Make sure to use the fastest data type for each column of a `pd.DataFrame`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### ⚠ Compile the calculation, 2x (3.5x on GTX 1060 for 10k cells)\n",
    "\n",
    "Finally code can be compiled in various ways. We choose [Theano](http://deeplearning.net/software/theano/) because it requires little rewritting (and can run code on the GPU as well):\n",
    "\n",
    "```python\n",
    "from theano import Tensor as T\n",
    "from theano import function\n",
    "\n",
    "\n",
    "X = T.matrix('X', dtype='floatX')\n",
    "N = T.imatrix('N')\n",
    "\n",
    "r = X[N] - X[:, None, :]\n",
    "norm_r = T.minimum(r.norm(2, axis=2), 1)\n",
    "norm_F = 2*(0.5 - norm_r)*(norm_r - 1) - (norm_r - 1)**2\n",
    "F = T.sum(r*(norm_F/norm_r).dimshuffle(0, 1, 'x'), axis=1)\n",
    "f = function([X, N], F.ravel(), allow_input_downcast=True)\n",
    "\n",
    "def forces(t, X, N):\n",
    "    return f(X.reshape(-1, 3), self.N)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Aftermath"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "p = pstats.Stats(\"theanostats\")\n",
    "p.strip_dirs().sort_stats(\"cumulative\").print_stats(10)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "Wed Aug  9 15:00:50 2017    theanostats\n",
    "\n",
    "         1384886 function calls (1371089 primitive calls) in 3.072 seconds\n",
    "\n",
    "   Ordered by: cumulative time\n",
    "   List reduced from 342 to 10 due to restriction <10>\n",
    "\n",
    "   ncalls  tottime  percall  cumtime  percall filename:lineno(function)\n",
    "        1    0.000    0.000    3.072    3.072 {built-in method builtins.exec}\n",
    "        1    0.000    0.000    3.072    3.072 <string>:1(<module>)\n",
    "        1    0.009    0.009    3.072    3.072 solve.py:36(scipy_ode)\n",
    "      300    0.001    0.000    2.286    0.008 _ode.py:396(integrate)\n",
    "      300    0.033    0.000    2.285    0.008 _ode.py:1030(run)\n",
    "    10249    0.032    0.000    2.252    0.000 round_up.py:96(squeezed_cubic_knn)\n",
    "    10249    0.026    0.000    1.849    0.000 round_up.py:68(__call__)\n",
    "    10249    1.610    0.000    1.817    0.000 function_module.py:725(__call__)\n",
    "      300    0.002    0.000    0.494    0.002 frame.py:4346(append)\n",
    "      300    0.001    0.000    0.492    0.002 merge.py:1396(concat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The file `function_module.py` belongs to `Theano`, there is likely not much to be gained. Still most time is spent calculating the derivatives, but allocating the `DataFrame` for the results could shave another 17% (35% when using a GPU) off in the best case. This would result in a ~50x (100x) faster code than the initial version.\n",
    "\n",
    "This prototype allowed us to simulate a few hundred interacting cells in seconds. However, we were aiming for at least half a million cells and thus built a `CUDA/C++` implementation after three months."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}