Merge pull request #232 from flatironinstitute/doctest

Doctest
flatironinstitute · Oct 2, 2024 · a2d5dcc · a2d5dcc
2 parents 6dcd891 + 9b1bb78
commit a2d5dcc
Show file tree

Hide file tree

Showing 16 changed files with 200 additions and 102 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -99,6 +99,10 @@ Lastly, you should make sure that the existing tests all run successfully and th
 ```bash
 # run tests and make sure they all pass
 pytest tests/
+
+# run doctest (run all examples in docstrings and match output)
+pytest --doctest-modules src/nemos/ 
+
 # format the code base
 black src/
 isort src
@@ -184,38 +188,86 @@ properly documented as outlined below.
 
 #### Adding documentation
 
-1) **Docstrings**
-
-All public-facing functions and classes should have complete docstrings, which start with a one-line short summary of the function, 
-a medium-length description of the function / class and what it does, and a complete description of all arguments and return values. 
-Math should be included in a `Notes` section when necessary to explain what the function is doing, and references to primary literature 
-should be included in a `References` section when appropriate. Docstrings should be relatively short, providing the information necessary 
-for a user to use the code.
-
-Private functions and classes should have sufficient explanation that other developers know what the function / class does and how to use it, 
-but do not need to be as extensive.
-
-We follow the [numpydoc](https://numpydoc.readthedocs.io/en/latest/) conventions for docstring structure.
-
-2) **Examples/Tutorials**
-
-If your changes are significant (add a new functionality or drastically change the current codebase), then the current examples may need to be updated or 
-a new example may need to be added.
-
-All examples live within the `docs/` subfolder of `nemos`.  These are written as `.py` files but are converted to 
-notebooks by [`mkdocs-gallery`](https://smarie.github.io/mkdocs-gallery/), and have a special syntax, as demonstrated in this [example 
-gallery](https://smarie.github.io/mkdocs-gallery/generated/gallery/).
-
-We avoid using `.ipynb` notebooks directly because their JSON-based format makes them difficult to read, interpret, and resolve merge conflicts in version control. 
-
-To see if changes you have made break the current documentation, you can build the documentation locally. 
-
-```bash
-# Clear the cached documentation pages
-# This step is only necessary if your changes affected the src/ directory
-rm -r docs/generated
-# build the docs within the nemos repo
-mkdocs build
-```
-
-If the build fails, you will see line-specific errors that prompted the failure.
+1. **Docstrings**
+
+    All public-facing functions and classes should have complete docstrings, which start with a one-line short summary of the function, a medium-length description of the function/class and what it does, a complete description of all arguments and return values, and an example to illustrate usage. Math should be included in a `Notes` section when necessary to explain what the function is doing, and references to primary literature should be included in a `References` section when appropriate. Docstrings should be relatively short, providing the information necessary for a user to use the code.
+    
+    Private functions and classes should have sufficient explanation that other developers know what the function/class does and how to use it, but do not need to be as extensive.
+    
+    We follow the [numpydoc](https://numpydoc.readthedocs.io/en/latest/) conventions for docstring structure.
+
+2. **Examples/Tutorials**
+
+    If your changes are significant (add a new functionality or drastically change the current codebase), then the current examples may need to be updated or a new example may need to be added.
+    
+    All examples live within the `docs/` subfolder of `nemos`. These are written as `.py` files but are converted to notebooks by [`mkdocs-gallery`](https://smarie.github.io/mkdocs-gallery/), and have a special syntax, as demonstrated in this [example gallery](https://smarie.github.io/mkdocs-gallery/generated/gallery/).
+    
+    We avoid using `.ipynb` notebooks directly because their JSON-based format makes them difficult to read, interpret, and resolve merge conflicts in version control.
+    
+    To see if changes you have made break the current documentation, you can build the documentation locally.
+    
+    ```
+    # Clear the cached documentation pages
+    # This step is only necessary if your changes affected the src/ directory
+    rm -r docs/generated
+    # build the docs within the nemos repo
+    mkdocs build
+    ```
+    
+    If the build fails, you will see line-specific errors that prompted the failure.
+
+3. **Doctest: Test the example code in your docs**
+
+    Doctests are a great way to ensure that code examples in your documentation remain accurate as the codebase evolves. With doctests, we will test any docstrings, Markdown files, or any other text-based documentation that contains code formatted as interactive Python sessions.
+    
+    - **Docstrings:**
+        To include doctests in your function and class docstrings you must add an `Examples` section. The examples should be formatted as if you were typing them into a Python interactive session, with `>>>` used to indicate commands and expected outputs listed immediately below.
+        
+        ```python
+        def add(a, b):
+            """
+            The sum of two numbers.
+                
+            ...Other docstrings sections (Parameters, Returns...)
+                
+            Examples
+            --------
+            An expected output is required.
+            >>> add(1, 2)
+            3
+            
+            Unless the output is captured.
+            >>> out = add(1, 2)
+            
+            """
+            return a + b
+        ```
+      
+        To validate all your docstrings examples, run pytest `--doctest-module` flag,
+        
+        ```
+        pytest --doctest-modules src/nemos/
+        ```
+      
+        This test is part of the Continuous Integration, every example must pass before we can merge a PR.
+    
+    - **Documentation Pages:**
+        Doctests can also be included in Markdown files by using code blocks with the `python` language identifier and interactive Python examples. To enable this functionality, ensure that code blocks follow the standard Python doctest format:
+        
+        ```markdown
+           ```python
+           >>> # Add any code
+           >>> x = 3 ** 2
+           >>> x + 1
+           10
+      
+           ```
+        ```
+      
+        To run doctests on a text file, use the following command:
+        
+        ```
+        python -m doctest -v path-to-your-text-file/file_name.md
+        ```
+        
+        All MarkDown files will be tested as part of the Continuous Integration.
diff --git a/README.md b/README.md
@@ -177,3 +177,8 @@ We communicate via several channels on Github:
 In all cases, we request that you respect our [code of
 conduct](CODE_OF_CONDUCT.md).
 
+## Support
+
+This package is supported by the Center for Computational Neuroscience, in the Flatiron Institute of the Simons Foundation.  
+
+<img src="docs/assets/CCN-logo-wText.png" width="20%" alt="Flatiron Center for Computational Neuroscience logo."> 
diff --git a/docs/assets/CCN-logo-wText.png b/docs/assets/CCN-logo-wText.png
diff --git a/docs/how_to_guide/plot_04_population_glm.py b/docs/how_to_guide/plot_04_population_glm.py
@@ -23,9 +23,10 @@
 """
 
 import jax.numpy as jnp
-import nemos as nmo
-import numpy as np
 import matplotlib.pyplot as plt
+import numpy as np
+
+import nemos as nmo
 
 np.random.seed(123)
 

diff --git a/docs/how_to_guide/plot_05_batch_glm.py b/docs/how_to_guide/plot_05_batch_glm.py
@@ -6,10 +6,11 @@
 
 """
 
+import matplotlib.pyplot as plt
+import numpy as np
 import pynapple as nap
+
 import nemos as nmo
-import numpy as np
-import matplotlib.pyplot as plt
 
 nap.nap_config.suppress_conversion_warnings = True
 

diff --git a/docs/how_to_guide/plot_06_sklearn_pipeline_cv_demo.py b/docs/how_to_guide/plot_06_sklearn_pipeline_cv_demo.py
@@ -71,20 +71,19 @@
 # ## Combining basis transformations and GLM in a pipeline
 # Let's start by creating some toy data.
 
-import nemos as nmo
+import matplotlib.pyplot as plt
 import numpy as np
 import pandas as pd
 import scipy.stats
-import matplotlib.pyplot as plt
 import seaborn as sns
-
-from sklearn.pipeline import Pipeline
 from sklearn.model_selection import GridSearchCV
+from sklearn.pipeline import Pipeline
+
+import nemos as nmo
 
 # some helper plotting functions
 from nemos import _documentation_utils as doc_plots
 
-
 # predictors, shape (n_samples, n_features)
 X = np.random.uniform(low=0, high=1, size=(1000, 1))
 # observed counts, shape (n_samples,)

diff --git a/docs/index.md b/docs/index.md
@@ -103,3 +103,10 @@ We provide a **Poisson GLM** for analyzing spike counts, and a **Gamma GLM** for
 ## :material-scale-balance:{ .lg } License
 
 Open source, [licensed under MIT](https://github.com/flatironinstitute/nemos/blob/main/LICENSE).
+
+
+## Support
+
+This package is supported by the Center for Computational Neuroscience, in the Flatiron Institute of the Simons Foundation.  
+
+<img src="assets/CCN-logo-wText.png" width="20%" alt="Flatiron Center for Computational Neuroscience logo."> 
diff --git a/pyproject.toml b/pyproject.toml
@@ -50,6 +50,10 @@ dev = [
     "pytest-cov",                   # Test coverage plugin for pytest
     "statsmodels",                  # Used to compare model pseudo-r2 in testing
     "scikit-learn",                 # Testing compatibility with CV & pipelines
+    "matplotlib>=3.7",              # Needed by doctest to run docstrings examples
+    "pooch",                        # Required by doctest for fetch module
+    "dandi",                        # Required by doctest for fetch module
+    "seaborn",                      # Required by doctest for _documentation_utils module
 ]
 docs = [
     "mkdocs",                       # Documentation generator
@@ -112,7 +116,7 @@ testpaths = ["tests"]             # Specify the directory where test files are l
 [tool.coverage.run]
 omit = [
     "src/nemos/fetch/*",
-    "src/nemos/_documentation_utils/*"
+    "src/nemos/_documentation_utils/*",
 ]
 
 [tool.coverage.report]

diff --git a/src/nemos/basis.py b/src/nemos/basis.py
@@ -152,15 +152,15 @@ class TransformerBasis:
     >>> # transformer can be used in pipelines
     >>> transformer = TransformerBasis(basis)
     >>> pipeline = Pipeline([ ("compute_features", transformer), ("glm", GLM()),])
-    >>> pipeline.fit(x[:, None], y)  # x need to be 2D for sklearn transformer API
-    >>> print(pipeline.predict(np.random.normal(size=(10, 1))))  # predict rate from new data
-
+    >>> pipeline = pipeline.fit(x[:, None], y)  # x need to be 2D for sklearn transformer API
+    >>> out = pipeline.predict(np.arange(10)[:, None]) # predict rate from new datas
     >>> # TransformerBasis parameter can be cross-validated.
     >>> # 5-fold cross-validate the number of basis
     >>> param_grid = dict(compute_features__n_basis_funcs=[4, 10])
     >>> grid_cv = GridSearchCV(pipeline, param_grid, cv=5)
-    >>> grid_cv.fit(x[:, None], y)
+    >>> grid_cv = grid_cv.fit(x[:, None], y)
     >>> print("Cross-validated number of basis:", grid_cv.best_params_)
+    Cross-validated number of basis: {'compute_features__n_basis_funcs': 10}
     """
 
     def __init__(self, basis: Basis):
@@ -289,7 +289,7 @@ def __getattr__(self, name: str):
         return getattr(self._basis, name)
 
     def __setattr__(self, name: str, value) -> None:
-        """
+        r"""
         Allow setting _basis or the attributes of _basis with a convenient dot assignment syntax.
 
         Setting any other attribute is not allowed.
@@ -312,10 +312,11 @@ def __setattr__(self, name: str, value) -> None:
         >>> # allowed
         >>> trans_bas.n_basis_funcs = 20
         >>> # not allowed
-        >>> tran_bas.random_attribute_name = "some value"
-        Traceback (most recent call last):
-        ...
-        ValueError: Only setting _basis or existing attributes of _basis is allowed.
+        >>> try:
+        ...     trans_bas.random_attribute_name = "some value"
+        ... except ValueError as e:
+        ...     print(repr(e))
+        ValueError('Only setting _basis or existing attributes of _basis is allowed.')
         """
         # allow self._basis = basis
         if name == "_basis":
@@ -343,7 +344,7 @@ def __sklearn_clone__(self) -> TransformerBasis:
         return cloned_obj
 
     def set_params(self, **parameters) -> TransformerBasis:
-        """
+        r"""
         Set TransformerBasis parameters.
 
         When used with `sklearn.model_selection`, users can set either the `_basis` attribute directly
@@ -357,12 +358,16 @@ def set_params(self, **parameters) -> TransformerBasis:
 
         >>> # setting parameters of _basis is allowed
         >>> print(transformer_basis.set_params(n_basis_funcs=8).n_basis_funcs)
-
+        8
         >>> # setting _basis directly is allowed
-        >>> print(transformer_basis.set_params(_basis=BSplineBasis(10))._basis)
-
+        >>> print(type(transformer_basis.set_params(_basis=BSplineBasis(10))._basis))
+        <class 'nemos.basis.BSplineBasis'>
         >>> # mixing is not allowed, this will raise an exception
-        >>> transformer_basis.set_params(_basis=BSplineBasis(10), n_basis_funcs=2)
+        >>> try:
+        ...     transformer_basis.set_params(_basis=BSplineBasis(10), n_basis_funcs=2)
+        ... except ValueError as e:
+        ...     print(repr(e))
+        ValueError('Set either new _basis object or parameters for existing _basis, not both.')
         """
         new_basis = parameters.pop("_basis", None)
         if new_basis is not None:
@@ -996,8 +1001,8 @@ def to_transformer(self) -> TransformerBasis:
         >>> from sklearn.pipeline import Pipeline
         >>> from sklearn.model_selection import GridSearchCV
         >>> # load some data
-        >>> X, y = ...  # X: features, y: neural activity
-        >>> basis = nmo.basis.RaisedCosineBasisLinear(10)
+        >>> X, y = np.random.normal(size=(30, 1)), np.random.poisson(size=30)
+        >>> basis = nmo.basis.RaisedCosineBasisLinear(10).to_transformer()
         >>> glm = nmo.glm.GLM(regularizer="Ridge")
         >>> pipeline = Pipeline([("basis", basis), ("glm", glm)])
         >>> param_grid = dict(
@@ -1009,7 +1014,7 @@ def to_transformer(self) -> TransformerBasis:
         ...     param_grid=param_grid,
         ...     cv=5,
         ... )
-        >>> gridsearch.fit(X, y)
+        >>> gridsearch = gridsearch.fit(X, y)
         """
 
         return TransformerBasis(copy.deepcopy(self))
@@ -1346,7 +1351,7 @@ def _check_n_basis_min(self) -> None:
 
 
 class MSplineBasis(SplineBasis):
-    r"""
+    """
     M-spline[$^{[1]}$](#references) basis functions for modeling and data transformation.
 
     M-splines are a type of spline basis function used for smooth curve fitting
@@ -1502,12 +1507,14 @@ def evaluate_on_grid(self, n_samples: int) -> Tuple[NDArray, NDArray]:
         >>> mspline_basis = MSplineBasis(n_basis_funcs=4, order=3)
         >>> sample_points, basis_values = mspline_basis.evaluate_on_grid(100)
         >>> for i in range(4):
-        ...     plt.plot(sample_points, basis_values[:, i], label=f'Function {i+1}')
+        ...     p = plt.plot(sample_points, basis_values[:, i], label=f'Function {i+1}')
         >>> plt.title('M-Spline Basis Functions')
+        Text(0.5, 1.0, 'M-Spline Basis Functions')
         >>> plt.xlabel('Domain')
+        Text(0.5, 0, 'Domain')
         >>> plt.ylabel('Basis Function Value')
-        >>> plt.legend()
-        >>> plt.show()
+        Text(0, 0.5, 'Basis Function Value')
+        >>> l = plt.legend()
         """
         return super().evaluate_on_grid(n_samples)
 

diff --git a/src/nemos/exceptions.py b/src/nemos/exceptions.py
@@ -15,6 +15,5 @@ class NotFittedError(ValueError, AttributeError):
     ...     GLM().predict([[[1, 2], [2, 3], [3, 4]]])
     ... except NotFittedError as e:
     ...     print(repr(e))
-    ... # NotFittedError("This GLM instance is not fitted yet. Call 'fit' with
-    ... # appropriate arguments.")
+    NotFittedError("This GLM instance is not fitted yet. Call 'fit' with appropriate arguments.")
     """
diff --git a/src/nemos/fetch/fetch_data.py b/src/nemos/fetch/fetch_data.py
@@ -143,10 +143,21 @@ def download_dandi_data(dandiset_id: str, filepath: str) -> NWBHDF5IO:
     Examples
     --------
     >>> import nemos as nmo
+    >>> import pynapple as nap
     >>> io = nmo.fetch.download_dandi_data("000582",
-                                           "sub-11265/sub-11265_ses-07020602_behavior+ecephys.nwb")
+    ...                                    "sub-11265/sub-11265_ses-07020602_behavior+ecephys.nwb")
     >>> nwb = nap.NWBFile(io.read(), lazy_loading=False)
     >>> print(nwb)
+    07020602
+    ┍━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━┑
+    │ Keys                │ Type     │
+    ┝━━━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━┥
+    │ units               │ TsGroup  │
+    │ ElectricalSeriesLFP │ Tsd      │
+    │ SpatialSeriesLED2   │ TsdFrame │
+    │ SpatialSeriesLED1   │ TsdFrame │
+    │ ElectricalSeries    │ Tsd      │
+    ┕━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━┙
 
     """
     if dandi is None: