Documentation upgrades (#352)

## Description

<!-- Provide a brief description of the PR's purpose here. -->

A bunch of tiny errors and upgrades in the documentation.

## TODO

<!-- Notable points that this PR has either accomplished or will
accomplish. -->

- [x]  Miscellaneous typos
- [x]  Remove obsolete references to `initialize` method in CVTArchive
- [x] Make parallel_axes_plot take shorter to run — before, this example
was taking very long to generate its plot
- [x] Rearrange tutorials page so that tutorials show up properly in the
sidebar — this involved making the globaltoc depth be 3 in docs/conf.py
- [x] Add a features section on tutorials page — in the future, we plan
to have a couple such tutorials
- [x]  Update CONTRIBUTING for tutorials — added some new details
- [x] Don't wait to open browser when reloading docs with
sphinx-autobuild
- [x] Note * syntax in constructor args — The * syntax to force keyword
arguments can be confusing. We now include a note in the lunar lander
tutorial about it.

## Questions

<!-- Any concerns or points of confusion? -->

## Status

- [x] I have read the guidelines in
[CONTRIBUTING.md](https://github.com/icaros-usc/pyribs/blob/master/CONTRIBUTING.md)
- [x] I have formatted my code using `yapf`
- [x] I have tested my code by running `pytest`
- [x] I have linted my code with `pylint`
- [ ] I have added a one-line description of my change to the changelog
in `HISTORY.md`
- [x] This PR is ready to go

CONTRIBUTING.md

@@ -118,7 +118,8 @@ to the docs.
 Tutorials are created in Jupyter notebooks that are stored under `tutorials/` in
 the repo. To create a tutorial:
 
-1. Write the notebook and save it under `tutorials/`.
+1. Write the notebook and save it under `tutorials/`. Notebooks may also be
+   saved in a subdirectory of `tutorials/`, e.g., `tutorials/features`.
 1. Use cell magic (e.g. `%pip install ribs`) to install pyribs and other
    dependencies.
    - Installation cells tend to produce a lot of output. Make sure to clear this
@@ -134,7 +135,9 @@ the repo. To create a tutorial:
    `## Level 2 Heading`) or higher.
 1. If linking to the pyribs documentation, make sure to link to pages in the
    `latest` version on ReadTheDocs, i.e. your links should start with
-   `https://docs.pyribs.org/en/latest/`
+   `https://docs.pyribs.org/en/latest/`. Note that we do not use Sphinx autodoc
+   (e.g., `:class:`) in the tutorials because such links do not work outside the
+   pyribs website (e.g., on Google Colab).
 1. Add an entry into the toctree in `docs/tutorials.md` and add it to one of the
    lists of tutorials.
 1. Check that the tutorial shows up on the Tutorials page when serving the docs.

Makefile

@@ -80,6 +80,7 @@ servedocs: ## compile the docs watching for changes
 		--watch ribs/ \
 		--watch examples/ \
 		--watch tutorials/ \
+		--delay 0 \
 		docs/ \
 		docs/_build/html
 .PHONY: servedocs

docs/conf.py

@@ -144,7 +144,8 @@
     "css_minify": not DEV_MODE,
     #  "logo_icon": "&#xe869",
     "repo_type": "github",
-    "globaltoc_depth": 2,
+    # Needs to be 3 so that tutorials show up in sidebar.
+    "globaltoc_depth": 3,
     "color_primary": "deep-purple",
     "color_accent": "purple",
     "touch_icon": None,

docs/tutorials.md

@@ -1,23 +1,20 @@
 # Tutorials
 
+Tutorials are Python notebooks with detailed explanations of pyribs usage. They
+may be [run locally](running-locally) or on
+[Google Colab](https://colab.research.google.com/notebooks/intro.ipynb). Each
+tutorial page has a link to open the tutorial in Google Colab.
+
+## Key Algorithms
+
 ```{toctree}
 :hidden:
 
 tutorials/lunar_lander
 tutorials/cma_mae
 tutorials/tom_cruise_dqd
-tutorials/lsi_mnist
-tutorials/arm_repertoire
-tutorials/fooling_mnist
 ```
 
-Tutorials are Python notebooks with detailed explanations of pyribs usage. They
-may be [run locally](running-locally) or on
-[Google Colab](https://colab.research.google.com/notebooks/intro.ipynb). Each
-tutorial page has a link to open the tutorial in Google Colab.
-
-## Key Algorithms
-
 We recommend new users start with these tutorials which demonstrate how to use
 the key algorithms in pyribs.
 
@@ -31,6 +28,14 @@ the key algorithms in pyribs.
 
 ## Applications
 
+```{toctree}
+:hidden:
+
+tutorials/lsi_mnist
+tutorials/arm_repertoire
+tutorials/fooling_mnist
+```
+
 The following tutorials show how pyribs can implement a variety of algorithms
 for different problems.
 
@@ -45,6 +50,19 @@ for different problems.
 
 (running-locally)=
 
+## Features
+
+Finally, these tutorials provide a closer look at some of the features of
+pyribs.
+
+```{toctree}
+:hidden:
+
+tutorials/features/example
+```
+
+- {doc}`tutorials/features/example`: Placeholder for upcoming tutorials!
+
 ## Running Locally
 
 If you would like to run the tutorials locally, follow these instructions:

ribs/archives/_cvt_archive.py

@@ -1,6 +1,5 @@
 """Contains the CVTArchive class."""
 import numpy as np
-import sklearn
 from scipy.spatial import cKDTree  # pylint: disable=no-name-in-module
 from sklearn.cluster import k_means
 
@@ -218,16 +217,14 @@ def samples(self):
         """(num_samples, measure_dim) numpy.ndarray: The samples used in
         creating the CVT.
 
-        May be None until :meth:`initialize` is called.
+        Will be None if custom centroids were passed in to the archive.
         """
         return self._samples
 
     @property
     def centroids(self):
         """(n_centroids, measure_dim) numpy.ndarray: The centroids used in the
         CVT.
-
-        None until :meth:`initialize` is called.
         """
         return self._centroids
 

ribs/emitters/_emitter_base.py

@@ -11,7 +11,7 @@ class EmitterBase(ABC):
     and a :meth:`tell` method that inserts solutions into the emitter's archive.
     Child classes are only required to override :meth:`ask`.
 
-    DQD emitter should override :meth:`ask_dqd` and :meth:`tell_dqd` methods.
+    DQD emitters should override :meth:`ask_dqd` and :meth:`tell_dqd` methods.
 
     Args:
         archive (ribs.archives.ArchiveBase): An archive to use when creating and

ribs/visualize.py

@@ -633,9 +633,9 @@ def parallel_axes_plot(archive,
             ...               ranges=[(-1, 1), (-1, 1), (-1, 1),
             ...                       (-1, 1), (-1, 1)],
             ...           )
-            >>> for x in np.linspace(-1, 1, 100):
-            ...     for y in np.linspace(0, 1, 100):
-            ...         for z in np.linspace(-1, 1, 100):
+            >>> for x in np.linspace(-1, 1, 10):
+            ...     for y in np.linspace(0, 1, 10):
+            ...         for z in np.linspace(-1, 1, 10):
             ...             archive.add_single(
             ...                 solution=np.array([x,y,z]),
             ...                 objective=-(x**2 + y**2 + z**2),

tests/tutorials.sh

@@ -61,7 +61,7 @@ function test_notebook {
 }
 
 if [ -z "$1" ]; then
-  TUTORIALS=($(ls tutorials/*.ipynb))  # Contains all notebooks.
+  TUTORIALS=($(ls tutorials/*.ipynb tutorials/*/*.ipynb))  # Contains all notebooks.
   for t in "${TUTORIALS[@]}"; do
     # Notebooks to exclude.
     case "$t" in

tutorials/features/example.ipynb

@@ -0,0 +1,35 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "6b5ef6e6-ed00-4f5a-bd13-89064b1482b0",
+   "metadata": {},
+   "source": [
+    "# Example Feature\n",
+    "\n",
+    "Feature tutorials coming soon!"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.17"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

tutorials/lunar_lander.ipynb

@@ -372,7 +372,11 @@
    "source": [
     "**Note: QD Score Offset**\n",
     "\n",
-    "> Above, we specified `qd_score_offset=-600` when initializing the archive. The QD score ([Pugh 2016](https://doi.org/10.3389/frobt.2016.00040)) is a metric for QD algorithms which sums the objective values of all elites in the archive. However, if objectives can be negative, this metric will penalize an algorithm for discovering new cells with a negative objective. To prevent this, it is common to normalize each objective to be non-negative by subtracting an offset, typically the minimum objective, before computing the QD score. While lunar lander does not have a predefined minimum objective, we know from previous experiments that almost all solutions score above -600, so we have set the offset accordingly. Thus, if a solution has, for example, an objective of -300, then its objective will be normalized to -300 - (-600) = 300."
+    "> Above, we specified `qd_score_offset=-600` when initializing the archive. The QD score ([Pugh 2016](https://doi.org/10.3389/frobt.2016.00040)) is a metric for QD algorithms which sums the objective values of all elites in the archive. However, if objectives can be negative, this metric will penalize an algorithm for discovering new cells with a negative objective. To prevent this, it is common to normalize each objective to be non-negative by subtracting an offset, typically the minimum objective, before computing the QD score. While lunar lander does not have a predefined minimum objective, we know from previous experiments that almost all solutions score above -600, so we have set the offset accordingly. Thus, if a solution has, for example, an objective of -300, then its objective will be normalized to -300 - (-600) = 300.\n",
+    "\n",
+    "**Note: Keyword Arguments**\n",
+    "\n",
+    "> In pyribs, most constructor arguments must be keyword arguments. For example, the `GridArchive` constructor looks like `GridArchive(*, solution_dim, ...)`. Notice the `*` in the parameter list. This `*` syntax indicates that any parameters listed after it must be passed as _keyword arguments_, e.g., `GridArchive(solution_dim=initial_model.size, ...)` as is done above. Furthermore, passing a _positional argument_ like `GridArchive(10, ...)` would throw a `TypeError`. We adopt this practice from [scikit-learn](https://scikit-learn-enhancement-proposals.readthedocs.io/en/latest/slep009/proposal.html) in an effort to increase code readability."
    ]
   },
   {
@@ -1127,7 +1131,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.8.17"
   }
  },
  "nbformat": 4,