From f5cb8f1ffab3fc132e1c819c97377d88723bac9c Mon Sep 17 00:00:00 2001
From: Fiona Naughton <fiona@mdanalysis.org>
Date: Wed, 17 Jan 2024 07:47:17 -0800
Subject: [PATCH 1/5] Create reviewersguide.rst

---
 docs/source/reviewersguide.rst | 265 +++++++++++++++++++++++++++++++++
 1 file changed, 265 insertions(+)
 create mode 100644 docs/source/reviewersguide.rst

diff --git a/docs/source/reviewersguide.rst b/docs/source/reviewersguide.rst
new file mode 100644
index 00000000..1c291ad7
--- /dev/null
+++ b/docs/source/reviewersguide.rst
@@ -0,0 +1,265 @@
+*******************
+Guide for reviewers
+*******************
+
+The MDAKit registration process aims to present a *low barrier of entry* for contributors to make their
+code available to the broader community via the `MDAKit registry`_. 
+However, there is still a minimal set of :ref:`requirements <requirements>` that must be met, and the
+MDAKit must pass a review before it is officially included in the Registry.
+
+A manual review also allows for more experienced reviewers to offer best-practice suggestions and
+improvements to newer contributors. As such, your **aims of a reviewer** are to:
+
+#. Ensure the MDAKit can be *merged* (all necessary details *present* and *correct*), and
+#. Encourage *best practices* and *future work*.
+                                      
+|
+This page provides a guide for reviewing an MDAKit submission, including a brief overview of the 
+registration process and:
+                                      
+* what **AUTOMATIC** checks are performed,
+* what to **CHECK** manually (things that must be present before the kit can be registered), and 
+* things to **RECOMMEND** (suggestions for improving the provided metadata, but that are not required).
+You can also see the :ref:`guide for submitters here<add-mdakit>`.
+
+.. note::   
+    The `MDAKit registry`_ is still evolving. This guide will be updated as changes to the registration 
+    process are made.
+
+Who can be a reviewer?
+======================
+We love for members of the community to get involved at all parts of the MDAKit process! Contact us if
+you’d like to be involved with reviewing MDAKit submissions.
+
+How does registration work?
+===========================
+A prospective MDAKit contributor will register their Kit by creating a Pull Request (PR) to the MDAKit
+github repository, consisting of a single `metadata.yaml` file (or one per Kit, if multiple Kits are
+being added/modified). Details on what is included in the metadata file, and what to look for as a 
+reviewer, are provided :ref:`below <metadatafile>`.
+
+Several checks will run automatically:
+                                      
+- *gen_matrix* generates a list of all MDAKits updated by the PR (usually, this will be a single Kit)
+- *mdakit-ci* will install the MDAKit and run its own tests (as detailed in
+  the `metadata.yaml` file) with the latest and/or develop versions of MDAnalysis
+- *readthedocs* will create a version of the MDAKit Registry website with the prospective kit added
+                                      
+Once the kit has passed both these automatic checks **and** the manual checks below, you can approve the
+PR - after merging, the Kit will officially be part of the MDAKit Registry!
+
+What to look for as a reviewer
+==============================
+
+On the PR
+*********
+                                      
+- **CHECK**: Are the automatic checks all passing?       
+                                      
+  - You many need to manually start the checks if the contributor is new to the organisation
+  - If the Kits’ tests use `tox`, these need to be manually checked
+
+  If checks are failing, you may need to help the contributor identify and fix the issue. A failure may 
+  be due to improper set up of `metadata.yaml` (see :ref:`below <metadatafile>`), or a local failure on
+  the MDAKit’s end. Follow the *Details* of the failed check to find out more. 
+                                      
+  Some other possible points of failure:
+                                      
+  - If the failure has a “the head commit is ahead of the base commit” error, tell the contributor to 
+    update the branch used in the PR so it is up-to-date with *main*
+  - A failure on installing dependencies may be due the things in the Kit’s `pyproject.toml` not
+    being properly defined
+
+|                                      
+- **CHECK**: Did the docs build correctly?
+                                      
+  View the page generated by `readthedocs` (by clicking *Details* next to the check) and find the 
+  MDAKit’s entry on the Registry page. Check the expected information and badges are present on both the
+  main list and the Kit’s individual page.
+
+|
+- **CHECK**: Is the metadata file named correctly and in the right location?
+                                      
+  The correct format is: `mdakits/<project_name>/metadata.yaml` (see `project_name`, below)
+
+|                                      
+At the MDAKit’s Project Home
+*****************************
+Follow the link to the project’s home provided in the metadata file and have a quick look at the MDAKit
+to see if it looks sensible. Much of the information provided in the metadata file should also be 
+available here (e.g. a LICENCE file containing licence information, installation instructions, etc). 
+Check these details match the metadata information.
+
+| 
+.. _metadatafile:
+Inside the metadata file
+************************
+                                      
+The metadata file is in `YAML format`_. Each entry is described briefly below; see also the 
+:ref:`template metadata file <template>` for more details and a demonstration of the
+proper formatting. In short, current metadata entries take the form of either a *string*, e.g.:
+                                      
+.. code-block:: yaml
+                                      
+    project_name: MDAKitForCats
+    description: 
+        A hypothetical and nonsensical MDAKit designed
+        to be used by cats.
+                                      
+or a *list of strings* (which may be only one item long):
+                                      
+.. code-block:: yaml
+                                      
+    instructions:
+      - obtain boxed MDAKit
+      - remove MDAKit contents
+      - sit in empty box
+
+                                      
+Entries *required* for registration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+                                      
+`project_name`: the name of the Kit
+                                      
+- **CHECK**: This is a *string* and must match the name of the directory the `metadata.yaml` file is placed in.
+- **RECOMMEND**: It is suggested that this name also matches the host repository name.
+
+|
+`authors`: the ‘creators’ of the Kit
+                                      
+- **CHECK**: This is a *list of strings*; either a list of names, or single entry with a link to an
+  AUTHORS file at the project’s home.
+- **RECOMMEND**: An AUTHORS file is preferred (as this will be easier to update).
+
+|
+`maintainers`: the individuals responsible for the Kit going forward; they will be pinged if the MDAKit 
+is failing
+                                      
+- **CHECK**: This is a *list of strings*. Each entry must be a github handle.
+- **RECOMMEND**: It’s expected the submitter will appear on this list. If the list contains individuals
+  not obviously associated with the submission/project, ping them to check their agreement to be 
+  included.
+
+|
+`description`: a free form description of the Kit
+
+- **CHECK**: This is a *string*. Give the Kit a quick look - is the description reasonable?
+- **RECOMMEND**: Suggest anything you think would be useful to add to the Kit’s description. There’s no 
+  upper limit on length, but ideally ~1-3 sentences should be sufficient.
+
+|
+`keywords`: keywords that describe the Kit
+
+- **CHECK**: This is a *list of strings*. 
+- **RECOMMEND**: Make any suggestions for things you think would be useful to add. See what
+  keywords current MDAKits use for examples (note that keywords are case-insensitive when searching). 
+
+|
+`licence`: the licence that the Kit falls under
+
+- **CHECK**: This is a *string*, which must be the SPDX ID of an 
+  `OSI approved licence <https://opensource.org/licenses/>`_. It should match the licence identified 
+  on the project’s home, e.g. in a LICENCE file.
+
+|
+`project_home`: a link to the Kit’s code
+
+- **CHECK**: This is a *string* and points to a reasonable location on a version-controlled repository
+  e.g. GitHub, GitLab, BitBucket, etc.
+
+|
+`documentation_home`: a link to the Kit’s documentation
+
+- **CHECK**: This is a *string* and points somewhere sensible, which could be a single file (e.g. a 
+  README), or a website. Minimal documentation is a requirement for an MDAKit: does the linked 
+  documentation detail what the code does, how to install it, and the basic usage?
+- **RECOMMEND**: While only basic documentation is required for registration, you can encourage the 
+  contributor to expand and improve their documentation in the future.
+
+|
+`documentation_type`: the type (i.e. “level of detail”) of documentation
+
+- **CHECK**: This is a *string* - e.g. 'README' (a basic overview), 'API' (description of the code) or
+  'UserGuide' (more thorough description and explanation of usage); or a combination ('API + UserGuide').
+- **RECOMMEND**: It is not strictly enforced for the “type” to match the current appearance of a Kit’s
+  docs. If you judge that it does not, see if the submitter intends to continue working on these (and 
+  encourage them to do so!)
+
+|
+*Optional* entries 
+~~~~~~~~~~~~~~~~~~
+
+These metadata entries are *optional*. Encourage the submitter to include them, but don’t block merging
+the PR over them. Many of these are tested by the automatic CI, so do not need to be checked manually once
+CI is passing.
+
+`install`: a list of commands to install the latest release of the Kit. This is a *list of strings* (*AUTOMATIC CHECK*).
+
+- **RECOMMEND**: If the installation uses e.g. github or is otherwise complicated (many steps involved),
+  encourage the contributor to make a release on conda-forge or PyPI. 
+
+|
+`src_install`: a list of commands to install the Kit from the source code. This is a *list of strings*
+(*AUTOMATIC CHECK*).
+
+|
+`import_name`: the package name, used to import the Kit in Python. This is a *string* (*AUTOMATIC CHECK*).
+
+|
+`python_requires`: range specifications for the versions of Python this Kit supports, e.g. “>=3.9”. This
+is a *string* (*AUTOMATIC CHECK*).
+
+|
+`mdanalysis_requires`: range specifications for the versions of MDAnalysis this Kit supports, e.g.
+“>=2.0.0”. This is a *string* (*AUTOMATIC CHECK*).
+                                      
+- **CHECK**: The automatic checks will test the upper bound provided, but not the lower bound. If 
+  provided, see if the lower bound seems reasonable - e.g. a newly-written Kit is likely to not actually 
+  work with early versions of MDAnalysis.
+- **RECOMMEND**: Ideally, the Kit works with the current version of MDAnalysis - if an upper bound to an
+  old version is given, enquire why, and recommend updating the Kit to work with a current version. 
+
+|
+`run_tests`: a list of commands to run the Kit’s tests. This is a *list of strings* (*AUTOMATIC CHECK*).
+
+- *note*: while (minimal) tests are one of the requirements of an MDAKit, providing instructions on how to run
+  tests in the metadata file is currently optional, in order to allow greater flexibility in
+  what format tests take and so lower the entry barrier for new contributors. However, it is *highly
+  recommended* here to provide this metadata.
+- **RECOMMEND**: While a MDAKit may be registered with only minimal tests, encourage the contributor 
+  to continue improving their tests in the future.
+                                      
+|
+`test_dependencies`: a list of commands for installing any dependencies required by the MDAKit’s tests.
+This is a *list of strings* (*AUTOMATIC CHECK*).
+
+|
+`project_org`: the account under which the code is found - this may be an individual user account, or an
+organisation like MDAnalysis. This is a *string*.
+
+|
+`development_status`: the development status of the MDAKit.
+
+- **CHECK**: This is a *string* and should match one of the `PyPI classifiers`_. 
+- **RECOMMEND**: If you don’t think the provided status matches the actual state of the Kit’s code, you 
+  can query this - but don’t let it be a blocker.
+
+|
+`publications`: list of publications to be cited when using this MDAKit.
+
+- **CHECK**: This is a *list of strings*, and should include any relevant publications for the Kit 
+  itself as well as key upstream publications (e.g. if the Kit heavily relies on another package with an
+  associated publication).
+
+|
+`changelog`: a link to the MDAKit’s changelog.
+
+- **CHECK**: This is a *string*. If included, check it points to a sensible place (e.g. a CHANGELOG 
+  file).
+
+
+.. _`MDAKit registry`: https://mdakits.mdanalysis.org/mdakits.html
+
+.. _YAML format: https://yaml.org/
+
+.. _PyPI classifiers: https://pypi.org/classifiers/

From 7792d019a052a326d3f126940af4a46fa2dd5b72 Mon Sep 17 00:00:00 2001
From: Fiona Naughton <fiona@mdanalysis.org>
Date: Wed, 17 Jan 2024 07:51:11 -0800
Subject: [PATCH 2/5] Update index.rst

---
 docs/source/index.rst | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/docs/source/index.rst b/docs/source/index.rst
index 517aa181..0f878401 100644
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@@ -11,16 +11,24 @@ Welcome to the MDAKit registry!
 Showcased here are a list of community generated packages which extend the
 functionality of the `MDAnalysis library`_
 
+.. include:: mdakits
 
 .. toctree::
    :maxdepth: 1
-   :caption: Table of Contents
+   :hidden:
 
    mdakits
+
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Resources
+
    about
    makingakit
    add
    troubleshooting
+   reviewersguide
    
 
 Acknowledgements

From 90452eb6aacccbd059f5c6c3fe4c7d16db3ae4a3 Mon Sep 17 00:00:00 2001
From: Fiona Naughton <fiona@mdanalysis.org>
Date: Wed, 17 Jan 2024 08:12:35 -0800
Subject: [PATCH 3/5] Update index.rst

---
 docs/source/index.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/source/index.rst b/docs/source/index.rst
index 0f878401..400ea076 100644
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@@ -11,11 +11,11 @@ Welcome to the MDAKit registry!
 Showcased here are a list of community generated packages which extend the
 functionality of the `MDAnalysis library`_
 
-.. include:: mdakits
 
 .. toctree::
    :maxdepth: 1
    :hidden:
+   :caption: MDAKit Registry
 
    mdakits
 

From f1e99ed9355e0abb86083d948b92e947c36a8692 Mon Sep 17 00:00:00 2001
From: Fiona Naughton <fiona@mdanalysis.org>
Date: Wed, 17 Jan 2024 08:14:08 -0800
Subject: [PATCH 4/5] fix formatting

---
 docs/source/reviewersguide.rst | 46 +++++++++++++++++-----------------
 1 file changed, 23 insertions(+), 23 deletions(-)

diff --git a/docs/source/reviewersguide.rst b/docs/source/reviewersguide.rst
index 1c291ad7..f39fc4f3 100644
--- a/docs/source/reviewersguide.rst
+++ b/docs/source/reviewersguide.rst
@@ -13,7 +13,7 @@ improvements to newer contributors. As such, your **aims of a reviewer** are to:
 #. Ensure the MDAKit can be *merged* (all necessary details *present* and *correct*), and
 #. Encourage *best practices* and *future work*.
                                       
-|
+
 This page provides a guide for reviewing an MDAKit submission, including a brief overview of the 
 registration process and:
                                       
@@ -70,19 +70,19 @@ On the PR
   - A failure on installing dependencies may be due the things in the Kit’s `pyproject.toml` not
     being properly defined
 
-|                                      
+
 - **CHECK**: Did the docs build correctly?
                                       
   View the page generated by `readthedocs` (by clicking *Details* next to the check) and find the 
   MDAKit’s entry on the Registry page. Check the expected information and badges are present on both the
   main list and the Kit’s individual page.
 
-|
+
 - **CHECK**: Is the metadata file named correctly and in the right location?
                                       
   The correct format is: `mdakits/<project_name>/metadata.yaml` (see `project_name`, below)
 
-|                                      
+
 At the MDAKit’s Project Home
 *****************************
 Follow the link to the project’s home provided in the metadata file and have a quick look at the MDAKit
@@ -124,14 +124,14 @@ Entries *required* for registration
 - **CHECK**: This is a *string* and must match the name of the directory the `metadata.yaml` file is placed in.
 - **RECOMMEND**: It is suggested that this name also matches the host repository name.
 
-|
+
 `authors`: the ‘creators’ of the Kit
                                       
 - **CHECK**: This is a *list of strings*; either a list of names, or single entry with a link to an
   AUTHORS file at the project’s home.
 - **RECOMMEND**: An AUTHORS file is preferred (as this will be easier to update).
 
-|
+
 `maintainers`: the individuals responsible for the Kit going forward; they will be pinged if the MDAKit 
 is failing
                                       
@@ -140,34 +140,34 @@ is failing
   not obviously associated with the submission/project, ping them to check their agreement to be 
   included.
 
-|
+
 `description`: a free form description of the Kit
 
 - **CHECK**: This is a *string*. Give the Kit a quick look - is the description reasonable?
 - **RECOMMEND**: Suggest anything you think would be useful to add to the Kit’s description. There’s no 
   upper limit on length, but ideally ~1-3 sentences should be sufficient.
 
-|
+
 `keywords`: keywords that describe the Kit
 
 - **CHECK**: This is a *list of strings*. 
 - **RECOMMEND**: Make any suggestions for things you think would be useful to add. See what
   keywords current MDAKits use for examples (note that keywords are case-insensitive when searching). 
 
-|
+
 `licence`: the licence that the Kit falls under
 
 - **CHECK**: This is a *string*, which must be the SPDX ID of an 
   `OSI approved licence <https://opensource.org/licenses/>`_. It should match the licence identified 
   on the project’s home, e.g. in a LICENCE file.
 
-|
+
 `project_home`: a link to the Kit’s code
 
 - **CHECK**: This is a *string* and points to a reasonable location on a version-controlled repository
   e.g. GitHub, GitLab, BitBucket, etc.
 
-|
+
 `documentation_home`: a link to the Kit’s documentation
 
 - **CHECK**: This is a *string* and points somewhere sensible, which could be a single file (e.g. a 
@@ -176,7 +176,7 @@ is failing
 - **RECOMMEND**: While only basic documentation is required for registration, you can encourage the 
   contributor to expand and improve their documentation in the future.
 
-|
+
 `documentation_type`: the type (i.e. “level of detail”) of documentation
 
 - **CHECK**: This is a *string* - e.g. 'README' (a basic overview), 'API' (description of the code) or
@@ -185,7 +185,7 @@ is failing
   docs. If you judge that it does not, see if the submitter intends to continue working on these (and 
   encourage them to do so!)
 
-|
+
 *Optional* entries 
 ~~~~~~~~~~~~~~~~~~
 
@@ -198,18 +198,18 @@ CI is passing.
 - **RECOMMEND**: If the installation uses e.g. github or is otherwise complicated (many steps involved),
   encourage the contributor to make a release on conda-forge or PyPI. 
 
-|
+
 `src_install`: a list of commands to install the Kit from the source code. This is a *list of strings*
 (*AUTOMATIC CHECK*).
 
-|
+
 `import_name`: the package name, used to import the Kit in Python. This is a *string* (*AUTOMATIC CHECK*).
 
-|
+
 `python_requires`: range specifications for the versions of Python this Kit supports, e.g. “>=3.9”. This
 is a *string* (*AUTOMATIC CHECK*).
 
-|
+
 `mdanalysis_requires`: range specifications for the versions of MDAnalysis this Kit supports, e.g.
 “>=2.0.0”. This is a *string* (*AUTOMATIC CHECK*).
                                       
@@ -219,7 +219,7 @@ is a *string* (*AUTOMATIC CHECK*).
 - **RECOMMEND**: Ideally, the Kit works with the current version of MDAnalysis - if an upper bound to an
   old version is given, enquire why, and recommend updating the Kit to work with a current version. 
 
-|
+
 `run_tests`: a list of commands to run the Kit’s tests. This is a *list of strings* (*AUTOMATIC CHECK*).
 
 - *note*: while (minimal) tests are one of the requirements of an MDAKit, providing instructions on how to run
@@ -229,29 +229,29 @@ is a *string* (*AUTOMATIC CHECK*).
 - **RECOMMEND**: While a MDAKit may be registered with only minimal tests, encourage the contributor 
   to continue improving their tests in the future.
                                       
-|
+
 `test_dependencies`: a list of commands for installing any dependencies required by the MDAKit’s tests.
 This is a *list of strings* (*AUTOMATIC CHECK*).
 
-|
+
 `project_org`: the account under which the code is found - this may be an individual user account, or an
 organisation like MDAnalysis. This is a *string*.
 
-|
+
 `development_status`: the development status of the MDAKit.
 
 - **CHECK**: This is a *string* and should match one of the `PyPI classifiers`_. 
 - **RECOMMEND**: If you don’t think the provided status matches the actual state of the Kit’s code, you 
   can query this - but don’t let it be a blocker.
 
-|
+
 `publications`: list of publications to be cited when using this MDAKit.
 
 - **CHECK**: This is a *list of strings*, and should include any relevant publications for the Kit 
   itself as well as key upstream publications (e.g. if the Kit heavily relies on another package with an
   associated publication).
 
-|
+
 `changelog`: a link to the MDAKit’s changelog.
 
 - **CHECK**: This is a *string*. If included, check it points to a sensible place (e.g. a CHANGELOG 

From 62135a5d44e8f3b39f54ea29e9a45100a7b12fd5 Mon Sep 17 00:00:00 2001
From: Fiona Naughton <fiona@mdanalysis.org>
Date: Fri, 19 Jan 2024 23:26:06 -0800
Subject: [PATCH 5/5] Update docs/source/reviewersguide.rst

Co-authored-by: Irfan Alibay <IAlibay@users.noreply.github.com>
---
 docs/source/reviewersguide.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/source/reviewersguide.rst b/docs/source/reviewersguide.rst
index f39fc4f3..cc2e2d38 100644
--- a/docs/source/reviewersguide.rst
+++ b/docs/source/reviewersguide.rst
@@ -67,7 +67,7 @@ On the PR
                                       
   - If the failure has a “the head commit is ahead of the base commit” error, tell the contributor to 
     update the branch used in the PR so it is up-to-date with *main*
-  - A failure on installing dependencies may be due the things in the Kit’s `pyproject.toml` not
+  - A failure on installing dependencies may be due the entires in the Kit’s `pyproject.toml` not
     being properly defined