Discussion: Generalization of package

[bendichter]

I think it's pretty clear that the features of this library and even more so the general goal of the library could be broadly useful for a large variety of software within and outside neuroscience. I'm opening this ticket as a place to discuss how we might generalize this package to be more broadly applicable. @oruebel, what exactly would need to be done to generalize this so it could be applied to any Python package? What are the places that are currently NWB-specific?

One option could be to refactor this into a sphinx extension so it could be auto-generated as an additional page on the ReadTheDocs of any package, similar to how auto API currently works. I suppose the challenge of that would be that we would need to use the RTD build services, which might fail if this takes too long. Is there a way to build docs using GitHub Actions instead?

cc @alejoe91 @rly @oruebel

[oruebel]

what exactly would need to be done to generalize this so it could be applied to any Python package

The main code is already designed as a python package. One of the main parts will be to make the creation of pages configurable.

1. create_codestat_pages.py implements the workflow for autogenerating the RST pages and plots. There are a few places here that would need to be updated to be configurable, instead of using hard-coded settings for NWB (in particular the parts with summaries for the main tools of a project):
   - https://github.com/NeurodataWithoutBorders/nwb-project-analytics/blob/71703211348775e11be6207275d5cca9220f3ca3/src/nwb_project_analytics/create_codestat_pages.py#L113C1-L113C1
   -

   nwb-project-analytics/src/nwb_project_analytics/create_codestat_pages.py

   Line 150 in 7170321

   def __create_nwb_codecov_summary_plot(

   
   -

   nwb-project-analytics/src/nwb_project_analytics/create_codestat_pages.py

   Line 188 in 7170321

   def __create_nwb_codestat_summary_rst(

   
   -

   nwb-project-analytics/src/nwb_project_analytics/create_codestat_pages.py

   Line 264 in 7170321

   def create_codestat_pages(out_dir: str,

2. NWBGitInfo essentially stores all the configurations for the project. We probably would need to define some form a base-class for this to define how to configure a new project and in turn update create_codestat_pages to be able to accept custom project configurations.

3. The functions used to configure the project in the conf.py in the docs would need to be updated and moved such that other projects can use and import them in their own sphinx docs.

One option could be to refactor this into a sphinx extension so it could be auto-generated as an additional page on the ReadTheDocs of any package

The package auto-generates RST sources. I.e., it should be straight forward to just include the index.rst file that is being generated here in another sphinx website. This is what we do for integration of the analytics with the nwb-overview website, see:

• https://github.com/NeurodataWithoutBorders/nwb-overview/blob/0dcf3434c5daa54c135ab55242779b81dd592b57/docs/source/conf.py#L75-L76
• https://github.com/NeurodataWithoutBorders/nwb-overview/blob/main/docs/source/conf_nwb_analytics.py
• https://github.com/NeurodataWithoutBorders/nwb-overview/blob/0dcf3434c5daa54c135ab55242779b81dd592b57/docs/source/index.rst?plain=1#L44