Merge pull request datajoint#1096 from kabilar/master

Migrate `docs.datajoint.org/python`

.github/workflows/development.yaml

@@ -93,6 +93,16 @@ jobs:
           black datajoint --check -v
           black tests --check -v
           black tests_old --check -v
+  codespell:
+    name: Check for spelling errors
+    permissions:
+      contents: read
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Codespell
+        uses: codespell-project/actions-codespell@v2
   publish-docs:
     if: |
       github.event_name == 'push' &&

.github/workflows/docs.yaml

@@ -0,0 +1,19 @@
+name: Manual docs release
+on:
+  workflow_dispatch:
+jobs:
+  publish-docs:
+    runs-on: ubuntu-latest
+    env:
+      DOCKER_CLIENT_TIMEOUT: "120"
+      COMPOSE_HTTP_TIMEOUT: "120"
+    steps:
+      - uses: actions/checkout@v3
+      - name: Deploy docs
+        run: |
+          export MODE=BUILD
+          export PACKAGE=datajoint
+          export UPSTREAM_REPO=https://github.com/${GITHUB_REPOSITORY}.git
+          export HOST_UID=$(id -u)
+          docker compose -f docs/docker-compose.yaml up --exit-code-from docs --build
+          git push origin gh-pages

CHANGELOG.md

@@ -1,5 +1,11 @@
 ## Release notes
 
+### Upcoming
+- Added - Codespell GitHub Actions workflow
+- Added - GitHub Actions workflow to manually release docs
+- Changed - Update `datajoint/nginx` to `v0.2.6`
+- Changed - Migrate docs from `https://docs.datajoint.org/python` to `https://datajoint.com/docs/core/datajoint-python`
+
 ### 0.14.1 -- Jun 02, 2023
 - Fixed - Fix altering a part table that uses the "master" keyword - PR [#991](https://github.com/datajoint/datajoint-python/pull/991)
 - Fixed - `.ipynb` output in tutorials is not visible in dark mode ([#1078](https://github.com/datajoint/datajoint-python/issues/1078)) PR [#1080](https://github.com/datajoint/datajoint-python/pull/1080)

README.md

@@ -5,10 +5,18 @@
 
 # Welcome to DataJoint for Python!
 
-DataJoint for Python is a framework for scientific workflow management based on relational principles. DataJoint is built on the foundation of the relational data model and prescribes a consistent method for organizing, populating, computing, and querying data.
-
-DataJoint was initially developed in 2009 by Dimitri Yatsenko in Andreas Tolias' Lab at Baylor College of Medicine for the distributed processing and management of large volumes of data streaming from regular experiments. Starting in 2011, DataJoint has been available as an open-source project adopted by other labs and improved through contributions from several developers.
-Presently, the primary developer of DataJoint open-source software is the company DataJoint (https://datajoint.com).
+DataJoint for Python is a framework for scientific workflow management based on 
+relational principles. DataJoint is built on the foundation of the relational data 
+model and prescribes a consistent method for organizing, populating, computing, and 
+querying data.
+
+DataJoint was initially developed in 2009 by Dimitri Yatsenko in Andreas Tolias' Lab at 
+Baylor College of Medicine for the distributed processing and management of large 
+volumes of data streaming from regular experiments. Starting in 2011, DataJoint has 
+been available as an open-source project adopted by other labs and improved through 
+contributions from several developers.
+Presently, the primary developer of DataJoint open-source software is the company 
+DataJoint (https://datajoint.com).
 
 ## Data Pipeline Example
 
@@ -18,7 +26,13 @@ Presently, the primary developer of DataJoint open-source software is the compan
 
 ## Getting Started
 
-- Install from PyPI
+- Install with Conda
+
+     ```bash
+     conda install -c conda-forge datajoint
+     ```
+
+- Install with pip
 
      ```bash
      pip install datajoint
@@ -33,9 +47,4 @@ Presently, the primary developer of DataJoint open-source software is the compan
 - Contribute
   - [Development Environment](https://datajoint.com/docs/core/datajoint-python/latest/develop/)
 
-  - [Guidelines](https://datajoint.com/docs/community/contribute/)
-
-- Legacy Resources (To be replaced by above)
-  - [Documentation](https://docs.datajoint.org)
-
-  - [Tutorials](https://tutorials.datajoint.org)
+  - [Guidelines](https://datajoint.com/docs/about/contribute/)

docs/.docker/pip_requirements.txt

@@ -8,3 +8,4 @@ mkdocs-gen-files
 mkdocs-literate-nav
 mkdocs-exclude-search
 mkdocs-jupyter
+mkdocs-section-index

docs/docker-compose.yaml

@@ -16,7 +16,7 @@ services:
       - ..:/main
     user: ${HOST_UID}:anaconda
     ports:
-      - 8080:80
+      - 80:80
     command:
       - sh
       - -c

docs/mkdocs.yaml

@@ -5,16 +5,20 @@ repo_url: https://github.com/datajoint/datajoint-python
 repo_name: datajoint/datajoint-python
 nav:
   - DataJoint Python: index.md
-  - Getting Started: getting-started/index.md
+  - Quick Start Guide: quick-start.md
   - Concepts: 
     - Principles: concepts/principles.md
-    - Glossary: concepts/glossary.md
+    - Data Model: concepts/data-model.md
+    - Data Pipelines: concepts/data-pipelines.md
+    - Teamwork: concepts/teamwork.md
+    - Terminology: concepts/terminology.md
   - System Administration: 
-    - Database Administration: sysadmin/dba.md
-    - File Storage: sysadmin/filestore.md
+    - Database Administration: sysadmin/database-admin.md
+    - Bulk Storage Systems: sysadmin/bulk-storage.md
+    - External Store: sysadmin/external-store.md
   - Client Configuration: 
     - Install: client/install.md
-    - Credentials: client/creds.md
+    - Credentials: client/credentials.md
     - Settings: client/settings.md
     - File Stores: client/stores.md
   - Schema Design:
@@ -25,6 +29,7 @@ nav:
       - Primary Key: design/tables/primary.md
       - Attributes: design/tables/attributes.md
       - Lookup Tables: design/tables/lookup.md
+      - Manual Tables: design/tables/manual.md
       - Blobs: design/tables/blobs.md
       - Attachments: design/tables/attach.md
       - Filepaths: design/tables/filepath.md
@@ -39,12 +44,14 @@ nav:
     - Schema Drop: design/drop.md
     - Schema Modification: design/alter.md
   - Data Manipulations:
+    - manipulation/index.md
     - Insert: manipulation/insert.md
     - Delete: manipulation/delete.md
     - Update: manipulation/update.md
     - Transactions: manipulation/transactions.md
   - Data Queries:
-    - Common Commands: query/common-commands.md
+    - Principles: query/principles.md
+    - Example Schema: query/example-schema.md
     - Fetch: query/fetch.md
     - Iteration: query/iteration.md
     - Operators: query/operators.md
@@ -60,16 +67,13 @@ nav:
     - Populate: compute/populate.md
     - Key Source: compute/key-source.md
     - Distributed Computing: compute/distributed.md
+  - Publish Data: publish-data.md
   - Internals:
     - SQL Transpilation: internal/transpilation.md
-  - Reproducibility:
-      - Table Tiers: reproduce/table-tiers.md
-      - Make Method: reproduce/make-method.md
-  - Existing Pipelines: existing-pipelines.md
   - Tutorials:
-      - tutorials/json.ipynb
+      - JSON Datatype: tutorials/json.ipynb
   - FAQ: faq.md
-  - Develop: develop.md
+  - Developer Guide: develop.md
   - Citation: citation.md
   - Changelog: changelog.md
   - API: api/ # defer to gen-files + literate-nav
@@ -125,6 +129,7 @@ plugins:
         - "*/archive/*md"
   - mkdocs-jupyter:
       include: ["*.ipynb"]
+  - section-index
 markdown_extensions:
   - attr_list
   - toc:
@@ -148,6 +153,7 @@ markdown_extensions:
   - pymdownx.magiclink # Displays bare URLs as links
   - pymdownx.tasklist: # Renders check boxes in tasks lists
       custom_checkbox: true
+  - md_in_html
 extra:
   generator: false # Disable watermark
   version:

docs/src/.overrides/partials/nav.html

@@ -14,7 +14,7 @@
         {#-
         Add DataJoint home link to navigation header, otherwise unchanged
         -#}
-        <a href="https://datajoint.com/docs/core/" title="DataJoint">
+        <a href="https://datajoint.com/docs/" title="DataJoint">
             ⬅ Home
         </a>
     </label>

docs/src/citation.md

@@ -1,7 +1,7 @@
 # Citation
 
-If your work uses the DataJoint API for Python, please cite the following manuscript and Research Resource Identifier (RRID):
+If your work uses the DataJoint for Python, please cite the following manuscript and Research Resource Identifier (RRID):
 
 - Yatsenko D, Reimer J, Ecker AS, Walker EY, Sinz F, Berens P, Hoenselaar A, Cotton RJ, Siapas AS, Tolias AS. DataJoint: managing big scientific data using MATLAB or Python. bioRxiv. 2015 Jan 1:031658. doi: https://doi.org/10.1101/031658
 
-- DataJoint API for Python - [RRID:SCR_014543](https://scicrunch.org/resolver/SCR_014543) - Version `Enter version here`
+- DataJoint for Python - [RRID:SCR_014543](https://scicrunch.org/resolver/SCR_014543) - Version `Enter version here`

docs/src/client/credentials.md

@@ -0,0 +1,46 @@
+# Credentials
+
+Configure the connection through DataJoint's `config` object:
+
+```python
+> import datajoint as dj
+DataJoint 0.4.9 (February 1, 2017)
+No configuration found. Use `dj.config` to configure and save the configuration.
+```
+
+You may now set the database credentials:
+
+```python
+dj.config['database.host'] = "alicelab.datajoint.io"
+dj.config['database.user'] = "alice"
+dj.config['database.password'] = "haha not my real password"
+```
+
+Skip setting the password to make DataJoint prompt to enter the password every time.
+
+You may save the configuration in the local work directory with 
+`dj.config.save_local()` or for all your projects in `dj.config.save_global()`.
+Configuration changes should be made through the `dj.config` interface; the config file 
+should not be modified directly by the user.
+
+You may leave the user or the password as `None`, in which case you will be prompted to 
+enter them manually for every session.
+Setting the password as an empty string allows access without a password.
+
+Note that the system environment variables `DJ_HOST`, `DJ_USER`, and `DJ_PASS` will 
+overwrite the settings in the config file.
+You can use them to set the connection credentials instead of config files.
+
+To change the password, the `dj.set_password` function will walk you through the 
+process:
+
+```python
+dj.set_password()
+```
+
+After that, update the password in the configuration and save it as described above:
+
+```python
+dj.config['database.password'] = 'my#cool!new*psswrd'
+dj.config.save_local()   # or dj.config.save_global()
+```