Skip to content

Commit

Permalink
Merge branch 'dev' into 66-check_unmatched_id_warnings-unit-test
Browse files Browse the repository at this point in the history
  • Loading branch information
CBROWN-ONS committed Sep 20, 2023
2 parents 7920f92 + 87216ed commit 12eb723
Show file tree
Hide file tree
Showing 44 changed files with 2,994 additions and 209 deletions.
34 changes: 34 additions & 0 deletions .github/workflows/all-os-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# This workflow will install OS dependencies and run a 'base' set of unit tests with Python 3.9
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Unit tests on macOS/Linux/Windows

on:
push:
pull_request:
branches: [ "dev", "main" ]

jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
python-version: ["3.9"]
os: ["macos-latest", "ubuntu-latest", "windows-latest"]
steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Run Integration Tests # run only those tests marked runinteg & with no osmosis deps
run: |
pytest -m runinteg --runinteg --deselect tests/osm/
- name: Test with pytest # run only tests with no osmosis deps
run: |
pytest --deselect tests/osm/
28 changes: 16 additions & 12 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# This workflow will install Python dependencies, run tests and lint with Python 3.9
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Python package
Expand All @@ -10,7 +10,6 @@ on:

jobs:
build:

runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
Expand All @@ -23,11 +22,6 @@ jobs:
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install geos # required for cartopy installation
uses: ConorMacBride/install-package@v1
with:
brew: geos
apt: libgeos-dev
- name: Install dependencies
run: |
python -m pip install --upgrade pip
Expand All @@ -40,11 +34,19 @@ jobs:
- name: Check Java Install
run: |
java --version
- name: Install Osmosis
uses: ConorMacBride/install-package@v1
with:
brew: osmosis
apt: osmosis
- name: Install mac depencies with brew
if: runner.os == 'macOS' # not updating brew version, issue with aom
run: |
brew install geos
brew install osmosis
shell: sh
- name: Install linux depencies with apt
if: runner.os == 'Linux'
run: |
sudo apt update
sudo apt install -y libgeos-dev
sudo apt install -y osmosis
shell: sh
- name: Run Integration Tests
run: |
pytest -m runinteg --runinteg # run only those tests marked runinteg
Expand All @@ -56,10 +58,12 @@ jobs:
run: |
pytest
- name: Generate Report
if: runner.os == 'macOS' # run coverage report only on macOS
run: |
coverage run -m pytest
coverage xml
- name: Upload coverage reports to Codecov
if: runner.os == 'macOS' # run coverage report only on macOS
uses: codecov/codecov-action@v3
with:
file: ./coverage.xml
Expand Down
34 changes: 34 additions & 0 deletions .github/workflows/sphinx-render.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: "Render docs"

on: push

env:
PYTHON_VERSION: "3.9"
PUSH_BRANCH: "refs/heads/dev"

jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v3
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Sphinx build # use -W to turn warnings into errors
run: |
make -C docs/ html SPHINXOPTS="-W"
- name: Deploy
uses: peaceiris/actions-gh-pages@v3
# deploy if it's being pushed only to this branch
if: ${{ github.ref == env.PUSH_BRANCH }}
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: docs/build/html
commit_message: ${{ github.event.head_commit.message }}
23 changes: 15 additions & 8 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,18 @@
# moved blanket rules above specific exceptions for test fixtures
*.zip
*.pkl
# except test fixtures
*.html

# exception for test fixtures
!tests/data/newport-2023-06-13.osm.pbf
!tests/data/newport-20230613_gtfs.zip
!tests/data/gtfs/route_lookup.pkl
!tests/data/gtfs/report/html_template.html

# exception for html templates
!src/transport_performance/gtfs/report/html_templates/evaluation_template.html
!src/transport_performance/gtfs/report/html_templates/stops_template.html
!src/transport_performance/gtfs/report/html_templates/summary_template.html

### Project structure ###
data/*
Expand All @@ -35,7 +43,6 @@ outputs/*

*.Rproj

*.html
*.pdf
*.csv
*.rds
Expand Down Expand Up @@ -167,7 +174,12 @@ instance/
.scrapy

# Sphinx documentation
docs/_build/
docs/source/reference/_autosummary/
docs/build/
!docs/README.md
!docs/gtfs
!docs/source/_static/dsc.png
!docs/source/_templates/footer.html

# PyBuilder
.pybuilder/
Expand Down Expand Up @@ -306,11 +318,6 @@ vignettes/*.pdf
# R Environment Variables
.Renviron

# pkgdown site
docs/*
!docs/README.md
!docs/gtfs

# translation temp files
po/*~

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!--- Badges start --->
<img src="https://img.shields.io/badge/repo%20status-in%20development%20(caution)-red" alt="Repository status is still in development (caution required)"/> <img src="https://github.com/datasciencecampus/transport-network-performance/actions/workflows/python-package.yml/badge.svg" alt="Build status badge"/> <a href="https://codecov.io/gh/datasciencecampus/transport-network-performance" > <img src="https://codecov.io/gh/datasciencecampus/transport-network-performance/branch/dev/graph/badge.svg?token=ZKJFT321CN"/></a>
<img src="https://img.shields.io/badge/repo%20status-in%20development%20(caution)-red" alt="Repository status is still in development (caution required)"/> <img src="https://github.com/datasciencecampus/transport-network-performance/actions/workflows/python-package.yml/badge.svg" alt="Build status badge"/> <img src="https://github.com/datasciencecampus/transport-network-performance/actions/workflows/sphinx-render.yml/badge.svg" alt="Docs status badge"/> <a href="https://codecov.io/gh/datasciencecampus/transport-network-performance" > <img src="https://codecov.io/gh/datasciencecampus/transport-network-performance/branch/dev/graph/badge.svg?token=ZKJFT321CN"/></a>

<!--- Badges end --->

Expand Down
51 changes: 30 additions & 21 deletions conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,28 +41,37 @@ def pytest_configure(config):
)


def pytest_collection_modifyitems(config, items): # noqa C901
def pytest_collection_modifyitems(config, items): # noqa:C901
"""Handle switching based on cli args."""
if config.getoption("--runsetup"):
# --runsetup given in cli: do not skip slow tests
if (
config.getoption("--runsetup")
& config.getoption("--runinteg")
& config.getoption("--runexpensive")
):
# do full test suite when all flags are given
return
skip_setup = pytest.mark.skip(reason="need --runsetup option to run")
for item in items:
if "setup" in item.keywords:
item.add_marker(skip_setup)

if config.getoption("--runinteg"):
return
skip_runinteg = pytest.mark.skip(reason="need --runinteg option to run")
for item in items:
if "runinteg" in item.keywords:
item.add_marker(skip_runinteg)
# do not add setup marks when the runsetup flag is given
if not config.getoption("--runsetup"):
skip_setup = pytest.mark.skip(reason="need --runsetup option to run")
for item in items:
if "setup" in item.keywords:
item.add_marker(skip_setup)

if config.getoption("--runexpensive"):
return
skip_runexpensive = pytest.mark.skip(
reason="need --runexpensive option to run"
)
for item in items:
if "runexpensive" in item.keywords:
item.add_marker(skip_runexpensive)
# do not add integ marks when the runinteg flag is given
if not config.getoption("--runinteg"):
skip_runinteg = pytest.mark.skip(
reason="need --runinteg option to run"
)
for item in items:
if "runinteg" in item.keywords:
item.add_marker(skip_runinteg)

# do not add expensive marks when the runexpensive flag is given
if not config.getoption("--runexpensive"):
skip_runexpensive = pytest.mark.skip(
reason="need --runexpensive option to run"
)
for item in items:
if "runexpensive" in item.keywords:
item.add_marker(skip_runexpensive)
20 changes: 20 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = source
BUILDDIR = build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
87 changes: 86 additions & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,88 @@
# `docs` folder overview

All documentation for the project should be included in this folder.
This folder contains all the source files needed to build package documentation
using [`sphinx`](https://www.sphinx-doc.org/en/master/).

## Building the documentation locally

This is useful whilst editing the documentation locally and previewing new
additions/edits. Following the steps below will render the documenation locally
allowing you to check for any warnings or errors during the build stage.

1. Ensure the dependencies in `requirements.txt` have been installed. This will
install `sphinx`, the necessary themes, and all the other Python dependecies
for this package.

2. Call the following from the project root:

```bash
make -C docs/ html
```

Or, from the within this docs directory:

```bash
make html
```

> Note: On Windows, if you are using PowerShell the make command may not
work. If this is the case, you should be able to run `.\make.bat html`
after navigating to this directory.

Calling one of the commands above will trigger `sphinx-build` and render
the documentaion in HTML format within the `build` directory.

3. Inside `docs/build/html/`, opening/refreshing `index.html` in a browser will
display the documentation landing page.

## Cleaning the docs folder

From time to time, it maybe necessary to clean the build folder (e.g., to
unpick some edits that have not made their way through to the browser for some
reason).

> Note: `sphinx-build` will only rebuild pages if the respective source file(s)
has changed. Calling clean maybe helpful to either force an entire rebuild of
all pages, or include an update that isn't picked up via a source (e.g. a CSS
file update).
To clean the build folder, call the following:
```bash
# from the project root
make -C docs/ clean
# or, from within the docs folder
make clean
```
It's also possible to combine both the clean and HTML build commands together
as follows:

```bash
# from the project root
make -C docs/ clean html
# or, from within the docs folder
make clean html
```

> Note: the contents of the `docs/build` folder are ignored by Git. Cleaning
the build folder locally will therefore only impact your local documentation
build.

## Building the documentation 'on push' to a remote branch

There is a GitHub action set-up (`.github/workflows/sphinx-render.yml`) that
runs on all pushes to any branch. This will attempt to build the `docs/source`
folder content and will fail if `sphinx-build` throws any errors or warnings.
This helps ensure the quality of the documentation on each push and allows
developers to correct any issues sooner.

The deployment stage of this GitHub action is only done when pushing to the
`dev` branch (i.e. after merging in a PR). Therefore, any changes made to
`docs` in a feature branch will not appear in the deployed documentation.

> Note: the current implementation of the GitHub action deploys on push to
`dev` but this is subject to change at a later date. It will likely be change
to puses to `main` once an inital release of this package is available.
Loading

0 comments on commit 12eb723

Please sign in to comment.