Skip to content

Commit

Permalink
Setup basic project config
Browse files Browse the repository at this point in the history
  • Loading branch information
berrydenhartog committed Apr 26, 2024
2 parents a86b65f + 94a7b92 commit 12eddd6
Show file tree
Hide file tree
Showing 18 changed files with 1,086 additions and 84 deletions.
3 changes: 0 additions & 3 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ updates:
time: "08:00"
timezone: "Europe/Amsterdam"
labels:
- "github-actions"
- "dependencies"

- package-ecosystem: "pip"
Expand All @@ -19,7 +18,6 @@ updates:
time: "08:00"
timezone: "Europe/Amsterdam"
labels:
- "pip"
- "dependencies"

- package-ecosystem: "devcontainers"
Expand All @@ -30,5 +28,4 @@ updates:
time: "08:00"
timezone: "Europe/Amsterdam"
labels:
- "devcontainers"
- "dependencies"
178 changes: 178 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,178 @@
name: CI

on:
push:
branches:
- main
tags:
- 'v*'
pull_request:
branches:
- 'main'

env:
REGISTRY: ghcr.io
POETRY_CACHE_DIR: ~/.cache/pypoetry
IMAGE_NAME: ${{ github.repository }}
PYTHON_VERSION: "3.11"

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install poetry
run: pipx install poetry

- name: Set up Python ${{ env.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: "poetry"

- name: Install dependencies
run: poetry install

- name: run ruff
run: poetry run ruff check --output-format=github

- name: run format
run: poetry run ruff format --check

- name: run pyright
run: poetry run pyright

security:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install poetry
run: pipx install poetry

- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: "poetry"

- name: Install dependencies
run: poetry install

- name: Generate SBOM
run: poetry run cyclonedx-py poetry > sbom.json

- name: Generace licenses file
run: |
poetry run pip-licenses --order=license --format=json --with-description > licenses.txt
- name: Upload SBOM and licenses
uses: actions/upload-artifact@v4
with:
name: sbom-licenses-${{ github.ref_name }}.json
path: |
sbom.json
licenses.txt
if-no-files-found: error
overwrite: true

- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
scan-type: 'fs'
scan-ref: '.'
trivy-config: trivy.yaml

test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4

- name: Install poetry
run: pipx install poetry

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "poetry"

- name: Install dependencies
run: poetry install

- name: run pytest
run: poetry run coverage run -m pytest

- name: run coverage
run: poetry run coverage report

- name: run coverage
run: poetry run coverage html

- name: Upload code coverage report
if: ${{ matrix.python-version }} == '3.11'
uses: actions/upload-artifact@v4
with:
name: codecoverage-${{ github.ref_name}}
path: htmlcov/
if-no-files-found: error
overwrite: true


build:
needs: test
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4

- name: Log in to the Container registry
uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Extract metadata for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}

- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
platforms: linux/amd64,linux/arm64,darwin/amd64

- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
image-ref: ${{ steps.meta.outputs.tags }}
trivy-config: trivy.yaml
github-pat: ${{ secrets.GITHUB_TOKEN }}

notifyMattermost:
runs-on: ubuntu-latest
steps:
- name: The job has failed
if: ${{ failure() }}
run: echo "The job has failed"
# todo: add mattermost notification

1 change: 1 addition & 0 deletions .github/workflows/first-interaction.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ on: [pull_request, issues]
jobs:
greeting:
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
issues: write
pull-requests: write
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/stale.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@ on:
jobs:
stale:
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: write # only for delete-branch option
contents: write
issues: write
pull-requests: write
steps:
Expand Down
File renamed without changes.
4 changes: 2 additions & 2 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
"version": "0.2.0",
"configurations": [
{
"name": "Project",
"name": "tad",
"type": "debugpy",
"request": "launch",
"module": "python_project",
"module": "tad",
"justMyCode": false,
"args": []
},
Expand Down
53 changes: 53 additions & 0 deletions BUILD.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Buiding TAD

There are several ways to build and run TAD.

1. poetry
2. container

## Building TAD with Poetry

Poetry is a python package and dependency manager. Before you can install poetry you first need to install python. Please follow [these](https://github.com/pyenv/pyenv?tab=readme-ov-file#installation) instructions.

Once you have python available you can install poetry. See [here](https://python-poetry.org/docs/#installation).

Once you have poetry and python install you can start installing the dependencies with the following shell command.

```shell
poetry install
```

when poetry is done installing all dependencies you can start using the tool.

```shell
poetry run python -m tad
```

## Building TAD with Containers

Containers allow use to package software and make it portable and isolated. Before you can run container you first need a container runtime. There are several available but allot of users use [docker desktop](https://www.docker.com/products/docker-desktop/).

Once you install a docker runtime like docker desktop you can start building the applications with this command:

```shell
docker compose build
```

to run the application you use this command:

```shell
docker compose up
```

## Testing, Linting etc

For testing, linting and other feature we use several tools. You can look up the documentation on how to use these:

* [pytest](https://docs.pytest.org/en/) `poetry run pytest`
* [ruff](https://docs.astral.sh/ruff/) `poetry run ruff format` or `poetry run ruff check --fix`
* [coverage](https://coverage.readthedocs.io/en/) `poetry run coverage report`
* [pyright](https://microsoft.github.io/pyright/#/) `poetry run pyright`

## Devcontainers

[VSCode](https://code.visualstudio.com/) has great support for devcontainers. If your editor had support for devcontainers you can also use them to start the devcontainer. Devcontaines offer great standardized environments for development.
12 changes: 6 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,7 @@ FROM --platform=$BUILDPLATFORM python:${PYTHON_VERSION} as project-base

LABEL [email protected] \
organization=MinBZK \
license=EUPL-1.2 \
io.docker.minbzk.name=python-project-template
license=EUPL-1.2

ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
Expand Down Expand Up @@ -37,14 +36,15 @@ FROM development AS lint

RUN ruff check
RUN ruff format --check
RUN pyright

FROM development AS test
RUN coverage run --rcfile ./pyproject.toml -m pytest ./tests
RUN coverage report --fail-under 95
RUN coverage run -m pytest ./tests
RUN coverage report

FROM project-base as production

COPY ./python_project /app/python_project
COPY ./tad /app/tad

# change this to a usefull command
CMD ["python", "-m", "python_project" ]
CMD ["python", "-m", "tad" ]
38 changes: 12 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,19 @@
# Python Project Template
# Transparency for Algorithm Decision making (TAD)

## Description
![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/minbzk/tad/ci.yml)
![Codecov (with branch)](https://img.shields.io/codecov/c/github/minbzk/tad/main)
![GitHub Downloads (all assets, all releases)](https://img.shields.io/github/downloads/minbzk/tad/total)
![GitHub License](https://img.shields.io/github/license/minbzk/tad)
![GitHub Release](https://img.shields.io/github/v/release/minbzk/tad)

This is a template repository that can be used for Python 3.11 projects and uses the Poetry package manager. By default this project sets up the following:
TAD is a modern tool to apply technical and non-technical tests for an AI model.

* Devcontainers for VSCode users
* Github community health files
* Github Dependabot
* VSCode configs
* Some scripts to adhere to programming standards
* A editorconfig file so editors enforce formatting
* A default .gitgignore
* A default pre-commit-config
* A EUPL v1.2 Licence
* A basic Docker setup
* publiccode.yml
Note: The main branch may be in an unstable or even broken state during development. Please use releases instead of the main branch.

## How to use this template repository
## How to contribute

When creating a new Repository select this template repository as the base.
See [contributing docs](CONTRIBUTING.md)

After the repository is created make sure to change the following (we may need to consider copier to automate this):
## How to build and run TAD

* change the owners in the the .github/CODEOWNERS
* run a global rename command where you rename new_name to your project name
* macos: `find . -type f -not -path "./.git/*" -exec sed -i '' "s/python_project/new_name/g" {} \;`
* linux: `find . -type f -not -path "./.git/*" -exec sed -i "s/python_project/new_name/g" {} \;`
* rename the python_project/ folder to your project name
* change author and name in pyproject.toml
* change labels in Dockerfile to appropriate values
* Verify the License used
* Change publiccode.yml to your needs
See [build docs](BUILD.md)
4 changes: 4 additions & 0 deletions compose.override.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
services:
tad:
build: .
image: ghcr.io/minbzk/tad:dev
4 changes: 4 additions & 0 deletions compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
services:
tad:
build: .
image: ghcr.io/minbzk/tad:latest
Loading

0 comments on commit 12eddd6

Please sign in to comment.