Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial Settings in the Code #1

Merged
merged 23 commits into from
Oct 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
0ffb533
environment updated
mpatrou Oct 28, 2024
c7d1c54
package name added, mantid logger replaced with python logger, enviro…
mpatrou Oct 28, 2024
7537830
configuration, version, and meta updates for new repo
mpatrou Oct 28, 2024
a4fe3dc
notebooks/ removed, github actions updated
mpatrou Oct 28, 2024
90f41a6
unittest updated
mpatrou Oct 28, 2024
535db69
additonal checks added in unittest job
mpatrou Oct 28, 2024
c6d91bb
job renamed in unittest job
mpatrou Oct 28, 2024
71561df
check-wheel-contents added for checking the wheels
mpatrou Oct 28, 2024
318341f
typo on unittest fixed
mpatrou Oct 28, 2024
aeea91d
action test
mpatrou Oct 28, 2024
2d4e48d
update job names
mpatrou Oct 28, 2024
1910703
pull request setting removed
mpatrou Oct 29, 2024
f346f24
pre-commit errors fixed
mpatrou Oct 29, 2024
ea77e5a
coverage and badges
mpatrou Oct 29, 2024
f9bd812
twine check added on github action
mpatrou Oct 29, 2024
41cbb0a
prototype script added, excluded from pre-commit
mpatrou Oct 29, 2024
433a1ad
run pytest after package installed in editable mode in github actions…
mpatrou Oct 29, 2024
6df09b1
typo on github actions, and removed unused steps, updates
mpatrou Oct 29, 2024
48266f7
environment name updated on actions
mpatrou Oct 29, 2024
b1041ab
pyproject updated to point only to gui, readme update with developmen…
mpatrou Oct 29, 2024
b1052b8
action restored
mpatrou Oct 29, 2024
aa1f424
conda environment instructions added
mpatrou Oct 29, 2024
9bcaba7
readme instruction updates
mpatrou Oct 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ on:
tags: ['v*']

jobs:
linux:
anaconda-publish:
name: upload release to Anaconda
runs-on: ubuntu-latest
defaults:
run:
Expand All @@ -20,18 +21,15 @@ jobs:
channels: conda-forge,defaults
use-mamba: true
environment-file: environment.yml
activate-environment: test
- name: install additional dependencies
run: |
echo "installing additional dependencies from environment_development.yml"
activate-environment: hyspecplanningtools_dev
- name: build conda package
run: |
# set up environment
cd conda.recipe
echo "versioningit $(versioningit ../)"
# build the package
VERSION=$(versioningit ../) conda mambabuild --channel conda-forge --output-folder . .
conda verify noarch/mypackagename*.tar.bz2
conda verify noarch/hyspecplanningtools*.tar.bz2
- name: upload conda package to anaconda
shell: bash -l {0}
if: startsWith(github.ref, 'refs/tags/v')
Expand All @@ -43,7 +41,7 @@ jobs:
CONDA_LABEL="main"
if [ "${IS_RC}" = "true" ]; then CONDA_LABEL="rc"; fi
echo pushing ${{ github.ref }} with label $CONDA_LABEL
anaconda upload --label $CONDA_LABEL conda.recipe/noarch/mypackagename*.tar.bz2
anaconda upload --label $CONDA_LABEL conda.recipe/noarch/hyspecplanningtools*.tar.bz2

pypi-publish:
name: upload release to PyPI
Expand All @@ -62,7 +60,7 @@ jobs:
channels: conda-forge,defaults
use-mamba: true
environment-file: environment.yml
activate-environment: test
activate-environment: hyspecplanningtools_dev
- name: build pypi package
run: |
# build the package
Expand Down
18 changes: 12 additions & 6 deletions .github/workflows/unittest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:
tags: ['v*']

jobs:
linux:
pytest_package_tests:
runs-on: ubuntu-latest
defaults:
run:
Expand All @@ -21,10 +21,11 @@ jobs:
channels: conda-forge,defaults
use-mamba: true
environment-file: environment.yml
activate-environment: test
- name: install additional dependencies
activate-environment: hyspecplanningtools_dev
- name: install in editable mode
run: |
echo "installing additional dependencies if cannot be installed from conda"
echo "installing in editable mode"
python -m pip install -e .
- name: run unit tests
run: |
echo "running unit tests"
Expand All @@ -35,13 +36,18 @@ jobs:
github.actor != 'dependabot[bot]'
with:
token: ${{ secrets.CODECOV_TOKEN }}
- name: build python wheel - pypi
run: |
# test that the python wheel builds
VERSION=$(versioningit .) python -m build --wheel --no-isolation
twine check dist/hyspecplanningtools-*.whl
- name: build conda package
run: |
# test that the conda package builds
cd conda.recipe
echo "versioningit $(versioningit ../)"
# conda channels could have been defined in the conda-incubator, but you can copy/paste the lines
# below to build the conda package in your local machine
CHANNELS="--channel mantid/label/main --channel conda-forge"
CHANNELS="--channel conda-forge"
VERSION=$(versioningit ../) conda mambabuild $CHANNELS --output-folder . .
conda verify noarch/mypackagename*.tar.bz2
conda verify noarch/hyspecplanningtools*.tar.bz2
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ share/python-wheels/
.installed.cfg
*.egg
MANIFEST
_version.py

# PyInstaller
# Usually these files are written by a python script from a template
Expand Down
2 changes: 2 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
exclude: "scripts/.*|.*\\.mat$"

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
Expand Down
155 changes: 14 additions & 141 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,154 +1,27 @@
python_project_template
=======================

This repository is a template repository for Python projects under neutrons.
After you create a new repository using this repo as template, please follow the following steps to adjust it for the new project.
HyspecPlanningTools
=====================

Codebase Adjustments
--------------------
Polarization tool for Single Crystal and Powder

1. Adjust the branch protection rules for the new repo. By default, we should protect the `main` (stable), `qa` (release candidate), and `next` (development) branches.
## Installation Process

1.1 Go to the `Settings` tab of the new repo.
Create the development conda environment

1.2 Click on `Branches` on the left side.
`conda env create`

1.3 Click on `Add rule` button.
Activate the environment

1.4 Follow the instructions from Github.
`conda activate hyspecplanningtools_dev`

Install the application in editable mode

2. Change the License if MIT license is not suitable for you project. For more information about licenses, please
refer to [Choose an open source license](https://choosealicense.com/).
`pip install -e .`

Start the tool

3. Update the environment dependency file `environment.yml`, which contain both runtime and development dependencies.
For more information about conda environment file, please refer to [Conda environment file](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#creating-an-environment-file-manually).
`hyspecplanningtools`

3.1 Specify environment 'name' field to match package name

3.2 We strongly recommended using a single `environment.yml` file to manage all the dependencies, including the runtime and development dependencies.

3.3 Please add comments to the `environment.yml` file to explain the dependencies.

3.4 Please prune the dependencies to the minimum when possible, we would like the solver to figure out the dependency tree for us.


4. Adjust pre-commit configuration file, `.pre-commit-config.yaml` to enable/disable the hooks you need. For more information about pre-commit, please refer to [pre-commit](https://pre-commit.com/).


5. Having code coverage, `codecov.yaml` is **strongly recommended**, please refer to [Code coverage](https://coverage.readthedocs.io/en/coverage-5.5/) for more information.


6. Adjust the demo Github action yaml files for CI/CD. For more information about Github action, please refer to [Github action](https://docs.github.com/en/actions).

6.1 Specify package name at: .github/workflows/package.yml#L34

6.2 Specify package name at: .github/workflows/package.yml#L46


7. Adjust the conda recipe, `conda-recipe/meta.yaml` to provide the meta information for the conda package. For more information about conda recipe, please refer to [Conda build](https://docs.conda.io/projects/conda-build/en/latest/).

7.1 Specify package name at: conda.recipe/meta.yaml#L15

7.2 Update license family, if necessary: conda.recipe/meta.yaml#L42


8. Adjust `pyproject.toml` to match your project. For more information about `pyproject.toml`,
please refer to [pyproject.toml](https://www.python.org/dev/peps/pep-0518/).

8.1 Specify package name at: pyproject.toml#L2

8.2 Specify package description at: pyproject.toml#L3

8.3 Specify package name at: pyproject.toml#L39

8.4 Specify any terminal entry points (terminal commands) at: pyproject.toml#48.

In the example, invoking `packagename-cli` in a terminal is equivalent to running the python script `from packagenamepy.packagename.import main; main()`

8.5 Projects will use a single `pyproject.toml` file to manage all the project metadata, including the project name, version, author, license, etc.

8.6 Python has moved away from `setup.cfg`/`setup.py`, and we would like to follow the trend for our new projects.


10. Specify package name at src/packagenamepy


11. Specify package name at: src/packagenamepy/packagename.py

12. If a GUI isn't used, delete the MVP structure at src/packagenamepy:
11.1: mainwindow.py
11.2: home/
11.3: help/


11. Clear the content of this file and add your own README.md as the project README file.
We recommend putting badges of the project status at the top of the README file.
For more information about badges, please refer to [shields.io](https://shields.io/).

Repository Adjustments
----------------------

### Add an access token to anaconda

Here we assume your intent is to upload the conda package to the [anaconda.org/neutrons](https://anaconda.org/neutrons) organization.
An administrator of `anaconda.org/neutrons` must create an access token for your repository in the [access settings](https://anaconda.org/neutrons/settings/access).

After created, the token must be stored in a `repository secret`:
1. Navigate to the main page of the repository on GitHub.com.
2. Click on the "Settings" tab.
3. In the left sidebar, navigate to the "Security" section and select "Secrets and variables" followed by "Actions".
4. Click on the "New repository secret" button.
5. Enter `ANACONDA_TOKEN` for the secret name
6. Paste the Anaconda access token
7. Click on the "Add secret" button
8. Test the setup by creating a release candidate tag,
which will result in a package built and uploaded to https://anaconda.org/neutrons/mypackagename

### Add an access token to codecov

Follow the instructions in the [Confluence page](https://ornl-neutrons.atlassian.net/wiki/spaces/NDPD/pages/103546883/Coverage+reports)
to create the access token.

Packaging building instructions
-------------------------------

The default package publishing service is anaconda.
However, we also support PyPI publishing as well.

### Instruction for publish to PyPI

1. Make sure you have the correct access to the project on PyPI.
2. Make sure `git status` returns a clean state.
3. At the root of the repo, use `python -m build` to generate the wheel.
4. Check the wheel with `twine check dist/*`, everything should pass before we move to next step.
5. When doing manual upload test, make sure to use testpypi instead of pypi.
6. Use `twine upload --repository testpypi dist/*` to upload to testpypi, you will need to specify the testpipy url in your `~/.pypirc`, i.e.

``````
[distutils]
index-servers = pypi, testpypi

[testpypi]
repository = https://test.pypi.org/legacy/
username = __token__
password = YOUR_TESTPYPI_TOKEN

``````

7. Test the package on testpypi with `pip install --index-url https://test.pypi.org/simple/ mypackagename`.
8. If everything is good, use the Github workflow, `package.yml` to trigger the publishing to PyPI.

### Instruction for publish to Anaconda

Publishing to Anaconda is handled via workflow, `package.yml`.

Development environment setup
-----------------------------

### Build development environment

1. By default, we recommend providing a single `environment.yml` that covers all necessary packages for development.
2. The runtime dependency should be in `meta.yaml` for anaconda packaging, and `pyproject.toml` for PyPI publishing.
3. When performing editable install for your feature branch, make sure to use `pip install --no-deps -e .` to ensure that `pip` does not install additional packages from `pyproject.toml` into development environment by accident.
[![CI](https://github.com/neutrons/HyspecPlanningTools/actions/workflows/unittest.yml/badge.svg?branch=next)](https://github.com/neutrons/HyspecPlanningTools/actions/workflows/unittest.yml)
[![codecov](https://codecov.io/gh/neutrons/HyspecPlanningTools/graph/badge.svg?token=GAQE3SS0HJ)](https://codecov.io/gh/neutrons/HyspecPlanningTools)
2 changes: 1 addition & 1 deletion conda.recipe/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
{% set build_number = 0 %}

package:
name: mypackagename
name: hyspecplanningtools
version: {{ version_number }}

source:
Expand Down
28 changes: 7 additions & 21 deletions environment.yml
Original file line number Diff line number Diff line change
@@ -1,23 +1,16 @@
name: mypythonapp
name: hyspecplanningtools_dev
channels:
- conda-forge
dependencies:
# -- Runtime dependencies
# base: list all base dependencies here
- python>=3.8 # please specify the minimum version of python here
- versioningit
# compute: list all compute dependencies here
- python>=3.10
- pyqt == 5.*
- qtpy
- numpy
- pandas
# plot: list all plot dependencies here, if applicable
- scipy
- matplotlib
# jupyter: list all jupyter dependencies here, if applicable
- jupyterlab
- ipympl
# -- Development dependencies
# utils:
- pre-commit
# package building:
- versioningit
- libmamba
- libarchive
- anaconda-client
Expand All @@ -29,16 +22,9 @@ dependencies:
# docs
- sphinx
- sphinx_rtd_theme
- sphinxcontrib-mermaid
- myst-parser # required for parsing markdown files
# test: list all test dependencies here
- pytest
- pytest-cov
- pytest-xdist
# --------------------------------------------------
# add additional sections such as Qt, etc. if needed
# --------------------------------------------------
# if packages are not available on conda, list them here
- pip
- pip:
- bm3d-streak-removal # example
- pytest-playwright
27 changes: 0 additions & 27 deletions notebooks/example.ipynb

This file was deleted.

Loading