Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add instructions about splitting header and runtime + run_export> packages #2257

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
37 changes: 37 additions & 0 deletions docs/maintainer/knowledge_base.md
Original file line number Diff line number Diff line change
Expand Up @@ -1695,6 +1695,8 @@ add some information on r packages which make heavy use of `noarch: generic`
- [importlib_metadata and importlib-metadata](https://github.com/conda-forge/importlib_metadata-feedstock/blob/main/recipe/meta.yaml)
- [typing_extensions and typing-extensions](https://github.com/conda-forge/typing_extensions-feedstock/blob/main/recipe/meta.yaml)



### Common pitfalls with `outputs`

This is a non-exhaustive list of common pitfalls when using `outputs`.
Expand All @@ -1704,6 +1706,41 @@ This is a non-exhaustive list of common pitfalls when using `outputs`.
- The `outputs[].script` field can only be set to a script name. If you prefer passing shell commands, you have to use `outputs[].build.script`. Compare [geopandas-feedstock](https://github.com/conda-forge/geopandas-feedstock/blob/8b985635a8538af1ee213900bd563085e3cdbd92/recipe/meta.yaml#L17) to [gym-feedstock](https://github.com/conda-forge/gym-feedstock/blob/2b47e0479923b7d49a39e9860ba30a28e263480b/recipe/meta.yaml#L31), respectively.
- Some `PIP_*` environment variables that are usually set for the top-level scripts are not automatically set for the outputs. If you are invoking `pip` in an output, you may need to pass additional flags. See [napari-feedstock](https://github.com/conda-forge/napari-feedstock/blob/32a4eb04ca7b6ccd2c4e146bde204f1dd5425a17/recipe/meta.yaml#L26). This issue is tracked in [conda/conda-build#3993](https://github.com/conda/conda-build/issues/3993).

- Pin as much as possible each output to the exact version of the subpackages. Pinning to the same minor version is often not enough since it may allow for different build numbers of packages built from the same recipe to be co-installed.

```yaml
requirements:
run:
- {{ pin_subpackage('subpackage_name', exact=True) }}
```

- Often multi-output recipes are used when maintainers wish to split the runtime files from the build time files. The runtime package will often include shared library files and any runtime configuration files, while the development package will include compilation time headers, and other compilation files. The intention is to make it necessary to have both the development and runtime package installed at compilation time, but at runtime, only the runtime package would be installed. We wish that maintainers of downstream packages depend on the "developement" package at build time in the host section. Thus the development package should have a run_export on the runtime package. This is done by adding the following to the development package's meta.yaml:
hmaarrfk marked this conversation as resolved.
Show resolved Hide resolved

```yaml
outputs:
- name: runtime_package_name
# no run_export should be added here. This package is missing the
# development headers and other compilation files.
# Typically this package is suffixed with `-devel`
- name: development_package_name
hmaarrfk marked this conversation as resolved.
Show resolved Hide resolved
build:
run_exports:
# the max_pin should be adjusted
- {{ pin_subpackage('runtime_package_name', max_pin='x.x') }}
requirements:
# Add the runtime package as a host dependency to ensure that files are deduplicated between the packages.
host:
- {{ pin_subpackage('runtime_package_name', exact=True) }}
run:
- {{ pin_subpackage('runtime_package_name', exact=True) }}
- other
- required
- compilation
- dependencies
Comment on lines +1718 to +1737
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
outputs:
- name: runtime_package_name
# no run_export should be added here. This package is missing the
# development headers and other compilation files.
# Typically this package is suffixed with `-devel`
- name: development_package_name
build:
run_exports:
# the max_pin should be adjusted
- {{ pin_subpackage('runtime_package_name', max_pin='x.x') }}
requirements:
# Add the runtime package as a host dependency to ensure that files are deduplicated between the packages.
host:
- {{ pin_subpackage('runtime_package_name', exact=True) }}
run:
- {{ pin_subpackage('runtime_package_name', exact=True) }}
- other
- required
- compilation
- dependencies
outputs:
- name: libfoo
# no run_export should be added here. This package is missing the
# development headers and other compilation files.
- name: libfoo-devel
build:
run_exports:
# the max_pin should be adjusted
- {{ pin_subpackage('libfoo', max_pin='x.x') }}
requirements:
# Add the runtime package as a host dependency to ensure that files are deduplicated between the packages.
host:
- {{ pin_subpackage('libfoo', exact=True) }}
run:
- {{ pin_subpackage('libfoo', exact=True) }}
- other
- required
- compilation
- dependencies

```

Once the package is created, it the development package should be added to the list of global pinnings for conda forge in the [`conda_build_config.yaml`](https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml) file.
hmaarrfk marked this conversation as resolved.
Show resolved Hide resolved

<a id="build-matrices"></a>

## Build matrices
Expand Down
Loading