Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build wheels without build isolation #1473

Merged
merged 31 commits into from
Oct 22, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
b983143
test wheel-build sccache fixes
jameslamb Oct 11, 2024
9e2ceef
fiddle with options
jameslamb Oct 11, 2024
8c747f5
run fewer build jobs
jameslamb Oct 11, 2024
7cf37ee
build with --no-build-isolation
jameslamb Oct 11, 2024
4477b70
paths
jameslamb Oct 11, 2024
d6f05c6
more paths
jameslamb Oct 11, 2024
d18c3f7
Merge branch 'wheel-build-sccache' of github.com:jameslamb/cuspatial …
jameslamb Oct 11, 2024
6c64e73
restore all CI
jameslamb Oct 11, 2024
cbfaf85
only pre-install build backend
jameslamb Oct 14, 2024
9744bcf
Merge branch 'wheel-build-sccache' of github.com:jameslamb/cuspatial …
jameslamb Oct 14, 2024
8256e35
only run wheel builds
jameslamb Oct 14, 2024
28e34eb
GHA fix
jameslamb Oct 14, 2024
8251aaa
go back to installing everything
jameslamb Oct 14, 2024
130e761
re-enable all CI
jameslamb Oct 14, 2024
29b005e
move PIP_CONSTRAINT stuff back, remove 'declare'
jameslamb Oct 14, 2024
fc8d575
Merge branch 'wheel-build-sccache' of github.com:jameslamb/cuspatial …
jameslamb Oct 14, 2024
ec06a4a
Merge branch 'branch-24.12' into wheel-build-sccache
bdice Oct 16, 2024
cb6b07d
only use an un-isolated build for libcuspatial
jameslamb Oct 16, 2024
75eaf12
Merge branch 'wheel-build-sccache' of github.com:jameslamb/cuspatial …
jameslamb Oct 16, 2024
7b87cd6
Merge branch 'branch-24.12' of github.com:rapidsai/cuspatial into whe…
jameslamb Oct 17, 2024
f59a067
test with new rapids-dependency-file-generator
jameslamb Oct 17, 2024
1199e31
enable all CI
jameslamb Oct 17, 2024
5e54225
shell variable
jameslamb Oct 17, 2024
9f979b5
actually turn off build isolation
jameslamb Oct 17, 2024
aa62971
more fiddling with PIP_NO_BUILD_ISOLATION
jameslamb Oct 17, 2024
b4cd177
ugh ok yeah it really is PIP_NO_BUILD_ISOLATION=0
jameslamb Oct 17, 2024
69fa9e8
try more libcuspatial builds
jameslamb Oct 17, 2024
8b841c2
restore all CI
jameslamb Oct 17, 2024
e5d02b9
use published rapids-dependency-file-generator packages
jameslamb Oct 22, 2024
dd7a7a6
run another build, hopefully with better cache hit rate
jameslamb Oct 22, 2024
336bc27
print sccache stats for conda builds
jameslamb Oct 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 41 additions & 1 deletion ci/build_wheel.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,49 @@ rapids-generate-version > ./VERSION

RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen ${RAPIDS_CUDA_VERSION})"

touch /tmp/constraints-build.txt

if [[ "${package_name}" == "cuspatial" ]]; then
# Downloads libcuspatial wheel from this current build,
# then ensures 'cuspatial' wheel builds always use the 'libcuspatial' just built in the same CI run.
RAPIDS_PY_WHEEL_NAME="libcuspatial_${RAPIDS_PY_CUDA_SUFFIX}" rapids-download-wheels-from-s3 cpp /tmp/libcuspatial_dist
echo "libcuspatial-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo /tmp/libcuspatial_dist/libcuspatial_*.whl)" > /tmp/constraints-build.txt
fi
jameslamb marked this conversation as resolved.
Show resolved Hide resolved

rapids-logger "Generating build requirements"
declare -r matrix_selectors="cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION};cuda_suffixed=true"
jameslamb marked this conversation as resolved.
Show resolved Hide resolved

rapids-dependency-file-generator \
--output requirements \
KyleFromNVIDIA marked this conversation as resolved.
Show resolved Hide resolved
--file-key "py_build_${package_name}" \
--matrix "${matrix_selectors}" \
| tee /tmp/requirements-build.txt

rapids-dependency-file-generator \
--output requirements \
--file-key "py_rapids_build_${package_name}" \
--matrix "${matrix_selectors}" \
| tee -a /tmp/requirements-build.txt

rapids-logger "Installing build requirements"
python -m pip install \
-v \
--prefer-binary \
--constraint /tmp/constraints-build.txt \
-r /tmp/requirements-build.txt

cd "${package_dir}"

python -m pip wheel . -w dist -vvv --no-deps --disable-pip-version-check
rapids-logger "Building '${package_name}' wheel"
python -m pip wheel \
-w dist \
-v \
--no-build-isolation \
--no-deps \
--disable-pip-version-check \
.

sccache --show-adv-stats
vyasr marked this conversation as resolved.
Show resolved Hide resolved

mkdir -p final_dist
python -m auditwheel repair \
Expand Down
11 changes: 0 additions & 11 deletions ci/build_wheel_cuspatial.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,4 @@

set -euo pipefail

RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen ${RAPIDS_CUDA_VERSION})"

# Downloads libcuspatial wheel from this current build,
# then ensures 'cuspatial' wheel builds always use the 'libcuspatial' just built in the same CI run.
#
# Using env variable PIP_CONSTRAINT is necessary to ensure the constraints
# are used when creating the isolated build environment.
RAPIDS_PY_WHEEL_NAME="libcuspatial_${RAPIDS_PY_CUDA_SUFFIX}" rapids-download-wheels-from-s3 cpp /tmp/libcuspatial_dist
echo "libcuspatial-${RAPIDS_PY_CUDA_SUFFIX} @ file://$(echo /tmp/libcuspatial_dist/libcuspatial_*.whl)" > /tmp/constraints.txt
export PIP_CONSTRAINT="/tmp/constraints.txt"

ci/build_wheel.sh cuspatial python/cuspatial python
Loading