Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

print sccache stats in builds #413

Merged
merged 8 commits into from
Oct 29, 2024
Merged

print sccache stats in builds #413

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e19fd6e
build wheels without build isolation
jameslamb Oct 15, 2024
4d9fbd8
enable all CI
jameslamb Oct 15, 2024
be73480
Merge branch 'branch-24.12' into wheel-build-no-isolation
jameslamb Oct 23, 2024
87daa62
print stats, new patterns, just run builds in CI
jameslamb Oct 23, 2024
cb9117d
run all CI
jameslamb Oct 23, 2024
11c5340
revert unnecessary changes
jameslamb Oct 23, 2024
e9a7310
Merge branch 'branch-24.12' into wheel-build-no-isolation
jameslamb Oct 29, 2024
07c8445
revert no-build-isolation stuff
jameslamb Oct 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ repos:
cpp/cmake/modules/FindAVX\.cmake|
- id: verify-alpha-spec
- repo: https://github.com/rapidsai/dependency-file-generator
rev: v1.13.11
rev: v1.16.0
hooks:
- id: rapids-dependency-file-generator
args: ["--clean"]
Expand Down
4 changes: 4 additions & 0 deletions ci/build_cpp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ rapids-print-env

rapids-logger "Begin cpp build"

sccache --zero-stats

RAPIDS_PACKAGE_VERSION=$(rapids-generate-version) rapids-conda-retry mambabuild conda/recipes/libcuvs

sccache --show-adv-stats

rapids-upload-conda-to-s3 cpp
10 changes: 10 additions & 0 deletions ci/build_python.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,28 @@ version=$(rapids-generate-version)
export RAPIDS_PACKAGE_VERSION=${version}
echo "${version}" > VERSION

sccache --zero-stats

# TODO: Remove `--no-test` flags once importing on a CPU
# node works correctly
rapids-conda-retry mambabuild \
--no-test \
--channel "${CPP_CHANNEL}" \
conda/recipes/cuvs

sccache --show-adv-stats
sccache --zero-stats

# Build cuvs-bench for each cuda and python version
rapids-conda-retry mambabuild \
--no-test \
--channel "${CPP_CHANNEL}" \
--channel "${RAPIDS_CONDA_BLD_OUTPUT_DIR}" \
conda/recipes/cuvs-bench

sccache --show-adv-stats
sccache --zero-stats

# Build cuvs-bench-cpu only in CUDA 12 jobs since it only depends on python
# version
RAPIDS_CUDA_MAJOR="${RAPIDS_CUDA_VERSION%%.*}"
Expand All @@ -47,6 +55,8 @@ if [[ ${RAPIDS_CUDA_MAJOR} == "12" ]]; then
--channel "${CPP_CHANNEL}" \
--channel "${RAPIDS_CONDA_BLD_OUTPUT_DIR}" \
conda/recipes/cuvs-bench-cpu

sccache --show-adv-stats
fi

rapids-upload-conda-to-s3 python
16 changes: 13 additions & 3 deletions ci/build_wheel.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,20 @@ case "${RAPIDS_CUDA_VERSION}" in
;;
esac

# Hardcode the output dir
python -m pip wheel . -w dist -vvv --no-deps --disable-pip-version-check
rapids-logger "Building '${package_name}' wheel"

sccache --zero-stats

python -m pip wheel \
-w dist \
-v \
--no-deps \
--disable-pip-version-check \
.

sccache --show-adv-stats

mkdir -p final_dist
python -m auditwheel repair -w final_dist "${EXCLUDE_ARGS[@]}" dist/*

RAPIDS_PY_WHEEL_NAME="${underscore_package_name}_${RAPIDS_PY_CUDA_SUFFIX}" rapids-upload-wheels-to-s3 final_dist
RAPIDS_PY_WHEEL_NAME="${underscore_package_name}_${RAPIDS_PY_CUDA_SUFFIX}" rapids-upload-wheels-to-s3 python final_dist
10 changes: 5 additions & 5 deletions dependencies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,14 +74,14 @@ files:
- rapids_build
- cuda
- rust
py_build_py_cuvs:
py_build_cuvs:
output: pyproject
pyproject_dir: python/cuvs
extras:
table: build-system
includes:
- build
py_rapids_build_py_cuvs:
py_rapids_build_cuvs:
output: pyproject
pyproject_dir: python/cuvs
extras:
Expand All @@ -90,7 +90,7 @@ files:
includes:
- rapids_build
- build_py_cuvs
py_run_py_cuvs:
py_run_cuvs:
output: pyproject
pyproject_dir: python/cuvs
extras:
Expand All @@ -99,7 +99,7 @@ files:
- cuda_wheels
- run_py_cuvs
- depends_on_pylibraft
py_test_py_cuvs:
py_test_cuvs:
output: pyproject
pyproject_dir: python/cuvs
extras:
Expand All @@ -116,7 +116,7 @@ files:
table: build-system
includes:
- rapids_build_setuptools
py_rapids_build_py_cuvs_bench:
py_rapids_build_cuvs_bench:
output: pyproject
pyproject_dir: python/cuvs_bench
extras:
Expand Down
Loading