Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve CI #44

Merged
merged 13 commits into from
Feb 9, 2024
36 changes: 18 additions & 18 deletions .github/workflows/__build_base.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Generate base images for each backend. Images are stored on Github registry
# as Docker format inconditionnaly and as Singularity format only if requested.
# Docker images are used for building Kokkos FFT, while Singularity images are
# used for test execution on the righteous hardware. This workflow can only be
# invoked through another workflows.
# Generate base images for each compiler environment. Images are stored on
# Github registry as Docker format inconditionnaly and as Singularity format
# only if requested. Docker images are used for building Kokkos FFT, while
# Singularity images are used for test execution on the righteous hardware.
# This workflow can only be invoked through another workflows.

name: Build base images

Expand All @@ -25,14 +25,14 @@ jobs:

strategy:
matrix:
backend:
- name: openmp
image:
- name: gcc
use_singularity: false
- name: cuda
- name: nvcc
use_singularity: true
- name: hip
- name: rocm
use_singularity: false
- name: sycl
- name: intel
use_singularity: false

steps:
Expand Down Expand Up @@ -61,19 +61,19 @@ jobs:
- name: Build Docker image
run: |
docker build \
-t ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.name }}_${{ inputs.image_name_suffix }} \
--cache-from ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.name }}_main \
-t ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.image.name }}_${{ inputs.image_name_suffix }} \
--cache-from ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.image.name }}_main \
--build-arg BUILDKIT_INLINE_CACHE=1 \
--progress=plain \
docker/${{ matrix.backend.name }}
docker/${{ matrix.image.name }}

- name: Push Docker image
run: docker push ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.name }}_${{ inputs.image_name_suffix }}
run: docker push ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.image.name }}_${{ inputs.image_name_suffix }}

- name: Convert Docker image to Singularity
run: singularity build base.sif docker://ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.name }}_${{ inputs.image_name_suffix }}
if: ${{ matrix.backend.use_singularity }}
run: singularity build base.sif docker://ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.image.name }}_${{ inputs.image_name_suffix }}
if: ${{ matrix.image.use_singularity }}

- name: Push Singularity image
run: singularity push base.sif oras://ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.name }}_singularity_${{ inputs.image_name_suffix }}
if: ${{ matrix.backend.use_singularity }}
run: singularity push base.sif oras://ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.image.name }}_singularity_${{ inputs.image_name_suffix }}
if: ${{ matrix.image.use_singularity }}
28 changes: 17 additions & 11 deletions .github/workflows/build_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -82,18 +82,22 @@ jobs:
matrix:
backend:
- name: openmp
image: gcc
c_compiler: gcc
cxx_compiler: g++
cmake_flags: -DKokkos_ENABLE_OPENMP=ON
- name: cuda
image: nvcc
c_compiler: gcc
cxx_compiler: g++
cmake_flags: -DKokkos_ENABLE_CUDA=ON -DKokkos_ARCH_AMPERE80=ON
- name: hip
image: rocm
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may prefer to call this image hipcc.

c_compiler: hipcc
cxx_compiler: hipcc
cmake_flags: -DKokkos_ENABLE_HIP=ON -DKokkos_ARCH_VEGA90A=ON
- name: sycl
image: intel
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As well, better to call this image dpcpp?

c_compiler: icx
cxx_compiler: icpx
# building for Intel PVC was unsuccessful without the proper device
Expand Down Expand Up @@ -124,7 +128,7 @@ jobs:

- name: Configure
run: |
docker run -v ${{ github.workspace }}:/work ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.name }}_${{ needs.check_docker_files.outputs.image_name_suffix }} \
docker run -v ${{ github.workspace }}:/work ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.image }}_${{ needs.check_docker_files.outputs.image_name_suffix }} \
cmake -B build \
-DCMAKE_INSTALL_PREFIX=/work/install \
-DCMAKE_BUILD_TYPE=${{ env.BUILD_TYPE }} \
Expand All @@ -139,12 +143,12 @@ jobs:

- name: Build
run: |
docker run -v ${{ github.workspace }}:/work ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.name }}_${{ needs.check_docker_files.outputs.image_name_suffix }} \
docker run -v ${{ github.workspace }}:/work ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.image }}_${{ needs.check_docker_files.outputs.image_name_suffix }} \
cmake --build build -j $(( $(nproc) * 2 + 1 ))

- name: Prepare artifacts
# this is mandatory to preserve execution rights
run: tar -cvf tests_${{ matrix.backend.name }}.tar build/
run: tar -cvf tests_${{ matrix.backend.image }}.tar build/
if: ${{ matrix.target.name == 'native' }}

- name: Save artifacts
Expand All @@ -157,21 +161,21 @@ jobs:

- name: Install
run: |
docker run -v ${{ github.workspace }}:/work ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.name }}_${{ needs.check_docker_files.outputs.image_name_suffix }} \
docker run -v ${{ github.workspace }}:/work ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.image }}_${{ needs.check_docker_files.outputs.image_name_suffix }} \
cmake --install build

- name: Configure and build test code
# Use the built and installed Kokkos FFT library to build a test code
run: |
docker run -v ${{ github.workspace }}:/work ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.name }}_${{ needs.check_docker_files.outputs.image_name_suffix }} \
docker run -v ${{ github.workspace }}:/work ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.image }}_${{ needs.check_docker_files.outputs.image_name_suffix }} \
cmake -B build_test \
-DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} \
-DCMAKE_C_COMPILER=${{ matrix.backend.c_compiler }} \
-DCMAKE_CXX_COMPILER=${{ matrix.backend.cxx_compiler }} \
-DCMAKE_CXX_STANDARD=17 \
-DCMAKE_PREFIX_PATH=/work/install \
install_test
docker run -v ${{ github.workspace }}:/work ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.name }}_${{ needs.check_docker_files.outputs.image_name_suffix }} \
docker run -v ${{ github.workspace }}:/work ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.image }}_${{ needs.check_docker_files.outputs.image_name_suffix }} \
cmake --build build_test -j $(( $(nproc) * 2 + 1 ))

test:
Expand All @@ -189,9 +193,11 @@ jobs:
backend:
# run CUDA tests on Ruche supercomputer
- name: cuda
image: nvcc
runner: [self-hosted, cuda]
# run OpenMP tests on Azure server
- name: openmp
image: gcc
runner: ubuntu-latest

steps:
Expand All @@ -206,18 +212,18 @@ jobs:

- name: Pull Singularity image
# pulling the image in advance seems necessary as sometimes invoking `singularity run` on the image URL fails because it cannot find ghcr.io
run: singularity pull oras://ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.name }}_singularity_${{ needs.check_docker_files.outputs.image_name_suffix }}:latest
run: singularity pull oras://ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.image }}_singularity_${{ needs.check_docker_files.outputs.image_name_suffix }}:latest
if: ${{ matrix.backend.name == 'cuda' }}

- name: Run CUDA tests within Slurm job and Singularity image
run: |
srun --nodes=1 --time=01:00:00 -p gpua100 --gres=gpu:1 \
singularity run --nv --bind $PWD/build:/work/build -H /work/build base_${{ matrix.backend.name }}_singularity_${{ needs.check_docker_files.outputs.image_name_suffix }}_latest.sif \
ctest
singularity run --nv --bind $PWD/build:/work/build -H /work/build base_${{ matrix.backend.image }}_singularity_${{ needs.check_docker_files.outputs.image_name_suffix }}_latest.sif \
ctest --verbose
if: ${{ matrix.backend.name == 'cuda' }}

- name: Run OpenMP tests within Docker image
run: |
docker run -v $PWD/build:/work/build -w /work/build ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.name }}_${{ needs.check_docker_files.outputs.image_name_suffix }} \
ctest
docker run -v $PWD/build:/work/build -w /work/build ghcr.io/cexa-project/kokkos-fft/base_${{ matrix.backend.image }}_${{ needs.check_docker_files.outputs.image_name_suffix }} \
ctest --verbose
cedricchevalier19 marked this conversation as resolved.
Show resolved Hide resolved
if: ${{ matrix.backend.name == 'openmp' }}
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Loading