Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update for GNU CI container and add dockerfile #1206

Merged
merged 25 commits into from
Jul 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .github/workflows/Dockerfile.gnu
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
#***********************************************************************
#* GNU Lesser General Public License
#*
#* This file is part of the GFDL Flexible Modeling System (FMS).
#*
#* FMS is free software: you can redistribute it and/or modify it under
#* the terms of the GNU Lesser General Public License as published by
#* the Free Software Foundation, either version 3 of the License, or (at
#* your option) any later version.
#*
#* FMS is distributed in the hope that it will be useful, but WITHOUT
#* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
#* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
#* for more details.
#*
#* You should have received a copy of the GNU Lesser General Public
#* License along with FMS. If not, see <http://www.gnu.org/licenses/>.
#***********************************************************************
# FMS CI image recipefile for GNU
# Runs on centos stream (builder has same base from redhat registry)
#
# arguments to specify versions to build can be given to docker or changed here (--build-arg name=val)
FROM spack/rockylinux9:latest as builder

ARG gcc_version=12.3.0
ARG netcdfc_version=4.9.0
ARG netcdff_version=4.6.0
ARG libyaml_version=0.2.5
ARG mpich_version=4.0.2

COPY spack.env /opt/deps/spack.env

# perl's download kept timing out
RUN sed -i 's/connect_timeout: 10/connect_timeout: 600/' /opt/spack/etc/spack/defaults/config.yaml && \
spack install gcc@${gcc_version} && \
source /opt/spack/share/spack/setup-env.sh && \
spack load gcc@${gcc_version} && \
spack compiler find && \
sed "s/COMPILER/gcc@$gcc_version/" /opt/deps/spack.env > spack.yaml && \
sed -i "s/NETCDF_C_VERSION/$netcdfc_version/" spack.yaml && \
sed -i "s/NETCDF_F_VERSION/$netcdff_version/" spack.yaml && \
sed -i "s/LIBYAML_VERSION/$libyaml_version/" spack.yaml && \
sed -i "s/MPI_LIB/mpich@$mpich_version/" spack.yaml && \
spack env activate -d . && \
spack -e . concretize -f > /opt/deps/deps.log && \
spack install --fail-fast

# copy built software to base from first image
FROM rockylinux:9

COPY --from=builder /opt/view/ /opt/view/
COPY --from=builder /opt/deps/ /opt/deps/

# input files used with --enable-input-tests
# need to be on the dev boxes if building
COPY ./fms_test_input /home/unit_tests_input

RUN dnf install -y autoconf make automake m4 libtool pkg-config zip

ENV FC="mpifort"
ENV CC="mpicc"
ENV MPICH_FC="/opt/view/bin/gfortran"
ENV MPICH_CC="/opt/view/bin/gcc"
ENV FCFLAGS="-I/opt/view/include"
ENV CFLAGS="-I/opt/view/include"
ENV LDFLAGS="-L/opt/view/lib"
ENV LD_LIBRARY_PATH="/opt/view/lib:/opt/view/lib64:/usr/local/lib:/usr/local/lib64"
ENV PATH="/opt/view/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin"
37 changes: 0 additions & 37 deletions .github/workflows/build_ubuntu_gnu.yml

This file was deleted.

39 changes: 39 additions & 0 deletions .github/workflows/github_autotools_gnu.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# 'main' required ci, does a distcheck (builds, tests, check install)
# image created off dockerfile in repo, compile/link flags are set there
name: Build libFMS test with autotools

on: [push, pull_request]

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
conf-flag: [ --disable-openmp, --enable-mixed-mode, --disable-setting-flags, --with-mpi=no]
input-flag: [--with-yaml, --enable-test-input=/home/unit_tests_input]
exclude:
- conf-flag: --with-mpi=no
input-flag: --enable-test-input=/home/unit_tests_input
container:
image: noaagfdl/fms-ci-rocky-gnu:12.3.0
env:
TEST_VERBOSE: 1
DISTCHECK_CONFIGURE_FLAGS: "${{ matrix.conf-flag }} ${{ matrix.input-flag }} ${{ matrix.io-flag }}"
SKIP_TESTS: "test_mpp_domains2.14 test_horiz_interp2.9 test_horiz_interp2.10 test_yaml_parser.5" # temporary till fixes are in
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Prepare GNU autoconf for build
run: autoreconf -if
- name: Configure the build
if: ${{ matrix.conf-flag != '--disable-setting-flags' }}
run: ./configure ${DISTCHECK_CONFIGURE_FLAGS} || cat config.log
- name: Configure the build with compiler flags
if: ${{ matrix.conf-flag == '--disable-setting-flags' }}
run: ./configure ${DISTCHECK_CONFIGURE_FLAGS} FCFLAGS="-fdefault-real-8 -fdefault-double-8 -fcray-pointer -ffree-line-length-none -I/usr/include $FCFLAGS" || cat config.log
- name: Build the library
run: make distcheck
if: ${{ matrix.conf-flag != '--with-mpi=no' }}
- name: Build the library (without test suite for serial build)
run: make
if: ${{ matrix.conf-flag == '--with-mpi=no' }}
File renamed without changes.
17 changes: 17 additions & 0 deletions .github/workflows/spack.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# template for spack environment yaml
# uppercase words get replaced before activating
spack:
specs:
- COMPILER
- MPI_LIB
- netcdf-c@NETCDF_C_VERSION ^MPI_LIB
- netcdf-fortran@NETCDF_F_VERSION
- libyaml@LIBYAML_VERSION
concretizer:
unify: true
packages:
all:
compiler: [ COMPILER ]
config:
install_tree: /opt/deps
view: /opt/view
4 changes: 3 additions & 1 deletion .github/workflows/version.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# appends -dev to the version upon release and opens pr
# CI won't run on generated PR, easiest workaround is to close + reopen
on:
release:
types: [published]
Expand All @@ -16,4 +18,4 @@ jobs:
branch-suffix: timestamp # add a timestamp to branch name
delete-branch: true # delete afer merge
title: Append dev to version number post-release
body: automated change, adds '-dev' to the version number upon releases
body: automated change, adds '-dev' to the version number upon releases. This PR will need to be closed and reopened to run CI testing.
22 changes: 14 additions & 8 deletions CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,30 @@ Required CI for pull requests are listed first.
## Pull Request CI and checks

### Build libFMS with autotools

Required GNU build test for all pull requests/pushes.
Runs `make distcheck` after configuring via GNU autotools.

Runs on a container image with spack installed dependencies, on top a rocky linux base.

Dockerfile for image is stored at .github/workflows/Dockerfile.gnu for more specific information on the CI environment.

Container environment:
gcc v7.3.0
mpich v3.3a2
netcdf v4.6.0
netcdf-fortran v4.4.4
gcc v12.3.0
mpich v4.0.2
netcdf v4.9.0
netcdf-fortran v4.6.0
autoconf v2.69
libyaml v0.2.5

container hosted at [noaagfdl/ubuntu_libfms_gnu:latest](https://hub.docker.com/r/noaagfdl/ubuntu_libfms_gnu)

`./configure` flags:
- `--enable-openmp`
`./configure` flags tested:
- `--disable-openmp`
- `--enable-mixed-mode`
- `--with-mpi=no` (disables unit testing)
- `--disable-setting-flags`
- `--with-yaml`
- `--enable-test-input=/home/unit_tests_input`


### Build libfms with cmake
Required GNU build test for all pull requests/pushes.
Expand Down
Loading