-
Notifications
You must be signed in to change notification settings - Fork 148
Fedora compiler environment
Noted differences between Fedora Core (FC) 21 and Fedora Core 23. Useful commands to find the software we need to compile MOM6.
- Package management: FC21: yum, FC23: dnf
- List files in a package: FC21: rpm -ql [pkg], FC23: dnf repoquery -l [pkg]
- FC21: pkg-config configuration files are not available for openmpi, see below.
These instructions closely follow the instructions outlined for GFDL's HPC platforms.
Execute the following commands to install the compilation prerequisites on an Fedora machine:
sudo dnf install csh make gcc-gfortran openmpi openmpi-devel netcdf netcdf-devel netcdf-fortran netcdf-fortran-devel netcdf-fortran-openmpi netcdf-fortran-openmpi-develOther recommended tools to use git (and unzip), X windows via SSH and syntax colored vim (restart shell session: logout and login again):
sudo dnf install git unzip xauth xterm vim-enhancedYou are ready to use git to download and setup the code repository on the AWS instance.
The following commands make use of a file build/gnu/env that can contain any local environment changes need for compilation:
mkdir -p build/gnu
echo > build/gnu/env# Need openmpi path before all other paths to get correct nc-config for netcdf with openmpi
export PATH=/usr/lib64/openmpi/bin:${PATH}
export PKG_CONFIG_PATH=${PKG_CONFIG_PATH}:/usr/lib64/openmpi/lib/pkgconfigFor FC21:
cd src/mkmf/templates
cp linux-gnu.mk linux-gnu-fedora21.mk
# The only real difference is supplying include file and library paths instead
# of using pkg-config. Replace (-) lines with (+) lines. Use different paths
# if you installed netcdf or openmpi in other locations. This demonstrates use
# of openmpi over mpich2.
-FFLAGS += $(shell pkg-config --cflags-only-I mpich2-c)
+FFLAGS += -I/usr/include/openmpi-x86_64 -I/usr/include/openmpi-x86_64/
-CFLAGS += $(shell pkg-config --cflags-only-I mpich2-c)
+CFLAGS += -I/usr/include/openmpi-x86_64 -I/usr/include/openmpi-x86_64/
-LIBS := $(shell nc-config --flibs) $(shell pkg-config --libs mpich2-f90)
+LIBS := $(shell nc-config --flibs) -L/usr/lib64/openmpi/libFor FC23:
cd src/mkmf/templates
cp linux-gnu.mk linux-gnu-fedora23.mk
# Since we chose to using openmpi in this example, change references of mpich2-* to ompi-*
# in linux-gnu-fedora23.mkNOTE: Change the template to linux-gnu-fedora21.mk for FC21 as needed in the following examples.
To build the Makefile use:
mkdir -p build/gnu/shared/repro/
(cd build/gnu/shared/repro/; rm -f path_names; \
../../../../src/mkmf/bin/list_paths -l ../../../../src/FMS; \
../../../../src/mkmf/bin/mkmf -t ../../../../src/mkmf/templates/linux-gnu-fedora23.mk -p libfms.a -c "-Duse_libMPI -Duse_netCDF -DSPMD" path_names)To build the FMS library (libfms.a):
csh/tcsh:
(cd build/gnu/shared/repro/; source ../../env; make NETCDF=3 REPRO=1 FC=mpif90 CC=mpicc libfms.a -j)bash:
(cd build/gnu/shared/repro/; . ../../env; make NETCDF=3 REPRO=1 FC=mpif90 CC=mpicc libfms.a -j)Notice the addition of FC=mpif90 and CC=mpicc above, which is necessary to find the MPI header files.
To create the Makefile for compiling MOM6:
mkdir -p build/gnu/ocean_only/repro/
(cd build/gnu/ocean_only/repro/; rm -f path_names; \
../../../../src/mkmf/bin/list_paths -l ./ ../../../../src/MOM6/{config_src/infra/FMS1,config_src/memory/dynamic_symmetric,config_src/drivers/solo_driver,config_src/external,src/{*,*/*}}/ ; \
../../../../src/mkmf/bin/mkmf -t ../../../../src/mkmf/templates/linux-gnu-fedora23.mk -o '-I../../shared/repro' -p 'MOM6 -L../../shared/repro -lfms' -c "-Duse_libMPI -Duse_netCDF -DSPMD" path_names)To build the MOM6 executable:
csh/tcsh:
(cd build/gnu/ocean_only/repro/; source ../../env; make NETCDF=3 REPRO=1 FC=mpif90 LD=mpif90 MOM6 -j)bash:
(cd build/gnu/ocean_only/repro/; . ../../env; make NETCDF=3 REPRO=1 FC=mpif90 LD=mpif90 MOM6 -j)This time note the addition of LD=mpif90.
For any example in ocean_only/ that does NOT have an INPUT/ directory, e.g. double_gyre:
csh/tcsh:
cd ocean_only/double_gyre/
mkdir RESTART
(source ./../../build/gnu/env ; mpirun -np 4 ../../build/gnu/ocean_only/repro/MOM6)bash:
cd ocean_only/double_gyre/
mkdir RESTART
(. ./../../build/gnu/env ; mpirun -np 4 ../../build/gnu/ocean_only/repro/MOM6)MOM Day 0.000 0: En 1.424361E-13, MaxCFL 0.00000, Mass 5.288178268008E+18
Total Energy: 4126FC96B8ADCFE7 7.5322736070108123E+05
Total Mass: 5.2881782680077681E+18, Change: 0.0000000000000000E+00 Error: 0.00000E+00 ( 0.0E+00)
MOM Day 1.000 72: En 4.580432E-06, MaxCFL 0.00023, Mass 5.288178268008E+18
Total Energy: 42B607A80A852EA4 2.4222139843886641E+13
Total Mass: 5.2881782680077681E+18, Change: -1.7202096100332227E+01 Error: -1.72021E+01 (-3.3E-18)
MOM Day 2.000 144: En 8.958206E-06, MaxCFL 0.00040, Mass 5.288178268008E+18
Total Energy: 42C58AE53347EF57 4.7372590026718680E+13
Total Mass: 5.2881782680077681E+18, Change: -6.7986416459937118E+01 Error: -6.79864E+01 (-1.3E-17)
MOM Day 3.000 216: En 1.166708E-05, MaxCFL 0.00045, Mass 5.288178268008E+18
Total Energy: 42CC0E8B5CE0D7C2 6.1697586479535516E+13
Total Mass: 5.2881782680077681E+18, Change: 1.2942244735786460E+01 Error: 1.29422E+01 ( 2.4E-18)
MOM Day 4.000 288: En 1.542296E-05, MaxCFL 0.00044, Mass 5.288178268008E+18
Total Energy: 42D28B60D589706D 8.1559337444801703E+13
Total Mass: 5.2881782680077681E+18, Change: -1.6385457801725806E+01 Error: -1.63855E+01 (-3.1E-18)
MOM Day 5.000 360: En 1.826022E-05, MaxCFL 0.00053, Mass 5.288178268008E+18
Total Energy: 42D5F4B97D5D6AA0 9.6563312817578500E+13
Total Mass: 5.2881782680077681E+18, Change: 3.7420734548125125E+01 Error: 3.74207E+01 ( 7.1E-18)
MOM Day 6.000 432: En 2.397431E-05, MaxCFL 0.00062, Mass 5.288178268008E+18
Total Energy: 42DCD397A5E1A4E8 1.2678043163201962E+14
Total Mass: 5.2881782680077681E+18, Change: -3.8078863737074414E+01 Error: -3.80789E+01 (-7.2E-18)
MOM Day 7.000 504: En 2.927962E-05, MaxCFL 0.00073, Mass 5.288178268008E+18
Total Energy: 42E19A5109E63388 1.5483585791222025E+14
Total Mass: 5.2881782680077681E+18, Change: -3.5114098448941945E+01 Error: -3.51141E+01 (-6.6E-18)
MOM Day 8.000 576: En 3.486441E-05, MaxCFL 0.00081, Mass 5.288178268008E+18
Total Energy: 42E4F5D995BB638B 1.8436920009398034E+14
Total Mass: 5.2881782680077681E+18, Change: -6.0992962031751858E+00 Error: -6.09930E+00 (-1.2E-18)
MOM Day 9.000 648: En 4.195565E-05, MaxCFL 0.00090, Mass 5.288178268008E+18
Total Energy: 42E9393C9CAC7A8E 2.2186896584802044E+14
Total Mass: 5.2881782680077681E+18, Change: 5.1411840814233216E+01 Error: 5.14118E+01 ( 9.7E-18)
MOM Day 10.000 720: En 4.604729E-05, MaxCFL 0.00100, Mass 5.288178268008E+18
Total Energy: 42EBAEF71B9AAA47 2.4350627232699422E+14
Total Mass: 5.2881782680077681E+18, Change: -1.9384762671898812E+01 Error: -1.93848E+01 (-3.7E-18)
20160512 184045.816: Memuse(MB) at Memory HiWaterMark= 2.965E+01 3.172E+01 8.756E-01 3.020E+01
MPP_DOMAINS_STACK high water mark= 3680Supplemental instructions on how to start a single instance on the Amazon Web Services (AWS) Elastic Compute Cloud (EC2)
For Fedora Core 23 (FC23), several pre-constructed base images are available from the Community AMIs. The ami name will be different for different compute regions and zones. By searching on "Fedora-Cloud-Base-23" you can see all the different zones and types available. Linux Amazon Machine Images use one of two types of virtualization: paravirtual (PV) or hardware virtual machine (HVM). The main difference between PV and HVM AMIs is the way in which they boot and whether they can take advantage of special hardware extensions (CPU, network, and storage) for better performance. For example, the image ami-20203441 is a HVM image which can take advantage of the special hardware extensions.
Reference: Nic Hannah, http://nh.id.au/run-an-ocean-model-in-30-mins.html
Author note: Once you have an account established with Amazon AWS and are familiar with how to request and login to the instance, it does literally take 30 minutes or less to run the above example.