-
Notifications
You must be signed in to change notification settings - Fork 622
Compiling FDS with GNU Fortran on x86 macOS
This tutorial should help the advanced users who are interested in compiling the FDS source code using the latest GNU Fortran distribution via Homebrew, Open MPI from source, and linking to the Intel Performance Libraries in macOS. The following instructions were developed for an x86 hardware system. If you are looking for compiling guidance for Apple chips please go here.
Navigate to the Homebrew website https://brew.sh and follow install instructions.
In your command window type:
$ brew update
$ brew install gcc
If you have openmpi installed through Homebrew you will need to uninstall it (incompatible with the MKL-MPI library wrapper compilation described here). Download the compressed OpenMPI source (for me stable release openmpi-4.0.2.tar.gz) from https://www.open-mpi.org/software/ompi/v4.0, then place it on a compilation directory of your choice (I have a Software directory in ~/Documents) and type:
$ tar -xvf openmpi-4.0.2.tar.gz
$ cd openmpi-4.0.2
In the openmpi directory configure the installation:
$ ./configure FC=gfortran CC=gcc --enable-mpi1-compatibility --prefix=/usr/local/openmpi4.0gcc --enable-mpirun-prefix-by-default --enable-mpi-fortran --enable-static --disable-shared --with-hwloc=internal --with-libevent=internal
Then make and install:
$ make -j 2
$ sudo make install
Your mpifort
and mpirun
executables will be located in /usr/local/openmpi4.0gcc/bin
, and the associated libraries to link FDS to will be located in /usr/local/openmpi4.0gcc/lib
. Make these directories available to your environment by adding to your ~/.zprofile (if using zsh) or ~/.bash_profile (if using bash) startup file:
# MPI Library:
export MPIDIST=/usr/local/openmpi4.0gcc
export PATH=$MPIDIST/bin:$PATH
export LD_LIBRARY_PATH=$MPIDIST/lib:$LD_LIBRARY_PATH
Finally, either log out and log in, or open a new terminal to apply your environment changes. Check you are locating these executables:
$ which mpirun
/usr/local/openmpi4.0gcc/bin/mpirun
This should show where your open-mpi executables (mpifort
, mpicc
compiler wrappers, mpirun
, etc.) are located. Note that if you do not see /usr/local/openmpi4.0gcc/bin/mpirun
installed from source, you should check that your .bash_profile is not already pointing to a different MPI distribution, possibly from an FDS bundle installation.
Follow the instructions provided on this wiki for macOS users.
To run finite-rate chemistry using the CVODE solver you need to install Sundials from Lawrence Livermore National Laboratory.
$ brew install llvm
Step 2: Download the sundials-6.7.0.tar.gz file (currently this is the only version that is supported in FDS) and unzip it in a local directory in your user account, <path-to-sundials>
.
$ cd <path-to-sundials>/sundials-6.7.0
$ mkdir BUILDDIR
$ mkdir INSTDIR
$ cd BUILDDIR
Next, copy the following to a text file and replace <path-to-sundials>
with the full path of the Sundials parent directory. Then copy it back to your terminal and hit enter
.
$ cmake ../ \
-DCMAKE_INSTALL_PREFIX=<path-to-sundials>/sundials-6.7.0/INSTDIR \
-DEXAMPLES_INSTALL_PATH=<path-to-sundials>/sundials-6.7.0/INSTDIR/examples \
-DCMAKE_C_COMPILER=/opt/homebrew/opt/llvm/bin/clang \
-DCMAKE_CXX_COMPILER=/opt/homebrew/opt/llvm/bin/clang++ \
-DBUILD_FORTRAN_MODULE_INTERFACE=ON \
-DEXAMPLES_ENABLE_CXX=ON \
-DEXAMPLES_ENABLE_CUDA=OFF \
-DEXAMPLES_ENABLE_F2003=ON \
-DENABLE_OPENMP=ON
Finally, make
and make install
.
$ make
$ make install
# Sundials:
export SUNDIALS_HOME=<path-to-sundials>/sundials-6.7.0/INSTDIR
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path-to-sundials>/sundials-6.7.0/INSTDIR/lib
It is advisable to open a new terminal to ensure your $PATH
is clean.
Make sure that your mpirun
command is the one installed previously in /usr/local/openmpi4.0gcc/bin/mpirun
.
$ which mpirun
/usr/local/openmpi4.0gcc/bin/mpirun
Also, make sure mpifort
points to gfortran
installed via Homebrew.
$ mpifort -v
Using built-in specs.
COLLECT_GCC=/usr/local/bin/gfortran
COLLECT_LTO_WRAPPER=/usr/local/Cellar/gcc/9.2.0_1/libexec/gcc/x86_64-apple-darwin18/9.2.0/lto-wrapper
Target: x86_64-apple-darwin18
Configured with: ../configure --build=x86_64-apple-darwin18 --prefix=/usr/local/Cellar/gcc/9.2.0_1 --libdir=/usr/local/Cellar/gcc/9.2.0_1/lib/gcc/9 --disable-nls --enable-checking=release --enable-languages=c,c++,objc,obj-c++,fortran --program-suffix=-9 --with-gmp=/usr/local/opt/gmp --with-mpfr=/usr/local/opt/mpfr --with-mpc=/usr/local/opt/libmpc --with-isl=/usr/local/opt/isl --with-system-zlib --with-pkgversion='Homebrew GCC 9.2.0_1' --with-bugurl=https://github.com/Homebrew/homebrew-core/issues --disable-multilib --with-native-system-header-dir=/usr/include --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
Thread model: posix
gcc version 9.2.0 (Homebrew GCC 9.2.0_1)
Next, go to your updated repo for FDS. You should be able to build the mpi_gnu_osx_64
and mpi_gnu_osx_64_db
targets by typing, for example,
$ cd Build/mpi_gnu_osx_64_db
$ ./make_fds.sh
Check that the preprocessor variable -DWITH_MKL
is being passed to the compiler in both compile and link phases. For example, for the first file being compiled (prec.f90):
mpifort -c -m64 -O0 -std=f2008 -ggdb -Wall -Wcharacter-truncation -Wno-target-lifetime -fcheck=all -fbacktrace
-ffpe-trap=invalid,zero,overflow -ffpe-summary=none -fall-intrinsics -cpp -DGITHASH_PP=\"FDS6.7.3-396-g6d163be33-dirty-master\"
-DGITDATE_PP=\""Tue Jan 21 15:46:34 2020 -0500\"" -DBUILDDATE_PP=\""Jan 28, 2020 15:16:42\"" -DCOMPVER_PP=\""Gnu gfortran GCC"\"
-DWITH_MKL -I/opt/intel18/compilers_and_libraries_2018.2.164/mac/mkl/include ../../Source/prec.f90
If you have multiple meshes, it is most efficient to use MPI to run the job in parallel. First, determine how many cores are available: Click the Apple icon in the upper left of your Desktop and go to About This Mac
. Then click on the System Report...
. My iMac has 4 cores. So, that is what I will use in my example below. I'm going to test the code on an example with 4 meshes. Cd to the fds/Verification/WRF/
directory. We will run the case wrf_time_ramp.fds
.
$ mpirun -n 4 /Users/mnv/Documents/FIREMODELS_FORK/fds/Build/mpi_gnu_osx_64/fds_mpi_gnu_osx_64 wrf_time_ramp.fds
Starting FDS ...
MPI Process 0 started on bevo.el.nist.gov
MPI Process 1 started on bevo.el.nist.gov
MPI Process 2 started on bevo.el.nist.gov
MPI Process 3 started on bevo.el.nist.gov
Reading FDS input file ...
Fire Dynamics Simulator
Current Date : January 28, 2020 17:00:10
Revision : FDS6.7.3-396-g6d163be33-dirty-master
Revision Date : Tue Jan 21 15:46:34 2020 -0500
Compiler : Gnu gfortran GCC
Compilation Date : Jan 28, 2020 16:13:43
MPI Enabled; Number of MPI Processes: 4
OpenMP Disabled
MPI version: 3.1
MPI library version: Open MPI v4.0.2, package: Open MPI [email protected] Distribution, ident: 4.0.2, repo rev: v4.0.2, Oct 07, 2019
Job TITLE : Test of mean velocity ramps for one-way WRF coupling with FDS
Job ID string : wrf_time_ramp
Time Step: 1, Simulation Time: 0.46 s
Time Step: 2, Simulation Time: 0.93 s
Time Step: 3, Simulation Time: 1.39 s
Time Step: 4, Simulation Time: 1.86 s
Time Step: 5, Simulation Time: 2.32 s
Time Step: 6, Simulation Time: 2.78 s
Time Step: 7, Simulation Time: 3.25 s
Time Step: 8, Simulation Time: 3.71 s
Time Step: 9, Simulation Time: 4.18 s
Time Step: 10, Simulation Time: 4.64 s
Time Step: 11, Simulation Time: 5.00 s
STOP: FDS completed successfully (CHID: wrf_time_ramp)