The cascade config structure

config/ structure

Use a cascade structure to separate concerns so that a task/job only need to define required environmental variables in runtime

config.* file sourcing order in runtime

exp.setup -> [config.ens if do_ensemble] -> config.${MACHINE} -> config.base -> [config.${task_id}]

resources/config.* file sourcing order in setup_xml.py

config.${MACHINE} -> config.base -> [config.realtime if it is a realtime run]
NOTE:

1. [ ] means optional
2. resources config files are only visible in the workflow setup process and are not needed in runtime
3. task-specific resource settings are through variable cascade in resources/config.base. For example, we define WALLTIME for all tasks first. If we want to set a different WALLTIME for spinup fcst, we can add a variable WALLTIME_FCST_SPINUP in config/resources/config.base. setup_xml.py has a function called get_cascade_env to get the correct environmental setting in a reverse cascade way. Examples:

export NODES=${NODES:-"<nodes>1:ppn=1</nodes>"} #the NODES variable has to include the tag <nodes> or <cores>
........
........
# fcst
export NODES_FCST=${NODES_FCST:-"<nodes>3:ppn=40</nodes>"}
export WALLTIME_FCST=${WALLTIME_FCST:-"00:50:00"}

exp.setup

Set up top-level options for an experiment, such as directories, VERSION, TAG, days for a realtime run or retro period for a retro run. Users (non-NCO environment) can also preempt some variables through this file.

config.base

Most system settings go into this file

config.${MACHINE}

Define machine-dependent settings, such as COMINgfs, COMINrap, etc.
For resources configuration, this file defines
ACCOUNT, QUEUE, PARTITION, RESERVATION, NODES, WALLTIME, NATIVE, MEMORY
on different machines

config.${task_id}

Define task specific settings, such as config.clean:

export CLEAN_HRS_LOG="1440" #60 days
#
export CLEAN_HRS_UNGRIB="72"
#
export CLEAN_HRS_IC="72"
export CLEAN_HRS_LBC="72"
#
export CLEAN_HRS_FCST="72"
export CLEAN_HRS_FCST_HISTORY="24"
export CLEAN_HRS_FCST_DIAG="24"
export CLEAN_HRS_FCST_RESTART="24"
#
export CLEAN_HRS_DA="72"
export CLEAN_HRS_ENSBE="72"
export CLEAN_HRS_IODA="72"

This file is optional. We only need this file if a task accumulates many setting options (e.g. 20+?) or needs complicated logic such as defining SDL/VDL DA parameters.

config.realtime

This file defines DEADLINE and STARTTIME for rocoto-based realtime runs, not needed by and does not affect NCO