Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow multiple data sources in the RTW #3854

Open
sloosvel opened this issue Jan 8, 2025 · 0 comments
Open

Allow multiple data sources in the RTW #3854

sloosvel opened this issue Jan 8, 2025 · 0 comments
Assignees
Labels
Recipe Test Workflow (RTW) Items relevant to the Recipe Test Workflow

Comments

@sloosvel
Copy link
Contributor

sloosvel commented Jan 8, 2025

Currently, the config-user.yml file that gets generated by the RTW in DKRZ only looks at one directory when finding the data for CMIP5 and CMIP6:

DRS_CMIP5="DKRZ"
DRS_CMIP6="DKRZ"
ENV_NAME="experimental"
KGO_ROOT_PATH="/work/bd0854/KGO/esmvaltool_output/"
ROOTPATH_CMIP5="/work/bd0854/DATA/ESMValTool2/CMIP5_DKRZ"
ROOTPATH_CMIP6="/work/bd0854/DATA/ESMValTool2/CMIP6_DKRZ"
ROOTPATH_OBS="/work/bd0854/DATA/ESMValTool2/OBS"
ROOTPATH_OBS4MIPS="/work/bd0854/DATA/ESMValTool2/OBS"
SITE="dkrz"
       

Whereas data in DKRZ is spread over multiple directories:

rootpath:
  CMIP6:
    /work/bd0854/DATA/ESMValTool2/CMIP6_DKRZ: DKRZ
    /work/bd0854/DATA/ESMValTool2/download: ESGF
  CMIP5:
    /work/bd0854/DATA/ESMValTool2/CMIP5_DKRZ: DKRZ
    /work/bd0854/DATA/ESMValTool2/download: ESGF
  CMIP3:
    /work/bd0854/DATA/ESMValTool2/CMIP3: DKRZ
    /work/bd0854/DATA/ESMValTool2/download: ESGF
  CORDEX:
    /work/ik1017/C3SCORDEX/data/c3s-cordex/output: BADC
    /work/bd0854/DATA/ESMValTool2/download: ESGF
  OBS: /work/bd0854/DATA/ESMValTool2/OBS
  OBS6: /work/bd0854/DATA/ESMValTool2/OBS
  obs4MIPs:
    /work/bd0854/DATA/ESMValTool2/OBS: default
    /work/bd0854/DATA/ESMValTool2/download: ESGF
  ana4mips: /work/bd0854/DATA/ESMValTool2/OBS
  native6:
    /work/bd0854/DATA/ESMValTool2/RAWOBS: default
    /pool/data/ERA5: DKRZ-ERA5-GRIB
  RAWOBS: /work/bd0854/DATA/ESMValTool2/RAWOBS

This makes recipes in the RTW fail due to missing data, when the real issue is that not all directories are being taken into account when finding the data.

@sloosvel sloosvel added the Recipe Test Workflow (RTW) Items relevant to the Recipe Test Workflow label Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Recipe Test Workflow (RTW) Items relevant to the Recipe Test Workflow
Projects
None yet
Development

No branches or pull requests

3 participants