Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: e3sm_diags_run.log was not properly generated #848

Open
chengzhuzhang opened this issue Sep 19, 2024 · 1 comment · May be fixed by #924
Open

[Bug]: e3sm_diags_run.log was not properly generated #848

chengzhuzhang opened this issue Sep 19, 2024 · 1 comment · May be fixed by #924
Labels
bug Bug fix (will increment patch version)
Milestone
FY25 Q2 (01/01/25...

Comments

@chengzhuzhang
Copy link
Contributor

chengzhuzhang commented Sep 19, 2024 •
edited
Loading

What happened?

When using runner.run_diags() in a loop, e3sm_diags_run.log was not properly generated:

Traceback (most recent call last):
  File "/global/u2/c/chengzhu/tests/tc_analysis/run_tc_analysis.py", line 47, in <module>
    runner.run_diags([param])
  File "/global/common/software/e3sm/anaconda_envs/base/envs/e3sm_unified_1.10.0_login/lib/python3.10/site-packages/e3sm_diags/run.py", line 95, in run_diags
    move_log_to_prov_dir(params[0].results_dir)
  File "/global/common/software/e3sm/anaconda_envs/base/envs/e3sm_unified_1.10.0_login/lib/python3.10/site-packages/e3sm_diags/logger.py", line 104, in move_log_to_prov_dir
    shutil.copy(LOG_FILENAME, provenance_dir)
  File "/global/common/software/e3sm/anaconda_envs/base/envs/e3sm_unified_1.10.0_login/lib/python3.10/shutil.py", line 417, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/global/common/software/e3sm/anaconda_envs/base/envs/e3sm_unified_1.10.0_login/lib/python3.10/shutil.py", line 254, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: 'e3sm_diags_run.log'

What did you expect to happen? Are there are possible answers you came across?

No response

Minimal Complete Verifiable Example (MVCE)

import os
from e3sm_diags.parameter.core_parameter import CoreParameter
from e3sm_diags.parameter.tc_analysis_parameter import TCAnalysisParameter
from e3sm_diags.run import runner

simulations = ['tc-v1.HR.0026_0035','tc-v2.LR.2000_2014','tc-v3.HR.0006_0025','tc-v3.LR.2000_2014']
sim_names = ['theta.20180906.branch_noCNT.A_WCYCL1950S_CMIP6_HR.ne120_oRRS18v3_ICG','v2.LR.historical_0101','20240609.piCtl.ne120pg2_r025_RRSwISC6to18E3r5.chrysalis.test1','extendedOutput.v3.LR.historical_0101']

data_path = '/global/homes/c/chengzhu/tests/tc_analysis/'

for idx, sim in enumerate(simulations):
    print(sim)
    param = CoreParameter()
    param.test_data_path = data_path+sim
    param.test_name = sim_names[idx]
    param.test_start_yr = sim.split('.')[-1][0:4]
    param.test_end_yr = sim.split('.')[-1][5:9]

    param.reference_data_path = '/global/cfs/cdirs/e3sm/diagnostics/observations/Atm/tc-analysis'
    param.ref_start_yr = '1979'
    param.ref_end_yr = '2018'

    prefix = f'/global/cfs/cdirs/e3sm/www/chengzhu/tc_analysis_test/'
    param.results_dir = os.path.join(prefix, sim)
    runner.sets_to_run = ['tc_analysis']
    runner.run_diags([param]

Environment

e3sm_unified_1.10.0
@chengzhuzhang chengzhuzhang added the bug Bug fix (will increment patch version) label Sep 19, 2024
@chengzhuzhang chengzhuzhang assigned forsyth2 and unassigned forsyth2 Sep 23, 2024
@tomvothecoder
Copy link
Collaborator

tomvothecoder commented Jan 22, 2025 •
edited
Loading

I found the root cause and a solution/workaround. I will open a PR to address this issue.

The root cause

  1. The run.py module creates a single logger object at the module level.
  1. The script imports runner from run.py.
  2. All simulations use the same runner, ensuring only one logger is instantiated across simulations.
  3. During the first simulation, the e3sm_diags_run.log file is moved.
  4. Subsequent simulations fail to locate e3sm_diags_run.log, resulting in a FileNotFoundError: [Errno 2] No such file or directory: 'e3sm_diags_run.log'.

Solution

  1. Move the logger inside the Run class to ensure a new logger object is created for each simulation in the for-loop.
class Run:
    """
    Used to run diagnostics.
    A class is needed because we often need to store some
    state regarding what we need to run, like the sets selected.
    """


    def __init__(self):
        self.parser = CoreParser()
		self.logger = custom_logger(__name__)

        # The list of sets to run based on diagnostic parameters.
        self.sets_to_run = []

        # The path to the user-specified `.cfg` file using `-d/--diags` or
        # the default diagnostics `.cfg` file.
        self.cfg_path = None
  1. Update the script to instantiate a new runner for each simulation.
    Example script with updates:
import os
from e3sm_diags.parameter.core_parameter import CoreParameter
from e3sm_diags.parameter.tc_analysis_parameter import TCAnalysisParameter
from e3sm_diags.run import Run

simulations = [
    "tc-v1.HR.0026_0035",
    "tc-v2.LR.2000_2014",
    "tc-v3.HR.0006_0025",
    "tc-v3.LR.2000_2014",
]
sim_names = [
    "theta.20180906.branch_noCNT.A_WCYCL1950S_CMIP6_HR.ne120_oRRS18v3_ICG",
    "v2.LR.historical_0101",
    "20240609.piCtl.ne120pg2_r025_RRSwISC6to18E3r5.chrysalis.test1",
    "extendedOutput.v3.LR.historical_0101",
]

data_path = "/global/homes/c/chengzhu/tests/tc_analysis/"

for idx, sim in enumerate(simulations):
    print(sim)
    runner = Run()
    param = CoreParameter()
    param.test_data_path = data_path + sim
    param.test_name = sim_names[idx]
    param.test_start_yr = sim.split(".")[-1][0:4]
    param.test_end_yr = sim.split(".")[-1][5:9]

    param.reference_data_path = (
        "/global/cfs/cdirs/e3sm/diagnostics/observations/Atm/tc-analysis"
    )
    param.ref_start_yr = "1979"
    param.ref_end_yr = "2018"

    prefix = "/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/"
    param.results_dir = os.path.join(prefix, sim)
    runner.sets_to_run = ["tc_analysis"]
    runner.run_diags([param])

Solution traceback with one log file per simulation

Ignore the permission errors. I don't have access to the directory with input data.

tc-v1.HR.0026_0035
2025-01-22 12:44:11,889 [INFO]: e3sm_diags_driver.py(_save_env_yml:58) >> Saved environment yml file to: [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v1.HR.0026_0035/prov/environment.yml](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v1.HR.0026_0035/prov/environment.yml)
2025-01-22 12:44:11,900 [INFO]: e3sm_diags_driver.py(_save_parameter_files:69) >> Saved command used to: [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v1.HR.0026_0035/prov/cmd_used.txt](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v1.HR.0026_0035/prov/cmd_used.txt)
2025-01-22 12:44:11,903 [INFO]: e3sm_diags_driver.py(_save_python_script:133) >> Saved Python script to: [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v1.HR.0026_0035/prov/ipykernel_launcher.py](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v1.HR.0026_0035/prov/ipykernel_launcher.py)
2025-01-22 12:44:12,003 [ERROR]: core_parameter.py(_run_diag:343) >> Error in e3sm_diags.driver.tc_analysis_driver
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/parameter/core_parameter.py", line 340, in _run_diag
    single_result = module.run_diag(self)
                    ^^^^^^^^^^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/driver/tc_analysis_driver.py", line 78, in run_diag
    test_cyclones_hist = xr.open_dataset(test_cyclones_file).sel(lat=slice(-60, 60))[
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/backends/api.py", line 660, in open_dataset
    engine = plugins.guess_engine(filename_or_obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/backends/plugins.py", line 147, in guess_engine
    if backend.guess_can_open(store_spec):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 633, in guess_can_open
    magic_number = try_read_magic_number_from_path(filename_or_obj)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/core/utils.py", line 646, in try_read_magic_number_from_path
    with open(path, "rb") as f:
         ^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/global/homes/c/chengzhu/tests/tc_analysis/tc-v1.HR.0026_0035/cyclones_hist_theta.20180906.branch_noCNT.A_WCYCL1950S_CMIP6_HR.ne120_oRRS18v3_ICG_0026_0035.nc'
2025-01-22 12:44:12,007 [WARNING]: e3sm_diags_driver.py(main:382) >> There was not a single valid diagnostics run, no viewer created.
2025-01-22 12:44:12,008 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:12,008 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:12,026 [INFO]: logger.py(move_log_to_prov_dir:107) >> Log file saved in [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v1.HR.0026_0035/prov/e3sm_diags_run.log](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v1.HR.0026_0035/prov/e3sm_diags_run.log)
tc-v2.LR.2000_2014
2025-01-22 12:44:13,930 [INFO]: e3sm_diags_driver.py(_save_env_yml:58) >> Saved environment yml file to: [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v2.LR.2000_2014/prov/environment.yml](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v2.LR.2000_2014/prov/environment.yml)
2025-01-22 12:44:13,941 [INFO]: e3sm_diags_driver.py(_save_parameter_files:69) >> Saved command used to: [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v2.LR.2000_2014/prov/cmd_used.txt](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v2.LR.2000_2014/prov/cmd_used.txt)
2025-01-22 12:44:13,950 [INFO]: e3sm_diags_driver.py(_save_python_script:133) >> Saved Python script to: [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v2.LR.2000_2014/prov/ipykernel_launcher.py](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v2.LR.2000_2014/prov/ipykernel_launcher.py)
2025-01-22 12:44:13,950 [ERROR]: core_parameter.py(_run_diag:343) >> Error in e3sm_diags.driver.tc_analysis_driver
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/parameter/core_parameter.py", line 340, in _run_diag
    single_result = module.run_diag(self)
                    ^^^^^^^^^^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/driver/tc_analysis_driver.py", line 78, in run_diag
    test_cyclones_hist = xr.open_dataset(test_cyclones_file).sel(lat=slice(-60, 60))[
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/backends/api.py", line 660, in open_dataset
    engine = plugins.guess_engine(filename_or_obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/backends/plugins.py", line 147, in guess_engine
    if backend.guess_can_open(store_spec):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 633, in guess_can_open
    magic_number = try_read_magic_number_from_path(filename_or_obj)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/core/utils.py", line 646, in try_read_magic_number_from_path
    with open(path, "rb") as f:
         ^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/global/homes/c/chengzhu/tests/tc_analysis/tc-v2.LR.2000_2014/cyclones_hist_v2.LR.historical_0101_2000_2014.nc'
2025-01-22 12:44:13,952 [WARNING]: e3sm_diags_driver.py(main:382) >> There was not a single valid diagnostics run, no viewer created.
2025-01-22 12:44:13,952 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:13,952 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:13,952 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:13,955 [INFO]: logger.py(move_log_to_prov_dir:107) >> Log file saved in [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v2.LR.2000_2014/prov/e3sm_diags_run.log](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v2.LR.2000_2014/prov/e3sm_diags_run.log)
tc-v3.HR.0006_0025
2025-01-22 12:44:15,860 [INFO]: e3sm_diags_driver.py(_save_env_yml:58) >> Saved environment yml file to: [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.HR.0006_0025/prov/environment.yml](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.HR.0006_0025/prov/environment.yml)
2025-01-22 12:44:15,862 [INFO]: e3sm_diags_driver.py(_save_parameter_files:69) >> Saved command used to: [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.HR.0006_0025/prov/cmd_used.txt](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.HR.0006_0025/prov/cmd_used.txt)
2025-01-22 12:44:15,864 [INFO]: e3sm_diags_driver.py(_save_python_script:133) >> Saved Python script to: [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.HR.0006_0025/prov/ipykernel_launcher.py](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.HR.0006_0025/prov/ipykernel_launcher.py)
2025-01-22 12:44:15,864 [ERROR]: core_parameter.py(_run_diag:343) >> Error in e3sm_diags.driver.tc_analysis_driver
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/parameter/core_parameter.py", line 340, in _run_diag
    single_result = module.run_diag(self)
                    ^^^^^^^^^^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/driver/tc_analysis_driver.py", line 78, in run_diag
    test_cyclones_hist = xr.open_dataset(test_cyclones_file).sel(lat=slice(-60, 60))[
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/backends/api.py", line 660, in open_dataset
    engine = plugins.guess_engine(filename_or_obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/backends/plugins.py", line 147, in guess_engine
    if backend.guess_can_open(store_spec):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 633, in guess_can_open
    magic_number = try_read_magic_number_from_path(filename_or_obj)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/core/utils.py", line 646, in try_read_magic_number_from_path
    with open(path, "rb") as f:
         ^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/global/homes/c/chengzhu/tests/tc_analysis/tc-v3.HR.0006_0025/cyclones_hist_20240609.piCtl.ne120pg2_r025_RRSwISC6to18E3r5.chrysalis.test1_0006_0025.nc'
2025-01-22 12:44:15,865 [WARNING]: e3sm_diags_driver.py(main:382) >> There was not a single valid diagnostics run, no viewer created.
2025-01-22 12:44:15,866 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:15,866 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:15,866 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:15,866 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:15,868 [INFO]: logger.py(move_log_to_prov_dir:107) >> Log file saved in [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.HR.0006_0025/prov/e3sm_diags_run.log](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.HR.0006_0025/prov/e3sm_diags_run.log)
tc-v3.LR.2000_2014
2025-01-22 12:44:17,785 [INFO]: e3sm_diags_driver.py(_save_env_yml:58) >> Saved environment yml file to: [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.LR.2000_2014/prov/environment.yml](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.LR.2000_2014/prov/environment.yml)
2025-01-22 12:44:17,787 [INFO]: e3sm_diags_driver.py(_save_parameter_files:69) >> Saved command used to: [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.LR.2000_2014/prov/cmd_used.txt](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.LR.2000_2014/prov/cmd_used.txt)
2025-01-22 12:44:17,789 [INFO]: e3sm_diags_driver.py(_save_python_script:133) >> Saved Python script to: [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.LR.2000_2014/prov/ipykernel_launcher.py](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.LR.2000_2014/prov/ipykernel_launcher.py)
2025-01-22 12:44:17,789 [ERROR]: core_parameter.py(_run_diag:343) >> Error in e3sm_diags.driver.tc_analysis_driver
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/parameter/core_parameter.py", line 340, in _run_diag
    single_result = module.run_diag(self)
                    ^^^^^^^^^^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/driver/tc_analysis_driver.py", line 78, in run_diag
    test_cyclones_hist = xr.open_dataset(test_cyclones_file).sel(lat=slice(-60, 60))[
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/backends/api.py", line 660, in open_dataset
    engine = plugins.guess_engine(filename_or_obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/backends/plugins.py", line 147, in guess_engine
    if backend.guess_can_open(store_spec):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", line 633, in guess_can_open
    magic_number = try_read_magic_number_from_path(filename_or_obj)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/global/homes/v/vo13/miniforge3/envs/e3sm_diags_dev_848/lib/python3.12/site-packages/xarray/core/utils.py", line 646, in try_read_magic_number_from_path
    with open(path, "rb") as f:
         ^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/global/homes/c/chengzhu/tests/tc_analysis/tc-v3.LR.2000_2014/cyclones_hist_extendedOutput.v3.LR.historical_0101_2000_2014.nc'
2025-01-22 12:44:17,790 [WARNING]: e3sm_diags_driver.py(main:382) >> There was not a single valid diagnostics run, no viewer created.
2025-01-22 12:44:17,791 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:17,791 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:17,791 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:17,791 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:17,791 [ERROR]: run.py(run_diags:91) >> Error traceback:
Traceback (most recent call last):
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/run.py", line 89, in run_diags
    params_results = main(params)
                     ^^^^^^^^^^^^
  File "/global/u2/v/vo13/E3SM-Project/e3sm_diags/e3sm_diags/e3sm_diags_driver.py", line 401, in main
    if parameters_results[0].fail_on_incomplete and (
       ~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
2025-01-22 12:44:17,793 [INFO]: logger.py(move_log_to_prov_dir:107) >> Log file saved in [/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.LR.2000_2014/prov/e3sm_diags_run.log](https://vscode-remote+ssh-002dremote-002bperlmutter.vscode-resource.vscode-cdn.net/global/cfs/cdirs/e3sm/www/vo13/tc_analysis_test/tc-v3.LR.2000_2014/prov/e3sm_diags_run.log)

@tomvothecoder tomvothecoder linked a pull request Jan 22, 2025 that will close this issue
Open
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Bug fix (will increment patch version)
Projects
None yet
Milestone
FY25 Q2 (01/01/25 - 03/31/25)
Development

Successfully merging a pull request may close this issue.

Revamp logger module for additional features and bug fixes E3SM-Project/e3sm_diags
3 participants
@chengzhuzhang @tomvothecoder @forsyth2