How to open BP series?

[pgrete]

Trying to open series with the viewer results in (same for trying the %T syntax directly):

from openpmd_viewer import OpenPMDTimeSeries
ts = OpenPMDTimeSeries("opmd.00007.bp/",backend="openpmd-api")
[AbstractIOHandlerImpl] IO Task READ_ATT failed with exception. Clearing IO queue and passing on the exception.

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[3], line 1
----> 1 ts = OpenPMDTimeSeries("opmd.00007.bp[/](http://localhost:8798/)",backend="openpmd-api")

File [~/venv/lib/python3.11/site-packages/openpmd_viewer/openpmd_timeseries/main.py:73](python3.11/site-packages/openpmd_viewer/openpmd_timeseries/main.py#line=72), in OpenPMDTimeSeries.__init__(self, path_to_dir, check_all_files, backend)
     70 self.data_reader = DataReader(backend)
     72 # Extract the iterations available in this timeseries
---> 73 self.iterations = self.data_reader.list_iterations(path_to_dir)
     75 # Check that there are files in this directory
     76 if len(self.iterations) == 0:

File [~/venv/lib/python3.11/site-packages/openpmd_viewer/openpmd_timeseries/data_reader/data_reader.py:116](http://localhost:8798/home/pgrete/venv/lib/python3.11/site-packages/openpmd_viewer/openpmd_timeseries/data_reader/data_reader.py#line=115), in DataReader.list_iterations(self, path_to_dir)
    113         file_path = re.sub(r'(\d+)(\.(?!\d).+$)', r'%T\2', first_file_name)
    114         series_name = os.path.join( path_to_dir, file_path)
--> 116     self.series = io.Series(
    117         series_name,
    118         #"opmd.%T.bp,
    119         io.Access.read_only )
    120     iterations = np.array( self.series.iterations )
    122 return iterations

RuntimeError: [json.exception.type_error.305] cannot use operator[] with a string argument with array

Hardcoding the series name fixes it:

            if is_single_file:
                file_path = path_to_dir
                series_name = file_path
            else:
                # match last occurance of integers and replace with %T wildcards
                # examples: data00000100.h5 diag4_00000500.h5 io12.0.bp
                #           te42st.1234.yolo.json scan7_run14_data123.h5
                file_path = re.sub(r'(\d+)(\.(?!\d).+$)', r'%T\2', first_file_name)
                series_name = os.path.join( path_to_dir, file_path)

            self.series = io.Series(
                #series_name,
                "opmd.%T.bp/",
                io.Access.read_only )
            iterations = np.array( self.series.iterations )

Am I doing sth wrong or is the regex parsing not working as expected?

[ax3l]

Hi @pgrete, do you have openpmd-api installed?
https://openpmd-api.readthedocs.io/en/0.15.2/install/install.html
openPMD-api is the backend we use to read ADIOS2 BP files.

What is the producing code of your ADIOS2 BP files? Is this BP3/BP4/BP5?

How did you install your post-processing environment and what exact package versions are used?

[ax3l]

It looks to me like you did some editing along the lines

--> 116     self.series = io.Series(
    117         series_name,
    118         #"opmd.%T.bp,
    119         io.Access.read_only )

of openpmd_viewer/openpmd_timeseries/data_reader/data_reader.py aka

openPMD-viewer/openpmd_viewer/openpmd_timeseries/data_reader/data_reader.py

Lines 114 to 119 in 4aac40b

	file_path = re.sub(r'(\d+)(\.(?!\d).+$)', r'%T\2', first_file_name)
	series_name = os.path.join( path_to_dir, file_path)
	self.series = io.Series(
	series_name,
	io.Access.read_only )

Before the line, can you print(series_name)?
Maybe the argument gets guessed in a funny way depending on how the files in the directory are named.

Did you try opening the series via

ts = OpenPMDTimeSeries("opmd.00007.bp",backend="openpmd-api")  # no / at the end

or

ts = OpenPMDTimeSeries("./",backend="openpmd-api")  # no / at the end

already?

In openPMD-viewer, we usually pass a path to a directory that contains an openPMD series.