Event iterators stuck at the last event of a given type

[mexanick]

Description
When iterating e.g. over the MC events the iterator seems to stuck at the last MC event low level block.

Reproduction

f = SimTelFile('path/to/test.simtel.gz')
n_total = 0
n_triggered = 0

for event_mc in f.iter_mc_events():
    n_total += 1

for event_array in f.iter_array_events():
    if event_array['type'] == 'calibration':
        continue
    if event_array['trigger_information']['n_triggered_telescopes'] > 0:
        n_triggered += 1
        
print(f'Total number of MC events: {n_total}')
print(f'Total number of triggered events: {n_triggered}')

yields smth like

Total number of MC events: 1000
Total number of triggered events: 1

while

f = SimTelFile('path/to/test.simtel.gz')
n_total = 0
n_triggered = 0

for event_array in f.iter_array_events():
    if event_array['type'] == 'calibration':
        continue
    if event_array['trigger_information']['n_triggered_telescopes'] > 0:
        n_triggered += 1
        
for event_mc in f.iter_mc_events():
    n_total += 1

print(f'Total number of MC events: {n_total}')
print(f'Total number of triggered events: {n_triggered}')

yields smth like

Total number of MC events: 0
Total number of triggered events: 1000

Environment
The test file is a dummy file for NSB study produced from dummy corsika events with 100 pedestal and 1000 MC events, all triggered. File was produced with corsika6.9_simtelarray_2018-11-07 SW release.
pyeventio is of the latest PyPi version

Possible fix
The function below (and alike) does not reset the iterator position before the loop.

pyeventio/eventio/simtel/simtelfile.py

Line 203 in c864d16

def iter_mc_events(self):

Thus, without reloading the SimTelFile object it is not possible to iterate over it correctly more than one time.

[maxnoe]

This is impossible for many file types, e.g. zstandard compressed files. These are not backwards seekable, only forward. Gzip compressed files are a middle ground, where two implementations exist, one that is faster without backseeking and one where backseeking is possible but very slow.

So you will just need to open the input file again, if you start a new iteration.

If you are just interested in the total numbers of events that triggered and events that were simulated,
have a look into the histograms at the end of a file.

I agree that this is a bit surprising, I will look into if it is possible to improve this by raising an error or a warning if you attempt this.

[mexanick]

Thanks for a reply. I provided the simplest possible example, just to show the behavior. Indeed, it was quite a bit surprising. Perhaps you can add a note or warning in the documentation?