Time Cutoff #2454

cfichtlscherer · 2023-04-04T20:02:24Z

Simulations for tallying events in specific time intervals can be made more efficient using time cutoffs. Particles older than this cutoff value are killed. Keep in mind that the time variable of a particle is inherited.

gridley · 2023-04-05T01:53:22Z

Does it make sense to have time treated differently among the different particles? What is a scenario where we would want all neutrons past a certain time to die, but continue propagating protons, for example?

One thing on my wishlist is a time boundary. For example, we run particles in a fixed source calculation until some boundary in time is hit, then stop them at that point, potentially putting them into a bank of SourceSites and then handing control back to the user with the python API. It would be useful for changing the geometry or densities in a time-dependent fashion. Would functionality like this accomplish what you're looking for? I think @stchaker was going to start looking at implementing something like this pretty soon.

One last point: particles will actually survive slightly longer than the time cutoff here. You'd have to be checking for the max time while advancing along a track segment to kill them at the exact cutoff time.

cfichtlscherer · 2023-04-05T02:50:31Z

Thanks for the feedback!

That's a good question, and I am not sure. I wanted to make it consistent with the energy cutoffs and thought it was the most flexible. That could make things easier in the future, but we can also make it a fixed variable for all particles.
That sounds interesting! So you plan a functionality to modify the model in dependency of the particle time and the goal is to simulate dynamical systems iteratively? Maybe it's easier to structure it more like the depletion code, that you can specify times and densities beforehand, which are then changed?
Yes, that's right - I was a bit lazy. I think we could probably introduce the if condition before the particle advanced?

gridley · 2023-04-05T03:44:10Z

As for why this shouldn't look like depletion, it's because the interesting dynamic reactor problems have multiphysics feedback. You can't know density or temperature over time until you have a history of neutron (and potentially photon)-driven heating. So it would be handy to do the feedbacks through the python API.

Maybe a similar functionality would be useful in depletion, e.g. adjusting for critical boron concentration.

Anyway, IMO a single time cutoff makes the most sense, because the problem is nonphysical if you're artificially killing some specific type of particle. Maybe others will have a differing opinion though.

cfichtlscherer · 2023-04-05T12:14:56Z

Sure, I suggest we wait a bit again to see if there are any other opinions. Otherwise, we go with the single time cutoff.

gridley · 2023-04-05T14:14:54Z

Interestingly, it seems that MCNP does have separate time cutoffs for the different particle types, p. 337 describes it.

https://permalink.lanl.gov/object/tr?what=info:lanl-repo/lareport/LA-UR-22-30006

So I guess it does make sense to do this, although I'm not sure what the application would be.

cfichtlscherer · 2023-04-05T15:03:59Z

Maybe we want to be able to vary the time variable in the simulation at some point? (kinda: #2355, even if this example won't be part of the code)
And, you can already give the particles different time variables in the source.

cfichtlscherer · 2023-04-14T03:08:48Z

I ran clang-format over it again.

gridley

Looks great to me! I just am suggesting a slight addition to the documentation that clarifies exactly when the particles get killed.

openmc/settings.py

docs/source/io_formats/settings.rst

src/particle.cpp

cfichtlscherer · 2023-04-21T22:25:02Z

@gridley Thanks a lot for your feedback! What do you think about it now?

gridley

Looks good to me!

gridley · 2023-04-24T14:56:26Z

I'm just going to wait for one more person to give it the OK before merging, Chris!

paulromano · 2023-04-24T14:58:31Z

I'll give this a look today 😄

paulromano · 2023-04-25T04:15:29Z

docs/source/io_formats/settings.rst

+  :time_neutrons 
+    The time above which neutrons will be killed.
+
+    *Default*: 1.7976931348623157E+308


Might be better to just say "Infinity"?

yes, thats better.

paulromano · 2023-04-25T04:23:05Z

src/particle.cpp

+  if (time() > settings::time_cutoff[static_cast<int>(type())]) {
+    wgt() = 0.0;
+  }


In both MCNP and Serpent, I believe it calculates the distance to the time cutoff and won't let the particle move past that time -- here we're not really stopping the particle at the time cutoff and in principle it could go much past the time specified, which makes me worried and seems like it could create surprising behavior for the user.

The documentation does clarify the expected behavior here. Despite this, I agree with you Paul. It costs nearly the same arithmetic to check along the pathlength rather than at collision sites, so why not take this more precise approach?

paulromano

Given that @gridley agrees with me on the time boundary (i.e., that particles should be stopped exactly at the time cutoff), I'm going to make that a formal request for this to be merged. FWIW, I could see two possible approaches:

Always calculate a "distance to time cutoff" and use that when comparing the distance to collision and distance to surface.
Use the distance to collision/surface as is, but have a separate check if the time exceeded the time cutoff and if it does, move the particle a distance back such that it would exactly match the time cutoff.

After you implement this, there should be a test with a TimeFilter having a bin corresponding to times past the time cutoff, for which there should be zero flux.

cfichtlscherer · 2023-04-26T14:23:36Z

Dear @paulromano, thanks for your feedback! Personally, I like option 2 better because I think option 1 requires a lot of unnecessary computing time that we can save (@gridley or do you think option 1 is better?). As I understand it, I would try to implement it with the idea that if the particle has exceeded the time (enters the if condition), I would just push the particle back a bit?

Maybe something like (not tested or tried, just for understanding):

  if (time() > settings::time_cutoff[static_cast<int>(type())]) {
    double time_dif = time() - settings::time_cutoff[static_cast<int>(type())];
    double push_back_distance = this->speed() * time_dif;
     
    for (int j = 0; j < n_coord(); ++j) {
       coord(j).r -= push_back_distance * coord(j).u;
    }
    wgt() = 0.0;
  }

gridley · 2023-04-26T14:51:49Z

Maybe this lump of code should go into an inlined function for Particle:

Particle::move_distance(double length) {
    for (int j = 0; j < n_coord(); ++j) {
       coord(j).r += length * coord(j).u;
    }
}

Then we don't have to write out two loops. But aside from that, yeah, looks good to me!

gridley · 2023-04-26T14:53:39Z

Also, I think that code is going to be prematurely killing the particle. The wgt() would have to be zero'd after doing the tracklength tally on this last little segment, if we are going to be strictly adhering to what Paul suggested. So you'd have to set some bool that's like: hit_time_boundary and then kill the particle after doing tracklength tallying.

cfichtlscherer · 2023-04-26T15:58:56Z

Yes, that's right! I was still thinking about whether you could do it differently, because you tear the functionality apart a bit. But I think this is the best method.

cfichtlscherer · 2023-05-11T21:12:39Z

To be honest, I think the problem is the test of the surface source. As described in the previous step, I believe that the routine

        if self._model.settings.surf_source_write:
            with h5py.File("surface_source_true.h5", 'r') as f:
                source_true = f['source_bank'][()]
                # Convert dtye from mixed to a float for comparison assertion
                source_true.dtype = 'float64'

does not work as it should and I propose to change the test slightly that the data is read out more explicitly.
@gridley, @paulromano What do you think about this change?

gridley · 2023-05-11T22:03:08Z

What do you think is wrong about how it's being read? It seems like your change would be just sorting the values before comparing, right?

If anything, we should fix the underlying file reader. That would be quite bad to have wrong!

cfichtlscherer · 2023-05-11T22:34:27Z

Thanks, @gridley!

I am not entirely sure what is going on (probably not a good sign in a pull request ;-) ).

I noticed that the array with all entries of the 1000 particles has too many entries at the end. It should have 12_000, but has 13_000 entries. The additional 1000 entries are causing the test to fail for the time cutoff (I am not sure why and where it could be triggered by the changes coming with the time cutoff).

The error (creation of 1000 additional entries) must occur somewhere in these lines:

        if self._model.settings.surf_source_write:
            with h5py.File("surface_source_true.h5", 'r') as f:
                source_true = f['source_bank'][()]
                # Convert dtye from mixed to a float for comparison assertion
                source_true.dtype = 'float64'

When you read out the individual lines of source_true you get 12 values in a 1d object, so I think the values are created in the

source_true.dtype = 'float64'

statement. I thought maybe it is not working correctly, because we are looking of an array that contains different <class 'numpy.void'> entries, whereby the first two carry 3 and the other only 1 element.

gridley · 2023-05-12T14:23:33Z

Hm, unfortunately I'm not too familiar with the HDF5 interface code. Paul will probably have some useful insight here.

cfichtlscherer · 2023-05-12T14:28:46Z

Thanks and sorry it's kind of a mess - I tried to explain it as I understood it...

paulromano · 2023-06-05T19:38:58Z

@cfichtlscherer What is the status of this PR? I just read through the comments but I'm confused because it looks like tests are passing fine?

cfichtlscherer · 2023-06-06T16:26:28Z

Hey @paulromano. Yes, it seems like everything is working fine. Anyway, this is only since I made the small modification in the tests/regression_tests/surface_source/test.py. As described above, this will compare for every particle only 12 entries instead of 13 (as described, I don't know where this mysterious value number 13 is coming from). Maybe you can have a quick check if you understand what's the issue here:

#2454 (comment)

Otherwise, I am happy to merge it into the code.

An additional comment on that: If we get this merged, we can try to add a score for the killed particles:
#2531

gridley

Looks good to me!

include/openmc/particle.h

gridley · 2023-06-30T14:46:43Z

Sorry to take so long to review this.

gridley · 2023-06-30T14:47:21Z

@paulromano what do you think about this now?

paulromano · 2023-06-30T14:55:25Z

Thanks for the ping @gridley, I'll take another look after the holiday. Appreciate your patience on this @cfichtlscherer!

paulromano

Sorry for the long delay @cfichtlscherer! While digging into the test failure you were seeing, I think I did find an issue with how source sites are written to HDF5 but will need to resolve that as part of a separate PR.

paulromano · 2023-07-14T01:29:56Z

openmc/settings.py

+            elif key in ['time_neutron', 'time_photon', 'time_electron',
+                         'time_positron']:


Slightly more efficient

Suggested change

elif key in ['time_neutron', 'time_photon', 'time_electron',

'time_positron']:

elif key in ('time_neutron', 'time_photon', 'time_electron',

'time_positron'):

yes, also changed it for the energy_cutoff

paulromano · 2023-07-14T01:34:47Z

src/particle.cpp

+  bool hit_time_boundary = false;
+  if (time() > settings::time_cutoff[static_cast<int>(type())]) {
+    double time_dif = time() - settings::time_cutoff[static_cast<int>(type())];
+    time() = settings::time_cutoff[static_cast<int>(type())];
+
+    double push_back_distance = speed() * time_dif;


Suggested change

bool hit_time_boundary = false;

if (time() > settings::time_cutoff[static_cast<int>(type())]) {

double time_dif = time() - settings::time_cutoff[static_cast<int>(type())];

time() = settings::time_cutoff[static_cast<int>(type())];

double push_back_distance = speed() * time_dif;

bool hit_time_boundary = false;

double time_cutoff = settings::time_cutoff[static_cast<int>(type())]

if (time() > time_cutoff) {

double dt = time() - time_cutoff;

time() = time_cutoff;

double push_back_distance = speed() * dt;

paulromano · 2023-07-14T01:40:59Z

tests/regression_tests/time_cutoff/test.py

+        s1 = openmc.Sphere(r=100, surface_id=1)
+        s2 = openmc.Sphere(r=200, surface_id=2, boundary_type='vacuum')
+        inner_sphere = openmc.Cell(cell_id=1, name='inner sphere')
+        inner_sphere.region = -s1
+        outer_sphere = openmc.Cell(cell_id=2, name='outer sphere')
+        outer_sphere.region = +s1 & -s2
+        root = openmc.Universe(universe_id=0, name='root universe')


Please remove manual assignment of surface, cell, and universe IDs

yes this is a bad habit of mine

paulromano · 2023-07-14T01:44:29Z

tests/regression_tests/time_cutoff/test.py

+class TimeCutoffTestHarness(PyAPITestHarness):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)


There's no need to have a custom class. Simply have a function that returns a Model instance, put a @pytest.fixture decorator above it, pass it as an argument to the test function, and then assign it to PyAPITestHarness. So something like:

@pytest.fixture def time_model(): model = openmc.Model() ... return model def test_time_cutoff(time_model): harness = PyAPITestHarness('statepoint.10.h5', time_model) harness.main()

much easier!

paulromano · 2023-07-14T01:46:27Z

tests/regression_tests/time_cutoff/test.py

+        cell_filter = openmc.CellFilter(outer_sphere)
+        tally.filters = [cell_filter]


Maybe instead have a time filter with two bins, one from 0 to cutoff, and one from cutoff to 2*cutoff. First bin should have non-zero result and second bin should have zero result.

yes I like the idea. Then we can also test everything in a single cell.

paulromano · 2023-07-14T01:46:41Z

tests/regression_tests/time_cutoff/test.py

+
+        # Tally flux under time cutoff
+        tallies = openmc.Tallies()
+        tally = openmc.Tally(1)


Again, please avoid assigning IDs manually

paulromano · 2023-07-14T01:51:56Z

src/simulation.cpp

+    std::sort(simulation::surf_source_bank.begin(),
+      simulation::surf_source_bank.end(),
+      [](const SourceSite& l, const SourceSite& r) {
+        return std::tie(l.parent_id, l.progeny_id) <
+               std::tie(r.parent_id, r.progeny_id);
+      });


The surface source test already includes a sort, so I don't see why we should be doing this sort by default and making the user always pay the cost.

This was added with the help of @gridley (to be more precise, he wrote this part of the code! 😃) - see #2500. We thought this would solve the surface source bug. Since you solved this one, I will remove this block again.

paulromano · 2023-07-14T05:31:35Z

src/particle.cpp

-  for (int j = 0; j < n_coord(); ++j) {
-    coord(j).r += distance * coord(j).u;
-  }
+  this->move_distance(distance);


After spending some time with this, I figured out that this change (event though it doesn't change the logic) was the cause of the failure in the surface_source test. Can you revert this change for now (along with the associated changes you made for that test)?

Okay. We use move_distance also in the time_cutoff to move the particle back to the place where it was killed. For now, I have only changed it at the point you specified. To be honest, I still don't understand exactly what this changes 😅.
I added a comment that we can set it back to this->move_distance(distance) in the future after fixing the surface source.

Christopher Fichtlscherer added 3 commits April 4, 2023 14:55

time cutoff

c9b99ef

remove empty line

c45da7f

read time cutoff from xml

6b6892d

ask for time before particle advances

015a2f1

clang-format

9e3ab6b

gridley requested changes Apr 21, 2023

View reviewed changes

openmc/settings.py Outdated Show resolved Hide resolved

docs/source/io_formats/settings.rst Outdated Show resolved Hide resolved

src/particle.cpp Outdated Show resolved Hide resolved

Christopher Fichtlscherer added 2 commits April 21, 2023 18:05

time cutoff after updating variable

f6707e1

documentation time cutoff

2e0f15d

gridley approved these changes Apr 22, 2023

View reviewed changes

paulromano reviewed Apr 25, 2023

View reviewed changes

paulromano requested changes Apr 26, 2023

View reviewed changes

cfichtlscherer closed this Apr 26, 2023

cfichtlscherer reopened this Apr 26, 2023

Christopher Fichtlscherer added 3 commits April 26, 2023 15:50

suggested changes by gridley/paulromano

2f21784

regression test time-cutoff

f1bcd7d

change dafault cutoff to INFTY

8ba821c

Christopher Fichtlscherer added 2 commits May 11, 2023 13:29

other readout of the surface source in test

fb56eb6

inputs xml time cutoff

36a09c3

cfichtlscherer requested review from paulromano and gridley May 11, 2023 21:15

cfichtlscherer closed this May 11, 2023

cfichtlscherer reopened this May 11, 2023

gridley approved these changes Jun 30, 2023

View reviewed changes

include/openmc/particle.h Show resolved Hide resolved

paulromano requested changes Jul 14, 2023

View reviewed changes

cfichtlscherer added 10 commits July 15, 2023 14:26

change

9f82601

Incorporate Paul's suggested changes

2607045

revert changes in surface_source_test

d811a14

merge develop & resolve merge conflict settings.py

33325af

typo

6675519

clang

8f06730

revert wrong clang changes in particle_data.h

8f42e04

clang particle data

c04b93f

typo

81cb17f

typo

6482ac9

cfichtlscherer mentioned this pull request Aug 6, 2023

time-cutoff version 2.0 #2631

Merged

5 tasks

cfichtlscherer closed this Aug 6, 2023

		elif key in ['time_neutron', 'time_photon', 'time_electron',
		'time_positron']:

		cell_filter = openmc.CellFilter(outer_sphere)
		tally.filters = [cell_filter]

Time Cutoff #2454

Time Cutoff #2454

Conversation

cfichtlscherer commented Apr 4, 2023

gridley commented Apr 5, 2023

cfichtlscherer commented Apr 5, 2023

gridley commented Apr 5, 2023 • edited Loading

cfichtlscherer commented Apr 5, 2023

gridley commented Apr 5, 2023

cfichtlscherer commented Apr 5, 2023

cfichtlscherer commented Apr 14, 2023

gridley left a comment

Choose a reason for hiding this comment

cfichtlscherer commented Apr 21, 2023

gridley left a comment

Choose a reason for hiding this comment

gridley commented Apr 24, 2023

paulromano commented Apr 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulromano left a comment

Choose a reason for hiding this comment

cfichtlscherer commented Apr 26, 2023 • edited Loading

gridley commented Apr 26, 2023

gridley commented Apr 26, 2023

cfichtlscherer commented Apr 26, 2023

cfichtlscherer commented May 11, 2023

gridley commented May 11, 2023

cfichtlscherer commented May 11, 2023 • edited Loading

gridley commented May 12, 2023

cfichtlscherer commented May 12, 2023

paulromano commented Jun 5, 2023

cfichtlscherer commented Jun 6, 2023

gridley left a comment

Choose a reason for hiding this comment

gridley commented Jun 30, 2023

gridley commented Jun 30, 2023

paulromano commented Jun 30, 2023

paulromano left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cfichtlscherer Aug 2, 2023 • edited Loading

Choose a reason for hiding this comment

gridley commented Apr 5, 2023 •

edited

Loading

cfichtlscherer commented Apr 26, 2023 •

edited

Loading

cfichtlscherer commented May 11, 2023 •

edited

Loading

cfichtlscherer Aug 2, 2023 •

edited

Loading