Make rrule fast forwarding stable

[fosterseth]

SUMMARY

By stable, we mean future occurrences of the rrule should be the same before and after the fast forward operation.

The problem before was that we were fast forwarding to exactly 7 days ago. For some rrules, this does not retain the old occurrences (for example, freq=HOURLY and INTERVAL=23). Thus, jobs would launch at unexpected times any time the schedule would fast forward.

This change makes sure we fast forward in increments of the rrule INTERVAL (converted to seconds), thus the new dtstart should be in the occurrence list of the old rrule.

DETAIL

Fast forward won't work for really large intervals. For example if you specify HOURLY and INTERVAL=1200 (that is 50 days worth of time), then we can't fast forward because one CHUNK of the interval doesn't fit in the period of time we are trying to fast forward (window is 30 days). In this case, we'll revert back to the old style of just updating dtstart to 7 days ago. We log a warning, so hopefully the user will update their rrule to be frequency DAYS instead of HOURS.

example log:

2024-10-24 20:01:55,628 WARNING  [-] awx.main.models.schedule Cannot fast forward rrule DTSTART:20190517T000000
RRULE:FREQ=HOURLY;INTERVAL=700000, interval is greater than the fast forward amount

PERFORMANCE

This change doesn't seem to hurt performance much. Since our fast forward window is larger (30 days vs 7), there will be more rrule objects generated, so we do expect it to cost more time. I attempted to capture it empirically.

create a bunch of old schedules

for i in range(2000):
    Schedule.objects.create(name=f's{i}', rrule='DTSTART;TZID=America/New_York:20190517T000000 RRULE:FREQ=HOURLY;INTERVAL=7', unified_job_template_id=7)

runit will update_computed_fields for each schedule

def runit():
    for sch in Schedule.objects.all():
        sch.update_computed_fields()

benchmark it

cProfile.run("runit()", "bench.txt")

before:

after:

So yeah, a slight slowdown, but I think it is worth it for the more accurate generation of occurrences.

ISSUE TYPE

• Bug, Docs Fix or other nominal change

COMPONENT NAME

• API

[PabloHiro]

Fast forward won't work for really large intervals. For example if you specify HOURLY and INTERVAL=1200 (that is 50 days worth of time), then we can't fast forward because one CHUNK of the interval doesn't fit in the period of time we are trying to fast forward (window is 30 days). In this case, we'll revert back to the old style of just updating dtstart to 7 days ago. We log a warning, so hopefully the user will update their rrule to be frequency DAYS instead of HOURS.

Can this be an HTTP 400 when a Schedule is created with an "invalid" rrule? Basically preventing this type of rrules to be created in the first place.

Also, I would specify the value of the fast forward amount in the logs, so the user does not need to guess the value by trial and error for existing "invalid" rrules

[pb82]

Ok, I think I understood what's going on:

1. To avoid computing a large number of events for an rrule with a dtstart way back in the past, we only compute from one week ago.
2. To not mess up the interval, we can't just subtract 7 days, we have to go back an amount of time that is dividable by the interval (that's what (fast_forward_seconds // interval) * interval does).

Did I get that right?

[pb82]

as @PabloHiro mentioned, this would lead to the exact problem that is being fixed. Can we just reject the rrule here?

[PabloHiro]

I wonder how we should handle the cases where a user has these "invalid" rules already. We can prevent further invalid rules, but this code has a solution to handle the existing ones.

[sonarcloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud