Charter 2: Explore backend performance to discover surprises

Charter:

Explore backend performance to discover surprises

Testers:

• Mike Walker-Rose

Date:

• 28/05/24, 04/06/24

Duration:

• 45 mins

Scope:

• This charter is focused on improvements in performance, which can be measured using the logging of the system. It used to take 10 minutes to perform ETL, this should be faster while having not broken the system.

References:

• N/A

Test Approach:

• running after CAMS update

Notes from 28/05

• running after CAMS update (00:02:17.985)

• running again after CAMS update (before next) (Result: 00:00:46.294)

• And again... (Result: 00:00:46.25)

• Delete database data & re-run (Result: 00:00:55.586)

• Delete grib files and re-run (Result: 00:01:24:509)

• Delete grib files & database data and re-run (Result: 00:02:03.754)

• Re-run forecast (Result: 00:00:46.201)

• Delete grib files and re-run (Result: 00:02:05.325)

• Delete grib files & database data and re-run (Result: 00:02:04.835)

• Delete database data & re-run (Result: 00:00:59.114)

• Delete grib files and re-run (Result: 00:01:32.182)

Conclusion

Version	Time
Initial run after new model base date	circa 2:17
subsequent runs with existing gribs and database data	circa 46s
running the process using the same grib files, but deleting the db	circa 55-60s
running the process using the same db data but deleting the grib files	circa 1:24 - 2:05 *
running the process after deleting db data and grib files	circa 2:04

(*however 2 of 3 test runs were in range of 1:24 - 1:32, possible human error in outlier?)

Notes from 04/06

• running after CAMS update (00:01:14.499)
• re-running (00:00:18:956)
• re-running (00:00:19:893)
• Delete database data & re-run (00:00:37.895)
• Delete database data & re-run (00:00:32.17)
• Delete grib files & database data & re-run (00:01:16.459)
• Delete grib files & database data & re-run (00:00:58.869)
• Delete grib files & re-run (00:00:55.886)
• Delete grib files & re-run (00:00:36.212)

Version	Time
Initial run after new model base date	circa 1:14
Subsequent runs with existing gribs and database data	19 - 20s
Delete only database data & re-run	32 - 37s
Delete only grib files & re-run	36 - 55s
running the process after deleting db data and grib files	58s - 1:16