Battle Testing

[lastmjs]

• I would like to make a long-running test, a test that can run for days, that battle-tests Azle in an environment with multiple canisters using many of the IC APIs. It should incorporate the following perhaps:
  • Fuzz testing
  • Memory leak testing
  • Property testing
• Design canisters and methods for testing
• Figure out fuzzing framework
• Figure out how to monitor for memory leaks
• Property test as well? Could we make a framework that works well for the property tests as well? I kind of want to generate a crazy set of canisters to do this
• create a general-purpose fuzz testing framework
• create fuzz workflow in the gui, spin up all tests, simply fuzz
• change benchmarks to not have a list but just have a key for each update method
• fuzzing could check for crashes and memory growth, and also do the benchmarks at the end
• Performance testing as well?
• It would be nice to have a workflow specifically called Fuzz Test
  • we should be able to explicitly say which tests to include instead of exclude
  • we need a way to understand if memory leaks are occurring...maybe if we know a test doesn't perform state changes like writing into a database, then in the cuzz.json we can ask it to check for any memory increase...and if there is, then we fail as a memory leak since it is unexpected?
• Make sure to run fuzz testing on the property tests...those will be perfect because of how crazy they are
• all child processes should be killed (spawn)...we need a better way to display multiple canisters
• After 6 hours the fuzz test jobs should succeed with green, right now they just fail
• Make a Fuzz Test workflow
• if the Fuzz Tests fail, see if you can get an error message to be reported, right now it just says that the process exited with exit code 1
• When logging the errors, please log the params as well so that we can at least attempt to reproduce that
• Check for memory leaks
  • cross-canister calls with uint8array params that are large, test both successful and unsuccessful
• cuzz.json should let you choose the length of all arrays and strings etc