Implement "boot time measurement" test

[a-wai]

Boot time is an important indicator for device performance. We should implement a simple "boot time" test measuring this point.

We should also avoid anything likely to affect performance, such as long logs being printed over the serial console, DHCP negociation and so on. This implies we should likely first run a minimal boot test, then restart it by disabling DHCP and adding the quiet command-line option.

Relevant topic on the LAVA mailing list: https://lists.lavasoftware.org/archives/list/[email protected]/thread/AYYRZA6XJUJC33UZKTBGKGFQ3MGGL3UB/?sort=thread

[padovan]

I wonder if kexec is helpful in this scenario. Booting another kernel from your kernel would skip all the LAVA/bootloader trouble. But otoh it could came with some pitfalls I don't know.

[a-wai]

I'm pretty sure it can be useful in some way, but will come with its own limitations (e.g. not resetting the hardware like a "real" reboot would do). Plus, from my experience it can be quite flaky on arm64.

That's still an interesting idea, definitely worth experimenting with, thanks!

[a-wai]

A few notes about my experiments here:

• LAVA already provides a way to retrieve boot time, or rather, kernel-startup-to-login-prompt time: the kernel-messages built-in tests both report their duration, either one would do
• those built-in tests need a minimal level of verbosity to identify the kernel startup; practically, this requires loglevel=6
• similarly, we want to ensure DHCP negociation time doesn't affect boot time, so we need to also append ip=none to the kernel cmdline

A simple (but working) LAVA template would be the following:

context:
  extra_kernel_args: ip=off loglevel=6

{% set base_template = 'base/' + runtime + '.jinja2' %}
{%- extends base_template %}

As it is extremely similar to the existing baseline template, the latter could probably be re-used. In particular, the extra_kernel_args is LAVA-specific, so we could extend the base lava template to include this when a specific job param is present, but that is left as an exercise for the reader ;)