Skip to content

Draft: Quick QEMU booting from saved VM state

David Brown requested to merge David.Brown/rules_labgrid:savevm into main

These commits demonstrate a hacky way of using QEMU's built-in support for saving (and then loading) the VM state to disk to significantly improve QEMU boot times (down to ~10 seconds)

It works by defining a labgrid_genrule and associated labgrid_config_toolchain target that runs a labgrid enabled python script. This script

  1. copies the read-only disk image into a local (writable) file and redirects QEMU to boot that instead
  2. Waits for the initial system setup to complete by using the command cloud-init status --wait on the remote shell
  3. Gets the QEMUDriver from labgrid and executes a 'monitor' command savevm boottag to save state into a named tag boottag

Then, another new labgrid_config_toolchain is created that depends upon the disk image that has the saved state in it, this uses config-booted.yaml which has arguments to boot QEMU from the saved tag instead of booting from scratch.

For this new image to work it also has to be writable so the labgrid_genrule that depends upon the image uses a preload script execute_with_writable_image.sh which copies the disk image locally, makes it writable and overrides the LG_QEMU_QCOW2_IMAGE environment variable to point to the writable disk before invoking the runner.

You should be able to test the flow with: bazelisk test qemu:run-booted

Running it the first time will have the normal 2-minute wait to generate the booted disk image, but if you then force the test to run a second time it will take approximately 20 seconds.

Merge request reports