-
Notifications
You must be signed in to change notification settings - Fork 73
Trying out the SEV flavor of libkrun
If you have an SEV-capable machine (that is, on that's equipped with a CPU from the AMD Naples, Rome or Milan series), you can give the SEV flavor a try by following this instructions.
Before installing any components, it's generally a good idea to use the sevctl to ensure the host is ready to run SEV guests:
$ ./target/debug/sevctl ok
[ PASS ] - AMD CPU
[ PASS ] - Microcode support
[ PASS ] - Secure Memory Encryption (SME)
[ PASS ] - Secure Encrypted Virtualization (SEV)
[ PASS ] - Encrypted State (SEV-ES)
[ FAIL ] - Secure Nested Paging (SEV-SNP)
[ SKIP ] - VM Permission Levels
[ SKIP ] - Number of VMPLs
[ PASS ] - Physical address bit reduction: 47
[ PASS ] - C-bit location: 15
[ PASS ] - Number of encrypted guests supported simultaneously: 15
[ PASS ] - Minimum ASID value for SEV-enabled, SEV-ES disabled guest: 8
[ PASS ] - SEV enabled in KVM: enabled
[ PASS ] - Reading /dev/sev: /dev/sev readable
[ PASS ] - Writing /dev/sev: /dev/sev writable
[ PASS ] - Page flush MSR
[ PASS ] - KVM supported: API version: 12
[ PASS ] - Memlock resource limit: Soft: 4294967296 | Hard: 4294967296
error: One or more tests in sevctl-ok reported a failure
caused by: invalid data
It's fine is sevctl indicates a failure on SEV-ES and/or SEV-SNP, as we'll be using just plain SEV.
This library bundles a custom Linux kernel, a minimal FW (based on qboot) and a small initramfs.
git clone https://github.com/containers/libkrunfw
cd libkrunfw
git checkout amdsev
make -j8
sudo make install
This will build the SEV flavor of the libkrun library itself and the examples included with it.
git clone https://github.com/containers/libkrun
cd libkrun
make SEV=1
sudo make install
cd examples
make SEV=1
This builds a simple example of an attestation server. This attestation server, in addition of verifying the contents of the guest, will be the one generating and sending the pre-encrypted LUKS passphrase and kernel command line (which includes the workload entry point, that is, the first binary to be executed inside the guest).
The attestation server can be build and run on the same machine where libkrun-SEV is going to be used to run SEV guests, or on a different one, doing the attestation over the network (HTTP). The only requirement, if you intend to use a different machine, is that this one must also have a copy of the libkrunfw library. You can either build it, following the same instructions as for the SEV machine, or just copy it to a directory that will be search for libraries (i.e., /usr/local/lib64, or any other directory as long it's specified in the LD_LIBRARY_PATH environment variable).
git clone https://github.com/slp/sev-attestation-server
cd sev-attestation-server
cargo build --release
The steps required to build a disk image encrypted with LUKS or LUKS2 is outside the scope of this documents, but I've prepared a couple of encrypted images for testing and demonstration purposes:
-
disk-nginx-tls.raw (LUKS1 with passphrase "mysecretpassphrase")
- This image is based on the NGINX container image
-
disk-fedora.raw (LUKS2+AEAD with passphrase "mysecretpassphrase")
- This image is based on the Fedora 34 container image
Execute the sev-noattest binary passing the disk image as first argument, the LUKS passphrase as the second, and lastly the workload entry point (the first binary to be executed in the VM):
cd libkrun/examples
./sev-noattest disk-nginx-tls.raw mysecretpassphrase /bin/sh
You need to start the remote attestation server, indicating at least the LUKS passphrase and the workload entry point (the first binary to be executed inside the guest) as command line arguments. You can also specify the port to listen for connections (8080 by default):
./target/release/sev-attestation-server -p mysecretpassphrase -e /bin/sh
If you want to request the VM to enable SEV-ES instead of plain SEV, just add the -s flag:
./target/release/sev-attestation-server -p mysecretpassphrase -e /bin/sh -s
Execute the sev-attest binary passing the disk image as first argument, and the base URL of the attestation server as the second:
cd libkrun/examples
./sev-attest disk-nginx-tls.raw http://127.0.0.1:8080
Executing one the examples fails with "error while loading shared libraries: libkrun.so: cannot open shared object file: No such file or directory"
Make sure the libraries were installed into a location that's included among the default search paths of the dynamic loader. For example, if you have installed the libraries in the /usr/local prefix, you might need to export the LD_LIBRARY_PATH environment variable pointing to /usr/local/lib64:
export LD_LIBRARY_PATH=/usr/local/lib64
Executing one the examples fails with "Building the microVM failed: Internal(Vm(SecVirtInit(OpenFirmware(Os { code: 13, kind: PermissionDenied, message: "Permission denied" }))))"
If you're running the example as an unprivileged, you might need to adjust the permissions of the /dev/sev node. A simple (but insecure, use only on a test machine) way of doing this is:
sudo chmod go+rw /dev/sev
Executing one the examples fails with "Building the microVM failed: SecureVirtPrepare(SecVirtPrepare(MemoryEncryptRegion))"
Check in dmesg if you have a message like this:
SEV: 524288 locked pages exceed the lock limit of 16
If that's the case, you need to edit /etc/security/limits.conf, and set up both a large enough (4194304 should do the trick) hard and soft memory lock (memlock) limit. This is an example doing exactly that for the user slp:
slp hard memlock 4194304
slp soft memlock 4194304
After editing the file, you'll need to log out and log in again to ensure the new limits have been applied.