[WIP] Update get/set Standard Registers to use VP register page #183

russell-islam · 2025-01-09T07:53:07Z

Summary of the PR

Please summarize here why the changes in this PR are needed.

Requirements

Before submitting your PR, please make sure you addressed the following
requirements:

All commits in this PR have Signed-Off-By trailers (with
git commit -s), and the commit message has max 60 characters for the
summary and max 75 characters for each description line.
All added/changed functionality has a corresponding unit/integration
test.
All added/changed public-facing functionality has entries in the "Upcoming
Release" section of CHANGELOG.md (if no such section exists, please create one).
Any newly added unsafe code is properly documented.

russell-islam · 2025-01-09T08:01:59Z

@jinankjain What do you think about arm64? I added the wrapper struct to lib.rs, I am open to add a similar regs.rs to arm64 and add the RegisterPage wrapper struct there.

jinankjain · 2025-01-10T08:57:35Z

Yes there would be similar regs for ARM64, but I haven't finalized the set of registers we would need.

NunoDasNeves

It looks like we need to handle the case where the register page is unavailable, e.g. in encrypted partitions.

NunoDasNeves · 2025-01-10T18:45:10Z

mshv-bindings/src/lib.rs

+#[derive(Debug)]
+pub struct RegisterPage(pub *mut hv_vp_register_page);
+
+// SAFETY: struct is based on reguster page in the hypervisor,


typo: reguser -> register

NunoDasNeves · 2025-01-10T18:47:53Z

mshv-ioctls/src/ioctls/vcpu.rs

@@ -64,6 +69,11 @@ impl AsRawFd for VcpuFd {
 }

 impl VcpuFd {
+    /// Get mut refeferenc eof VP register page


typo: refeferenc -> reference

NunoDasNeves · 2025-01-10T18:50:39Z

mshv-ioctls/src/ioctls/vcpu.rs

@@ -46,15 +46,20 @@ macro_rules! set_registers_64 {
 pub struct VcpuFd {
    index: u32,
    vcpu: File,
+    vp_page: RegisterPage,


Have you checked that this will always be available? Wondering if we should make it an Option

I just checked the driver code to be sure. We don't map the register page if it's an encrypted partition, so we need to make this optional.

Yes. This should be optional

NunoDasNeves · 2025-01-10T18:57:09Z

mshv-ioctls/src/ioctls/vcpu.rs

    /// Sets the vCPU general purpose registers
    #[cfg(not(target_arch = "aarch64"))]
    pub fn set_regs(&self, regs: &StandardRegisters) -> Result<()> {
-        let reg_assocs = [


I had suggested keeping the existing set_regs() and adding a new API. Maybe RegisterPage.set_gpregs() or something.
The semantics are not exactly the same - e.g. if you set a register using the register page, then before restarting the VP, get a register using the IOCTL, you won't get the value you just put in the page. So it might be better to differentiate these cases or be explicit that this is using the register page method.

We don't have the page in some cases, so we need to do this a different way.
We could have set_regs() dispatch to either the IOCTL or the register page, if it is available.
Then it will always pick the fastest way. It would still be nice to differentiate using the IOCTL vs using the register page, for reasons I mentioned above, as well.

We don't have the page in some cases, so we need to do this a different way. We could have set_regs() dispatch to either the IOCTL or the register page, if it is available. Then it will always pick the fastest way. It would still be nice to differentiate using the IOCTL vs using the register page, for reasons I mentioned above, as well.

Ar the moment it is always available except encrypted VM. How dowe exactly check is the page is available here? Checking the option?

I had suggested keeping the existing set_regs() and adding a new API. Maybe RegisterPage.set_gpregs() or something. The semantics are not exactly the same - e.g. if you set a register using the register page, then before restarting the VP, get a register using the IOCTL, you won't get the value you just put in the page. So it might be better to differentiate these cases or be explicit that this is using the register page method.

We need to have this part of vcpu API for vmm to use. Are you suggesting make the set_regs conditioonal inside? For encrypted case this API will never be called BTW.

How dowe exactly check is the page is available here? Checking the option?

Yes

We need to have this part of vcpu API for vmm to use.

We can change it however we like and as long as we make the matching changes on the VMM side.

Are you suggesting make the set_regs conditioonal inside

It might be the easiest way to do it, because we wouldn't have to change much in the VMM.

But, making it transparent to the VMM like that could lead to cases where IOCTL and register page usage are mixed, so differentiating those methods might save some trouble in future.

Ar the moment it is always available except encrypted VM.
...
For encrypted case this API will never be called BTW.

Ok, maybe just replacing the current set_regs() etc is fine then... I think I'd still like it better if the API made it clear this was a different method of setting the registers than using set_reg()/set_registers_64!.
E.g. renaming set_regs to set_vp_page_gp_regs or something

This sounds good.

Signed-off-by: Muminul Islam <[email protected]>

Define a new struct RegisterPage with mutable pointer to hv_vp_register_page. Signed-off-by: Muminul Islam <[email protected]>

Include the register page in vcpu object for later use to read and write the VP registers. Signed-off-by: Muminul Islam <[email protected]>

One more API to VCPU implementation sin signature get_vp_reg_page, that return mutable reference to VP register page. This API could be used by VMM or other VCPU implementations. Signed-off-by: Muminul Islam <[email protected]>

This API use mapped VP register page to set the registers. Signed-off-by: Muminul Islam <[email protected]>

This API uses VP register page to populate the registers. Signed-off-by: Muminul Islam <[email protected]>

russell-islam · 2025-01-13T17:31:43Z

@NunoDasNeves Please take another look.

liuw · 2025-01-13T18:41:45Z

If KVM supports a similar interface, can you check if they require the VMM to explicitly opt-in / enable to use this feature?

We want to be careful here. The kernel may decide some day that it also wants to touch that page.

russell-islam · 2025-01-13T19:12:13Z

If KVM supports a similar interface, can you check if they require the VMM to explicitly opt-in / enable to use this feature?

We want to be careful here. The kernel may decide some day that it also wants to touch that page.

I do not see any such feature in KVM. @NunoDasNeves What is the kernel scenario in our driver code. Can the driver change the page contents?

liuw · 2025-01-14T00:51:07Z

If KVM supports a similar interface, can you check if they require the VMM to explicitly opt-in / enable to use this feature?
We want to be careful here. The kernel may decide some day that it also wants to touch that page.

I do not see any such feature in KVM. @NunoDasNeves What is the kernel scenario in our driver code. Can the driver change the page contents?

It is all very nebulous in my head, but supposedly one day we may decide we want to do code emulation in Linux kernel, or make the kernel to directly inspect / modify guest register states.

We need to be thorough here. If such use cases arise, we need to make sure correctness is maintained. With the current code, the VMM blindly maps the page. It may or may not be okay. I'm looking for the reasoning why the current scheme is okay; if not, what needs to be done.

russell-islam · 2025-01-14T00:59:48Z

If KVM supports a similar interface, can you check if they require the VMM to explicitly opt-in / enable to use this feature?
We want to be careful here. The kernel may decide some day that it also wants to touch that page.

I do not see any such feature in KVM. @NunoDasNeves What is the kernel scenario in our driver code. Can the driver change the page contents?

It is all very nebulous in my head, but supposedly one day we may decide we want to do code emulation in Linux kernel, or make the kernel to directly inspect / modify guest register states.

We need to be thorough here. If such use cases arise, we need to make sure correctness is maintained. With the current code, the VMM blindly maps the page. It may or may not be okay. I'm looking for the reasoning why the current scheme is okay; if not, what needs to be done.

The whole purpose of this PR is to speed up the boot process. Mainly the emulation code is accessing GP and Control registers. If you move the emulation in the Kernel later we might remove this mapping from the VMM.

liuw · 2025-01-14T00:59:55Z

Actually the code as-is already makes a case for gating that feature with a flag. Mapping the register page can be separated from the guest type.

You don't need to carry the knowledge of the guest type in that particular code path.
You don't prevent a possible improvement in the future if hardware allows you to access a snapshot of the register page for an encrypted guest.

In general, you should always provide a mechanism to discover features across component boundary.

liuw · 2025-01-14T01:02:25Z

mshv-ioctls/src/ioctls/vcpu.rs

@@ -64,6 +69,13 @@ impl AsRawFd for VcpuFd {
 }

 impl VcpuFd {
+    /// Get mut reference of VP register page
+    pub fn get_vp_reg_page(&self) -> *mut hv_vp_register_page {


If this can fail, you should return an Option. That's far better than expecting someone to read your comment in the function.

match self.vp_page { Some(p) => p, None => None }.

We can just return as is self.vp_page.

In your example two incompatible return type.

russell-islam · 2025-01-14T01:15:27Z

Actually the code as-is already makes a case for gating that feature with a flag. Mapping the register page can be separated from the guest type.
1. You don't need to carry the knowledge of the guest type in that particular code path.

2. You don't prevent a possible improvement in the future if hardware allows you to access a snapshot of the register page for an encrypted guest.
In general, you should always provide a mechanism to discover features across component boundary.

For encrypted guest, there is no register page in the driver. that means mmap will fail and we have to store None.
In your comment I am not sure about the flag you are referring.

skinsbursky · 2025-01-14T03:04:10Z

If KVM supports a similar interface, can you check if they require the VMM to explicitly opt-in / enable to use this feature?

We want to be careful here. The kernel may decide some day that it also wants to touch that page.

Why this might be a problem from your POV?
Kernel does touch the page to inject the interrupt vectors upon VP dispatch and it's not mutually exclusive with other potential setters.
At the end it's just a memory page with some numbers until the VP is dispatched, isn't it?

russell-islam · 2025-01-14T23:42:58Z

After a discussion with Wei, I think we need to have some feature sets returned from the Kernel driver and make appropriate features available in the VMM. As a result, I am making this PR draft and continue to work on the Kernel and resume here once Kernel implementation is done.

liuw · 2025-01-15T00:00:25Z

If KVM supports a similar interface, can you check if they require the VMM to explicitly opt-in / enable to use this feature?
We want to be careful here. The kernel may decide some day that it also wants to touch that page.

Why this might be a problem from your POV? Kernel does touch the page to inject the interrupt vectors upon VP dispatch and it's not mutually exclusive with other potential setters. At the end it's just a memory page with some numbers until the VP is dispatched, isn't it?

When two entities modify the same page in an overlapping manner, there needs to be a way to synchronize them. We don't have a clear model yet.

Having a mechanism to clearly denote what feature is available gives us some leeway if something happens in the future.

russell-islam requested review from liuw, NunoDasNeves and jinankjain as code owners January 9, 2025 07:53

russell-islam force-pushed the muislam/reg-page-initial branch from 53c3cea to 349a0b5 Compare January 9, 2025 07:55

russell-islam changed the title ~~Update get/set Standard register to use VP register page~~ Update get/set Standard Registers to use VP register page Jan 9, 2025

russell-islam force-pushed the muislam/reg-page-initial branch from 349a0b5 to 05d26e5 Compare January 9, 2025 20:31

NunoDasNeves requested changes Jan 10, 2025

View reviewed changes

russell-islam added 4 commits January 12, 2025 14:39

mshv-ioctls: keep vm_type in the Vm object

6242304

Signed-off-by: Muminul Islam <[email protected]>

mshv-bindings: add wrapper struct around hv_vp_register_page

160e11b

Define a new struct RegisterPage with mutable pointer to hv_vp_register_page. Signed-off-by: Muminul Islam <[email protected]>

mshv-{bindings-ioctls}: redfine vcpu struct and new_vcpu function

2493887

Include the register page in vcpu object for later use to read and write the VP registers. Signed-off-by: Muminul Islam <[email protected]>

mshv-ioctls: add an API to return register page

7a41231

One more API to VCPU implementation sin signature get_vp_reg_page, that return mutable reference to VP register page. This API could be used by VMM or other VCPU implementations. Signed-off-by: Muminul Islam <[email protected]>

russell-islam force-pushed the muislam/reg-page-initial branch from 05d26e5 to e42898d Compare January 12, 2025 22:41

mshv-ioctls: add new API to set standard registers

63e982d

This API use mapped VP register page to set the registers. Signed-off-by: Muminul Islam <[email protected]>

russell-islam force-pushed the muislam/reg-page-initial branch from e42898d to 5d883df Compare January 12, 2025 22:48

mshv-ioctls: add a new API to get standard registers

5a74c89

This API uses VP register page to populate the registers. Signed-off-by: Muminul Islam <[email protected]>

russell-islam force-pushed the muislam/reg-page-initial branch from 5d883df to 5a74c89 Compare January 12, 2025 22:54

liuw reviewed Jan 14, 2025

View reviewed changes

russell-islam changed the title ~~Update get/set Standard Registers to use VP register page~~ [WIP] Update get/set Standard Registers to use VP register page Jan 14, 2025

russell-islam marked this pull request as draft January 14, 2025 23:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Update get/set Standard Registers to use VP register page #183

[WIP] Update get/set Standard Registers to use VP register page #183

russell-islam commented Jan 9, 2025

russell-islam commented Jan 9, 2025 •

edited

Loading

jinankjain commented Jan 10, 2025

NunoDasNeves left a comment

NunoDasNeves Jan 10, 2025

NunoDasNeves Jan 10, 2025

NunoDasNeves Jan 10, 2025

NunoDasNeves Jan 10, 2025

russell-islam Jan 10, 2025

NunoDasNeves Jan 10, 2025

NunoDasNeves Jan 10, 2025

russell-islam Jan 10, 2025

russell-islam Jan 10, 2025

NunoDasNeves Jan 10, 2025

russell-islam Jan 11, 2025

russell-islam commented Jan 13, 2025

liuw commented Jan 13, 2025

russell-islam commented Jan 13, 2025 •

edited

Loading

liuw commented Jan 14, 2025

russell-islam commented Jan 14, 2025

liuw commented Jan 14, 2025

liuw Jan 14, 2025

russell-islam Jan 14, 2025

russell-islam commented Jan 14, 2025 •

edited

Loading

skinsbursky commented Jan 14, 2025

russell-islam commented Jan 14, 2025

liuw commented Jan 15, 2025

[WIP] Update get/set Standard Registers to use VP register page #183

Are you sure you want to change the base?

[WIP] Update get/set Standard Registers to use VP register page #183

Conversation

russell-islam commented Jan 9, 2025

Summary of the PR

Requirements

russell-islam commented Jan 9, 2025 • edited Loading

jinankjain commented Jan 10, 2025

NunoDasNeves left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

russell-islam commented Jan 13, 2025

liuw commented Jan 13, 2025

russell-islam commented Jan 13, 2025 • edited Loading

liuw commented Jan 14, 2025

russell-islam commented Jan 14, 2025

liuw commented Jan 14, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

russell-islam commented Jan 14, 2025 • edited Loading

skinsbursky commented Jan 14, 2025

russell-islam commented Jan 14, 2025

liuw commented Jan 15, 2025

russell-islam commented Jan 9, 2025 •

edited

Loading

russell-islam commented Jan 13, 2025 •

edited

Loading

russell-islam commented Jan 14, 2025 •

edited

Loading