Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP][DO NOT MERGE] Partial SCMI support in Xen #209

Open
wants to merge 25 commits into
base: xen-4.19-xt0.1
Choose a base branch
from

Conversation

lorc
Copy link
Collaborator

@lorc lorc commented Aug 15, 2024

TODOs:

  1. Check TODO for every commit made be me
  2. Revert original authorship for all patches, because for some reason Ihor Usik made himself an author for all patches
  3. Complete dom0less support
  4. Fold up changes into original patches
  5. Clean all the patches

To enable new features, enable the following options:

CONFIG_ARM_SCI=y
CONFIG_SCMI_SMC=y
CONFIG_ACCESS_CONTROLLER=y

lorc and others added 25 commits August 16, 2024 02:18
Signed-off-by: Volodymyr Babchuk <[email protected]>
Signed-off-by: Volodymyr Babchuk <[email protected]>
This is to avoid conflicts with QCOM devices

Signed-off-by: Volodymyr Babchuk <[email protected]>
…ervisor

There are use cases (for example using the PV driver) in Dom0less
setup that require Dom0less DomUs start immediately with Dom0, but
initialize XenStore later after Dom0's successful boot and call to
the init-dom0less application.

An error message can seen from the init-dom0less application on
1:1 direct-mapped domains:
```
Allocating magic pages
memory.c:238:d0v0 mfn 0x39000 doesn't belong to d1
Error on alloc magic pages
```
This is because currently the magic pages for Dom0less DomUs are
populated by the init-dom0less app through populate_physmap(), and
populate_physmap() automatically assumes gfn == mfn for 1:1 direct
mapped domains. This cannot be true for the magic pages that are
allocated later from the init-dom0less application executed in Dom0.
For domain using statically allocated memory but not 1:1 direct-mapped,
similar error "failed to retrieve a reserved page" can be seen as the
reserved memory list is empty at that time.

To solve above issue, this commit allocates the magic pages for
Dom0less DomUs at the domain construction time. The base address/PFN
of the magic page region will be noted and communicated to the
init-dom0less application in Dom0.

Reported-by: Alec Kwapis <[email protected]>
Suggested-by: Daniel P. Smith <[email protected]>
Signed-off-by: Henry Wang <[email protected]>
Message-ID: <[email protected]>
For use cases such as Dom0less PV drivers, a mechanism to communicate
Dom0less DomU's static data with the runtime control plane (Dom0) is
needed. Since on Arm HVMOP is already the existing approach to address
such use cases (for example the allocation of HVM_PARAM_CALLBACK_IRQ),
add a new HVMOP key HVM_PARAM_MAGIC_BASE_PFN for storing the magic
page region base PFN. The value will be set at Dom0less DomU
construction time after Dom0less DomU's magic page region has been
allocated.

To keep consistent, also set the value for HVM_PARAM_MAGIC_BASE_PFN
for libxl guests in alloc_magic_pages().

Reported-by: Alec Kwapis <[email protected]>
Signed-off-by: Henry Wang <[email protected]>
Message-ID: <[email protected]>
Currently the GUEST_MAGIC_BASE in the init-dom0less application is
hardcoded, which will lead to failures for 1:1 direct-mapped Dom0less
DomUs.

Since the guest magic region is now allocated from the hypervisor,
instead of hardcoding the guest magic pages region, use
xc_hvm_param_get() to get the guest magic region PFN, and based on
that the XenStore PFN can be calculated. Also, we don't need to set
the max mem anymore, so drop the call to xc_domain_setmaxmem(). Rename
the alloc_xs_page() to get_xs_page() to reflect the changes.

Take the opportunity to do some coding style improvements when possible.

Reported-by: Alec Kwapis <[email protected]>
Signed-off-by: Henry Wang <[email protected]>
Message-ID: <[email protected]>
This patch adds the basic framework for SCI mediator.
SCI is System Control Interface, which is designed to redirect
requests from the Domains to Firmware. This will allow the devices,
passed-through to the different Domains, to access to the System Controls
(such as clocks/resets etc) by sending requests to the firmware.
Xen mediates requests between the Domains and the Firmware, also it is
responsible for permission handling during Domain crateion.

SCI mediator register itself with REGISTER_SCI_MEDIATOR() macro.

At run-time, during initialization, framework calls probe for the first
matching device in the device-tree. When no device-tree is present - the
first registered mediator should be probed.

Signed-off-by: Oleksii Moisieiev <[email protected]>
Signed-off-by: Ihor Usyk <[email protected]>
This is the implementation of SCI interface, called SCMI-SMC driver,
which works as the mediator between XEN Domains and Firmware (SCP, ATF etc).
This allows devices from the Domains to work with clocks, resets and
power-domains without access to CPG.

Originally, cpg should be passed to the domain so it can work with
power-domains/clocks/resets etc. Considering that cpg can't be split between
the Domains, we get the limitation that the devices, which are using
power-domains/clocks/resets etc, couldn't be split between the domains.
The solution is to move the power-domain/clock/resets etc to the
Firmware (such as SCP firmware or ATF) and provide interface for the
Domains. XEN should have an entity, caled SCI-Mediator, which is
responsible for messages redirection between Domains and Firmware and
for permission handling.

The following features are implemented:
- request SCMI channels from ATF and pass channels to Domains;
- set device permissions for Domains based on the Domain partial
device-tree. Devices with permissions are able to work with clocks,
resets and power-domains via SCMI;
- redirect scmi messages from Domains to ATF.

Signed-off-by: Oleksii Moisieiev <[email protected]>
Signed-off-by: Ihor Usyk <[email protected]>
If set, Xen is allowed to assign the devices even if they are not under
IOMMU.
Can be confugired from dom.cfg in the following format:
force_assign_without_iommu = 1

This parameter has the same purpose as xen,force-assign-without-iommu
property in dom0less archtecture.

Signed-off-by: Oleksii Moisieiev <[email protected]>
Signed-off-by: Ihor Usyk <[email protected]>
This enumeration sets SCI type for the domain. Currently there is
two possible options: either 'none' or 'scmi_smc'.

'none' is the default value and it disables SCI support at all.

'scmi_smc' enables access to the Firmware from the domains using SCMI
protocol and SMC as transport.

Signed-off-by: Oleksii Moisieiev <[email protected]>
Integration of the SCMI mediator with xen libs:
- add hypercalls to aquire SCI channel and set device permissions
for DomUs;
- add SCMI_SMC nodes to DomUs device-tree based on partial device-tree;
- SCI requests redirection from DomUs to Firmware.

Signed-off-by: Oleksii Moisieiev <[email protected]>
The function is used in scmi_smc.c

Signed-off-by: Ihor Usyk <[email protected]>
There are some code wrapped in the blocks #ifdef..#endif, inside of the blocks
the ret value is compared with '-ENOSYS' and if it is, start the code execution.
It means that in the code below, function iommu_do_dt_domctl can not be executed
if CONFIG_HAS_PCI not defined

int ret = -ENOSYS;

if ( !is_iommu_enabled(d) )
    return -EOPNOTSUPP;

ret = iommu_do_pci_domctl(domctl, d, u_domctl);

if ( ret == -ENOSYS )
   ret = iommu_do_dt_domctl(domctl, d, u_domctl);

So, it is reasonable to have '-ENOSYS' as initial value.

Signed-off-by: Ihor Usyk <[email protected]>
Function sci_domain_init is called from the two places:
1. construct_dom0 - for domain Dom0
2. arch_domain_create - for all domains.

Double call of sci_domain_init for dom0 assigns the two sci channels
to one domain 'Dom0'. It means that 3rd domain (DomA) can not get the sci channel
because all of them are busy.
To avoid this situation, the call of sci_domain_init in arch_domain_create is moved under
the condition !is_hardware_domain.

Signed-off-by: Ihor Usyk <[email protected]>
The code used the bool variable 'found' was used
to identify that channel had been found. The additional condition was used
to return the required result, like
if ( found )
    return curr;
return NULL;

It is proposed to user variable 'ret' and remove the condition;
like:
int *ret = NULL;
/* here the loop and if value is found - assign value to ret*/
return ret;

Signed-off-by: Ihor Usyk <[email protected]>
Allow to provide TEE type for a Dom0less guest via "xen,tee"
property. Create appropriate nodes in the guests' device tree and
initialize tee subsystem for it.

Signed-off-by: Volodymyr Babchuk <[email protected]>
access-controllers are generic term for service/hardware  that
allow device partitioning. We use it to provide SCMI device ID.

Signed-off-by: Volodymyr Babchuk <[email protected]>
manipulation with dt entries.

Those changed should be folded into original patches that add SCMI
support in Xen.

Signed-off-by: Volodymyr Babchuk <[email protected]>
- use "epam,secondary-agents" property to collect secondary agents,

- drop notion of channel and use agent_id everywhere

- use different SMC function IDs for different channels

Those changes should be wrapped into original patches

Signed-off-by: Volodymyr Babchuk <[email protected]>
assign this device to Xen, to prevent failures

TODO: add proper commit description or wrap into patch that adds
initial SCMI support

Signed-off-by: Volodymyr Babchuk <[email protected]>
TODO:

 - make this conditional
 - create device tree entries for SCMI nodes

Signed-off-by: Volodymyr Babchuk <[email protected]>
@oleksiimoisieiev
Copy link

tools build fails if

CONFIG_ARM_SCI=y
CONFIG_SCMI_SMC=y
CONFIG_ACCESS_CONTROLLER=y

are not enabled
Leaving here as a reminder

@GrygiriiS
Copy link

GrygiriiS commented Aug 25, 2024

I've tried this and can boot both dom0 and domD - I've added SCMI DT nodes statically to both of them.
Following my testing I have following requirements to this solution:

  1. Dom0 - SCMI agent id have to be assigned statically to avoid Zephyr app DT changing every time number of agents or agent allocation algo are changing
  2. dom0less code has to made a high priority for RPI5 project
  3. DomD - SCMI DT generation code should take into account that RPI5 DomD memory is static and direct mapped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants