Subsys/Sensor Power Management Usage

[JordanYates]

Topic

How should applications and subsystems interact with lower level API's in the context of power management

Note this document is only concerned with device runtime PM (CONFIG_PM_DEVICE_RUNTIME, pm_device_runtime_get, pm_device_runtime_put) as device PM should never be used directly on devices.

Point to resolve

According to Device Power Management, devices need to be in PM_DEVICE_STATE_ACTIVE to perform useful work. For an application concerned about power consumption, the default state of a device will be PM_DEVICE_STATE_OFF or PM_DEVICE_STATE_SUSPEND, depending on hardware configuration.

Therefore the device needs to transition to PM_DEVICE_STATE_ACTIVE when it is performing work. There are two potential methods to transition.

Implicit Transitions

The act of calling an API function (e.g. flash_read) automatically requests the device to be in active mode at the start of a function, and releases that request at the end of the function.

void api_user(const struct device *dev) {
  (void)flash_read(dev, ...);
}

int device_flash_read(const struct device *dev, ...) {
  pm_device_runtime_get(dev);
  // Do flash_read
  pm_device_runtime_put(dev);
}

Advantages:

• Simpler user interface
• No thought about PM required

Disadvantages:

• Device can transition between ACTIVE and SUSPEND on every call, potentially losing internal state

Explicit Transitions

Users of an API function are required to ensure the device is in active mode before calling API functions.

void api_user(const struct device *dev) {
  pm_device_runtime_get(dev);
  (void)flash_read(dev, ...);
  pm_device_runtime_put(dev);
}

int device_flash_read(const struct device *dev, ...) {
  // Do flash_read
}

Advantages:

• Device does not need to worry about losing internal state (user responsible for keeping ACTIVE as long as required)

Disadvantages:

• API user need to be power management aware
• Power management is present at all API call sites

Axioms

1. Usage of a common API must be implementation agnostic

If function calls or ordering need to differ based on the backend implementing the API, then all advantages of having a common API disappear.

2. Requirements that make sense in the context of one API do not necessarily apply in the context of another

It is okay for the usage patterns to differ between API's, as long as the pattern within an API is consistent (See 1).

3. The user of an API is always in a position to know when a device needs to be active

Whether calling uart_rx or sensor_fetch, the user is always in a position to know that the device needs to be active, as they are asking the API to do something. This does not mean that the user should always need to manually control PM, just that it is possible.

4. Some API's have more complicated usage patterns than others

The Entropy API has two function calls, independent of each other, with no pre or post call requirements.

The Sensor API has a range of synchronous and asynchronous function calls, with the result of function calls (sensor_channel_get, sensor_channel_fetch, etc) depending on the contents of previous function calls (sensor_attr_set, sensor_channel_fetch, etc). Furthermore, there is no function that explicitly 'starts' or 'ends' usage.

5. Any proposal needs to behave "as expected" no matter the PM configuration of an application

Code can't break depending on whether CONFIG_PM_DEVICE_RUNTIME is enabled or not, whether the backend device supports PM, or whether PM is enabled on the backend device. Happily this is provided out of the box by the runtime PM implementation, PM calls always return success in every case where device runtime PM is not enabled on a device.

General Thoughts

From a users perspective, the implicit mechanism is cleaner. They don't need to think about power management and the logic is all contained within the driver. Additionally, if the user wants to minimise transitions between ACTIVE and SUSPEND, they can always wrap N API calls with pm_device_runtime_get/put themselves, as demonstrated below:

void fewer_transitions(const struct device *dev) {
  pm_device_runtime_get(dev);
  flash_read(dev, ...);
  flash_read(dev, ...);
  flash_read(dev, ...);
  pm_device_runtime_put(dev);
}

The problems appear when API calls aren't self-contained, but instead depend on previous calls.

void sensor_operations(const struct device *dev) {
  sensor_attr_set(dev, SENSOR_ATTR_FULL_SCALE, 4);
  sensor_sample_fetch(dev);
  sensor_channel_get(dev, ...);
}

If PM in this situation is implicit, and the sensor device loses state when in suspend mode, then any value set in the sensor_attr_set call will be lost in a naive driver. This forces any sensor with this behaviour to cache any and all attributes in RAM, and either apply every attribute in the fetch implementation, or every time it transitions to ACTIVE mode (every function call in implicit mode). This is a significant overhead.

Proposal

For the purposes of power management, split APIs into two classes, simple and complex (defined below).

Simple APIs can be documented as implementing the easier to use 'implicit transition' method (which still enabled explicit control for optimisation as mentioned previously).

Complex APIs are documented as requiring power management to be externally controlled. This eliminates the requirement for drivers to perform complex guesswork about when the device really needs to be powered, and eliminates the need to cache device parameters in RAM, as we can assume that the device will not be powered down until it is no longer in use.

Simple APIs

A simple API is one in which function calls in the API are independent from each other. Examples:

• Flash
• Entropy
• SPI/I2C

Complex APIs

A complex API is one in which function calls in the API depend on each other, and potentially asynchronous external events: Examples:

• Sensor
• UART

[ceolin]

I agree with the axioms and as far as I understand they are achievable today using a building block approach, where complex subsystems control the device pm (since it is referenced counted). My concern in making this distinction, simple/complex APIs is that devices can behave different. Some device may need to wait some event before suspending (or something similar) that the subsystem does not know.

That's said, this approach definitely introduce an overhead, there will be multiple put/get calls that will just increase/decrease reference count. But do we have numbers to confirm this is a problem ? Sensors are probably the most sensitive to this approach, but it is not clear if that is a major performance problem.

[ceolin]

Btw, thanks @JordanYates to put all together !

[JordanYates]

achievable today using a building block approach, where complex subsystems control the device pm (since it is referenced counted)

Please provide an example of how this works in the sensor context (sensor_attr_set, sensor_fetch, sensor_channel_get) without the device transitioning through SUSPEND and losing state.

[ceolin]

I am assuming that the proposal here is that the user will encapsulate these calls inside runtime_get/runtime_put calls, in this case it does not matter if the driver does pm_device_runtime_get(self) ... pm_device_runtime_put(self) in these methods since the reference will just increment/decrement but never reaches 0 which will trigger a suspension. Am I missing something here ? Some situation where this won't work ? The problem I can see is if the driver does unbalanced pm_device_runtime_get / pm_device_runtime_put calls, but that looks another issue.

[JordanYates]

Yes, sorry I thought you were proposing some other mechanism at the subsys level.

[JordanYates]

in this case it does not matter if the driver does pm_device_runtime_get(self) ... pm_device_runtime_put(self) in these methods

The problem is that if the API is requiring the calls to be wrapped with PM, calling them inside the driver implementation is useless, because the device is by definition already in ACTIVE. Worse, it gives the wrong impression to a user that the driver is handling PM, which will likely break things if they change the backend driver.

[teburd]

Sensor WG 12/4/2023:

Flavio: We should do get/put in all calls, if power is needed to stay on the caller can also use get/put
Jordan: Won't attribute set be effectively a no-op?
Anas: We need to do this consistently not just for sensors, though sensors shows the issue
Tom: What do we do about attributes, do we cache them during PM suspend? Behavior can vary in a large way with a simple PM enable/disable
Yuval: Certainly some attributes should be cached and restored, and documented as such
Brix: The state should be cached and restored
Jordan: When do they get restored?
Flavio: If the driver can't cache state, then only the application can do get/put.

[ceolin]

The biggest problem with sensors (if I understood correctly) is that some operations are not persistent (and/or are too cost) across power states changes and neither the subsystem or driver have control of this. e.g: application can set an attribute and use it only later on in a different context. In this situation, the application should be pm aware. So we have two paths:

1. Sensors don't do device runtime pm.
2. Sensors do device runtime pm and explicitly document about this limitation and we document what an application should do as well (how to they need to use device runtime pm).

[nashif]

What I was referring to in the meeting:

https://www.kernel.org/doc/Documentation/power/runtime_pm.txt

9. Autosuspend, or automatically-delayed suspends

Changing a device's power state isn't free; it requires both time and energy.
A device should be put in a low-power state only when there's some reason to
think it will remain in that state for a substantial time. A common heuristic
says that a device which hasn't been used for a while is liable to remain
unused; following this advice, drivers should not allow devices to be suspended
at runtime until they have been inactive for some minimum period. Even when
the heuristic ends up being non-optimal, it will still prevent devices from
"bouncing" too rapidly between low-power and full-power states.

The term "autosuspend" is an historical remnant. It doesn't mean that the
device is automatically suspended (the subsystem or driver still has to call
the appropriate PM routines); rather it means that runtime suspends will
automatically be delayed until the desired period of inactivity has elapsed.

Inactivity is determined based on the power.last_busy field. Drivers should
call pm_runtime_mark_last_busy() to update this field after carrying out I/O,
typically just before calling pm_runtime_put_autosuspend(). The desired length
of the inactivity period is a matter of policy. Subsystems can set this length
initially by calling pm_runtime_set_autosuspend_delay(), but after device
registration the length should be controlled by user space, using the
/sys/devices/.../power/autosuspend_delay_ms attribute.

[ceolin]

that is a good thing to have but does not solve the problem when the subsystem / driver does not know when it will be used and can't preserve state (as it seems to be the sensors case). But for many other situations that is good to have.

[teburd]

No doubt having a delay on when to suspend power since last use is generally a good idea. It would at least solve the issue that many devices take significant amounts of time to turn on (maybe off if there's some sort of fifo/storage flush out).

But as @ceolin notes this doesn't solve the issue with sensors and internal state being lost on power suspension.

It also does not solve the issue where the device state is unknowable by the driver itself (DMA with sof).

In both cases the caller is basically the only thing that knows when power should be on or off.

That's why I was asking about keeping the state of the device in the driver for sensors as way of solving this. That has a RAM cost though and RAM costs are usually very sensitive on the kinds of devices Zephyr runs on, not the types of devices Linux runs on.

All that said, the behavior completely changing between PM_DEVICE being on or not in Kconfig for what happens with a function like...

void do_sensor_stuff() {
  sensor_attr_set(...);
  sensor_read(...);
}

Means something isn't right regardless of where pm get/put are called. That inconsistent behavior is the most concerning part of all of this to me. The behavior of this shouldn't vary on runtime device PM Kconfig options.