Refactoring to make EOS more generally applicable

[michaelosthege]

Over the past weeks there were many changes, but the project largely remained tailored to one specific setup.

I would like to propose changing how loads, parameters and the optimization problem are set up, such that it becomes easier to customize the involved devices and parameters.

Step 1: Understand what is important to move the project forward

Is the current goal to make EOS an "enterprise product" with documentation, strong type hints, full test coverage, state of the art dev tooling, self-describing REST API and so on?

That would certainly be nice for Andreas, but hardly useful for anybody else.

Instead, I would personally prefer an EOS that can be customized to different energy setups.
Large or small, with a battery or without, with or without (non-)smart household loads and so on.

To achieve this, we need to change how the setup (aka the optimization problem) is defined.

Step 2: Separate components, simulation and optimization

Let's start by mapping components/tasks in EOS to see if we can find patterns by which we might organize the code.

mindmap
  root((EOS))
    forecasts
        PV generation
        electricity price
        outside temperature
    load profiles
        car
        battery
        dishwasher
        fridge
        heat pump
        ...
    simulation
        model parametrization
        initial state
        system trajectories
    optimization
        free parameters
        cost function
        diagnostics

Loading

In words:

EOS simulates an energy system comprising various (positive or negative) load profiles, some of which may be parametrized.
From a parametrization of load profiles, in combination with forecasted PV or temperature, it simulates the energy system (charges, power, ...) creating trajectories.
The trajectories and forecasted energy prices are combined in a cost function.
An optimization algorithm is used to tweak the energy system model's free parameters such that the cost function is minimized.

We can now try to morph the Python codebase to reflect this separation of concerns too.
I have opened PR #188 to do this.

Step 3: Increase abstraction level of model components

Right now optimization_problem and EnergieManagementSystem know PVAkku, Haushaltsgeraet and Wechselrichter by name. Oddly enough they don't know about Heatpump and eauto = PVAkku(...).
Looks like an organically grown design, which is fair enough for a prototype.

Looking at the mindmap above, however, we can cluster these system components and call them Load, LoadModel, LoadProfile or whatever.

From load profiles to a energy system model

What do load profiles have in common?

• A power trajectory (positive or negative)
• Some have an internal state that is relevant for visualization or cost function (e.g. state_of_charge, skip_penalty)
• Most of them are parametrized
  • By fixed parameters (e.g. on_power)
  • By initial state (e.g. state_of_charge)
  • By free parameters (e.g. charge_start, charge_duration)
  • By external forecasts (e.g. temperature or pv_max_power)

With this, our EnergySystem model is simply a container that sums the load profiles in all aspects:

• Power balance trajectory (aka Einspeisung/Netzbezug)
• Internal state trajectories (state of charge, water temperature, ...)
• All free parameters

The other EOS components no longer have to know any details!

They only care about the abstract properties of a LoadProfile:

• free_parameters
• simulate(params, forecasts) -> power, cost, states

Refactoring for abstraction

The current load model implementations are stateful and do not share a common superclass.
Some have a .reset() method, Haushaltsgeraet already has a .get_lastkurve() but other than that they are quite ... individual.

I would recommend to start by defining the structural protocols needed by EnergyManagementSystem and other components.

For example:

class PVForecastConsumer(Protocol):
    def set_pv_forecast(self, fc: pd.Series) -> None: ...

class TemperatureForecastConsumer(Protocol):
    def set_temperature_forecast(self, fc: pd.Series) -> None: ...

class ParametrizableLoad(Protocol):
    free_parameters: Mapping[str, ParameterSettings]  # 👈 in some way that's useful to set up the deap optimizer

    def set_free_parameters(self, params: Mapping[str, Any]) -> None: ...

class LoadProfileSimuator(Protocol):
    def simulate_load_profile(self) -> pd.Series: ...

This would enable more abstract initialization of the energy system model:

for load in self.loads:
    if isinstance(load, PVForecastConsumer):
        load.set_pv_forecast(pv_forecast)
    if isinstance(load, TemperatureForecastConsumer):
        load.set_temperature_forecast(temp_forecast)

---

Enough thinking about this for today.

I'm curious what people think about this proposal.

[b0661]

When creating a proof of concept for a prediction submodule I recognized some things to be handy in the new structure:

An utilities submodule for concepts/ functions used or to be used over the complete code base. What I needed that could be useful for other code parts, and that centralizes maintenenace for these utilities:

• datetimeutil: for convenient date, time, timezone handling
• singletonutil: For a mixin class to create (threadsafe) singletons
• cacheutil: For file caching or memory caching
• logutil: Generic logger function
• configutil: Automatic config path generation according to platform rules.
• pydanticutil: A pydantic ModelBase class extended by custom type serialization/ deserialization. In my case pendulum.DateTime
• mathutil: I think there is already a usage for this.

A submodule for configuration that provides a mixin class so that all classes get easy access to the configuration with the concept of a priority of configuration data:

• EOS configuration file
• environment configuration,
• application initialization settings,
• class fields,
• class defaults,
• module defaults

A server submodule that is part of the akkudoktoreos package (not in an extra package).

In the end my proposal for the submodule/ directory structure is:

• config
• devices
• optimization
• prediction
• server
• utils

I would like to use this structure to go on with the proof of concept prediction submodule. Any comments welcome.