ECHO scheduler support on Linux 6.9

[anh0516]

The ECHO 6.8 patch works on vanilla Linux 6.9, at least for me.

I already opened an issue upstream.

Is it worth adding to TkG as experimental ASAP or waiting for confirmed support?

[ptr1337]

https://github.com/CachyOS/kernel-patches/blob/master/6.9/sched/0001-echo.patch

Here is a tested patch.

[anh0516]

That is the same exact patch, just with different commit info. So the CachyOS's consensus is also that it just works. Let's make this available too then?

…

On Wed, May 15, 2024, 9:23 AM Peter Jung ***@***.***> wrote: https://github.com/CachyOS/kernel-patches/blob/master/6.9/sched/0001-echo.patch Here is a tested patch. — Reply to this email directly, view it on GitHub <#944 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATSSZDCSUA7I3ZTX76MGJD3ZCNOWVAVCNFSM6AAAAABHXK4AZ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJSGUYTMMZYHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[ptr1337]

That is the same exact patch, just with different commit info. So the CachyOS's consensus is also that it just works. Let's make this available too then?
…
On Wed, May 15, 2024, 9:23 AM Peter Jung @.> wrote: https://github.com/CachyOS/kernel-patches/blob/master/6.9/sched/0001-echo.patch Here is a tested patch. — Reply to this email directly, view it on GitHub <#944 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATSSZDCSUA7I3ZTX76MGJD3ZCNOWVAVCNFSM6AAAAABHXK4AZ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJSGUYTMMZYHA . You are receiving this because you authored the thread.Message ID: @.>

Yes, it works fine. Weve tested it.
Actually, what is once again problematic - the maintainer of the patch seems not to be active again and feels like it is happening the same as what happened to cacule and the TT Scheduler.

[hamadmarri]

Hello,

Thank you all for you efforts. I assume no diff between 6.8 and 6.9 since the old patch is working. I will have a look on 6.9 changes and see if any changes are needed.

[ptr1337]

Hello,

Thank you all for you efforts. I assume no diff between 6.8 and 6.9 since the old patch is working. I will have a look on 6.9 changes and see if any changes are needed.

6.10 will not work anymore. The patch does apply, but fails at compilation time.

[anh0516]

CachyOS is carrying a Linux 6.10 patch.

It fails for me with Clang 18.1 because implicit function declarations are errors as of Clang 16, with a bunch of related errors (ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]). This is probably because of other broken parts of the patch.

I changed -Werror=implicit-function-declaration to -Wno-error=implicit-function-declaration at usr/include/Makefile:9 and scripts/Makefile.extrawarn, and now it is happy and we can see the real problem:

In file included from kernel/sched/bs.c:49:
kernel/sched/fair_dep_funcs.h:532:42: warning: call to undeclared function 'arch_scale_thermal_pressure'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
  532 |         capacity_orig_thermal = capacity_orig - arch_scale_thermal_pressure(cpu);
      |                                                 ^
kernel/sched/fair_dep_funcs.h:625:36: error: use of undeclared identifier 'SG_OVERUTILIZED'
  625 |                 WRITE_ONCE(rq->rd->overutilized, SG_OVERUTILIZED);
      |                                                  ^
kernel/sched/fair_dep_funcs.h:626:39: error: use of undeclared identifier 'SG_OVERUTILIZED'
  626 |                 trace_sched_overutilized_tp(rq->rd, SG_OVERUTILIZED);
      |                                                     ^
In file included from kernel/sched/bs.c:747:
kernel/sched/balancer.h:146:6: warning: call to undeclared function 'thermal_load_avg'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
  146 |         if (thermal_load_avg(rq))
      |             ^
kernel/sched/balancer.h:256:21: warning: call to undeclared function 'arch_scale_thermal_pressure'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
  256 |         thermal_pressure = arch_scale_thermal_pressure(cpu_of(rq));
      |                            ^
kernel/sched/balancer.h:260:5: warning: call to undeclared function 'update_thermal_load_avg'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
  260 |                   update_thermal_load_avg(rq_clock_thermal(rq), rq, thermal_pressure) |
      |                   ^
kernel/sched/balancer.h:260:29: warning: call to undeclared function 'rq_clock_thermal'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
  260 |                   update_thermal_load_avg(rq_clock_thermal(rq), rq, thermal_pressure) |
      |                                           ^
kernel/sched/balancer.h:459:46: error: use of undeclared identifier 'CPU_NOT_IDLE'; did you mean '__CPU_NOT_IDLE'?
  459 |                 .idle           = dst_rq->idle_balance ? CPU_IDLE : CPU_NOT_IDLE,
      |                                                                     ^~~~~~~~~~~~
      |                                                                     __CPU_NOT_IDLE
./include/linux/sched/idle.h:8:2: note: '__CPU_NOT_IDLE' declared here
    8 |         __CPU_NOT_IDLE = 0,
      |         ^
In file included from kernel/sched/bs.c:747:
kernel/sched/balancer.h:676:17: warning: unused variable 'cfs_rq' [-Wunused-variable]
  676 |         struct cfs_rq *cfs_rq;
      |                        ^~~~~~
kernel/sched/balancer.h:858:6: warning: no previous prototype for function 'trigger_load_balance' [-Wmissing-prototypes]
  858 | void trigger_load_balance(struct rq *this_rq)
      |      ^
kernel/sched/balancer.h:858:1: note: declare 'static' if the function is not intended to be used outside of this translation unit
  858 | void trigger_load_balance(struct rq *this_rq)
      | ^
      | static 
In file included from kernel/sched/bs.c:747:
In file included from kernel/sched/balancer.h:878:
kernel/sched/nohz.h:296:18: error: use of undeclared identifier 'CPU_NOT_IDLE'; did you mean '__CPU_NOT_IDLE'?
  296 |                                                 CPU_IDLE : CPU_NOT_IDLE;
      |                                                            ^~~~~~~~~~~~
      |                                                            __CPU_NOT_IDLE
./include/linux/sched/idle.h:8:2: note: '__CPU_NOT_IDLE' declared here
    8 |         __CPU_NOT_IDLE = 0,
      |         ^
In file included from kernel/sched/bs.c:747:
In file included from kernel/sched/balancer.h:878:
kernel/sched/nohz.h:387:47: error: no member named 'max_cpu_capacity' in 'struct root_domain'
  387 |                 (arch_scale_cpu_capacity(rq->cpu) < rq->rd->max_cpu_capacity ||
      |                                                     ~~~~~~  ^
  AR      kernel/power/built-in.a
7 warnings and 5 errors generated.

A quick ripgrep of the upstream sources shows that SG_OVERUTILIZED and CPU_NOT_IDLE were removed from 6.10.

linux-6.9.10/kernel/sched/sched.h
843:#define SG_OVERUTILIZED		0x2 /* One or more CPUs are over-utilized. */

linux-6.9.10/kernel/sched/fair.c
6710:		set_rd_overutilized_status(rq->rd, SG_OVERUTILIZED);
9934:			*sg_status |= SG_OVERUTILIZED;
10632:					  sg_status & SG_OVERUTILIZED);
10633:	} else if (sg_status & SG_OVERUTILIZED) {
10634:		set_rd_overutilized_status(env->dst_rq->rd, SG_OVERUTILIZED);


linux-6.9.10/kernel/sched/fair.c
9105:		if (env->idle != CPU_NOT_IDLE && env->src_rq->nr_running <= 1)
9834:	if (env->idle == CPU_NOT_IDLE)
9858:	if (env->idle == CPU_NOT_IDLE || !busiest->sum_nr_running)
9958:		} else if ((env->idle != CPU_NOT_IDLE) &&
9971:	if (!local_group && env->idle != CPU_NOT_IDLE && sgs->sum_h_nr_running &&
10724:			if (env->idle != CPU_NOT_IDLE && env->imbalance == 0) {
10939:		if (env->idle == CPU_NOT_IDLE) {
11147:	return env->idle != CPU_NOT_IDLE && sched_use_asym_prio(env->sd, env->dst_cpu) &&
11185:	if ((env->idle != CPU_NOT_IDLE) &&
11746:				idle = idle_cpu(cpu) ? CPU_IDLE : CPU_NOT_IDLE;
12428:						CPU_IDLE : CPU_NOT_IDLE;

linux-6.9.10/include/linux/sched/idle.h
9:	CPU_NOT_IDLE,

vs. 6.10

linux-6.10/include/linux/sched/idle.h
8:	__CPU_NOT_IDLE = 0,

So the TL;DR here is that ECHO has to be significantly reworked to build with Linux 6.10. Should we bring this to @hamadmarri's repo?