From 03298a2f9ee0d575b3ea970f2504450f352fb0ac Mon Sep 17 00:00:00 2001 From: Andrea Righi Date: Fri, 20 Mar 2026 18:28:31 +0100 Subject: [PATCH 1/2] sched_ext: idle: Prioritize idle SMT sibling [Upstream commit 2197cecdb02c57b08340059452540fcf101fa30d] In the default built-in idle CPU selection policy, when @prev_cpu is busy and no fully idle core is available, try to place the task on its SMT sibling if that sibling is idle, before searching any other idle CPU in the same LLC. Migration to the sibling is cheap and keeps the task on the same core, preserving L1 cache and reducing wakeup latency. On large SMT systems this appears to consistently boost throughput by roughly 2-3% on CPU-bound workloads (running a number of tasks equal to the number of SMT cores). Cc: Cheng-Yang Chou Signed-off-by: Andrea Righi Signed-off-by: Tejun Heo Signed-off-by: WangYuli --- kernel/sched/ext_idle.c | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/kernel/sched/ext_idle.c b/kernel/sched/ext_idle.c index d16a1de07865..ad311aa124e5 100644 --- a/kernel/sched/ext_idle.c +++ b/kernel/sched/ext_idle.c @@ -424,18 +424,24 @@ static inline bool task_affinity_all(const struct task_struct *p) * - prefer the last used CPU to take advantage of cached data (L1, L2) and * branch prediction optimizations. * - * 3. Pick a CPU within the same LLC (Last-Level Cache): + * 3. Prefer @prev_cpu's SMT sibling: + * - if @prev_cpu is busy and no fully idle core is available, try to + * place the task on an idle SMT sibling of @prev_cpu; keeping the + * task on the same core makes migration cheaper, preserves L1 cache + * locality and reduces wakeup latency. + * + * 4. Pick a CPU within the same LLC (Last-Level Cache): * - if the above conditions aren't met, pick a CPU that shares the same * LLC, if the LLC domain is a subset of @cpus_allowed, to maintain * cache locality. * - * 4. Pick a CPU within the same NUMA node, if enabled: + * 5. Pick a CPU within the same NUMA node, if enabled: * - choose a CPU from the same NUMA node, if the node cpumask is a * subset of @cpus_allowed, to reduce memory access latency. * - * 5. Pick any idle CPU within the @cpus_allowed domain. + * 6. Pick any idle CPU within the @cpus_allowed domain. * - * Step 3 and 4 are performed only if the system has, respectively, + * Step 4 and 5 are performed only if the system has, respectively, * multiple LLCs / multiple NUMA nodes (see scx_selcpu_topo_llc and * scx_selcpu_topo_numa) and they don't contain the same subset of CPUs. * @@ -616,6 +622,18 @@ s32 scx_select_cpu_dfl(struct task_struct *p, s32 prev_cpu, u64 wake_flags, goto out_unlock; } + /* + * Use @prev_cpu's sibling if it's idle. + */ + if (sched_smt_active()) { + for_each_cpu_and(cpu, cpu_smt_mask(prev_cpu), allowed) { + if (cpu == prev_cpu) + continue; + if (scx_idle_test_and_clear_cpu(cpu)) + goto out_unlock; + } + } + /* * Search for any idle CPU in the same LLC domain. */ From b762a605308d29b7567efea24e284eaf2a782a4f Mon Sep 17 00:00:00 2001 From: Andrea Righi Date: Sun, 22 Mar 2026 07:51:46 +0100 Subject: [PATCH 2/2] sched_ext: Guard cpu_smt_mask() with CONFIG_SCHED_SMT [Upstream commit 63f500c32a37d490ec623a3130e488cdb9bd6cf7] Wrap cpu_smt_mask() usage with CONFIG_SCHED_SMT to avoid build failures on kernels built without SMT support. Fixes: 2197cecdb02c ("sched_ext: idle: Prioritize idle SMT sibling") Reported-by: kernel test robot Closes: https://lore.kernel.org/oe-kbuild-all/202603221422.XIueJOE9-lkp@intel.com/ Signed-off-by: Andrea Righi Reviewed-by: Cheng-Yang Chou Signed-off-by: Tejun Heo Suggested-by: Wentao Guan Signed-off-by: WangYuli --- kernel/sched/ext_idle.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/sched/ext_idle.c b/kernel/sched/ext_idle.c index ad311aa124e5..ce39c04c1c92 100644 --- a/kernel/sched/ext_idle.c +++ b/kernel/sched/ext_idle.c @@ -622,6 +622,7 @@ s32 scx_select_cpu_dfl(struct task_struct *p, s32 prev_cpu, u64 wake_flags, goto out_unlock; } +#ifdef CONFIG_SCHED_SMT /* * Use @prev_cpu's sibling if it's idle. */ @@ -633,6 +634,7 @@ s32 scx_select_cpu_dfl(struct task_struct *p, s32 prev_cpu, u64 wake_flags, goto out_unlock; } } +#endif /* * Search for any idle CPU in the same LLC domain.