diff options
| author |  Sebastian Andrzej Siewior <bigeasy@linutronix.de>
| 2020-08-19 22:00:25 +0200 |
|---|---|---|
| committer |  Peter Zijlstra <peterz@infradead.org>
| 2020-08-26 12:41:58 +0200 |
| commit | c1cecf884ad748f63f9139d5a18ee265ee2f70fb (patch) | |
| tree | 9fdcfe0c0b9d905d20932c1aba3971ac14bd2b4a /kernel/sched | |
| parent | sched: Bring the PF_IO_WORKER and PF_WQ_WORKER bits closer together (diff) | |
| download | linux-c1cecf884ad748f63f9139d5a18ee265ee2f70fb.tar.gz linux-c1cecf884ad748f63f9139d5a18ee265ee2f70fb.tar.bz2 linux-c1cecf884ad748f63f9139d5a18ee265ee2f70fb.zip | |
sched: Cache task_struct::flags in sched_submit_work()
sched_submit_work() is considered to be a hot path. The preempt_disable()
instruction is a compiler barrier and forces the compiler to load
task_struct::flags for the second comparison.
By using a local variable, the compiler can load the value once and keep it in
a register for the second comparison.
Verified on x86-64 with gcc-10.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20200819200025.lqvmyefqnbok5i4f@linutronix.de
Diffstat (limited to 'kernel/sched')
| -rw-r--r-- | kernel/sched/core.c | 7 |
1 files changed, 5 insertions, 2 deletions
diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 8471a0f7eb32..c36dc1ae58be 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -4551,9 +4551,12 @@ void __noreturn do_task_dead(void) static inline void sched_submit_work(struct task_struct *tsk) { + unsigned int task_flags; + if (!tsk->state) return; + task_flags = tsk->flags; /* * If a worker went to sleep, notify and ask workqueue whether * it wants to wake up a task to maintain concurrency. @@ -4562,9 +4565,9 @@ static inline void sched_submit_work(struct task_struct *tsk) * in the possible wakeup of a kworker and because wq_worker_sleeping() * requires it. */ - if (tsk->flags & (PF_WQ_WORKER | PF_IO_WORKER)) { + if (task_flags & (PF_WQ_WORKER | PF_IO_WORKER)) { preempt_disable(); - if (tsk->flags & PF_WQ_WORKER) + if (task_flags & PF_WQ_WORKER) wq_worker_sleeping(tsk); else io_wq_worker_sleeping(tsk); |