nohz: Reduce overhead under high-freq idling patterns

One testbox of mine (Intel Nehalem, 16-way) uses MWAIT for its idle routine,
which apparently can break out of its idle loop rather frequently, with
high frequency.

In that case NO_HZ_FULL=y kernels show high ksoftirqd overhead and constant
context switching, because tick_nohz_stop_sched_tick() will, if
delta_jiffies == 0, mis-identify this as a timer event - activating the
TIMER_SOFTIRQ, which wakes up ksoftirqd.

Fix this by treating delta_jiffies == 0 the same way we treat other short
wakeups, delta_jiffies == 1.

Cc: Frederic Weisbecker <[email protected]>
Cc: Chris Metcalf <[email protected]>
Cc: Christoph Lameter <[email protected]>
Cc: Geoff Levand <[email protected]>
Cc: Gilad Ben Yossef <[email protected]>
Cc: Hakan Akkan <[email protected]>
Cc: Kevin Hilman <[email protected]>
Cc: Li Zhong <[email protected]>
Cc: Oleg Nesterov <[email protected]>
Cc: Paul E. McKenney <[email protected]>
Cc: Paul Gortmaker <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Steven Rostedt <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 85e05ab9825388d9a5dc4dfe3c6f2a74077dce43..da53c8f2beb555081d9a04fcfd41d89df9b40b7f 100644 (file)
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -565,11 +565,12 @@ static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts,
                        delta_jiffies = rcu_delta_jiffies;
                }
        }
+
        /*
-        * Do not stop the tick, if we are only one off
-        * or if the cpu is required for rcu
+        * Do not stop the tick, if we are only one off (or less)
+        * or if the cpu is required for RCU:
         */
-       if (!ts->tick_stopped && delta_jiffies == 1)
+       if (!ts->tick_stopped && delta_jiffies <= 1)
                goto out;
 
        /* Schedule the tick, if we are at least one jiffie off */