x86/mm: Remove preempt_disable/enable() from __native_flush_tlb()

The preempt_disable/enable() pair in __native_flush_tlb() was added in
commit:

  5cf0791da5c1 ("x86/mm: Disable preemption during CR3 read+write")

... to protect the UP variant of flush_tlb_mm_range().

That preempt_disable/enable() pair should have been added to the UP variant
of flush_tlb_mm_range() instead.

The UP variant was removed with commit:

  ce4a4e565f52 ("x86/mm: Remove the UP asm/tlbflush.h code, always use the (formerly) SMP code")

... but the preempt_disable/enable() pair stayed around.

The latest change to __native_flush_tlb() in commit:

  6fd166aae78c ("x86/mm: Use/Fix PCID to optimize user/kernel switches")

... added an access to a per CPU variable outside the preempt disabled
regions, which makes no sense at all. __native_flush_tlb() must always
be called with at least preemption disabled.

Remove the preempt_disable/enable() pair and add a WARN_ON_ONCE() to catch
bad callers independent of the smp_processor_id() debugging.

Signed-off-by: Thomas Gleixner <[email protected]>
Cc: <[email protected]>
Cc: Andy Lutomirski <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: Dominik Brodowski <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>

diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index b519da4fc03c7c26c4f722ea2c5d97b091cbfd15..f9b48ce152ebac21d9f8d67824234d7ead0325cc 100644 (file)
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -345,15 +345,17 @@ static inline void invalidate_user_asid(u16 asid)
  */
 static inline void __native_flush_tlb(void)
 {
  */
 static inline void __native_flush_tlb(void)
 {

-       invalidate_user_asid(this_cpu_read(cpu_tlbstate.loaded_mm_asid));

        /*
        /*

-        * If current->mm == NULL then we borrow a mm which may change
-        * during a task switch and therefore we must not be preempted
-        * while we write CR3 back:
+        * Preemption or interrupts must be disabled to protect the access
+        * to the per CPU variable and to prevent being preempted between
+        * read_cr3() and write_cr3().

         */
         */

-       preempt_disable();
+       WARN_ON_ONCE(preemptible());
+
+       invalidate_user_asid(this_cpu_read(cpu_tlbstate.loaded_mm_asid));
+
+       /* If current->mm == NULL then the read_cr3() "borrows" an mm */

        native_write_cr3(__native_read_cr3());
        native_write_cr3(__native_read_cr3());

-       preempt_enable();

 }
 
 /*
 }
 
 /*