powerpc/64s: __find_linux_pte() synchronization vs pmdp_invalidate()

The change to pmdp_invalidate() to mark the pmd with _PAGE_INVALID
broke the synchronisation against lock free lookups,
__find_linux_pte()'s pmd_none() check no longer returns true for such
cases.

Fix this by adding a check for this condition as well.

Fixes: da7ad366b497 ("powerpc/mm/book3s: Update pmd_present to look at _PAGE_PRESENT bit")
Cc: [email protected] # v4.20+
Suggested-by: Aneesh Kumar K.V <[email protected]>
Signed-off-by: Nicholas Piggin <[email protected]>
Reviewed-by: Aneesh Kumar K.V <[email protected]>
Signed-off-by: Michael Ellerman <[email protected]>

diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c
index db4a6253df92a8713c406944dfda6c7affb09c63..533fc6fa672670614c193339f2dd4285963781b3 100644 (file)
--- a/arch/powerpc/mm/pgtable.c
+++ b/arch/powerpc/mm/pgtable.c
@@ -372,13 +372,25 @@ pte_t *__find_linux_pte(pgd_t *pgdir, unsigned long ea,
        pdshift = PMD_SHIFT;
        pmdp = pmd_offset(&pud, ea);
        pmd  = READ_ONCE(*pmdp);
+
        /*
-        * A hugepage collapse is captured by pmd_none, because
-        * it mark the pmd none and do a hpte invalidate.
+        * A hugepage collapse is captured by this condition, see
+        * pmdp_collapse_flush.
         */
        if (pmd_none(pmd))
                return NULL;
 
+#ifdef CONFIG_PPC_BOOK3S_64
+       /*
+        * A hugepage split is captured by this condition, see
+        * pmdp_invalidate.
+        *
+        * Huge page modification can be caught here too.
+        */
+       if (pmd_is_serializing(pmd))
+               return NULL;
+#endif
+
        if (pmd_trans_huge(pmd) || pmd_devmap(pmd)) {
                if (is_thp)
                        *is_thp = true;