mm: cachestat: fix two shmem bugs

When cachestat on shmem races with swapping and invalidation, there
are two possible bugs:

1) A swapin error can have resulted in a poisoned swap entry in the
   shmem inode's xarray. Calling get_shadow_from_swap_cache() on it
   will result in an out-of-bounds access to swapper_spaces[].

   Validate the entry with non_swap_entry() before going further.

2) When we find a valid swap entry in the shmem's inode, the shadow
   entry in the swapcache might not exist yet: swap IO is still in
   progress and we're before __remove_mapping; swapin, invalidation,
   or swapoff have removed the shadow from swapcache after we saw the
   shmem swap entry.

   This will send a NULL to workingset_test_recent(). The latter
   purely operates on pointer bits, so it won't crash - node 0, memcg
   ID 0, eviction timestamp 0, etc. are all valid inputs - but it's a
   bogus test. In theory that could result in a false "recently
   evicted" count.

   Such a false positive wouldn't be the end of the world. But for
   code clarity and (future) robustness, be explicit about this case.

   Bail on get_shadow_from_swap_cache() returning NULL.

Link: https://lkml.kernel.org/r/[email protected]
Fixes: cf264e1329fb ("cachestat: implement cachestat syscall")
Signed-off-by: Johannes Weiner <[email protected]>
Reported-by: Chengming Zhou <[email protected]>	[Bug #1]
Reported-by: Jann Horn <[email protected]>		[Bug #2]
Reviewed-by: Chengming Zhou <[email protected]>
Reviewed-by: Nhat Pham <[email protected]>
Cc: <[email protected]>				[v6.5+]
Signed-off-by: Andrew Morton <[email protected]>

diff --git a/mm/filemap.c b/mm/filemap.c
index 7437b2bd75c1ab48b093d390017e809af963f1e2..30de18c4fd28a907184695e3cf0f60079e956d89 100644 (file)
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -4197,7 +4197,23 @@ static void filemap_cachestat(struct address_space *mapping,
                                /* shmem file - in swap cache */
                                swp_entry_t swp = radix_to_swp_entry(folio);
 
+                               /* swapin error results in poisoned entry */
+                               if (non_swap_entry(swp))
+                                       goto resched;
+
+                               /*
+                                * Getting a swap entry from the shmem
+                                * inode means we beat
+                                * shmem_unuse(). rcu_read_lock()
+                                * ensures swapoff waits for us before
+                                * freeing the swapper space. However,
+                                * we can race with swapping and
+                                * invalidation, so there might not be
+                                * a shadow in the swapcache (yet).
+                                */
                                shadow = get_shadow_from_swap_cache(swp);
+                               if (!shadow)
+                                       goto resched;
                        }
 #endif
                        if (workingset_test_recent(shadow, true, &workingset))