ARM: 9016/2: Initialize the mapping of KASan shadow memory
This patch initializes KASan shadow region's page table and memory.
There are two stage for KASan initializing:
1. At early boot stage the whole shadow region is mapped to just
one physical page (kasan_zero_page). It is finished by the function
kasan_early_init which is called by __mmap_switched(arch/arm/kernel/
head-common.S)
2. After the calling of paging_init, we use kasan_zero_page as zero
shadow for some memory that KASan does not need to track, and we
allocate a new shadow space for the other memory that KASan need to
track. These issues are finished by the function kasan_init which is
call by setup_arch.
When using KASan we also need to increase the THREAD_SIZE_ORDER
from 1 to 2 as the extra calls for shadow memory uses quite a bit
of stack.
As we need to make a temporary copy of the PGD when setting up
shadow memory we create a helpful PGD_SIZE definition for both
LPAE and non-LPAE setups.
The KASan core code unconditionally calls pud_populate() so this
needs to be changed from BUG() to do {} while (0) when building
with KASan enabled.
After the initial development by Andre Ryabinin several modifications
have been made to this code:
Abbott Liu <
[email protected]>
- Add support ARM LPAE: If LPAE is enabled, KASan shadow region's
mapping table need be copied in the pgd_alloc() function.
- Change kasan_pte_populate,kasan_pmd_populate,kasan_pud_populate,
kasan_pgd_populate from .meminit.text section to .init.text section.
Reported by Florian Fainelli <
[email protected]>
Linus Walleij <
[email protected]>:
- Drop the custom mainpulation of TTBR0 and just use
cpu_switch_mm() to switch the pgd table.
- Adopt to handle 4th level page tabel folding.
- Rewrite the entire page directory and page entry initialization
sequence to be recursive based on ARM64:s kasan_init.c.
Ard Biesheuvel <
[email protected]>:
- Necessary underlying fixes.
- Crucial bug fixes to the memory set-up code.
Co-developed-by: Andrey Ryabinin <[email protected]>
Co-developed-by: Abbott Liu <[email protected]>
Co-developed-by: Ard Biesheuvel <[email protected]>
Cc: Alexander Potapenko <[email protected]>
Cc: Dmitry Vyukov <[email protected]>
Cc: [email protected]
Cc: Mike Rapoport <[email protected]>
Acked-by: Mike Rapoport <[email protected]>
Reviewed-by: Ard Biesheuvel <[email protected]>
Tested-by: Ard Biesheuvel <[email protected]> # QEMU/KVM/mach-virt/LPAE/8G
Tested-by: Florian Fainelli <[email protected]> # Brahma SoCs
Tested-by: Ahmad Fatoum <[email protected]> # i.MX6Q
Reported-by: Russell King - ARM Linux <[email protected]>
Reported-by: Florian Fainelli <[email protected]>
Signed-off-by: Andrey Ryabinin <[email protected]>
Signed-off-by: Abbott Liu <[email protected]>
Signed-off-by: Florian Fainelli <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Linus Walleij <[email protected]>
Signed-off-by: Russell King <[email protected]>