Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm

Pull kvm updates from Paolo Bonzini:
 "Loongarch:

   - Clear LLBCTL if secondary mmu mapping changes

   - Add hypercall service support for usermode VMM

  x86:

   - Add a comment to kvm_mmu_do_page_fault() to explain why KVM
     performs a direct call to kvm_tdp_page_fault() when RETPOLINE is
     enabled

   - Ensure that all SEV code is compiled out when disabled in Kconfig,
     even if building with less brilliant compilers

   - Remove a redundant TLB flush on AMD processors when guest CR4.PGE
     changes

   - Use str_enabled_disabled() to replace open coded strings

   - Drop kvm_x86_ops.hwapic_irr_update() as KVM updates hardware's
     APICv cache prior to every VM-Enter

   - Overhaul KVM's CPUID feature infrastructure to track all vCPU
     capabilities instead of just those where KVM needs to manage state
     and/or explicitly enable the feature in hardware. Along the way,
     refactor the code to make it easier to add features, and to make it
     more self-documenting how KVM is handling each feature

   - Rework KVM's handling of VM-Exits during event vectoring; this
     plugs holes where KVM unintentionally puts the vCPU into infinite
     loops in some scenarios (e.g. if emulation is triggered by the
     exit), and brings parity between VMX and SVM

   - Add pending request and interrupt injection information to the
     kvm_exit and kvm_entry tracepoints respectively

   - Fix a relatively benign flaw where KVM would end up redoing RDPKRU
     when loading guest/host PKRU, due to a refactoring of the kernel
     helpers that didn't account for KVM's pre-checking of the need to
     do WRPKRU

   - Make the completion of hypercalls go through the complete_hypercall
     function pointer argument, no matter if the hypercall exits to
     userspace or not.

     Previously, the code assumed that KVM_HC_MAP_GPA_RANGE specifically
     went to userspace, and all the others did not; the new code need
     not special case KVM_HC_MAP_GPA_RANGE and in fact does not care at
     all whether there was an exit to userspace or not

   - As part of enabling TDX virtual machines, support support
     separation of private/shared EPT into separate roots.

     When TDX will be enabled, operations on private pages will need to
     go through the privileged TDX Module via SEAMCALLs; as a result,
     they are limited and relatively slow compared to reading a PTE.

     The patches included in 6.14 allow KVM to keep a mirror of the
     private EPT in host memory, and define entries in kvm_x86_ops to
     operate on external page tables such as the TDX private EPT

   - The recently introduced conversion of the NX-page reclamation
     kthread to vhost_task moved the task under the main process. The
     task is created as soon as KVM_CREATE_VM was invoked and this, of
     course, broke userspace that didn't expect to see any child task of
     the VM process until it started creating its own userspace threads.

     In particular crosvm refuses to fork() if procfs shows any child
     task, so unbreak it by creating the task lazily. This is arguably a
     userspace bug, as there can be other kinds of legitimate worker
     tasks and they wouldn't impede fork(); but it's not like userspace
     has a way to distinguish kernel worker tasks right now. Should they
     show as "Kthread: 1" in proc/.../status?

  x86 - Intel:

   - Fix a bug where KVM updates hardware's APICv cache of the highest
     ISR bit while L2 is active, while ultimately results in a
     hardware-accelerated L1 EOI effectively being lost

   - Honor event priority when emulating Posted Interrupt delivery
     during nested VM-Enter by queueing KVM_REQ_EVENT instead of
     immediately handling the interrupt

   - Rework KVM's processing of the Page-Modification Logging buffer to
     reap entries in the same order they were created, i.e. to mark gfns
     dirty in the same order that hardware marked the page/PTE dirty

   - Misc cleanups

  Generic:

   - Cleanup and harden kvm_set_memory_region(); add proper lockdep
     assertions when setting memory regions and add a dedicated API for
     setting KVM-internal memory regions. The API can then explicitly
     disallow all flags for KVM-internal memory regions

   - Explicitly verify the target vCPU is online in kvm_get_vcpu() to
     fix a bug where KVM would return a pointer to a vCPU prior to it
     being fully online, and give kvm_for_each_vcpu() similar treatment
     to fix a similar flaw

   - Wait for a vCPU to come online prior to executing a vCPU ioctl, to
     fix a bug where userspace could coerce KVM into handling the ioctl
     on a vCPU that isn't yet onlined

   - Gracefully handle xarray insertion failures; even though such
     failures are impossible in practice after xa_reserve(), reserving
     an entry is always followed by xa_store() which does not know (or
     differentiate) whether there was an xa_reserve() before or not

  RISC-V:

   - Zabha, Svvptc, and Ziccrse extension support for guests. None of
     them require anything in KVM except for detecting them and marking
     them as supported; Zabha adds byte and halfword atomic operations,
     while the others are markers for specific operation of the TLB and
     of LL/SC instructions respectively

   - Virtualize SBI system suspend extension for Guest/VM

   - Support firmware counters which can be used by the guests to
     collect statistics about traps that occur in the host

  Selftests:

   - Rework vcpu_get_reg() to return a value instead of using an
     out-param, and update all affected arch code accordingly

   - Convert the max_guest_memory_test into a more generic
     mmu_stress_test. The basic gist of the "conversion" is to have the
     test do mprotect() on guest memory while vCPUs are accessing said
     memory, e.g. to verify KVM and mmu_notifiers are working as
     intended

   - Play nice with treewrite builds of unsupported architectures, e.g.
     arm (32-bit), as KVM selftests' Makefile doesn't do anything to
     ensure the target architecture is actually one KVM selftests
     supports

   - Use the kernel's $(ARCH) definition instead of the target triple
     for arch specific directories, e.g. arm64 instead of aarch64,
     mainly so as not to be different from the rest of the kernel

   - Ensure that format strings for logging statements are checked by
     the compiler even when the logging statement itself is disabled

   - Attempt to whack the last LLC references/misses mole in the Intel
     PMU counters test by adding a data load and doing CLFLUSH{OPT} on
     the data instead of the code being executed. It seems that modern
     Intel CPUs have learned new code prefetching tricks that bypass the
     PMU counters

   - Fix a flaw in the Intel PMU counters test where it asserts that
     events are counting correctly without actually knowing what the
     events count given the underlying hardware; this can happen if
     Intel reuses a formerly microarchitecture-specific event encoding
     as an architectural event, as was the case for Top-Down Slots"

* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (151 commits)
  kvm: defer huge page recovery vhost task to later
  KVM: x86/mmu: Return RET_PF* instead of 1 in kvm_mmu_page_fault()
  KVM: Disallow all flags for KVM-internal memslots
  KVM: x86: Drop double-underscores from __kvm_set_memory_region()
  KVM: Add a dedicated API for setting KVM-internal memslots
  KVM: Assert slots_lock is held when setting memory regions
  KVM: Open code kvm_set_memory_region() into its sole caller (ioctl() API)
  LoongArch: KVM: Add hypercall service support for usermode VMM
  LoongArch: KVM: Clear LLBCTL if secondary mmu mapping is changed
  KVM: SVM: Use str_enabled_disabled() helper in svm_hardware_setup()
  KVM: VMX: read the PML log in the same order as it was written
  KVM: VMX: refactor PML terminology
  KVM: VMX: Fix comment of handle_vmx_instruction()
  KVM: VMX: Reinstate __exit attribute for vmx_exit()
  KVM: SVM: Use str_enabled_disabled() helper in sev_hardware_setup()
  KVM: x86: Avoid double RDPKRU when loading host/guest PKRU
  KVM: x86: Use LVT_TIMER instead of an open coded literal
  RISC-V: KVM: Add new exit statstics for redirected traps
  RISC-V: KVM: Update firmware counters for various events
  RISC-V: KVM: Redirect instruction access fault trap to guest
  ...

diff --cc Documentation/virt/kvm/api.rst
Simple merge

diff --cc MAINTAINERS
Simple merge

diff --cc arch/x86/include/asm/kvm_host.h
index e8aeb4b4f868e87ff00e02277d4600133c2ac228,f378cd43241c6570411e2452ddbdbc060d3a41ba..b15cde0a9b5ca7ae2c90b15fed4a4debd34f825c
--- 1/arch/x86/include/asm/kvm_host.h
--- 2/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@@ -24,8 -24,10 +24,9 @@@
  #include <linux/pvclock_gtod.h>
  #include <linux/clocksource.h>
  #include <linux/irqbypass.h>
 -#include <linux/hyperv.h>
  #include <linux/kfifo.h>
  #include <linux/sched/vhost_task.h>
+ #include <linux/call_once.h>
  
  #include <asm/apic.h>
  #include <asm/pvclock-abi.h>

diff --cc arch/x86/kvm/cpuid.c
index f7e222953cab766da585988ba73392f139b04a9e,edef30359c198d28556c140f1043ab282c6f801b..2cbb3874ad3984d53c92b06eb1983f62c7d8aef6
--- 1/arch/x86/kvm/cpuid.c
--- 2/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@@ -808,50 -1134,72 +1134,73 @@@ void kvm_set_cpu_caps(void
            !boot_cpu_has(X86_FEATURE_AMD_SSBD))
                kvm_cpu_cap_set(X86_FEATURE_VIRT_SSBD);
  
-       /*
-        * Hide all SVM features by default, SVM will set the cap bits for
-        * features it emulates and/or exposes for L1.
-        */
-       kvm_cpu_cap_mask(CPUID_8000_000A_EDX, 0);
- 
-       kvm_cpu_cap_mask(CPUID_8000_001F_EAX,
-               0 /* SME */ | 0 /* SEV */ | 0 /* VM_PAGE_FLUSH */ | 0 /* SEV_ES */ |
-               F(SME_COHERENT));
+       /* All SVM features required additional vendor module enabling. */
+       kvm_cpu_cap_init(CPUID_8000_000A_EDX,
+               VENDOR_F(NPT),
+               VENDOR_F(VMCBCLEAN),
+               VENDOR_F(FLUSHBYASID),
+               VENDOR_F(NRIPS),
+               VENDOR_F(TSCRATEMSR),
+               VENDOR_F(V_VMSAVE_VMLOAD),
+               VENDOR_F(LBRV),
+               VENDOR_F(PAUSEFILTER),
+               VENDOR_F(PFTHRESHOLD),
+               VENDOR_F(VGIF),
+               VENDOR_F(VNMI),
+               VENDOR_F(SVME_ADDR_CHK),
+       );
  
-       kvm_cpu_cap_mask(CPUID_8000_0021_EAX,
-               F(NO_NESTED_DATA_BP) | F(LFENCE_RDTSC) | 0 /* SmmPgCfgLock */ |
-               F(NULL_SEL_CLR_BASE) | F(AUTOIBRS) | 0 /* PrefetchCtlMsr */ |
-               F(WRMSR_XX_BASE_NS) | F(SRSO_USER_KERNEL_NO)
+       kvm_cpu_cap_init(CPUID_8000_001F_EAX,
+               VENDOR_F(SME),
+               VENDOR_F(SEV),
+               /* VM_PAGE_FLUSH */
+               VENDOR_F(SEV_ES),
+               F(SME_COHERENT),
        );
  
-       kvm_cpu_cap_check_and_set(X86_FEATURE_SBPB);
-       kvm_cpu_cap_check_and_set(X86_FEATURE_IBPB_BRTYPE);
-       kvm_cpu_cap_check_and_set(X86_FEATURE_SRSO_NO);
+       kvm_cpu_cap_init(CPUID_8000_0021_EAX,
+               F(NO_NESTED_DATA_BP),
+               /*
+                * Synthesize "LFENCE is serializing" into the AMD-defined entry
+                * in KVM's supported CPUID, i.e. if the feature is reported as
+                * supported by the kernel.  LFENCE_RDTSC was a Linux-defined
+                * synthetic feature long before AMD joined the bandwagon, e.g.
+                * LFENCE is serializing on most CPUs that support SSE2.  On
+                * CPUs that don't support AMD's leaf, ANDing with the raw host
+                * CPUID will drop the flags, and reporting support in AMD's
+                * leaf can make it easier for userspace to detect the feature.
+                */
+               SYNTHESIZED_F(LFENCE_RDTSC),
+               /* SmmPgCfgLock */
+               F(NULL_SEL_CLR_BASE),
+               F(AUTOIBRS),
+               EMULATED_F(NO_SMM_CTL_MSR),
+               /* PrefetchCtlMsr */
+               F(WRMSR_XX_BASE_NS),
+               SYNTHESIZED_F(SBPB),
+               SYNTHESIZED_F(IBPB_BRTYPE),
+               SYNTHESIZED_F(SRSO_NO),
++              SYNTHESIZED_F(SRSO_USER_KERNEL_NO),
+       );
  
-       kvm_cpu_cap_init_kvm_defined(CPUID_8000_0022_EAX,
-               F(PERFMON_V2)
+       kvm_cpu_cap_init(CPUID_8000_0022_EAX,
+               F(PERFMON_V2),
        );
  
-       /*
-        * Synthesize "LFENCE is serializing" into the AMD-defined entry in
-        * KVM's supported CPUID if the feature is reported as supported by the
-        * kernel.  LFENCE_RDTSC was a Linux-defined synthetic feature long
-        * before AMD joined the bandwagon, e.g. LFENCE is serializing on most
-        * CPUs that support SSE2.  On CPUs that don't support AMD's leaf,
-        * kvm_cpu_cap_mask() will unfortunately drop the flag due to ANDing
-        * the mask with the raw host CPUID, and reporting support in AMD's
-        * leaf can make it easier for userspace to detect the feature.
-        */
-       if (cpu_feature_enabled(X86_FEATURE_LFENCE_RDTSC))
-               kvm_cpu_cap_set(X86_FEATURE_LFENCE_RDTSC);
        if (!static_cpu_has_bug(X86_BUG_NULL_SEG))
                kvm_cpu_cap_set(X86_FEATURE_NULL_SEL_CLR_BASE);
-       kvm_cpu_cap_set(X86_FEATURE_NO_SMM_CTL_MSR);
  
-       kvm_cpu_cap_mask(CPUID_C000_0001_EDX,
-               F(XSTORE) | F(XSTORE_EN) | F(XCRYPT) | F(XCRYPT_EN) |
-               F(ACE2) | F(ACE2_EN) | F(PHE) | F(PHE_EN) |
-               F(PMM) | F(PMM_EN)
+       kvm_cpu_cap_init(CPUID_C000_0001_EDX,
+               F(XSTORE),
+               F(XSTORE_EN),
+               F(XCRYPT),
+               F(XCRYPT_EN),
+               F(ACE2),
+               F(ACE2_EN),
+               F(PHE),
+               F(PHE_EN),
+               F(PMM),
+               F(PMM_EN),
        );
  
        /*

diff --cc arch/x86/kvm/svm/sev.c
Simple merge