KVM: x86: Ensure a full memory barrier is emitted in the VM-Exit path

Ensure a full memory barrier is emitted in the VM-Exit path, as a full
barrier is required on Intel CPUs to evict WC buffers.  This will allow
unconditionally honoring guest PAT on Intel CPUs that support self-snoop.

As srcu_read_lock() is always called in the VM-Exit path and it internally
has a smp_mb(), call smp_mb__after_srcu_read_lock() to avoid adding a
second fence and make sure smp_mb() is called without dependency on
implementation details of srcu_read_lock().

Cc: Paolo Bonzini <[email protected]>
Cc: Sean Christopherson <[email protected]>
Cc: Kevin Tian <[email protected]>
Signed-off-by: Yan Zhao <[email protected]>
[sean: massage changelog]
Tested-by: Xiangfei Ma <[email protected]>
Tested-by: Yongwei Ma <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Sean Christopherson <[email protected]>

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 869cfaa980bdeca45d04df4f6402a8e3992418b8..121907049190f72e8ea183362b3a72c6ad066518 100644 (file)
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -11135,6 +11135,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 
        kvm_vcpu_srcu_read_lock(vcpu);
 
+       /*
+        * Call this to ensure WC buffers in guest are evicted after each VM
+        * Exit, so that the evicted WC writes can be snooped across all cpus
+        */
+       smp_mb__after_srcu_read_lock();
+
        /*
         * Profile KVM exit RIPs:
         */