target-i386: fix segment limit check in ljmp

The current implementation has three bugs,
 * segment limits are not enforced in protected mode if the L bit is set
   in the target segment descriptor
 * segment limits are not enforced in compatibility mode (ljmp to 32-bit
   code segment in long mode)
 * #GP(new_cs) is generated rather than #GP(0)

Now the segment limits are enforced if we're not in long mode OR the
target code segment doesn't have the L bit set.

Signed-off-by: Andrew Oates <[email protected]>
Message-Id: <20180816011903[email protected]>
Signed-off-by: Paolo Bonzini <[email protected]>

diff --git a/target/i386/seg_helper.c b/target/i386/seg_helper.c
index b2adddcd7fa0a7747b7a3204bf2cb5c51dcf9469..d1cbc6ebf0d696cd7d50100f9ee27c7a1f3981b1 100644 (file)
--- a/target/i386/seg_helper.c
+++ b/target/i386/seg_helper.c
@@ -1633,8 +1633,8 @@ void helper_ljmp_protected(CPUX86State *env, int new_cs, target_ulong new_eip,
         }
         limit = get_seg_limit(e1, e2);
         if (new_eip > limit &&
-            !(env->hflags & HF_LMA_MASK) && !(e2 & DESC_L_MASK)) {
-            raise_exception_err_ra(env, EXCP0D_GPF, new_cs & 0xfffc, GETPC());
+            (!(env->hflags & HF_LMA_MASK) || !(e2 & DESC_L_MASK))) {
+            raise_exception_err_ra(env, EXCP0D_GPF, 0, GETPC());
         }
         cpu_x86_load_seg_cache(env, R_CS, (new_cs & 0xfffc) | cpl,
                        get_seg_base(e1, e2), limit, e2);