x86: do the user address masking outside the user access area

In any normal situation this really shouldn't matter, but in case the
address passed in to masked_user_access_begin() were to be some complex
expression, we should evaluate it fully before doing the 'stac'
instruction.

And even without that issue (which objdump would pick up on for any
really bad case), just in general we should strive to minimize the
amount of code we run with user accesses enabled.

For example, even for the trivial pselect6() case, the code generation
(obviously with a non-debug build) just diff with this ends up being

  -	stac
 	mov    %rax,%rcx
 	sar    $0x3f,%rcx
 	or     %rax,%rcx
  +	stac
 	mov    (%rcx),%r13
 	mov    0x8(%rcx),%r14
 	clac

so the area delimeted by the 'stac / clac' pair is now literally just
the two user access instructions, and the address generation has been
moved out to before that code.

This will be much more noticeable if we end up deciding that we can go
back to just inlining "get_user()" using the new masked user access
model.  The get_user() pointers can often be more complex expressions
involving kernel memory accesses or even function calls.

Signed-off-by: Linus Torvalds <[email protected]>

diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uaccess_64.h
index a10149a96d9e9992fadb4c983427a087161fc99f..92859c1ef59c4cc8862c3e769caf5e20a43a2b53 100644 (file)
--- a/arch/x86/include/asm/uaccess_64.h
+++ b/arch/x86/include/asm/uaccess_64.h
@@ -59,7 +59,9 @@ static inline unsigned long __untagged_addr_remote(struct mm_struct *mm,
  * for dense accesses starting at the address.
  */
 #define mask_user_address(x) ((typeof(x))((long)(x)|((long)(x)>>63)))
-#define masked_user_access_begin(x) ({ __uaccess_begin(); mask_user_address(x); })
+#define masked_user_access_begin(x) ({ \
+       __auto_type __masked_ptr = mask_user_address(x); \
+       __uaccess_begin(); __masked_ptr; })
 
 /*
  * User pointers can have tag bits on x86-64.  This scheme tolerates