crush: don't normalize input of crush_ln iteratively

Use __builtin_clz() supported by GCC and Clang to figure out
how many bits we should shift instead of shifting by a bit
in a loop until the value gets normalized. Improves performance
of this function by up to 3x in worst-case scenario and overall
straw2 performance by ~10%.

Reflects ceph.git commit 110de33ca497d94fc4737e5154d3fe781fa84a0a.

Signed-off-by: Ilya Dryomov <[email protected]>

diff --git a/net/ceph/crush/mapper.c b/net/ceph/crush/mapper.c
index 5fcfb98f309efb5018f84628902cd1711fc36705..511ade95339ad80c4eab12af814f07f7c4a40d39 100644 (file)
--- a/net/ceph/crush/mapper.c
+++ b/net/ceph/crush/mapper.c
@@ -253,9 +253,15 @@ static __u64 crush_ln(unsigned int xin)
 
        /* normalize input */
        iexpon = 15;
-       while (!(x & 0x18000)) {
-               x <<= 1;
-               iexpon--;
+
+       /*
+        * figure out number of bits we need to shift and
+        * do it in one step instead of iteratively
+        */
+       if (!(x & 0x18000)) {
+               int bits = __builtin_clz(x & 0x1FFFF) - 16;
+               x <<= bits;
+               iexpon = 15 - bits;
        }
 
        index1 = (x >> 8) << 1;