The only remaining user of the fallback implementation of 64x64
polynomial multiplication using 8x8 PMULL instructions is the final
reduction from a 16 byte vector to a 16-bit CRC.
The fallback code is complicated and messy, and this reduction has
little impact on the overall performance, so instead, let's calculate
the final CRC by passing the 16 byte vector to the generic CRC-T10DIF
implementation when running the fallback version.
Ard Biesheuvel [Tue, 5 Nov 2024 16:09:02 +0000 (17:09 +0100)]
crypto: arm64/crct10dif - Use faster 16x64 bit polynomial multiply
The CRC-T10DIF implementation for arm64 has a version that uses 8x8
polynomial multiplication, for cores that lack the crypto extensions,
which cover the 64x64 polynomial multiplication instruction that the
algorithm was built around.
This fallback version rather naively adopted the 64x64 polynomial
multiplication algorithm that I ported from ARM for the GHASH driver,
which needs 8 PMULL8 instructions to implement one PMULL64. This is
reasonable, given that each 8-bit vector element needs to be multiplied
with each element in the other vector, producing 8 vectors with partial
results that need to be combined to yield the correct result.
However, most PMULL64 invocations in the CRC-T10DIF code involve
multiplication by a pair of 16-bit folding coefficients, and so all the
partial results from higher order bytes will be zero, and there is no
need to calculate them to begin with.
Then, the CRC-T10DIF algorithm always XORs the output values of the
PMULL64 instructions being issued in pairs, and so there is no need to
faithfully implement each individual PMULL64 instruction, as long as
XORing the results pairwise produces the expected result.
Implementing these improvements results in a speedup of 3.3x on low-end
platforms such as Raspberry Pi 4 (Cortex-A72)
This is a partial revert of commit fc754c024a343b, which moved the logic
into C code which ensures that kernel mode NEON code does not hog the
CPU for too long.
This is no longer needed now that kernel mode NEON no longer disables
preemption, so we can drop this.
Chen Ridong [Mon, 4 Nov 2024 12:17:45 +0000 (12:17 +0000)]
crypto: bcm - add error check in the ahash_hmac_init function
The ahash_init functions may return fails. The ahash_hmac_init should
not return ok when ahash_init returns error. For an example, ahash_init
will return -ENOMEM when allocation memory is error.
Zicheng Qu [Fri, 1 Nov 2024 09:13:24 +0000 (09:13 +0000)]
padata: Clean up in padata_do_multithreaded()
In commit 24cc57d8faaa ("padata: Honor the caller's alignment in case of
chunk_size 0"), the line 'ps.chunk_size = max(ps.chunk_size, 1ul)' was
added, making 'ps.chunk_size = 1U' redundant and never executed.
Li Huafei [Thu, 31 Oct 2024 11:27:55 +0000 (19:27 +0800)]
crypto: inside-secure - Fix the return value of safexcel_xcbcmac_cra_init()
The commit 320406cb60b6 ("crypto: inside-secure - Replace generic aes
with libaes") replaced crypto_alloc_cipher() with kmalloc(), but did not
modify the handling of the return value. When kmalloc() returns NULL,
PTR_ERR_OR_ZERO(NULL) returns 0, but in fact, the memory allocation has
failed, and -ENOMEM should be returned.
Fixes: 320406cb60b6 ("crypto: inside-secure - Replace generic aes with libaes") Signed-off-by: Li Huafei <[email protected]> Acked-by: Antoine Tenart <[email protected]> Signed-off-by: Herbert Xu <[email protected]>
Wang Hai [Tue, 29 Oct 2024 15:05:23 +0000 (23:05 +0800)]
crypto: qat - Fix missing destroy_workqueue in adf_init_aer()
The adf_init_aer() won't destroy device_reset_wq when alloc_workqueue()
for device_sriov_wq failed. Add destroy_workqueue for device_reset_wq to
fix this issue.
Fixes: 4469f9b23468 ("crypto: qat - re-enable sriov after pf reset") Signed-off-by: Wang Hai <[email protected]> Signed-off-by: Herbert Xu <[email protected]>
Lukas Wunner [Tue, 29 Oct 2024 10:24:57 +0000 (11:24 +0100)]
crypto: rsassa-pkcs1 - Reinstate support for legacy protocols
Commit 1e562deacecc ("crypto: rsassa-pkcs1 - Migrate to sig_alg backend")
enforced that rsassa-pkcs1 sign/verify operations specify a hash
algorithm. That is necessary because per RFC 8017 sec 8.2, a hash
algorithm identifier must be prepended to the hash before generating or
verifying the signature ("Full Hash Prefix").
However the commit went too far in that it changed user space behavior:
KEYCTL_PKEY_QUERY system calls now return -EINVAL unless they specify a
hash algorithm. Intel Wireless Daemon (iwd) is one application issuing
such system calls (for EAP-TLS).
Closer analysis of the Embedded Linux Library (ell) used by iwd reveals
that the problem runs even deeper: When iwd uses TLS 1.1 or earlier, it
not only queries for keys, but performs sign/verify operations without
specifying a hash algorithm. These legacy TLS versions concatenate an
MD5 to a SHA-1 hash and omit the Full Hash Prefix:
TLS 1.1 was deprecated in 2021 by RFC 8996, but removal of support was
inadvertent in this case. It probably should be coordinated with iwd
maintainers first.
So reinstate support for such legacy protocols by defaulting to hash
algorithm "none" which uses an empty Full Hash Prefix.
If it is later on decided to remove TLS 1.1 support but still allow
KEYCTL_PKEY_QUERY without a hash algorithm, that can be achieved by
reverting the present commit and replacing it with the following patch:
It's worth noting that Python's cryptography library gained support for
such legacy use cases very recently, so they do seem to still be a thing.
The Python developers identified IKE version 1 as another protocol
omitting the Full Hash Prefix:
The author of those issues, Zoltan Kelemen, spent considerable effort
searching for test vectors but only found one in a 2019 blog post by
Kevin Jones. Add it to testmgr.h to verify correctness of this feature.
Examination of wpa_supplicant as well as various IKE daemons (libreswan,
strongswan, isakmpd, raccoon) has determined that none of them seems to
use the kernel's Key Retention Service, so iwd is the only affected user
space application known so far.
Weili Qian [Sat, 26 Oct 2024 11:44:29 +0000 (19:44 +0800)]
crypto: hisilicon/qm - disable same error report before resetting
If an error indicating that the device needs to be reset is reported,
disable the error reporting before device reset is complete,
enable the error reporting after the reset is complete to prevent
the same error from being reported repeatedly.
Qi Tao [Sat, 26 Oct 2024 09:46:51 +0000 (17:46 +0800)]
crypto: hisilicon - support querying the capability register
Query the capability register status of accelerator devices
(SEC, HPRE and ZIP) through the debugfs interface, for example:
cat cap_regs. The purpose is to improve the robustness and
locability of hardware devices and drivers.
encrypt_blob(), decrypt_blob() and create_signature() were some of the
functions added in 2018 by
commit 5a30771832aa ("KEYS: Provide missing asymmetric key subops for new
key type ops [ver #2]")
however, they've not been used.
Uwe Kleine-König [Mon, 21 Oct 2024 10:48:55 +0000 (12:48 +0200)]
hwrng: drivers - Switch back to struct platform_driver::remove()
After commit 0edb555a65d1 ("platform: Make platform_driver::remove()
return void") .remove() is (again) the right callback to implement for
platform drivers.
Convert all platform drivers below drivers/char/hw_random to use
.remove(), with the eventual goal to drop struct
platform_driver::remove_new(). As .remove() and .remove_new() have the
same prototypes, conversion is done by just changing the structure
member name in the driver initializer.
Eric Biggers [Fri, 18 Oct 2024 23:53:43 +0000 (16:53 -0700)]
crypto: api - move crypto_simd_disabled_for_test to lib
Move crypto_simd_disabled_for_test to lib/ so that crypto_simd_usable()
can be used by library code.
This was discussed previously
(https://lore.kernel.org/linux-crypto/20220716062920[email protected]/)
but was not done because there was no use case yet. However, this is
now needed for the arm64 CRC32 library code.
Everest K.C [Fri, 18 Oct 2024 16:23:10 +0000 (10:23 -0600)]
crypto: cavium - Fix the if condition to exit loop after timeout
The while loop breaks in the first run because of incorrect
if condition. It also causes the statements after the if to
appear dead.
Fix this by changing the condition from if(timeout--) to
if(!timeout--).
This bug was reported by Coverity Scan.
Report:
CID 1600859: (#1 of 1): Logically dead code (DEADCODE)
dead_error_line: Execution cannot reach this statement: udelay(30UL);
Fixes: 9e2c7d99941d ("crypto: cavium - Add Support for Octeon-tx CPT Engine") Signed-off-by: Everest K.C. <[email protected]> Signed-off-by: Herbert Xu <[email protected]>
dt-bindings: rng: add support for Airoha EN7581 TRNG
Add support for Airoha EN7581 True Random Number generator.
This module can generate up to 4bytes of raw data at times and support
self health test at startup. The module gets noise for randomness from
various source from ADC, AP, dedicated clocks and other devices attached
to the SoC producing true random numbers.
Eric Biggers [Thu, 17 Oct 2024 00:00:50 +0000 (17:00 -0700)]
crypto: x86/aegis128 - remove unneeded FRAME_BEGIN and FRAME_END
Stop using FRAME_BEGIN and FRAME_END in the AEGIS assembly functions,
since all these functions are now leaf functions. This eliminates some
unnecessary instructions.
Eric Biggers [Thu, 17 Oct 2024 00:00:49 +0000 (17:00 -0700)]
crypto: x86/aegis128 - take advantage of block-aligned len
Update a caller of aegis128_aesni_ad() to round down the length to a
block boundary. After that, aegis128_aesni_ad(), aegis128_aesni_enc(),
and aegis128_aesni_dec() are only passed whole blocks. Update the
assembly code to take advantage of that, which eliminates some unneeded
instructions. For aegis128_aesni_enc() and aegis128_aesni_dec(), the
length is also always nonzero, so stop checking for zero length.
Eric Biggers [Thu, 17 Oct 2024 00:00:47 +0000 (17:00 -0700)]
crypto: x86/aegis128 - improve assembly function prototypes
Adjust the prototypes of the AEGIS assembly functions:
- Use proper types instead of 'void *', when applicable.
- Move the length parameter to after the buffers it describes rather
than before, to match the usual convention. Also shorten its name to
just len (which is the name used in the assembly code).
- Declare register aliases at the beginning of each function rather than
once per file. This was necessary because len was moved, but also it
allows adding some aliases where raw registers were used before.
- Put assoclen and cryptlen in the correct order when declaring the
finalization function in the .c file.
Eric Biggers [Thu, 17 Oct 2024 00:00:46 +0000 (17:00 -0700)]
crypto: x86/aegis128 - optimize length block preparation using SSE4.1
Start using SSE4.1 instructions in the AES-NI AEGIS code, with the first
use case being preparing the length block in fewer instructions.
In practice this does not reduce the set of CPUs on which the code can
run, because all Intel and AMD CPUs with AES-NI also have SSE4.1.
Upgrade the existing SSE2 feature check to SSE4.1, though it seems this
check is not strictly necessary; the aesni-intel module has been getting
away with using SSE4.1 despite checking for AES-NI only.
Eric Biggers [Thu, 17 Oct 2024 00:00:45 +0000 (17:00 -0700)]
crypto: x86/aegis128 - don't bother with special code for aligned data
Remove the AEGIS assembly code paths that were "optimized" to operate on
16-byte aligned data using movdqa, and instead just use the code paths
that use movdqu and can handle data with any alignment.
This does not reduce performance. movdqa is basically a historical
artifact; on aligned data, movdqu and movdqa have had the same
performance since Intel Nehalem (2008) and AMD Bulldozer (2011). And
code that requires AES-NI cannot run on CPUs older than those anyway.
Eric Biggers [Thu, 17 Oct 2024 00:00:44 +0000 (17:00 -0700)]
crypto: x86/aegis128 - eliminate some indirect calls
Instead of using a struct of function pointers to decide whether to call
the encryption or decryption assembly functions, use a conditional
branch on a bool. Force-inline the functions to avoid actually
generating the branch. This improves performance slightly since
indirect calls are slow. Remove the now-unnecessary CFI stubs.
Note that just force-inlining the existing functions might cause the
compiler to optimize out the indirect branches, but that would not be a
reliable way to do it and the CFI stubs would still be required.
Eric Biggers [Thu, 17 Oct 2024 00:00:42 +0000 (17:00 -0700)]
crypto: x86/aegis128 - access 32-bit arguments as 32-bit
Fix the AEGIS assembly code to access 'unsigned int' arguments as 32-bit
values instead of 64-bit, since the upper bits of the corresponding
64-bit registers are not guaranteed to be zero.
Note: there haven't been any reports of this bug actually causing
incorrect behavior. Neither gcc nor clang guarantee zero-extension to
64 bits, but zero-extension is likely to happen in practice because most
instructions that operate on 32-bit registers zero-extend to 64 bits.
Ard Biesheuvel [Wed, 16 Oct 2024 18:57:25 +0000 (20:57 +0200)]
crypto: crc32c - Provide crc32c-arch driver for accelerated library code
crc32c-generic is currently backed by the architecture's CRC-32c library
code, which may offer a variety of implementations depending on the
capabilities of the platform. These are not covered by the crypto
subsystem's fuzz testing capabilities because crc32c-generic is the
reference driver that the fuzzing logic uses as a source of truth.
Fix this by providing a crc32c-arch implementation which is based on the
arch library code if available, and modify crc32c-generic so it is
always based on the generic C implementation. If the arch has no CRC-32c
library code, this change does nothing.
Ard Biesheuvel [Wed, 16 Oct 2024 18:57:24 +0000 (20:57 +0200)]
crypto: crc32 - Provide crc32-arch driver for accelerated library code
crc32-generic is currently backed by the architecture's CRC-32 library
code, which may offer a variety of implementations depending on the
capabilities of the platform. These are not covered by the crypto
subsystem's fuzz testing capabilities because crc32-generic is the
reference driver that the fuzzing logic uses as a source of truth.
Fix this by providing a crc32-arch implementation which is based on the
arch library code if available, and modify crc32-generic so it is
always based on the generic C implementation. If the arch has no CRC-32
library code, this change does nothing.
hwrng: stm32 - implement support for STM32MP25x platforms
Implement the support for STM32MP25x platforms. On this platform, a
security clock is shared between some hardware blocks. For the RNG,
it is the RNG kernel clock. Therefore, the gate is no more shared
between the RNG bus and kernel clocks as on STM32MP1x platforms and
the bus clock has to be managed on its own.
Colin Ian King [Tue, 15 Oct 2024 13:11:22 +0000 (14:11 +0100)]
crypto: tegra - remove redundant error check on ret
Currently there is an unnecessary error check on ret without a proceeding
assignment to ret that needs checking. The check is redundant and can be
removed.
Vishal Chourasia [Tue, 15 Oct 2024 10:55:52 +0000 (16:25 +0530)]
crypto: nx - Fix invalid wait context during kexec reboot
nx842_remove() call of_reconfig_notifier_unregister while holding the
devdata_spinlock. This could lead to an invalid wait context error during
kexec reboot, as of_reconfig_notifier_unregister tries to acquire a read-write
semaphore (check logs) while holding a spinlock.
Move the of_reconfig_notifier_unregister() call before acquiring the
spinlock to prevent this race condition invalid wait contexts during system
shutdown or kexec operations.
Since commit 8f4f68e788c3 ("crypto: pcrypt - Fix hungtask for
PADATA_RESET"), the pcrypt encryption and decryption operations return
-EAGAIN when the CPU goes online or offline. In alg_test(), a WARN is
generated when pcrypt_aead_decrypt() or pcrypt_aead_encrypt() returns
-EAGAIN, the unnecessary panic will occur when panic_on_warn set 1.
Fix this issue by calling crypto layer directly without parallelization
in that case.
Fixes: 8f4f68e788c3 ("crypto: pcrypt - Fix hungtask for PADATA_RESET") Signed-off-by: Yi Yang <[email protected]> Signed-off-by: Herbert Xu <[email protected]>
'struct pm_status_row' are not modified in this driver.
Constifying this structure moves some data to a read-only section, so
increases overall security.
Update the prototype of some functions accordingly.
On a x86_64, with allmodconfig, as an example:
Before:
======
text data bss dec hex filename
4400 1059 0 5459 1553 drivers/crypto/intel/qat/qat_common/adf_gen4_pm_debugfs.o
After:
=====
text data bss dec hex filename
5216 243 0 5459 1553 drivers/crypto/intel/qat/qat_common/adf_gen4_pm_debugfs.o
The Marvell Armada RNG uses the same IP as TI from Inside Secure and is
already using the binding. The only missing part is the
"marvell,armada-8k-rng" compatible string.
Rename the binding to inside-secure,safexcel-eip76.yaml to better
reflect it is multi-vendor, licensed IP and to follow the naming
convention using compatible string.
Lukas Wunner [Mon, 14 Oct 2024 10:04:41 +0000 (12:04 +0200)]
crypto: ecdsa - Update Kconfig help text for NIST P521
Commit a7d45ba77d3d ("crypto: ecdsa - Register NIST P521 and extend test
suite") added support for ECDSA signature verification using NIST P521,
but forgot to amend the Kconfig help text. Fix it.
Fixes: a7d45ba77d3d ("crypto: ecdsa - Register NIST P521 and extend test suite") Signed-off-by: Lukas Wunner <[email protected]> Reviewed-by: Stefan Berger <[email protected]> Signed-off-by: Herbert Xu <[email protected]>
Lukas Wunner [Mon, 14 Oct 2024 09:43:01 +0000 (11:43 +0200)]
crypto: sig - Fix oops on KEYCTL_PKEY_QUERY for RSA keys
Commit a2471684dae2 ("crypto: ecdsa - Move X9.62 signature size
calculation into template") introduced ->max_size() and ->digest_size()
callbacks to struct sig_alg. They return an algorithm's maximum
signature size and digest size, respectively.
For algorithms which lack these callbacks, crypto_register_sig() was
amended to use the ->key_size() callback instead.
However the commit neglected to also amend sig_register_instance().
As a result, the ->max_size() and ->digest_size() callbacks remain NULL
pointers if instances do not define them. A KEYCTL_PKEY_QUERY system
call results in an oops for such instances:
The only instances affected by this are "pkcs1(rsa, ...)".
Fix by moving the callback checks from crypto_register_sig() to
sig_prepare_alg(), which is also invoked by sig_register_instance().
Change the return type of sig_prepare_alg() from void to int to be able
to return errors. This matches other algorithm types, see e.g.
aead_prepare_alg() or ahash_prepare_alg().
Eric Biggers [Mon, 14 Oct 2024 04:24:47 +0000 (21:24 -0700)]
crypto: x86/crc32c - eliminate jump table and excessive unrolling
crc32c-pcl-intel-asm_64.S has a loop with 1 to 127 iterations fully
unrolled and uses a jump table to jump into the correct location. This
optimization is misguided, as it bloats the binary code size and
introduces an indirect call. x86_64 CPUs can predict loops well, so it
is fine to just use a loop instead. Loop bookkeeping instructions can
compete with the crc instructions for the ALUs, but this is easily
mitigated by unrolling the loop by a smaller amount, such as 4 times.
Therefore, re-roll the loop and make related tweaks to the code.
This reduces the binary code size of crc_pclmul() from 4546 bytes to 418
bytes, a 91% reduction. In general it also makes the code faster, with
some large improvements seen when retpoline is enabled.
More detailed performance results are shown below. They are given as
percent improvement in throughput (negative means regressed) for CPU
microarchitecture vs. input length in bytes. E.g. an improvement from
40 GB/s to 50 GB/s would be listed as 25%.
Table 1: Results with retpoline enabled (the default):
Eric Biggers [Mon, 14 Oct 2024 04:24:46 +0000 (21:24 -0700)]
crypto: x86/crc32c - access 32-bit arguments as 32-bit
Fix crc32c-pcl-intel-asm_64.S to access 32-bit arguments as 32-bit
values instead of 64-bit, since the upper bits of the corresponding
64-bit registers are not guaranteed to be zero. Also update the type of
the length argument to be unsigned int rather than int, as the assembly
code treats it as unsigned.
Note: there haven't been any reports of this bug actually causing
incorrect behavior. Neither gcc nor clang guarantee zero-extension to
64 bits, but zero-extension is likely to happen in practice because most
instructions that operate on 32-bit registers zero-extend to 64 bits.
Eric Biggers [Mon, 14 Oct 2024 04:24:45 +0000 (21:24 -0700)]
crypto: x86/crc32c - simplify code for handling fewer than 200 bytes
The assembly code in crc32c-pcl-intel-asm_64.S is invoked only for
lengths >= 512, due to the overhead of saving and restoring FPU state.
Therefore, it is unnecessary for this code to be excessively "optimized"
for lengths < 200. Eliminate the excessive unrolling of this part of
the code and use a more straightforward qword-at-a-time loop.
Note: the part of the code in question is not entirely redundant, as it
is still used to process any remainder mod 24, as well as any remaining
data when fewer than 200 bytes remain after least one 3072-byte chunk.
crypto: jitter - output full sample from test interface
The Jitter RNG time delta is computed based on the difference of two
high-resolution, 64-bit time stamps. However, the test interface added
in 69f1c387ba only outputs the lower 32 bits of those time stamps. To
ensure all information is available during the evaluation process of
the Jitter RNG, output the full 64-bit time stamps.
Any clients collecting data from the test interface will need to be
updated to take this change into account.
Additionally, the size of the temporary buffer that holds the data for
user space has been clarified. Previously, this buffer was
JENT_TEST_RINGBUFFER_SIZE (= 1000) bytes in size, however that value
represents the number of samples held in the kernel space ring buffer,
with each sample taking 8 (previously 4) bytes.
Rather than increasing the size to allow for all 1000 samples to be
output, we keep it at 1000 bytes, but clarify that this means at most
125 64-bit samples will be output every time this interface is called.
Fabio Estevam [Wed, 9 Oct 2024 12:51:44 +0000 (09:51 -0300)]
dt-bindings: imx-rng: Allow passing only "fsl,imx31-rnga"
On imx31.dtsi the rng compatible string contains "fsl,imx31-rnga" only.
Adjust the binding to accept passing "fsl,imx31-rnga" only.
This fixes the following dt-schema warning:
imx31-lite.dtb: rng@53fb0000: compatible: 'oneOf' conditional failed, one must be fixed:
['fsl,imx31-rnga'] is too short
'fsl,imx21-rnga' was expected
'fsl,imx25-rngb' was expected
'fsl,imx31-rnga' is not one of ['fsl,imx6sl-rngb', 'fsl,imx6sll-rngb', 'fsl,imx6ull-rngb']
'fsl,imx35-rngc' was expected
crypto: drivers - Switch back to struct platform_driver::remove()
After commit 0edb555a65d1 ("platform: Make platform_driver::remove()
return void") .remove() is (again) the right callback to implement for
platform drivers.
Convert all platform drivers below drivers/crypto to use .remove(), with
the eventual goal to drop struct platform_driver::remove_new(). As
.remove() and .remove_new() have the same prototypes, conversion is done
by just changing the structure member name in the driver initializer.
Ahsan Atta [Mon, 7 Oct 2024 13:42:40 +0000 (14:42 +0100)]
crypto: qat - remove faulty arbiter config reset
Resetting the service arbiter config can cause potential issues
related to response ordering and ring flow control check in the
event of AER or device hang. This is because it results in changing
the default response ring size from 32 bytes to 16 bytes. The service
arbiter config reset also disables response ring flow control check.
Thus, by removing this reset we can prevent the service arbiter from
being configured inappropriately, which leads to undesired device
behaviour in the event of errors.
Lukas Wunner [Tue, 8 Oct 2024 09:12:01 +0000 (11:12 +0200)]
crypto: doc - Fix akcipher title reference
Stephen reports a documentation build warning for "make htmldocs"
introduced by recent commit 6b34562f0cfe ("crypto: akcipher - Drop
sign/verify operations").
The commit renamed a paragraph title in a header file, but neglected to
amend the title reference in the API documentation. Fix it.
Dan Carpenter [Sat, 28 Sep 2024 10:05:01 +0000 (13:05 +0300)]
crypto: qat/qat_420xx - fix off by one in uof_get_name()
This is called from uof_get_name_420xx() where "num_objs" is the
ARRAY_SIZE() of fw_objs[]. The > needs to be >= to prevent an out of
bounds access.
Fixes: fcf60f4bcf54 ("crypto: qat - add support for 420xx devices") Signed-off-by: Dan Carpenter <[email protected]> Acked-by: Giovanni Cabiddu <[email protected]> Signed-off-by: Herbert Xu <[email protected]>
Zanussi, Tom [Fri, 27 Sep 2024 18:46:49 +0000 (13:46 -0500)]
crypto: iaa - Remove potential infinite loop in check_completion()
For iaa_crypto operations, it's assumed that if an operation doesn't
make progress, the IAA watchdog timer will kick in and set the
completion status bit to failure and the reason to completion timeout.
Some systems may have broken hardware that doesn't even do that, which
can result in an infinite status-checking loop. Add a check for that
in the loop, and disable the driver if it occurs.
Danny Tsen [Mon, 23 Sep 2024 13:30:39 +0000 (09:30 -0400)]
crypto: powerpc/p10-aes-gcm - Register modules as SIMD
This patch is to fix an issue when simd is not usable that data mismatch
may occur. The fix is to register algs as SIMD modules so that the
algorithm is excecuted when SIMD instructions is usable. Called
gcm_update() to generate the final digest if needed.
A new module rfc4106(gcm(aes)) is also added.
Fixes: cdcecfd9991f ("crypto: p10-aes-gcm - Glue code for AES/GCM stitched implementation") Signed-off-by: Danny Tsen <[email protected]> Signed-off-by: Herbert Xu <[email protected]>
Re-write AES/GCM assembly codes with smaller footprints and
small performance gain. Handling the partial blocks differently that
computes partial block to AES states and re-assembles to a complete
block and then computes a full-block hash.
Added gcm_update() to update the last partial block hash value and
generate the final digest.
Fixes: fd0e9b3e2ee6 ("crypto: p10-aes-gcm - An accelerated AES/GCM stitched implementation") Signed-off-by: Danny Tsen <[email protected]> Signed-off-by: Herbert Xu <[email protected]>
crypto: atmel - Drop explicit initialization of struct i2c_device_id::driver_data to 0
These drivers don't use the driver_data member of struct i2c_device_id,
so don't explicitly initialize this member.
This prepares putting driver_data in an anonymous union which requires
either no initialization or named designators. But it's also a nice
cleanup on its own.
crypto: qat - remove check after debugfs_create_dir()
The debugfs functions are guaranteed to return a valid error code
instead of NULL upon failure. Consequently, the driver can directly
propagate any error returned without additional checks.
Remove the unnecessary `if` statement after debugfs_create_dir(). If
this function fails, the error code is stored in accel_dev->debugfs_dir
and utilized in subsequent debugfs calls.
Additionally, since accel_dev->debugfs_dir is assured to be non-NULL,
remove the superfluous NULL pointer checks within the adf_dbgfs_add()
and adf_dbgfs_rm().
Fixes: 9260db6640a6 ("crypto: qat - move dbgfs init to separate file") Signed-off-by: Giovanni Cabiddu <[email protected]> Signed-off-by: Herbert Xu <[email protected]>
crypto: caam - Fix the pointer passed to caam_qi_shutdown()
The type of the last parameter given to devm_add_action_or_reset() is
"struct caam_drv_private *", but in caam_qi_shutdown(), it is casted to
"struct device *".
Pass the correct parameter to devm_add_action_or_reset() so that the
resources are released as expected.
Fixes: f414de2e2fff ("crypto: caam - use devres to de-initialize QI") Signed-off-by: Christophe JAILLET <[email protected]> Signed-off-by: Herbert Xu <[email protected]>
crypto: hisilicon/hpre - enable all clusters clock gating
Currently, the driver enables clock gating for only one cluster.
However, the new hardware has three clusters. Therefore, clock
gating needs to be enabled based on the number of clusters on the
current hardware.
Tomas Paukrt [Fri, 13 Sep 2024 09:11:43 +0000 (11:11 +0200)]
crypto: mxs-dcp - Fix AES-CBC with hardware-bound keys
Fix passing an initialization vector in the payload field which
is necessary for AES in CBC mode even with hardware-bound keys.
Fixes: 3d16af0b4cfa ("crypto: mxs-dcp: Add support for hardware-bound keys") Signed-off-by: Tomas Paukrt <[email protected]> Signed-off-by: Herbert Xu <[email protected]>
Tang Bin [Wed, 11 Sep 2024 12:17:51 +0000 (20:17 +0800)]
crypto: amlogic - Remove redundant assignment and error messages
In the function meson_crypto_probe, devm_platform_ioremap_resource()
have already contains error message, so remove the
redundant assignment and error messages.
software_key_query() returns the curve size as maximum signature size
for ecrdsa. However it should return twice as much.
It's only the maximum signature size that seems to be off. The maximum
digest size is likewise set to the curve size, but that's correct as it
matches the checks in ecrdsa_set_pub_key() and ecrdsa_verify().
Alternatively to the X9.62 encoding of ecdsa signatures, which uses
ASN.1 and is already supported by the kernel, there's another common
encoding called P1363. It stores r and s as the concatenation of two
big endian, unsigned integers. The name originates from IEEE P1363.
Add a P1363 template in support of the forthcoming SPDM library
(Security Protocol and Data Model) for PCI device authentication.
P1363 is prescribed by SPDM 1.2.1 margin no 44:
"For ECDSA signatures, excluding SM2, in SPDM, the signature shall be
the concatenation of r and s. The size of r shall be the size of
the selected curve. Likewise, the size of s shall be the size of
the selected curve. See BaseAsymAlgo in NEGOTIATE_ALGORITHMS for
the size of r and s. The byte order for r and s shall be in big
endian order. When placing ECDSA signatures into an SPDM signature
field, r shall come first followed by s."
crypto: ecdsa - Move X9.62 signature size calculation into template
software_key_query() returns the maximum signature and digest size for a
given key to user space. When it only supported RSA keys, calculating
those sizes was trivial as they were always equivalent to the key size.
However when ECDSA was added, the function grew somewhat complicated
calculations which take the ASN.1 encoding and curve into account.
This doesn't scale well and adjusting the calculations is easily
forgotten when adding support for new encodings or curves. In fact,
when NIST P521 support was recently added, the function was initially
not amended:
Introduce a ->max_size() callback to struct sig_alg and take advantage
of it to move the signature size calculations to ecdsa-x962.c.
Introduce a ->digest_size() callback to struct sig_alg and move the
maximum ECDSA digest size to ecdsa.c. It is common across ecdsa-x962.c
and the upcoming ecdsa-p1363.c and thus inherited by both of them.
For all other algorithms, continue using the key size as maximum
signature and digest size.
crypto: sig - Rename crypto_sig_maxsize() to crypto_sig_keysize()
crypto_sig_maxsize() is a bit of a misnomer as it doesn't return the
maximum signature size, but rather the key size.
Rename it as well as all implementations of the ->max_size callback.
A subsequent commit introduces a crypto_sig_maxsize() function which
returns the actual maximum signature size.
While at it, change the return type of crypto_sig_keysize() from int to
unsigned int for consistency with crypto_akcipher_maxsize(). None of
the callers checks for a negative return value and an error condition
can always be indicated by returning zero.
crypto: ecdsa - Move X9.62 signature decoding into template
Unlike the rsa driver, which separates signature decoding and
signature verification into two steps, the ecdsa driver does both in one.
This restricts users to the one signature format currently supported
(X9.62) and prevents addition of others such as P1363, which is needed
by the forthcoming SPDM library (Security Protocol and Data Model) for
PCI device authentication.
Per Herbert's suggestion, change ecdsa to use a "raw" signature encoding
and then implement X9.62 and P1363 as templates which convert their
respective encodings to the raw one. One may then specify
"x962(ecdsa-nist-XXX)" or "p1363(ecdsa-nist-XXX)" to pick the encoding.
The present commit moves X9.62 decoding to a template. A separate
commit is going to introduce another template for P1363 decoding.
The ecdsa driver internally represents a signature as two u64 arrays of
size ECC_MAX_BYTES. This appears to be the most natural choice for the
raw format as it can directly be used for verification without having to
further decode signature data or copy it around.
Repurpose all the existing test vectors for "x962(ecdsa-nist-XXX)" and
create a duplicate of them to test the raw encoding.
crypto: ecdsa - Avoid signed integer overflow on signature decoding
When extracting a signature component r or s from an ASN.1-encoded
integer, ecdsa_get_signature_rs() subtracts the expected length
"bufsize" from the ASN.1 length "vlen" (both of unsigned type size_t)
and stores the result in "diff" (of signed type ssize_t).
This results in a signed integer overflow if vlen > SSIZE_MAX + bufsize.
The kernel is compiled with -fno-strict-overflow, which implies -fwrapv,
meaning signed integer overflow is not undefined behavior. And the
function does check for overflow:
if (-diff >= bufsize)
return -EINVAL;
So the code is fine in principle but not very obvious. In the future it
might trigger a false-positive with CONFIG_UBSAN_SIGNED_WRAP=y.
Avoid by comparing the two unsigned variables directly and erroring out
if "vlen" is too large.
ASN.1: Clean up include statements in public headers
If <linux/asn1_decoder.h> is the first header included from a .c file
(due to headers being sorted alphabetically), the compiler complains:
include/linux/asn1_decoder.h:18:29: error: unknown type name 'size_t'
Avoid by including <linux/types.h>.
Jonathan notes that the counterpart <linux/asn1_encoder.h> already
includes <linux/types.h>, but additionally includes the unnecessary
<linux/bug.h>. Drop it.
crypto: sig - Move crypto_sig_*() API calls to include file
The crypto_sig_*() API calls lived in sig.c so far because they needed
access to struct crypto_sig_type: This was necessary to differentiate
between signature algorithms that had already been migrated from
crypto_akcipher to crypto_sig and those that hadn't yet.
Now that all algorithms have been migrated, the API calls can become
static inlines in <crypto/sig.h> to mimic what <crypto/akcipher.h> is
doing.
A sig_alg backend has just been introduced and all asymmetric
sign/verify algorithms have been migrated to it.
The sign/verify operations can thus be dropped from akcipher_alg.
It is now purely for asymmetric encrypt/decrypt.
Move struct crypto_akcipher_sync_data from internal.h to akcipher.c and
unexport crypto_akcipher_sync_{prep,post}(): They're no longer used by
sig.c but only locally in akcipher.c.
In crypto_akcipher_sync_{prep,post}(), drop various NULL pointer checks
for data->dst as they were only necessary for the verify operation.
In the crypto_sig_*() API calls, remove the forks that were necessary
while algorithms were converted from crypto_akcipher to crypto_sig
one by one.
In struct akcipher_testvec, remove the "params", "param_len" and "algo"
elements as they were only needed for the ecrdsa verify operation.
Remove corresponding dead code from test_akcipher_one() as well.
The drivers aspeed-acry.c, hpre_crypto.c and jh7110-rsa.c purport to
implement sign/verify operations for raw (unpadded) "rsa".
But there is no such thing as message digests generally need to be
padded according to a predefined scheme (such as PSS or PKCS#1) to
match the size of the usually much larger RSA keys.
The bogus sign/verify operations defined by these drivers are never
called but block removal of sign/verify from akcipher_alg. Drop them.
The virtio crypto driver exposes akcipher sign/verify operations in a
user space ABI. This blocks removal of sign/verify from akcipher_alg.
Herbert opines:
"I would say that this is something that we can break. Breaking it
is no different to running virtio on a host that does not support
these algorithms. After all, a software implementation must always
be present.
I deliberately left akcipher out of crypto_user because the API
is still in flux. We should not let virtio constrain ourselves."
https://lore.kernel.org/all/[email protected]/
"I would remove virtio akcipher support in its entirety. This API
was never meant to be exposed outside of the kernel."
https://lore.kernel.org/all/[email protected]/
Drop sign/verify support from virtio crypto. There's no strong reason
to also remove encrypt/decrypt support, so keep it.
A key selling point of virtio crypto is to allow guest access to crypto
accelerators on the host. So far the only akcipher algorithm supported
by virtio crypto is RSA. Dropping sign/verify merely means that the
PKCS#1 padding is now always generated or verified inside the guest,
but the actual signature generation/verification (which is an RSA
decrypt/encrypt operation) may still use an accelerator on the host.
Generating or verifying the PKCS#1 padding is cheap, so a hardware
accelerator won't be of much help there. Which begs the question
whether virtio crypto support for sign/verify makes sense at all.
It would make sense for the sign operation if the host has a security
chip to store asymmetric private keys. But the kernel doesn't even
have an asymmetric_key_subtype yet for hardware-based private keys.
There's at least one rudimentary driver for such chips (atmel-ecc.c for
ATECC508A), but it doesn't implement the sign operation. The kernel
would first have to grow support for a hardware asymmetric_key_subtype
and at least one driver implementing the sign operation before exposure
to guests via virtio makes sense.
When constructing the EMSA-PKCS1-v1_5 padding for the sign operation,
a buffer for the padding is allocated and the Full Hash Prefix is copied
into it. The padding is then passed to the RSA decrypt operation as an
sglist entry which is succeeded by a second sglist entry for the hash.
Actually copying the hash prefix around is completely unnecessary.
It can simply be referenced from a third sglist entry which sits
in-between the padding and the digest.
The RSASSA-PKCS1-v1_5 sign operation currently only checks that the
digest length is less than "key_size - hash_prefix->size - 11".
The verify operation merely checks that it's more than zero.
Actually the precise digest length is known because the hash algorithm
is specified upon instance creation and the digest length is encoded
into the final byte of the hash algorithm's Full Hash Prefix.
So check for the exact digest length rather than solely relying on
imprecise maximum/minimum checks.
Keep the maximum length check for the sign operation as a safety net,
but drop the now unnecessary minimum check for the verify operation.
A sig_alg backend has just been introduced with the intent of moving all
asymmetric sign/verify algorithms to it one by one.
Migrate the sign/verify operations from rsa-pkcs1pad.c to a separate
rsassa-pkcs1.c which uses the new backend.
Consequently there are now two templates which build on the "rsa"
akcipher_alg:
* The existing "pkcs1pad" template, which is instantiated as an
akcipher_instance and retains the encrypt/decrypt operations of
RSAES-PKCS1-v1_5 (RFC 8017 sec 7.2).
* The new "pkcs1" template, which is instantiated as a sig_instance
and contains the sign/verify operations of RSASSA-PKCS1-v1_5
(RFC 8017 sec 8.2).
In a separate step, rsa-pkcs1pad.c could optionally be renamed to
rsaes-pkcs1.c for clarity. Additional "oaep" and "pss" templates
could be added for RSAES-OAEP and RSASSA-PSS.
Note that it's currently allowed to allocate a "pkcs1pad(rsa)" transform
without specifying a hash algorithm. That makes sense if the transform
is only used for encrypt/decrypt and continues to be supported. But for
sign/verify, such transforms previously did not insert the Full Hash
Prefix into the padding. The resulting message encoding was incompliant
with EMSA-PKCS1-v1_5 (RFC 8017 sec 9.2) and therefore nonsensical.
From here on in, it is no longer allowed to allocate a transform without
specifying a hash algorithm if the transform is used for sign/verify
operations. This simplifies the code because the insertion of the Full
Hash Prefix is no longer optional, so various "if (digest_info)" clauses
can be removed.
There has been a previous attempt to forbid transform allocation without
specifying a hash algorithm, namely by commit c0d20d22e0ad ("crypto:
rsa-pkcs1pad - Require hash to be present"). It had to be rolled back
with commit b3a8c8a5ebb5 ("crypto: rsa-pkcs1pad: Allow hash to be
optional [ver #2]"), presumably because it broke allocation of a
transform which was solely used for encrypt/decrypt, not sign/verify.
Avoid such breakage by allowing transform allocation for encrypt/decrypt
with and without specifying a hash algorithm (and simply ignoring the
hash algorithm in the former case).
So again, specifying a hash algorithm is now mandatory for sign/verify,
but optional and ignored for encrypt/decrypt.
The new sig_alg API uses kernel buffers instead of sglists, which
avoids the overhead of copying signature and digest from sglists back
into kernel buffers. rsassa-pkcs1.c is thus simplified quite a bit.
sig_alg is always synchronous, whereas the underlying "rsa" akcipher_alg
may be asynchronous. So await the result of the akcipher_alg, similar
to crypto_akcipher_sync_{en,de}crypt().
As part of the migration, rename "rsa_digest_info" to "hash_prefix" to
adhere to the spec language in RFC 9580. Otherwise keep the code
unmodified wherever possible to ease reviewing and bisecting. Leave
several simplification and hardening opportunities to separate commits.
rsassa-pkcs1.c uses modern __free() syntax for allocation of buffers
which need to be freed by kfree_sensitive(), hence a DEFINE_FREE()
clause for kfree_sensitive() is introduced herein as a byproduct.
pkcs1pad_set_pub_key() and pkcs1pad_set_priv_key() are almost identical.
The upcoming migration of sign/verify operations from rsa-pkcs1pad.c
into a separate crypto_template will require another copy of the exact
same functions. When RSASSA-PSS and RSAES-OAEP are introduced, each
will need yet another copy.
Deduplicate the functions into a single one which lives in a common
header file for reuse by RSASSA-PKCS1-v1_5, RSASSA-PSS and RSAES-OAEP.
A sig_alg backend has just been introduced with the intent of moving all
asymmetric sign/verify algorithms to it one by one.
Migrate ecrdsa.c to the new backend.
One benefit of the new API is the use of kernel buffers instead of
sglists, which avoids the overhead of copying signature and digest
sglists back into kernel buffers. ecrdsa.c is thus simplified quite
a bit.
projects / linux.git / log