Gregory Maxwell [Wed, 29 May 2019 10:35:10 +0000 (10:35 +0000)]
Merge #578: Avoid implementation-defined and undefined behavior when dealing with sizes
14c7dbd Simplify control flow in DER parsing (Tim Ruffing) ec8f20b Avoid out-of-bound pointers and integer overflows in size comparisons (Tim Ruffing) 01ee1b3 Parse DER-enconded length into a size_t instead of an int (Tim Ruffing) 3cb057f Fix possible integer overflow in DER parsing (Tim Ruffing)
Pull request description:
This is a result of auditing the code for overflow issues at random places. None of this is critical but I think all of it should be fixed.
I know this touches "red" code. I double-checked and triple-checked this but I can understand if some of the changes are not desirable because they change well-tested code.
Gregory Maxwell [Mon, 27 May 2019 07:30:33 +0000 (07:30 +0000)]
Merge #595: Allow to use external default callbacks
e49f799 Add missing #(un)defines to base-config.h (Tim Ruffing) 77defd2 Add secp256k1_ prefix to default callback functions (Tim Ruffing) 908bdce Include stdio.h and stdlib.h explicitly in secp256k1.c (Tim Ruffing) 5db782e Allow usage of external default callbacks (Tim Ruffing) 6095a86 Replace CHECKs for no_precomp ctx by ARG_CHECKs without a return (Tim Ruffing)
Pull request description:
This is intended for environments without implementations for `abort()`, `fprintf()`, and `stderr`. e.g., embedded systems. Those can provide their own implementations of `default_illegal_callback_fn` and `default_error_callback_fn` at compile time.
If you want to use your own default callback, things will be somewhat inconsistent unfortunately: We cannot make the callback data `extern` too, because then the initialization lists for `default_illegal_callback` won't contain only constants. (`const` variables are not compile-time constants). So you cannot take callback data in your own default callback function.
As a more drastic/breaking alternative I suggest to remove the callback data entirely. I don't think it's a big loss and I would be surprised if anyone uses it. Additionally, we could even remove the possibility to set the callback function at runtime after this PR. This will simplify things a lot, and again I don't think it's a big loss.
Note that `abort()`, `fprintf()`, and `stderr` are also used in `CHECK`, which is still used in production code if we rely on gmp for scalar and field inversions (e.g., https://github.com/bitcoin-core/secp256k1/blob/master/src/scalar_impl.h#L240). This is not an issue for embedded system which probably don't want to use gmp anyway, but it is probably an issue for the reasons explained in https://github.com/bitcoin-core/secp256k1/pull/566#issuecomment-469111901.
Gregory Maxwell [Sun, 26 May 2019 07:37:54 +0000 (07:37 +0000)]
Merge #600: scratch space: use single allocation
98836b1 scratch: replace frames with "checkpoint" system (Andrew Poelstra) 7623cf2 scratch: save a couple bytes of unnecessarily-allocated memory (Andrew Poelstra) a7a164f scratch: rename `max_size` to `size`, document that extra will actually be allocated (Andrew Poelstra) 5a4bc0b scratch: unify allocations (Andrew Poelstra) c2b028a scratch space: thread `error_callback` into all scratch space functions (Andrew Poelstra) 0be1a4a scratch: add magic bytes to beginning of structure (Andrew Poelstra) 92a48a7 scratch space: use single allocation (Andrew Poelstra)
Gregory Maxwell [Sat, 25 May 2019 22:34:47 +0000 (22:34 +0000)]
Merge #592: Use trivial algorithm in ecmult_multi if scratch space is small
9ab96f7 Use trivial algorithm in ecmult_multi if scratch space is small (Jonas Nick)
Pull request description:
`ecmult_multi` already selects the trivial algorithm if the scratch space is NULL. With this PR the trivial algorithm is also selected if the scratch space is too small to use pippenger or strauss instead of returning 0. That makes it more easier to avoid consensus relevant inconsistencies just because scratch space construction was messed up.
ACKs for commit 9ab96f:
real-or-random:
utACK 9ab96f7
Gregory Maxwell [Sat, 25 May 2019 21:16:07 +0000 (21:16 +0000)]
Merge #566: Enable context creation in preallocated memory
0522caa Explain caller's obligations for preallocated memory (Tim Ruffing) 238305f Move _preallocated functions to separate header (Tim Ruffing) 695feb6 Export _preallocated functions (Tim Ruffing) 814cc78 Add tests for contexts in preallocated memory (Tim Ruffing) ba12dd0 Check arguments of _preallocated functions (Tim Ruffing) 5feadde Support cloning a context into preallocated memory (Tim Ruffing) c4fd5da Switch to a single malloc call (Tim Ruffing) ef020de Add size constants for preallocated memory (Tim Ruffing) 1bf7c05 Prepare for manual memory management in preallocated memory (Tim Ruffing)
Pull request description:
@apoelstra
This builds on #557.
Manually managing memory is always a pain in the ass in some way. I tried to keep the pain manageable. I'm open to suggestions to make this less ugly or error-prone.
Gregory Maxwell [Sat, 25 May 2019 10:15:59 +0000 (10:15 +0000)]
Merge #596: Make WINDOW_G configurable
a61a93f Clean up ./configure help strings (Tim Ruffing) 2842dc5 Make WINDOW_G configurable (Tim Ruffing)
Pull request description:
This makes WINDOW_G a configurable value in the range of [2..24].
The upper limit of 24 is a defensive choice. The code is probably
correct for values up to 33 but those larger values yield in huge
tables (>= 256MiB), which are i) unlikely to be really beneficial
in practice and ii) increasingly difficult to test.
The main point of this is not to make the window size configurable (using ./configure) but rather to use an external #define for the window size, which makes it configurable for embedded system that rely on their own build system (like in #595).
Tim Ruffing [Wed, 6 Mar 2019 12:12:33 +0000 (13:12 +0100)]
Make WINDOW_G configurable
This makes WINDOW_G a configurable value in the range of [2..24].
The upper limit of 24 is a defensive choice. The code is probably
correct for values up to 27 but those larger values yield in huge
tables (>= 256MiB), which are i) unlikely to be really beneficial
in practice and ii) increasingly difficult to test.
Tim Ruffing [Wed, 7 Nov 2018 15:17:57 +0000 (16:17 +0100)]
Avoid out-of-bound pointers and integer overflows in size comparisons
This changes pointer calculations in size comparions to a form that
ensures that no out-of-bound pointers are computed, because even their
computation yields undefined behavior.
Also, this changes size comparions to a form that ensures that neither
the left-hand side nor the right-hand side can overflow.
Gregory Maxwell [Thu, 23 May 2019 00:36:27 +0000 (00:36 +0000)]
Merge #561: Respect LDFLAGS and #undef STATIC_PRECOMPUTATION if using basic config
dbed75d Undefine `STATIC_PRECOMPUTATION` if using the basic config (DesWurstes) 310111e Keep LDFLAGS if `--coverage` (DesWurstes)
Pull request description:
Update: **This is a trimmed pull request with strong rationale.**
- Adding `--coverage` shouldn't reset `LDFLAGS`, this is definitely a typo
- The basic configuration should undefine `STATIC_PRECOMPUTATION`, as generating it is not supported and it complicates #549
This fix install all the headers under include/ into
/usr/local/include. The fix solves problems that arise
when building libraries that depend on secp256k1 such
as libbitcoin-system which require all the headers
Gregory Maxwell [Wed, 22 May 2019 04:43:53 +0000 (04:43 +0000)]
Merge #533: Make sure we're not using an uninitialized variable in secp256k1_wnaf_const(...)
248f046 Make sure we're not using an uninitialized variable in secp256k1_wnaf_const(...) (practicalswift)
Pull request description:
Make sure we're not using an uninitialized variable in `secp256k1_wnaf_const(...)`:
```
In file included from src/secp256k1.c:15:0,
from src/tests.c:17:
src/ecmult_const_impl.h: In function ‘secp256k1_wnaf_const’:
src/ecmult_const_impl.h:117:20: warning: ‘u’ may be used uninitialized in this function [-Wmaybe-uninitialized]
wnaf[word] = u * global_sign;
^
```
**Note to reviewers:** Perhaps an `assert(…);` is a bit drastic. What would be a more graceful way to handle this? :-)
This fix install all the headers under include/ into
/usr/local/include. The fix solves problems that arise
when building libraries that depend on secp256k1 such
as bitcoin-system which require all the headers
Gregory Maxwell [Thu, 9 May 2019 22:23:41 +0000 (22:23 +0000)]
Merge #612: Allow field_10x26_arm.s to compile for ARMv7 architecture
d4d270a Allow field_10x26_arm.s to compile for ARMv7 architecture (Roman Zeyde)
Pull request description:
It would allow using optimized field operations on the TREZOR device, which is using ARMv7 Cortex-M4.
Following https://github.com/trezor/trezor-core/pull/500 and part of https://github.com/trezor/trezor-firmware/issues/66.
Gregory Maxwell [Mon, 25 Feb 2019 21:00:08 +0000 (21:00 +0000)]
Merge #568: Fix integer overflow in ecmult_multi_var when n is large
2277af5 Fix integer overflow in ecmult_multi_var when n is large (Jonas Nick)
Pull request description:
Without this PR ecmult_multi could return wrong results. If the number of points `n` is large enough then some or all multiplications could be skipped or the function could end up in an infinite loop. This PR adds two checks to prevent `n` from wrapping around.
Gregory Maxwell [Sun, 24 Feb 2019 03:01:31 +0000 (03:01 +0000)]
Merge #580: Add trivial ecmult_multi algorithm which does not require a scratch space
a697d82 Add trivial ecmult_multi to the benchmark tool (Jonas Nick) bade617 Add trivial ecmult_multi algorithm. It is selected when no scratch space is given and just multiplies and adds the points. (Jonas Nick)
Pull request description:
This commit adds a new ecmult_multi algorithm that is automatically selected when `ecmult_multi_var` is called with scratch space set to `NULL`. This is a trivial algorithm that simply multiplies the points with the corresponding scalars and adds them up.
The use case is to allow creating exposed function that uses `ecmult_multi` but without requiring a scratch space argument. For example, in MuSig when computing the combined public key we need to compute a weighted sum of points but we most likely don't care about the performance. And if we do we can still provide a scratch space. Having the option of not providing a scratch space is useful because creating a scratch space is not entirely trivial. One needs to decide on a size and it needs to be destroyed.
Gregory Maxwell [Fri, 22 Feb 2019 01:28:04 +0000 (01:28 +0000)]
Merge #584: configure: Use CFLAGS_FOR_BUILD when checking native compiler
a34bcaa Actually pass CFLAGS_FOR_BUILD and LDFLAGS_FOR_BUILD to linker (Tim Ruffing) 2d5f4ce configure: Use CFLAGS_FOR_BUILD when checking native compiler (Tim Ruffing)
Pull request description:
This fixes a bug where configure would fail or disable static
ecmult tables because it wrongly checks the native compiler using
the target CFLAGS (instead of the native CFLAGS_FOR_BUILD).
Moreover, this commit adds tests to figure out whether the native
compiler supports the warning flags passed during the build, and it
contains a few minor improvements to the code that checks the native
compiler.
Gregory Maxwell [Thu, 21 Feb 2019 11:42:08 +0000 (11:42 +0000)]
Merge #516: improvements to random seed in src/tests.c
be40c4d Fixup for C90 mixed declarations. (Gregory Maxwell) 8b3841c fix bug in fread() failure check (Don Viszneki) cddef0c tests: add warning message when /dev/urandom fails (Don Viszneki)
Pull request description:
I've made two small changes to `src/tests.c` circa random seed generation.
Added a warning when `/dev/urandom` fails, mostly to defend against the case that someone should use the code verbatim, but also to enhance its illustrative power.
Also I fixed a bug with how the return value of `fread()` was being evaluated. In fact, `/dev/urandom` was never being applied before as the check on the return value of `fread()` always failed!
Gregory Maxwell [Thu, 21 Feb 2019 04:31:26 +0000 (04:31 +0000)]
Merge #587: Make randomization of a non-signing context a noop
6198375 Make randomization of a non-signing context a noop (Tim Ruffing)
Pull request description:
Before this commit secp256k1_context_randomize called illegal_callback
when called on a context not initialized for signing. This is not
documented. Moreover, it is not desirable because non-signing contexts
may use randomization in the future.
This commit makes secp256k1_context_randomize a noop in this case. This
is safe because the context cannot be used for signing anyway.
This fixes #573 and it fixes rust-bitcoin/rust-secp256k1#82.
Gregory Maxwell [Thu, 21 Feb 2019 04:17:54 +0000 (04:17 +0000)]
Merge #539: Assorted minor corrections
52ab96f clean dependendies in field_*_impl.h (Russell O'Connor) deff5ed Correct math typos in field_*.h (Russell O'Connor) 4efb3f8 Add check that restrict pointers don't alias with all parameters. (Russell O'Connor)
Pull request description:
* add more checks for restrict pointers.
* correct math typos.
* refine dependencies on "num.h"
9bd89c8 Optimize secp256k1_fe_normalize_weak calls. Move secp256k1_fe_normalize_weak calls out of ECMULT_TABLE_GET_GE and ECMULT_TABLE_GET_GE_STORAGE and into secp256k1_ge_globalz_set_table_gej instead. (Russell O'Connor)
Pull request description:
Move secp256k1_fe_normalize_weak calls out of ECMULT_TABLE_GET_GE and ECMULT_TABLE_GET_GE_STORAGE and into secp256k1_ge_globalz_set_table_gej instead.
Tim Ruffing [Wed, 16 Jan 2019 16:12:38 +0000 (17:12 +0100)]
configure: Use CFLAGS_FOR_BUILD when checking native compiler
This fixes a bug where configure would fail or disable static
ecmult tables because it wrongly checks the native compiler using
the target CFLAGS (instead of the native CFLAGS_FOR_BUILD), and
similar for CPPFLAGS and LDFLAGS.
Moreover, this commit adds tests to figure out whether the native
compiler supports the warning flags passed during the build, and it
contains a few minor improvements to the code that checks the native
compiler.
Tim Ruffing [Sun, 27 Jan 2019 12:17:37 +0000 (13:17 +0100)]
Make randomization of a non-signing context a noop
Before this commit secp256k1_context_randomize called illegal_callback
when called on a context not initialized for signing. This is not
documented. Moreover, it is not desirable because non-signing contexts
may use randomization in the future.
This commit makes secp256k1_context_randomize a noop in this case. This
is safe because the context cannot be used for signing anyway.
This fixes #573 and it fixes rust-bitcoin/rust-secp256k1#82.
Tim Ruffing [Thu, 1 Nov 2018 11:15:28 +0000 (12:15 +0100)]
Fix possible integer overflow in DER parsing
If we’re in the last loop iteration, then `lenleft == 1` and it could
be the case that `ret == MAX_SIZE`, and so `ret + lenleft` will
overflow to 0 and the sanity check will not catch it. Then we will
return `(int) MAX_SIZE`, which should be avoided because this value is
implementation-defined. (However, this is harmless because
`(int) MAX_SIZE == -1` on all supported platforms.)
Pieter Wuille [Mon, 26 Nov 2018 17:12:55 +0000 (09:12 -0800)]
Merge #557: Eliminate scratch memory used when generating contexts
b3bf5f9 ecmult_impl: expand comment to explain how effective affine interacts with everything (Andrew Poelstra) efa783f Store z-ratios in the 'x' coord they'll recover (Peter Dettman) ffd3b34 add `secp256k1_ge_set_all_gej_var` test which deals with many infinite points (Andrew Poelstra) 84740ac ecmult_impl: save one fe_inv_var (Andrew Poelstra) 4704527 ecmult_impl: eliminate scratch memory used when generating context (Andrew Poelstra) 7f7a2ed ecmult_gen_impl: eliminate scratch memory used when generating context (Andrew Poelstra)
Pieter Wuille [Tue, 6 Nov 2018 02:23:52 +0000 (18:23 -0800)]
Merge #553: add static context object which has no capabilities
40fde61 prevent attempts to modify `secp256k1_context_no_precomp` (Andrew Poelstra) ed7c084 add static context object which has no capabilities (Andrew Poelstra)
Pieter Wuille [Wed, 17 Oct 2018 19:12:31 +0000 (12:12 -0700)]
Merge #354: [ECDH API change] Support custom hash function
c8fbc3c [ECDH API change] Allow pass arbitrary data to hash function (Kirill Fomichev) b00be65 [ECDH API change] Support custom hash function (Kirill Fomichev)
Russell O'Connor [Mon, 13 Aug 2018 03:47:38 +0000 (23:47 -0400)]
Optimize secp256k1_fe_normalize_weak calls.
Move secp256k1_fe_normalize_weak calls out of ECMULT_TABLE_GET_GE and ECMULT_TABLE_GET_GE_STORAGE and into secp256k1_ge_globalz_set_table_gej instead.