Implement and integrate new Incremental Merkle Tree
This supersedes #823.
----
This is an implementation of a new incremental merkle tree with
* no memory safety issues
* a more sensible internal design
* better space efficiency (tree representation, witnessing)
* simpler API
It is intended that this tracks the behavior of the previous tree, which it does, as verified by tests. I even wrote a little circuit for testing that all the paths work.
This PR also integrates the tree into the codebase and deprecates the old tree in almost all of our code. (I left it alone in `zerocashTest` but everything else has been changed.)
This change is compatible with the testnet but you will need to clear your *local* blockchain data out since the serialized representation of the merkle tree is now different.
Closes#517, Closes#519, Closes#591, Closes#460, Closes#473
This is a new implementation of the incremental merkle tree used by our
scheme to witness commitments to spendable value. It serves as a fixed-sized
accumulator.
This new construction has a much simpler API surface area, avoids memory
safety issues, remains pruned at all times, avoids serialization edge cases,
has more efficient insertion, and is abstract over the depth and hash
function used at the type level.
Further, it lays the groundwork for efficient "fast-forwarding" of witnesses
into the tree as the treestate is updated.
All blocks before the first subsidy halving block, with the exception of
the genesis block, must contain an output which sends 20% of the block
subsidy value to a scriptPubKey `FOUNDERS_REWARD_SCRIPT`.
Changing the order of difficulty calculation operations to divide first doesn't
affect the result significantly, but ensures we never overflow the arith_uint256
during multiplication and get an artificial jump in difficulty.
Changes to bloom tests were done by running the following commands:
sed -i 's/\(CDataStream stream(ParseHex(".\{152\}\)\(.\{8\}\)/\100000000000000000000000000000000000000000000000000000000\200/' src/test/bloom_tests.cpp
sed -i 's/\(vector<unsigned char> vch = ParseHex(".\{152\}\)\(.\{8\}\)/\100000000000000000000000000000000000000000000000000000000\200/' src/test/bloom_tests.cpp
and then reverting the single change to the transaction line.
The main and test networks are configured to use parameters that are currently
low-memory but usable with the basic solver; they will be increased once the
solver is optimised. The regtest network is configured to have extremely low
memory usage for speed.
Note that Bitcoin's double-hasher is used for the difficulty check. This does
not match the paper, but is simpler than changing the block header
serialization. Single hashing is kept for the EquiHash solver because there is
no requirement on execution time there, only on memory usage.
* PourTxs cannot appear in coinbase transactions.
* Transactions can only contain empty vin/vouts if they contain a PourTx.
* PourTx public values must be well-formed (not negative or too large).
* Transactions cannot have the same serial twice throughout all PourTxs.
Transactions of version 2 and above contain a `vpour` field which is a vector of `CPourTx`
objects that embody our protocol. We introduce serialization primitives for boost::array
(we intend for changing the amount of inputs and outputs in the circuit to be simple).
SIGHASH_* operations hash this field like any other for now.
af6edac *: alias -h for --help (Daniel Cousens)
131d7f9 Change URLs to https in debian/control (Matt Corallo)
7ce2c91 Update debian/changelog and slight tweak to debian/control (Matt Corallo)
4fbfebe Correct spelling mistakes in doc folder (Mitchell Cash)
e42bf16 Clarification of unit test build instructions. (Eric Lombrozo)
54f9dee Update bluematt-key, the old one is long-since revoked (Matt Corallo)
bfc6154 [Trivial] Fixed typo when referring to a previous section in depends/README.md [skip ci] (Chris Kleeschulte)
9e45157 build: disable -Wself-assign (Wladimir J. van der Laan)
33d6825 Bugfix: Allow mining on top of old tip blocks for testnet (fixes testnet-in-a-box use case) (Luke Dashjr)
87a797a build: Remove dependency of bitcoin-cli on secp256k1 (Wladimir J. van der Laan)
a33cd5b [trivial] Fix rpc message "help generate" (MarcoFalke)
6fd0019 Drop "with minimal dependencies" from description (Zak Wilcox)
2394f4d Split bitcoin-tx into its own package (Zak Wilcox)
1e672ae Include bitcoin-tx binary on Debian/Ubuntu (Zak Wilcox)
b3eaa30 [Qt] Raise debug window when requested (MarcoFalke)
01878c9 Fix locking in GetTransaction. (Alex Morcos)
9b9acc2 Fix spelling of Qt (Diego Viola)
Transactions are not allowed in the memory pool or selected for inclusion in a block until their lock times exceed chainActive.Tip()->GetMedianTimePast(). However blocks including transactions which are only mature under the old rules are still accepted; this is *not* the soft-fork required to actually rely on the new constraint in production.
6af25b0 Add BIP65 to getblockchaininfo softforks list (Peter Todd)
ba1da90 Show softfork status in getblockchaininfo (Wladimir J. van der Laan)
70a427b CLTV: Add more tests to improve coverage (Esteban Ordano)
c5a27f4 Add RPC tests for the CHECKLOCKTIMEVERIFY (BIP65) soft-fork (Peter Todd)
5e82e1c Add CHECKLOCKTIMEVERIFY (BIP65) soft-fork logic (Peter Todd)
6ea5ca4 Enable CHECKLOCKTIMEVERIFY as a standard script verify flag (Peter Todd)
4fa7a04 Replace NOP2 with CHECKLOCKTIMEVERIFY (BIP65) (Peter Todd)
6ec08db Move LOCKTIME_THRESHOLD to src/script/script.h (Peter Todd)
684636b Make CScriptNum() take nMaxNumSize as an argument (Peter Todd)
To bridge the time until a dynamic method for determining this fee is
merged.
This is especially aimed at the stable releases (0.10, 0.11) because
full mempool limiting, as will be in 0.12, is too invasive and risky to
backport.
Github-Pull: #6793
Rebased-From: 28e3249e53b8ef7516636df0f1406466a513095d 4e2efb3c5fde4b1e332cc032e3dc4082ec4e3cac
Four cases included:
* The CLTV operand type mismatches the tx locktime. In the script it is
1 (interpreted as block height), but in the tx is 500000000
(interpreted as date)
* The stack is empty when executing OP_CLTV
* The tx is final by having only one input with MAX_INT sequence number
* The operand for CLTV is negative (after OP_0 OP_1 OP_SUB)
Rebased-From: cb54d17355864fa08826d6511a0d7692b21ef2c9
<nLockTime> CHECKLOCKTIMEVERIFY -> <nLockTime>
Fails if tx.nLockTime < nLockTime, allowing the funds in a txout to be
locked until some block height or block time in the future is reached.
Only the logic and unittests are implemented; this commit does not have
any actual soft-fork logic in it.
Thanks to Pieter Wuille for rebase.
Credit goes to Gregory Maxwell for the suggestion of comparing the
argument against the transaction nLockTime rather than the current
time/blockheight directly.
Rebased-From: bc60b2b4b401f0adff5b8b9678903ff8feb5867b
While the existing numeric opcodes are all limited to 4-byte bignum
arguments, new opcodes will need different limits.
Rebased-From: 99088d60d8a7747c6d1a7fd5d8cd388be1b3e138
