RPC calls to manage zkey creation, management and backup
Rebased on PR #1205 for keystore functionality.
For #1197, implements:
z_getnewaddress
z_importkey
z_exportkey
For #1198, spending key support added to:
z_exportwallet
z_importwallet
Example dump file containing zkeys:
https://gist.github.com/bitcartel/d7013b30b19419c6a550bc5d6ff6cc47
Also adds:
z_listaddresses
Tests in:
gtest/test_wallet_zkeys
test/rpc_wallet_tests
Fix formatting of first paragraph of payment API document
The description of taddr and zaddr were bunched up one line. This PR fixes the issue.
Self-ACK. Please merge.
Fix inconsistent -O1/-O2, fix libzcash flags, add -fwrapv -fno-strict-aliasing
Closes#1168. In that ticket I decided the optimization flags for dependencies are out of scope, i.e. we go with whatever the upstream package maintainer chose.
Implement zkSNARK compression
Implement zkSNARK compression.
The serialization format of zkSNARKs has now been specified in our protocol specification. G1/G2 elements are now compressed into 33 byte and 65 byte formats, repsectively. `MONTGOMERY_OUTPUT` is enabled everywhere to make proving/verifying keys load faster and be consistent with the MPC.
Proofs are now 296 bytes. This PR also updates the proving and verifying keys.
Closes#1103. Closes#471.
See also https://github.com/zcash/libsnark/pull/3 and https://github.com/zcash/libsnark/pull/4
Improve speed and accuracy of zcbenchmark validatelargetx
The verification test, in a loop, passes `spending_tx` (a `CMutableTransaction`) to the constructor of `MutableTransactionSignatureChecker`, which immediately uses it to create a non-mutable `CTransaction` object, which is used for the actual verification process.
Since `spending_tx` is not mutated during the verification loop & process, we can instead convert it to a `CTransaction` just once, and use it with `TransactionSignatureChecker`.
This removes the time to create `CTransaction` objects from the benchmark itself.
Results show an improvement in running time to complete the verification phase of the test and consistent times across z7 and z8 releases.
```
Benchmarks on i3 processor:
z7 old 228.67205900 --> z7 new 49.27225200
z7 old 229.90048900 --> z7 new 48.38650700
z8 old 295.77963800 --> z8 new 48.37695100
z8 old 294.32640100 --> z8 new 49.93216100
```
Add gtest for non-malleable txids.
1. Test txid of normal transaction is double sha256 with sigscript input data removed
2. Test txid of coinbase transaction is double sha256 over whole tx
You can run a single test with:
./zcash-gtest --gtest_filter="txid_tests*"
A fix for transaction malleability
This PR fixes transaction malleability by not including the sigscript of transaction inputs and joinsplit sigs when hashing the txid.
This PR supercedes PR #1101 which was a minimal solution based on a new serialization flag.
This PR introduces GetTxid() to distinguish between getting a transaction id and the double sha256 hash.
The key changes are:
- Adding GetTxid() method to CTransaction which makes a copy of the transaction, clearing out the sigscript and joinsplitsig fields, before hashing.
- Verifying that every call to GetHash() actually wants a txid, and replacing with GetTxid().
- Renaming GetHash() to GetSerializeHash()
- Rationale: In future, upstream code we want to merge will use GetHash() but we don't know the intent. We should check to see if the intent is to receive a txid (most likely) in which case we replace with GetTxid(), or if upstream actually wants a double hash of the transaction we can use GetSerializeHash().
- Updated genesis data in chainparams.cpp
Note that coinbase transactions are excluded as they need the sigscript hashed to help avoid duplicate txids per BIP34:
- This modification is related to a question from @ebfull on PR #1101 - "Can we think of a way this change allows us to construct two transactions with the same txid which can simultaneously appear in the blockchain? My guess is it would be possible to construct a coinbase transaction of such a form... this surely breaks invariants."
This PR Passes all tests in test_bitcoin (test data was updated in bloom_tests, miner_tests and script_tests).
Remove the assumption that n/(k+1) is a multiple of 8
This version works, but generates the initial rows in a way that is probably
not what we want to specify.
Closes#1148
[WIP] Add more commands to run unit tests under valgrind.
This runs both zcash-gtest and test_bitcoin under valgrind. There's a corresponding PR to the buildbot config https://github.com/Electric-Coin-Company/bbotzc/pull/23. Closes#761.
Extend try catch block around call to libsnark verifier
As discussed in #1126.
@daira Per your [comment](https://github.com/zcash/zcash/pull/1126#issuecomment-234714939):
> I would like assertion errors during tests to cause a test failure (unless the test explicitly expects them). Can we split this into verify and verify_internal, where the former does the try/catch around a call to verify_internal, and the latter is called by verification unit tests?
Did you mean move everything inside the extended try/catch to verify_internal, or just the call to r1cs_ppzksnark_verifier_strong_IC?
Bump the (minimum) protocol version to avoid invoking legacy behavior
This sets the current protocol version to `170002` (an order of magnitude larger than Bitcoin's) and the minimum to the same.
Closes#1107
Collect all permutations of final solutions
This fixes a small bug where if there was a three-way (or more) collision in the final step, one or more valid solutions would be left out.
Remove the constraint system from the alpha proving key.
This removes the constraint system from the `z5-proving.key`, shortening it by about 35%. It appeared at the end of the file, which is why we didn't need to change public parameters in #1104.
Add security warnings doc with warning about side channels.
Closes#5. Closes#785. Closes#488. Closes#784.
Let's only merge this once we're sure the warning is at least as strong as it needs to be (and thus sufficient to close those tickets).
New private/public key pairs for broadcasting alert messages
Implements #424
Fixes and integrates method of sending alerts as described by upstream here:
- https://gist.github.com/laanwj/0e689cfa37b52bcbbb44
To send an alert:
- Copy private keys into alertkeys.h.
- Modify alert parameters and message found in sendalert.cpp
- Build and run to send the alert e.g. ./zcashd -printtoconsole -sendalert
Tested and verified with local nodes on alpha 6 testnet.
Rename things to match protocol specification
This is a pass through the code to eliminate usage of "pour" and "serial" and any other outdated terminology.
Closes#602
Add getblocksubsidy RPC command to return block reward taking into account mining slow start
This PR adds a new RPC command to return the block reward as defined by function `GetBlockSubsidy`.
Usage:
`zcash-cli getblocksubsidy blockheight
`
The basis for this PR is that some users have been unaware of the mining slow start and they subsequently sought help to clarify if they were mining blocks correctly or if there was a bug in the reward schedule.
A script to remove "unofficial" tags from a remote, such as github.
Officialness is determined by a regular expression. ;-)
This is handy because we often want upstream Bitcoin tags for local
diffs, but sometimes we accidentally upload them to github which then
claims they are our releases, which is confusing and misleading.
Flush to disk more consistently by accounting memory usage of serials/anchors in cache.
Closes#626.
It's important that this at least *approximates* the memory usage, so that we flush the cache to disk as expected. It's okay that we overestimate. The serials are stored in keys in the `boost::unordered_map`, so we can simply use that map's `DynamicMemoryUsage`. The anchors are another story.
Only compare the first n/(k+1) bits when sorting
We only need to sort based on the bits we are colliding. In earlier solver rounds, this speeds up the comparison considerably (calling `memcmp()` on 3 bytes instead of 12 in the first round for the current parameters).
Randomise the nonce in the block header
The top and bottom 16 bits of the nonce are left clear for local use as thread
flags and counters. This does not leak any more local information about the
miner than is currently exposed.
The cleared bits should not be considered a consensus rule, as miners are free
to set all bits of the nonce however they wish.
Closes#1033
Zcash address encoding
We need to encode Zcash addresses so they aren't as large and unweildy. We're using Base58Check just like upstream does, and to ensure the first character is "z" in our addresses we must use two bytes for the version string. Two bytes gives us an extra character for free, so this PR targets the beginning of addresses to have "zc".
```
$ ./src/zcash-cli zcrawkeygen
{
"zcaddress" : "tnvaj4ZbZG83tj4RwZcFeLgJoSt8nw1ZvSCG8EMyowAsXTQgJPat77Y43BVdVCrwrbLy7GG9msJDYdn5hmreHmkXAkX17hb",
"zcsecretkey" : "SKzkxCRWvscKnroSFyhCqhY332KcDMH4LLNdK2TsSvbmr3CGAB8B",
"zcviewingkey" : "10aa74046f31cbe5eaa8965d1e104853234c3d6c6e45f9c497ca3a025d159755"
}
```
This PR also encodes the spending keys with a prefix that targets "SK". The spec needs to be updated with these changes.
Testnet addresses will start with "tn".
Closes#572
Final changes to the circuit
* Remove [redundant](https://github.com/scipr-lab/libsnark/issues/37) bitness constraints that I pointed out in #908.
* The depth is increased from 20 to 29. I chose 29 because the QAP degree ends up being a power of two, which is probably going to be useful for the FFT in our MPC. (Closes#16)
If we're happy with depth 29 (over 268 million possible joinsplits), this will probably be the final change to the constraint system before 1.0.
Enforce that coinbases must be protected
This PR forces miners to place their funds in the private value transfer system, making "transparent coins" opt-in for users, and increasing privacy for all participants on a systemic level.
Closes#101.
Use depth-first scan for eliminating partial solutions instead of breadth-first
This reduces the peak number of lists in-memory from 2^k to k, and enables the
solver to eliminate most duplicates before it has instantiated the full set of
leaves.
Part of #857.
Fix issue #717 t
Fix issue #717 where if addrman is starved of addresses (e.g. on testnet)
the Select_() function will loop endlessly trying to find an address,
and therefore eat up 100% cpu time on the 'opencon' thread.
Solution is to (1) add a delay to the loop and (2) restrict the number
of attempts to find an address. On exiting the loop, we return
to an outer loop in net.cpp which will sleep, add seed nodes and
calculate new addresses.
Signing pours with ed25519
This is an alternative to #964 which uses ed25519 instead of secp256k1, and avoids the separate hash for fitting the public key into the `h_sig` block. It's based on @defuse's work in that branch.
Closes#808.
Remove Bitcoin mainnet and testnet seeds.
Simple patch for issue #651 to remove hard-coded mainnet and testnet seeds
- they currently point to bitcoin nodes
- some are .onion addresses but Tor support has not yet been added
- gets rid of error messages in debug.log which might confuse people testing the alpha builds
CPourTx structural changes
* Enable binary serialization of proofs and the proving key (closes#799) and make the proofs fixed-size.
* Reorder fields of CPourTx to match the spec (closes#927)
Enforce first four bits are zero for all spending keys and phi.
This introduces a `uint252` to ensure these fields are 252-bits for the PRFs.
Closes#899
libzcash and new zkSNARK circuit implementation
This PR completes [`libzcash`](https://github.com/zcash/zcash/tree/zc.v0.11.2.latest/src/zcash), the implementation of the [Zcash protocol specification](https://github.com/zcash/zips/blob/9bb4410e45394c8b7e43e35622f54dd4041a4ddd/protocol/protocol.pdf) and replacement of [`libzerocash`](https://github.com/Zerocash/libzerocash), our old Zerocash protocol implementation. The new spec comes with some improvements to security and terminology, with minimal differences from the original academic design.
This implementation includes:
* A rewrite of the zkSNARK circuit for `JoinSplit` operations. This rewrite is cleaner, broken up into separate gadgets, easier to audit and review, and fixes some security bugs. (Closes#822, Closes#809, Closes#500, Closes#854)
* A minimal API for interacting with `JoinSplit`s and surrounding primitives. This PR removes almost twice as much code as it introduces. (Closes#877, Closes#315, Closes#824, Closes#798, Closes#707, Closes#512, Closes#247, Closes#128, Closes#514)
Initialize libsodium in the gtest suite.
We left behind this initialization routine when we switched to gtest. I would rather we had moved those tests over in a separate PR instead of changing existing PRs at the last second -- we would have paid more attention to the consequences.
Run `zcash-gtest` in `make check` and fix performance tests.
* gtest tests weren't being run by make check
* performance tests were broken
* We need to automatically upload graphs of performance, [see this PR](https://github.com/Electric-Coin-Company/bbotzc/pull/15).
* Moves zerocash tests into `zcash`'s full test suite, we're removing them anyway later and it'd be nice to remove them in the PR instead of from buildbot
Implement and integrate new Incremental Merkle Tree
This supersedes #823.
----
This is an implementation of a new incremental merkle tree with
* no memory safety issues
* a more sensible internal design
* better space efficiency (tree representation, witnessing)
* simpler API
It is intended that this tracks the behavior of the previous tree, which it does, as verified by tests. I even wrote a little circuit for testing that all the paths work.
This PR also integrates the tree into the codebase and deprecates the old tree in almost all of our code. (I left it alone in `zerocashTest` but everything else has been changed.)
This change is compatible with the testnet but you will need to clear your *local* blockchain data out since the serialized representation of the merkle tree is now different.
Closes#517, Closes#519, Closes#591, Closes#460, Closes#473
Automated performance measurement
Supersedes #843 because that one would have merged into the wrong branch. (Oh yeah and I rebased).
**REBASED AND FORCE PUSHED**
Implement Founders' Reward
More info: https://z.cash/blog/funding.html
The consensus rule is as follows:
```
All blocks before the first subsidy halving block, with the exception of
the genesis block, must contain an output which sends 20% of the block
subsidy value to a scriptPubKey `FOUNDERS_REWARD_SCRIPT`.
```
Right now, `FOUNDERS_REWARD_SCRIPT` is a 2-of-3 multisig P2SH.
Closes#125
zkbot