DragonX/dragonx
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 4 Wiki Activity

Compare commits

base: DragonX/dragonx:fa16e740b602ee94de5a194e1be25583f340e968
Branches Tags
DragonX/dragonx:master DragonX/dragonx:dev DragonX/dragonx:perf/wallet-scan-rpc-costs DragonX/dragonx:audit-fixes-20260901 DragonX/dragonx:autoshield-safety DragonX/dragonx:feature/autoshield-coinbase
DragonX/dragonx:v1.3.0 DragonX/dragonx:v1.2.0 DragonX/dragonx:v1.1.0 DragonX/dragonx:v1.0.3 DragonX/dragonx:v1.0.2 DragonX/dragonx:v1.0.1 DragonX/dragonx:v1.0.0
...
compare: DragonX/dragonx:perf/wallet-scan-rpc-costs
Branches Tags
DragonX/dragonx:dev DragonX/dragonx:perf/wallet-scan-rpc-costs DragonX/dragonx:master DragonX/dragonx:audit-fixes-20260901 DragonX/dragonx:autoshield-safety DragonX/dragonx:feature/autoshield-coinbase
DragonX/dragonx:v1.3.0 DragonX/dragonx:v1.2.0 DragonX/dragonx:v1.1.0 DragonX/dragonx:v1.0.3 DragonX/dragonx:v1.0.2 DragonX/dragonx:v1.0.1 DragonX/dragonx:v1.0.0
24 Commits
fa16e740b6 ... perf/wallet-scan-rpc-costs
Author SHA1 Message Date
DanSandClaude Opus 4.8 416f1ffd6d rpc: read vNodes under cs_vNodes in getinfo/getnetworkinfo 
getinfo and getnetworkinfo read vNodes.size() and counted the TLS peers
under cs_main (getinfo also cs_wallet), not cs_vNodes, racing the socket
handler that pushes to and erases from vNodes. That is a torn read and
iteration over a vector being reallocated: undefined behaviour, in
practice a rare crash or a stale, inconsistent count.

Add GetNodeCounts(), which takes cs_vNodes once and returns a consistent
snapshot (tls_connections <= connections). Both RPCs now read the counts
through it before taking cs_main, so the two locks are never held at once
(no lock-order change, no extended hold of cs_vNodes) and the
"connections"/"tls_connections" output fields are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-10-14 09:09:12 -05:00
DanSandClaude Opus 5.5 fd67a1a15a wallet: don't abort building witnesses for viewing-key-only notes 
A note found with only an incoming viewing key (z_importviewingkey) never
gets a nullifier: UpdateSaplingNullifierNoteMapWithTx needs the full
viewing key. VerifyAndSetInitialWitness built a witness for such a note
anyway and then dereferenced *item.second.nullifier, aborting the daemon
on the boost::optional assertion.

Importing a viewing key that had received funds therefore crashed the
node during the import's rescan, and again at every start and on every
block, since witnesses are rebuilt then too. Reproduced on regtest
(z_importviewingkey -> ScanForWalletTransactions -> BuildWitnessCache ->
VerifyAndSetInitialWitness).

Skip notes that have no nullifier and no full viewing key: they cannot
be spent and need no witness. Only call SaplingWitnessMinimumHeight when
the nullifier is set, and log the corrupt-data case where a note with a
full viewing key gets no witness, which used to abort as well. A note
whose spending key is imported later is picked up on the next rebuild.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:31:01 -05:00
DanSandClaude Opus 5.5 1faa349c27 gtest: check GetFilteredNotes and operation status against the old code 
- test_wallet_scan_notes.cpp compares the new GetFilteredNotes with a
  verbatim copy of the original over a fabricated wallet: spending keys,
  diversified, unrecorded, ivk-only, fvk-only, mismatched and aliased
  addresses; spent, unspent, locked, conflicted and mempool notes at many
  depths; every filter/flag combination (about 1,400 calls), plus an
  encrypted and locked wallet. It also checks decrypt_with_pk_d against
  decrypt.
- test_operation_status.cpp compares z_getoperationstatus,
  z_getoperationresult and z_listoperationids with the original algorithm
  (up to 300 ops, same-second ties), shows the status calls finish while
  another thread holds cs_main and cs_wallet, and checks that concurrent
  pops hand each operation to exactly one caller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:31:01 -05:00
DanSandClaude Opus 5.5 7f64df008e rpc: answer z_getoperationstatus without cs_main and cs_wallet 
z_getoperationstatus took LOCK2(cs_main, cs_wallet) for a pure read of the
async operation queue, so a GUI polling it every few seconds queued behind
every wallet scan and block connect. It also sorted UniValues with a
by-value comparator.

- z_getoperationstatus_IMPL takes no wallet locks. z_getoperationresult
  takes LOCK2 itself, so the removing path stays serialised with the
  wallet's swap of automated operations (cancel then remove, under
  cs_wallet).
- Sort (creation_time, index) pairs comparing the time only. libstdc++'s
  sort depends only on the comparator results, so the order, ties
  included, is the same as before.
- AsyncRPCQueue::popOperationForId finds and erases under one lock.
- z_listoperationids takes no wallet locks; the status-name map is const
  and read with at().

Output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:31:01 -05:00
DanSandClaude Opus 5.5 c32c85b7f0 wallet: count the entries listtransactions skips instead of building them 
listtransactions formatted every entry before "from" and then threw it
away. Paging through the history that way is quadratic: ObsidianDragon
pages up to 5000 rows at a time, so a 30k-row mining wallet built about
105k entries per full refresh (30k with this change). Skipped entries are
now counted after the same account and watch-only checks as before and
never formatted.

Output is unchanged for every valid input. When count + from overflows
(the build uses -fwrapv), the result is [] instead of an out-of-bounds read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:31:01 -05:00
DanSandClaude Opus 5.5 1541d045ee wallet: cut per-entry copies in listtransactions and gettransaction 
- ListTransactions computes a tx's "size" once, lazily, through a const
  CTransaction reference instead of a slicing copy per entry.
- listtransactions copies the returned window once from the built array.
- OrderedTxItems can open the wallet DB read-only without a flush on close;
  listtransactions uses it, so a read-only RPC no longer forces a BDB
  checkpoint (ThreadFlushWalletDB still flushes about 2 s later).
- gettransaction hex-encodes the tx without copying it.

Output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:31:01 -05:00
DanSandClaude Opus 5.5 23f3e26d2f wallet: stop zsTxSpendsToJSON inserting into the nullifier and note maps 
zsTxSpendsToJSON looked spends up with operator[]: every foreign nullifier
in a listed tx inserted a null outpoint into mapSaplingNullifiersToNotes,
and the note-data lookup could insert an empty SaplingNoteData. A read-only
RPC was mutating wallet state.

z_viewtransaction then found those null outpoints and failed with RPC -1
"map::at" for every tx with a foreign Sapling spend (most incoming z->z
payments) once z_listsentbyaddress or getalldata had run, until restart.
ObsidianDragon treats that error as transient and retried those txs on
every refresh.

Use find() with the same test as IsSaplingNullifierFromMe(). This call
produces the same rows in every case; z_viewtransaction now answers as it
does on a freshly started daemon.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:31:01 -05:00
DanSandClaude Opus 5.5 08afe5bb3b wallet: read wallet txs and note data by reference in more RPCs 
z_listreceivedbyaddress copied a whole CWalletTx per returned note to call
GetTxTime(). z_viewtransaction copied the previous tx of each own spend,
CWalletTx::DecryptSaplingNote copied the output and the note data (witness
list included) to read one field, and zsTxSpendsToJSON copied the note
data. All of them now bind const references to the same elements. The
lookups that throw or assert still do, and output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:31:01 -05:00
DanSandClaude Opus 5.5 375b995b38 wallet: stop z_listsentbyaddress re-deriving keys and copying the wallet 
z_listsentbyaddress, getalldata, z_getbalances and z_listreceivedaddress
deep-copied every CWalletTx (witness lists included) into an ordered map,
and the JSON helpers re-derived keys and scanned the whole wallet for every
output: zsTxReceivedToJSON trial-decrypted each output with every spending
key, and each change check called GetNullifiersForAddresses, a full
mapWallet note scan, once per decrypted output.

- ZsTxJSONContext memoises per call the spending keys with their ivk/ovk,
  the HD-seed ovk and the wallet's nullifier set.
- Wallet txs are ordered by pointer. stable_sort + unique on nOrderPos keeps
  the first entry, exactly like the old map::insert.
- A received output is first decrypted with the ivk the wallet recorded for
  it when that ivk belongs to one of the usable keys, then with the
  remaining keys in the old order.
- "size" is measured without a slicing copy; the result is reversed in one
  pass; getalldata's note loop reads note data by reference and reuses the
  memoised keys.

Output is unchanged. New opt-in 6th argument sendsOnly (default false):
true omits each entry's "spends" and "received" sections and the trial
decryption they need. Older daemons reject a 6th argument with the help
text, so a client can fall back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:31:01 -05:00
DanSandClaude Opus 5.5 106fc1ceac wallet: filter notes before decrypting them in GetFilteredNotes 
GetFilteredNotes copied every CWalletTx and every SaplingNoteData (witness
list included) by value, and trial-decrypted every note before it applied
the address, spent and lock filters. It sits under z_listunspent,
z_gettotalbalance, z_getbalance, z_listreceivedbyaddress, z_mergetoaddress
and sweep/consolidation, all with cs_main and cs_wallet held, so on a wallet
with thousands of notes it is most of the time those RPCs block the
block-connect thread.

- Read wallet txs and note data by const reference, and skip txs with no
  Sapling notes before the finality, maturity and depth checks.
- Check IsSaplingSpent before decrypting, so spent notes are not decrypted
  when ignoreSpent is set.
- Address pre-filter: when 1-16 filter addresses all have a recorded ivk that
  is canonical (below 2^251) and actually generates that address, a note
  whose canonical ivk is none of them cannot pay a filter address and is
  skipped without decrypting. z_importviewingkey stores (ivk, address) pairs
  unchecked, hence the verification; an aliased ivk + r_J would pass it,
  hence the canonical check. Any other case takes the old path.
- SaplingNotePlaintext::decrypt_with_pk_d returns the pk_d that decryption
  already computes, so the note's address and SaplingNote are built without
  two more scalar multiplications. decrypt() is now a wrapper around it.

RPC output is unchanged. The one behaviour difference: a spent note that
cannot be decrypted no longer trips assert(maybe_pt) when ignoreSpent is set.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:31:01 -05:00
DanSandClaude Opus 5 c1040028e4 wallet: checkpoint rescan progress when the scan is interrupted 
An aborted or shutdown-interrupted rescan discarded all of its progress: nothing
advanced the on-disk best-block locator, so the next run started over. On a large
shielded wallet that is a multi-minute witness rebuild repeated from scratch.

Writes the locator ONCE, at the interrupt, rather than periodically through the
scan. That delivers the whole stated benefit -- "resume where it stopped" -- at
the cost of exactly one SetBestChain per interrupted scan. A periodic checkpoint
would additionally survive SIGKILL and power loss, a much weaker requirement, and
it is the part that carries all the cost: SetBestChainINTERNAL is O(whole wallet)
regardless of progress, so on the reported 5.3k-tx / 7.5k-note wallet a checkpoint
every 2500 blocks would plausibly cost more than the rebuild it was meant to save.
If someone later demonstrates a need for crash resilience mid-scan, add it then,
with a measured interval and fFlush=false -- not before.

Two guards, both necessary:

  - CONTIGUITY. The RPC entry points (rescan / importprivkey / z_importkey /
    z_importviewingkey) take a caller-supplied start height validated only against
    chainActive.Height(), never against the wallet's own persisted locator. A scan
    beginning above that locator must not checkpoint at all, or it would record
    the skipped range as scanned and hide any funds in it. Logged once when it
    applies, so an operator can see why an interrupt did not persist.

  - MONOTONICITY. A checkpoint may only advance the locator, never move it back.

The locator points at the last FULLY PROCESSED block -- the parent of the block we
were about to scan -- so resume restarts one block early. CChain::GetLocator pushes
its argument first and FindForkInGlobalIndex returns that same block, so resume
begins AT it; the one-block overlap is deliberate and idempotent, since AddToWallet
only takes its merge path when the transaction is already present.

No witness work is done at the interrupt, which answers the two "//TODO: should we
update witness caches?" comments this replaces: witnesses are re-derived from each
note's own witnessHeight independently of the locator, and witnessRootValidated is
never serialized, so every note is revalidated against hashFinalSaplingRoot on the
next start. A mid-scan checkpoint does capture notes at mixed witnessHeights -- the
in-loop BuildWitnessCache(pindex, true) returns before the extension phase and only
seeds new notes at their own transaction's height -- but that state is recoverable:
the post-loop BuildWitnessCache(tip, false) levels every note, and it now occurs at
most once per scan instead of hundreds of times.

Uses SetBestChainNoFlush so the checkpoint does not trigger a full BDB
txn_checkpoint over the whole cache, and reports failure rather than silently
leaving the operator to discover the replay.

Depends on the fLastRescanCompleted gate in the previous commit: without it
init.cpp would overwrite this checkpoint with a tip locator immediately after the
scan returns.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
 2026-08-31 20:54:37 -05:00
DanSandClaude Opus 5 a6b6f80db0 wallet: stop an interrupted rescan from hiding funds, and un-latch fAbortRescan 
Two independent fund-visibility bugs on the rescan path, both pre-existing.

1. AN INTERRUPTED RESCAN RECORDED ITSELF AS COMPLETE.

ScanForWalletTransactions returns a bare `int ret` (a found-tx count) on both its
abort and shutdown bail-outs, so the caller could not tell "finished" from
"stopped at block H". init.cpp then ran, unconditionally:

    pwalletMain->ScanForWalletTransactions(pindexRescan, true);
    pwalletMain->SetBestChain(chainActive.GetLocator());   // TIP locator

An interrupted scan therefore stamped the wallet as scanned all the way to the
chain tip. On the next start, the `chainActive.Tip() != pindexRescan` guard just
above sees no work to do and skips the rescan entirely, so every transaction in
the never-scanned range stays out of mapWallet -- permanently invisible to
getbalance and unspendable. That is exactly the case `-rescan` exists for: a key
import, right after ClearNoteWitnessCache has run.

Adds CWallet::fLastRescanCompleted, set false at scan entry and true only on the
normal exit, and gates the init.cpp locator write on it. Deliberately NOT
overloading the int return, which callers already use as a tx count.

2. fAbortRescan WAS NEVER RESET.

wallet.h declares it, AbortRescan() sets it true, and nothing in src/ ever sets
it false. The abortrescan RPC is live. After one call, for the remaining lifetime
of the process:
  - every ScanForWalletTransactions returns immediately, so re-running an import
    silently no-ops while the RPC still reports success;
  - BuildWitnessCache bails on the same flag on EVERY call, including the routine
    per-block extension from ChainTip.
Witness heights then diverge across notes while the chain advances, and
GetSaplingNoteWitnesses elects the majority root as the anchor and returns
boost::none for every note that disagrees. Those notes still show in the balance
but cannot be spent. Cleared at scan entry and consumed in the abort branch.

Also in BuildWitnessCache's abort branch: stop clearing fRescanning. A witness
rebuild is not a rescan and must not touch a flag ScanForWalletTransactions owns.

3. Both bail-out log lines had two format specifiers and one argument:

    LogPrintf("%s: Rescan aborted at block %d\n", pwalletMain->rescanHeight);

tinyformat's "too many conversion specifiers" guard is disabled in this tree
(tinyformat.h: `if(*fmt != '\0' && 0 ) // disabled due to complaints`), so this
does not throw -- verified by compiling the exact call against this tree's
tinyformat.h. It emits "3841207: Rescan aborted at block " with the height in the
__func__ slot, the real height dropped, and the trailing newline swallowed so the
next log line concatenates onto it. On the one path an operator has to diagnose
an interrupted rescan, the only record was corrupt. Fixed at both sites, and the
state updates moved above the log call.

4. Makes SetBestChainINTERNAL report whether the atomic write committed, so a
checkpointing caller can tell (six failure paths returned void). Adds
SetBestChainNoFlush, which opens the wallet DB with fFlushOnClose=false: the
default ctor makes ~CDB run a full BDB txn_checkpoint over the whole cache, which
is fine hourly but not from inside a scan -- the same flush wallet.cpp already
documents avoiding elsewhere "for performance reasons". Nothing calls it yet.

5. SetBestChainINTERNAL took each CWalletTx by value, deep-copying every note's
witness deque (WITNESS_CACHE_SIZE entries) per transaction per call, purely to
serialize it. Now by const reference.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
 2026-08-31 20:52:40 -05:00
DanSandClaude Opus 5 3aac75e94f cli: report the port actually dialled, not a stale global 
dragonx-cli resolves -rpcport correctly -- CallRPC does
  int port = GetArg("-rpcport", BaseParams().RPCPort());
and dials that port. But both connection-failure messages printed
ASSETCHAINS_RPCPORT instead, a separate global defined at the top of
bitcoin-cli.cpp, initialised to the mainnet default 21769, and never assigned
anywhere in the CLI.

So every failure claimed port 21769 regardless of what was asked for:

  $ dragonx-cli -rpcport=21799 getblockcount
  error: couldn't connect to server at port 21769

That reads as "your -rpcport was ignored", which is a much more alarming and
much more misleading diagnosis than "nothing is listening yet". It cost an hour
of debugging on a node that was simply still loading a 15 GB txindex, and led to
the wrong conclusion that the CLI could be talking to the production daemon when
it was not.

Both sites now print the local `port`. Verified against the previous binary:

  new:  -rpcport=21799 -> "couldn't connect to server at port 21799"
        -rpcport=59999 -> "couldn't connect to server at port 59999"
  old:  both            -> "couldn't connect to server at port 21769"

and the CLI still reaches a live daemon on a non-default port.

ASSETCHAINS_RPCPORT is left defined; it is referenced by other translation units.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
 2026-08-31 17:35:52 -05:00
DanSandClaude Opus 5 5634aed750 wallet: open salvaged wallets in a degraded mode, and detect a mismatched hdchain 
A wallet.dat repaired by v1.0.3-or-earlier -salvagewallet could not be opened by
this build at all. Those builds' IsKeyType has no "hdchain" case, so salvage
dropped the record; LoadWallet then returned DB_CORRUPT and told the user to
restore from a seed phrase. There is no such recovery path: the node aborts
before the RPC server exists, -usemnemonic defaulted to 0 in v1.0.3 so many of
these wallets never had a phrase, and SetHDSeedFromMnemonic refuses a non-empty
wallet, so "move wallet.dat aside" discards every non-HD key salvage preserved.

The guard itself was right to refuse to DERIVE -- a missing hdchain falls back to
SetNull defaults, clearing fMnemonicSeed and switching derivation from the
64-byte BIP39 seed to the raw 32-byte entropy, an entirely different key tree.
It was wrong to refuse to OPEN. Split the two:

  - "hdchain" record ABSENT (the salvage signature, tracked by a new
    wss.fHDChainSeen set before any parse attempt) -> open DEGRADED.
  - "hdchain" record PRESENT but unreadable -> still DB_CORRUPT; that signals
    wider file damage.

Degraded means the wallet spends, receives and rescans normally but refuses to
derive any new HD key. Gated at the single choke point GetHDSeedForDerivation
rather than at the two generators, so z_getnewaddress/sendmany/shieldcoinbase
raise a clean error and autoshield skips. hdChain.seedFp is deliberately left
null, which keeps IsHDTransparentEnabled false so getnewaddress falls back to the
legacy random-key path -- gating it instead would break TopUpKeyPool, change
addresses and block templates. -rescan is soft-set because an old salvage also
dropped defaultkey and bestblock, which would otherwise look like a first run and
skip the rescan, leaving a permanently zero balance.

NEW DETECTOR, covering strictly more damage than the guard beside it. A wallet
salvaged by v1.0.3 and then USED on v1.0.3 persists a SetNull-derived hdchain
carrying seedFp=null. On this build fHDChainRead is then true, the guard never
fires, the node starts perfectly clean -- and derives into the wrong key tree
forever, silently. That state is unforgeable by any legitimate writer, since
InstallHDSeed always stores seedFp = seed.Fingerprint(), so compare the loaded
chain's seedFp against the fingerprint taken from the hdseed/chdseed record KEY
(which works while an encrypted wallet is locked) and degrade on mismatch.

Also:
  - init.cpp: abort in the DB_CORRUPT branch itself, as DB_NEED_REWRITE already
    does. Falling through ran several hundred more lines against a wallet just
    declared corrupt, including SetHDSeedOrigin(), which WRITES to it. Error text
    no longer advises -mnemonic (wrong twice over, see above).
  - rpcdump.cpp: z_exportwallet discarded GetHDSeedForDerivation's return and
    emitted the line regardless, so a failure wrote a blank seed next to a
    legitimate-looking BLAKE2b-of-empty fingerprint -- a backup that looks valid
    and restores nothing.
  - Corrected the guard's comment: the saplingAccountCounter half was overstated.
    Both generators loop while(Have...Key(...)), so a low counter walks forward
    past existing accounts rather than colliding with them.

qa/r4-salvaged-wallet-harness.sh builds the victim wallets with the on-disk
v1.0.1 release binary and asserts the behaviour end to end. It REFUSES to run
outside a network namespace and re-checks getconnectioncount==0, because those
old binaries predate the regtest seed-injection fix and would otherwise dial the
live network. 11/11 pass:

  C healthy wallet opens and is NOT flagged   (no false positives)
  A salvaged wallet opens, flagged, persists no hdchain, z_getnewaddress refused,
    getnewaddress still works, z_exportwallet emits no bogus HDSeed line
  B poisoned wallet opens and the seedFp detector fires

Note the harness asserts "no hdchain synthesised", NOT "wallet.dat unchanged":
a control run showed normal startup rewrites wallet.dat for healthy wallets too.

This is the access/detection half. Proving fMnemonicSeed by re-deriving against
held keys, and repairing the record, is deliberately left out: it requires a
write, and the affected population is unmeasured. dev's own salvage already
preserves hdchain, so the class is closed going forward.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
 2026-08-31 13:29:43 -05:00
DanSandClaude Opus 5 db42091ce3 rpc/net: gate and harden stratummine, keep regtest off the live network, init nSPV filter 
stratummine (rpc/mining.cpp), a test-only reference miner, was registered
unconditionally behind a bare #ifndef WIN32 with okSafeMode=true, so it shipped
as a live RPC on every non-Windows release build. Four fixes:

  - Gated behind an explicit, default-off -stratummine, with regtest exempt so
    qa/rpc-tests can still drive it. Deliberately NOT gated on fExperimentalMode:
    that defaults to TRUE (init.cpp:1195), so such a gate is a no-op -- an error
    made and caught while testing this change.
  - mining.notify's three hash fields went straight into uint256(ParseHex(...)).
    uint256's vector ctor asserts on a wrong size (uint256.cpp:30) and NDEBUG is
    defined nowhere in this build, so that assert is live in release; ParseHex
    also truncates silently at the first non-hex character. A short or garbled
    field therefore ABORTED THE DAEMON. Added StratumHex256(), which requires 64
    hex chars and a 32-byte result, and all three fields are validated before any
    is committed so a bad job is rejected rather than half-applied.
  - processLine ran inside the window where the RandomX cache, the VM and the
    socket are live, all released on the normal path only, and every get_str()
    throws on a type mismatch -- so a malformed message leaked 256 MB and the fd.
    Body wrapped in try/catch: ignore the line, keep mining.
  - Caller-supplied timeout clamped to [1, 3600]; it was unbounded, pinning an
    RPC worker and the cache indefinitely. okSafeMode -> false.

Verified against a hostile stratum server on regtest:
  - 4 malformed mining.notify payloads (short hex, non-hex, wrong JSON type,
    31 bytes) -> all rejected, daemon alive, 0 assertions. The 31-byte case is
    the one that previously hit the uint256 assert.
  - valid job first (so RandomX actually allocates) then garbage mid-mine ->
    RSS delta +2.2 MB, i.e. cache and VM released, not the ~256 MB a leak leaves.
  - regtest exemption confirmed: stratummine runs past the gate there.
  The non-regtest refusal path is by code reading only -- a testnet node on this
  host collides with the production daemon's RPC port, so it was not exercised.

hush_utils.h: stop injecting the mainnet node1-node10.dragonx.is addnode seeds
when -regtest or -testnet is set. regtest reuses mainnet's network magic, so a
supposedly isolated node was handshaking production peers and pulling their
headers into its own index. Gated at the injection site only -- isdragonx itself
must stay true, because it also selects ac_private, ac_algo, blocktime and the
reward/halving schedule (an earlier version of this patch gated isdragonx itself
and silently turned ac_private off on regtest). hush_args() runs between
ParseParameters() and ReadConfigFile() (bitcoind.cpp:115/144/158), so this sees a
command-line -regtest, as qa/rpc-tests uses, but not a config-file regtest=1.

hush_nSPV_fullnode.h: initialize `filter` at both sites. It was assigned only on
the len-11 request form; the other two passed uninitialized stack memory to
NSPV_getaddressutxos/NSPV_getaddresstxids, remotely reachable since
HUSH_NSPV_FULLNODE is on by default.

getblocktemplate_proposals.py still passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
 2026-08-31 02:38:38 -05:00
DanSandClaude Opus 5 6d282db216 consensus: stop BLOCK_VALID_CONTEXT overwriting the block validity level 
BLOCK_VALID_CONTEXT was 6. The validity levels above it are sequential VALUES
packed into a 3-bit field, not independent bits, so BLOCK_VALID_MASK is
1|2|3|4|5 == 7 and the value 6 sat entirely inside it. `pindex->nStatus |=
BLOCK_VALID_CONTEXT` (main.cpp:5616, and :3285) therefore did not set a flag --
it overwrote the validity level.

Consequences, all long-standing:
  - A header-only block raised to BLOCK_VALID_TREE(2) became 2|6 == 6, which
    reads as >= BLOCK_VALID_CHAIN(4) and >= BLOCK_VALID_SCRIPTS(5). A block
    merely written to disk reported full script validity.
  - Every later RaiseValidity() silently no-opped, because 6 >= every level.
    ConnectBlock's RaiseValidity(BLOCK_VALID_SCRIPTS) was a permanent no-op.
  - CheckBlockIndex's "CHAIN valid implies all parents are CHAIN valid" invariant
    was violated whenever a stored block sat above a still-header-only ancestor,
    i.e. ordinary out-of-order parallel block download. fDefaultConsistencyChecks
    is true only for regtest, so the abort was regtest-only -- but the garbled
    index is written identically on mainnet, where only the detection is off.

Not a v1.3.0 regression: introduced upstream in Komodo fa309e5b0 (2019-04-02),
inherited via Hush, and byte-identical in v1.0.3, which the production network
runs today. Validity was only ever INFLATED, never deflated, so no valid block
was rejected and no invalid block skipped validation -- ConnectBlock's CheckBlock
and full script/proof verification always ran. The user-visible effects were
misreports: getchaintips labelling never-connected forks "valid-fork",
submitblock answering "duplicate" for unvalidated blocks, and ProcessGetData
serving them. The material cost was to QA: multi-node regtest tests aborted the
syncing node at random, making the rpc-test suite unusable.

Moves the flag to 512, the next free bit above BLOCK_IN_TMPFILE(256), and adds
static_asserts that every nStatus flag is disjoint from BLOCK_VALID_MASK so this
cannot be reintroduced silently. nStatus is serialized as VARINT, so the wider
value needs no format change.

Verified under gdb on regtest. A connected block's nStatus:
  before   0x1e  validity field 6            (measured on the previous binary)
  after   0x21d  validity field 5 = SCRIPTS, context bit set
wallet_sapling.py, which aborted the syncing node deterministically, now runs to
completion with no assert.

No migration: the original level is unrecoverable from a polluted entry (1|6,
3|6 and 5|6 all give 7; 2|6 and 4|6 both give 6), and the only safe guess is
downward, which would risk revalidation work on a 3.25M-block index for a
cosmetic gain. Legacy entries simply have the flag bit clear, so they re-run the
contextual check they previously skipped -- more checking, not less -- and
resolve on any reindex.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
 2026-08-31 02:12:17 -05:00
DanSandClaude Opus 5 c5fde12485 qa: port 12 wallet/mining tests to python3 and fix three framework blockers 
Ports qa/rpc-tests from 6 python3 files to 18. Ran every ported test against
the freshly built v1.3.0 dragonxd. Results, honestly:

  PASS (1)            getblocktemplate_proposals.py
  NOT APPLICABLE (7)  wallet.py, walletbackup.py, wallet_protectcoinbase.py,
                      wallet_listnotes.py, wallet_mergetoaddress.py,
                      getblocktemplate.py, wallet_shieldcoinbase.py
  BLOCKED (4)         wallet_sapling, wallet_nullifiers, wallet_persistence,
                      wallet_treestate

The "not applicable" seven are inherited Zcash/Hush-era tests that exercise
features DragonX deliberately removed. They assume transparent t->t value
transfer, but ASSETCHAINS_PRIVATE=1 (hush_utils.h:1826) makes sendtoaddress
and sendmany consensus-refuse; they assume Sprout joinsplits, which are gone
from the RPC layer entirely; and they hardcode Bitcoin economics (10 coin/block,
100-block maturity) against DragonX's 3 DRGX and COINBASE_MATURITY=1. They are
ported and left in place rather than deleted, but they cannot pass on this chain
without being rewritten around z_shieldcoinbase/autoshield.

Three framework fixes in test_framework/util.py, each of which broke every
multi-node test:
  - initialize_chain() passed -connect=0, and init.cpp soft-sets -listen=0 when
    -connect is present, so cache node0 never opened its p2p port and nodes 1-3
    could never sync to it -- initialize_chain() hung forever in sync_blocks().
    Now passes -listen=1 -bind=127.0.0.1 -dnsseed=0 explicitly (an explicit arg
    beats SoftSetBoolArg) while keeping the cache nodes off the public network.
  - cache cleanup removed files from <datadir> when dragonxd writes them one
    level deeper into the net-specific <datadir>/regtest.
  - set_node_times() did print("..." + t) with t an int -> TypeError.
  - default binary paths corrected to src/dragonxd and src/dragonx-cli.

The four BLOCKED tests are blocked by a daemon assert, not by the port; see the
follow-up commit/report on BLOCK_VALID_CONTEXT.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
 2026-08-31 01:53:43 -05:00
DanSandClaude Opus 5 e2e10f6ef8 doc: regenerate man pages from the v1.3.0 binaries 
Previously the version strings were restamped by hand in af7d9e230 because
help2man and a built binary were both unavailable. Regenerated properly via
util/gen-manpages.sh against freshly built v1.3.0 binaries, which also picks
up two options that were never documented:

  -sietch-min-zouts=<n>   decoy Sapling outputs added to each z_sendmany
  -stratumtarget=<hex>    pool share target, for solo/low-difficulty mining

NOTE: the version lines read "v1.3.0-af7d9e230" because `git describe` has
no annotated tag to find yet. Once v1.3.0 is tagged annotated, these must be
regenerated once more so they read a clean "v1.3.0" -- that step stays open
on the release checklist.

Generated on seed 176 (glibc 2.35 runs the binaries; help2man is installed
there and not on the primary), with an isolated HOME so nothing touched the
node's datadir. The reindex running on that box was not disturbed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
 2026-08-31 00:23:56 -05:00
DanSandClaude Opus 5 af7d9e2300 release: bump to v1.3.0 and document the signing step that was never performed 
The tree stamped 1.2.0 in both configure.ac and src/clientversion.h, but
v1.2.0 is an annotated tag already pushed at fad05d3ab and dev is 27
commits past it. Both trees therefore produced CLIENT_VERSION 1020050 and
announced an identical /DragonX:1.2.0/ subversion, so a released binary
would have been indistinguishable from the tag on the wire, in
getnetworkinfo, and to the wallet's in-app updater -- destroying the only
provenance check users have: build the tag, compare the binary.

Bump MINOR rather than REVISION: the delta since v1.2.0 adds a subsystem
(RandomX stratum) and a new RPC (stratummine).

  configure.ac, src/clientversion.h  1.2.0 -> 1.3.0 (CLIENT_VERSION 1030050)
  doc/man/*.1                        version strings restamped
  contrib/debian/changelog           1.3.0 entry for the 27 commits
  src/chain.h                        stale "CLIENT_VERSION is 1010050" comment

Man page *content* still needs a real regeneration: util/gen-manpages.sh
requires help2man and built 1.3.0 binaries, so it belongs in the release
build, where it will also pick up the new stratum options.

doc/release-process.md is why v1.1.0 and v1.2.0 were tagged but never
became releases. Followed verbatim it produced a release the wallet
refuses to install:

  - it directed releases to a branch named `dragonx`, which does not
    exist; releases are cut on `master`
  - it never once mentioned signing, yet the updater pins an ed25519 key
    and sets kDaemonRequireSignature = true, so an unsigned release is
    refused outright and every user silently stays on their old daemon
  - it did not require the release tag to be annotated, and genbuild.sh
    calls `git describe` without --tags, so a lightweight tag stamps the
    build `v<older-tag>-<sha>` instead of the release version -- which is
    exactly what happened to v1.0.0 through v1.0.3
  - it referenced util/build-debian-package-ARM.sh, which is not in tree

Adds the signing and checksum-table steps, the annotated-tag requirement
with a `git describe` verification, and the rule that a new version must
exceed every existing tag including unpublished ones.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
 2026-08-30 22:48:52 -05:00
DanS 4cc7e0491a qa: finish the python3 port far enough to build the shared test chain 
Follow-up to 5e0a70683, which got start_node() working. Three more
defects sat behind it:

- initialize_chain() builds the 4-node cache with its own daemon
  invocation, which 5e0a70683 did not touch. It therefore still omitted
  -regtest (so those cache nodes ran on MAINNET) and -asmap (so they
  refused to start at all). Every test that uses the cache -- which is
  most of the wallet suite -- died there.

- reindex.py and getblocktemplate_longpoll.py each carried a single
  python2 print statement, which is the whole reason they would not even
  parse under python3. Shebangs updated to match.

The suite still does not pass: initialize_chain hits a remaining py2
str+int concatenation, and getblocktemplate.py reaches a real test
assertion. Both are beyond this commit, but the harness now gets far
enough to start nodes, answer RPC and begin building the shared chain,
which it could not do before.
 2026-08-30 21:29:08 -05:00
DanS 5e0a706839 qa: repair the rpc-test harness so it can start a DragonX node at all 
The integration suite has never run against DragonX. Six independent
defects stacked up, each only visible once the previous was fixed:

1. test_framework used python2 implicit relative imports
   ("from authproxy import ..."), removed in python3, so every test died
   at import. Made explicit relative imports.

2. It wrote ZZZ.conf -- a Komodo assetchain convention -- while DragonX
   reads DRAGONX.conf. The daemon therefore never saw the generated
   config, fell back to mainnet defaults and tried to bind RPC 21769,
   which on a seed node is already held by the real node.

3. start_node() was hard-wired for the -ac_name=ZZZ assetchain tests: it
   took the RPC port from extra_args[3], passed extra_args[0] as argv[0]
   of the CLI, and only wrote a config when extra_args[0] matched. Any
   test that passes no extra args crashed on len(None). The generic path
   now takes the port from rpc_port(i) -- the same helper
   initialize_datadir() already used -- and drives the CLI with -datadir.

4. dragonxd refuses to start without an asmap file, which no test datadir
   had. initialize_datadir() now provisions one.

5. -asmap relative paths resolve against the NET-SPECIFIC datadir, so a
   copy in <datadir> is never found. Pass an absolute path.

6. Worst: -regtest was only ever set as "regtest=1" in the conf file,
   which DragonX ignores. Every "regtest" node therefore ran on MAINNET:
   real genesis, real seeds, real peers. An observed run synced 196,180
   live blocks and 679MB into /tmp before the test timed out. -regtest is
   now passed as a command-line flag, with -connect=0 so an isolated
   regtest node stays off the public network.

With these, nodes start, RPC answers, and tests run to a real result.
They do not all pass yet -- getblocktemplate.py reaches an assertion --
but that is now a test outcome rather than a harness failure.
 2026-08-30 19:44:27 -05:00
DanSandClaude Opus 4.8 60d66022f6 regtest: default -checkpoints off so an isolated node leaves IBD 
chainparams_commandline() applies the DRAGONX mainnet checkpoint set to
every SMART_CHAIN_SYMBOL=="DRAGONX" network, -regtest included, so an
isolated regtest node (height ~hundreds) sits far below the top checkpoint
(~3.2M). IsInitialBlockDownload() latches true whenever fCheckpointsEnabled
&& height < GetTotalBlocksEstimate(), so regtest was permanently in IBD --
disabling every ChainTip auto-op that gates on !IBD (autoshield, z_sweep,
consolidation) and the below-checkpoint script-check skip, and forcing
every regtest test to pass -checkpoints=0 by hand.

Default -checkpoints to false on regtest (still overridable with
-checkpoints=1). Verified: a fresh regtest node logs "Leaving
InitialBlockDownload" after a few blocks with no flag, and -checkpoints=1
keeps it in IBD as before.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PGhWvdBSgt6UxxHANr7gfN
 2026-08-30 17:26:21 -05:00
DanSandClaude Opus 4.8 3b2aa866aa rpc: honor a command-line rpcpassword across restarts; fix dead -rpcusername key 
hush_configfile(), on any restart where the auto-generated DRAGONX.conf
already exists, hard-assigned mapArgs["-rpcpassword"] from the conf and
wrote the username to mapArgs["-rpcusername"] -- a key nothing reads
(InitRPCAuthentication in httprpc.cpp and bitcoin-cli.cpp both read
"-rpcuser"). The hard assignment silently overwrote a -rpcpassword passed
on the command line, so after the first run the effective RPC credentials
became {cmdline-user}:{conf-password}, matching neither the command-line
pair the operator passed nor the full conf pair. Automation or external
clients that connect with the known command-line password broke on every
restart.

Use SoftSetArg for both, so a command-line (or explicitly configured)
value wins and the conf-derived credential is only a fallback. Verified on
regtest: after a restart an external client with the command-line password
gets HTTP 200 and the conf's random password gets 401; the conf-only
operator path (no command-line creds) still authenticates.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PGhWvdBSgt6UxxHANr7gfN
 2026-08-30 16:32:20 -05:00
DanSandClaude Opus 4.8 2d6359ea74 gtest: add job_id parser regression test for the 63-char abort fix 
Covers b3e81f1ed: a 63-character mining.submit job_id containing any
non-hex byte must never complete to a 32-byte vector, so the EWBF
completion loop skips it instead of constructing uint256() (which
asserts vch.size()==32 and would abort the daemon from an
unauthenticated client).

Five cases pin the invariant using the real ParseHex/uint256 primitives:
all-spaces, a non-hex byte mid-string, and a non-hex byte at the end
never yield 32 bytes; a genuine 63-hex job_id completes to exactly 32
bytes for all 16 digits; plus ParseHex odd/even-length anchors. Wired
gtest/test_stratum_jobid.cpp into Makefile.gtest.include (5 tests, all
pass; full hush-gtest suite now 18/18).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PGhWvdBSgt6UxxHANr7gfN
 2026-08-30 16:24:55 -05:00
52 changed files with 3455 additions and 409 deletions
Expand all files Collapse all files
configure.ac
+1 -1
View File
@@ -2,7 +2,7 @@ dnl require autoconf 2.60 (AS_ECHO/AS_ECHO_N)
AC_PREREQ([2.60])
define(_CLIENT_VERSION_MAJOR, 1)
dnl Must be kept in sync with src/clientversion.h , ugh!
define(_CLIENT_VERSION_MINOR, 2)
define(_CLIENT_VERSION_MINOR, 3)
define(_CLIENT_VERSION_REVISION, 0)
define(_CLIENT_VERSION_BUILD, 50)
define(_ZC_BUILD_VAL, m4_if(m4_eval(_CLIENT_VERSION_BUILD < 25), 1, m4_incr(_CLIENT_VERSION_BUILD), m4_eval(_CLIENT_VERSION_BUILD < 50), 1, m4_eval(_CLIENT_VERSION_BUILD - 24), m4_eval(_CLIENT_VERSION_BUILD == 50), 1, , m4_eval(_CLIENT_VERSION_BUILD - 50)))
contrib/debian/changelog
+28
View File
@@ -1,3 +1,31 @@
dragonx (1.3.0) stable; urgency=medium
* RandomX stratum mining support: the daemon can serve stratum clients
directly, with a reference miner behind the stratummine RPC for testing.
* Stratum fixes found by audit: a malformed 63-character job_id no longer
aborts the daemon; each block is paid to the miner that actually found it
rather than to whichever client asked for work first; and a low-difficulty
share is now rejected before it costs a RandomX hash.
* Fix -connect never dialing its targets, so a node pinned to specific peers
reaches them instead of silently falling through to peer discovery.
* Remove roughly 7,100 lines of dead code, including the CBOPRET price
validation in the coinbase check, whose guard could not be true on any
chain, and adaptive-PoW difficulty logic that DragonX does not enable.
* Honor a command-line -rpcpassword across restarts, and fix a misspelled
-rpcusername key that silently discarded the configured RPC user.
* Default -checkpoints off on regtest so an isolated node leaves initial
block download, instead of staying in IBD forever and disabling every
operation gated on it.
* Release cs_main and the mempool lock on the miner's isStake error paths;
the leak presented as a permanent stall rather than a slow response.
* Windows cross-build: the mingw target links and finds librustzcash, and a
fs::path::c_str() regression in init no longer breaks the build.
* Stamp container builds with the real version instead of "-unk".
* Repair the qa/rpc-tests harness far enough to start a DragonX node and
build the shared test chain; it previously started mainnet nodes.
-- DragonX Developers <dev@dragonx.is> Mon, 31 Aug 2026 03:46:19 +0000
dragonx (1.2.0) stable; urgency=medium
* Auto-shield matured coinbase into a wallet-owned Sapling address on a block
doc/man/dragonx-cli.1
+3 -3
View File
@@ -1,9 +1,9 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.1.
.TH DRAGONX-CLI "1" "August 2026" "dragonx-cli v1.2.0" "User Commands"
.TH DRAGONX-CLI "1" "August 2026" "dragonx-cli v1.3.0" "User Commands"
.SH NAME
dragonx-cli \- manual page for dragonx-cli v1.2.0
dragonx-cli \- manual page for dragonx-cli v1.3.0
.SH DESCRIPTION
DragonX RPC client version v1.2.0
DragonX RPC client version v1.3.0\-af7d9e230
.PP
In order to ensure you are adequately protecting your privacy when using
DragonX, please see <https://dragonx.is/security/>.
doc/man/dragonx-tx.1
+3 -3
View File
@@ -1,9 +1,9 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.1.
.TH DRAGONX-TX "1" "August 2026" "dragonx-tx v1.2.0" "User Commands"
.TH DRAGONX-TX "1" "August 2026" "dragonx-tx v1.3.0" "User Commands"
.SH NAME
dragonx-tx \- manual page for dragonx-tx v1.2.0
dragonx-tx \- manual page for dragonx-tx v1.3.0
.SH DESCRIPTION
hush\-tx utility version v1.2.0
hush\-tx utility version v1.3.0\-af7d9e230
.SS "Usage:"
.TP
hush\-tx [options] <hex\-tx> [commands]
doc/man/dragonxd.1
+15 -3
View File
@@ -1,9 +1,9 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.49.1.
.TH DRAGONXD "1" "August 2026" "dragonxd v1.2.0" "User Commands"
.TH DRAGONXD "1" "August 2026" "dragonxd v1.3.0" "User Commands"
.SH NAME
dragonxd \- manual page for dragonxd v1.2.0
dragonxd \- manual page for dragonxd v1.3.0
.SH DESCRIPTION
DragonX Daemon version v1.2.0
DragonX Daemon version v1.3.0\-af7d9e230
.PP
In order to ensure you are adequately protecting your privacy when using
DragonX, please see <https://dragonx.is/security/>.
@@ -452,6 +452,13 @@ or create a wallet z\-address). Must be spendable by this wallet.
Fee in puposhis for automatic coinbase\-shielding transactions (default:
10000)
.HP
\fB\-sietch\-min\-zouts=\fR<n>
.IP
Minimum number of shielded (Sapling) outputs Sietch adds to each
z_sendmany transaction as decoys, strengthening
amount/linkability privacy. Higher values add privacy at the cost
of larger transactions (default: 7, clamped to the range 3\-50)
.HP
\fB\-autoshieldminutxos\fR
.IP
Only auto\-shield once at least this many matured coinbase UTXOs exist
@@ -710,6 +717,11 @@ Stratum server options:
.IP
Enable stratum server (default: off)
.HP
\fB\-stratumtarget=\fR<hex>
.IP
Pool share target (64\-hex, big\-endian; larger = easier). Default is the
diff\-1 target. Useful for solo/low\-difficulty mining.
.HP
\fB\-stratumaddress=\fR<address>
.IP
Mining address to use when special address of 'x' is sent by miner
doc/release-process.md
+27 -16
View File
@@ -8,32 +8,32 @@ It is best to keep doc/relnotes/README.md up to date as changes and bug fixes ar
## Branch model
Development happens on the `dev` branch. Releases are cut on the default branch, `dragonx`. There is no `master` branch. Code changes should land on `dev` first and undergo testing before being merged into `dragonx`.
Development happens on the `dev` branch. Releases are cut on the default branch, `master`. Code changes should land on `dev` first and undergo testing before being merged into `master`.
## Check for changes on dragonx that should be on dev
## Check for changes on master that should be on dev
Occasionally trivial changes are made directly on the `dragonx` branch, such as documentation changes. In theory, no code changes should happen on `dragonx` without being on `dev` first, but it's better to be safe than sorry. We want the `dev` branch which undergoes testing to be as close as possible to what the `dragonx` branch will become, so we don't want to merge `dev` into `dragonx` and just assume everything works. So it's best to merge the `dragonx` branch into `dev` just before merging the `dev` branch into `dragonx`.
Occasionally trivial changes are made directly on the `master` branch, such as documentation changes. In theory, no code changes should happen on `master` without being on `dev` first, but it's better to be safe than sorry. We want the `dev` branch which undergoes testing to be as close as possible to what the `master` branch will become, so we don't want to merge `dev` into `master` and just assume everything works. So it's best to merge the `master` branch into `dev` just before merging the `dev` branch into `master`.
To check if the `dragonx` branch has any changes that the `dev` branch does not:
To check if the `master` branch has any changes that the `dev` branch does not:
```
# this assumes you are working with https://git.dragonx.is/DragonX/dragonx as your remote
git checkout dev
git pull # make sure dev is up to date
git checkout dragonx
git pull # make sure dragonx is up to date
git diff dev...dragonx # look at the set of changes which exist in dragonx but not dev
git checkout master
git pull # make sure master is up to date
git diff dev...master # look at the set of changes which exist in master but not dev
```
If the last command has no output, congrats, there is nothing to do. If the last command has output, then you should merge `dragonx` into `dev`:
If the last command has no output, congrats, there is nothing to do. If the last command has output, then you should merge `master` into `dev`:
```
git checkout dev
git merge dragonx
git merge master
git push origin dev
```
Use the `--no-ff` flag when merging `dev` into `dragonx` for a release (see below). The `--no-ff` flag makes sure to make a merge commit, no matter what, even if a "fast forward" could be done. For those in the future looking back, it's much better to see evidence of when branches were merged.
Use the `--no-ff` flag when merging `dev` into `master` for a release (see below). The `--no-ff` flag makes sure to make a merge commit, no matter what, even if a "fast forward" could be done. For those in the future looking back, it's much better to see evidence of when branches were merged.
### Git Issues
@@ -66,6 +66,7 @@ Install deps on Linux:
- Run "make seeds"
- Commit the result
- Update version in configure.ac and src/clientversion.h to update the dragonxd version
- **The new version MUST be higher than every version already tagged**, including tags that were never built or published. Check with `git tag -l --sort=-v:refname | head`. Two trees stamped with the same `CLIENT_VERSION` are indistinguishable on the wire, in `getnetworkinfo`, and to the wallet's in-app updater — and a published archive that does not match its tag destroys the only provenance check users have.
- In src/clientversion.h you update `CLIENT_VERSION_*` variables. Usually you will just update `CLIENT_VERSION_REVISION`
- If there is a consensus change, it may be a good idea to update `CLIENT_VERSION_MINOR` or `CLIENT_VERSION_MAJOR`
- To make a pre-release "beta" you can modify `CLIENT_VERSION_BUILD` but that is rarely done.
@@ -97,17 +98,27 @@ Install deps on Linux:
- Try to generate checkpoints as close to the release as possible, so you can have a recent block height be protected.
- For instance, don't update checkpoints and then do a release a month later. You can always update checkpoint data again or multiple times
- Update doc/relnotes/README.md
- To get the stats of file changes: `git diff --stat dragonx...dev`
- To get the stats of file changes: `git diff --stat master...dev`
- Do a fresh clone and fresh sync with new checkpoints
- Stop node, wait 20 minutes, and then do a partial sync with new checkpoints
- Merge dev into dragonx: `git checkout dev && git pull && git checkout dragonx && git pull && git merge --no-ff dev && git push`
- Merge dev into master: `git checkout dev && git pull && git checkout master && git pull && git merge --no-ff dev && git push`
- The above command makes sure that your local dev branch is up to date before doing anything
- The above command will not merge if "git pull" creates a merge conflict
- The above command will not push if there is a problem with merging dev
- Make Gitea release with git tag from the dragonx branch (make sure to merge dev in first)
- Make Gitea release with git tag from the master branch (make sure to merge dev in first)
- Make sure git tag starts with a `v` such as `v1.0.3`
- Use util/gen-linux-binary-release.sh to make a Linux release binary
- Upload Linux binary to Gitea release and add SHA256 sum
- **The tag MUST be annotated** (`git tag -a v1.3.0 -m 'DragonX v1.3.0'`), not lightweight. `util/genbuild.sh` calls `git describe` *without* `--tags`, which only ever sees annotated tags — a lightweight tag makes the build stamp itself `v<older-tag>-<sha>` instead of the release version. v1.0.0 through v1.0.3 are lightweight, which is why their builds are labelled that way.
- Verify before building: `git describe` must print exactly the tag, with no `-<n>-g<sha>` suffix.
- Use `./build.sh` (container-based, see doc/build-containers.md) or util/gen-linux-binary-release.sh to make a Linux release binary
- **Sign every archive and publish the signatures.** This step is mandatory and was missing from this document until v1.3.0 — its absence is why v1.1.0 and v1.2.0 were tagged but never became installable releases.
- The wallet's in-app daemon updater pins an ed25519 public key in `ObsidianDragon/src/util/daemon_updater.h` and sets `kDaemonRequireSignature = true`. **An update is refused outright unless a valid `<archive>.sig` is published beside the archive.** No signature means every existing user silently stays on their old daemon.
- Sign with `ObsidianDragon/scripts/sign-daemon-release.sh`:
- `scripts/sign-daemon-release.sh sign <secret.key> <archive>...` produces `<archive>.sig` (base64 of a detached 64-byte ed25519 signature over the exact archive bytes)
- or `scripts/sign-daemon-release.sh release <secret.key> <version>` to zip, sign, and print the checksum table in one step
- Keep the secret key offline, mode 600. The matching base64 public key must already be pinned in `kDaemonSignaturePublicKeyBase64`.
- Upload each Linux binary archive **and its `.sig`** to the Gitea release
- **Paste the SHA-256 checksum table into the release body** as markdown rows of the form `| <archive>.zip | `<sha256hex>` |`. The updater parses this table and will not install an archive that is absent from it.
- Confirm the release is actually consumable before announcing it: the updater looks for an asset whose name contains `"-" + platformToken + ".zip"` (`linux-amd64`, `macos`, `win64`). An archive named for a distro variant instead of the platform token is invisible to it.
- Create an x86 Debian package for the release:
- Edit contrib/debian/changelog to add information about the new release
- Use `util/build-debian-package.sh` to make an x86 Debian package for the release
@@ -115,7 +126,7 @@ Install deps on Linux:
- `lintian` is an optional dependency, it's not needed to build the .deb
- Upload .deb to Gitea release
- Add SHA256 checksum of .deb to release
- Use util/build-debian-package-ARM.sh (does this still work?) to make an ARM Debian package for the release
- ARM Debian package: `util/build-debian-package-ARM.sh` is referenced here historically but **is not present in the tree**. Skip, or restore the script first.
- Upload the debian packages to the Gitea release page, with SHA256 sums
## Platform-specific notes
qa/r4-salvaged-wallet-harness.sh
Executable
+87
View File
@@ -0,0 +1,87 @@
#!/usr/bin/env bash
# r4 now-tier acceptance harness. MUST run inside a network namespace: the v1.0.x binaries
# predate the regtest seed-injection fix and would otherwise dial the live DragonX network.
set -u
OLD=/home/dev/dragonx/release/dragonx-1.0.1-linux-amd64
NEW=/home/dev/dragonx-dev/src
ROOT=/tmp/claude-1000/-home-dev/45a644d6-ea0b-4b7c-8ec7-6ccb1a64afa7/scratchpad/r4lab
PASS=0; FAIL=0
ok(){ echo " PASS $1"; PASS=$((PASS+1)); }
no(){ echo " FAIL $1"; FAIL=$((FAIL+1)); }
# --- hard guardrail: refuse to run unisolated ---
if [ "$(ip route show 2>/dev/null | wc -l)" != "0" ]; then
echo "REFUSING: not in an isolated netns (routes present). Run under: unshare -rn"; exit 90
fi
ip link set lo up 2>/dev/null
echo "isolation: $(ip route show | wc -l) routes, $(ip -o link | wc -l) interface(s)"
conf(){ printf 'regtest=1\nrpcuser=t\nrpcpassword=t\nlisten=0\ndnsseed=0\n' > "$1/DRAGONX.conf"; }
start(){ # $1=bindir $2=datadir $3=extra
"$1/dragonxd" -regtest -datadir="$2" -connect=0 -listen=0 -dnsseed=0 $3 -daemon >/dev/null 2>&1
for i in $(seq 40); do "$1/dragonx-cli" -regtest -datadir="$2" -rpcuser=t -rpcpassword=t getblockcount >/dev/null 2>&1 && return 0; sleep 2; done
return 1; }
cli(){ "$1/dragonx-cli" -regtest -datadir="$2" -rpcuser=t -rpcpassword=t "${@:3}" 2>&1; }
stopn(){ cli "$1" "$2" stop >/dev/null 2>&1; sleep 6; }
rm -rf "$ROOT"; mkdir -p "$ROOT"
echo; echo "### build victim wallets with the OLD binary (v1.0.1) ###"
mkdir -p "$ROOT/base/regtest"; conf "$ROOT/base/regtest"
start "$OLD" "$ROOT/base/regtest" "" || { echo "old node failed to start"; exit 91; }
[ "$(cli "$OLD" "$ROOT/base/regtest" getconnectioncount)" = "0" ] && ok "victim-maker has 0 peers (isolated)" || no "victim-maker NOT isolated -- ABORT"
cli "$OLD" "$ROOT/base/regtest" getnewaddress >/dev/null
cli "$OLD" "$ROOT/base/regtest" z_getnewaddress >/dev/null
ZBEFORE=$(cli "$OLD" "$ROOT/base/regtest" z_listaddresses | tr -d ' \n')
stopn "$OLD" "$ROOT/base/regtest"
for v in C A B; do cp -a "$ROOT/base" "$ROOT/$v"; done
# A = salvaged by v1.0.1 (drops hdchain). B = A then USED on v1.0.1 (persists a bogus chain).
start "$OLD" "$ROOT/A/regtest" "-salvagewallet" && stopn "$OLD" "$ROOT/A/regtest"
start "$OLD" "$ROOT/B/regtest" "-salvagewallet" && stopn "$OLD" "$ROOT/B/regtest"
start "$OLD" "$ROOT/B/regtest" "" && { cli "$OLD" "$ROOT/B/regtest" z_getnewaddress >/dev/null; stopn "$OLD" "$ROOT/B/regtest"; }
echo " A hdchain records: $(strings "$ROOT/A/regtest/regtest/wallet.dat" | grep -c hdchain) (expect 0)"
echo " B hdchain records: $(strings "$ROOT/B/regtest/regtest/wallet.dat" | grep -c hdchain) (expect >=1)"
echo; echo "### open each on the NEW binary ###"
for v in C A B; do
D="$ROOT/$v/regtest"; L="$D/regtest/debug.log"
WDAT="$D/regtest/wallet.dat"
[ -f "$WDAT" ] || { echo " FAIL $v: wallet.dat not found at $WDAT"; exit 92; }
# NOT a byte-identical check: normal startup (keypool top-up, bestblock) rewrites wallet.dat for
# ANY wallet, healthy ones included -- verified with a control. The precise claim is that the
# degraded path never SYNTHESISES an hdchain record, so count that instead.
HD1=$(strings "$WDAT" | grep -c hdchain)
: > "$L" 2>/dev/null
if start "$NEW" "$D" "-exportdir=$D/exp"; then
STARTED=yes; DEG=$(grep -c "DEGRADED" "$L" 2>/dev/null)
MISS=$(grep -c "hdchain record is missing" "$L" 2>/dev/null)
MISM=$(grep -c "does not belong to this wallet" "$L" 2>/dev/null)
if [ "$v" = "A" ]; then
mkdir -p "$D/exp"; EXP=$(cli "$NEW" "$D" z_exportwallet r4dump 2>&1 | head -1)
DUMP=$(find "$D" -name 'r4dump' 2>/dev/null | head -1)
ZNEW=$(cli "$NEW" "$D" z_getnewaddress); TNEW=$(cli "$NEW" "$D" getnewaddress)
fi
stopn "$NEW" "$D"
else STARTED=no; DEG=0; MISS=0; MISM=0; fi
HD2=$(strings "$WDAT" | grep -c hdchain)
case $v in
C) [ "$STARTED" = yes ] && ok "C healthy wallet opens" || no "C healthy wallet failed to open"
[ "$DEG" = "0" ] && ok "C no false positive (not flagged degraded)" || no "C FALSE POSITIVE: healthy wallet flagged" ;;
A) [ "$STARTED" = yes ] && ok "A salvaged wallet opens (was DB_CORRUPT before)" || no "A salvaged wallet still refuses to open"
[ "$MISS" -ge 1 ] && ok "A flagged: hdchain missing" || no "A not flagged as missing-hdchain"
[ "$HD1" = "0" ] && [ "$HD2" = "0" ] && ok "A no hdchain synthesised (degraded path persists nothing)" || no "A hdchain record appeared ($HD1 -> $HD2)"
echo "$ZNEW" | grep -qi 'error' && ok "A z_getnewaddress refused cleanly (derivation gated)" || no "A z_getnewaddress derived anyway: $ZNEW"
echo "$TNEW" | grep -qiE '^R[a-zA-Z0-9]+$' && ok "A getnewaddress still works (legacy random t-key)" || no "A getnewaddress broke: $TNEW"
if [ -n "${DUMP:-}" ] && [ -f "$DUMP" ]; then
grep -qE '^# HDSeed=[0-9a-f]' "$DUMP" && no "E z_exportwallet emitted an HDSeed line on a degraded wallet" || ok "E z_exportwallet emitted no bogus HDSeed line"
else echo " SKIP E (no dump produced: $EXP)"; fi ;;
B) [ "$STARTED" = yes ] && ok "B poisoned wallet opens" || no "B poisoned wallet failed to open"
[ "$MISM" -ge 1 ] && ok "B CASE-3 DETECTOR FIRED (seedFp mismatch)" || no "B case-3 detector did NOT fire" ;;
esac
done
echo; echo "### $PASS passed, $FAIL failed ###"
exit $FAIL
qa/rpc-tests/getblocktemplate.py
+1 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2016 The Zcash developers
# Distributed under the GPLv3 software license, see the accompanying
qa/rpc-tests/getblocktemplate_longpoll.py
+2 -2
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2014 The Bitcoin Core developers
# Distributed under the GPLv3 software license, see the accompanying
@@ -52,7 +52,7 @@ class GetBlockTemplateLPTest(BitcoinTestFramework):
'''
def run_test(self):
print "Warning: this test will take about 70 seconds in the best case. Be patient."
print("Warning: this test will take about 70 seconds in the best case. Be patient.")
self.nodes[0].generate(10)
templat = self.nodes[0].getblocktemplate()
longpollid = templat['longpollid']
qa/rpc-tests/getblocktemplate_proposals.py
+74 -9
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2014 The Bitcoin Core developers
# Distributed under the GPLv3 software license, see the accompanying
@@ -6,6 +6,7 @@
from test_framework.test_framework import BitcoinTestFramework
from test_framework.authproxy import JSONRPCException
from test_framework.util import initialize_chain_clean, start_node
from binascii import a2b_hex, b2a_hex
from hashlib import sha256
@@ -69,14 +70,43 @@ def genmrklroot(leaflist):
cur = n
return cur[0]
# ---------------------------------------------------------------------------
# Sapling v4 transaction layout.
#
# This test was written against the pre-Overwinter serialization, where a tx
# began with a 4-byte nVersion immediately followed by the vin count, so the
# first input's prevout hash lived at offset 4+1. DragonX transactions are
# Sapling v4: 4-byte header (nVersion | fOverwintered) + 4-byte nVersionGroupId
# + vin count, so the prevout hash starts 4 bytes further in. Poking the old
# offset corrupts nVersionGroupId and every proposal below just comes back
# "Block decode failed" instead of exercising any consensus rule.
CB_PREVOUT_OFF = 4+4+1
# Likewise the tx no longer ends at nLockTime: nExpiryHeight (4), valueBalance
# (8) and the empty vShieldedSpend/vShieldedOutput/vJoinSplit counts (1 each)
# trail it, so nLockTime is the 4 bytes at [-19:-15].
TX_TAIL_AFTER_LOCKTIME = 4+8+1+1+1
def tx_seq_off(tx):
"""Offset of the first input's nSequence in a Sapling v4 tx."""
scriptlen_off = CB_PREVOUT_OFF + 32 + 4 # after prevout hash + prevout.n
return scriptlen_off + 1 + tx[scriptlen_off]
def tx_vout0_value_off(tx):
"""Offset of the first output's 8-byte value in a Sapling v4 tx."""
return tx_seq_off(tx) + 4 + 1 # after nSequence + vout count
def template_to_bytes(tmpl, txlist):
blkver = pack('<L', tmpl['version'])
mrklroot = genmrklroot(list(dblsha(a) for a in txlist))
reserved = b'\0'*32
# hashFinalSaplingRoot. The all-zeroes placeholder this test used predates
# Sapling; a header carrying the wrong root is rejected with
# 'bad-sapling-root-in-block', which would sink even Test 11 (valid block).
reserved = a2b_hex(tmpl['finalsaplingroothash'])[::-1]
timestamp = pack('<L', tmpl['curtime'])
nonce = b'\0'*32
soln = b'\0'
blk = blkver + a2b_hex(tmpl['previousblockhash'])[::-1] + mrklroot + reserved + timestamp + a2b_hex(tmpl['bits'])[::-1] + nonce + soln
# bytearray, not bytes: Test 9 mutates one byte of the result in place.
blk = bytearray(blkver + a2b_hex(tmpl['previousblockhash'])[::-1] + mrklroot + reserved + timestamp + a2b_hex(tmpl['bits'])[::-1] + nonce + soln)
blk += varlenEncode(len(txlist))
for tx in txlist:
blk += tx
@@ -95,6 +125,20 @@ class GetBlockTemplateProposalTest(BitcoinTestFramework):
Test block proposals with getblocktemplate.
'''
def setup_chain(self):
print("Initializing test directory "+self.options.tmpdir)
initialize_chain_clean(self.options.tmpdir, 1)
def setup_network(self, split=False):
# -daaforkheight=0: ContextualCheckBlockHeader only enforces nBits for a
# smart chain above daaForkHeight, which defaults to
# ASSETCHAINS_RANDOMX_VALIDATION+62000 (millions of blocks) so that a
# fresh sync accepts DragonX's historical bad-nBits window. Without this
# flag Test 8 (bad bits) is unreachable at regtest heights and the
# daemon happily accepts a block with arbitrary nBits.
self.nodes = [ start_node(0, self.options.tmpdir, ['-daaforkheight=0']) ]
self.is_network_split = False
def run_test(self):
node = self.nodes[0]
node.generate(1) # Mine a block to leave initial block download
@@ -117,9 +161,9 @@ class GetBlockTemplateProposalTest(BitcoinTestFramework):
#txlist[0][4+1+36+1+1] -= 1
# Test 2: Bad input hash for gen tx
txlist[0][4+1] += 1
txlist[0][CB_PREVOUT_OFF] += 1
assert_template(node, tmpl, txlist, 'bad-cb-missing')
txlist[0][4+1] -= 1
txlist[0][CB_PREVOUT_OFF] -= 1
# Test 3: Truncated final tx
lastbyte = txlist[-1].pop()
@@ -136,14 +180,33 @@ class GetBlockTemplateProposalTest(BitcoinTestFramework):
# Test 5: Add an invalid tx to the end (non-duplicate)
txlist.append(bytearray(txlist[0]))
txlist[-1][4+1] = b'\xff'
txlist[-1][CB_PREVOUT_OFF] = 0xff
# DragonX is a fully private chain (ASSETCHAINS_PRIVATE): a non-coinbase
# tx paying a positive amount to a t-addr is killed in CheckTransaction
# with 'bad-txns-acprivacy-chain' before ConnectBlock ever looks up its
# inputs. Zero the output value -- CheckTransaction exempts zero-value
# vouts -- so the tx survives to the missing-input check this case is
# actually about.
_val_off = tx_vout0_value_off(txlist[-1])
txlist[-1][_val_off:_val_off+8] = b'\0'*8
assert_template(node, tmpl, txlist, 'bad-txns-inputs-missingorspent')
txlist.pop()
# Test 6: Future tx lock time
txlist[0][-4:] = b'\xff\xff\xff\xff'
# The server-supplied coinbase carries nSequence=0xffffffff, and
# IsFinalTx() short-circuits on all-final inputs, so a future nLockTime
# alone leaves the tx final and the block valid. (DragonX's IsFinalTx
# also whitelists 0xfffffffe below the Hush hardfork height, which
# regtest is.) Make the input genuinely non-final so the future
# nLockTime is the thing under test.
seq_off = tx_seq_off(txlist[0])
realseq = txlist[0][seq_off:seq_off+4]
txlist[0][seq_off:seq_off+4] = b'\0\0\0\0'
reallocktime = txlist[0][-TX_TAIL_AFTER_LOCKTIME-4:-TX_TAIL_AFTER_LOCKTIME]
txlist[0][-TX_TAIL_AFTER_LOCKTIME-4:-TX_TAIL_AFTER_LOCKTIME] = b'\xff\xff\xff\xff'
assert_template(node, tmpl, txlist, 'bad-txns-nonfinal')
txlist[0][-4:] = b'\0\0\0\0'
txlist[0][-TX_TAIL_AFTER_LOCKTIME-4:-TX_TAIL_AFTER_LOCKTIME] = reallocktime
txlist[0][seq_off:seq_off+4] = realseq
# Test 7: Bad tx count
txlist.append(b'')
@@ -171,7 +234,9 @@ class GetBlockTemplateProposalTest(BitcoinTestFramework):
tmpl['curtime'] = 0x7fffffff
assert_template(node, tmpl, txlist, 'time-too-new')
tmpl['curtime'] = 0
assert_template(node, tmpl, txlist, 'time-too-old')
# DragonX split Bitcoin's 'time-too-old' into 'time-too-old-median'
# (block time <= prev MedianTimePast) and 'time-too-old-prevblock'.
assert_template(node, tmpl, txlist, 'time-too-old-median')
tmpl['curtime'] = realtime
# Test 11: Valid block
qa/rpc-tests/reindex.py
+2 -2
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2014 The Bitcoin Core developers
# Copyright (c) 2016-2024 The Hush developers
# Released under the GPLv3
@@ -28,7 +28,7 @@ class ReindexTest(BitcoinTestFramework):
wait_bitcoinds()
self.nodes[0]=start_node(0, self.options.tmpdir, ["-debug", "-reindex", "-checkblockindex=1"])
assert_equal(self.nodes[0].getblockcount(), 3)
print "Success"
print("Success")
if __name__ == '__main__':
ReindexTest().main()
qa/rpc-tests/test_framework/blockstore.py
+1 -1
View File
@@ -7,7 +7,7 @@
# and for constructing a getheaders message
#
from mininode import CBlock, CBlockHeader, CBlockLocator, CTransaction, msg_block, msg_headers, msg_tx
from .mininode import CBlock, CBlockHeader, CBlockLocator, CTransaction, msg_block, msg_headers, msg_tx
import sys
import cStringIO
qa/rpc-tests/test_framework/blocktools.py
+2 -2
View File
@@ -3,8 +3,8 @@
# Distributed under the GPLv3 software license, see the accompanying
# file COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html
# blocktools.py - utilities for manipulating blocks and transactions
from mininode import CBlock, CTransaction, CTxIn, CTxOut, COutPoint
from script import CScript, OP_0, OP_EQUAL, OP_HASH160
from .mininode import CBlock, CTransaction, CTxIn, CTxOut, COutPoint
from .script import CScript, OP_0, OP_EQUAL, OP_HASH160
# Create a block (with regtest difficulty)
def create_block(hashprev, coinbase, nTime=None, nBits=None):
qa/rpc-tests/test_framework/comptool.py
+3 -3
View File
@@ -3,10 +3,10 @@
# Distributed under the GPLv3 software license, see the accompanying
# file COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html
from mininode import CBlock, CTransaction, CInv, NodeConn, NodeConnCB, \
from .mininode import CBlock, CTransaction, CInv, NodeConn, NodeConnCB, \
msg_inv, msg_getheaders, msg_ping, msg_mempool, mininode_lock, MAX_INV_SZ
from blockstore import BlockStore, TxStore
from util import p2p_port
from .blockstore import BlockStore, TxStore
from .util import p2p_port
import time
qa/rpc-tests/test_framework/test_framework.py
+3 -3
View File
@@ -11,8 +11,8 @@ import shutil
import tempfile
import traceback
from authproxy import JSONRPCException
from util import assert_equal, check_json_precision, \
from .authproxy import JSONRPCException
from .util import assert_equal, check_json_precision, \
initialize_chain, initialize_chain_clean, \
start_nodes, connect_nodes_bi, stop_nodes, \
sync_blocks, sync_mempools, wait_bitcoinds
@@ -91,7 +91,7 @@ class BitcoinTestFramework(object):
parser.add_option("--noshutdown", dest="noshutdown", default=False, action="store_true",
help="Don't stop nodes after the test execution")
parser.add_option("--srcdir", dest="srcdir", default="../../src",
help="Source directory containing hushd/hush-cli (default: %default)")
help="Source directory containing dragonxd/dragonx-cli (default: %default)")
parser.add_option("--tmpdir", dest="tmpdir", default=tempfile.mkdtemp(prefix="test"),
help="Root directory for datadirs")
parser.add_option("--tracerpc", dest="trace_rpc", default=False, action="store_true",
qa/rpc-tests/test_framework/util.py
+70 -20
View File
@@ -18,7 +18,7 @@ import subprocess
import time
import re
from authproxy import AuthServiceProxy
from .authproxy import AuthServiceProxy
def p2p_port(n):
return 11000 + n + os.getpid()%999
@@ -97,8 +97,8 @@ def initialize_datadir(dirname, n):
print("Creating dirs %s" % datadir)
os.makedirs(datadir)
print("Writing to " + os.path.join(datadir,"ZZZ.conf"))
with open(os.path.join(datadir, "ZZZ.conf"), 'w') as f:
print("Writing to " + os.path.join(datadir,"DRAGONX.conf"))
with open(os.path.join(datadir, "DRAGONX.conf"), 'w') as f:
f.write("regtest=1\n");
f.write("txindex=1\n");
#f.write("testnode=1\n");
@@ -116,7 +116,19 @@ def initialize_datadir(dirname, n):
f.write("spentindex=1\n");
f.write("timestampindex=1\n");
#f.write("zindex=1\n");
print("Done writing to %s" % os.path.join(datadir,"ZZZ.conf") )
print("Done writing to %s" % os.path.join(datadir,"DRAGONX.conf") )
# dragonxd refuses to start without an asmap file ("Could not find any asmap file!"),
# so every regtest datadir needs one. Link the tree's copy rather than duplicating it.
for src in ("../../../asmap.dat", "../../../src/asmap.dat",
os.path.expanduser("~/.hush/DRAGONX/asmap.dat")):
cand = src if os.path.isabs(src) else os.path.join(os.path.dirname(os.path.abspath(__file__)), src)
if os.path.exists(cand):
dst = os.path.join(datadir, "asmap.dat")
if not os.path.exists(dst):
try: os.symlink(os.path.realpath(cand), dst)
except OSError: shutil.copyfile(cand, dst)
break
return datadir
@@ -133,11 +145,23 @@ def initialize_chain(test_dir):
# Create cache directories, run hushds:
for i in range(4):
datadir=initialize_datadir("cache", i)
args = [ os.getenv("BITCOIND", "hushd"), "-keypool=1", "-datadir="+datadir, "-discover=0" ]
# Same two requirements as start_node(): -regtest must be a command-line flag (the
# conf key is ignored, and without it this cache node runs on MAINNET), and -asmap
# must be absolute or dragonxd refuses to start.
# -connect=<anything> makes init.cpp soft-set -listen=0 ("parameter interaction: -connect
# set -> setting -listen=0"), so cache node0 never opened its p2p port and nodes 1-3 could
# never sync to it -- initialize_chain() then hung forever in sync_blocks(). Pass -listen
# and -bind explicitly (an explicit arg beats SoftSetBoolArg) and keep the loopback bind so
# the cache nodes stay off the public network.
args = [ os.getenv("BITCOIND", "src/dragonxd"), "-regtest", "-connect=0", "-keypool=1", "-datadir="+datadir, "-discover=0",
"-listen=1", "-bind=127.0.0.1", "-dnsseed=0" ]
_am = os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../../asmap.dat")
if os.path.exists(_am):
args.append("-asmap=" + os.path.realpath(_am))
if i > 0:
args.append("-connect=127.0.0.1:"+str(p2p_port(0)))
bitcoind_processes[i] = subprocess.Popen(args)
cmd = os.getenv("BITCOINCLI", "hush-cli")
cmd = os.getenv("BITCOINCLI", "src/dragonx-cli")
cmd_args = cmd + " -datadir="+datadir + " -rpcwait getblockcount"
if os.getenv("PYTHON_DEBUG", ""):
print("initialize_chain: hushd started, calling: " + cmd_args)
@@ -180,10 +204,17 @@ def initialize_chain(test_dir):
wait_bitcoinds()
for i in range(4):
print("Cleaning up cache dir files")
os.remove(log_filename("cache", i, "debug.log"))
os.remove(log_filename("cache", i, "db.log"))
os.remove(log_filename("cache", i, "peers.dat"))
os.remove(log_filename("cache", i, "fee_estimates.dat"))
# log_filename() points at <cache>/node<i>/regtest, but that IS the -datadir we passed;
# dragonxd writes its logs/peers.dat one level deeper, into the net-specific
# <datadir>/regtest subdir (same datadir-vs-netdir split that forced -asmap to be
# absolute in start_node). Try both, and tolerate files a node never created.
for name in ("debug.log", "db.log", "peers.dat", "fee_estimates.dat"):
for cand in (log_filename("cache", i, os.path.join("regtest", name)),
log_filename("cache", i, name)):
try:
os.remove(cand)
except OSError:
pass
for i in range(4):
from_dir = os.path.join("cache", "node"+str(i))
@@ -227,9 +258,10 @@ def start_node(i, dirname, extra_args=None, rpchost=None, timewait=None, binary=
"""
print("Starting node " + str(i) + " in dir " + dirname)
datadir = os.path.join(dirname, "node"+str(i), "regtest")
if extra_args is None: extra_args = []
# creating special config
if len(extra_args) > 0 and extra_args[0] == '-ac_name=ZZZ':
configpath = datadir + "/ZZZ.conf"
configpath = datadir + "/DRAGONX.conf"
with open(configpath, "w+") as config:
config.write("rpcuser=hush\n")
config.write("rpcpassword=puppy\n")
@@ -247,16 +279,30 @@ def start_node(i, dirname, extra_args=None, rpchost=None, timewait=None, binary=
print("Done writing to %s" % configpath)
if binary is None:
binary = os.getenv("BITCOIND", "src/hushd")
args = [ binary, "-datadir="+datadir, "-keypool=1", "-discover=0", "-rest" ]
binary = os.getenv("BITCOIND", "src/dragonxd")
# -regtest MUST be a command-line flag. DragonX ignores "regtest=1" in the conf file, so
# without this the node silently runs on MAINNET: it loads the real genesis, dials the real
# seeds and starts syncing the live chain into the test datadir (observed: 196k blocks and
# 679MB before a test timed out). -connect=0 keeps the regtest node off the public network.
args = [ binary, "-regtest", "-connect=0", "-datadir="+datadir, "-keypool=1", "-discover=0", "-rest" ]
# -asmap relative paths are resolved against the NET-SPECIFIC datadir (init.cpp), which for
# regtest is <datadir>/regtest -- so a copy sitting in <datadir> is never found. Pass an
# absolute path; without it dragonxd exits with "Could not find any asmap file!".
_asmap = os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../../asmap.dat")
if os.path.exists(_asmap):
args.append("-asmap=" + os.path.realpath(_asmap))
if extra_args is not None: args.extend(extra_args)
print("args=" + ' '.join(args))
bitcoind_processes[i] = subprocess.Popen(args)
devnull = open("/dev/null", "w+")
cmd = os.getenv("BITCOINCLI", "src/hush-cli")
cmd = os.getenv("BITCOINCLI", "src/dragonx-cli")
print("cmd=" + cmd)
args = [ extra_args[0], "-datadir="+datadir, "-keypool=1", "-discover=0", "-rest" ]
# The CLI only needs the datadir: initialize_datadir() already wrote DRAGONX.conf there
# with the right rpcport/user/password. The old form passed extra_args[0] as argv[0] and
# replayed daemon-only flags at the CLI, which only worked for the -ac_name=ZZZ assetchain
# tests and broke every test that passes no extra_args.
args = [ "-regtest", "-datadir="+datadir ]
cmd_args = ' '.join(args) + " -rpcwait getblockcount "
if os.getenv("PYTHON_DEBUG", ""):
print("start_node: hushd started, calling : " + cmd + " " + cmd_args)
@@ -266,18 +312,22 @@ def start_node(i, dirname, extra_args=None, rpchost=None, timewait=None, binary=
import time
time.sleep(2)
subprocess.check_call(strcmd, shell=True);
#subprocess.check_call([ os.getenv("BITCOINCLI", "hush-cli"), "-datadir="+datadir] +
#subprocess.check_call([ os.getenv("BITCOINCLI", "dragonx-cli"), "-datadir="+datadir] +
# _rpchost_to_args(rpchost) +
# ["-rpcwait", "-rpcport=6438", "getblockcount"], stdout=devnull)
if os.getenv("PYTHON_DEBUG", ""):
print("start_node: calling hush-cli -rpcwait getblockcount returned")
devnull.close()
port = extra_args[3]
#port = rpc_port(i)
# Port comes from the same helper initialize_datadir() used, except for the assetchain
# tests which pass it positionally as extra_args[3] == "-rpcport=NNNN".
if len(extra_args) > 3 and str(extra_args[0]) == '-ac_name=ZZZ':
port = extra_args[3][9:]
else:
port = str(rpc_port(i))
#print("port=%s" % port)
username = rpc_username()
password = rpc_password()
url = "http://%s:%s@%s:%s" % (username, password, rpchost or '127.0.0.1', port[9:])
url = "http://%s:%s@%s:%s" % (username, password, rpchost or '127.0.0.1', port)
print("connecting to " + url)
if timewait is not None:
proxy = AuthServiceProxy(url, timeout=timewait)
@@ -315,7 +365,7 @@ def stop_nodes(nodes):
del nodes[:] # Emptying array closes connections as a side effect
def set_node_times(nodes, t):
print("Setting nodes time to " + t)
print("Setting nodes time to " + str(t))
for node in nodes:
node.setmocktime(t)
qa/rpc-tests/wallet.py
+35 -23
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2014 The Bitcoin Core developers
# Distributed under the GPLv3 software license, see the accompanying
@@ -20,8 +20,20 @@ class WalletTest (BitcoinTestFramework):
print("Initializing test directory "+self.options.tmpdir)
initialize_chain_clean(self.options.tmpdir, 4)
# PORT NOTE (DragonX/regtest, not a change of test intent):
# test_framework.start_node() hardcodes "-connect=0", and init.cpp turns that into
# "-connect set -> setting -listen=0". With listening off the nodes never bind their
# p2p port, so connect_nodes_bi() connects nothing at all (its version==0 poll loop
# exits immediately because there are no local peers) and the first sync_all() hangs
# forever. Passing -bind forces -listen back to 1. -dnsseed=0 keeps these regtest
# nodes from dialing the live DragonX network.
NET_ARGS = ["-listen=1", "-bind=127.0.0.1", "-dnsseed=0"]
def net_args(self, n, extra=None):
return [list(self.NET_ARGS) + list(extra or []) for _ in range(n)]
def setup_network(self, split=False):
self.nodes = start_nodes(3, self.options.tmpdir)
self.nodes = start_nodes(3, self.options.tmpdir, self.net_args(3))
connect_nodes_bi(self.nodes,0,1)
connect_nodes_bi(self.nodes,1,2)
connect_nodes_bi(self.nodes,0,2)
@@ -29,7 +41,7 @@ class WalletTest (BitcoinTestFramework):
self.sync_all()
def run_test (self):
print "Mining blocks..."
print("Mining blocks...")
self.nodes[0].generate(4)
self.sync_all()
@@ -106,7 +118,7 @@ class WalletTest (BitcoinTestFramework):
signed_tx = self.nodes[2].signrawtransaction(raw_tx)
try:
self.nodes[2].sendrawtransaction(signed_tx["hex"])
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert("absurdly high fees" in errorString)
assert("900000000 > 190000" in errorString)
@@ -186,7 +198,7 @@ class WalletTest (BitcoinTestFramework):
txid2 = self.nodes[1].sendtoaddress(self.nodes[0].getnewaddress(), 1)
sync_mempools(self.nodes)
self.nodes.append(start_node(3, self.options.tmpdir))
self.nodes.append(start_node(3, self.options.tmpdir, list(self.NET_ARGS)))
connect_nodes_bi(self.nodes, 0, 3)
sync_blocks(self.nodes)
@@ -227,7 +239,7 @@ class WalletTest (BitcoinTestFramework):
#do some -walletbroadcast tests
stop_nodes(self.nodes)
wait_bitcoinds()
self.nodes = start_nodes(3, self.options.tmpdir, [["-walletbroadcast=0"],["-walletbroadcast=0"],["-walletbroadcast=0"]])
self.nodes = start_nodes(3, self.options.tmpdir, self.net_args(3, ["-walletbroadcast=0"]))
connect_nodes_bi(self.nodes,0,1)
connect_nodes_bi(self.nodes,1,2)
connect_nodes_bi(self.nodes,0,2)
@@ -256,7 +268,7 @@ class WalletTest (BitcoinTestFramework):
#restart the nodes with -walletbroadcast=1
stop_nodes(self.nodes)
wait_bitcoinds()
self.nodes = start_nodes(3, self.options.tmpdir)
self.nodes = start_nodes(3, self.options.tmpdir, self.net_args(3))
connect_nodes_bi(self.nodes,0,1)
connect_nodes_bi(self.nodes,1,2)
connect_nodes_bi(self.nodes,0,2)
@@ -290,7 +302,7 @@ class WalletTest (BitcoinTestFramework):
num_t_recipients = 3000
amount_per_recipient = Decimal('0.00000001')
errorString = ''
for i in xrange(0,num_t_recipients):
for i in range(0,num_t_recipients):
newtaddr = self.nodes[2].getnewaddress()
recipients.append({"address":newtaddr, "amount":amount_per_recipient})
@@ -305,7 +317,7 @@ class WalletTest (BitcoinTestFramework):
try:
self.nodes[0].z_sendmany(myzaddr, recipients)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert("Too many outputs, size of raw transaction" in errorString)
@@ -314,10 +326,10 @@ class WalletTest (BitcoinTestFramework):
num_z_recipients = 50
amount_per_recipient = Decimal('0.00000001')
errorString = ''
for i in xrange(0,num_t_recipients):
for i in range(0,num_t_recipients):
newtaddr = self.nodes[2].getnewaddress()
recipients.append({"address":newtaddr, "amount":amount_per_recipient})
for i in xrange(0,num_z_recipients):
for i in range(0,num_z_recipients):
newzaddr = self.nodes[2].z_getnewaddress()
recipients.append({"address":newzaddr, "amount":amount_per_recipient})
@@ -327,7 +339,7 @@ class WalletTest (BitcoinTestFramework):
try:
self.nodes[0].z_sendmany(myzaddr, recipients)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert("size of raw transaction would be larger than limit" in errorString)
@@ -335,12 +347,12 @@ class WalletTest (BitcoinTestFramework):
num_z_recipients = 100
amount_per_recipient = Decimal('0.00000001')
errorString = ''
for i in xrange(0,num_z_recipients):
for i in range(0,num_z_recipients):
newzaddr = self.nodes[2].z_getnewaddress()
recipients.append({"address":newzaddr, "amount":amount_per_recipient})
try:
self.nodes[0].z_sendmany(myzaddr, recipients)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert("Invalid parameter, too many zaddr outputs" in errorString)
@@ -426,7 +438,7 @@ class WalletTest (BitcoinTestFramework):
errorString = ""
try:
txId = self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), "1f-4")
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Invalid amount" in errorString, True)
@@ -434,7 +446,7 @@ class WalletTest (BitcoinTestFramework):
errorString = ""
try:
self.nodes[0].generate("2") #use a string to as block amount parameter must fail because it's not interpreted as amount
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("not an integer" in errorString, True)
@@ -448,9 +460,9 @@ class WalletTest (BitcoinTestFramework):
try:
myopid = self.nodes[0].z_sendmany(myzaddr, recipients)
assert(myopid)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
print errorString
print(errorString)
assert(False)
# This fee is larger than the default fee and since amount=0
@@ -462,7 +474,7 @@ class WalletTest (BitcoinTestFramework):
try:
myopid = self.nodes[0].z_sendmany(myzaddr, recipients, minconf, fee)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert('Small transaction amount' in errorString)
@@ -475,9 +487,9 @@ class WalletTest (BitcoinTestFramework):
try:
myopid = self.nodes[0].z_sendmany(myzaddr, recipients, minconf, fee)
assert(myopid)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
print errorString
print(errorString)
assert(False)
# Make sure amount=0, fee=0 transaction are valid to add to mempool
@@ -490,9 +502,9 @@ class WalletTest (BitcoinTestFramework):
try:
myopid = self.nodes[0].z_sendmany(myzaddr, recipients, minconf, fee)
assert(myopid)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
print errorString
print(errorString)
assert(False)
qa/rpc-tests/wallet_listnotes.py
+1 -1
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2018 The Zcash developers
# Distributed under the GPLv3 software license, see the accompanying
qa/rpc-tests/wallet_mergetoaddress.py
+12 -12
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2017 The Zcash developers
# Distributed under the GPLv3 software license, see the accompanying
@@ -32,7 +32,7 @@ class WalletMergeToAddressTest (BitcoinTestFramework):
self.sync_all()
def run_test (self):
print "Mining blocks..."
print("Mining blocks...")
self.nodes[0].generate(1)
do_not_shield_taddr = self.nodes[0].getnewaddress()
@@ -81,7 +81,7 @@ class WalletMergeToAddressTest (BitcoinTestFramework):
try:
self.nodes[0].z_mergetoaddress("*", myzaddr)
assert(False)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("JSON value is not an array as expected" in errorString, True)
@@ -90,7 +90,7 @@ class WalletMergeToAddressTest (BitcoinTestFramework):
try:
self.nodes[2].z_mergetoaddress([mytaddr], myzaddr)
assert(False)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Could not find any funds to merge" in errorString, True)
@@ -98,7 +98,7 @@ class WalletMergeToAddressTest (BitcoinTestFramework):
try:
self.nodes[0].z_mergetoaddress(["*"], myzaddr, -1)
assert(False)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Amount out of range" in errorString, True)
@@ -106,7 +106,7 @@ class WalletMergeToAddressTest (BitcoinTestFramework):
try:
self.nodes[0].z_mergetoaddress(["*"], myzaddr, Decimal('21000000.00000001'))
assert(False)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Amount out of range" in errorString, True)
@@ -114,7 +114,7 @@ class WalletMergeToAddressTest (BitcoinTestFramework):
try:
self.nodes[0].z_mergetoaddress(["*"], myzaddr, 999)
assert(False)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Insufficient funds" in errorString, True)
@@ -122,7 +122,7 @@ class WalletMergeToAddressTest (BitcoinTestFramework):
try:
self.nodes[0].z_mergetoaddress(["*"], myzaddr, Decimal('0.001'), -1)
assert(False)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Limit on maximum number of UTXOs cannot be negative" in errorString, True)
@@ -130,7 +130,7 @@ class WalletMergeToAddressTest (BitcoinTestFramework):
try:
self.nodes[0].z_mergetoaddress(["*"], myzaddr, Decimal('0.001'), 99999999999999)
assert(False)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("JSON integer out of range" in errorString, True)
@@ -138,7 +138,7 @@ class WalletMergeToAddressTest (BitcoinTestFramework):
try:
self.nodes[0].z_mergetoaddress(["*"], myzaddr, Decimal('0.001'), 50, -1)
assert(False)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Limit on maximum number of notes cannot be negative" in errorString, True)
@@ -146,7 +146,7 @@ class WalletMergeToAddressTest (BitcoinTestFramework):
try:
self.nodes[0].z_mergetoaddress(["*"], myzaddr, Decimal('0.001'), 50, 99999999999999)
assert(False)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("JSON integer out of range" in errorString, True)
@@ -154,7 +154,7 @@ class WalletMergeToAddressTest (BitcoinTestFramework):
try:
self.nodes[0].z_mergetoaddress([mytaddr], mytaddr)
assert(False)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Destination address is also the only source address, and all its funds are already merged" in errorString, True)
qa/rpc-tests/wallet_nullifiers.py
+63 -7
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2016 The Zcash developers
# Distributed under the GPLv3 software license, see the accompanying
@@ -7,15 +7,71 @@
from test_framework.test_framework import BitcoinTestFramework
from test_framework.util import assert_equal, assert_true, bitcoind_processes, \
connect_nodes_bi, start_node, start_nodes, wait_and_assert_operationid_status
connect_nodes_bi, initialize_chain_clean, p2p_port, start_node, start_nodes, \
sync_blocks, wait_and_assert_operationid_status
from decimal import Decimal
class WalletNullifiersTest (BitcoinTestFramework):
# The framework default setup_chain() calls initialize_chain(), which pre-builds a
# 200-block chain in a *relative* "cache/" directory shared by every test process
# running out of this tree. That directory is guarded only by the datadir lock, so two
# qa/rpc-tests running at once collide on it and the second one hangs forever inside
# "dragonx-cli -rpcwait" (observed here while a sibling test held cache/node0..3).
# Build the identical pre-condition -- 4 nodes, two rounds of 25 blocks each, i.e. 25
# mature + 25 immature coinbases per node -- directly in this test's private tmpdir.
# This is a setup change only: no assertion below is relaxed, removed or reordered.
def setup_chain(self):
print("Initializing test directory "+self.options.tmpdir)
initialize_chain_clean(self.options.tmpdir, 4)
# Three networking facts about this daemon force extra flags here. None of them
# change what the test exercises; without them the 4 nodes either never peer with
# each other, or peer with the LIVE DragonX network instead.
#
# 1. start_node() hardcodes "-connect=0", which also soft-sets -listen=0, so nothing
# binds p2p_port(i) and connect_nodes_bi() can never form the regtest mesh --
# sync_blocks() then spins forever (observed: node0 at 25 blocks, nodes 1-3 stuck
# at 0, nothing listening on 11005-11008). -listen=1 -bind=127.0.0.1 restores the
# mesh and keeps it on loopback.
# 2. hush_args() appends node1..node10.dragonx.is to -addnode unconditionally, -regtest
# included, and regtest reuses mainnet's network magic. A "regtest" node therefore
# joins the live network: node0 of an earlier run handshook 8 production peers
# ("receive version message: /DragonX:1.0.3/ ... blocks=3254266") and ingested their
# headers. -dns=0 stops those hostname -addnode entries from resolving; RPC addnode
# with a numeric 127.0.0.1:port is unaffected.
# 3. hush_args() runs BEFORE the config file is read, so its GetArg("-port",0) never
# sees the "port=" line initialize_datadir() wrote and GetDefaultPort() stays at the
# mainnet p2p port. "-connect=0" is then parsed as the address 0.0.0.0:<mainnet
# port>, i.e. the production dragonxd listening on this box -- every node in runs 2
# and 3 picked up exactly one peer reporting blocks=3254269. Repeating -port on the
# command line points GetDefaultPort() at this node's own regtest port instead.
#
# -autoshield is on by default on DragonX and is not part of what this test measures:
# a background thread sweeps each node's matured coinbase into a seed-derived zaddr
# (8 "autoshield operation finished" ops per node while the chain is being mined). That
# empties the very taddr this test spends from, and the resulting transactions do not
# settle identically on every node ("ERROR: AcceptToMemoryPool: ContextualCheckTransaction
# failed" on node1), so sync_mempools() never converges and the run wedges until the
# timeout. Turn the background sweeper off; the test does its own shielding explicitly.
def net_args(self, i):
return ['-listen=1', '-bind=127.0.0.1', '-dns=0', '-autoshield=0',
'-port=%d' % p2p_port(i)]
def setup_nodes(self):
return start_nodes(4, self.options.tmpdir,
extra_args=[['-experimentalfeatures', '-developerencryptwallet']] * 4)
extra_args=[['-experimentalfeatures', '-developerencryptwallet']
+ self.net_args(i) for i in range(4)])
def setup_network(self, split = False):
super().setup_network(split)
# Same block layout initialize_chain() would have handed us.
for _ in range(2):
for peer in range(4):
self.nodes[peer].generate(25)
sync_blocks(self.nodes)
self.sync_all()
def run_test (self):
# add zaddr to node 0
@@ -25,7 +81,7 @@ class WalletNullifiersTest (BitcoinTestFramework):
mytaddr = self.nodes[0].getnewaddress()
recipients = []
recipients.append({"address":myzaddr0, "amount":Decimal('10.0')-Decimal('0.0001')}) # utxo amount less fee
wait_and_assert_operationid_status(self.nodes[0], self.nodes[0].z_sendmany(mytaddr, recipients), timeout=120)
self.sync_all()
@@ -44,7 +100,7 @@ class WalletNullifiersTest (BitcoinTestFramework):
bitcoind_processes[1].wait()
# restart node 1
self.nodes[1] = start_node(1, self.options.tmpdir)
self.nodes[1] = start_node(1, self.options.tmpdir, self.net_args(1))
connect_nodes_bi(self.nodes, 0, 1)
connect_nodes_bi(self.nodes, 1, 2)
self.sync_all()
@@ -52,7 +108,7 @@ class WalletNullifiersTest (BitcoinTestFramework):
# send node 0 zaddr to note 2 zaddr
recipients = []
recipients.append({"address":myzaddr, "amount":7.0})
wait_and_assert_operationid_status(self.nodes[0], self.nodes[0].z_sendmany(myzaddr0, recipients), timeout=120)
self.sync_all()
@@ -97,7 +153,7 @@ class WalletNullifiersTest (BitcoinTestFramework):
mytaddr1 = self.nodes[1].getnewaddress()
recipients = []
recipients.append({"address":mytaddr1, "amount":1.0})
wait_and_assert_operationid_status(self.nodes[1], self.nodes[1].z_sendmany(myzaddr, recipients), timeout=120)
self.sync_all()
qa/rpc-tests/wallet_persistence.py
+40 -7
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2018 The Zcash developers
# Distributed under the GPLv3 software license, see the accompanying
@@ -13,6 +13,29 @@ from test_framework.util import (
)
from decimal import Decimal
def get_value_pool(pools, pool_id):
"""
Return the valuePools entry with the given id, or None if this chain does
not have that pool. DragonX's getblockchaininfo only reports the Sapling
pool (no Sprout history exists on this chain), so the pools list can no
longer be indexed positionally the way the upstream test did.
"""
for pool in pools:
if pool['id'] == pool_id:
return pool
return None
def assert_pool_values(pools, sprout_value, sapling_value):
sprout = get_value_pool(pools, 'sprout')
if sprout is not None:
assert_equal(sprout['chainValue'], sprout_value)
else:
# No Sprout pool at all is the same statement as "the Sprout pool holds nothing"
assert_equal(sprout_value, Decimal('0'))
sapling = get_value_pool(pools, 'sapling')
assert_true(sapling is not None, "Sapling value pool missing from getblockchaininfo")
assert_equal(sapling['chainValue'], sapling_value)
class WalletPersistenceTest (BitcoinTestFramework):
def setup_chain(self):
@@ -20,8 +43,20 @@ class WalletPersistenceTest (BitcoinTestFramework):
initialize_chain_clean(self.options.tmpdir, 3)
def setup_network(self, split=False):
# -listen=1/-bind: the framework's start_node() passes -connect=0, and DragonX (like
# Bitcoin) reacts to -connect by soft-setting -listen=0. A non-listening node can never
# accept the "addnode 127.0.0.1:<port>" that connect_nodes_bi() issues, so without this
# the three nodes stay isolated and sync_all() spins forever. -bind keeps the listener on
# loopback so a regtest node never becomes reachable from the public internet.
self.nodes = start_nodes(3, self.options.tmpdir,
extra_args=[[
'-listen=1',
'-bind=127.0.0.1',
# -dns=0: DragonX appends node1..node10.dragonx.is to -addnode for every chain
# named DRAGONX (hush_utils.h), and -connect=0 does not suppress -addnode. Without
# this a regtest node dials the LIVE DragonX network and is fed mainnet headers.
# The addnode calls connect_nodes_bi() makes use literal IPs, so they still work.
'-dns=0',
'-nuparams=5ba81b19:100', # Overwinter
'-nuparams=76b809bb:201', # Sapling
]] * 3)
@@ -69,12 +104,11 @@ class WalletPersistenceTest (BitcoinTestFramework):
# Verify shielded balance
assert_equal(self.nodes[0].z_getbalance(sapling_addr), Decimal('20'))
# Verify size of shielded pools
pools = self.nodes[0].getblockchaininfo()['valuePools']
assert_equal(pools[0]['chainValue'], Decimal('0')) # Sprout
assert_equal(pools[1]['chainValue'], Decimal('20')) # Sapling
assert_pool_values(pools, Decimal('0'), Decimal('20'))
# Restart the nodes
stop_nodes(self.nodes)
wait_bitcoinds()
@@ -82,8 +116,7 @@ class WalletPersistenceTest (BitcoinTestFramework):
# Verify size of shielded pools
pools = self.nodes[0].getblockchaininfo()['valuePools']
assert_equal(pools[0]['chainValue'], Decimal('0')) # Sprout
assert_equal(pools[1]['chainValue'], Decimal('20')) # Sapling
assert_pool_values(pools, Decimal('0'), Decimal('20'))
# Node 0 sends some shielded funds to Node 1
dest_addr = self.nodes[1].z_getnewaddress('sapling')
qa/rpc-tests/wallet_protectcoinbase.py
+52 -34
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2016 The Zcash developers
# Distributed under the GPLv3 software license, see the accompanying
@@ -6,7 +6,6 @@
from test_framework.test_framework import BitcoinTestFramework
from test_framework.authproxy import JSONRPCException
from test_framework.mininode import COIN
from test_framework.util import assert_equal, initialize_chain_clean, \
start_nodes, connect_nodes_bi, wait_and_assert_operationid_status
@@ -14,6 +13,20 @@ import sys
import timeit
from decimal import Decimal
# Upstream imported this from test_framework.mininode, which is still python2 and
# fails to even parse under python3 (0x100000000L literals). mininode is a p2p
# serialisation module this test does not otherwise use, so rather than drag a
# large unrelated port into test_framework/ the one constant needed is inlined.
# Same value as test_framework/mininode.py:52.
COIN = 100000000 # 1 DRGX in puposhis
# DragonX has no Sprout pool: getblockchaininfo/getblock only ever emit a
# "sapling" entry in valuePools (see rpc/blockchain.cpp), and z_getnewaddress
# only makes Sapling addresses. The shielded value this test moves therefore
# lands in the Sapling pool, so every check that upstream made against 'sprout'
# is made against 'sapling' here. The assertion itself is unchanged.
SHIELDED_POOL = 'sapling'
def check_value_pool(node, name, total):
value_pools = node.getblockchaininfo()['valuePools']
found = False
@@ -42,7 +55,7 @@ class WalletProtectCoinbaseTest (BitcoinTestFramework):
self.sync_all()
def run_test (self):
print "Mining blocks..."
print("Mining blocks...")
self.nodes[0].generate(4)
@@ -59,17 +72,17 @@ class WalletProtectCoinbaseTest (BitcoinTestFramework):
assert_equal(self.nodes[2].getbalance(), 0)
assert_equal(self.nodes[3].getbalance(), 0)
check_value_pool(self.nodes[0], 'sprout', 0)
check_value_pool(self.nodes[1], 'sprout', 0)
check_value_pool(self.nodes[2], 'sprout', 0)
check_value_pool(self.nodes[3], 'sprout', 0)
check_value_pool(self.nodes[0], SHIELDED_POOL, 0)
check_value_pool(self.nodes[1], SHIELDED_POOL, 0)
check_value_pool(self.nodes[2], SHIELDED_POOL, 0)
check_value_pool(self.nodes[3], SHIELDED_POOL, 0)
# Send will fail because we are enforcing the consensus rule that
# coinbase utxos can only be sent to a zaddr.
errorString = ""
try:
self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 1)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Coinbase funds can only be sent to a zaddr" in errorString, True)
@@ -88,18 +101,18 @@ class WalletProtectCoinbaseTest (BitcoinTestFramework):
# as it's currently not possible to specify a change address in z_sendmany.
recipients = []
recipients.append({"address":myzaddr, "amount":Decimal('1.23456789')})
myopid = self.nodes[0].z_sendmany(mytaddr, recipients)
error_result = wait_and_assert_operationid_status(self.nodes[0], myopid, "failed", "wallet does not allow any change", 10)
# Test that the returned status object contains a params field with the operation's input parameters
assert_equal(error_result["method"], "z_sendmany")
params = error_result["params"]
assert_equal(params["fee"], Decimal('0.0001')) # default
assert_equal(params["minconf"], Decimal('1')) # default
assert_equal(Decimal(params["fee"]), Decimal('0.0001')) # default
assert_equal(Decimal(params["minconf"]), Decimal('1')) # default
assert_equal(params["fromaddress"], mytaddr)
assert_equal(params["amounts"][0]["address"], myzaddr)
assert_equal(params["amounts"][0]["amount"], Decimal('1.23456789'))
assert_equal(Decimal(params["amounts"][0]["amount"]), Decimal('1.23456789'))
# Add viewing key for myzaddr to Node 3
myviewingkey = self.nodes[0].z_exportviewingkey(myzaddr)
@@ -169,14 +182,17 @@ class WalletProtectCoinbaseTest (BitcoinTestFramework):
assert_equal(Decimal(resp["private"]), Decimal('19.9999'))
assert_equal(Decimal(resp["total"]), Decimal('39.9999'))
# The Sprout value pool should reflect the send
sproutvalue = shieldvalue
check_value_pool(self.nodes[0], 'sprout', sproutvalue)
# The shielded value pool should reflect the send
shieldedvalue = shieldvalue
check_value_pool(self.nodes[0], SHIELDED_POOL, shieldedvalue)
# A custom fee of 0 is okay. Here the node will send the note value back to itself.
recipients = []
recipients.append({"address":myzaddr, "amount": Decimal('19.9999')})
myopid = self.nodes[0].z_sendmany(myzaddr, recipients, 1, Decimal('0.0'))
# NB: the fee is passed as a JSON number, not a Decimal. authproxy serialises
# Decimal as a JSON *string* and z_sendmany reads the fee with params[3].get_real(),
# which only accepts VNUM -- see port notes. The value is unchanged.
myopid = self.nodes[0].z_sendmany(myzaddr, recipients, 1, 0.0)
mytxid = wait_and_assert_operationid_status(self.nodes[0], myopid)
self.sync_all()
self.nodes[1].generate(1)
@@ -186,8 +202,8 @@ class WalletProtectCoinbaseTest (BitcoinTestFramework):
assert_equal(Decimal(resp["private"]), Decimal('19.9999'))
assert_equal(Decimal(resp["total"]), Decimal('39.9999'))
# The Sprout value pool should be unchanged
check_value_pool(self.nodes[0], 'sprout', sproutvalue)
# The shielded value pool should be unchanged
check_value_pool(self.nodes[0], SHIELDED_POOL, shieldedvalue)
# convert note to transparent funds
unshieldvalue = Decimal('10.0')
@@ -206,12 +222,12 @@ class WalletProtectCoinbaseTest (BitcoinTestFramework):
self.sync_all()
# check balances
sproutvalue -= unshieldvalue + Decimal('0.0001')
shieldedvalue -= unshieldvalue + Decimal('0.0001')
resp = self.nodes[0].z_gettotalbalance()
assert_equal(Decimal(resp["transparent"]), Decimal('30.0'))
assert_equal(Decimal(resp["private"]), Decimal('9.9998'))
assert_equal(Decimal(resp["total"]), Decimal('39.9998'))
check_value_pool(self.nodes[0], 'sprout', sproutvalue)
check_value_pool(self.nodes[0], SHIELDED_POOL, shieldedvalue)
# z_sendmany will return an error if there is transparent change output considered dust.
# UTXO selection in z_sendmany sorts in ascending order, so smallest utxos are consumed first.
@@ -226,7 +242,7 @@ class WalletProtectCoinbaseTest (BitcoinTestFramework):
errorString = ""
try:
self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 99999)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Insufficient funds" in errorString, True)
@@ -241,7 +257,7 @@ class WalletProtectCoinbaseTest (BitcoinTestFramework):
# Send will fail because of insufficient funds unless sender uses coinbase utxos
try:
self.nodes[0].sendtoaddress(self.nodes[2].getnewaddress(), 21)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Insufficient funds, coinbase funds can only be spent after they have been sent to a zaddr" in errorString, True)
@@ -256,7 +272,7 @@ class WalletProtectCoinbaseTest (BitcoinTestFramework):
# Note that regtest chainparams does not require standard tx, so setting the amount to be
# less than the dust threshold, e.g. 0.00000001 will not result in mempool rejection.
start_time = timeit.default_timer()
for i in xrange(0,num_t_recipients):
for i in range(0,num_t_recipients):
newtaddr = self.nodes[2].getnewaddress()
recipients.append({"address":newtaddr, "amount":amount_per_recipient})
elapsed = timeit.default_timer() - start_time
@@ -287,28 +303,30 @@ class WalletProtectCoinbaseTest (BitcoinTestFramework):
# check balance
node2balance = amount_per_recipient * num_t_recipients
sproutvalue -= node2balance + Decimal('0.0001')
shieldedvalue -= node2balance + Decimal('0.0001')
assert_equal(self.nodes[2].getbalance(), node2balance)
check_value_pool(self.nodes[0], 'sprout', sproutvalue)
check_value_pool(self.nodes[0], SHIELDED_POOL, shieldedvalue)
# Send will fail because fee is negative
try:
self.nodes[0].z_sendmany(myzaddr, recipients, 1, -1)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Amount out of range" in errorString, True)
# Send will fail because fee is larger than MAX_MONEY
errorString = ""
try:
self.nodes[0].z_sendmany(myzaddr, recipients, 1, Decimal('21000000.00000001'))
except JSONRPCException,e:
self.nodes[0].z_sendmany(myzaddr, recipients, 1, float(Decimal('21000000.00000001')))
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Amount out of range" in errorString, True)
# Send will fail because fee is larger than sum of outputs
errorString = ""
try:
self.nodes[0].z_sendmany(myzaddr, recipients, 1, (amount_per_recipient * num_t_recipients) + Decimal('0.00000001'))
except JSONRPCException,e:
self.nodes[0].z_sendmany(myzaddr, recipients, 1, float((amount_per_recipient * num_t_recipients) + Decimal('0.00000001')))
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("is greater than the sum of outputs" in errorString, True)
@@ -334,10 +352,10 @@ class WalletProtectCoinbaseTest (BitcoinTestFramework):
send_amount = num_recipients * amount_per_recipient
custom_fee = Decimal('0.00012345')
zbalance = self.nodes[0].z_getbalance(myzaddr)
for i in xrange(0,num_recipients):
for i in range(0,num_recipients):
newzaddr = self.nodes[2].z_getnewaddress()
recipients.append({"address":newzaddr, "amount":amount_per_recipient})
myopid = self.nodes[0].z_sendmany(myzaddr, recipients, minconf, custom_fee)
myopid = self.nodes[0].z_sendmany(myzaddr, recipients, minconf, float(custom_fee))
wait_and_assert_operationid_status(self.nodes[0], myopid)
self.sync_all()
self.nodes[1].generate(1)
@@ -353,8 +371,8 @@ class WalletProtectCoinbaseTest (BitcoinTestFramework):
resp = self.nodes[0].z_getbalance(myzaddr)
assert_equal(Decimal(resp), zbalance - custom_fee - send_amount)
sproutvalue -= custom_fee
check_value_pool(self.nodes[0], 'sprout', sproutvalue)
shieldedvalue -= custom_fee
check_value_pool(self.nodes[0], SHIELDED_POOL, shieldedvalue)
notes = self.nodes[0].z_listunspent(1, 99999, False, [myzaddr])
sum_of_notes = sum([note["amount"] for note in notes])
qa/rpc-tests/wallet_sapling.py
+85 -7
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2018 The Zcash developers
# Distributed under the GPLv3 software license, see the accompanying
@@ -8,21 +8,99 @@ from test_framework.test_framework import BitcoinTestFramework
from test_framework.authproxy import JSONRPCException
from test_framework.util import (
assert_equal,
start_nodes,
initialize_chain_clean,
p2p_port,
set_node_times,
start_node,
sync_blocks,
wait_and_assert_operationid_status,
)
import os
import stat
from decimal import Decimal
# Test wallet behaviour with Sapling addresses
class WalletSaplingTest(BitcoinTestFramework):
def setup_chain(self):
# The shared initialize_chain() cache builder in test_framework/util.py has not been
# repaired for DragonX (it spawns the cache nodes by bare name off PATH, drives the
# CLI without -regtest so it dials the assetchain RPC port, and deletes debug.log
# from the non-net-specific datadir). Build the same starting state here instead --
# see _generate_starting_chain() -- so this test does not depend on it.
print("Initializing test directory " + self.options.tmpdir)
initialize_chain_clean(self.options.tmpdir, 4)
# !!! test_framework/util.py:start_node() hardcodes "-connect=0" into every regtest
# node's argv. Modern Bitcoin Core special-cases that value to mean "make no automatic
# connections", but THIS codebase does not (net.cpp ThreadOpenConnections just iterates
# mapMultiArgs["-connect"]), so "0" is dialled as a hostname: it resolves to 0.0.0.0,
# which on Linux connects to localhost on Params().GetDefaultPort() -- 21768, the live
# DRAGONX p2p port. Observed directly: a regtest node started by the unmodified
# framework peered with the production dragonxd on this host and with seven public
# mainnet nodes (heights ~3.25M) and began ingesting mainnet headers. "-connect" also
# soft-sets "-listen=0", so the framework's own connect_nodes_bi() can never establish
# the local links a multi-node test needs.
#
# Both problems are in the shared framework, which this port is not allowed to touch, so
# they are worked around per-node here: the daemon is launched through a tiny wrapper
# that strips the "-connect=0" argument, and the real topology/listening flags are passed
# as extra_args (which start_node appends after its own).
def _daemon_wrapper(self):
srcdir = os.path.join(
os.path.dirname(os.path.abspath(__file__)), "..", "..", "src", "dragonxd")
path = os.path.join(self.options.tmpdir, "dragonxd-no-connect0")
with open(path, "w") as f:
f.write("#!/usr/bin/env bash\n")
f.write("args=()\n")
f.write('for a in "$@"; do\n')
f.write(' if [ "$a" = "-connect=0" ]; then continue; fi\n')
f.write(' args+=("$a")\n')
f.write("done\n")
f.write('exec %s "${args[@]}"\n' % os.path.realpath(srcdir))
os.chmod(path, os.stat(path).st_mode | stat.S_IXUSR)
return path
def setup_nodes(self):
return start_nodes(4, self.options.tmpdir, [[
#'-nuparams=5ba81b19:201', # Overwinter
#'-nuparams=76b809bb:203', # Sapling
#'-experimentalfeatures', '-zmergetoaddress',
]] * 4)
binary = self._daemon_wrapper()
nodes = []
for i in range(4):
extra_args = [
#'-nuparams=5ba81b19:201', # Overwinter
#'-nuparams=76b809bb:203', # Sapling
#'-experimentalfeatures', '-zmergetoaddress',
# Listen on this test's PID-keyed port so the nodes can actually peer with
# each other, and only ever dial each other -- never the public network.
'-listen=1',
'-bind=127.0.0.1',
'-port=%d' % p2p_port(i),
] + ['-connect=127.0.0.1:%d' % p2p_port(j) for j in range(4) if j != i]
nodes.append(start_node(i, self.options.tmpdir, extra_args, binary=binary))
return nodes
def setup_network(self, split=False):
super(WalletSaplingTest, self).setup_network(split)
self._generate_starting_chain()
def _generate_starting_chain(self):
# Equivalent of test_framework.util.initialize_chain(): a 200-block chain where each
# of the 4 nodes mined 25 blocks twice, so every node holds 25 mature and 25 immature
# coinbases. Block timestamps are 10 minutes apart starting 1 Jan 2014, as there.
block_time = 1388534400
for _round in range(2):
for peer in range(4):
for _j in range(25):
set_node_times(self.nodes, block_time)
self.nodes[peer].generate(1)
block_time += 10 * 75
# Must sync before next peer starts generating blocks
sync_blocks(self.nodes)
# Drop back to wall-clock time: initialize_chain() stops the cache nodes and the test
# then runs against freshly started nodes that have no mocktime set.
set_node_times(self.nodes, 0)
self.sync_all()
def run_test(self):
# Sanity-check the test harness
qa/rpc-tests/wallet_shieldcoinbase.py
+36 -10
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2017 The Zcash developers
# Distributed under the GPLv3 software license, see the accompanying
@@ -22,8 +22,24 @@ class WalletShieldCoinbaseTest (BitcoinTestFramework):
initialize_chain_clean(self.options.tmpdir, 4)
def setup_network(self, split=False):
args = ['-regtestprotectcoinbase', '-debug=zrpcunsafe']
args2 = ['-regtestprotectcoinbase', '-debug=zrpcunsafe', "-mempooltxinputlimit=7"]
# DragonX-specific environment flags. None of these change what the test
# asserts; without them the test cannot run at all on this daemon:
# -listen=1/-bind=127.0.0.1: the framework's start_node() always passes
# -connect=0, and AppInit2 soft-sets -listen=0 whenever -connect is
# present, so the nodes never listen and connect_nodes_bi() silently
# builds an empty topology (sync_all() then spins forever). An explicit
# -listen=1 beats the SoftSetBoolArg.
# -dnsseed=0: chainparams_commandline() keeps DRAGONX's DNS seeds and
# overwrites pchMessageStart with the DRAGONX chain magic on *every*
# network including regtest, so a regtest node otherwise dials and
# handshakes with live mainnet peers (observed: 8 mainnet peers,
# blocks=3254237, feeding mainnet headers into the regtest node).
# -autoshield=0: DragonX auto-shields matured coinbase every 25 blocks
# by default, which would race the manual z_shieldcoinbase calls under
# test and move the balances this test checks.
isolate = ['-listen=1', '-bind=127.0.0.1', '-dnsseed=0', '-autoshield=0']
args = ['-regtestprotectcoinbase', '-debug=zrpcunsafe'] + isolate
args2 = ['-regtestprotectcoinbase', '-debug=zrpcunsafe', "-mempooltxinputlimit=7"] + isolate
if self.addr_type != 'sprout':
nu = [
'-nuparams=5ba81b19:0', # Overwinter
@@ -42,7 +58,7 @@ class WalletShieldCoinbaseTest (BitcoinTestFramework):
self.sync_all()
def run_test (self):
print "Mining blocks..."
print("Mining blocks...")
self.nodes[0].generate(1)
self.sync_all()
@@ -73,42 +89,42 @@ class WalletShieldCoinbaseTest (BitcoinTestFramework):
self.nodes[2].importaddress(mytaddr)
try:
self.nodes[2].z_shieldcoinbase(mytaddr, myzaddr)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Could not find any coinbase funds to shield" in errorString, True)
# Shielding will fail because fee is negative
try:
self.nodes[0].z_shieldcoinbase("*", myzaddr, -1)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Amount out of range" in errorString, True)
# Shielding will fail because fee is larger than MAX_MONEY
try:
self.nodes[0].z_shieldcoinbase("*", myzaddr, Decimal('21000000.00000001'))
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Amount out of range" in errorString, True)
# Shielding will fail because fee is larger than sum of utxos
try:
self.nodes[0].z_shieldcoinbase("*", myzaddr, 999)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Insufficient coinbase funds" in errorString, True)
# Shielding will fail because limit parameter must be at least 0
try:
self.nodes[0].z_shieldcoinbase("*", myzaddr, Decimal('0.001'), -1)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("Limit on maximum number of utxos cannot be negative" in errorString, True)
# Shielding will fail because limit parameter is absurdly large
try:
self.nodes[0].z_shieldcoinbase("*", myzaddr, Decimal('0.001'), 99999999999999)
except JSONRPCException,e:
except JSONRPCException as e:
errorString = e.error['message']
assert_equal("JSON integer out of range" in errorString, True)
@@ -214,3 +230,13 @@ class WalletShieldCoinbaseTest (BitcoinTestFramework):
sync_mempools(self.nodes[:2])
self.nodes[1].generate(1)
self.sync_all()
if __name__ == '__main__':
# Upstream (Zcash/Hush) ran this test twice: once with Sprout zaddrs and once
# with Sapling. DragonX has no Sprout support at all -- z_getnewaddress only
# accepts "sapling" or "amnesia" (src/wallet/rpcwallet.cpp z_getnewaddress),
# so WalletShieldCoinbaseTest('sprout') cannot even allocate its target
# address. The sprout-only branches inside run_test() are kept intact for
# reference but only the sapling variant is executed.
print("Running for sapling...")
WalletShieldCoinbaseTest('sapling').main()
qa/rpc-tests/wallet_treestate.py
+13 -4
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2016 The Zcash developers
# Distributed under the GPLv3 software license, see the accompanying
@@ -20,7 +20,16 @@ class WalletTreeStateTest (BitcoinTestFramework):
# Start nodes with -regtestprotectcoinbase to set fCoinbaseMustBeProtected to true.
def setup_network(self, split=False):
self.nodes = start_nodes(3, self.options.tmpdir, extra_args=[['-regtestprotectcoinbase','-debug=zrpc']] * 3 )
# -listen=1 and -dns=0 are DragonX-specific harness requirements, not part of the
# original test: start_node() passes -connect=0, which trips the "-connect set ->
# setting -listen=0" parameter interaction, so without -listen=1 the three nodes
# cannot open the p2p links connect_nodes_bi() asks for (node1/node2 stay at height
# 0 forever and sync_all() can never converge). -dns=0 blocks the unconditional
# node1..node10.dragonx.is -addnode injection in hush_args(), which otherwise dials
# the real DragonX seed nodes from regtest and floods these nodes with mainnet headers.
self.nodes = start_nodes(3, self.options.tmpdir,
extra_args=[['-regtestprotectcoinbase','-debug=zrpc',
'-listen=1','-dns=0']] * 3 )
connect_nodes_bi(self.nodes,0,1)
connect_nodes_bi(self.nodes,1,2)
connect_nodes_bi(self.nodes,0,2)
@@ -28,7 +37,7 @@ class WalletTreeStateTest (BitcoinTestFramework):
self.sync_all()
def run_test (self):
print "Mining blocks..."
print("Mining blocks...")
self.nodes[0].generate(100)
self.sync_all()
@@ -79,7 +88,7 @@ class WalletTreeStateTest (BitcoinTestFramework):
myopid = self.nodes[0].z_sendmany(myzaddr, recipients)
# Wait for Tx 2 to begin executing...
for x in xrange(1, 60):
for x in range(1, 60):
results = self.nodes[0].z_getoperationstatus([myopid])
status = results[0]["status"]
if status == "executing":
qa/rpc-tests/walletbackup.py
+53 -10
View File
@@ -1,4 +1,4 @@
#!/usr/bin/env python2
#!/usr/bin/env python3
# Copyright (c) 2016-2024 The Hush developers
# Copyright (c) 2014 The Bitcoin Core developers
# Distributed under the GPLv3 software license, see the accompanying
@@ -37,8 +37,8 @@ and confirm again balances are correct.
from test_framework.test_framework import BitcoinTestFramework
from test_framework.authproxy import JSONRPCException
from test_framework.util import assert_equal, initialize_chain_clean, \
start_nodes, start_node, connect_nodes, stop_node, \
sync_blocks, sync_mempools
start_nodes, start_node, connect_nodes, \
sync_blocks, sync_mempools, bitcoind_processes
import os
import shutil
@@ -48,6 +48,46 @@ import logging
logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.INFO)
# Three node flags this test has to supply for itself, because test_framework.util.start_node()
# cannot give it a usable isolated regtest network on DragonX:
#
# -listen=1: start_node() passes "-connect=0" to every node. In DragonX (as in Bitcoin) the
# presence of a -connect argument SoftSetBoolArg()s -listen to false, so no test node ever
# binds its p2p port and connect_nodes() cannot build the loopback topology this test needs.
# Verified on a live run: only the RPC ports were listening and getpeerinfo showed zero
# 127.0.0.1 peers on all four nodes. SoftSetBoolArg does not override an explicit value.
#
# -dns=0: hush_args() unconditionally appends node1..node10.dragonx.is to -addnode whenever the
# chain name is DRAGONX, regardless of network, and -connect=0 does not suppress it. On a live
# run every "regtest" node ended up with 7-8 established connections to production mainnet
# nodes on port 21768, was flooded with mainnet headers ("AcceptBlockHeader: hashPrevBlock ...
# not found"), and one of them aborted on CheckBlockIndex(). -dns=0 stops the hostnames from
# resolving while leaving the literal 127.0.0.1:PORT addnodes connect_nodes() uses intact.
#
# -allowlist=127.0.0.1: "-connect=0" does not mean "no connections". The daemon resolves the
# literal "0" to 0.0.0.0 and dials it on the chain's default p2p port, i.e. 127.0.0.1:21768 --
# which on a machine that also runs a real node is the PRODUCTION daemon (observed:
# ESTAB 127.0.0.1:41282 -> 127.0.0.1:21768 from every test node, peer subver /DragonX:1.0.3/,
# startingheight 3254254). That peer is outbound, so it is the node's only preferred-download
# peer; main.cpp:8398 then computes fFetch=false for every inbound peer, and node3 -- which in
# this test's topology is dialed by everyone and dials no one -- never downloads an announced
# block, so sync_blocks() hangs forever. Allowlisting loopback makes inbound test peers
# preferred-download too (main.cpp:381), which restores block propagation.
LISTEN = "-listen=1"
NODNS = "-dns=0"
ALLOWLIST = "-allowlist=127.0.0.1"
def stop_node_and_reap(node, i):
# Equivalent to test_framework.util.stop_node(), which cannot be called: it does
# print("Stopping node " + i) with the int index that every caller passes, which raises
# TypeError. Reimplemented here rather than editing shared framework code other tests use.
print("Stopping node %d" % i)
node.stop()
bitcoind_processes[i].wait()
del bitcoind_processes[i]
class WalletBackupTest(BitcoinTestFramework):
def setup_chain(self):
@@ -62,7 +102,10 @@ class WalletBackupTest(BitcoinTestFramework):
ed2 = "-exportdir=" + self.options.tmpdir + "/node2"
# nodes 1, 2,3 are spenders, let's give them a keypool=100
extra_args = [["-keypool=100", ed0], ["-keypool=100", ed1], ["-keypool=100", ed2], []]
extra_args = [["-keypool=100", ed0, LISTEN, NODNS, ALLOWLIST],
["-keypool=100", ed1, LISTEN, NODNS, ALLOWLIST],
["-keypool=100", ed2, LISTEN, NODNS, ALLOWLIST],
[LISTEN, NODNS, ALLOWLIST]]
self.nodes = start_nodes(4, self.options.tmpdir, extra_args)
connect_nodes(self.nodes[0], 3)
connect_nodes(self.nodes[1], 3)
@@ -95,18 +138,18 @@ class WalletBackupTest(BitcoinTestFramework):
# As above, this mirrors the original bash test.
def start_three(self):
self.nodes[0] = start_node(0, self.options.tmpdir)
self.nodes[1] = start_node(1, self.options.tmpdir)
self.nodes[2] = start_node(2, self.options.tmpdir)
self.nodes[0] = start_node(0, self.options.tmpdir, [LISTEN, NODNS, ALLOWLIST])
self.nodes[1] = start_node(1, self.options.tmpdir, [LISTEN, NODNS, ALLOWLIST])
self.nodes[2] = start_node(2, self.options.tmpdir, [LISTEN, NODNS, ALLOWLIST])
connect_nodes(self.nodes[0], 3)
connect_nodes(self.nodes[1], 3)
connect_nodes(self.nodes[2], 3)
connect_nodes(self.nodes[2], 0)
def stop_three(self):
stop_node(self.nodes[0], 0)
stop_node(self.nodes[1], 1)
stop_node(self.nodes[2], 2)
stop_node_and_reap(self.nodes[0], 0)
stop_node_and_reap(self.nodes[1], 1)
stop_node_and_reap(self.nodes[2], 2)
def erase_three(self):
os.remove(self.options.tmpdir + "/node0/regtest/wallet.dat")
src/Makefile.gtest.include
+4 -1
View File
@@ -13,7 +13,10 @@ hush_gtest_SOURCES = \
gtest/utils.cpp \
gtest/test_randomx_preverify.cpp \
gtest/test_hdtransparent.cpp \
gtest/test_mnemonic_compat.cpp
gtest/test_mnemonic_compat.cpp \
gtest/test_stratum_jobid.cpp \
gtest/test_wallet_scan_notes.cpp \
gtest/test_operation_status.cpp
hush_gtest_CPPFLAGS = $(AM_CPPFLAGS) -DMULTICORE -fopenmp -DBINARY_OUTPUT -DCURVE_ALT_BN128 -DSTATIC $(BITCOIN_INCLUDES)
hush_gtest_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)
src/asyncrpcoperation.cpp
+3 -3
View File
@@ -32,7 +32,7 @@ using namespace std;
static boost::uuids::random_generator uuidgen;
static std::map<OperationStatus, std::string> OperationStatusMap = {
static const std::map<OperationStatus, std::string> OperationStatusMap = {
{OperationStatus::READY, "queued"},
{OperationStatus::EXECUTING, "executing"},
{OperationStatus::CANCELLED, "cancelled"},
@@ -194,7 +194,7 @@ UniValue AsyncRPCOperation::getStatus() const {
OperationStatus status = this->getState();
UniValue obj(UniValue::VOBJ);
obj.push_back(Pair("id", this->id_));
obj.push_back(Pair("status", OperationStatusMap[status]));
obj.push_back(Pair("status", OperationStatusMap.at(status)));
obj.push_back(Pair("creation_time", this->creation_time_));
// TODO: Issue #1354: There may be other useful metadata to return to the user.
UniValue err = this->getError();
@@ -218,5 +218,5 @@ UniValue AsyncRPCOperation::getStatus() const {
*/
std::string AsyncRPCOperation::getStateAsString() const {
OperationStatus status = this->getState();
return OperationStatusMap[status];
return OperationStatusMap.at(status);
}
src/asyncrpcqueue.cpp
+8 -4
View File
@@ -129,14 +129,18 @@ std::shared_ptr<AsyncRPCOperation> AsyncRPCQueue::getOperationForId(AsyncRPCOper
/**
* Return the operation for a given operation id and then remove the operation from internal storage.
* The lookup and the erase happen under one hold of lock_, so when several threads pop the same id
* exactly one of them receives the operation and the others get a null pointer.
*/
std::shared_ptr<AsyncRPCOperation> AsyncRPCQueue::popOperationForId(AsyncRPCOperationId id) {
std::shared_ptr<AsyncRPCOperation> ptr = getOperationForId(id);
if (ptr) {
std::lock_guard<std::mutex> guard(lock_);
std::shared_ptr<AsyncRPCOperation> ptr; // declared before guard, so it is released after unlock
std::lock_guard<std::mutex> guard(lock_);
AsyncRPCOperationMap::iterator iter = operation_map_.find(id);
if (iter != operation_map_.end()) {
ptr = iter->second;
// Note: if the id still exists in the operationIdQueue, when it gets processed by a worker
// there will no operation in the map to execute, so nothing will happen.
operation_map_.erase(id);
operation_map_.erase(iter);
}
return ptr;
}
src/bitcoin-cli.cpp
+6 -2
View File
@@ -257,14 +257,18 @@ UniValue CallRPC(const std::string& strMethod, const UniValue& params)
event_base_dispatch(base.get());
if (response.status == 0) {
// Report the port we ACTUALLY dialled. ASSETCHAINS_RPCPORT is a separate global that is
// initialised to the mainnet default at the top of this file and never assigned here, so
// using it made every failure claim port 21769 no matter what -rpcport was given -- which
// reads as "your -rpcport was ignored" and sends you chasing a config bug that isn't there.
throw CConnectionFailed(strprintf("couldn't connect to server at port %d : %s (code %d)\n(make sure server is running and you are connecting to the correct RPC port)",
ASSETCHAINS_RPCPORT, http_errorstring(response.error), response.error));
port, http_errorstring(response.error), response.error));
} else if (response.status == HTTP_UNAUTHORIZED) {
throw std::runtime_error("incorrect rpcuser or rpcpassword (authorization failed)");
} else if (response.status >= 400 && response.status != HTTP_BAD_REQUEST && response.status != HTTP_NOT_FOUND && response.status != HTTP_INTERNAL_SERVER_ERROR) {
throw std::runtime_error(strprintf("server returned HTTP error %d", response.status));
} else if (response.body.empty()) {
throw std::runtime_error(strprintf("no response from server at port %d", ASSETCHAINS_RPCPORT ));
throw std::runtime_error(strprintf("no response from server at port %d", port));
}
// Parse reply
src/chain.h
+24 -5
View File
@@ -35,7 +35,7 @@ extern bool fZindex;
// These version thresholds control whether nSproutValue/nSaplingValue are
// serialized in the block index. They must be <= CLIENT_VERSION or the
// values will never be persisted, causing nChainSaplingValue to reset
// to 0 after node restart. DragonX CLIENT_VERSION is 1010050 (v1.1.0.50).
// to 0 after node restart. DragonX CLIENT_VERSION is 1030050 (v1.3.0.50).
static const int SPROUT_VALUE_VERSION = 1000000;
static const int SAPLING_VALUE_VERSION = 1000000;
// Block-index records written at >= this version store nSaplingValue as a boost::optional
@@ -113,10 +113,9 @@ enum BlockStatus: uint32_t {
//! Scripts & signatures ok. Implies all parents are also at least SCRIPTS.
BLOCK_VALID_SCRIPTS = 5,
// flag to check if contextual check block has passed in Accept block, if it has not check at connect block.
BLOCK_VALID_CONTEXT = 6,
//! All validity bits.
//! NOTE: the levels above are sequential VALUES occupying this 3-bit field, not independent
//! bits, so any flag stored in nStatus must live entirely outside this mask.
BLOCK_VALID_MASK = BLOCK_VALID_HEADER | BLOCK_VALID_TREE | BLOCK_VALID_TRANSACTIONS |
BLOCK_VALID_CHAIN | BLOCK_VALID_SCRIPTS,
@@ -129,9 +128,29 @@ enum BlockStatus: uint32_t {
BLOCK_FAILED_MASK = BLOCK_FAILED_VALID | BLOCK_FAILED_CHILD,
BLOCK_ACTIVATES_UPGRADE = 128, //! block activates a network upgrade
BLOCK_IN_TMPFILE = 256
BLOCK_IN_TMPFILE = 256,
//! ContextualCheckBlock already passed in AcceptBlock, so ConnectBlock may skip re-running it.
//! Was 6 until v1.3.0, which put it INSIDE BLOCK_VALID_MASK (1|2|3|4|5 == 7): `nStatus |=
//! BLOCK_VALID_CONTEXT` then overwrote the validity level rather than setting a flag, so a
//! block that was only written to disk read back as BLOCK_VALID_SCRIPTS and every later
//! RaiseValidity() silently no-opped. Detected by CheckBlockIndex's "CHAIN valid implies all
//! parents are CHAIN valid" assert, which aborts any node doing out-of-order block download
//! (regtest only, where fDefaultConsistencyChecks is true). Validity was only ever inflated,
//! never deflated, so ConnectBlock's full validation was never skipped -- see git history.
//! Legacy block indexes still carry the polluted low bits; they resolve on reindex, and until
//! then simply re-run the contextual check they used to skip.
BLOCK_VALID_CONTEXT = 512
};
//! The validity level is a small integer packed into BLOCK_VALID_MASK, so every other nStatus flag
//! must be disjoint from it. Enforced here so this class of bug cannot be reintroduced silently.
static_assert((BLOCK_VALID_CONTEXT & BLOCK_VALID_MASK) == 0, "BLOCK_VALID_CONTEXT overlaps the validity-level field");
static_assert((BLOCK_HAVE_MASK & BLOCK_VALID_MASK) == 0, "BLOCK_HAVE_MASK overlaps the validity-level field");
static_assert((BLOCK_FAILED_MASK & BLOCK_VALID_MASK) == 0, "BLOCK_FAILED_MASK overlaps the validity-level field");
static_assert((BLOCK_ACTIVATES_UPGRADE & BLOCK_VALID_MASK) == 0, "BLOCK_ACTIVATES_UPGRADE overlaps the validity-level field");
static_assert((BLOCK_IN_TMPFILE & BLOCK_VALID_MASK) == 0, "BLOCK_IN_TMPFILE overlaps the validity-level field");
//! Short-hand for the highest consensus validity we implement.
//! Blocks with this validity are assumed to satisfy all consensus rules.
static const BlockStatus BLOCK_VALID_CONSENSUS = BLOCK_VALID_SCRIPTS;
src/clientversion.h
+1 -1
View File
@@ -29,7 +29,7 @@
//! These need to be macros, as clientversion.cpp's and bitcoin*-res.rc's voodoo requires it
// Must be kept in sync with configure.ac , ugh!
#define CLIENT_VERSION_MAJOR 1
#define CLIENT_VERSION_MINOR 2
#define CLIENT_VERSION_MINOR 3
#define CLIENT_VERSION_REVISION 0
#define CLIENT_VERSION_BUILD 50
src/gtest/test_operation_status.cpp
+936
View File
@@ -0,0 +1,936 @@
// Copyright (c) 2016-2026 The Hush developers
// Distributed under the GPLv3 software license, see the accompanying
// file COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html
//
// z_getoperationstatus / z_getoperationresult / z_listoperationids after they stopped sorting
// whole UniValues, and after z_getoperationstatus and z_listoperationids stopped taking
// cs_main + cs_wallet. These tests show that:
// * both status calls return byte for byte what the original algorithm (copied below from
// c1040028e) returns, for queued, executing, successful, failed and cancelled operations,
// with and without id filters, with unknown and duplicate ids, with more than 16
// operations and with creation_time ties. std::sort reorders equal keys once there are
// more than 16 of them, so the order of same-second operations is part of the contract;
// * z_getoperationresult removes exactly the finished operations that it reports;
// * z_getoperationstatus and z_listoperationids answer while another thread holds
// cs_main + cs_wallet, and z_getoperationresult still waits for those locks;
// * AsyncRPCQueue::popOperationForId hands an operation to exactly one of several concurrent
// callers, so concurrent removers never report the same operation twice.
//
// No worker runs on the shared async queue in hush-gtest, so every operation keeps the state the
// test gives it. Without pwalletMain the handlers throw "Method not found", and
// z_getoperationresult locks pwalletMain->cs_wallet, so each test installs an empty in-memory
// CWallet. These tests use TEST() and a TestEnv guard rather than a fixture, because a fixture
// class cannot be named OperationStatus: that name is the operation state enum in
// asyncrpcoperation.h.
#include <gtest/gtest.h>
#include <univalue.h>
#include <algorithm>
#include <atomic>
#include <chrono>
#include <condition_variable>
#include <ctime>
#include <future>
#include <map>
#include <memory>
#include <mutex>
#include <set>
#include <stdexcept>
#include <string>
#include <thread>
#include <vector>
#include "asyncrpcoperation.h"
#include "asyncrpcqueue.h"
#include "init.h"
#include "main.h"
#include "pubkey.h"
#include "rpc/server.h"
#include "sync.h"
#include "wallet/wallet.h"
// Defined in wallet/rpcwallet.cpp (non-static, no header). Called directly so that removers can
// run without the cs_main + cs_wallet that z_getoperationresult takes.
extern UniValue z_getoperationstatus_IMPL(const UniValue& params, bool fRemoveFinishedOperations);
namespace {
// Bound for every wait that must succeed, so a regression fails instead of hanging.
const std::chrono::milliseconds kGuardTimeout(5000);
const char* const kQueueNotEmpty = "the shared async queue already holds operations from elsewhere";
const char* const kPoolFailed = "could not create operations with controlled creation_time ties";
enum FakeKind {
FAKE_QUEUED = 0,
FAKE_EXECUTING,
FAKE_SUCCESS,
FAKE_FAILED,
FAKE_CANCELLED,
FAKE_KINDS
};
// An operation whose state is set by the test and never changes after it is queued: main() does
// nothing, and nothing runs it anyway. getStatus() appends method/params exactly the way
// z_sendmany, z_mergetoaddress and z_shieldcoinbase append their contextinfo_.
class FixedStateOperation : public AsyncRPCOperation {
public:
virtual void main() {}
virtual UniValue getStatus() const {
UniValue v = AsyncRPCOperation::getStatus();
if (contextinfo_.isNull()) {
return v;
}
UniValue obj = v.get_obj();
obj.push_back(Pair("method", "z_fixedstate"));
obj.push_back(Pair("params", contextinfo_));
return obj;
}
// Only called before the operation is queued, like a real operation's constructor.
void SetContext(const UniValue& context) {
contextinfo_ = context;
}
void ForceExecuting() {
set_state(OperationStatus::EXECUTING);
}
// Same order as the real operations: stop the clock, then publish the result and SUCCESS.
void ForceSuccess(const UniValue& result) {
start_execution_clock();
stop_execution_clock();
set_result(result);
set_state(OperationStatus::SUCCESS);
}
void ForceFailed(int code, const std::string& message) {
set_error_code(code);
set_error_message(message);
set_state(OperationStatus::FAILED);
}
private:
UniValue contextinfo_;
};
std::shared_ptr<FixedStateOperation> MakeOperation(int kind, int tag)
{
std::shared_ptr<FixedStateOperation> op(new FixedStateOperation());
if (tag % 3 == 0) {
UniValue context(UniValue::VOBJ);
context.push_back(Pair("fromaddress", "zs1fixedstate"));
context.push_back(Pair("minconf", tag));
op->SetContext(context);
}
switch (kind) {
case FAKE_QUEUED:
break;
case FAKE_EXECUTING:
op->ForceExecuting();
break;
case FAKE_SUCCESS:
if (tag % 2) {
UniValue result(UniValue::VOBJ);
result.push_back(Pair("txid", "txid-" + std::to_string(tag)));
op->ForceSuccess(result);
} else {
op->ForceSuccess(UniValue("done"));
}
break;
case FAKE_FAILED:
op->ForceFailed(-6, "boom " + std::to_string(tag));
break;
case FAKE_CANCELLED:
op->cancel(); // READY -> CANCELLED, the same path shutdown and wallet.cpp use
break;
}
return op;
}
bool IsFinished(const AsyncRPCOperation& op)
{
return op.isSuccess() || op.isFailed() || op.isCancelled();
}
// The z_getoperationstatus_IMPL of c1040028e, verbatim except that the LOCK2 and the
// popOperationForId call are left out. Evaluated on an unchanged queue just before the call under
// test, it sees the same getAllOperationIds() enumeration and returns what the original returned:
// the original removing pass popped each operation only after reading it, so it reported exactly
// what this non-removing pass reports.
UniValue ReferenceOperationStatus(const UniValue& params, bool fRemoveFinishedOperations)
{
std::set<AsyncRPCOperationId> filter;
if (params.size()==1) {
UniValue ids = params[0].get_array();
for (const UniValue & v : ids.getValues()) {
filter.insert(v.get_str());
}
}
bool useFilter = (filter.size()>0);
UniValue ret(UniValue::VARR);
std::shared_ptr<AsyncRPCQueue> q = getAsyncRPCQueue();
std::vector<AsyncRPCOperationId> ids = q->getAllOperationIds();
for (auto id : ids) {
if (useFilter && !filter.count(id))
continue;
std::shared_ptr<AsyncRPCOperation> operation = q->getOperationForId(id);
if (!operation) {
continue;
}
UniValue obj = operation->getStatus();
std::string s = obj["status"].get_str();
if (fRemoveFinishedOperations) {
// Caller is only interested in retrieving finished results
if ("success"==s || "failed"==s || "cancelled"==s) {
ret.push_back(obj);
}
} else {
ret.push_back(obj);
}
}
std::vector<UniValue> arrTmp = ret.getValues();
// sort results chronologically by creation_time
std::sort(arrTmp.begin(), arrTmp.end(), [](UniValue a, UniValue b) -> bool {
const int64_t t1 = find_value(a.get_obj(), "creation_time").get_int64();
const int64_t t2 = find_value(b.get_obj(), "creation_time").get_int64();
return t1 < t2;
});
ret.clear();
ret.setArray();
ret.push_backV(arrTmp);
return ret;
}
// The body of c1040028e's z_listoperationids after its argument check, verbatim without the LOCK2.
UniValue ReferenceListOperationIds(const UniValue& params)
{
std::string filter;
bool useFilter = false;
if (params.size()==1) {
filter = params[0].get_str();
useFilter = true;
}
UniValue ret(UniValue::VARR);
std::shared_ptr<AsyncRPCQueue> q = getAsyncRPCQueue();
std::vector<AsyncRPCOperationId> ids = q->getAllOperationIds();
for (auto id : ids) {
std::shared_ptr<AsyncRPCOperation> operation = q->getOperationForId(id);
if (!operation) {
continue;
}
std::string state = operation->getStateAsString();
if (useFilter && filter.compare(state)!=0)
continue;
ret.push_back(id);
}
return ret;
}
// Results and errors as one comparable string. These helpers make no gtest calls, so helper
// threads can use them.
std::string CallOutcome(rpcfn_type fn, const UniValue& params)
{
try {
return fn(params, false, CPubKey()).write();
} catch (const UniValue& objError) {
return "rpc error: " + objError.write();
} catch (const std::exception& e) {
return std::string("exception: ") + e.what();
}
}
std::string ReferenceOutcome(const UniValue& params, bool fRemoveFinishedOperations)
{
try {
return ReferenceOperationStatus(params, fRemoveFinishedOperations).write();
} catch (const UniValue& objError) {
return "rpc error: " + objError.write();
} catch (const std::exception& e) {
return std::string("exception: ") + e.what();
}
}
std::string ReferenceListOutcome(const UniValue& params)
{
try {
return ReferenceListOperationIds(params).write();
} catch (const UniValue& objError) {
return "rpc error: " + objError.write();
} catch (const std::exception& e) {
return std::string("exception: ") + e.what();
}
}
// Byte comparison that reports where two (possibly very long) JSON texts first differ.
::testing::AssertionResult SameText(const std::string& original, const std::string& patched)
{
if (original == patched)
return ::testing::AssertionSuccess();
size_t i = 0;
while (i < original.size() && i < patched.size() && original[i] == patched[i])
i++;
const size_t from = i < 40 ? 0 : i - 40;
return ::testing::AssertionFailure()
<< "outputs differ at byte " << i << " (lengths " << original.size() << " / " << patched.size() << ")"
<< "\n original: ..." << original.substr(from, 120)
<< "\n patched: ..." << patched.substr(from, 120);
}
UniValue NoParams()
{
return UniValue(UniValue::VARR);
}
// [[id, ...]]
UniValue IdParams(const std::vector<std::string>& ids)
{
UniValue arr(UniValue::VARR);
for (size_t i = 0; i < ids.size(); i++)
arr.push_back(ids[i]);
UniValue params(UniValue::VARR);
params.push_back(arr);
return params;
}
// The ids a params array asks for. Empty means "all", as in the handlers.
std::set<std::string> RequestedIds(const UniValue& params)
{
std::set<std::string> ids;
if (params.size() == 1) {
const std::vector<UniValue>& v = params[0].get_array().getValues();
for (size_t i = 0; i < v.size(); i++)
ids.insert(v[i].get_str());
}
return ids;
}
std::set<std::string> ListedIds()
{
UniValue listed = z_listoperationids(NoParams(), false, CPubKey());
std::set<std::string> ids;
for (size_t i = 0; i < listed.size(); i++)
ids.insert(listed[i].get_str());
return ids;
}
// Installs an empty in-memory wallet as pwalletMain for one test, and when it goes out of scope
// removes every operation the test added to the shared queue and restores pwalletMain.
class TestEnv {
public:
TestEnv() : queue_(getAsyncRPCQueue()), savedWallet_(pwalletMain) {
wasEmpty_ = queue_->getAllOperationIds().empty();
pwalletMain = &wallet_;
}
~TestEnv() {
RemoveAll();
pwalletMain = savedWallet_;
}
bool QueueWasEmpty() const { return wasEmpty_; }
const std::shared_ptr<AsyncRPCQueue>& Queue() const { return queue_; }
// False if the queue refused an operation.
bool AddAll(const std::vector<std::shared_ptr<FixedStateOperation> >& ops) {
for (size_t i = 0; i < ops.size(); i++) {
if (!queue_->addOperation(ops[i]))
return false;
added_.push_back(ops[i]->getId());
}
return true;
}
void RemoveAll() {
for (size_t i = 0; i < added_.size(); i++)
queue_->popOperationForId(added_[i]);
added_.clear();
}
private:
std::shared_ptr<AsyncRPCQueue> queue_;
CWallet wallet_;
CWallet* savedWallet_;
bool wasEmpty_;
std::vector<AsyncRPCOperationId> added_;
};
// Holds cs_main and pwalletMain->cs_wallet on a helper thread, as a long wallet scan does, until
// Release() or destruction.
class WalletLockHolder {
public:
WalletLockHolder() : held_(false), release_(false), thread_(&WalletLockHolder::Run, this) {}
~WalletLockHolder() {
Release();
thread_.join();
}
bool WaitUntilHeld(std::chrono::milliseconds timeout) {
std::unique_lock<std::mutex> lk(m_);
return cv_.wait_for(lk, timeout, [this]() { return held_; });
}
void Release() {
{
std::lock_guard<std::mutex> guard(m_);
release_ = true;
}
cv_.notify_all();
}
private:
void Run() {
LOCK2(cs_main, pwalletMain->cs_wallet);
std::unique_lock<std::mutex> lk(m_);
held_ = true;
cv_.notify_all();
cv_.wait(lk, [this]() { return release_; });
}
std::mutex m_;
std::condition_variable cv_;
bool held_;
bool release_;
std::thread thread_; // last, so that everything Run() touches exists before it starts
};
// creation_time is time(NULL) at construction and cannot be set, so ties and distinct times are
// made by creating operations inside chosen seconds. Waits in 10 ms steps for the next second.
void WaitForNextSecond()
{
const time_t start = time(NULL);
for (int i = 0; i < 150 && time(NULL) == start; i++)
std::this_thread::sleep_for(std::chrono::milliseconds(10));
}
bool AllCreatedInOneSecond(const std::vector<std::shared_ptr<FixedStateOperation> >& ops)
{
for (size_t i = 1; i < ops.size(); i++) {
if (ops[i]->getCreationTime() != ops[0]->getCreationTime())
return false;
}
return true;
}
const int kPoolGroups = 3;
const size_t kPoolGroupSize = 300;
// kPoolGroups groups of operations in all five states. Every operation in a group has the same
// creation_time, and the groups' creation_times strictly increase.
struct OperationPool {
bool ok;
std::vector<std::vector<std::shared_ptr<FixedStateOperation> > > groups;
};
OperationPool BuildPool()
{
OperationPool pool;
pool.ok = true;
for (int g = 0; g < kPoolGroups && pool.ok; g++) {
std::vector<std::shared_ptr<FixedStateOperation> > group;
bool tied = false;
for (int attempt = 0; attempt < 3 && !tied; attempt++) {
if (g > 0 || attempt > 0)
WaitForNextSecond();
group.clear();
for (size_t i = 0; i < kPoolGroupSize; i++) {
group.push_back(MakeOperation(static_cast<int>((i + 2 * g) % FAKE_KINDS),
static_cast<int>(g * kPoolGroupSize + i)));
}
tied = AllCreatedInOneSecond(group);
}
const bool later = pool.groups.empty() ||
group.front()->getCreationTime() > pool.groups.back().front()->getCreationTime();
pool.ok = tied && later;
pool.groups.push_back(group);
}
return pool;
}
// Built once (it waits for up to about two second boundaries) and shared by the tests that need
// it. The operations are never modified after they are built.
const OperationPool& SharedPool()
{
static const OperationPool pool = BuildPool();
return pool;
}
struct Population {
std::string name;
std::vector<std::shared_ptr<FixedStateOperation> > ops;
};
// For each size: "ties-K", K operations from one second, and "mixed-K", K operations spread
// round robin over three seconds (ties within each second). 16 is the largest size that
// libstdc++'s std::sort handles with insertion sort alone; from 17 on, introsort reorders equal keys.
std::vector<Population> Populations(const OperationPool& pool)
{
static const size_t sizes[] = {0, 1, 2, 16, 17, 40, 300};
std::vector<Population> pops;
for (size_t s = 0; s < sizeof(sizes) / sizeof(sizes[0]); s++) {
const size_t k = sizes[s];
Population ties;
ties.name = "ties-" + std::to_string(k);
ties.ops.assign(pool.groups[1].begin(), pool.groups[1].begin() + k);
pops.push_back(ties);
Population mixed;
mixed.name = "mixed-" + std::to_string(k);
for (size_t i = 0; i < k; i++)
mixed.ops.push_back(pool.groups[i % kPoolGroups][i / kPoolGroups]);
pops.push_back(mixed);
}
return pops;
}
// Arguments for z_getoperationstatus/z_getoperationresult over a population: no argument, an
// empty list (no filter), an unknown id, two ids, a duplicated id, every other id plus an
// unknown one, and every id.
std::vector<UniValue> StatusParamSets(const std::vector<std::shared_ptr<FixedStateOperation> >& ops)
{
std::vector<UniValue> sets;
sets.push_back(NoParams());
sets.push_back(IdParams(std::vector<std::string>()));
sets.push_back(IdParams(std::vector<std::string>(1, "opid-unknown")));
if (!ops.empty()) {
std::vector<std::string> two;
two.push_back(ops.front()->getId());
two.push_back(ops.back()->getId());
sets.push_back(IdParams(two));
sets.push_back(IdParams(std::vector<std::string>(2, ops.front()->getId())));
std::vector<std::string> half, all;
for (size_t i = 0; i < ops.size(); i++) {
all.push_back(ops[i]->getId());
if (i % 2 == 0)
half.push_back(ops[i]->getId());
}
half.push_back("opid-unknown");
sets.push_back(IdParams(half));
sets.push_back(IdParams(all));
}
return sets;
}
// Pops every id once, as fast as it can, after the start signal.
void PopEach(AsyncRPCQueue* q, const std::vector<AsyncRPCOperationId>* ids,
const std::atomic<bool>* go, std::vector<AsyncRPCOperationId>* out)
{
while (!go->load())
std::this_thread::yield();
for (size_t i = 0; i < ids->size(); i++) {
if (q->popOperationForId((*ids)[i]) != nullptr)
out->push_back((*ids)[i]);
}
}
// Removes finished operations until the queue is empty (or kGuardTimeout passes) and records
// every id it was handed. If direct is true it calls the IMPL without cs_main/cs_wallet, so only
// the queue's atomic pop and the IMPL's checked pop keep it apart from the other removers.
// Otherwise it calls the z_getoperationresult handler, which takes the locks.
void DrainFinished(bool direct, const std::atomic<bool>* go, std::vector<std::string>* out)
{
while (!go->load())
std::this_thread::yield();
const UniValue params(UniValue::VARR);
std::shared_ptr<AsyncRPCQueue> q = getAsyncRPCQueue();
const std::chrono::steady_clock::time_point deadline = std::chrono::steady_clock::now() + kGuardTimeout;
while (std::chrono::steady_clock::now() < deadline) {
UniValue ret;
try {
ret = direct ? z_getoperationstatus_IMPL(params, true)
: z_getoperationresult(params, false, CPubKey());
for (size_t i = 0; i < ret.size(); i++)
out->push_back(find_value(ret[i], "id").get_str());
} catch (...) {
out->push_back("<exception>");
return;
}
if (ret.empty() && q->getAllOperationIds().empty())
return;
}
}
// Polls z_getoperationstatus while the removers run. Counts results that break its contract
// (only the test's finished operations exist): an unfinished status, a repeated id, or
// creation_time out of order.
void PollStatus(const std::atomic<bool>* go, const std::atomic<bool>* stop, int* faults)
{
while (!go->load())
std::this_thread::yield();
const UniValue params(UniValue::VARR);
while (!stop->load()) {
try {
UniValue ret = z_getoperationstatus(params, false, CPubKey());
std::set<std::string> ids;
int64_t last = 0;
for (size_t i = 0; i < ret.size(); i++) {
const std::string id = find_value(ret[i], "id").get_str();
const std::string status = find_value(ret[i], "status").get_str();
const int64_t t = find_value(ret[i], "creation_time").get_int64();
if (!ids.insert(id).second || t < last ||
(status != "success" && status != "failed" && status != "cancelled"))
(*faults)++;
last = t;
}
} catch (...) {
(*faults)++;
}
}
}
} // namespace
// The read path returns exactly what the original algorithm returned, including the order of
// same-second operations, and removes nothing.
TEST(OperationStatus, StatusMatchesOriginalAlgorithm)
{
const OperationPool& pool = SharedPool();
ASSERT_TRUE(pool.ok) << kPoolFailed;
TestEnv env;
ASSERT_TRUE(env.QueueWasEmpty()) << kQueueNotEmpty;
const std::vector<Population> pops = Populations(pool);
for (size_t p = 0; p < pops.size(); p++) {
ASSERT_TRUE(env.AddAll(pops[p].ops));
const std::vector<UniValue> paramSets = StatusParamSets(pops[p].ops);
for (size_t i = 0; i < paramSets.size(); i++) {
EXPECT_TRUE(SameText(ReferenceOutcome(paramSets[i], false),
CallOutcome(&z_getoperationstatus, paramSets[i])))
<< pops[p].name << " params=" << paramSets[i].write();
}
EXPECT_EQ(pops[p].ops.size(), env.Queue()->getAllOperationIds().size()) << pops[p].name;
env.RemoveAll();
}
#if defined(__GLIBCXX__)
// Check that the comparison above can fail. With 40 operations in one second, libstdc++'s
// introsort reorders the equal keys, so the original output is not the enumeration order.
// A (creation_time, index) comparator would keep the enumeration order and fail above.
std::vector<std::shared_ptr<FixedStateOperation> > tied(pool.groups[1].begin(), pool.groups[1].begin() + 40);
ASSERT_TRUE(env.AddAll(tied));
const UniValue ref = ReferenceOperationStatus(NoParams(), false);
std::vector<std::string> reported;
for (size_t i = 0; i < ref.size(); i++)
reported.push_back(find_value(ref[i], "id").get_str());
const std::vector<std::string> enumerated = env.Queue()->getAllOperationIds();
EXPECT_EQ(enumerated.size(), reported.size());
EXPECT_NE(enumerated, reported) << "40 tied operations kept their enumeration order";
env.RemoveAll();
#endif
}
// The removing path reports exactly what the original reported, removes exactly the finished
// operations it was asked about, and leaves the rest visible to the other two calls.
TEST(OperationStatus, ResultMatchesOriginalAlgorithm)
{
const OperationPool& pool = SharedPool();
ASSERT_TRUE(pool.ok) << kPoolFailed;
TestEnv env;
ASSERT_TRUE(env.QueueWasEmpty()) << kQueueNotEmpty;
const std::vector<Population> pops = Populations(pool);
for (size_t p = 0; p < pops.size(); p++) {
const std::vector<UniValue> paramSets = StatusParamSets(pops[p].ops);
for (size_t i = 0; i < paramSets.size(); i++) {
const UniValue& params = paramSets[i];
const std::string where = pops[p].name + " params=" + params.write();
ASSERT_TRUE(env.AddAll(pops[p].ops));
const std::string ref = ReferenceOutcome(params, true);
EXPECT_TRUE(SameText(ref, CallOutcome(&z_getoperationresult, params))) << where;
const std::set<std::string> requested = RequestedIds(params);
std::set<std::string> remaining;
int wrong = 0;
for (size_t j = 0; j < pops[p].ops.size(); j++) {
const std::shared_ptr<FixedStateOperation>& op = pops[p].ops[j];
const bool selected = requested.empty() || requested.count(op->getId()) > 0;
const bool present = env.Queue()->getOperationForId(op->getId()) != nullptr;
if (present != !(IsFinished(*op) && selected))
wrong++;
if (present)
remaining.insert(op->getId());
}
EXPECT_EQ(0, wrong) << where << ": operations removed or kept wrongly";
// Nothing is handed out twice, and the survivors are still listed and reported.
EXPECT_EQ("[]", CallOutcome(&z_getoperationresult, params)) << where;
EXPECT_EQ(remaining, ListedIds()) << where;
EXPECT_TRUE(SameText(ReferenceOutcome(NoParams(), false),
CallOutcome(&z_getoperationstatus, NoParams()))) << where;
env.RemoveAll();
}
}
}
TEST(OperationStatus, ListOperationIdsMatchesOriginalAlgorithm)
{
const OperationPool& pool = SharedPool();
ASSERT_TRUE(pool.ok) << kPoolFailed;
TestEnv env;
ASSERT_TRUE(env.QueueWasEmpty()) << kQueueNotEmpty;
const char* const filters[] = {NULL, "queued", "executing", "success", "failed", "cancelled", "bogus"};
const std::vector<Population> pops = Populations(pool);
for (size_t p = 0; p < pops.size(); p++) {
ASSERT_TRUE(env.AddAll(pops[p].ops));
for (size_t f = 0; f < sizeof(filters) / sizeof(filters[0]); f++) {
UniValue params(UniValue::VARR);
if (filters[f] != NULL)
params.push_back(filters[f]);
EXPECT_TRUE(SameText(ReferenceListOutcome(params), CallOutcome(&z_listoperationids, params)))
<< pops[p].name << " params=" << params.write();
}
env.RemoveAll();
}
}
// Malformed arguments fail exactly as before and remove nothing; no wallet still means
// "Method not found".
TEST(OperationStatus, ParameterErrorsUnchanged)
{
TestEnv env;
ASSERT_TRUE(env.QueueWasEmpty()) << kQueueNotEmpty;
std::vector<std::shared_ptr<FixedStateOperation> > ops;
for (int k = 0; k < FAKE_KINDS; k++)
ops.push_back(MakeOperation(k, k));
ASSERT_TRUE(env.AddAll(ops));
UniValue notArray(UniValue::VARR);
notArray.push_back("not_an_array");
UniValue inner(UniValue::VARR);
inner.push_back(ops[FAKE_SUCCESS]->getId());
inner.push_back(1);
UniValue notString(UniValue::VARR);
notString.push_back(inner);
EXPECT_EQ("exception: JSON value is not an array as expected", ReferenceOutcome(notArray, false));
EXPECT_EQ("exception: JSON value is not a string as expected", ReferenceOutcome(notString, false));
EXPECT_EQ(ReferenceOutcome(notArray, false), CallOutcome(&z_getoperationstatus, notArray));
EXPECT_EQ(ReferenceOutcome(notString, false), CallOutcome(&z_getoperationstatus, notString));
EXPECT_EQ(ReferenceOutcome(notArray, true), CallOutcome(&z_getoperationresult, notArray));
EXPECT_EQ(ReferenceOutcome(notString, true), CallOutcome(&z_getoperationresult, notString));
UniValue tooMany(UniValue::VARR);
tooMany.push_back(UniValue(UniValue::VARR));
tooMany.push_back("toomanyargs");
EXPECT_THROW(z_getoperationstatus(tooMany, false, CPubKey()), std::runtime_error);
EXPECT_THROW(z_getoperationresult(tooMany, false, CPubKey()), std::runtime_error);
EXPECT_THROW(z_listoperationids(tooMany, false, CPubKey()), std::runtime_error);
EXPECT_EQ(ops.size(), env.Queue()->getAllOperationIds().size());
CWallet* wallet = pwalletMain;
pwalletMain = NULL;
EXPECT_THROW(z_getoperationstatus(NoParams(), false, CPubKey()), UniValue);
EXPECT_THROW(z_getoperationresult(NoParams(), false, CPubKey()), UniValue);
EXPECT_THROW(z_listoperationids(NoParams(), false, CPubKey()), UniValue);
pwalletMain = wallet;
}
// While another thread holds cs_main + cs_wallet, z_getoperationstatus (unfiltered and filtered,
// as the GUI polls it) and z_listoperationids still answer, with unchanged output, and
// z_getoperationresult waits for the locks and then answers as before.
TEST(OperationStatus, StatusDoesNotWaitForWalletLocks)
{
TestEnv env;
ASSERT_TRUE(env.QueueWasEmpty()) << kQueueNotEmpty;
std::vector<std::shared_ptr<FixedStateOperation> > ops;
for (int k = 0; k < FAKE_KINDS; k++)
ops.push_back(MakeOperation(k, k));
ASSERT_TRUE(env.AddAll(ops));
const UniValue filtered = IdParams(std::vector<std::string>(1, ops[FAKE_SUCCESS]->getId()));
const std::string refStatus = ReferenceOutcome(NoParams(), false);
const std::string refFiltered = ReferenceOutcome(filtered, false);
const std::string refList = ReferenceListOutcome(NoParams());
const std::string refResult = ReferenceOutcome(NoParams(), true);
// Declared before the holder: on an early return the holder is destroyed first, which
// releases the locks, and only then do the futures' destructors wait for their calls.
std::future<std::string> status, statusFiltered, listed, result;
WalletLockHolder holder;
ASSERT_TRUE(holder.WaitUntilHeld(kGuardTimeout)) << "helper thread could not take cs_main + cs_wallet";
status = std::async(std::launch::async, CallOutcome, &z_getoperationstatus, NoParams());
statusFiltered = std::async(std::launch::async, CallOutcome, &z_getoperationstatus, filtered);
listed = std::async(std::launch::async, CallOutcome, &z_listoperationids, NoParams());
result = std::async(std::launch::async, CallOutcome, &z_getoperationresult, NoParams());
const bool statusDone = status.wait_for(kGuardTimeout) == std::future_status::ready;
const bool filteredDone = statusFiltered.wait_for(kGuardTimeout) == std::future_status::ready;
const bool listDone = listed.wait_for(kGuardTimeout) == std::future_status::ready;
// The locks are still held, so a z_getoperationresult that takes them cannot have finished.
const bool resultWaited = result.wait_for(std::chrono::milliseconds(100)) == std::future_status::timeout;
holder.Release();
const bool resultDone = result.wait_for(kGuardTimeout) == std::future_status::ready;
EXPECT_TRUE(statusDone) << "z_getoperationstatus waited for cs_main/cs_wallet";
EXPECT_TRUE(filteredDone) << "filtered z_getoperationstatus waited for cs_main/cs_wallet";
EXPECT_TRUE(listDone) << "z_listoperationids waited for cs_main/cs_wallet";
EXPECT_TRUE(resultWaited) << "z_getoperationresult no longer waits for cs_main/cs_wallet";
EXPECT_TRUE(resultDone) << "z_getoperationresult did not finish after the locks were released";
if (statusDone) {
EXPECT_EQ(refStatus, status.get());
}
if (filteredDone) {
EXPECT_EQ(refFiltered, statusFiltered.get());
}
if (listDone) {
EXPECT_EQ(refList, listed.get());
}
if (resultDone) {
EXPECT_EQ(refResult, result.get());
}
// The result call removed the three finished operations and kept the queued and executing ones.
std::set<std::string> expected;
expected.insert(ops[FAKE_QUEUED]->getId());
expected.insert(ops[FAKE_EXECUTING]->getId());
EXPECT_EQ(expected, ListedIds());
}
// SC-1: popOperationForId looks up and erases under one hold of the queue lock, so when several
// threads pop the same ids each operation goes to exactly one of them. The old two-step pop
// could hand the same operation to two callers; this catches that probabilistically.
TEST(OperationStatus, PopOperationForIdHandsEachOperationToOneCaller)
{
{
std::shared_ptr<AsyncRPCQueue> q = std::make_shared<AsyncRPCQueue>();
std::shared_ptr<FixedStateOperation> op = MakeOperation(FAKE_SUCCESS, 1);
ASSERT_TRUE(q->addOperation(op));
EXPECT_TRUE(q->popOperationForId("opid-unknown") == nullptr);
EXPECT_TRUE(q->popOperationForId(op->getId()).get() == op.get());
EXPECT_TRUE(q->getOperationForId(op->getId()) == nullptr);
EXPECT_TRUE(q->popOperationForId(op->getId()) == nullptr);
}
const int kThreads = 8;
const int kOps = 1000;
const int kRounds = 10;
for (int round = 0; round < kRounds; round++) {
// A private queue with no workers.
std::shared_ptr<AsyncRPCQueue> q = std::make_shared<AsyncRPCQueue>();
std::vector<AsyncRPCOperationId> ids;
for (int i = 0; i < kOps; i++) {
std::shared_ptr<AsyncRPCOperation> op(new FixedStateOperation());
ASSERT_TRUE(q->addOperation(op));
ids.push_back(op->getId());
}
std::atomic<bool> go(false);
std::vector<std::vector<AsyncRPCOperationId> > popped(kThreads);
std::vector<std::thread> threads;
for (int t = 0; t < kThreads; t++)
threads.push_back(std::thread(PopEach, q.get(), &ids, &go, &popped[t]));
go.store(true);
for (size_t t = 0; t < threads.size(); t++)
threads[t].join();
std::map<AsyncRPCOperationId, int> handed;
size_t total = 0;
for (int t = 0; t < kThreads; t++) {
for (size_t i = 0; i < popped[t].size(); i++) {
handed[popped[t][i]]++;
total++;
}
}
int wrong = 0;
for (size_t i = 0; i < ids.size(); i++) {
std::map<AsyncRPCOperationId, int>::const_iterator it = handed.find(ids[i]);
if (it == handed.end() || it->second != 1)
wrong++;
}
EXPECT_EQ(0, wrong) << "round " << round << ": operations not handed to exactly one caller";
EXPECT_EQ(static_cast<size_t>(kOps), total) << "round " << round;
EXPECT_TRUE(q->getAllOperationIds().empty()) << "round " << round;
}
}
// Removers through the locked handler and straight through the IMPL (no wallet locks) drain the
// same finished operations while two lock-free z_getoperationstatus pollers run. Every operation
// is reported exactly once, and no poll returns a malformed or unsorted list.
TEST(OperationStatus, ConcurrentRemoversReportEachOperationOnce)
{
TestEnv env;
ASSERT_TRUE(env.QueueWasEmpty()) << kQueueNotEmpty;
const int kOps = 200;
std::vector<std::shared_ptr<FixedStateOperation> > ops;
std::set<std::string> expected;
for (int i = 0; i < kOps; i++) {
ops.push_back(MakeOperation(FAKE_SUCCESS + i % 3, i)); // success, failed or cancelled
expected.insert(ops.back()->getId());
}
ASSERT_TRUE(env.AddAll(ops));
const int kRemovers = 8;
const int kReaders = 2;
std::atomic<bool> go(false);
std::atomic<bool> stop(false);
std::vector<std::vector<std::string> > removed(kRemovers);
std::vector<int> readerFaults(kReaders, 0);
std::vector<std::thread> removers, readers;
for (int t = 0; t < kRemovers; t++)
removers.push_back(std::thread(DrainFinished, t % 2 == 0, &go, &removed[t]));
for (int r = 0; r < kReaders; r++)
readers.push_back(std::thread(PollStatus, &go, &stop, &readerFaults[r]));
go.store(true);
for (size_t t = 0; t < removers.size(); t++)
removers[t].join();
stop.store(true);
for (size_t r = 0; r < readers.size(); r++)
readers[r].join();
std::map<std::string, int> reported;
size_t total = 0;
for (int t = 0; t < kRemovers; t++) {
for (size_t i = 0; i < removed[t].size(); i++) {
reported[removed[t][i]]++;
total++;
}
}
int duplicated = 0, unknown = 0;
for (std::map<std::string, int>::const_iterator it = reported.begin(); it != reported.end(); ++it) {
if (it->second != 1)
duplicated++;
if (!expected.count(it->first))
unknown++;
}
EXPECT_EQ(0, duplicated) << "an operation was reported by more than one remover";
EXPECT_EQ(0, unknown) << "a remover reported an unknown id or threw";
EXPECT_EQ(static_cast<size_t>(kOps), reported.size());
EXPECT_EQ(static_cast<size_t>(kOps), total);
EXPECT_TRUE(env.Queue()->getAllOperationIds().empty());
for (int r = 0; r < kReaders; r++)
EXPECT_EQ(0, readerFaults[r]) << "reader " << r;
}
// SC-4: the const status map with .at() names every state exactly as before.
TEST(OperationStatus, StateNamesUnchanged)
{
const char* const names[FAKE_KINDS] = {"queued", "executing", "success", "failed", "cancelled"};
for (int k = 0; k < FAKE_KINDS; k++) {
std::shared_ptr<FixedStateOperation> op = MakeOperation(k, k + 1);
EXPECT_EQ(std::string(names[k]), op->getStateAsString());
const UniValue status = op->getStatus();
EXPECT_EQ(std::string(names[k]), find_value(status, "status").get_str());
}
}
src/gtest/test_stratum_jobid.cpp
+98
View File
@@ -0,0 +1,98 @@
// Copyright (c) 2016-2026 The Hush developers
// Distributed under the GPLv3 software license, see the accompanying
// file COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html
//
// Regression coverage for the b3e81f1ed fix:
// "stratum: do not abort the daemon on a malformed 63-character job_id"
//
// The EWBF "31 bytes job_id" path in stratum_mining_submit() completes a
// 63-character job_id with each hex digit in turn and feeds the result to
// uint256(). ParseHex() stops at the first non-hex character and returns a
// SHORT vector without signalling an error, and base_blob's vector ctor
// asserts vch.size() == 32. asserts are live in release builds here, and the
// job_id arrives from an unauthenticated client -- so a single mining.submit
// whose 63-character job_id contains any non-hex byte (63 spaces will do)
// used to abort the node.
//
// The fix size-checks each candidate before constructing uint256. These tests
// pin the exact invariant that guard relies on, using the real ParseHex and
// uint256 primitives, without ever constructing a uint256 from a short vector
// (which would still abort under the guard we are protecting).
#include <gtest/gtest.h>
#include <string>
#include <vector>
#include "uint256.h"
#include "utilstrencodings.h"
namespace {
const std::string HEXDIGITS = "0123456789abcdef";
// Mirrors the guarded completion loop in stratum_mining_submit(): try every
// single-hex-digit completion and report whether ANY of them parses to a
// whole 32-byte value. Only then is uint256() construction reached.
bool AnyCompletionParsesTo32(const std::string& jobid) {
for (char d : HEXDIGITS) {
std::vector<unsigned char> vch = ParseHex(jobid + d);
if (vch.size() == 32) return true;
}
return false;
}
} // namespace
// The exact example from the fix commit: a 63-character job_id of spaces.
// No completion may reach a 32-byte vector, so the daemon never constructs
// uint256() and never aborts.
TEST(StratumJobId, AllSpacesNeverYields32Bytes) {
std::string spaces(63, ' ');
ASSERT_EQ(spaces.size(), 63u);
for (char d : HEXDIGITS) {
EXPECT_NE(ParseHex(spaces + d).size(), 32u)
<< "completion '" << d << "' unexpectedly produced 32 bytes";
}
EXPECT_FALSE(AnyCompletionParsesTo32(spaces));
}
// A single non-hex byte embedded in an otherwise-hex 63-char job_id is enough:
// ParseHex stops at it, so every completion is short.
TEST(StratumJobId, SingleNonHexByteInMiddleIsRejected) {
std::string jobid(63, 'a');
jobid[30] = 'g'; // 'g' is not a hex digit
ASSERT_EQ(jobid.size(), 63u);
EXPECT_FALSE(AnyCompletionParsesTo32(jobid));
}
// A non-hex byte at the very end (position 62) is likewise rejected: the last
// hex pair can never complete to a whole byte.
TEST(StratumJobId, NonHexByteAtEndIsRejected) {
std::string jobid(62, 'a');
jobid.push_back('z'); // length 63, last char non-hex
ASSERT_EQ(jobid.size(), 63u);
EXPECT_FALSE(AnyCompletionParsesTo32(jobid));
}
// A genuine truncated EWBF job_id -- 63 real hex characters -- must complete
// to exactly 32 bytes for every digit, so uint256() construction is safe.
TEST(StratumJobId, ValidSixtyThreeHexCompletesToExactly32Bytes) {
std::string jobid(63, 'a');
ASSERT_EQ(jobid.size(), 63u);
for (char d : HEXDIGITS) {
std::vector<unsigned char> vch = ParseHex(jobid + d);
ASSERT_EQ(vch.size(), 32u) << "completion '" << d << "'";
// 32-byte vector: construction must not trip the size assertion.
uint256 h(vch);
EXPECT_EQ(h.size(), 32u);
}
EXPECT_TRUE(AnyCompletionParsesTo32(jobid));
}
// Sanity anchors for ParseHex's odd/even handling that the loop depends on:
// an odd hex length drops the trailing nibble (63 hex -> 31 bytes), and one
// more hex char fills the 32nd byte (64 hex -> 32 bytes).
TEST(StratumJobId, ParseHexOddLengthDropsTrailingNibble) {
EXPECT_EQ(ParseHex(std::string(63, 'a')).size(), 31u);
EXPECT_EQ(ParseHex(std::string(64, 'a')).size(), 32u);
}
src/gtest/test_wallet_scan_notes.cpp
+848
View File
@@ -0,0 +1,848 @@
// Copyright (c) 2016-2026 The Hush developers
// Distributed under the GPLv3 software license, see the accompanying
// file COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html
//
// Proves that the optimised CWallet::GetFilteredNotes returns exactly what the original
// did: the same entries, in the same order, with every field equal. The optimisations
// are references instead of copies, skipping txs without note data, the spent check
// before trial decryption, the exact ivk pre-filter for address-filtered calls, and
// building the note and address from decrypt_with_pk_d. RefGetFilteredNotes below is a
// verbatim copy of the original body (c1040028e), and both run over the same hand-built
// wallet for a grid of filters and parameters. The file also tests
// SaplingNotePlaintext::decrypt_with_pk_d against decrypt().
//
// No proofs are needed: the wallet never verifies them, so notes are encrypted by hand
// into fabricated OutputDescriptions and added with AddToWallet(wtx, true, NULL). Depth
// is faked with a mapBlockIndex chain (depth >= 1), the mempool (depth 0) or neither
// (depth -1). minDepth > 1 is not covered: that branch calls tx_height(), which reads
// the tx from disk (the regtest differential covers it).
#include <gtest/gtest.h>
#include "arith_uint256.h"
#include "chain.h"
#include "chainparams.h"
#include "key_io.h"
#include "main.h"
#include "primitives/transaction.h"
#include "txmempool.h"
#include "uint256.h"
#include "util.h"
#include "wallet/crypter.h"
#include "wallet/wallet.h"
#include "zcash/Address.hpp"
#include "zcash/Note.hpp"
#include "zcash/zip32.h"
#include <algorithm>
#include <array>
#include <cassert>
#include <climits>
#include <list>
#include <memory>
#include <set>
#include <string>
#include <vector>
#include <boost/filesystem.hpp>
// The original body calls these; wallet.cpp declares them the same way (wallet.cpp:77-78).
// Both are defined in main.cpp (via hush.h). Neither is reached here (minDepth <= 1).
int32_t hush_dpowconfs(int32_t height,int32_t numconfs);
int tx_height( const uint256 &hash );
/**
* Verbatim copy of the original
* CWallet::GetFilteredNotes(std::vector<SaplingNoteEntry>&, std::set<PaymentAddress>&,
* int, int, bool, bool, bool) at c1040028e (wallet.cpp:5637-5721). The only changes:
* it is a free function, so wallet members are `wallet.`-qualified (all of them are
* public: cs_wallet, mapWallet, IsSaplingSpent, GetSaplingIncomingViewingKey,
* GetSaplingFullViewingKey, HaveSaplingSpendingKey, IsLockedNote), and the two LogPrintf
* calls are dropped (they do not affect the output).
*/
static void RefGetFilteredNotes(
CWallet& wallet,
std::vector<SaplingNoteEntry>& saplingEntries,
std::set<libzcash::PaymentAddress>& filterAddresses,
int minDepth,
int maxDepth,
bool ignoreSpent,
bool requireSpendingKey,
bool ignoreLocked)
{
using namespace libzcash;
LOCK2(cs_main, wallet.cs_wallet);
for (auto & p : wallet.mapWallet) {
CWalletTx wtx = p.second;
// Filter the transactions before checking for notes
if (!CheckFinalTx(wtx) || wtx.GetBlocksToMaturity() > 0)
continue;
if (minDepth > 1) {
int nHeight = tx_height(wtx.GetHash());
if ( nHeight == 0 ) {
continue;
}
int nDepth = wtx.GetDepthInMainChain();
int dpowconfs = hush_dpowconfs(nHeight,nDepth);
if ( dpowconfs < minDepth || dpowconfs > maxDepth) {
continue;
}
} else {
if ( wtx.GetDepthInMainChain() < minDepth ||
wtx.GetDepthInMainChain() > maxDepth) {
continue;
}
}
for (auto & pair : wtx.mapSaplingNoteData) {
SaplingOutPoint op = pair.first;
SaplingNoteData nd = pair.second;
auto maybe_pt = SaplingNotePlaintext::decrypt(
wtx.vShieldedOutput[op.n].encCiphertext,
nd.ivk,
wtx.vShieldedOutput[op.n].ephemeralKey,
wtx.vShieldedOutput[op.n].cm);
assert(static_cast<bool>(maybe_pt));
auto notePt = maybe_pt.get();
auto maybe_pa = nd.ivk.address(notePt.d);
assert(static_cast<bool>(maybe_pa));
auto pa = maybe_pa.get();
// skip notes which belong to a different payment address in the wallet
if (!(filterAddresses.empty() || filterAddresses.count(pa))) {
continue;
}
if (ignoreSpent && nd.nullifier && wallet.IsSaplingSpent(*nd.nullifier)) {
continue;
}
// skip notes which cannot be spent
if (requireSpendingKey) {
libzcash::SaplingIncomingViewingKey ivk;
libzcash::SaplingFullViewingKey fvk;
if (!(wallet.GetSaplingIncomingViewingKey(pa, ivk) &&
wallet.GetSaplingFullViewingKey(ivk, fvk) &&
wallet.HaveSaplingSpendingKey(fvk))) {
continue;
}
}
// skip locked notes
if (ignoreLocked && wallet.IsLockedNote(op)) {
continue;
}
auto note = notePt.note(nd.ivk).get();
saplingEntries.push_back(SaplingNoteEntry {
op, pa, note, notePt.memo(), wtx.GetDepthInMainChain() });
}
}
}
namespace {
// Heights 0..CHAIN_LENGTH-1, so a tx in the block at height h has depth CHAIN_LENGTH - h.
const int CHAIN_LENGTH = 10;
// The Jubjub subgroup order r_J. An ivk + r_J generates the same addresses as ivk,
// because librustzcash_ivk_to_pkd multiplies by the raw 256-bit value.
const char JUBJUB_R_HEX[] = "0e7db4ea6533afa906673b0101343b00a6682093ccc81082d0970e5ed6f72cb7";
// Keys and addresses of the test wallet (see WalletScanNotes::BuildWallet).
struct TestKeys {
libzcash::SaplingFullViewingKey fvk1, fvk2;
libzcash::SaplingIncomingViewingKey ivk1, ivk2, ivk3, ivk4, ivkN, aliasIvk1;
libzcash::SaplingPaymentAddress a1; // K1 default address (spending key)
libzcash::SaplingPaymentAddress d1; // K1 diversified address, recorded correctly
libzcash::SaplingPaymentAddress u1; // K1 diversified address, never recorded
libzcash::SaplingPaymentAddress x1; // K1 diversified address, recorded with K2's ivk
libzcash::SaplingPaymentAddress y1; // K1 diversified address, recorded with ivk1 + r_J
libzcash::SaplingPaymentAddress a2; // K2 default address (spending key)
libzcash::SaplingPaymentAddress a3; // K3: incoming viewing key only
libzcash::SaplingPaymentAddress a4; // K4: full viewing key only
libzcash::SaplingPaymentAddress aN; // address of a non-canonical ivk (>= 2^251)
libzcash::SaplingPaymentAddress foreign; // not in the wallet at all
std::vector<libzcash::SaplingPaymentAddress> empty; // spending keys that hold no notes
size_t nNotes;
};
// One output of a fabricated tx that the wallet owns: the address it pays, the ivk the
// wallet records for it (SaplingNoteData::ivk) and its cached nullifier, if any.
struct OwnedOutput {
libzcash::SaplingPaymentAddress addr;
libzcash::SaplingIncomingViewingKey ivk;
uint64_t value;
boost::optional<uint256> nullifier;
};
struct NamedFilter {
std::string name;
std::set<libzcash::PaymentAddress> addrs;
};
// CCryptoKeyStore::EncryptKeys is protected, and CWallet::EncryptWallet needs a wallet file.
class EncryptableWallet : public CWallet {
public:
bool EncryptKeysForTest(CKeyingMaterial& vMasterKeyIn) { return EncryptKeys(vMasterKeyIn); }
};
libzcash::SaplingExtendedSpendingKey KeyFromSeedByte(unsigned char b)
{
RawHDSeed raw(32, b);
HDSeed seed(raw);
return libzcash::SaplingExtendedSpendingKey::Master(seed);
}
// A diversified address of xsk other than its default one: the first valid diversifier
// index at or after `start` (the default address uses the first one from 0).
libzcash::SaplingPaymentAddress DiversifiedAddress(
const libzcash::SaplingExtendedSpendingKey& xsk, unsigned char start)
{
libzcash::diversifier_index_t j;
j.begin()[0] = start;
return xsk.ToXFVK().Address(j).get().second;
}
// A synthetic nullifier. GetFilteredNotes only compares nullifiers, it never recomputes them.
uint256 Nf(unsigned char n)
{
uint256 nf;
nf.begin()[0] = n;
nf.begin()[31] = 0x4e;
return nf;
}
// A shielded output paying `value` to `addr`, encrypted the way a real sender does
// (SaplingNotePlaintext::encrypt), with a random rcm and a memo derived from the value.
// There is no proof, value commitment or outgoing ciphertext: the wallet reads only cm,
// ephemeralKey and encCiphertext.
OutputDescription MakeOutput(const libzcash::SaplingPaymentAddress& addr, uint64_t value)
{
libzcash::SaplingNote note(addr, value);
std::array<unsigned char, HUSH_MEMO_SIZE> memo;
memo.fill(0);
memo[0] = value & 0xff;
memo[1] = (value >> 8) & 0xff;
libzcash::SaplingNotePlaintext pt(note, memo);
libzcash::SaplingNotePlaintextEncryptionResult res = pt.encrypt(note.pk_d).get();
OutputDescription od;
od.cv = uint256();
od.cm = note.cm().get();
od.ephemeralKey = res.second.get_epk();
od.encCiphertext = res.first;
od.outCiphertext.fill(0);
od.zkproof.fill(0);
return od;
}
std::set<libzcash::PaymentAddress> Filter(const std::vector<libzcash::SaplingPaymentAddress>& addrs)
{
std::set<libzcash::PaymentAddress> s;
for (const libzcash::SaplingPaymentAddress& a : addrs) {
s.insert(a);
}
return s;
}
void ExpectSameEntries(const std::vector<SaplingNoteEntry>& expected,
const std::vector<SaplingNoteEntry>& actual)
{
ASSERT_EQ(expected.size(), actual.size());
for (size_t i = 0; i < expected.size(); i++) {
SCOPED_TRACE(testing::Message() << "entry " << i);
const SaplingNoteEntry& e = expected[i];
const SaplingNoteEntry& a = actual[i];
EXPECT_EQ(e.op.hash, a.op.hash);
EXPECT_EQ(e.op.n, a.op.n);
EXPECT_TRUE(e.address == a.address);
EXPECT_TRUE(e.note.d == a.note.d);
EXPECT_EQ(e.note.pk_d, a.note.pk_d);
EXPECT_EQ(e.note.value(), a.note.value());
EXPECT_EQ(e.note.r, a.note.r);
EXPECT_TRUE(e.memo == a.memo);
EXPECT_EQ(e.confirmations, a.confirmations);
}
}
// Runs the original and the optimised GetFilteredNotes with the same arguments and
// expects the same entries in the same order. Returns the number of entries.
size_t ExpectMatchesReference(CWallet& wallet, NamedFilter& filter,
int minDepth, int maxDepth,
bool ignoreSpent, bool requireSpendingKey, bool ignoreLocked)
{
SCOPED_TRACE(testing::Message() << "filter=" << filter.name
<< " minDepth=" << minDepth << " maxDepth=" << maxDepth
<< " ignoreSpent=" << ignoreSpent
<< " requireSpendingKey=" << requireSpendingKey
<< " ignoreLocked=" << ignoreLocked);
std::vector<SaplingNoteEntry> expected;
std::vector<SaplingNoteEntry> actual;
RefGetFilteredNotes(wallet, expected, filter.addrs,
minDepth, maxDepth, ignoreSpent, requireSpendingKey, ignoreLocked);
wallet.GetFilteredNotes(actual, filter.addrs,
minDepth, maxDepth, ignoreSpent, requireSpendingKey, ignoreLocked);
ExpectSameEntries(expected, actual);
return expected.size();
}
// The reference's result size with maxDepth INT_MAX.
size_t RefCount(CWallet& wallet, std::set<libzcash::PaymentAddress> filter, int minDepth,
bool ignoreSpent, bool requireSpendingKey, bool ignoreLocked)
{
std::vector<SaplingNoteEntry> entries;
RefGetFilteredNotes(wallet, entries, filter,
minDepth, INT_MAX, ignoreSpent, requireSpendingKey, ignoreLocked);
return entries.size();
}
// Every filter shape the pre-filter distinguishes.
std::vector<NamedFilter> AllFilters(const TestKeys& k)
{
// 16 recorded, verifiable addresses: the pre-filter is on (the cap is inclusive).
std::vector<libzcash::SaplingPaymentAddress> sixteen = {k.a1, k.d1, k.a2, k.a3, k.a4};
sixteen.insert(sixteen.end(), k.empty.begin(), k.empty.begin() + 11);
// 17: the pre-filter is off only because of the cap.
std::vector<libzcash::SaplingPaymentAddress> seventeen = sixteen;
seventeen.push_back(k.empty[11]);
std::set<libzcash::PaymentAddress> invalidOnly;
invalidOnly.insert(libzcash::InvalidEncoding());
std::set<libzcash::PaymentAddress> invalidAndA2 = invalidOnly;
invalidAndA2.insert(k.a2);
std::vector<NamedFilter> f;
f.push_back(NamedFilter{"{}", Filter({})});
f.push_back(NamedFilter{"{a1}", Filter({k.a1})});
f.push_back(NamedFilter{"{d1}", Filter({k.d1})});
f.push_back(NamedFilter{"{a2}", Filter({k.a2})});
f.push_back(NamedFilter{"{a3 ivk-only}", Filter({k.a3})});
f.push_back(NamedFilter{"{a4 fvk-only}", Filter({k.a4})});
f.push_back(NamedFilter{"{a1,a2,a3,a4,d1}", Filter({k.a1, k.a2, k.a3, k.a4, k.d1})});
f.push_back(NamedFilter{"{u1 unrecorded}", Filter({k.u1})});
f.push_back(NamedFilter{"{a1,u1 unrecorded}", Filter({k.a1, k.u1})});
f.push_back(NamedFilter{"{x1 mismatched}", Filter({k.x1})});
f.push_back(NamedFilter{"{x1 mismatched,a2}", Filter({k.x1, k.a2})});
f.push_back(NamedFilter{"{y1 aliased}", Filter({k.y1})});
f.push_back(NamedFilter{"{aN non-canonical}", Filter({k.aN})});
f.push_back(NamedFilter{"{no notes}", Filter({k.empty[0]})});
f.push_back(NamedFilter{"{foreign}", Filter({k.foreign})});
f.push_back(NamedFilter{"{invalid}", invalidOnly});
f.push_back(NamedFilter{"{invalid,a2}", invalidAndA2});
f.push_back(NamedFilter{"16 addresses", Filter(sixteen)});
f.push_back(NamedFilter{"17 addresses", Filter(seventeen)});
return f;
}
// decrypt_with_pk_d must succeed exactly when decrypt() does, and must leave pk_d_out
// alone when it fails. Returns whether decryption succeeded.
bool DecryptBothWays(const libzcash::SaplingEncCiphertext& ct, const uint256& ivk,
const uint256& epk, const uint256& cmu)
{
uint256 sentinel;
std::fill(sentinel.begin(), sentinel.end(), 0xa5);
uint256 pk_d = sentinel;
auto expected = libzcash::SaplingNotePlaintext::decrypt(ct, ivk, epk, cmu);
auto actual = libzcash::SaplingNotePlaintext::decrypt_with_pk_d(ct, ivk, epk, cmu, pk_d);
EXPECT_EQ(static_cast<bool>(expected), static_cast<bool>(actual));
if (!actual) {
EXPECT_EQ(pk_d, sentinel);
}
return static_cast<bool>(actual);
}
} // namespace
class WalletScanNotes : public ::testing::Test {
protected:
boost::filesystem::path datadir;
bool hadDatadirArg;
std::string oldDatadirArg;
bool oldPrintToDebugLog;
CBlockIndex* oldTip;
std::vector<std::unique_ptr<CBlockIndex>> blocks;
std::vector<CTransaction> mempoolTxs;
void SetUp() override
{
SelectParams(CBaseChainParams::REGTEST);
// GetFilteredNotes logs on every call: keep that away from any real datadir and
// out of debug.log.
hadDatadirArg = mapArgs.count("-datadir") > 0;
if (hadDatadirArg) {
oldDatadirArg = mapArgs["-datadir"];
}
datadir = boost::filesystem::temp_directory_path() / boost::filesystem::unique_path();
boost::filesystem::create_directories(datadir);
mapArgs["-datadir"] = datadir.string();
ClearDatadirCache();
oldPrintToDebugLog = fPrintToDebugLog;
fPrintToDebugLog = false;
// A fake active chain. Only what CMerkleTx::GetDepthInMainChain reads is set: the
// mapBlockIndex entry, phashBlock, the height and pprev.
LOCK(cs_main);
oldTip = chainActive.Tip();
for (int h = 0; h < CHAIN_LENGTH; h++) {
std::unique_ptr<CBlockIndex> pindex(new CBlockIndex());
pindex->SetHeight(h);
pindex->pprev = blocks.empty() ? NULL : blocks.back().get();
uint256 hash;
hash.begin()[0] = h;
hash.begin()[30] = 0x5c;
hash.begin()[31] = 0xa9;
auto ret = mapBlockIndex.insert(std::make_pair(hash, pindex.get()));
ASSERT_TRUE(ret.second);
pindex->phashBlock = &ret.first->first;
blocks.push_back(std::move(pindex));
}
chainActive.SetTip(blocks.back().get());
}
void TearDown() override
{
{
LOCK(cs_main);
for (const CTransaction& tx : mempoolTxs) {
std::list<CTransaction> removed;
mempool.remove(tx, removed, false);
}
mempoolTxs.clear();
chainActive.SetTip(oldTip);
for (const std::unique_ptr<CBlockIndex>& pindex : blocks) {
mapBlockIndex.erase(pindex->GetBlockHash());
}
blocks.clear();
}
fPrintToDebugLog = oldPrintToDebugLog;
if (hadDatadirArg) {
mapArgs["-datadir"] = oldDatadirArg;
} else {
mapArgs.erase("-datadir");
}
ClearDatadirCache();
boost::system::error_code ec;
boost::filesystem::remove_all(datadir, ec);
// The network every other test in this binary selects for itself.
SelectParams(CBaseChainParams::MAIN);
}
// Adds a fabricated tx to the wallet the way LoadWallet does: AddToWallet(wtx, true,
// NULL) skips the database and proof checks but fills the nullifier and spend maps.
// `owned` become outputs 0..n-1 with wallet note data, `foreign` outputs follow with
// none, and `spends` are the nullifiers the tx reveals. depth >= 1 puts the tx in the
// fake chain, 0 in the mempool, -1 nowhere (like a conflicted tx).
uint256 AddTx(CWallet& wallet, int depth,
const std::vector<OwnedOutput>& owned,
const std::vector<libzcash::SaplingPaymentAddress>& foreign,
const std::vector<uint256>& spends,
bool transparentOutput = false)
{
assert(depth >= -1 && depth <= CHAIN_LENGTH);
CMutableTransaction mtx;
mtx.fOverwintered = true;
mtx.nVersionGroupId = SAPLING_VERSION_GROUP_ID;
mtx.nVersion = SAPLING_TX_VERSION;
for (const OwnedOutput& o : owned) {
mtx.vShieldedOutput.push_back(MakeOutput(o.addr, o.value));
}
for (const libzcash::SaplingPaymentAddress& addr : foreign) {
mtx.vShieldedOutput.push_back(MakeOutput(addr, 1));
}
for (const uint256& nf : spends) {
SpendDescription sd;
sd.nullifier = nf;
sd.zkproof.fill(0);
sd.spendAuthSig.fill(0);
mtx.vShieldedSpend.push_back(sd);
}
if (transparentOutput) {
mtx.vout.push_back(CTxOut(1000, CScript() << OP_TRUE));
}
CTransaction tx(mtx);
CWalletTx wtx(&wallet, tx);
for (size_t i = 0; i < owned.size(); i++) {
SaplingNoteData nd(owned[i].ivk);
nd.nullifier = owned[i].nullifier;
wtx.mapSaplingNoteData[SaplingOutPoint(tx.GetHash(), i)] = nd;
}
if (depth >= 1) {
wtx.hashBlock = blocks[CHAIN_LENGTH - depth]->GetBlockHash();
wtx.nIndex = 0;
// There is no real block to check a merkle branch against.
wtx.fMerkleVerified = true;
} else if (depth == 0) {
mempool.addUnchecked(tx.GetHash(),
CTxMemPoolEntry(tx, 0, GetTime(), 0.0, chainActive.Height(), true, false, SAPLING_BRANCH_ID),
false);
mempoolTxs.push_back(tx);
}
wallet.AddToWallet(wtx, true, NULL);
return tx.GetHash();
}
// The test wallet. Every key state, every recorded-address state and every note state
// GetFilteredNotes distinguishes, spread over several depths:
// K1 spending key: a1 (default), d1 (recorded diversified), u1 (unrecorded),
// x1 (recorded with K2's ivk, as z_importviewingkey allows), y1 (recorded with
// ivk1 + r_J, which still generates y1);
// K2 spending key: a2; K3 ivk only: a3; K4 fvk only: a4;
// ivkN: a non-canonical ivk in [2^251, r_J) and its address aN;
// 12 more spending keys with no notes; one foreign key.
// Notes are spent in a block, spent in the mempool, "spent" by a conflicted tx (so
// still unspent), unspent, without a cached nullifier although spent on chain, and
// locked. There are also txs without note data (spends only, transparent only, and
// a shielded output to a foreign address).
void BuildWallet(CWallet& wallet, TestKeys& k)
{
LOCK2(cs_main, wallet.cs_wallet);
libzcash::SaplingExtendedSpendingKey xsk1 = KeyFromSeedByte(0x11);
libzcash::SaplingExtendedSpendingKey xsk2 = KeyFromSeedByte(0x22);
libzcash::SaplingExtendedSpendingKey xsk3 = KeyFromSeedByte(0x33);
libzcash::SaplingExtendedSpendingKey xsk4 = KeyFromSeedByte(0x44);
libzcash::SaplingExtendedSpendingKey xskN = KeyFromSeedByte(0x55);
k.fvk1 = xsk1.expsk.full_viewing_key();
k.fvk2 = xsk2.expsk.full_viewing_key();
libzcash::SaplingFullViewingKey fvk4 = xsk4.expsk.full_viewing_key();
k.ivk1 = k.fvk1.in_viewing_key();
k.ivk2 = k.fvk2.in_viewing_key();
k.ivk3 = xsk3.expsk.full_viewing_key().in_viewing_key();
k.ivk4 = fvk4.in_viewing_key();
k.a1 = xsk1.DefaultAddress();
k.d1 = DiversifiedAddress(xsk1, 50);
k.u1 = DiversifiedAddress(xsk1, 100);
k.x1 = DiversifiedAddress(xsk1, 150);
k.y1 = DiversifiedAddress(xsk1, 200);
std::set<libzcash::SaplingPaymentAddress> k1addrs = {k.a1, k.d1, k.u1, k.x1, k.y1};
ASSERT_EQ(k1addrs.size(), (size_t)5);
k.a2 = xsk2.DefaultAddress();
k.a3 = xsk3.DefaultAddress();
k.a4 = xsk4.DefaultAddress();
k.foreign = KeyFromSeedByte(0x66).DefaultAddress();
// Genuine ivks are canonical (top 5 bits clear, so below 2^251).
ASSERT_EQ(k.ivk1.begin()[31] & 0xF8, 0);
ASSERT_EQ(k.ivk2.begin()[31] & 0xF8, 0);
// ivk1 + r_J generates exactly ivk1's addresses. If this ever stops holding, the
// y1 case no longer exercises the pre-filter's canonical-ivk guard.
k.aliasIvk1 = libzcash::SaplingIncomingViewingKey(ArithToUint256(
UintToArith256(k.ivk1) + UintToArith256(uint256S(JUBJUB_R_HEX))));
ASSERT_NE(k.aliasIvk1.begin()[31] & 0xF8, 0);
auto aliasY1 = k.aliasIvk1.address(k.y1.d);
ASSERT_TRUE(static_cast<bool>(aliasY1));
EXPECT_TRUE(aliasY1.get() == k.y1);
// ivkN in [2^251, 2^251 + 2^248), below r_J: ka_agree accepts it, so notes to aN
// decrypt normally, but it is not canonical.
uint256 rawN = xskN.expsk.full_viewing_key().in_viewing_key();
rawN.begin()[31] = 0x08;
k.ivkN = libzcash::SaplingIncomingViewingKey(rawN);
k.aN = k.ivkN.address(k.a1.d).get();
ASSERT_TRUE(wallet.AddSaplingZKey(xsk1, k.a1));
ASSERT_TRUE(wallet.AddSaplingIncomingViewingKey(k.ivk1, k.d1));
ASSERT_TRUE(wallet.AddSaplingZKey(xsk2, k.a2));
ASSERT_TRUE(wallet.AddSaplingIncomingViewingKey(k.ivk3, k.a3));
ASSERT_TRUE(wallet.AddSaplingFullViewingKey(fvk4, k.a4));
ASSERT_TRUE(wallet.AddSaplingIncomingViewingKey(k.ivk2, k.x1));
ASSERT_TRUE(wallet.AddSaplingIncomingViewingKey(k.aliasIvk1, k.y1));
ASSERT_TRUE(wallet.AddSaplingIncomingViewingKey(k.ivkN, k.aN));
for (int i = 0; i < 12; i++) {
libzcash::SaplingExtendedSpendingKey xsk = KeyFromSeedByte(0x70 + i);
libzcash::SaplingPaymentAddress addr = xsk.DefaultAddress();
ASSERT_TRUE(wallet.AddSaplingZKey(xsk, addr));
k.empty.push_back(addr);
}
const boost::optional<uint256> noNf;
std::vector<uint256> noSpends;
std::vector<libzcash::SaplingPaymentAddress> noForeign;
// Mixed ownership in one tx; Nf(2) is spent in a block.
AddTx(wallet, 5, {{k.a1, k.ivk1, 1001, Nf(1)},
{k.a1, k.ivk1, 1002, Nf(2)},
{k.a2, k.ivk2, 2001, Nf(3)}}, noForeign, noSpends);
// Nf(4) is spent in the mempool. The IVK-only note has no cached nullifier, but its
// on-chain nullifier Nf(5) is spent in a block: it must still count as unspent.
AddTx(wallet, 1, {{k.d1, k.ivk1, 1003, Nf(4)},
{k.a3, k.ivk3, 3001, noNf}}, noForeign, noSpends);
// u1's note is locked; x1's note is "spent" only by a conflicted tx.
uint256 hashUX = AddTx(wallet, 3, {{k.u1, k.ivk1, 1004, Nf(6)},
{k.x1, k.ivk1, 1005, Nf(7)}}, noForeign, noSpends);
// In the mempool (depth 0).
AddTx(wallet, 0, {{k.a1, k.ivk1, 1006, Nf(8)},
{k.a4, k.ivk4, 4001, Nf(9)}}, noForeign, noSpends);
// Conflicted (depth -1).
AddTx(wallet, -1, {{k.a2, k.ivk2, 2002, Nf(10)}}, noForeign, noSpends);
// Deepest block; Nf(11) is spent in a block.
AddTx(wallet, 10, {{k.y1, k.ivk1, 1007, Nf(11)},
{k.aN, k.ivkN, 5001, noNf}}, noForeign, noSpends);
// a1's note is locked; one output goes to a foreign address (no note data).
uint256 hashLocked = AddTx(wallet, 2, {{k.a1, k.ivk1, 1008, Nf(12)},
{k.a2, k.ivk2, 2003, Nf(13)}},
{k.foreign}, noSpends);
// Nf(14) is spent in a block.
AddTx(wallet, 4, {{k.a3, k.ivk3, 3002, noNf},
{k.aN, k.ivkN, 5002, Nf(14)},
{k.x1, k.ivk1, 1009, Nf(15)}}, noForeign, noSpends);
AddTx(wallet, 1, {{k.d1, k.ivk1, 1010, Nf(16)}}, noForeign, noSpends);
k.nNotes = 18;
// Txs without note data: the spenders (in a block, in the mempool, conflicted), a
// transparent-only tx and a tx whose only output pays a foreign address.
AddTx(wallet, 2, {}, noForeign, {Nf(2), Nf(5), Nf(11), Nf(14)});
AddTx(wallet, 0, {}, noForeign, {Nf(4)});
AddTx(wallet, -1, {}, noForeign, {Nf(7)});
AddTx(wallet, 3, {}, noForeign, noSpends, true);
AddTx(wallet, 1, {}, {k.foreign}, noSpends);
wallet.LockNote(SaplingOutPoint(hashUX, 0));
wallet.LockNote(SaplingOutPoint(hashLocked, 0));
}
};
// The fixture really has what the equivalence tests rely on: every note is visible, the
// spent, depth, key and lock states are the intended ones, and the regression cases
// (unrecorded, mismatched, aliased and non-canonical addresses) hold notes, so equivalence
// on them is not vacuous.
TEST_F(WalletScanNotes, ReferenceSeesTheFixture)
{
CWallet wallet;
TestKeys k;
ASSERT_NO_FATAL_FAILURE(BuildWallet(wallet, k));
std::set<libzcash::PaymentAddress> none;
std::vector<SaplingNoteEntry> all;
RefGetFilteredNotes(wallet, all, none, -1, INT_MAX, false, false, false);
EXPECT_EQ(all.size(), k.nNotes);
std::set<int> depths;
for (const SaplingNoteEntry& e : all) {
depths.insert(e.confirmations);
}
EXPECT_EQ(depths, (std::set<int>{-1, 0, 1, 2, 3, 4, 5, 10}));
// Spent: Nf(2), Nf(11) and Nf(14) in a block, Nf(4) in the mempool. Not spent: Nf(7)
// (conflicted spender) and the IVK-only note without a cached nullifier.
EXPECT_EQ(RefCount(wallet, none, -1, true, false, false), k.nNotes - 4);
// minDepth 0 drops the conflicted tx (1 note), minDepth 1 also the mempool tx (2).
EXPECT_EQ(RefCount(wallet, none, 0, false, false, false), k.nNotes - 1);
EXPECT_EQ(RefCount(wallet, none, 1, false, false, false), k.nNotes - 3);
EXPECT_EQ(RefCount(wallet, none, -1, false, false, true), k.nNotes - 2);
// Spending key reachable through the recorded address: a1 (4), d1 (2), a2 (3), and
// x1 (2), because x1 is recorded with K2's ivk and the wallet has K2's spending key.
EXPECT_EQ(RefCount(wallet, none, -1, false, true, false), (size_t)11);
EXPECT_EQ(RefCount(wallet, Filter({k.a1}), -1, false, false, false), (size_t)4);
EXPECT_EQ(RefCount(wallet, Filter({k.d1}), -1, false, false, false), (size_t)2);
EXPECT_EQ(RefCount(wallet, Filter({k.u1}), -1, false, false, false), (size_t)1);
EXPECT_EQ(RefCount(wallet, Filter({k.x1}), -1, false, false, false), (size_t)2);
EXPECT_EQ(RefCount(wallet, Filter({k.y1}), -1, false, false, false), (size_t)1);
EXPECT_EQ(RefCount(wallet, Filter({k.a2}), -1, false, false, false), (size_t)3);
EXPECT_EQ(RefCount(wallet, Filter({k.a3}), -1, false, false, false), (size_t)2);
EXPECT_EQ(RefCount(wallet, Filter({k.a4}), -1, false, false, false), (size_t)1);
EXPECT_EQ(RefCount(wallet, Filter({k.aN}), -1, false, false, false), (size_t)2);
EXPECT_EQ(RefCount(wallet, Filter({k.empty[0]}), -1, false, false, false), (size_t)0);
EXPECT_EQ(RefCount(wallet, Filter({k.foreign}), -1, false, false, false), (size_t)0);
}
// Every filter shape x ignoreSpent x requireSpendingKey x ignoreLocked x minDepth {0, 1}.
TEST_F(WalletScanNotes, MatchesReferenceAcrossFilters)
{
CWallet wallet;
TestKeys k;
ASSERT_NO_FATAL_FAILURE(BuildWallet(wallet, k));
std::vector<NamedFilter> filters = AllFilters(k);
size_t total = 0;
for (NamedFilter& f : filters) {
for (int minDepth : {0, 1}) {
for (bool ignoreSpent : {false, true}) {
for (bool requireSpendingKey : {false, true}) {
for (bool ignoreLocked : {false, true}) {
total += ExpectMatchesReference(wallet, f, minDepth, INT_MAX,
ignoreSpent, requireSpendingKey, ignoreLocked);
}
}
}
}
}
EXPECT_GT(total, (size_t)0);
}
// The depth window: minDepth {-1, 0, 1} x maxDepth {0, 1, 3, INT_MAX} x the three flags,
// unfiltered and with a single (pre-filtered) address.
TEST_F(WalletScanNotes, MatchesReferenceAcrossDepths)
{
CWallet wallet;
TestKeys k;
ASSERT_NO_FATAL_FAILURE(BuildWallet(wallet, k));
std::vector<NamedFilter> filters;
filters.push_back(NamedFilter{"{}", Filter({})});
filters.push_back(NamedFilter{"{a1}", Filter({k.a1})});
size_t total = 0;
for (NamedFilter& f : filters) {
for (int minDepth : {-1, 0, 1}) {
for (int maxDepth : {0, 1, 3, INT_MAX}) {
for (bool ignoreSpent : {false, true}) {
for (bool requireSpendingKey : {false, true}) {
for (bool ignoreLocked : {false, true}) {
total += ExpectMatchesReference(wallet, f, minDepth, maxDepth,
ignoreSpent, requireSpendingKey, ignoreLocked);
}
}
}
}
}
}
EXPECT_GT(total, (size_t)0);
}
// The single-address string overload (z_getbalance and friends): it decodes the address
// and calls the set overload with maxDepth INT_MAX and ignoreLocked true.
TEST_F(WalletScanNotes, MatchesReferenceForAddressString)
{
CWallet wallet;
TestKeys k;
ASSERT_NO_FATAL_FAILURE(BuildWallet(wallet, k));
std::vector<std::string> addresses = {
"",
EncodePaymentAddress(k.a1),
EncodePaymentAddress(k.u1),
EncodePaymentAddress(k.x1),
EncodePaymentAddress(k.foreign),
"not-an-address",
};
size_t total = 0;
for (const std::string& address : addresses) {
for (int minDepth : {0, 1}) {
for (bool ignoreSpent : {false, true}) {
for (bool requireSpendingKey : {false, true}) {
SCOPED_TRACE(testing::Message() << "address=\"" << address << "\""
<< " minDepth=" << minDepth << " ignoreSpent=" << ignoreSpent
<< " requireSpendingKey=" << requireSpendingKey);
std::set<libzcash::PaymentAddress> filter;
if (address.length() > 0) {
filter.insert(DecodePaymentAddress(address));
}
std::vector<SaplingNoteEntry> expected;
std::vector<SaplingNoteEntry> actual;
RefGetFilteredNotes(wallet, expected, filter,
minDepth, INT_MAX, ignoreSpent, requireSpendingKey, true);
wallet.GetFilteredNotes(actual, address, minDepth, ignoreSpent, requireSpendingKey);
ExpectSameEntries(expected, actual);
total += expected.size();
}
}
}
}
EXPECT_GT(total, (size_t)0);
}
// An encrypted, locked wallet: crypted spending keys still satisfy requireSpendingKey and
// the ivk/fvk maps are not encrypted, so both implementations must still agree.
TEST_F(WalletScanNotes, MatchesReferenceWhenEncryptedAndLocked)
{
EncryptableWallet wallet;
TestKeys k;
ASSERT_NO_FATAL_FAILURE(BuildWallet(wallet, k));
CKeyingMaterial masterKey(WALLET_CRYPTO_KEY_SIZE, 0x5a);
ASSERT_TRUE(wallet.EncryptKeysForTest(masterKey));
ASSERT_TRUE(wallet.Lock());
ASSERT_TRUE(wallet.IsCrypted());
ASSERT_TRUE(wallet.IsLocked());
EXPECT_TRUE(wallet.HaveSaplingSpendingKey(k.fvk1));
std::vector<NamedFilter> filters = AllFilters(k);
size_t total = 0;
size_t spendable = 0;
for (NamedFilter& f : filters) {
for (bool ignoreSpent : {false, true}) {
for (bool requireSpendingKey : {false, true}) {
for (bool ignoreLocked : {false, true}) {
size_t n = ExpectMatchesReference(wallet, f, 0, INT_MAX,
ignoreSpent, requireSpendingKey, ignoreLocked);
total += n;
if (requireSpendingKey) {
spendable += n;
}
}
}
}
}
EXPECT_GT(total, (size_t)0);
EXPECT_GT(spendable, (size_t)0);
}
// decrypt_with_pk_d returns the same plaintext as decrypt() and the pk_d that
// ivk.address(d) recomputes, and fails exactly when decrypt() fails.
TEST_F(WalletScanNotes, DecryptWithPkDMatchesDecrypt)
{
CWallet wallet;
TestKeys k;
ASSERT_NO_FATAL_FAILURE(BuildWallet(wallet, k));
LOCK2(cs_main, wallet.cs_wallet);
size_t checked = 0;
for (const auto& p : wallet.mapWallet) {
const CWalletTx& wtx = p.second;
for (const auto& item : wtx.mapSaplingNoteData) {
SCOPED_TRACE(testing::Message() << "note " << item.first.ToString());
const OutputDescription& od = wtx.vShieldedOutput[item.first.n];
const libzcash::SaplingIncomingViewingKey& ivk = item.second.ivk;
uint256 pk_d;
auto expected = libzcash::SaplingNotePlaintext::decrypt(
od.encCiphertext, ivk, od.ephemeralKey, od.cm);
auto actual = libzcash::SaplingNotePlaintext::decrypt_with_pk_d(
od.encCiphertext, ivk, od.ephemeralKey, od.cm, pk_d);
ASSERT_TRUE(static_cast<bool>(expected));
ASSERT_TRUE(static_cast<bool>(actual));
EXPECT_TRUE(actual->d == expected->d);
EXPECT_EQ(actual->value(), expected->value());
EXPECT_EQ(actual->rcm, expected->rcm);
EXPECT_TRUE(actual->memo() == expected->memo());
auto addr = ivk.address(actual->d);
ASSERT_TRUE(static_cast<bool>(addr));
EXPECT_EQ(pk_d, addr->pk_d);
checked++;
// Wrong ivk: the AEAD fails.
const libzcash::SaplingIncomingViewingKey& wrongIvk = (ivk == k.ivk2) ? k.ivk1 : k.ivk2;
EXPECT_FALSE(DecryptBothWays(od.encCiphertext, wrongIvk, od.ephemeralKey, od.cm));
// Wrong cmu: decryption and ivk_to_pkd succeed, the commitment check fails, so
// pk_d has been computed but must not reach pk_d_out.
uint256 badCm = od.cm;
badCm.begin()[0] ^= 1;
EXPECT_FALSE(DecryptBothWays(od.encCiphertext, ivk, od.ephemeralKey, badCm));
// Tampered ciphertext.
libzcash::SaplingEncCiphertext badCt = od.encCiphertext;
badCt[0] ^= 1;
EXPECT_FALSE(DecryptBothWays(badCt, ivk, od.ephemeralKey, od.cm));
// ivk1 + r_J: whatever librustzcash decides for a non-canonical scalar, both agree.
if (ivk == k.ivk1) {
DecryptBothWays(od.encCiphertext, k.aliasIvk1, od.ephemeralKey, od.cm);
}
}
}
EXPECT_EQ(checked, k.nNotes);
}
src/hush_nSPV_fullnode.h
+2 -2
View File
@@ -660,7 +660,7 @@ void hush_nSPVreq(CNode *pfrom,std::vector<uint8_t> request) // received a reque
struct NSPV_utxosresp U;
if ( len >= 2 && len < 64+5 && request[1] < 64 && (request[1] == len-3 || request[1] == len-7 || request[1] == len-11) )
{
int32_t skipcount = 0; char coinaddr[64]; uint32_t filter; uint8_t isCC = 0;
int32_t skipcount = 0; char coinaddr[64]; uint32_t filter = 0; uint8_t isCC = 0; // only assigned on the len-11 form; the other two passed it on uninitialized
memcpy(coinaddr,&request[2],request[1]); // request[1] < 64 bounds the copy + the terminator write
coinaddr[request[1]] = 0;
if ( request[1] == len-3 )
@@ -698,7 +698,7 @@ void hush_nSPVreq(CNode *pfrom,std::vector<uint8_t> request) // received a reque
struct NSPV_txidsresp T;
if ( len >= 2 && len < 64+5 && request[1] < 64 && (request[1] == len-3 || request[1] == len-7 || request[1] == len-11) )
{
int32_t skipcount = 0; char coinaddr[64]; uint32_t filter; uint8_t isCC = 0;
int32_t skipcount = 0; char coinaddr[64]; uint32_t filter = 0; uint8_t isCC = 0; // only assigned on the len-11 form; the other two passed it on uninitialized
memcpy(coinaddr,&request[2],request[1]); // request[1] < 64 bounds the copy + the terminator write
coinaddr[request[1]] = 0;
if ( request[1] == len-3 )
src/hush_utils.h
+21 -5
View File
@@ -1383,8 +1383,14 @@ void hush_configfile(char *symbol,uint16_t rpcport)
#endif
} else {
_hush_userpass(myusername,mypassword,fp);
mapArgs["-rpcpassword"] = mypassword;
mapArgs["-rpcusername"] = myusername;
// Feed the credentials read by InitRPCAuthentication (httprpc.cpp) and the
// CLI (bitcoin-cli.cpp) -- both read "-rpcuser"/"-rpcpassword". Use SoftSetArg
// so a value passed on the command line (or an explicit -rpcuser/-rpcpassword)
// still wins: the old direct assignment silently overwrote a command-line
// -rpcpassword on every restart once this conf existed, and the username was
// written to a misspelled "-rpcusername" key that nothing ever reads.
SoftSetArg("-rpcpassword", mypassword);
SoftSetArg("-rpcuser", myusername);
fclose(fp);
}
}
@@ -1777,11 +1783,21 @@ void hush_args(char *argv0)
fprintf(stderr,".oO Starting %s Full Node (Extreme Privacy!) with genproc=%d notary=%d\n",name.c_str(),HUSH_MININGTHREADS, IS_HUSH_NOTARY);
vector<string> DRAGONX_nodes = {};
// Only DRAGONX connects to these by default, other chains must opt-in via -connect/-addnode
// Only DRAGONX connects to these by default, other chains must opt-in via -connect/-addnode.
// Never on regtest or testnet: regtest reuses mainnet's network magic, so injecting the mainnet
// seeds here makes a supposedly isolated node handshake production peers and pull their headers
// into its own index -- which is exactly what it did, and why multi-node rpc-tests were talking
// to the live chain. NOTE: hush_args() runs between ParseParameters() and ReadConfigFile()
// (bitcoind.cpp:115/144/158), so this sees a command-line -regtest (how qa/rpc-tests starts
// nodes) but NOT a bare "regtest=1" in the config file.
const bool isdragonx = strncmp(name.c_str(), "DRAGONX",7) == 0 ? true : false;
LogPrint("net", "%s: isdragonx=%d\n", __func__, isdragonx);
if (isdragonx) {
// Seed injection only -- isdragonx itself must stay true here, because it also selects
// ac_private, ac_algo, blocktime and the reward/halving schedule below.
const bool isnotmainnet = GetBoolArg("-regtest", false) || GetBoolArg("-testnet", false);
LogPrint("net", "%s: isdragonx=%d isnotmainnet=%d\n", __func__, isdragonx, isnotmainnet);
if (isdragonx && !isnotmainnet) {
// node8-node10 are PLACEHOLDERS with no DNS records yet. A hostname that
// does not resolve is harmless here: ThreadOpenAddedConnections just fails
// to open the connection and retries on its 2-minute cycle. Reserving the
src/init.cpp
+36 -7
View File
@@ -1432,7 +1432,13 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler)
mempool.setSanityCheck(1.0 / ratio);
}
fCheckBlockIndex = GetBoolArg("-checkblockindex", chainparams.DefaultConsistencyChecks());
fCheckpointsEnabled = GetBoolArg("-checkpoints", true);
// Regtest inherits the DRAGONX *mainnet* checkpoint set (chainparams_commandline applies it
// for every SMART_CHAIN_SYMBOL=="DRAGONX" network, regardless of -regtest), whose top height
// ~3.2M would otherwise pin an isolated regtest chain in IsInitialBlockDownload() forever --
// disabling the ChainTip auto-ops (autoshield/sweep/consolidation) and the below-checkpoint
// script-check skip. Default checkpoints OFF on regtest so a fresh regtest node leaves IBD
// normally; still overridable with -checkpoints=1.
fCheckpointsEnabled = GetBoolArg("-checkpoints", chainparams.NetworkIDString() != "regtest");
// -par=0 means autodetect, but nScriptCheckThreads==0 means no concurrency
nScriptCheckThreads = GetArg("-par", DEFAULT_SCRIPTCHECK_THREADS);
@@ -2254,10 +2260,22 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler)
if (nLoadWalletRet != DB_LOAD_OK)
{
if (nLoadWalletRet == DB_CORRUPT)
strErrors << _("Error loading wallet.dat: Wallet corrupted. If this wallet was last opened "
"by an older version, move wallet.dat aside and restore from your seed "
"phrase with -mnemonic=\"<your seed phrase>\" -rescan (see debug.log for "
"the specific record at fault).") << "\n";
{
// Abort HERE, as the DB_NEED_REWRITE branch below already does. Falling through
// runs several hundred more lines of initialisation against a wallet we have just
// declared corrupt -- including SetHDSeedOrigin(), which WRITES to it, and the
// rescan and SetBestChain that follow.
//
// The old text advised restoring with -mnemonic. That is wrong twice over:
// -usemnemonic defaulted to 0 in v1.0.3 so many such wallets never had a phrase,
// and SetHDSeedFromMnemonic refuses a non-empty wallet, so "move wallet.dat aside"
// would discard every non-HD key the salvage preserved.
strErrors << _("Error loading wallet.dat: the wallet database is corrupt. Your keys may "
"still be intact -- do NOT delete or replace wallet.dat. Back it up now, "
"and see debug.log for the specific record at fault.") << "\n";
LogPrintf("%s", strErrors.str());
return InitError(strErrors.str());
}
else if (nLoadWalletRet == DB_NONCRITICAL_ERROR)
{
string msg(_("Warning: error reading wallet.dat! All keys read correctly, but transaction data"
@@ -2636,8 +2654,19 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler)
nStart = GetTimeMillis();
pwalletMain->ScanForWalletTransactions(pindexRescan, true);
LogPrintf(" rescan %15dms\n", GetTimeMillis() - nStart);
pwalletMain->SetBestChain(chainActive.GetLocator());
nWalletDBUpdated++;
// ONLY record "scanned to the tip" if the scan actually reached it. An aborted or
// shutdown-interrupted scan that stamps a tip locator tells the next startup there is
// nothing left to scan (the chainActive.Tip() != pindexRescan guard above then skips
// the rescan entirely), so every transaction in the unscanned range stays out of
// mapWallet permanently: invisible to getbalance and unspendable. The scan writes its
// own locator at the interrupt point instead.
if (pwalletMain->fLastRescanCompleted) {
pwalletMain->SetBestChain(chainActive.GetLocator());
nWalletDBUpdated++;
} else {
LogPrintf("Rescan did not complete; leaving the best-block locator at the scan's own "
"checkpoint so the remaining range is rescanned on the next start\n");
}
// Restore wallet transaction metadata after -zapwallettxes=1
if (GetBoolArg("-zapwallettxes", false) && GetArg("-zapwallettxes", "1") != "2")
src/net.cpp
+10
View File
@@ -2951,6 +2951,16 @@ void CNode::PushAddrMessage(CSerializedNetMsg&& msg)
}
}
void GetNodeCounts(int& nAll, int& nTls)
{
LOCK(cs_vNodes);
nAll = (int)vNodes.size();
nTls = 0;
for (const CNode* n : vNodes)
if (n->ssl != NULL)
nTls++;
}
size_t GetNodeCount(NumConnections flags)
{
LOCK(cs_vNodes);
src/net.h
+1
View File
@@ -165,6 +165,7 @@ enum NumConnections {
};
size_t GetNodeCount(NumConnections num);
void GetNodeCounts(int& nAll, int& nTls);
bool GetNetworkActive();
void SetNetworkActive(bool active);
src/rpc/mining.cpp
+55 -5
View File
@@ -1112,6 +1112,27 @@ static bool StratumMinerRecvLines(int fd, std::string& buf, int timeout_ms, std:
// authorize, receive work + the per-height RandomX key, then vary the block nNonce, hash with
// RandomX (byte-identical to CheckRandomXSolution via GetRandomXInput), and submit a 32-byte
// solution when the block hash meets target. Exists to validate the -stratum RandomX pool path.
//! Upper bound on how long stratummine will hold an RPC worker thread (and a 256 MB RandomX
//! cache). An unbounded caller-supplied deadline pins both indefinitely.
static const int64_t MAX_STRATUMMINE_TIMEOUT = 3600;
//! Parse a 64-char hex field from mining.notify into a uint256.
//! These fields come from whatever host the operator pointed us at, and both primitives below are
//! unforgiving: uint256's vector constructor asserts on a wrong-size input (uint256.cpp:30, and
//! NDEBUG is never defined for this build so the assert is live in release), while ParseHex
//! silently truncates at the first non-hex character. A short or garbled field would therefore
//! abort the daemon rather than be rejected. Returns false instead.
static bool StratumHex256(const std::string& hex, uint256& out)
{
if (hex.size() != 64 || !IsHex(hex))
return false;
std::vector<unsigned char> v = ParseHex(hex);
if (v.size() != 32)
return false;
out = uint256(v);
return true;
}
UniValue stratummine(const UniValue& params, bool fHelp, const CPubKey& mypk)
{
if (fHelp || params.size() < 2 || params.size() > 4)
@@ -1129,10 +1150,24 @@ UniValue stratummine(const UniValue& params, bool fHelp, const CPubKey& mypk)
if (ASSETCHAINS_ALGO != ASSETCHAINS_RANDOMX)
throw JSONRPCError(RPC_MISC_ERROR, "stratummine only supports RandomX chains");
// This is a reference miner for exercising -stratum, not a production facility: it dials an
// operator-supplied host, blocks an RPC worker for the whole run, and parses whatever that host
// chooses to send back. Require an explicit opt-in, and exempt regtest so the test suite can
// still drive it. NOTE: do NOT gate this on fExperimentalMode -- that defaults to TRUE
// (init.cpp:1195), so it would leave the RPC exposed on every node and the gate would be a no-op.
if (!GetBoolArg("-stratummine", false) && Params().NetworkIDString() != "regtest")
throw JSONRPCError(RPC_MISC_ERROR,
"stratummine is a test-only reference miner and is disabled by default; "
"restart with -stratummine to enable it");
const std::string host = params[0].get_str();
const int port = params[1].get_int();
const std::string addr = params.size() > 2 ? params[2].get_str() : "x";
const int64_t timeout = params.size() > 3 ? params[3].get_int64() : 120;
int64_t timeout = params.size() > 3 ? params[3].get_int64() : 120;
if (timeout < 1)
timeout = 1;
if (timeout > MAX_STRATUMMINE_TIMEOUT)
timeout = MAX_STRATUMMINE_TIMEOUT;
const int64_t deadline = GetTime() + timeout;
// connect (blocking TCP)
@@ -1166,6 +1201,11 @@ UniValue stratummine(const UniValue& params, bool fHelp, const CPubKey& mypk)
uint256 hashPrevBlock, hashMerkleRoot, hashReserved;
auto processLine = [&](const std::string& line) {
// Every get_str()/get_int() below throws on a type mismatch, and this lambda runs inside the
// window where the RandomX cache and VM are allocated and the socket is open -- all of which
// are released only on the normal path. A malformed server message must therefore never
// escape from here, or it leaks 256 MB and the fd on its way out.
try {
UniValue v;
if (!v.read(line)) return;
const UniValue& id = find_value(v, "id");
@@ -1185,16 +1225,26 @@ UniValue stratummine(const UniValue& params, bool fHelp, const CPubKey& mypk)
poolTarget = UintToArith256(uint256S(p[0].get_str()));
haveTarget = true;
} else if (m == "mining.notify" && p.size() >= 7) {
// Validate every fixed-width field before committing any of it, so a malformed job is
// ignored outright rather than half-applied over the previous one.
uint256 prev, merkle, reserved;
if (!StratumHex256(p[2].get_str(), prev) ||
!StratumHex256(p[3].get_str(), merkle) ||
!StratumHex256(p[4].get_str(), reserved))
return;
jobId = p[0].get_str();
nVersion = bswap_32((uint32_t)strtoul(p[1].get_str().c_str(), NULL, 16));
hashPrevBlock = uint256(ParseHex(p[2].get_str()));
hashMerkleRoot = uint256(ParseHex(p[3].get_str()));
hashReserved = uint256(ParseHex(p[4].get_str()));
hashPrevBlock = prev;
hashMerkleRoot = merkle;
hashReserved = reserved;
timeHex = p[5].get_str();
nTime = bswap_32((uint32_t)strtoul(timeHex.c_str(), NULL, 16));
nBits = bswap_32((uint32_t)strtoul(p[6].get_str().c_str(), NULL, 16));
haveJob = true;
}
} catch (const std::exception&) {
// Malformed message from the server: ignore the line and keep mining.
}
};
std::string buf;
@@ -1302,7 +1352,7 @@ static const CRPCCommand commands[] =
{ // category name actor (function) okSafeMode
// --------------------- ------------------------ ----------------------- ----------
#ifndef WIN32
{ "mining", "stratummine", &stratummine, true },
{ "mining", "stratummine", &stratummine, false },
#endif
{ "mining", "getlocalsolps", &getlocalsolps, true },
{ "mining", "getnetworksolps", &getnetworksolps, true },
src/rpc/misc.cpp
+6 -2
View File
@@ -132,6 +132,10 @@ UniValue getinfo(const UniValue& params, bool fHelp, const CPubKey& mypk)
+ HelpExampleRpc("getinfo", "")
);
// C268: snapshot peer counts under cs_vNodes (sequentially, before the cs_main/cs_wallet lock)
int nAllConns = 0, nTlsConns = 0;
GetNodeCounts(nAllConns, nTlsConns);
#ifdef ENABLE_WALLET
LOCK2(cs_main, pwalletMain ? &pwalletMain->cs_wallet : NULL);
#else
@@ -175,8 +179,8 @@ UniValue getinfo(const UniValue& params, bool fHelp, const CPubKey& mypk)
}
//obj.push_back(Pair("build_date", BUILD_DATE));
obj.push_back(Pair("timeoffset", 0));
obj.push_back(Pair("connections", (int)vNodes.size()));
obj.push_back(Pair("tls_connections", (int)std::count_if(vNodes.begin(), vNodes.end(), [](CNode* n) {return n->ssl != NULL;})));
obj.push_back(Pair("connections", nAllConns));
obj.push_back(Pair("tls_connections", nTlsConns));
obj.push_back(Pair("proxy", (proxy.IsValid() ? proxy.proxy.ToStringIPPort() : string())));
obj.push_back(Pair("testnet", Params().TestnetToBeDeprecatedFieldRPC()));
obj.push_back(Pair("relayfee", ValueFromAmount(::minRelayTxFee.GetFeePerK())));
src/rpc/net.cpp
+5 -2
View File
@@ -533,6 +533,9 @@ UniValue getnetworkinfo(const UniValue& params, bool fHelp, const CPubKey& mypk)
+ HelpExampleRpc("getnetworkinfo", "")
);
int nAllConns = 0, nTlsConns = 0;
GetNodeCounts(nAllConns, nTlsConns);
LOCK(cs_main);
UniValue obj(UniValue::VOBJ);
@@ -541,8 +544,8 @@ UniValue getnetworkinfo(const UniValue& params, bool fHelp, const CPubKey& mypk)
obj.push_back(Pair("protocolversion",PROTOCOL_VERSION));
obj.push_back(Pair("localservices", strprintf("%016x", nLocalServices)));
obj.push_back(Pair("timeoffset", 0));
obj.push_back(Pair("connections", (int)vNodes.size()));
obj.push_back(Pair("tls_connections", (int)std::count_if(vNodes.begin(), vNodes.end(), [](CNode* n) {return n->ssl != NULL;})));
obj.push_back(Pair("connections", nAllConns));
obj.push_back(Pair("tls_connections", nTlsConns));
obj.push_back(Pair("networks", GetNetworksInfo()));
obj.push_back(Pair("relayfee", ValueFromAmount(::minRelayTxFee.GetFeePerK())));
UniValue localAddresses(UniValue::VARR);
src/wallet/rpcdump.cpp
+11 -4
View File
@@ -746,10 +746,17 @@ UniValue dumpwallet_impl(const UniValue& params, bool fHelp, bool fDumpZKeys)
HDSeed hdSeed;
// Dump the 64-byte derivation seed (for mnemonic wallets this is the
// expanded BIP39 seed), so re-importing the hex reproduces the same keys.
pwalletMain->GetHDSeedForDerivation(hdSeed);
auto rawSeed = hdSeed.RawSeed();
file << strprintf("# HDSeed=%s fingerprint=%s", HexStr(rawSeed.begin(), rawSeed.end()), hdSeed.Fingerprint().GetHex());
file << "\n";
// The return MUST be checked: on failure hdSeed is default-constructed, and emitting it
// anyway writes a blank seed next to a legitimate-looking BLAKE2b-of-empty fingerprint --
// a backup that looks valid and restores nothing. The per-key dump below is still a
// complete backup without this line.
if (pwalletMain->GetHDSeedForDerivation(hdSeed)) {
auto rawSeed = hdSeed.RawSeed();
file << strprintf("# HDSeed=%s fingerprint=%s", HexStr(rawSeed.begin(), rawSeed.end()), hdSeed.Fingerprint().GetHex());
file << "\n";
} else {
file << "# HDSeed unavailable (wallet locked, no HD seed, or hdchain unproven)\n";
}
}
file << "\n";
for (std::vector<std::pair<int64_t, CKeyID> >::const_iterator it = vKeyBirth.begin(); it != vKeyBirth.end(); it++) {
src/wallet/rpchushwallet.cpp
+120 -31
View File
@@ -11,11 +11,72 @@
#include "utilmoneystr.h"
#include <utf8.h>
#include <map>
#include <set>
#include <utility>
#include <vector>
using namespace std;
using namespace libzcash;
bool EnsureWalletIsAvailable(bool avoidException);
const std::vector<ZsTxJSONContext::Key>& ZsTxJSONContext::keys()
{
if (!keysBuilt) {
// The calls, order and filter of the per-output key loops this replaces.
std::set<libzcash::SaplingPaymentAddress> addresses;
pwalletMain->GetSaplingPaymentAddresses(addresses);
for (std::set<libzcash::SaplingPaymentAddress>::const_iterator it = addresses.begin(); it != addresses.end(); ++it) {
libzcash::SaplingExtendedSpendingKey extsk;
if (!pwalletMain->GetSaplingExtendedSpendingKey(*it, extsk))
continue; // viewing-key-only, or encrypted and locked: skipped, as before
libzcash::SaplingFullViewingKey fvk = extsk.expsk.full_viewing_key();
Key key;
key.addr = *it;
key.ivk = fvk.in_viewing_key();
key.ovk = fvk.ovk;
vKeys.push_back(key);
}
keysBuilt = true;
}
return vKeys;
}
bool ZsTxJSONContext::hasNullifier(const libzcash::SaplingPaymentAddress& addr, const uint256& nullifier)
{
if (!nullifiersBuilt) {
// The population GetNullifiersForAddresses() scans: every mapWallet note that has a nullifier.
for (std::map<uint256, CWalletTx>::const_iterator it = pwalletMain->mapWallet.begin(); it != pwalletMain->mapWallet.end(); ++it) {
for (mapSaplingNoteData_t::const_iterator nit = it->second.mapSaplingNoteData.begin(); nit != it->second.mapSaplingNoteData.end(); ++nit) {
if (nit->second.nullifier)
nfByIvk.insert(std::make_pair(nit->second.ivk, nit->second.nullifier.get()));
}
}
nullifiersBuilt = true;
}
// GetNullifiersForAddresses() ignores the result of this lookup, so an address the wallet does not
// know is matched against the default (null) ivk. Keep that.
libzcash::SaplingIncomingViewingKey ivk;
pwalletMain->GetSaplingIncomingViewingKey(addr, ivk);
return nfByIvk.count(std::make_pair(ivk, nullifier)) > 0;
}
bool ZsTxJSONContext::seedOvk(uint256& ovkOut)
{
if (!seedOvkTried) {
HDSeed seed;
if (pwalletMain->GetHDSeedForDerivation(seed)) {
seedOvkValue = ovkForShieldingFromTaddr(seed);
seedOvkOk = true;
}
seedOvkTried = true;
}
if (seedOvkOk)
ovkOut = seedOvkValue;
return seedOvkOk;
}
void zsTxSpendsToJSON(const CWalletTx& wtx, UniValue& spends, CAmount& totalSpends, CAmount& filteredSpends, const std::string& strAddress, bool filterByAddress) {
LOCK2(cs_main, pwalletMain->cs_wallet);
@@ -87,12 +148,16 @@ void zsTxSpendsToJSON(const CWalletTx& wtx, UniValue& spends, CAmount& totalSpen
const SpendDescription& spendDesc = wtx.vShieldedSpend[i];
UniValue obj(UniValue::VOBJ);
SaplingOutPoint op = pwalletMain->mapSaplingNullifiersToNotes[spendDesc.nullifier];
std::map<uint256, SaplingOutPoint>::const_iterator nfIt = pwalletMain->mapSaplingNullifiersToNotes.find(spendDesc.nullifier);
if (pwalletMain->IsSaplingNullifierFromMe(spendDesc.nullifier)) {
const CWalletTx* parent = pwalletMain->GetWalletTx(pwalletMain->mapSaplingNullifiersToNotes[spendDesc.nullifier].hash);
// Same test as IsSaplingNullifierFromMe(), but find() so a read-only RPC never inserts into wallet maps.
if (nfIt != pwalletMain->mapSaplingNullifiersToNotes.end() && pwalletMain->mapWallet.count(nfIt->second.hash)) {
const SaplingOutPoint op = nfIt->second;
const CWalletTx* parent = pwalletMain->GetWalletTx(op.hash);
mapSaplingNoteData_t::const_iterator ndIt = parent->mapSaplingNoteData.find(op);
if (ndIt == parent->mapSaplingNoteData.end()) continue; // stale entry: old code inserted an empty note that never decrypts
const OutputDescription& output = parent->vShieldedOutput[op.n];
auto nd = pwalletMain->mapWallet[pwalletMain->mapSaplingNullifiersToNotes[spendDesc.nullifier].hash].mapSaplingNoteData[op];
const SaplingNoteData& nd = ndIt->second;
auto pt = libzcash::SaplingNotePlaintext::decrypt(output.encCiphertext,nd.ivk,output.ephemeralKey,output.cm);
if (pt) {
@@ -132,13 +197,16 @@ void zsTxSpendsToJSON(const CWalletTx& wtx, UniValue& spends, CAmount& totalSpen
}
void zsTxReceivedToJSON(const CWalletTx& wtx, UniValue& received, CAmount& totalReceived, const std::string& strAddress, bool filterByAddress) {
void zsTxReceivedToJSON(const CWalletTx& wtx, UniValue& received, CAmount& totalReceived, const std::string& strAddress, bool filterByAddress, ZsTxJSONContext* ctx) {
if(fZdebug)
fprintf(stderr,"%s: txid=%s\n", __func__, wtx.GetHash().ToString().c_str() );
LOCK2(cs_main, pwalletMain->cs_wallet);
ZsTxJSONContext localCtx; // used only when the caller passed no per-call memo
ZsTxJSONContext& zctx = ctx ? *ctx : localCtx;
//Check address
bool isTAddress = false;
bool isZsAddress = false;
@@ -192,13 +260,36 @@ void zsTxReceivedToJSON(const CWalletTx& wtx, UniValue& received, CAmount& total
UniValue obj(UniValue::VOBJ);
bool changeTx = false;
//Decrypt sapling incoming commitments using IVK
std::set<libzcash::SaplingPaymentAddress> addresses;
pwalletMain->GetSaplingPaymentAddresses(addresses);
for (auto addr : addresses) {
libzcash::SaplingExtendedSpendingKey extsk;
if (pwalletMain->GetSaplingExtendedSpendingKey(addr, extsk)) {
const std::vector<ZsTxJSONContext::Key>& keys = zctx.keys();
// An output opens under at most one ivk (decrypt() checks the note commitment, which binds
// pk_d = [ivk]g_d), and FindMySaplingNotes records that ivk in mapSaplingNoteData. When the recorded
// ivk is one of keys(), pass 0 tries only the keys with that ivk: every other key would fail, as in
// the old single loop. If pass 0 opens nothing (stale note data), pass 1 tries the remaining keys in
// the same order. Without a usable record only pass 1 runs, over every key, exactly as before.
libzcash::SaplingIncomingViewingKey recordedIvk;
bool narrow = false;
{
mapSaplingNoteData_t::const_iterator ndIt = wtx.mapSaplingNoteData.find(SaplingOutPoint(wtx.GetHash(), i));
if (ndIt != wtx.mapSaplingNoteData.end()) {
for (size_t k = 0; k < keys.size() && !narrow; k++) {
if (keys[k].ivk == ndIt->second.ivk) {
recordedIvk = ndIt->second.ivk;
narrow = true;
}
}
}
}
bool openedByRecordedIvk = false;
for (int pass = narrow ? 0 : 1; pass < 2 && !openedByRecordedIvk; pass++) {
for (size_t k = 0; k < keys.size(); k++) {
if (narrow && ((pass == 0) != (keys[k].ivk == recordedIvk)))
continue; // pass 0: only the recorded ivk; pass 1: only the others
const libzcash::SaplingPaymentAddress& addr = keys[k].addr;
{ // spending-key availability is resolved once per call, in ZsTxJSONContext::keys()
auto pt = libzcash::SaplingNotePlaintext::decrypt(
outputDesc.encCiphertext, extsk.expsk.full_viewing_key().in_viewing_key(), outputDesc.ephemeralKey, outputDesc.cm);
outputDesc.encCiphertext, keys[k].ivk, outputDesc.ephemeralKey, outputDesc.cm);
if (pt && pass == 0)
openedByRecordedIvk = true;
if (pt) {
auto note = pt.get();
@@ -219,10 +310,8 @@ void zsTxReceivedToJSON(const CWalletTx& wtx, UniValue& received, CAmount& total
//Check Change Status
if (wtx.vShieldedSpend.size()!=0) {
std::set<std::pair<PaymentAddress, uint256>> nullifierSet;
nullifierSet = pwalletMain->GetNullifiersForAddresses({addr});
BOOST_FOREACH(const SpendDescription& spendDesc, wtx.vShieldedSpend) {
if (nullifierSet.count(std::make_pair(addr, spendDesc.nullifier))) {
if (zctx.hasNullifier(addr, spendDesc.nullifier)) {
changeTx = true;
}
}
@@ -235,6 +324,7 @@ void zsTxReceivedToJSON(const CWalletTx& wtx, UniValue& received, CAmount& total
}
}
}
} // pass
}
}
received.push_back(Pair("saplingReceived",zsReceived));
@@ -243,10 +333,13 @@ void zsTxReceivedToJSON(const CWalletTx& wtx, UniValue& received, CAmount& total
}
void zsTxSendsToJSON(const CWalletTx& wtx, UniValue& sends, CAmount& totalSends, const std::string& strAddress, bool filterByAddress) {
void zsTxSendsToJSON(const CWalletTx& wtx, UniValue& sends, CAmount& totalSends, const std::string& strAddress, bool filterByAddress, ZsTxJSONContext* ctx) {
LOCK2(cs_main, pwalletMain->cs_wallet);
ZsTxJSONContext localCtx; // used only when the caller passed no per-call memo
ZsTxJSONContext& zctx = ctx ? *ctx : localCtx;
//Used to identify incomplete key sets
int shieldedOutputCount = 0;
int shieldedOutputDecryptedCount = 0;
@@ -304,10 +397,10 @@ void zsTxSendsToJSON(const CWalletTx& wtx, UniValue& sends, CAmount& totalSends,
//Decrypt sapling outgoing t to z transaction using HDseed
if (wtx.vShieldedSpend.size()==0) {
HDSeed seed;
if (pwalletMain->GetHDSeedForDerivation(seed)) {
uint256 taddrOvk;
if (zctx.seedOvk(taddrOvk)) {
auto opt = libzcash::SaplingOutgoingPlaintext::decrypt(
outputDesc.outCiphertext,ovkForShieldingFromTaddr(seed),outputDesc.cv,outputDesc.cm,outputDesc.ephemeralKey);
outputDesc.outCiphertext,taddrOvk,outputDesc.cv,outputDesc.cm,outputDesc.ephemeralKey);
if (opt) {
auto opt_unwrapped = opt.get();
@@ -332,13 +425,11 @@ void zsTxSendsToJSON(const CWalletTx& wtx, UniValue& sends, CAmount& totalSends,
//attempt Decryption of Outgoing Sapling using wallet extended spending keys
} else {
std::set<libzcash::SaplingPaymentAddress> addresses;
pwalletMain->GetSaplingPaymentAddresses(addresses);
for (auto addr : addresses) {
libzcash::SaplingExtendedSpendingKey extsk;
if (pwalletMain->GetSaplingExtendedSpendingKey(addr, extsk)) {
const std::vector<ZsTxJSONContext::Key>& keys = zctx.keys();
for (size_t k = 0; k < keys.size(); k++) {
{ // spending-key availability is resolved once per call, in ZsTxJSONContext::keys()
auto opt = libzcash::SaplingOutgoingPlaintext::decrypt(
outputDesc.outCiphertext,extsk.expsk.full_viewing_key().ovk,outputDesc.cv,outputDesc.cm,outputDesc.ephemeralKey);
outputDesc.outCiphertext,keys[k].ovk,outputDesc.cv,outputDesc.cm,outputDesc.ephemeralKey);
if (opt) {
auto opt_unwrapped = opt.get();
@@ -365,10 +456,8 @@ void zsTxSendsToJSON(const CWalletTx& wtx, UniValue& sends, CAmount& totalSends,
//Check Change Status
if (wtx.vShieldedSpend.size()!=0) {
std::set<std::pair<PaymentAddress, uint256>> nullifierSet;
nullifierSet = pwalletMain->GetNullifiersForAddresses({sentAddr});
BOOST_FOREACH(const SpendDescription& spendDesc, wtx.vShieldedSpend) {
if (nullifierSet.count(std::make_pair(sentAddr, spendDesc.nullifier))) {
if (zctx.hasNullifier(sentAddr, spendDesc.nullifier)) {
changeTx = true;
}
}
@@ -397,7 +486,7 @@ void zsTxSendsToJSON(const CWalletTx& wtx, UniValue& sends, CAmount& totalSends,
}
void zsWalletTxJSON(const CWalletTx& wtx, UniValue& ret, const std::string strAddress, bool fBool, const int returnType) {
void zsWalletTxJSON(const CWalletTx& wtx, UniValue& ret, const std::string strAddress, bool fBool, const int returnType, ZsTxJSONContext* ctx) {
LOCK2(cs_main, pwalletMain->cs_wallet);
@@ -445,7 +534,7 @@ void zsWalletTxJSON(const CWalletTx& wtx, UniValue& ret, const std::string strAd
tx.push_back(Pair("expiryheight", (int64_t)wtx.nExpiryHeight));
tx.push_back(Pair("confirmations", confirms));
tx.push_back(Pair("time", wtx.GetTxTime()));
tx.push_back(Pair("size", static_cast<uint64_t>(GetSerializeSize(static_cast<CTransaction>(wtx), SER_NETWORK, PROTOCOL_VERSION))));
tx.push_back(Pair("size", static_cast<uint64_t>(GetSerializeSize(static_cast<const CTransaction&>(wtx), SER_NETWORK, PROTOCOL_VERSION))));
//Wallet Conflicts
UniValue conflicts(UniValue::VARR);
@@ -469,7 +558,7 @@ void zsWalletTxJSON(const CWalletTx& wtx, UniValue& ret, const std::string strAd
}
// Get Received
if (returnType == 0 || returnType == 2) {
zsTxReceivedToJSON(wtx, received, totalReceived, strAddress, fBool);
zsTxReceivedToJSON(wtx, received, totalReceived, strAddress, fBool, ctx);
if (!fBool || totalReceived != 0) {
tx.push_back(Pair("received",received));
}
@@ -479,7 +568,7 @@ void zsWalletTxJSON(const CWalletTx& wtx, UniValue& ret, const std::string strAd
if (returnType == 0 || returnType == 3) {
//Only include sends if there are spends that belong to the wallet.
if (totalSpends != 0 || fBool) {
zsTxSendsToJSON(wtx, sends, totalSends, strAddress, fBool);
zsTxSendsToJSON(wtx, sends, totalSends, strAddress, fBool, ctx);
}
if (!fBool || totalSends != 0) {
tx.push_back(Pair("sends",sends));
src/wallet/rpchushwallet.h
+46 -3
View File
@@ -6,6 +6,13 @@
#ifndef HUSH_WALLET_RPCHUSHWALLET_H
#define HUSH_WALLET_RPCHUSHWALLET_H
#include "uint256.h"
#include "zcash/Address.hpp"
#include <set>
#include <utility>
#include <vector>
struct balancestruct {
CAmount confirmed;
CAmount unconfirmed;
@@ -13,10 +20,46 @@ struct balancestruct {
CAmount immature;
};
// Per-RPC-call memo for the zs*ToJSON helpers: the wallet's usable Sapling keys, the HD-seed ovk and a
// nullifier index, derived once per call instead of once per output. Every member is a copy; nothing
// points into wallet maps. Valid only while the caller holds LOCK2(cs_main, pwalletMain->cs_wallet) for
// the whole call: declare it after that lock and never keep it across calls. Passing NULL to the helpers
// gives each helper call its own memo (the old per-call cost profile, minus the per-output repeats).
class ZsTxJSONContext {
public:
struct Key {
libzcash::SaplingPaymentAddress addr;
libzcash::SaplingIncomingViewingKey ivk;
uint256 ovk;
};
ZsTxJSONContext() : keysBuilt(false), nullifiersBuilt(false), seedOvkTried(false), seedOvkOk(false) {}
ZsTxJSONContext(const ZsTxJSONContext&) = delete;
ZsTxJSONContext& operator=(const ZsTxJSONContext&) = delete;
// Wallet z-addresses (std::set order) whose extended spending key is available now, with that key's
// extsk.expsk.full_viewing_key().in_viewing_key() and .ovk. Empty for an encrypted+locked wallet or
// one that holds only viewing keys.
const std::vector<Key>& keys();
// == GetNullifiersForAddresses({addr}).count(std::make_pair(addr, nullifier)) > 0
bool hasNullifier(const libzcash::SaplingPaymentAddress& addr, const uint256& nullifier);
// == GetHDSeedForDerivation(seed) ? (ovkOut = ovkForShieldingFromTaddr(seed), true) : false
bool seedOvk(uint256& ovkOut);
private:
bool keysBuilt;
std::vector<Key> vKeys;
bool nullifiersBuilt;
std::set<std::pair<libzcash::SaplingIncomingViewingKey, uint256> > nfByIvk;
bool seedOvkTried;
bool seedOvkOk;
uint256 seedOvkValue;
};
void zsTxSpendsToJSON(const CWalletTx& wtx, UniValue& spends, CAmount& totalSpends, CAmount& filteredSpends, const std::string& strAddress, bool filterByAddress);
void zsTxReceivedToJSON(const CWalletTx& wtx, UniValue& received, CAmount& totalReceived, const std::string& strAddress, bool filterByAddress);
void zsTxSendsToJSON(const CWalletTx& wtx, UniValue& sends, CAmount& totalSends, const std::string& strAddress, bool filterByAddress);
void zsWalletTxJSON(const CWalletTx& wtx, UniValue& ret, const std::string strAddress, bool fBool, const int returnType);
void zsTxReceivedToJSON(const CWalletTx& wtx, UniValue& received, CAmount& totalReceived, const std::string& strAddress, bool filterByAddress, ZsTxJSONContext* ctx = NULL);
void zsTxSendsToJSON(const CWalletTx& wtx, UniValue& sends, CAmount& totalSends, const std::string& strAddress, bool filterByAddress, ZsTxJSONContext* ctx = NULL);
void zsWalletTxJSON(const CWalletTx& wtx, UniValue& ret, const std::string strAddress, bool fBool, const int returnType, ZsTxJSONContext* ctx = NULL);
#endif //HUSH_WALLET_RPCWALLET_H
src/wallet/rpcwallet.cpp
+176 -92
View File
@@ -48,6 +48,7 @@
#include <boost/assign/list_of.hpp>
#include <utf8.h>
#include <univalue.h>
#include <algorithm>
#include <numeric>
#include "hush_defs.h"
#include <string.h>
@@ -1773,7 +1774,19 @@ static void MaybePushAddress(UniValue & entry, const CTxDestination &dest)
}
}
void ListTransactions(const CWalletTx& wtx, const string& strAccount, int nMinDepth, bool fLong, UniValue& ret, const isminefilter& filter)
// listtransactions skips its first `from` entries without building them: while *pnSkip > 0, an
// entry that would be emitted only decrements *pnSkip. The counter is shared across calls, so the
// skipped entries are exactly the first `from` ones in emission order. NULL: emit everything.
static bool ConsumeSkip(int* pnSkip)
{
if (pnSkip != NULL && *pnSkip > 0) {
--*pnSkip;
return true;
}
return false;
}
void ListTransactions(const CWalletTx& wtx, const string& strAccount, int nMinDepth, bool fLong, UniValue& ret, const isminefilter& filter, int* pnSkip = NULL)
{
CAmount nFee;
string strSentAccount;
@@ -1787,11 +1800,18 @@ void ListTransactions(const CWalletTx& wtx, const string& strAccount, int nMinDe
bool fAllAccounts = (strAccount == string("*"));
bool involvesWatchonly = wtx.IsFromMe(ISMINE_WATCH_ONLY);
// "size" is a property of the transaction, not of an entry: compute it once, when the first
// entry is built, through a const reference instead of a by-value CTransaction copy.
bool fHaveTxSize = false;
uint64_t nTxSize = 0;
// Sent
if ((!listSent.empty() || nFee != 0) && (fAllAccounts || strAccount == strSentAccount))
{
BOOST_FOREACH(const COutputEntry& s, listSent)
{
if (ConsumeSkip(pnSkip))
continue;
UniValue entry(UniValue::VOBJ);
if(involvesWatchonly || (::IsMine(*pwalletMain, s.destination) & ISMINE_WATCH_ONLY))
entry.push_back(Pair("involvesWatchonly", true));
@@ -1803,7 +1823,11 @@ void ListTransactions(const CWalletTx& wtx, const string& strAccount, int nMinDe
entry.push_back(Pair("fee", ValueFromAmount(-nFee)));
if (fLong)
WalletTxToJSON(wtx, entry);
entry.push_back(Pair("size", static_cast<uint64_t>(GetSerializeSize(static_cast<CTransaction>(wtx), SER_NETWORK, PROTOCOL_VERSION))));
if (!fHaveTxSize) {
nTxSize = static_cast<uint64_t>(GetSerializeSize(static_cast<const CTransaction&>(wtx), SER_NETWORK, PROTOCOL_VERSION));
fHaveTxSize = true;
}
entry.push_back(Pair("size", nTxSize));
ret.push_back(entry);
}
}
@@ -1818,6 +1842,8 @@ void ListTransactions(const CWalletTx& wtx, const string& strAccount, int nMinDe
account = pwalletMain->mapAddressBook[r.destination].name;
if (fAllAccounts || (account == strAccount))
{
if (ConsumeSkip(pnSkip))
continue;
UniValue entry(UniValue::VOBJ);
if(involvesWatchonly || (::IsMine(*pwalletMain, r.destination) & ISMINE_WATCH_ONLY))
entry.push_back(Pair("involvesWatchonly", true));
@@ -1848,19 +1874,25 @@ void ListTransactions(const CWalletTx& wtx, const string& strAccount, int nMinDe
entry.push_back(Pair("vout", r.vout));
if (fLong)
WalletTxToJSON(wtx, entry);
entry.push_back(Pair("size", static_cast<uint64_t>(GetSerializeSize(static_cast<CTransaction>(wtx), SER_NETWORK, PROTOCOL_VERSION))));
if (!fHaveTxSize) {
nTxSize = static_cast<uint64_t>(GetSerializeSize(static_cast<const CTransaction&>(wtx), SER_NETWORK, PROTOCOL_VERSION));
fHaveTxSize = true;
}
entry.push_back(Pair("size", nTxSize));
ret.push_back(entry);
}
}
}
}
void AcentryToJSON(const CAccountingEntry& acentry, const string& strAccount, UniValue& ret)
void AcentryToJSON(const CAccountingEntry& acentry, const string& strAccount, UniValue& ret, int* pnSkip = NULL)
{
bool fAllAccounts = (strAccount == string("*"));
if (fAllAccounts || acentry.strAccount == strAccount)
{
if (ConsumeSkip(pnSkip))
return;
UniValue entry(UniValue::VOBJ);
entry.push_back(Pair("account", acentry.strAccount));
entry.push_back(Pair("category", "move"));
@@ -1954,46 +1986,42 @@ UniValue listtransactions(const UniValue& params, bool fHelp, const CPubKey& myp
UniValue ret(UniValue::VARR);
std::list<CAccountingEntry> acentries;
CWallet::TxItems txOrdered = pwalletMain->OrderedTxItems(acentries, strAccount);
CWallet::TxItems txOrdered = pwalletMain->OrderedTxItems(acentries, strAccount, true);
// iterate backwards until we have nCount items to return:
// iterate backwards until we have nCount items to return. The first nFrom entries are only
// counted (nSkipLeft), never built, since they are not returned.
int nSkipLeft = nFrom;
for (CWallet::TxItems::reverse_iterator it = txOrdered.rbegin(); it != txOrdered.rend(); ++it)
{
CWalletTx *const pwtx = (*it).second.first;
if (pwtx != 0)
{
ListTransactions(*pwtx, strAccount, 0, true, ret, filter);
ListTransactions(*pwtx, strAccount, 0, true, ret, filter, &nSkipLeft);
}
CAccountingEntry *const pacentry = (*it).second.second;
if (pacentry != 0)
AcentryToJSON(*pacentry, strAccount, ret);
AcentryToJSON(*pacentry, strAccount, ret, &nSkipLeft);
if ((int)ret.size() >= (nCount+nFrom)) break;
// (skipped + built) is the entry count the loop used to test
if ((nFrom - nSkipLeft) + (int)ret.size() >= (nCount+nFrom)) break;
}
// ret is newest to oldest
// ret holds the entries after the first nSkipped ones, newest to oldest
const int nSkipped = nFrom - nSkipLeft;
const int nSeen = nSkipped + (int)ret.size();
if (nFrom > (int)ret.size())
nFrom = ret.size();
if ((nFrom + nCount) > (int)ret.size())
nCount = ret.size() - nFrom;
if (nFrom > nSeen)
nFrom = nSeen;
if ((nFrom + nCount) > nSeen)
nCount = nSeen - nFrom;
vector<UniValue> arrTmp = ret.getValues();
// The reply is entries [nFrom, nFrom+nCount) of the newest-first sequence, oldest to newest.
// Entry k of that sequence is ret[k - nSkipped] (nFrom >= nSkipped holds here).
UniValue result(UniValue::VARR);
const std::vector<UniValue>& vNewestFirst = ret.getValues();
for (int i = nFrom + nCount; i > nFrom; --i)
result.push_back(vNewestFirst[i - 1 - nSkipped]);
vector<UniValue>::iterator first = arrTmp.begin();
std::advance(first, nFrom);
vector<UniValue>::iterator last = arrTmp.begin();
std::advance(last, nFrom+nCount);
if (last != arrTmp.end()) arrTmp.erase(last, arrTmp.end());
if (first != arrTmp.begin()) arrTmp.erase(arrTmp.begin(), first);
std::reverse(arrTmp.begin(), arrTmp.end()); // Return oldest to newest
ret.clear();
ret.setArray();
ret.push_backV(arrTmp);
return ret;
return result;
}
UniValue listaccounts(const UniValue& params, bool fHelp, const CPubKey& mypk)
@@ -2238,7 +2266,7 @@ UniValue gettransaction(const UniValue& params, bool fHelp, const CPubKey& mypk)
ListTransactions(wtx, "*", 0, false, details, filter);
entry.push_back(Pair("details", details));
string strHex = EncodeHexTx(static_cast<CTransaction>(wtx));
string strHex = EncodeHexTx(static_cast<const CTransaction&>(wtx));
entry.push_back(Pair("hex", strHex));
return entry;
@@ -2960,6 +2988,26 @@ UniValue listunspent(const UniValue& params, bool fHelp, const CPubKey& mypk)
return results;
}
// Wallet transactions ordered by nOrderPos without copying them (the old code copied every CWalletTx,
// with every note's witness list, into a map<int64_t,CWalletTx>). The pointers point into
// pwalletMain->mapWallet and are valid only while cs_wallet is held and nothing erases from mapWallet
// (std::map insertions do not invalidate them). Reproduces map<int64_t,CWalletTx>::insert exactly:
// ascending nOrderPos and, for equal nOrderPos, only the first tx in mapWallet (txid) order, because
// insert() never replaces an existing key; hence stable_sort + unique.
typedef std::pair<int64_t, const CWalletTx*> OrderedWtx;
static bool OrderedWtxLess(const OrderedWtx& a, const OrderedWtx& b) { return a.first < b.first; }
static bool OrderedWtxSamePos(const OrderedWtx& a, const OrderedWtx& b) { return a.first == b.first; }
static void BuildOrderedWalletTxs(std::vector<OrderedWtx>& out)
{
AssertLockHeld(pwalletMain->cs_wallet);
out.clear();
out.reserve(pwalletMain->mapWallet.size());
for (std::map<uint256, CWalletTx>::const_iterator it = pwalletMain->mapWallet.begin(); it != pwalletMain->mapWallet.end(); ++it)
out.push_back(OrderedWtx(it->second.nOrderPos, &it->second));
std::stable_sort(out.begin(), out.end(), OrderedWtxLess);
out.erase(std::unique(out.begin(), out.end(), OrderedWtxSamePos), out.end());
}
// Return current blockchain status, wallet balance, address balance and the last 200 transactions
UniValue getalldata(const UniValue& params, bool fHelp,const CPubKey&)
{
@@ -3031,12 +3079,14 @@ UniValue getalldata(const UniValue& params, bool fHelp,const CPubKey&)
addressBalances.insert(make_pair(addressString,txAmounts));
}
//Create Ordered List
map<int64_t,CWalletTx> orderedTxs;
// Per-call key/nullifier memo for the note loop below and for zsWalletTxJSON (rpchushwallet.h).
// Declared after the LOCK2 above, so it is destroyed before cs_wallet is released.
ZsTxJSONContext zsCtx;
//Classify every wallet tx for the balances (the ordered list for listtransactions is built below)
for (map<uint256, CWalletTx>::iterator it = pwalletMain->mapWallet.begin(); it != pwalletMain->mapWallet.end(); ++it) {
const uint256& wtxid = it->first;
const CWalletTx& wtx = (*it).second;
orderedTxs.insert(std::pair<int64_t,CWalletTx>(wtx.nOrderPos, wtx));
unsigned int txType = 0;
// 0 Unassigend
@@ -3105,18 +3155,19 @@ UniValue getalldata(const UniValue& params, bool fHelp,const CPubKey&)
for (auto & pair : wtx.mapSaplingNoteData) {
SaplingOutPoint op = pair.first;
SaplingNoteData nd = pair.second;
const SaplingNoteData& nd = pair.second;
//Skip Spent
if (nd.nullifier && pwalletMain->IsSaplingSpent(*nd.nullifier))
continue;
//Decrypt sapling incoming commitments using IVK
for (auto addr : zs_addresses) {
libzcash::SaplingExtendedSpendingKey extsk;
if (pwalletMain->GetSaplingExtendedSpendingKey(addr, extsk)) {
const std::vector<ZsTxJSONContext::Key>& zsKeys = zsCtx.keys();
for (size_t k = 0; k < zsKeys.size(); k++) {
const libzcash::SaplingPaymentAddress& addr = zsKeys[k].addr;
{ // spending-key availability is resolved once per call, in ZsTxJSONContext::keys()
auto pt = libzcash::SaplingNotePlaintext::decrypt(
wtx.vShieldedOutput[op.n].encCiphertext,extsk.expsk.full_viewing_key().in_viewing_key(),wtx.vShieldedOutput[op.n].ephemeralKey,wtx.vShieldedOutput[op.n].cm);
wtx.vShieldedOutput[op.n].encCiphertext,zsKeys[k].ivk,wtx.vShieldedOutput[op.n].ephemeralKey,wtx.vShieldedOutput[op.n].cm);
if (txType == 0 && pwalletMain->IsLockedNote(op))
txType == 3;
@@ -3228,10 +3279,14 @@ UniValue getalldata(const UniValue& params, bool fHelp,const CPubKey&)
}
//Ordered list of pointers into mapWallet (cs_wallet is held for the whole call)
std::vector<OrderedWtx> orderedTxs;
BuildOrderedWalletTxs(orderedTxs);
uint64_t t = GetTime();
for (map<int64_t,CWalletTx>::reverse_iterator it = orderedTxs.rbegin(); it != orderedTxs.rend(); ++it)
for (std::vector<OrderedWtx>::reverse_iterator it = orderedTxs.rbegin(); it != orderedTxs.rend(); ++it)
{
const CWalletTx& wtx = (*it).second;
const CWalletTx& wtx = *(it->second);
if (!CheckFinalTx(wtx))
continue;
@@ -3248,7 +3303,7 @@ UniValue getalldata(const UniValue& params, bool fHelp,const CPubKey&)
continue;
}
zsWalletTxJSON(wtx, trans, "*", false, 0);
zsWalletTxJSON(wtx, trans, "*", false, 0, &zsCtx);
if (trans.size() >= nCount) break;
}
@@ -3514,18 +3569,16 @@ UniValue z_listreceivedaddress(const UniValue& params, bool fHelp,const CPubKey&
if (nFilter < 0)
throw runtime_error("Filter must be greater that 0.");
//Created Ordered Transaction Map
map<int64_t,CWalletTx> orderedTxs;
for (map<uint256, CWalletTx>::iterator it = pwalletMain->mapWallet.begin(); it != pwalletMain->mapWallet.end(); ++it) {
const CWalletTx& wtx = (*it).second;
orderedTxs.insert(std::pair<int64_t,CWalletTx>(wtx.nOrderPos, wtx));
}
//Ordered list of pointers into mapWallet (cs_wallet is held for the whole call)
std::vector<OrderedWtx> orderedTxs;
BuildOrderedWalletTxs(orderedTxs);
ZsTxJSONContext zsCtx; // per-call key/nullifier memo (rpchushwallet.h)
uint64_t t = GetTime();
//Reverse Iterate thru transactions
for (map<int64_t,CWalletTx>::reverse_iterator it = orderedTxs.rbegin(); it != orderedTxs.rend(); ++it) {
const CWalletTx& wtx = (*it).second;
for (std::vector<OrderedWtx>::reverse_iterator it = orderedTxs.rbegin(); it != orderedTxs.rend(); ++it) {
const CWalletTx& wtx = *(it->second);
bool includeTransaction = true;
//Excude transactions with less confirmations than required
@@ -3546,7 +3599,7 @@ UniValue z_listreceivedaddress(const UniValue& params, bool fHelp,const CPubKey&
}
if (includeTransaction) {
zsWalletTxJSON(wtx, ret, params[0].get_str() , true, 2);
zsWalletTxJSON(wtx, ret, params[0].get_str() , true, 2, &zsCtx);
}
if (ret.size() > nCount) break;
@@ -3597,7 +3650,7 @@ UniValue z_listsentbyaddress(const UniValue& params, bool fHelp,const CPubKey&)
if (!EnsureWalletIsAvailable(fHelp))
return NullUniValue;
if (fHelp || params.size() > 5 || params.size() == 3)
if (fHelp || params.size() > 6 || params.size() == 3)
throw runtime_error(
"z_listsentbyaddress\n"
"\nReturns decrypted DragonX outputs sent to a single address.\n"
@@ -3622,11 +3675,16 @@ UniValue z_listsentbyaddress(const UniValue& params, bool fHelp,const CPubKey&)
"5. \"Count:\" (numeric, optional, default=9999999) \n"
" Last n number of transactions returned\n"
"\n"
"6. \"Sends only:\" (boolean, optional, default=false) \n"
" true: omit the \"spends\" and \"received\" keys of every entry and skip the\n"
" trial decryption they need. false: the full historic output.\n"
"\n"
"Default Parameters:\n"
"2. 0 - O confimations required\n"
"3. 0 - Returns all transactions\n"
"4. 9999999 - Ignored\n"
"5. 9999999 - Return the last 9,999,999 transactions.\n"
"6. false - Include the spends and received sections.\n"
"\n"
"\nResult:\n"
" \"txid\": \"transactionid\", (string) The transaction id.\n"
@@ -3675,6 +3733,7 @@ UniValue z_listsentbyaddress(const UniValue& params, bool fHelp,const CPubKey&)
int64_t nFilterType = 0;
int64_t nFilter = 9999999;
int64_t nCount = 9999999;
bool fSendsOnly = false; // opt-in, see help: omits the "spends"/"received" keys
if (params.size() >= 2)
nMinConfirms = params[1].get_int64();
@@ -3684,10 +3743,14 @@ UniValue z_listsentbyaddress(const UniValue& params, bool fHelp,const CPubKey&)
nFilter = params[3].get_int64();
}
if (params.size() == 5) {
if (params.size() >= 5) {
nCount = params[4].get_int64();
}
if (params.size() == 6) {
fSendsOnly = params[5].get_bool();
}
if (nMinConfirms < 0)
throw runtime_error("Minimum confimations must be greater that 0");
@@ -3697,18 +3760,16 @@ UniValue z_listsentbyaddress(const UniValue& params, bool fHelp,const CPubKey&)
if (nFilter < 0)
throw runtime_error("Filter must be greater that 0.");
//Created Ordered Transaction Map
map<int64_t,CWalletTx> orderedTxs;
for (map<uint256, CWalletTx>::iterator it = pwalletMain->mapWallet.begin(); it != pwalletMain->mapWallet.end(); ++it) {
const CWalletTx& wtx = (*it).second;
orderedTxs.insert(std::pair<int64_t,CWalletTx>(wtx.nOrderPos, wtx));
}
//Ordered list of pointers into mapWallet (cs_wallet is held for the whole call)
std::vector<OrderedWtx> orderedTxs;
BuildOrderedWalletTxs(orderedTxs);
ZsTxJSONContext zsCtx; // per-call key/nullifier memo (rpchushwallet.h)
uint64_t t = GetTime();
//Reverse Iterate thru transactions
for (map<int64_t,CWalletTx>::reverse_iterator it = orderedTxs.rbegin(); it != orderedTxs.rend(); ++it) {
const CWalletTx& wtx = (*it).second;
for (std::vector<OrderedWtx>::reverse_iterator it = orderedTxs.rbegin(); it != orderedTxs.rend(); ++it) {
const CWalletTx& wtx = *(it->second);
if (!CheckFinalTx(wtx))
continue;
@@ -3731,21 +3792,21 @@ UniValue z_listsentbyaddress(const UniValue& params, bool fHelp,const CPubKey&)
if (nFilterType == 2 && wtx.GetDepthInMainChain() > nFilter)
continue;
zsWalletTxJSON(wtx, ret, "*", false, 0);
zsWalletTxJSON(wtx, ret, "*", false, fSendsOnly ? 3 : 0, &zsCtx);
if (ret.size() >= nCount) break;
}
vector<UniValue> arrTmp = ret.getValues();
std::reverse(arrTmp.begin(), arrTmp.end()); // Return oldest to newest
ret.clear();
ret.setArray();
ret.push_backV(arrTmp);
return ret;
// Return oldest to newest (ret was filled newest first): one copy of each entry instead of
// getValues() + std::reverse + push_backV
UniValue oldestFirst(UniValue::VARR);
{
const std::vector<UniValue>& newestFirst = ret.getValues();
for (std::vector<UniValue>::const_reverse_iterator rit = newestFirst.rbegin(); rit != newestFirst.rend(); ++rit)
oldestFirst.push_back(*rit);
}
return oldestFirst;
}
UniValue z_getstats(const UniValue& params, bool fHelp, const CPubKey& mypk)
@@ -4662,7 +4723,7 @@ UniValue z_listreceivedbyaddress(const UniValue& params, bool fHelp, const CPubK
}
obj.push_back(Pair("outindex", (int)entry.op.n));
obj.push_back(Pair("rawconfirmations", entry.confirmations));
auto wtx = pwalletMain->mapWallet.at(entry.op.hash); //.ToString());
const CWalletTx& wtx = pwalletMain->mapWallet.at(entry.op.hash); //.ToString());
obj.push_back(Pair("time", wtx.GetTxTime()));
obj.push_back(Pair("confirmations", dpowconfs));
@@ -4915,7 +4976,7 @@ UniValue z_viewtransaction(const UniValue& params, bool fHelp, const CPubKey& my
continue;
}
auto op = res->second;
auto wtxPrev = pwalletMain->mapWallet.at(op.hash);
const CWalletTx& wtxPrev = pwalletMain->mapWallet.at(op.hash);
auto decrypted = wtxPrev.DecryptSaplingNote(op).get();
auto notePt = decrypted.first;
auto pa = decrypted.second;
@@ -5018,7 +5079,12 @@ UniValue z_getoperationresult(const UniValue& params, bool fHelp, const CPubKey&
+ HelpExampleRpc("z_getoperationresult", "'[\"operationid\", ... ]'")
);
// This call will remove finished operations
// This call will remove finished operations. Unlike z_getoperationstatus it keeps cs_main +
// cs_wallet: CWallet::RunSaplingSweep/RunSaplingConsolidation/RunAutoShieldCoinbase cancel and
// remove the previous automated operation inside one cs_wallet critical section, and holding
// cs_wallet here keeps this removal serialised with that and with other z_getoperationresult
// callers, exactly as before.
LOCK2(cs_main, pwalletMain->cs_wallet);
return z_getoperationstatus_IMPL(params, true);
}
@@ -5047,7 +5113,12 @@ UniValue z_getoperationstatus(const UniValue& params, bool fHelp, const CPubKey&
UniValue z_getoperationstatus_IMPL(const UniValue& params, bool fRemoveFinishedOperations=false)
{
LOCK2(cs_main, pwalletMain->cs_wallet);
// No cs_main/cs_wallet here: nothing below reads wallet or chain state. AsyncRPCQueue guards
// its operation map with its own mutex, and AsyncRPCOperation::getStatus() (and every
// override) reads only the atomic state_, members fixed at construction (id_, creation_time_,
// contextinfo_, targetHeight_) and result/error under the operation's own lock_. Holding the
// wallet locks only made this poll wait behind wallet scans and block connection. The removing
// form is serialised by its caller, z_getoperationresult, which still holds cs_main + cs_wallet.
std::set<AsyncRPCOperationId> filter;
if (params.size()==1) {
@@ -5058,11 +5129,18 @@ UniValue z_getoperationstatus_IMPL(const UniValue& params, bool fRemoveFinishedO
}
bool useFilter = (filter.size()>0);
UniValue ret(UniValue::VARR);
std::shared_ptr<AsyncRPCQueue> q = getAsyncRPCQueue();
std::vector<AsyncRPCOperationId> ids = q->getAllOperationIds();
for (auto id : ids) {
// Status objects in enumeration order, and one (creation_time, index) key per object for the
// sort. UniValue has no move operations, so sorting the objects themselves deep-copied them on
// every comparison and every element move.
std::vector<UniValue> objs;
objs.reserve(ids.size());
std::vector<std::pair<int64_t, size_t> > order;
order.reserve(ids.size());
for (const AsyncRPCOperationId& id : ids) {
if (useFilter && !filter.count(id))
continue;
@@ -5078,26 +5156,32 @@ UniValue z_getoperationstatus_IMPL(const UniValue& params, bool fRemoveFinishedO
if (fRemoveFinishedOperations) {
// Caller is only interested in retrieving finished results
if ("success"==s || "failed"==s || "cancelled"==s) {
ret.push_back(obj);
q->popOperationForId(id);
objs.push_back(obj);
// Report the operation only if this call is the one that removed it.
if (q->popOperationForId(id)) {
order.push_back(std::make_pair(operation->getCreationTime(), objs.size() - 1));
} else {
objs.pop_back();
}
}
} else {
ret.push_back(obj);
objs.push_back(obj);
order.push_back(std::make_pair(operation->getCreationTime(), objs.size() - 1));
}
}
std::vector<UniValue> arrTmp = ret.getValues();
// sort results chronologically by creation_time
std::sort(arrTmp.begin(), arrTmp.end(), [](UniValue a, UniValue b) -> bool {
const int64_t t1 = find_value(a.get_obj(), "creation_time").get_int64();
const int64_t t2 = find_value(b.get_obj(), "creation_time").get_int64();
return t1 < t2;
// sort results chronologically by creation_time. Compare creation_time only, never the index:
// std::sort then makes exactly the comparisons and moves it made on the status objects before,
// so operations created in the same second keep the order earlier releases returned.
std::sort(order.begin(), order.end(),
[](const std::pair<int64_t, size_t>& a, const std::pair<int64_t, size_t>& b) -> bool {
return a.first < b.first;
});
ret.clear();
ret.setArray();
ret.push_backV(arrTmp);
UniValue ret(UniValue::VARR);
for (size_t i = 0; i < order.size(); i++) {
ret.push_back(objs[order[i].second]);
}
return ret;
}
@@ -6184,8 +6268,8 @@ UniValue z_listoperationids(const UniValue& params, bool fHelp, const CPubKey& m
+ HelpExampleRpc("z_listoperationids", "")
);
LOCK2(cs_main, pwalletMain->cs_wallet);
// No cs_main/cs_wallet: like z_getoperationstatus this only reads the self-locking async
// operation queue and each operation's atomic state.
std::string filter;
bool useFilter = false;
if (params.size()==1) {
src/wallet/wallet.cpp
+196 -33
View File
@@ -801,10 +801,18 @@ bool CWallet::CommitAutomatedTx(const CTransaction& tx) {
void CWallet::SetBestChain(const CBlockLocator& loc)
{
// Default ctor => fFlushOnClose=true => ~CDB runs a full BDB txn_checkpoint over the entire
// cache. Fine for the hourly/shutdown callers; ruinous inside a scan (see SetBestChainNoFlush).
CWalletDB walletdb(strWalletFile);
SetBestChainINTERNAL(walletdb, loc);
}
bool CWallet::SetBestChainNoFlush(const CBlockLocator& loc)
{
CWalletDB walletdb(strWalletFile, "r+", false);
return SetBestChainINTERNAL(walletdb, loc);
}
std::set<std::pair<libzcash::PaymentAddress, uint256>> CWallet::GetNullifiersForAddresses(
const std::set<libzcash::PaymentAddress> & addresses)
{
@@ -1258,8 +1266,15 @@ int CWallet::VerifyAndSetInitialWitness(const CBlockIndex* pindex, bool witnessO
uint256 blockRoot;
uint256 witnessRoot;
if (!nd->nullifier)
if (!nd->nullifier) {
::ClearSingleNoteWitnessCache(nd);
// A note found with only an incoming viewing key (z_importviewingkey) never gets a
// nullifier (UpdateSaplingNullifierNoteMapWithTx needs the full viewing key) and can't be
// spent, so it needs no witness. Building one re-read its block on every call and then
// aborted on *item.second.nullifier below.
if (!HaveSaplingFullViewingKey(nd->ivk))
continue;
}
if (!nd->witnesses.empty() && nd->witnessHeight > 0) {
@@ -1350,7 +1365,10 @@ int CWallet::VerifyAndSetInitialWitness(const CBlockIndex* pindex, bool witnessO
}
nd->witnessHeight = pblockindex->GetHeight();
UpdateSaplingNullifierNoteMapWithTx(wtxItem.second);
nMinimumHeight = SaplingWitnessMinimumHeight(*item.second.nullifier, nd->witnessHeight, nMinimumHeight);
if (item.second.nullifier)
nMinimumHeight = SaplingWitnessMinimumHeight(*item.second.nullifier, nd->witnessHeight, nMinimumHeight);
else
LogPrintf("%s: no witness found for note %s:%d in its block; it stays unspendable until a rescan\n", __func__, wtxHash.ToString(), op.n);
}
}
}
@@ -1436,8 +1454,9 @@ void CWallet::BuildWitnessCache(const CBlockIndex* pindex, bool witnessOnly)
return;
}
if (pwalletMain->fAbortRescan) {
// Do NOT clear fRescanning here: a witness rebuild is not a rescan, and clearing it from
// this path desynchronises the flag from ScanForWalletTransactions, which owns it.
LogPrintf("%s: rescan aborted during witness rebuild\n", __func__);
pwalletMain->fRescanning = false;
return;
}
int h = pbi->GetHeight();
@@ -1692,10 +1711,12 @@ int64_t CWallet::IncOrderPosNext(CWalletDB *pwalletdb)
return nRet;
}
CWallet::TxItems CWallet::OrderedTxItems(std::list<CAccountingEntry>& acentries, std::string strAccount)
CWallet::TxItems CWallet::OrderedTxItems(std::list<CAccountingEntry>& acentries, std::string strAccount, bool fReadOnlyDb)
{
AssertLockHeld(cs_wallet); // mapWallet
CWalletDB walletdb(strWalletFile);
// Only a read cursor over the accounting entries runs on this handle. Pure readers open it
// read-only and without flush-on-close, so the call does not end in a txn_checkpoint.
CWalletDB walletdb(strWalletFile, fReadOnlyDb ? "r" : "r+", !fReadOnlyDb);
// First: get all CWalletTx and CAccountingEntry into a sorted-by-order multimap.
TxItems txOrdered;
@@ -2813,6 +2834,15 @@ bool CWallet::SetHDSeedFromMnemonic(const std::string& phrase)
bool CWallet::GetHDSeedForDerivation(HDSeed& seedOut) const
{
// Single choke point for every HD derivation in the wallet. When the hdchain record could not
// be trusted at load time we do not know whether fMnemonicSeed should be true, and guessing
// wrong derives into an entirely different key tree -- so refuse rather than guess. Callers
// surface this as a clean error (z_getnewaddress, sendmany, shieldcoinbase) or skip
// (autoshield). Transparent address generation falls back to the legacy random-key path,
// because hdChain.seedFp stays null and so IsHDTransparentEnabled() is false.
if (fHDChainUnproven)
return false;
HDSeed stored;
if (!GetHDSeed(stored))
return false;
@@ -2917,12 +2947,13 @@ boost::optional<std::pair<
SaplingPaymentAddress>> CWalletTx::DecryptSaplingNote(SaplingOutPoint op) const
{
// Check whether we can decrypt this SaplingOutPoint
if (this->mapSaplingNoteData.count(op) == 0) {
mapSaplingNoteData_t::const_iterator ndIt = this->mapSaplingNoteData.find(op);
if (ndIt == this->mapSaplingNoteData.end()) {
return boost::none;
}
auto output = this->vShieldedOutput[op.n];
auto nd = this->mapSaplingNoteData.at(op);
const OutputDescription& output = this->vShieldedOutput[op.n];
const SaplingNoteData& nd = ndIt->second;
auto maybe_pt = SaplingNotePlaintext::decrypt(
output.encCiphertext,
@@ -3425,6 +3456,12 @@ int CWallet::ScanForWalletTransactions(CBlockIndex* pindexStart, bool fUpdate)
if(fZdebug)
LogPrintf("%s: fUpdate=%d now=%li\n",__func__,fUpdate,nNow);
// fAbortRescan is sticky: nothing else in the tree ever clears it, so without this a single
// `abortrescan` RPC would disable every later scan AND every later BuildWitnessCache for the
// lifetime of the process (BuildWitnessCache bails on the same flag), freezing witness heights
// while the chain advances and progressively rendering notes unspendable.
pwalletMain->fAbortRescan = false;
pwalletMain->fLastRescanCompleted = false;
pwalletMain->fRescanning = true;
CBlockIndex* pindex = pindexStart;
pwalletMain->rescanStartHeight = pindex->GetHeight();
@@ -3439,6 +3476,51 @@ int CWallet::ScanForWalletTransactions(CBlockIndex* pindexStart, bool fUpdate)
pwalletMain->rescanHeight = pindex ? pindex->GetHeight() : 0;
}
// --- interrupt checkpoint setup -------------------------------------------------------
// Where does the wallet currently believe it has scanned to? A checkpoint may only ever
// ADVANCE that point, and only if this scan is contiguous with it. The RPC entry points
// (rescan / importprivkey / z_importkey / z_importviewingkey) take a caller-supplied start
// height validated only against chainActive.Height(), so a scan can legitimately begin far
// ABOVE the persisted locator -- writing a checkpoint from such a scan would mark the
// skipped range as scanned and hide any funds in it.
CBlockIndex* pindexPersisted = NULL;
{
CWalletDB walletdb(strWalletFile, "r+", false);
CBlockLocator locPersisted;
if (walletdb.ReadBestBlock(locPersisted))
pindexPersisted = FindForkInGlobalIndex(chainActive, locPersisted);
}
const bool fMayCheckpoint = pindexPersisted != NULL &&
pindexStart->GetHeight() <= pindexPersisted->GetHeight() + 1;
if (!fMayCheckpoint) {
LogPrintf("%s: scan starts at %d but the wallet is persisted at %d; progress will NOT be "
"checkpointed on interrupt (a non-contiguous scan cannot safely advance the locator)\n",
__func__, pindexStart->GetHeight(),
pindexPersisted ? pindexPersisted->GetHeight() : -1);
}
// Persist progress when the scan is cut short. `pindexStopped` is the block we were ABOUT to
// scan, so the last fully-processed block is its parent. Resume restarts AT the locator's own
// block (CChain::GetLocator pushes it first; FindForkInGlobalIndex returns it), giving one
// block of deliberate overlap -- idempotent, because AddToWallet only merges when the tx is
// already present. No witness work is needed: witnesses are re-derived from each note's own
// witnessHeight, and witnessRootValidated is in-memory-only so every note is revalidated
// against hashFinalSaplingRoot on the next start.
auto checkpointProgress = [&](const CBlockIndex* pindexStopped) {
if (!fMayCheckpoint || !pindexStopped || !pindexStopped->pprev)
return;
const CBlockIndex* pindexDone = pindexStopped->pprev;
if (pindexDone->GetHeight() <= pindexPersisted->GetHeight())
return; // never move the locator backwards
if (SetBestChainNoFlush(chainActive.GetLocator(pindexDone))) {
LogPrintf("%s: checkpointed scan progress at height %d\n", __func__, pindexDone->GetHeight());
} else {
LogPrintf("%s: FAILED to checkpoint scan progress at height %d; the scan will replay "
"from height %d on the next start\n", __func__, pindexDone->GetHeight(),
pindexPersisted->GetHeight());
}
};
ShowProgress(_("Rescanning..."), 0); // show rescan progress in GUI as dialog or on splashscreen, if -rescan on startup
double dProgressStart = Checkpoints::GuessVerificationProgress(chainParams.Checkpoints(), pindex, false);
double dProgressTip = Checkpoints::GuessVerificationProgress(chainParams.Checkpoints(), chainActive.LastTip(), false);
@@ -3447,15 +3529,20 @@ int CWallet::ScanForWalletTransactions(CBlockIndex* pindexStart, bool fUpdate)
{
pwalletMain->rescanHeight = pindex->GetHeight();
if(pwalletMain->fAbortRescan) {
//TODO: should we update witness caches?
LogPrintf("%s: Rescan aborted at block %d\n", pwalletMain->rescanHeight);
pwalletMain->fRescanning = false;
// The witness caches do NOT need updating here: on resume each note's witnesses are
// re-derived from its own witnessHeight, independently of the locator, and
// witnessRootValidated is in-memory-only so a full validation pass runs next boot.
// What DOES need saving is the locator -- see the checkpoint below.
pwalletMain->fRescanning = false;
pwalletMain->fAbortRescan = false; // consume it; see the note at scan entry
LogPrintf("%s: Rescan aborted at block %d\n", __func__, pwalletMain->rescanHeight);
checkpointProgress(pindex);
return ret;
}
if (ShutdownRequested()) {
//TODO: should we update witness caches?
LogPrintf("%s: Rescan interrupted by shutdown request at block %d\n", pwalletMain->rescanHeight);
pwalletMain->fRescanning = false;
LogPrintf("%s: Rescan interrupted by shutdown request at block %d\n", __func__, pwalletMain->rescanHeight);
checkpointProgress(pindex);
return ret;
}
@@ -3516,6 +3603,9 @@ int CWallet::ScanForWalletTransactions(CBlockIndex* pindexStart, bool fUpdate)
// we are no longer rescanning
pwalletMain->fRescanning = false;
// Reached only by running the loop to the end of the active chain. Callers use this to decide
// whether it is honest to record the wallet as scanned up to the tip.
pwalletMain->fLastRescanCompleted = true;
return ret;
}
@@ -5552,6 +5642,26 @@ void CWallet::GetFilteredNotes(
GetFilteredNotes(saplingEntries, filterAddresses, minDepth, INT_MAX, ignoreSpent, requireSpendingKey);
}
namespace {
// GetFilteredNotes' decrypt-free address pre-filter is only built for at most this many
// filter addresses. Verifying each one costs a Jubjub scalar multiplication, which buys
// nothing when the filter is "every address in the wallet" (z_listunspent, z_getbalances).
// Setting it to 0 disables the pre-filter entirely.
const size_t GETFILTEREDNOTES_PREFILTER_MAX_ADDRESSES = 16;
// True if ivk < 2^251. A genuine Sapling ivk is CRH_ivk output with its top 5 bits cleared
// (librustzcash_crh_ivk), so it is below 2^251 < r_J (the Jubjub subgroup order). Below r_J,
// pk_d = [ivk] g_d determines ivk uniquely. librustzcash_ivk_to_pkd multiplies by the raw
// 256-bit value without reducing it, so a value >= r_J aliases ivk - r_J. Byte 31 of the
// little-endian representation is the most significant byte.
bool IsCanonicalSaplingIvk(const libzcash::SaplingIncomingViewingKey& ivk)
{
return (ivk.begin()[31] & 0xF8) == 0;
}
} // namespace
/**
* Find notes in the wallet filtered by payment addresses, min depth, max depth,
* if the note is spent, if a spending key is required, and if the notes are locked.
@@ -5570,8 +5680,47 @@ void CWallet::GetFilteredNotes(
LogPrintf("%s ignoreLocked=%d\n", __func__, ignoreLocked);
for (auto & p : mapWallet) {
CWalletTx wtx = p.second;
// Decrypt-free address pre-filter. A note matches a Sapling filter address a only if
// nd.ivk.address(a.d) == a. If a's recorded ivk r is canonical and provably generates a
// (r.address(a.d) == a), then any other canonical ivk cannot generate a, so a note whose
// canonical nd.ivk is not one of these r can be skipped without decrypting it. Recorded
// pairs are not trusted as-is (z_importviewingkey stores any (ivk, zaddr) pair the user
// supplies), so each one is verified. If any Sapling filter address has no recorded ivk,
// a non-canonical one, or one that does not generate it, the pre-filter is off and every
// note takes the old decrypt path. InvalidEncoding entries never equal a note's address.
std::set<libzcash::SaplingIncomingViewingKey> prefilterIvks;
bool usePrefilter = !filterAddresses.empty() &&
filterAddresses.size() <= GETFILTEREDNOTES_PREFILTER_MAX_ADDRESSES;
if (usePrefilter) {
for (const libzcash::PaymentAddress& fa : filterAddresses) {
const libzcash::SaplingPaymentAddress* sa = boost::get<libzcash::SaplingPaymentAddress>(&fa);
if (sa == nullptr) {
continue;
}
libzcash::SaplingIncomingViewingKey rivk;
if (!GetSaplingIncomingViewingKey(*sa, rivk) || !IsCanonicalSaplingIvk(rivk)) {
usePrefilter = false;
break;
}
auto derived = rivk.address(sa->d);
if (!derived || !(derived.get() == *sa)) {
usePrefilter = false;
break;
}
prefilterIvks.insert(rivk);
}
}
for (const auto & p : mapWallet) {
// A reference, not a copy: copying a CWalletTx deep-copies every note's witness list.
// Nothing in this loop mutates mapWallet or any CWalletTx, and cs_main + cs_wallet are
// held for the whole function, so the reference stays valid.
const CWalletTx& wtx = p.second;
// A tx without Sapling note data can never yield an entry (the note loop below runs
// zero times), so skip it before the finality, maturity and depth work.
if (wtx.mapSaplingNoteData.empty())
continue;
// Filter the transactions before checking for notes
if (!CheckFinalTx(wtx) || wtx.GetBlocksToMaturity() > 0)
@@ -5594,31 +5743,43 @@ void CWallet::GetFilteredNotes(
}
}
for (auto & pair : wtx.mapSaplingNoteData) {
SaplingOutPoint op = pair.first;
SaplingNoteData nd = pair.second;
for (const auto & pair : wtx.mapSaplingNoteData) {
const SaplingOutPoint& op = pair.first;
const SaplingNoteData& nd = pair.second;
auto maybe_pt = SaplingNotePlaintext::decrypt(
wtx.vShieldedOutput[op.n].encCiphertext,
nd.ivk,
wtx.vShieldedOutput[op.n].ephemeralKey,
wtx.vShieldedOutput[op.n].cm);
assert(static_cast<bool>(maybe_pt));
auto notePt = maybe_pt.get();
auto maybe_pa = nd.ivk.address(notePt.d);
assert(static_cast<bool>(maybe_pa));
auto pa = maybe_pa.get();
// skip notes which belong to a different payment address in the wallet
if (!(filterAddresses.empty() || filterAddresses.count(pa))) {
// Address pre-filter (see prefilterIvks above): a canonical ivk that is not one of
// the verified filter ivks cannot generate any filter address, so the
// filterAddresses check after decryption would reject this note anyway.
if (usePrefilter && IsCanonicalSaplingIvk(nd.ivk) && prefilterIvks.count(nd.ivk) == 0) {
continue;
}
// Spent check before trial decryption: it reads only nd.nullifier and wallet
// state (not the plaintext or the address), so spent notes are never decrypted.
if (ignoreSpent && nd.nullifier && IsSaplingSpent(*nd.nullifier)) {
continue;
}
uint256 pk_d;
auto maybe_pt = SaplingNotePlaintext::decrypt_with_pk_d(
wtx.vShieldedOutput[op.n].encCiphertext,
nd.ivk,
wtx.vShieldedOutput[op.n].ephemeralKey,
wtx.vShieldedOutput[op.n].cm,
pk_d);
assert(static_cast<bool>(maybe_pt));
auto notePt = maybe_pt.get();
// Same value as nd.ivk.address(notePt.d).get(): decryption succeeded, so
// librustzcash_ivk_to_pkd(nd.ivk, d) returned true (d is a valid diversifier, which
// is all check_diversifier tests) and wrote pk_d, the value address() recomputes.
SaplingPaymentAddress pa(notePt.d, pk_d);
// skip notes which belong to a different payment address in the wallet
if (!(filterAddresses.empty() || filterAddresses.count(pa))) {
continue;
}
// skip notes which cannot be spent
if (requireSpendingKey) {
libzcash::SaplingIncomingViewingKey ivk;
@@ -5636,7 +5797,9 @@ void CWallet::GetFilteredNotes(
continue;
}
auto note = notePt.note(nd.ivk).get();
// Same value as notePt.note(nd.ivk).get(), which is
// SaplingNote(d, nd.ivk.address(d)->pk_d, value, rcm): pa is nd.ivk.address(d).
libzcash::SaplingNote note(notePt.d, pa.pk_d, notePt.value(), notePt.rcm);
saplingEntries.push_back(SaplingNoteEntry {
op, pa, note, notePt.memo(), wtx.GetDepthInMainChain() });
}
src/wallet/wallet.h
+37 -10
View File
@@ -833,6 +833,10 @@ public:
bool fAutoShieldRunning = false;
std::atomic<bool> fAbortRescan{false};
//! True only when the last ScanForWalletTransactions ran to completion. An aborted or
//! shutdown-interrupted scan leaves this false, so callers must not record the wallet as
//! scanned up to the chain tip -- doing so makes the unscanned range permanently invisible.
bool fLastRescanCompleted = false;
// abort current rescan
void AbortRescan() { fAbortRescan = true; }
// Are we currently aborting a rescan?
@@ -904,17 +908,22 @@ protected:
*/
void DecrementNoteWitnesses(const CBlockIndex* pindex);
//! Returns true only if the atomic write actually committed. Callers that checkpoint during a
//! long scan need to know: a silently-failing checkpoint would otherwise be retried forever at
//! full cost while never making progress.
template <typename WalletDB>
void SetBestChainINTERNAL(WalletDB& walletdb, const CBlockLocator& loc) {
bool SetBestChainINTERNAL(WalletDB& walletdb, const CBlockLocator& loc) {
if (!walletdb.TxnBegin()) {
// This needs to be done atomically, so don't do it at all
LogPrintf("SetBestChain(): Couldn't start atomic write\n");
return;
return false;
}
try {
LOCK(cs_wallet);
for (std::pair<const uint256, CWalletTx>& wtxItem : mapWallet) {
auto wtx = wtxItem.second;
// By reference: a copy here deep-copies every note's witness deque
// (WITNESS_CACHE_SIZE entries) for every transaction, on every call.
const CWalletTx& wtx = wtxItem.second;
// We skip transactions for which mapSaplingNoteData
// is empty. This covers transactions that have no Sapling data
// (i.e. are purely transparent), as well as shielding and unshielding
@@ -923,32 +932,33 @@ protected:
if (!walletdb.WriteTx(wtxItem.first, wtx)) {
LogPrintf("SetBestChain(): Failed to write CWalletTx, aborting atomic write\n");
walletdb.TxnAbort();
return;
return false;
}
}
}
if (!walletdb.WriteWitnessCacheSize(nWitnessCacheSize)) {
LogPrintf("SetBestChain(): Failed to write nWitnessCacheSize, aborting atomic write\n");
walletdb.TxnAbort();
return;
return false;
}
if (!walletdb.WriteBestBlock(loc)) {
LogPrintf("SetBestChain(): Failed to write best block, aborting atomic write\n");
walletdb.TxnAbort();
return;
return false;
}
} catch (const std::exception &exc) {
// Unexpected failure
LogPrintf("SetBestChain(): Unexpected error during atomic write:\n");
LogPrintf("%s\n", exc.what());
walletdb.TxnAbort();
return;
return false;
}
if (!walletdb.TxnCommit()) {
// Couldn't commit all to db, but in-memory state is fine
LogPrintf("SetBestChain(): Couldn't commit atomic write\n");
return;
return false;
}
return true;
}
private:
@@ -961,6 +971,11 @@ protected:
/* the hd chain data model (chain counters) */
CHDChain hdChain;
//! Set at load time when hdChain cannot be trusted to describe this wallet's HD seed -- the
//! record was missing, or its seedFp does not match the seed actually loaded. While true the
//! wallet is fully usable for existing keys but refuses to DERIVE new ones, because it cannot
//! tell which key tree it belongs to. Never serialized; recomputed on every load.
bool fHDChainUnproven = false;
public:
/*
@@ -1195,10 +1210,12 @@ public:
/**
* Get the wallet's activity log
* @param fReadOnlyDb open wallet.dat read-only and without flush-on-close (pure readers such as
* listtransactions: no txn_checkpoint per call); false = read-write, flush-on-close
* @return multimap of ordered transactions and accounting entries
* @warning Returned pointers are *only* valid within the scope of passed acentries
*/
TxItems OrderedTxItems(std::list<CAccountingEntry>& acentries, std::string strAccount = "");
TxItems OrderedTxItems(std::list<CAccountingEntry>& acentries, std::string strAccount = "", bool fReadOnlyDb = false);
void MarkDirty();
bool UpdateNullifierNoteMap();
@@ -1277,8 +1294,12 @@ public:
void RunSaplingConsolidation(int blockHeight);
void RunAutoShieldCoinbase(int blockHeight);
bool CommitAutomatedTx(const CTransaction& tx);
/** Saves witness caches and best block locator to disk. */
/** Saves witness caches and best block locator to disk. Overrides CValidationInterface. */
void SetBestChain(const CBlockLocator& loc);
/** As SetBestChain, but for use INSIDE a long scan: opens the wallet DB with fFlushOnClose=false
* so the call does not trigger a full BDB txn_checkpoint over the whole cache (see the comment
* at CWallet::SetBestChain), and reports whether the write actually committed. */
bool SetBestChainNoFlush(const CBlockLocator& loc);
std::set<std::pair<libzcash::PaymentAddress, uint256>> GetNullifiersForAddresses(const std::set<libzcash::PaymentAddress> & addresses);
bool IsNoteSaplingChange(const std::set<std::pair<libzcash::PaymentAddress, uint256>> & nullifierSet, const libzcash::PaymentAddress & address, const SaplingOutPoint & entry);
@@ -1415,6 +1436,12 @@ public:
void SetHDChain(const CHDChain& chain, bool memonly);
const CHDChain& GetHDChain() const { return hdChain; }
//! Mark hdChain as untrustworthy for derivation (see the member's declaration). Set only by
//! CWalletDB::LoadWallet; there is deliberately no way to clear it short of reloading, so a
//! degraded wallet cannot be talked back into deriving without a real repair.
void SetHDChainUnproven() { fHDChainUnproven = true; }
bool IsHDChainUnproven() const { return fHDChainUnproven; }
/* Record (in memory and in wallet.dat) how this wallet's HD seed came to
exist. Best-effort: a failed write is logged, not fatal — the next start
simply re-classifies, and re-classification always errs toward
src/wallet/walletdb.cpp
+69 -13
View File
@@ -421,6 +421,15 @@ public:
// True when that record had to be repaired on read (see the "hdchain" case
// in ReadKeyValue); LoadWallet rewrites it in full form afterwards.
bool fHDChainRepaired;
// True once an "hdchain" record was ENCOUNTERED, whether or not it parsed. This is what
// separates the two failure shapes: a v1.0.3-or-earlier -salvagewallet drops the record
// entirely (its IsKeyType has no "hdchain" case), so absent == salvaged and recoverable,
// while present-but-unreadable means wider file damage.
bool fHDChainSeen;
// Fingerprint taken from the KEY of the hdseed/chdseed record. Available even for an
// encrypted wallet, where the seed itself cannot be read at load time.
bool fHDSeedSeen;
uint256 hdSeedFpSeen;
CWalletScanState() {
nKeys = nCKeys = nKeyMeta = nZKeys = nCZKeys = nZKeyMeta = nSapZAddrs = 0;
@@ -429,6 +438,8 @@ public:
nFileVersion = 0;
fHDChainRead = false;
fHDChainRepaired = false;
fHDChainSeen = false;
fHDSeedSeen = false;
}
};
@@ -836,6 +847,8 @@ ReadKeyValue(CWallet* pwallet, CDataStream& ssKey, CDataStream& ssValue,
strErr = "Error reading wallet database: LoadHDSeed failed";
return false;
}
wss.fHDSeedSeen = true;
wss.hdSeedFpSeen = seedFp;
}
else if (strType == "chdseed")
{
@@ -849,9 +862,15 @@ ReadKeyValue(CWallet* pwallet, CDataStream& ssKey, CDataStream& ssValue,
return false;
}
wss.fIsEncrypted = true;
// The fingerprint is the plaintext KEY of the chdseed record, so this works while
// the wallet is locked and the seed itself is unreadable.
wss.fHDSeedSeen = true;
wss.hdSeedFpSeen = seedFp;
}
else if (strType == "hdchain")
{
// Record the ENCOUNTER before any parsing can fail.
wss.fHDChainSeen = true;
CHDChain chain;
// Keep an untouched copy: a failed >> has already consumed part of ssValue.
CDataStream ssRetry(ssValue.begin(), ssValue.end(), ssValue.GetType(), ssValue.GetVersion());
@@ -1046,23 +1065,60 @@ DBErrors CWalletDB::LoadWallet(CWallet* pwallet)
}
}
// A wallet that holds an HD seed but whose hdchain record is missing or
// unreadable is NOT safe to run. hdChain would fall back to its SetNull
// defaults (walletdb.h:105-113), which (a) clears fMnemonicSeed, switching
// HD derivation from the 64-byte BIP39 seed to the raw 32-byte entropy
// (CWallet::GetHDSeedForDerivation, wallet.cpp:2615-2633) -> an entirely
// different key tree, and (b) resets saplingAccountCounter to 0, so the
// next GenerateNewSaplingZKey walks back over accounts that already exist.
// Both are silent today (a bad hdchain read is only DB_NONCRITICAL_ERROR).
// Fail loud instead of quietly deriving into the wrong tree.
if (pwallet->HaveHDSeed() && !wss.fHDChainRead)
// A wallet that holds an HD seed but no usable hdchain record cannot safely DERIVE: hdChain
// falls back to its SetNull defaults, clearing fMnemonicSeed and so switching derivation from
// the 64-byte BIP39 seed to the raw 32-byte entropy -- an entirely different key tree.
//
// saplingAccountCounter was previously listed here as a second hazard. It is not one:
// GenerateNewSaplingZKey loops `do {...} while (HaveSaplingSpendingKey(...))` and
// DeriveNewChildKey loops `while (HaveKey(...))`, so a counter that starts low walks forward
// past accounts that already exist rather than colliding with them.
//
// Two shapes reach here and only one is real corruption:
// (a) the record is ABSENT -- the signature of a -salvagewallet run by v1.0.3 or earlier,
// whose IsKeyType has no "hdchain" case, so salvage dropped it. The keys are intact.
// Refusing strands the wallet with no way back: the node aborts before the RPC server
// exists, and -mnemonic refuses a non-empty wallet, so there is no user-executable
// recovery path at all.
// (b) the record is PRESENT but unreadable -- wider file damage. Keep refusing.
if (pwallet->HaveHDSeed() && !wss.fHDChainRead && wss.fHDChainSeen)
{
LogPrintf("Error loading wallet.dat: HD seed present but the hdchain record is missing or corrupt. "
"Recover by restoring from the seed phrase: move wallet.dat aside and start with "
"-mnemonic=\"<your seed phrase>\" -rescan\n");
LogPrintf("Error loading wallet.dat: the hdchain record is present but unreadable. Your keys "
"are intact -- do NOT delete or replace wallet.dat. Back it up and see debug.log.\n");
return DB_CORRUPT;
}
{
const char* pszDegraded = NULL;
if (pwallet->HaveHDSeed() && !wss.fHDChainRead)
{
pszDegraded = "the hdchain record is missing (an older -salvagewallet drops it)";
}
// A chain WAS read, but it does not belong to the seed we loaded. No legitimate writer can
// produce that -- InstallHDSeed always stores seedFp = seed.Fingerprint(). It is the mark
// of a wallet that lost its hdchain to an old salvage and was then USED on that old build,
// which persists a SetNull-derived chain carrying a null seedFp. Such a wallet otherwise
// starts up perfectly clean and derives into the WRONG TREE forever, silently -- strictly
// worse than failing to open, which is why it is worth detecting here.
else if (pwallet->HaveHDSeed() && wss.fHDSeedSeen &&
pwallet->GetHDChain().seedFp != wss.hdSeedFpSeen)
{
pszDegraded = "the hdchain record does not belong to this wallet's HD seed";
}
if (pszDegraded != NULL)
{
pwallet->SetHDChainUnproven();
LogPrintf("Wallet opened in DEGRADED mode: %s. Existing keys are intact, spendable and "
"receivable, but no NEW HD-derived key can be generated and new transparent "
"addresses will not be recoverable from a seed phrase. Back up wallet.dat now "
"and do NOT delete or replace it.\n", pszDegraded);
// An old salvage also dropped defaultkey and bestblock, which would make this look like
// a first run and skip the rescan, leaving a permanently zero balance.
SoftSetBoolArg("-rescan", true);
}
}
// Any wallet corruption at all: skip any rewriting or
// upgrading, we don't want to make it worse.
if (result != DB_LOAD_OK)
src/zcash/Note.cpp
+13
View File
@@ -121,6 +121,18 @@ boost::optional<SaplingNotePlaintext> SaplingNotePlaintext::decrypt(
const uint256 &epk,
const uint256 &cmu
)
{
uint256 pk_d;
return decrypt_with_pk_d(ciphertext, ivk, epk, cmu, pk_d);
}
boost::optional<SaplingNotePlaintext> SaplingNotePlaintext::decrypt_with_pk_d(
const SaplingEncCiphertext &ciphertext,
const uint256 &ivk,
const uint256 &epk,
const uint256 &cmu,
uint256 &pk_d_out
)
{
auto pt = AttemptSaplingEncDecryption(ciphertext, ivk, epk);
if (!pt) {
@@ -161,6 +173,7 @@ boost::optional<SaplingNotePlaintext> SaplingNotePlaintext::decrypt(
return boost::none;
}
pk_d_out = pk_d;
return ret;
}
src/zcash/Note.hpp
+11
View File
@@ -77,6 +77,17 @@ public:
const uint256 &cmu
);
// Same as decrypt(ciphertext, ivk, epk, cmu). On success it also stores in pk_d_out the
// pk_d = [ivk] g_d that decryption computed, which equals ivk.address(d)->pk_d, so the
// caller does not have to recompute it. pk_d_out is not written on failure.
static boost::optional<SaplingNotePlaintext> decrypt_with_pk_d(
const SaplingEncCiphertext &ciphertext,
const uint256 &ivk,
const uint256 &epk,
const uint256 &cmu,
uint256 &pk_d_out
);
static boost::optional<SaplingNotePlaintext> decrypt(
const SaplingEncCiphertext &ciphertext,
const uint256 &epk,