wallet/main: break the replay -> witness-rebuild -> kill loop on large wallets
A v1.3.0 node with a ~13.6k-note wallet (-dbcache=4084) looked hung for hours:
every start replayed ~62k blocks from a chainstate 26 days stale, then ran a
47-60 min all-core witness rebuild holding LOCK2(cs_main, cs_wallet) (RPC answered
HTTP 500 "Work queue depth exceeded", peers timed out), the GUI killed it before
anything was flushed, and the next start did it all again. Even after escaping,
each new block cost ~30s under LOCK2, so the node fell behind a 36s chain.
D1 BuildWitnessCache: the post-rebuild latch computed witnesses.front().root()
(~2.4ms) for every pending note on every block, only to feed a log line that
is printed for bulk rebuilds alone. The anchor check now runs on the Phase-2
workers and only for a bulk rebuild; the per-block path just sets the latch.
Phase 2 also runs in parallel whenever the job is big (notes x commitments
>= 2000), not only for a bulk range, so busy blocks no longer extend every
note serially. If the OS refuses a worker thread, the remaining shards run
inline instead of unwinding past joinable threads (std::terminate).
D2 FlushStateToDisk: the coins cache was only flushed on shutdown, gettxoutsetinfo
or after 24h; on this chain it stays a few MiB, so the size triggers never
fire and a node that never reached 24h uptime or a clean stop kept its
chainstate (and block index) weeks behind. Flush at least hourly and every
DATABASE_FLUSH_BLOCKS (5000) connected blocks -- the latter also inside one
long ActivateBestChain -- and right after the startup ActivateBestChain when
it moved the tip.
D3 UpdatedNoteData: a replayed or rescanned tx arrives with FindMySaplingNotes
data (no nullifier). Witnesses and witnessHeight were carried over but the
nullifier was dropped, so VerifyAndSetInitialWitness cleared and reseeded the
witnesses on the next block -- the replay above rewrote 3,750 txs and forced
the full rebuild. Carry the nullifier with the witnesses, unless the tx was
re-mined in a different block (its notes moved; reseed as before). When the
merged data is identical, don't rewrite the tx; just clear the in-memory
witnessRootValidated so the next block re-checks the kept roots once (a
rescan still finds and reseeds a witness that went bad, without rebuilding
the healthy ones). The carry loop no longer deep-copies every note.
SetBestChain: with hourly coin flushes the wallet locator must not run ahead
of the async notifier, or a crash permanently skips the blocks it had not yet
delivered. Stamp the block the wallet's state reflects (pindexWalletTip,
seeded in init before ThreadNotifyWallets starts -- at the startup rescan's
checkpoint if that rescan was interrupted -- and advanced contiguously by
ChainTip), like zcashd's ChainTipAdded. ThreadNotifyWallets no longer hands
the wallet an empty block after a fatal block-read failure.
Tests: new gtest test_wallet_notedata.cpp (5 tests); 23/23 gtests pass, and the
two core D3 tests fail with D3 reverted. Regtest A/B against master with a
1,008-note wallet: notify-thread CPU per block 1,780ms -> 3ms; worst LOCK2 RPC
stall 1.6-2.9s -> 10-15ms on ordinary blocks and 4.6-7.1s -> ~1.6s on a block
carrying a 40-output self-send; a z_importkey rescan went from 35 reseeds plus a
bulk rebuild (10.6s) to 0 reseeds, same balance; after kill -9 at 150 blocks a
build with a 100-block flush threshold restarted at 100, master at 0. Two
adversarial review rounds; every confirmed finding is addressed here.
Not covered: the hourly SetBestChain/flush paths were reviewed, not exercised
(GetTimeMicros ignores mocktime); no run yet against a copy of a real large wallet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+7
-1
@@ -965,8 +965,12 @@ private:
|
||||
template <class T>
|
||||
void SyncMetaData(std::pair<typename TxSpendMap<T>::iterator, typename TxSpendMap<T>::iterator>);
|
||||
|
||||
// Last block ThreadNotifyWallets delivered to ChainTip, i.e. the block the wallet's state
|
||||
// reflects (guarded by cs_wallet). SetBestChain stamps this instead of the chain tip.
|
||||
const CBlockIndex* pindexWalletTip = nullptr;
|
||||
|
||||
protected:
|
||||
bool UpdatedNoteData(const CWalletTx& wtxIn, CWalletTx& wtx);
|
||||
bool UpdatedNoteData(const CWalletTx& wtxIn, CWalletTx& wtx, bool fSameBlock);
|
||||
void MarkAffectedTransactionsDirty(const CTransaction& tx);
|
||||
|
||||
/* the hd chain data model (chain counters) */
|
||||
@@ -1294,6 +1298,8 @@ public:
|
||||
bool CommitAutomatedTx(const CTransaction& tx);
|
||||
/** Saves witness caches and best block locator to disk. Overrides CValidationInterface. */
|
||||
void SetBestChain(const CBlockLocator& loc);
|
||||
// Seed the block the wallet's state reflects, before ThreadNotifyWallets starts delivering.
|
||||
void SetWalletTip(const CBlockIndex* pindex);
|
||||
/** As SetBestChain, but for use INSIDE a long scan: opens the wallet DB with fFlushOnClose=false
|
||||
* so the call does not trigger a full BDB txn_checkpoint over the whole cache (see the comment
|
||||
* at CWallet::SetBestChain), and reports whether the write actually committed. */
|
||||
|
||||
Reference in New Issue
Block a user