Notes spent by a send still in the mempool stopped getting witness
updates. After a restart (pending flags are not saved), or a reorg
that un-mined the send, those notes were spendable again but with
stale witnesses, so the next send from them used an anchor that is not
on chain and failed until a rescan. Deterministic note selection made
this sticky: the stale note was picked every time it was the best fit.
Witnesses now advance for every note that is not spent on chain,
pending or not. Witness clearing still keys on the mined spend only,
so pending notes are never cleared. The wallet file format is
unchanged; an old build reading a file written while a send was
pending can now spend from it (verified offline).
The oracle reloads and reorgs inside the pending window and compares
every unspent note's witnesses byte for byte; reverting the fix fails
9 tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The block scan appended each shielded output's leaf to the witness of
every live (unspent, not pending) note as it went, one witness at a time
on the sync thread, while holding the wallet's txs write lock: live
notes x outputs appends per block, all on one core.
With 16 or more live notes, the scan now collects the block's leaves in
output order and appends them to those witnesses once, at the end of the
block, still under the same lock and before anything reads them. When
that is at least 128 appends, the witnesses are split into disjoint
chunks over min(8, cores, live notes / 8) scoped threads; otherwise the
appends run inline. Every witness receives the same leaves in the same
order, including a leaf whose epk does not parse; an output whose cmu
does not parse still adds nothing. The hashing work is the same, only
spread over cores; the speed-up has not been benchmarked yet.
No output change. Each qualifying block starts up to min(8, cores)
short-lived threads.
Tests check that the threaded helper matches the plain double loop byte
for byte for 0-64 witnesses, and that the deferred path handles bad-epk
and bad-cmu outputs. The 1000-seed oracle passes, as do the suite and
125 oracle seeds pinned to 2 cores, and a differential run against the
unmodified backend is byte-identical, apart from the lists cleared two
commits earlier, for wallets below and above 16 live notes and blocks of
0, 1 and 50 outputs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every note keeps a list of up to 100 witnesses, one per block, so that a
reorg can roll it back; a full list takes roughly 40-90 KB of the wallet
file. The block scan kept duplicating and trimming that list for every
note that had one, spent notes included, although a spent note never
reads its history again once its spend can no longer be rolled back. So
every note the wallet ever spent kept costing memory, per-block scan
time and file size.
Once the block window is full (101 blocks), a spent note whose spending
transaction's block is below the window's first block now gets an empty
witness list. This runs before each block's witness update and once when
a wallet is loaded. Such a spend can only be undone by a reorg that pops
the whole window, and that already empties every witness list, so reorgs
behave exactly as before.
Behaviour changes:
- notes whose spend lies below a full reorg window lose their witness
history in memory, at load and in the next save; existing large
wallets shrink on their first save (which, like every save, copies the
previous file to wallet.dat.bak);
- the wallet format (v9) is unchanged; old builds and SilentDragonXLite
read an empty list as an unspendable note, which a spent note already
is;
- new info log at load: "Dropped witness history of N notes whose spend
is deeper than 100 blocks".
The oracle now also accepts an empty list for a spent note that was
cleared before a later partial reorg.
New witness tests compare every block's lists with a model of the old
algorithm, including reorgs across the clear point and an old file with
full lists, which loads and re-saves to the new bytes. The 1000-seed
oracle run passes (12,207 reorgs, 993 sends verified offline), and a
differential run against the unmodified backend is byte-identical apart
from exactly the cleared lists; the old build reads the new files, and
17 sends it built from them verified offline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DanSandClaude Opus 5.5