Persisting the sent-sweep watermark means a relaunch runs a windowed sweep,
which can't see a change made outside the GUI (a Console z_importviewingkey,
an external same-seed restore) that adds an old send below the window. This
adds a periodic full re-sweep so that case is still caught within ~7 days.
- sentSweepSafetyNetDue(now, lastFull) = lastFull > 0 && now - lastFull > 7d
(a wallet that has never full-swept doesn't trigger one; the seeding sweep
records the time). lastFullTime / lastFullHeight ride in the sent_sweep
snapshot; any successful full sweep (seed, verify-mismatch or safety-net)
resets the clock. On load, a valid watermark without a clock starts it now,
so even watermarks seeded before this field eventually re-sweep.
- When due, the snapshot forces a full sweep for that cycle. Gated on: a
watermark exists, a routine (non-syncing) cycle, not the Mining page, and
not in full-sweep backoff -- so a full sweep that can't finish on a slow
daemon backs off instead of retrying every cycle.
Degrades gracefully: ~0.4 s on a note-cache daemon; callLong(120 s) with
backoff on an old one (the deferred banding would keep each cs_main hold
short so it fits there too). Helper + persistence covered by tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
z_listsentbyaddress re-walks the whole wallet when there's no usable
watermark, which the daemon does under cs_main for every send -- hundreds
of seconds, 400+ s and ~19 GB RSS on a large wallet. The watermark lived
only in memory, so every relaunch, reconnect or daemon restart paid for a
full sweep. This persists it so those cases run a bounded windowed sweep
([z, 0, 2, tip - W + margin]) instead (SENT-HISTORY-SWEEP Part 1).
- The watermark (W = last swept tip), its anchor (A = W - margin) and anchor
hash (H_A = getblockhash(A)), plus the enrichment backlog and the full-
sweep backoff, ride in the encrypted history snapshot as an optional
`sent_sweep` object (schema stays v1; a missing/inconsistent one reads as
"no watermark" = today's behaviour, so older builds and payloads are safe).
- Before the first windowed sweep after a load, the anchor is verified
(getblockhash(A) == H_A). A match means any fork is inside the window; a
mismatch (reorg deeper than the margin, different chain/datadir) or an
inability to check drops the watermark and sweeps the whole wallet.
- The first sweep after a load uses a bigger margin (absorbs notifier lag),
and a watermark older than 2400 blocks uses the long timeout (callLong).
- The backlog is persisted WITH W: a pure z->z send's txid lives only in a
sweep result and would be lost if the window moved past an un-enriched one.
- Watermark changes reach disk (folded into the store signature), and every
hard case still forces a full sweep: reorg (anchor verify), GUI rescan /
zap / import / address change, restoreOriginalWallet, and disconnect.
Deferred (follow-ups, each only turns a correct-but-slow case faster, never a
correctness risk): the banded full sweep for seeding on a pre-note-cache
daemon, the external-import tripwire, and the weekly safety-net re-sweep.
Part 2 (the daemon's decrypt cost) is already on the daemon's
perf/sapling-note-cache branch.
Pure helpers + cache round-trip + anchor-verify/large-window covered by
tests; the getblockhash-after-sweep updates the pinned RPC sequences.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Mining tab shows earnings derived from transaction history, but it
was not treated as a wallet-data page, so newly mined or paid-out
earnings did not appear until the user left the tab and came back. A
coinbase row also changes category in place (immature becomes generate
at maturity, or orphan on a reorg), and the Recent merge kept both the
old and the new row, double-counting the earnings.
- The Mining tab now gets the cheap between-blocks Recent poll: one
listtransactions per new block, bounded by the 15 s transaction-age
timer, once history has loaded. It never runs a full routine scan and
never pre-empts an owed dirty cycle; tracked sends still take their
normal cycle on any page.
- The shielded-payout probe runs only while the pool miner is running,
since pool payouts are shielded receives while a solo miner's earnings
are transparent coinbase rows.
- The Recent merge drops the previous row of the other coinbase category
when a coinbase transitions, so maturing or reorged rewards are no
longer counted twice. A non-coinbase row sharing the txid is left
alone.
Behaviour change: mining earnings update on the Mining tab per block
instead of only on tab re-entry, and maturing or reorged coinbase rows
no longer double-count.
Tests: the per-block poll gate (one poll per block, due only after
history loads, held while dirty or addresses are dirty), the Mining-page
cycle shaping, and the coinbase transition merge for immature to
generate, generate to orphan, and a shared-txid non-coinbase row.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The balance, address scan, history and probe were throttled one by one at
10% each, which added up to 30-40% of dragonxd's cs_main with all of them
due. services::WalletScanGovernor runs one wallet scan at a time across
both RPC lanes and keeps one shared quiet clock at 15% of wall-clock (cut
when the node is behind). Waiters are served first in, first out, with a
head start for what the page shows and a starvation floor of max(30 s
visible / 900 s hidden, 15x the stream's cost). Explicit work (a dirty
set, the user's send, a first load) is never refused but is charged.
Routine scans run only when the tip, the mempool size (getmempoolinfo:
no cs_main) or a wallet-mutation epoch changed since their last good scan,
or a safety sweep is due. Pages that don't show the balance poll it at
most every max(10 min, 50x its cost) (2 min with client-side auto-shield);
coming back into view with an old value fetches it at once. The staleness
badge counts from the last poll that confirmed nothing changed, and hides
while the budget holds the balance within its guarantee.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every block re-ran z_listreceivedbyaddress for every z-address (O(wallet)
each, under cs_main) and a whole-wallet z_listsentbyaddress. On a 40-address
wallet that was ~80% of wall-clock. The history now advances in passes:
- an address is rescanned when it is stale by a number of blocks scaled to
its measured cost (8 to 500), never-scanned and hinted addresses first
(a hint: z_listunspent shows a note the history doesn't);
- z_listsentbyaddress covers only the blocks since the last good sweep
(+100 reorg margin); the whole-wallet sweep runs once per connect, with
a 10 min / 1 h / 6 h backoff after real failures;
- listtransactions pages are sized from last time, and an unchanged tip
reuses the transparent rows;
- found-but-unenriched txids carry over as a backlog; only -5 is a
definitive "not ours" for z_viewtransaction;
- between blocks, a cheap recent poll plus a one-address probe.
Duty model for a 40-address wallet at 10 s per scan: ~9% instead of ~80%.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DanSandClaude Opus 4.8