perf(net): one scan budget for all wallet scans, gated on what changed

The balance, address scan, history and probe were throttled one by one at
10% each, which added up to 30-40% of dragonxd's cs_main with all of them
due. services::WalletScanGovernor runs one wallet scan at a time across
both RPC lanes and keeps one shared quiet clock at 15% of wall-clock (cut
when the node is behind). Waiters are served first in, first out, with a
head start for what the page shows and a starvation floor of max(30 s
visible / 900 s hidden, 15x the stream's cost). Explicit work (a dirty
set, the user's send, a first load) is never refused but is charged.

Routine scans run only when the tip, the mempool size (getmempoolinfo:
no cs_main) or a wallet-mutation epoch changed since their last good scan,
or a safety sweep is due. Pages that don't show the balance poll it at
most every max(10 min, 50x its cost) (2 min with client-side auto-shield);
coming back into view with an old value fetches it at once. The staleness
badge counts from the last poll that confirmed nothing changed, and hides
while the budget holds the balance within its guarantee.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-09 11:06:34 -05:00
co-authored by Claude Opus 5.5
parent 879de37d18
commit 323143e111
19 changed files with 1470 additions and 121 deletions
+35 -22
View File
@@ -27,6 +27,7 @@
#include "daemon/daemon_busy_detect.h"
#include "rpc/rpc_busy_gate.h"
#include "services/history_scan_policy.h"
#include "services/wallet_scan_governor.h"
#include "services/network_refresh_service.h"
#include "services/wallet_security_controller.h"
#include "services/wallet_security_workflow.h"
@@ -439,6 +440,17 @@ public:
// form (every retained op) for the rest of this connection.
bool opidFilterUnsupported() const { return opid_filter_unsupported_.load(std::memory_order_relaxed); }
// Something changed the wallet that the scans' change signals (tip, mempool) don't show: a send or other
// operation finished, an import, a Console command. The change-gated scans run again.
void noteWalletMutation() { ++wallet_mutation_epoch_; }
// Minimized, hidden or occluded: no page is visible, so no scan stream gets the visible head start.
void setWindowBackground(bool background) { window_background_ = background; }
// A balance scan is in flight (the staleness badge waits for it).
bool isBalanceRefreshInFlight() const { return scan_governor_.inFlight(services::ScanStream::Balance); }
// The balance is waiting for the scan budget and still within what it promises (its floor plus one scan):
// an old value then is the budget at work, not a failing node.
bool isBalanceHeldByScanBudget() const;
// RAII around one long ("maintenance") rpc_ call made on a worker thread with a timeout above the
// default 30 s, e.g. a merge or a whole-wallet scan given 120 s. rpc_ holds its curl lock for the
// whole transfer and disconnect() takes the same lock, so these are counted for
@@ -1348,7 +1360,6 @@ private:
// off in proportion to its own measured cost. See effectivelySyncing() / balanceRefreshDue().
bool was_core_syncing_ = false; // previous Core-refresh sync state, to detect the caught-up edge
std::time_t sync_settle_until_ = 0; // hold the sync-throttle until this wall-clock time (0 = not settling)
double last_balance_scan_ms_ = 0.0; // measured cost of the last z_gettotalbalance scan
bool force_balance_refresh_ = false; // a wallet mutation forces the next balance poll through the throttle
// Bumped with each forced-refresh request, so the poll that carried the forced balance clears the
// flag only if no newer request came in meanwhile (see requestForcedBalanceRefresh()).
@@ -1360,17 +1371,21 @@ private:
// that actually sent the forced balance call, not at dispatch, so a dispatch the job queue refuses
// (a Core job already in flight) or one that skips the balance (node behind or busy) keeps it.
void requestForcedBalanceRefresh() { force_balance_refresh_ = true; ++force_balance_seq_; }
// Same adaptive back-off applied to the other two O(mapWallet) scans that hold the daemon's cs_main:
// the address scan (z_listunspent) and the history scan (z_listreceivedbyaddress). Without this, a
// large shielded wallet re-scans them every tab cadence (~seconds each), saturating cs_main and
// starving block connection near the tip (where effectivelySyncing() reads false). See
// addressRefreshDue() / txRefreshDue(); only the routine periodic poll is throttled — explicit
// refreshes (tab switch, dirty set, in-flight send) call the refresh directly and bypass this.
double last_address_scan_ms_ = 0.0; // measured cost of the last address scan
double last_tx_scan_ms_ = 0.0; // measured cost of the last wallet-wide history scan
// When that scan ran (wall-clock): the history throttle's clock. Not state_.last_tx_update, which
// every pending-send row edit and probe-less poll also stamps (that pushed the next scan out).
std::int64_t last_tx_scan_at_ = 0;
// One budget for every wallet-wide scan the GUI sends on its own (balance, addresses, history, the
// between-blocks probe): one in flight at a time across both lanes, ~15% of wall-clock in total. They
// used to be throttled one by one at 10% each, which added up to 30-40% with all of them due.
// Explicit needs (dirty sets, the user's own send, a never-loaded list) bypass it but are charged.
services::WalletScanGovernor scan_governor_;
// The change signals the routine scans are gated on, besides the tip hash (state_.sync.best_blockhash).
std::uint64_t wallet_mutation_epoch_ = 0; // bumped by noteWalletMutation()
bool mempool_known_ = false; // the last Core poll's getmempoolinfo
std::int64_t mempool_txs_ = 0;
std::int64_t mempool_bytes_ = 0;
bool window_background_ = false; // setWindowBackground()
static constexpr float kScanRetrySec = 2.0f; // a routine scan the governor held back asks again this soon
services::ScanSignals currentScanSignals() const;
services::ScanVisibility currentScanVisibility() const;
static double scanClockNow(); // the governor's clock: steady-clock seconds
// Auto-clear for secrets copied to the clipboard. Only a hash of the copied secret is kept.
std::uint64_t clipboard_secret_hash_ = 0;
double clipboard_clear_deadline_ = 0.0;
@@ -1520,6 +1535,8 @@ private:
// mined (a mempool hint doesn't use that up, so the row also catches its confirmation).
std::unordered_map<std::string, int> history_hint_floors_;
std::unordered_set<std::string> history_hinted_txids_;
// The between-blocks probe's progress through the z-addresses since what it watches last changed.
services::ProbeRotation recent_probe_rotation_;
// The z-addresses of the last successful address enumeration this session: a never-scanned address new
// to it (an imported key) resets the sent sweep's watermark, even if the list in between came back blank.
std::unordered_set<std::string> history_listed_z_;
@@ -1761,12 +1778,14 @@ private:
// Private methods - data refresh
void refreshData(); // Orchestrator: dispatches per-category refreshes
void refreshCoreData(bool allowBalance = true); // Balance + blockchain info (can use fast_worker_)
void refreshAddressData(); // Address lists + balances
// Address lists + balances. With no ticket it is an explicit refresh (charged to the scan budget).
void refreshAddressData(services::ScanTicket ticket = {});
// Transaction history: decides the cycle (services::decideTxCycle) and starts it. Safe to call any
// time: it refuses when the node can't be scanned (not connected, locked, warming up, rescanning, busy).
void refreshTransactionData();
void startTransactionCycle(bool full, bool routineAllowed); // Full or Light cycle
void startRecentTransactionPoll(bool probe); // between-blocks poll
// Returns true when a routine scan was wanted but the scan budget held it back (ask again soon).
bool refreshTransactionData();
void startTransactionCycle(bool full, bool routineAllowed, services::ScanTicket ticket); // Full or Light
void startRecentTransactionPoll(bool probe, services::ScanTicket ticket); // between blocks
void markTransactionsDirty() { transactions_dirty_ = true; ++tx_dirty_generation_; }
std::vector<std::string> shieldedAddressList() const;
services::ShieldedHistoryPass::Work historyScanWork() const;
@@ -1780,12 +1799,6 @@ private:
void refreshWalletEncryptionState();
void applyRefreshPolicy(ui::NavPage page);
bool effectivelySyncing() const; // syncing, or within the post-sync settle window (hysteresis)
bool balanceRefreshDue() const; // adaptive: enough time elapsed given the last balance-scan cost?
bool addressRefreshDue() const; // same adaptive back-off for the address scan (z_listunspent)
bool txRefreshDue() const; // same adaptive back-off for wallet-wide history scans
// Shared duty-cycle rule: a scan may resume only once (lastScanMs / kScanDutyCycle) has elapsed since
// lastUpdate, so any single O(mapWallet) scan can occupy at most ~kScanDutyCycle of wall-clock.
bool scanRefreshDue(std::int64_t lastUpdate, double lastScanMs) const;
bool currentPageNeedsWalletDataRefresh() const;
bool shouldRunWalletTransactionRefresh() const;
bool shouldRefreshRecentTransactions() const;