DragonX/ObsidianDragon
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 10 Wiki Activity

perf(history): scan shielded history in bounded passes, not every block

Browse Source 
Every block re-ran z_listreceivedbyaddress for every z-address (O(wallet)
each, under cs_main) and a whole-wallet z_listsentbyaddress. On a 40-address
wallet that was ~80% of wall-clock. The history now advances in passes:

- an address is rescanned when it is stale by a number of blocks scaled to
  its measured cost (8 to 500), never-scanned and hinted addresses first
  (a hint: z_listunspent shows a note the history doesn't);
- z_listsentbyaddress covers only the blocks since the last good sweep
  (+100 reorg margin); the whole-wallet sweep runs once per connect, with
  a 10 min / 1 h / 6 h backoff after real failures;
- listtransactions pages are sized from last time, and an unchanged tip
  reuses the transparent rows;
- found-but-unenriched txids carry over as a backlog; only -5 is a
  definitive "not ours" for z_viewtransaction;
- between blocks, a cheap recent poll plus a one-address probe.

Duty model for a 40-address wallet at 10 s per scan: ~9% instead of ~80%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DanS
2026-10-09 11:06:34 -05:00
co-authored by Claude Opus 5.5
parent 8b9a498eee
commit 879de37d18
10 changed files with 1645 additions and 233 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/app_network.cpp
+328 -101
View File
@@ -99,8 +99,9 @@ std::string unencryptedTransactionHistoryCacheKey(const std::string& walletIdent
class AppRefreshRpcGateway final : public NetworkRefreshService::RefreshRpcGateway {
public:
AppRefreshRpcGateway(rpc::RPCClient& rpc, std::string source)
: rpc_(rpc), source_(std::move(source)) {}
// `maintenanceOwner`: the App whose rpc_ this is, for callLong() (see App::MaintenanceCallScope).
AppRefreshRpcGateway(rpc::RPCClient& rpc, std::string source, App* maintenanceOwner = nullptr)
: rpc_(rpc), source_(std::move(source)), owner_(maintenanceOwner) {}
json call(const std::string& method, const json& params) override
{
@@ -108,9 +109,20 @@ public:
return rpc_.call(method, params);
}
// Holds rpc_'s curl lock for up to timeoutSec: as a maintenance call, a main-thread disconnect
// aborts it rather than waiting for it (App::abortMaintenanceCalls()).
json callLong(const std::string& method, const json& params, long timeoutSec) override
{
std::optional<App::MaintenanceCallScope> maintenance;
if (owner_) maintenance.emplace(*owner_);
rpc::RPCClient::TraceScope trace(source_);
return rpc_.call(method, params, timeoutSec);
}
private:
rpc::RPCClient& rpc_;
std::string source_;
App* owner_ = nullptr;
};
const char* tracePageName(ui::NavPage page)
@@ -150,19 +162,6 @@ std::size_t shieldedReceiveScanBudget(ui::NavPage page)
return page == ui::NavPage::History ? 8u : 4u;
}
// How far the tip may drift past an address's last shielded scan before we re-scan it. A full pass
// scans ~budget addresses per refresh cycle (≈96 per block of wall time), so a wallet with many
// z-addresses takes several blocks to scan fully. With a strict (tolerance 0) "scanned at tip"
// check, new blocks arriving mid-pass would invalidate already-scanned addresses and the pass would
// never complete — leaving transactions_dirty_ (and its "refreshing history" banner + send-progress
// gate) stuck on forever. Scaling the tolerance with the address count lets the pass complete while
// keeping shielded-receive latency minimal for small wallets; it's capped for pathological sizes.
int shieldedScanTipTolerance(std::size_t shieldedAddressCount)
{
int t = 2 + static_cast<int>(shieldedAddressCount / 96);
return std::min(t, 50);
}
} // namespace
// ============================================================================
@@ -841,6 +840,9 @@ void App::onDisconnected(const std::string& reason)
opid_poll_in_progress_ = false;
opid_filter_unsupported_ = false; // a reconnect may be to a different daemon
last_total_balance_zat_.reset();
history_transparent_rows_ = 0; // the history's scan state (the pass, epochs and watermark are
history_transparent_tip_ = -1; // reset with the cache session below); cost estimates are kept
last_tx_scan_at_ = 0;
ui::ShieldDialog::invalidateScope();
address_validation_cache_dirty_ = true;
resetTransactionHistoryCacheSession();
@@ -1043,7 +1045,9 @@ bool App::addressRefreshDue() const
bool App::txRefreshDue() const
{
return scanRefreshDue(state_.last_tx_update, last_tx_scan_ms_);
// Keyed on the last wallet-wide scan, not state_.last_tx_update (pending-send row edits and probe-less
// polls stamp that too, which kept pushing the next scan out).
return scanRefreshDue(last_tx_scan_at_, last_tx_scan_ms_);
}
bool App::currentPageNeedsWalletDataRefresh() const
@@ -1059,7 +1063,7 @@ bool App::shouldRunWalletTransactionRefresh() const
{
if (currentPageNeedsWalletDataRefresh()) return true;
if (hasTransactionSendProgress() || !send_txids_.empty()) return true;
return transactions_dirty_ && !shielded_history_scan_pending_;
return transactions_dirty_;
}
void App::setCurrentPage(ui::NavPage page)
@@ -1083,7 +1087,7 @@ void App::setCurrentPage(ui::NavPage page)
network_refresh_.reset(RefreshTimer::Core);
break;
case NP::History:
transactions_dirty_ = true;
markTransactionsDirty();
refreshTransactionData();
network_refresh_.reset(RefreshTimer::Transactions);
break;
@@ -1108,25 +1112,14 @@ void App::setCurrentPage(ui::NavPage page)
// Screenshot sweep (tab-only + full-UI) + its demo data live in src/app_sweep.cpp.
bool App::shouldRefreshTransactions() const
{
// NOTE: this is block-height / dirty driven, NOT interval-gated. It returns true only when a new
// block arrived (currentBlocks != last_tx_block_height_), the history was never fetched, or
// something marked it dirty (tab entry, a send, a reorg, etc.). The Transactions timer only
// controls how often this CHECK runs; between blocks the lightweight recent-poll
// (shouldRefreshRecentTransactions / TxAge) handles mempool + unconfirmed deltas instead.
const int currentBlocks = state_.sync.blocks;
return network_refresh_.shouldRefreshTransactions(last_tx_block_height_,
currentBlocks,
transactions_dirty_);
}
bool App::shouldRefreshRecentTransactions() const
{
// No same-tip condition: at a new tip services::decideTxCycle runs a Full or Light cycle first when it
// can, so the cheap poll runs there only while the scan throttle holds the Full cycle back (it keeps
// transparent rows and confirmations live meanwhile, and doesn't consume the new tip).
using RefreshTimer = services::NetworkRefreshService::Timer;
return network_refresh_.isDue(RefreshTimer::TxAge)
&& last_tx_block_height_ >= 0
&& state_.sync.blocks == last_tx_block_height_
&& !state_.transactions.empty()
&& !transactions_dirty_
&& !addresses_dirty_;
@@ -1643,6 +1636,14 @@ void App::resetTransactionHistoryCacheSession()
wipePendingTransactionHistoryCachePassphrase();
transaction_history_cache_loaded_ = false;
invalidateShieldedHistoryScanProgress(false);
history_listed_z_.clear();
}
void App::lockTransactionHistoryCacheSession()
{
transaction_history_cache_.lockKey();
wipePendingTransactionHistoryCachePassphrase();
transaction_history_cache_loaded_ = false;
}
void App::pruneShieldedHistoryScanProgress()
@@ -1662,15 +1663,70 @@ void App::pruneShieldedHistoryScanProgress()
}
}
void App::invalidateShieldedHistoryScanProgress(bool persistCache)
void App::invalidateShieldedHistoryScanProgress(bool persistCache, bool keepSentScan)
{
shielded_history_scan_cursor_ = 0;
shielded_history_scan_pending_ = false;
initial_history_scan_complete_ = false; // a full re-scan is coming — show load progress again
shielded_history_scan_heights_.clear();
history_pass_.reset();
++history_scan_epoch_; // results of scans already in flight no longer apply
history_hint_floors_.clear();
history_hinted_txids_.clear();
if (!keepSentScan) {
history_sent_scan_height_ = -1; // the next sweep covers the whole wallet again
history_enrich_backlog_.clear();
history_full_sweep_failures_ = 0;
history_full_sweep_retry_at_ = 0;
++history_sent_epoch_;
}
if (persistCache) storeTransactionHistoryCacheIfAvailable();
}
std::vector<std::string> App::shieldedAddressList() const
{
// The same order as NetworkRefreshService::buildTransactionRefreshSnapshot.
std::vector<std::string> addresses;
addresses.reserve(state_.z_addresses.size());
for (const auto& address : state_.z_addresses) {
if (!address.address.empty()) addresses.push_back(address.address);
}
return addresses;
}
void App::noteHistoryHintsFromUnspent(
const std::vector<services::NetworkRefreshService::UnspentNoteLite>& notes)
{
const int tip = state_.sync.blocks;
if (tip <= 0 || notes.empty()) return;
std::unordered_map<std::string, bool> rowConfirmed; // txid -> any row of it is mined
for (const auto& transaction : state_.transactions) {
if (transaction.txid.empty()) continue;
bool& confirmed = rowConfirmed[transaction.txid];
confirmed = confirmed || transaction.confirmations >= 1;
}
for (const auto& note : notes) {
if (note.change || note.txid.empty() || note.address.empty()) continue;
if (history_hinted_txids_.count(note.txid)) continue; // once per txid: change/merges can't loop
const auto row = rowConfirmed.find(note.txid);
const bool noRow = row == rowConfirmed.end();
const bool staleUnconfirmed = !noRow && !row->second && note.confirmations >= 1;
if (!noRow && !staleUnconfirmed) continue;
int& floor = history_hint_floors_[note.address];
floor = std::max(floor, tip);
// A mempool hint doesn't use up the txid: once mined, its row (still at 0 conf) needs one more scan.
// No loop: an unmined note with a row matches neither case, and a mined one hints once.
if (note.confirmations >= 1) history_hinted_txids_.insert(note.txid);
}
}
services::ShieldedHistoryPass::Work App::historyScanWork() const
{
const auto addresses = shieldedAddressList();
return history_pass_.evaluate(state_.sync.blocks, addresses, shielded_history_scan_heights_,
services::historyStaleAfterBlocks(addresses.size(), history_addr_scan_ms_),
initial_history_scan_complete_);
}
bool App::ensureTransactionHistoryCacheUnlockedFor(const std::string& walletIdentity)
{
if (walletIdentity.empty()) return false;
@@ -1741,7 +1797,7 @@ void App::loadTransactionHistoryCacheIfAvailable()
}
confirmed_cache_block_ = loaded.tipHeight;
transaction_history_cache_loaded_ = true;
transactions_dirty_ = true;
markTransactionsDirty();
network_refresh_.markDue(services::NetworkRefreshService::Timer::Transactions);
}
@@ -1790,15 +1846,12 @@ void App::refreshData()
// as each completes, rather than waiting for the slowest phase.
refreshCoreData();
bool addressRefreshNeeded = addresses_dirty_;
bool walletDataPage = currentPageNeedsWalletDataRefresh();
if (addressRefreshNeeded)
// The address list first: the history needs it (the address callback marks the history due).
// refreshTransactionData() picks the cycle itself (Full / Light / the between-blocks poll / none).
if (addresses_dirty_)
refreshAddressData();
if (!addressRefreshNeeded && shouldRunWalletTransactionRefresh() && shouldRefreshTransactions())
else if (shouldRunWalletTransactionRefresh())
refreshTransactionData();
else if (!addressRefreshNeeded && walletDataPage && shouldRefreshRecentTransactions())
refreshRecentTransactionData();
// Always refresh peers here — refreshData() only runs on one-shot transitions (connect,
// warmup-complete, unlock), so this populates the status-bar peer count immediately on open
@@ -1857,7 +1910,7 @@ void App::refreshCoreData(bool allowBalance)
resetWitnessRescanProgress();
state_.sync.rescan_progress = 1.0f;
// Notes/witnesses were rebuilt — force a fresh history + balance pull.
transactions_dirty_ = true;
markTransactionsDirty();
last_tx_block_height_ = -1;
invalidateShieldedHistoryScanProgress(true);
if (user_initiated_rescan_) {
@@ -2077,15 +2130,17 @@ void App::refreshAddressData()
if (!worker_ || !rpc_ || !state_.connected) return;
const std::size_t previousAddressCount = state_.z_addresses.size() + state_.t_addresses.size();
const std::string previousWalletIdentity = transactionHistoryCacheWalletIdentity();
const auto zList = shieldedAddressList();
std::unordered_set<std::string> previousZ(zList.begin(), zList.end());
auto addressSnapshot = address_validation_cache_dirty_
? NetworkRefreshService::AddressRefreshSnapshot{}
: NetworkRefreshService::buildAddressRefreshSnapshot(state_);
ui::NavPage tracePage = current_page_;
auto enqueued = network_refresh_.enqueue(services::NetworkRefreshService::Job::Addresses, *worker_, [this, previousAddressCount, previousWalletIdentity, addressSnapshot = std::move(addressSnapshot), tracePage]() -> rpc::RPCWorker::MainCb {
auto enqueued = network_refresh_.enqueue(services::NetworkRefreshService::Job::Addresses, *worker_, [this, previousAddressCount, previousWalletIdentity, previousZ = std::move(previousZ), addressSnapshot = std::move(addressSnapshot), tracePage]() -> rpc::RPCWorker::MainCb {
AppRefreshRpcGateway refreshRpc(*rpc_, traceSource(tracePage, "Address refresh"));
auto result = NetworkRefreshService::collectAddressRefreshResult(refreshRpc, addressSnapshot);
return [this, previousAddressCount, previousWalletIdentity, result = std::move(result)]() mutable {
return [this, previousAddressCount, previousWalletIdentity, previousZ, result = std::move(result)]() mutable {
const bool addrListOk = result.addressListOk; // capture before the move
const double scanMs = result.scanMs; // feed the adaptive throttle (addressRefreshDue)
if (scanMs > 0.0) last_address_scan_ms_ = scanMs;
@@ -2105,16 +2160,44 @@ void App::refreshAddressData()
const std::size_t currentAddressCount = state_.z_addresses.size() + state_.t_addresses.size();
const bool addressSetChanged = currentAddressCount != previousAddressCount ||
transactionHistoryCacheWalletIdentity() != previousWalletIdentity;
if (state_.transactions.empty() || addressSetChanged) {
if (addressSetChanged) {
invalidateShieldedHistoryScanProgress(false);
if (addrListOk) {
// Drop removed addresses' scan heights; the others stay valid (a changed set used to wipe
// every height, so one new address re-scanned the whole wallet). A z-address we never
// scanned that wasn't listed before (a key imported from the Console, say) may bring
// older sends with it, which a windowed sweep can't reach: sweep the whole wallet once.
pruneShieldedHistoryScanProgress();
// New: in neither the list at dispatch (GUI-created addresses are added to it before the
// refresh, and have no history) nor the last good listing (a blank list in between proves
// nothing), and never scanned.
bool newUnscannedZ = false;
std::unordered_set<std::string> listedZ;
for (const auto& address : state_.z_addresses) {
if (address.address.empty()) continue;
listedZ.insert(address.address);
if (!previousZ.count(address.address) && !history_listed_z_.count(address.address) &&
!shielded_history_scan_heights_.count(address.address))
newUnscannedZ = true;
}
transactions_dirty_ = true;
if (newUnscannedZ && (!previousZ.empty() || !history_listed_z_.empty())) {
history_sent_scan_height_ = -1;
history_full_sweep_failures_ = 0;
history_full_sweep_retry_at_ = 0;
++history_sent_epoch_;
}
history_listed_z_ = std::move(listedZ);
}
if (state_.transactions.empty() || addressSetChanged) {
markTransactionsDirty();
last_tx_block_height_ = -1;
network_refresh_.markDue(services::NetworkRefreshService::Timer::Transactions);
}
if (state_.transactions.empty()) loadTransactionHistoryCacheIfAvailable();
else storeTransactionHistoryCacheIfAvailable();
else if (addrListOk) storeTransactionHistoryCacheIfAvailable();
// Hints compare the notes with the history on screen: only once it's there (the cache loaded
// just above, or a pass is done). Before that every note looks missing, and the first pass scans
// every address anyway.
if (addrListOk && (!state_.transactions.empty() || initial_history_scan_complete_))
noteHistoryHintsFromUnspent(unspentNotes);
maybeFinishTransactionSendProgress();
// Addresses (hence identity + count) are now known — refresh the wallet index's cached
// metadata for the active wallet. Throttled: upsert() only writes on a real change.
@@ -2130,46 +2213,161 @@ void App::refreshAddressData()
void App::refreshTransactionData()
{
if (!worker_ || !rpc_ || !state_.connected) return;
if (addresses_dirty_) {
refreshAddressData();
network_refresh_.markDue(services::NetworkRefreshService::Timer::Transactions);
// The one entry point for history refreshes (the periodic timer, connect, tab entry, the address
// callback's re-route), so every caller gets the same rules. Wallet-wide scans hold the daemon's
// cs_main: never while it can't take them.
if (!worker_ || !rpc_ || !rpc_->isConnected() || !state_.connected) return;
if (state_.isLocked() || state_.warming_up || runtime_rescan_active_ || rpc_busy_gate_->busy()) return;
const int tip = state_.sync.blocks;
const auto work = historyScanWork();
services::TxCycleInputs in;
in.neverFetched = last_tx_block_height_ < 0;
in.newTip = tip != last_tx_block_height_;
in.dirty = transactions_dirty_;
in.sendTracking = !send_txids_.empty();
in.explicitScanWork = work.explicitWork;
in.routineScanWork = work.routineWork;
in.gateOpen = txRefreshDue();
in.syncing = effectivelySyncing();
in.heightKnown = tip > 0;
in.recentDue = currentPageNeedsWalletDataRefresh() && shouldRefreshRecentTransactions();
const auto cycle = services::decideTxCycle(in);
switch (cycle) {
case services::TxCycle::None:
return;
case services::TxCycle::Recent:
case services::TxCycle::RecentWithProbe:
startRecentTransactionPoll(cycle == services::TxCycle::RecentWithProbe);
return;
case services::TxCycle::Full:
case services::TxCycle::Light:
if (addresses_dirty_) {
// The history needs the current address list; the address callback marks it due again.
refreshAddressData();
network_refresh_.markDue(services::NetworkRefreshService::Timer::Transactions);
return;
}
startTransactionCycle(cycle == services::TxCycle::Full, in.gateOpen);
return;
}
}
void App::startTransactionCycle(bool full, bool routineAllowed)
{
const int currentBlocks = state_.sync.blocks;
if (last_tx_block_height_ < 0 || currentBlocks != last_tx_block_height_ ||
!shielded_history_scan_pending_) {
shielded_history_scan_cursor_ = 0;
shielded_history_scan_pending_ = false;
}
auto transactionSnapshot = NetworkRefreshService::buildTransactionRefreshSnapshot(
state_, viewtx_cache_, send_txids_);
transactionSnapshot.pendingOpids.insert(pending_opids_.begin(), pending_opids_.end());
if (settings_) transactionSnapshot.miningAddresses = settings_->getMiningAddresses();
transactionSnapshot.shieldedScanHeights = shielded_history_scan_heights_;
transactionSnapshot.shieldedScanStartIndex = shielded_history_scan_cursor_;
transactionSnapshot.maxShieldedReceiveScans = shieldedReceiveScanBudget(current_page_);
transactionSnapshot.shieldedScanTipTolerance =
shieldedScanTipTolerance(transactionSnapshot.shieldedAddresses.size());
// When fully synced, widen the re-scan tolerance so the O(mapWallet) per-address history scan runs
// every several blocks instead of ~every 2 — the heaviest residual synced-state cs_main cost. The
// balance poll still surfaces incoming funds on its own cadence; only the detailed history list lags a
// few minutes. (During catch-up the tx refresh is suppressed by kSyncProfile, so this is synced-only.)
if (!effectivelySyncing())
transactionSnapshot.shieldedScanTipTolerance =
std::max(transactionSnapshot.shieldedScanTipTolerance, 8);
transactionSnapshot.transparentRowHint = history_transparent_rows_;
transactionSnapshot.reuseTransparentRows = !transactions_dirty_ && currentBlocks > 0 &&
history_transparent_tip_ == currentBlocks && !state_.transactions.empty();
transactionSnapshot.enrichBacklogTxids = history_enrich_backlog_;
if (full) {
const auto& addresses = transactionSnapshot.shieldedAddresses;
const bool wasActive = history_pass_.active();
history_pass_.begin(currentBlocks, addresses, shielded_history_scan_heights_,
services::historyStaleAfterBlocks(addresses.size(), history_addr_scan_ms_),
initial_history_scan_complete_, routineAllowed);
if (!wasActive && history_pass_.active()) shielded_history_scan_cursor_ = 0; // a new pass: from the top
transactionSnapshot.shieldedScanFloorHeight = history_pass_.scanFloor(currentBlocks, routineAllowed);
transactionSnapshot.shieldedScanStartIndex = shielded_history_scan_cursor_;
transactionSnapshot.maxShieldedReceiveScans = shieldedReceiveScanBudget(current_page_);
auto listed = [&](const std::string& address) {
return std::find(addresses.begin(), addresses.end(), address) != addresses.end();
};
// Never-scanned addresses first, whatever the cursor (the explicit work: a new or imported
// address costs one scan, not a pass).
for (const auto& address : addresses) {
if (!shielded_history_scan_heights_.count(address)) transactionSnapshot.priorityScanFloors[address] = currentBlocks;
}
if (routineAllowed) {
// Addresses holding notes the history doesn't show yet (noteHistoryHintsFromUnspent).
for (const auto& [address, floor] : history_hint_floors_) {
if (listed(address)) transactionSnapshot.priorityScanFloors.emplace(address, floor);
}
// The chat reply address once per new tip, so messages also surface off the Chat tab.
if (!lite_wallet_ && chat::hushChatFeatureEnabledAtBuild() && chat_service_.hasIdentity() && settings_) {
const std::string chatZ = settings_->getChatReplyZaddr();
if (!chatZ.empty() && listed(chatZ)) transactionSnapshot.priorityScanFloors[chatZ] = currentBlocks;
}
}
transactionSnapshot.sentScanFromHeight = history_sent_scan_height_;
// A windowed sweep mid-pass too, now and then, so an untracked send (the Console, another wallet on
// the same seed) doesn't wait for a whole pass on a large wallet: at most every quarter of the pass
// cadence, and only in cycles the throttle allows.
const int staleAfter = services::historyStaleAfterBlocks(addresses.size(), history_addr_scan_ms_);
transactionSnapshot.allowMidPassSentSweep =
routineAllowed && history_sent_scan_height_ >= 0 && history_sent_scan_height_ < currentBlocks &&
currentBlocks - history_sent_scan_height_ >= std::max(services::kHistoryMinStaleBlocks, staleAfter / 4);
// The sweep runs once per tip, when the throttle allows (or the first load is still running),
// and backs off after whole-wallet sweeps that failed.
transactionSnapshot.skipSentSweep =
(!routineAllowed && initial_history_scan_complete_) ||
(history_sent_scan_height_ >= 0 && history_sent_scan_height_ == currentBlocks) ||
(history_sent_scan_height_ < 0 && std::time(nullptr) < history_full_sweep_retry_at_);
} else {
// Light: rows now (a send, a mutation), no wallet-wide z-scans.
transactionSnapshot.skipShieldedScans = true;
transactionSnapshot.skipSentSweep = true;
transactionSnapshot.shieldedScanStartIndex = shielded_history_scan_cursor_;
}
const std::uint64_t scanEpoch = history_scan_epoch_;
const std::uint64_t sentEpoch = history_sent_epoch_;
const std::uint64_t dirtyGen = tx_dirty_generation_;
const bool snapshotHadZ = !transactionSnapshot.shieldedAddresses.empty();
ui::NavPage tracePage = current_page_;
auto enqueued = network_refresh_.enqueue(services::NetworkRefreshService::Job::Transactions, *worker_, [this, currentBlocks,
transactionSnapshot = std::move(transactionSnapshot), tracePage]() -> rpc::RPCWorker::MainCb {
AppRefreshRpcGateway refreshRpc(*rpc_, traceSource(tracePage, "Transaction refresh"));
auto enqueued = network_refresh_.enqueue(services::NetworkRefreshService::Job::Transactions, *worker_, [this, currentBlocks, full,
transactionSnapshot = std::move(transactionSnapshot), tracePage, scanEpoch, sentEpoch, dirtyGen,
snapshotHadZ]() -> rpc::RPCWorker::MainCb {
AppRefreshRpcGateway refreshRpc(*rpc_, traceSource(tracePage, full ? "Transaction refresh" : "Transaction refresh (light)"),
this);
auto result = NetworkRefreshService::collectTransactionRefreshResult(
refreshRpc, transactionSnapshot, currentBlocks, MAX_VIEWTX_PER_CYCLE);
return [this, result = std::move(result)]() mutable {
bool shieldedScanComplete = result.shieldedScanComplete;
if (result.scanMs > 0.0) last_tx_scan_ms_ = result.scanMs; // feed the adaptive throttle (txRefreshDue)
return [this, full, scanEpoch, sentEpoch, dirtyGen, snapshotHadZ, result = std::move(result)]() mutable {
const std::time_t now = std::time(nullptr);
const int resultTip = result.blockHeight;
const std::size_t attempted = result.shieldedAddressesScanned;
const std::size_t succeeded = result.shieldedScansSucceeded;
const bool sameScanEpoch = scanEpoch == history_scan_epoch_;
// Only history_sent_epoch_ invalidates the sent side (watermark, backoff, backlog): a plain
// refresh keeps the sent scan, so it doesn't throw away an in-flight whole-wallet sweep.
const bool sameSentEpoch = sentEpoch == history_sent_epoch_;
// Feed the throttle (txRefreshDue): Full cycles are the wallet-wide scans it meters. Not the
// whole-wallet sweep, which has its own cadence (once per connect or full invalidation, with a
// backoff): charging up to 120 s of it held routine scans back for 10x that after every connect.
if (full) {
double charged = result.scanMs;
if (result.sentSweepRan && result.sentSweepFull) charged = std::max(1.0, result.scanMs - result.sentSweepMs);
if (charged > 0.0) last_tx_scan_ms_ = charged;
last_tx_scan_at_ = now;
}
if (sameScanEpoch) {
if (succeeded > 0) {
history_addr_scan_ms_ = services::updateScanCostEstimateMs(
history_addr_scan_ms_, result.receiveScanOkMs / static_cast<double>(succeeded));
}
if (result.transparentChainOk) {
history_transparent_rows_ = result.transparentRowCount;
history_transparent_tip_ = resultTip;
}
}
if (sameSentEpoch) history_enrich_backlog_ = std::move(result.unenrichedTxids);
if (sameSentEpoch) {
if (result.sentSweepOk && result.sentSweepHeight > 0) {
history_sent_scan_height_ = result.sentSweepHeight;
history_full_sweep_failures_ = 0;
history_full_sweep_retry_at_ = 0;
} else if (result.sentSweepRan && result.sentSweepFull &&
services::fullSentSweepFailureCounts(result.sentSweepErrKind)) {
++history_full_sweep_failures_;
history_full_sweep_retry_at_ = now + services::fullSentSweepRetryDelaySec(history_full_sweep_failures_);
}
}
std::size_t nextShieldedScanStartIndex = result.nextShieldedScanStartIndex;
auto shieldedScanHeights = std::move(result.shieldedScanHeights);
NetworkRefreshService::TransactionCacheUpdate cacheUpdate{
@@ -2207,12 +2405,32 @@ void App::refreshTransactionData()
NetworkRefreshService::applyTransactionRefreshResult(
state_, cacheUpdate, std::move(result), std::time(nullptr));
pruneTrackedSendTxids();
shielded_history_scan_heights_ = std::move(shieldedScanHeights);
// Pass bookkeeping on the main thread, from the merged heights. Not from a cycle whose address
// list came back blank (enumeration failed): it scanned nothing and knows nothing.
if (full && sameScanEpoch && (snapshotHadZ || shielded_history_scan_heights_.empty())) {
shielded_history_scan_heights_ = std::move(shieldedScanHeights);
shielded_history_scan_cursor_ = nextShieldedScanStartIndex;
const auto addresses = shieldedAddressList();
// A list blanked by a failed enumeration after dispatch proves nothing: don't end the pass
// or call the first load complete from it.
if (!(snapshotHadZ && addresses.empty())) {
history_pass_.finish(resultTip, addresses, shielded_history_scan_heights_, attempted, succeeded);
for (auto it = history_hint_floors_.begin(); it != history_hint_floors_.end();) {
const auto height = shielded_history_scan_heights_.find(it->first);
if (height != shielded_history_scan_heights_.end() && height->second >= it->second)
it = history_hint_floors_.erase(it);
else
++it;
}
const auto work = historyScanWork();
if (work.total > 0 && work.missing == 0 &&
!(history_pass_.active() && history_pass_.explicitPass()))
initial_history_scan_complete_ = true;
}
}
storeTransactionHistoryCacheIfAvailable();
shielded_history_scan_cursor_ = nextShieldedScanStartIndex;
shielded_history_scan_pending_ = !shieldedScanComplete;
if (shieldedScanComplete) initial_history_scan_complete_ = true;
transactions_dirty_ = !shieldedScanComplete;
// A mutation that landed while this cycle ran isn't covered by it: stay dirty for it.
if (tx_dirty_generation_ == dirtyGen) transactions_dirty_ = false;
maybeFinishTransactionSendProgress();
};
}, 3);
@@ -2245,34 +2463,44 @@ void App::pruneTrackedSendTxids()
}
}
void App::refreshRecentTransactionData()
void App::startRecentTransactionPoll(bool probe)
{
if (!worker_ || !rpc_ || !state_.connected) return;
if (!shouldRefreshRecentTransactions()) return;
const int currentBlocks = state_.sync.blocks;
auto transactionSnapshot = NetworkRefreshService::buildTransactionRefreshSnapshot(
state_, viewtx_cache_, send_txids_);
transactionSnapshot.pendingOpids.insert(pending_opids_.begin(), pending_opids_.end());
if (settings_) transactionSnapshot.miningAddresses = settings_->getMiningAddresses();
transactionSnapshot.shieldedScanHeights = shielded_history_scan_heights_;
transactionSnapshot.shieldedScanStartIndex = shielded_history_scan_cursor_;
// Between blocks: recent listtransactions rows, plus (when the scan throttle allows) one receive scan
// of the address scanned longest ago. It no longer advances the pass cursor.
transactionSnapshot.skipShieldedScans = !probe;
transactionSnapshot.maxShieldedReceiveScans = 1;
transactionSnapshot.shieldedScanTipTolerance =
shieldedScanTipTolerance(transactionSnapshot.shieldedAddresses.size());
if (!effectivelySyncing()) // synced: widen the re-scan tolerance (see refreshTransactionData)
transactionSnapshot.shieldedScanTipTolerance =
std::max(transactionSnapshot.shieldedScanTipTolerance, 8);
transactionSnapshot.shieldedScanStartIndex = probe
? services::oldestScannedAddressIndex(transactionSnapshot.shieldedAddresses, shielded_history_scan_heights_)
: 0;
const std::uint64_t scanEpoch = history_scan_epoch_;
ui::NavPage tracePage = current_page_;
auto enqueued = network_refresh_.enqueue(services::NetworkRefreshService::Job::Transactions, *worker_, [this, currentBlocks,
transactionSnapshot = std::move(transactionSnapshot), tracePage]() -> rpc::RPCWorker::MainCb {
transactionSnapshot = std::move(transactionSnapshot), tracePage, probe, scanEpoch]() -> rpc::RPCWorker::MainCb {
AppRefreshRpcGateway refreshRpc(*rpc_, traceSource(tracePage, "Recent transaction poll"));
auto result = NetworkRefreshService::collectRecentTransactionRefreshResult(
refreshRpc, transactionSnapshot, currentBlocks);
return [this, result = std::move(result)]() mutable {
std::size_t nextShieldedScanStartIndex = result.nextShieldedScanStartIndex;
return [this, probe, scanEpoch, result = std::move(result)]() mutable {
// The poll doesn't consume the tip: newTip stays set until a Full or Light cycle applies at it,
// so the Full cycle still runs once the throttle allows. (Read here, not at dispatch, so a reset
// to -1 meanwhile is kept.)
int pollTip = last_tx_block_height_;
if (probe && result.shieldedAddressesScanned > 0) {
// The probe is a wallet-wide scan: it counts for the throttle.
if (result.scanMs > 0.0) last_tx_scan_ms_ = result.scanMs;
last_tx_scan_at_ = std::time(nullptr);
if (scanEpoch == history_scan_epoch_ && result.shieldedScansSucceeded > 0) {
history_addr_scan_ms_ = services::updateScanCostEstimateMs(
history_addr_scan_ms_, result.receiveScanOkMs / static_cast<double>(result.shieldedScansSucceeded));
}
}
auto shieldedScanHeights = std::move(result.shieldedScanHeights);
NetworkRefreshService::TransactionCacheUpdate cacheUpdate{
viewtx_cache_,
@@ -2280,7 +2508,7 @@ void App::refreshRecentTransactionData()
confirmed_tx_cache_,
confirmed_tx_ids_,
confirmed_cache_block_,
last_tx_block_height_
pollTip
};
// HushChat: decrypt & thread any harvested chat memos BEFORE `result` is moved (see the
// full-refresh path above for rationale; the store dedups across both paths).
@@ -2307,8 +2535,7 @@ void App::refreshRecentTransactionData()
}
NetworkRefreshService::applyTransactionRefreshResult(
state_, cacheUpdate, std::move(result), std::time(nullptr));
shielded_history_scan_heights_ = std::move(shieldedScanHeights);
shielded_history_scan_cursor_ = nextShieldedScanStartIndex;
if (scanEpoch == history_scan_epoch_) shielded_history_scan_heights_ = std::move(shieldedScanHeights);
storeTransactionHistoryCacheIfAvailable();
maybeFinishTransactionSendProgress();
};
@@ -2341,7 +2568,7 @@ bool App::refreshEncryptionState()
return [this, result]() {
NetworkRefreshService::applyWalletEncryptionResult(state_, result);
if (state_.isLocked()) {
resetTransactionHistoryCacheSession();
lockTransactionHistoryCacheSession();
} else if (state_.transactions.empty()) {
loadTransactionHistoryCacheIfAvailable();
} else {
@@ -5855,7 +6082,7 @@ void App::submitZSendMany(const std::string& from, const std::string& to, double
// A send changes address balances — refresh on next cycle
addresses_dirty_ = true;
// Force transaction list refresh so the sent tx appears immediately
transactions_dirty_ = true;
markTransactionsDirty();
last_tx_block_height_ = -1;
requestForcedBalanceRefresh(); // the user's own send must update the balance now, past the throttle
network_refresh_.markWalletMutationRefresh();
@@ -6030,7 +6257,7 @@ void App::runtimeRescan(int startHeight)
user_initiated_rescan_ = true; // user clicked Rescan → completion should toast
state_.sync.rescan_progress = 0.0f;
state_.sync.rescan_status = "Rescanning from block " + std::to_string(startHeight) + "...";
transactions_dirty_ = true;
markTransactionsDirty();
last_tx_block_height_ = -1;
invalidateShieldedHistoryScanProgress(true);
@@ -6052,7 +6279,7 @@ void App::runtimeRescan(int startHeight)
resetWitnessRescanProgress();
if (ok) {
state_.sync.rescan_progress = 1.0f;
transactions_dirty_ = true;
markTransactionsDirty();
last_tx_block_height_ = -1;
invalidateShieldedHistoryScanProgress(true);
ui::Notifications::instance().success("Blockchain rescan complete");