DragonX/ObsidianDragon
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 10 Wiki Activity

fix(sync): prioritize getblockchaininfo and pause chat scans while behind

Browse Source 
The node kept falling behind the network near the tip because sync
DETECTION was starved: getblockchaininfo was queued behind the
O(mapWallet) wallet RPCs (z_gettotalbalance / z_listunspent), so
longestchain went stale, the wallet decided it was "synced", and it
resumed hammering cs_main — a feedback loop.

- Issue getblockchaininfo FIRST each cycle and skip the balance/address/
  tx refresh entirely while behind, so sync state (and kSyncProfile)
  updates before any heavy wallet scan runs.
- Gate the two chat note scans (refreshChatNoteBudgetNode /
  fastScanChatMemos) on effectivelySyncing() and the active page, so chat
  memo scanning no longer competes with block connection during sync.
- Windows debug.log tailer: reset the read offset when dragonxd truncates
  the log on startup (it was stranding at Block:0 with no witness/rescan
  progress).
- Tests cover the getblockchaininfo-first ordering and behind-skip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
DanS
2026-09-01 19:36:30 -05:00
co-authored by Claude Opus 4.8
parent 56d93b6128
commit 4492aa3425
5 changed files with 231 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/app_network.cpp
+122 -12
View File
@@ -882,13 +882,32 @@ bool App::effectivelySyncing() const
// cheap wallet (sub-cadence cost) is unaffected — the tab's Core timer stays the real cadence; a ~20s
// scan on a large wallet backs off to roughly every ~200s instead of every 2s, freeing cs_main for
// block connection. A wallet mutation bypasses this via force_balance_refresh_.
bool App::scanRefreshDue(std::int64_t lastUpdate, double lastScanMs) const
{
constexpr double kScanDutyCycle = 0.10;
if (lastUpdate == 0) return true; // never fetched
if (lastScanMs <= 0.0) return true; // no cost measured yet
const double minInterval = (lastScanMs / 1000.0) / kScanDutyCycle;
return std::difftime(std::time(nullptr), static_cast<std::time_t>(lastUpdate)) >= minInterval;
}
bool App::balanceRefreshDue() const
{
constexpr double kBalanceDutyCycle = 0.10;
if (state_.last_balance_update == 0) return true; // never fetched
if (last_balance_scan_ms_ <= 0.0) return true; // no cost measured yet
const double minInterval = (last_balance_scan_ms_ / 1000.0) / kBalanceDutyCycle;
return std::difftime(std::time(nullptr), state_.last_balance_update) >= minInterval;
return scanRefreshDue(state_.last_balance_update, last_balance_scan_ms_);
}
// The address scan (z_listunspent) and history scan (z_listreceivedbyaddress) are just as O(mapWallet)
// and cs_main-bound as the balance scan, so they get the same adaptive back-off. This is what keeps a
// large synced wallet from re-scanning them every tab cadence — the near-tip starvation the balance-only
// throttle missed (only the periodic poll is gated; explicit refreshes call the refresh directly).
bool App::addressRefreshDue() const
{
return scanRefreshDue(state_.last_address_update, last_address_scan_ms_);
}
bool App::txRefreshDue() const
{
return scanRefreshDue(state_.last_tx_update, last_tx_scan_ms_);
}
bool App::currentPageNeedsWalletDataRefresh() const
@@ -1739,11 +1758,19 @@ void App::refreshCoreData()
// cost (0 when balance was skipped), and arm the settle window only on the
// syncing→caught-up edge so a wallet synced from the start is never throttled at connect.
if (result.balanceScanMs > 0.0) last_balance_scan_ms_ = result.balanceScanMs;
// Sticky-behind hysteresis: hold the low-impact profile for kSyncSettleSeconds after the
// LAST time we read "behind" — not only on the caught-up edge. While actually syncing we
// re-arm every Core tick, so the profile stays engaged continuously; once we truly reach
// the tip (within the 2-block tolerance in applyCoreRefreshResult) we stop re-arming and it
// lapses after the window. This is what breaks the near-tip feedback loop: a single stale or
// optimistic "caught up" reading can no longer immediately unleash the heavy O(mapWallet)
// scans and re-starve the tail (which dropped us behind again → oscillation). With the
// scans suppressed, kSyncProfile keeps sync detection itself cheap, so we keep getting fresh
// "behind" readings and stay latched until genuinely caught up. Window must exceed the
// worst-case Core-refresh interval so it can't lapse between two behind readings mid-catchup.
const bool nowSyncing = state_.sync.syncing;
if (nowSyncing) {
sync_settle_until_ = 0;
} else if (was_core_syncing_) {
constexpr double kSyncSettleSeconds = 8.0;
constexpr double kSyncSettleSeconds = 30.0;
if (nowSyncing || was_core_syncing_) {
sync_settle_until_ = std::time(nullptr) + static_cast<std::time_t>(kSyncSettleSeconds);
}
was_core_syncing_ = nowSyncing;
@@ -1872,7 +1899,13 @@ void App::refreshAddressData()
return [this, previousAddressCount, previousWalletIdentity, result = std::move(result)]() mutable {
const bool addrListOk = result.addressListOk; // capture before the move
const double scanMs = result.scanMs; // feed the adaptive throttle (addressRefreshDue)
if (scanMs > 0.0) last_address_scan_ms_ = scanMs;
// #3: this scan already listed the wallet's unspent notes — feed the chat send-budget from it
// (dedups the dedicated chat z_listunspent). Grab it before `result` is moved below.
auto unspentNotes = std::move(result.unspentNotes);
NetworkRefreshService::applyAddressRefreshResult(state_, std::move(result));
updateChatNoteBudgetFromUnspent(unspentNotes);
// Mark the address list as loaded ONLY if enumeration actually succeeded — a swallowed
// z_listaddresses/getaddressesbyaccount failure returns a falsely-short list, and stamping it
// would let the empty-wallet warning trust a spurious 0 count (see maybeWarnEmptyWallet…).
@@ -1931,6 +1964,13 @@ void App::refreshTransactionData()
transactionSnapshot.maxShieldedReceiveScans = shieldedReceiveScanBudget(current_page_);
transactionSnapshot.shieldedScanTipTolerance =
shieldedScanTipTolerance(transactionSnapshot.shieldedAddresses.size());
// When fully synced, widen the re-scan tolerance so the O(mapWallet) per-address history scan runs
// every several blocks instead of ~every 2 — the heaviest residual synced-state cs_main cost. The
// balance poll still surfaces incoming funds on its own cadence; only the detailed history list lags a
// few minutes. (During catch-up the tx refresh is suppressed by kSyncProfile, so this is synced-only.)
if (!effectivelySyncing())
transactionSnapshot.shieldedScanTipTolerance =
std::max(transactionSnapshot.shieldedScanTipTolerance, 8);
ui::NavPage tracePage = current_page_;
auto enqueued = network_refresh_.enqueue(services::NetworkRefreshService::Job::Transactions, *worker_, [this, currentBlocks,
@@ -1941,6 +1981,7 @@ void App::refreshTransactionData()
return [this, result = std::move(result)]() mutable {
bool shieldedScanComplete = result.shieldedScanComplete;
if (result.scanMs > 0.0) last_tx_scan_ms_ = result.scanMs; // feed the adaptive throttle (txRefreshDue)
std::size_t nextShieldedScanStartIndex = result.nextShieldedScanStartIndex;
auto shieldedScanHeights = std::move(result.shieldedScanHeights);
NetworkRefreshService::TransactionCacheUpdate cacheUpdate{
@@ -2006,6 +2047,9 @@ void App::refreshRecentTransactionData()
transactionSnapshot.maxShieldedReceiveScans = 1;
transactionSnapshot.shieldedScanTipTolerance =
shieldedScanTipTolerance(transactionSnapshot.shieldedAddresses.size());
if (!effectivelySyncing()) // synced: widen the re-scan tolerance (see refreshTransactionData)
transactionSnapshot.shieldedScanTipTolerance =
std::max(transactionSnapshot.shieldedScanTipTolerance, 8);
ui::NavPage tracePage = current_page_;
auto enqueued = network_refresh_.enqueue(services::NetworkRefreshService::Job::Transactions, *worker_, [this, currentBlocks,
@@ -3535,9 +3579,25 @@ void App::refreshChatNoteBudget(const wallet::LiteWalletAppRefreshModel& model)
void App::refreshChatNoteBudgetNode() {
if (lite_wallet_ || !chat::hushChatFeatureEnabledAtBuild() || !chat_service_.hasIdentity()) return;
if (!state_.connected || !rpc_ || !worker_ || state_.isLocked()) return;
// Never contend the daemon's cs_main while the node is catching up. z_listunspent is O(mapWallet)
// and holds cs_main for its whole duration — seconds on a large wallet — so an ungated per-few-second
// chat scan here starves block connection and keeps the node from ever pinning to the tip. Chat
// send-readiness can wait until we're synced; this budget is only advisory (the send path re-checks).
if (effectivelySyncing()) return;
// Only pre-warm the send-budget while the user is actually in chat. Off the Chat tab this heavy
// z_listunspent is unnecessary — and on a wallet-data tab, refreshAddressData's own z_listunspent
// already feeds the budget for free (updateChatNoteBudgetFromUnspent), so this dedicated scan is a
// fallback for the Chat tab (where the address refresh doesn't run).
if (current_page_ != ui::NavPage::Chat) return;
if (chat_note_scan_in_flight_) return; // one scan at a time
const double now = ImGui::GetTime();
if (now - chat_note_scan_last_ < 4.0) return; // rate limit; self-throttles via the in-flight flag too
// Adaptive back-off keyed on the last scan's own cost: on a large wallet where the scan takes many
// seconds, this stretches the interval so the scan can't occupy more than ~10% of wall-clock (same
// duty-cycle discipline as the balance/address/tx polls). A cheap wallet stays at the 4s floor.
double minInterval = 4.0;
if (chat_note_scan_ms_ > 0.0)
minInterval = std::max(4.0, (chat_note_scan_ms_ / 1000.0) / 0.10);
if (now - chat_note_scan_last_ < minInterval) return;
chat_note_scan_last_ = now;
chat_note_scan_in_flight_ = true;
@@ -3548,6 +3608,7 @@ void App::refreshChatNoteBudgetNode() {
int verified = 0, pipeline = 0;
std::uint64_t verifiedZat = 0;
bool ok = false;
const auto scanStart = std::chrono::steady_clock::now();
try {
rpc::RPCClient::TraceScope trace("HushChat / note-buffer scan");
nlohmann::json notes = rpc_->call("z_listunspent", nlohmann::json::array({0})); // 0 = include maturing
@@ -3567,9 +3628,12 @@ void App::refreshChatNoteBudgetNode() {
}
}
} catch (const std::exception&) {}
return [this, scanGen, ok, verified, pipeline, verifiedZat]() {
const double scanMs = std::chrono::duration<double, std::milli>(
std::chrono::steady_clock::now() - scanStart).count();
return [this, scanGen, ok, verified, pipeline, verifiedZat, scanMs]() {
if (scanGen != chat_session_generation_) return; // wallet switched — drop (reset cleared the flag)
chat_note_scan_in_flight_ = false;
if (scanMs > 0.0) chat_note_scan_ms_ = scanMs; // feed the adaptive back-off
if (!ok) return; // z_listunspent unavailable — leave caches as-is
chat_note_model_seen_ = true;
chat_verified_note_budget_ = verified;
@@ -3584,6 +3648,36 @@ void App::refreshChatNoteBudgetNode() {
});
}
// #3: derive the chat send-budget from an already-collected z_listunspent (refreshAddressData's scan),
// so we don't issue a duplicate z_listunspent. Runs on the main thread; the notes are pre-parsed to the
// fields we need (amount / locked / TRUE confirmations), so there's no RPC and no worker hop. Mirrors the
// apply in refreshChatNoteBudgetNode and stamps chat_note_scan_last_ so the dedicated scan backs off.
void App::updateChatNoteBudgetFromUnspent(
const std::vector<services::NetworkRefreshService::UnspentNoteLite>& unspentNotes) {
if (lite_wallet_ || !chat::hushChatFeatureEnabledAtBuild() || !chat_service_.hasIdentity()) return;
const int confsNeeded = chatConfsRequired();
const std::uint64_t minVal = chatDrgxToZat(kChatMinFeeDrgx);
int verified = 0, pipeline = 0;
std::uint64_t verifiedZat = 0;
for (const auto& nz : unspentNotes) {
if (nz.locked) continue; // tied up by an in-flight send
const std::uint64_t amtZat = chatDrgxToZat(nz.amount);
if (amtZat < minVal) continue; // skip dust below a fee
++pipeline; // unspent self-note (verified or maturing)
if (nz.confirmations >= confsNeeded) { ++verified; verifiedZat += amtZat; }
}
chat_note_model_seen_ = true;
chat_verified_note_budget_ = verified;
chat_pipeline_note_count_ = pipeline;
chat_verified_shielded_zat_ = verifiedZat;
chat_note_scan_last_ = ImGui::GetTime(); // counts as a fresh scan → dedicated scan backs off
if (chat_split_outstanding_ &&
(chat_pipeline_note_count_ >= kChatRefillTrigger ||
ImGui::GetTime() - chat_split_submitted_at_ > kChatSplitWatchdogSecs)) {
chat_split_outstanding_ = false;
}
}
void App::enqueueChatSend(LiteOpKind kind, const chat::OutgoingChatMemos& memos, const std::string& echoLocalId) {
QueuedChatOp op;
op.kind = kind;
@@ -3825,15 +3919,28 @@ void App::fastScanChatMemos()
if (!chat::hushChatFeatureEnabledAtBuild() || !chat_service_.hasIdentity()) return;
if (!state_.connected || !rpc_ || !worker_) return;
if (chat_fast_scan_in_flight_) return; // don't stack RPCs if a previous scan is still running
// z_listreceivedbyaddress is O(mapWallet) (the daemon iterates every wtx even when filtered to one
// address) — seconds on a large wallet — so this 0-conf poll must NOT run while the node is catching
// up, and even when synced it backs off to ~10% wall-clock keyed on its own cost. The block-tip
// harvest still ingests every message on confirmation; this only trades away mempool-speed delivery.
if (effectivelySyncing()) return;
// Mempool-speed 0-conf delivery is only worth its heavy cost while the user is watching chat. Off the
// Chat tab, incoming messages still arrive via the confirmed block-tip harvest (+ toast) within ~1 block.
if (current_page_ != ui::NavPage::Chat) return;
const double nowT = ImGui::GetTime();
if (chat_fast_scan_ms_ > 0.0 &&
nowT - chat_fast_scan_last_ < (chat_fast_scan_ms_ / 1000.0) / 0.10) return;
const std::string addr = chatReplyZaddr();
if (addr.empty()) return;
chat_fast_scan_last_ = nowT;
chat_fast_scan_in_flight_ = true;
const int scanGen = chat_session_generation_; // guard: drop the result if the wallet switches/locks
worker_->post([this, addr, scanGen]() -> rpc::RPCWorker::MainCb {
std::vector<chat::HushChatTransactionMetadata> metadata;
int rawMemoCount = 0; // received notes carrying a memo at the reply addr (0-conf visibility signal)
std::string scanError;
const auto scanStart = std::chrono::steady_clock::now();
try {
rpc::RPCClient::TraceScope trace("HushChat / 0-conf fast scan");
nlohmann::json received = rpc_->call("z_listreceivedbyaddress", {addr, 0}); // 0 = include mempool
@@ -3864,12 +3971,15 @@ void App::fastScanChatMemos()
}
} catch (const std::exception& e) { scanError = e.what(); }
const int parsedCount = static_cast<int>(metadata.size());
return [this, scanGen, metadata = std::move(metadata), rawMemoCount, parsedCount, scanError]() mutable {
const double scanMs = std::chrono::duration<double, std::milli>(
std::chrono::steady_clock::now() - scanStart).count();
return [this, scanGen, metadata = std::move(metadata), rawMemoCount, parsedCount, scanError, scanMs]() mutable {
// The wallet was switched/locked between post and now — this metadata belongs to the previous
// session. resetChatSession already reset the in-flight flag, so just drop; clearing it here
// would clobber a new session's own in-flight scan (mirrors the broadcast/identity guards).
if (scanGen != chat_session_generation_) return;
chat_fast_scan_in_flight_ = false;
if (scanMs > 0.0) chat_fast_scan_ms_ = scanMs; // feed the adaptive back-off
// Diagnostic (console App/chat channel): log when the memo-note count at the reply address
// CHANGES, so a stable mempool doesn't spam every 2.5s. Reveals whether inbound 0-conf messages
// are reaching us at all, and how many parse as chat vs are unrelated memos.