DragonX/ObsidianDragon
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 10 Wiki Activity

perf(chat): sends re-read by origin, chat scans under the scan budget

Browse Source 
A z_sendmany submit set off a burst of wallet scans before anything had
changed. Ops now re-read the wallet when they finish, by who started them:
the user's sends and dialog ops get a re-read plus one forced balance once
none of their ops is pending; client auto-shield re-reads unforced; chat
messages and splits refresh only the chat note view. A pending send no
longer forces scans, and chat txids no longer enter the tracked-send set.

Chat ingest skips stored messages before decrypting and caches
undecryptable ones per identity. The inbox scan runs when getwalletinfo's
txcount changed (a cheap probe, at most 2% duty) or a cost-scaled safety
net is due; the note-buffer scan runs on need. Both are admitted by the
scan budget, on a steady clock. The chat payer comes from a per-address
view of verified notes, with debits re-applied across in-flight scans;
splits are sized to one payer and only when they can refill the buffer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DanS
2026-10-09 11:06:34 -05:00
co-authored by Claude Opus 5.5
parent f86cc4d6b7
commit b29a09eaa0
15 changed files with 1037 additions and 177 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/app.h
+66 -19
View File
@@ -30,6 +30,7 @@
#include "services/connect_scan_gate.h"
#include "services/scan_policy.h"
#include "services/wallet_scan_governor.h"
#include "services/chat_scan_policy.h"
#include "services/network_refresh_service.h"
#include "services/wallet_security_controller.h"
#include "services/wallet_security_workflow.h"
@@ -436,8 +437,10 @@ public:
// auto-shield) with the shared opid poller so its eventual success/failure is
// surfaced and balances/transactions refresh on completion. z_sendmany uses the
// richer pending-send path internally; this is for operations with no optimistic
// transaction row of their own.
void trackOperation(const std::string& opid);
// transaction row of their own. The origin decides what its completion re-reads: the client
// auto-shield is Auto (re-read, balance unforced); everything the user starts is User.
void trackOperation(const std::string& opid,
services::NetworkRefreshService::OperationOrigin origin = services::NetworkRefreshService::OperationOrigin::User);
// The connected daemon rejected the opid-filtered z_getoperationstatus, so polls use the unfiltered
// form (every retained op) for the rest of this connection.
bool opidFilterUnsupported() const { return opid_filter_unsupported_.load(std::memory_order_relaxed); }
@@ -935,12 +938,38 @@ private:
// the pipeline recovers (outputs mined) or a watchdog expires (a split that never mines mustn't wedge
// refill forever). Prevents runaway splitting that would drain balance into fees.
bool chat_split_outstanding_ = false;
double chat_split_submitted_at_ = 0.0; // ImGui time the outstanding split was submitted (watchdog)
double chat_split_submitted_at_ = 0.0; // scanClockNow() when the outstanding split was submitted (watchdog)
// Full-node only: a coordinator-owned z_listunspent worker scan feeds the per-note counts (the shared
// balance poll discards per-note data). Mirrors chat_fast_scan_in_flight_.
// balance poll discards per-note data). Set with its scan ticket; cleared only by its reply.
bool chat_note_scan_in_flight_ = false;
double chat_note_scan_last_ = 0.0; // ImGui time of the last note scan (rate limit)
double chat_note_scan_ms_ = 0.0; // measured cost of the last note scan (adaptive back-off)
double chat_note_scan_last_ = 0.0; // scanClockNow() of the last note-scan dispatch or address-scan feed
std::int64_t chat_note_scan_tip_ = -1; // the settled tip the note view was read at (-1: look again)
bool chat_note_rescan_requested_ = false; // a chat op finished: its notes changed
bool chat_note_first_look_sent_ = false; // the session's one explicit look (Chat page, no model)
services::ChatPayerMap chat_verified_funds_by_addr_; // verified notes by address (the chat payer view)
std::int64_t chat_split_hold_tip_ = -1; // a split failed at this tip: none before the next block
// Chat spends the payer view's producers (a note scan, an address pass) posted earlier can't see: worker_ is
// FIFO, so a reply reflects every spend submitted before it was posted, none after. Re-applied on its reply.
struct ChatSpendLogEntry {
std::uint64_t seq = 0;
std::string payer;
services::ChatPayerSpend spend = services::ChatPayerSpend::OneNote;
};
std::uint64_t chat_spend_seq_ = 0;
std::deque<ChatSpendLogEntry> chat_spend_log_;
std::int64_t chat_note_scan_spend_seq_ = -1; // the spend seq an outstanding note scan was posted at
std::int64_t addr_pass_spend_seq_ = -1; // ...and an outstanding address pass
void debitChatPayerLogged(const std::string& payer, services::ChatPayerSpend spend);
void reapplyChatSpendsAfter(std::uint64_t seq);
// The 0-conf inbox: a txcount probe per chat tick, the inbox scan when the wallet gained a tx.
std::int64_t chat_inbox_basis_txcount_ = -1; // txcount the last successful inbox scan is current for
double chat_inbox_last_scan_at_ = 0.0; // scanClockNow() at that scan's dispatch (safety net)
bool chat_inbox_scan_wanted_ = false; // the probe found a need the budget hasn't admitted yet
std::int64_t chat_inbox_wanted_txcount_ = -1; // the count that need was found at: the next basis
double chat_inbox_probe_at_ = 0.0; // scanClockNow() of the last probe
double chat_inbox_probe_ms_ = 0.0; // its cost (self-throttle)
bool chatInboxScanAllowed() const; // the inbox's gates (identity, the hard gates, the Chat page)
void admitChatInboxScan(); // ask the scan budget; post the inbox scan when admitted
// Most-recent chain tip, cached across refreshes: a lite refresh model that carries spendableOutputs
// may NOT carry sync status that same cycle (tolerated partial refresh), so verifiedSelfNoteCount reads
// this cache rather than requiring the current model to have both — else the budget flickers to 0.
@@ -949,14 +978,21 @@ private:
// Coordinator helpers (both variants unless noted; see app_network.cpp).
void refreshChatNoteBudget(const wallet::LiteWalletAppRefreshModel& model); // lite: recompute caches on a fresh model
void refreshChatNoteBudgetNode(); // full node: rate-limited z_listunspent worker scan
// Full node: the z_listunspent note-buffer scan, on need (services::shouldScanChatNotes), under the scan
// budget (ChatNotes); also off the Chat page while a queued send or a split waits on it.
void refreshChatNoteBudgetNode();
// Feeds the SAME chat send-budget from an already-collected z_listunspent (the address refresh),
// so the dedicated scan above is skipped while the address refresh is active (dedup — see #3).
void updateChatNoteBudgetFromUnspent(const std::vector<services::NetworkRefreshService::UnspentNoteLite>& unspentNotes);
// feedTip: the tip that pass may vouch for (-1 when the tip moved too recently for the wallet notifier).
// spendSeq: the chat spend seq the pass was posted at (its reply re-applies the spends after it).
void updateChatNoteBudgetFromUnspent(const std::vector<services::NetworkRefreshService::UnspentNoteLite>& unspentNotes,
std::int64_t feedTip, std::uint64_t spendSeq);
void pumpChatNoteBuffer(); // per-frame: drain the queue / build the buffer
int verifiedSelfNoteCount(const wallet::LiteWalletAppRefreshModel& model); // lite
int pipelineSelfNoteCount(const wallet::LiteWalletAppRefreshModel& model); // lite
bool shouldSplitChatBuffer();
// What one split can spend: on the full node the richest verified payer (z_sendmany pays from ONE address).
std::uint64_t chatSplitAffordableZat() const;
bool broadcastSelfSplitLite(int noteCount); // lite: self-send minting noteCount reply-address notes
bool broadcastSelfSplitNode(int noteCount); // full node: z_sendmany self-send minting noteCount notes
void enqueueChatSend(LiteOpKind kind, const chat::OutgoingChatMemos& memos, const std::string& echoLocalId);
@@ -1010,23 +1046,23 @@ private:
void beginAdoptSeedWallet(); // stop daemon -> swap wallet.dat -> restart with -rescan
void provisionChatIdentityFromSecret(std::string secret);
std::string chatReplyZaddr(); // a stable (persisted) wallet z-addr — chat IDENTITY (reply-to)
// A spendable z-address that can actually PAY the fee (balance >= fee), preferring the identity
// reply address. The reply-to in the memo stays the identity address, so paying from a different
// funded note is transport-transparent. Empty if no z-address can cover the fee.
std::string chatPayFromZaddr(double fee) const;
// A spendable z-address that can actually PAY `need` (a fee, or a split's notes plus its fee), preferring
// the identity reply address; from the chat note view's verified notes when there is one. The reply-to in
// the memo stays the identity address, so paying from a different funded note is transport-transparent.
// Empty if no z-address can cover it.
std::string chatPayFromZaddr(double need) const;
std::string generateChatLocalId(const char* prefix, int numBytes) const; // unique echo id / cid
// Broadcast the memos and, when the async op resolves, flip the echo (echoLocalId) to Sent/Failed.
bool broadcastChatMemos(const chat::OutgoingChatMemos& memos, const std::string& echoLocalId); // true if submitted
bool broadcastChatMemosLite(const chat::OutgoingChatMemos& memos); // lite two-recipient send
void ingestLiteChatMemos(const wallet::LiteWalletAppRefreshModel& model); // lite chat receive harvest
// Full-node 0-conf fast path: re-scan just the chat reply address at minconf=0 every refresh cycle
// so incoming messages surface at mempool speed (before a block). Hidden conversations are skipped
// (they still un-hide via the normal confirmed harvest). Self-gated; no-op without a chat identity.
// Full-node 0-conf fast path on the Chat page: a txcount probe each chat tick, and the inbox scan (the chat
// reply address at minconf=0) when the wallet gained a tx, as the scan budget admits it (ChatFast), so
// incoming messages surface at mempool speed. Hidden conversations are skipped (they still un-hide via the
// normal confirmed harvest). Self-gated; no-op without a chat identity.
void fastScanChatMemos();
bool chat_fast_scan_in_flight_ = false; // guard against overlapping fast-scan RPCs
float chat_fast_scan_accum_ = 0.0f; // seconds since the last fast-scan (dedicated ~2.5s poll)
double chat_fast_scan_last_ = 0.0; // ImGui time of the last 0-conf fast scan (adaptive back-off)
double chat_fast_scan_ms_ = 0.0; // measured cost of the last fast scan (z_listreceivedbyaddress)
bool chat_fast_scan_in_flight_ = false; // the inbox probe or scan is out; cleared only by its reply
float chat_fast_scan_accum_ = 0.0f; // seconds since the last chat tick
bool font_rebuild_requested_ = false; // set by requestFontRebuild(); consumed in preFrame()
// Lite first-run welcome prompt: dismissed for the session once the user picks an action.
bool lite_firstrun_dismissed_ = false;
@@ -1377,6 +1413,11 @@ private:
// that actually sent the forced balance call, not at dispatch, so a dispatch the job queue refuses
// (a Core job already in flight) or one that skips the balance (node behind or busy) keeps it.
void requestForcedBalanceRefresh() { force_balance_refresh_ = true; ++force_balance_seq_; }
// A wallet mutation committed (a User/Auto op succeeded, or a send returned no opid): re-read what it changed.
void markWalletMutated(bool forceBalance);
bool hasPendingUserOperations() const {
return services::NetworkRefreshService::anyUserOperationPending(pending_opids_, op_origin_);
}
// One budget for every wallet-wide scan the GUI sends on its own (balance, addresses, history, the
// between-blocks probe): one in flight at a time across both lanes, ~15% of wall-clock in total. They
// used to be throttled one by one at 10% each, which added up to 30-40% with all of them due.
@@ -1385,6 +1426,7 @@ private:
// The change signals the routine scans are gated on, besides the tip hash (state_.sync.best_blockhash).
std::uint64_t wallet_mutation_epoch_ = 0; // bumped by noteWalletMutation()
bool mempool_known_ = false; // the last Core poll's getmempoolinfo
double tip_changed_at_ = 0.0; // scanClockNow() when a Core poll saw a new tip hash (chat settle)
std::int64_t mempool_txs_ = 0;
std::int64_t mempool_bytes_ = 0;
bool window_background_ = false; // setWindowBackground()
@@ -1556,6 +1598,7 @@ private:
std::size_t history_transparent_rows_ = 0; // rows the last listtransactions chain returned
int history_transparent_tip_ = -1; // ...and the tip it ran at (same tip: rows can be reused)
std::uint64_t tx_dirty_generation_ = 0; // bumped by markTransactionsDirty()
std::uint64_t addr_dirty_generation_ = 0; // bumped by markAddressesDirty()
// Addresses holding notes the history doesn't show yet (from the address scan's z_listunspent): scanned
// first in the next wallet-wide cycle, so a receive or payout shows within one cycle on a large wallet
// instead of when the routine pass reaches its address. Each txid hints once per session after it is
@@ -1626,6 +1669,8 @@ private:
bool send_progress_active_ = false;
int send_submissions_in_flight_ = 0;
std::vector<std::string> pending_opids_; // opids to poll for completion
// Non-User ops only (absent = User): what each op's completion re-reads.
std::unordered_map<std::string, services::NetworkRefreshService::OperationOrigin> op_origin_;
struct PendingSendInfo {
std::string from;
std::string to;
@@ -1814,6 +1859,8 @@ private:
void startTransactionCycle(bool full, bool routineAllowed, services::ScanTicket ticket); // Full or Light
void startRecentTransactionPoll(bool probe, services::ScanTicket ticket); // between blocks
void markTransactionsDirty() { transactions_dirty_ = true; ++tx_dirty_generation_; }
// A pass sent before this can't clear it (refreshAddressData compares the generation).
void markAddressesDirty() { addresses_dirty_ = true; ++addr_dirty_generation_; }
std::vector<std::string> shieldedAddressList() const;
services::ShieldedHistoryPass::Work historyScanWork() const;
// From the address scan's z_listunspent: addresses holding received notes the history doesn't show