perf(history): persist the sent-sweep watermark (windowed relaunch)
z_listsentbyaddress re-walks the whole wallet when there's no usable watermark, which the daemon does under cs_main for every send -- hundreds of seconds, 400+ s and ~19 GB RSS on a large wallet. The watermark lived only in memory, so every relaunch, reconnect or daemon restart paid for a full sweep. This persists it so those cases run a bounded windowed sweep ([z, 0, 2, tip - W + margin]) instead (SENT-HISTORY-SWEEP Part 1). - The watermark (W = last swept tip), its anchor (A = W - margin) and anchor hash (H_A = getblockhash(A)), plus the enrichment backlog and the full- sweep backoff, ride in the encrypted history snapshot as an optional `sent_sweep` object (schema stays v1; a missing/inconsistent one reads as "no watermark" = today's behaviour, so older builds and payloads are safe). - Before the first windowed sweep after a load, the anchor is verified (getblockhash(A) == H_A). A match means any fork is inside the window; a mismatch (reorg deeper than the margin, different chain/datadir) or an inability to check drops the watermark and sweeps the whole wallet. - The first sweep after a load uses a bigger margin (absorbs notifier lag), and a watermark older than 2400 blocks uses the long timeout (callLong). - The backlog is persisted WITH W: a pure z->z send's txid lives only in a sweep result and would be lost if the window moved past an un-enriched one. - Watermark changes reach disk (folded into the store signature), and every hard case still forces a full sweep: reorg (anchor verify), GUI rescan / zap / import / address change, restoreOriginalWallet, and disconnect. Deferred (follow-ups, each only turns a correct-but-slow case faster, never a correctness risk): the banded full sweep for seeding on a pre-note-cache daemon, the external-import tripwire, and the weekly safety-net re-sweep. Part 2 (the daemon's decrypt cost) is already on the daemon's perf/sapling-note-cache branch. Pure helpers + cache round-trip + anchor-verify/large-window covered by tests; the getblockhash-after-sweep updates the pinned RPC sequences. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -15,6 +15,24 @@ struct sqlite3;
|
||||
namespace dragonx {
|
||||
namespace data {
|
||||
|
||||
// Persisted z_listsentbyaddress watermark, so a relaunch runs a windowed sweep instead of a full one.
|
||||
// height (W) = tip of the last successful sweep; anchorHeight (A) = W - margin, the bottom edge the window
|
||||
// covered; anchorHash (H_A) = getblockhash(A) at that time, verified on the next load. backlog = txids found
|
||||
// but not yet enriched (must persist WITH W, or a pure z->z send whose txid lives only in a sweep result is
|
||||
// lost once the window moves past it). The backoff fields keep a relaunch from immediately re-running a full
|
||||
// sweep that keeps failing; daemonId resets the backoff on a version change. (Namespace-scope, not nested, so
|
||||
// it is a complete type for the default argument of replace().)
|
||||
struct SentSweepState {
|
||||
int height = -1;
|
||||
int anchorHeight = -1;
|
||||
std::string anchorHash;
|
||||
std::vector<std::string> backlog;
|
||||
int fullFailures = 0;
|
||||
long fullRetryDelaySec = 0;
|
||||
std::string daemonId;
|
||||
bool valid() const { return height >= 0; }
|
||||
};
|
||||
|
||||
class TransactionHistoryCache {
|
||||
public:
|
||||
struct LoadResult {
|
||||
@@ -25,6 +43,7 @@ public:
|
||||
std::time_t updatedAt = 0;
|
||||
std::vector<TransactionInfo> transactions;
|
||||
std::unordered_map<std::string, int> shieldedScanHeights;
|
||||
SentSweepState sentSweep;
|
||||
};
|
||||
|
||||
// Everything one snapshot write needs, BY VALUE, so it can run on the history writer thread after the UI
|
||||
@@ -38,6 +57,7 @@ public:
|
||||
std::vector<TransactionInfo> transactions;
|
||||
std::time_t updatedAt = 0;
|
||||
std::unordered_map<std::string, int> shieldedScanHeights;
|
||||
SentSweepState sentSweep;
|
||||
|
||||
SnapshotJob() = default;
|
||||
SnapshotJob(const SnapshotJob&) = delete;
|
||||
@@ -81,7 +101,8 @@ public:
|
||||
const std::string& tipHash,
|
||||
const std::vector<TransactionInfo>& transactions,
|
||||
std::time_t updatedAt,
|
||||
const std::unordered_map<std::string, int>& shieldedScanHeights = {});
|
||||
const std::unordered_map<std::string, int>& shieldedScanHeights = {},
|
||||
const SentSweepState& sentSweep = SentSweepState{});
|
||||
// Thread-safe snapshot write (the history writer thread; replace() wraps it): JSON + AEAD with no lock
|
||||
// held, then INSERT OR REPLACE + prune under the DB lock. Always wipes job.key. False for a keyless job.
|
||||
bool writeSnapshot(SnapshotJob&& job);
|
||||
|
||||
Reference in New Issue
Block a user