DragonX/ObsidianDragon
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 10 Wiki Activity

feat(merge): run per-address consolidation as a paced background job

Browse Source 
"Keep in each address" posted one task that called z_mergetoaddress for
every batch of every address back to back; each call is a whole-wallet
scan under cs_main, and the daemon runs the ops one at a time anyway.
It is now an App-owned services::ConsolidationRun:

- one RPC at a time; the next merge only after the previous batch left the
  queue and a gap as long as the last call took (2-120 s);
- pauses while the node syncs, the wallet is locked, a send of ours is in
  progress, or another wallet scan is running;
- failures classified: nothing that may have queued is ever resent; a
  breaker stops after 3 dead addresses or 3 failed ops in a row;
- each merge is charged to the scan budget; one re-read and one forced
  balance at the end;
- it keeps running when the dialog closes: a status-bar chip, a run view
  with progress, the wait reason and a Stop button.

The "Into one" merge can no longer be confirmed twice while its call is
in flight. fake_dragonxd gains FAKE_SCAN_SECS, FAKE_MERGE_SERIAL and
FAKE_MERGE_FAIL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DanS
2026-10-09 11:06:34 -05:00
co-authored by Claude Opus 5.5
parent b29a09eaa0
commit cee8b9d7f6
19 changed files with 1875 additions and 202 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/fake_dragonxd/README.md
+3
View File
@@ -122,6 +122,9 @@ inherits the GUI's environment.
| `FAKE_STUCK_SHUTDOWN` | 0 | the Sep 29 orphan: hold the datadir lock, no listener, ignore SIGTERM, never exit |
| `FAKE_WALLET_NOTES` | (off) | a note wallet: `addr:count:amount,...` shielded notes; enables `z_mergetoaddress` + its operations (dragonxd rules: confirmed unlocked notes, a batch locked on return, merged note next block) |
| `FAKE_MERGE_SECS` | 6 | how long a `z_mergetoaddress` operation runs before it succeeds |
| `FAKE_MERGE_SERIAL` | 0 | `1` = ops run one at a time, like dragonxd's single async worker: a new op is `queued` until the previous one finishes |
| `FAKE_MERGE_FAIL` | (off) | `syncing:N` (the Nth and later merges fail with the THROW_IF_SYNCING text), `lockedkey:N` (-8 "Could not find spending key…" after the scan), `opfail:N` (the Nth and later ops end `failed`, their notes unlocked) |
| `FAKE_SCAN_SECS` | 0 | the GUI's wallet scans (`z_listunspent`, `listunspent`, `z_gettotalbalance`, `z_listreceivedbyaddress`, `z_mergetoaddress`) hold cs_main this long; `scan-begin`/`scan-end` events |
| `FAKE_TIP_UNKNOWN_SECS` | 0 | `longestchain` reads 0 this long after `Done Loading` (a node that hasn't seen an UpdateTip yet) |
| `FAKE_LONGEST_AHEAD` / `FAKE_CATCHUP_SECS` | 0 / 0 | the network tip (`longestchain`, `headers`) is N blocks ahead, for that long after `Done Loading` (0 = for good) |
| `FAKE_TXCOUNT` | 42 | `getwalletinfo` `txcount` (the GUI's connect gate waits for the tip on wallets of 500+) |
tools/fake_dragonxd/fake_dragonxd.cpp
+62 -3
View File
@@ -424,8 +424,10 @@ struct FakeOp {
std::vector<long> noteIds;
std::string to;
double fee = 0.0;
double startAt = 0.0; // FAKE_MERGE_SERIAL: queued until the previous op finished
double finishAt = 0.0; // unixMs()/1000.0
std::string status = "executing";
std::string error; // a failed op's message (FAKE_MERGE_FAIL=opfail:N)
long long created = 0;
};
bool g_noteWallet = false;
@@ -434,6 +436,12 @@ std::vector<FakeOp> g_ops;
long g_noteSeq = 0;
long g_opSeq = 0;
double g_mergeSecs = 6.0;
// Consolidation-run knobs (C210 manual checks).
double g_scanSecs = 0.0; // FAKE_SCAN_SECS: a wallet scan holds cs_main this long
bool g_mergeSerial = false; // FAKE_MERGE_SERIAL=1: one op at a time, like dragonxd's single async worker
std::string g_mergeFailKind; // FAKE_MERGE_FAIL=syncing:N | lockedkey:N | opfail:N
long g_mergeFailFrom = 0;
long g_mergeCalls = 0;
void loadNoteWallet()
{
@@ -441,6 +449,13 @@ void loadNoteWallet()
if (!spec || !*spec) return;
g_noteWallet = true;
g_mergeSecs = envDouble("FAKE_MERGE_SECS", 6.0);
g_mergeSerial = envLong("FAKE_MERGE_SERIAL", 0) == 1;
if (const char* f = std::getenv("FAKE_MERGE_FAIL")) {
const std::string spec(f);
const size_t colon = spec.find(':');
g_mergeFailKind = spec.substr(0, colon);
g_mergeFailFrom = colon == std::string::npos ? 1 : std::max(1L, std::strtol(spec.c_str() + colon + 1, nullptr, 10));
}
std::string s(spec);
size_t pos = 0;
while (pos <= s.size()) {
@@ -472,7 +487,15 @@ void settleOps()
{
const double now = unixMs() / 1000.0;
for (auto& op : g_ops) {
if (op.status == "queued" && now >= op.startAt) op.status = "executing";
if (op.status != "executing" || now < op.finishAt) continue;
if (!op.error.empty()) { // FAKE_MERGE_FAIL=opfail: the op fails and its notes are unlocked again
for (auto& n : g_notes)
if (std::find(op.noteIds.begin(), op.noteIds.end(), n.id) != op.noteIds.end()) n.locked = false;
op.status = "failed";
event("merge-failed", op.id);
continue;
}
double sum = 0.0;
for (long id : op.noteIds) {
auto it = std::find_if(g_notes.begin(), g_notes.end(), [&](const FakeNote& n) { return n.id == id; });
@@ -488,6 +511,7 @@ json opStatus(const FakeOp& op)
{
json o = {{"id", op.id}, {"status", op.status}, {"creation_time", op.created}, {"method", "z_mergetoaddress"}};
if (op.status == "success") o["result"] = {{"txid", fakeHash(1000000 + std::stol(op.id.substr(5)))}};
if (op.status == "failed") o["error"] = {{"code", -6}, {"message", op.error}};
return o;
}
@@ -503,7 +527,7 @@ Canned noteWalletReply(const std::string& m, const json& params)
const long conf = std::max(0L, h - n.height + 1);
arr.push_back({{"txid", fakeHash(5000000 + n.id)}, {"outindex", 0}, {"confirmations", conf},
{"spendable", true}, {"address", n.address}, {"amount", n.amount}, {"memo", "f6"},
{"change", false}});
{"change", false}, {"locked", n.locked}});
}
c.result = arr;
} else if (m == "z_listaddresses") {
@@ -535,6 +559,24 @@ Canned noteWalletReply(const std::string& m, const json& params)
else from.push_back(a.get<std::string>());
}
const std::string to = params[1].get<std::string>();
++g_mergeCalls;
const bool failNow = !g_mergeFailKind.empty() && g_mergeCalls >= g_mergeFailFrom;
if (failNow && g_mergeFailKind == "syncing") { // THROW_IF_SYNCING runs before the scan (cheap)
c.errCode = -1;
c.errMsg = "z_mergetoaddress: Extreme Privacy! Chain still syncing at height " + std::to_string(h) +
", aborting to prevent linkability analysis. Please wait until FULLY SYNCED and try again.";
return c;
}
if (g_scanSecs > 0.0) { // GetFilteredNotes: a whole-wallet scan under cs_main
event("scan-begin", "m=z_mergetoaddress");
std::this_thread::sleep_for(std::chrono::duration<double>(g_scanSecs));
event("scan-end", "m=z_mergetoaddress");
}
if (failNow && g_mergeFailKind == "lockedkey") {
c.errCode = -8;
c.errMsg = "Could not find spending key for payment address.";
return c;
}
const double fee = params.size() > 2 && params[2].is_number() ? params[2].get<double>() : 0.0001;
long limit = params.size() > 4 && params[4].is_number() ? params[4].get<long>() : 90;
if (limit <= 0) limit = 1000000;
@@ -565,7 +607,16 @@ Canned noteWalletReply(const std::string& m, const json& params)
op.to = to;
op.fee = fee;
op.created = unixMs() / 1000;
op.finishAt = unixMs() / 1000.0 + g_mergeSecs;
const double nowSec = unixMs() / 1000.0;
double prevFinish = nowSec;
if (g_mergeSerial) {
for (const auto& o : g_ops)
if (o.status == "queued" || o.status == "executing") prevFinish = std::max(prevFinish, o.finishAt);
}
op.startAt = prevFinish;
op.finishAt = op.startAt + g_mergeSecs;
op.status = op.startAt > nowSec ? "queued" : "executing";
if (failNow && g_mergeFailKind == "opfail") op.error = "Proof failed (FAKE_MERGE_FAIL)";
g_ops.push_back(op);
event("merge", op.id + " notes=" + std::to_string(picked.size()) + " to=" + to);
c.result = {{"remainingUTXOs", 0}, {"remainingTransparentValue", 0.0},
@@ -581,7 +632,7 @@ Canned noteWalletReply(const std::string& m, const json& params)
std::vector<FakeOp> keep;
for (const auto& op : g_ops) {
const bool match = filter.empty() || std::find(filter.begin(), filter.end(), op.id) != filter.end();
const bool finished = op.status != "executing";
const bool finished = op.status == "success" || op.status == "failed";
if (match && (m == "z_getoperationstatus" || finished)) arr.push_back(opStatus(op));
if (!(m == "z_getoperationresult" && match && finished)) keep.push_back(op);
}
@@ -792,6 +843,13 @@ json execOne(const json& req, int& status)
std::lock_guard<std::mutex> lk(g_csMain);
c = cannedReply(m, params);
if (m == "setgenerate") g_minerOn = !params.empty() && params[0].is_boolean() && params[0].get<bool>();
// FAKE_SCAN_SECS: the GUI's wallet-wide scans hold cs_main (z_mergetoaddress sleeps in its own handler).
if (g_scanSecs > 0.0 && (m == "z_listunspent" || m == "listunspent" || m == "z_gettotalbalance" ||
m == "z_listreceivedbyaddress")) {
event("scan-begin", "m=" + m);
std::this_thread::sleep_for(std::chrono::duration<double>(g_scanSecs));
event("scan-end", "m=" + m);
}
if (g_noteWallet) {
Canned w = noteWalletReply(m, params);
if (w.known) c = w;
@@ -1363,6 +1421,7 @@ void loadConfig(int argc, char** argv)
g_cfg.tipUnknownSecs = envDouble("FAKE_TIP_UNKNOWN_SECS", g_cfg.tipUnknownSecs);
g_cfg.longestAhead = std::max(0L, envLong("FAKE_LONGEST_AHEAD", g_cfg.longestAhead));
g_cfg.catchupSecs = envDouble("FAKE_CATCHUP_SECS", g_cfg.catchupSecs);
g_scanSecs = envDouble("FAKE_SCAN_SECS", 0.0);
g_cfg.txcount = envLong("FAKE_TXCOUNT", g_cfg.txcount);
g_cfg.height = envLong("FAKE_HEIGHT", g_cfg.height);
g_cfg.rebuildTotal = std::max(1L, envLong("FAKE_REBUILD_TOTAL", g_cfg.rebuildTotal));