DragonX/ObsidianDragon
1
1
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases 10 Wiki Activity

feat(node): in-app "Rebuild wallet database" recovery for a BDB-inconsistent wallet

Browse Source 
Automates the manual recovery that fixed a wallet.dat with stale Berkeley DB
extent metadata (the "main" subdb metapage records a low last_pgno while its live
data spans thousands of pages beyond it). A tolerant page-walk reads every record,
but the daemon's BDB verify rejects the file and auto-salvages it — finding nothing
and shrinking the wallet to empty on each restart (the salvage cascade that looks
like fund loss). Plain "Restore original" can't fix it (hands the same broken file
back → re-salvage); a rebuild must produce a fresh, consistent DB.

Pieces (Approach A from the design workflow — out-of-process helper keeps AGPL
Berkeley DB out of the GPLv3 GUI):
- util/wallet_file_probe.h: extractWalletBtreeRecords() — sibling to parseWalletBtree
  that collects raw (key,value) bytes (same bounds-checked, subdb-aware walk).
  Records copied verbatim → encrypted key material passes through as opaque
  ciphertext (no passphrase). Overflow-page values (only large tx history) are
  skipped + counted; a rescan rebuilds history — funds unaffected.
- tools/wallet_rebuild/main.cpp: dragonx-wallet-rebuild CLI — reads via the tolerant
  reader, writes the records into a fresh BDB "main" btree via libdb (DB_EXCL, never
  overwrites), prints a JSON summary. New BDB-guarded CMake target.
- App::rebuildWalletDatabase(): picks the largest readable wallet/.bak as source,
  stops the daemon, runs the helper, VERIFIES the output (readable BDB with keys)
  before swapping, moves the current wallet aside (kept, timestamped), installs the
  rebuilt one, clears the stale BDB env, sets -rescan, restarts. Copy/rename only —
  never deletes. Result surfaced via the existing pumpWalletRestore channel.
- Wired as the preferred action on the existing wallet-auto-recovery dialog
  (shown only when the helper is present). Full-node only; lite-safe.

Verified end-to-end against the real broken wallet: helper reads 3,808 t-keys + 1
z-key + HD seed and the daemon LOADS the rebuilt output with no salvage. Adds
extractWalletBtreeRecords coverage. Build clean, suite green (1/1).

Remaining (follow-up): release packaging — build.sh bundling the helper built
against the vendored per-platform static libdb (DRAGONX_BDB_ROOT), and a macOS
Berkeley DB port (no in-tree artifact).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
DanS
2026-08-10 11:26:55 -05:00
parent b2e037bb67
commit bc183257b7
8 changed files with 449 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
src/util/i18n.cpp
View File

@@ -1211,6 +1211,14 @@ void I18n::loadBuiltinEnglish()
strings_["wallet_restore_move_failed"] = "Couldn't set the current wallet aside — nothing was changed.";
strings_["wallet_restore_copy_failed"] = "Couldn't install the backup wallet; your current wallet was left in place.";
strings_["wallet_restore_no_restart"] = "Your original wallet was restored, but the node didn't restart — start it from Settings.";
// One-click "Rebuild wallet database" flow (fixes a BDB-inconsistent wallet that keeps getting salvaged).
strings_["wallet_recovered_rebuild"] = "Rebuild wallet database (recommended)";
strings_["wallet_rebuild_started"] = "Rebuilding your wallet database and restarting the node…";
strings_["wallet_rebuild_ok"] = "Wallet database rebuilt — the node is loading it and rescanning for your balance.";
strings_["wallet_rebuild_no_helper"] = "The wallet-rebuild helper isn't available in this build. Use Restore, or rebuild manually.";
strings_["wallet_rebuild_no_source"] = "Couldn't find a readable wallet to rebuild. Nothing was changed.";
strings_["wallet_rebuild_failed"] = "The rebuild didn't produce a valid wallet, so nothing was changed. Your wallet is untouched.";
strings_["wallet_rebuild_install_failed"] = "Couldn't install the rebuilt wallet; your current wallet was left in place.";
// Receive Tab
strings_["receiving_addresses"] = "Your Receiving Addresses";

150
src/util/wallet_file_probe.h
View File

@@ -339,5 +339,155 @@ inline WalletBtreeStats parseWalletBtree(const std::string& path,
return st;
}
// ─────────────────────────────────────────────────────────────────────────────────────────────────
// Tier 3: collect the raw (key,value) record BYTES — the read half of the offline wallet REBUILD that
// recovers a BDB-inconsistent wallet.dat (stale extent metadata: our tolerant walk reads records the
// daemon's Berkeley DB verify rejects and auto-salvages). A helper then writes these verbatim into a
// fresh, consistent BDB so the daemon loads it cleanly. Records are copied byte-for-byte — encrypted
// key material (ckey/csapzkey/mkey) passes through as opaque ciphertext, so no passphrase is needed.
// Values that live in BDB OVERFLOW pages (only large `tx` history records) are NOT captured (skipped +
// counted); they are irrelevant to funds — a rescan rebuilds transaction history. Same bounds-checked,
// subdb-aware, visited-set-capped walk as parseWalletBtree.
struct WalletRawRecords {
bool parsed = false; ///< the btree walked cleanly
bool complete = false; ///< the whole file was read (not cap-truncated)
std::vector<std::pair<std::string, std::string>> records; ///< inline (key,value) bytes, verbatim
int keyRecords = 0; ///< fund-critical key-type records captured (key/wkey/ckey/z*/sap*/hdseed)
int skippedOverflow = 0; ///< records whose value spilled to overflow pages (tx history) — not captured
std::size_t bytesRead = 0;
};
inline WalletRawRecords extractWalletBtreeRecords(const std::string& path,
std::size_t maxBytes = 512u * 1024u * 1024u) {
WalletRawRecords out;
std::ifstream f(path, std::ios::binary);
if (!f) return out;
std::string buf;
{
f.seekg(0, std::ios::end);
std::streamoff sz = f.tellg();
if (sz < 512) return out;
const std::size_t want = std::min<std::size_t>(static_cast<std::size_t>(sz), maxBytes);
f.seekg(0, std::ios::beg);
buf.resize(want);
f.read(&buf[0], static_cast<std::streamsize>(want));
buf.resize(static_cast<std::size_t>(std::max<std::streamsize>(0, f.gcount())));
if (buf.size() < 512) return out;
out.bytesRead = buf.size();
out.complete = (buf.size() == static_cast<std::size_t>(sz));
}
const unsigned char* B = reinterpret_cast<const unsigned char*>(buf.data());
const std::size_t N = buf.size();
auto rd32at = [&](std::size_t o, bool le) -> uint32_t {
return le ? (uint32_t)B[o] | ((uint32_t)B[o+1]<<8) | ((uint32_t)B[o+2]<<16) | ((uint32_t)B[o+3]<<24)
: (uint32_t)B[o+3] | ((uint32_t)B[o+2]<<8) | ((uint32_t)B[o+1]<<16) | ((uint32_t)B[o]<<24);
};
constexpr uint32_t kBtreeMagic = 0x00053162u;
bool le;
if (rd32at(12, true) == kBtreeMagic) le = true;
else if (rd32at(12, false) == kBtreeMagic) le = false;
else return out;
auto r32 = [&](std::size_t o) { return o + 4 <= N ? rd32at(o, le) : 0u; };
auto r16 = [&](std::size_t o) -> uint32_t {
if (o + 2 > N) return 0;
return le ? (uint32_t)B[o] | ((uint32_t)B[o+1]<<8) : (uint32_t)B[o+1] | ((uint32_t)B[o]<<8);
};
const uint32_t pagesize = r32(20);
if (pagesize < 512 || pagesize > 65536 || (pagesize & (pagesize - 1)) != 0) return out;
const uint32_t npages = static_cast<uint32_t>(N / pagesize);
const uint32_t root = r32(88);
if (npages == 0 || root == 0 || root >= npages) return out;
if (B[24] != 0 || (B[26] & 0x01)) return out; // page checksum/encryption — offsets shift; bail
constexpr uint8_t P_IBTREE = 3, P_LBTREE = 5, P_BTREEMETA = 9, B_KEYDATA = 1;
constexpr std::size_t kMaxPagesVisited = 600000;
constexpr int kMaxKeys = 4000000;
std::vector<bool> visited(npages, false);
std::size_t pagesVisited = 0;
int keys = 0;
bool aborted = false;
auto rdpgno = [&](const unsigned char* p) -> uint32_t {
return le ? (uint32_t)p[0] | ((uint32_t)p[1]<<8) | ((uint32_t)p[2]<<16) | ((uint32_t)p[3]<<24)
: (uint32_t)p[3] | ((uint32_t)p[2]<<8) | ((uint32_t)p[1]<<16) | ((uint32_t)p[0]<<24);
};
auto traverse = [&](uint32_t rootPg, auto&& fn) {
std::fill(visited.begin(), visited.end(), false);
std::vector<uint32_t> stack;
if (rootPg < npages && !visited[rootPg]) { visited[rootPg] = true; stack.push_back(rootPg); }
while (!stack.empty()) {
const uint32_t pg = stack.back(); stack.pop_back();
if (pg >= npages) continue;
if (++pagesVisited > kMaxPagesVisited) { aborted = true; return; }
const std::size_t base = static_cast<std::size_t>(pg) * pagesize;
if (base + 26 > N) continue;
const uint8_t type = B[base + 25];
const uint32_t entries = r16(base + 20);
if (26 + static_cast<std::size_t>(entries) * 2 > pagesize) continue;
if (type == P_IBTREE) {
for (uint32_t i = 0; i < entries; ++i) {
const uint32_t off = r16(base + 26 + i * 2);
if (off + 8 > pagesize) continue;
const uint32_t child = r32(base + off + 4);
if (child > 0 && child < npages && !visited[child]) { visited[child] = true; stack.push_back(child); }
}
} else if (type == P_LBTREE) {
for (uint32_t i = 0; i + 1 < entries; i += 2) {
if (++keys > kMaxKeys) { aborted = true; return; }
const uint32_t ko = r16(base + 26 + i * 2);
const uint32_t dO = r16(base + 26 + (i + 1) * 2);
if (ko + 3 > pagesize || dO + 3 > pagesize) continue;
if (B[base + ko + 2] != B_KEYDATA) continue; // overflow/dup key — never a record name
const uint32_t kl = r16(base + ko);
if (kl < 1 || ko + 3 + kl > pagesize) continue;
const uint8_t dtype = B[base + dO + 2];
const uint32_t dl = r16(base + dO);
const unsigned char* dp = (dO + 3 + dl <= pagesize) ? B + base + dO + 3 : nullptr;
fn(B + base + ko + 3, kl, dp, dl, dtype);
}
}
}
};
// Master DB: subdb-name → subdb meta/root pgno (big-endian value; native fallback), same as parseWalletBtree.
std::vector<uint32_t> subRoots;
traverse(root, [&](const unsigned char*, uint32_t, const unsigned char* dp, uint32_t dl, uint8_t dtype) {
if (dtype != B_KEYDATA || dl != 4 || !dp) return;
const uint32_t cand[2] = {
(uint32_t)dp[3] | ((uint32_t)dp[2]<<8) | ((uint32_t)dp[1]<<16) | ((uint32_t)dp[0]<<24),
rdpgno(dp),
};
for (const uint32_t pgno : cand) {
if (pgno == 0 || pgno >= npages) continue;
const uint8_t pt = B[static_cast<std::size_t>(pgno) * pagesize + 25];
if (pt == P_BTREEMETA) {
const uint32_t sr = r32(static_cast<std::size_t>(pgno) * pagesize + 88);
if (sr > 0 && sr < npages) { subRoots.push_back(sr); break; }
} else if (pt == P_LBTREE || pt == P_IBTREE) { subRoots.push_back(pgno); break; }
}
});
if (aborted) return out;
if (subRoots.empty()) subRoots.push_back(root);
auto isKeyType = [](const char* nm, uint32_t nl) {
auto is = [&](const char* s) { return std::strlen(s) == nl && std::memcmp(nm, s, nl) == 0; };
return is("key") || is("wkey") || is("ckey") || is("zkey") || is("czkey")
|| is("sapzkey") || is("csapzkey") || is("hdseed") || is("chdseed");
};
for (const uint32_t sr : subRoots) {
traverse(sr, [&](const unsigned char* kp, uint32_t kl, const unsigned char* dp, uint32_t dl, uint8_t dt) {
if (dt != B_KEYDATA || !dp) { out.skippedOverflow++; return; } // overflow value (tx history) — skip
out.records.emplace_back(std::string(reinterpret_cast<const char*>(kp), kl),
std::string(reinterpret_cast<const char*>(dp), dl));
const uint32_t nlen = kp[0];
if (nlen >= 2 && nlen <= 20 && 1u + nlen <= kl && isKeyType(reinterpret_cast<const char*>(kp + 1), nlen))
out.keyRecords++;
});
if (aborted) return out;
}
out.parsed = true;
return out;
}
} // namespace util
} // namespace dragonx