DragonX/ObsidianDragon
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 10 Wiki Activity

perf(startup): prepare first-run node files in the background

Browse Source 
On a first run or after an upgrade, Windows release builds wrote the
bundled Sapling params, asmap, node and miner (about 88 MB) in
App::init, before the first frame: 0.5-3 s of frozen blank window, 5-30 s
on a slow disk or under antivirus. Zip, AppImage and .app builds copied
the bundled files into the daemon dir and hashed the 51.5 MB of params on
the UI thread at the first node start, a 0.3-1.5 s freeze in the wizard
or the connect loop.

A stat-only check at startup decides whether anything is missing, stale
or unverified; a normal launch starts no thread and shows nothing.
Otherwise one background job extracts (Windows) or copies (Linux, macOS)
the files, then verifies the params once:

- The first frame paints at once. If the job takes longer than 0.4 s, a
  chip at the bottom shows "Preparing wallet files... N%", then
  "Verifying wallet files...".
- A node start on the UI thread while the job runs is deferred and
  replayed when it ends (never into a bootstrap download, a restart or a
  live connection); worker threads wait, and give up on quit. After an
  upgrade that ships a new miner, the node starts after the miner is
  written instead of before. Pool-miner starts and the miner version
  check wait too, so the old miner is never run or reported while it is
  being replaced.
- Each file goes to "<name>.partial", is flushed and renamed into place
  under one lock shared with the daemon start, the bundled-node
  reinstall and the miner extraction, and the job stops between 4 MB
  chunks at shutdown. An interrupted write used to leave a truncated
  binary that was trusted forever; a leftover .partial is now removed.

Other behaviour changes:

- Linux and macOS full-node builds copy the bundled node files (about
  77 MB) into the daemon dir at first launch, even if the wallet ends up
  on an external node; before, this happened at the first embedded start.
- Lite no longer writes the 53 MB of params and asmap it never reads.
- A disk-full or permission failure that leaves the node files unusable
  shows a warning; a miner blocked by antivirus alone does not.
- A corrupt or truncated param in a zip, AppImage or .app install is
  replaced from the bundle and verified again instead of trusted forever.
- Reinstalling the bundled node holds off the connect loop and a deferred
  start for its whole run (either could launch the old binary while the
  new one was written), and is refused while another restart is running.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DanS
2026-10-09 21:09:48 -05:00
co-authored by Claude Opus 5.5
parent 47180ff840
commit 971f2abae5
19 changed files with 1245 additions and 360 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/resources/embedded_resources.cpp
+436 -242
View File
@@ -1,10 +1,15 @@
#include "embedded_resources.h"
#include "../util/platform.h"
#include "../util/logger.h"
#include <algorithm>
#include <cerrno>
#include <fstream>
#include <filesystem>
#include <functional>
#include <mutex>
#include <vector>
#include <cstdio>
#include <cstring>
#include <cctype>
#include <chrono>
@@ -12,6 +17,7 @@
#include <windows.h>
#include <shlobj.h>
#else
#include <fcntl.h>
#include <unistd.h>
#include <pwd.h>
#include <sys/types.h>
@@ -205,272 +211,461 @@ std::string getParamsDirectory()
#endif
}
// Forward declaration — defined below extractResource()
static bool resourceNeedsUpdate(const EmbeddedResource* res, const std::string& destPath);
// ---------------------------------------------------------------------------------------------------
// Writing into the daemon directory: one lock, atomic (temp + flush + rename) files.
// ---------------------------------------------------------------------------------------------------
namespace {
bool needsParamsExtraction()
// Every writer of the daemon directory (startup job, getDaemonPath(), getXmrigPath(), forceExtractXmrig(),
// reextractBundledDaemon(), ensureWalletRebuildHelperExtracted(), copyBundledFiles()) holds this for its whole
// write; whoever comes second finds the files complete and does nothing. Never taken re-entrantly. Lock order:
// provisionMutex -> Logger mutex (DEBUG_LOGF) only. The in-app updaters do not take it (see residual risks).
std::mutex& provisionMutex() { static std::mutex m; return m; }
struct ProvisionEntry {
const char* name;
bool refreshOnSizeMismatch; // re-extract when the bundled size differs (params, asmap, xmrig)
bool executable;
bool fullNodeOnly; // not written by Lite builds (the lite backend bakes in its own params)
bool startupTrigger; // makes needsParamsExtraction() true
};
// The same files and rules as the old hand-written extraction: params/asmap/xmrig follow the bundled size;
// the node binaries and the rebuild helper are placed only when MISSING (the user may run a specific dragonxd;
// replacing it is the explicit Settings action). asmap/cli/tx/rebuild are written along, never a trigger.
const ProvisionEntry kProvisionEntries[] = {
{ RESOURCE_SAPLING_SPEND, true, false, true, true },
{ RESOURCE_SAPLING_OUTPUT, true, false, true, true },
{ RESOURCE_ASMAP, true, false, true, false },
#ifdef HAS_EMBEDDED_DAEMON
{ RESOURCE_DRAGONXD, false, true, true, true },
{ RESOURCE_DRAGONX_CLI, false, true, true, false },
{ RESOURCE_DRAGONX_TX, false, true, true, false },
#endif
#ifdef HAS_EMBEDDED_XMRIG
{ RESOURCE_XMRIG, true, true, false, true },
#endif
#ifdef HAS_EMBEDDED_WALLET_REBUILD
{ RESOURCE_DRAGONX_WALLET_REBUILD, false, true, true, false },
#endif
};
bool entryActive(const ProvisionEntry& e)
{
if (!hasEmbeddedResources()) {
#if defined(DRAGONX_LITE_BUILD) && DRAGONX_LITE_BUILD
return !e.fullNodeOnly;
#else
(void)e;
return true;
#endif
}
std::string joinDir(const std::string& dir, const char* name)
{
#ifdef _WIN32
return dir + "\\" + name;
#else
return dir + "/" + name;
#endif
}
constexpr std::size_t kCopyChunk = 4u * 1024u * 1024u;
bool prepareParentDir(const std::string& destPath)
{
const std::filesystem::path dest(destPath);
if (!dest.has_parent_path()) return true;
std::error_code ec;
std::filesystem::create_directories(dest.parent_path(), ec);
if (ec) {
DEBUG_LOGF("[ERROR] Failed to create directory %s: %s\n", dest.parent_path().string().c_str(),
ec.message().c_str());
return false;
}
// Check daemon directory (dragonx/) — the only extraction target
std::string daemonDir = getDaemonDirectory();
return true;
}
// Temp-file sink for writeFileAtomic / copyFileAtomic. Windows: ONE CreateFileA handle with no SECURITY_ATTRIBUTES,
// i.e. NOT inheritable: std::ofstream's CRT handle is, and a child created with bInheritHandles=TRUE meanwhile
// (xmrig, runHiddenCapture) would hold "<name>.partial" without FILE_SHARE_DELETE, so the MoveFileEx in
// replaceFromTemp fails while it runs (same fix as Platform::writeFileAtomically). commit() flushes on the same
// handle (FlushFileBuffers / fsync) before close, so the bytes are on disk before the rename. POSIX: O_CLOEXEC.
class TempSink {
public:
explicit TempSink(const std::string& path)
{
#ifdef _WIN32
const char pathSep = '\\';
h_ = CreateFileA(path.c_str(), GENERIC_WRITE, 0, nullptr, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, nullptr);
#else
const char pathSep = '/';
#endif
std::string spendPath = daemonDir + pathSep + RESOURCE_SAPLING_SPEND;
std::string outputPath = daemonDir + pathSep + RESOURCE_SAPLING_OUTPUT;
// Check if params are missing or stale (size mismatch → updated in newer build)
const auto* spendRes = getEmbeddedResource(RESOURCE_SAPLING_SPEND);
const auto* outputRes = getEmbeddedResource(RESOURCE_SAPLING_OUTPUT);
if (spendRes && resourceNeedsUpdate(spendRes, spendPath)) return true;
if (outputRes && resourceNeedsUpdate(outputRes, outputPath)) return true;
// Daemon binaries are only auto-placed when MISSING (never auto-overwritten on a size
// mismatch) — the user may be running a specific dragonxd. Replacing the bundled daemon is
// an explicit action via Settings → daemon binary. So only trigger extraction if it's absent.
#ifdef HAS_EMBEDDED_DAEMON
const auto* daemonRes = getEmbeddedResource(RESOURCE_DRAGONXD);
std::string daemonPath = daemonDir + pathSep + RESOURCE_DRAGONXD;
if (daemonRes && !std::filesystem::exists(daemonPath)) return true;
fd_ = ::open(path.c_str(), O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 0644);
#endif
}
~TempSink() { (void)close(); }
TempSink(const TempSink&) = delete;
TempSink& operator=(const TempSink&) = delete;
#ifdef HAS_EMBEDDED_XMRIG
const auto* xmrigRes = getEmbeddedResource(RESOURCE_XMRIG);
std::string xmrigPath = daemonDir + pathSep + RESOURCE_XMRIG;
if (xmrigRes && resourceNeedsUpdate(xmrigRes, xmrigPath)) return true;
bool ok() const
{
#ifdef _WIN32
return h_ != INVALID_HANDLE_VALUE;
#else
return fd_ >= 0;
#endif
}
bool write(const void* data, std::size_t n)
{
const char* p = static_cast<const char*>(data);
while (n > 0) {
#ifdef _WIN32
const DWORD chunk = static_cast<DWORD>(n > (1u << 30) ? (1u << 30) : n);
DWORD w = 0;
if (!WriteFile(h_, p, chunk, &w, nullptr) || w == 0) return false;
#else
const ssize_t w = ::write(fd_, p, n);
if (w < 0) { if (errno == EINTR) continue; return false; }
if (w == 0) return false;
#endif
p += w;
n -= static_cast<std::size_t>(w);
}
return true;
}
// Flush to stable storage, then close. False on any error.
bool commit()
{
bool good = ok();
#ifdef _WIN32
if (good && !FlushFileBuffers(h_)) good = false;
#else
if (good && ::fsync(fd_) != 0) good = false;
#endif
if (!close()) good = false;
return good;
}
private:
bool close()
{
#ifdef _WIN32
if (h_ == INVALID_HANDLE_VALUE) return true;
const bool r = CloseHandle(h_) != 0;
h_ = INVALID_HANDLE_VALUE;
return r;
#else
if (fd_ < 0) return true;
const bool r = ::close(fd_) == 0;
fd_ = -1;
return r;
#endif
}
#ifdef _WIN32
HANDLE h_ = INVALID_HANDLE_VALUE;
#else
int fd_ = -1;
#endif
};
// Move `tmp` over `dest`. POSIX: rename() replaces atomically (even a running binary) + best-effort dir fsync.
// Windows (same primitive as Platform::writeFileAtomically): MoveFileEx REPLACE_EXISTING|WRITE_THROUGH. A
// running .exe cannot be replaced but can be renamed, so it is moved aside to ".old" once (the running process
// keeps it; cleaned on a later run). A just-closed file is often held for a moment by antivirus (sharing
// violation), so retry for up to ~2 s, but stop at once if the temp vanished (quarantined). On failure the
// temp is removed and a moved-aside original is put back.
bool replaceFromTemp(const std::string& tmp, const std::string& dest)
{
#ifdef _WIN32
const std::string aside = dest + ".old";
bool movedAside = false;
DWORD err = 0;
for (int attempt = 0; attempt < 20; ++attempt) {
if (MoveFileExA(tmp.c_str(), dest.c_str(), MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH))
return true;
err = GetLastError();
if (GetFileAttributesA(tmp.c_str()) == INVALID_FILE_ATTRIBUTES) break; // temp gone (AV quarantine)
if (!movedAside && GetFileAttributesA(dest.c_str()) != INVALID_FILE_ATTRIBUTES) {
DeleteFileA(aside.c_str());
if (MoveFileExA(dest.c_str(), aside.c_str(), MOVEFILE_REPLACE_EXISTING)) {
movedAside = true;
DEBUG_LOGF("[INFO] %s is in use - moved the old copy to .old\n", dest.c_str());
continue;
}
}
Sleep(100);
}
DEBUG_LOGF("[ERROR] Could not move %s into place (Win32 error %lu)\n", dest.c_str(), (unsigned long)err);
DeleteFileA(tmp.c_str());
if (movedAside) MoveFileExA(aside.c_str(), dest.c_str(), MOVEFILE_REPLACE_EXISTING);
return false;
#else
if (::rename(tmp.c_str(), dest.c_str()) != 0) {
DEBUG_LOGF("[ERROR] Could not move %s into place: %s\n", dest.c_str(), std::strerror(errno));
::remove(tmp.c_str());
return false;
}
const std::string dir = std::filesystem::path(dest).parent_path().string();
const int dfd = ::open(dir.empty() ? "." : dir.c_str(), O_RDONLY | O_CLOEXEC);
if (dfd >= 0) { ::fsync(dfd); ::close(dfd); }
return true;
#endif
}
} // namespace
bool resourceNeedsWrite(std::uint64_t embeddedSize, bool refreshOnSizeMismatch, const std::string& destPath)
{
if (embeddedSize == 0) return false;
std::error_code ec;
if (!std::filesystem::exists(destPath, ec)) return true;
if (!refreshOnSizeMismatch) return false;
const auto diskSize = std::filesystem::file_size(destPath, ec);
if (ec) return true; // can't stat -> rewrite
return static_cast<std::uint64_t>(diskSize) != embeddedSize;
}
// Check if an on-disk file is missing or differs in size from the embedded resource.
// A size mismatch means the binary was updated in a newer wallet build.
static bool resourceNeedsUpdate(const EmbeddedResource* res, const std::string& destPath)
[[maybe_unused]] static bool resourceNeedsUpdate(const EmbeddedResource* res, const std::string& destPath)
{
if (!res || !res->data || res->size == 0) return false;
if (!std::filesystem::exists(destPath)) return true;
std::error_code ec;
auto diskSize = std::filesystem::file_size(destPath, ec);
if (ec) return true; // can't stat → re-extract
return diskSize != static_cast<std::uintmax_t>(res->size);
return resourceNeedsWrite(res->size, /*refreshOnSizeMismatch=*/true, destPath);
}
static bool extractResource(const EmbeddedResource* res, const std::string& destPath)
bool needsParamsExtraction()
{
if (!res || !res->data || res->size == 0) {
return false;
if (!hasEmbeddedResources()) return false;
const std::string dir = getDaemonDirectory(); // the only extraction target
for (const auto& e : kProvisionEntries) {
if (!entryActive(e) || !e.startupTrigger) continue;
const EmbeddedResource* res = getEmbeddedResource(e.name);
if (res && resourceNeedsWrite(res->size, e.refreshOnSizeMismatch, joinDir(dir, e.name))) return true;
}
// Create parent directories
std::filesystem::path path(destPath);
if (path.has_parent_path()) {
std::error_code ec;
std::filesystem::create_directories(path.parent_path(), ec);
if (ec) {
DEBUG_LOGF("[ERROR] Failed to create directory %s: %s\n",
path.parent_path().string().c_str(), ec.message().c_str());
return false;
}
}
// Write file
std::ofstream file(destPath, std::ios::binary);
if (!file) {
// The destination may be locked because the previous daemon is still using the binary:
// Windows locks a running .exe against truncation, Linux returns ETXTBSY. Both platforms
// DO allow renaming/moving such a file — the running process keeps the moved copy — so move
// the stale binary aside and write a fresh one at the original path.
std::error_code ec;
if (std::filesystem::exists(destPath)) {
std::string sidelined = destPath + ".old";
std::filesystem::remove(sidelined, ec); // clear any leftover from a prior swap
ec.clear();
std::filesystem::rename(destPath, sidelined, ec);
if (!ec) {
file.clear();
file.open(destPath, std::ios::binary);
if (file)
DEBUG_LOGF("[INFO] Replaced in-use %s (old copy moved to .old)\n", destPath.c_str());
} else {
DEBUG_LOGF("[WARN] Could not move stale %s aside: %s\n",
destPath.c_str(), ec.message().c_str());
return false;
}
bool writeFileAtomic(const std::string& destPath, const std::uint8_t* data, std::size_t size, bool executable,
ProvisionProgress* progress, const CancelFn& cancelled)
{
if (!data || size == 0) return false;
if (!prepareParentDir(destPath)) return false;
const std::string tmp = destPath + kPartialFileSuffix;
std::error_code ec;
bool good = true;
{
TempSink f(tmp);
if (!f.ok()) { DEBUG_LOGF("[ERROR] Failed to open %s for writing\n", tmp.c_str()); return false; }
for (std::size_t off = 0; off < size; off += kCopyChunk) {
if (cancelled && cancelled()) { good = false; break; }
const std::size_t n = std::min(kCopyChunk, size - off);
if (!f.write(data + off, n)) {
DEBUG_LOGF("[ERROR] Failed to write %zu bytes to %s\n", n, tmp.c_str());
good = false; break;
}
if (progress) progress->bytesDone.fetch_add(n, std::memory_order_relaxed);
}
if (!file) {
DEBUG_LOGF("[ERROR] Failed to open %s for writing\n", destPath.c_str());
return false;
}
}
file.write(reinterpret_cast<const char*>(res->data), res->size);
if (!file) {
DEBUG_LOGF("[ERROR] Failed to write %zu bytes to %s\n", res->size, destPath.c_str());
return false;
}
DEBUG_LOGF("[INFO] Extracted %s (%zu bytes)\n", destPath.c_str(), res->size);
// Flush on the same handle BEFORE the rename: a power cut can't leave a truncated file under the final name.
if (good && !f.commit()) { DEBUG_LOGF("[ERROR] Failed to finish writing %s\n", tmp.c_str()); good = false; }
} // closed here: Windows can't delete a file that is still open without FILE_SHARE_DELETE
if (!good) { std::filesystem::remove(tmp, ec); return false; }
#ifndef _WIN32
if (executable) ::chmod(tmp.c_str(), 0755); // before it becomes visible under the final name
#else
(void)executable;
#endif
if (!replaceFromTemp(tmp, destPath)) return false;
DEBUG_LOGF("[INFO] Extracted %s (%zu bytes)\n", destPath.c_str(), size);
return true;
}
bool extractEmbeddedResources()
bool copyFileAtomic(const std::string& srcPath, const std::string& destPath,
ProvisionProgress* progress, const CancelFn& cancelled)
{
namespace fs = std::filesystem;
if (!prepareParentDir(destPath)) return false;
const std::string tmp = destPath + kPartialFileSuffix;
std::error_code ec;
bool good = true;
{
std::ifstream in(srcPath, std::ios::binary); // a read handle on the source is harmless if inherited
if (!in) { DEBUG_LOGF("[ERROR] Failed to open %s\n", srcPath.c_str()); return false; }
TempSink out(tmp);
if (!out.ok()) { DEBUG_LOGF("[ERROR] Failed to open %s for writing\n", tmp.c_str()); return false; }
std::vector<char> buf(kCopyChunk);
for (;;) {
if (cancelled && cancelled()) { good = false; break; }
in.read(buf.data(), static_cast<std::streamsize>(buf.size()));
const std::streamsize got = in.gcount();
if (got > 0) {
if (!out.write(buf.data(), static_cast<std::size_t>(got))) {
DEBUG_LOGF("[ERROR] Failed to write %s\n", tmp.c_str());
good = false; break;
}
if (progress) progress->bytesDone.fetch_add(static_cast<std::uint64_t>(got), std::memory_order_relaxed);
}
if (!in) break; // EOF (eof|fail) or a read error (bad, checked below)
}
if (good && in.bad()) {
DEBUG_LOGF("[ERROR] Failed to read %s\n", srcPath.c_str());
good = false;
}
// Flush on the same handle BEFORE the rename: a power cut can't leave a truncated file under the final name.
if (good && !out.commit()) { DEBUG_LOGF("[ERROR] Failed to finish writing %s\n", tmp.c_str()); good = false; }
} // closed here: Windows can't delete a file that is still open without FILE_SHARE_DELETE
if (!good) { fs::remove(tmp, ec); return false; }
const auto st = fs::status(srcPath, ec);
if (!ec) fs::permissions(tmp, st.permissions(), ec); // keeps dragonxd/cli/tx executable (as copy_file did)
return replaceFromTemp(tmp, destPath);
}
// Requires provisionMutex() held.
static bool extractResourceLocked(const EmbeddedResource* res, const std::string& destPath, bool executable,
ProvisionProgress* progress = nullptr, const CancelFn& cancelled = {})
{
if (!res || !res->data || res->size == 0) return false;
return writeFileAtomic(destPath, res->data, res->size, executable, progress, cancelled);
}
bool extractEmbeddedResources(ProvisionProgress* progress, const CancelFn& cancelled)
{
if (!hasEmbeddedResources()) {
DEBUG_LOGF("[ERROR] No embedded resources available\n");
return false;
}
bool success = true;
#ifdef _WIN32
const char pathSep = '\\';
#else
const char pathSep = '/';
#endif
// All files go to <ObsidianDragonDir>/dragonx/
std::string daemonDir = getDaemonDirectory();
std::lock_guard<std::mutex> lock(provisionMutex());
// Extract Sapling params to daemon directory alongside dragonxd
const EmbeddedResource* spendRes = getEmbeddedResource(RESOURCE_SAPLING_SPEND);
if (spendRes) {
std::string dest = daemonDir + pathSep + RESOURCE_SAPLING_SPEND;
if (resourceNeedsUpdate(spendRes, dest)) {
DEBUG_LOGF("[INFO] Extracting sapling-spend.params (%zu MB)...\n", spendRes->size / (1024*1024));
if (!extractResource(spendRes, dest)) {
success = false;
}
}
}
const EmbeddedResource* outputRes = getEmbeddedResource(RESOURCE_SAPLING_OUTPUT);
if (outputRes) {
std::string dest = daemonDir + pathSep + RESOURCE_SAPLING_OUTPUT;
if (resourceNeedsUpdate(outputRes, dest)) {
DEBUG_LOGF("[INFO] Extracting sapling-output.params (%zu MB)...\n", outputRes->size / (1024*1024));
if (!extractResource(outputRes, dest)) {
success = false;
}
}
}
// Extract asmap.dat to daemon directory
const EmbeddedResource* asmapRes = getEmbeddedResource(RESOURCE_ASMAP);
if (asmapRes) {
std::string dest = daemonDir + pathSep + RESOURCE_ASMAP;
if (resourceNeedsUpdate(asmapRes, dest)) {
DEBUG_LOGF("[INFO] Extracting asmap.dat...\n");
if (!extractResource(asmapRes, dest)) {
success = false;
}
}
}
// Extract daemon binaries — NOT the data directory.
// Running dragonxd.exe from inside the data directory (where it writes blockchain
// data, debug.log, lock files, etc.) causes crashes on some Windows machines.
#ifdef HAS_EMBEDDED_DAEMON
// All files go to <ObsidianDragonDir>/dragonx/ — NOT the data directory (running dragonxd.exe from inside
// the data directory, where it writes chain data / debug.log / lock files, crashes on some Windows machines).
const std::string daemonDir = getDaemonDirectory();
DEBUG_LOGF("[INFO] Daemon extraction directory: %s\n", daemonDir.c_str());
// Daemon binaries are placed ONLY when missing — never auto-overwritten on a size mismatch
// (the user may run a specific dragonxd; replacing it is an explicit Settings action).
const EmbeddedResource* daemonRes = getEmbeddedResource(RESOURCE_DRAGONXD);
if (daemonRes) {
std::string dest = daemonDir + pathSep + RESOURCE_DRAGONXD;
if (!std::filesystem::exists(dest)) {
DEBUG_LOGF("[INFO] Extracting dragonxd (%zu MB)...\n", daemonRes->size / (1024*1024));
if (!extractResource(daemonRes, dest)) {
success = false;
}
#ifndef _WIN32
else { chmod(dest.c_str(), 0755); }
#endif
std::error_code ec;
std::filesystem::create_directories(daemonDir, ec);
ec.clear();
// Temp files left by an interrupted run (killed process, power cut). Only OUR suffix.
for (const auto& e : kProvisionEntries) {
std::filesystem::remove(joinDir(daemonDir, e.name) + kPartialFileSuffix, ec);
ec.clear();
}
struct Todo { const ProvisionEntry* entry; const EmbeddedResource* res; std::string dest; };
std::vector<Todo> todo;
std::uint64_t total = 0;
for (const auto& e : kProvisionEntries) {
if (!entryActive(e)) continue;
const EmbeddedResource* res = getEmbeddedResource(e.name);
if (!res) continue;
std::string dest = joinDir(daemonDir, e.name);
if (!resourceNeedsWrite(res->size, e.refreshOnSizeMismatch, dest)) continue;
total += res->size;
todo.push_back({ &e, res, std::move(dest) });
}
if (progress) {
progress->bytesDone.store(0, std::memory_order_relaxed);
progress->bytesTotal.store(total, std::memory_order_relaxed);
progress->setPhase(ProvisionPhase::Extracting);
}
bool success = true;
for (const auto& t : todo) {
if (cancelled && cancelled()) return false;
DEBUG_LOGF("[INFO] Extracting %s (%zu MB)...\n", t.entry->name, t.res->size / (1024 * 1024));
if (!extractResourceLocked(t.res, t.dest, t.entry->executable, progress, cancelled)) {
if (cancelled && cancelled()) return false;
success = false;
}
}
const EmbeddedResource* cliRes = getEmbeddedResource(RESOURCE_DRAGONX_CLI);
if (cliRes) {
std::string dest = daemonDir + pathSep + RESOURCE_DRAGONX_CLI;
if (!std::filesystem::exists(dest)) {
DEBUG_LOGF("[INFO] Extracting dragonx-cli (%zu MB)...\n", cliRes->size / (1024*1024));
if (!extractResource(cliRes, dest)) {
success = false;
}
#ifndef _WIN32
else { chmod(dest.c_str(), 0755); }
#endif
}
// Best-effort cleanup of ".old" copies left by an in-use replacement. Once the old process has exited the
// file is no longer locked and removes cleanly; if it still runs, the remove fails and we retry next run.
for (const auto& e : kProvisionEntries) {
std::filesystem::remove(joinDir(daemonDir, e.name) + std::string(".old"), ec);
ec.clear();
}
const EmbeddedResource* txRes = getEmbeddedResource(RESOURCE_DRAGONX_TX);
if (txRes) {
std::string dest = daemonDir + pathSep + RESOURCE_DRAGONX_TX;
if (!std::filesystem::exists(dest)) {
DEBUG_LOGF("[INFO] Extracting dragonx-tx (%zu MB)...\n", txRes->size / (1024*1024));
if (!extractResource(txRes, dest)) {
success = false;
}
#ifndef _WIN32
else { chmod(dest.c_str(), 0755); }
#endif
}
}
#endif
#ifdef HAS_EMBEDDED_XMRIG
const EmbeddedResource* xmrigRes = getEmbeddedResource(RESOURCE_XMRIG);
if (xmrigRes) {
std::string dest = daemonDir + pathSep + RESOURCE_XMRIG;
if (resourceNeedsUpdate(xmrigRes, dest)) {
if (std::filesystem::exists(dest))
DEBUG_LOGF("[INFO] Updating stale xmrig (size mismatch)...\n");
DEBUG_LOGF("[INFO] Extracting xmrig (%zu MB)...\n", xmrigRes->size / (1024*1024));
if (!extractResource(xmrigRes, dest)) {
success = false;
}
#ifndef _WIN32
else { chmod(dest.c_str(), 0755); }
#endif
}
}
#endif
#ifdef HAS_EMBEDDED_WALLET_REBUILD
// Offline wallet-rebuild recovery helper — extracted next to the daemon so a bare, self-extracting
// ObsidianDragon.exe still offers "Repair automatically" (findWalletRebuildHelper() checks this dir).
const EmbeddedResource* rebuildRes = getEmbeddedResource(RESOURCE_DRAGONX_WALLET_REBUILD);
if (rebuildRes) {
std::string dest = daemonDir + pathSep + RESOURCE_DRAGONX_WALLET_REBUILD;
if (!std::filesystem::exists(dest)) {
DEBUG_LOGF("[INFO] Extracting dragonx-wallet-rebuild (%zu MB)...\n", rebuildRes->size / (1024*1024));
if (!extractResource(rebuildRes, dest)) {
success = false;
}
#ifndef _WIN32
else { chmod(dest.c_str(), 0755); }
#endif
}
}
#endif
// Best-effort cleanup of any ".old" binaries left behind by a previous in-use replacement.
// Once the old daemon/xmrig process has exited, the file is no longer locked and removes cleanly;
// if it's still running, the remove fails harmlessly and we retry on the next startup.
{
std::error_code ec;
for (const char* name : { RESOURCE_DRAGONXD, RESOURCE_XMRIG }) {
std::filesystem::remove(daemonDir + pathSep + name + std::string(".old"), ec);
ec.clear();
}
}
return success;
}
std::vector<std::string> bundleSearchDirs()
{
std::vector<std::string> dirs;
const std::string exeDir = util::Platform::getExecutableDirectory();
if (exeDir.empty()) return dirs;
#ifdef __APPLE__
// Inside a .app the params live in Contents/Resources while the executable is in Contents/MacOS.
std::error_code ec;
const std::filesystem::path res = std::filesystem::path(exeDir).parent_path() / "Resources";
if (std::filesystem::is_directory(res, ec)) dirs.push_back(res.string());
#endif
dirs.push_back(exeDir);
return dirs;
}
const std::vector<std::string>& saplingParamFileNames()
{
static const std::vector<std::string> v = { "sapling-spend.params", "sapling-output.params" };
return v;
}
const std::vector<std::string>& nodeBundleFileNames()
{
static const std::vector<std::string> v = { "sapling-spend.params", "sapling-output.params", "asmap.dat",
"dragonxd", "dragonx-cli", "dragonx-tx" };
return v;
}
std::vector<BundleCopyItem> planBundleCopy(const std::vector<std::string>& searchDirs, const std::string& destDir,
const std::vector<std::string>& names, bool replaceExisting)
{
namespace fs = std::filesystem;
std::vector<BundleCopyItem> plan;
std::error_code ec;
for (const auto& name : names) {
const fs::path dst = fs::path(destDir) / name;
if (!replaceExisting && fs::exists(dst, ec)) continue;
for (const auto& dir : searchDirs) {
const fs::path src = fs::path(dir) / name;
if (fs::exists(src, ec)) { plan.push_back({ name, src.string(), dst.string() }); break; }
}
}
return plan;
}
BundleCopyResult copyBundledFiles(const std::vector<std::string>& searchDirs, const std::string& destDir,
const std::vector<std::string>& names, bool replaceExisting,
ProvisionProgress* progress, const CancelFn& cancelled)
{
BundleCopyResult r;
// Stat-only check first: the steady-state call (every daemon start) never touches the lock.
if (planBundleCopy(searchDirs, destDir, names, replaceExisting).empty()) return r;
std::lock_guard<std::mutex> lock(provisionMutex());
const auto plan = planBundleCopy(searchDirs, destDir, names, replaceExisting); // re-check under the lock
std::uint64_t total = 0;
std::error_code ec;
for (const auto& it : plan) {
std::filesystem::remove(it.dst + kPartialFileSuffix, ec); // stale temp from an interrupted copy
ec.clear();
const auto sz = std::filesystem::file_size(it.src, ec);
if (!ec) total += sz;
ec.clear();
}
if (progress) {
progress->bytesDone.store(0, std::memory_order_relaxed);
progress->bytesTotal.store(total, std::memory_order_relaxed);
progress->setPhase(ProvisionPhase::Copying);
}
for (const auto& it : plan) {
if (cancelled && cancelled()) { r.ok = false; return r; }
DEBUG_LOGF("Copying bundled %s to %s\n", it.name.c_str(), destDir.c_str());
if (copyFileAtomic(it.src, it.dst, progress, cancelled)) {
++r.copied;
} else {
if (cancelled && cancelled()) { r.ok = false; return r; }
DEBUG_LOGF("[ERROR] Failed to copy %s\n", it.name.c_str());
r.ok = false;
if (r.firstFailure.empty()) r.firstFailure = it.name;
}
}
return r;
}
std::string ensureWalletRebuildHelperExtracted()
{
#ifdef HAS_EMBEDDED_WALLET_REBUILD
@@ -484,12 +679,11 @@ std::string ensureWalletRebuildHelperExtracted()
const std::string dir = getDaemonDirectory();
const std::string dest = dir + sep + RESOURCE_DRAGONX_WALLET_REBUILD;
std::error_code ec;
if (std::filesystem::exists(dest, ec)) return dest; // already extracted
if (std::filesystem::exists(dest, ec)) return dest; // already extracted (atomic write: exists == complete)
std::lock_guard<std::mutex> lock(provisionMutex()); // serialise with the startup job / other writers
if (std::filesystem::exists(dest, ec)) return dest; // written meanwhile by the startup job
std::filesystem::create_directories(dir, ec);
if (!extractResource(res, dest)) return {};
#ifndef _WIN32
chmod(dest.c_str(), 0755);
#endif
if (!extractResourceLocked(res, dest, /*executable=*/true)) return {};
DEBUG_LOGF("[INFO] Extracted wallet-rebuild helper on demand: %s\n", dest.c_str());
return dest;
#else
@@ -623,14 +817,11 @@ bool forceExtractXmrig()
}
DEBUG_LOGF("[INFO] forceExtractXmrig: extracting xmrig (%zu MB) to %s\n",
xmrigRes->size / (1024*1024), dest.c_str());
if (!extractResource(xmrigRes, dest)) {
std::lock_guard<std::mutex> lock(provisionMutex());
if (!extractResourceLocked(xmrigRes, dest, /*executable=*/true)) {
DEBUG_LOGF("[ERROR] forceExtractXmrig: extraction failed\n");
return false;
}
#ifndef _WIN32
// Set executable permission on Linux
chmod(dest.c_str(), 0755);
#endif
return true;
#else
return false;
@@ -722,6 +913,7 @@ BundledDaemonInfo getBundledDaemonInfo()
bool reextractBundledDaemon()
{
#ifdef HAS_EMBEDDED_DAEMON
std::lock_guard<std::mutex> lock(provisionMutex());
std::string daemonDir = getDaemonDirectory();
#ifdef _WIN32
const char pathSep = '\\';
@@ -736,15 +928,12 @@ bool reextractBundledDaemon()
if (!res) continue;
std::string dest = daemonDir + pathSep + name;
DEBUG_LOGF("[INFO] reextractBundledDaemon: writing %s (%zu MB)\n", name, res->size / (1024*1024));
if (!extractResource(res, dest)) {
if (!extractResourceLocked(res, dest, /*executable=*/true)) {
DEBUG_LOGF("[ERROR] reextractBundledDaemon: failed to write %s\n", name);
ok = false;
continue;
}
wroteAny = true;
#ifndef _WIN32
chmod(dest.c_str(), 0755);
#endif
}
return ok && wroteAny;
#else
@@ -770,8 +959,13 @@ std::string getXmrigPath()
std::string embeddedPath = daemonDir + pathSep + xmrigName;
DEBUG_LOGF("[DEBUG] getXmrigPath: checking embedded path %s\n", embeddedPath.c_str());
if (!std::filesystem::exists(embeddedPath)) {
DEBUG_LOGF("[DEBUG] getXmrigPath: not found, force re-extracting xmrig\n");
forceExtractXmrig();
// Serialise with the startup job: if it just wrote xmrig, the re-check finds it complete.
std::lock_guard<std::mutex> lock(provisionMutex());
if (!std::filesystem::exists(embeddedPath)) {
DEBUG_LOGF("[DEBUG] getXmrigPath: not found, force re-extracting xmrig\n");
const EmbeddedResource* xr = getEmbeddedResource(RESOURCE_XMRIG);
if (xr) extractResourceLocked(xr, embeddedPath, /*executable=*/true);
}
}
if (std::filesystem::exists(embeddedPath)) {
DEBUG_LOGF("[INFO] getXmrigPath: found at %s\n", embeddedPath.c_str());
src/resources/embedded_resources.h
+47 -4
View File
@@ -1,7 +1,12 @@
#pragma once
#include <string>
#include <cstddef>
#include <cstdint>
#include <functional>
#include <string>
#include <vector>
#include "provision_state.h"
namespace dragonx {
namespace resources {
@@ -20,9 +25,47 @@ bool hasEmbeddedResources();
// Returns nullptr if not found or not embedded
const EmbeddedResource* getEmbeddedResource(const std::string& name);
// Extract all embedded resources to appropriate locations
// Returns true if all resources extracted successfully
bool extractEmbeddedResources();
using CancelFn = std::function<bool()>; // polled between 4 MB chunks; true = stop now
// Suffix of our own temp files in the daemon dir ("<name>.partial"). NOT ".tmp": the in-app miner/node
// updaters stage "<name>.tmp" in the same directory and must never have it deleted under them.
inline constexpr const char* kPartialFileSuffix = ".partial";
// Extract all embedded resources to the daemon directory. Serialised with every other writer of that
// directory by an internal mutex; atomic per file (temp + flush + rename), so a destination is either the old
// complete file or the new one. `progress` (optional) receives bytes written; `cancelled` (optional) aborts
// between chunks leaving no partial destination. Returns true if every needed file was written (false also on
// cancel). Safe on any thread.
bool extractEmbeddedResources(ProvisionProgress* progress = nullptr, const CancelFn& cancelled = {});
// Does `destPath` need (re)writing? Missing -> yes. Present: only when `refreshOnSizeMismatch` and the size
// differs. embeddedSize == 0 -> never. (Pure apart from the stat; exposed for tests.)
bool resourceNeedsWrite(std::uint64_t embeddedSize, bool refreshOnSizeMismatch, const std::string& destPath);
// Write `data` to `destPath` via `destPath + kPartialFileSuffix`, flush it to disk, then move it into place
// (POSIX rename; Windows MoveFileEx with retry and move-aside of a running .exe to ".old"). POSIX: chmod 0755
// first when `executable`. Returns false (temp removed, original kept) on any failure or on cancel.
// Takes no lock itself (extractEmbeddedResources / copyBundledFiles / the on-demand extractors do).
bool writeFileAtomic(const std::string& destPath, const std::uint8_t* data, std::size_t size, bool executable,
ProvisionProgress* progress = nullptr, const CancelFn& cancelled = {});
// Same contract, streaming from a source file in 4 MB chunks; keeps the source's permissions.
bool copyFileAtomic(const std::string& srcPath, const std::string& destPath,
ProvisionProgress* progress = nullptr, const CancelFn& cancelled = {});
// Node files shipped NEXT TO the executable (zip / AppImage / .app; no embedded_data.h).
struct BundleCopyItem { std::string name, src, dst; };
std::vector<std::string> bundleSearchDirs(); // macOS: .app Contents/Resources first; then exe dir
const std::vector<std::string>& saplingParamFileNames(); // sapling-spend.params, sapling-output.params
const std::vector<std::string>& nodeBundleFileNames(); // params + asmap.dat + dragonxd + dragonx-cli + dragonx-tx
// First search dir holding a name wins; names with no source are skipped (optional); names whose destination
// exists are skipped unless replaceExisting. Stat-only.
std::vector<BundleCopyItem> planBundleCopy(const std::vector<std::string>& searchDirs, const std::string& destDir,
const std::vector<std::string>& names, bool replaceExisting);
struct BundleCopyResult { bool ok = true; int copied = 0; std::string firstFailure; };
// Executes planBundleCopy under the provisioning lock (checks the plan lock-free first: the common
// "nothing to copy" call never touches the lock).
BundleCopyResult copyBundledFiles(const std::vector<std::string>& searchDirs, const std::string& destDir,
const std::vector<std::string>& names, bool replaceExisting,
ProvisionProgress* progress = nullptr, const CancelFn& cancelled = {});
// Check if params need to be extracted (not already present)
bool needsParamsExtraction();
src/resources/provision_state.h
+118
View File
@@ -0,0 +1,118 @@
// DragonX Wallet - ImGui Edition
// Copyright 2024-2026 The Hush Developers
// Released under the GPLv3
//
// provision_state.h - state of the startup "node files" job (App::startStartupFilesProvision): on Windows it
// extracts the bundled Sapling params / asmap / node / miner from the .exe, on Linux/macOS it copies them
// from the executable's directory, and on full-node builds it then verifies the Sapling params. The job runs
// on a worker thread; the UI thread only polls this. No app dependencies, so tests include it directly.
#pragma once
#include <atomic>
#include <chrono>
#include <condition_variable>
#include <cstdint>
#include <mutex>
namespace dragonx {
namespace resources {
enum class ProvisionPhase : int { Idle = 0, Extracting, Copying, Verifying };
// Lock-free progress: written by the worker, read by the UI every frame.
struct ProvisionProgress {
std::atomic<std::uint64_t> bytesDone{0};
std::atomic<std::uint64_t> bytesTotal{0};
std::atomic<int> phase{static_cast<int>(ProvisionPhase::Idle)};
void setPhase(ProvisionPhase p) { phase.store(static_cast<int>(p), std::memory_order_relaxed); }
ProvisionPhase getPhase() const { return static_cast<ProvisionPhase>(phase.load(std::memory_order_relaxed)); }
};
// 0..100; 0 while the total is unknown.
inline int provisionPercent(std::uint64_t done, std::uint64_t total)
{
if (total == 0) return 0;
if (done >= total) return 100;
return static_cast<int>((done * 100u) / total);
}
// The chip stays hidden for a short grace period so a job that ends in a few ms never flashes.
inline bool provisionChipVisible(bool running, double elapsedSeconds, double delaySeconds)
{
return running && elapsedSeconds >= delaySeconds;
}
// What App::startEmbeddedDaemon() does while the job may still be writing the node files: never launch the
// node over half-written files, never block the UI thread (defer + replay), workers simply wait.
enum class DaemonStartGate { Proceed, Defer, Wait };
inline DaemonStartGate daemonStartGate(bool jobRunning, bool onUiThread)
{
if (!jobRunning) return DaemonStartGate::Proceed;
return onUiThread ? DaemonStartGate::Defer : DaemonStartGate::Wait;
}
class StartupProvision {
public:
struct Result {
bool extractOk = true; // every needed file written/copied (false: disk full / permissions / AV)
bool verifyOk = true; // Sapling SHA-256 check (full node only)
bool cancelled = false; // shutdown stopped it; nothing is wrong
};
enum class State : int { Idle, Running, Finished, Reaped };
ProvisionProgress progress;
// The Sapling param repair copy (replace the daemon-dir params from the bundle, ~51.5 MB) ran this session:
// set by the job and by App::startEmbeddedDaemon(), so a still-failing verify never repeats it on every
// connect retry (the UI thread).
std::atomic<bool> paramRepairTried{false};
void begin() {
std::lock_guard<std::mutex> lk(m_);
state_ = State::Running;
startedAt_ = std::chrono::steady_clock::now();
}
// Nothing needed doing (the steady-state launch): no thread was started.
void markNothingToDo() { std::lock_guard<std::mutex> lk(m_); state_ = State::Reaped; }
void finish(const Result& r) {
{ std::lock_guard<std::mutex> lk(m_); result_ = r; state_ = State::Finished; }
cv_.notify_all();
}
bool isRunning() const { std::lock_guard<std::mutex> lk(m_); return state_ == State::Running; }
// Non-reaping peek: the job has ended and shutdown cancelled it (synchronized with finish() through m_).
bool endedCancelled() const {
std::lock_guard<std::mutex> lk(m_);
return state_ != State::Running && result_.cancelled;
}
// App::hasPendingBackgroundResults(): finished and not yet reaped (the reap needs one update() tick).
// NOT while Running: nothing is reapable then, and the chip already requests progress frames when visible.
bool hasCompletion() const { std::lock_guard<std::mutex> lk(m_); return state_ == State::Finished; }
// UI thread: true exactly once, after the job finished.
bool takeCompletion(Result* out) {
std::lock_guard<std::mutex> lk(m_);
if (state_ != State::Finished) return false;
if (out) *out = result_;
state_ = State::Reaped;
return true;
}
// Worker threads: true when the job is no longer running.
bool waitUntilDone(int timeoutMs) {
std::unique_lock<std::mutex> lk(m_);
cv_.wait_for(lk, std::chrono::milliseconds(timeoutMs), [&] { return state_ != State::Running; });
return state_ != State::Running;
}
double elapsedSeconds() const {
std::lock_guard<std::mutex> lk(m_);
if (state_ != State::Running) return 0.0;
return std::chrono::duration<double>(std::chrono::steady_clock::now() - startedAt_).count();
}
private:
mutable std::mutex m_;
std::condition_variable cv_;
State state_ = State::Idle;
Result result_;
std::chrono::steady_clock::time_point startedAt_{};
};
} // namespace resources
} // namespace dragonx