ObsidianDragon/tools/fake_dragonxd

fake_dragonxd

A deliberately slow, scriptable stand-in for dragonxd, used to reproduce and regression-test how ObsidianDragon stops (or kills) a busy node. It is test tooling: off by default, never packaged.

The incident it models (2026-09):

1. A 13.6k-note wallet spent ~1 h in a witness rebuild that held cs_main.
2. The GUI's own abandoned polls filled the node's 24 RPC slots, so stop was never delivered.
3. On close, the GUI hard-killed the node mid-rebuild, so every restart replayed ~62k blocks and redid the rebuild.

What it copies from the real daemon (DragonX v1.3.0):

• HTTP-RPC queue. 8 workers plus a 16-deep queue (or -rpcthreads / -rpcworkqueue from the command line or DRAGONX.conf). When the queue is full the request gets an immediate plain-text HTTP 500 "Work queue depth exceeded". A request whose client timed out keeps its slot until a worker reaches it.
• cs_main. One global lock. Every method except stop, help and getrescaninfo (lock-free in dragonxd too) takes it. A "witness rebuild" holds it for FAKE_REBUILD_SECS and logs the real markers:
  • Reading blocks for witness rebuild: <off> / <total> (Phase 1, abortable);
  • nothing during Phase 2, which ignores shutdown;
  • BuildWitnessCache: rebuilt N note witness cache(s) to height H in Tms using 8 thread(s) at the end.
• Warmup. -28 errors carrying the current init message, using the daemon's warmup allow-list. Optionally an Activating best chain... replay, which logs UpdateTip lines and stops between steps on shutdown.
• Shutdown. Triggered by stop, or by SIGTERM/SIGINT on POSIX (Windows has no signal channel, as in v1.3.0). In order, the stub:
  1. unbinds the listener at once;
  2. answers new requests on open connections with 503;
  3. waits for the rebuild to finish;
  4. logs Shutdown: In progress... and renames its main thread to hush-shutoff (Linux comm);
  5. answers never-run queued requests with 500 Unhandled request;
  6. flushes for FAKE_FLUSH_SECS, logs Shutdown: done, and exits 0.
• Logging. With -printtoconsole, untimestamped lines go to stdout (this is what the GUI uses on Linux and macOS). Otherwise, timestamped UTC lines go to <datadir>/debug.log (the Windows path the GUI tails).
• Datadir. It takes .lock, writes hushd.pid (POSIX) and .cookie (when there is no rpcpassword), and uses the same default datadir as dragonxd (~/.hush/DRAGONX, %APPDATA%\Hush\DRAGONX).

Its replies are canned: getinfo reports version 1030050, the balance is fixed, and there are two peers on TEST-NET addresses. Unknown methods get the real -32601 Method not found (HTTP 404).

Events file

Everything that matters for a test is appended to <datadir>/fake_dragonxd.events, one <name> t=<unix ms> [k=v ...] per line:

Event	Meaning
start, listening, warmup-end	startup; start lists every knob
rebuild-start, rebuild-end, rebuild-aborted	the cs_main hold
queue-full	the first Work queue depth exceeded
max-abandoned n=	a new high for client-abandoned requests still holding a slot
stop-received n=	a stop RPC ran (every one is logged, so resends are visible)
stop-rejected status=	a stop never reached a worker (500 queue-full, or 503 after interrupt)
sigterm	SIGTERM/SIGINT received (POSIX)
http-interrupted, shutdown-begin	shutdown progress
stats ...	counters, every 30 s when they change
methods ...	the 10 most-requested JSON-RPC methods with counts (with stats, and at exit): what the GUI is polling
clean-exit ...	graceful exit, with final counters. If this is missing, the process was killed.

Build

cmake -S . -B build/linux -DDRAGONX_BUILD_FAKE_DAEMON=ON
cmake --build build/linux --target fake_dragonxd          # -> build/linux/fake_dragonxd/dragonxd

# Windows (mingw cross, from Linux/WSL): builds it with the Windows test suite in build/windows-tests
scripts/test-windows.sh --no-run                          # -> build/windows-tests/fake_dragonxd/dragonxd.exe

The output goes to its own fake_dragonxd/ directory so it never replaces the real bin/dragonxd.

Run the GUI against it (Linux)

FAKE_REBUILD_SECS=300 tools/fake_dragonxd/run-gui-with-fake.sh [--build-dir build/linux] [--scratch DIR]

The script builds a throwaway HOME with:

• copies of the GUI and the stub (as dragonxd, next to the GUI, where findDaemonBinary() looks first);
• a DRAGONX.conf and an empty blocks/, so the first-run wizard is skipped;
• settings.json with verbose_logging.

It then starts the GUI and prints the events file when the GUI exits. It refuses to run if something already listens on 21769, because the GUI would adopt that process as an external node. It also refuses a Lite-configured --build-dir: Lite has no embedded daemon, and any bin/ObsidianDragon in such a tree is stale. To "close the window" from a script, send SIGTERM to the GUI (kill -TERM <gui pid>); SDL turns it into a normal quit. For scripted runs nobody needs to watch, add SDL_VIDEODRIVER=offscreen: the GUI runs with no window at all (OpenGL still loads, and connecting and shutting down work as usual).

To run the stub alone: FAKE_... build/linux/fake_dragonxd/dragonxd -datadir=/some/dir [-printtoconsole] with a DRAGONX.conf (rpcuser/rpcpassword/rpcport) in that dir.

Windows

Use a separate Windows user or a VM:

• The datadir is %APPDATA%\Hush\DRAGONX and can't be redirected for the GUI.
• Until S3 lands, the GUI TerminateProcesses any dragonxd.exe of the same user, including a real node.

Put dragonxd.exe (the stub) next to ObsidianDragon.exe. Set the FAKE_* variables in that user's environment, or start the stub by hand from a console for case E. Each new GUI-launched stub inherits the GUI's environment.

Knobs (environment)

Variable	Default	Meaning
FAKE_WARMUP_SECS	3	-28 Loading block index... phase
FAKE_REPLAY_SECS / FAKE_REPLAY_BPS	0 / 12	startup replay (Activating best chain..., UpdateTip lines)
FAKE_REBUILD_DELAY_SECS	20	wait after Done Loading before the rebuild, so the GUI connects first
FAKE_REBUILD_SECS	300	how long the rebuild holds cs_main; 0 = healthy node, no rebuild
FAKE_PHASE1_SECS	2	the logged, abortable part of the rebuild
FAKE_POST_REBUILD_LOCK_SECS	0	cs_main held after the end line (models the 18–36 s flush + wallet SetBestChain)
FAKE_FLUSH_SECS	3	Shutdown: In progress... → Shutdown: done
FAKE_UPDATETIP_SECS	36	block interval once loaded
FAKE_NOISE_SECS	20	an unrelated log line this often (defeats "newest line" heuristics)
FAKE_SERVER_TIMEOUT_SECS	30	idle keep-alive connections are closed (-rpcservertimeout)
FAKE_PREFILL	0	inject N abandoned requests when the rebuild starts (24 = saturated)
FAKE_MINING	0	internal miner on: stop waits for cs_main first (GenerateBitcoins join)
FAKE_REBUILD_SILENT	0	hold cs_main like a rebuild but log no rebuild markers (a busy node whose output says nothing)
FAKE_STUCK_SHUTDOWN	0	the Sep 29 orphan: hold the datadir lock, no listener, ignore SIGTERM, never exit
FAKE_WALLET_NOTES	(off)	a note wallet: addr:count:amount,... shielded notes; enables z_mergetoaddress + its operations (dragonxd rules: confirmed unlocked notes, a batch locked on return, merged note next block)
FAKE_MERGE_SECS	6	how long a z_mergetoaddress operation runs before it succeeds
FAKE_MERGE_SERIAL	0	1 = ops run one at a time, like dragonxd's single async worker: a new op is queued until the previous one finishes
FAKE_MERGE_FAIL	(off)	syncing:N (the Nth and later merges fail with the THROW_IF_SYNCING text), lockedkey:N (-8 "Could not find spending key…" after the scan), opfail:N (the Nth and later ops end failed, their notes unlocked)
FAKE_SCAN_SECS	0	the GUI's wallet scans (z_listunspent, listunspent, z_gettotalbalance, z_listreceivedbyaddress, z_mergetoaddress) hold cs_main this long; scan-begin/scan-end events
FAKE_TIP_UNKNOWN_SECS	0	longestchain reads 0 this long after Done Loading (a node that hasn't seen an UpdateTip yet)
FAKE_LONGEST_AHEAD / FAKE_CATCHUP_SECS	0 / 0	the network tip (longestchain, headers) is N blocks ahead, for that long after Done Loading (0 = for good)
FAKE_TXCOUNT	42	getwalletinfo txcount (the GUI's connect gate waits for the tip on wallets of 500+)
FAKE_METHODS_ALL	0	1 = one call m=<method> p=<params> event per request (the GUI's RPC cadence; the methods lines keep only the top 10 counts)
FAKE_HEIGHT, FAKE_REBUILD_TOTAL, FAKE_NOTES, FAKE_VERSION	3327145, 62583, 13632, 1030050	cosmetic numbers

Test cases

"Today" is the behaviour of v2.0.1 / master @ 46189da. "Fixed" is the target behaviour of the daemon-stop hardening work.

Case	Setup	Today (bug)	Fixed
A	defaults; leave the GUI open ~5 min into the rebuild	~24 abandoned, then queue-full 500s; overlay "Launching dragonxd…"	busy overlay; max-abandoned ≤ 7
B	close the GUI during the rebuild	Linux: sigterm at +20 s, killed at +30 s, no clean-exit. Windows: killed at +20 s, no stop-received	modal; stop-received/sigterm within 2 s; waits; clean-exit
C	close so the rebuild ends ~8 s later	killed	stop retried, clean-exit
D	FAKE_PREFILL=24, close	stop never delivered	POSIX via SIGTERM; Windows stop retried until it lands
E	Windows: start the stub by hand with FAKE_STUCK_SHUTDOWN=1, launch the GUI, close it	"Found orphaned dragonxd.exe … forcing termination"	no kill
F	Detach on the shutdown screen	n/a	GUI exits at once; stub runs on to clean-exit
G	Overview page during the 500 phase; count [connect #N] per minute in the GUI log	~1,800–3,200/min	≤ ~2–4/min