Review follow-ups to 73a93f1.
Bound in-flight calls. rpcclient's single sendPostHandler goroutine
imposed an accidental ceiling of one concurrent RPC; removing it without
putting anything in its place left no bound at all. grpc-go supplies
none either -- this server sets no MaxConcurrentStreams, so the default
is math.MaxUint32 -- and dragonxd answers RPC with 8 worker threads
behind a 4096-deep queue, shared on the pool node with getblocktemplate.
Overload would therefore surface as mining latency rather than as an
error we could back off on. MaxConnsPerHost blocks the caller at the
limit instead of dialling more, which is the backpressure wanted;
MaxIdleConnsPerHost alone would only cap reuse and let us exceed the
limit while churning connections. Default 8, matching the node's
DEFAULT_HTTP_THREADS, tunable with -rpc-max-concurrent. Even 8 removes
all of the head-of-line blocking this work set out to fix.
IdleConnTimeout 90s -> 20s. dragonxd closes idle connections at 30s
(DEFAULT_HTTP_SERVER_TIMEOUT, applied via evhttp_set_timeout and not
overridden in DRAGONX.conf). At 90s we were always the second to close,
so a request could be written into a connection the server had already
sent a FIN for, and Go will not retry a POST once bytes are on the wire.
Closing first removes the race.
Reject a negative -rpc-timeout, which silently meant "unbounded", the
same as the documented 0. The check has to run after flag.Parse(); it
was initially placed before it and never fired.
Also correct the coinsupply note: hush_coinsupply walks the block index
back to genesis, loading each block from disk and memoising newcoins and
zfunds into the CBlockIndex, so the first call pays for the whole chain
and later ones are nearly free. It is not a UTXO-set scan, as the
earlier comment claimed. The measured 48s/3s figures are unchanged.
Verified: five concurrent GetLightdInfo calls all return grpc-status 0
with no errors, 46 blocks ingested, and the daemon holds 2 sockets to
the node rather than one per request; a negative timeout exits 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
lightwalletd used github.com/btcsuite/btcd/rpcclient for exactly one
method, RawRequest, and that package has two defects that together
produce the failure seen on the primary node:
1. It has no timeout and no way to set one. The http.Client is built
in an unexported newHTTPClient() and ConnConfig exposes no Timeout
field, so a call can hang forever. Calls were observed running past
five minutes against a healthy node that answered the same query
from dragonx-cli in 2ms.
2. In HTTP POST mode it runs ONE sendPostHandler goroutine which calls
handleSendPostMessage synchronously, so the whole process has at
most one RPC in flight. One stuck call therefore blocks the block
ingestor, the mempool monitor and every user-facing gRPC handler at
once.
These compound: a timeout alone would not have been enough, because
bounding the caller's wait still leaves the shared goroutine stuck
inside http.Client.Do with everything queued behind it. Only a timeout
on the HTTP client aborts the in-flight request, and only dropping the
shared goroutine lets independent callers proceed.
Patching the vendored copy is not an option here: go.mod declares
go 1.12, so automatic vendor mode (go >= 1.14) is off and the committed
vendor/ tree is silently ignored in favour of the module cache. It is
also stale -- vendor/modules.txt disagrees with go.mod on btcd,
protobuf, logrus, sqlite3 and six other modules, so -mod=vendor cannot
build at all, and `go mod vendor` would discard any patch.
Replace it with package zrpc: a ~150-line JSON-RPC client that is safe
for concurrent use and takes a timeout. The request envelope, ID
sequence, basic auth and error semantics are deliberately identical.
RPCError.Error() still renders as "<code>: <message>" because callers
recover the numeric code from the string -- common.GetSaplingInfo
checks for -8 via strings.SplitN(err.Error(), ":", 2) -- and the
non-JSON body path still reports `status code: %d, response: %q`.
Default timeout 120s, tunable with -rpc-timeout (0 disables). The bound
is set by the slowest legitimate call: coinsupply measured 48s against
a cold UTXO set, 3s once cached, so a tighter timeout would turn a
slow-but-working call into a hard failure.
Also drops rpcclient's Close=true, which opened a fresh TCP connection
per request and left hundreds of sockets in TIME_WAIT on a busy node;
idle connections are now reused and capped.
Verified against a live dragonxd: getblockchaininfo returns chain=main;
a bad height yields exactly "-8: Block height out of range" and parses
back to -8; a 1ns timeout aborts in 54us instead of hanging; and the
built daemon serves GetLightdInfo at the current height with no errors.
The pre-existing parser TestCompactBlocks failure is unrelated and
reproduces on the base commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
/home/dev/lightwalletd/monitor_lwd.sh was carrying two uncommitted
production fixes, and the copy on disk had been reverted to the broken
committed version. The monitor that is actually running was executing a
deleted inode, so the fixes survived only as long as that process did --
any restart would have picked up the broken file.
The two fixes that were nearly lost:
* `-cache-size 5000` on the launch line. Without it a relaunch warms
the block cache from tip-400000 instead of tip-5000, which is
several minutes of getblock storm against the local node and several
minutes during which every wallet errors "Server's latest block is
behind ours".
* `EXIT_CODE=0; wait "$LWD_PID" || EXIT_CODE=$?` instead of
`wait "$LWD_PID" || true; EXIT_CODE=$?`. The latter reads the status
of `|| true` and is therefore always 0, so the monitor logged
"exited cleanly. Not restarting." and broke its loop on every exit
including crashes. That bug produced an 11h48m outage on 2026-08-21.
Recovered byte-identical from the running monitor via /proc/<pid>/fd/255
(md5 1823440d0af509c92583796af075b657) and committed so a checkout
cannot discard it again. An out-of-repo copy is kept at
/home/dev/monitor_lwd.sh.good.
Note the other branches still carry the broken blob; checking one out in
this working tree will clobber this file again. This working tree is a
live operational directory, not just a source checkout.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
Bump for the GetTransaction crash fix, and make the version one value
instead of two literals that could drift.
It was duplicated: cmd/server/main.go had `var version = "0.1.1"` for
--version, and frontend/service.go had "0.1.1-dragonxlightd" inline in
the LightdInfo reply. The gRPC one is the load-bearing copy -- it is
walletrpc/service.proto:48, so every client reads it, and it is the only
way to tell from off-box which build a node is running.
That property is the point of bumping now rather than later. With it, a
rollout can be verified by probing each endpoint over TLS and reading
the advertised version, instead of shelling in to compare binary
checksums, and instead of the only alternative positive test -- calling
GetTransaction on a mempool txid, which proves the fix by crashing any
node that does not have it.
Verified: --version prints 0.1.2, and a GetLightdInfo probe against a
test instance returns 0.1.2-dragonxlightd where production still returns
0.1.1-dragonxlightd.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
GetTransaction asserted the height out of getrawtransaction's reply
without checking it:
txHeight = txinfo.(map[string]interface{})["height"].(float64)
dragonxd emits "height" only for a transaction that is in a block --
rawtransaction.cpp puts it inside `if (!hashBlock.IsNull())` -- so every
mempool transaction comes back without the key. The assertion then runs
nil.(float64) and panics. Nothing recovers it: grpc-go v1.24.0 installs
no recovery interceptor (there is no recover() in its server.go) and
this daemon adds none, so the panic takes down the whole process and
every wallet connected to that endpoint with it.
Any client can trigger it deliberately: broadcast a transaction, then
ask for it before it is mined. GetMempoolStream, added in b1b0d45,
hands out unconfirmed txids by design, so ordinary 0-conf use walks
straight into it.
Verified against a live mempool transaction on this node, whose reply
has neither "height" nor "blockhash": the old expression panics with
"interface conversion: interface {} is nil, not float64"; the new one
returns cleanly.
An unconfirmed transaction is now reported as tip+1, which is what
GetMempoolStream already advertises for the same transactions
(mempool.go:109).
Also harden GetSaplingInfo, which had six more unchecked assertions on
the getblockchaininfo reply. Those run on the block-ingestor goroutine,
where a panic is equally fatal. The top-level object is asserted once
and every field is read with the comma-ok form; a missing field now
degrades instead of crashing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
Adaptive-lag: advertise a tip that trails the real tip by a reorg-rate-driven
lag (GetLatestBlock/GetLightdInfo) so wallets anchor shielded spends at a
settled height during reorg churn. Configurable via
-adaptive-lag/-lag-min/-lag-max/-lag-window; monitor_lwd.sh runs
-lag-min 4 -lag-max 12 -lag-window 30.
Startup coinsupply: the informational startup coinsupply RPC (result used only
for a log line) blocked the gRPC bind for minutes on a node whose supply index
is cold, keeping the lite endpoint down on every restart while it also starved
the block-cache ingestor. Run it in a goroutine so the bind is never blocked;
clients still get coinsupply on demand via the GetCoinsupply RPC.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The CompactTxStreamer service did not implement any mempool RPC, so
lightwallets (SilentDragonXLite / ObsidianDragonLite) received UNIMPLEMENTED
from get_mempool_stream and permanently stopped their mempool monitor. As a
result the lite wallet could not see 0-confirmation transactions and only
surfaced incoming shielded chat messages after they were mined (~1 block).
Add GetMempoolStream(Empty) returns (stream RawTransaction). A single
process-wide monitor (frontend/mempool.go) polls the node's getrawmempool and
fetches each new tx via getrawtransaction (full serialized bytes, so shielded
memos are preserved -- CompactTx's 52-byte prefix would not carry a memo), then
fans the tx out to every subscribed stream. This avoids polling the node once
per connected wallet: N clients share one poll loop. The monitor starts lazily
on the first subscription, idles without touching the node when no clients are
connected, and closes all subscriber streams when a new block is mined (the
semantics the client's monitor loop expects; the client re-syncs and reconnects).
Purely additive and wire-compatible: RawTransaction field numbers and the
cash.z.wallet.sdk.rpc package match the deployed client; existing clients that
never call GetMempoolStream are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Handle missing Sapling upgrade in getblockchaininfo response by
defaulting activation height to 1 (DragonX has Sapling active from
genesis but uses NO_ACTIVATION_HEIGHT in chainparams)
- Override consensus branch ID to Sapling (76b809bb) when node reports
Sprout (00000000) due to disabled activation heights
- Include full serialized block header in CompactBlock so the SDK can
validate hashFinalSaplingRoot against the Sapling commitment tree
- Set CompactBlock ProtoVersion to 1
