DragonX/ObsidianDragon
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 10 Wiki Activity

fix(rpc): run rescans and key imports without the 30 s timeout

Browse Source 
A rescan or key import holds the node's cs_main for the whole wallet
scan. On the shared clients it hit the 30 s curl timeout, reported a
failure that wasn't one, abandoned a daemon RPC slot and flipped the app
into "node busy".

- One long wallet RPC at a time (rescan, importprivkey, importaddress,
  importwallet, z_importkey, z_importviewingkey, z_importwallet) runs on
  its own ungated client with no client timeout (rpc/long_call_lane.h).
  It can be aborted from any thread, and an abort is never lost, even
  one that lands before the client opens.
- The GUI rescan, key import and sweep use it. The sweep's fund-moving
  body is unchanged; only the client it calls differs. runtimeRescan()
  returns bool, and a refused post-bootstrap reconcile stays armed.
- The console routes rescan and the import commands through the lane;
  its input stays usable meanwhile, and a refusal is printed like any
  other result. RPCClient gains callRaw(.., timeoutSec).
- Restart, wallet switch, delete-chain, bootstrap (now also from
  Settings) and the other node-stopping flows are refused while it runs,
  with a message saying a key import or rescan is still running; quit
  asks for confirmation. The lane is aborted at quit.
- A second import or rescan is refused before the key is copied anywhere.
  An import cut off after the request reached the node (a lost reply,
  or a proxy's 502/504 while the node still imports) says the key may
  already be in the wallet and to run Settings -> Rescan. Only the
  node's own HTTP-layer refusals (401, 503, work queue full, unhandled
  request) say that the key was not imported. A pure
  classifyLongCallCutOff() makes that choice.
- New strings in all 8 languages; console results stay English.

Tests: the long-method list and the console's routing; one lane at a
time; an abort lands whether it comes before the client opens or during
the call; a call with no client timeout outlasts a 3 s scan, where a 1 s
gated client times out and marks the node busy; classifyLongCallCutOff()
for every failure kind with and without a sent request, and from raw
HTTP replies (502, 504, a plain 500 -> cut off; 401, 503, queue full,
unhandled -> refused).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DanS
2026-10-12 13:22:00 -05:00
co-authored by Claude Opus 5.5
parent b3a57b07ad
commit 81733f5603
20 changed files with 776 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_phase4.cpp
+319
View File
@@ -4,8 +4,13 @@
#endif
#include <windows.h>
#else
#include <arpa/inet.h>
#include <netinet/in.h>
#include <poll.h>
#include <signal.h>
#include <sys/socket.h>
#include <sys/wait.h>
#include <unistd.h>
#endif
#include "chat/chat_crypto.h"
#include "chat/chat_identity.h"
@@ -29,6 +34,7 @@
#include "rpc/connection.h"
#include "rpc/rpc_failure.h"
#include "rpc/rpc_busy_gate.h"
#include "rpc/long_call_lane.h"
#include "resources/embedded_resources.h"
#include "resources/provision_state.h"
#include "services/history_scan_policy.h"
@@ -5458,6 +5464,312 @@ void testRpcBusyGate()
EXPECT_TRUE(hour.abandoned() <= 2);
}
// ---------------------------------------------------------------------------------------------------
// C61: long wallet RPCs (rescan, key imports) run one at a time on their own ungated client with no client
// timeout, and an abort is never lost, whenever it lands.
// ---------------------------------------------------------------------------------------------------
void testIsLongWalletMethod()
{
using dragonx::rpc::isLongWalletMethod;
for (const char* m : {"rescan", "importprivkey", "importaddress", "importwallet", "z_importkey",
"z_importviewingkey", "z_importwallet"})
EXPECT_TRUE(isLongWalletMethod(m));
for (const char* m : {"getinfo", "z_sendmany", "z_exportkey", "dumpprivkey", "getrescaninfo", "stop", ""})
EXPECT_FALSE(isLongWalletMethod(m));
// The console routes a typed rescan / import through the lane.
const auto rescan = dragonx::ui::BuildConsoleRpcCall("rescan 100");
EXPECT_TRUE(rescan.valid);
EXPECT_TRUE(isLongWalletMethod(rescan.method));
EXPECT_TRUE(isLongWalletMethod(dragonx::ui::BuildConsoleRpcCall("z_importkey \"secret-extended-key-main1x\" yes 5").method));
EXPECT_FALSE(isLongWalletMethod(dragonx::ui::BuildConsoleRpcCall("getinfo").method));
// One at a time; a new call starts un-aborted.
dragonx::rpc::LongCallLane lane;
EXPECT_FALSE(lane.active());
lane.abort(); // nothing running: sticky until the next tryBegin()
EXPECT_TRUE(lane.aborted());
EXPECT_TRUE(lane.tryBegin());
EXPECT_FALSE(lane.aborted());
EXPECT_TRUE(lane.active());
EXPECT_FALSE(lane.tryBegin());
lane.end();
EXPECT_FALSE(lane.active());
EXPECT_TRUE(lane.tryBegin());
lane.end();
}
// The wording of a key import that ended in a transport error: only the node's own HTTP-layer refusals say
// nothing was imported; a sent request with any other ending (a proxy's 502/504 while the node still runs
// the import, a dropped reply) says the key may be in and to run Rescan.
void testLongCallCutOff()
{
using dragonx::rpc::LongCallCutOff;
using dragonx::rpc::RpcFailureKind;
using dragonx::rpc::classifyLongCallCutOff;
const RpcFailureKind all[] = {
RpcFailureKind::None, RpcFailureKind::NotListening, RpcFailureKind::ConnectTimeout,
RpcFailureKind::ReplyTimeout, RpcFailureKind::QueueFull, RpcFailureKind::ShuttingDown,
RpcFailureKind::Unhandled, RpcFailureKind::Auth, RpcFailureKind::Warmup,
RpcFailureKind::AppError, RpcFailureKind::Aborted, RpcFailureKind::Deferred,
RpcFailureKind::Other,
};
for (RpcFailureKind k : all) {
for (bool sent : {false, true}) {
LongCallCutOff want;
if (k == RpcFailureKind::Aborted)
want = LongCallCutOff::Cancelled;
else if (k == RpcFailureKind::Auth || k == RpcFailureKind::ShuttingDown ||
k == RpcFailureKind::QueueFull || k == RpcFailureKind::Unhandled)
want = LongCallCutOff::Refused;
else
want = sent ? LongCallCutOff::Interrupted : LongCallCutOff::None;
const LongCallCutOff got = classifyLongCallCutOff(k, sent);
EXPECT_TRUE(got == want || (std::cerr << " classifyLongCallCutOff(" << dragonx::rpc::rpcFailureKindName(k)
<< ", sent=" << sent << ") = " << (int)got << "\n", false));
}
}
// End to end from the HTTP reply, as the import sees it.
using dragonx::rpc::classifyRpcFailure;
const auto viaHttp = [](long http, const std::string& body) {
return classifyLongCallCutOff(classifyRpcFailure(CURLE_OK, true, http, 0, body), true);
};
EXPECT_TRUE(viaHttp(502, "<html>Bad Gateway</html>") == LongCallCutOff::Interrupted);
EXPECT_TRUE(viaHttp(504, "Gateway Timeout") == LongCallCutOff::Interrupted);
EXPECT_TRUE(viaHttp(500, "Internal Server Error") == LongCallCutOff::Interrupted);
EXPECT_TRUE(viaHttp(401, "") == LongCallCutOff::Refused);
EXPECT_TRUE(viaHttp(503, "") == LongCallCutOff::Refused);
EXPECT_TRUE(viaHttp(500, "Work queue depth exceeded") == LongCallCutOff::Refused);
EXPECT_TRUE(viaHttp(500, "Unhandled request") == LongCallCutOff::Refused);
EXPECT_TRUE(classifyLongCallCutOff(classifyRpcFailure(CURLE_OPERATION_TIMEDOUT, true, 0, 0, ""), true) ==
LongCallCutOff::Interrupted);
EXPECT_TRUE(classifyLongCallCutOff(classifyRpcFailure(CURLE_COULDNT_CONNECT, false, 0, 0, ""), false) ==
LongCallCutOff::None);
EXPECT_TRUE(classifyLongCallCutOff(classifyRpcFailure(CURLE_ABORTED_BY_CALLBACK, true, 0, 0, ""), true) ==
LongCallCutOff::Cancelled);
}
#ifndef _WIN32
// A loopback JSON-RPC stub: answers each request delayMs after reading it (never, when delayMs < 0) with
// {"result":null,"error":null,"id":"x"}. One connection at a time; requests() counts those it has read.
class LoopbackRpcStub {
public:
explicit LoopbackRpcStub(int delayMs) : delay_ms_(delayMs)
{
fd_ = ::socket(AF_INET, SOCK_STREAM, 0);
sockaddr_in a{};
a.sin_family = AF_INET;
a.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
socklen_t len = sizeof(a);
if (fd_ < 0 || ::bind(fd_, reinterpret_cast<sockaddr*>(&a), sizeof(a)) != 0 || ::listen(fd_, 4) != 0 ||
::getsockname(fd_, reinterpret_cast<sockaddr*>(&a), &len) != 0)
return;
port_ = ntohs(a.sin_port);
thread_ = std::thread([this] { serve(); });
}
~LoopbackRpcStub()
{
stop_ = true;
if (thread_.joinable()) thread_.join();
if (fd_ >= 0) ::close(fd_);
}
int port() const { return port_; }
int requests() const { return requests_.load(); }
private:
void serve()
{
while (!stop_) {
pollfd lp{fd_, POLLIN, 0};
if (::poll(&lp, 1, 20) <= 0) continue;
const int c = ::accept(fd_, nullptr, nullptr);
if (c < 0) continue;
std::string req; // headers, then Content-Length bytes of body
for (;;) {
pollfd p{c, POLLIN, 0};
if (stop_ || ::poll(&p, 1, 20) < 0) break;
if (!(p.revents & POLLIN)) continue;
char buf[2048];
const ssize_t n = ::recv(c, buf, sizeof(buf), 0);
if (n <= 0) break;
req.append(buf, static_cast<size_t>(n));
const size_t h = req.find("\r\n\r\n");
const size_t cl = req.find("Content-Length:");
if (h != std::string::npos && cl != std::string::npos &&
req.size() >= h + 4 + std::strtoul(req.c_str() + cl + 15, nullptr, 10))
break;
}
++requests_;
const auto until = std::chrono::steady_clock::now() + std::chrono::milliseconds(std::max(delay_ms_, 0));
while (!stop_ && (delay_ms_ < 0 || std::chrono::steady_clock::now() < until))
std::this_thread::sleep_for(std::chrono::milliseconds(10));
if (!stop_) {
const std::string body = "{\"result\":null,\"error\":null,\"id\":\"x\"}";
const std::string resp = "HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: " +
std::to_string(body.size()) + "\r\nConnection: close\r\n\r\n" + body;
::send(c, resp.data(), resp.size(), MSG_NOSIGNAL); // the client may have given up: no SIGPIPE
}
::close(c);
}
}
int delay_ms_;
int fd_ = -1;
int port_ = 0;
std::atomic<bool> stop_{false};
std::atomic<int> requests_{0};
std::thread thread_;
};
dragonx::rpc::ConnectionConfig loopbackStubConfig(const LoopbackRpcStub& stub)
{
dragonx::rpc::ConnectionConfig cfg;
cfg.host = "127.0.0.1";
cfg.port = std::to_string(stub.port());
cfg.rpcuser = "user";
cfg.rpcpassword = "pass";
return cfg;
}
// lane.run() + a no-timeout "rescan" on a thread; returns whether it finished within `seconds` and how the call
// ended. A call still blocked after that (an abort was lost) is cut through the client directly so the test
// can finish, and counts as a failure.
struct LaneCallResult {
bool finishedInTime = false;
dragonx::rpc::RpcFailureKind kind = dragonx::rpc::RpcFailureKind::Other;
double seconds = 0.0;
};
LaneCallResult runLaneCall(dragonx::rpc::LongCallLane& lane, const dragonx::rpc::ConnectionConfig& cfg,
const std::function<void()>& whileRunning, double seconds)
{
using namespace dragonx::rpc;
RPCClient client;
std::atomic<bool> done{false};
std::atomic<int> kind{static_cast<int>(RpcFailureKind::Other)};
const auto t0 = std::chrono::steady_clock::now();
std::thread t([&] {
lane.run(client, cfg, [&](RPCClient& c) {
try {
c.call("rescan", json::array({0}), 0L);
kind = static_cast<int>(RpcFailureKind::None);
} catch (const std::exception& e) {
kind = static_cast<int>(rpcFailureKindOf(e));
}
});
done = true;
});
if (whileRunning) whileRunning();
LaneCallResult r;
r.finishedInTime = waitUntilTrue([&] { return done.load(); }, seconds);
r.seconds = secondsSince(t0);
if (!r.finishedInTime) client.requestAbort(); // rescue a lost abort so the test ends
t.join();
r.kind = static_cast<RpcFailureKind>(kind.load());
return r;
}
void testLongCallLaneAbortBeforeOpen()
{
using namespace dragonx::rpc;
LoopbackRpcStub stub(-1); // never answers
EXPECT_TRUE(stub.port() > 0);
LongCallLane lane;
EXPECT_TRUE(lane.tryBegin());
lane.abort(); // e.g. quit right after the task was submitted: open() must not erase it
const LaneCallResult r = runLaneCall(lane, loopbackStubConfig(stub), nullptr, 3.0);
EXPECT_TRUE(r.finishedInTime);
EXPECT_TRUE(r.kind == RpcFailureKind::Aborted);
lane.end();
EXPECT_FALSE(lane.active());
}
void testLongCallLaneAbortDuringCall()
{
using namespace dragonx::rpc;
LoopbackRpcStub stub(-1); // never answers
LongCallLane lane;
EXPECT_TRUE(lane.tryBegin());
double abortAt = -1.0;
const auto t0 = std::chrono::steady_clock::now();
const LaneCallResult r = runLaneCall(lane, loopbackStubConfig(stub), [&] {
// The request is in flight (the stub has read it), then the abort lands from this thread.
EXPECT_TRUE(waitUntilTrue([&] { return stub.requests() >= 1; }, 3.0));
std::this_thread::sleep_for(std::chrono::milliseconds(300));
abortAt = secondsSince(t0);
lane.abort();
}, 6.0);
EXPECT_TRUE(r.finishedInTime);
EXPECT_TRUE(r.kind == RpcFailureKind::Aborted);
EXPECT_TRUE(abortAt >= 0.0 && r.seconds - abortAt < 3.0); // returns within 3 s of the abort
lane.end();
}
void testLongCallNoTimeout()
{
using namespace dragonx::rpc;
{
LoopbackRpcStub stub(3000); // a "scan" that takes 3 s
const ConnectionConfig cfg = loopbackStubConfig(stub);
RPCClient lane; // ungated, like the lane's client
EXPECT_TRUE(lane.open(cfg.host, cfg.port, cfg.rpcuser, cfg.rpcpassword, false));
const auto t0 = std::chrono::steady_clock::now();
bool ok = false;
try {
ok = lane.call("rescan", json::array({0}), 0L).is_null(); // 0 = no client timeout
} catch (const std::exception& e) {
std::cerr << "no-timeout call threw: " << e.what() << "\n";
}
EXPECT_TRUE(ok);
EXPECT_TRUE(secondsSince(t0) >= 2.5);
// Control: a 1 s client timeout gives up on the same call (ReplyTimeout), and on a client that shares the
// busy gate it flips the gate to busy and abandons a daemon slot. Why the lane has no gate and no timeout.
auto gate = std::make_shared<RpcBusyGate>();
RPCClient gated;
gated.setBusyGate(gate);
EXPECT_TRUE(gated.open(cfg.host, cfg.port, cfg.rpcuser, cfg.rpcpassword, false));
RpcFailureKind kind = RpcFailureKind::None;
try {
gated.call("rescan", json::array({0}), 1L);
} catch (const std::exception& e) {
kind = rpcFailureKindOf(e);
}
EXPECT_TRUE(kind == RpcFailureKind::ReplyTimeout);
EXPECT_TRUE(gate->busy());
EXPECT_EQ(gate->abandoned(), 1);
}
{
// The console's callRaw(.., timeoutSec) override, and both restore the 30 s default afterwards.
LoopbackRpcStub stub(1500);
const ConnectionConfig cfg = loopbackStubConfig(stub);
RPCClient c;
EXPECT_TRUE(c.open(cfg.host, cfg.port, cfg.rpcuser, cfg.rpcpassword, false));
std::string raw;
try {
raw = c.callRaw("rescan", json::array({0}), 0L);
} catch (const std::exception& e) {
raw = std::string("threw: ") + e.what();
}
EXPECT_EQ(raw, std::string("null"));
RpcFailureKind kind = RpcFailureKind::None;
try {
c.callRaw("rescan", json::array({0}), 1L);
} catch (const std::exception& e) {
kind = rpcFailureKindOf(e);
}
EXPECT_TRUE(kind == RpcFailureKind::ReplyTimeout);
bool ok = false;
try {
ok = c.call("getinfo").is_null(); // the default timeout again (30 s), not the 1 s override
} catch (const std::exception& e) {
std::cerr << "default-timeout call threw: " << e.what() << "\n";
}
EXPECT_TRUE(ok);
}
}
#endif // !_WIN32
void testRpcKindClasses()
{
using namespace dragonx::rpc;
@@ -16566,6 +16878,13 @@ int main()
testRefreshScheduler();
testRefreshSchedulerDeferFor();
testRpcBusyGate();
testIsLongWalletMethod();
testLongCallCutOff();
#ifndef _WIN32
testLongCallLaneAbortBeforeOpen();
testLongCallLaneAbortDuringCall();
testLongCallNoTimeout();
#endif
testRpcKindClasses();
testConnectRetryBackoff();
testFetchBackoff();