From a8b5d2f6a3d392188486338954951ec2c5de9961 Mon Sep 17 00:00:00 2001 From: DanS Date: Thu, 4 Jun 2026 22:38:34 -0500 Subject: [PATCH] =?UTF-8?q?feat(lite):=20M2b-3=20=E2=80=94=20background=20?= =?UTF-8?q?refresh=20worker=20+=20App::update=20hook?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - LiteWalletController owns a background std::thread worker that, once a wallet is ready, refreshes every ~4s and publishes a copyable LiteWalletAppRefreshModel under a mutex. Worker auto-starts on lifecycle-ready and is stopped+joined in the destructor. status_ is written only on the main thread; walletOpen_/syncStarted_ are atomic. - App::update() calls takeRefreshedModel() and applies it into state_ on the main thread (WalletState is non-copyable, so the model crosses the thread boundary, not the state), so the existing Balance/Receive/Transactions tabs populate from lite data. - refreshWalletState() refactored onto refreshModel() (pure, worker-safe). - testLiteWalletControllerWorkerProducesModel verifies the worker publishes a populated model (stable across repeated runs). Builds clean in all configs. Real-backend smoke (lite_smoke --refresh now runs real output through the parsers) found two integration bugs, documented in the plan for follow-up: - syncstatus parser requires synced_blocks/total_blocks but the real idle response is {"syncing":"false"} (string), so it fails to parse when not actively syncing. - the first data query (balance/list) blocks on a full chain sync, which would hang the worker's shutdown join — needs a cancel/timeout path. Co-Authored-By: Claude Opus 4.8 (1M context) --- CMakeLists.txt | 1 + ...allet-implementation-plan-v2-2026-06-04.md | 7 +- src/app.cpp | 8 +++ src/wallet/lite_wallet_controller.cpp | 68 ++++++++++++++++--- src/wallet/lite_wallet_controller.h | 38 ++++++++++- tests/test_phase4.cpp | 34 ++++++++++ tools/lite_smoke.cpp | 49 +++++++++++++ 7 files changed, 190 insertions(+), 15 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 15194bf..dfe0d6e 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -733,6 +733,7 @@ if(DRAGONX_LITE_BACKEND_READY) src/wallet/lite_backend_artifact_contract.cpp src/wallet/lite_backend_artifact_resolver.cpp src/wallet/lite_connection_service.cpp + src/wallet/lite_result_parsers.cpp ) target_include_directories(lite_smoke PRIVATE ${CMAKE_SOURCE_DIR}/src diff --git a/docs/lite-wallet-implementation-plan-v2-2026-06-04.md b/docs/lite-wallet-implementation-plan-v2-2026-06-04.md index 89d5c81..9f508ab 100644 --- a/docs/lite-wallet-implementation-plan-v2-2026-06-04.md +++ b/docs/lite-wallet-implementation-plan-v2-2026-06-04.md @@ -95,7 +95,12 @@ Each milestone is independently demoable and gated by a fake-backend test. Order > - ℹ️ The fetch/parse/assemble pipeline already exists and works: `LiteWalletGateway::refresh()` → `LiteWalletRefreshBundle` → `mapLiteWalletRefreshBundle()` → `LiteWalletAppRefreshModel`. M2 just needed the final `→ WalletState` hop (above) plus live wiring. > - ✅ **M2b-1 — shared-bridge refactor (done).** `litelib` is a global singleton and every `LiteClientBridge` calls `litelib_shutdown()` on destruction, so services must not each own one. `LiteWalletLifecycleService`, `LiteWalletGateway`, and `LiteSyncService` now take a **non-owning `LiteClientBridge*`**; `LiteWalletController` owns the single bridge and passes `&bridge_`. Builds clean in all configs; existing tests stay green. > - ✅ **M2b-2 — sync + controller refresh (done + tested).** `LiteSyncService::startSync` now executes the `sync` command (was a stub). `LiteWalletController` gained `startSync()` (auto-invoked when a wallet becomes ready) and `refreshWalletState(WalletState&)` which polls `syncstatus`, runs `gateway.refresh()`, maps the bundle, and applies it into `WalletState`. `testLiteWalletControllerRefreshPopulatesState()` drives the full path against the real-shape fake (balances/addresses/transactions/sync populated; no-op when no wallet open). The fake harness now returns command-shaped JSON per `tests/fixtures/lite/result_parsers.json`. (Surfaced a real bug: `info` requires `latest_block_height`, and the gateway aborts the whole refresh on the first command's parse failure — fixed in the fake; worth noting the gateway's abort-on-first-failure is fragile against partial backend responses.) -> - ⏳ **M2b-3 — threaded App hook (remaining).** Drive `refreshWalletState` from a controller-owned background worker (mirror `NetworkRefreshService`/`RefreshScheduler`): worker produces a copyable `LiteWalletAppRefreshModel`; `App::update()` applies it into `state_` on the main thread (WalletState is non-copyable, so pass the model, not the state, across threads). `litelib_execute` is `catch_unwind`-safe so the worker won't abort the app. Also: per-address balances (notes-correlation) and a real-backend refresh smoke (the real backend's JSON shapes may differ from the hand-built fixture). +> - ✅ **M2b-3 — threaded App hook (done + tested).** `LiteWalletController` owns a background worker (`std::thread`) that, once a wallet is ready, refreshes every ~4s and publishes a copyable `LiteWalletAppRefreshModel` under a mutex; `App::update()` calls `takeRefreshedModel()` and applies it into `state_` on the main thread (WalletState is non-copyable, so the model crosses the thread boundary, not the state). Worker auto-starts on lifecycle-ready and is stopped+joined in the controller destructor. `status_` is written only on the main thread to avoid races; `walletOpen_`/`syncStarted_` are atomic. `testLiteWalletControllerWorkerProducesModel()` opens a wallet and asserts the worker publishes a populated model (stable across repeated runs). Builds clean in all configs. +> **Real-backend refresh smoke (2026-06-04): ran `lite_smoke --create --refresh` against the live backend — found two real bugs** the fake/fixture couldn't (smoke now links `lite_result_parsers` and runs each command's real output through the parser): +> 1. **OPEN — `syncstatus` parser mismatch.** `parseLiteSyncStatusResponse` hard-requires `synced_blocks`/`total_blocks`, but the real backend (per `commands.rs:83-87`) returns **idle = `{"syncing":"false"}`** (string!) and only **in-progress = `{"syncing":"true","synced_blocks":N,"total_blocks":M}`**. So syncstatus fails to parse whenever the wallet isn't actively syncing → sync/progress never updates in the real app. Fix: read `syncing` as a string, make the block fields optional. (info/balance/addresses parsers verified OK against real output.) +> 2. **OPEN — first data query blocks on a full chain sync.** `execute("balance"/"list")` on a fresh wallet triggers a synchronous multi-million-block sync (observed "Syncing 1.76M/2.99M…"). On the M2b-3 worker thread that means the controller's destructor `join()` would hang at app shutdown. Needs: a cancel/timeout path for in-flight refresh (e.g., don't block shutdown on the worker), and likely gating data fetches until sync has progressed. **This is the main blocker for a usable real lite wallet** and should lead M2 polish / M3. +> +> - ⏳ **Remaining for M2 polish:** fix the syncstatus parser (above), address the blocking-sync/worker-shutdown issue (above), per-address balances (notes-correlation; currently aggregate-only), and harden the gateway's abort-on-first-failure (skip-and-continue per command). - Implement `LiteSyncService::startSync` (replace the "not implemented" stub) + a background worker polling `syncstatus`, mirroring `NetworkRefreshService`/`RefreshScheduler` (enqueue → worker → apply on main thread). - Drive `LiteWalletGateway` refresh (info/height/balance/addresses/notes/list/transactions) through `lite_result_parsers` → `lite_wallet_state_mapper` → `App` `WalletState` (`privateBalance`, `transparentBalance`, `addresses`, `transactions`, `sync`). - Hook the controller into `App::update()`'s refresh dispatch alongside (not inside) the full-node path. diff --git a/src/app.cpp b/src/app.cpp index a979e44..22ef018 100644 --- a/src/app.cpp +++ b/src/app.cpp @@ -390,6 +390,14 @@ void App::update() if (fast_worker_) { fast_worker_->drainResults(); } + + // Apply any lite-wallet refresh the controller's background worker produced (main thread). + if (lite_wallet_) { + wallet::LiteWalletAppRefreshModel liteModel; + if (lite_wallet_->takeRefreshedModel(liteModel)) { + wallet::applyLiteRefreshModelToWalletState(liteModel, state_); + } + } async_tasks_.reapCompleted(); // Auto-lock check (only when connected + encrypted + unlocked) diff --git a/src/wallet/lite_wallet_controller.cpp b/src/wallet/lite_wallet_controller.cpp index c94ff4c..154129b 100644 --- a/src/wallet/lite_wallet_controller.cpp +++ b/src/wallet/lite_wallet_controller.cpp @@ -6,6 +6,7 @@ #include "../data/wallet_state.h" +#include #include #include @@ -105,6 +106,11 @@ LiteWalletController::LiteWalletController(WalletCapabilities capabilities, status_ = lifecycle_.status(); } +LiteWalletController::~LiteWalletController() +{ + stopWorker(); +} + std::unique_ptr LiteWalletController::createLinked( WalletCapabilities capabilities, LiteConnectionSettings connectionSettings) @@ -122,7 +128,8 @@ void LiteWalletController::onLifecycleResult(const LiteWalletLifecycleResult& re if (result.walletReady) { walletOpen_ = true; if (persist_) persist_(); - startSync(); // begin background sync now that a wallet is ready + startSync(); // begin background sync on the backend + startWorker(); // begin periodic refresh -> WalletState (via takeRefreshedModel) } } @@ -133,9 +140,9 @@ LiteSyncStartResult LiteWalletController::startSync() return result; } -bool LiteWalletController::refreshWalletState(dragonx::WalletState& state) +std::optional LiteWalletController::refreshModel() { - if (!walletOpen_) return false; + if (!walletOpen_.load()) return std::nullopt; // Poll sync status first so the refresh bundle (and the mapped sync model) carries it. LiteWalletRefreshRequest request; @@ -147,21 +154,60 @@ bool LiteWalletController::refreshWalletState(dragonx::WalletState& state) const auto refreshResult = gateway_.refresh(request); if (refreshResult.bundle.successfulCommandCount == 0 && !request.haveSyncStatus) { - status_ = refreshResult.status; - return false; + return std::nullopt; } const auto mapped = mapLiteWalletRefreshResult(refreshResult); - if (!mapped.ok) { - status_ = refreshResult.status; - return false; - } + if (!mapped.ok) return std::nullopt; + return mapped.model; +} - applyLiteRefreshModelToWalletState(mapped.model, state); - status_ = refreshResult.status; +bool LiteWalletController::refreshWalletState(dragonx::WalletState& state) +{ + auto model = refreshModel(); + if (!model) return false; + applyLiteRefreshModelToWalletState(*model, state); return true; } +bool LiteWalletController::takeRefreshedModel(LiteWalletAppRefreshModel& out) +{ + std::lock_guard lock(modelMutex_); + if (!pendingModel_) return false; + out = std::move(*pendingModel_); + pendingModel_.reset(); + return true; +} + +void LiteWalletController::startWorker() +{ + if (running_.exchange(true)) return; // already running + worker_ = std::thread([this] { workerLoop(); }); +} + +void LiteWalletController::stopWorker() +{ + if (!running_.exchange(false)) return; // not running + wakeCv_.notify_all(); + if (worker_.joinable()) worker_.join(); +} + +void LiteWalletController::workerLoop() +{ + while (running_.load()) { + if (walletOpen_.load()) { + auto model = refreshModel(); + if (model) { + std::lock_guard lock(modelMutex_); + pendingModel_ = std::move(model); + } + } + std::unique_lock lock(wakeMutex_); + wakeCv_.wait_for(lock, std::chrono::milliseconds(kRefreshIntervalMs), + [this] { return !running_.load(); }); + } +} + LiteWalletLifecycleResult LiteWalletController::createWallet(LiteWalletCreateRequest request) { auto result = lifecycle_.createWallet(request); diff --git a/src/wallet/lite_wallet_controller.h b/src/wallet/lite_wallet_controller.h index f7cd605..beafe44 100644 --- a/src/wallet/lite_wallet_controller.h +++ b/src/wallet/lite_wallet_controller.h @@ -25,9 +25,14 @@ #include "wallet_backend.h" #include "wallet_capabilities.h" +#include +#include #include #include +#include +#include #include +#include namespace dragonx { struct WalletState; // data/wallet_state.h @@ -55,6 +60,11 @@ public: LiteClientBridge bridge, LiteWalletControllerOptions options = LiteWalletControllerOptions{}); + ~LiteWalletController(); // stops + joins the background refresh worker + + LiteWalletController(const LiteWalletController&) = delete; + LiteWalletController& operator=(const LiteWalletController&) = delete; + // Production factory: links the SDXL backend compiled into this build. static std::unique_ptr createLinked( WalletCapabilities capabilities, @@ -82,19 +92,41 @@ public: // Poll sync status + fetch balance/addresses/transactions, and apply the result into the // app's WalletState. Returns true if state was updated. Safe no-op when no wallet is open. + // Synchronous (blocks on the backend); used by tests and as the worker's unit of work. bool refreshWalletState(dragonx::WalletState& state); + // Synchronous refresh that returns the mapped model (or nullopt). Pure w.r.t. shared + // state; safe to call from the background worker. nullopt when no wallet is open or the + // refresh produced nothing usable. + std::optional refreshModel(); + + // Main-thread handoff: if the background worker has produced a fresh model since the last + // call, move it into `out` and return true. Apply it with applyLiteRefreshModelToWalletState. + bool takeRefreshedModel(LiteWalletAppRefreshModel& out); + private: void onLifecycleResult(const LiteWalletLifecycleResult& result); + void startWorker(); + void stopWorker(); + void workerLoop(); LiteClientBridge bridge_; // the single owned bridge; services below borrow &bridge_ LiteWalletLifecycleService lifecycle_; LiteWalletGateway gateway_; LiteSyncService sync_; std::function persist_; - bool walletOpen_ = false; - bool syncStarted_ = false; - WalletBackendStatus status_; + std::atomic walletOpen_{false}; + std::atomic syncStarted_{false}; + WalletBackendStatus status_; // written only on the main thread (lifecycle ops) + + // Background refresh worker. + std::thread worker_; + std::atomic running_{false}; + std::mutex wakeMutex_; + std::condition_variable wakeCv_; + std::mutex modelMutex_; + std::optional pendingModel_; // guarded by modelMutex_ + static constexpr int kRefreshIntervalMs = 4000; }; } // namespace wallet diff --git a/tests/test_phase4.cpp b/tests/test_phase4.cpp index 709f36b..a114908 100644 --- a/tests/test_phase4.cpp +++ b/tests/test_phase4.cpp @@ -31,6 +31,7 @@ #include #include +#include #include #include #include @@ -4650,6 +4651,38 @@ void testLiteWalletControllerRefreshPopulatesState() EXPECT_FALSE(fresh.refreshWalletState(empty)); } +// M2b-3: opening a wallet auto-starts the background worker, which produces a refresh model +// the main thread can pick up via takeRefreshedModel() and apply to WalletState. +void testLiteWalletControllerWorkerProducesModel() +{ + using namespace dragonx::wallet; + const auto caps = makeWalletCapabilities(WalletBuildKind::Lite, /*embeddedDaemon*/ false, /*liteBackendLinked*/ true); + const auto conn = defaultLiteConnectionSettings(); + + LiteWalletController controller(caps, conn, LiteClientBridge::fromApi(dragonx::test::makeFakeLiteApi())); + EXPECT_TRUE(controller.createWallet(LiteWalletCreateRequest{}).walletReady); // auto-starts the worker + + // The worker refreshes immediately on start; poll briefly (<=2s) for the produced model. + LiteWalletAppRefreshModel model; + bool got = false; + for (int i = 0; i < 200 && !got; ++i) { + got = controller.takeRefreshedModel(model); + if (!got) std::this_thread::sleep_for(std::chrono::milliseconds(10)); + } + EXPECT_TRUE(got); + EXPECT_TRUE(model.hasBalance); + EXPECT_TRUE(model.hasAddresses); + + dragonx::WalletState state; + applyLiteRefreshModelToWalletState(model, state); + EXPECT_NEAR(state.privateBalance, 2.0, 1e-9); + + // Idle controller (no wallet -> no worker) has nothing pending. + LiteWalletController idle(caps, conn, LiteClientBridge::fromApi(dragonx::test::makeFakeLiteApi())); + LiteWalletAppRefreshModel none; + EXPECT_FALSE(idle.takeRefreshedModel(none)); +} + } // namespace int main() @@ -4687,6 +4720,7 @@ int main() testLiteChainNameMigration(); testLiteRefreshModelAppliesToWalletState(); testLiteWalletControllerRefreshPopulatesState(); + testLiteWalletControllerWorkerProducesModel(); testLiteBridgeRuntimeShutdownIsIdempotent(); testLiteBridgeRuntimeDestructorCallsShutdownOnce(); testLiteBridgeRuntimeShutdownWaitsForOwnedStringRelease(); diff --git a/tools/lite_smoke.cpp b/tools/lite_smoke.cpp index 1b5394e..172e915 100644 --- a/tools/lite_smoke.cpp +++ b/tools/lite_smoke.cpp @@ -15,6 +15,7 @@ #include "wallet/lite_client_bridge.h" #include "wallet/lite_connection_service.h" +#include "wallet/lite_result_parsers.h" #include #include @@ -22,12 +23,17 @@ int main(int argc, char** argv) { using namespace dragonx::wallet; + std::setvbuf(stdout, nullptr, _IONBF, 0); // unbuffered so output survives a timeout kill std::string server = "https://lite.dragonx.is"; bool doCreate = false; + bool doRefresh = false; + bool doFull = false; for (int i = 1; i < argc; ++i) { const std::string arg = argv[i]; if (arg == "--create") doCreate = true; + else if (arg == "--refresh") doRefresh = true; + else if (arg == "--full") doFull = true; else server = arg; } @@ -61,6 +67,49 @@ int main(int argc, char** argv) } } + if (doRefresh) { + // Run each refresh command through the real parser to verify the LIVE backend's + // JSON shapes match what the gateway expects. Prints flags/counts only (no secrets, + // addresses, or amounts). + std::printf("[lite-smoke] --- refresh shape check (real backend output -> parsers) ---\n"); + { + auto r = bridge.execute("info", ""); + auto p = parseLiteInfoResponse(r.value); + std::printf("[lite-smoke] info bridge_ok=%d parse_ok=%d err=%s\n", + r.ok, p.ok, liteResultParserErrorName(p.error)); + } + { + auto r = bridge.execute("syncstatus", ""); + auto p = parseLiteSyncStatusResponse(r.value); + std::printf("[lite-smoke] syncstatus bridge_ok=%d parse_ok=%d err=%s synced=%llu/%llu\n", + r.ok, p.ok, liteResultParserErrorName(p.error), + (unsigned long long)p.syncStatus.syncedBlocks, (unsigned long long)p.syncStatus.totalBlocks); + std::printf("[lite-smoke] syncstatus RAW = %.200s\n", r.value.c_str()); + } + // The data commands below trigger a blocking full-chain sync on a fresh wallet; + // gate them behind --full so the shape check stays fast by default. + if (!doFull) { bridge.shutdown(); std::printf("[lite-smoke] (skipping balance/addresses/list; pass --full)\n"); return 0; } + { + auto r = bridge.execute("balance", ""); + auto p = parseLiteBalanceResponse(r.value); + std::printf("[lite-smoke] balance bridge_ok=%d parse_ok=%d err=%s\n", + r.ok, p.ok, liteResultParserErrorName(p.error)); + } + { + auto r = bridge.execute("addresses", ""); + auto p = parseLiteAddressesResponse(r.value); + std::printf("[lite-smoke] addresses bridge_ok=%d parse_ok=%d err=%s z=%zu t=%zu\n", + r.ok, p.ok, liteResultParserErrorName(p.error), + p.addresses.zAddresses.size(), p.addresses.tAddresses.size()); + } + { + auto r = bridge.execute("list", ""); + auto p = parseLiteTransactionsResponse(r.value); + std::printf("[lite-smoke] list bridge_ok=%d parse_ok=%d err=%s count=%zu\n", + r.ok, p.ok, liteResultParserErrorName(p.error), p.transactions.transactions.size()); + } + } + bridge.shutdown(); std::printf("[lite-smoke] done (real litelib_* symbols are callable; results above are live)\n"); return 0;