a5db2e9a3d44d205c632c0675d4eef968041af0d
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  DanSandClaude Opus 5.5
|
a5db2e9a3d |
fix(lite-backend): never hang or panic a sync on malformed server data
The lightwalletd server controls the compact ciphertext length, the block, prev-block and tx hash lengths, and (through the chain) the full txs' epks. Each of these could wedge or poison a sync: - try_sapling_compact_note_decryption asserts a 52-byte ciphertext. A shorter (or full 580-byte) one panicked a pool job in scan_output_internal while the original sender stayed alive, so the collector waited forever with sync_lock, txs.write() and a wallet read guard held. - scan_taddress_txids kept its own sender alive too: one panicking t-address job (scan_full_tx) left crx.iter().take(n) blocked forever. - CompactBlock::hash()/prev_hash() assert 32 bytes when no header is sent, and every vtx hash is copied into a [u8; 32], one of them under txs.write(): a 31-byte tx hash poisoned the wallet for good, and a bad hash in the block's txid list panicked after the witnesses had moved. - scan_full_tx and scan_full_mempool_tx unwrapped as_prime_order() on the epk. Consensus rejects only a small-order epk, so a mixed-order one can be mined: an own tx then failed every sync, and the mempool receiver thread died silently. Changes: - append_leaf(node: Node, ..) commits a leaf to the existing, block and new witnesses and the tree; the epk-error path, the new short-ciphertext path and the normal path all use it (a behaviour-neutral extraction). - A compact ciphertext under 52 bytes is skipped with its cmu still committed; a longer one is truncated to 52 bytes, which is exact (the compact plaintext is those bytes and the cmu re-check validates it). - Each decryption job catches a panic and sends exactly one message; drop(tx) after the dispatch, and the collector logs a vanished or panicked job and treats the output as not ours. - scan_taddress_txids runs through the private seam scan_taddress_txids_with(job): catch_unwind per job, drop(ctx), and a count check, so a dead job is an error, never a hang or a silent partial success. - scan_block_with_pool checks the block, prev-block and vtx hash lengths right after the parse and returns Err(-1), as for an unparsable block, before any change (Err(height) would refetch the same block forever under sync_lock). - scan_full_tx and scan_full_mempool_tx skip an output whose epk is not in the prime-order subgroup (one check per output, used by every decryption and the outgoing recovery). - The mempool receiver thread catches a panic of scan_full_mempool_tx. Flagged behaviour changes: - a compact ciphertext under 52 bytes is skipped (its cmu is still committed); one over 52 bytes is truncated; - pool-job panics are reported (logged; the output is treated as not ours, which a rescan repairs) instead of hanging the sync; - bad hash lengths return Err(-1) before any change; - outputs with a non-prime-order epk are skipped; - the mempool thread survives a bad tx. Wallet format v9 is unchanged; no ABI or symbol change. Honest data takes the same path as before (an old-vs-new run over generated honest chains is byte-identical). Tests (each on its own thread under a 15 s watchdog): 0/51/52/53/580-byte ciphertexts in one block on the inline and deferred witness paths (the 51-byte notes are skipped, the 53- and 580-byte notes are found, the tree and every witness match the reference); a 31-byte tx hash, an empty block hash, a 31-byte prev hash and a 33-byte foreign tx hash each return Err(-1) with the txs lock usable and last_scanned_height, trees and witnesses unchanged; a mixed-order epk in scan_full_tx and scan_full_mempool_tx does not panic and decrypts nothing while the real epk still decrypts; a t-address job panicking for 1 of 3 addresses is an Err. The oracle generator gains short (0..=51-byte) and full (580-byte) ciphertext outputs without an extra random draw, so each seed keeps its sequence of blocks, values and keys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
20b63bf45d |
fix(lite-backend): validate WIFs, refuse key import while encrypted
import_taddr decoded a WIF with bs58::decode(..).unwrap(), a [2..66] slice of its hex and SecretKey::from_slice(..).unwrap(), all under the wallet write guard that do_import_tk holds. A z-key, a typo or any short string panicked there and poisoned the wallet until restart, and the checksum was never verified, so a mistyped WIF that was still valid base58 imported a different key. A private key imported into an encrypted wallet was stored without an enc_key. The import then failed in do_save's lock(), which had already zeroed the seed while the wallet stayed "unlocked": seed export showed the all-zero-entropy phrase. - decode_taddr_wif replaces the old decode: Base58Check with the checksum verified, version 0xBC (the config prefix) or 0x80, and the 0x01 compressed flag. It decodes into a buffer it owns and wipes it with sodiumoxide::utils::memzero on every path, including a failed checksum. Errors are fixed strings that never contain the key. - import_taddr and add_imported_sk refuse while the wallet is encrypted, as add_taddr and add_zaddr already do. - do_import_tk and do_import_sk check is_encrypted() before the "locked" check, so a locked wallet does not send the user to unlock and retry. Flagged behaviour changes: - Base58Check, version 0xBC/0x80 and the compressed flag are required; - uncompressed and foreign-network WIFs are rejected with fixed messages (an uncompressed key used to import under the compressed address, where its funds are not); - no panic and no poisoned lock; - private-key import (t and z) is refused while the wallet is encrypted, with a reason that says to remove encryption, import, then encrypt again; viewing keys still import; - the key is never echoed. No ABI, symbol or wallet format change. Tests: garbage, z-keys, bad checksums, a transposed or flipped character, a zero scalar, a wrong flag, uncompressed, 0x05 and 0xEF WIFs are rejected with their message and leave tkeys usable; the wallet's own export, a 0x80 WIF and the published key-1 vector still import; the encrypted refusal in the wallet and the client, locked and unlocked, leaves lock, save and seed export intact; a viewing key still imports into an encrypted wallet; a bad timport leaves the wallet lock unpoisoned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
026d0e92a6 |
fix(lite-backend): serialize clear and rescan with a running sync
clear_state_from wiped the wallet's blocks, txs and mempool maps without taking sync_lock. The console `clear` and `rescan` commands run concurrently with the refresh worker's sync (litelib_execute drops the client mutex before a command runs), so a clear could land after a sync had already taken its nullifier snapshot. scan_block then unwrapped a note that was no longer in the wallet and panicked under the txs write guard, which poisons the wallet until restart. - clear_state_from now takes sync_lock first (lock order sync_lock -> wallet, as in do_save and do_sync). No caller holds sync_lock, so it cannot self-deadlock; do_rescan holds no lock when it clears. - clear_state reads the birthday into a local before calling clear_state_from. Before, the wallet read guard was a temporary that lived across the call, which read-locks the wallet again: with a writer (a save, an import, a new address) queued in between, the writer-preferring RwLock on Linux and Windows could deadlock. Holding it while waiting for sync_lock would also have inverted the lock order against do_save. - scan_block resolves each spent note with and_then instead of three unwraps. A spend of a note that is no longer in the wallet logs a warning and is skipped. Flagged behaviour changes: - console clear/rescan and the GUI rescan wait for an in-flight sync or save to finish instead of racing it; - the recursive wallet read lock that could deadlock a concurrent save is gone; - a spend of a vanished note now warns and is skipped instead of panicking; it is counted after the next rescan. No ABI, symbol or wallet format change. Tests: clear_state_from waits for sync_lock, clear_state holds no wallet guard while it waits, neither leaks the lock, and a 200x clear/write loop finishes within a 20 s watchdog (a regression guard only). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
0a72b47ca6 |
fix(lite-backend): verify TLS for the t-address scan and mempool monitor
The transparent-address txid scan in the sync loop and the mempool monitor both hard-coded no_cert = true, so they connected to the lightwalletd through get_client's NoCertificateVerification path in every Lite build, while the app always passes dangerous = false. Block sync, info, broadcast and the full-tx fetcher already honour LightClientConfig::no_cert_verification. An active man in the middle on those two channels could hide or invent transparent balance, UTXOs and pending transactions (signing stays local, so no funds can be taken). Both calls now take the setting from the config: - the sync loop passes self.config.no_cert_verification to scan_taddress_txids; - start_mempool_monitor reads config.no_cert_verification once and passes it to monitor_mempool. Flagged behaviour change: a lightwalletd with an invalid or self-signed certificate now fails the t-address stage (do_sync retries, then returns the error) and the mempool monitor (it logs and retries), as block sync already does. `dangerous` is the only bypass. No ABI, symbol or wallet format change. A tripwire test checks that lightclient.rs never again hard-codes the bypass for either call; its needles are joined at runtime so the test cannot match its own source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
c233d4d503 |
perf(lite-backend): fetch full transactions over one shared gRPC channel
At the end of a sync the wallet fetches the full transaction of each new wallet tx, together with every other transaction in the blocks that contain them, as decoys. Every txid got its own pool job that built a fresh Tokio runtime (one thread per CPU), a new rustls config and a new TCP + TLS + HTTP/2 connection for a single GetTransaction call, so a restore or rescan opened one connection per fetched transaction. The first error returned from the sync while the remaining jobs kept fetching and scanning in the background, and a stalled connection, an unparsable transaction or a panicking scan could hang the sync forever with its lock held. fulltx_fetch.rs now works through the same (txid, height)-sorted, de-duplicated list on the sync's runtime, with as many workers as the sync pool has threads (2-8, the same fan-out as before), over one gRPC channel that is dialed lazily and re-dialed on retry. Parsing and scanning stay on the sync pool, and the call returns once the whole batch has finished. Behaviour changes: - a failure on a decoy no longer fails the sync; it fails only when one of the wallet's own txs could not be fetched and scanned; - new error texts "Error reading transaction: ...", "full tx scan job panicked" and "fetched only X of Y wallet txs", all after the existing "Error fetching all txns for memos: " prefix; - each tx gets up to 3 fetch attempts with a 120 s timeout each, where the old code could wait forever; a request that legitimately takes longer now fails; - nothing keeps running in the background after the sync returns; - all GetTransaction calls share one TLS/HTTP-2 connection (up to one dial per worker at the start, plus a re-dial per retry); the request list and its order are unchanged; - if the network silently drops packets mid-batch, every attempt waits out its timeout until 16 consecutive failed txs stop the batch: up to about 12 minutes with 8 workers (48 minutes with 2) with the sync lock held, where the old code gave up at its first failed connection. Which txs are requested, in what order, how often they are retried and when the batch stops do not depend on which txs are the wallet's own; only the final result does. A server that keeps failing one tx can see that result through the sync's retry, but it could already tell, because an own tx that failed is requested again on the next sync without its decoys; nothing new leaks. tests/fulltx_fetch.rs drives the engine with a fake fetcher: the old request list, each item fetched and scanned once, bounded concurrency, a request sequence independent of ownership, retries, timeouts, the breaker, panicking and unparsable scans, and a saved wallet byte-identical to the old serial fill. The gRPC fetcher itself is not covered by these tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
6de9b5cc90 |
perf(lite-backend): borrow embedded Sapling params instead of copying
Every LightClient copied the Sapling spend and output parameters, which rust-embed already compiles into the binary, onto the heap: 51,551,256 bytes (49.2 MiB) of private memory per live client, whether or not it ever sent anything. LightClient::sapling_output and sapling_spend are now Cow<'static, [u8]> and borrow the embedded image. set_sapling_params still stores an owned copy, with its is-empty guard, hash checks and disk write unchanged. The prover receives the same bytes. Behaviour changes: - about 49 MiB less anonymous memory per live LightClient; the wallet file and JSON are unchanged; - the first send or shield pages the 51.5 MB of embedded params in from the binary, where client creation used to copy them (an estimated 0.3-1 s from the AppImage); - the public fields sapling_output and sapling_spend change type from Vec<u8> to Cow<'static, [u8]>: a Rust API change, with no C ABI or exported-symbol change. tests/sapling_params.rs checks that a restored, a new and a reloaded client all borrow the embedded image (same pointer, pinned lengths and SHA-256), that set_sapling_params keeps its guard and hash checks, and that the borrowed params parse like a heap copy. An ignored test measures 576 KiB of RssAnon growth for a second client, against 46.5 MiB with the old heap copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
bd4c06dfd0 |
perf(lite-backend): encode note addresses from pk_d instead of deriving
note_address() re-derived a note's payment address from its viewing key and diversifier (a group hash, an ivk hash and a 252-bit scalar multiplication) for every note in `notes` and `list` and for each new note in the block scan. The z-balance sums did the same derivation in their address filter, so `balance`, which sums four balances for each of Z z-addresses, cost Z x (N + 2U) derivations for N notes, U of them unspent. do_list_notes also encoded up to Z + Z^2 addresses to find the spendable ones and cloned that set once per transaction. Now: - note_address() encodes the note's stored pk_d with its diversifier. Both decryption and wallet load build that pk_d from the same key and diversifier, so the string is byte-identical; - the three z-balance sums share one address filter. The per-address sums behind `balance` compare (diversifier, pk_d) with each z-address directly; any other address string keeps the old exact string compare; - those sums work from a copy of the z-addresses whose lock is released before any note is read, so the txs-then-zkeys lock order that sync uses is unchanged; - do_list_notes builds the spendable-address set with Z encodes and borrows it. No output changes: balances and the notes/list/balance JSON are byte-identical. The address filter no longer contains an unwrap (it could not fail). Tests compare against verbatim copies of the old functions over randomized chains (note addresses, also after a reload; every filtered balance for own, diversified, foreign and junk address strings; the per-address balances; the spendable set), and a concurrency test fails with a deadlock if the zkeys lock is held across the sums. A differential run against the unmodified backend compared per-address balances and the JSON after every operation, through reorgs and reloads, and the old build re-saves the new files byte for byte. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
8895038c40 |
test(lite-backend): port the core-crate tests to the current API
The lite backend core's in-crate tests no longer compiled (119 errors in
lightwallet/tests.rs and the lightclient and walletzkey test modules):
they still used the old LightWallet::new and send_to_address signatures,
the removed extfvks/extsks/zaddress fields and the old new_from_phrase
and do_send arities. So `cargo test` built nothing and no backend change
could be checked against them.
Port them, and the commands and checkpoints tests, to the current API,
touching only #[cfg(test)] code:
- LightWallet::new takes 4 arguments, send_to_address 7 (with an
explicit default fee, returning (txid, raw tx)), new_from_phrase 5 and
do_send 2; keys are read through zkeys and tkeys;
- every config uses the "main" chain: creating a wallet on the "test"
chain panics in base58_pubkey_address, so LightClient::unconnected
(which hard-codes "test") is no longer used;
- every LightClient lives in a temporary data dir, so a test cannot
write to the real ~/.silentdragonxlite or ~/.zcash-params;
- ten send_to_address recipient lists garbled by an upstream change in
2020 are restored from the upstream history.
Expectations that contradicted deliberate current behaviour are updated;
none is ignored:
- a pending send's mempool entry sits at the tip, not tip+1, and
zbalance no longer counts a note with a pending spend;
- keys cannot be added to an encrypted wallet, so the encrypted-receive
tests derive their second address before encrypting;
- the t-address vectors are m/44'/141'/0'/0/{0,1}, checked against an
independent BIP32 derivation; the z-address vectors record the current
output;
- test_addresses and test_checkpoints follow the current address count
and checkpoint tables.
All 43 tests pass on the unchanged production code (three runs) and
leave the real wallet and params directories untouched; that list is the
regression baseline for the following commits. They cover balances,
address derivation, bookkeeping, the mempool, encryption, serialization
and rollback, but none verifies a proof, an anchor or a witness.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DanSandClaude Opus 4.8
|
1393d9ae1a |
feat(lite): vendor SDXL backend source; source deps from git.dragonx.is
Vendor the two local Rust crates that build the lite backend artifact into third_party/silentdragonxlite/ (the qtlib C-ABI wrapper + the silentdragonxlitelib core, with proto/res and all the lite-send fixes), and point build-lite-backend-artifact.sh's default --backend-dir there, so the lite wallet builds without the upstream SilentDragonXLite repo. External build inputs are now only the Rust toolchain + git.dragonx.is + crates.io: - the 6 librustzcash git deps point at the git.dragonx.is/DragonX/librustzcash mirror (pinned rev acff1444), not git.hush.is; - the Sapling params are gitignored (not committed, no Git LFS) - the build fetches them from the git.dragonx.is/DragonX/zcash-params 'sapling-v1' release and verifies their SHA-256 before rust-embed bakes them in (ensure_sapling_params). For fully offline builds, cargo vendor into lib/vendor/ and add a vendored-sources redirect (vendor/ is gitignored; the script symlinks it into the prepared dir). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |