39292b1b1f1abbbe58bcbc69816e96e9a857f0bb
16
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  DanSandClaude Opus 4.8
|
39292b1b1f |
feat(litebe): salted Argon2id KDF for new encrypted wallets
New Lite wallets now encrypt their seed and keys with a salted Argon2id key instead of the unsalted double-SHA256 of the passphrase, which was cheap to brute-force from a stolen wallet file. - A shared derive_wallet_key() replaces double-SHA256 across encrypt, unlock, remove-encryption and seed recovery. KDF 0 is the old double-SHA256 (unchanged); KDF 1 is Argon2id (moderate: 3 passes, 256 MiB), using sodiumoxide's existing pwhash -- no new dependency. - The KDF id, salt and parameters are stored in the wallet. A wallet is written at version 10 only while it is actually Argon2id-encrypted; every pre-existing wallet (plaintext or double-SHA256) is read and written byte-for-byte as version 9. - New-wallets-only: a freshly created or restored wallet encrypts with Argon2id; a wallet read from disk never changes its KDF, and removing encryption never changes it. Set DRAGONX_LITE_KDF=legacy to keep new wallets on double-SHA256. - An older build (or SilentDragonXLite) opening a version-10 wallet fails cleanly with "don't know how to read wallet version 10" before parsing anything, never a silent misread; the seed phrase stays the cross-wallet recovery path. Compatibility break (by design, opt-in): a wallet newly encrypted with Argon2id cannot be opened by older ObsidianDragon or SilentDragonXLite. Tests: the existing encryption tests now exercise the Argon2id path, plus the version-10 header, legacy byte-identity and derive_wallet_key parameter bounds. Validated by the lite oracle suite and the old-vs-new serialization differential (legacy wallets byte-identical; version-10 round-trips). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 4.8
|
b49f85bcbe |
perf(litebe): take a read lock for wallet serialization in do_save
Saving the wallet serializes it to a temporary file under the sync lock, but it took a write guard on the wallet to do so. Serialization only reads wallet state, so the write guard needlessly excluded other readers for the whole duration of the write, buffering, and flush. - The save path now takes a read guard to serialize. A read guard still excludes writers, so nothing can mutate the wallet mid-write; it only lets other readers proceed in parallel. The surrounding sync lock still serializes saves against syncs, and the atomic temp-file-then-rename sequence is unchanged. The serialized bytes are identical, verified against the differential harness. The wallet file format is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 4.8
|
c045e56d4e |
perf(litebe-sync): bound the gRPC block channel with backpressure
Each block range is streamed over one gRPC connection into a channel that a worker thread drains, encodes, and scans. That channel was unbounded, so when the server delivered blocks faster than the scanner consumed them the backlog piled up in memory with no ceiling - on a fast link against a large range it could grow without limit. - The channel is now a bounded synchronous channel holding at most 2048 blocks. When it fills, the fetch side waits, applying backpressure up the stream instead of buffering endlessly, so peak memory is capped no matter how far ahead the server runs. - The batch size is untouched: it still streams a whole range over a single connection, which is what keeps the connection count low enough to avoid socket-buffer exhaustion on Windows. Only the in-flight depth between fetch and scan is now bounded. The same blocks are fetched and scanned in the same order, so the resulting wallet is byte-identical, verified against the differential harness. The wallet file format is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 4.8
|
be69c9bafe |
fix(litebe-sync): bound the reorg recovery window to the max depth
When sync hit a reorg it stepped the scan back to just before the invalidated block, then set the top of the re-scan range to that point plus the full forward batch size. That window is far larger than any reorg can be: the batch size is tuned for forward streaming (tens of thousands of blocks), while a reorg is capped at the maximum reorg depth and the pass aborts once it is exceeded. So the recovery pass re-fetched and re-scanned a large span of already-good blocks before it could resume moving forward. - The recovery range now reaches only the maximum reorg depth (plus one) above the rewind point, which is the most a reorg can ever disturb. Forward syncing still uses the full batch size; only the pass immediately after a reorg is bounded this way. - The set of blocks that ultimately get scanned is unchanged - the next pass picks up from where recovery stops - so the resulting wallet is identical, just reached with far less redundant fetching and scanning. The wallet file format is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 4.8
|
3f7af2d028 |
fix(lite-backend): fix t-scan rewind underflow and retry the owed scan
When a sync pass's transparent (t-address) scan failed, the backend mishandled it three ways: the retry computed its start height as the block height minus the batch size, an unsigned subtraction that wrapped to a huge value for low heights; the block range left un-scanned was not remembered, so a transient failure left a permanent hole in the transparent history that nothing ever revisited; and the block times needed to date the transactions in that range were discarded. - The owed range and the block times seen so far are now kept and folded into the next pass, which resumes from the lower of its own first block and any owed range. - The scan start never subtracts, so it can no longer underflow. - A t-address scan owed by an earlier failed pass is paid on the next sync even when no new block is pending. Behaviour change: a transparent scan that fails is retried in-process on the next sync rather than being lost. The owed range lives only in memory and is never written to the wallet file, so the file format is unchanged and remains interchangeable with other builds. The debt is lost if the process exits or the server is switched before a retry completes; Re-download blocks (rescan) recovers it, as today. Tests: library tests on the extracted t-scan seam cover a failed pass keeping its range and block times, a later success clearing the debt, and the scan start never exceeding the pass start across the batch-size boundary (the old formula wrapped there). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
31304a1a63 |
fix(lite-backend): durable wallet saves, non-destructive backup recovery
do_save flushed the temp wallet and renamed it over the wallet with no
fsync of the temp file, the .bak or the directory, so after a power cut
the rename could be on disk while the data was not, leaving a torn
wallet. The readers then panicked on malformed bytes instead of
returning an error (the gzip Decoder and WalletTx unwraps in
LightWallet::read, the version asserts in data.rs and walletzkey.rs,
from_utf8/diversifier unwraps, and vec![0; len] on a corrupt length
prefix, which can abort the process outright), so read_from_disk's .bak
fallback was never reached. And the fallback itself was destructive: it
copied the .bak over the wallet before reading it, on ANY error, so the
damaged file was lost, a transient I/O error replaced a good newer
wallet with an older .bak, and a wallet written by a newer app ("Don't
know how to read wallet version") was overwritten too, dropping imported
keys that are not in the seed.
- do_save: sync_all the temp file (removed on error), copy the .bak,
tighten and sync it through a write handle (FlushFileBuffers needs
write access on Windows), rename, sync the copy-fallback destination,
then fsync the directory on unix. No MoveFileEx write-through on
Windows.
- read_from_disk: each parse runs in read_wallet_file under
catch_unwind (panic = "unwind" is kept for release builds). The .bak
is used only when the wallet fails with InvalidData, UnexpectedEof or
InvalidInput, and it is read first, so a bad backup leaves both files
as they were. A damaged wallet whose header says encrypted is not
replaced by a plaintext .bak (only an older build leaves one): the
open fails with an explanation and nothing changes. Otherwise the
damaged wallet is moved (or, failing that, copied) to
<wallet>.dat.corrupt-<unix secs>, with a -N suffix while that name is
taken (a dangling symlink counts as taken), and is never deleted. The
recovered wallet is tightened as before.
- read_from_disk_reporting returns the same client plus "recovered from
backup; damaged file kept as <name>" after a recovery, for the FFI;
read_from_disk wraps it. Both stages are also written to stderr,
since the logger is not initialised yet when the wallet is opened.
- Readers return Err: a newer wallet version is ErrorKind::Unsupported
(was InvalidData), and so is a newer WalletTx or key version;
Decoder::new and WalletTx::read propagate with ?; a bad Utxo version,
a negative or non-UTF-8 or non-R Utxo address, non-UTF-8 metadata
addresses, an invalid diversifier and a viewing key with no default
address are InvalidData. Every length-prefixed string and address is
read through read_bytes_bounded (take(len).read_to_end, no cap, std's
own short-read error), so a corrupt length is never allocated up
front.
Flagged behaviour changes:
- fsync order: tmp, then .bak, then rename, then the dir (unix); saves
at exit, encrypt and decrypt still run on the UI thread and get
slower on slow disks;
- the readers return Err instead of panicking;
- the .bak fallback runs only for corrupt files, and the damaged file
is kept as .dat.corrupt-<ts> (every copy is kept; one from a
plaintext era keeps its seed, inside the 0700 dir);
- a wallet with a newer format errors (Unsupported) and is never
replaced;
- a damaged encrypted wallet whose backup predates encryption is not
opened.
Wallet format v9 is unchanged; valid files read and write byte for byte
as before. No ABI or symbol change.
Tests: a wallet whose parse panicked recovers from the .bak, the
garbage is kept as .dat.corrupt-<ts> and the report names it, and the
next open reports nothing; a newer-version wallet is Unsupported and
both files are unchanged (this fails when only the version error kind is
reverted); an unreadable (0000) wallet is a PermissionDenied error that
changes nothing either; a corrupt wallet with
a corrupt or no .bak changes nothing; a damaged encrypted wallet with a
pre-encryption plaintext .bak is refused with both files unchanged and
nothing moved aside; a recovery never overwrites an earlier
.dat.corrupt-<ts> or follows a dangling symlink; every truncation of a
wallet and of a WalletTx is an Err, crafted Utxo, metadata, note and key
records are the expected Err kinds, and a u64::MAX length returns at
once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DanSandClaude Opus 5.5
|
39333f80b7 |
fix(lite-backend): keep wallet files and logs private to the owner
On Linux and macOS the Lite data dir (~/.silentdragonxlite) was created
with create_dir_all (0755 under the default umask) on every
get_zcash_data_path() call, even by the startup wallet_exists() probe.
do_save wrote the wallet through File::create (0644) and renamed it into
place, and fs::copy gave the .bak the same mode, so any local account
could read the wallet file, and the plaintext seed with it when the
wallet was not encrypted. The debug log was 0644 and its roller pattern
was relative, so rolled logs landed in the process's working directory.
A crash-leftover .dat.tmp was truncated and reused: an fd another
account had opened on it read the next wallet. And after encrypt(), the
first save copied the previous, plaintext wallet to .bak, where the seed
stayed until the next save, or for good if the app quit right away.
- New helpers, each an item-level #[cfg(unix)] / #[cfg(not(unix))]
pair: create_private_dir_all (0700 components), tighten (strips only
the group/other bits of an existing dir or file, keeps the owner bits,
never follows a symlink), create_private_file_new (unlinks a stale
file, then create_new with 0600), and a once-per-process warning when
a chmod fails (drvfs, FAT, NFS).
- get_zcash_data_path creates the default data dir 0700 and tightens an
existing one (for test/regtest also its parent). A caller-supplied
data_dir is created as before and never chmodded, and the params dir
is left alone.
- do_save writes a fresh 0600 temp file and tightens the .bak after the
copy. At the plaintext -> encrypted transition (the wallet is
encrypted and the on-disk header says plaintext) the .bak is copied
from the new, encrypted file. wallet_header_encrypted reads the flag
byte for versions 4 and 6-9 only (v5 is gzip) and returns None when it
cannot tell, which keeps the old behaviour.
- read_from_disk tightens a wallet restored from .bak, and the wallet
and .bak after every successful read.
- get_log_config pre-creates the log 0600 on unix and anchors the
roller in the data dir; a data path containing "{}" keeps the old
relative pattern. log4rs re-creates the log with the umask mode after
a 5 MB roll, so the 0700 data dir is the lasting control.
Flagged behaviour changes:
- on unix, the data dir is 0700 and existing files lose their
group/other bits (owner bits kept, symlinks and a caller-supplied
data_dir untouched); a shared group or a backup user under another uid
loses read access;
- a fresh 0600 temp file is used;
- the first encrypted save backs up the encrypted file;
- the log rolls inside the data dir (old rolled logs in former working
directories are not cleaned up).
On Windows the file modes are unchanged; the encrypted first .bak and
the log-roll location apply there too. Wallet format v9 is unchanged;
no ABI or symbol change.
Tests: the wallet and .bak are 0600 after every save and after an open
from 0644, and a wallet restored from a loose .bak is 0600, while the
caller's 0755 data dir is unchanged; a stale 0644 .dat.tmp is replaced
by a new inode and an fd held on it still reads the stale bytes; tighten
keeps owner bits (0555 -> 0500), ignores the other kind and symlinks,
create_private_dir_all makes every component 0700, and
create_private_file_new never reuses or follows a stale file or link;
after encrypt the first save's .bak is encrypted, byte-identical to the
wallet and unlocks to the seed; the header probe answers only for v4 and
v6-v9; the log is 0600, the roller pattern and a real roll stay in the
data dir, and a "{}" path falls back. An opt-in test
(DRAGONX_LITE_SCRATCH_HOME=1, temp HOME) checks the default data dir,
its testnet parent, a symlinked data dir and the params dir.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DanSandClaude Opus 5.5
|
a5db2e9a3d |
fix(lite-backend): never hang or panic a sync on malformed server data
The lightwalletd server controls the compact ciphertext length, the block, prev-block and tx hash lengths, and (through the chain) the full txs' epks. Each of these could wedge or poison a sync: - try_sapling_compact_note_decryption asserts a 52-byte ciphertext. A shorter (or full 580-byte) one panicked a pool job in scan_output_internal while the original sender stayed alive, so the collector waited forever with sync_lock, txs.write() and a wallet read guard held. - scan_taddress_txids kept its own sender alive too: one panicking t-address job (scan_full_tx) left crx.iter().take(n) blocked forever. - CompactBlock::hash()/prev_hash() assert 32 bytes when no header is sent, and every vtx hash is copied into a [u8; 32], one of them under txs.write(): a 31-byte tx hash poisoned the wallet for good, and a bad hash in the block's txid list panicked after the witnesses had moved. - scan_full_tx and scan_full_mempool_tx unwrapped as_prime_order() on the epk. Consensus rejects only a small-order epk, so a mixed-order one can be mined: an own tx then failed every sync, and the mempool receiver thread died silently. Changes: - append_leaf(node: Node, ..) commits a leaf to the existing, block and new witnesses and the tree; the epk-error path, the new short-ciphertext path and the normal path all use it (a behaviour-neutral extraction). - A compact ciphertext under 52 bytes is skipped with its cmu still committed; a longer one is truncated to 52 bytes, which is exact (the compact plaintext is those bytes and the cmu re-check validates it). - Each decryption job catches a panic and sends exactly one message; drop(tx) after the dispatch, and the collector logs a vanished or panicked job and treats the output as not ours. - scan_taddress_txids runs through the private seam scan_taddress_txids_with(job): catch_unwind per job, drop(ctx), and a count check, so a dead job is an error, never a hang or a silent partial success. - scan_block_with_pool checks the block, prev-block and vtx hash lengths right after the parse and returns Err(-1), as for an unparsable block, before any change (Err(height) would refetch the same block forever under sync_lock). - scan_full_tx and scan_full_mempool_tx skip an output whose epk is not in the prime-order subgroup (one check per output, used by every decryption and the outgoing recovery). - The mempool receiver thread catches a panic of scan_full_mempool_tx. Flagged behaviour changes: - a compact ciphertext under 52 bytes is skipped (its cmu is still committed); one over 52 bytes is truncated; - pool-job panics are reported (logged; the output is treated as not ours, which a rescan repairs) instead of hanging the sync; - bad hash lengths return Err(-1) before any change; - outputs with a non-prime-order epk are skipped; - the mempool thread survives a bad tx. Wallet format v9 is unchanged; no ABI or symbol change. Honest data takes the same path as before (an old-vs-new run over generated honest chains is byte-identical). Tests (each on its own thread under a 15 s watchdog): 0/51/52/53/580-byte ciphertexts in one block on the inline and deferred witness paths (the 51-byte notes are skipped, the 53- and 580-byte notes are found, the tree and every witness match the reference); a 31-byte tx hash, an empty block hash, a 31-byte prev hash and a 33-byte foreign tx hash each return Err(-1) with the txs lock usable and last_scanned_height, trees and witnesses unchanged; a mixed-order epk in scan_full_tx and scan_full_mempool_tx does not panic and decrypts nothing while the real epk still decrypts; a t-address job panicking for 1 of 3 addresses is an Err. The oracle generator gains short (0..=51-byte) and full (580-byte) ciphertext outputs without an extra random draw, so each seed keeps its sequence of blocks, values and keys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
20b63bf45d |
fix(lite-backend): validate WIFs, refuse key import while encrypted
import_taddr decoded a WIF with bs58::decode(..).unwrap(), a [2..66] slice of its hex and SecretKey::from_slice(..).unwrap(), all under the wallet write guard that do_import_tk holds. A z-key, a typo or any short string panicked there and poisoned the wallet until restart, and the checksum was never verified, so a mistyped WIF that was still valid base58 imported a different key. A private key imported into an encrypted wallet was stored without an enc_key. The import then failed in do_save's lock(), which had already zeroed the seed while the wallet stayed "unlocked": seed export showed the all-zero-entropy phrase. - decode_taddr_wif replaces the old decode: Base58Check with the checksum verified, version 0xBC (the config prefix) or 0x80, and the 0x01 compressed flag. It decodes into a buffer it owns and wipes it with sodiumoxide::utils::memzero on every path, including a failed checksum. Errors are fixed strings that never contain the key. - import_taddr and add_imported_sk refuse while the wallet is encrypted, as add_taddr and add_zaddr already do. - do_import_tk and do_import_sk check is_encrypted() before the "locked" check, so a locked wallet does not send the user to unlock and retry. Flagged behaviour changes: - Base58Check, version 0xBC/0x80 and the compressed flag are required; - uncompressed and foreign-network WIFs are rejected with fixed messages (an uncompressed key used to import under the compressed address, where its funds are not); - no panic and no poisoned lock; - private-key import (t and z) is refused while the wallet is encrypted, with a reason that says to remove encryption, import, then encrypt again; viewing keys still import; - the key is never echoed. No ABI, symbol or wallet format change. Tests: garbage, z-keys, bad checksums, a transposed or flipped character, a zero scalar, a wrong flag, uncompressed, 0x05 and 0xEF WIFs are rejected with their message and leave tkeys usable; the wallet's own export, a 0x80 WIF and the published key-1 vector still import; the encrypted refusal in the wallet and the client, locked and unlocked, leaves lock, save and seed export intact; a viewing key still imports into an encrypted wallet; a bad timport leaves the wallet lock unpoisoned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
026d0e92a6 |
fix(lite-backend): serialize clear and rescan with a running sync
clear_state_from wiped the wallet's blocks, txs and mempool maps without taking sync_lock. The console `clear` and `rescan` commands run concurrently with the refresh worker's sync (litelib_execute drops the client mutex before a command runs), so a clear could land after a sync had already taken its nullifier snapshot. scan_block then unwrapped a note that was no longer in the wallet and panicked under the txs write guard, which poisons the wallet until restart. - clear_state_from now takes sync_lock first (lock order sync_lock -> wallet, as in do_save and do_sync). No caller holds sync_lock, so it cannot self-deadlock; do_rescan holds no lock when it clears. - clear_state reads the birthday into a local before calling clear_state_from. Before, the wallet read guard was a temporary that lived across the call, which read-locks the wallet again: with a writer (a save, an import, a new address) queued in between, the writer-preferring RwLock on Linux and Windows could deadlock. Holding it while waiting for sync_lock would also have inverted the lock order against do_save. - scan_block resolves each spent note with and_then instead of three unwraps. A spend of a note that is no longer in the wallet logs a warning and is skipped. Flagged behaviour changes: - console clear/rescan and the GUI rescan wait for an in-flight sync or save to finish instead of racing it; - the recursive wallet read lock that could deadlock a concurrent save is gone; - a spend of a vanished note now warns and is skipped instead of panicking; it is counted after the next rescan. No ABI, symbol or wallet format change. Tests: clear_state_from waits for sync_lock, clear_state holds no wallet guard while it waits, neither leaks the lock, and a 200x clear/write loop finishes within a 20 s watchdog (a regression guard only). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
0a72b47ca6 |
fix(lite-backend): verify TLS for the t-address scan and mempool monitor
The transparent-address txid scan in the sync loop and the mempool monitor both hard-coded no_cert = true, so they connected to the lightwalletd through get_client's NoCertificateVerification path in every Lite build, while the app always passes dangerous = false. Block sync, info, broadcast and the full-tx fetcher already honour LightClientConfig::no_cert_verification. An active man in the middle on those two channels could hide or invent transparent balance, UTXOs and pending transactions (signing stays local, so no funds can be taken). Both calls now take the setting from the config: - the sync loop passes self.config.no_cert_verification to scan_taddress_txids; - start_mempool_monitor reads config.no_cert_verification once and passes it to monitor_mempool. Flagged behaviour change: a lightwalletd with an invalid or self-signed certificate now fails the t-address stage (do_sync retries, then returns the error) and the mempool monitor (it logs and retries), as block sync already does. `dangerous` is the only bypass. No ABI, symbol or wallet format change. A tripwire test checks that lightclient.rs never again hard-codes the bypass for either call; its needles are joined at runtime so the test cannot match its own source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
c233d4d503 |
perf(lite-backend): fetch full transactions over one shared gRPC channel
At the end of a sync the wallet fetches the full transaction of each new wallet tx, together with every other transaction in the blocks that contain them, as decoys. Every txid got its own pool job that built a fresh Tokio runtime (one thread per CPU), a new rustls config and a new TCP + TLS + HTTP/2 connection for a single GetTransaction call, so a restore or rescan opened one connection per fetched transaction. The first error returned from the sync while the remaining jobs kept fetching and scanning in the background, and a stalled connection, an unparsable transaction or a panicking scan could hang the sync forever with its lock held. fulltx_fetch.rs now works through the same (txid, height)-sorted, de-duplicated list on the sync's runtime, with as many workers as the sync pool has threads (2-8, the same fan-out as before), over one gRPC channel that is dialed lazily and re-dialed on retry. Parsing and scanning stay on the sync pool, and the call returns once the whole batch has finished. Behaviour changes: - a failure on a decoy no longer fails the sync; it fails only when one of the wallet's own txs could not be fetched and scanned; - new error texts "Error reading transaction: ...", "full tx scan job panicked" and "fetched only X of Y wallet txs", all after the existing "Error fetching all txns for memos: " prefix; - each tx gets up to 3 fetch attempts with a 120 s timeout each, where the old code could wait forever; a request that legitimately takes longer now fails; - nothing keeps running in the background after the sync returns; - all GetTransaction calls share one TLS/HTTP-2 connection (up to one dial per worker at the start, plus a re-dial per retry); the request list and its order are unchanged; - if the network silently drops packets mid-batch, every attempt waits out its timeout until 16 consecutive failed txs stop the batch: up to about 12 minutes with 8 workers (48 minutes with 2) with the sync lock held, where the old code gave up at its first failed connection. Which txs are requested, in what order, how often they are retried and when the batch stops do not depend on which txs are the wallet's own; only the final result does. A server that keeps failing one tx can see that result through the sync's retry, but it could already tell, because an own tx that failed is requested again on the next sync without its decoys; nothing new leaks. tests/fulltx_fetch.rs drives the engine with a fake fetcher: the old request list, each item fetched and scanned once, bounded concurrency, a request sequence independent of ownership, retries, timeouts, the breaker, panicking and unparsable scans, and a saved wallet byte-identical to the old serial fill. The gRPC fetcher itself is not covered by these tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
6de9b5cc90 |
perf(lite-backend): borrow embedded Sapling params instead of copying
Every LightClient copied the Sapling spend and output parameters, which rust-embed already compiles into the binary, onto the heap: 51,551,256 bytes (49.2 MiB) of private memory per live client, whether or not it ever sent anything. LightClient::sapling_output and sapling_spend are now Cow<'static, [u8]> and borrow the embedded image. set_sapling_params still stores an owned copy, with its is-empty guard, hash checks and disk write unchanged. The prover receives the same bytes. Behaviour changes: - about 49 MiB less anonymous memory per live LightClient; the wallet file and JSON are unchanged; - the first send or shield pages the 51.5 MB of embedded params in from the binary, where client creation used to copy them (an estimated 0.3-1 s from the AppImage); - the public fields sapling_output and sapling_spend change type from Vec<u8> to Cow<'static, [u8]>: a Rust API change, with no C ABI or exported-symbol change. tests/sapling_params.rs checks that a restored, a new and a reloaded client all borrow the embedded image (same pointer, pinned lengths and SHA-256), that set_sapling_params keeps its guard and hash checks, and that the borrowed params parse like a heap copy. An ignored test measures 576 KiB of RssAnon growth for a second client, against 46.5 MiB with the old heap copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
bd4c06dfd0 |
perf(lite-backend): encode note addresses from pk_d instead of deriving
note_address() re-derived a note's payment address from its viewing key and diversifier (a group hash, an ivk hash and a 252-bit scalar multiplication) for every note in `notes` and `list` and for each new note in the block scan. The z-balance sums did the same derivation in their address filter, so `balance`, which sums four balances for each of Z z-addresses, cost Z x (N + 2U) derivations for N notes, U of them unspent. do_list_notes also encoded up to Z + Z^2 addresses to find the spendable ones and cloned that set once per transaction. Now: - note_address() encodes the note's stored pk_d with its diversifier. Both decryption and wallet load build that pk_d from the same key and diversifier, so the string is byte-identical; - the three z-balance sums share one address filter. The per-address sums behind `balance` compare (diversifier, pk_d) with each z-address directly; any other address string keeps the old exact string compare; - those sums work from a copy of the z-addresses whose lock is released before any note is read, so the txs-then-zkeys lock order that sync uses is unchanged; - do_list_notes builds the spendable-address set with Z encodes and borrows it. No output changes: balances and the notes/list/balance JSON are byte-identical. The address filter no longer contains an unwrap (it could not fail). Tests compare against verbatim copies of the old functions over randomized chains (note addresses, also after a reload; every filtered balance for own, diversified, foreign and junk address strings; the per-address balances; the spendable set), and a concurrency test fails with a deadlock if the zkeys lock is held across the sums. A differential run against the unmodified backend compared per-address balances and the JSON after every operation, through reorgs and reloads, and the old build re-saves the new files byte for byte. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DanSandClaude Opus 5.5
|
8895038c40 |
test(lite-backend): port the core-crate tests to the current API
The lite backend core's in-crate tests no longer compiled (119 errors in
lightwallet/tests.rs and the lightclient and walletzkey test modules):
they still used the old LightWallet::new and send_to_address signatures,
the removed extfvks/extsks/zaddress fields and the old new_from_phrase
and do_send arities. So `cargo test` built nothing and no backend change
could be checked against them.
Port them, and the commands and checkpoints tests, to the current API,
touching only #[cfg(test)] code:
- LightWallet::new takes 4 arguments, send_to_address 7 (with an
explicit default fee, returning (txid, raw tx)), new_from_phrase 5 and
do_send 2; keys are read through zkeys and tkeys;
- every config uses the "main" chain: creating a wallet on the "test"
chain panics in base58_pubkey_address, so LightClient::unconnected
(which hard-codes "test") is no longer used;
- every LightClient lives in a temporary data dir, so a test cannot
write to the real ~/.silentdragonxlite or ~/.zcash-params;
- ten send_to_address recipient lists garbled by an upstream change in
2020 are restored from the upstream history.
Expectations that contradicted deliberate current behaviour are updated;
none is ignored:
- a pending send's mempool entry sits at the tip, not tip+1, and
zbalance no longer counts a note with a pending spend;
- keys cannot be added to an encrypted wallet, so the encrypted-receive
tests derive their second address before encrypting;
- the t-address vectors are m/44'/141'/0'/0/{0,1}, checked against an
independent BIP32 derivation; the z-address vectors record the current
output;
- test_addresses and test_checkpoints follow the current address count
and checkpoint tables.
All 43 tests pass on the unchanged production code (three runs) and
leave the real wallet and params directories untouched; that list is the
regression baseline for the following commits. They cover balances,
address derivation, bookkeeping, the mempool, encryption, serialization
and rollback, but none verifies a proof, an anchor or a witness.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DanSandClaude Opus 4.8
|
1393d9ae1a |
feat(lite): vendor SDXL backend source; source deps from git.dragonx.is
Vendor the two local Rust crates that build the lite backend artifact into third_party/silentdragonxlite/ (the qtlib C-ABI wrapper + the silentdragonxlitelib core, with proto/res and all the lite-send fixes), and point build-lite-backend-artifact.sh's default --backend-dir there, so the lite wallet builds without the upstream SilentDragonXLite repo. External build inputs are now only the Rust toolchain + git.dragonx.is + crates.io: - the 6 librustzcash git deps point at the git.dragonx.is/DragonX/librustzcash mirror (pinned rev acff1444), not git.hush.is; - the Sapling params are gitignored (not committed, no Git LFS) - the build fetches them from the git.dragonx.is/DragonX/zcash-params 'sapling-v1' release and verifies their SHA-256 before rust-embed bakes them in (ensure_sapling_params). For fully offline builds, cargo vendor into lib/vendor/ and add a vendored-sources redirect (vendor/ is gitignored; the script symlinks it into the prepared dir). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |