11 Commits

Author SHA1 Message Date
4caf2fc68f Add BIP39 seed phrases (SilentDragonXLite-compatible) and HD transparent keys
Derive transparent (t-addr) keys from the HD seed and add BIP39 mnemonic seed
phrases that are byte-for-byte compatible with SilentDragonXLite, so the same
24 words recover the same shielded and transparent addresses in either wallet.

HD transparent keys:
- Derive t-keys from the seed at m/44'/coin'/0'/0/i (were random CKeys).
- CHDChain gains a version-gated transparent counter; existing wallets load
  unchanged. GenerateNewKey routes through DeriveNewChildKey when enabled
  (-hdtransparent, default on).
- Restore from a seed hex via -hdseed with gap-limit pre-derivation; birthday
  pinned to genesis so the rescan is not clipped.

BIP39 seed phrases:
- Wire the vendored trezor BIP39 lib (src/crypto/bip39) into the build, fix its
  BIP39_WORDS guard, and disable the insecure mnemonic cache.
- Match SDXLite exactly: English wordlist, empty passphrase, PBKDF2 64-byte
  seed, coin type 141, ZIP-32 m/32'/141'/i' and BIP44 m/44'/141'/0'/0/i. Store
  the 32-byte entropy and expand to the 64-byte seed on demand.
- Restore via -mnemonic, create via -usemnemonic, reveal via z_exportmnemonic.

Verified by gtests including a known-answer BIP39 seed vector and z/t address
derivation checks (src/gtest/test_hdtransparent.cpp, test_mnemonic_compat.cpp).
Docs in doc/hd-transparent-keys.md and doc/seed-phrase.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 01:57:18 -05:00
84aefb5475 Remove assumeutxo / UTXO-snapshot feature
Removes the dumptxoutset RPC, -loadutxosnapshot / -loadutxosnapshotunsafe,
the CCoinsViewDB Dump/LoadSnapshot machinery + CUTXOSnapshotHeader, the
AssumeutxoData chainparams anchor, the LoadSnapshotChainstate activation +
reorg-below-H guard, the persisted assumeutxo-height flag, and the gtest.

Rationale: it duplicated the existing bootstrap (same skip-the-genesis-grind
fast-sync, no speed advantage), its only real edge was a trust model we don't
need for this chain, and it was inert anyway (no published snapshot hash in
chainparams). The -loadutxosnapshot load path adopted an external UTXO set and
bypassed genesis validation, so removing it also drops that attack surface.
Builds clean (no dangling references); the kept IBD speedups (RandomX
pre-verify, adaptive dbcache, tlsmanager) are untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 16:42:37 -05:00
1f2b109d95 Add opt-in bulk block streaming (-bulkblocksync)
A single getblockstrm request makes a peer stream a contiguous range of old
blocks back-to-back as ordinary BLOCK messages, amortizing the per-block
round-trip over the whole range instead of the MAX_BLOCKS_IN_TRANSIT_PER_PEER
window. This targets the bandwidth-delay-product ceiling that dominates IBD
from few/high-latency peers below the checkpoint.

Design (off by default; negotiated via a NODE_BULKBLOCKS service bit; the
default getdata IBD path is untouched when disabled):
- protocol: NODE_BULKBLOCKS service bit + getblockstrm/blockstream messages.
- requester: in SendMessages, after FindNextBlocksToDownload, when the first
  needed block is >= BULK_TIP_MARGIN (5000) below the network tip and the peer
  advertises the bit and we are in IBD, request a contiguous range (<=128
  blocks) instead of per-block getdata; mark the range in-flight.
- server: stream the range (caps 128 blocks / 8 MiB; reads outside cs_main;
  per-peer flood throttle), then a trailing blockstream header with the actual
  count sent. Self-suppresses while the server itself is in IBD.
- received blocks ride the existing BLOCK -> ProcessNewBlock path (fully
  validated; checkpoints below 2.84M still apply); the trailing header
  reconciles partial deliveries and the range is freed on a 90s timeout, so a
  partial/withheld/refused batch falls back to the normal path (no leak, no
  permanent gap, no disconnect). In-flight tracking is by literal hash, so a
  reorg cannot orphan range entries.

Hardened against the issues found in two adversarial review passes (drain vs
timeout, partial reconciliation, ownership-guarded frees, one-shot header,
reorg-proof helpers, cs_main hold). Validated end-to-end between two local
v1.0.3 nodes (128/128 and partial serves; height advanced; no errors).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 21:22:54 -05:00
78ea2aac5b Add -maxblocksintransit: tunable per-peer block-download window
The per-peer in-flight block window (MAX_BLOCKS_IN_TRANSIT_PER_PEER) was a
hardcoded 16. On a single, high-latency peer during IBD the transfer is
bandwidth-delay-product bound (window / RTT), so with tiny sub-checkpoint
blocks the window, not bandwidth, is the ceiling — measured ~4x throughput
going 16 -> 64 on a 350ms-RTT peer. Make it a runtime flag (default 16,
clamped 1..4096), logged at startup. No behavior change at the default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 21:22:32 -05:00
2419ed7bf7 Fix flaky build: make version-probe pipes SIGPIPE-safe
util/build.sh runs with `set -eu -o pipefail`. `eval "$MAKE" --version | head -n2`
(and the analogous `as --version | head`) can race: head closes the pipe after N
lines, make/as catch SIGPIPE and exit non-zero, pipefail propagates the failure,
and errexit aborts the build before any compilation. Append `|| true` so these
purely-informational version prints can never fail the build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 21:21:29 -05:00
a9b1b4085f Fix automake -lcurl portability lint in Makefile
LIBBITCOIN_SERVER was fed into both EXTRA_LIBRARIES (a list of buildable
library files) and several _LDADD link lines. Embedding the -lcurl linker
flag inside it made automake reject it in the EXTRA_LIBRARIES context
("'-lcurl' is not a standard library name"). Make LIBBITCOIN_SERVER a pure
file and route -lcurl through its own LIBCURL variable, added to the
dragonxd, hush-gtest, and test_bitcoin link lines after libbitcoin_server.a
(whose objects reference curl symbols) so static link order stays correct.

Verified with a clean Windows cross-build (-DCURL_STATICLIB) and a native
Linux build: both link cleanly and the automake lint is gone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 16:58:31 -05:00
f763e3f1e7 Merge ibd-sync-speedups into dev
Parallel RandomX PoW pre-verify, adaptive dbcache, UTXO snapshot, P2P/TLS sync fixes.
2026-06-28 16:11:38 -05:00
f8f13f9027 Merge sapling-witness-rebuild-fix into dev
Sapling witness desync fix + parallel witness-cache rebuild + version bump to 1.0.3.
2026-06-28 16:11:38 -05:00
bf1b4cffe0 Bump version to 1.0.3
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 16:04:51 -05:00
82d77344d2 Fix Sapling witness desync and parallelize witness cache rebuild
Wallets upgraded across the 1.0.1->1.0.2 network transition could end up
with note witnesses stuck at a stale height, causing z_sendmany /
z_mergetoaddress to fail to build a valid spend. Root cause was a trio of
issues that let a desynced witnessHeight perpetuate instead of self-healing:

- DecrementNoteWitnesses left witnessRootValidated and the witness deque in
  an asymmetric state on the size<=1 path.
- VerifyAndSetInitialWitness blindly trusted witnessHeight instead of
  validating the cached root against the chain, so a bad height survived.
- UpdatedNoteData copied witnessHeight even when no witnesses were present.
- witnessRootValidated was uninitialized and never serialized, so a garbage
  true value could short-circuit the self-heal.

Fixes:
- Default witnessRootValidated to false (in-memory only; never serialized).
- VerifyAndSetInitialWitness now validates the cached witness root against
  the block's hashFinalSaplingRoot and reseeds on mismatch.
- Symmetric reset of witness state in DecrementNoteWitnesses.
- Guard the witnessHeight copy in UpdatedNoteData behind a non-empty
  witnesses check.
- Defensive majority-root guard in GetSaplingNoteWitnesses.

Also rewrites BuildWitnessCache to rebuild the witness cache in parallel
(per-block commitment extraction + worker pool), cutting a full repair from
~28 min to ~2 min. Tunable via -witnessbuildthreads and -witnessfastrebuild;
output verified byte-identical to the serial path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 16:03:23 -05:00
1673cfb6dc IBD/sync speedups: parallel RandomX pre-verify, adaptive dbcache, P2P download fixes
- Parallel RandomX PoW pre-verification pool (CCheckQueue) run ahead of the serial
  connect; consensus-neutral (inline CheckRandomXSolution fallback still verifies
  anything not pre-verified). New -randomxverifythreads (default = -par).
- Adaptive dbcache: default sizes the UTXO/coins cache to most of RAM and shrinks
  under memory pressure, always leaving a reserve free; -dbcache pins a fixed value.
- P2P block download: bounded socket recv-drain loop (tlsmanager); frontier-block
  reassignment to break head-of-line stalls (-blockreassigntimeout); ProcessGetData
  serves a bounded batch of blocks per pass instead of one (fixes the serve-side
  one-block-per-tick throttle that caps download network-wide).
- assumeutxo: dumptxoutset RPC + LoadSnapshot machinery + AssumeutxoData chainparams.
- Signed bootstrap verification (util/bootstrap-dragonx.sh, util/sign-bootstrap.md).
- gtest: RandomX pre-verify consensus-equivalence test + UTXO-snapshot round-trip;
  revived the gtest harness (Makefile.am include fix, Makefile.gtest.include).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 12:30:10 -05:00
42 changed files with 2518 additions and 282 deletions

View File

@@ -27,6 +27,21 @@ the entire history of Hush transactions; depending on the speed of your
computer and network connection, it will likely take a few hours at least, but
some people report full nodes syncing in less than 1.5 hours.
# Fastest way to sync (bootstrap)
The quickest way to get a fully-synced node is the signed bootstrap snapshot, which
installs a pre-built blockchain so you skip re-validating the whole chain from genesis:
```sh
# Stop dragonxd first if it is running, then:
./util/bootstrap-dragonx.sh
```
The script preserves your `wallet.dat` and `DRAGONX.conf`, verifies the download's
checksums and (once a release key is published) its cryptographic signature, then starts
you near the chain tip. If you prefer to sync from the network instead, a larger
`-dbcache` (e.g. `-dbcache=2048`) noticeably speeds up the initial block download.
# Banned by GitHub
In working on this release, Duke Leto was suspended from Github, which gave Hush developers

View File

@@ -6,7 +6,7 @@
set -eu -o pipefail
VERSION="1.0.2"
VERSION="1.0.3"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
RELEASE_DIR="$SCRIPT_DIR/release"

View File

@@ -3,7 +3,7 @@ AC_PREREQ([2.60])
define(_CLIENT_VERSION_MAJOR, 1)
dnl Must be kept in sync with src/clientversion.h , ugh!
define(_CLIENT_VERSION_MINOR, 0)
define(_CLIENT_VERSION_REVISION, 2)
define(_CLIENT_VERSION_REVISION, 3)
define(_CLIENT_VERSION_BUILD, 50)
define(_ZC_BUILD_VAL, m4_if(m4_eval(_CLIENT_VERSION_BUILD < 25), 1, m4_incr(_CLIENT_VERSION_BUILD), m4_eval(_CLIENT_VERSION_BUILD < 50), 1, m4_eval(_CLIENT_VERSION_BUILD - 24), m4_eval(_CLIENT_VERSION_BUILD == 50), 1, , m4_eval(_CLIENT_VERSION_BUILD - 50)))
define(_CLIENT_VERSION_SUFFIX, m4_if(m4_eval(_CLIENT_VERSION_BUILD < 25), 1, _CLIENT_VERSION_REVISION-beta$1, m4_eval(_CLIENT_VERSION_BUILD < 50), 1, _CLIENT_VERSION_REVISION-rc$1, m4_eval(_CLIENT_VERSION_BUILD == 50), 1, _CLIENT_VERSION_REVISION, _CLIENT_VERSION_REVISION-$1)))

View File

@@ -0,0 +1,87 @@
# HD transparent keys
DragonX derives **transparent** (t-address) keys deterministically from the
wallet's HD seed, so they can be recovered from the seed alone — the same way
Sapling (shielded) keys already are.
## Derivation
Transparent keys are derived over secp256k1 using BIP32/BIP44:
```
m / 44' / coin_type' / 0' / 0 / i
```
* `coin_type` is `Params().BIP44CoinType()`**141** on mainnet, **1** on
test/regtest.
* Account is fixed at `0'` and the chain at `0` (external). The internal/change
chain (`1`) is **not** used: on this `ac_private=1` chain a non-coinbase
transparent output is consensus-invalid, so transparent change can never carry
value.
* `i` is `CHDChain.transparentChildCounter`, a monotonic index persisted in the
wallet so the same addresses regenerate after a seed-only restore.
Each derived key records its `hdKeypath` and the seed fingerprint (`seedFp`) in
its `CKeyMetadata`, matching the Sapling scheme.
## Why this matters on a private chain
On DragonX (`ac_private=1` from genesis) a normal user can never *receive* to a
transparent address — inbound t-payments are rejected by consensus. The only
thing that legitimately lands spendable value on a t-address is a **mining
coinbase** (plus notary/burn special cases). There is no "coinbase must be
shielded" rule, so mature coinbase is directly spendable.
So HD transparent keys exist to let a **miner recover coinbase rewards** that
were paid to wallet-derived t-addresses, using only the seed.
## Enabling / disabling
Controlled by `-hdtransparent` (default **on**). When on and the wallet has an
HD seed, every newly generated transparent key (receive address, change,
coinbase payout drawn from the keypool) is HD-derived.
```
-hdtransparent=0 # keep the legacy behaviour (random transparent keys)
```
## Backing up and restoring
* **Back up the seed.** `z_exportwallet <file>` writes the 32-byte HD seed as a
`# HDSeed=<hex>` line. Guard this value like a private key.
* **Restore into a fresh/empty wallet** by starting the node with:
```
-hdseed=<64-hex-character seed>
-hdtransparentgaplimit=<n> # HD transparent keys to pre-derive (default 1000)
```
On restore the node injects the seed, pre-derives `n` transparent keys with a
genesis birthday, and the normal startup rescan finds any coinbase paid to
them. Raise `-hdtransparentgaplimit` if the wallet minted more than `n`
distinct coinbase addresses.
> **Warning:** passing `-hdseed` on the command line exposes the seed to your
> shell history and the process list. Prefer putting it in `DRAGONX.conf` with
> tight file permissions, and remove it after the restore completes.
## Limitations (read before relying on recovery)
* **Legacy random keys are not recoverable.** Any transparent key created before
this feature (or with `-hdtransparent=0`) came from the CSPRNG, not the seed,
and the phrase/seed will **not** regenerate it. Keep `wallet.dat` /
`dumpwallet` backups for those. A wallet that predates the feature and then
enables it becomes a *mix* of random (old) and HD (new) keys.
* **Gap limit.** A rescan only discovers keys already present in the wallet.
Restore pre-derives `-hdtransparentgaplimit` keys; coinbase paid to an index
beyond that window is not found until you derive further and rescan again.
* **Scope.** Recovers transparent **coinbase** value only, per the consensus
rules above. Shielded funds are recovered separately via the Sapling HD keys.
## On-disk compatibility
The transparent counter is stored in `CHDChain` under a new serialization
version (`VERSION_HD_TRANSPARENT = 2`). Existing v1 `wallet.dat` records load
unchanged (the counter defaults to 0); the record is rewritten as v2 the first
time an HD transparent key is derived. Downgrading a v2 wallet to an older
binary is not supported.

90
doc/seed-phrase.md Normal file
View File

@@ -0,0 +1,90 @@
# BIP39 seed phrases (SilentDragonXLite-compatible)
DragonX full-node wallets can be created from and restored to a **BIP39 24-word
seed phrase** that is **byte-for-byte compatible with SilentDragonXLite**: the
same words produce the same transparent (t-) and shielded (z-) addresses in
either wallet, so funds move between the light wallet and the full node with one
backup.
## What makes them compatible
Compatibility requires the mnemonic, the seed derivation, and every HD path to
match exactly. They do:
| Detail | Value (both wallets) |
|---|---|
| Word list | BIP39 English, 2048 words |
| Passphrase | empty (no "25th word") |
| Mnemonic → seed | PBKDF2-HMAC-SHA512, 2048 rounds, salt `"mnemonic"`, 64-byte output |
| Coin type | 141 (KMD SLIP-0044) |
| Shielded path | `m/32'/141'/i'` (ZIP-32) |
| Transparent path | `m/44'/141'/0'/0/i` (BIP44) |
The node stores the 32-byte BIP39 **entropy** (SilentDragonXLite's on-disk
convention) and expands it to the 64-byte seed on demand for derivation. The
node's vendored BIP39 library (`src/crypto/bip39`) is byte-identical to
SilentDragonXLite's `tiny-bip39` 0.6.2, and the derivation is anchored by a
known-answer test (`src/gtest/test_mnemonic_compat.cpp`).
## Restore from a phrase
Start the node once, on a **fresh/empty datadir**, with the phrase:
```
dragonxd -mnemonic="word1 word2 ... word24"
```
or, preferably (keeps the phrase out of your shell history and process list),
put it in `DRAGONX.conf` with tight permissions:
```
mnemonic=word1 word2 ... word24
```
On restore the node pre-derives keys and rescans from genesis to recover funds:
* `-hdtransparentgaplimit=<n>` — HD transparent keys to pre-derive (default 1000)
* `-mnemonicsaplinggap=<n>` — shielded addresses to pre-derive (default 100)
Raise these if the wallet used many addresses. Restore only works on a wallet
with no seed yet (a brand-new datadir); it refuses to overwrite an existing seed.
## Create a new phrase on the node
By default new node wallets use a random (non-mnemonic) seed. To create a new
wallet from a fresh 24-word phrase instead — so you can export it and use it in
SilentDragonXLite — start with:
```
dragonxd -usemnemonic
```
## Show / back up the phrase
For a mnemonic wallet (created with `-usemnemonic` or restored with `-mnemonic`):
```
dragonx-cli z_exportmnemonic
```
returns the 24 words and the seed fingerprint. The wallet must be unlocked.
Guard the phrase like a private key.
## Limitations
* **English + empty passphrase only.** Any other word list or a BIP39 passphrase
would break compatibility, so they are not accepted.
* **Legacy / random-seed wallets have no phrase.** A wallet created before this
feature (or without `-usemnemonic`) has a random seed; `z_exportmnemonic`
returns an error for it — use `z_exportwallet` to back up the raw seed. Such
wallets are not SilentDragonXLite-compatible.
* **Scope.** Recovers HD-derived shielded funds and transparent coinbase (see
[hd-transparent-keys.md](hd-transparent-keys.md) for why only coinbase lands on
t-addresses on this `ac_private=1` chain). Keys imported with `z_importkey` are
not seed-derived and are not recovered by the phrase.
## On-disk compatibility
Mnemonic wallets set `CHDChain` version 3 (`VERSION_HD_MNEMONIC`). Older wallet
records load unchanged. Downgrading a mnemonic wallet to an older binary is not
supported.

View File

@@ -39,15 +39,22 @@ BITCOIN_INCLUDES += -I$(srcdir)/univalue/include
BITCOIN_INCLUDES += -I$(srcdir)/leveldb/include
if TARGET_WINDOWS
LIBBITCOIN_SERVER=libbitcoin_server.a -lcurl
LIBBITCOIN_SERVER=libbitcoin_server.a
endif
if TARGET_DARWIN
LIBBITCOIN_SERVER=libbitcoin_server.a -lcurl
LIBBITCOIN_SERVER=libbitcoin_server.a
endif
if TARGET_LINUX
LIBBITCOIN_SERVER=libbitcoin_server.a -lcurl
LIBBITCOIN_SERVER=libbitcoin_server.a
endif
# libcurl is a linker flag, not a buildable library file. It must NOT live inside
# LIBBITCOIN_SERVER, which is also fed into EXTRA_LIBRARIES (a list of files automake
# builds) where a -l flag is illegal and triggers a portability error. Keep it as its
# own variable, added to each binary's _LDADD after libbitcoin_server.a (whose objects
# reference curl symbols) so static link order stays correct.
LIBCURL = -lcurl
LIBBITCOIN_WALLET=libbitcoin_wallet.a
LIBBITCOIN_COMMON=libbitcoin_common.a
LIBBITCOIN_CLI=libbitcoin_cli.a
@@ -318,6 +325,7 @@ libbitcoin_wallet_a_SOURCES = \
wallet/asyncrpcoperation_shieldcoinbase.cpp \
wallet/crypter.cpp \
wallet/db.cpp \
wallet/mnemonic.cpp \
zcash/Note.cpp \
transaction_builder.cpp \
wallet/rpcdump.cpp \
@@ -354,6 +362,23 @@ crypto_libbitcoin_crypto_a_SOURCES = \
crypto/sha512.cpp \
crypto/sha512.h
# Vendored trezor-crypto BIP39 (mnemonic seed phrases). Kept self-contained so
# the same 24 words are compatible with SilentDragonXLite (tiny-bip39 0.6.2).
crypto_libbitcoin_crypto_a_SOURCES += \
crypto/bip39/bip39.c \
crypto/bip39/bip39.h \
crypto/bip39/bip39_english.h \
crypto/bip39/pbkdf2.c \
crypto/bip39/pbkdf2.h \
crypto/bip39/hmac.c \
crypto/bip39/hmac.h \
crypto/bip39/sha2.c \
crypto/bip39/sha2.h \
crypto/bip39/memzero.c \
crypto/bip39/memzero.h \
crypto/bip39/options.h \
crypto/bip39/rand.h
if EXPERIMENTAL_ASM
crypto_libbitcoin_crypto_a_SOURCES += crypto/sha256_sse4.cpp
endif
@@ -464,6 +489,7 @@ endif
dragonxd_LDADD = \
$(LIBBITCOIN_SERVER) \
$(LIBCURL) \
$(LIBBITCOIN_COMMON) \
$(LIBUNIVALUE) \
$(LIBBITCOIN_UTIL) \
@@ -685,5 +711,5 @@ endif
if ENABLE_TESTS
#include Makefile.test-hush.include
#include Makefile.test.include
#include Makefile.gtest.include
include Makefile.gtest.include
endif

View File

@@ -4,65 +4,61 @@ TESTS += hush-gtest
bin_PROGRAMS += hush-gtest
# tool for generating our public parameters
# NOTE: the original test list used an invalid automake form (comment after a trailing
# backslash, and `zcash_gtest_SOURCES +=` with no prior `=`), which is why the whole
# gtest harness was disabled via a `#include`. Minimal valid set: the harness + the
# Re-add other gtest sources here as they are revived.
hush_gtest_SOURCES = \
gtest/main.cpp \
gtest/utils.cpp \
gtest/test_checktransaction.cpp \
gtest/json_test_vectors.cpp \
gtest/json_test_vectors.h \
gtest/test_wallet_zkeys.cpp \
# These tests are order-dependent, because they
# depend on global state (see #1539)
if ENABLE_WALLET
zcash_gtest_SOURCES += \
wallet/gtest/test_wallet_zkeys.cpp
endif
zcash_gtest_SOURCES += \
gtest/test_tautology.cpp \
gtest/test_deprecation.cpp \
gtest/test_equihash.cpp \
gtest/test_httprpc.cpp \
gtest/test_keys.cpp \
gtest/test_keystore.cpp \
gtest/test_noteencryption.cpp \
gtest/test_mempool.cpp \
gtest/test_merkletree.cpp \
gtest/test_metrics.cpp \
gtest/test_miner.cpp \
gtest/test_pow.cpp \
gtest/test_random.cpp \
gtest/test_rpc.cpp \
gtest/test_sapling_note.cpp \
gtest/test_transaction.cpp \
gtest/test_transaction_builder.cpp \
gtest/test_upgrades.cpp \
gtest/test_validation.cpp \
gtest/test_circuit.cpp \
gtest/test_txid.cpp \
gtest/test_libzcash_utils.cpp \
gtest/test_proofs.cpp \
gtest/test_pedersen_hash.cpp \
gtest/test_checkblock.cpp \
gtest/test_zip32.cpp
if ENABLE_WALLET
zcash_gtest_SOURCES += \
wallet/gtest/test_wallet.cpp
endif
gtest/test_randomx_preverify.cpp \
gtest/test_hdtransparent.cpp \
gtest/test_mnemonic_compat.cpp
hush_gtest_CPPFLAGS = $(AM_CPPFLAGS) -DMULTICORE -fopenmp -DBINARY_OUTPUT -DCURVE_ALT_BN128 -DSTATIC $(BITCOIN_INCLUDES)
hush_gtest_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)
hush_gtest_LDADD = -lgtest -lgmock $(LIBBITCOIN_SERVER) $(LIBBITCOIN_CLI) $(LIBBITCOIN_COMMON) $(LIBBITCOIN_UTIL) $(LIBBITCOIN_CRYPTO) $(LIBBITCOIN_UNIVALUE) $(LIBLEVELDB) $(LIBMEMENV) \
$(BOOST_LIBS) $(BOOST_UNIT_TEST_FRAMEWORK_LIB) $(LIBSECP256K1)
# Mirror dragonxd_LDADD's working library set/order (the old list used a non-existent
# $(LIBBITCOIN_UNIVALUE) so univalue was never linked, and omitted LIBHUSH/LIBRANDOMX/libcc).
hush_gtest_LDADD = -lgtest -lgmock \
$(LIBBITCOIN_SERVER) \
$(LIBCURL) \
$(LIBBITCOIN_COMMON) \
$(LIBUNIVALUE) \
$(LIBBITCOIN_UTIL) \
$(LIBBITCOIN_CRYPTO) \
$(LIBZCASH) \
$(LIBHUSH) \
$(LIBLEVELDB) \
$(LIBMEMENV) \
$(LIBSECP256K1) \
$(LIBRANDOMX)
if ENABLE_WALLET
hush_gtest_LDADD += $(LIBBITCOIN_WALLET)
endif
hush_gtest_LDADD += $(LIBZCASH_CONSENSUS) $(BDB_LIBS) $(SSL_LIBS) $(CRYPTO_LIBS) $(EVENT_PTHREADS_LIBS) $(EVENT_LIBS) $(LIBZCASH) $(LIBZCASH_LIBS)
hush_gtest_LDADD += \
$(BOOST_LIBS) \
$(BOOST_UNIT_TEST_FRAMEWORK_LIB) \
$(BDB_LIBS) \
$(SSL_LIBS) \
$(CRYPTO_LIBS) \
$(EVENT_PTHREADS_LIBS) \
$(EVENT_LIBS) \
$(LIBBITCOIN_CRYPTO) \
$(LIBZCASH_LIBS)
hush_gtest_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) -static
if TARGET_DARWIN
hush_gtest_LDADD += libcc.dylib $(LIBSECP256K1)
endif
if TARGET_WINDOWS
hush_gtest_LDADD += libcc.dll $(LIBSECP256K1)
endif
if TARGET_LINUX
hush_gtest_LDADD += libcc.so $(LIBSECP256K1)
endif
hush_gtest_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) -static
hush_gtest_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS)
hush-gtest-expected-failures: hush-gtest FORCE
./hush-gtest --gtest_filter=*DISABLED_* --gtest_also_run_disabled_tests

View File

@@ -112,13 +112,13 @@ endif
test_test_bitcoin_SOURCES = $(BITCOIN_TESTS) $(JSON_TEST_FILES) $(RAW_TEST_FILES)
test_test_bitcoin_CPPFLAGS = $(AM_CPPFLAGS) -fopenmp $(BITCOIN_INCLUDES) -I$(builddir)/test/ $(TESTDEFS) $(EVENT_CFLAGS)
test_test_bitcoin_LDADD = $(LIBBITCOIN_SERVER) $(LIBBITCOIN_CLI) $(LIBBITCOIN_COMMON) $(LIBBITCOIN_UTIL) $(LIBBITCOIN_CRYPTO) $(LIBUNIVALUE) $(LIBLEVELDB) $(LIBMEMENV) \
test_test_bitcoin_LDADD = $(LIBBITCOIN_SERVER) $(LIBCURL) $(LIBBITCOIN_CLI) $(LIBBITCOIN_COMMON) $(LIBBITCOIN_UTIL) $(LIBBITCOIN_CRYPTO) $(LIBUNIVALUE) $(LIBLEVELDB) $(LIBMEMENV) \
$(BOOST_LIBS) $(BOOST_UNIT_TEST_FRAMEWORK_LIB) $(LIBSECP256K1) $(EVENT_PTHREADS_LIBS) $(EVENT_LIBS)
test_test_bitcoin_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)
if ENABLE_WALLET
test_test_bitcoin_LDADD += $(LIBBITCOIN_WALLET)
endif
test_test_bitcoin_LDADD += $(LIBBITCOIN_SERVER) $(LIBBITCOIN_CLI) $(LIBBITCOIN_COMMON) $(LIBBITCOIN_UTIL) $(LIBBITCOIN_CRYPTO) $(LIBUNIVALUE) \
test_test_bitcoin_LDADD += $(LIBBITCOIN_SERVER) $(LIBCURL) $(LIBBITCOIN_CLI) $(LIBBITCOIN_COMMON) $(LIBBITCOIN_UTIL) $(LIBBITCOIN_CRYPTO) $(LIBUNIVALUE) \
$(LIBLEVELDB) $(LIBMEMENV) $(BOOST_LIBS) $(BOOST_UNIT_TEST_FRAMEWORK_LIB) $(LIBSECP256K1) $(EVENT_LIBS) $(EVENT_PTHREADS_LIBS)
test_test_bitcoin_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)

View File

@@ -35,7 +35,7 @@ extern bool fZindex;
// These version thresholds control whether nSproutValue/nSaplingValue are
// serialized in the block index. They must be <= CLIENT_VERSION or the
// values will never be persisted, causing nChainSaplingValue to reset
// to 0 after node restart. DragonX CLIENT_VERSION is 1000250 (v1.0.2.50).
// to 0 after node restart. DragonX CLIENT_VERSION is 1000350 (v1.0.3.50).
static const int SPROUT_VALUE_VERSION = 1000000;
static const int SAPLING_VALUE_VERSION = 1000000;
extern int32_t ASSETCHAINS_LWMAPOS;
@@ -399,7 +399,16 @@ public:
//! (memory only) Sequential id assigned to distinguish order in which blocks are received.
uint32_t nSequenceId;
//! (memory only) Set true once this block's RandomX PoW has been verified by the parallel
//! pre-verification pool, letting the inline check in CheckBlockHeader skip the recompute.
//! Written by exactly one pre-verify worker (1:1 with the block) and read by the connect
//! thread only AFTER the pool barrier (CCheckQueue::Wait provides the happens-before), so a
//! plain bool is race-free here. NOT serialized — a pure optimization hint; the inline
//! CheckRandomXSolution remains the consensus authority. (Plain bool, not std::atomic, so
//! CBlockIndex stays copyable for CDiskBlockIndex's `CBlockIndex(*pindex)` construction.)
bool fRandomXVerified;
void SetNull()
{
phashBlock = NULL;
@@ -414,6 +423,7 @@ public:
chainPower = CChainPower();
nTx = 0;
nChainTx = 0;
fRandomXVerified = false;
// Shieldex Index chain stats
nChainPayments = 0;

View File

@@ -30,7 +30,7 @@
// Must be kept in sync with configure.ac , ugh!
#define CLIENT_VERSION_MAJOR 1
#define CLIENT_VERSION_MINOR 0
#define CLIENT_VERSION_REVISION 2
#define CLIENT_VERSION_REVISION 3
#define CLIENT_VERSION_BUILD 50
//! Set to true for release, false for prerelease or test build

View File

@@ -33,10 +33,13 @@
#include "rand.h"
#include "sha2.h"
#if USE_BIP39_CACHE
// BIP39_WORDS is used unconditionally by the wordlist helpers below, so it must
// be defined even when the BIP39 cache is disabled (upstream places it inside
// the cache block by mistake).
int BIP39_WORDS = 2048;
#if USE_BIP39_CACHE
static int bip39_cache_index = 0;
static CONFIDENTIAL struct {

View File

@@ -56,8 +56,10 @@
#endif
// implement BIP39 caching
// Disabled: caching keeps the plaintext mnemonic/passphrase/seed in a static
// process-lifetime buffer, which we do not want in a wallet daemon.
#ifndef USE_BIP39_CACHE
#define USE_BIP39_CACHE 1
#define USE_BIP39_CACHE 0
#define BIP39_CACHE_SIZE 4
#endif

View File

@@ -0,0 +1,171 @@
// Copyright (c) 2016-2024 The Hush developers
// Distributed under the GPLv3 software license, see the accompanying
// file COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html
//
// Tests for HD-derived transparent keys (m/44'/coin'/0'/0/i) and the
// version-gated CHDChain serialization used to persist the transparent counter.
#include <gtest/gtest.h>
#include "key.h"
#include "chainparams.h"
#include "streams.h"
#include "uint256.h"
#include "util.h"
#include "version.h"
#include "wallet/wallet.h"
#include "wallet/walletdb.h"
#include "zcash/zip32.h"
// Build an in-memory wallet with a known seed + hdChain so that the
// HD-transparent path (IsHDTransparentEnabled) is active.
static void LoadSeedForTest(CWallet& wallet, const HDSeed& seed)
{
wallet.LoadHDSeed(seed);
CHDChain chain;
chain.nVersion = CHDChain::VERSION_HD_TRANSPARENT;
chain.seedFp = seed.Fingerprint();
chain.nCreateTime = 1;
wallet.SetHDChain(chain, true /* memonly */);
}
// Same seed must reproduce the same transparent addresses in the same order:
// this is the recovery guarantee that lets a seed-only restore find coinbase.
TEST(hdtransparent_tests, DeterministicFromSeed)
{
SelectParams(CBaseChainParams::MAIN);
RawHDSeed rawSeed(HD_WALLET_SEED_LENGTH, 0x42);
HDSeed seed(rawSeed);
std::vector<CKeyID> keysA;
{
CWallet wallet;
LoadSeedForTest(wallet, seed);
ASSERT_TRUE(wallet.IsHDTransparentEnabled());
LOCK(wallet.cs_wallet);
for (int i = 0; i < 5; i++) {
CPubKey pk = wallet.GenerateNewKey();
keysA.push_back(pk.GetID());
const CKeyMetadata& md = wallet.mapKeyMetadata[pk.GetID()];
EXPECT_EQ(md.seedFp, seed.Fingerprint());
EXPECT_EQ(md.hdKeypath, std::string("m/44'/141'/0'/0/") + std::to_string(i));
}
}
// Fresh wallet, same seed -> identical keys.
{
CWallet wallet;
LoadSeedForTest(wallet, seed);
LOCK(wallet.cs_wallet);
for (int i = 0; i < 5; i++) {
CPubKey pk = wallet.GenerateNewKey();
EXPECT_EQ(pk.GetID(), keysA[i]);
}
}
}
// Pin the exact derivation path so it can never silently change.
TEST(hdtransparent_tests, KnownDerivationPath)
{
SelectParams(CBaseChainParams::MAIN);
RawHDSeed rawSeed(HD_WALLET_SEED_LENGTH, 0x42);
HDSeed seed(rawSeed);
// Independently derive m/44'/141'/0'/0/0.
RawHDSeed raw = seed.RawSeed();
CExtKey m, purpose, coinType, account, external, child;
m.SetMaster(raw.data(), raw.size());
m.Derive(purpose, 44 | BIP32_HARDENED_KEY_LIMIT);
purpose.Derive(coinType, 141 | BIP32_HARDENED_KEY_LIMIT);
coinType.Derive(account, 0 | BIP32_HARDENED_KEY_LIMIT);
account.Derive(external, 0);
external.Derive(child, 0);
CKeyID expected = child.key.GetPubKey().GetID();
CWallet wallet;
LoadSeedForTest(wallet, seed);
LOCK(wallet.cs_wallet);
CPubKey pk = wallet.GenerateNewKey();
EXPECT_EQ(pk.GetID(), expected);
}
// A pre-existing v1 CHDChain record (no transparent counter) must still
// deserialize under v2 code, leaving transparentChildCounter at 0; and a v2
// record must round-trip the counter.
TEST(hdtransparent_tests, HDChainVersionCompat)
{
CHDChain v1;
v1.nVersion = CHDChain::VERSION_HD_BASE; // 1: transparentChildCounter not serialized
v1.seedFp = uint256S("0000000000000000000000000000000000000000000000000000000000000001");
v1.nCreateTime = 12345;
v1.saplingAccountCounter = 7;
CDataStream ss(SER_DISK, PROTOCOL_VERSION);
ss << v1;
CHDChain out; // default-constructed: SetNull() zeroes transparentChildCounter
ss >> out;
EXPECT_EQ(out.nVersion, +CHDChain::VERSION_HD_BASE); // unary + -> rvalue, avoid ODR-use of static const
EXPECT_EQ(out.seedFp, v1.seedFp);
EXPECT_EQ(out.nCreateTime, (int64_t)12345);
EXPECT_EQ(out.saplingAccountCounter, (uint32_t)7);
EXPECT_EQ(out.transparentChildCounter, (uint32_t)0);
CHDChain v2;
v2.nVersion = CHDChain::VERSION_HD_TRANSPARENT;
v2.saplingAccountCounter = 3;
v2.transparentChildCounter = 42;
CDataStream ss2(SER_DISK, PROTOCOL_VERSION);
ss2 << v2;
CHDChain out2;
ss2 >> out2;
EXPECT_EQ(out2.nVersion, +CHDChain::VERSION_HD_TRANSPARENT);
EXPECT_EQ(out2.saplingAccountCounter, (uint32_t)3);
EXPECT_EQ(out2.transparentChildCounter, (uint32_t)42);
}
// Restoring from a 32-byte seed hex reproduces the same keys as the source
// wallet, and refuses to run when a seed already exists.
TEST(hdtransparent_tests, RestoreFromSeedHex)
{
SelectParams(CBaseChainParams::MAIN);
RawHDSeed rawSeed(HD_WALLET_SEED_LENGTH, 0x7a);
HDSeed seed(rawSeed);
std::string seedHex = HexStr(seed.RawSeed());
// Source wallet: derive some keys.
std::vector<CKeyID> expected;
{
CWallet wallet;
LoadSeedForTest(wallet, seed);
LOCK(wallet.cs_wallet);
for (int i = 0; i < 3; i++)
expected.push_back(wallet.GenerateNewKey().GetID());
}
// Restored wallet: inject the seed hex, pre-derive, and compare.
{
CWallet wallet;
ASSERT_TRUE(wallet.SetHDSeedFromHex(seedHex));
// Second attempt must fail: a seed already exists.
EXPECT_FALSE(wallet.SetHDSeedFromHex(seedHex));
wallet.TopUpHDTransparentKeys(3, 1);
LOCK(wallet.cs_wallet);
for (int i = 0; i < 3; i++)
EXPECT_TRUE(wallet.HaveKey(expected[i]));
}
// Bad input is rejected.
{
CWallet wallet;
EXPECT_FALSE(wallet.SetHDSeedFromHex("nothex"));
EXPECT_FALSE(wallet.SetHDSeedFromHex("abcd")); // too short
}
}

View File

@@ -0,0 +1,141 @@
// Copyright (c) 2016-2024 The Hush developers
// Distributed under the GPLv3 software license, see the accompanying
// file COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html
//
// Proves that a BIP39 seed phrase produces the SAME transparent and shielded
// addresses on the DragonX full node as in SilentDragonXLite. The proof chain:
// phrase -> entropy (round-trip) -> 64-byte BIP39 seed (known-answer)
// -> z/t addresses (wallet path == direct ZIP-32/BIP44 derivation).
// The 64-byte seed is anchored to the well-known BIP39 value for the all-zero
// "abandon...art" entropy with an EMPTY passphrase, which is exactly what
// SilentDragonXLite's tiny-bip39 0.6.2 feeds into the same coin_type=141 paths.
#include <gtest/gtest.h>
#include "chainparams.h"
#include "key.h"
#include "key_io.h"
#include "util.h"
#include "wallet/mnemonic.h"
#include "wallet/wallet.h"
#include "zcash/Address.hpp"
#include "zcash/zip32.h"
// The canonical 24-word phrase for 32 bytes of all-zero entropy.
static const char* ABANDON_ART =
"abandon abandon abandon abandon abandon abandon abandon abandon "
"abandon abandon abandon abandon abandon abandon abandon abandon "
"abandon abandon abandon abandon abandon abandon abandon art";
// The standard BIP39 seed for that phrase with an EMPTY passphrase
// (PBKDF2-HMAC-SHA512, 2048 rounds, salt "mnemonic"). Matches tiny-bip39.
static const char* SEED64_HEX =
"408b285c123836004f4b8842c89324c1f01382450c0d439af345ba7fc49acf70"
"5489c6fc77dbd4e3dc1dd8cc6bc9f043db8ada1e243c4a0eafb290d399480840";
// First shielded address for a 64-byte seed: m/32'/141'/0' default address.
static std::string DeriveZAddrFromSeed64(RawHDSeed seed64)
{
HDSeed s(seed64);
auto m = libzcash::SaplingExtendedSpendingKey::Master(s);
auto xsk = m.Derive(32 | ZIP32_HARDENED_KEY_LIMIT)
.Derive(141 | ZIP32_HARDENED_KEY_LIMIT)
.Derive(0 | ZIP32_HARDENED_KEY_LIMIT);
return EncodePaymentAddress(xsk.DefaultAddress());
}
// First transparent address for a BIP32 master over `seedBytes`:
// m/44'/141'/0'/0/0.
static std::string DeriveTAddrFromSeedBytes(RawHDSeed seedBytes)
{
CExtKey master, purpose, coinType, account, external, child;
master.SetMaster(seedBytes.data(), seedBytes.size());
master.Derive(purpose, 44 | BIP32_HARDENED_KEY_LIMIT);
purpose.Derive(coinType, 141 | BIP32_HARDENED_KEY_LIMIT);
coinType.Derive(account, 0 | BIP32_HARDENED_KEY_LIMIT);
account.Derive(external, 0);
external.Derive(child, 0);
return EncodeDestination(child.key.GetPubKey().GetID());
}
// The 64-byte seed derived from the mnemonic must equal the known BIP39 value.
// This is the cross-wallet anchor: SilentDragonXLite feeds the identical seed.
TEST(mnemonic_compat, Bip39SeedKnownAnswer)
{
RawHDSeed entropy(32, 0);
RawHDSeed seed64;
ASSERT_TRUE(Bip39SeedFromEntropy(entropy, seed64));
ASSERT_EQ(seed64.size(), (size_t)64);
EXPECT_EQ(HexStr(seed64.begin(), seed64.end()), std::string(SEED64_HEX));
}
TEST(mnemonic_compat, EntropyPhraseRoundTrip)
{
RawHDSeed zeros(32, 0);
std::string phrase;
ASSERT_TRUE(EntropyToMnemonic(zeros, phrase));
EXPECT_EQ(phrase, std::string(ABANDON_ART));
EXPECT_TRUE(MnemonicIsValid(ABANDON_ART));
RawHDSeed entropy;
ASSERT_TRUE(MnemonicToEntropy(ABANDON_ART, entropy));
EXPECT_EQ(entropy.size(), (size_t)32);
EXPECT_EQ(HexStr(entropy.begin(), entropy.end()), std::string(64, '0'));
// Bad checksum / unknown words are rejected.
EXPECT_FALSE(MnemonicIsValid("abandon abandon abandon"));
EXPECT_FALSE(MnemonicIsValid("clearly not valid bip39 words at all here"));
RawHDSeed junk;
EXPECT_FALSE(MnemonicToEntropy("clearly not valid bip39 words at all here", junk));
}
// The wallet's mnemonic derivation must reproduce the exact addresses obtained
// by driving ZIP-32 / BIP44 directly from the known 64-byte seed, and must be
// deterministic across wallets.
TEST(mnemonic_compat, WalletDerivesSdxliteAddresses)
{
SelectParams(CBaseChainParams::MAIN);
RawHDSeed zeros(32, 0), seed64;
ASSERT_TRUE(Bip39SeedFromEntropy(zeros, seed64));
const std::string expZ = DeriveZAddrFromSeed64(seed64);
const std::string expT = DeriveTAddrFromSeedBytes(seed64);
EXPECT_EQ(expZ.substr(0, 2), "zs"); // sapling HRP for mainnet
CWallet wallet;
ASSERT_TRUE(wallet.SetHDSeedFromMnemonic(ABANDON_ART));
ASSERT_TRUE(wallet.IsMnemonicSeed());
{
LOCK(wallet.cs_wallet);
EXPECT_EQ(EncodePaymentAddress(wallet.GenerateNewSaplingZKey()), expZ);
EXPECT_EQ(EncodeDestination(wallet.GenerateNewKey().GetID()), expT);
}
// Same phrase, fresh wallet -> identical first addresses.
CWallet wallet2;
ASSERT_TRUE(wallet2.SetHDSeedFromMnemonic(ABANDON_ART));
{
LOCK(wallet2.cs_wallet);
EXPECT_EQ(EncodePaymentAddress(wallet2.GenerateNewSaplingZKey()), expZ);
EXPECT_EQ(EncodeDestination(wallet2.GenerateNewKey().GetID()), expT);
}
// The phrase round-trips out of the wallet.
std::string exported;
ASSERT_TRUE(wallet.GetMnemonicPhrase(exported));
EXPECT_EQ(exported, std::string(ABANDON_ART));
}
// Negative: feeding the 32-byte entropy DIRECTLY as the seed (the classic
// interop bug) must produce a different address than the 64-byte BIP39 seed.
TEST(mnemonic_compat, RawEntropyDiffersFromMnemonicSeed)
{
SelectParams(CBaseChainParams::MAIN);
RawHDSeed zeros(32, 0), seed64;
ASSERT_TRUE(Bip39SeedFromEntropy(zeros, seed64));
const std::string seedT = DeriveTAddrFromSeedBytes(seed64); // correct (64-byte)
const std::string entropyT = DeriveTAddrFromSeedBytes(zeros); // wrong (32-byte)
EXPECT_NE(seedT, entropyT);
}

View File

@@ -0,0 +1,172 @@
// Copyright (c) 2024-2026 The DragonX developers
// Distributed under the GPLv3 software license, see the accompanying
// file COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html
//
// Consensus-equivalence test for the parallel RandomX pre-verification pool. The pool is purely an
// optimization: a block's transient fRandomXVerified flag (set by CRandomXCheck on a real hash
// match) only lets CheckBlockHeader SKIP the inline recompute. So for every block the pool's
// outcome must equal the inline CheckRandomXSolution outcome — `(preVerified || inline) == inline`.
// We exercise a valid solution, a corrupted solution, and confirm the pool never "succeeds" on a
// block the inline check would reject.
#include <gtest/gtest.h>
#include <string>
#include <vector>
#include "arith_uint256.h"
#include "chain.h"
#include "chainparams.h"
#include "pow.h"
#include "primitives/block.h"
#include "RandomX/src/randomx.h"
#include "hush_defs.h"
#include "util.h"
#include <boost/thread.hpp>
#include <memory>
extern int32_t HUSH_LOADINGBLOCKS;
extern bool fCheckpointsEnabled;
namespace {
// Compute the correct RandomX solution for a header using a standalone reference light VM, via the
// SAME key + input helpers the validator uses (so the bytes/key match exactly).
void ReferenceRandomXHash(const CBlockHeader& hdr, const std::string& key, unsigned char out[RANDOMX_HASH_SIZE])
{
std::vector<unsigned char> in = GetRandomXInput(hdr);
randomx_flags flags = randomx_get_flags();
randomx_cache* c = randomx_alloc_cache(flags);
ASSERT_NE(c, nullptr);
randomx_init_cache(c, key.data(), key.size());
randomx_vm* vm = randomx_create_vm(flags, c, nullptr);
ASSERT_NE(vm, nullptr);
randomx_calculate_hash(vm, in.data(), in.size(), out);
randomx_destroy_vm(vm);
randomx_release_cache(c);
}
} // namespace
TEST(RandomXPreVerify, ConsensusEquivalence)
{
// Force RandomX validation to actually run at low heights in the test harness.
uint32_t savedAlgo = ASSETCHAINS_ALGO, savedRx = ASSETCHAINS_RANDOMX;
int32_t savedVal = ASSETCHAINS_RANDOMX_VALIDATION, savedLoad = HUSH_LOADINGBLOCKS;
bool savedCkpt = fCheckpointsEnabled;
ASSETCHAINS_RANDOMX = 2; // a distinct nonzero algo id
ASSETCHAINS_ALGO = ASSETCHAINS_RANDOMX;
ASSETCHAINS_RANDOMX_VALIDATION = 1; // enforce from height 1
HUSH_LOADINGBLOCKS = 0; // not in initial-load (else RandomX skipped)
fCheckpointsEnabled = false; // avoid the below-checkpoint skip
const int32_t height = 10; // < interval+lag -> the chain-params initial key (no chainActive needed)
CBlockHeader hdr;
hdr.nVersion = 4;
hdr.hashPrevBlock = uint256S("0x0000000000000000000000000000000000000000000000000000000000000001");
hdr.hashMerkleRoot = uint256S("0x0000000000000000000000000000000000000000000000000000000000000002");
hdr.hashFinalSaplingRoot = uint256S("0x0000000000000000000000000000000000000000000000000000000000000003");
hdr.nTime = 1700000000;
hdr.nBits = 0x200f0f0f;
hdr.nNonce = uint256S("0x0000000000000000000000000000000000000000000000000000000000000004");
std::string key = GetRandomXKey(height);
ASSERT_FALSE(key.empty());
unsigned char good[RANDOMX_HASH_SIZE];
ReferenceRandomXHash(hdr, key, good);
// Run the pool path synchronously on this thread (CRandomXCheck creates its own thread_local VM).
auto poolVerifies = [&](const CBlockHeader& h) -> bool {
RandomXValidatorPrepareKey(key); // load the shared cache with this key
bool slot = false;
CRandomXCheck chk(key, GetRandomXInput(h), h.nSolution.data(), &slot);
chk();
return slot;
};
// Case 1 — valid solution: both inline and pool accept; equivalence holds.
hdr.nSolution.assign(good, good + RANDOMX_HASH_SIZE);
EXPECT_TRUE(CheckRandomXSolution(&hdr, height));
EXPECT_TRUE(poolVerifies(hdr));
EXPECT_EQ(poolVerifies(hdr) || CheckRandomXSolution(&hdr, height), CheckRandomXSolution(&hdr, height));
// Case 2 — corrupted solution: both reject; the pool must NOT set verified.
{
CBlockHeader bad = hdr;
bad.nSolution[0] ^= 0xff;
EXPECT_FALSE(CheckRandomXSolution(&bad, height));
EXPECT_FALSE(poolVerifies(bad));
EXPECT_EQ(poolVerifies(bad) || CheckRandomXSolution(&bad, height), CheckRandomXSolution(&bad, height));
}
// Case 3 — a verified flag on the block lets CheckBlockHeader skip, but verified is only ever set
// by a real hash match, so it can never mask an invalid block. (Pool returns false for the bad
// block above, so its fRandomXVerified stays false and the inline path rejects it at connect.)
ASSETCHAINS_ALGO = savedAlgo; ASSETCHAINS_RANDOMX = savedRx;
ASSETCHAINS_RANDOMX_VALIDATION = savedVal; HUSH_LOADINGBLOCKS = savedLoad;
fCheckpointsEnabled = savedCkpt;
}
// A/B: serial inline verification (single VM) vs the parallel pool (worker threads). Directly
// measures the speedup the pool delivers. We don't care about validity here (mismatched solutions
// still cost a full hash), only wall-clock. parallel must beat serial whenever >1 core is used.
TEST(RandomXPreVerify, ParallelSpeedup)
{
uint32_t savedAlgo = ASSETCHAINS_ALGO, savedRx = ASSETCHAINS_RANDOMX;
int32_t savedVal = ASSETCHAINS_RANDOMX_VALIDATION, savedLoad = HUSH_LOADINGBLOCKS;
bool savedCkpt = fCheckpointsEnabled;
ASSETCHAINS_RANDOMX = 2; ASSETCHAINS_ALGO = ASSETCHAINS_RANDOMX;
ASSETCHAINS_RANDOMX_VALIDATION = 1; HUSH_LOADINGBLOCKS = 0; fCheckpointsEnabled = false;
const int32_t height = 10;
std::string key = GetRandomXKey(height);
ASSERT_FALSE(key.empty());
ASSERT_TRUE(RandomXValidatorPrepareKey(key));
const int M = 16; // blocks to verify in the window
std::vector<CBlockHeader> hdrs(M);
for (int i = 0; i < M; i++) {
hdrs[i].nVersion = 4;
hdrs[i].nTime = 1700000000 + i;
hdrs[i].nBits = 0x200f0f0f;
hdrs[i].nNonce = ArithToUint256(arith_uint256(i + 1)); // distinct inputs
hdrs[i].nSolution.assign(RANDOMX_HASH_SIZE, 0); // arbitrary; we time the hash
}
// Serial baseline: inline single-VM verification (each call hashes, then mismatches -> false).
int64_t t0 = GetTimeMicros();
for (int i = 0; i < M; i++) CheckRandomXSolution(&hdrs[i], height);
int64_t serialUs = GetTimeMicros() - t0;
// Parallel: spawn K-1 workers + the master (this thread) joining via Wait().
int K = std::min(8, std::max(2, (int)boost::thread::hardware_concurrency()));
boost::thread_group workers;
for (int i = 0; i < K - 1; i++) workers.create_thread(&ThreadRandomXVerify);
std::unique_ptr<bool[]> slots(new bool[M]());
std::vector<CRandomXCheck> checks;
checks.reserve(M);
for (int i = 0; i < M; i++)
checks.push_back(CRandomXCheck(key, GetRandomXInput(hdrs[i]), hdrs[i].nSolution.data(), &slots[i]));
int64_t t1 = GetTimeMicros();
{
CCheckQueueControl<CRandomXCheck> control(&rxCheckQueue);
control.Add(checks);
control.Wait();
}
int64_t parallelUs = GetTimeMicros() - t1;
workers.interrupt_all();
workers.join_all();
printf("[ RandomX A/B ] %d blocks: serial(1 VM)=%ldms, parallel(%d threads)=%ldms, speedup=%.1fx\n",
M, (long)(serialUs / 1000), K, (long)(parallelUs / 1000),
(double)serialUs / (double)std::max<int64_t>(1, parallelUs));
EXPECT_LT(parallelUs, serialUs); // parallel must be faster than serial on a multi-core box
ASSETCHAINS_ALGO = savedAlgo; ASSETCHAINS_RANDOMX = savedRx;
ASSETCHAINS_RANDOMX_VALIDATION = savedVal; HUSH_LOADINGBLOCKS = savedLoad;
fCheckpointsEnabled = savedCkpt;
}

View File

@@ -580,70 +580,98 @@ int TLSManager::threadSocketHandler(CNode* pnode, fd_set& fdsetRecv, fd_set& fds
char pchBuf[0x10000];
bool bIsSSL = false;
int nBytes = 0, nRet = 0;
// Drain the socket in a bounded loop rather than one read per select pass: a single
// 64K read per pass underfills high-bandwidth/high-latency links. Cap the reads per
// pass and honor the receive-flood back-pressure so one peer can neither exhaust
// memory nor starve other peers within this pass.
int nDrainReads = 0;
const int MAX_DRAIN_READS = 16; // up to ~1 MiB per peer per pass (fairness across peers)
// Pre-read back-pressure: gate on the flood ceiling BEFORE each read so the per-peer
// recv buffer high-water stays at ReceiveFloodSize()+one read (matching the select()
// FD_SET gate), and track bytes locally to avoid the O(n) GetTotalRecvSize() per pass.
const int64_t nRecvBase = (int64_t)pnode->GetTotalRecvSize();
int64_t nPassBytes = 0;
bool fKeepReading = true;
while (fKeepReading) {
if (nRecvBase + nPassBytes > (int64_t)ReceiveFloodSize())
break;
{
LOCK(pnode->cs_hSocket);
{
LOCK(pnode->cs_hSocket);
if (pnode->hSocket == INVALID_SOCKET) {
LogPrint("tls", "Receive: connection with %s is already closed\n", pnode->addr.ToString());
return -1;
}
if (pnode->hSocket == INVALID_SOCKET) {
LogPrint("tls", "Receive: connection with %s is already closed\n", pnode->addr.ToString());
return -1;
bIsSSL = (pnode->ssl != NULL);
if (bIsSSL) {
wolfSSL_ERR_clear_error(); // clear the error queue, otherwise we may be reading an old error that occurred previously in the current thread
nBytes = wolfSSL_read(pnode->ssl, pchBuf, sizeof(pchBuf));
nRet = wolfSSL_get_error(pnode->ssl, nBytes);
} else {
nBytes = recv(pnode->hSocket, pchBuf, sizeof(pchBuf), MSG_DONTWAIT);
nRet = WSAGetLastError();
}
}
bIsSSL = (pnode->ssl != NULL);
if (bIsSSL) {
wolfSSL_ERR_clear_error(); // clear the error queue, otherwise we may be reading an old error that occurred previously in the current thread
nBytes = wolfSSL_read(pnode->ssl, pchBuf, sizeof(pchBuf));
nRet = wolfSSL_get_error(pnode->ssl, nBytes);
} else {
nBytes = recv(pnode->hSocket, pchBuf, sizeof(pchBuf), MSG_DONTWAIT);
nRet = WSAGetLastError();
}
}
if (nBytes > 0) {
if (!pnode->ReceiveMsgBytes(pchBuf, nBytes))
pnode->CloseSocketDisconnect();
pnode->nLastRecv = GetTime();
pnode->nRecvBytes += nBytes;
pnode->RecordBytesRecv(nBytes);
} else if (nBytes == 0) {
if (bIsSSL) {
unsigned long error = ERR_get_error();
const char* error_str = ERR_error_string(error, NULL);
LogPrint("tls", "TLS: WARNING: %s: %s():%d - SSL_read err: %s\n",
__FILE__, __func__, __LINE__, error_str);
}
// socket closed gracefully (peer disconnected)
if (!pnode->fDisconnect)
LogPrint("tls", "socket closed (%s)\n", pnode->addr.ToString());
pnode->CloseSocketDisconnect();
} else if (nBytes < 0) {
// error
if (bIsSSL) {
if (nRet != WOLFSSL_ERROR_WANT_READ && nRet != WOLFSSL_ERROR_WANT_WRITE)
{
if (!pnode->fDisconnect)
LogPrintf("TLS: ERROR: SSL_read %s\n", ERR_error_string(nRet, NULL));
if (nBytes > 0) {
if (!pnode->ReceiveMsgBytes(pchBuf, nBytes)) {
pnode->CloseSocketDisconnect();
fKeepReading = false;
}
pnode->nLastRecv = GetTime();
pnode->nRecvBytes += nBytes;
pnode->RecordBytesRecv(nBytes);
nPassBytes += nBytes;
// Keep draining only while the socket likely has more data (we filled the
// buffer, or TLS has buffered decrypted bytes) and within the per-pass cap.
// The flood ceiling is enforced pre-read at the top of the loop.
if (fKeepReading) {
bool fMore = (nBytes == (int)sizeof(pchBuf)) || (bIsSSL && wolfSSL_pending(pnode->ssl) > 0);
if (!fMore || ++nDrainReads >= MAX_DRAIN_READS)
fKeepReading = false;
}
} else if (nBytes == 0) {
if (bIsSSL) {
unsigned long error = ERR_get_error();
const char* error_str = ERR_error_string(error, NULL);
LogPrint("tls", "TLS: WARNING: %s: %s():%d - SSL_read - code[0x%x], err: %s\n",
__FILE__, __func__, __LINE__, nRet, error_str);
LogPrint("tls", "TLS: WARNING: %s: %s():%d - SSL_read err: %s\n",
__FILE__, __func__, __LINE__, error_str);
}
// socket closed gracefully (peer disconnected)
if (!pnode->fDisconnect)
LogPrint("tls", "socket closed (%s)\n", pnode->addr.ToString());
pnode->CloseSocketDisconnect();
fKeepReading = false;
} else if (nBytes < 0) {
// error
if (bIsSSL) {
if (nRet != WOLFSSL_ERROR_WANT_READ && nRet != WOLFSSL_ERROR_WANT_WRITE)
{
if (!pnode->fDisconnect)
LogPrintf("TLS: ERROR: SSL_read %s\n", ERR_error_string(nRet, NULL));
pnode->CloseSocketDisconnect();
unsigned long error = ERR_get_error();
const char* error_str = ERR_error_string(error, NULL);
LogPrint("tls", "TLS: WARNING: %s: %s():%d - SSL_read - code[0x%x], err: %s\n",
__FILE__, __func__, __LINE__, nRet, error_str);
} else {
// preventive measure from exhausting CPU usage
MilliSleep(1); // 1 msec
}
} else {
// preventive measure from exhausting CPU usage
MilliSleep(1); // 1 msec
}
} else {
if (nRet != WSAEWOULDBLOCK && nRet != WSAEMSGSIZE && nRet != WSAEINTR && nRet != WSAEINPROGRESS) {
if (!pnode->fDisconnect)
LogPrintf("TLS: ERROR: socket recv %s\n", NetworkErrorString(nRet));
pnode->CloseSocketDisconnect();
if (nRet != WSAEWOULDBLOCK && nRet != WSAEMSGSIZE && nRet != WSAEINTR && nRet != WSAEINPROGRESS) {
if (!pnode->fDisconnect)
LogPrintf("TLS: ERROR: socket recv %s\n", NetworkErrorString(nRet));
pnode->CloseSocketDisconnect();
}
}
fKeepReading = false;
}
}
}

View File

@@ -43,6 +43,7 @@
#endif
#include "main.h"
#include "metrics.h"
#include "pow.h"
#include "miner.h"
#include "net.h"
#include "rpc/server.h"
@@ -176,7 +177,7 @@ public:
// Writes do not need similar protection, as failure to write is handled by the caller.
};
static CCoinsViewDB *pcoinsdbview = NULL;
CCoinsViewDB *pcoinsdbview = NULL; // global (declared extern in main.h) for UTXO-snapshot dump/load
static CCoinsViewErrorCatcher *pcoinscatcher = NULL;
static boost::scoped_ptr<ECCVerifyHandle> globalVerifyHandle;
@@ -387,7 +388,7 @@ std::string HelpMessage(HelpMessageMode mode)
}
strUsage += HelpMessageOpt("-datadir=<dir>", _("Specify data directory (this path cannot use '~')"));
strUsage += HelpMessageOpt("-exportdir=<dir>", _("Specify directory to be used when exporting data"));
strUsage += HelpMessageOpt("-dbcache=<n>", strprintf(_("Set database cache size in megabytes (%d to %d, default: %d)"), nMinDbCache, nMaxDbCache, nDefaultDbCache));
strUsage += HelpMessageOpt("-dbcache=<n>", strprintf(_("Set database cache size in megabytes (%d to %d). Default: adaptive - uses most free RAM to speed up initial block download (far fewer UTXO flushes to disk) and automatically shrinks if other applications need memory, always leaving a reserve free. Setting a fixed value disables adaptive sizing."), nMinDbCache, nMaxDbCache));
strUsage += HelpMessageOpt("-loadblock=<file>", _("Imports blocks from external blk000??.dat file") + " " + _("on startup"));
strUsage += HelpMessageOpt("-maxdebugfilesize=<n>", strprintf(_("Set the max size of the debug.log file (default: %u)"), 15));
strUsage += HelpMessageOpt("-maxorphantx=<n>", strprintf(_("Keep at most <n> unconnectable transactions in memory (default: %u)"), DEFAULT_MAX_ORPHAN_TRANSACTIONS));
@@ -395,6 +396,7 @@ std::string HelpMessage(HelpMessageMode mode)
strUsage += HelpMessageOpt("-mempooltxinputlimit=<n>", _("[DEPRECATED/IGNORED] Set the maximum number of transparent inputs in a transaction that the mempool will accept (default: 0 = no limit applied)"));
strUsage += HelpMessageOpt("-par=<n>", strprintf(_("Set the number of script verification threads (%u to %d, 0 = auto, <0 = leave that many cores free, default: %d)"),
-(int)boost::thread::hardware_concurrency(), MAX_SCRIPTCHECK_THREADS, DEFAULT_SCRIPTCHECK_THREADS));
strUsage += HelpMessageOpt("-randomxverifythreads=<n>", strprintf(_("Number of threads for parallel RandomX PoW pre-verification of post-checkpoint blocks during sync (0 = inline only, max %d, default: same as -par)"), MAX_SCRIPTCHECK_THREADS));
#ifndef _WIN32
strUsage += HelpMessageOpt("-pid=<file>", strprintf(_("Specify pid file (default: %s)"), "hushd.pid"));
#endif
@@ -465,6 +467,12 @@ std::string HelpMessage(HelpMessageMode mode)
strUsage += HelpMessageGroup(_("Wallet options:"));
strUsage += HelpMessageOpt("-disablewallet", _("Do not load the wallet and disable wallet RPC calls"));
strUsage += HelpMessageOpt("-keypool=<n>", strprintf(_("Set key pool size to <n> (default: %u)"), 100));
strUsage += HelpMessageOpt("-hdtransparent", strprintf(_("Derive transparent addresses from the HD seed so they can be recovered from it (default: %u)"), 1));
strUsage += HelpMessageOpt("-hdseed=<hex>", _("Restore a fresh/empty wallet from a 32- or 64-byte HD seed hex (the value shown in z_exportwallet's '# HDSeed=' line). WARNING: exposes the seed to your shell history and process list."));
strUsage += HelpMessageOpt("-mnemonic=<words>", _("Restore/create a fresh/empty wallet from a BIP39 seed phrase, compatible with SilentDragonXLite (English, no passphrase). WARNING: exposes the phrase to your shell history and process list; prefer DRAGONX.conf with tight permissions."));
strUsage += HelpMessageOpt("-usemnemonic", strprintf(_("Create new wallets from a fresh BIP39 seed phrase so the 24 words can be exported (z_exportmnemonic) and used in SilentDragonXLite (default: %u)"), 0));
strUsage += HelpMessageOpt("-hdtransparentgaplimit=<n>", strprintf(_("On -mnemonic/-hdseed restore, pre-derive this many HD transparent keys so a rescan can find coinbase paid to them (default: %u)"), 1000));
strUsage += HelpMessageOpt("-mnemonicsaplinggap=<n>", strprintf(_("On -mnemonic/-hdseed restore, pre-derive this many shielded (Sapling) addresses so a rescan can find notes sent to them (default: %u)"), 100));
strUsage += HelpMessageOpt("-consolidation", _("Enable auto Sapling note consolidation (default: false)"));
strUsage += HelpMessageOpt("-consolidationinterval", _("Block interval between consolidations (default: 25)"));
strUsage += HelpMessageOpt("-consolidatesaplingaddress=<zaddr>", _("Specify Sapling Address to Consolidate. (default: all)"));
@@ -987,6 +995,123 @@ bool AppInitServers(boost::thread_group& threadGroup)
*/
extern int32_t HUSH_REWIND;
// --- Adaptive coins-cache sizing -------------------------------------------------------------
// The in-memory UTXO/coins cache (nCoinCacheUsage) is the biggest lever on IBD speed: a bigger
// cache means far fewer chainstate flushes to disk. We size it to use most of RAM, but a scheduled
// background task (AdjustCoinCacheForMemoryPressure, registered in AppInit2) shrinks the target when
// free system memory runs low — e.g. the user opens other apps — and grows it back when memory frees
// up, always leaving a reserve free for the rest of the system. The existing per-block flush
// (FlushStateToDisk, FLUSH_STATE_IF_NEEDED, which fires when cacheSize > nCoinCacheUsage) enforces
// whatever target is current, so the task only moves the threshold: it never touches cs_main or the
// flush path. NOTE: the coins cache is application heap, not OS file cache — "freeing" it means an
// early flush that clears the map; on Linux the allocator returns the pages, on Windows the heap
// returns them best-effort (RSS may lag), but either way the node stops growing past the target.
// windows.h / <unistd.h> arrive via compat.h (net.h). Memory helpers return 0 if undeterminable.
static int64_t GetPhysicalMemoryMB()
{
#ifdef WIN32
MEMORYSTATUSEX status;
status.dwLength = sizeof(status);
if (GlobalMemoryStatusEx(&status))
return (int64_t)(status.ullTotalPhys / (1024 * 1024));
return 0;
#elif defined(_SC_PHYS_PAGES) && defined(_SC_PAGESIZE)
long pages = sysconf(_SC_PHYS_PAGES);
long pageSize = sysconf(_SC_PAGESIZE);
if (pages > 0 && pageSize > 0)
return (int64_t)((int64_t)pages * (int64_t)pageSize / (1024 * 1024));
return 0;
#else
return 0;
#endif
}
// Currently-available (allocatable) physical RAM in MiB. On Linux uses MemAvailable (counts
// reclaimable page cache), falling back to truly-free pages.
static int64_t GetAvailableMemoryMB()
{
#ifdef WIN32
MEMORYSTATUSEX status;
status.dwLength = sizeof(status);
if (GlobalMemoryStatusEx(&status))
return (int64_t)(status.ullAvailPhys / (1024 * 1024));
return 0;
#else
FILE* f = fopen("/proc/meminfo", "r");
if (f) {
char line[256];
long long availKB = -1;
while (fgets(line, sizeof(line), f)) {
if (sscanf(line, "MemAvailable: %lld kB", &availKB) == 1)
break;
}
fclose(f);
if (availKB >= 0)
return (int64_t)(availKB / 1024);
}
#if defined(_SC_AVPHYS_PAGES) && defined(_SC_PAGESIZE)
long pages = sysconf(_SC_AVPHYS_PAGES);
long pageSize = sysconf(_SC_PAGESIZE);
if (pages > 0 && pageSize > 0)
return (int64_t)((int64_t)pages * (int64_t)pageSize / (1024 * 1024));
#endif
return 0;
#endif
}
// RAM (MiB) to always keep free for the OS and other applications: 20% of total, at least 2 GiB.
static int64_t GetMemoryReserveMB()
{
int64_t ramMB = GetPhysicalMemoryMB();
int64_t reserve = (ramMB > 0) ? ramMB / 5 : 2048; // 20%
if (reserve < 2048) reserve = 2048;
return reserve;
}
// Startup -dbcache default: use most of RAM (total minus the reserve), clamped to
// [nDefaultDbCache, nMaxDbCache] MiB. Falls back to the fixed default if RAM can't be detected.
static int64_t GetDefaultDbCacheMB()
{
int64_t ramMB = GetPhysicalMemoryMB();
if (ramMB <= 0)
return nDefaultDbCache;
int64_t cacheMB = ramMB - GetMemoryReserveMB();
if (cacheMB < nDefaultDbCache) cacheMB = nDefaultDbCache;
if (cacheMB > nMaxDbCache) cacheMB = nMaxDbCache;
return cacheMB;
}
// Ceiling (bytes) the adaptive task may grow the coins cache back up to (the startup nCoinCacheUsage).
static size_t g_nMaxCoinCacheUsage = 0;
static const int64_t g_nMinCoinCacheMB = 256; // never thrash below this working set
// Scheduled task: nudge nCoinCacheUsage toward "use all RAM except the reserve". If free RAM is below
// the reserve we shrink the target (the next per-block flush releases the excess); if there is spare
// RAM we grow it back toward the startup ceiling. Lock-free: it only reads system memory and writes
// the aligned size_t threshold that the flush path reads.
static void AdjustCoinCacheForMemoryPressure()
{
if (g_nMaxCoinCacheUsage == 0)
return; // adaptive sizing disabled (user pinned -dbcache) or RAM undetectable
int64_t availMB = GetAvailableMemoryMB();
if (availMB <= 0)
return; // can't measure pressure; leave the target untouched
int64_t reserveMB = GetMemoryReserveMB();
// Error term: free RAM beyond the reserve. >0 => spare, grow; <0 => pressure, shrink.
int64_t errMB = availMB - reserveMB;
// Deadband: ignore small fluctuations so the target settles instead of oscillating.
if (errMB > -256 && errMB < 256)
return;
int64_t curTargetMB = (int64_t)(nCoinCacheUsage >> 20);
// Damped proportional step (gain 1/4) toward "free RAM == reserve"; the clamps bound it and the
// per-block flush (FLUSH_STATE_IF_NEEDED) enforces a lowered target within ~one block during IBD.
int64_t newTargetMB = curTargetMB + errMB / 4;
int64_t ceilMB = (int64_t)(g_nMaxCoinCacheUsage >> 20);
if (newTargetMB > ceilMB) newTargetMB = ceilMB;
if (newTargetMB < g_nMinCoinCacheMB) newTargetMB = g_nMinCoinCacheMB;
nCoinCacheUsage = (size_t)(newTargetMB << 20);
}
bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler)
{
//fprintf(stderr,"%s start\n", __FUNCTION__);
@@ -1309,6 +1434,29 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler)
else if (nScriptCheckThreads > MAX_SCRIPTCHECK_THREADS)
nScriptCheckThreads = MAX_SCRIPTCHECK_THREADS;
// Parallel RandomX pre-verification threads (speeds up post-checkpoint sync). Defaults to the
// script-check thread count — RandomX pre-verify and script checks do not run simultaneously
// within a single connect, so they can share the same budget. 0 disables (inline-only).
nRandomXVerifyThreads = GetArg("-randomxverifythreads", nScriptCheckThreads);
if (nRandomXVerifyThreads < 0)
nRandomXVerifyThreads = 0;
else if (nRandomXVerifyThreads > MAX_SCRIPTCHECK_THREADS)
nRandomXVerifyThreads = MAX_SCRIPTCHECK_THREADS;
// Per-peer block-download window (see MAX_BLOCKS_IN_TRANSIT_PER_PEER). Raising this lifts
// the bandwidth-delay-product ceiling on high-latency peers during IBD. Clamp to a sane range.
MAX_BLOCKS_IN_TRANSIT_PER_PEER = GetArg("-maxblocksintransit", DEFAULT_MAX_BLOCKS_IN_TRANSIT_PER_PEER);
if (MAX_BLOCKS_IN_TRANSIT_PER_PEER < 1)
MAX_BLOCKS_IN_TRANSIT_PER_PEER = 1;
else if (MAX_BLOCKS_IN_TRANSIT_PER_PEER > 4096)
MAX_BLOCKS_IN_TRANSIT_PER_PEER = 4096;
LogPrintf("Per-peer max blocks in transit: %d\n", MAX_BLOCKS_IN_TRANSIT_PER_PEER);
// Opt-in bulk block streaming (DragonX). Drives the requester branch in SendMessages and, when
// set, also advertises NODE_BULKBLOCKS below so we serve bulk ranges to peers. OFF by default.
fBulkBlockSync = GetBoolArg("-bulkblocksync", DEFAULT_BULKBLOCKSYNC);
LogPrintf("Bulk block streaming: %s\n", fBulkBlockSync ? "enabled" : "disabled");
fServer = GetBoolArg("-server", false);
//fprintf(stderr,"%s tik6\n", __FUNCTION__);
@@ -1545,6 +1693,14 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler)
threadGroup.create_thread(&ThreadScriptCheck);
}
// Spawn the parallel RandomX pre-verification worker pool (the connect thread joins as the Nth
// worker via CCheckQueueControl::Wait, so spawn N-1 here, mirroring ThreadScriptCheck).
if (ASSETCHAINS_ALGO == ASSETCHAINS_RANDOMX && nRandomXVerifyThreads > 0) {
LogPrintf("Using %u threads for parallel RandomX pre-verification\n", nRandomXVerifyThreads);
for (int i = 0; i < nRandomXVerifyThreads - 1; i++)
threadGroup.create_thread(&ThreadRandomXVerify);
}
//fprintf(stderr,"%s tik13\n", __FUNCTION__);
// Start the lightweight task scheduler thread
@@ -1840,7 +1996,7 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler)
LogPrintf("* Compression is %s\n", dbCompression ? "enabled" : "disabled");
// cache size calculations
int64_t nTotalCache = (GetArg("-dbcache", nDefaultDbCache) << 20);
int64_t nTotalCache = (GetArg("-dbcache", GetDefaultDbCacheMB()) << 20);
nTotalCache = std::max(nTotalCache, nMinDbCache << 20); // total cache cannot be less than nMinDbCache
nTotalCache = std::min(nTotalCache, nMaxDbCache << 20); // total cache cannot be greated than nMaxDbcache
int64_t nBlockTreeDBCache = nTotalCache / 8;
@@ -1857,6 +2013,14 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler)
int64_t nCoinDBCache = std::min(nTotalCache / 2, (nTotalCache / 4) + (1 << 23)); // use 25%-50% of the remainder for disk cache
nTotalCache -= nCoinDBCache;
nCoinCacheUsage = nTotalCache; // the rest goes to in-memory cache
// Adaptive sizing: unless the user pinned -dbcache, grow/shrink the coins cache with free system
// memory (AdjustCoinCacheForMemoryPressure), using the startup size as the ceiling.
if (!mapArgs.count("-dbcache")) {
g_nMaxCoinCacheUsage = nCoinCacheUsage;
scheduler.scheduleEvery(&AdjustCoinCacheForMemoryPressure, 5);
LogPrintf("* Adaptive dbcache enabled: ceiling %.0fMiB, keeping >= %lldMiB RAM free for the system\n",
nCoinCacheUsage * (1.0 / 1024 / 1024), (long long)GetMemoryReserveMB());
}
LogPrintf("Cache configuration:\n");
LogPrintf("* Max cache setting possible %.1fMiB\n", nMaxDbCache);
LogPrintf("* Using %.1fMiB for block index database\n", nBlockTreeDBCache * (1.0 / 1024 / 1024));
@@ -1938,6 +2102,7 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler)
strLoadError = _("Error initializing block database");
break;
}
HUSH_LOADINGBLOCKS = 0;
// Check for changed -txindex state
if (fTxIndex != GetBoolArg("-txindex", true)) {
@@ -2107,8 +2272,54 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler)
if (!pwalletMain->HaveHDSeed())
{
// generate a new HD seed
pwalletMain->GenerateNewSeed();
std::string mnemonic = GetArg("-mnemonic", "");
std::string hdSeedHex = GetArg("-hdseed", "");
bool restoring = false;
if (!mnemonic.empty() && !hdSeedHex.empty())
return InitError(_("Specify only one of -mnemonic or -hdseed, not both"));
if (!mnemonic.empty())
{
// Restore/create a wallet from a BIP39 seed phrase, byte-compatible
// with SilentDragonXLite. Must be a fresh/empty wallet.
if (!pwalletMain->SetHDSeedFromMnemonic(mnemonic))
return InitError(_("Invalid -mnemonic: expected a valid BIP39 English phrase on a fresh/empty wallet"));
LogPrintf("%s: restoring wallet from -mnemonic seed phrase\n", __func__);
restoring = true;
}
else if (!hdSeedHex.empty())
{
// Restore from a previously exported HD seed hex (z_exportwallet's
// "# HDSeed=" line): 32 bytes (raw) or 64 bytes (BIP39-derived).
if (!pwalletMain->SetHDSeedFromHex(hdSeedHex))
return InitError(_("Invalid -hdseed: expected a 32- or 64-hex-character seed on a fresh/empty wallet"));
LogPrintf("%s: restoring wallet from -hdseed\n", __func__);
restoring = true;
}
else
{
// generate a new HD seed
pwalletMain->GenerateNewSeed();
}
if (restoring)
{
// Pre-derive keys (birthday = genesis) so the startup rescan finds
// funds paid to them: transparent coinbase + shielded notes.
int64_t tGap = GetArg("-hdtransparentgaplimit", 1000);
if (tGap < 0) tGap = 0;
pwalletMain->TopUpHDTransparentKeys((unsigned int)tGap, 1);
int64_t zGap = GetArg("-mnemonicsaplinggap", 100);
if (zGap < 0) zGap = 0;
{
LOCK(pwalletMain->cs_wallet);
for (int i = 0; i < (int)zGap; i++)
pwalletMain->GenerateNewSaplingZKey();
}
LogPrintf("%s: pre-derived %d transparent and %d sapling keys for restore rescan\n", __func__, (int)tGap, (int)zGap);
}
}
//Set Sapling Consolidation
@@ -2380,6 +2591,9 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler)
nLocalServices |= NODE_ADDRINDEX;
if ( GetBoolArg("-spentindex", DEFAULT_SPENTINDEX) != 0 )
nLocalServices |= NODE_SPENTINDEX;
// Advertise willingness to SERVE bulk block streams (full nodes only) when opted in.
if ( fBulkBlockSync )
nLocalServices |= NODE_BULKBLOCKS;
fprintf(stderr,"nLocalServices %llx %d, %d\n",(long long)nLocalServices,GetBoolArg("-addressindex", DEFAULT_ADDRESSINDEX),GetBoolArg("-spentindex", DEFAULT_SPENTINDEX));
}
// ********************************************************* Step 10: import blocks

View File

@@ -39,6 +39,9 @@
*/
typedef std::vector<unsigned char, secure_allocator<unsigned char> > CPrivKey;
/** BIP32: child indices at or above this are hardened. */
const unsigned int BIP32_HARDENED_KEY_LIMIT = 0x80000000;
/** An encapsulated private key. */
class CKey
{

View File

@@ -89,6 +89,12 @@ static int64_t nTimeBestReceived = 0;
CWaitableCriticalSection csBestBlock;
CConditionVariable cvBlockChange;
int nScriptCheckThreads = 0;
int MAX_BLOCKS_IN_TRANSIT_PER_PEER = DEFAULT_MAX_BLOCKS_IN_TRANSIT_PER_PEER;
bool fBulkBlockSync = DEFAULT_BULKBLOCKSYNC;
// Server-side flood throttle: minimum interval between bulk serves to the same peer (main.cpp-local
// since only the serve handler uses it; kept out of main.h to avoid a full-tree recompile).
static const int64_t BULK_MIN_SERVE_INTERVAL_US = 50000; // 50 ms => <= 20 bulk serves/s/peer
int nRandomXVerifyThreads = 0; // parallel RandomX pre-verification worker count (0 = inline only)
bool fExperimentalMode = true;
bool fImporting = false;
bool fReindex = false;
@@ -247,6 +253,7 @@ namespace {
int64_t nTime; //! Time of "getdata" request in microseconds.
bool fValidatedHeaders; //! Whether this block has validated headers at the time of request.
int64_t nTimeDisconnect; //! The timeout for this block request (for disconnecting a slow peer)
bool fBulk; //! Requested as part of a bulk stream range (exempt from the front() stall-disconnect).
};
map<uint256, pair<NodeId, list<QueuedBlock>::iterator> > mapBlocksInFlight;
@@ -306,6 +313,21 @@ namespace {
int nBlocksInFlightValidHeaders;
//! Whether we consider this a preferred download peer.
bool fPreferredDownload;
//! Opt-in bulk block streaming (DragonX): whether a bulk range request is outstanding to this peer.
bool fBulkInFlight;
//! Time (us) the outstanding bulk request was issued, for the response timeout/fallback.
int64_t nBulkSince;
//! Height of the first block in the outstanding bulk range.
int nBulkRangeStart;
//! Number of blocks requested in the outstanding bulk range.
int nBulkRangeCount;
//! Hash of the first block of the outstanding bulk range (request identity; the server echoes it
//! in the BLOCKSTREAM header so a stale/duplicate header for an old request can be ignored).
uint256 nBulkHashStart;
//! Whether the (one-shot) trailing BLOCKSTREAM header for the outstanding request was processed.
bool fBulkHeaderSeen;
//! (server side) time (us) we last served a bulk stream to this peer, for flood throttling.
int64_t nLastBulkServeTime;
CNodeState() {
fCurrentlyConnected = false;
@@ -319,6 +341,13 @@ namespace {
nBlocksInFlight = 0;
nBlocksInFlightValidHeaders = 0;
fPreferredDownload = false;
fBulkInFlight = false;
nBulkSince = 0;
nBulkRangeStart = 0;
nBulkRangeCount = 0;
nBulkHashStart.SetNull();
fBulkHeaderSeen = false;
nLastBulkServeTime = 0;
}
};
@@ -413,7 +442,7 @@ namespace {
}
// Requires cs_main.
void MarkBlockAsInFlight(NodeId nodeid, const uint256& hash, const Consensus::Params& consensusParams, CBlockIndex *pindex = NULL) {
void MarkBlockAsInFlight(NodeId nodeid, const uint256& hash, const Consensus::Params& consensusParams, CBlockIndex *pindex = NULL, bool fBulk = false) {
CNodeState *state = State(nodeid);
assert(state != NULL);
@@ -421,7 +450,7 @@ namespace {
MarkBlockAsReceived(hash);
int64_t nNow = GetTimeMicros();
QueuedBlock newentry = {hash, pindex, nNow, pindex != NULL, GetBlockTimeout(nNow, nQueuedValidatedHeaders, consensusParams)};
QueuedBlock newentry = {hash, pindex, nNow, pindex != NULL, GetBlockTimeout(nNow, nQueuedValidatedHeaders, consensusParams), fBulk};
nQueuedValidatedHeaders += newentry.fValidatedHeaders;
list<QueuedBlock>::iterator it = state->vBlocksInFlight.insert(state->vBlocksInFlight.end(), newentry);
state->nBlocksInFlight++;
@@ -429,6 +458,36 @@ namespace {
mapBlocksInFlight[hash] = std::make_pair(nodeid, it);
}
// Opt-in bulk block streaming (DragonX): free this peer's still-in-flight bulk blocks whose height
// falls in [hStart, hEnd), so the normal per-block path re-fetches them. We scan the peer's OWN
// vBlocksInFlight by the LITERAL hash marked at request time (via the stored pindex) rather than
// re-deriving hashes from the mutable pindexBestKnownBlock - the latter would miss the real entries
// after a reorg (leaking in-flight slots) and can never touch another peer's blocks. Requires cs_main.
void FreeBulkRangeInFlight(CNodeState* state, int hStart, int hEnd) {
if (state == NULL) return;
std::vector<uint256> toFree; // collect first: MarkBlockAsReceived erases from vBlocksInFlight
BOOST_FOREACH(const QueuedBlock& q, state->vBlocksInFlight) {
if (q.fBulk && q.pindex != NULL) {
int h = q.pindex->GetHeight();
if (h >= hStart && h < hEnd) toFree.push_back(q.hash);
}
}
BOOST_FOREACH(const uint256& hh, toFree)
MarkBlockAsReceived(hh);
}
// True if any of this peer's bulk blocks with height in [hStart, hEnd) is still in flight (range not
// fully drained). Completion is decided by the RANGE draining, not the global per-peer window count.
bool BulkRangeInFlight(CNodeState* state, int hStart, int hEnd) {
if (state == NULL) return false;
BOOST_FOREACH(const QueuedBlock& q, state->vBlocksInFlight) {
if (q.fBulk && q.pindex != NULL) {
int h = q.pindex->GetHeight();
if (h >= hStart && h < hEnd) return true;
}
}
return false;
}
/** Check whether the last unknown block a peer advertized is not yet known. */
void ProcessBlockAvailability(NodeId nodeid) {
CNodeState *state = State(nodeid);
@@ -485,7 +544,7 @@ namespace {
/** Update pindexLastCommonBlock and add not-in-flight missing successors to vBlocks, until it has
* at most count entries. */
void FindNextBlocksToDownload(NodeId nodeid, unsigned int count, std::vector<CBlockIndex*>& vBlocks, NodeId& nodeStaller) {
void FindNextBlocksToDownload(NodeId nodeid, unsigned int count, std::vector<CBlockIndex*>& vBlocks, NodeId& nodeStaller, CBlockIndex** pFrontierStuck = NULL) {
if (count == 0)
return;
@@ -562,8 +621,9 @@ namespace {
return;
}
} else if (waitingfor == -1) {
// This is the first already-in-flight block.
// This is the first already-in-flight block (the download frontier).
waitingfor = mapBlocksInFlight[pindex->GetBlockHash()].first;
if (pFrontierStuck) *pFrontierStuck = pindex;
}
}
}
@@ -4188,6 +4248,7 @@ static bool ActivateBestChainStep(bool fSkipdpow, CValidationState &state, CBloc
return state.DoS(100, error("ActivateBestChainStep(): pindexOldTip->GetHeight().%d > notarizedht %d && pindexFork->GetHeight().%d is < notarizedht %d, so ignore it",(int32_t)pindexOldTip->GetHeight(),notarizedht,(int32_t)pindexFork->GetHeight(),notarizedht),
REJECT_INVALID, "past-notarized-height");
}
// - On ChainDB initialization, pindexOldTip will be null, so there are no removable blocks.
// - If pindexMostWork is in a chain that doesn't have the same genesis block as our chain,
// then pindexFork will be null, and we would need to remove the entire chain including
@@ -4258,6 +4319,36 @@ static bool ActivateBestChainStep(bool fSkipdpow, CValidationState &state, CBloc
}
nHeight = nTargetHeight;
// Parallel RandomX pre-verification (Stage 4): verify this about-to-be-connected window's
// PoW on the worker pool BEFORE the serial connect, so ConnectBlock rarely pays the
// ~tens-of-ms light-mode hash. Pure optimization — CheckBlockHeader's inline
// CheckRandomXSolution still verifies anything not pre-verified, so consensus is unchanged.
// We hold cs_main; key derivation + the disk reads happen here on the main thread, and the
// pool workers receive only value-type work items (no cs_main, no chainstate pointers).
if (nRandomXVerifyThreads > 0 && rxCheckQueue.IsIdle()) {
std::map<std::string, std::vector<CRandomXCheck> > rxGroups; // grouped by RandomX key
BOOST_FOREACH(CBlockIndex *pidx, vpindexToConnect) {
if (pidx->fRandomXVerified || !RandomXValidationRequired(pidx->GetHeight()))
continue;
std::string rxKey = GetRandomXKey(pidx->GetHeight());
if (rxKey.empty())
continue; // can't derive key -> inline fallback
CBlock blk;
if (!ReadBlockFromDisk(blk, pidx, false))
continue; // -> inline fallback
if (blk.nSolution.size() != 32) // RANDOMX_HASH_SIZE; wrong size -> inline (will error)
continue;
rxGroups[rxKey].push_back(CRandomXCheck(rxKey, GetRandomXInput(blk), blk.nSolution.data(), &pidx->fRandomXVerified));
}
for (std::map<std::string, std::vector<CRandomXCheck> >::iterator it = rxGroups.begin(); it != rxGroups.end(); ++it) {
if (!RandomXValidatorPrepareKey(it->first))
break; // cache alloc failed -> leave the rest for the inline fallback
CCheckQueueControl<CRandomXCheck> control(&rxCheckQueue);
control.Add(it->second);
control.Wait();
}
}
// Connect new blocks.
BOOST_REVERSE_FOREACH(CBlockIndex *pindexConnect, vpindexToConnect) {
if (!ConnectTip(state, pindexConnect, pindexConnect == pindexMostWork ? pblock : NULL)) {
@@ -4993,7 +5084,11 @@ bool CheckBlockHeader(int32_t *futureblockp,int32_t height,CBlockIndex *pindex,
{
if ( !CheckEquihashSolution(&blockhdr, Params()) )
return state.DoS(100, error("CheckBlockHeader(): Equihash solution invalid"),REJECT_INVALID, "invalid-solution");
if ( !CheckRandomXSolution(&blockhdr, height) )
// Skip the inline RandomX recompute only if the parallel pre-verify pool already verified
// THIS block (fRandomXVerified set 1:1 on a real hash match). Every other case — pool miss,
// straggler, disabled pool, or any pindex==NULL caller (TestBlockValidity/VerifyDB/header
// accept) — falls through to the inline check, so consensus is unchanged.
if ( !(pindex && pindex->fRandomXVerified) && !CheckRandomXSolution(&blockhdr, height) )
return state.DoS(100, error("CheckBlockHeader(): RandomX solution invalid"),REJECT_INVALID, "invalid-randomx-solution");
}
// Check proof of work matches claimed amount
@@ -6746,6 +6841,13 @@ void static ProcessGetData(CNode* pfrom)
std::deque<CInv>::iterator it = pfrom->vRecvGetData.begin();
vector<CInv> vNotFound;
// Serve up to this many blocks per ProcessGetData pass. The old code broke after a SINGLE block,
// so a 16-block getdata was dribbled out one block per message-handler tick (~100ms), throttling
// block download for every peer fetching from us. Bound the per-pass work (cs_main is held while
// reading blocks from disk); any remainder is served on the next pass (the message handler keeps
// fSleep=false while vRecvGetData is non-empty, so there is no 100ms park between passes).
const unsigned int nMaxBlocksServedPerPass = 16;
unsigned int nBlocksServed = 0;
LOCK(cs_main);
@@ -6863,7 +6965,10 @@ void static ProcessGetData(CNode* pfrom)
}
}
if (inv.type == MSG_BLOCK || inv.type == MSG_FILTERED_BLOCK)
// Serve a bounded batch of blocks per pass rather than one (see nMaxBlocksServedPerPass
// above). The send-buffer gate at the top of the loop still pauses us if the buffer fills;
// this counter bounds the cs_main hold for a (possibly malicious) large getdata.
if ((inv.type == MSG_BLOCK || inv.type == MSG_FILTERED_BLOCK) && ++nBlocksServed >= nMaxBlocksServedPerPass)
break;
}
}
@@ -7688,6 +7793,118 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv,
}
CheckBlockIndex();
} else if (strCommand == NetMsgType::GETBLOCKSTREAM) {
// Opt-in bulk block streaming (DragonX): a peer asks us to stream a contiguous range of
// old blocks as back-to-back BLOCK messages. We only honor it if we advertised the bit
// (i.e. were started with -bulkblocksync) and we are not mid-import/reindex.
if ((nLocalServices & NODE_BULKBLOCKS) == 0 || fImporting || fReindex)
return true;
uint256 hashStart; int32_t nStartHeight; uint16_t nCount;
vRecv >> hashStart >> nStartHeight >> nCount;
// Resolve the range under cs_main (cheap, no disk I/O), then read + stream the blocks WITHOUT
// holding the lock, so a 128-block / 8 MiB serve never holds cs_main across disk reads (the
// analogous ProcessGetData caps per-pass work precisely because it reads under cs_main).
std::vector<CBlockIndex*> vSend;
int firstH = -1;
bool refuse = false;
{
LOCK(cs_main);
if (nCount == 0 || nCount > BULK_MAX_BLOCKS_PER_REQUEST) {
Misbehaving(pfrom->GetId(), 20); // mirrors the getdata MAX_INV_SZ penalty
return true;
}
// Light flood throttle: at most one bulk serve per peer per BULK_MIN_SERVE_INTERVAL_US. On
// throttle, send a refusal header so the requester falls back immediately (not after 90s).
int64_t nNowServe = GetTimeMicros();
CNodeState* sst = State(pfrom->GetId());
if (sst != NULL && sst->nLastBulkServeTime > nNowServe - BULK_MIN_SERVE_INTERVAL_US) {
pfrom->PushMessage(NetMsgType::BLOCKSTREAM, hashStart, (int32_t)-1, (uint16_t)0);
return true;
}
if (sst != NULL) sst->nLastBulkServeTime = nNowServe;
BlockMap::iterator mi = mapBlockIndex.find(hashStart);
// Don't flood old blocks while WE are still syncing (unless allowlisted); only serve blocks
// on our active chain at the height the requester expects (nStartHeight, tamper-checked).
if ((IsInitialBlockDownload() && !pfrom->fAllowlisted) ||
mi == mapBlockIndex.end() || !chainActive.Contains(mi->second) ||
mi->second->GetHeight() != nStartHeight) {
refuse = true;
} else {
CBlockIndex* pindex = mi->second;
firstH = pindex->GetHeight();
for (uint16_t i = 0; i < nCount && pindex != NULL; i++, pindex = chainActive.Next(pindex)) {
if ((pindex->nStatus & BLOCK_HAVE_DATA) == 0) break; // pruned/missing
vSend.push_back(pindex);
}
}
}
if (refuse) {
pfrom->PushMessage(NetMsgType::BLOCKSTREAM, hashStart, (int32_t)-1, (uint16_t)0);
return true;
}
// Read from disk + stream OUTSIDE cs_main. CBlockIndex pointers are stable and block files are
// append-only, so reading by pindex without the lock is safe (a concurrent reorg cannot delete
// block data, and the requester validates every block against its own headers regardless).
uint16_t nSent = 0;
size_t cumBytes = 0;
BOOST_FOREACH(CBlockIndex* pb, vSend) {
if (pfrom->nSendSize >= SendBufferSize()) break; // send-buffer backpressure
boost::this_thread::interruption_point();
CBlock block;
if (!ReadBlockFromDisk(block, pb, 1)) break; // graceful, never assert
size_t sz = GetSerializeSize(block, SER_NETWORK, PROTOCOL_VERSION);
if (nSent > 0 && cumBytes + sz > BULK_MAX_RESPONSE_BYTES) break; // total byte cap
cumBytes += sz;
pfrom->PushMessage(NetMsgType::BLOCK, block);
nSent++;
}
// Trailing control header carries the ACTUAL count sent (authoritative), so the requester can
// free any undelivered tail immediately rather than waiting for the bulk response timeout.
pfrom->PushMessage(NetMsgType::BLOCKSTREAM, hashStart, (int32_t)firstH, nSent);
LogPrint("net", "Bulk stream serve: %u/%u blocks from height %d (%lu bytes) peer=%d\n",
(unsigned)nSent, (unsigned)nCount, firstH, (unsigned long)cumBytes, pfrom->id);
return true;
} else if (strCommand == NetMsgType::BLOCKSTREAM) {
// Opt-in bulk block streaming (DragonX): the trailing control header for a streamed range. The
// blocks themselves arrive as ordinary BLOCK messages (handled below); this reconciles what the
// peer actually delivered so the undelivered tail (or a refusal) falls back at once instead of
// waiting for the bulk timeout. Service bits are unauthenticated, so we ignore anything that
// doesn't match our exact outstanding request.
uint256 hashStart; int32_t nFirstHeight; uint16_t nBlocks;
vRecv >> hashStart >> nFirstHeight >> nBlocks;
LOCK(cs_main);
CNodeState* state = State(pfrom->GetId());
if (state == NULL || !state->fBulkInFlight)
return true; // nothing outstanding
if (hashStart != state->nBulkHashStart)
return true; // header for a different/stale request; ignore
if (state->fBulkHeaderSeen)
return true; // one-shot: already reconciled this request
state->fBulkHeaderSeen = true;
// nBlocks==0 (refusal) or an over-count => free our whole outstanding range and fall back.
// 0 < nBlocks <= count => the peer commits to that many; free only the undelivered tail now.
// FreeBulkRangeInFlight scans THIS peer's vBlocksInFlight by literal hash, so it only ever frees
// heights still genuinely in flight to this peer (no cross-peer effect, reorg-proof).
bool refuse = (nBlocks == 0 || nBlocks > state->nBulkRangeCount);
int deliver = refuse ? 0 : (int)nBlocks;
FreeBulkRangeInFlight(state, state->nBulkRangeStart + deliver,
state->nBulkRangeStart + state->nBulkRangeCount);
if (refuse) {
state->fBulkInFlight = false;
pfrom->nServices &= ~(uint64_t)NODE_BULKBLOCKS; // local hint: don't retry bulk on this peer
LogPrint("net", "Bulk stream refused by peer=%d (nBlocks=%u), falling back\n", pfrom->id, (unsigned)nBlocks);
} else {
// Track only what was promised; fBulkInFlight clears once that prefix fully drains
// (range-drain check in SendMessages) or via the timeout fallback.
state->nBulkRangeCount = deliver;
if (deliver == 0)
state->fBulkInFlight = false;
}
return true;
} else if (strCommand == NetMsgType::BLOCK && !fImporting && !fReindex) // Ignore blocks received while importing
{
CBlock block;
@@ -8133,24 +8350,108 @@ bool SendMessages(CNode* pto, bool fSendTrickle)
LogPrint("net", "Reducing block download timeout for peer=%d block=%s, orig=%d new=%d\n", pto->id, queuedBlock.hash.ToString(), queuedBlock.nTimeDisconnect, nTimeoutIfRequestedNow);
queuedBlock.nTimeDisconnect = nTimeoutIfRequestedNow;
}
if (queuedBlock.nTimeDisconnect < nNow) {
if (queuedBlock.nTimeDisconnect < nNow && !queuedBlock.fBulk) {
// Bulk-stream blocks are exempt: a 128-block batch shares one request time, so the
// front() entry could expire before the tail streams in. The bulk response timeout
// below frees the range without disconnecting instead.
LogPrintf("Timeout downloading block %s from peer=%d, disconnecting\n", queuedBlock.hash.ToString(), pto->id);
pto->fDisconnect = true;
}
}
// Opt-in bulk block streaming (DragonX): manage the outstanding bulk range, then (below)
// possibly issue a new one. Clearing fBulkInFlight once the batch has drained below the
// normal window re-enables the next bulk request; a never-fully-delivered batch is freed
// after BULK_RESPONSE_TIMEOUT_US so the normal per-block path re-fetches it (no disconnect).
if (state.fBulkInFlight) {
int hEnd = state.nBulkRangeStart + state.nBulkRangeCount;
if (!BulkRangeInFlight(&state, state.nBulkRangeStart, hEnd)) {
// Whole (possibly shrunk) range received -> done. Completion is keyed on the RANGE
// draining, NOT on the global in-flight count crossing the window, so a partially
// delivered batch can never leave undelivered heights stuck in-flight.
state.fBulkInFlight = false;
} else if (state.nBulkSince > 0 && state.nBulkSince < nNow - BULK_RESPONSE_TIMEOUT_US) {
// Promised blocks never fully arrived: free the still-in-flight remainder (the normal
// per-block path re-fetches it), give up bulk on this unresponsive peer. No disconnect.
FreeBulkRangeInFlight(&state, state.nBulkRangeStart, hEnd);
state.fBulkInFlight = false;
pto->nServices &= ~(uint64_t)NODE_BULKBLOCKS;
LogPrint("net", "Bulk stream timeout peer=%d, freed range [%d,%d)\n",
pto->id, state.nBulkRangeStart, hEnd);
}
}
// Message: getdata (blocks)
static uint256 zero;
vector<CInv> vGetData;
if (!pto->fDisconnect && !pto->fClient && (fFetch || !IsInitialBlockDownload()) && state.nBlocksInFlight < MAX_BLOCKS_IN_TRANSIT_PER_PEER) {
if (!pto->fDisconnect && !pto->fClient && (fFetch || !IsInitialBlockDownload()) && state.nBlocksInFlight < MAX_BLOCKS_IN_TRANSIT_PER_PEER && !state.fBulkInFlight) {
vector<CBlockIndex*> vToDownload;
NodeId staller = -1;
FindNextBlocksToDownload(pto->GetId(), MAX_BLOCKS_IN_TRANSIT_PER_PEER - state.nBlocksInFlight, vToDownload, staller);
CBlockIndex *pFrontierStuck = NULL;
FindNextBlocksToDownload(pto->GetId(), MAX_BLOCKS_IN_TRANSIT_PER_PEER - state.nBlocksInFlight, vToDownload, staller, &pFrontierStuck);
// Opt-in bulk block streaming (DragonX): if the first block we need is in the deep,
// stable region (>= BULK_TIP_MARGIN below the NETWORK tip) and the peer advertised the
// capability, request a whole contiguous range in one shot instead of per-block getdata.
// FindNextBlocksToDownload already advanced the cursor past what we have, so
// vToDownload.front() is the correct, cursor-managed starting point.
bool didBulk = false;
if (fBulkBlockSync && (pto->nServices & NODE_BULKBLOCKS) && IsInitialBlockDownload()
&& !vToDownload.empty() && state.pindexBestKnownBlock != NULL) {
CBlockIndex* pfirst = vToDownload.front();
int cursorH = pfirst->GetHeight();
int maxH = state.pindexBestKnownBlock->GetHeight() - BULK_TIP_MARGIN;
if (cursorH <= maxH) {
int want = std::min(maxH - cursorH + 1, (int)BULK_MAX_BLOCKS_PER_REQUEST);
uint16_t n = 0;
for (int i = 0; i < want; i++) {
CBlockIndex* pb = state.pindexBestKnownBlock->GetAncestor(cursorH + i);
if (pb == NULL || mapBlocksInFlight.count(pb->GetBlockHash())) break;
MarkBlockAsInFlight(pto->GetId(), pb->GetBlockHash(), consensusParams, pb, true);
n++;
}
if (n > 0) {
pto->PushMessage(NetMsgType::GETBLOCKSTREAM, pfirst->GetBlockHash(), (int32_t)cursorH, n);
state.fBulkInFlight = true;
state.nBulkSince = nNow;
state.nBulkRangeStart = cursorH;
state.nBulkRangeCount = n;
state.nBulkHashStart = pfirst->GetBlockHash(); // request identity (matched in BLOCKSTREAM)
state.fBulkHeaderSeen = false; // arm the one-shot header reconciliation
didBulk = true;
LogPrint("net", "Requesting bulk stream [%d..%d] (%u blocks) peer=%d\n",
cursorH, cursorH + n - 1, (unsigned)n, pto->id);
}
}
}
if (!didBulk) {
BOOST_FOREACH(CBlockIndex *pindex, vToDownload) {
vGetData.push_back(CInv(MSG_BLOCK, pindex->GetBlockHash()));
MarkBlockAsInFlight(pto->GetId(), pindex->GetBlockHash(), consensusParams, pindex);
LogPrint("net", "Requesting block %s (%d) peer=%d\n", pindex->GetBlockHash().ToString(), pindex->GetHeight(), pto->id);
}
}
// Frontier reassignment: when this peer has nothing new to fetch because the next-needed
// (frontier) block is in flight from another, slow peer and has been stuck beyond a short
// threshold, re-request it from THIS (responsive) peer instead of waiting out the long
// (~72s) timeout or disconnecting the slow peer. This breaks the head-of-line stall that
// throttles IBD when downloading from few, distant peers. Trustless: the block is still
// fully validated on arrival - we only change which peer serves it. -blockreassigntimeout
// = seconds (0 disables; default 5).
static const int64_t nReassignUs = GetArg("-blockreassigntimeout", 5) * 1000000LL;
if (nReassignUs > 0 && vToDownload.empty() && pFrontierStuck != NULL &&
staller != -1 && staller != pto->GetId()) {
map<uint256, pair<NodeId, list<QueuedBlock>::iterator> >::iterator itF =
mapBlocksInFlight.find(pFrontierStuck->GetBlockHash());
if (itF != mapBlocksInFlight.end() && itF->second.first == staller &&
itF->second.second->nTime < nNow - nReassignUs) {
uint256 hReassign = pFrontierStuck->GetBlockHash();
LogPrint("net", "Reassigning stalled frontier block %s (%d) from peer=%d to peer=%d\n",
hReassign.ToString(), pFrontierStuck->GetHeight(), staller, pto->id);
MarkBlockAsReceived(hReassign); // free from slow peer (no disconnect)
vGetData.push_back(CInv(MSG_BLOCK, hReassign));
MarkBlockAsInFlight(pto->GetId(), hReassign, consensusParams, pFrontierStuck); // re-request from this peer
}
}
if (state.nBlocksInFlight == 0 && staller != -1) {
if (State(staller)->nStallingSince == 0) {
State(staller)->nStallingSince = nNow;

View File

@@ -95,12 +95,37 @@ static const unsigned int UNDOFILE_CHUNK_SIZE = 0x100000; // 1 MiB
static const int MAX_SCRIPTCHECK_THREADS = 16;
/** -par default (number of script-checking threads, 0 = auto) */
static const int DEFAULT_SCRIPTCHECK_THREADS = 0;
/** Number of blocks that can be requested at any given time from a single peer. */
static const int MAX_BLOCKS_IN_TRANSIT_PER_PEER = 16;
/** Number of blocks that can be requested at any given time from a single peer.
* Runtime-tunable via -maxblocksintransit. The default of 16 caps single-peer IBD
* throughput at (window / RTT): on a high-latency peer with tiny (sub-checkpoint)
* blocks the transfer is bandwidth-delay-product bound, so a larger window lifts the
* ceiling at negligible bandwidth cost. */
static const int DEFAULT_MAX_BLOCKS_IN_TRANSIT_PER_PEER = 16;
extern int MAX_BLOCKS_IN_TRANSIT_PER_PEER;
/** Opt-in bulk block streaming (DragonX, -bulkblocksync). A single GETBLOCKSTREAM request makes a
* peer stream a contiguous range of old blocks as back-to-back BLOCK messages, amortizing the
* per-block round-trip over the whole range instead of the MAX_BLOCKS_IN_TRANSIT_PER_PEER window.
* OFF by default; negotiated via NODE_BULKBLOCKS; only used during IBD for blocks more than
* BULK_TIP_MARGIN below the active tip; never alters the default getdata path. */
static const bool DEFAULT_BULKBLOCKSYNC = false;
extern bool fBulkBlockSync;
/** Only bulk-stream blocks at least this far below the active tip (near-tip uses the normal path). */
static const int BULK_TIP_MARGIN = 5000;
/** Hard DoS cap: max blocks a single GETBLOCKSTREAM may request/serve. */
static const uint16_t BULK_MAX_BLOCKS_PER_REQUEST = 128;
/** Hard DoS cap: max total bytes streamed in response to one GETBLOCKSTREAM. */
static const size_t BULK_MAX_RESPONSE_BYTES = 8 * 1024 * 1024;
/** Requester fallback: if a promised bulk range doesn't fully arrive within this many microseconds,
* free the in-flight range so the normal per-block path re-fetches it. */
static const int64_t BULK_RESPONSE_TIMEOUT_US = 90 * 1000000LL;
/** Timeout in seconds during which a peer must stall block download progress before being disconnected. */
static const unsigned int BLOCK_STALLING_TIMEOUT = 2;
/** Number of headers sent in one getheaders result. We rely on the assumption that if a peer sends
* less than this number, we reached its tip. Changing this value is a protocol upgrade. */
* less than this number, we reached its tip. Changing this value is a protocol upgrade: the
* continuation logic (main.cpp, "nCount == MAX_HEADERS_RESULTS") and the serve-side limit must
* match across the network, so a single node raising it unilaterally would mis-detect a stock
* peer's 160-header reply as "tip reached" and stall header sync. Raise only as a coordinated
* network upgrade (with a protocol-version bump). */
static const unsigned int MAX_HEADERS_RESULTS = 160;
/** Size of the "block download window": how far ahead of our current height do we fetch?
* Larger windows tolerate larger download speed differences between peer, but increase the potential
@@ -155,6 +180,7 @@ extern bool fExperimentalMode;
extern bool fImporting;
extern bool fReindex;
extern int nScriptCheckThreads;
extern int nRandomXVerifyThreads;
extern bool fTxIndex;
extern bool fZindex;
extern bool fIsBareMultisigStd;
@@ -930,6 +956,10 @@ extern CChain chainActive;
/** Global variable that points to the active CCoinsView (protected by cs_main) */
extern CCoinsViewCache *pcoinsTip;
/** Global variable that points to the coins database (chainstate/, protected by cs_main). */
class CCoinsViewDB;
extern CCoinsViewDB *pcoinsdbview;
/** Global variable that points to the active block tree (protected by cs_main) */
extern CBlockTreeDB *pblocktree;

View File

@@ -18,6 +18,7 @@
* *
******************************************************************************/
#include "pow.h"
#include "checkpoints.h"
#include "consensus/upgrades.h"
#include "arith_uint256.h"
#include "chain.h"
@@ -30,6 +31,8 @@
#include "sodium.h"
#include "RandomX/src/randomx.h"
#include <mutex>
#include <boost/thread/shared_mutex.hpp>
#include <boost/thread/locks.hpp>
#ifdef ENABLE_RUST
#include "librustzcash.h"
@@ -704,6 +707,7 @@ static std::mutex cs_randomx_validator;
static randomx_cache *s_rxCache = nullptr;
static randomx_vm *s_rxVM = nullptr;
static std::string s_rxCurrentKey; // tracks current key to avoid re-init
static int64_t nTimeRandomX = 0; // cumulative RandomX validation time (us), reported under -debug=bench
// Thread-local flag: skip CheckRandomXSolution when the miner is validating its own block
// The miner already computed the correct RandomX hash — re-verifying with a separate
@@ -714,26 +718,70 @@ void SetSkipRandomXValidation(bool skip) { fSkipRandomXValidation = skip; }
CBlockIndex *hush_chainactive(int32_t height);
bool CheckRandomXSolution(const CBlockHeader *pblock, int32_t height)
// Centralized predicate: does a block at this height actually require a RandomX hash check?
// Shared by CheckRandomXSolution (inline path) and the parallel pre-verify pool so the two can
// never drift. Returns false when the recompute is unnecessary:
// - non-RandomX chain, or RandomX validation disabled (activation height < 0)
// - below the RandomX activation height (those blocks used Equihash, validated elsewhere)
// - during initial on-disk block loading / reindex (HUSH_LOADINGBLOCKS)
// - below the last hardcoded checkpoint (chain pinned by checkpoint hash + linkage + work)
// Deliberately does NOT consider the thread-local fSkipRandomXValidation (miner self-check) — that
// is a property of the calling thread, handled only in the inline CheckRandomXSolution below.
bool RandomXValidationRequired(int32_t height)
{
// Only applies to RandomX chains
if (ASSETCHAINS_ALGO != ASSETCHAINS_RANDOMX)
return true;
// Disabled if activation height is negative
return false;
if (ASSETCHAINS_RANDOMX_VALIDATION < 0)
return true;
// Not yet at activation height
return false;
if (height < ASSETCHAINS_RANDOMX_VALIDATION)
return true;
// Do not affect initial block loading
return false;
extern int32_t HUSH_LOADINGBLOCKS;
if (HUSH_LOADINGBLOCKS != 0)
return false;
extern bool fCheckpointsEnabled;
if (fCheckpointsEnabled && height < Checkpoints::GetTotalBlocksEstimate(Params().Checkpoints()))
return false;
return true;
}
// Serialize the RandomX hash input: the block header without nSolution (but with nNonce). Used by
// both the inline CheckRandomXSolution and the parallel pre-verify pool, so the bytes are identical.
std::vector<unsigned char> GetRandomXInput(const CBlockHeader& block)
{
CRandomXInput rxInput(block);
CDataStream ss(SER_NETWORK, PROTOCOL_VERSION);
ss << rxInput;
return std::vector<unsigned char>(ss.begin(), ss.end());
}
// Derive the RandomX key string for a block at `height`. Below interval+lag it is the chain-params
// initial key; otherwise the block hash at the key-rotation height. MUST be called under cs_main
// (reads chainActive via hush_chainactive). Returns empty if the key-height block is unavailable.
std::string GetRandomXKey(int32_t height)
{
static int randomxInterval = GetRandomXInterval();
static int randomxBlockLag = GetRandomXBlockLag();
if (height < randomxInterval + randomxBlockLag) {
char initialKey[82];
snprintf(initialKey, 81, "%08x%s%08x", ASSETCHAINS_MAGIC, SMART_CHAIN_SYMBOL, ASSETCHAINS_RPCPORT);
return std::string(initialKey, strlen(initialKey));
}
int keyHeight = ((height - randomxBlockLag) / randomxInterval) * randomxInterval;
CBlockIndex *pKeyIndex = hush_chainactive(keyHeight);
if (pKeyIndex == nullptr)
return std::string();
uint256 blockKey = pKeyIndex->GetBlockHash();
return std::string((const char*)&blockKey, sizeof(blockKey));
}
bool CheckRandomXSolution(const CBlockHeader *pblock, int32_t height)
{
// Centralized height gate (shared with the parallel pre-verify pool, Stage 0).
if (!RandomXValidationRequired(height))
return true;
// Skip when miner is validating its own block via TestBlockValidity
// Skip when the miner is validating its own freshly-mined block via TestBlockValidity
// (thread-local; never set on the connect thread or the pre-verify worker threads).
if (fSkipRandomXValidation)
return true;
@@ -743,47 +791,44 @@ bool CheckRandomXSolution(const CBlockHeader *pblock, int32_t height)
pblock->nSolution.size(), RANDOMX_HASH_SIZE, height);
}
static int randomxInterval = GetRandomXInterval();
static int randomxBlockLag = GetRandomXBlockLag();
// Determine the correct RandomX key for this height
char initialKey[82];
snprintf(initialKey, 81, "%08x%s%08x", ASSETCHAINS_MAGIC, SMART_CHAIN_SYMBOL, ASSETCHAINS_RPCPORT);
std::string rxKey;
if (height < randomxInterval + randomxBlockLag) {
// Use initial key derived from chain params
rxKey = std::string(initialKey, strlen(initialKey));
} else {
// Use block hash at the key height
int keyHeight = ((height - randomxBlockLag) / randomxInterval) * randomxInterval;
CBlockIndex *pKeyIndex = hush_chainactive(keyHeight);
if (pKeyIndex == nullptr) {
return error("CheckRandomXSolution(): cannot get block index at key height %d for block %d", keyHeight, height);
}
uint256 blockKey = pKeyIndex->GetBlockHash();
rxKey = std::string((const char*)&blockKey, sizeof(blockKey));
}
// Serialize the block header without nSolution (but with nNonce) as RandomX input
CRandomXInput rxInput(*pblock);
CDataStream ss(SER_NETWORK, PROTOCOL_VERSION);
ss << rxInput;
// Derive the key (shared helper) and serialize the input (identical bytes to the pool path).
std::string rxKey = GetRandomXKey(height);
if (rxKey.empty())
return error("CheckRandomXSolution(): cannot derive RandomX key for height %d", height);
std::vector<unsigned char> ssInput = GetRandomXInput(*pblock);
char computedHash[RANDOMX_HASH_SIZE];
// Measurement (Track 1): isolate RandomX verification cost during IBD. The
// expensive parts are the per-key cache (re)init (~every GetRandomXInterval()
// blocks) and the hash computation itself; both happen under the lock below.
int64_t nTimeRxStart = GetTimeMicros();
bool fKeyInit = false;
{
std::lock_guard<std::mutex> lock(cs_randomx_validator);
// Initialize cache + VM if needed, or re-init if key changed
if (s_rxCache == nullptr) {
randomx_flags flags = randomx_get_flags();
s_rxCache = randomx_alloc_cache(flags);
// Try large pages for the 256MB validator cache: fewer TLB misses → ~15-30% faster
// light-mode validation where the OS has hugepages configured. Falls back transparently
// when unavailable, exactly as the miner does (miner.cpp:1097). Page size does not affect
// the computed hash, so this is consensus-neutral.
bool fLargePages = true;
s_rxCache = randomx_alloc_cache(flags | RANDOMX_FLAG_LARGE_PAGES);
if (s_rxCache == nullptr) {
fLargePages = false;
s_rxCache = randomx_alloc_cache(flags);
}
if (s_rxCache == nullptr) {
return error("CheckRandomXSolution(): failed to allocate RandomX cache");
}
// Confirm the fast paths are active (JIT off would be ~9x slower; see randomx-benchmark).
LogPrint("bench", "CheckRandomXSolution: RandomX flags=0x%x JIT=%d HARD_AES=%d largePages=%d\n",
(unsigned int)flags, !!(flags & RANDOMX_FLAG_JIT), !!(flags & RANDOMX_FLAG_HARD_AES), (int)fLargePages);
randomx_init_cache(s_rxCache, rxKey.data(), rxKey.size());
s_rxCurrentKey = rxKey;
fKeyInit = true;
s_rxVM = randomx_create_vm(flags, s_rxCache, nullptr);
if (s_rxVM == nullptr) {
randomx_release_cache(s_rxCache);
@@ -793,11 +838,17 @@ bool CheckRandomXSolution(const CBlockHeader *pblock, int32_t height)
} else if (s_rxCurrentKey != rxKey) {
randomx_init_cache(s_rxCache, rxKey.data(), rxKey.size());
s_rxCurrentKey = rxKey;
fKeyInit = true;
randomx_vm_set_cache(s_rxVM, s_rxCache);
}
randomx_calculate_hash(s_rxVM, &ss[0], ss.size(), computedHash);
randomx_calculate_hash(s_rxVM, ssInput.data(), ssInput.size(), computedHash);
}
int64_t nTimeRxEnd = GetTimeMicros();
nTimeRandomX += nTimeRxEnd - nTimeRxStart;
LogPrint("bench", " - RandomX verify ht=%d: %.2fms%s [%.2fs]\n",
height, (nTimeRxEnd - nTimeRxStart) * 0.001,
fKeyInit ? " (key-init)" : "", nTimeRandomX * 0.000001);
// Compare computed hash against nSolution
if (memcmp(computedHash, pblock->nSolution.data(), RANDOMX_HASH_SIZE) != 0) {
@@ -814,7 +865,7 @@ bool CheckRandomXSolution(const CBlockHeader *pblock, int32_t height)
fprintf(stderr, " computed : %s\n", computedHex.c_str());
fprintf(stderr, " nSolution: %s\n", solutionHex.c_str());
fprintf(stderr, " rxKey size=%lu, input size=%lu, nNonce=%s\n",
rxKey.size(), ss.size(), pblock->nNonce.ToString().c_str());
rxKey.size(), ssInput.size(), pblock->nNonce.ToString().c_str());
fprintf(stderr, " nSolution.size()=%lu, RANDOMX_HASH_SIZE=%d\n",
pblock->nSolution.size(), RANDOMX_HASH_SIZE);
// Also log to debug.log
@@ -822,7 +873,7 @@ bool CheckRandomXSolution(const CBlockHeader *pblock, int32_t height)
LogPrintf(" computed : %s\n", computedHex);
LogPrintf(" nSolution: %s\n", solutionHex);
LogPrintf(" rxKey size=%lu, input size=%lu, nNonce=%s\n",
rxKey.size(), ss.size(), pblock->nNonce.ToString());
rxKey.size(), ssInput.size(), pblock->nNonce.ToString());
return false;
}
@@ -830,6 +881,88 @@ bool CheckRandomXSolution(const CBlockHeader *pblock, int32_t height)
return true;
}
// ============================================================================================
// Parallel RandomX pre-verification pool (Stage 2).
// One shared light-mode cache (holding a single key at a time) + per-thread VMs, mirroring the
// miner's RandomXDatasetManager pattern (miner.cpp). The connect thread (ActivateBestChainStep)
// loads the cache key for a same-key group of about-to-be-connected blocks, dispatches them to
// this pool, and barrier-waits; each worker hashes on its own VM (sharing the read-only cache)
// and, on a match, sets the block's transient fRandomXVerified flag so the inline check in
// CheckBlockHeader can be skipped. The inline path remains the consensus authority for anything
// not pre-verified, so the pool can only ever flip false->true on a real hash match.
static boost::shared_mutex g_rxvMutex; // shared = hashing; exclusive = cache (re)init
static randomx_cache* g_rxvCache = nullptr; // shared, read-only during hashing
static std::string g_rxvKey; // key currently loaded into g_rxvCache
static randomx_flags g_rxvFlags;
static thread_local randomx_vm* tls_rxvVM = nullptr;
static thread_local std::string tls_rxvVMKey;
CCheckQueue<CRandomXCheck> rxCheckQueue(1); // batch size 1: each item is ~tens of ms
bool RandomXValidatorPrepareKey(const std::string& rxKey)
{
boost::unique_lock<boost::shared_mutex> lock(g_rxvMutex);
if (g_rxvCache == nullptr) {
g_rxvFlags = randomx_get_flags();
g_rxvCache = randomx_alloc_cache(g_rxvFlags | RANDOMX_FLAG_LARGE_PAGES);
if (g_rxvCache == nullptr)
g_rxvCache = randomx_alloc_cache(g_rxvFlags);
if (g_rxvCache == nullptr) {
LogPrintf("RandomXValidatorPrepareKey: cache alloc failed; parallel pre-verify disabled\n");
return false;
}
randomx_init_cache(g_rxvCache, rxKey.data(), rxKey.size());
g_rxvKey = rxKey;
return true;
}
if (g_rxvKey != rxKey) {
randomx_init_cache(g_rxvCache, rxKey.data(), rxKey.size());
g_rxvKey = rxKey;
}
return true;
}
bool CRandomXCheck::operator()()
{
boost::shared_lock<boost::shared_mutex> lock(g_rxvMutex);
// The connect thread set the shared cache to one key before dispatching this group. If this
// item's key doesn't match (e.g. a key-rotation straggler) or the cache is unavailable, skip it
// and leave *presult false — the inline CheckRandomXSolution will verify it.
if (g_rxvCache == nullptr || g_rxvKey != rxKey)
return true;
if (tls_rxvVM == nullptr) {
tls_rxvVM = randomx_create_vm(g_rxvFlags, g_rxvCache, nullptr);
if (tls_rxvVM == nullptr)
return true; // cannot verify here -> inline fallback
tls_rxvVMKey = g_rxvKey;
} else if (tls_rxvVMKey != g_rxvKey) {
// Cache was re-initialized to a new key since this VM last ran; rebind.
randomx_vm_set_cache(tls_rxvVM, g_rxvCache);
tls_rxvVMKey = g_rxvKey;
}
unsigned char h[RANDOMX_HASH_SIZE];
randomx_calculate_hash(tls_rxvVM, input.data(), input.size(), h);
if (memcmp(h, expected, RANDOMX_HASH_SIZE) == 0 && presult != nullptr)
*presult = true;
return true; // ALWAYS true: never short-circuit the queue; per-block result is in *presult
}
void ThreadRandomXVerify()
{
RenameThread("hush-rxverify");
rxCheckQueue.Thread();
}
void RandomXValidatorShutdown()
{
boost::unique_lock<boost::shared_mutex> lock(g_rxvMutex);
// Per-thread VMs are intentionally leaked (process exiting); release the shared cache.
if (g_rxvCache != nullptr) {
randomx_release_cache(g_rxvCache);
g_rxvCache = nullptr;
}
}
int32_t hush_chosennotary(int32_t *notaryidp,int32_t height,uint8_t *pubkey33,uint32_t timestamp);
int32_t hush_currentheight();
void hush_index2pubkey33(uint8_t *pubkey33,CBlockIndex *pindex,int32_t height);

View File

@@ -21,8 +21,13 @@
#define HUSH_POW_H
#include "chain.h"
#include "checkqueue.h"
#include "consensus/params.h"
#include <stdint.h>
#include <cstring>
#include <string>
#include <utility>
#include <vector>
class CBlockHeader;
class CBlockIndex;
@@ -41,6 +46,55 @@ bool CheckEquihashSolution(const CBlockHeader *pblock, const CChainParams&);
/** Check whether a block header contains a valid RandomX solution */
bool CheckRandomXSolution(const CBlockHeader *pblock, int32_t height);
/** Whether a block at this height requires a RandomX hash check (shared gate used by both the
* inline CheckRandomXSolution and the parallel pre-verification pool). */
bool RandomXValidationRequired(int32_t height);
/** Serialize the RandomX hash input (block header without nSolution) — identical bytes to the
* inline CheckRandomXSolution path, so the parallel pool computes the same hash. */
std::vector<unsigned char> GetRandomXInput(const CBlockHeader& block);
/** Derive the RandomX key string for a block at `height`. MUST be called under cs_main (reads
* chainActive). Returns empty string if the key-height block is unavailable. */
std::string GetRandomXKey(int32_t height);
/** A single RandomX pre-verification work item for the parallel validator pool. Pure value type
* (no chainstate pointers) so workers need no cs_main. On a hash match it sets *presult=true; on
* any failure it leaves *presult untouched — the inline CheckRandomXSolution remains the
* consensus authority and re-verifies anything not pre-verified. operator() ALWAYS returns true,
* so one block's failure never short-circuits the rest of the CCheckQueue batch. */
class CRandomXCheck
{
private:
std::string rxKey; // RandomX key for this block's height
std::vector<unsigned char> input; // serialized CRandomXInput(header)
unsigned char expected[32]; // block.nSolution (claimed RandomX hash)
bool* presult; // -> pindex->fRandomXVerified (set true only on a hash match)
public:
CRandomXCheck() : presult(nullptr) { memset(expected, 0, sizeof(expected)); }
CRandomXCheck(const std::string& keyIn, std::vector<unsigned char> inputIn,
const unsigned char* expectedIn, bool* presultIn)
: rxKey(keyIn), input(std::move(inputIn)), presult(presultIn)
{ memcpy(expected, expectedIn, sizeof(expected)); }
bool operator()();
void swap(CRandomXCheck& c) {
rxKey.swap(c.rxKey);
input.swap(c.input);
std::swap(presult, c.presult);
for (int i = 0; i < 32; i++) std::swap(expected[i], c.expected[i]);
}
};
/** The RandomX pre-verification check queue (parallel pool). */
extern CCheckQueue<CRandomXCheck> rxCheckQueue;
/** Worker entry point (spawn N at startup, mirrors ThreadScriptCheck). */
void ThreadRandomXVerify();
/** Load `rxKey` into the shared validator cache (alloc on first use); call before dispatching a
* same-key group of checks. Returns false on allocation failure. */
bool RandomXValidatorPrepareKey(const std::string& rxKey);
/** Release the shared validator cache at shutdown. */
void RandomXValidatorShutdown();
/** Set thread-local flag to skip RandomX validation (used by miner during TestBlockValidity) */
void SetSkipRandomXValidation(bool skip);

View File

@@ -75,6 +75,8 @@ const char *GETNSPV="getnSPV"; //used
const char *NSPV="nSPV"; //used
const char *ALERT="alert"; //used
const char *REJECT="reject"; //used
const char *GETBLOCKSTREAM="getblockstrm"; // 12 chars (COMMAND_SIZE max); "getblockstream" would truncate
const char *BLOCKSTREAM="blockstream";
} // namespace NetMsgType
/** All known message types. Keep this in the same order as the list of
@@ -119,6 +121,8 @@ const static std::string allNetMessageTypes[] = {
NetMsgType::NSPV,
NetMsgType::ALERT,
NetMsgType::REJECT,
NetMsgType::GETBLOCKSTREAM,
NetMsgType::BLOCKSTREAM,
};
CMessageHeader::CMessageHeader(const MessageStartChars& pchMessageStartIn)

View File

@@ -285,6 +285,10 @@ extern const char* GETNSPV;
extern const char* NSPV;
extern const char* ALERT;
extern const char* REJECT;
/** Opt-in bulk block streaming (DragonX): request a contiguous range of old blocks. */
extern const char* GETBLOCKSTREAM;
/** Opt-in bulk block streaming (DragonX): control header preceding a streamed block range. */
extern const char* BLOCKSTREAM;
}; // namespace NetMsgType
/* Get a vector of all valid message types (see above) */
@@ -304,6 +308,9 @@ enum ServiceFlags : uint64_t {
NODE_NSPV = (1 << 30),
NODE_ADDRINDEX = (1 << 29),
NODE_SPENTINDEX = (1 << 28),
// Opt-in bulk block streaming (DragonX). Unauthenticated advertisement; serve/request
// handlers validate every block regardless, so robustness against false advertisement holds.
NODE_BULKBLOCKS = (1 << 27),
// Bits 24-31 are reserved for temporary experiments. Just pick a bit that
// isn't getting used, or one not being used much, and notify the

View File

@@ -30,7 +30,9 @@
#include "rpc/server.h"
#include "streams.h"
#include "sync.h"
#include "txdb.h"
#include "util.h"
#include <boost/filesystem.hpp>
#include "script/script.h"
#include "script/script_error.h"
#include "script/sign.h"
@@ -860,6 +862,7 @@ UniValue gettxoutsetinfo(const UniValue& params, bool fHelp, const CPubKey& mypk
return ret;
}
UniValue getblockmerkletree(const UniValue& params, bool fHelp, const CPubKey& mypk)
{
if (fHelp || params.size() != 1 )

View File

@@ -474,6 +474,7 @@ static const CRPCCommand vRPCCommands[] =
{ "wallet", "z_listaddresses", &z_listaddresses, true },
{ "wallet", "z_listnullifiers", &z_listnullifiers, true },
{ "wallet", "z_exportkey", &z_exportkey, true },
{ "wallet", "z_exportmnemonic", &z_exportmnemonic, true },
{ "wallet", "z_importkey", &z_importkey, true },
{ "wallet", "z_exportviewingkey", &z_exportviewingkey, true },
{ "wallet", "z_importviewingkey", &z_importviewingkey, true },

View File

@@ -353,6 +353,7 @@ extern UniValue nspv_listccmoduleunspent(const UniValue& params, bool fHelp, con
extern UniValue getblocksubsidy(const UniValue& params, bool fHelp, const CPubKey& mypk);
extern UniValue z_exportkey(const UniValue& params, bool fHelp, const CPubKey& mypk); // in rpcdump.cpp
extern UniValue z_exportmnemonic(const UniValue& params, bool fHelp, const CPubKey& mypk); // in rpcdump.cpp
extern UniValue z_importkey(const UniValue& params, bool fHelp, const CPubKey& mypk); // in rpcdump.cpp
extern UniValue z_exportviewingkey(const UniValue& params, bool fHelp, const CPubKey& mypk); // in rpcdump.cpp
extern UniValue z_importviewingkey(const UniValue& params, bool fHelp, const CPubKey& mypk); // in rpcdump.cpp

View File

@@ -21,9 +21,11 @@
#include "txdb.h"
#include "chainparams.h"
#include "clientversion.h"
#include "hash.h"
#include "main.h"
#include "pow.h"
#include "streams.h"
#include "uint256.h"
#include "core_io.h"
#include <stdint.h>

View File

@@ -312,7 +312,7 @@ bool AsyncRPCOperation_mergetoaddress::main_impl()
// recoverable, while keeping it logically separate from the ZIP 32
// Sapling key hierarchy, which the user might not be using.
HDSeed seed;
if (!pwalletMain->GetHDSeed(seed)) {
if (!pwalletMain->GetHDSeedForDerivation(seed)) {
throw JSONRPCError(
RPC_WALLET_ERROR,
"AsyncRPCOperation_sendmany: HD seed not found");

View File

@@ -377,7 +377,7 @@ bool AsyncRPCOperation_sendmany::main_impl() {
// recoverable, while keeping it logically separate from the ZIP 32
// Sapling key hierarchy, which the user might not be using.
HDSeed seed;
if (!pwalletMain->GetHDSeed(seed)) {
if (!pwalletMain->GetHDSeedForDerivation(seed)) {
throw JSONRPCError(
RPC_WALLET_ERROR,
"AsyncRPCOperation_sendmany::main_impl(): HD seed not found");

View File

@@ -197,7 +197,7 @@ bool ShieldToAddress::operator()(const libzcash::SaplingPaymentAddress &zaddr) c
// recoverable, while keeping it logically separate from the ZIP 32
// Sapling key hierarchy, which the user might not be using.
HDSeed seed;
if (!pwalletMain->GetHDSeed(seed)) {
if (!pwalletMain->GetHDSeedForDerivation(seed)) {
throw JSONRPCError(
RPC_WALLET_ERROR,
"CWallet::GenerateNewSaplingZKey(): HD seed not found");

99
src/wallet/mnemonic.cpp Normal file
View File

@@ -0,0 +1,99 @@
// Copyright (c) 2016-2024 The Hush developers
// Distributed under the GPLv3 software license, see the accompanying
// file COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html
#include "wallet/mnemonic.h"
#include "random.h"
#include "support/cleanse.h"
#include <mutex>
#include <cstring>
extern "C" {
#include "crypto/bip39/bip39.h"
}
// The vendored BIP39 library references random_buffer() (used by its
// mnemonic_generate()). We do not compile trezor's insecure rand.c; instead we
// route it to the node CSPRNG so any BIP39 randomness is cryptographically
// sound. random_buffer is declared weak in rand.c, so this strong definition
// is the one that links.
extern "C" void random_buffer(uint8_t* buf, size_t len)
{
GetRandBytes(buf, (int)len);
}
// mnemonic_from_data()/mnemonic_to_seed() use process-static scratch buffers,
// so serialize all access behind one lock and copy results out immediately.
static std::mutex cs_bip39;
bool MnemonicIsValid(const std::string& phrase)
{
std::lock_guard<std::mutex> lock(cs_bip39);
return mnemonic_check(phrase.c_str()) != 0;
}
bool MnemonicToEntropy(const std::string& phrase, RawHDSeed& entropyOut)
{
std::lock_guard<std::mutex> lock(cs_bip39);
// Reject bad checksum / unknown words first.
if (mnemonic_check(phrase.c_str()) == 0) {
return false;
}
// mnemonic_to_entropy() writes 33 bytes (entropy || 1 checksum byte) and
// returns the total bit count (words * 11).
uint8_t buf[33];
int totalBits = mnemonic_to_entropy(phrase.c_str(), buf);
if (totalBits <= 0) {
return false;
}
int words = totalBits / 11;
if (words != 12 && words != 18 && words != 24) {
memory_cleanse(buf, sizeof(buf));
return false;
}
int entropyBytes = words * 4 / 3; // 12->16, 18->24, 24->32
entropyOut.assign(buf, buf + entropyBytes);
memory_cleanse(buf, sizeof(buf));
return true;
}
bool EntropyToMnemonic(const RawHDSeed& entropy, std::string& phraseOut)
{
std::lock_guard<std::mutex> lock(cs_bip39);
const char* phrase = mnemonic_from_data(entropy.data(), (int)entropy.size());
if (phrase == nullptr) {
return false;
}
phraseOut.assign(phrase);
mnemonic_clear(); // wipe the static buffer
return true;
}
bool Bip39SeedFromEntropy(const RawHDSeed& entropy, RawHDSeed& seed64Out)
{
std::lock_guard<std::mutex> lock(cs_bip39);
// Regenerate the canonical phrase from entropy (matches SDXLite's
// Mnemonic::from_entropy(entropy).phrase()), then PBKDF2 with an EMPTY
// passphrase to get the standard 64-byte BIP39 seed.
const char* phrase = mnemonic_from_data(entropy.data(), (int)entropy.size());
if (phrase == nullptr) {
return false;
}
uint8_t seed[64];
mnemonic_to_seed(phrase, "", seed, nullptr);
mnemonic_clear();
seed64Out.assign(seed, seed + 64);
memory_cleanse(seed, sizeof(seed));
return true;
}
bool GenerateMnemonicEntropy(int bits, RawHDSeed& entropyOut)
{
if (bits != 128 && bits != 160 && bits != 192 && bits != 224 && bits != 256) {
return false;
}
entropyOut.resize(bits / 8);
GetRandBytes(entropyOut.data(), (int)entropyOut.size());
return true;
}

39
src/wallet/mnemonic.h Normal file
View File

@@ -0,0 +1,39 @@
// Copyright (c) 2016-2024 The Hush developers
// Distributed under the GPLv3 software license, see the accompanying
// file COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html
#ifndef HUSH_WALLET_MNEMONIC_H
#define HUSH_WALLET_MNEMONIC_H
#include <string>
#include "zcash/zip32.h" // RawHDSeed
// Thin, thread-safe C++ wrapper over the vendored BIP39 (trezor-crypto) library.
// It reproduces SilentDragonXLite's tiny-bip39 0.6.2 conventions EXACTLY so the
// same 24 words yield the same addresses in both wallets:
// - English wordlist only (byte-identical to tiny-bip39's english.txt)
// - empty BIP39 passphrase (no "25th word")
// - PBKDF2-HMAC-SHA512, 2048 rounds, 64-byte seed
// - the seed is derived from the CANONICAL phrase regenerated from entropy,
// matching SDXLite's Mnemonic::from_entropy(entropy).phrase() round-trip.
//! True if `phrase` is a valid BIP39 mnemonic (word list + checksum).
bool MnemonicIsValid(const std::string& phrase);
//! Parse `phrase` into its BIP39 entropy (16/20/24/28/32 bytes). Validates the
//! checksum first. Returns false on any invalid input.
bool MnemonicToEntropy(const std::string& phrase, RawHDSeed& entropyOut);
//! Regenerate the canonical English mnemonic phrase from `entropy`.
bool EntropyToMnemonic(const RawHDSeed& entropy, std::string& phraseOut);
//! Derive the 64-byte BIP39 seed used for HD derivation from `entropy`, exactly
//! as SilentDragonXLite does: canonical phrase from entropy, then PBKDF2 with an
//! empty passphrase.
bool Bip39SeedFromEntropy(const RawHDSeed& entropy, RawHDSeed& seed64Out);
//! Generate fresh BIP39 entropy of `bits` (128/160/192/224/256) from the node
//! CSPRNG, for creating a new mnemonic wallet.
bool GenerateMnemonicEntropy(int bits, RawHDSeed& entropyOut);
#endif // HUSH_WALLET_MNEMONIC_H

View File

@@ -742,7 +742,9 @@ UniValue dumpwallet_impl(const UniValue& params, bool fHelp, bool fDumpZKeys)
file << strprintf("# mined on %s\n", EncodeDumpTime(chainActive.Tip()->GetBlockTime()));
{
HDSeed hdSeed;
pwalletMain->GetHDSeed(hdSeed);
// Dump the 64-byte derivation seed (for mnemonic wallets this is the
// expanded BIP39 seed), so re-importing the hex reproduces the same keys.
pwalletMain->GetHDSeedForDerivation(hdSeed);
auto rawSeed = hdSeed.RawSeed();
file << strprintf("# HDSeed=%s fingerprint=%s", HexStr(rawSeed.begin(), rawSeed.end()), hdSeed.Fingerprint().GetHex());
file << "\n";
@@ -1026,6 +1028,50 @@ UniValue z_exportkey(const UniValue& params, bool fHelp, const CPubKey& mypk)
return EncodeSpendingKey(sk.get());
}
UniValue z_exportmnemonic(const UniValue& params, bool fHelp, const CPubKey& mypk)
{
if (!EnsureWalletIsAvailable(fHelp))
return NullUniValue;
if (fHelp || params.size() != 0)
throw runtime_error(
"z_exportmnemonic\n"
"\nReveal the wallet's BIP39 seed phrase (24 words).\n"
"The phrase is byte-compatible with SilentDragonXLite: the same words\n"
"restore the same transparent and shielded addresses in either wallet.\n"
"Only works for wallets created or restored from a mnemonic (see the\n"
"-mnemonic and -usemnemonic options). Requires the wallet be unlocked.\n"
"\nResult:\n"
"{\n"
" \"mnemonic\" : \"word1 ... word24\", (string) the BIP39 seed phrase\n"
" \"seedfp\" : \"hex\" (string) the HD seed fingerprint\n"
"}\n"
"\nExamples:\n"
+ HelpExampleCli("z_exportmnemonic", "")
+ HelpExampleRpc("z_exportmnemonic", "")
);
LOCK2(cs_main, pwalletMain->cs_wallet);
EnsureWalletIsUnlocked();
if (!pwalletMain->IsMnemonicSeed()) {
throw JSONRPCError(RPC_WALLET_ERROR,
"This wallet's seed was not derived from a mnemonic, so no seed phrase is available. "
"Use z_exportwallet to back up the raw HD seed instead.");
}
std::string phrase;
if (!pwalletMain->GetMnemonicPhrase(phrase)) {
throw JSONRPCError(RPC_WALLET_ERROR, "Could not retrieve the seed phrase (is the wallet unlocked?)");
}
UniValue ret(UniValue::VOBJ);
ret.push_back(Pair("mnemonic", phrase));
ret.push_back(Pair("seedfp", pwalletMain->GetHDChain().seedFp.GetHex()));
return ret;
}
UniValue z_exportviewingkey(const UniValue& params, bool fHelp, const CPubKey& mypk)
{
if (!EnsureWalletIsAvailable(fHelp))

View File

@@ -305,7 +305,7 @@ void zsTxSendsToJSON(const CWalletTx& wtx, UniValue& sends, CAmount& totalSends,
//Decrypt sapling outgoing t to z transaction using HDseed
if (wtx.vShieldedSpend.size()==0) {
HDSeed seed;
if (pwalletMain->GetHDSeed(seed)) {
if (pwalletMain->GetHDSeedForDerivation(seed)) {
auto opt = libzcash::SaplingOutgoingPlaintext::decrypt(
outputDesc.outCiphertext,ovkForShieldingFromTaddr(seed),outputDesc.cv,outputDesc.cm,outputDesc.ephemeralKey);

View File

@@ -6272,6 +6272,7 @@ extern UniValue importaddress(const UniValue& params, bool fHelp, const CPubKey&
extern UniValue dumpwallet(const UniValue& params, bool fHelp, const CPubKey& mypk);
extern UniValue importwallet(const UniValue& params, bool fHelp, const CPubKey& mypk);
extern UniValue z_exportkey(const UniValue& params, bool fHelp, const CPubKey& mypk);
extern UniValue z_exportmnemonic(const UniValue& params, bool fHelp, const CPubKey& mypk);
extern UniValue z_importkey(const UniValue& params, bool fHelp, const CPubKey& mypk);
extern UniValue z_exportviewingkey(const UniValue& params, bool fHelp, const CPubKey& mypk);
extern UniValue z_importviewingkey(const UniValue& params, bool fHelp, const CPubKey& mypk);
@@ -6351,6 +6352,7 @@ static const CRPCCommand commands[] =
{ "wallet", "z_getnewaddress", &z_getnewaddress, true },
{ "wallet", "z_listaddresses", &z_listaddresses, true },
{ "wallet", "z_exportkey", &z_exportkey, true },
{ "wallet", "z_exportmnemonic", &z_exportmnemonic, true },
{ "wallet", "z_importkey", &z_importkey, true },
{ "wallet", "z_exportviewingkey", &z_exportviewingkey, true },
{ "wallet", "z_importviewingkey", &z_importviewingkey, true },

View File

@@ -36,11 +36,15 @@
#include "utilmoneystr.h"
#include "zcash/Note.hpp"
#include "crypter.h"
#include "wallet/mnemonic.h"
#include "coins.h"
#include "wallet/asyncrpcoperation_saplingconsolidation.h"
#include "wallet/asyncrpcoperation_sweep.h"
#include <random>
#include <limits>
#include <thread>
#include <atomic>
#include <mutex>
#include "zcash/zip32.h"
#include "cc/CCinclude.h"
#include <assert.h>
@@ -128,7 +132,7 @@ SaplingPaymentAddress CWallet::GenerateNewSaplingZKey(bool addToWallet)
// Try to get the seed
HDSeed seed;
if (!GetHDSeed(seed))
if (!GetHDSeedForDerivation(seed))
throw std::runtime_error("CWallet::GenerateNewSaplingZKey(): HD seed not found");
auto m = libzcash::SaplingExtendedSpendingKey::Master(seed);
@@ -219,7 +223,20 @@ CPubKey CWallet::GenerateNewKey()
bool fCompressed = CanSupportFeature(FEATURE_COMPRPUBKEY); // default to compressed public keys if we want 0.6.0 wallets
CKey secret;
secret.MakeNewKey(fCompressed);
// Create new metadata
int64_t nCreationTime = GetTime();
CKeyMetadata metadata(nCreationTime);
// Derive the transparent key deterministically from the HD seed when the
// feature is enabled, so it can be recovered from the seed alone. Otherwise
// fall back to a random key (e.g. legacy wallets that have no HD seed).
if (IsHDTransparentEnabled()) {
DeriveNewChildKey(metadata, secret);
fCompressed = true; // BIP32-derived keys are always compressed
} else {
secret.MakeNewKey(fCompressed);
}
// Compressed public keys were introduced in version 0.6.0
if (fCompressed)
@@ -228,9 +245,7 @@ CPubKey CWallet::GenerateNewKey()
CPubKey pubkey = secret.GetPubKey();
assert(secret.VerifyPubKey(pubkey));
// Create new metadata
int64_t nCreationTime = GetTime();
mapKeyMetadata[pubkey.GetID()] = CKeyMetadata(nCreationTime);
mapKeyMetadata[pubkey.GetID()] = metadata;
if (!nTimeFirstKey || nCreationTime < nTimeFirstKey)
nTimeFirstKey = nCreationTime;
@@ -239,6 +254,57 @@ CPubKey CWallet::GenerateNewKey()
return pubkey;
}
// Derive a new transparent key from the HD seed along the BIP44 external chain
// m/44'/coin_type'/0'/0/i. The child index is taken from (and advances)
// hdChain.transparentChildCounter, which is persisted so the same keys can be
// regenerated after a seed-only restore. Mirrors GenerateNewSaplingZKey.
void CWallet::DeriveNewChildKey(CKeyMetadata& metadata, CKey& secretRet)
{
AssertLockHeld(cs_wallet); // mapKeyMetadata / hdChain
HDSeed seed;
if (!GetHDSeedForDerivation(seed))
throw std::runtime_error("CWallet::DeriveNewChildKey(): HD seed not found");
RawHDSeed rawSeed = seed.RawSeed();
CExtKey masterKey; // m
CExtKey purposeKey; // m/44'
CExtKey coinTypeKey; // m/44'/coin_type'
CExtKey accountKey; // m/44'/coin_type'/0'
CExtKey externalChainKey; // m/44'/coin_type'/0'/0
CExtKey childKey; // m/44'/coin_type'/0'/0/i
masterKey.SetMaster(rawSeed.data(), rawSeed.size());
uint32_t bip44CoinType = Params().BIP44CoinType();
// BIP44 path, single account (0'), external chain (0). On this ac_private=1
// chain the internal/change chain can never hold value, so it is unused.
masterKey.Derive(purposeKey, 44 | BIP32_HARDENED_KEY_LIMIT);
purposeKey.Derive(coinTypeKey, bip44CoinType | BIP32_HARDENED_KEY_LIMIT);
coinTypeKey.Derive(accountKey, 0 | BIP32_HARDENED_KEY_LIMIT);
accountKey.Derive(externalChainKey, 0);
// Derive the next child index, skipping any key already in the wallet.
do {
externalChainKey.Derive(childKey, hdChain.transparentChildCounter);
metadata.hdKeypath = "m/44'/" + std::to_string(bip44CoinType) + "'/0'/0/" + std::to_string(hdChain.transparentChildCounter);
metadata.seedFp = hdChain.seedFp;
hdChain.transparentChildCounter++;
} while (HaveKey(childKey.key.GetPubKey().GetID()));
secretRet = childKey.key;
// Bump a legacy v1 chain to v2 so the transparent counter gets persisted.
if (hdChain.nVersion < CHDChain::VERSION_HD_TRANSPARENT)
hdChain.nVersion = CHDChain::VERSION_HD_TRANSPARENT;
// Persist the advanced counter so restarts / restores don't reuse indices.
if (fFileBacked && !CWalletDB(strWalletFile).WriteHDChain(hdChain))
throw std::runtime_error("CWallet::DeriveNewChildKey(): Writing HD chain model failed");
}
bool CWallet::AddKeyPubKey(const CKey& secret, const CPubKey &pubkey)
{
AssertLockHeld(cs_wallet); // mapKeyMetadata
@@ -990,11 +1056,22 @@ void CWallet::DecrementNoteWitnesses(const CBlockIndex* pindex)
if (nd->nullifier && pwalletMain->GetSaplingSpendDepth(*item.second.nullifier) <= WITNESS_CACHE_SIZE) {
// Only decrement witnesses that are not above the current height
if (nd->witnessHeight <= pindex->GetHeight()) {
//PART B1: a rolled-back note must re-validate on reconnect (flag is in-memory only).
nd->witnessRootValidated = false;
if (nd->witnesses.size() > 1) {
// indexHeight is the height of the block being removed, so
// the new witness cache height is one below it.
nd->witnesses.pop_front();
nd->witnessHeight = pindex->GetHeight() - 1;
} else {
//PART B1: with only the base witness left we cannot pop_front without emptying
//the cache, but we must NOT leave witnessHeight stranded above the disconnected
//tip (that high-height/stale-witness state is the desync originator). Force a
//clean reseed on reconnect instead of lying about the height. (Inlined because
//ClearSingleNoteWitnessCache is defined later in this file.)
nd->witnesses.clear();
nd->witnessHeight = -1;
nd->witnessRootValidated = false;
}
}
}
@@ -1060,10 +1137,23 @@ int CWallet::VerifyAndSetInitialWitness(const CBlockIndex* pindex, bool witnessO
continue;
}
//Skip Validation when witness height is greater that block height
//PART A: a witness whose height is at/above the build height must NOT be blindly trusted.
//Validate its root against the canonical sapling root at witnessHeight when that block is
//on the active chain. Match -> safe to skip. Mismatch (witnessHeight advanced past the real
//witness state = the desync signature) -> fall through to ClearSingleNoteWitnessCache + reseed.
if (nd->witnessHeight > pindex->GetHeight() - 1) {
nMinimumHeight = SaplingWitnessMinimumHeight(*item.second.nullifier, nd->witnessHeight, nMinimumHeight);
continue;
CBlockIndex* whIndex = chainActive[nd->witnessHeight];
if (whIndex == NULL) {
//witnessHeight strictly above the active chain (transient catch-up): cannot validate yet
nMinimumHeight = SaplingWitnessMinimumHeight(*item.second.nullifier, nd->witnessHeight, nMinimumHeight);
continue;
}
if (nd->witnesses.front().root() == whIndex->hashFinalSaplingRoot) {
nd->witnessRootValidated = true;
nMinimumHeight = SaplingWitnessMinimumHeight(*item.second.nullifier, nd->witnessHeight, nMinimumHeight);
continue;
}
//root mismatch on the active chain -> desynced; fall through to rebuild below
}
//Validate the witness at the witness height
@@ -1145,74 +1235,206 @@ void CWallet::BuildWitnessCache(const CBlockIndex* pindex, bool witnessOnly)
return;
}
uint256 saplingRoot;
CBlockIndex* pblockindex = chainActive[startHeight];
int height = chainActive.Height();
if(fZdebug)
LogPrintf("%s: height=%d, startHeight=%d\n", __func__, height, startHeight);
LogPrintf("%s: startHeight=%d, tip=%d\n", __func__, startHeight, chainActive.Height());
while (pblockindex) {
if (ShutdownRequested()) {
LogPrintf("%s: shutdown requested, aborting building witnesses\n", __func__);
break;
// Tier 1 optimization: build the set of notes that still need extension ONCE instead of
// rescanning all of mapWallet for every block. A note's gate conditions (nullifier present,
// spend depth <= WITNESS_CACHE_SIZE, tx confirmed) are invariant across this loop (the active
// chain is fixed under cs_main), so they are evaluated once here rather than per block. A note
// becomes "active" at the block where witnessHeight == GetHeight()-1 (exactly the original
// per-block gate) and is then extended every subsequent block, so the produced witnesses are
// byte-for-byte identical to the original full-rescan implementation -- only the bookkeeping
// cost changes from O(blocks * walletSize) to O(blocks + activeNotes).
struct PendingNote { int startWitnessHeight; SaplingNoteData* nd; };
std::vector<PendingNote> pending;
for (std::pair<const uint256, CWalletTx>& wtxItem : mapWallet) {
if (wtxItem.second.mapSaplingNoteData.empty())
continue;
if (wtxItem.second.GetDepthInMainChain() <= 0)
continue;
for (mapSaplingNoteData_t::value_type& item : wtxItem.second.mapSaplingNoteData) {
SaplingNoteData* nd = &(item.second);
if (!nd->nullifier)
continue;
if (nd->witnesses.empty()) // cannot extend (front() would be UB); the original gate also never matched these in practice
continue;
if (GetSaplingSpendDepth(*nd->nullifier) > WITNESS_CACHE_SIZE)
continue;
// Only notes that still lag the build target need extension. The lowest such witnessHeight
// is exactly startHeight-1 (startHeight = nMinimumHeight+1 from SaplingWitnessMinimumHeight).
if (nd->witnessHeight >= startHeight - 1 && nd->witnessHeight <= pindex->GetHeight() - 1)
pending.push_back({ nd->witnessHeight, nd });
}
if(pwalletMain->fAbortRescan) {
LogPrintf("%s: rescan aborted at block %d, stopping witness building\n", pwalletMain->rescanHeight);
}
std::sort(pending.begin(), pending.end(),
[](const PendingNote& a, const PendingNote& b) { return a.startWitnessHeight < b.startWitnessHeight; });
// Phase 1 (serial, main thread under cs_main/cs_wallet): extract the per-block Sapling
// commitments for [startHeight, tip] into memory. This is the only part touching chain/disk;
// profiling showed it is ~1% of rebuild time. ~9MB for a full-chain range.
const int tipHeight = pindex->GetHeight();
const int rangeLen = tipHeight - startHeight + 1;
std::vector<std::vector<uint256>> blockCms(rangeLen > 0 ? rangeLen : 0);
int64_t tRead = 0;
{
int64_t r0 = GetTimeMicros();
CBlockIndex* pbi = chainActive[startHeight];
while (pbi) {
if (ShutdownRequested()) {
LogPrintf("%s: shutdown requested, aborting witness rebuild\n", __func__);
return;
}
if (pwalletMain->fAbortRescan) {
LogPrintf("%s: rescan aborted during witness rebuild\n", __func__);
pwalletMain->fRescanning = false;
return;
}
if (pblockindex->GetHeight() % 100 == 0 && pblockindex->GetHeight() < height - 5) {
LogPrintf("Building Witnesses for block %i %.4f complete, %d remaining\n", pblockindex->GetHeight(), pblockindex->GetHeight() / double(height), height - pblockindex->GetHeight() );
}
SaplingMerkleTree saplingTree;
saplingRoot = pblockindex->pprev->hashFinalSaplingRoot;
pcoinsTip->GetSaplingAnchorAt(saplingRoot, saplingTree);
//Cycle through blocks and transactions building sapling tree until the commitment needed is reached
CBlock block;
if (!ReadBlockFromDisk(block, pblockindex, 1)) {
throw std::runtime_error(
strprintf("Cannot read block height %d (%s) from disk", pindex->GetHeight(), pindex->GetBlockHash().GetHex()));
}
for (std::pair<const uint256, CWalletTx>& wtxItem : mapWallet) {
if (wtxItem.second.mapSaplingNoteData.empty())
continue;
if (wtxItem.second.GetDepthInMainChain() > 0) {
//Sapling
for (mapSaplingNoteData_t::value_type& item : wtxItem.second.mapSaplingNoteData) {
auto* nd = &(item.second);
if (nd->nullifier && nd->witnessHeight == pblockindex->GetHeight() - 1
&& GetSaplingSpendDepth(*item.second.nullifier) <= WITNESS_CACHE_SIZE) {
nd->witnesses.push_front(nd->witnesses.front());
while (nd->witnesses.size() > WITNESS_CACHE_SIZE) {
nd->witnesses.pop_back();
}
for (const CTransaction& tx : block.vtx) {
for (uint32_t i = 0; i < tx.vShieldedOutput.size(); i++) {
const uint256& note_commitment = tx.vShieldedOutput[i].cm;
nd->witnesses.front().append(note_commitment);
}
}
nd->witnessHeight = pblockindex->GetHeight();
}
}
}
int h = pbi->GetHeight();
if (h % 5000 == 0 && h < tipHeight - 5)
LogPrintf("Reading blocks for witness rebuild: %d / %d\n", h - startHeight, rangeLen);
CBlock block;
if (!ReadBlockFromDisk(block, pbi, 1)) {
throw std::runtime_error(strprintf("Cannot read block height %d from disk", h));
}
std::vector<uint256>& cms = blockCms[h - startHeight];
for (const CTransaction& tx : block.vtx)
for (uint32_t i = 0; i < tx.vShieldedOutput.size(); i++)
cms.push_back(tx.vShieldedOutput[i].cm);
if (pbi == pindex) break;
pbi = chainActive.Next(pbi);
}
tRead = GetTimeMicros() - r0;
}
if (pblockindex == pindex)
break;
// Phase 2 (parallel): each note's witness extension is independent, so partition the lagging
// notes across worker threads. Workers touch ONLY their own notes' witness lists plus the
// read-only commitment cache -- no locks, no chain access -- while the main thread keeps
// cs_main/cs_wallet held throughout. Round-robin assignment over the start-sorted `pending`
// spreads the long (low-start) notes across threads. Produces witnesses byte-identical to the
// serial path: every note is extended over exactly its [witnessHeight+1, tip] block range, in
// order, appending each block's commitments.
std::vector<SaplingNoteData*> work;
work.reserve(pending.size());
for (const PendingNote& p : pending) work.push_back(p.nd);
pblockindex = chainActive.Next(pblockindex);
// Only a substantial bulk rebuild (e.g. a one-time post-upgrade repair) is worth parallelizing
// and logging; routine 1-block tip extension runs serially to avoid per-block thread-spawn
// overhead and log spam.
const bool bulkRebuild = (rangeLen > 100);
int nPar = (int)GetArg("-witnessbuildthreads", 0);
if (nPar <= 0) nPar = (int)std::thread::hardware_concurrency();
if (nPar <= 0) nPar = 1;
if (nPar > (int)work.size()) nPar = (int)work.size();
if (nPar < 1) nPar = 1;
if (!bulkRebuild) nPar = 1;
std::atomic<bool> failed(false);
std::mutex failMtx;
std::string failMsg;
const int CACHE = (int)WITNESS_CACHE_SIZE;
// Tier 2b: advance one witness in place for the deep part (no per-block heap clone), then
// materialize only the final CACHE snapshots. -witnessfastrebuild=0 forces the reference
// clone-every-block path for A/B verification.
bool fastRebuild = GetBoolArg("-witnessfastrebuild", true);
auto worker = [&](int tid) {
try {
for (size_t k = (size_t)tid; k < work.size(); k += (size_t)nPar) {
SaplingNoteData* nd = work[k];
int startH = nd->witnessHeight;
int nBlocks = tipHeight - startH;
if (nBlocks <= 0)
continue;
if (!fastRebuild || nBlocks <= CACHE) {
// Reference / shallow path: clone every block (preserves pre-existing older snapshots).
for (int h = startH + 1; h <= tipHeight; h++) {
nd->witnesses.push_front(nd->witnesses.front());
while ((int)nd->witnesses.size() > CACHE)
nd->witnesses.pop_back();
const std::vector<uint256>& cms = blockCms[h - startHeight];
for (size_t c = 0; c < cms.size(); c++)
nd->witnesses.front().append(cms[c]);
nd->witnessHeight = h;
}
} else {
// Deep fast path: advance ONE witness in place through [startH+1, tipHeight-CACHE]
// with no per-block clone, then build only the final CACHE snapshots. The reference
// loop pops all but the last CACHE snapshots, so the resulting deque is identical
// ([W@tip .. W@(tip-CACHE+1)]), but with ~CACHE heap allocations instead of ~nBlocks.
int deepEnd = tipHeight - CACHE;
{
SaplingWitness& w = nd->witnesses.front();
for (int h = startH + 1; h <= deepEnd; h++) {
const std::vector<uint256>& cms = blockCms[h - startHeight];
for (size_t c = 0; c < cms.size(); c++)
w.append(cms[c]);
}
}
// Drop pre-existing older snapshots; keep only the advanced front (W@deepEnd).
while (nd->witnesses.size() > 1)
nd->witnesses.pop_back();
// Materialize the last CACHE snapshots (heights deepEnd+1 .. tip).
for (int h = deepEnd + 1; h <= tipHeight; h++) {
nd->witnesses.push_front(nd->witnesses.front());
const std::vector<uint256>& cms = blockCms[h - startHeight];
for (size_t c = 0; c < cms.size(); c++)
nd->witnesses.front().append(cms[c]);
}
while ((int)nd->witnesses.size() > CACHE)
nd->witnesses.pop_back();
nd->witnessHeight = tipHeight;
}
}
} catch (const std::exception& e) {
std::lock_guard<std::mutex> lk(failMtx);
if (failMsg.empty()) failMsg = e.what();
failed = true;
} catch (...) {
failed = true;
}
};
int64_t e0 = GetTimeMicros();
if (nPar <= 1) {
worker(0);
} else {
std::vector<std::thread> threads;
threads.reserve(nPar);
for (int t = 0; t < nPar; t++) threads.emplace_back(worker, t);
for (std::thread& th : threads) th.join();
}
int64_t tExtend = GetTimeMicros() - e0;
if (failed)
throw std::runtime_error(std::string("Witness rebuild worker failed: ") + (failMsg.empty() ? "unknown" : failMsg));
// Latch the rebuilt notes as validated so they are not re-validated and reseeded on every
// subsequent block connect. Without this, any note whose witness cannot be reconstructed to
// the canonical anchor (e.g. legacy corruption) would be reseeded and fully replayed on every
// block forever. A note whose rebuilt root still disagrees with the canonical finalsaplingroot
// is unrecoverable here: it is left flagged (the GetSaplingNoteWitnesses majority-anchor guard
// skips it for spends) and reported, rather than spun on indefinitely. witnessRootValidated is
// in-memory only, so a fresh validation pass still runs on each restart and after any reorg
// (DecrementNoteWitnesses clears it), keeping the heal self-correcting.
if (!work.empty()) {
const uint256& canonicalRoot = pindex->hashFinalSaplingRoot;
int nUnrecoverable = 0;
for (SaplingNoteData* nd : work) {
if (nd->witnesses.empty() || nd->witnesses.front().root() != canonicalRoot)
nUnrecoverable++;
nd->witnessRootValidated = true;
}
if (bulkRebuild) {
LogPrintf("%s: rebuilt %u note witness cache(s) to height %d in %ldms using %d thread(s)%s\n",
__func__, (unsigned)work.size(), tipHeight, (long)((tRead + tExtend) / 1000), nPar,
nUnrecoverable
? strprintf(" [WARNING: %d note(s) could not be rebuilt to the canonical anchor and were skipped]", nUnrecoverable).c_str()
: "");
}
}
}
@@ -1593,8 +1815,11 @@ bool CWallet::UpdatedNoteData(const CWalletTx& wtxIn, CWalletTx& wtx)
if (tmp.count(nd.first) && nd.second.witnesses.size() > 0) {
tmp.at(nd.first).witnesses.assign(
nd.second.witnesses.cbegin(), nd.second.witnesses.cend());
//PART B2: only carry over witnessHeight TOGETHER with the witnesses it describes.
//Copying it unconditionally (when witnesses are NOT copied) advances the height past
//the actual witness state for a whole tx's notes at once = the batch desync originator.
tmp.at(nd.first).witnessHeight = nd.second.witnessHeight;
}
tmp.at(nd.first).witnessHeight = nd.second.witnessHeight;
}
// Now copy over the updated note data
@@ -1833,37 +2058,54 @@ void CWallet::GetSaplingNoteWitnesses(std::vector<SaplingOutPoint> notes,
uint256 &final_anchor)
{
LOCK(cs_wallet);
witnesses.clear();
witnesses.resize(notes.size());
boost::optional<uint256> rt;
int i = 0;
for (SaplingOutPoint note : notes) {
//fprintf(stderr,"%s: i=%d\n", __func__,i);
auto noteData = mapWallet[note.hash].mapSaplingNoteData;
auto nWitnesses = noteData[note].witnesses.size();
if (mapWallet.count(note.hash) && noteData.count(note) && nWitnesses > 0) {
fprintf(stderr,"%s: Found %lu witnesses for note %s...\n", __func__, nWitnesses, note.hash.ToString().substr(0,8).c_str() );
witnesses[i] = noteData[note].witnesses.front();
if (!rt) {
//fprintf(stderr,"%s: Setting witness root\n",__func__);
rt = witnesses[i]->root();
} else {
if(*rt == witnesses[i]->root()) {
} else {
// Something is fucky
std::string err = string("CWallet::GetSaplingNoteWitnesses: Invalid witness root! rt=") + rt.get().ToString();
err += string("\n!= witness[i]->root()=") + witnesses[i]->root().ToString();
fprintf(stderr,"%s: IGNORING %s\n", __func__,err.c_str());
}
}
// Pass 1: collect each note's most-recent cached witness and tally roots. Use find() so we do
// NOT default-construct mapWallet / mapSaplingNoteData entries (the original indexed
// mapWallet[note.hash] BEFORE its own count() guard, silently inserting empty entries).
std::vector<boost::optional<SaplingWitness>> cand(notes.size());
std::map<uint256, int> rootVotes;
for (size_t i = 0; i < notes.size(); i++) {
const SaplingOutPoint& note = notes[i];
auto wi = mapWallet.find(note.hash);
if (wi == mapWallet.end())
continue;
const mapSaplingNoteData_t& noteData = wi->second.mapSaplingNoteData;
auto ni = noteData.find(note);
if (ni == noteData.end() || ni->second.witnesses.empty())
continue;
cand[i] = ni->second.witnesses.front();
rootVotes[cand[i]->root()]++;
}
// Choose the anchor that the most witnesses agree on (robust even when the FIRST note is the
// desynced one - the original code blindly took the first note's root as the anchor).
boost::optional<uint256> anchor;
int bestVotes = 0;
for (const std::pair<uint256, int>& rv : rootVotes) {
if (rv.second > bestVotes) { bestVotes = rv.second; anchor = rv.first; }
}
// Pass 2: only emit witnesses whose root matches the common anchor. A desynced witness is left
// as boost::none rather than returned: handing the spend prover a witness whose root disagrees
// with the anchor guarantees a "Failed to build transaction". Note selection / callers skip
// notes that have no usable witness (see asyncrpcoperation_*: "Missing witness for Sapling note").
for (size_t i = 0; i < notes.size(); i++) {
if (cand[i] && anchor && cand[i]->root() == *anchor) {
witnesses[i] = cand[i];
} else {
if (cand[i])
LogPrintf("%s: note %s has a desynced witness (root=%s != anchor=%s); skipping it\n",
__func__, notes[i].hash.ToString().substr(0, 16).c_str(),
cand[i]->root().ToString().c_str(),
anchor ? anchor->ToString().c_str() : "none");
witnesses[i] = boost::none;
}
i++;
}
// All returned witnesses have the same anchor
if (rt) {
final_anchor = *rt;
//fprintf(stderr,"%s: final_anchor=%s\n", __func__, rt.get().ToString().c_str() );
}
if (anchor)
final_anchor = *anchor;
}
isminetype CWallet::IsMine(const CTxIn &txin) const
@@ -2111,18 +2353,39 @@ CAmount CWallet::GetChange(const CTransaction& tx) const
bool CWallet::IsHDFullyEnabled() const
{
// Only Sapling addresses are HD for now
return false;
// Both Sapling and transparent addresses are HD when transparent HD is on.
return IsHDTransparentEnabled();
}
bool CWallet::IsHDTransparentEnabled() const
{
// Transparent keys are HD-derived when the wallet has an HD seed and the
// feature is enabled (default on). Legacy wallets keep any pre-existing
// random t-keys; only newly generated keys become HD (and those old random
// keys are NOT seed-recoverable, so wallet.dat backups remain necessary).
return !hdChain.seedFp.IsNull() && GetBoolArg("-hdtransparent", true);
}
void CWallet::GenerateNewSeed()
{
LOCK(cs_wallet);
auto seed = HDSeed::Random(HD_WALLET_SEED_LENGTH);
int64_t nCreationTime = GetTime();
// Opt-in: create the wallet from a fresh BIP39 mnemonic so its 24 words can
// be exported (z_exportmnemonic) and used in SilentDragonXLite.
if (GetBoolArg("-usemnemonic", false)) {
RawHDSeed entropy;
if (GenerateMnemonicEntropy(256, entropy)) {
HDSeed seed(entropy);
if (InstallHDSeed(seed, true, nCreationTime))
return;
}
LogPrintf("%s: -usemnemonic seed generation failed, falling back to a random seed\n", __func__);
}
auto seed = HDSeed::Random(HD_WALLET_SEED_LENGTH);
// If the wallet is encrypted and locked, this will fail.
if (!SetHDSeed(seed))
throw std::runtime_error(std::string(__func__) + ": SetHDSeed failed");
@@ -2131,7 +2394,7 @@ void CWallet::GenerateNewSeed()
// the child index counter in the database
// as a hdchain object
CHDChain newHdChain;
newHdChain.nVersion = CHDChain::VERSION_HD_BASE;
newHdChain.nVersion = CHDChain::VERSION_HD_TRANSPARENT;
newHdChain.seedFp = seed.Fingerprint();
newHdChain.nCreateTime = nCreationTime;
SetHDChain(newHdChain, false);
@@ -2195,6 +2458,122 @@ bool CWallet::LoadCryptedHDSeed(const uint256& seedFp, const std::vector<unsigne
return CCryptoKeyStore::SetCryptedHDSeed(seedFp, seed);
}
bool CWallet::InstallHDSeed(const HDSeed& seed, bool fMnemonic, int64_t nCreateTime)
{
AssertLockHeld(cs_wallet);
if (!SetHDSeed(seed))
return false;
CHDChain newHdChain;
newHdChain.nVersion = fMnemonic ? CHDChain::VERSION_HD_MNEMONIC
: CHDChain::VERSION_HD_TRANSPARENT;
newHdChain.seedFp = seed.Fingerprint();
newHdChain.nCreateTime = nCreateTime;
newHdChain.fMnemonicSeed = fMnemonic;
SetHDChain(newHdChain, false);
return true;
}
bool CWallet::SetHDSeedFromHex(const std::string& seedHex)
{
LOCK(cs_wallet);
// Refuse to clobber an existing seed (the keystore refuses too); restore
// must run on a fresh/empty wallet.
if (HaveHDSeed())
return false;
if (!IsHex(seedHex))
return false;
std::vector<unsigned char> raw = ParseHex(seedHex);
// 32 = legacy raw seed; 64 = BIP39-derived seed (as exported by a mnemonic
// wallet). Either is used directly for derivation (fMnemonicSeed = false).
if (raw.size() != 32 && raw.size() != 64)
return false;
RawHDSeed rawSeed(raw.begin(), raw.end());
HDSeed seed(rawSeed);
return InstallHDSeed(seed, false, 1); // birthday = genesis for a restore
}
bool CWallet::SetHDSeedFromMnemonic(const std::string& phrase)
{
LOCK(cs_wallet);
if (HaveHDSeed())
return false;
RawHDSeed entropy;
if (!MnemonicToEntropy(phrase, entropy))
return false;
// Store the BIP39 entropy as the HDSeed (SilentDragonXLite's on-disk
// convention); the 64-byte seed is expanded from it on demand.
HDSeed seed(entropy);
return InstallHDSeed(seed, true, 1); // birthday = genesis for a restore
}
bool CWallet::GetHDSeedForDerivation(HDSeed& seedOut) const
{
HDSeed stored;
if (!GetHDSeed(stored))
return false;
if (!hdChain.fMnemonicSeed) {
seedOut = stored; // legacy / hex seed: fed to derivation directly
return true;
}
// Mnemonic wallet: the stored seed is the 32-byte BIP39 entropy. Expand it
// to the 64-byte BIP39 seed exactly as SilentDragonXLite does.
RawHDSeed seed64;
if (!Bip39SeedFromEntropy(stored.RawSeed(), seed64))
return false;
seedOut = HDSeed(seed64);
return true;
}
bool CWallet::GetMnemonicPhrase(std::string& phraseOut) const
{
if (!hdChain.fMnemonicSeed)
return false;
HDSeed stored;
if (!GetHDSeed(stored)) // fails on an encrypted+locked wallet
return false;
return EntropyToMnemonic(stored.RawSeed(), phraseOut);
}
void CWallet::TopUpHDTransparentKeys(unsigned int count, int64_t nBirthday)
{
LOCK(cs_wallet);
if (!IsHDTransparentEnabled())
return;
for (unsigned int i = 0; i < count; i++) {
CKey secret;
CKeyMetadata metadata(nBirthday);
DeriveNewChildKey(metadata, secret);
CPubKey pubkey = secret.GetPubKey();
assert(secret.VerifyPubKey(pubkey));
mapKeyMetadata[pubkey.GetID()] = metadata;
// Keep the birthday floor at nBirthday so the rescan is not clipped
// (derived keys are stamped nBirthday, not "now", precisely for this).
if (!nTimeFirstKey || nBirthday < nTimeFirstKey)
nTimeFirstKey = nBirthday;
if (!AddKeyPubKey(secret, pubkey))
throw std::runtime_error("CWallet::TopUpHDTransparentKeys(): AddKeyPubKey failed");
}
}
void CWalletTx::SetSaplingNoteData(mapSaplingNoteData_t &noteData)
{
mapSaplingNoteData.clear();

View File

@@ -311,7 +311,9 @@ public:
boost::optional<uint256> nullifier;
//In Memory Only
bool witnessRootValidated;
// Never serialized (see SerializationOp): must default false so a garbage value can't
// read true and short-circuit the witness self-heal in VerifyAndSetInitialWitness.
bool witnessRootValidated = false;
ADD_SERIALIZE_METHODS;
@@ -1061,6 +1063,9 @@ public:
* Generate a new key
*/
CPubKey GenerateNewKey();
//! Derive a new transparent key from the HD seed along the BIP44 external
//! chain m/44'/coin_type'/0'/0/i, advancing hdChain.transparentChildCounter.
void DeriveNewChildKey(CKeyMetadata& metadata, CKey& secretRet);
//! Adds a key to the store, and saves it to disk.
bool AddKeyPubKey(const CKey& key, const CPubKey &pubkey);
//! Adds a key to the store, without saving it to disk (used by LoadWallet)
@@ -1292,6 +1297,10 @@ public:
/* Returns true if HD is enabled for all address types, false if only for Sapling */
bool IsHDFullyEnabled() const;
/* Returns true if transparent keys should be HD-derived from the seed.
Requires an HD seed and the -hdtransparent option (default on). */
bool IsHDTransparentEnabled() const;
/* Generates a new HD seed (will reset the chain child index counters)
Sets the seed's version based on the current wallet version (so the
caller must ensure the current wallet version is correct before calling
@@ -1301,6 +1310,41 @@ public:
bool SetHDSeed(const HDSeed& seed);
bool SetCryptedHDSeed(const uint256& seedFp, const std::vector<unsigned char> &vchCryptedSecret);
/* Restore a wallet's HD seed from a hex string (as exported in the
z_exportwallet "# HDSeed=" comment): 32 bytes for a legacy raw seed, or
64 bytes for a BIP39-derived seed. Only succeeds on a wallet that has no
seed yet. Sets the chain birthday to genesis so a rescan finds all
historical (coinbase) funds. Returns false on bad input or existing seed. */
bool SetHDSeedFromHex(const std::string& seedHex);
/* Restore/create a wallet from a BIP39 mnemonic phrase, byte-compatible with
SilentDragonXLite: stores the 32-byte entropy, marks the chain mnemonic,
and derives the 64-byte BIP39 seed on demand. Only succeeds on a wallet
with no seed yet. Returns false on an invalid phrase or existing seed. */
bool SetHDSeedFromMnemonic(const std::string& phrase);
/* Return the wallet's 24-word BIP39 recovery phrase, if this is a mnemonic
wallet and the seed is available (unlocked). Returns false otherwise. */
bool GetMnemonicPhrase(std::string& phraseOut) const;
/* True if the HD seed was derived from a BIP39 mnemonic (stored as entropy). */
bool IsMnemonicSeed() const { return hdChain.fMnemonicSeed; }
/* Return the seed to feed into HD derivation. For mnemonic wallets this
expands the stored 32-byte entropy into the 64-byte BIP39 seed; for legacy
wallets it is the stored seed unchanged. Use this everywhere keys/OVKs are
derived so behaviour matches SilentDragonXLite. */
bool GetHDSeedForDerivation(HDSeed& seedOut) const;
/* Shared tail of the seed-install paths: stores `seed` and a fresh CHDChain
(mnemonic vs raw) with the given birthday. Caller must hold cs_wallet. */
bool InstallHDSeed(const HDSeed& seed, bool fMnemonic, int64_t nCreateTime);
/* Pre-derive `count` HD transparent keys (external chain) into the keystore,
stamped with creation time `nBirthday`, so a subsequent rescan can find
funds paid to them after a seed-only restore. */
void TopUpHDTransparentKeys(unsigned int count, int64_t nBirthday);
/* Set the HD chain model (chain child index counters) */
void SetHDChain(const CHDChain& chain, bool memonly);
const CHDChain& GetHDChain() const { return hdChain; }

View File

@@ -62,11 +62,24 @@ class CHDChain
{
public:
static const int VERSION_HD_BASE = 1;
static const int CURRENT_VERSION = VERSION_HD_BASE;
// Version 2 adds the transparent (secp256k1/BIP44) external-chain counter.
static const int VERSION_HD_TRANSPARENT = 2;
// Version 3 marks a seed derived from a BIP39 mnemonic: the stored HDSeed is
// the 32-byte BIP39 entropy, expanded to the 64-byte seed for derivation
// (matches SilentDragonXLite's on-disk convention).
static const int VERSION_HD_MNEMONIC = 3;
static const int CURRENT_VERSION = VERSION_HD_MNEMONIC;
int nVersion;
uint256 seedFp;
int64_t nCreateTime; // 0 means unknown
uint32_t saplingAccountCounter;
// Next index on the HD transparent external chain m/44'/coin'/0'/0/i.
// Only serialized/consulted when nVersion >= VERSION_HD_TRANSPARENT.
uint32_t transparentChildCounter;
// True when the stored HDSeed is BIP39 entropy that must be expanded to the
// 64-byte BIP39 seed before HD derivation. Only serialized when
// nVersion >= VERSION_HD_MNEMONIC (false for all pre-existing wallets).
bool fMnemonicSeed;
CHDChain() { SetNull(); }
@@ -79,6 +92,14 @@ public:
READWRITE(seedFp);
READWRITE(nCreateTime);
READWRITE(saplingAccountCounter);
// Version-gated so pre-existing v1 wallet.dat records still deserialize
// (they simply leave the newer fields at their SetNull defaults).
if (this->nVersion >= VERSION_HD_TRANSPARENT) {
READWRITE(transparentChildCounter);
}
if (this->nVersion >= VERSION_HD_MNEMONIC) {
READWRITE(fMnemonicSeed);
}
}
void SetNull()
@@ -87,6 +108,8 @@ public:
seedFp.SetNull();
nCreateTime = 0;
saplingAccountCounter = 0;
transparentChildCounter = 0;
fMnemonicSeed = false;
}
};

View File

@@ -13,6 +13,20 @@ BOOTSTRAP_FALLBACK_URL="https://bootstrap2.dragonx.is"
BOOTSTRAP_FILE="DRAGONX.zip"
CHAIN_NAME="DRAGONX"
# DragonX bootstrap signing public key (PEM, openssl-compatible).
# WHY: the .md5/.sha256 files are served from the same host as the archive, so they
# only detect transmission corruption — a compromised bootstrap server could publish a
# malicious archive with matching checksums. A detached signature verified against THIS
# embedded public key (shipped in the repo, not downloaded) closes that gap: a bad server
# cannot forge a signature without the maintainer's offline private key.
#
# ROLLOUT: until the maintainer embeds a real key here and publishes DRAGONX.zip.sig,
# this stays as the placeholder and signature enforcement is skipped (with a loud warning),
# so existing users are unaffected. Once a real key is pasted in, an unsigned/invalid
# bootstrap is refused (fail-closed). See util/sign-bootstrap.md for the signing procedure.
BOOTSTRAP_PUBKEY_PLACEHOLDER="REPLACE_WITH_DRAGONX_BOOTSTRAP_PUBLIC_KEY_PEM"
BOOTSTRAP_PUBKEY="$BOOTSTRAP_PUBKEY_PLACEHOLDER"
# Determine data directory
if [[ "$OSTYPE" == "darwin"* ]]; then
DATADIR="$HOME/Library/Application Support/Hush/$CHAIN_NAME"
@@ -139,6 +153,7 @@ download_from() {
local outfile="$DATADIR/$BOOTSTRAP_FILE"
local md5file="$DATADIR/${BOOTSTRAP_FILE}.md5"
local sha256file="$DATADIR/${BOOTSTRAP_FILE}.sha256"
local sigfile="$DATADIR/${BOOTSTRAP_FILE}.sig"
info "Downloading bootstrap from $base_url ..."
info "This may take a while depending on your connection speed."
@@ -149,14 +164,50 @@ download_from() {
info "Downloading checksums..."
download_file "$base_url/${BOOTSTRAP_FILE}.md5" "$md5file" || return 1
download_file "$base_url/${BOOTSTRAP_FILE}.sha256" "$sha256file" || return 1
# Detached signature is optional during rollout (non-fatal if absent); enforcement
# is decided in verify_signature() based on whether a real public key is embedded.
rm -f "$sigfile"
download_file "$base_url/${BOOTSTRAP_FILE}.sig" "$sigfile" || warn "No signature file at $base_url (${BOOTSTRAP_FILE}.sig)"
return 0
}
# Verify the detached signature of the archive against the embedded release public key.
# Fail-closed once a real key is configured; skip (with warning) while the placeholder is in place.
verify_signature() {
local archive="$1"
local sigfile="$2"
if [[ "$BOOTSTRAP_PUBKEY" == "$BOOTSTRAP_PUBKEY_PLACEHOLDER" ]]; then
warn "Bootstrap signature verification is not yet configured (no maintainer key embedded)."
warn "Relying on TLS + checksum integrity only. See util/sign-bootstrap.md."
return 0
fi
if ! command -v openssl &>/dev/null; then
error "openssl is required to verify the bootstrap signature but was not found. Install openssl and retry."
fi
if [[ ! -s "$sigfile" ]]; then
error "Bootstrap signature (${BOOTSTRAP_FILE}.sig) is missing; refusing to use an unsigned bootstrap."
fi
local pubfile
pubfile=$(mktemp)
printf '%s\n' "$BOOTSTRAP_PUBKEY" > "$pubfile"
if openssl dgst -sha256 -verify "$pubfile" -signature "$sigfile" "$archive" >&2; then
rm -f "$pubfile"
info "Bootstrap signature verified against embedded DragonX release key."
else
rm -f "$pubfile"
error "Bootstrap signature verification FAILED — the archive is NOT signed by the DragonX release key. Aborting; do not use this bootstrap."
fi
}
# Download the bootstrap and verify checksums
download_bootstrap() {
local outfile="$DATADIR/$BOOTSTRAP_FILE"
local md5file="$DATADIR/${BOOTSTRAP_FILE}.md5"
local sha256file="$DATADIR/${BOOTSTRAP_FILE}.sha256"
local sigfile="$DATADIR/${BOOTSTRAP_FILE}.sig"
if ! download_from "$BOOTSTRAP_BASE_URL"; then
warn "Primary download failed, trying fallback $BOOTSTRAP_FALLBACK_URL ..."
@@ -187,8 +238,11 @@ download_bootstrap() {
warn "sha256sum not found, skipping SHA256 verification."
fi
# Clean up checksum files
rm -f "$md5file" "$sha256file"
# Verify the cryptographic signature (fail-closed once a release key is embedded).
verify_signature "$outfile" "$sigfile"
# Clean up checksum + signature files
rm -f "$md5file" "$sha256file" "$sigfile"
echo "$outfile"
}

View File

@@ -121,8 +121,8 @@ then
fi
# Just show the useful info
eval "$MAKE" --version | head -n2
as --version | head -n1
eval "$MAKE" --version | head -n2 || true
as --version | head -n1 || true
as --version | tail -n1
ld -v
autoconf --version

56
util/sign-bootstrap.md Normal file
View File

@@ -0,0 +1,56 @@
# Signing the DragonX bootstrap archive
`util/bootstrap-dragonx.sh` verifies a detached signature of `DRAGONX.zip` against a
public key **embedded in the script** (`BOOTSTRAP_PUBKEY`). Because the key ships in the
repo/binary and is not downloaded from the bootstrap server, a compromised bootstrap host
cannot forge a valid signature — unlike the `.md5`/`.sha256` files, which are served from
the same host and only detect corruption.
Until a real key is embedded, `BOOTSTRAP_PUBKEY` is the placeholder and the script skips
signature enforcement (with a warning), so existing users are unaffected. Once a real key
is pasted in, an unsigned or invalid bootstrap is **refused**.
## One-time: create the signing keypair (offline)
Keep the private key OFFLINE (air-gapped if possible). Ed25519 or RSA-4096 both work with
the `openssl dgst -sha256 -verify` check the script uses; RSA-4096 maximizes compatibility:
```sh
# Private key — keep secret, never publish
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:4096 -out dragonx-bootstrap.key
# Public key — paste into bootstrap-dragonx.sh
openssl pkey -in dragonx-bootstrap.key -pubout -out dragonx-bootstrap.pub
cat dragonx-bootstrap.pub
```
Paste the full PEM (including the `-----BEGIN/END PUBLIC KEY-----` lines) into
`BOOTSTRAP_PUBKEY` in `util/bootstrap-dragonx.sh`, e.g.:
```sh
BOOTSTRAP_PUBKEY="$(cat <<'PEM'
-----BEGIN PUBLIC KEY-----
... base64 ...
-----END PUBLIC KEY-----
PEM
)"
```
## Each release: sign the archive and publish the signature
```sh
openssl dgst -sha256 -sign dragonx-bootstrap.key -out DRAGONX.zip.sig DRAGONX.zip
```
Upload `DRAGONX.zip.sig` next to `DRAGONX.zip` (and its `.md5`/`.sha256`) on every
bootstrap host (`bootstrap.dragonx.is`, `bootstrap2.dragonx.is`). Verify locally first:
```sh
openssl dgst -sha256 -verify dragonx-bootstrap.pub -signature DRAGONX.zip.sig DRAGONX.zip
# -> "Verified OK"
```
## Rotating the key
Embed the new public key in the script, sign future archives with the new private key, and
release a new client version. Old clients keep trusting the old key; coordinate the cutover
with a release so users upgrade before the old key is retired.