IBD/sync speedups: parallel RandomX pre-verify, adaptive dbcache, P2P download fixes

- Parallel RandomX PoW pre-verification pool (CCheckQueue) run ahead of the serial
  connect; consensus-neutral (inline CheckRandomXSolution fallback still verifies
  anything not pre-verified). New -randomxverifythreads (default = -par).
- Adaptive dbcache: default sizes the UTXO/coins cache to most of RAM and shrinks
  under memory pressure, always leaving a reserve free; -dbcache pins a fixed value.
- P2P block download: bounded socket recv-drain loop (tlsmanager); frontier-block
  reassignment to break head-of-line stalls (-blockreassigntimeout); ProcessGetData
  serves a bounded batch of blocks per pass instead of one (fixes the serve-side
  one-block-per-tick throttle that caps download network-wide).
- assumeutxo: dumptxoutset RPC + LoadSnapshot machinery + AssumeutxoData chainparams.
- Signed bootstrap verification (util/bootstrap-dragonx.sh, util/sign-bootstrap.md).
- gtest: RandomX pre-verify consensus-equivalence test + UTXO-snapshot round-trip;
  revived the gtest harness (Makefile.am include fix, Makefile.gtest.include).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-19 12:30:10 -05:00
parent 2b011d6ee2
commit 1673cfb6dc
18 changed files with 1599 additions and 154 deletions

View File

@@ -13,6 +13,20 @@ BOOTSTRAP_FALLBACK_URL="https://bootstrap2.dragonx.is"
BOOTSTRAP_FILE="DRAGONX.zip"
CHAIN_NAME="DRAGONX"
# DragonX bootstrap signing public key (PEM, openssl-compatible).
# WHY: the .md5/.sha256 files are served from the same host as the archive, so they
# only detect transmission corruption — a compromised bootstrap server could publish a
# malicious archive with matching checksums. A detached signature verified against THIS
# embedded public key (shipped in the repo, not downloaded) closes that gap: a bad server
# cannot forge a signature without the maintainer's offline private key.
#
# ROLLOUT: until the maintainer embeds a real key here and publishes DRAGONX.zip.sig,
# this stays as the placeholder and signature enforcement is skipped (with a loud warning),
# so existing users are unaffected. Once a real key is pasted in, an unsigned/invalid
# bootstrap is refused (fail-closed). See util/sign-bootstrap.md for the signing procedure.
BOOTSTRAP_PUBKEY_PLACEHOLDER="REPLACE_WITH_DRAGONX_BOOTSTRAP_PUBLIC_KEY_PEM"
BOOTSTRAP_PUBKEY="$BOOTSTRAP_PUBKEY_PLACEHOLDER"
# Determine data directory
if [[ "$OSTYPE" == "darwin"* ]]; then
DATADIR="$HOME/Library/Application Support/Hush/$CHAIN_NAME"
@@ -139,6 +153,7 @@ download_from() {
local outfile="$DATADIR/$BOOTSTRAP_FILE"
local md5file="$DATADIR/${BOOTSTRAP_FILE}.md5"
local sha256file="$DATADIR/${BOOTSTRAP_FILE}.sha256"
local sigfile="$DATADIR/${BOOTSTRAP_FILE}.sig"
info "Downloading bootstrap from $base_url ..."
info "This may take a while depending on your connection speed."
@@ -149,14 +164,50 @@ download_from() {
info "Downloading checksums..."
download_file "$base_url/${BOOTSTRAP_FILE}.md5" "$md5file" || return 1
download_file "$base_url/${BOOTSTRAP_FILE}.sha256" "$sha256file" || return 1
# Detached signature is optional during rollout (non-fatal if absent); enforcement
# is decided in verify_signature() based on whether a real public key is embedded.
rm -f "$sigfile"
download_file "$base_url/${BOOTSTRAP_FILE}.sig" "$sigfile" || warn "No signature file at $base_url (${BOOTSTRAP_FILE}.sig)"
return 0
}
# Verify the detached signature of the archive against the embedded release public key.
# Fail-closed once a real key is configured; skip (with warning) while the placeholder is in place.
verify_signature() {
local archive="$1"
local sigfile="$2"
if [[ "$BOOTSTRAP_PUBKEY" == "$BOOTSTRAP_PUBKEY_PLACEHOLDER" ]]; then
warn "Bootstrap signature verification is not yet configured (no maintainer key embedded)."
warn "Relying on TLS + checksum integrity only. See util/sign-bootstrap.md."
return 0
fi
if ! command -v openssl &>/dev/null; then
error "openssl is required to verify the bootstrap signature but was not found. Install openssl and retry."
fi
if [[ ! -s "$sigfile" ]]; then
error "Bootstrap signature (${BOOTSTRAP_FILE}.sig) is missing; refusing to use an unsigned bootstrap."
fi
local pubfile
pubfile=$(mktemp)
printf '%s\n' "$BOOTSTRAP_PUBKEY" > "$pubfile"
if openssl dgst -sha256 -verify "$pubfile" -signature "$sigfile" "$archive" >&2; then
rm -f "$pubfile"
info "Bootstrap signature verified against embedded DragonX release key."
else
rm -f "$pubfile"
error "Bootstrap signature verification FAILED — the archive is NOT signed by the DragonX release key. Aborting; do not use this bootstrap."
fi
}
# Download the bootstrap and verify checksums
download_bootstrap() {
local outfile="$DATADIR/$BOOTSTRAP_FILE"
local md5file="$DATADIR/${BOOTSTRAP_FILE}.md5"
local sha256file="$DATADIR/${BOOTSTRAP_FILE}.sha256"
local sigfile="$DATADIR/${BOOTSTRAP_FILE}.sig"
if ! download_from "$BOOTSTRAP_BASE_URL"; then
warn "Primary download failed, trying fallback $BOOTSTRAP_FALLBACK_URL ..."
@@ -187,8 +238,11 @@ download_bootstrap() {
warn "sha256sum not found, skipping SHA256 verification."
fi
# Clean up checksum files
rm -f "$md5file" "$sha256file"
# Verify the cryptographic signature (fail-closed once a release key is embedded).
verify_signature "$outfile" "$sigfile"
# Clean up checksum + signature files
rm -f "$md5file" "$sha256file" "$sigfile"
echo "$outfile"
}

56
util/sign-bootstrap.md Normal file
View File

@@ -0,0 +1,56 @@
# Signing the DragonX bootstrap archive
`util/bootstrap-dragonx.sh` verifies a detached signature of `DRAGONX.zip` against a
public key **embedded in the script** (`BOOTSTRAP_PUBKEY`). Because the key ships in the
repo/binary and is not downloaded from the bootstrap server, a compromised bootstrap host
cannot forge a valid signature — unlike the `.md5`/`.sha256` files, which are served from
the same host and only detect corruption.
Until a real key is embedded, `BOOTSTRAP_PUBKEY` is the placeholder and the script skips
signature enforcement (with a warning), so existing users are unaffected. Once a real key
is pasted in, an unsigned or invalid bootstrap is **refused**.
## One-time: create the signing keypair (offline)
Keep the private key OFFLINE (air-gapped if possible). Ed25519 or RSA-4096 both work with
the `openssl dgst -sha256 -verify` check the script uses; RSA-4096 maximizes compatibility:
```sh
# Private key — keep secret, never publish
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:4096 -out dragonx-bootstrap.key
# Public key — paste into bootstrap-dragonx.sh
openssl pkey -in dragonx-bootstrap.key -pubout -out dragonx-bootstrap.pub
cat dragonx-bootstrap.pub
```
Paste the full PEM (including the `-----BEGIN/END PUBLIC KEY-----` lines) into
`BOOTSTRAP_PUBKEY` in `util/bootstrap-dragonx.sh`, e.g.:
```sh
BOOTSTRAP_PUBKEY="$(cat <<'PEM'
-----BEGIN PUBLIC KEY-----
... base64 ...
-----END PUBLIC KEY-----
PEM
)"
```
## Each release: sign the archive and publish the signature
```sh
openssl dgst -sha256 -sign dragonx-bootstrap.key -out DRAGONX.zip.sig DRAGONX.zip
```
Upload `DRAGONX.zip.sig` next to `DRAGONX.zip` (and its `.md5`/`.sha256`) on every
bootstrap host (`bootstrap.dragonx.is`, `bootstrap2.dragonx.is`). Verify locally first:
```sh
openssl dgst -sha256 -verify dragonx-bootstrap.pub -signature DRAGONX.zip.sig DRAGONX.zip
# -> "Verified OK"
```
## Rotating the key
Embed the new public key in the script, sign future archives with the new private key, and
release a new client version. Old clients keep trusting the old key; coordinate the cutover
with a release so users upgrade before the old key is retired.