IBD/sync speedups: parallel RandomX pre-verify, adaptive dbcache, P2P download fixes
- Parallel RandomX PoW pre-verification pool (CCheckQueue) run ahead of the serial connect; consensus-neutral (inline CheckRandomXSolution fallback still verifies anything not pre-verified). New -randomxverifythreads (default = -par). - Adaptive dbcache: default sizes the UTXO/coins cache to most of RAM and shrinks under memory pressure, always leaving a reserve free; -dbcache pins a fixed value. - P2P block download: bounded socket recv-drain loop (tlsmanager); frontier-block reassignment to break head-of-line stalls (-blockreassigntimeout); ProcessGetData serves a bounded batch of blocks per pass instead of one (fixes the serve-side one-block-per-tick throttle that caps download network-wide). - assumeutxo: dumptxoutset RPC + LoadSnapshot machinery + AssumeutxoData chainparams. - Signed bootstrap verification (util/bootstrap-dragonx.sh, util/sign-bootstrap.md). - gtest: RandomX pre-verify consensus-equivalence test + UTXO-snapshot round-trip; revived the gtest harness (Makefile.am include fix, Makefile.gtest.include). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -13,6 +13,20 @@ BOOTSTRAP_FALLBACK_URL="https://bootstrap2.dragonx.is"
|
||||
BOOTSTRAP_FILE="DRAGONX.zip"
|
||||
CHAIN_NAME="DRAGONX"
|
||||
|
||||
# DragonX bootstrap signing public key (PEM, openssl-compatible).
|
||||
# WHY: the .md5/.sha256 files are served from the same host as the archive, so they
|
||||
# only detect transmission corruption — a compromised bootstrap server could publish a
|
||||
# malicious archive with matching checksums. A detached signature verified against THIS
|
||||
# embedded public key (shipped in the repo, not downloaded) closes that gap: a bad server
|
||||
# cannot forge a signature without the maintainer's offline private key.
|
||||
#
|
||||
# ROLLOUT: until the maintainer embeds a real key here and publishes DRAGONX.zip.sig,
|
||||
# this stays as the placeholder and signature enforcement is skipped (with a loud warning),
|
||||
# so existing users are unaffected. Once a real key is pasted in, an unsigned/invalid
|
||||
# bootstrap is refused (fail-closed). See util/sign-bootstrap.md for the signing procedure.
|
||||
BOOTSTRAP_PUBKEY_PLACEHOLDER="REPLACE_WITH_DRAGONX_BOOTSTRAP_PUBLIC_KEY_PEM"
|
||||
BOOTSTRAP_PUBKEY="$BOOTSTRAP_PUBKEY_PLACEHOLDER"
|
||||
|
||||
# Determine data directory
|
||||
if [[ "$OSTYPE" == "darwin"* ]]; then
|
||||
DATADIR="$HOME/Library/Application Support/Hush/$CHAIN_NAME"
|
||||
@@ -139,6 +153,7 @@ download_from() {
|
||||
local outfile="$DATADIR/$BOOTSTRAP_FILE"
|
||||
local md5file="$DATADIR/${BOOTSTRAP_FILE}.md5"
|
||||
local sha256file="$DATADIR/${BOOTSTRAP_FILE}.sha256"
|
||||
local sigfile="$DATADIR/${BOOTSTRAP_FILE}.sig"
|
||||
|
||||
info "Downloading bootstrap from $base_url ..."
|
||||
info "This may take a while depending on your connection speed."
|
||||
@@ -149,14 +164,50 @@ download_from() {
|
||||
info "Downloading checksums..."
|
||||
download_file "$base_url/${BOOTSTRAP_FILE}.md5" "$md5file" || return 1
|
||||
download_file "$base_url/${BOOTSTRAP_FILE}.sha256" "$sha256file" || return 1
|
||||
# Detached signature is optional during rollout (non-fatal if absent); enforcement
|
||||
# is decided in verify_signature() based on whether a real public key is embedded.
|
||||
rm -f "$sigfile"
|
||||
download_file "$base_url/${BOOTSTRAP_FILE}.sig" "$sigfile" || warn "No signature file at $base_url (${BOOTSTRAP_FILE}.sig)"
|
||||
return 0
|
||||
}
|
||||
|
||||
# Verify the detached signature of the archive against the embedded release public key.
|
||||
# Fail-closed once a real key is configured; skip (with warning) while the placeholder is in place.
|
||||
verify_signature() {
|
||||
local archive="$1"
|
||||
local sigfile="$2"
|
||||
|
||||
if [[ "$BOOTSTRAP_PUBKEY" == "$BOOTSTRAP_PUBKEY_PLACEHOLDER" ]]; then
|
||||
warn "Bootstrap signature verification is not yet configured (no maintainer key embedded)."
|
||||
warn "Relying on TLS + checksum integrity only. See util/sign-bootstrap.md."
|
||||
return 0
|
||||
fi
|
||||
|
||||
if ! command -v openssl &>/dev/null; then
|
||||
error "openssl is required to verify the bootstrap signature but was not found. Install openssl and retry."
|
||||
fi
|
||||
if [[ ! -s "$sigfile" ]]; then
|
||||
error "Bootstrap signature (${BOOTSTRAP_FILE}.sig) is missing; refusing to use an unsigned bootstrap."
|
||||
fi
|
||||
|
||||
local pubfile
|
||||
pubfile=$(mktemp)
|
||||
printf '%s\n' "$BOOTSTRAP_PUBKEY" > "$pubfile"
|
||||
if openssl dgst -sha256 -verify "$pubfile" -signature "$sigfile" "$archive" >&2; then
|
||||
rm -f "$pubfile"
|
||||
info "Bootstrap signature verified against embedded DragonX release key."
|
||||
else
|
||||
rm -f "$pubfile"
|
||||
error "Bootstrap signature verification FAILED — the archive is NOT signed by the DragonX release key. Aborting; do not use this bootstrap."
|
||||
fi
|
||||
}
|
||||
|
||||
# Download the bootstrap and verify checksums
|
||||
download_bootstrap() {
|
||||
local outfile="$DATADIR/$BOOTSTRAP_FILE"
|
||||
local md5file="$DATADIR/${BOOTSTRAP_FILE}.md5"
|
||||
local sha256file="$DATADIR/${BOOTSTRAP_FILE}.sha256"
|
||||
local sigfile="$DATADIR/${BOOTSTRAP_FILE}.sig"
|
||||
|
||||
if ! download_from "$BOOTSTRAP_BASE_URL"; then
|
||||
warn "Primary download failed, trying fallback $BOOTSTRAP_FALLBACK_URL ..."
|
||||
@@ -187,8 +238,11 @@ download_bootstrap() {
|
||||
warn "sha256sum not found, skipping SHA256 verification."
|
||||
fi
|
||||
|
||||
# Clean up checksum files
|
||||
rm -f "$md5file" "$sha256file"
|
||||
# Verify the cryptographic signature (fail-closed once a release key is embedded).
|
||||
verify_signature "$outfile" "$sigfile"
|
||||
|
||||
# Clean up checksum + signature files
|
||||
rm -f "$md5file" "$sha256file" "$sigfile"
|
||||
|
||||
echo "$outfile"
|
||||
}
|
||||
|
||||
56
util/sign-bootstrap.md
Normal file
56
util/sign-bootstrap.md
Normal file
@@ -0,0 +1,56 @@
|
||||
# Signing the DragonX bootstrap archive
|
||||
|
||||
`util/bootstrap-dragonx.sh` verifies a detached signature of `DRAGONX.zip` against a
|
||||
public key **embedded in the script** (`BOOTSTRAP_PUBKEY`). Because the key ships in the
|
||||
repo/binary and is not downloaded from the bootstrap server, a compromised bootstrap host
|
||||
cannot forge a valid signature — unlike the `.md5`/`.sha256` files, which are served from
|
||||
the same host and only detect corruption.
|
||||
|
||||
Until a real key is embedded, `BOOTSTRAP_PUBKEY` is the placeholder and the script skips
|
||||
signature enforcement (with a warning), so existing users are unaffected. Once a real key
|
||||
is pasted in, an unsigned or invalid bootstrap is **refused**.
|
||||
|
||||
## One-time: create the signing keypair (offline)
|
||||
|
||||
Keep the private key OFFLINE (air-gapped if possible). Ed25519 or RSA-4096 both work with
|
||||
the `openssl dgst -sha256 -verify` check the script uses; RSA-4096 maximizes compatibility:
|
||||
|
||||
```sh
|
||||
# Private key — keep secret, never publish
|
||||
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:4096 -out dragonx-bootstrap.key
|
||||
# Public key — paste into bootstrap-dragonx.sh
|
||||
openssl pkey -in dragonx-bootstrap.key -pubout -out dragonx-bootstrap.pub
|
||||
cat dragonx-bootstrap.pub
|
||||
```
|
||||
|
||||
Paste the full PEM (including the `-----BEGIN/END PUBLIC KEY-----` lines) into
|
||||
`BOOTSTRAP_PUBKEY` in `util/bootstrap-dragonx.sh`, e.g.:
|
||||
|
||||
```sh
|
||||
BOOTSTRAP_PUBKEY="$(cat <<'PEM'
|
||||
-----BEGIN PUBLIC KEY-----
|
||||
... base64 ...
|
||||
-----END PUBLIC KEY-----
|
||||
PEM
|
||||
)"
|
||||
```
|
||||
|
||||
## Each release: sign the archive and publish the signature
|
||||
|
||||
```sh
|
||||
openssl dgst -sha256 -sign dragonx-bootstrap.key -out DRAGONX.zip.sig DRAGONX.zip
|
||||
```
|
||||
|
||||
Upload `DRAGONX.zip.sig` next to `DRAGONX.zip` (and its `.md5`/`.sha256`) on every
|
||||
bootstrap host (`bootstrap.dragonx.is`, `bootstrap2.dragonx.is`). Verify locally first:
|
||||
|
||||
```sh
|
||||
openssl dgst -sha256 -verify dragonx-bootstrap.pub -signature DRAGONX.zip.sig DRAGONX.zip
|
||||
# -> "Verified OK"
|
||||
```
|
||||
|
||||
## Rotating the key
|
||||
|
||||
Embed the new public key in the script, sign future archives with the new private key, and
|
||||
release a new client version. Old clients keep trusting the old key; coordinate the cutover
|
||||
with a release so users upgrade before the old key is retired.
|
||||
Reference in New Issue
Block a user