Binaries built on Ubuntu 22.04 require GLIBC_2.34 and GLIBCXX_3.4.30 and will not
start on Ubuntu 20.04 -- which is four of our five seeds, and an unknown share of
users. The binary the fleet actually runs today needs only GLIBC_2.29, so it was
built somewhere older; seed 176 has since been upgraded to 22.04 and now produces
binaries it is the only seed able to run.
--linux-compat and Dockerfile.compat already solved this (6d56ad854) but were
undocumented outside the build script and pinned to one base image. Parameterise
the base via ARG BASE_IMAGE (default unchanged, so --linux-compat behaves exactly
as before) and document the whole path.
doc/build-containers.md is written to be executed by a person or an agent starting
from a machine with nothing installed: why the glibc direction matters, with the
measured numbers; what already exists in the repo so nobody writes a second build
system; prerequisites and honest cost (~15GB, 4GB RAM, 1-2h per base because
depends/ builds boost, BDB, wolfssl and rust from source); one-target and
multi-target recipes; which base to pick and why 20.04 is the recommended floor
while 18.04 needs verifying (GCC 7 against -std=c++17); a mandatory verification
step with the exact objdump/readelf commands and the expected ceilings; and the
traps.
The traps are the part worth having written down: ETXTBSY when installing over a
running daemon (cp fails even after the process exits -- stage and rename, then
sha256-verify before starting); never touching configure.ac in a configured tree,
because the mtime alone triggers a reconfigure that dies on libdb_cxx; never
blind-touching a path that may not exist, which silently creates stray empty files;
RandomX needing ARCH=default or it emits AVX-512 that SIGILLs the fleet; build-win.sh
silently discarding every argument; and macOS being uncontainerisable because
depends/ has no darwin cross path at all.
Also records that full static linking is NOT the answer here: the daemon resolves
node1..node5.dragonx.is via getaddrinfo, and static glibc pushes that through NSS,
which dlopens libnss_dns at runtime and reintroduces the dependency it was meant to
remove.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
10 KiB
Building release binaries in containers
Release binaries must be built in a container based on an old Linux distribution. This document is written to be executed, by a person or an agent, on a machine that has nothing set up yet.
1. Why this exists
glibc compatibility runs one way only. A binary linked against glibc 2.35 demands symbol versions that glibc 2.31 does not have, and refuses to start. A binary linked against glibc 2.29 runs on 2.29, 2.31 and 2.35 alike.
Measured on the actual fleet, 2026-08-25:
| binary | max GLIBC required | runs on |
|---|---|---|
what all four 20.04 seeds run today (v1.0.3-d159e7208) |
GLIBC_2.29 |
18.04, 20.04, 22.04 |
| anything built on seed 176 today (Ubuntu 22.04) | GLIBC_2.34 |
22.04 only |
The second binary will not start on four of our own five seeds. The loader reports:
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found
/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.30' not found
Nothing new is being called. glibc 2.34 merged libpthread and libdl into libc and
re-versioned every pthread_*, dlsym and dladdr symbol; 2.33 replaced the old
__xstat inlines with real stat/fstat/lstat64. All of those functions exist in
2.31 under older tags. Building against older headers is the entire fix.
Do not try to solve this with full static linking. The daemon calls getaddrinfo,
gethostbyname and getnameinfo, and it must resolve node1..node5.dragonx.is, which
are hard-coded and injected into -addnode on every node. Under a fully static glibc
binary those go through NSS, which dlopens libnss_dns.so.2 at run time — it either
fails or silently requires the target to have the same glibc you linked against, which
defeats the purpose.
2. What already exists in this repo
Do not write a new build system. Two pieces are already here:
Dockerfile.compat— an Ubuntu base image that installs the toolchain, copies the tree, deletes any host-builtdepends/and object files, runs./util/build.sh, and strips the three binaries../build.sh --linux-compat— builds that image, creates a throwaway container, copiesdragonxd,dragonx-clianddragonx-txout intorelease/dragonx-<version>-linux-amd64-ubuntu2004/, addsbootstrap-dragonx.sh,asmap.datand the two sapling params, fixes ownership, and prints the binary's maximum required GLIBC version.
The base image is parameterised via ARG BASE_IMAGE (default ubuntu:20.04), so the
same Dockerfile can target several glibc floors.
3. Prerequisites
Docker (the scripted path uses docker specifically; podman works for the manual path
if you alias or substitute it).
sudo apt-get update
sudo apt-get install -y docker.io git
sudo usermod -aG docker "$USER" # then log out and back in, or every command needs sudo
Budget, measured on a 4-core box:
| resource | needs |
|---|---|
| disk | ~15 GB free (the depends/ tree alone is ~1.6 GB per target, plus image layers) |
| RAM | 4 GB minimum, 8 GB comfortable — the link step is the peak |
| time | 1–2 hours per base image on first build. depends/ builds boost, BDB, wolfssl, libevent, libsodium, libcurl and rust from source. Later builds reuse Docker layer cache unless the tree changed. |
depends/ downloads and builds its own rust toolchain, so the host's rust (or absence
of it) is irrelevant.
4. Build one target
git clone https://git.dragonx.is/DragonX/dragonx
cd dragonx
git checkout <the tag or branch you are releasing>
./build.sh --linux-compat
Output lands in release/dragonx-<version>-linux-amd64-ubuntu2004/ and the script
prints the max GLIBC at the end. <version> is read from configure.ac, not
hardcoded, so it always matches what the binaries report.
5. Build several targets
for BASE in ubuntu:18.04 ubuntu:20.04 ubuntu:22.04; do
TAG="dragonx-compat-${BASE#ubuntu:}"
TAG="${TAG//./}"
docker build --build-arg "BASE_IMAGE=$BASE" -f Dockerfile.compat -t "$TAG" .
OUT="release/dragonx-$(grep -oP 'define\(_CLIENT_VERSION_MAJOR, \K[0-9]+' configure.ac).$(grep -oP 'define\(_CLIENT_VERSION_MINOR, \K[0-9]+' configure.ac).$(grep -oP 'define\(_CLIENT_VERSION_REVISION, \K[0-9]+' configure.ac)-linux-amd64-${BASE#ubuntu:}"
mkdir -p "$OUT"
CID=$(docker create "$TAG")
for b in dragonxd dragonx-cli dragonx-tx; do docker cp "$CID:/build/src/$b" "$OUT/$b"; done
docker rm "$CID" >/dev/null
cp util/bootstrap-dragonx.sh contrib/asmap/asmap.dat sapling-output.params sapling-spend.params "$OUT/" 2>/dev/null || true
done
Which base to choose
| base | glibc it provides | default GCC | verdict |
|---|---|---|---|
ubuntu:18.04 |
2.27 | 7 | Verify before relying on it. The tree is built with -std=c++17; GCC 7's C++17 support is incomplete and its cmake (3.10) may be too old for RandomX. Attempt only if you need to reach 18.04 users, and treat a successful build as the proof. |
ubuntu:20.04 |
2.31 | 9 | Recommended floor. GCC 9 covers C++17 fully. Evidence it works: the binary the fleet runs today requires only GLIBC_2.29, i.e. the code touches nothing newer, so a 20.04 build reaches 18.04 machines anyway. |
ubuntu:22.04 |
2.35 | 11 | Do not ship this. It is what we already have and what excludes four of our own seeds. Useful only for development. |
Ubuntu 20.04 left standard support in April 2025, which is precisely why it belongs in a container on a patched host rather than on a build box someone has to maintain.
6. Verify — this step is not optional
A build that silently targets the wrong glibc looks completely normal until a user reports that nothing starts.
BIN=release/dragonx-<version>-linux-amd64-ubuntu2004/dragonxd
# The ceiling. Must be <= the glibc of the OLDEST system you intend to support.
objdump -p "$BIN" | grep -oE 'GLIBC_2\.[0-9]+' | sort -t. -k2 -n | tail -1
objdump -p "$BIN" | grep -oE 'GLIBCXX_3\.4\.[0-9]+' | sort -t. -k3 -n | tail -1
# If the ceiling is too high, this names the symbols responsible.
readelf --dyn-syms --wide "$BIN" | grep -E '@GLIBC_2\.(3[2-9])'
Expected for a 20.04 build: GLIBC_2.29 or lower, GLIBCXX_3.4.26 or lower.
Then actually run it somewhere old. A ceiling check proves the loader will resolve the
symbols; it does not prove the binary works. ./dragonxd --version on a real 20.04 box
is a ten-second confirmation.
7. Traps
Each of these has cost real time.
ETXTBSY when installing over a running daemon. cp onto the binary fails with
"Text file busy" even after the process has exited — pgrep returning nothing is not
sufficient, the kernel still holds the text mapping. Stage into the same directory and
mv (rename is not blocked), allow ~10 s to settle, and sha256-verify the installed
file before starting it. A failed copy that goes unnoticed leaves the old binary
running and looks like a successful deploy.
Never touch configure.ac in a configured tree. Even cp-ing back a byte-identical
copy updates its mtime, which makes make regenerate aclocal.m4 and configure and
then re-run configure, which fails with libdb_cxx headers missing because the
depends prefix is not on the command line. If it happens: confirm
git diff --quiet HEAD -- configure.ac, then restore mtime order oldest-to-newest with
one-second gaps — configure.ac/Makefile.am, then aclocal.m4, then
configure/Makefile.in, then config.status, then Makefile. Inside a container
this cannot happen, which is one more reason to build there.
Never blind-touch a path that might not exist. touch src/config/hush-config.h
silently creates an empty stray file; the real header is bitcoin-config.h. Check
git status after any timestamp surgery.
RandomX must be built with ARCH=default. util/build.sh already passes it and the
comment there explains why: ARCH=native tunes to the build machine, and a build on an
AVX-512 host emitted 746 zmm instructions into librandomx.a, which SIGILLs on the
entire fleet. If you ever invoke cmake by hand, pass -DARCH=default.
Strip before distributing. Unstripped is ~220 MB, stripped ~16 MB. Dockerfile.compat
already strips inside the container.
util/build-win.sh discards every argument. There is no "$@" handling in it, so
-j$(nproc) and --disable-tests are dropped on the floor and the Windows build is
single-threaded. Expect it to be far slower than you planned.
Windows also needs -Wa,-mbig-obj and -DARCH=default. Both are in
util/build-win.sh today. The mingw flag was missing from dev for a month; without it
the cross-compile fails at link because boost-heavy translation units exceed the
PE/COFF section limit. Do not lose it on a re-branch.
macOS cannot be containerised. util/build-mac.sh is a native-Mac script, and there
is no darwin cross-compile path in depends/ at all: hosts/darwin.mk wants
native_cctools, which has no package definition, and there is no SDK in the tree. It
also hardcodes an Intel Homebrew GCC path, so it produces x86_64 only — no arm64, no
universal binary. macOS needs a real Mac.
The contrib/gitian-descriptors/ files are not a build path. They are unmodified
upstream Bitcoin files (name: "bitcoin-win-0.11", suite trusty) with zero DragonX
content. Ignore them.
8. Handoff checklist
- Docker installed, user in the
dockergroup, ~15 GB free - Correct tag or branch checked out, tree clean (
git status) - Version in
configure.acis the one you intend to release ./build.sh --linux-compatcompletes- GLIBC ceiling is 2.31 or lower (2.29 expected)
- GLIBCXX ceiling is 3.4.28 or lower (3.4.26 expected)
dragonxd --versionruns on a real machine of the oldest supported distro- Binaries stripped,
release/contains the bootstrap script,asmap.datand both sapling params - sha256 recorded for each artifact
One more thing that is not a build step but belongs in the same conversation: the
in-app daemon updater refuses any release without a detached signature
(kDaemonRequireSignature = true). Publishing checksums alone means no existing user
can update in place.