DanSandClaude Opus 5.5
7131b3673f
shim/ is an LD_PRELOAD guard around close/fclose/dup2/dup3: - before each close it resolves /proc/self/fd/N; - it reports, with a backtrace, a non-owner close of wallet.dat, database/log.* or blocks/*.dat, and every close that fails with EBADF (the second half of a double close); - FDGUARD_ABORT=1 aborts on the first hit; - run_tests.sh self-tests it (51 checks). repro/ runs a shimmed dragonxd on isolated regtest. Each run gets its own unprivileged network namespace (loopback only). The loads are: - rpc: a hammer with impatient/RST/half-open clients, work-queue bursts and long-polls; - p2p: TLS peers churned by disconnectnode, ban and kill, plus a raw TCP/TLS misbehaver; - tor: a fake control port; - mine: mining with blocknotify and autoshield. tools/tally.py groups the reports by closer. tools/scan_bdb.py finds BDB pages written into block files. With the wolfSSL CloseSocket macro in effect: - p2pmin (300 s) gives one EBADF double close per disconnect; - the full load (1200 s) gives 930 EBADF closes, 186 non-owner closes (37 of wallet/log/blk files) and a 632 KB db.log with BDB0060 PANIC; - RPC-only and Tor-only loads give none. After the fix, every count is 0 and db.log stays empty. The README has the commands and the numbers. Paths come from the environment (GOLDEN, BIN, RUNS, SHIM, ASMAP and the Sapling params). Run output goes to $RUNS, outside the tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
56 lines
3.3 KiB
Markdown
56 lines
3.3 KiB
Markdown
# fd-guard: catch stray close() of files a thread doesn't own
|
||
|
||
This is the harness that found the wolfSSL `CloseSocket` macro bug. With that bug, P2P disconnects closed their fd number a second time, after another thread had reused it, and BDB then wrote wallet pages into `blocks/*.dat`.
|
||
|
||
It has two parts:
|
||
|
||
- `shim/`: an `LD_PRELOAD` guard around `close`, `fclose`, `dup2` and `dup3`.
|
||
- Before each close it resolves `/proc/self/fd/N`.
|
||
- It logs a backtrace when a non-owner closes `wallet.dat`, `database/log.*` or `blocks/*.dat`, and for every close that fails with `EBADF` (the second half of a double close).
|
||
- `FDGUARD_ABORT=1` makes the first hit abort.
|
||
- `repro/`: isolated regtest runs that put a shimmed `dragonxd` under load:
|
||
- **rpc:** impatient and RST clients, half-open connections, work-queue bursts and long-polls;
|
||
- **p2p:** TLS peers churned by `disconnectnode`/ban/kill, plus misbehaving raw TCP/TLS clients;
|
||
- **tor:** a fake control port;
|
||
- **mining:** with `blocknotify` and autoshield.
|
||
|
||
Each run executes in its own unprivileged network namespace: loopback only, no route.
|
||
|
||
## Requirements
|
||
|
||
- Linux.
|
||
- `unshare -rn` (unprivileged user namespaces), `nsenter`, `ip`, python3, gcc, and `addr2line` for symbolizing.
|
||
- A `dragonxd`/`dragonx-cli` built with `-g`. Adding `-rdynamic` makes backtraces readable without `addr2line`.
|
||
- A regtest datadir with a grown wallet to copy from (`GOLDEN`). It is never modified.
|
||
|
||
## Running
|
||
|
||
```sh
|
||
qa/fd-guard/shim/build.sh # builds libfdguard.so
|
||
qa/fd-guard/shim/run_tests.sh # self-test: double close, foreign close of a guarded file
|
||
cd qa/fd-guard/repro
|
||
export GOLDEN=/path/to/regtest/datadir BIN=/path/to/build/src # RUNS defaults to ~/fd-guard-runs
|
||
LOADS=p2pmin,mine DISC_INT=5 GUARD=all ./go.sh p2pmin 300 # 1 peer, disconnectnode every 5 s
|
||
LOADS=rpc,p2p,tor,mine WQ=64 RT=8 GUARD=all ./go.sh full 1200 # everything, production-like work queue
|
||
python3 tools/tally.py "$RUNS/<name>/fdguard.log" # who closed what
|
||
python3 ../shim/symbolize.py --summary "$RUNS/<name>/fdguard.log"
|
||
python3 tools/scan_bdb.py "$RUNS/<name>"/target/regtest/blocks/*.dat # BDB pages inside block files
|
||
```
|
||
|
||
Each run writes `run.txt` (full log plus summary), `fdguard.log` and the target's `debug.log` and `db.log` under `$RUNS/<name>/`.
|
||
|
||
## What to expect
|
||
|
||
Measured on a 2041-note regtest wallet, before and after the fix (`net: stop wolfSSL's CloseSocket macro ...`), under the same loads:
|
||
|
||
| Run | Binary | EBADF closes | Non-owner closes (wallet/log/blk) | libevent `Epoll` failures | `db.log` |
|
||
|---|---|---|---|---|---|
|
||
| `p2pmin`, 300 s | before | 51 (≈ 1 per disconnect) | 1 | 0 | empty |
|
||
| | after | 0 | 0 | 0 | empty |
|
||
| `full`, 1200 s, `WQ=64 RT=8` | before | 930 | 186 (37) | 111 | 632 KB: 7500 × `BDB0060 PANIC`, `write failed for page N` |
|
||
| | after | 0 | 0 | 0 | empty |
|
||
|
||
- RPC-only and Tor-only loads give 0 reports, even before the fix: P2P disconnects are the only origin.
|
||
- `scan_bdb.py` found no BDB pages in block files in any run. The landing of a page in a block file needs the reused number to be a writable block file at the moment BDB writes. Production did hit that (blk00011/blk00020), but it is rare.
|
||
- `SSL_write ... closing connection` lines are expected under `tcpchurn`. They are how the load triggers disconnects, not a symptom.
|