4b5a5e5a67
/home/dev/lightwalletd is simultaneously a git checkout and the runtime directory, and monitor_lwd.sh -- the primary's live supervisor -- was a tracked file inside it. So a routine `git reset --hard` during an unrelated cherry-pick reverted the running supervisor to an older committed revision. That happened on 2026-08-26. Nothing noticed, because the running monitor was executing an already-deleted inode: the file on disk was broken while the live process was fine. It was recovered from /proc/<pid>/fd/255. What the reverted copy would have reintroduced, had it ever restarted: the loss of `-cache-size 5000`, so every relaunch warms the block cache from tip-400000 instead of tip-5000; and `wait "$LWD_PID" || true; EXIT_CODE=$?`, which reads the exit status of `|| true` and is therefore always 0, so the monitor logs "exited cleanly. Not restarting." and breaks its loop on every exit including crashes -- the bug behind an 11h48m outage on 2026-08-21. Move it to deploy/, from where it is copied to /home/dev/ and run. The runtime directory now holds only the binary and its logs, so no checkout, reset, rebase or branch switch can reach a running supervisor. The script no longer derives its paths from its own location: SCRIPT_DIR became an explicit LWD_DIR, because the script and the runtime directory are deliberately no longer the same place. lwd_watchdog.sh launches "$MONITOR" by absolute path and refuses to run if it is missing. Verified: run from /tmp, the relocated monitor resolves its binary and log through LWD_DIR rather than its own directory, and launches with -cache-size 5000 intact. The running monitor was not restarted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FU87LdsJZiZkfq1eXubpeo
3.9 KiB
Executable File
3.9 KiB
Executable File