pastilhasandClaude Opus 5
16da10150b
the 2026.08.25.01 tag was pushed and nothing picked it up. commander 40's loop-pid held 291464, dead; its last state file was written 2026-08-24 03:50, and the machine came up at 12:40 the same day. no cron, and deploy-watch.sh was not running either. commander 54 says "watching" only because it was armed after that boot. the watcher is a bare background process — not pm2, not systemd, parented to whichever agent turn armed it. so a reboot ends it silently, and the only signal is the absence of a word on a card. that breaks the one promise that makes a commander worth having, quietly. ranked by what it buys: SAY IT (a row that claims armed with a dead pid is already detectable — router.ts computes watcherArmed and watcherRunning separately for exactly this reason, so notify once and nothing else on the list matters), then re-arm on boot, then stop it being a bare process at all. interim: re-arm by hand after a reboot, and trust watcherRunning over yesterday. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Officer-Session: 656f5c85-e49a-4f1f-b9f3-042a04b4d043
3.3 KiB
id, title, theme, status, priority, created, date_source, order
| id | title | theme | status | priority | created | date_source | order |
|---|---|---|---|---|---|---|---|
| agent-coordination/a-commander-s-watcher-does-not-survive-a-reboot | A commander's watcher does not survive a reboot, and nothing says so | agent-coordination | open | urgent | 2026-08-25 | stated | 3 |
A commander's watcher loop is a bare background process, so a reboot ends it and nothing brings it
back or mentions it. Found 2026-08-25 while trying to deploy 2026.08.25.01: the tag was pushed and
nothing picked it up.
What was verified
commanders/40/state/loop-pidheld 291464, dead. Its last state file was written 2026-08-24 03:50 — the2026.08.24.02rollout.uptime -ssays the machine came up 2026-08-24 12:40:11.commanders/54(Voice Memos) was armed after that boot and its pid is alive, which is why one card reads "watching" and the other does not.- Nothing else was watching either: no cron entry, and
data/deploy-watch.shwas not running.
So a release sat on the remote with nobody looking at it, and the only signal was the absence of a word
on a card. watcher_armed in command_hq_sessions was already false for both rows, so even the
database did not think it was armed — the UI is honest, it just says nothing.
Why it is worse than an ordinary crash
The commander doctrine is that you arm it and walk away for months. A watcher that dies on reboot
breaks exactly the promise that makes a commander worth having, and it breaks it quietly — no
notification, no failed run, no error anywhere. The owner discovers it by noticing that something which
should have happened did not, which is the failure mode docs/agent-coordination.md exists to avoid.
Note it is not only reboots. Anything that kills the process tree the loop was spawned under does the same thing, and the loop's parentage is an accident of which agent turn armed it.
What to build, in order of how much it buys
-
Say it out loud. The cheapest fix and most of the value: a commander whose row says it was armed and whose pid is dead is a detectable state —
liveLoopPid()already answers it, androuter.tsalready computeswatcherArmedandwatcherRunningseparately for exactly this reason. Notify the owner once when a watcher that was armed is found not running. Nothing else on this list matters if the owner is told. -
Bring it back on boot. A loop that records its pid can record its command line beside it, and something at platform start can re-arm every commander whose row says armed and whose pid is gone. The awkward part is that the loop is authored by the commander itself, per commander — so what gets restarted has to be recorded when it is armed, not reconstructed afterwards.
-
Stop it being a bare process at all.
docs/commander-directory.mdalready argues a commander is a directory rather than a session. A watcher belongs to that directory and could be supervised the way sidecars are, rather than inheriting whichever agent turn happened to spawn it.
Interim, until any of it exists
After a reboot, re-arm every commander by hand. watcherRunning on the Command HQ grid is the
truthful field — a card that does not say "watching" is not watching, whatever it did yesterday.
Related: agent-coordination docs, and data/deploy-watch.sh, which is the deploy commander's own
loop script and equally unsupervised.