Files
platform/todo/fleet/an-orchestrator-that-acts-on-every-server-at.md
T
pastilhasandClaude Opus 5 530e5880e1 todo: what tonight's deploy work left behind, and two items it closed
Closed:

- fleet/an-orchestrator-that-acts-on-every-server-at — built, and as none of the three shapes it
  proposed. Not a script, not a screen, not a chat capability: an agent handed a written procedure and a
  server list it has to read rather than one baked into it. All three verified facts in the item earned
  their place; two things it did not anticipate are now new items.
- fleet/a-way-to-mark-a-commit-this-one-ships-so-a — built as a TAG, which the item explicitly ruled out.
  The owner reversed it, and the argument that changed his mind is recorded: a codeword can be written by
  accident, because every document about deploying contains it, so the trigger needs rules about where in
  a subject line it sits and a rule like that is one somebody eventually gets wrong.

Opened, all from tonight and all with the evidence attached:

- a deploy verifies pm2 and not that the app serves anything — the commander raised this against itself
- the commander cannot finish its own host, so one machine is always a build behind
- the watcher loop is rewritten by an agent every arming, so its correctness is re-rolled each time
- plugins:update restarts every sidecar, so a report naming only officer understates what moved
- an agent spawned by a background job is invisible in the UI while being perfectly readable
- AskUserQuestion can fail to parse its own call
- a commander opened from /chat loses its commander controls, because isCommander comes from the screen
  rather than the session

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-21 20:27:00 +00:00

2.7 KiB

id, title, theme, status, created, date_source, order
id title theme status created date_source order
fleet/an-orchestrator-that-acts-on-every-server-at An orchestrator that acts on every server at once fleet done 2026-08-20 assumed 1

An orchestrator that acts on every server at once — pull and update every platform, restart a process everywhere, report which host is behind. The list is the input; this is the thing that reads it.

Three facts from writing the list decide the design, and all three are verified: the checkout path differs per host (take it from PM2's pm_cwd for the officer process, never a hard-coded path); alpha carries a second, stale checkout so pulling the wrong one succeeds while changing nothing; and the git remotes differ — one host pulls over HTTPS and one points at a different Gitea altogether. An orchestrator built on "ssh, cd to the same path, git pull" is wrong on three counts before it starts.

Open: whether this is a script over SSH, an Officer screen, or a chat capability; and whether headscale and omega — checkout present, nothing running — are targets at all. Note the platform already has an Update button for a single server (03abdeb2); this is that, fanned out, and it should probably share the code rather than reimplement the pull.


Done 2026-08-21, and not as any of the three shapes this item proposed. It is not a script, not a screen and not a chat capability — it is an agent given a written procedure (prompts/deploy-commander.md) and a list it has to read rather than a list baked into it. Tonight it rolled 2026.08.21.02 and 2026.08.21.03 across six hosts unattended and wrote its own reports.

All three verified facts in this item survived contact and all three earned their place: it takes each checkout from PM2's pm_cwd, so alpha's stale second checkout was never touched; it pulls in place rather than assuming a remote; and it takes the login user per host, so deadpool was reached as br.

Two things this item did not anticipate, both now in the procedure. It must skip its own host entirely — restarting officer there kills the rollout mid-flight — which leaves that one machine needing a deliberate restart afterwards (todo/fleet/the-deploy-commander-cannot-finish-its-own-host). And a release needs one attempt only: the first watcher retried a failed deploy every two minutes, so that rule is now scripts/deploy-claim.sh rather than a paragraph anyone re-implements.

It does not share code with the single-server Update button. Worth revisiting, but the overlap turned out to be smaller than expected: the button restarts a process, and almost all of this is the reading, skipping and reporting around that.