Closed: - fleet/an-orchestrator-that-acts-on-every-server-at — built, and as none of the three shapes it proposed. Not a script, not a screen, not a chat capability: an agent handed a written procedure and a server list it has to read rather than one baked into it. All three verified facts in the item earned their place; two things it did not anticipate are now new items. - fleet/a-way-to-mark-a-commit-this-one-ships-so-a — built as a TAG, which the item explicitly ruled out. The owner reversed it, and the argument that changed his mind is recorded: a codeword can be written by accident, because every document about deploying contains it, so the trigger needs rules about where in a subject line it sits and a rule like that is one somebody eventually gets wrong. Opened, all from tonight and all with the evidence attached: - a deploy verifies pm2 and not that the app serves anything — the commander raised this against itself - the commander cannot finish its own host, so one machine is always a build behind - the watcher loop is rewritten by an agent every arming, so its correctness is re-rolled each time - plugins:update restarts every sidecar, so a report naming only officer understates what moved - an agent spawned by a background job is invisible in the UI while being perfectly readable - AskUserQuestion can fail to parse its own call - a commander opened from /chat loses its commander controls, because isCommander comes from the screen rather than the session Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
47 lines
2.7 KiB
Markdown
47 lines
2.7 KiB
Markdown
---
|
|
id: fleet/an-orchestrator-that-acts-on-every-server-at
|
|
title: "An orchestrator that acts on every server at once"
|
|
theme: fleet
|
|
status: done
|
|
created: 2026-08-20
|
|
date_source: assumed
|
|
order: 1
|
|
---
|
|
|
|
**An orchestrator that acts on every server at once** — pull and update every platform, restart a
|
|
process everywhere, report which host is behind. The list is the input; this is the thing that
|
|
reads it.
|
|
|
|
Three facts from writing the list decide the design, and all three are verified: the checkout
|
|
**path differs per host** (take it from PM2's `pm_cwd` for the `officer` process, never a
|
|
hard-coded path); `alpha` carries a **second, stale checkout** so pulling the wrong one succeeds
|
|
while changing nothing; and the **git remotes differ** — one host pulls over HTTPS and one points
|
|
at a different Gitea altogether. An orchestrator built on "ssh, cd to the same path, git pull"
|
|
is wrong on three counts before it starts.
|
|
|
|
Open: whether this is a script over SSH, an Officer screen, or a chat capability; and whether
|
|
`headscale` and `omega` — checkout present, nothing running — are targets at all. Note the
|
|
platform already has an **Update button** for a single server (`03abdeb2`); this is that, fanned
|
|
out, and it should probably share the code rather than reimplement the pull.
|
|
|
|
---
|
|
|
|
**Done 2026-08-21, and not as any of the three shapes this item proposed.** It is not a script, not a
|
|
screen and not a chat capability — it is an **agent** given a written procedure
|
|
(`prompts/deploy-commander.md`) and a list it has to read rather than a list baked into it. Tonight it
|
|
rolled `2026.08.21.02` and `2026.08.21.03` across six hosts unattended and wrote its own reports.
|
|
|
|
All three verified facts in this item survived contact and all three earned their place: it takes each
|
|
checkout from PM2's `pm_cwd`, so `alpha`'s stale second checkout was never touched; it pulls in place
|
|
rather than assuming a remote; and it takes the login user per host, so `deadpool` was reached as `br`.
|
|
|
|
Two things this item did not anticipate, both now in the procedure. It must **skip its own host entirely**
|
|
— restarting `officer` there kills the rollout mid-flight — which leaves that one machine needing a
|
|
deliberate restart afterwards (`todo/fleet/the-deploy-commander-cannot-finish-its-own-host`). And a
|
|
release needs **one attempt only**: the first watcher retried a failed deploy every two minutes, so that
|
|
rule is now `scripts/deploy-claim.sh` rather than a paragraph anyone re-implements.
|
|
|
|
It does **not** share code with the single-server Update button. Worth revisiting, but the overlap turned
|
|
out to be smaller than expected: the button restarts a process, and almost all of this is the reading,
|
|
skipping and reporting around that.
|