---
id: fleet/an-orchestrator-that-acts-on-every-server-at
title: "An orchestrator that acts on every server at once"
theme: fleet
status: done
created: 2026-08-20
date_source: assumed
order: 1
---

**An orchestrator that acts on every server at once** — pull and update every platform, restart a
process everywhere, report which host is behind. The list is the input; this is the thing that
reads it.

Three facts from writing the list decide the design, and all three are verified: the checkout
**path differs per host** (take it from PM2's `pm_cwd` for the `officer` process, never a
hard-coded path); `alpha` carries a **second, stale checkout** so pulling the wrong one succeeds
while changing nothing; and the **git remotes differ** — one host pulls over HTTPS and one points
at a different Gitea altogether. An orchestrator built on "ssh, cd to the same path, git pull"
is wrong on three counts before it starts.

Open: whether this is a script over SSH, an Officer screen, or a chat capability; and whether
`headscale` and `omega` — checkout present, nothing running — are targets at all. Note the
platform already has an **Update button** for a single server (`03abdeb2`); this is that, fanned
out, and it should probably share the code rather than reimplement the pull.

---

**Done 2026-08-21, and not as any of the three shapes this item proposed.** It is not a script, not a
screen and not a chat capability — it is an **agent** given a written procedure
(`prompts/deploy-commander.md`) and a list it has to read rather than a list baked into it. Tonight it
rolled `2026.08.21.02` and `2026.08.21.03` across six hosts unattended and wrote its own reports.

All three verified facts in this item survived contact and all three earned their place: it takes each
checkout from PM2's `pm_cwd`, so `alpha`'s stale second checkout was never touched; it pulls in place
rather than assuming a remote; and it takes the login user per host, so `deadpool` was reached as `br`.

Two things this item did not anticipate, both now in the procedure. It must **skip its own host entirely**
— restarting `officer` there kills the rollout mid-flight — which leaves that one machine needing a
deliberate restart afterwards (`todo/fleet/the-deploy-commander-cannot-finish-its-own-host`). And a
release needs **one attempt only**: the first watcher retried a failed deploy every two minutes, so that
rule is now `scripts/deploy-claim.sh` rather than a paragraph anyone re-implements.

It does **not** share code with the single-server Update button. Worth revisiting, but the overlap turned
out to be smaller than expected: the button restarts a process, and almost all of this is the reading,
skipping and reporting around that.