one attempt per release, and the rule is code rather than a paragraph
The first watcher had an unbounded retry in it. A release whose deploy failed left no .done file, so the next poll two minutes later spawned a fresh headless agent, and the one after that, all night, each one costing money. The loop was careful and its logging was honest; the defect was structural. It is structural because the loop is written FRESH by an agent every time one arms. A rule that lives only in the prompt gets re-implemented from scratch each time, and the wrong implementation is the one that looks right: test the .done file. That file only exists on success, so testing it is exactly the retry storm. So scripts/deploy-claim.sh owns it. `claim <version>` exits 0 once and 1 for ever after, whatever became of the attempt that held it — a failed release is not retried, it is looked at. Handing the claim back is `rm -rf` by a person, on purpose; a loop cannot undo its own claim, which is what makes the storm impossible rather than unlikely. Three choices in it worth keeping: - `mkdir` is the claim. One atomic syscall that both tests and takes, with no window between checking and acting — two pollers a second apart is not hypothetical when a watcher gets restarted. - Bash, not bun. The thing that stops a retry storm must not need a healthy checkout; a deploy that dies on `bun install` is precisely when it has to work. - A version is digits and dots or it is refused. It arrives off a git tag and is concatenated into a path. A pre-existing .done with no claim beside it still counts as deployed, so deploy/2026.08.21.01 — rolled out by hand before any of this existed — is not re-attempted. docs/servers.md corrected from the commander's own report, which was right and specific: on macbookpro APPEND /opt/homebrew/bin for bun rather than replacing the PATH (this file said "do not help the PATH", which is right for pm2 and is what made plugins:update fail there), `ls` is aliased to an eza that is not installed, plugins:update bounces every sidecar before the plan's own restart step, and fofos cannot publish affordances for what is probably the same ownership problem as its capabilities sync. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -30,6 +30,37 @@ names its own version, and `git ls-remote` answers "is there one" in a single ca
|
||||
- **No untagged release** → say "no release pending" and stop. Do not pull, do not touch a server.
|
||||
- **A `deploy/*` tag with no local record** → that is the release. Continue.
|
||||
|
||||
### One attempt per release, and you do not enforce that yourself
|
||||
|
||||
**Before you spawn or run anything for a version, claim it:**
|
||||
|
||||
```bash
|
||||
scripts/deploy-claim.sh claim <version> # exit 0 = it is yours, exit 1 = do not proceed
|
||||
```
|
||||
|
||||
Exit 1 means somebody already attempted this release. **Stop. Do not retry it, do not deploy it, and do
|
||||
not decide the previous attempt does not count.** Say what the script said and move on.
|
||||
|
||||
On success, record it through the same script:
|
||||
|
||||
```bash
|
||||
scripts/deploy-claim.sh done <version>
|
||||
```
|
||||
|
||||
**Why this is a script and not a rule you follow.** A watcher loop is written fresh each time one arms, so
|
||||
a rule that lives only in this file gets re-implemented from scratch every time — and the wrong version of
|
||||
it looks right: test for the `.done` file, which exists only on SUCCESS. Write that and a failing release
|
||||
spawns a new agent every poll, all night, each one costing money. That is not hypothetical. It is what the
|
||||
first watcher did on 2026-08-21: a careful loop, honest logging, and an unbounded retry inside it.
|
||||
|
||||
So the rule is code. `mkdir` is the claim — atomic, one syscall that both tests and takes, no window
|
||||
between checking and acting. The script refuses a second claim and keeps refusing, whatever happened to the
|
||||
first attempt.
|
||||
|
||||
**A failed release is therefore not retried. It is looked at.** Clearing a claim is a deliberate human act
|
||||
(`rm -rf …/<version>.claim`) or a new tag. A loop cannot undo its own claim, which is the property that
|
||||
makes the storm impossible rather than unlikely.
|
||||
|
||||
### The record is LOCAL, not in git
|
||||
|
||||
```
|
||||
@@ -155,7 +186,24 @@ that are hard to reconstruct afterwards.
|
||||
|
||||
---
|
||||
|
||||
## What is missing, as of writing
|
||||
## Being woken
|
||||
|
||||
**Nothing wakes you yet.** A watcher, a cron entry or a repeating prompt has to exist and does not. Until
|
||||
then this runs when a person starts it.
|
||||
A background job does it, armed from the composer's background-job dialog with **Exit and hand back to the
|
||||
agent, then re-arm**. The wait happens in the shell, not in a model tick — a loop polling
|
||||
`git ls-remote`, which is one network call and no tokens.
|
||||
|
||||
The loop cannot do a deploy: it cannot read a plan, judge a host or write a report. So on firing it spawns
|
||||
a headless agent — `claude -p "<read this file and follow it>" --dangerously-skip-permissions
|
||||
--output-format=stream-json --verbose` — and pipes the events through `jq` so the job panel shows progress
|
||||
instead of a blank box for twenty minutes.
|
||||
|
||||
Two things that loop must get right, both learned by getting them wrong on 2026-08-21:
|
||||
|
||||
- **Claim the version first**, per the section above. This is the one that costs money when missed.
|
||||
- **Set `OFFICER_ROOT` explicitly.** It is empty in the sidecar's environment, so the lock path resolves to
|
||||
`/data/deploys` and the release looks permanently undeployed.
|
||||
|
||||
The deploy therefore runs in a **separate** agent, not in the commander's own conversation: no Command HQ
|
||||
row, no cost on the commander's line, and its reasoning lives in that child's transcript. That is the price
|
||||
of putting the wait in the shell, and it is worth it — a model-side tick that re-reads a whole conversation
|
||||
to learn nothing is the thing being avoided.
|
||||
|
||||
Reference in New Issue
Block a user