pastilhasandClaude Opus 5
aac8af5e1f
The first watcher had an unbounded retry in it. A release whose deploy failed left no .done file, so the next poll two minutes later spawned a fresh headless agent, and the one after that, all night, each one costing money. The loop was careful and its logging was honest; the defect was structural. It is structural because the loop is written FRESH by an agent every time one arms. A rule that lives only in the prompt gets re-implemented from scratch each time, and the wrong implementation is the one that looks right: test the .done file. That file only exists on success, so testing it is exactly the retry storm. So scripts/deploy-claim.sh owns it. `claim <version>` exits 0 once and 1 for ever after, whatever became of the attempt that held it — a failed release is not retried, it is looked at. Handing the claim back is `rm -rf` by a person, on purpose; a loop cannot undo its own claim, which is what makes the storm impossible rather than unlikely. Three choices in it worth keeping: - `mkdir` is the claim. One atomic syscall that both tests and takes, with no window between checking and acting — two pollers a second apart is not hypothetical when a watcher gets restarted. - Bash, not bun. The thing that stops a retry storm must not need a healthy checkout; a deploy that dies on `bun install` is precisely when it has to work. - A version is digits and dots or it is refused. It arrives off a git tag and is concatenated into a path. A pre-existing .done with no claim beside it still counts as deployed, so deploy/2026.08.21.01 — rolled out by hand before any of this existed — is not re-attempted. docs/servers.md corrected from the commander's own report, which was right and specific: on macbookpro APPEND /opt/homebrew/bin for bun rather than replacing the PATH (this file said "do not help the PATH", which is right for pm2 and is what made plugins:update fail there), `ls` is aliased to an eza that is not installed, plugins:update bounces every sidecar before the plan's own restart step, and fofos cannot publish affordances for what is probably the same ownership problem as its capabilities sync. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
108 lines
4.0 KiB
Bash
Executable File
108 lines
4.0 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# One attempt per release, enforced by the filesystem.
|
|
#
|
|
# A watcher loop asks this before it spawns anything. Exit 0 means "you have the claim, go". Exit 1 means
|
|
# "somebody already had it" — and that answer never changes back, whatever happened to the attempt that
|
|
# held it. A release that fails is not retried; it is looked at.
|
|
#
|
|
# WHY THIS IS A SCRIPT AND NOT A PARAGRAPH IN prompts/deploy-commander.md
|
|
#
|
|
# The watcher loop is written fresh by an agent each time it arms, so a rule that lives only in prose is
|
|
# re-implemented from scratch on every arming, and "skip a version already attempted" has a wrong version
|
|
# that looks right: test the .done file, which only exists on SUCCESS. Write that and a failing deploy
|
|
# spawns a fresh agent every poll, all night, each one costing money. That is exactly what happened on
|
|
# 2026-08-21 — the loop was careful, logged honestly, and still had an unbounded retry in it.
|
|
#
|
|
# WHY BASH AND NOT bun
|
|
#
|
|
# The thing that stops a retry storm must not depend on the checkout being healthy. A deploy that dies on
|
|
# `bun install` is precisely when this has to work.
|
|
#
|
|
# WHY mkdir
|
|
#
|
|
# It is atomic on every POSIX filesystem and it fails if the target exists — one syscall that both tests
|
|
# and claims. `[ -f x ] && ...` is two steps with a race between them, and two pollers a second apart is
|
|
# not a hypothetical when a watcher is restarted.
|
|
set -u
|
|
|
|
DEPLOYS_DIR="${DEPLOYS_DIR:-$HOME/officer-dev/data/deploys}"
|
|
|
|
usage() {
|
|
cat >&2 <<'USAGE'
|
|
usage: deploy-claim.sh claim <version> take the one attempt for this release (exit 0 = yours, 1 = taken)
|
|
deploy-claim.sh done <version> record that the attempt succeeded
|
|
deploy-claim.sh state <version> print unclaimed | claimed | done
|
|
deploy-claim.sh list every release this host knows about
|
|
|
|
To retry a release deliberately: rm -rf <deploys-dir>/<version>.claim
|
|
That is a human act on purpose. A loop cannot undo its own claim.
|
|
USAGE
|
|
exit 2
|
|
}
|
|
|
|
version_ok() {
|
|
# The version reaches here from a git tag, so it is attacker-adjacent input used to build a path.
|
|
case "$1" in
|
|
*[!0-9.]* | '' | *..*) return 1 ;;
|
|
*) return 0 ;;
|
|
esac
|
|
}
|
|
|
|
CMD="${1:-}"
|
|
[ -n "$CMD" ] || usage
|
|
|
|
case "$CMD" in
|
|
list)
|
|
mkdir -p "$DEPLOYS_DIR"
|
|
found=0
|
|
for d in "$DEPLOYS_DIR"/*.claim; do
|
|
[ -e "$d" ] || continue
|
|
found=1
|
|
v=$(basename "$d" .claim)
|
|
if [ -f "$DEPLOYS_DIR/$v.done" ]; then s="done"; else s="claimed, no success recorded"; fi
|
|
printf '%-16s %s\n' "$v" "$s"
|
|
done
|
|
[ "$found" = 1 ] || echo "no releases attempted on this host"
|
|
;;
|
|
|
|
claim | done | state)
|
|
V="${2:-}"
|
|
[ -n "$V" ] || usage
|
|
version_ok "$V" || { echo "deploy-claim: refusing a version that is not digits and dots: $V" >&2; exit 2; }
|
|
mkdir -p "$DEPLOYS_DIR" || exit 2
|
|
CLAIM="$DEPLOYS_DIR/$V.claim"
|
|
DONE="$DEPLOYS_DIR/$V.done"
|
|
|
|
case "$CMD" in
|
|
state)
|
|
if [ -f "$DONE" ]; then echo done
|
|
elif [ -d "$CLAIM" ]; then echo claimed
|
|
else echo unclaimed; fi
|
|
;;
|
|
claim)
|
|
# A pre-existing .done with no .claim is a release deployed before this script existed. Honour it.
|
|
if [ -f "$DONE" ]; then
|
|
echo "deploy-claim: $V is already deployed" >&2
|
|
exit 1
|
|
fi
|
|
if mkdir "$CLAIM" 2>/dev/null; then
|
|
printf '%s claimed on %s by pid %s\n' "$(date -Iseconds)" "$(hostname)" "$$" > "$CLAIM/by"
|
|
echo "deploy-claim: $V is yours"
|
|
exit 0
|
|
fi
|
|
printf 'deploy-claim: %s was already attempted — %s\n' "$V" "$(cat "$CLAIM/by" 2>/dev/null || echo 'claim exists')" >&2
|
|
echo "deploy-claim: NOT retrying. Look at what failed, then rm -rf $CLAIM to allow another attempt." >&2
|
|
exit 1
|
|
;;
|
|
done)
|
|
[ -d "$CLAIM" ] || echo "deploy-claim: warning — recording success for $V that was never claimed" >&2
|
|
printf '%s succeeded on %s\n' "$(date -Iseconds)" "$(hostname)" >> "$DONE"
|
|
echo "deploy-claim: $V recorded as deployed"
|
|
;;
|
|
esac
|
|
;;
|
|
|
|
*) usage ;;
|
|
esac
|