Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3343b21a17 | ||
|
|
aac8af5e1f | ||
|
|
8be09a012b |
@@ -0,0 +1,76 @@
|
||||
# 2026.08.21.02 — deployed
|
||||
|
||||
**6/6 hosts.** Every platform the plan reaches is on `55ecbf4f`, `officer` restarted and back `online`,
|
||||
and no other process on any host was touched by the restart step.
|
||||
|
||||
`officer-dev` is the seventh platform and was **skipped entirely** — it is the host this commander runs
|
||||
on. See below.
|
||||
|
||||
Rolled out one host at a time from `officer-dev`, by the procedure in `prompts/deploy-commander.md`.
|
||||
|
||||
## The release
|
||||
|
||||
Tag `deploy/2026.08.21.02` → `55ecbf4f`; `package.json` says `2026.08.21.02`. They agree.
|
||||
Plan: `deploy/2026.08.21.02.md` — pull, `plugins:update`, restart `officer` only. No `bun install`,
|
||||
no `db:push`, and neither was run anywhere.
|
||||
|
||||
| Host | Before | After | plugins | restarted | restart_time | back |
|
||||
| ---------------------------- | ---------- | ---------- | -------------- | --------- | ------------ | ------- |
|
||||
| `alpha` | `eee73e77` | `55ecbf4f` | 12/12 | officer | 60 → 61 | online |
|
||||
| `fofos` | `eee73e77` | `55ecbf4f` | 1/1 | officer | 15 → 16 | online |
|
||||
| `macbookpro` | `eee73e77` | `55ecbf4f` | none installed | officer | 2 → 3 | online |
|
||||
| `edge-pertento-bridge` | `eee73e77` | `55ecbf4f` | 2/2 | officer | 4 → 5 | online |
|
||||
| `production-pertento-bridge` | `eee73e77` | `55ecbf4f` | 1/1 | officer | 6 → 7 | online |
|
||||
| `deadpool-rezio-bridge` | `bd41dff5` | `55ecbf4f` | none installed | officer | 0 → 1 | online |
|
||||
| `officer-dev` | `55ecbf4f` | `55ecbf4f` | not run | **none** | 155 | skipped |
|
||||
|
||||
Every host was clean of TRACKED changes and every pull was a real fast-forward. `pm2 jlist` after each
|
||||
restart showed zero processes in any state other than `online`, so nothing was collaterally bounced.
|
||||
|
||||
Checkout paths were taken from each `officer` process's `pm_cwd`, not assumed — so `alpha`'s stale
|
||||
second checkout at `~/dockers/officer.dev/platform` was never touched.
|
||||
|
||||
## What was NOT done, and why
|
||||
|
||||
- **`officer-dev` was skipped completely** — no pull, no `plugins:update`, no restart. This session runs
|
||||
inside `officer-claude-code` on it, and its `officer` process would have been restarted out from under
|
||||
the rollout. Its checkout is already at `55ecbf4f` (the tag was cut here), so **the code is present and
|
||||
the running process is a build behind**. `pm2 restart officer` on `officer-dev`, when the owner is happy
|
||||
with the rest, finishes this release. **Outstanding.**
|
||||
|
||||
- **`deadpool-rezio-bridge` was two releases behind, not one.** It came in at `bd41dff5` — the
|
||||
`deploy/2026.08.21.01` commit — while every other host was at `eee73e77`. The fast-forward carried both,
|
||||
14 files, and it is now level. Nothing was wrong with it; it simply had not had the interim commits.
|
||||
|
||||
- **`fofos` still cannot sync its capabilities.** Same `dubious ownership` refusal as `2026.08.21.01`,
|
||||
same unexplained cause, still not papered over with a `safe.directory` entry. New this time and probably
|
||||
the same root: its gitea plugin could not publish affordances —
|
||||
`EACCES: permission denied, mkdir '/home/pastilhas/officerdev/capabilities/affordances/officerdev'`.
|
||||
Both point at that capabilities checkout being owned by somebody other than `pastilhas`, which is worth
|
||||
one `stat` before the next deploy. `plugins:update` exited 0 and the plugin itself updated. **Unexplained,
|
||||
and now costing two things instead of one.**
|
||||
|
||||
- **`fofos` carries one untracked file**, `scripts/chk.ts`. A note, not a skip — that is the rule the last
|
||||
report asked for and it worked.
|
||||
|
||||
- **No `bun install` and no `db:push` ran anywhere.** The plan says this release needs neither.
|
||||
|
||||
## Notes for the next one
|
||||
|
||||
- **`macbookpro` needs the PATH helped after all — carefully.** The last report said "`$SHELL -ic` and
|
||||
nothing else"; that is right for `pm2` and wrong for `bun`. `bun` is at `/opt/homebrew/bin/bun`, and
|
||||
`.zshrc` does not put `/opt/homebrew/bin` on the interactive PATH, so `bun run plugins:update` failed
|
||||
`command not found` before it ran a line. What works is **appending** that one directory for that one
|
||||
command (`export PATH=$PATH:/opt/homebrew/bin`) and running `pm2` as a separate command. Appending
|
||||
shadows nothing — `pm2` stays at `~/.local/bin/pm2` and `node` at `node@22`. **Replacing** the PATH is
|
||||
what broke `pm2` last time. `docs/servers.md` says "do not help the PATH" and should say "append, never
|
||||
replace, and only for `bun`".
|
||||
|
||||
- **`$SHELL -ic` on `macbookpro` also aliases `ls` to `eza`**, which is not installed, so a probe using
|
||||
`ls` reports a missing file that is plainly there. Use `command ls` or a plain non-interactive shell for
|
||||
file tests.
|
||||
|
||||
- **`plugins:update` restarts sidecars itself.** It prints `sidecar: started` per plugin, so on `alpha`
|
||||
twelve sidecars bounced during step 3 — before the plan's restart step, and not something the plan asked
|
||||
for. That is the script's own behaviour and it is not new, but a report that only lists `officer` under
|
||||
"restarted" is understating it. Worth a line in the plan format.
|
||||
@@ -0,0 +1,30 @@
|
||||
# 2026.08.21.03 — a drill
|
||||
|
||||
**No code change.** This release exists to watch the watcher fire, and to exercise the claim rule that
|
||||
landed after the last one.
|
||||
|
||||
There is nothing to bump but the version and this file, and that is deliberate rather than lazy: a tag
|
||||
whose `package.json` disagrees with it makes the commander stop, and a tag with no plan beside it makes it
|
||||
stop too, so the smallest honest release is still a version and a plan.
|
||||
|
||||
## What is being tested
|
||||
|
||||
- The watcher noticing a tag and spawning an agent on its own, with nobody watching the composer.
|
||||
- `scripts/deploy-claim.sh` taking the claim before anything is deployed, and the deploy recording success
|
||||
through the same script.
|
||||
- A second pass over the same version being refused rather than repeated.
|
||||
|
||||
## Every host
|
||||
|
||||
1. `git pull --ff-only`
|
||||
2. `bun run plugins:update`
|
||||
3. `pm2 restart officer`
|
||||
|
||||
`officer` only. No `bun install`, no `db:push` — nothing in this range touches a schema, a dependency or
|
||||
the lockfile, and `git diff deploy/2026.08.21.02..HEAD` is two files.
|
||||
|
||||
## Then check
|
||||
|
||||
`pm2 jlist`: `officer` is `online` and its `restart_time` moved. Nothing else restarted.
|
||||
|
||||
`scripts/deploy-claim.sh state 2026.08.21.03` says `done`.
|
||||
+16
-2
@@ -60,13 +60,27 @@ All verified, none cosmetic.
|
||||
_login_ shell does not read. Use `ssh <host> '$SHELL -ic "pm2 jlist"'`. `-lc` is **not** enough — it
|
||||
reports "command not found" on machines where pm2 is plainly running, which cost two wrong conclusions
|
||||
on 2026-08-21.
|
||||
- **On `macbookpro`, do not help the PATH.** Exporting one to reach `bun` breaks `pm2`, which is only on
|
||||
the interactive shell's. `$SHELL -ic` and nothing else.
|
||||
- **On `macbookpro`, APPEND to the PATH for `bun`. Never replace it.** This said "do not help the PATH"
|
||||
until 2026-08-21, and that was half right in a way that cost a deploy step: `pm2` is only on the
|
||||
interactive shell's PATH, so replacing it breaks `pm2` — but `bun` is at `/opt/homebrew/bin/bun` and
|
||||
`.zshrc` does not put that directory on the interactive PATH either, so `bun run plugins:update` fails
|
||||
`command not found` before it runs a line. `export PATH=$PATH:/opt/homebrew/bin` for that one command
|
||||
shadows nothing: `pm2` stays at `~/.local/bin/pm2` and `node` at `node@22`. Run `pm2` as a separate
|
||||
command.
|
||||
- **`$SHELL -ic` on `macbookpro` aliases `ls` to `eza`, which is not installed.** A probe using `ls`
|
||||
reports a missing file that is plainly there. Use `command ls`, or a non-interactive shell for file tests.
|
||||
- **The git remotes differ.** `fofos` pulls over HTTPS; the others over SSH. Never assume a URL — pull in
|
||||
place.
|
||||
- **A "dirty tree" means TRACKED changes.** `git status --porcelain` counts untracked files, and a stray
|
||||
script a fast-forward could never touch is not a reason to skip a host. Use `--untracked-files=no`.
|
||||
- **`macbookpro` is a laptop.** It may be asleep. Unreachable is a skip with a note, not a failed deploy.
|
||||
- **`bun run plugins:update` restarts sidecars itself**, printing `sidecar: started` per plugin — on
|
||||
`alpha` that is twelve processes bouncing during the plugins step, before any restart the plan asked for.
|
||||
Not a fault, but a deploy report that lists only `officer` under "restarted" is understating what moved.
|
||||
- **`fofos` cannot sync its capabilities**, and could not publish gitea affordances on 2026-08-21:
|
||||
`dubious ownership` from git and `EACCES … mkdir '…/capabilities/affordances/officerdev'`. Both point at
|
||||
that capabilities checkout being owned by somebody other than `pastilhas`. Deliberately not papered over
|
||||
with a `safe.directory` entry until somebody has run one `stat` on it. **Unexplained.**
|
||||
|
||||
## Not platforms — ignore for now
|
||||
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "officer",
|
||||
"version": "2026.08.21.02",
|
||||
"version": "2026.08.21.03",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"workspaces": [
|
||||
|
||||
@@ -30,6 +30,37 @@ names its own version, and `git ls-remote` answers "is there one" in a single ca
|
||||
- **No untagged release** → say "no release pending" and stop. Do not pull, do not touch a server.
|
||||
- **A `deploy/*` tag with no local record** → that is the release. Continue.
|
||||
|
||||
### One attempt per release, and you do not enforce that yourself
|
||||
|
||||
**Before you spawn or run anything for a version, claim it:**
|
||||
|
||||
```bash
|
||||
scripts/deploy-claim.sh claim <version> # exit 0 = it is yours, exit 1 = do not proceed
|
||||
```
|
||||
|
||||
Exit 1 means somebody already attempted this release. **Stop. Do not retry it, do not deploy it, and do
|
||||
not decide the previous attempt does not count.** Say what the script said and move on.
|
||||
|
||||
On success, record it through the same script:
|
||||
|
||||
```bash
|
||||
scripts/deploy-claim.sh done <version>
|
||||
```
|
||||
|
||||
**Why this is a script and not a rule you follow.** A watcher loop is written fresh each time one arms, so
|
||||
a rule that lives only in this file gets re-implemented from scratch every time — and the wrong version of
|
||||
it looks right: test for the `.done` file, which exists only on SUCCESS. Write that and a failing release
|
||||
spawns a new agent every poll, all night, each one costing money. That is not hypothetical. It is what the
|
||||
first watcher did on 2026-08-21: a careful loop, honest logging, and an unbounded retry inside it.
|
||||
|
||||
So the rule is code. `mkdir` is the claim — atomic, one syscall that both tests and takes, no window
|
||||
between checking and acting. The script refuses a second claim and keeps refusing, whatever happened to the
|
||||
first attempt.
|
||||
|
||||
**A failed release is therefore not retried. It is looked at.** Clearing a claim is a deliberate human act
|
||||
(`rm -rf …/<version>.claim`) or a new tag. A loop cannot undo its own claim, which is the property that
|
||||
makes the storm impossible rather than unlikely.
|
||||
|
||||
### The record is LOCAL, not in git
|
||||
|
||||
```
|
||||
@@ -155,7 +186,24 @@ that are hard to reconstruct afterwards.
|
||||
|
||||
---
|
||||
|
||||
## What is missing, as of writing
|
||||
## Being woken
|
||||
|
||||
**Nothing wakes you yet.** A watcher, a cron entry or a repeating prompt has to exist and does not. Until
|
||||
then this runs when a person starts it.
|
||||
A background job does it, armed from the composer's background-job dialog with **Exit and hand back to the
|
||||
agent, then re-arm**. The wait happens in the shell, not in a model tick — a loop polling
|
||||
`git ls-remote`, which is one network call and no tokens.
|
||||
|
||||
The loop cannot do a deploy: it cannot read a plan, judge a host or write a report. So on firing it spawns
|
||||
a headless agent — `claude -p "<read this file and follow it>" --dangerously-skip-permissions
|
||||
--output-format=stream-json --verbose` — and pipes the events through `jq` so the job panel shows progress
|
||||
instead of a blank box for twenty minutes.
|
||||
|
||||
Two things that loop must get right, both learned by getting them wrong on 2026-08-21:
|
||||
|
||||
- **Claim the version first**, per the section above. This is the one that costs money when missed.
|
||||
- **Set `OFFICER_ROOT` explicitly.** It is empty in the sidecar's environment, so the lock path resolves to
|
||||
`/data/deploys` and the release looks permanently undeployed.
|
||||
|
||||
The deploy therefore runs in a **separate** agent, not in the commander's own conversation: no Command HQ
|
||||
row, no cost on the commander's line, and its reasoning lives in that child's transcript. That is the price
|
||||
of putting the wait in the shell, and it is worth it — a model-side tick that re-reads a whole conversation
|
||||
to learn nothing is the thing being avoided.
|
||||
|
||||
Executable
+107
@@ -0,0 +1,107 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# One attempt per release, enforced by the filesystem.
|
||||
#
|
||||
# A watcher loop asks this before it spawns anything. Exit 0 means "you have the claim, go". Exit 1 means
|
||||
# "somebody already had it" — and that answer never changes back, whatever happened to the attempt that
|
||||
# held it. A release that fails is not retried; it is looked at.
|
||||
#
|
||||
# WHY THIS IS A SCRIPT AND NOT A PARAGRAPH IN prompts/deploy-commander.md
|
||||
#
|
||||
# The watcher loop is written fresh by an agent each time it arms, so a rule that lives only in prose is
|
||||
# re-implemented from scratch on every arming, and "skip a version already attempted" has a wrong version
|
||||
# that looks right: test the .done file, which only exists on SUCCESS. Write that and a failing deploy
|
||||
# spawns a fresh agent every poll, all night, each one costing money. That is exactly what happened on
|
||||
# 2026-08-21 — the loop was careful, logged honestly, and still had an unbounded retry in it.
|
||||
#
|
||||
# WHY BASH AND NOT bun
|
||||
#
|
||||
# The thing that stops a retry storm must not depend on the checkout being healthy. A deploy that dies on
|
||||
# `bun install` is precisely when this has to work.
|
||||
#
|
||||
# WHY mkdir
|
||||
#
|
||||
# It is atomic on every POSIX filesystem and it fails if the target exists — one syscall that both tests
|
||||
# and claims. `[ -f x ] && ...` is two steps with a race between them, and two pollers a second apart is
|
||||
# not a hypothetical when a watcher is restarted.
|
||||
set -u
|
||||
|
||||
DEPLOYS_DIR="${DEPLOYS_DIR:-$HOME/officer-dev/data/deploys}"
|
||||
|
||||
usage() {
|
||||
cat >&2 <<'USAGE'
|
||||
usage: deploy-claim.sh claim <version> take the one attempt for this release (exit 0 = yours, 1 = taken)
|
||||
deploy-claim.sh done <version> record that the attempt succeeded
|
||||
deploy-claim.sh state <version> print unclaimed | claimed | done
|
||||
deploy-claim.sh list every release this host knows about
|
||||
|
||||
To retry a release deliberately: rm -rf <deploys-dir>/<version>.claim
|
||||
That is a human act on purpose. A loop cannot undo its own claim.
|
||||
USAGE
|
||||
exit 2
|
||||
}
|
||||
|
||||
version_ok() {
|
||||
# The version reaches here from a git tag, so it is attacker-adjacent input used to build a path.
|
||||
case "$1" in
|
||||
*[!0-9.]* | '' | *..*) return 1 ;;
|
||||
*) return 0 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
CMD="${1:-}"
|
||||
[ -n "$CMD" ] || usage
|
||||
|
||||
case "$CMD" in
|
||||
list)
|
||||
mkdir -p "$DEPLOYS_DIR"
|
||||
found=0
|
||||
for d in "$DEPLOYS_DIR"/*.claim; do
|
||||
[ -e "$d" ] || continue
|
||||
found=1
|
||||
v=$(basename "$d" .claim)
|
||||
if [ -f "$DEPLOYS_DIR/$v.done" ]; then s="done"; else s="claimed, no success recorded"; fi
|
||||
printf '%-16s %s\n' "$v" "$s"
|
||||
done
|
||||
[ "$found" = 1 ] || echo "no releases attempted on this host"
|
||||
;;
|
||||
|
||||
claim | done | state)
|
||||
V="${2:-}"
|
||||
[ -n "$V" ] || usage
|
||||
version_ok "$V" || { echo "deploy-claim: refusing a version that is not digits and dots: $V" >&2; exit 2; }
|
||||
mkdir -p "$DEPLOYS_DIR" || exit 2
|
||||
CLAIM="$DEPLOYS_DIR/$V.claim"
|
||||
DONE="$DEPLOYS_DIR/$V.done"
|
||||
|
||||
case "$CMD" in
|
||||
state)
|
||||
if [ -f "$DONE" ]; then echo done
|
||||
elif [ -d "$CLAIM" ]; then echo claimed
|
||||
else echo unclaimed; fi
|
||||
;;
|
||||
claim)
|
||||
# A pre-existing .done with no .claim is a release deployed before this script existed. Honour it.
|
||||
if [ -f "$DONE" ]; then
|
||||
echo "deploy-claim: $V is already deployed" >&2
|
||||
exit 1
|
||||
fi
|
||||
if mkdir "$CLAIM" 2>/dev/null; then
|
||||
printf '%s claimed on %s by pid %s\n' "$(date -Iseconds)" "$(hostname)" "$$" > "$CLAIM/by"
|
||||
echo "deploy-claim: $V is yours"
|
||||
exit 0
|
||||
fi
|
||||
printf 'deploy-claim: %s was already attempted — %s\n' "$V" "$(cat "$CLAIM/by" 2>/dev/null || echo 'claim exists')" >&2
|
||||
echo "deploy-claim: NOT retrying. Look at what failed, then rm -rf $CLAIM to allow another attempt." >&2
|
||||
exit 1
|
||||
;;
|
||||
done)
|
||||
[ -d "$CLAIM" ] || echo "deploy-claim: warning — recording success for $V that was never claimed" >&2
|
||||
printf '%s succeeded on %s\n' "$(date -Iseconds)" "$(hostname)" >> "$DONE"
|
||||
echo "deploy-claim: $V recorded as deployed"
|
||||
;;
|
||||
esac
|
||||
;;
|
||||
|
||||
*) usage ;;
|
||||
esac
|
||||
@@ -0,0 +1,60 @@
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'bun:test';
|
||||
import { existsSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
|
||||
const SCRIPT = join(import.meta.dir, 'deploy-claim.sh');
|
||||
let dir: string;
|
||||
|
||||
const run = (...args: string[]) => {
|
||||
const p = Bun.spawnSync([SCRIPT, ...args], { env: { ...process.env, DEPLOYS_DIR: dir } });
|
||||
return { code: p.exitCode, out: p.stdout.toString(), err: p.stderr.toString() };
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
dir = mkdtempSync(join(tmpdir(), 'deploy-claim-'));
|
||||
});
|
||||
afterEach(() => rmSync(dir, { recursive: true, force: true }));
|
||||
|
||||
// The whole point of this script is that the SECOND caller is refused. Everything else is scaffolding.
|
||||
describe('deploy-claim', () => {
|
||||
it('grants once and refuses after, whatever the outcome was', () => {
|
||||
expect(run('claim', '2026.09.01.01').code).toBe(0);
|
||||
expect(run('claim', '2026.09.01.01').code).toBe(1);
|
||||
// Still refused after a failure — no .done was ever written. This is the retry storm not happening.
|
||||
expect(run('claim', '2026.09.01.01').code).toBe(1);
|
||||
expect(run('state', '2026.09.01.01').out.trim()).toBe('claimed');
|
||||
});
|
||||
|
||||
it('refuses a release already deployed, even with no claim beside it', () => {
|
||||
// A release deployed by hand before this script existed. `deploy/2026.08.21.01` is exactly this case.
|
||||
writeFileSync(join(dir, '2026.08.21.01.done'), 'deployed by hand\n');
|
||||
expect(run('claim', '2026.08.21.01').code).toBe(1);
|
||||
expect(run('claim', '2026.08.21.01').err).toContain('already deployed');
|
||||
});
|
||||
|
||||
it('lets a human hand the claim back, and only a human', () => {
|
||||
expect(run('claim', '2026.09.01.02').code).toBe(0);
|
||||
rmSync(join(dir, '2026.09.01.02.claim'), { recursive: true });
|
||||
expect(run('claim', '2026.09.01.02').code).toBe(0);
|
||||
});
|
||||
|
||||
it('records success separately from the claim', () => {
|
||||
run('claim', '2026.09.01.03');
|
||||
run('done', '2026.09.01.03');
|
||||
expect(run('state', '2026.09.01.03').out.trim()).toBe('done');
|
||||
expect(existsSync(join(dir, '2026.09.01.03.done'))).toBe(true);
|
||||
});
|
||||
|
||||
// The version comes off a git tag, and it is concatenated into a path.
|
||||
it('refuses anything that is not digits and dots', () => {
|
||||
for (const bad of ['../../etc/passwd', 'x; rm -rf /', '2026.08.21.01/..', '', 'v1']) {
|
||||
expect(run('claim', bad).code).toBe(2);
|
||||
}
|
||||
expect(existsSync(join(dir, '..', 'etc'))).toBe(false);
|
||||
});
|
||||
|
||||
it('says so plainly when nothing has been attempted', () => {
|
||||
expect(run('list').out).toContain('no releases attempted');
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user