watcher loop: the agent re-arms itself, same turn — final mechanism, confirmed against real usage

second correction to the re-arm design tonight, and this one came from the
owner, not from me. my plan (platform sends a synthetic follow-up turn once
the acting turn's result lands, instructing re-arm) was technically sound —
it keeps the agent inside the sdk's own Monitor tracking, which is the thing
the FIRST wrong plan (silent respawn) broke — but it costs a real extra turn
every cycle. pushed back on with hard evidence: the mobile dev's own
watcher, on this repo, this session, had been running with THEM re-arming
themselves, same turn, for hours already — zero failures. that beats a
reliability concern I raised without having tested it.

so: handoffContract now instructs the agent to re-arm itself, as the last
action of the SAME turn that handles the notification — no synthetic turn,
no platform spawn, no extra cost. exitContract's handoff branch corrected to
match (was still saying "the harness re-arms a firing on its own", exactly
backwards now).

server side simplifies to match: createMessageHandler's task:started hook
now handles both cases with one rule — pendingArmWatcher (session-object
flag, not closure-captured, since arming happens turns after the closure
was built) claims the FIRST task:started as the watcher; every SUBSEQUENT
task:started while already armed is read as the agent's own self-re-arm and
just updates watcherTaskId to match. task:notification records the fire
when it matches the tracked id. the platform never drives anything — it
only notices what the agent already did.

pulled COMMS/command-hq/mobile-3.md, which answered the PREVIOUS (synthetic
turn) version — crossed in transit. their transcript-bloat concern is moot
now (no synthetic turn exists to bloat anything). their `watcherArmed &&
!running` detectable-failure pair stands regardless of which mechanism won
and is exactly right — kept as-is, matches their own three-state dot
(running/armed-but-not/neither). replied in platform.md.

all four proposal items now built: prompt+dialog, stopped-vs-died, the
re-arm mechanism (this commit), arming state+plumbing.

bunx tsgo clean, suite unchanged (1038 pass, same 7 pre-existing baseline
failures).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-18 08:33:47 +00:00
co-authored by Claude Opus 5
parent cc5340b8e0
commit 624d67ed52
7 changed files with 232 additions and 59 deletions
+52 -38
View File
@@ -311,14 +311,33 @@ process reaches the model only when it EXITS. The owner's call, confirmed 2026-0
proposal as written, before the first real tests** — this governs the agent's behaviour generally, not
only Command HQ, so it is not optional groundwork the way most of this document is.
**The re-arm mechanism, settled 2026-08-18, after two false starts recorded here rather than quietly
fixed:**
1. First instinct: the PLATFORM silently respawns the watch command when it fires, no agent turn involved.
Wrong — the SDK only ever notifies the agent about a process it is itself tracking via `Monitor`. A
platform-side respawn exists outside that tracking, so it would fire once, work, and then go dark
forever — the agent is never told about the second fire. Not a cost tradeoff, a dead end.
2. Second instinct: keep the agent in the loop, but have the PLATFORM drive it — detect the acting turn's
`result` and send a synthetic follow-up message telling the agent to re-arm. Technically correct, but
costs a real extra turn every single cycle, and the owner pushed back on that cost before it was built:
an actual multi-hour, real-world watcher (the mobile side's own, on this repo, this session) had
already been running with the agent re-arming ITSELF, same turn, for hours — zero failures. That is
hard evidence to override with a hypothetical reliability argument.
3. **What is actually built:** the agent re-arms itself, as the LAST action of the SAME turn that handles
the notification — no separate turn, no platform-side spawn. The platform's role is narrower than
either false start: it only WATCHES for this happening and can say so if it doesn't (see "Built" below)
— it does not drive the re-arm and does not respawn anything itself.
**Built:**
- **A third dialog option**, `FireBehavior = 'stop' | 'handoff' | 'keep-going'` (was a `stopOnFire`
boolean). `handoff` — "Exit and hand back to the agent, then re-arm" — carries the proposal's one-sentence
contract (`handoffContract` in `background-job-prompt.ts`) verbatim: _"When the condition fires you will
be handed back, with the change. Do the work. The harness re-arms you — do not re-arm yourself, and do
not build your own polling loop on top of this one."_ The closing "IF IT EXITS" section is rewritten for
`handoff` too, so it does not contradict rule 7 by telling the model an exit means stay-stopped.
boolean). `handoff` — "Exit and hand back to the agent, then re-arm" — carries the contract
(`handoffContract` in `background-job-prompt.ts`) verbatim: _"When the condition fires you will be handed
back, with the change. Do the work — and before you end this turn, start the SAME watch again yourself,
exactly as you did the first time: same condition, same command. That is the whole loop, every time: act,
then re-arm, in the one turn."_ The closing "IF IT EXITS" section is rewritten for `handoff` too, so it
does not contradict rule 7 by telling the model an exit means stay-stopped.
- **`isCommander`, gating the promise to where it can be kept.** The proposal frames this entirely in terms
of commanders — "a commander already has a row," arming state's only home — and outside one there is
nowhere to persist "this should keep re-arming itself," so nothing would fulfil the promise. Rather than
@@ -335,8 +354,6 @@ only Command HQ, so it is not optional groundwork the way most of this document
Known gap: this is in-memory client state, so it does not survive a page refresh between clicking Stop
and the notification arriving — an acceptable, narrow window, not attempted to be closed tonight.
**Built, item 4 (ahead of item 3 — plumbing needed to exist before the trigger could be tested):**
- **`command_hq_sessions` gained `watcherArmed`, `watcherTaskId`, `watcherLastFiredAt`.** Unlike `running`,
`watcherArmed` is allowed to be stored — it is the owner's INTENT ("keep re-arming this"), not a live
process fact the platform is reporting on, so nothing external can silently falsify it the way a sidecar
@@ -346,39 +363,36 @@ only Command HQ, so it is not optional groundwork the way most of this document
read-only on `GET /commanders` and `GET /commanders/:sessionId`, and `PATCH /commanders/:sessionId` accepts
`{watcherArmed: false}` to disarm (arming itself has no route — it needs a live task id, which only a
running turn has).
- **`armWatcher` threaded end-to-end, client to server, and CONSUMED NOWHERE YET.** `BackgroundJobDialog`
computes it (`watching && onFire === 'handoff'`) → `sendWithHiddenPrefix` → `QueuedPrompt.armWatcher` →
`sendPrompt`'s new last parameter → the chat websocket message → `ClientMessage`'s `'chat'` variant →
`startClaudeTurn`'s `msg.armWatcher`. Every step type-checks and the full suite is unaffected, because
nothing reads the field on the server yet — safely inert plumbing, not a shipped mechanism.
- **`armWatcher` threaded end-to-end, client to server.** `BackgroundJobDialog` computes it (`watching &&
onFire === 'handoff'`) → `sendWithHiddenPrefix` → `QueuedPrompt.armWatcher` → `sendPrompt`'s new last
parameter → the chat websocket message → `ClientMessage`'s `'chat'` variant → `startClaudeTurn`'s
`msg.armWatcher`, which sets `session.pendingArmWatcher = true` — on the SESSION object, not captured by
`createMessageHandler`'s closure, because the dialog is routinely submitted turns after that closure was
built (the session-scoped event subscription wires ONCE, on the session's first-ever turn) and a value
captured then would never see a later arm.
- **The noticing, in `createMessageHandler` (`websocket.ts`).** On `task:started`: if `pendingArmWatcher`
is set, consume it (one-shot, the same arm/claim shape `background-job-origin.ts` already uses
client-side for the identical problem) and call `armCommandHqWatcher` — this is what turns "some task
started" into "this is the watcher" in the first place. If the commander is ALREADY armed instead, a new
task starting is what the agent's own self-re-arm produces, so the tracked `watcherTaskId` is simply
updated to it — the platform is not driving anything here, only keeping up with what the agent already
did. On `task:notification`: if it matches the tracked `watcherTaskId` for an armed commander, record the
fire (`watcherLastFiredAt`). Both read `getCommandHqSession` live rather than trusting any closure-local
state, for the same reason as above.
- **Known, accepted imprecision:** the "already armed → treat the next task:started as the re-arm" rule
cannot tell a genuine self-re-arm apart from an unrelated background task the agent happens to start in
the same commander conversation. A commander is one watcher by design, so this is a narrow gap rather
than the common case, and not attempted to be closed tonight.
**Not yet built — item 3, and a real design question found while trying to build it:**
Suggested order from the proposal, all four items now done: (1) prompt + dialog, (2) stopped-vs-died, (3)
the re-arm mechanism itself — landed as agent-driven, same-turn, after two rejected designs recorded above
— (4) arming state and the plumbing that watches for it.
The original plan here was wrong and is worth recording as wrong rather than quietly fixed: `task-restart.ts`'s
`restartBackgroundTask` looked like the obvious reuse — it already recovers a command from a transcript and
respawns it detached. **It is not reusable for this.** A restarted job is deliberately outside the SDK's
own `Monitor` tracking — `chat.ts`'s restart route emits its own `task:notification` via `emitToSession`
specifically because the harness stopped watching. That path updates the UI; it does not reach back into
the agent's own context, because there is no mechanism by which it could. A `handoff` watcher's whole
point is the OPPOSITE: the agent must be told again when it fires again, which only happens through the
SDK's own notification delivery, which only exists for a job the SDK itself is tracking.
So harness-owned re-arm cannot be a silent platform-side respawn. It has to drive the AGENT to reissue its
own `run_in_background`/`Monitor` call — the platform's job is to make that reissue AUTOMATIC and
CORRECT (the right command, the right moment, no double-arming) rather than something the model has to
remember across turns, not to do the reissuing itself. Concretely, likely shape: once a fired watcher's
"acting" turn ends (`result`, for a commander whose `watcherTaskId` just produced a `task:notification`),
the platform sends a SYNTHETIC follow-up turn — not from the user — instructing the agent to re-arm using
the exact command `commandForTask` recovers from the original transcript. `commandForTask` is still the
right reuse; `restartBackgroundTask` is not. Not implemented — raised with the owner before writing it, for
the second time tonight, because getting the mechanism itself wrong is more expensive to discover after
the fact than before.
Suggested order from the proposal: (1) prompt + dialog — done, (2) stopped-vs-died — done, (4) arming
state and plumbing — done ahead of schedule, (3) the actual re-arm trigger — open, blocked on confirming
the synthetic-turn design above. "Nothing here blocks the first live test of the apps; all of it blocks
trusting a watcher that is meant to run for months" — the proposal's own words, and the reason this is
still open rather than rushed.
**Deliberately not built: the platform verifying re-arm happened.** Raised as a possible cheap addition —
since the platform already knows `watcherLastFiredAt` and can already compute `running` from the same fold
`GET /commanders` uses for every card, a client could flag "fired N ago, nothing has run since" without
any new mechanism, purely by reading what already exists. Not built tonight; noted here so it is not lost,
not because it was rejected.
## Later — explicitly deferred, not forgotten