watcher loop: the agent re-arms itself, same turn — final mechanism, confirmed against real usage
second correction to the re-arm design tonight, and this one came from the owner, not from me. my plan (platform sends a synthetic follow-up turn once the acting turn's result lands, instructing re-arm) was technically sound — it keeps the agent inside the sdk's own Monitor tracking, which is the thing the FIRST wrong plan (silent respawn) broke — but it costs a real extra turn every cycle. pushed back on with hard evidence: the mobile dev's own watcher, on this repo, this session, had been running with THEM re-arming themselves, same turn, for hours already — zero failures. that beats a reliability concern I raised without having tested it. so: handoffContract now instructs the agent to re-arm itself, as the last action of the SAME turn that handles the notification — no synthetic turn, no platform spawn, no extra cost. exitContract's handoff branch corrected to match (was still saying "the harness re-arms a firing on its own", exactly backwards now). server side simplifies to match: createMessageHandler's task:started hook now handles both cases with one rule — pendingArmWatcher (session-object flag, not closure-captured, since arming happens turns after the closure was built) claims the FIRST task:started as the watcher; every SUBSEQUENT task:started while already armed is read as the agent's own self-re-arm and just updates watcherTaskId to match. task:notification records the fire when it matches the tracked id. the platform never drives anything — it only notices what the agent already did. pulled COMMS/command-hq/mobile-3.md, which answered the PREVIOUS (synthetic turn) version — crossed in transit. their transcript-bloat concern is moot now (no synthetic turn exists to bloat anything). their `watcherArmed && !running` detectable-failure pair stands regardless of which mechanism won and is exactly right — kept as-is, matches their own three-state dot (running/armed-but-not/neither). replied in platform.md. all four proposal items now built: prompt+dialog, stopped-vs-died, the re-arm mechanism (this commit), arming state+plumbing. bunx tsgo clean, suite unchanged (1038 pass, same 7 pre-existing baseline failures). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -133,9 +133,11 @@ export async function getCommandHqSession(userId: number, sessionId: string): Pr
|
||||
}
|
||||
|
||||
/**
|
||||
* Arm (or re-arm) a commander's watcher — called both the first time a `handoff` job starts and every
|
||||
* time the harness re-arms it after acting on a fire. Idempotent in effect: arming with the same task id
|
||||
* twice just writes the same values again.
|
||||
* Arm (or re-arm) a commander's watcher — called on the first `task:started` after the dialog submits a
|
||||
* `handoff` job, and again on every SUBSEQUENT `task:started` while already armed, which is how the
|
||||
* AGENT's own self-re-arm (same turn, per `handoffContract`) is picked up: the platform does not drive
|
||||
* the re-arm, it only notices the new task id the agent's own next watch produced. Idempotent in effect:
|
||||
* arming with the same task id twice just writes the same values again.
|
||||
*/
|
||||
export async function armCommandHqWatcher(userId: number, sessionId: string, taskId: string): Promise<boolean> {
|
||||
const rows = await db
|
||||
|
||||
@@ -421,6 +421,15 @@ export type UserSession = {
|
||||
meta: SessionMeta;
|
||||
_sidecarUnsub?: () => void;
|
||||
_claudeKill?: () => void;
|
||||
/**
|
||||
* A `handoff` watcher was just requested and has not yet been claimed by a `task:started`. See
|
||||
* `docs/command-hq.md` → "The watcher loop". Lives on the session object rather than being captured by
|
||||
* `createMessageHandler`'s closure, because arming routinely happens turns after that closure was first
|
||||
* built — the session-scoped event subscription is wired ONCE, on the session's first-ever turn
|
||||
* (`websocket.ts`, the `!session._claudeKill` branch), so a closure built on a LATER turn is never
|
||||
* actually the one receiving events. The session object is the one thing every turn shares.
|
||||
*/
|
||||
pendingArmWatcher?: boolean;
|
||||
};
|
||||
|
||||
export type ModelInfo = {
|
||||
|
||||
@@ -30,6 +30,9 @@ import {
|
||||
setChatSessionModel,
|
||||
setChatSessionPlanMode,
|
||||
createCommandHqSession,
|
||||
getCommandHqSession,
|
||||
armCommandHqWatcher,
|
||||
recordCommandHqWatcherFired,
|
||||
} from 'officerdb';
|
||||
import { resolveHomeDir } from '@@/user-home';
|
||||
import { claudeLoginState } from '@@/os-user-claude';
|
||||
@@ -513,6 +516,74 @@ function createMessageHandler(
|
||||
fileAsCommander();
|
||||
}
|
||||
|
||||
/*
|
||||
The watcher loop — see `docs/command-hq.md` → "The watcher loop" and `handoffContract` in
|
||||
`background-job-prompt.ts`. The platform does not DRIVE re-arming; the agent does, itself, same turn,
|
||||
per that prompt. All the platform does is NOTICE: which task is the current watcher, and when it
|
||||
fired. Both read `session` live (never a value captured when this closure was built) because arming
|
||||
routinely happens turns after the closure was created — the dialog is submitted well into an
|
||||
already-open commander conversation, not necessarily on its first message.
|
||||
*/
|
||||
if (msg.type === 'task:started' && commandHq && transcriptId) {
|
||||
const userId = session.userId;
|
||||
if (userId) {
|
||||
// Consumed here, one-shot — the SAME arm/claim shape `background-job-origin.ts` already uses
|
||||
// client-side for the identical problem (which task is "the one", among however many a turn
|
||||
// might start). Only the FIRST arm needs this: it is what turns "some task started" into "this is
|
||||
// the watcher" in the first place. Every later re-arm is recognised the fallback way, below,
|
||||
// because by then the row already says armed and there is nothing left to claim.
|
||||
if (session.pendingArmWatcher) {
|
||||
session.pendingArmWatcher = false;
|
||||
const taskId = msg.taskId;
|
||||
void armCommandHqWatcher(userId, transcriptId, taskId).catch((err: unknown) => {
|
||||
logger.warn('Could not arm commander watcher', { sessionId, transcriptId, error: String(err) });
|
||||
});
|
||||
} else {
|
||||
// Not a fresh arm — but if this commander is ALREADY armed, a new task starting is exactly what
|
||||
// the agent's own self-re-arm produces (`handoffContract`: same turn, same command, new task id).
|
||||
// Update the tracked id so the NEXT fire is still recognised as this commander's watcher.
|
||||
//
|
||||
// Known imprecision, accepted rather than solved tonight: this cannot tell a genuine re-arm apart
|
||||
// from an unrelated background task the agent happens to start in the same commander conversation
|
||||
// — a commander is one watcher by design, so this is a real but narrow gap, not the common case.
|
||||
const capturedTranscriptId = transcriptId;
|
||||
const taskId = msg.taskId;
|
||||
void getCommandHqSession(userId, capturedTranscriptId)
|
||||
.then((row) => {
|
||||
if (row?.watcherArmed) return armCommandHqWatcher(userId, capturedTranscriptId, taskId);
|
||||
})
|
||||
.catch((err: unknown) => {
|
||||
logger.warn('Could not check commander watcher state', {
|
||||
sessionId,
|
||||
transcriptId: capturedTranscriptId,
|
||||
error: String(err),
|
||||
});
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (msg.type === 'task:notification' && commandHq && transcriptId) {
|
||||
const userId = session.userId;
|
||||
if (userId) {
|
||||
const capturedTranscriptId = transcriptId;
|
||||
const taskId = msg.taskId;
|
||||
void getCommandHqSession(userId, capturedTranscriptId)
|
||||
.then((row) => {
|
||||
if (row?.watcherArmed && row.watcherTaskId === taskId) {
|
||||
return recordCommandHqWatcherFired(userId, capturedTranscriptId);
|
||||
}
|
||||
})
|
||||
.catch((err: unknown) => {
|
||||
logger.warn('Could not record commander watcher fire', {
|
||||
sessionId,
|
||||
transcriptId: capturedTranscriptId,
|
||||
error: String(err),
|
||||
});
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
A turn ended — take a usage snapshot while the person is reading the answer.
|
||||
|
||||
@@ -768,6 +839,9 @@ export async function startClaudeTurn(params: {
|
||||
|
||||
const session = sessionManager.getOrCreate(sessionId, email, cwd, model, groupSlug, msg.context, msg.contextId);
|
||||
session.userId = userId;
|
||||
// Set on the SESSION, not captured by this turn's `createMessageHandler` closure below — see
|
||||
// `pendingArmWatcher`'s own docstring for why that distinction matters here specifically.
|
||||
if (msg.armWatcher) session.pendingArmWatcher = true;
|
||||
// No live socket to attach for an HTTP-originated turn — the caller attaches later, by transcript id,
|
||||
// through the ordinary `/chat` websocket `attach` path, exactly as a reconnect already does.
|
||||
if (ws) {
|
||||
|
||||
@@ -11,21 +11,21 @@ const base: BackgroundJobSpec = {
|
||||
};
|
||||
|
||||
describe('buildBackgroundJobPrompt — fire behavior', () => {
|
||||
test('handoff exits AND carries the harness-owns-re-arm contract verbatim', () => {
|
||||
test('handoff exits AND carries the self-re-arm contract verbatim', () => {
|
||||
const prompt = buildBackgroundJobPrompt(base);
|
||||
expect(prompt).toContain('7. EXIT when it fires');
|
||||
expect(prompt).toContain(handoffContract);
|
||||
// The whole point: the model must not be told to re-arm itself.
|
||||
// The fix this exists for: re-arming is immediate, same turn — not deferred to be remembered later.
|
||||
expect(prompt).not.toContain('Re-arm only if I ask');
|
||||
});
|
||||
|
||||
test("handoff's closing contract does not contradict rule 7 by telling the model to stay stopped", () => {
|
||||
const prompt = buildBackgroundJobPrompt(base);
|
||||
// The generic closing ("Do not restart it... I will start it again") would directly contradict a
|
||||
// mechanism whose entire point is coming back on its own — asserting both would tell the model to
|
||||
// distrust whichever one it read second.
|
||||
// mechanism whose entire point is that the AGENT brings it back itself — asserting both would tell
|
||||
// the model to distrust whichever one it read second.
|
||||
expect(prompt).not.toContain('If I want it running again I will start it again');
|
||||
expect(prompt).toContain('the harness re-arms a');
|
||||
expect(prompt).toContain('re-arming is the');
|
||||
});
|
||||
|
||||
test('stop exits once and says so — no handoff language, no auto-restart language', () => {
|
||||
|
||||
@@ -18,9 +18,14 @@
|
||||
* — a background PROCESS reaches the model only when it EXITS, so a loop that never exits can do shell
|
||||
* work and nothing else. It cannot read a diff, judge whether a push matters, or write code.
|
||||
*
|
||||
* `handoff` is the fix: exit is still the notification (the one part of `keep-going` that always worked),
|
||||
* but re-arming is no longer the model's job. See `handoffContract` below — the harness owns the loop now,
|
||||
* which is the part `watchers-what-broke.md` found the model could not reliably hold across turns anyway.
|
||||
* `handoff` is the fix: exit is still the notification (the one part of `keep-going` that always worked).
|
||||
* Re-arming is still the AGENT's own act — a platform-silent respawn was considered and rejected, because
|
||||
* the harness only ever notifies about a process IT is tracking, so anything spawned outside that tracking
|
||||
* would fire exactly once and then go dark forever. What changes from the ORIGINAL two-option prompt is
|
||||
* WHEN the agent is told to re-arm: immediately, as the last step of the SAME turn that is already
|
||||
* handling the notification, spelled out explicitly in `handoffContract` — not left to be remembered
|
||||
* unprompted across a later turn, which is the part `watchers-what-broke.md` found unreliable. Confirmed
|
||||
* against real usage 2026-08-18: run this way for hours, self-re-armed every time, no failures.
|
||||
*/
|
||||
export type FireBehavior = 'stop' | 'handoff' | 'keep-going';
|
||||
|
||||
@@ -53,12 +58,18 @@ const durationLine = (d: BackgroundJobSpec['duration']): string =>
|
||||
/**
|
||||
* Rule 7 — what happens on fire. `stop` and `handoff` both exit; only the sentence after that differs,
|
||||
* and that sentence is the entire fix `watcher-loop-proposal.md` asked for. Kept as one exported constant
|
||||
* (`handoffContract`) rather than inlined, because it is meant to change meaning at exactly one place —
|
||||
* the day the harness actually owns re-arming — and a prompt asserting that has to say the true thing.
|
||||
* so it is asserted in exactly one place rather than paraphrased differently wherever it is needed.
|
||||
*
|
||||
* The re-arm is explicitly the AGENT's own action, same turn, last step — not deferred, not something a
|
||||
* later turn has to remember unprompted. That immediacy is the whole fix: the failure mode observed live
|
||||
* was a model losing the thread across a context gap, not a model declining an instruction it was just
|
||||
* given. Confirmed against real usage: run this way for hours, self-re-armed every time.
|
||||
*/
|
||||
export const handoffContract =
|
||||
'When the condition fires you will be handed back, with the change. Do the work. The harness re-arms ' +
|
||||
'you — do not re-arm yourself, and do not build your own polling loop on top of this one.';
|
||||
'When the condition fires you will be handed back, with the change. Do the work — and before you end ' +
|
||||
'this turn, start the SAME watch again yourself, exactly as you did the first time: same condition, ' +
|
||||
'same command. That is the whole loop, every time: act, then re-arm, in the one turn. Never build a ' +
|
||||
'separate polling loop on top of this — the exit is still what tells me it fired.';
|
||||
|
||||
function fireRule(onFire: FireBehavior): string {
|
||||
if (onFire === 'keep-going') {
|
||||
@@ -167,18 +178,18 @@ export function buildBackgroundJobPrompt(spec: BackgroundJobSpec): string {
|
||||
|
||||
/**
|
||||
* The closing contract differs for `handoff`, and has to: the generic version tells the model an exit
|
||||
* means STOP and never comes back on its own, which is exactly backwards for a job whose whole design is
|
||||
* that it comes back on its own. Two different promises, so two different closings — asserting the wrong
|
||||
* one would tell the model to distrust the very mechanism rule 7 just described.
|
||||
* means STOP, which is exactly backwards for a job whose whole design is that ITS OWN action, taken
|
||||
* immediately per rule 7, is what brings it back. Two different promises, so two different closings —
|
||||
* asserting the wrong one would tell the model to distrust the very mechanism rule 7 just described.
|
||||
*/
|
||||
function exitContract(isHandoff: boolean): string[] {
|
||||
if (isHandoff) {
|
||||
return [
|
||||
`IF IT EXITS:`,
|
||||
``,
|
||||
`That is either the condition firing — expected, and rule 7 already told you what to do — or me`,
|
||||
`stopping it by hand from its chip. Either way, do not restart it yourself: the harness re-arms a`,
|
||||
`firing on its own, and a stop I asked for staying stopped is the whole point of the button.`,
|
||||
`That is either the condition firing — rule 7 already told you what to do, and re-arming is the`,
|
||||
`last step of that, not a separate decision — or me stopping it by hand from its chip. If it was`,
|
||||
`me, do not start it again: I will, if I want it back.`,
|
||||
];
|
||||
}
|
||||
return [
|
||||
|
||||
Reference in New Issue
Block a user