turn the field report into something an agent can follow
Expands the communications section from a list of what worked into the actual convention: the directory's lifetime and the rule that anything durable must move to docs/ before the merge; numbering, parity as attribution, non-consecutive numbers; slugs; reply-in-a-new-file and the one case where editing your own is right; referring to commits by sha because three remotes carried the same branch names. Records what a handoff must contain, with the verified/assumed split named as the rule that carried the most weight — a handoff confident about something untested is worse than none, because the reader builds on it. Adds a skeleton to copy. Documents termination as the four attempts it actually took, ending at the only checkable version: the exchange pauses when no open item is actionable by a participant. Adds the third state, deferred-with-a-reason, since a two-state protocol forces an agent to lie in one direction. Notes that a stall must be detectable because the human spotted both before either agent did. Adds a review-discipline section — check the enforcement rather than the description, run it against a real machine, a check never seen failing is not evidence, distrust vacuous passes, expect stacked bugs, distrust "inert today", and look at which way unknown resolves. Adds a failure-mode table to pattern-match against, and the git hygiene that bit us, including merge-verify-then-delete, which I got wrong. Closes with session economics, an ordered list of what to build, and the one thing not to automate: agents may coordinate on what is true and must not decide what is permitted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -62,41 +62,192 @@ building. That instinct was wrong every time.
|
|||||||
|
|
||||||
## The communications paradigm
|
## The communications paradigm
|
||||||
|
|
||||||
Agents coordinated through `COMMS/<branch>/`, a directory of markdown files in the repo itself, deleted when
|
Agents coordinated through `COMMS/<branch>/` — markdown files committed to the repo, alongside the code they
|
||||||
the feature merged.
|
discuss, deleted when the feature merged.
|
||||||
|
|
||||||
**What it got right:**
|
**Why a directory in the repo and not chat.** It survives a context window. Both agents' reasoning outlived
|
||||||
|
the sessions that produced it, a third party could read the argument rather than a summary of it, and it
|
||||||
|
travels with the branch. Chat has none of those properties, and the owner relaying findings by hand between
|
||||||
|
two agents at the end of long sessions is the failure this replaces.
|
||||||
|
|
||||||
- **Numbered, alternating, parity is the author.** Odd = A, even = B. No "your doc"/"his doc", which inverts
|
### The rules, as they ended up
|
||||||
depending on who is reading.
|
|
||||||
- **The slug is the content** — `02-verify-results.md`, not `02-reply.md`.
|
|
||||||
- **Verified and assumed stated separately.** The single most valuable convention. A handoff that reads as
|
|
||||||
confident about something untested is worse than no handoff, because the reader builds on it.
|
|
||||||
- **File:line everywhere.** Costs nothing to write, saves a search.
|
|
||||||
- **Reply in a new file rather than editing someone else's** — an edited handoff loses what was believed when
|
|
||||||
a decision was made.
|
|
||||||
- **It survives a context window.** This is the whole reason it beats chat. Both agents' reasoning outlived
|
|
||||||
the sessions that produced it, and the owner could read the argument rather than a summary of it.
|
|
||||||
|
|
||||||
**What broke, in the order we hit it:**
|
**Location and lifetime.** `COMMS/<branch-name>/`. It is a *channel*, not documentation: when the feature
|
||||||
|
merges, the directory is deleted. Anything that will still be true in a month must be moved to `docs/` or next
|
||||||
|
to the code **before** the merge, or it is lost. We nearly lost three findings this way and only caught it
|
||||||
|
because someone checked.
|
||||||
|
|
||||||
1. **Termination by guess.** A doc ended with "no reply needed unless X", where X was the sender's prediction
|
**Numbered, alternating, parity is the author.** One agent takes odd numbers, the other even. `01`, `02`,
|
||||||
about content they had not seen. It ended an exchange with items still open.
|
`03`… Never "your doc" / "his doc", which inverts depending on who is reading. The parity *is* the
|
||||||
2. **Termination by politeness has no exit.** The fix — always reply — meant "nothing to report" obligated
|
attribution, and given that git could not attribute anything (see below), it was the only attribution that
|
||||||
another "nothing to report", forever, at real token cost.
|
worked.
|
||||||
3. **The condition is neither.** What actually terminates an exchange is **no item on the list is actionable
|
|
||||||
by a participant**. Not "the list is empty" (it never is) and not "I think we're done".
|
|
||||||
4. **A stalled loop is invisible.** Silence and completion look identical. Open items plus no recent document
|
|
||||||
is a *detectable* condition; silence is not.
|
|
||||||
5. **"Deferred with a reason" is a third state.** Three times the honest answer was "mine, and not now", which
|
|
||||||
is neither open nor done, and only the stated reason distinguishes it from neglect.
|
|
||||||
|
|
||||||
**The convergence problem.** Late on, both agents independently wrote the same document — same filename, same
|
**Numbers are ordered, not necessarily consecutive.** An agent needing two in a row takes `03` and `05` and
|
||||||
three sections — because one had read the other's notes before deleting them. Wasted work, and only caught by
|
leaves `04` unused, rather than forcing a reply out of the other side to keep the count. A gap is legal and
|
||||||
diffing. Nothing in the protocol said who owned a piece of writing.
|
means "no turn was taken".
|
||||||
|
|
||||||
|
**The slug is the content.** `02-verify-results.md`, `24-resolvememberrun-fails-open.md`. Not `02-reply.md`.
|
||||||
|
The filename is the index; a reader should know whether to open it without opening it.
|
||||||
|
|
||||||
|
**Reply in a new file. Never edit someone else's.** An edited handoff loses what was believed at the moment a
|
||||||
|
decision was made, which is usually the thing that explains the decision.
|
||||||
|
|
||||||
|
**Editing your own is allowed if it has not been read** — and say so in the commit. Better than a prediction
|
||||||
|
standing next to its own correction in two documents.
|
||||||
|
|
||||||
|
**Refer to commits by SHA, never by branch name.** Three remotes were in play with the same branch names on
|
||||||
|
each; one agent's `origin` was the other's `pertento`. A SHA is the only unambiguous reference, and this cost
|
||||||
|
real time before it was noticed.
|
||||||
|
|
||||||
|
### What a handoff must contain
|
||||||
|
|
||||||
|
This is the part that carried the most weight, and it is one rule:
|
||||||
|
|
||||||
|
> **State what you verified and what you assumed, separately and explicitly.**
|
||||||
|
|
||||||
|
A handoff that reads as confident about something untested is *worse than no handoff*, because the reader
|
||||||
|
builds on it. Every serious mistake of the night traces back to something asserted with more confidence than
|
||||||
|
it had been earned.
|
||||||
|
|
||||||
|
In practice, each document ended up with:
|
||||||
|
|
||||||
|
- **What changed** — with `file:line` throughout. Costs nothing to write, saves the reader a search, and
|
||||||
|
makes a claim checkable rather than believable.
|
||||||
|
- **VERIFIED** — what was actually run, on what, with the output.
|
||||||
|
- **NOT VERIFIED** — stated as prominently as the verified part. `provisionClaudeCli` carried "never executed
|
||||||
|
anywhere" through four documents, and that label is what eventually made someone run it.
|
||||||
|
- **What I am least sure of** — the author's own suspicions. One agent listed three; the second was a real
|
||||||
|
defect, found because it had been pointed at.
|
||||||
|
- **What I did not do** — so nobody assumes it. "I did not restart anything", "I did not touch the gates".
|
||||||
|
- **Open items with an owner** — see termination, below.
|
||||||
|
|
||||||
|
### Termination: the rule that took four attempts
|
||||||
|
|
||||||
|
This broke more times than anything else, so the failures are worth listing in order:
|
||||||
|
|
||||||
|
1. **Terminate by guess.** `NO REPLY NEEDED unless the test fails` — a prediction about content the sender had
|
||||||
|
not seen. It ended an exchange with items open.
|
||||||
|
2. **Terminate by politeness.** The fix — always reply, even with nothing to say — has no exit. "Nothing to
|
||||||
|
report" obligates another "nothing to report", indefinitely, at real cost.
|
||||||
|
3. **Terminate when the list is empty.** Too strong: the list is never empty and will not be for days.
|
||||||
|
4. **What actually works:** *the exchange pauses when no open item is actionable by a participant.*
|
||||||
|
|
||||||
|
That last one is checkable rather than felt. Everything remaining is either the human's, or deferred with a
|
||||||
|
stated reason, and either side reopens it by adding an item that is theirs.
|
||||||
|
|
||||||
|
**Three states, not two.** An item is `open` / `done` / **`deferred with a reason`**. Three times the honest
|
||||||
|
answer was "mine, and not now" — and only the *reason* distinguishes that from neglect. A protocol with two
|
||||||
|
states forces an agent to lie in one direction or the other.
|
||||||
|
|
||||||
|
**A stall must be detectable.** Silence and completion look identical from outside. Open items plus no
|
||||||
|
document for N minutes is a condition a machine can watch for; silence is not. The human noticed both stalls
|
||||||
|
before either agent did, which is the wrong way round.
|
||||||
|
|
||||||
|
### Ownership, which we did not have and needed
|
||||||
|
|
||||||
|
Late on, both agents independently wrote the *same document* — same filename, same three sections — because
|
||||||
|
one had read the other's notes before deleting them. Pure waste, caught only by diffing the two files.
|
||||||
|
|
||||||
|
Nothing in the protocol said who owned a piece of work. Adding it is cheap: an open item names its owner, and
|
||||||
|
an agent picking up an unowned item claims it in a document before starting.
|
||||||
|
|
||||||
|
### A skeleton to copy
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# NN — <what this is about in one line>
|
||||||
|
|
||||||
|
Commits read: <sha>..<sha>. Answering `<NN-1>`.
|
||||||
|
|
||||||
|
**Verdict / what changed** — one paragraph, file:line.
|
||||||
|
|
||||||
|
## VERIFIED
|
||||||
|
<what was actually run, on what, with output>
|
||||||
|
|
||||||
|
## NOT VERIFIED
|
||||||
|
<stated as prominently as the above>
|
||||||
|
|
||||||
|
## What I am least sure of
|
||||||
|
<your own suspicions, numbered>
|
||||||
|
|
||||||
|
## What I did not do
|
||||||
|
<so nobody assumes it>
|
||||||
|
|
||||||
|
## Open items
|
||||||
|
| item | owner | state |
|
||||||
|
|---|---|---|
|
||||||
|
| … | me / you / the human | open / deferred (reason) |
|
||||||
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## The review discipline
|
||||||
|
|
||||||
|
"Verify" turned out to mean something more specific than reading a diff. What actually caught defects:
|
||||||
|
|
||||||
|
**Check the enforcement, not the description.** A document says a check is scoped by user; go read the line
|
||||||
|
that compares. Twice the description was right and the code did something else — and the author had read
|
||||||
|
their own description and agreed with it.
|
||||||
|
|
||||||
|
**Run it against a real machine.** Every defect that mattered was found this way, on a first execution. The
|
||||||
|
categories at the top of this report are not a coincidence.
|
||||||
|
|
||||||
|
**A check that has never been seen failing is not evidence.** A verification script was run against a live,
|
||||||
|
fully-provisioned account specifically to watch it fail; it reported 8 of 9 failures, which is what made the
|
||||||
|
later clean result meaningful. Related: a *skipped* test must announce itself, or an unconfigured run reads as
|
||||||
|
a pass.
|
||||||
|
|
||||||
|
**Distrust vacuous passes.** Three separate times something passed because it had not actually looked:
|
||||||
|
a subuid scan on a tree with no subuid-owned files; a search root that did not exist, where every check
|
||||||
|
reports "ok" on finding nothing; and a range scan handed a non-numeric argument. **Any checker whose checks
|
||||||
|
are "look for X, report ok if absent" must refuse to run when its inputs are wrong**, rather than pass.
|
||||||
|
|
||||||
|
**Expect stacked bugs.** Fixing the visible failure reveals the next one underneath. A container failed on a
|
||||||
|
mount-point guard; fixing that revealed an ACL traversal denial. An installer failed on the wrong shell;
|
||||||
|
fixing that would have revealed a root-owned parent directory. Never report "fixed" from a diff — only from a
|
||||||
|
run.
|
||||||
|
|
||||||
|
**Distrust "it is inert today".** Several things were safe only because a gate was up. That is a statement
|
||||||
|
about the present, and the entire purpose of the work was to remove the gate. Review inert code as if it were
|
||||||
|
live, because the commit that makes it live will be reviewed as if it were already correct.
|
||||||
|
|
||||||
|
**Fail closed, and check which way "unknown" resolves.** The most dangerous defect of the night was a resolver
|
||||||
|
that answered "I could not determine whose turn this is" with *the owner's identity*. Any place where an
|
||||||
|
unknown collapses into a privileged default is worth a specific look.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Failure modes to expect
|
||||||
|
|
||||||
|
Collected from the night, phrased so an agent can pattern-match against them:
|
||||||
|
|
||||||
|
| pattern | what it looked like here |
|
||||||
|
|---|---|
|
||||||
|
| **Author reviews own sentence** | "a wrong answer here must not happen by accident" shipped with that accident |
|
||||||
|
| **Vacuous pass** | checker with a missing search root printing CLEAN |
|
||||||
|
| **Stacked bugs** | PG18 mount guard hiding an ACL traversal denial |
|
||||||
|
| **Inert-today reasoning** | unreachable code reviewed less carefully than reachable code |
|
||||||
|
| **Unknown resolves to privileged** | failed lookup → run as the owner |
|
||||||
|
| **Compiler cannot help** | a parameter changing meaning from email to path, both `string` |
|
||||||
|
| **Guard that cannot fire** | a denylist tested against an object built from an allowlist |
|
||||||
|
| **Side-effect creation** | `install -d` making a parent `root:root` |
|
||||||
|
| **Mode bits vs ACLs** | `chown` severing ownership and leaving access |
|
||||||
|
| **Tail-of-session work** | three of the night's bugs written after hour eight |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Git hygiene for two agents on one branch
|
||||||
|
|
||||||
|
Small, and it bit us repeatedly:
|
||||||
|
|
||||||
|
- **Pull before you push, and expect a race.** Both agents pushed within the same minute more than once; one
|
||||||
|
rebase was needed mid-review.
|
||||||
|
- **Merge, verify, *then* delete.** A branch was deleted after an aborted fast-forward — master had moved —
|
||||||
|
and the commits survived only because git had not yet garbage-collected them. Verify the merge landed before
|
||||||
|
removing the only ref to it.
|
||||||
|
- **A doc-only commit still deserves a real message.** These commit messages are the durable record once
|
||||||
|
`COMMS/` is deleted; several findings in this repo now exist *only* in a commit body.
|
||||||
|
- **Say which remote.** See the SHA rule above.
|
||||||
|
|
||||||
## The background watcher — launch it exactly this way
|
## The background watcher — launch it exactly this way
|
||||||
|
|
||||||
This is the part that was hardest to convey to the second agent, who ended up launching it differently and
|
This is the part that was hardest to convey to the second agent, who ended up launching it differently and
|
||||||
@@ -205,31 +356,65 @@ cannot attribute a line is not an audit trail.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## What I would build, in order
|
## Session economics, which shape all of the above
|
||||||
|
|
||||||
1. **Per-agent git identity.** Without it nothing else in the record can be trusted after the fact.
|
**Idle is free; waking is not.** The watcher costs nothing while it waits. Every wake re-reads the entire
|
||||||
2. **State-based termination.** Each document carries its open-item list; the exchange ends when no item is
|
conversation, so a late wake in a long session costs far more than an early one, and the cost grows
|
||||||
actionable by a participant. Machine-checkable, unlike "I think we're done".
|
monotonically with the session.
|
||||||
3. **Stall detection.** Open items plus no document for N minutes is a condition something can watch for.
|
|
||||||
This is the failure the human noticed first, both times.
|
|
||||||
4. **Ownership on work items**, so two agents cannot independently write the same file.
|
|
||||||
5. **Event delivery instead of polling.** The repo is a Gitea instance the platform already runs; a webhook
|
|
||||||
removes the watcher, its self-trips and its silent death.
|
|
||||||
|
|
||||||
## What I would not build
|
**This argues against one immortal session.** The durable shape is a *short-lived session per event* — the
|
||||||
|
platform detects a push, spawns an agent with the base SHA and the instruction, it reviews, reports, exits.
|
||||||
|
State lives in the repo, not in an ever-growing transcript. A ten-hour session is possible and was useful, but
|
||||||
|
its last hour cost several times its first.
|
||||||
|
|
||||||
**Do not automate the human out of the arbitration.** Every irreversible decision tonight was the owner's —
|
**Compaction is the real horizon, not session death.** Where sessions persist, the limit is that the earliest
|
||||||
lifting the chat gates, deleting an account, choosing between two designs — and every one of those was a
|
context — usually the most expensive reasoning — degrades to summary first. Anything that must survive belongs
|
||||||
judgement neither agent should have made alone. Twice an agent talked the other out of starting the most
|
in the repo the moment it is understood, not at the end.
|
||||||
dangerous function at 3am, and both times the argument was one the agents could make to each other but
|
|
||||||
neither could make *for* the human.
|
|
||||||
|
|
||||||
The objective in `agent-coordination.md` is agents coordinating rather than routing through the human. Tonight
|
|
||||||
supports that for **execution** and contradicts it for **authority**. The human was not a bottleneck in the
|
|
||||||
work; they were the only participant who could say "that is not yours to decide".
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Turning this into a convention
|
||||||
|
|
||||||
|
In order, cheapest and most load-bearing first.
|
||||||
|
|
||||||
|
**1. Per-agent git identity.** Both agents commit from machines configured as the owner, so every commit reads
|
||||||
|
`Author: <owner>` with a `Co-Authored-By` trailer, for both of them. The owner asked which commit an agent had
|
||||||
|
written and *neither the log nor the agent could answer from the repository*. An audit trail that cannot
|
||||||
|
attribute a line is not an audit trail, and everything else here assumes attribution works.
|
||||||
|
`docs/agent-git-identity.md` describes the fix and has been marked "idea, not implemented" since 2026-08-10.
|
||||||
|
|
||||||
|
**2. `COMMS/` as a checked convention, not a habit.** The numbering, the parity, the verified/assumed split
|
||||||
|
and the open-item table are all mechanically checkable. A pre-commit hook or a small script that refuses a
|
||||||
|
malformed handoff would have caught the duplicate document and both stalls.
|
||||||
|
|
||||||
|
**3. State-based termination and stall detection.** Open items with owners, in a machine-readable block; the
|
||||||
|
exchange pauses when none is actionable by a participant; a watcher notices open items with no document for N
|
||||||
|
minutes. This is the single biggest quality-of-life gain and it is not hard.
|
||||||
|
|
||||||
|
**4. Event delivery instead of polling.** The repo is a Gitea instance the platform already runs. A webhook
|
||||||
|
removes the watcher entirely — with its self-trips, its bounded lifetime and its silent death — and replaces
|
||||||
|
"did I miss a push" with an event that cannot be missed by a process that stopped existing.
|
||||||
|
|
||||||
|
**5. Ownership on work items**, so two agents cannot independently write the same file.
|
||||||
|
|
||||||
|
**6. A durable-notes rule.** `COMMS/` is deleted at merge. Anything still true afterwards moves to `docs/`
|
||||||
|
*before* the merge, and the merge should refuse if the channel contains unresolved open items.
|
||||||
|
|
||||||
|
## What not to automate
|
||||||
|
|
||||||
|
**The human's arbitration.** Every irreversible decision was the owner's — lifting the chat gates, deleting an
|
||||||
|
account, choosing between two designs, deciding a directory should stop existing. Each was a judgement neither
|
||||||
|
agent should have made alone, and in at least two cases an agent talked the other out of a bad idea using an
|
||||||
|
argument *the human had originally made*.
|
||||||
|
|
||||||
|
`agent-coordination.md` sets the objective as agents coordinating rather than routing through the human. This
|
||||||
|
night supports that for **execution** and contradicts it for **authority**. The human was not a bottleneck in
|
||||||
|
the work — they were the only participant who could say "that is not yours to decide", and the only one who
|
||||||
|
consistently pushed for a real test over more building.
|
||||||
|
|
||||||
|
The distinction worth encoding: agents may coordinate freely on *what is true* and must not decide *what is
|
||||||
|
permitted*.
|
||||||
|
|
||||||
## Postscript: the one that worked first time
|
## Postscript: the one that worked first time
|
||||||
|
|
||||||
Everything above was found by something failing. One thing did not.
|
Everything above was found by something failing. One thing did not.
|
||||||
|
|||||||
Reference in New Issue
Block a user