From 79da78008ad043ed95aef70e0a5f479dbb845fa0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9=20Padez?= Date: Wed, 12 Aug 2026 16:52:27 +0000 Subject: [PATCH] turn the field report into something an agent can follow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Expands the communications section from a list of what worked into the actual convention: the directory's lifetime and the rule that anything durable must move to docs/ before the merge; numbering, parity as attribution, non-consecutive numbers; slugs; reply-in-a-new-file and the one case where editing your own is right; referring to commits by sha because three remotes carried the same branch names. Records what a handoff must contain, with the verified/assumed split named as the rule that carried the most weight — a handoff confident about something untested is worse than none, because the reader builds on it. Adds a skeleton to copy. Documents termination as the four attempts it actually took, ending at the only checkable version: the exchange pauses when no open item is actionable by a participant. Adds the third state, deferred-with-a-reason, since a two-state protocol forces an agent to lie in one direction. Notes that a stall must be detectable because the human spotted both before either agent did. Adds a review-discipline section — check the enforcement rather than the description, run it against a real machine, a check never seen failing is not evidence, distrust vacuous passes, expect stacked bugs, distrust "inert today", and look at which way unknown resolves. Adds a failure-mode table to pattern-match against, and the git hygiene that bit us, including merge-verify-then-delete, which I got wrong. Closes with session economics, an ordered list of what to build, and the one thing not to automate: agents may coordinate on what is true and must not decide what is permitted. Co-Authored-By: Claude Opus 5 --- docs/two-agent-field-report-2026-08-12.md | 277 ++++++++++++++++++---- 1 file changed, 231 insertions(+), 46 deletions(-) diff --git a/docs/two-agent-field-report-2026-08-12.md b/docs/two-agent-field-report-2026-08-12.md index 4522ea42..4dc2a376 100644 --- a/docs/two-agent-field-report-2026-08-12.md +++ b/docs/two-agent-field-report-2026-08-12.md @@ -62,41 +62,192 @@ building. That instinct was wrong every time. ## The communications paradigm -Agents coordinated through `COMMS//`, a directory of markdown files in the repo itself, deleted when -the feature merged. +Agents coordinated through `COMMS//` — markdown files committed to the repo, alongside the code they +discuss, deleted when the feature merged. -**What it got right:** +**Why a directory in the repo and not chat.** It survives a context window. Both agents' reasoning outlived +the sessions that produced it, a third party could read the argument rather than a summary of it, and it +travels with the branch. Chat has none of those properties, and the owner relaying findings by hand between +two agents at the end of long sessions is the failure this replaces. -- **Numbered, alternating, parity is the author.** Odd = A, even = B. No "your doc"/"his doc", which inverts - depending on who is reading. -- **The slug is the content** — `02-verify-results.md`, not `02-reply.md`. -- **Verified and assumed stated separately.** The single most valuable convention. A handoff that reads as - confident about something untested is worse than no handoff, because the reader builds on it. -- **File:line everywhere.** Costs nothing to write, saves a search. -- **Reply in a new file rather than editing someone else's** — an edited handoff loses what was believed when - a decision was made. -- **It survives a context window.** This is the whole reason it beats chat. Both agents' reasoning outlived - the sessions that produced it, and the owner could read the argument rather than a summary of it. +### The rules, as they ended up -**What broke, in the order we hit it:** +**Location and lifetime.** `COMMS//`. It is a *channel*, not documentation: when the feature +merges, the directory is deleted. Anything that will still be true in a month must be moved to `docs/` or next +to the code **before** the merge, or it is lost. We nearly lost three findings this way and only caught it +because someone checked. -1. **Termination by guess.** A doc ended with "no reply needed unless X", where X was the sender's prediction - about content they had not seen. It ended an exchange with items still open. -2. **Termination by politeness has no exit.** The fix — always reply — meant "nothing to report" obligated - another "nothing to report", forever, at real token cost. -3. **The condition is neither.** What actually terminates an exchange is **no item on the list is actionable - by a participant**. Not "the list is empty" (it never is) and not "I think we're done". -4. **A stalled loop is invisible.** Silence and completion look identical. Open items plus no recent document - is a *detectable* condition; silence is not. -5. **"Deferred with a reason" is a third state.** Three times the honest answer was "mine, and not now", which - is neither open nor done, and only the stated reason distinguishes it from neglect. +**Numbered, alternating, parity is the author.** One agent takes odd numbers, the other even. `01`, `02`, +`03`… Never "your doc" / "his doc", which inverts depending on who is reading. The parity *is* the +attribution, and given that git could not attribute anything (see below), it was the only attribution that +worked. -**The convergence problem.** Late on, both agents independently wrote the same document — same filename, same -three sections — because one had read the other's notes before deleting them. Wasted work, and only caught by -diffing. Nothing in the protocol said who owned a piece of writing. +**Numbers are ordered, not necessarily consecutive.** An agent needing two in a row takes `03` and `05` and +leaves `04` unused, rather than forcing a reply out of the other side to keep the count. A gap is legal and +means "no turn was taken". + +**The slug is the content.** `02-verify-results.md`, `24-resolvememberrun-fails-open.md`. Not `02-reply.md`. +The filename is the index; a reader should know whether to open it without opening it. + +**Reply in a new file. Never edit someone else's.** An edited handoff loses what was believed at the moment a +decision was made, which is usually the thing that explains the decision. + +**Editing your own is allowed if it has not been read** — and say so in the commit. Better than a prediction +standing next to its own correction in two documents. + +**Refer to commits by SHA, never by branch name.** Three remotes were in play with the same branch names on +each; one agent's `origin` was the other's `pertento`. A SHA is the only unambiguous reference, and this cost +real time before it was noticed. + +### What a handoff must contain + +This is the part that carried the most weight, and it is one rule: + +> **State what you verified and what you assumed, separately and explicitly.** + +A handoff that reads as confident about something untested is *worse than no handoff*, because the reader +builds on it. Every serious mistake of the night traces back to something asserted with more confidence than +it had been earned. + +In practice, each document ended up with: + +- **What changed** — with `file:line` throughout. Costs nothing to write, saves the reader a search, and + makes a claim checkable rather than believable. +- **VERIFIED** — what was actually run, on what, with the output. +- **NOT VERIFIED** — stated as prominently as the verified part. `provisionClaudeCli` carried "never executed + anywhere" through four documents, and that label is what eventually made someone run it. +- **What I am least sure of** — the author's own suspicions. One agent listed three; the second was a real + defect, found because it had been pointed at. +- **What I did not do** — so nobody assumes it. "I did not restart anything", "I did not touch the gates". +- **Open items with an owner** — see termination, below. + +### Termination: the rule that took four attempts + +This broke more times than anything else, so the failures are worth listing in order: + +1. **Terminate by guess.** `NO REPLY NEEDED unless the test fails` — a prediction about content the sender had + not seen. It ended an exchange with items open. +2. **Terminate by politeness.** The fix — always reply, even with nothing to say — has no exit. "Nothing to + report" obligates another "nothing to report", indefinitely, at real cost. +3. **Terminate when the list is empty.** Too strong: the list is never empty and will not be for days. +4. **What actually works:** *the exchange pauses when no open item is actionable by a participant.* + +That last one is checkable rather than felt. Everything remaining is either the human's, or deferred with a +stated reason, and either side reopens it by adding an item that is theirs. + +**Three states, not two.** An item is `open` / `done` / **`deferred with a reason`**. Three times the honest +answer was "mine, and not now" — and only the *reason* distinguishes that from neglect. A protocol with two +states forces an agent to lie in one direction or the other. + +**A stall must be detectable.** Silence and completion look identical from outside. Open items plus no +document for N minutes is a condition a machine can watch for; silence is not. The human noticed both stalls +before either agent did, which is the wrong way round. + +### Ownership, which we did not have and needed + +Late on, both agents independently wrote the *same document* — same filename, same three sections — because +one had read the other's notes before deleting them. Pure waste, caught only by diffing the two files. + +Nothing in the protocol said who owned a piece of work. Adding it is cheap: an open item names its owner, and +an agent picking up an unowned item claims it in a document before starting. + +### A skeleton to copy + +```markdown +# NN — + +Commits read: ... Answering ``. + +**Verdict / what changed** — one paragraph, file:line. + +## VERIFIED + + +## NOT VERIFIED + + +## What I am least sure of + + +## What I did not do + + +## Open items +| item | owner | state | +|---|---|---| +| … | me / you / the human | open / deferred (reason) | +``` --- +## The review discipline + +"Verify" turned out to mean something more specific than reading a diff. What actually caught defects: + +**Check the enforcement, not the description.** A document says a check is scoped by user; go read the line +that compares. Twice the description was right and the code did something else — and the author had read +their own description and agreed with it. + +**Run it against a real machine.** Every defect that mattered was found this way, on a first execution. The +categories at the top of this report are not a coincidence. + +**A check that has never been seen failing is not evidence.** A verification script was run against a live, +fully-provisioned account specifically to watch it fail; it reported 8 of 9 failures, which is what made the +later clean result meaningful. Related: a *skipped* test must announce itself, or an unconfigured run reads as +a pass. + +**Distrust vacuous passes.** Three separate times something passed because it had not actually looked: +a subuid scan on a tree with no subuid-owned files; a search root that did not exist, where every check +reports "ok" on finding nothing; and a range scan handed a non-numeric argument. **Any checker whose checks +are "look for X, report ok if absent" must refuse to run when its inputs are wrong**, rather than pass. + +**Expect stacked bugs.** Fixing the visible failure reveals the next one underneath. A container failed on a +mount-point guard; fixing that revealed an ACL traversal denial. An installer failed on the wrong shell; +fixing that would have revealed a root-owned parent directory. Never report "fixed" from a diff — only from a +run. + +**Distrust "it is inert today".** Several things were safe only because a gate was up. That is a statement +about the present, and the entire purpose of the work was to remove the gate. Review inert code as if it were +live, because the commit that makes it live will be reviewed as if it were already correct. + +**Fail closed, and check which way "unknown" resolves.** The most dangerous defect of the night was a resolver +that answered "I could not determine whose turn this is" with *the owner's identity*. Any place where an +unknown collapses into a privileged default is worth a specific look. + +--- + +## Failure modes to expect + +Collected from the night, phrased so an agent can pattern-match against them: + +| pattern | what it looked like here | +|---|---| +| **Author reviews own sentence** | "a wrong answer here must not happen by accident" shipped with that accident | +| **Vacuous pass** | checker with a missing search root printing CLEAN | +| **Stacked bugs** | PG18 mount guard hiding an ACL traversal denial | +| **Inert-today reasoning** | unreachable code reviewed less carefully than reachable code | +| **Unknown resolves to privileged** | failed lookup → run as the owner | +| **Compiler cannot help** | a parameter changing meaning from email to path, both `string` | +| **Guard that cannot fire** | a denylist tested against an object built from an allowlist | +| **Side-effect creation** | `install -d` making a parent `root:root` | +| **Mode bits vs ACLs** | `chown` severing ownership and leaving access | +| **Tail-of-session work** | three of the night's bugs written after hour eight | + +--- + +## Git hygiene for two agents on one branch + +Small, and it bit us repeatedly: + +- **Pull before you push, and expect a race.** Both agents pushed within the same minute more than once; one + rebase was needed mid-review. +- **Merge, verify, *then* delete.** A branch was deleted after an aborted fast-forward — master had moved — + and the commits survived only because git had not yet garbage-collected them. Verify the merge landed before + removing the only ref to it. +- **A doc-only commit still deserves a real message.** These commit messages are the durable record once + `COMMS/` is deleted; several findings in this repo now exist *only* in a commit body. +- **Say which remote.** See the SHA rule above. + ## The background watcher — launch it exactly this way This is the part that was hardest to convey to the second agent, who ended up launching it differently and @@ -205,31 +356,65 @@ cannot attribute a line is not an audit trail. --- -## What I would build, in order +## Session economics, which shape all of the above -1. **Per-agent git identity.** Without it nothing else in the record can be trusted after the fact. -2. **State-based termination.** Each document carries its open-item list; the exchange ends when no item is - actionable by a participant. Machine-checkable, unlike "I think we're done". -3. **Stall detection.** Open items plus no document for N minutes is a condition something can watch for. - This is the failure the human noticed first, both times. -4. **Ownership on work items**, so two agents cannot independently write the same file. -5. **Event delivery instead of polling.** The repo is a Gitea instance the platform already runs; a webhook - removes the watcher, its self-trips and its silent death. +**Idle is free; waking is not.** The watcher costs nothing while it waits. Every wake re-reads the entire +conversation, so a late wake in a long session costs far more than an early one, and the cost grows +monotonically with the session. -## What I would not build +**This argues against one immortal session.** The durable shape is a *short-lived session per event* — the +platform detects a push, spawns an agent with the base SHA and the instruction, it reviews, reports, exits. +State lives in the repo, not in an ever-growing transcript. A ten-hour session is possible and was useful, but +its last hour cost several times its first. -**Do not automate the human out of the arbitration.** Every irreversible decision tonight was the owner's — -lifting the chat gates, deleting an account, choosing between two designs — and every one of those was a -judgement neither agent should have made alone. Twice an agent talked the other out of starting the most -dangerous function at 3am, and both times the argument was one the agents could make to each other but -neither could make *for* the human. - -The objective in `agent-coordination.md` is agents coordinating rather than routing through the human. Tonight -supports that for **execution** and contradicts it for **authority**. The human was not a bottleneck in the -work; they were the only participant who could say "that is not yours to decide". +**Compaction is the real horizon, not session death.** Where sessions persist, the limit is that the earliest +context — usually the most expensive reasoning — degrades to summary first. Anything that must survive belongs +in the repo the moment it is understood, not at the end. --- +## Turning this into a convention + +In order, cheapest and most load-bearing first. + +**1. Per-agent git identity.** Both agents commit from machines configured as the owner, so every commit reads +`Author: ` with a `Co-Authored-By` trailer, for both of them. The owner asked which commit an agent had +written and *neither the log nor the agent could answer from the repository*. An audit trail that cannot +attribute a line is not an audit trail, and everything else here assumes attribution works. +`docs/agent-git-identity.md` describes the fix and has been marked "idea, not implemented" since 2026-08-10. + +**2. `COMMS/` as a checked convention, not a habit.** The numbering, the parity, the verified/assumed split +and the open-item table are all mechanically checkable. A pre-commit hook or a small script that refuses a +malformed handoff would have caught the duplicate document and both stalls. + +**3. State-based termination and stall detection.** Open items with owners, in a machine-readable block; the +exchange pauses when none is actionable by a participant; a watcher notices open items with no document for N +minutes. This is the single biggest quality-of-life gain and it is not hard. + +**4. Event delivery instead of polling.** The repo is a Gitea instance the platform already runs. A webhook +removes the watcher entirely — with its self-trips, its bounded lifetime and its silent death — and replaces +"did I miss a push" with an event that cannot be missed by a process that stopped existing. + +**5. Ownership on work items**, so two agents cannot independently write the same file. + +**6. A durable-notes rule.** `COMMS/` is deleted at merge. Anything still true afterwards moves to `docs/` +*before* the merge, and the merge should refuse if the channel contains unresolved open items. + +## What not to automate + +**The human's arbitration.** Every irreversible decision was the owner's — lifting the chat gates, deleting an +account, choosing between two designs, deciding a directory should stop existing. Each was a judgement neither +agent should have made alone, and in at least two cases an agent talked the other out of a bad idea using an +argument *the human had originally made*. + +`agent-coordination.md` sets the objective as agents coordinating rather than routing through the human. This +night supports that for **execution** and contradicts it for **authority**. The human was not a bottleneck in +the work — they were the only participant who could say "that is not yours to decide", and the only one who +consistently pushed for a real test over more building. + +The distinction worth encoding: agents may coordinate freely on *what is true* and must not decide *what is +permitted*. + ## Postscript: the one that worked first time Everything above was found by something failing. One thing did not.