write down what to test, and what is most likely broken

The serve is the only path now, so this is not a comparison against a fallback.

Split by what I have actually driven end to end versus what probes cannot answer. The second
list is the real testing: resume from history (never exercised against the serve, and my
pick for most likely broken), an idle session, a sidecar restart mid-turn, an officer restart
mid-turn, and two conversations at once — that last one because the live event stream is
global and a wrong sessionID filter would splice one conversation into another.

Known gaps are listed so they do not get reported as bugs, and the one silent failure mode
with a single cause — a turn producing nothing at all — points at the credential line from
boot, which I have chased twice already.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-10 19:34:48 +01:00
co-authored by Claude Opus 5
parent a3dbda7d3b
commit 45e3e86d72
2 changed files with 92 additions and 2 deletions
+6 -2
View File
@@ -67,9 +67,13 @@ Each of these cost time to find. None is in the API docs.
- **Phase C — server half done.** A message sent while a turn runs is injected with `delivery: "steer"` - **Phase C — server half done.** A message sent while a turn runs is injected with `delivery: "steer"`
into the RUNNING turn, verified end to end. No client change was needed: officer's composer already into the RUNNING turn, verified end to end. No client change was needed: officer's composer already
sends mid-turn, and the subprocess path was superseding where the serve steers. sends mid-turn, and the subprocess path was superseding where the serve steers.
- **Phase D — not started, deliberately.** - **Phase D — done.** The subprocess is deleted: no `runner.ts`, no `OPENCODE_TURNS` switch, no fallback
engine. Andre called it — nothing depends on OpenCode, so the cost of removing the escape hatch is
near zero and the recovery is git. 818 lines went with it, all of them workarounds for `stdin` being
`/dev/null`.
Not yet lived with. Nothing here has run a real conversation with a person at the other end. Not yet lived with. **Nothing here has run a real conversation with a person at the other end** — see
`docs/opencode-testing-checklist.md` for what to try and what is most likely to be broken.
## Shape of the work ## Shape of the work
+86
View File
@@ -0,0 +1,86 @@
# OpenCode: what to test tonight
Every OpenCode turn now runs through the serve (`serve-runner.ts`). The `opencode run` subprocess is
deleted, so this is not a comparison against a fallback — it is the only path.
I have driven each item in **Should already work** end to end through the real chat socket, with a
script rather than a browser. Nothing below has been used by a person in a real conversation, and the
things in **Unproven** are unproven because probes cannot answer them.
---
## Before you start
```bash
cd ~/projects/officer-suite/platform
pm2 logs officer-opencode --lines 40 # the sidecar's own account of what happened
```
On boot you should see three lines: `serve healthy on port …`, `connected the opencode credential to the
api surface`, and the sidecar registering. **If the credential line is missing or says it could not
connect, stop** — paid models will silently do nothing, which is the failure that cost most of an
afternoon (`docs/opencode-fork-decision.md`).
Rolling back is `git revert` of `a3dbda7` (Phase D) and a restart. There is no config flag any more.
Alpha is unaffected until you pull.
---
## Should already work — confirm, do not investigate
Each verified by me end to end. If one fails, that is new information and worth stopping on.
- [ ] **A plain turn.** Ask for something short. Text arrives.
- [ ] **Streaming.** Ask for something long — "count slowly to 50". Text should appear **progressively**,
not in one block at the end. This is the headline change; the old path could not do it.
- [ ] **A tool call.** "Run `echo hi` with bash." A tool row appears with the command as its arguments,
then its output.
- [ ] **Cost.** The turn ends with a token count attached.
- [ ] **An image.** Drop a screenshot in and ask what it shows. It must actually be described — the
failure mode is a confident answer about nothing, which is what B4 was.
- [ ] **Stop.** Press stop mid-turn. The turn ends **and the conversation stays usable** — send another
message straight after and it should answer. Previously stop destroyed the session.
- [ ] **Mid-turn injection.** While a turn runs, send another message. It should join the RUNNING turn
rather than starting a new one or superseding it.
## Unproven — this is the actual testing
- [ ] **Resume from history.** Open an older OpenCode conversation from the list and continue it. This
exercises `resumeSessionId`, which I never tested against the serve. **Most likely thing to be
broken.**
- [ ] **An idle session, an hour later.** Send a message to a conversation you have not touched for a
while. Warm sessions are new here — the subprocess had nothing to go stale.
- [ ] **A sidecar restart mid-turn.** `pm2 restart officer-opencode` while a turn is generating. Expect
the transcript to say the turn stopped. **The turn itself keeps running inside the serve** — that
is intended, not a bug, but nobody has watched what it looks like from the browser.
- [ ] **An officer restart mid-turn.** Different from the above: officer is the relay, the sidecar keeps
committing to `chat_session_events`. On reload the transcript should be intact. This is the one I
would least like to be wrong about.
- [ ] **Two conversations at once.** The live event stream is GLOBAL — one socket carries every session
and `serve-runner` filters on `sessionID`. If that filter is wrong, output from one conversation
appears in another. Two panels side by side is the test.
- [ ] **The Live panel.** A running OpenCode turn should appear, named, and disappear when it ends.
- [ ] **A long turn.** Ten-plus minutes. The old path had watchdogs (inactivity, hard cap) that are gone
with it; the serve has its own ideas about timeouts and I have not found their edges.
- [ ] **A failing tool.** Ask it to run a command that does not exist. The error should land in the
transcript as a failed tool, not as a dead turn.
## Known gaps, so you do not report them as bugs
- **No durable replay.** `serve-runner` reads the live stream only. Events are still committed to
`chat_session_events` as they arrive, so the transcript survives — but recovering a turn *this sidecar
process never saw* would need the `?after=` cursor, and that is not built.
- **Notifications/thinking/background tasks.** Not implemented on this harness. Thinking is deliberately
out of scope for both harnesses.
- **`messageCount` shows nothing** for OpenCode rows. The UI renders an `OpenCode` badge instead; not a
gap (`docs/opencode-parity.md`).
## What is useful to tell me
For anything that misbehaves: what you did, what appeared, and the last twenty lines of
`pm2 logs officer-opencode`. The sidecar logs its own errors, and a silent failure with clean logs is a
different diagnosis from a loud one.
If a turn produces **nothing at all** — no text, no error, no spinner ending — check the credential line
from boot first. That specific silence has one cause and I have chased it twice.