From 45e3e86d72a442dbe89ebb4fb1f00fc123147e71 Mon Sep 17 00:00:00 2001 From: Andre Padez Date: Mon, 10 Aug 2026 19:34:48 +0100 Subject: [PATCH] write down what to test, and what is most likely broken MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The serve is the only path now, so this is not a comparison against a fallback. Split by what I have actually driven end to end versus what probes cannot answer. The second list is the real testing: resume from history (never exercised against the serve, and my pick for most likely broken), an idle session, a sidecar restart mid-turn, an officer restart mid-turn, and two conversations at once — that last one because the live event stream is global and a wrong sessionID filter would splice one conversation into another. Known gaps are listed so they do not get reported as bugs, and the one silent failure mode with a single cause — a turn producing nothing at all — points at the credential line from boot, which I have chased twice already. Co-Authored-By: Claude Opus 5 --- docs/opencode-serve-migration-plan.md | 8 ++- docs/opencode-testing-checklist.md | 86 +++++++++++++++++++++++++++ 2 files changed, 92 insertions(+), 2 deletions(-) create mode 100644 docs/opencode-testing-checklist.md diff --git a/docs/opencode-serve-migration-plan.md b/docs/opencode-serve-migration-plan.md index 92208fa9..3ab2a073 100644 --- a/docs/opencode-serve-migration-plan.md +++ b/docs/opencode-serve-migration-plan.md @@ -67,9 +67,13 @@ Each of these cost time to find. None is in the API docs. - **Phase C — server half done.** A message sent while a turn runs is injected with `delivery: "steer"` into the RUNNING turn, verified end to end. No client change was needed: officer's composer already sends mid-turn, and the subprocess path was superseding where the serve steers. -- **Phase D — not started, deliberately.** +- **Phase D — done.** The subprocess is deleted: no `runner.ts`, no `OPENCODE_TURNS` switch, no fallback + engine. Andre called it — nothing depends on OpenCode, so the cost of removing the escape hatch is + near zero and the recovery is git. 818 lines went with it, all of them workarounds for `stdin` being + `/dev/null`. -Not yet lived with. Nothing here has run a real conversation with a person at the other end. +Not yet lived with. **Nothing here has run a real conversation with a person at the other end** — see +`docs/opencode-testing-checklist.md` for what to try and what is most likely to be broken. ## Shape of the work diff --git a/docs/opencode-testing-checklist.md b/docs/opencode-testing-checklist.md new file mode 100644 index 00000000..597759db --- /dev/null +++ b/docs/opencode-testing-checklist.md @@ -0,0 +1,86 @@ +# OpenCode: what to test tonight + +Every OpenCode turn now runs through the serve (`serve-runner.ts`). The `opencode run` subprocess is +deleted, so this is not a comparison against a fallback — it is the only path. + +I have driven each item in **Should already work** end to end through the real chat socket, with a +script rather than a browser. Nothing below has been used by a person in a real conversation, and the +things in **Unproven** are unproven because probes cannot answer them. + +--- + +## Before you start + +```bash +cd ~/projects/officer-suite/platform +pm2 logs officer-opencode --lines 40 # the sidecar's own account of what happened +``` + +On boot you should see three lines: `serve healthy on port …`, `connected the opencode credential to the +api surface`, and the sidecar registering. **If the credential line is missing or says it could not +connect, stop** — paid models will silently do nothing, which is the failure that cost most of an +afternoon (`docs/opencode-fork-decision.md`). + +Rolling back is `git revert` of `a3dbda7` (Phase D) and a restart. There is no config flag any more. + +Alpha is unaffected until you pull. + +--- + +## Should already work — confirm, do not investigate + +Each verified by me end to end. If one fails, that is new information and worth stopping on. + +- [ ] **A plain turn.** Ask for something short. Text arrives. +- [ ] **Streaming.** Ask for something long — "count slowly to 50". Text should appear **progressively**, + not in one block at the end. This is the headline change; the old path could not do it. +- [ ] **A tool call.** "Run `echo hi` with bash." A tool row appears with the command as its arguments, + then its output. +- [ ] **Cost.** The turn ends with a token count attached. +- [ ] **An image.** Drop a screenshot in and ask what it shows. It must actually be described — the + failure mode is a confident answer about nothing, which is what B4 was. +- [ ] **Stop.** Press stop mid-turn. The turn ends **and the conversation stays usable** — send another + message straight after and it should answer. Previously stop destroyed the session. +- [ ] **Mid-turn injection.** While a turn runs, send another message. It should join the RUNNING turn + rather than starting a new one or superseding it. + +## Unproven — this is the actual testing + +- [ ] **Resume from history.** Open an older OpenCode conversation from the list and continue it. This + exercises `resumeSessionId`, which I never tested against the serve. **Most likely thing to be + broken.** +- [ ] **An idle session, an hour later.** Send a message to a conversation you have not touched for a + while. Warm sessions are new here — the subprocess had nothing to go stale. +- [ ] **A sidecar restart mid-turn.** `pm2 restart officer-opencode` while a turn is generating. Expect + the transcript to say the turn stopped. **The turn itself keeps running inside the serve** — that + is intended, not a bug, but nobody has watched what it looks like from the browser. +- [ ] **An officer restart mid-turn.** Different from the above: officer is the relay, the sidecar keeps + committing to `chat_session_events`. On reload the transcript should be intact. This is the one I + would least like to be wrong about. +- [ ] **Two conversations at once.** The live event stream is GLOBAL — one socket carries every session + and `serve-runner` filters on `sessionID`. If that filter is wrong, output from one conversation + appears in another. Two panels side by side is the test. +- [ ] **The Live panel.** A running OpenCode turn should appear, named, and disappear when it ends. +- [ ] **A long turn.** Ten-plus minutes. The old path had watchdogs (inactivity, hard cap) that are gone + with it; the serve has its own ideas about timeouts and I have not found their edges. +- [ ] **A failing tool.** Ask it to run a command that does not exist. The error should land in the + transcript as a failed tool, not as a dead turn. + +## Known gaps, so you do not report them as bugs + +- **No durable replay.** `serve-runner` reads the live stream only. Events are still committed to + `chat_session_events` as they arrive, so the transcript survives — but recovering a turn *this sidecar + process never saw* would need the `?after=` cursor, and that is not built. +- **Notifications/thinking/background tasks.** Not implemented on this harness. Thinking is deliberately + out of scope for both harnesses. +- **`messageCount` shows nothing** for OpenCode rows. The UI renders an `OpenCode` badge instead; not a + gap (`docs/opencode-parity.md`). + +## What is useful to tell me + +For anything that misbehaves: what you did, what appeared, and the last twenty lines of +`pm2 logs officer-opencode`. The sidecar logs its own errors, and a silent failure with clean logs is a +different diagnosis from a loud one. + +If a turn produces **nothing at all** — no text, no error, no spinner ending — check the credential line +from boot first. That specific silence has one cause and I have chased it twice.