Files

T

pastilhasandClaude Opus 5 45e3e86d72 write down what to test, and what is most likely broken

The serve is the only path now, so this is not a comparison against a fallback.

Split by what I have actually driven end to end versus what probes cannot answer. The second
list is the real testing: resume from history (never exercised against the serve, and my
pick for most likely broken), an idle session, a sidecar restart mid-turn, an officer restart
mid-turn, and two conversations at once — that last one because the live event stream is
global and a wrong sessionID filter would splice one conversation into another.

Known gaps are listed so they do not get reported as bugs, and the one silent failure mode
with a single cause — a turn producing nothing at all — points at the credential line from
boot, which I have chased twice already.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-08-10 19:34:48 +01:00

5.0 KiB

Raw Permalink Blame History

OpenCode: what to test tonight

Every OpenCode turn now runs through the serve (serve-runner.ts). The opencode run subprocess is deleted, so this is not a comparison against a fallback — it is the only path.

I have driven each item in Should already work end to end through the real chat socket, with a script rather than a browser. Nothing below has been used by a person in a real conversation, and the things in Unproven are unproven because probes cannot answer them.

Before you start

cd ~/projects/officer-suite/platform
pm2 logs officer-opencode --lines 40      # the sidecar's own account of what happened

On boot you should see three lines: serve healthy on port …, connected the opencode credential to the api surface, and the sidecar registering. If the credential line is missing or says it could not connect, stop — paid models will silently do nothing, which is the failure that cost most of an afternoon (docs/opencode-fork-decision.md).

Rolling back is git revert of a3dbda7 (Phase D) and a restart. There is no config flag any more.

Alpha is unaffected until you pull.

Should already work — confirm, do not investigate

Each verified by me end to end. If one fails, that is new information and worth stopping on.

A plain turn. Ask for something short. Text arrives.
Streaming. Ask for something long — "count slowly to 50". Text should appear progressively, not in one block at the end. This is the headline change; the old path could not do it.
A tool call. "Run echo hi with bash." A tool row appears with the command as its arguments, then its output.
Cost. The turn ends with a token count attached.
An image. Drop a screenshot in and ask what it shows. It must actually be described — the failure mode is a confident answer about nothing, which is what B4 was.
Stop. Press stop mid-turn. The turn ends and the conversation stays usable — send another message straight after and it should answer. Previously stop destroyed the session.
Mid-turn injection. While a turn runs, send another message. It should join the RUNNING turn rather than starting a new one or superseding it.

Unproven — this is the actual testing

Resume from history. Open an older OpenCode conversation from the list and continue it. This exercises resumeSessionId, which I never tested against the serve. Most likely thing to be broken.
An idle session, an hour later. Send a message to a conversation you have not touched for a while. Warm sessions are new here — the subprocess had nothing to go stale.
A sidecar restart mid-turn. pm2 restart officer-opencode while a turn is generating. Expect the transcript to say the turn stopped. The turn itself keeps running inside the serve — that is intended, not a bug, but nobody has watched what it looks like from the browser.
An officer restart mid-turn. Different from the above: officer is the relay, the sidecar keeps committing to chat_session_events. On reload the transcript should be intact. This is the one I would least like to be wrong about.
Two conversations at once. The live event stream is GLOBAL — one socket carries every session and serve-runner filters on sessionID. If that filter is wrong, output from one conversation appears in another. Two panels side by side is the test.
The Live panel. A running OpenCode turn should appear, named, and disappear when it ends.
A long turn. Ten-plus minutes. The old path had watchdogs (inactivity, hard cap) that are gone with it; the serve has its own ideas about timeouts and I have not found their edges.
A failing tool. Ask it to run a command that does not exist. The error should land in the transcript as a failed tool, not as a dead turn.

Known gaps, so you do not report them as bugs

No durable replay. serve-runner reads the live stream only. Events are still committed to chat_session_events as they arrive, so the transcript survives — but recovering a turn this sidecar process never saw would need the ?after= cursor, and that is not built.
Notifications/thinking/background tasks. Not implemented on this harness. Thinking is deliberately out of scope for both harnesses.
messageCount shows nothing for OpenCode rows. The UI renders an OpenCode badge instead; not a gap (docs/opencode-parity.md).

What is useful to tell me

For anything that misbehaves: what you did, what appeared, and the last twenty lines of pm2 logs officer-opencode. The sidecar logs its own errors, and a silent failure with clean logs is a different diagnosis from a loud one.

If a turn produces nothing at all — no text, no error, no spinner ending — check the credential line from boot first. That specific silence has one cause and I have chased it twice.

5.0 KiB Raw Permalink Blame History