write down what to test, and what is most likely broken
The serve is the only path now, so this is not a comparison against a fallback. Split by what I have actually driven end to end versus what probes cannot answer. The second list is the real testing: resume from history (never exercised against the serve, and my pick for most likely broken), an idle session, a sidecar restart mid-turn, an officer restart mid-turn, and two conversations at once — that last one because the live event stream is global and a wrong sessionID filter would splice one conversation into another. Known gaps are listed so they do not get reported as bugs, and the one silent failure mode with a single cause — a turn producing nothing at all — points at the credential line from boot, which I have chased twice already. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -67,9 +67,13 @@ Each of these cost time to find. None is in the API docs.
|
|||||||
- **Phase C — server half done.** A message sent while a turn runs is injected with `delivery: "steer"`
|
- **Phase C — server half done.** A message sent while a turn runs is injected with `delivery: "steer"`
|
||||||
into the RUNNING turn, verified end to end. No client change was needed: officer's composer already
|
into the RUNNING turn, verified end to end. No client change was needed: officer's composer already
|
||||||
sends mid-turn, and the subprocess path was superseding where the serve steers.
|
sends mid-turn, and the subprocess path was superseding where the serve steers.
|
||||||
- **Phase D — not started, deliberately.**
|
- **Phase D — done.** The subprocess is deleted: no `runner.ts`, no `OPENCODE_TURNS` switch, no fallback
|
||||||
|
engine. Andre called it — nothing depends on OpenCode, so the cost of removing the escape hatch is
|
||||||
|
near zero and the recovery is git. 818 lines went with it, all of them workarounds for `stdin` being
|
||||||
|
`/dev/null`.
|
||||||
|
|
||||||
Not yet lived with. Nothing here has run a real conversation with a person at the other end.
|
Not yet lived with. **Nothing here has run a real conversation with a person at the other end** — see
|
||||||
|
`docs/opencode-testing-checklist.md` for what to try and what is most likely to be broken.
|
||||||
|
|
||||||
## Shape of the work
|
## Shape of the work
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,86 @@
|
|||||||
|
# OpenCode: what to test tonight
|
||||||
|
|
||||||
|
Every OpenCode turn now runs through the serve (`serve-runner.ts`). The `opencode run` subprocess is
|
||||||
|
deleted, so this is not a comparison against a fallback — it is the only path.
|
||||||
|
|
||||||
|
I have driven each item in **Should already work** end to end through the real chat socket, with a
|
||||||
|
script rather than a browser. Nothing below has been used by a person in a real conversation, and the
|
||||||
|
things in **Unproven** are unproven because probes cannot answer them.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Before you start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/projects/officer-suite/platform
|
||||||
|
pm2 logs officer-opencode --lines 40 # the sidecar's own account of what happened
|
||||||
|
```
|
||||||
|
|
||||||
|
On boot you should see three lines: `serve healthy on port …`, `connected the opencode credential to the
|
||||||
|
api surface`, and the sidecar registering. **If the credential line is missing or says it could not
|
||||||
|
connect, stop** — paid models will silently do nothing, which is the failure that cost most of an
|
||||||
|
afternoon (`docs/opencode-fork-decision.md`).
|
||||||
|
|
||||||
|
Rolling back is `git revert` of `a3dbda7` (Phase D) and a restart. There is no config flag any more.
|
||||||
|
|
||||||
|
Alpha is unaffected until you pull.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Should already work — confirm, do not investigate
|
||||||
|
|
||||||
|
Each verified by me end to end. If one fails, that is new information and worth stopping on.
|
||||||
|
|
||||||
|
- [ ] **A plain turn.** Ask for something short. Text arrives.
|
||||||
|
- [ ] **Streaming.** Ask for something long — "count slowly to 50". Text should appear **progressively**,
|
||||||
|
not in one block at the end. This is the headline change; the old path could not do it.
|
||||||
|
- [ ] **A tool call.** "Run `echo hi` with bash." A tool row appears with the command as its arguments,
|
||||||
|
then its output.
|
||||||
|
- [ ] **Cost.** The turn ends with a token count attached.
|
||||||
|
- [ ] **An image.** Drop a screenshot in and ask what it shows. It must actually be described — the
|
||||||
|
failure mode is a confident answer about nothing, which is what B4 was.
|
||||||
|
- [ ] **Stop.** Press stop mid-turn. The turn ends **and the conversation stays usable** — send another
|
||||||
|
message straight after and it should answer. Previously stop destroyed the session.
|
||||||
|
- [ ] **Mid-turn injection.** While a turn runs, send another message. It should join the RUNNING turn
|
||||||
|
rather than starting a new one or superseding it.
|
||||||
|
|
||||||
|
## Unproven — this is the actual testing
|
||||||
|
|
||||||
|
- [ ] **Resume from history.** Open an older OpenCode conversation from the list and continue it. This
|
||||||
|
exercises `resumeSessionId`, which I never tested against the serve. **Most likely thing to be
|
||||||
|
broken.**
|
||||||
|
- [ ] **An idle session, an hour later.** Send a message to a conversation you have not touched for a
|
||||||
|
while. Warm sessions are new here — the subprocess had nothing to go stale.
|
||||||
|
- [ ] **A sidecar restart mid-turn.** `pm2 restart officer-opencode` while a turn is generating. Expect
|
||||||
|
the transcript to say the turn stopped. **The turn itself keeps running inside the serve** — that
|
||||||
|
is intended, not a bug, but nobody has watched what it looks like from the browser.
|
||||||
|
- [ ] **An officer restart mid-turn.** Different from the above: officer is the relay, the sidecar keeps
|
||||||
|
committing to `chat_session_events`. On reload the transcript should be intact. This is the one I
|
||||||
|
would least like to be wrong about.
|
||||||
|
- [ ] **Two conversations at once.** The live event stream is GLOBAL — one socket carries every session
|
||||||
|
and `serve-runner` filters on `sessionID`. If that filter is wrong, output from one conversation
|
||||||
|
appears in another. Two panels side by side is the test.
|
||||||
|
- [ ] **The Live panel.** A running OpenCode turn should appear, named, and disappear when it ends.
|
||||||
|
- [ ] **A long turn.** Ten-plus minutes. The old path had watchdogs (inactivity, hard cap) that are gone
|
||||||
|
with it; the serve has its own ideas about timeouts and I have not found their edges.
|
||||||
|
- [ ] **A failing tool.** Ask it to run a command that does not exist. The error should land in the
|
||||||
|
transcript as a failed tool, not as a dead turn.
|
||||||
|
|
||||||
|
## Known gaps, so you do not report them as bugs
|
||||||
|
|
||||||
|
- **No durable replay.** `serve-runner` reads the live stream only. Events are still committed to
|
||||||
|
`chat_session_events` as they arrive, so the transcript survives — but recovering a turn *this sidecar
|
||||||
|
process never saw* would need the `?after=` cursor, and that is not built.
|
||||||
|
- **Notifications/thinking/background tasks.** Not implemented on this harness. Thinking is deliberately
|
||||||
|
out of scope for both harnesses.
|
||||||
|
- **`messageCount` shows nothing** for OpenCode rows. The UI renders an `OpenCode` badge instead; not a
|
||||||
|
gap (`docs/opencode-parity.md`).
|
||||||
|
|
||||||
|
## What is useful to tell me
|
||||||
|
|
||||||
|
For anything that misbehaves: what you did, what appeared, and the last twenty lines of
|
||||||
|
`pm2 logs officer-opencode`. The sidecar logs its own errors, and a silent failure with clean logs is a
|
||||||
|
different diagnosis from a loud one.
|
||||||
|
|
||||||
|
If a turn produces **nothing at all** — no text, no error, no spinner ending — check the credential line
|
||||||
|
from boot first. That specific silence has one cause and I have chased it twice.
|
||||||
Reference in New Issue
Block a user