diff --git a/docs/agent-git-identity.md b/docs/agent-git-identity.md new file mode 100644 index 00000000..c660e38d --- /dev/null +++ b/docs/agent-git-identity.md @@ -0,0 +1,186 @@ +# Per-agent git identity + +**Status: idea, not implemented. Nothing in this document has been built.** Written 2026-08-10 from a +read of the live spawn path; the file:line references were verified against `dc6b623`. + +## The goal + +A team of agents works on this project, sometimes several of them in the same repository at once. Each +one should commit under its own identity, so `git log` answers "which agent wrote this" without anybody +having to remember to say so. + +Today it cannot. Every agent commits as the owner, because every agent *is* the owner as far as the OS +is concerned. + +## How git identity can be overridden at all + +Identity is unverified metadata, not authentication — a default, never a constraint. Anyone who can +commit can claim any name and email, by any of: + +```bash +git -c user.name=X -c user.email=x@y.z commit # per-invocation config +git commit --author="X " # author only; committer stays whoever ran it +GIT_AUTHOR_NAME=X GIT_AUTHOR_EMAIL=x@y.z \ +GIT_COMMITTER_NAME=X GIT_COMMITTER_EMAIL=x@y.z git commit # env; both identities +``` + +Precedence: `--author` > `GIT_AUTHOR_*` env > `-c user.email` > local config > global config. + +Two consequences that shape the design below: + +- **Author and committer are different fields.** `--author` alone leaves the committer as the owner, and + `git log` shows only the author by default. Set both, or the attribution is half-fiction. To read + both: `git log --format='%an <%ae> | %cn <%ce>'`. +- **Environment beats instruction.** Telling an agent "commit as X" in its prompt is a rule it can + forget. `GIT_AUTHOR_*` in the process environment applies to every git invocation in that process + whether or not anyone remembered. The whole point is to make the identity unforgettable rather than + well-intentioned, so this belongs in the environment. + +## Why the obvious approach does not work here + +The first instinct is a PM2 `env` block per agent, or setting `process.env.GIT_AUTHOR_NAME` in the +sidecar at boot. **Both are wrong on this platform**, and for the same reason: + +`officer-agent` is **one process running many concurrent sessions.** `sessions` is a +`Map` (`src/servers/sidecar/claude/claude-manager.ts:213`); each session owns +its own long-lived `query()` and its own warm `claude` child, reused across turns and collected after 30 +minutes idle. A singleton lockfile enforces one sidecar per owner (`user-instance.ts:95-98`). + +So a process-level variable — whether from PM2, from `.env`, or assigned in `user-instance.ts` — is +shared by every agent and every session on the box. It can say "an agent did this". It cannot say which. + +The `ecosystem.config.cjs` entry is bare anyway, which is worth recording since it looks like a place +where env might already be happening: + +```js +{ name: 'officer-agent', script: 'bun', args: 'run src/servers/sidecar/claude/user-instance.ts', watch: false }, +``` + +No `env`, no `cwd`, no `interpreter` — for any app in the file. + +## What the spawn path actually looks like + +The live path is the Agent SDK, not `Bun.spawn`. `claude-manager.ts:320-360`: + +```ts +const q = query({ + prompt: input.gen as AsyncIterable, + options: { + cwd: params.cwd ?? HOST_HOME, + permissionMode: 'bypassPermissions', + pathToClaudeCodeExecutable: CLAUDE_BIN, + env: cleanEnv as Record, + ... + }, +}); +``` + +`cleanEnv` is built once, at `claude-manager.ts:315`, by destructuring three keys back out of the +sidecar's own environment: + +```ts +const { CLAUDECODE: _c, CLAUDE_CODE_ENTRYPOINT: _e, CLAUDE_CODE_SSE_PORT: _s, ...cleanEnv } = process.env; +``` + +That is the whole story: the child gets the sidecar's full `process.env` minus the three nested-session +guards, and nothing is added per turn. + +**This is the good news.** `env` is *already* a per-`query()` option. It is built once today, but there +is no structural reason it has to be — which makes `claude-manager.ts:315` the single injection point +for everything below. + +### What identity exists today + +Almost none, and none of it at the OS level. + +- `sessionKey` — officer's uuid, the key in the `sessions` map. Reaches the child only as a transport + field on the pushed message. +- **Agent name and persona are prompt-only.** `buildAgentPrompt` + (`src/servers/api/agents/agent-runner.ts:71-79`) inlines the agent's `AGENT.md` into the *first user + message*. There is no `systemPrompt`, no `--agents`, no per-agent settings file. +- The one durable per-agent handle is the working directory: `getAgentRunsDir(agent.dirName)` + (`agent-runner.ts:144`), deliberately shared across all runs of that agent so the CLI groups their + transcripts. + +`grep -rn "GIT_AUTHOR\|GIT_COMMITTER" src/` returns nothing. Verified. Any commit an agent makes today +is attributed to whatever `~/.gitconfig` says — the human owner, identically for every agent. + +## The proposed change + +Three edits, all on the claude path: + +1. **`src/servers/sidecar/protocol.ts`** — add an optional `gitIdentity: { name: string; email: string }` + to `ClaudeSpawnStreamingParams` (the type begins at line 170). Optional so every existing caller is + untouched. +2. **`claude-manager.ts:315`** — build `cleanEnv` per session rather than once, merging the identity in + as all four variables when present: + ```ts + ...(params.gitIdentity && { + GIT_AUTHOR_NAME: params.gitIdentity.name, + GIT_AUTHOR_EMAIL: params.gitIdentity.email, + GIT_COMMITTER_NAME: params.gitIdentity.name, + GIT_COMMITTER_EMAIL: params.gitIdentity.email, + }), + ``` +3. **`agent-runner.ts:144`** — populate it from `agent.dirName`, next to where the cwd pin is already + derived from the same field. + +Chat sessions (`api/chat/websocket.ts`) would pass nothing and keep committing as the owner, which is +almost certainly right: a chat turn is the human driving directly. + +Use a domain that is actually controlled — `@officer.dev` — so Gitea can be made to map or +deliberately not-map these authors later. + +## Known limitations of the proposal + +### opencode cannot do this at all + +Since the `serve` migration (`a3dbda7`, phase D) there is **no process spawned per turn**. One shared +`opencode serve` starts at sidecar boot (`src/servers/sidecar/opencode/index.ts:112-116`) with no `env` +key — full inheritance — and turns are driven over HTTP against it, with per-turn cwd carried as a +request header (`serve-runner.ts:155`). Every session shares that one process environment. + +Per-agent git identity on the opencode path therefore requires either a serve per agent, or an upstream +API field. Neither is a small change, and this document does not propose one. + +### Attribution is not isolation, and isolation is the real problem + +There is **no filesystem isolation** between agents. They share one real `HOME` +(`HOME_DIR=/home/pastilhas`), one `~/.claude`, one credential store; `user-instance.ts:75-78` says this +outright, and it is the stated reason `chat` is an `execution` capability that can never be granted. +`grep -ril worktree src/` returns nothing — worktrees are used nowhere. + +cwd is the only per-session variation and it is not a boundary, since absolute paths escape it freely. + +So two agents told to work on the same repository will share one working tree: fighting over +`index.lock`, staging each other's half-finished edits, interleaving commits. Per-agent identity makes +that **legible after the fact**. It does nothing to prevent it. + +If agents are genuinely to work the same repo concurrently, the isolation question is the larger and +more urgent one — a worktree or a clone per agent — and per-agent identity composes naturally with it +(a worktree per agent is also the cleanest place to put a per-worktree git identity). + +### If worktrees do arrive, note this trap + +`git worktree` shares one `.git/config`, so `git config --local user.email` in one worktree changes it +for **all** of them. Per-worktree config needs `extensions.worktreeConfig true` and then +`git config --worktree user.email …`. Environment variables sidestep the whole issue, which is another +argument for the env approach above. + +## Deliberately not proposed + +- **Signing.** Per-agent SSH signing keys would make attribution unforgeable rather than conventional. + For a trusted local fleet where every agent already runs as the owner with + `permissionMode: 'bypassPermissions'`, an agent that wanted to forge another's identity has far easier + routes. Revisit only if agents stop being equally trusted. +- **`Co-Authored-By:` trailers.** Useful when a human and an agent genuinely share a commit, and both + Gitea and GitHub attribute them. Orthogonal to this, and a prompt-level convention rather than an + environment one. + +## Open questions + +- Should pipeline steps (`pipeline-executor.ts`) get an identity too, or only named agents? +- Is `dirName` the right identity, or should an agent's `AGENT.md` declare its own name and email — so + the identity is authored where the persona is, rather than derived from a directory? +- Does anything downstream — Gitea webhooks, activity feeds, the dashboards — assume commits belong to + the owner and break when they do not? diff --git a/docs/opencode-api-2-assessment.md b/docs/opencode-api-2-assessment.md new file mode 100644 index 00000000..d04fe3ac --- /dev/null +++ b/docs/opencode-api-2-assessment.md @@ -0,0 +1,457 @@ +# OpenCode's newer API — what it is, what it would cost, what it buys + +Written 2026-08-11 against **opencode 1.18.16**, from three sources: the running server's own OpenAPI +document (`GET /doc` on `opencode serve`), live probes against a real serve, and upstream docs/npm. +Every claim below is marked by where it came from. Measurements were taken on the local serve +(port 49698, the `officer-opencode` sidecar's own) and cleaned up afterwards — the session store is +back to the 50 rows it started with. + +Read this before starting any opencode work. Two live defects fell out of writing it (§1), and the +naming is actively misleading (§2). + +--- + +## 1. Two live defects, found while measuring + +Neither is a migration concern. Both are broken right now, in production, and both are consequences of +being half-migrated. + +### 1a. Every OpenCode conversation created since 2026-08-10 opens EMPTY + +Since Phase D, turns run through `POST /api/session/{id}/prompt`, so the session belongs to the newer +engine. But `loadOpenCodeSession` reads the transcript through the legacy route +(`client.ts:51` → `GET /session/{id}/message`). + +**The two surfaces are mutually blind.** Measured, both directions, on a session created via `/api` and +run to completion with a real model reply: + +| read | api-created session | legacy-created session | +|---|---|---| +| `GET /session/{id}/message` (what we call) | **`[]` — 0 messages** | 200, full transcript | +| `GET /api/session/{id}/message` | 200, 3 messages | **500** | +| `GET /session/{id}` (the record) | 200, title + directory | 200 | + +So the row appears in the list with its title and directory, and opens with nothing in it. And the +inverse is equally true: switching the reader to `/api` without keeping the old one would empty every +conversation from before 2026-08-10. + +The fix is not "swap the endpoint" — it is "route by which engine owns the session", and there is no +field that says so. The one usable discriminator found tonight is that the legacy read returns `[]` +rather than erroring. + +### 1b. The session list silently truncates at 50 + +`GET /api/session` defaults to **50 rows** and returns a `cursor.next`. Measured: with 50 sessions in +the store the list returns 50 *and still offers a next cursor*; adding a 51st and asking `?limit=200` +returns 51 (and `limit` is capped at 100 — 200 is accepted for the list but `/history` rejects >100 +with `Expected a value less than or equal to 100`). + +`client.ts:36` sends neither `limit` nor `cursor`, so **once the store passes 50 sessions the oldest +stop appearing in `/chat`**. The local store is at exactly 50 today. This is in code shipped this +morning (`adaaba6`). + +The same endpoint takes `directory=` — verified filtering correctly (`?directory=/tmp/oc-cap` → 11 +rows, all in that directory). We fetch everything and filter client-side in `opencode-sessions.ts:49`. +Pushing the filter down fixes the normal case and brings `search=`, `order=`, `project=` with it. + +--- + +## 2. The naming, because "API 2.0" means two different things + +There is no version string "2.0" in the running server. `GET /doc` self-reports +`{"openapi":"3.1.0","info":{"title":"opencode","version":"1.0.0"}}`. What actually exists: + +| | **legacy** | **the `/api/*` surface** | **OpenCode 2.0 beta** | +|---|---|---|---| +| where | in 1.18.16 | in 1.18.16 | separate product, binary `opencode2`, npm `@next` | +| routes | 111 paths | 51 paths | ~100 paths, still moving | +| operationIds | `session.list` | **`v2.session.list`** | — | +| we use it | reads: transcript, delete, rename | writes: every turn since 2026-08-10 | not at all | +| docs | opencode.ai/docs/server (stale — never mentions `/api/*`) | undocumented publicly | opencode.ai/v2/docs | + +So "API 2.0" most likely means **the `/api/*` surface — which we already run on for turns**. Its +operation ids are literally `v2.*`. It is not something to adopt; it is something to *finish*. + +Two qualifications, both from the source at tag `v1.18.16`: + +- **Upstream calls it experimental.** `packages/protocol/src/api.ts` titles it `"opencode HttpApi"`, + version `"0.0.1"`, described as *"Experimental HttpApi surface for selected instance routes"*, with + every group annotated the same way. Meanwhile `/session/*` is the surface the public docs actually + document, and it is not deprecated. The internal direction is unambiguous; the external commitment is + nil. +- **`session.next` is the event family of that rewritten engine, and the name is already dead + upstream.** It arrived in **1.15.0** (PR #27415, "Add Effect-native core event system", merged + 2026-05-15) as an interim prefix. On the `v2` branch all 36 session events have dropped `.next.` — + `session.step.started`, `session.text.delta` — along with renames: `agent.switched` → + `agent.selected`, `model.switched` → `model.selected`, `prompted` → `prompt.promoted`. Those renames + are **v2-branch only**; the 1.x line we run still emits `session.next.*`. Code against + `session.next.*` today, but put the names behind one mapping table, because they are scheduled to + change wholesale. + +Same for the `v2` suffix itself. `packages/schema/AGENTS.md`: *"V1 coexistence is temporary… delete the +V1 subtree when the legacy runtime is retired"* and *"Do not preserve `V2` as the permanent name for the +replacement architecture."* Both halves of today's naming are transitional. + +**OpenCode 2.0 the product is a different question**, and the answer tonight is not yet: the beta docs +carry the banner *"we may wipe your data, things may break, and APIs, configuration, and plugin APIs +may change"*, releases ship ~6/day, and the migration guide states three intentional breaking changes +(plugin API, server API contracts, TUI config), with *"Integrations that call the V1 server API must +migrate to the V2 API"*. No deprecation date for the legacy surface is published anywhere. + +Two facts worth knowing regardless: + +- **The repo moved.** `github.com/sst/opencode` 301s to **`github.com/anomalyco/opencode`**. Every npm + package now points there. No announcement was found explaining it. +- **There is already a typed client for the surface we run.** `@opencode-ai/sdk@1.18.16` ships two + generated clients: the default export covers legacy only, and **`@opencode-ai/sdk/v2` covers all 51 + `/api/*` routes**. We have no opencode dependency at all today — every call is hand-rolled `fetch`. + +--- + +## 3. What we call today + +Two of our processes talk to one serve, with no shared client. + +**Sidecar (`src/servers/sidecar/opencode/`) — already on `/api/*`:** `POST /api/session` +(`serve-runner.ts:222`), `POST …/model` (`:236`), `POST …/prompt` (`:266`, `:199`), `POST …/interrupt` +(`:328`), `GET /api/event` (`:86`), `GET /api/health` (`index.ts:79`), +`POST /api/integration/{provider}/connect/key` (`connect-credential.ts:66`). + +**API server (`src/servers/api/chat/opencode/client.ts`) — still legacy:** `GET /session/{id}` (`:45`), +`GET /session/{id}/message` (`:51`), `DELETE /session/{id}` (`:57`), `PATCH /session/{id}` (`:62`), +plus `GET /config/providers` for the model list (`list-models.ts:58`). The one exception is +`GET /api/session` for the list (`:36`), moved this morning. + +51 routes exist. We call 7. + +--- + +## 4. What the newer surface has that we don't use + +### 4a. Adding context to a turn that is already running + +The capability the subprocess path could never have, and the reason the migration happened. + +``` +POST /api/session/{id}/prompt +{ "id": "msg_…", "prompt": { "text", "files": [{uri,name,description,source}], + "agents": [{name,source}] }, + "delivery": "steer" | "queue", "resume": true|false } +``` + +Spec description: *"Durably admit one session input and schedule agent-loop execution unless resume is +false."* + +- **`delivery: "steer"` injects into the RUNNING turn** — the model takes the new text as part of the + work in flight. No kill, no restart, no lost context. We already send it (`serve-runner.ts:199`) but + only on the accidental path: a message that happens to arrive mid-turn. Nothing in the UI *asks* for + it, and nothing distinguishes "add this to what you're doing" from "here's my next message". +- **`delivery: "queue"`** runs after the current turn. It must be stated explicitly — **the field + defaults to `steer`** — or two quick messages merge into one turn (`serve-runner.ts:268`). +- **`prompt.files[]`** attaches content to that same input; measured last night, it must be a `data:` + URI (a `file://` one is accepted with 200 and dies inside the provider). Each attachment also takes + a `description`, which we don't send. +- **`prompt.agents[]`** attaches an agent to the input. Unused, unexplored. +- **`id`** lets the caller mint the `msg_…` id, which is how a send survives a retry without + double-posting. We let the server mint it and therefore can't. + +Measured: the POST returns in **22 ms** with `{"admittedSeq":1,"id":"msg_…","delivery":"queue"}`. It is +an admission receipt, not a turn — and `admittedSeq` is the durable cursor for everything that follows. + +### 4b. Surviving a restart mid-turn — verified working + +``` +GET /api/session/{id}/event?after= "Replay durable events after an aggregate sequence, + then continue with new durable events." +GET /api/session/{id}/history?limit=&after= "Read one finite page of public durable Session events + after an exclusive aggregate sequence." +``` + +Driven end to end tonight on a real turn (free model, "Reply with exactly: hi"): + +``` +seq 1 session.next.prompt.admitted seq 6 session.next.context.updated +seq 2 session.next.prompted seq 7 session.next.step.started +seq 3 session.next.model.switched seq 8 session.next.text.started +seq 4 session.next.prompt.admitted seq 9 session.next.text.ended +seq 5 session.next.prompted seq 10 session.next.step.ended +``` + +`?after=5` returned exactly 6–10. `GET …/event?after=7` replayed 8, 9, 10 and then held the socket open +for more. Every durable event carries `{aggregateID, seq: integer, version}`, so `after=` is that +integer. This is the documented, working answer to the gap Phase B left open and +`docs/opencode-testing-checklist.md` calls the most likely thing to be broken. + +The upstream implementation (`packages/core/src/event.ts`, `durable()`) makes three things explicit +that matter for building on it: + +- `after` is an **exclusive** lower bound on the durable seq, and the aggregate is the session. + Omitting it replays the session from 0. +- **Replay-then-live is gap-free by construction**: it reads `WHERE seq > after ORDER BY seq ASC`, + advances its cursor to the last row, and on every wake re-reads *the database* rather than draining a + pubsub buffer. Sequences are strictly monotonic and contiguous per session, enforced with explicit + `Sequence mismatch` / `Replay diverged` errors. +- **The first cursor is free.** `POST …/prompt` returns `{admittedSeq, id, sessionID, prompt, delivery, + timeCreated, promotedSeq?}` — measured at 22 ms — and `admittedSeq` feeds straight back as `after`. + +Note the two cursor kinds are unrelated: the session *list* uses an opaque base64url cursor +(`cursor.previous` / `cursor.next`), this one is a plain integer. + +**But the two streams are not interchangeable, and the schema says why.** `SessionDurableEvent` is a +`oneOf` of exactly 28 members, and the five it omits are `text.delta`, `tool.input.delta`, +`reasoning.delta`, `compaction.delta` and the retry error. **Deltas are live-only by design; the +durable log stores whole values.** So a client that wants both token streaming and restart recovery +must read both streams: the global live one for deltas, the per-session durable one for the replayable +spine. Last night's 13-vs-21 event count was this same fact, found by counting instead of by reading. + +### 4c. Knowing what is running, without having started it + +``` +GET /api/session/active "Retrieve foreground Session drains currently owned by this OpenCode + process. Sessions absent from the result are inactive." +POST /api/session/{id}/wait "Wait for a session agent loop to become idle." +``` + +Today "what is running" is an in-memory map in our sidecar (`serve-runner.ts:66`). Restart the sidecar +and the truth is gone — which is why `/chat/live` can be wrong after a restart. `session/active` is the +server's own answer and survives us. + +### 4d. Permissions and questions — nothing in officer models this + +``` +GET|POST /api/session/{id}/permission POST …/permission/{requestID}/reply +GET /api/permission/saved DELETE /api/permission/saved/{id} +GET /api/session/{id}/question POST …/question/{requestID}/reply | /reject +``` + +Plus `permission.v2.asked` / `question.v2.asked` events (the v1 families still exist alongside; the +only `deprecated: true` operation in the entire document is `POST /session/{id}/permissions/{id}`). + +The two "v2"s are not the same kind of change, which matters if we implement one of them: + +- **Permissions v2 is a real contract change.** A rule goes from `{permission, pattern, action}` to + `{action, resource, effect}`; a request from `{permission, patterns[], metadata, always[], tool?}` to + `{action, resources[], save?[], metadata?, source?}`, with the tool linkage becoming a tagged union + `source: {type:"tool", messageID, callID}`; and the reply loses its free-text `message`. The public + V2 docs say the same in config terms: *"Do not use `permission`, `bash`, or `task` in V2 + configuration."* +- **Questions v2 is a re-homing.** Field shapes are byte-identical to v1 — `questions[]` of + `{question, header, options[], multiple?, custom?}`, answers as `string[][]`. Only the namespace and + event names changed. + +Which family a 1.18.16 agent actually emits is worth measuring before building UI: the manifest the +`/api` protocol is *built* from excludes the v1 families, but the server wires the **full** manifest +(`makeApi({definitions: EventManifest.Latest.values()})`), which is why both appear in the `/api/event` +union on our own `/doc`. + +An opencode agent that wants consent, or that asks a question mid-turn, gets no answer from officer. We +don't subscribe to those events and have no route to reply on. Claude's harness runs +`--dangerously-skip-permissions`, so this has never been modelled for either harness. Largest single +behavioural gap. + +### 4e. Undo, compaction, context + +``` +POST /api/session/{id}/revert/stage {messageID, files?} …/revert/commit …/revert/clear +POST /api/session/{id}/compact GET /api/session/{id}/context +``` + +Stage a revert to a message, then commit or discard. Explicit compaction with +`compaction.started/delta/ended` events, and a readable context state. Officer has none of this. + +### 4f. The rest + +`GET /api/agent`, `/api/skill`, `/api/command`, `/api/model`, `/api/provider`, `/api/fs/{list,find,read}`, +`GET|POST /api/pty` (+`connect`, `connect-token`), `POST /api/session/{id}/agent` (switch agent +mid-session), `/api/reference`, `/api/location`, `/api/integration`, `/api/credential/{id}`. + +`GET /api/model` and `/api/provider` are the `/api` equivalents of the `/config/providers` call our +model list is built on (87 models locally). `/api/pty` overlaps our own pty sidecar. + +--- + +## 5. What the event stream carries that we drop + +Our mapper recognises 18 names and maps 7. The server emits **130 event type strings**, 32 in the +`session.next.*` family plus eight plain `session.*` (`idle`, `status`, `error`, `compacted`, +`created`, `deleted`, `updated`, `diff`). + +| dropped | what it would give | +|---|---| +| `reasoning.started/delta/ended` | thinking, streamed — we show none for opencode | +| `tool.input.delta` / `.started` / `.ended` | a tool call rendering as its arguments arrive | +| `tool.progress` | long tools reporting instead of appearing hung | +| `shell.started/ended` | shell commands as a first-class thing | +| `compaction.*` | telling the user the context was compacted | +| `revert.*` | §4e | +| `retried` | a retry that currently looks like a stall | +| `prompt.admitted` / `prompted` | acknowledgement — the exact window where silence has twice cost an afternoon | +| `session.idle` | the real turn-end signal (see below) | + +We end a turn on `step.ended` with `finish !== 'tool-calls'` (`serve-runner.ts:139`), because there is +no turn-ended event in what we read. `session.idle` looks like what that rule approximates, and it is +not in our `KNOWN` set. + +--- + +## 6. Two silent-failure modes, both reproduced tonight + +Both produce the identical signature — `prompt.admitted`, `prompted`, then **nothing, forever**: + +1. **No credential connected** for the `/api` surface. Already known and fixed at boot + (`connect-credential.ts`), but the failure has no error. +2. **No model on the session and no server default.** New tonight: my first probe sat at + `admitted → prompted` and stopped. `GET /config` reports `model: None`, and the session had no model + because I hadn't set one. `POST …/model` then re-prompting produced the full 10-event turn above. + +Our runner only sends `POST …/model` when `params.model` is set (`serve-runner.ts:231`). **A turn sent +with no model, against a serve with no configured default, hangs silently.** Worth an explicit check. + +--- + +## 7. What finishing the migration would cost + +- **Both readers stay.** §1a: `/api` reads 500 on legacy-owned sessions, legacy reads `[]` on + api-owned ones. Routing by ownership is required, and no field declares ownership. +- **Delete and rename cannot move.** `/api/session/{sessionID}` is **GET only**; `DELETE` and `PATCH` + exist only on the legacy route (spec-verified, and a live `DELETE` returned 200). +- **The transcript shape differs.** Legacy items are `{info:{role,…}, parts:[…]}` — what + `opencode-sessions.ts:81` parses. `/api` items are + `{id, time, type:'assistant', agent, model:{id,providerID,variant}, content:[{type:'text',id,text}], + finish, cost, tokens}`. A second mapper, or a shared normaliser. +- **The SSE parser needs to grow up.** `serve-runner.ts:89` is `data:`-only: no `event:`, no `id:`, no + comments, no `retry:`, no multi-line frames, fixed 1 s reconnect with no backoff. A cursored stream + must resume at `?after=`, not restart. +- **Two envelope unwrappers and three hand-written type sets** (`serve-runner.ts:161`, `client.ts:38`; + types pinned by comment to two different opencode versions, 1.17.9 and 1.18.16). This is the part a + dependency would delete outright — see below. +- **Stale comments in at least nine files** still describe the deleted `opencode run` subprocess path + (`protocol.ts:198`, `serve-events.ts:5`, `connect-credential.ts:24`, `index.ts:137`, + `websocket.ts:454`, `chat.ts:127`, `list-models.ts:75`, `sidecar-server.ts:8`, `send-opencode.ts:7`). + Two of them actively lie: they say turns read `auth.json` and don't depend on the credential connect. + They now do. + +Unrelated but found while inventorying: the settings UI writes provider keys to `~/.pi/agent/auth.json` +(`chat-providers.ts:10`) while the credential connect reads `~/.local/share/opencode/auth.json` +(`connect-credential.ts:29`). Two different files. + +--- + +## 7b. The SDK is generated from the document we have been reading by hand + +`@opencode-ai/sdk@1.18.16` (published 2026-08-10, versioned in lockstep with the CLI) is built by +`packages/sdk/js/script/build.ts`, which runs opencode's own `generate` to produce the OpenAPI document +and feeds it to `@hey-api/openapi-ts`. **It is generated from the same `/doc` we probed**, which is +about as good a guarantee of shape-agreement as exists. + +It ships two clients. The default export is the legacy surface. `@opencode-ai/sdk/v2` is ours: + +```ts +import { createOpencodeClient } from '@opencode-ai/sdk/v2'; + +const client = createOpencodeClient({ baseUrl }); +const admitted = await client.v2.session.prompt({ sessionID, prompt: { text }, delivery: 'steer' }); +const events = await client.v2.session.events({ sessionID, after: admitted.data.admittedSeq }); +for await (const ev of events.stream) { /* ev.type, ev.durable.seq */ } +``` + +`client.v2.session.*` covers list/create/active/get/switchAgent/switchModel/prompt/compact/wait/ +context/history/events/interrupt/message(s); there is also `client.v2.event.subscribe`, +`client.v2.permission.*`, `client.v2.question.*`, `fs`, `model`, `provider`, `agent`, `skill`, `pty`. +`createOpencodeClient` takes `directory` and injects it as both the `x-opencode-*` headers and the +`location[directory]` query param — the thing we hand-roll in two places. + +That would delete: our hand-rolled SSE reader, both envelope unwrappers, three hand-written type sets, +and the model-id string splitting. It is a dependency change, and installs here are frozen, so it is a +deliberate `bun install --no-frozen-lockfile` plus a read of the lockfile diff. Worth noting the +package's only dependency is `cross-spawn`. + +Not to be confused with two siblings the v2 docs mention: `@opencode-ai/sdk-next` is marked private and +is not on npm, and `@opencode-ai/client` is a private generation target for the beta line. + +--- + +## 8. What we'd gain immediately + +Ordered by value over effort. 1–3 are bug fixes, not features. + +1. **Transcripts that aren't empty** — route the read by session ownership. This is broken in + production now. (§1a) +2. **A list that doesn't stop at 50**, filtered server-side by `?directory=`. One call site. (§1b) +3. **A turn that can't hang silently** — set a model explicitly, or check `/config` for a default, and + say so out loud when neither exists. (§6) +4. **"Add to what you're doing" as a real control** — `delivery: "steer"` on a deliberate trigger + rather than only when a message happens to land mid-turn. The plumbing already exists. (§4a) +5. **Idempotent sends** — mint our own `msg_…`. One field. (§4a) +6. **Restart recovery** — subscribe `?after=` alongside the live stream. Verified working. (§4b) +7. **A truthful live panel** — `GET /api/session/active`. (§4c) +8. **Richer streaming for free** — reasoning deltas, tool-input deltas, tool progress, retries. Already + arriving on the socket we already read, and dropped in a `default:` case. (§5) +9. **Explicit compaction and context** instead of a long conversation quietly getting more expensive. + +Then the two that are real features needing UI: **permissions/questions** (§4d) and **revert** (§4e). + +--- + +## 9. What this does NOT get us + +- It does not retire the legacy surface: delete, rename and every pre-2026-08-10 transcript stay there, + with no deprecation date published. +- It does not touch the Claude harness — a different sidecar, a different protocol. Every gain above + lands on one harness only, while the chat UI assumes the two behave alike. +- It does not put us on OpenCode 2.0. Note the direction of travel there: the beta **removes** + `/api/session/{id}/history` and `/api/session/{id}/event` — the two durable routes item 6 depends on + — replacing them with `GET /api/experimental/session/{id}/log?after=&follow=`. Same idea, new path, + `experimental/` prefix. So item 6 is worth doing *and* worth writing behind one function. + +--- + +## 10. Open questions + +1. Is there a field that says which engine owns a session? Tonight's only discriminator is behavioural + (legacy returns `[]`). If not, we need our own record — we already store `sessionKey → ses_…` in + `opencode/state.ts` and could record the surface with it. +2. Which permission/question family does a 1.18.16 agent actually emit? Both are declared and both + appear in our `/doc`, because the server wires the full manifest. Measure before building UI. +3. What is `/api/*`'s auth story? The spec declares no `securitySchemes` yet every route declares a + `401`. The v1 docs describe HTTP Basic via `OPENCODE_SERVER_PASSWORD`; we run with none, on + loopback. In the 2.0 beta this is formalised as basic auth read from + `~/.local/state/opencode/service.json`. +4. Does `@opencode-ai/sdk/v2` work against 1.18.16 exactly? It is generated from this exact server's + OpenAPI output and versioned in lockstep, so it should — but nobody here has run it. +5. When did `/api/*` first appear in the 1.x line? UNKNOWN; the changelog names no `/api/` additions. + The event family underneath it landed in 1.15.0. And no dated removal plan exists for `/session/*`, + `permission.asked/replied` or `question.*` — only the undated internal intent quoted in §2. +6. Will the `v2`-branch event renames reach the 1.x line, or only ship with OpenCode 2.0? No merge + found, no statement either way. This decides whether the mapping table in §2 is a one-off or a + permanent seam. +7. Is a durable `seq` stable across a server restart or a session move? It is a database column, so it + should be, but no durability guarantee is documented and we have not tested it. Item 6 in §8 depends + on the answer. + +Version state at the time of writing: **1.18.16 is the newest release** (2026-08-10) and contains +nothing API-facing. The active stream is the 2.0 beta, cutting releases continuously — the most recent +was published hours before this file was written. + +--- + +## 11. Sources + +- Live: `GET /doc` on `opencode serve` 1.18.16 (162 paths, 51 under `/api/`), plus the probes recorded + above against the local sidecar's serve on port 49698. +- Code: `src/servers/sidecar/opencode/*`, `src/servers/api/chat/opencode/*`, `opencode-sessions.ts`, + `list-models.ts`, `send-opencode.ts`. +- Upstream, docs: opencode.ai/v2/docs/migrate-v1, opencode.ai/v2/docs, opencode.ai/v2/docs/permissions, + opencode.ai/docs/server, opencode.ai/changelog. +- Upstream, source at tag `v1.18.16` in **github.com/anomalyco/opencode** (formerly `sst/opencode`, + which 301s): `packages/protocol/src/api.ts` and `groups/session.ts` (the surface's own "experimental" + self-description, the `after` parameter), `packages/schema/src/session-event.ts` (`DurableDefinitions` + vs `Definitions` — the delta exclusion), `packages/schema/src/{permission,question}.ts` and their + `v1/` counterparts, `packages/schema/src/session-input.ts` (`admittedSeq`), `packages/schema/AGENTS.md` + (the V1/V2 naming intent), `packages/core/src/event.ts` (replay-then-live), `packages/sdk/js/script/ + build.ts` and `src/v2/client.ts`. PRs #27415 (the engine landing in 1.15.0), #33993, #35217, #35229 + (the renames). +- npm: `@opencode-ai/sdk` 1.18.16, `@opencode-ai/client@next`. +- Prior art in this repo: `docs/opencode-parity.md`, `-fork-decision.md`, `-serve-migration-plan.md`, + `-serve-path.md`, `-testing-checklist.md`, `-phase0-review.md`, `-phase1-report.md`, + `-phase1-review.md`. diff --git a/docs/opencode-fork-decision.md b/docs/opencode-fork-decision.md index 58f250a6..14f4ab4b 100644 --- a/docs/opencode-fork-decision.md +++ b/docs/opencode-fork-decision.md @@ -5,7 +5,8 @@ > The conclusion is reversed below. The mistake is written up rather than deleted, because the shape of > it is the useful part. -**Decision: the fork is worth taking. Not started, and gated on one open question.** +**Decision: the fork is worth taking, and it is no longer blocked.** Not started; the blocker turned out +to be a missing credential and is fixed — see RESOLVED below. --- @@ -41,19 +42,81 @@ Steer and queue are exactly the two features we hand-built for Claude and that A changer. Having them as primitives, plus a resumable per-session cursor that mirrors officer's durable replay, is a strong argument for migrating. -## The open question, which is now the only blocker +## RESOLVED — it was a missing credential, not a bug -**Why does `claude-sonnet-4-6` run under `opencode run --model` and silently die under the new API?** +Andre said he had a paid Zen key working in his terminal and suggested it simply was not set up here. He +was right, and this is the second wrong conclusion I reached on this page. -Until that is answered, the migration cannot be adopted: model choice is a user-facing control, and a -harness where picking the wrong model produces silence rather than an error is worse than one without -streaming. Things to try, cheapest first: +**The new pipeline has its own credential store, separate from `auth.json`.** `opencode run`, the CLI and +the legacy `/session` surface all read `~/.local/share/opencode/auth.json`, which holds the Zen key — so +they reach paid models. The `/api/*` surface reads integrations instead (`/api/integration`, +`/api/credential`), and ours had **none connected**. With no credential it silently fell back to what +needs none, which is exactly the free tier. -1. `variant` on `ModelRef` — it is the one field of the three we never set. -2. Compare `/config/providers` against what `run` resolves, to see whether the ids differ in form. -3. Whether it is auth-scoped: alpha's default model failed `401 missing_api_key` on a *different* - provider (`nano-gpt`), which proves the new pipeline surfaces provider auth errors properly when it - gets that far — so sonnet's silence is not a generic auth failure. +The fix was one call, and it persists across a serve restart (verified — a paid model still ran after +`pm2 restart officer-opencode`): + +``` +POST /api/integration/opencode/connect/key { "key": "", "label": "…" } → 204 +``` + +Afterwards `claude-sonnet-4-6` and `claude-haiku-4-5` both run on the new pipeline. **The fork is +unblocked**, and everything the table above promises — steer, queue, interrupt, resumable per-session +SSE — is available with real models. + +Two consequences worth carrying: + +- **alpha needs the same one-time connect** before it can use the new pipeline. +- The sidecar should do this itself at boot rather than relying on someone having run it by hand, since + a missing credential degrades to "only free models work" with no error anywhere. + +### What the evidence looked like while I was getting it wrong + +Recorded because the shape repeats: the failure was **silent and total** for paid models, and the cost +table drew a perfect line — every cost-0 model ran, every cost>0 model did not. I read that as a billing +boundary inside a broken pipeline. It was a billing boundary caused by an absent credential, which is a +far more ordinary explanation and one Andre reached from knowing his own setup rather than from the API. + +The tell I had and did not use: the configured default is `opencode/big-pickle`, and a session created +with no model ran on `ling-3.0-tiny-free` **instead of the default**. A pipeline ignoring its configured +default is a pipeline that cannot use it — that is a credential symptom, and it was sitting in the +`/config/providers` output the whole time. + +## The original diagnosis, kept for the record: only free models run + +It is not sonnet, and it is not `variant`. Swept four models through `POST /api/session/{id}/model` +followed by a prompt: + +| Model | New pipeline | +| -------------------- | ------------ | +| `longcat-2.0-free` | **ran** | +| `ling-3.0-tiny-free` | **ran** | +| `claude-haiku-4-5` | never ran | +| `claude-sonnet-4-6` | never ran | +| `gpt-5.1-codex-mini` | never ran | + +**Every `-free` model runs; every paid model silently does not.** Ruled out along the way: + +- **Not `variant`.** `claude-sonnet-4-6` advertises `["low","medium","high","max"]` and session create + echoes back `variant: "default"`, which is not among them — a promising theory that turned out to be + wrong: setting `variant: "high"` explicitly also never ran. Tested rather than assumed, which is the + whole lesson of this file. +- **Not missing credentials.** `opencode auth list` shows an OpenCode Zen API key in + `~/.local/share/opencode/auth.json` plus `ANTHROPIC_API_KEY` in the environment. +- **Not the sidecar's environment.** The *same* sidecar process runs `claude-sonnet-4-6` correctly + through `opencode run --model`, verified end to end. Same user, same home, same auth file. + +So the new pipeline does not resolve paid-model credentials, and fails **silently** rather than +reporting it — while the legacy path and `run` both authenticate fine. Note it *can* surface provider +auth errors when it reaches that far: alpha's default (`nano-gpt`) returned a clean +`401 missing_api_key`. The silence is specific to opencode-zen paid models. + +This reads as an upstream bug in an in-progress pipeline, not something configurable on our side. + +**Consequence for the fork:** blocked, but precisely. Officer's users pick real models; a harness that +works only on free tiers is not adoptable. Re-run the sweep above after each `opencode upgrade` — the +day a paid model runs there, the migration is unblocked and worth doing immediately, because steer and +queue are already proven. ## Revised recommendation diff --git a/docs/opencode-parity.md b/docs/opencode-parity.md index 948f396c..39405bd0 100644 --- a/docs/opencode-parity.md +++ b/docs/opencode-parity.md @@ -91,7 +91,7 @@ is gone. What remains is bucket 1 — capabilities Claude has and OpenCode does visible ones (token streaming, mid-turn injection, background tasks, interrupt-without-teardown) are downstream of `stdin: 'ignore'` and therefore of the Phase 2 fork. -**The fork is REOPENED and worth taking.** The serve publishes a newer `/api/session/*` surface offering +**The fork is REOPENED, unblocked, and worth taking.** The serve publishes a newer `/api/session/*` surface offering those capabilities natively, and on 1.18.16 **`delivery: "steer"` and `delivery: "queue"` are both verified working** — mid-turn injection and queueing, as primitives, plus `/interrupt` and a resumable per-session event stream. One blocker remains: `claude-sonnet-4-6` silently does not run on that surface @@ -102,6 +102,30 @@ passed that one model. Until the model question is answered, turns stay on `opencode run --dir`, which is verified working on 1.18.16. +**Crash-recovery state is not a gap either.** `state:sync` is sent to the `proxy` capability and carries +`proxySecret` — it is the Anthropic proxy s state, not a chat recovery record — and `syncState` / +`getCachedState` have **no callers at all** outside `sidecar-registry.ts`. The row compared OpenCode +against a mechanism officer never consults. The real recovery story now exists and is better: a sidecar +restart stops in-flight turns and writes the reason to `chat_session_events`, and `/chat/live` +enumerates what is running. + +**Identity is correctly deferred, not forgotten.** `TODO.md:40-47` already records that `pty`, `vault` +and `opencode` receive no identity and are covered today only because those capabilities are owner-only — +"a correct outcome resting on the wrong layer". `chat` is `kind: execution`, which the grants API refuses +to share at any level, so this cannot be reached by a member. It is latent by construction. + +**`messageCount` is a non-issue, not a gap.** `SessionList.tsx:197-203` renders an `OpenCode` badge in +place of the count for OpenCode rows, so the hardcoded `0` is never displayed. Computing a real count +would cost one HTTP call per listed session — the session record carries no count field — to populate +something nothing renders. Left alone deliberately. + +**Images are done, and they never needed the fork** (bucket 1 lists them as "No — see B4", and Phase 4 +put them behind the migration). `opencode run` takes attachments with `--file`, so the subprocess path +carries them today: the sidecar spills each image to a temp file for the turn and removes it in +`settle`. Verified end to end — a red PNG over the chat socket to `opencode/claude-sonnet-4-6` came back +"Red". `list-models` now reports each model's own `capabilities.input.image` instead of a hardcoded +`false`, so the composer gate became load-bearing in the right direction. + --- ## Bucket 1 — Claude has it, OpenCode does not @@ -123,7 +147,7 @@ Ordered roughly by user-visible value. | MCP tools | `--mcp-config` | **Nothing** — no MCP anywhere in the OpenCode path | No | | `messageCount` on the list | from the transcript | hardcoded `0` | No | | Idle GC / warm-session lifetime | 30-min heartbeat, task-aware | N/A — nothing warm to collect | **Yes** | -| Crash-recovery state on disk | `claude-state.json` | **None** — `state:sync` returns an error | No | +| Crash-recovery state on disk | `claude-state.json` | **not a gap — see below** | No | | Identity | validates `X-Officer-User` | **None** — flagged in `TODO.md:42-47` | No | | Tests | 4 test files on the pure pieces | **Zero** | No | diff --git a/docs/opencode-serve-migration-plan.md b/docs/opencode-serve-migration-plan.md new file mode 100644 index 00000000..3ab2a073 --- /dev/null +++ b/docs/opencode-serve-migration-plan.md @@ -0,0 +1,119 @@ +# Moving OpenCode turns onto the serve — the plan + +Written 2026-08-10, after the fork was unblocked (`docs/opencode-fork-decision.md`). **Nothing here is +implemented.** It exists so the work can start from verified facts rather than from the API docs, which +have been wrong or misleading three times on this path. + +Andre should read "What changes for the user" and "The risk I would not take blind" before this starts. + +--- + +## What we are moving from and to + +Today every turn is `opencode run --dir --format json`, a subprocess with `stdin: 'ignore'`. It +works, it is verified end to end, and its limits are all consequences of that one closed pipe. + +The serve's `/api/session/*` surface offers, and I have run each of these against 1.18.16: + +| Capability | How | Verified | +| ------------------------ | --------------------------------------------------- | --------------------------------------------------- | +| Mid-turn injection | `POST /prompt` `{delivery: "steer"}` | yes — steered a running turn | +| Queue behind a turn | `POST /prompt` `{delivery: "queue"}` | yes — "ONE" then "TWO", no errors | +| Token streaming | `GET /api/event` (GLOBAL, live) — `text.delta` | yes — deltas reassemble to the committed text | +| Reconnect + replay | `GET /api/session/{id}/event?after=` (durable) | yes — replayed a finished session | +| Interrupt, session lives | `POST /interrupt` → 204 | endpoint only, not exercised | +| Model selection | `POST /model` → 204 | yes — runs on the chosen model | +| Images | `prompt.files` | not exercised (we have images via `--file` already) | + +## There are TWO streams, and this is the thing to get right + +Corrected after Phase A; the table above originally implied one. The serve publishes each turn twice: + +- **`GET /api/session/{id}/event?after=`** — durable, per session, replayable, every event carrying + `durable.seq`. Whole values only (`text.ended` with the full text). **No deltas.** +- **`GET /api/event`** — live, **global**, ephemeral. Carries `text.delta` and `tool.input.delta`. No cursor. + +Measured on one real turn: 13 events durable, 21 live, the difference being 3 `text.delta` and 5 +`tool.input.delta`. **Reading only the per-session stream — which is what I did first — makes it look +like the serve cannot stream at all**, and would have quietly removed the main reason to migrate. + +The split maps exactly onto what officer already does for Claude: durable → `chat_session_events`, live → +UI deltas. The cost is that the live stream is GLOBAL, so a consumer must filter on `sessionID` and +cannot assume it owns the socket. + +## Facts that will bite whoever implements this + +Each of these cost time to find. None is in the API docs. + +1. **The location is per REQUEST, not per session.** `x-opencode-directory: ` header, or + `?location[directory]=` as a deepObject query. A session created with `location` in the body and then + prompted without the header does not behave. +2. **Responses wrap in `{"data": …}`** on this surface; the legacy `/session/*` returns bare objects. + Reading `body.id` instead of `body.data.id` yields `undefined` silently. +3. **`delivery` defaults to `"steer"`.** Omitting it injects into a running turn, which is NOT the safe + default for an ordinary "send" — it must be set explicitly per intent. +4. **A model with no connected credential fails silently.** Prompt admitted, `prompt.admitted` and + `prompted` emitted, then nothing, forever. The sidecar now connects the credential at boot + (`connect-credential.ts`), and this failure mode is why that exists. +5. **The event names are `session.next.*`** — `step.started`, `text.started`, `text.ended`, + `tool.called`, `tool.success`, `step.ended`, `step.failed`. Not the shapes `mapRunLine` handles. + +## Status, 2026-08-10 + +- **Phase A — done.** `serve-events.ts` + tests, fixtures captured from real turns. +- **Phase B — done, behind `OPENCODE_TURNS=serve` (default: subprocess).** `serve-runner.ts`. Verified + end to end through the chat socket: tool call, tool result, **3 streaming deltas**, text, cost. Stop is + an interrupt and the session survives it. +- **Phase C — server half done.** A message sent while a turn runs is injected with `delivery: "steer"` + into the RUNNING turn, verified end to end. No client change was needed: officer's composer already + sends mid-turn, and the subprocess path was superseding where the serve steers. +- **Phase D — done.** The subprocess is deleted: no `runner.ts`, no `OPENCODE_TURNS` switch, no fallback + engine. Andre called it — nothing depends on OpenCode, so the cost of removing the escape hatch is + near zero and the recovery is git. 818 lines went with it, all of them workarounds for `stdin` being + `/dev/null`. + +Not yet lived with. **Nothing here has run a real conversation with a person at the other end** — see +`docs/opencode-testing-checklist.md` for what to try and what is most likely to be broken. + +## Shape of the work + +**Phase A — read the stream without depending on it.** Add a serve-based reader alongside the existing +runner: subscribe to `/api/session/{id}/event`, map `session.next.*` → `ChatEvent`, and prove the mapping +against real turns. Do not route any user traffic through it. This is where `mapRunLine`'s successor gets +written and tested, and it is the only phase with no user-visible risk. + +**Phase B — turns through the serve, behind a switch.** `POST /prompt` for the turn, events from Phase A, +`POST /interrupt` for stop. Keep `opencode run` reachable by config so a bad day is one restart from the +known-good path. The switch is the deliverable, not a detail. + +**Phase C — the capabilities that motivated it.** `delivery: "steer"` wired to the existing "send now" +button, `delivery: "queue"` to the queue, streaming deltas to the composer. These are the visible wins +and they are cheap once B holds. + +**Phase D — retire the subprocess**, only after B has run for a while. Deleting it early converts every +future problem into an emergency. + +## What changes for the user + +Better: text appears as it is generated instead of in blocks; the queue and "send now" work on OpenCode +exactly as they do on Claude; stop interrupts without destroying the session. + +Worse, potentially: the serve becomes load-bearing. Today a serve crash costs session listing and nothing +else, because turns are subprocesses. After this it costs every turn in flight. That trade is the whole +decision. + +## The risk I would not take blind + +**Warm sessions bring a lifetime problem OpenCode does not currently have.** A subprocess ends when the +turn ends; there is nothing to garbage-collect, adopt after a restart, or leak. A serve session persists, +so this migration imports the entire class of problems the Claude path spent months getting right — idle +GC, orphan adoption, releasing versus killing, the supersede race I fixed this morning. + +That is not an argument against doing it. It is an argument for Phase B keeping the old path one config +flip away, and for not doing Phase D on the same day as Phase B. + +## Where to start + +Phase A, `runner.ts`'s sibling, with the `session.next.*` fixtures captured from a real turn rather than +hand-written — `docs/opencode-fork-decision.md` records how to drive one with plain `curl`, and +`runner.test.ts` is the pattern for pinning a mapping without spawning anything. diff --git a/docs/opencode-testing-checklist.md b/docs/opencode-testing-checklist.md new file mode 100644 index 00000000..597759db --- /dev/null +++ b/docs/opencode-testing-checklist.md @@ -0,0 +1,86 @@ +# OpenCode: what to test tonight + +Every OpenCode turn now runs through the serve (`serve-runner.ts`). The `opencode run` subprocess is +deleted, so this is not a comparison against a fallback — it is the only path. + +I have driven each item in **Should already work** end to end through the real chat socket, with a +script rather than a browser. Nothing below has been used by a person in a real conversation, and the +things in **Unproven** are unproven because probes cannot answer them. + +--- + +## Before you start + +```bash +cd ~/projects/officer-suite/platform +pm2 logs officer-opencode --lines 40 # the sidecar's own account of what happened +``` + +On boot you should see three lines: `serve healthy on port …`, `connected the opencode credential to the +api surface`, and the sidecar registering. **If the credential line is missing or says it could not +connect, stop** — paid models will silently do nothing, which is the failure that cost most of an +afternoon (`docs/opencode-fork-decision.md`). + +Rolling back is `git revert` of `a3dbda7` (Phase D) and a restart. There is no config flag any more. + +Alpha is unaffected until you pull. + +--- + +## Should already work — confirm, do not investigate + +Each verified by me end to end. If one fails, that is new information and worth stopping on. + +- [ ] **A plain turn.** Ask for something short. Text arrives. +- [ ] **Streaming.** Ask for something long — "count slowly to 50". Text should appear **progressively**, + not in one block at the end. This is the headline change; the old path could not do it. +- [ ] **A tool call.** "Run `echo hi` with bash." A tool row appears with the command as its arguments, + then its output. +- [ ] **Cost.** The turn ends with a token count attached. +- [ ] **An image.** Drop a screenshot in and ask what it shows. It must actually be described — the + failure mode is a confident answer about nothing, which is what B4 was. +- [ ] **Stop.** Press stop mid-turn. The turn ends **and the conversation stays usable** — send another + message straight after and it should answer. Previously stop destroyed the session. +- [ ] **Mid-turn injection.** While a turn runs, send another message. It should join the RUNNING turn + rather than starting a new one or superseding it. + +## Unproven — this is the actual testing + +- [ ] **Resume from history.** Open an older OpenCode conversation from the list and continue it. This + exercises `resumeSessionId`, which I never tested against the serve. **Most likely thing to be + broken.** +- [ ] **An idle session, an hour later.** Send a message to a conversation you have not touched for a + while. Warm sessions are new here — the subprocess had nothing to go stale. +- [ ] **A sidecar restart mid-turn.** `pm2 restart officer-opencode` while a turn is generating. Expect + the transcript to say the turn stopped. **The turn itself keeps running inside the serve** — that + is intended, not a bug, but nobody has watched what it looks like from the browser. +- [ ] **An officer restart mid-turn.** Different from the above: officer is the relay, the sidecar keeps + committing to `chat_session_events`. On reload the transcript should be intact. This is the one I + would least like to be wrong about. +- [ ] **Two conversations at once.** The live event stream is GLOBAL — one socket carries every session + and `serve-runner` filters on `sessionID`. If that filter is wrong, output from one conversation + appears in another. Two panels side by side is the test. +- [ ] **The Live panel.** A running OpenCode turn should appear, named, and disappear when it ends. +- [ ] **A long turn.** Ten-plus minutes. The old path had watchdogs (inactivity, hard cap) that are gone + with it; the serve has its own ideas about timeouts and I have not found their edges. +- [ ] **A failing tool.** Ask it to run a command that does not exist. The error should land in the + transcript as a failed tool, not as a dead turn. + +## Known gaps, so you do not report them as bugs + +- **No durable replay.** `serve-runner` reads the live stream only. Events are still committed to + `chat_session_events` as they arrive, so the transcript survives — but recovering a turn *this sidecar + process never saw* would need the `?after=` cursor, and that is not built. +- **Notifications/thinking/background tasks.** Not implemented on this harness. Thinking is deliberately + out of scope for both harnesses. +- **`messageCount` shows nothing** for OpenCode rows. The UI renders an `OpenCode` badge instead; not a + gap (`docs/opencode-parity.md`). + +## What is useful to tell me + +For anything that misbehaves: what you did, what appeared, and the last twenty lines of +`pm2 logs officer-opencode`. The sidecar logs its own errors, and a silent failure with clean logs is a +different diagnosis from a loud one. + +If a turn produces **nothing at all** — no text, no error, no spinner ending — check the credential line +from boot first. That specific silence has one cause and I have chased it twice. diff --git a/src/server.tsx b/src/server.tsx index ac2ca1a1..dec9337d 100644 --- a/src/server.tsx +++ b/src/server.tsx @@ -304,6 +304,18 @@ const server = serve({ '/.well-known/carddav': honoServer.fetch, '/dav': honoServer.fetch, '/dav/*': honoServer.fetch, + // Vaultwarden for third-party Bitwarden clients, unauthenticated at Officer's layer. Needs its own + // entry for the same reason /dav does: only the paths listed here reach hono, and anything else + // falls through to the SPA — which answers 200 with the React shell, so a missing line here looks + // like a working endpoint returning nonsense rather than a 404. + '/vaultwarden/*': honoServer.fetch, + // The same proxy at the root, so the extension needs only the bare Officer URL. These four prefixes + // are Vaultwarden's alone — nothing in Officer answers on them — so routing them here costs nothing. + // `/api/*` already reaches hono below, where a Bitwarden client header diverts it. + '/identity/*': honoServer.fetch, + '/notifications/*': honoServer.fetch, + '/icons/*': honoServer.fetch, + '/events/*': honoServer.fetch, '/': officerWeb, '/*': officerWeb, '/api': honoServer.fetch, diff --git a/src/servers/api/chat/chat.ts b/src/servers/api/chat/chat.ts index fd2b8bef..dd93e3a6 100644 --- a/src/servers/api/chat/chat.ts +++ b/src/servers/api/chat/chat.ts @@ -23,7 +23,7 @@ import { renameOpenCodeSession, isOpenCodeSessionId, } from './opencode-sessions'; -import { getOpenCodeSession } from './opencode/state'; +import { getOpenCodePrompt, getOpenCodeSession } from './opencode/state'; import { listChatModels } from './list-models'; import { logger } from './logger'; import { readSttConfig } from '../server-settings/stt'; @@ -55,7 +55,8 @@ chatRouter.use(async (ctx, next) => { // The working directory a request operates on: an explicit ?cwd= (a chosen pwd), else the default // general_chat_sessions dir. Claude groups sessions by cwd, so this selects which project group we read. -// (OpenCode sessions all live in the one fixed server and ignore cwd.) +// OpenCode runs on one fixed serve, but each session records the directory its turn ran in, so cwd +// selects there too. const cwdOf = (ctx: Context, email: string): string => ctx.req.query('cwd')?.trim() || getGeneralChatSessionsCwd(email); // GET /chat/pwds — the default /chat dir plus every directory that already has Claude sessions. @@ -173,9 +174,12 @@ chatRouter.get('/live', async (ctx) => { isGenerating: true, pendingTasks: 0, harness: 'opencode' as const, - // Still null for the first seconds of a turn: the id is only known once the subprocess prints it. - // That window is real and short, and showing nothing beats showing a key the user has never seen. - title: meta?.title ?? null, + // OpenCode's own title wins as soon as it exists — it is derived from the conversation and is + // better than anything we would compose. Until then (and it titles asynchronously, so "until + // then" covers the whole time a turn is RUNNING, which is exactly what this panel shows) fall + // back to the prompt that started the session. Same shape as the Claude side, which has never + // shown a live row without a name. + title: meta?.title && meta.title !== '(untitled)' ? meta.title : (getOpenCodePrompt(sessionKey) ?? null), cwd: meta?.cwd || null, }; }); diff --git a/src/servers/api/chat/list-models.ts b/src/servers/api/chat/list-models.ts index 8d2b1a8b..1e545dcc 100644 --- a/src/servers/api/chat/list-models.ts +++ b/src/servers/api/chat/list-models.ts @@ -39,8 +39,12 @@ export function invalidateModelCache(): void { openCodeCache = null; } +type OpenCodeModel = { + capabilities?: { input?: { image?: boolean }; reasoning?: boolean }; +}; + type ProvidersResponse = { - providers?: Array<{ id?: string; models?: Record }>; + providers?: Array<{ id?: string; models?: Record }>; }; // Enumerate OpenCode models from the fixed server's GET /config/providers (reliable — no subprocess). @@ -58,19 +62,20 @@ async function listOpenCodeModels(): Promise { const models: ModelInfo[] = []; for (const provider of data.providers ?? []) { const providerId = provider.id ?? ''; - for (const modelId of Object.keys(provider.models ?? {})) { + for (const [modelId, model] of Object.entries(provider.models ?? {})) { models.push({ id: `${providerId}/${modelId}`, name: modelId, provider: providerId, contextWindow: 200000, maxTokens: 8192, - reasoning: false, - // False because nothing carries them: `handleOpenCodeChat`'s message type has no `images` - // field, so an attached image is rendered in the bubble, never sent, and silently dropped. - // The composer gates on this flag, so advertising `true` offered a capability that did not - // exist. Flip it back when images are plumbed through OpenCodeRunParams (parity doc, Phase 4). - images: false, + reasoning: model?.capabilities?.reasoning ?? false, + // Was hardcoded `false`, correctly, while nothing carried images — the composer gates on this + // flag, so advertising `true` offered a capability that did not exist. Images are now plumbed + // through `OpenCodeRunParams` to `opencode run --file`, so the honest answer is the model's + // own: OpenCode publishes it per model and we had never read it. Defaults to false, so a model + // that does not declare the capability keeps the affordance hidden rather than offering it. + images: model?.capabilities?.input?.image ?? false, }); } } diff --git a/src/servers/api/chat/opencode-sessions.test.ts b/src/servers/api/chat/opencode-sessions.test.ts new file mode 100644 index 00000000..e41e531b --- /dev/null +++ b/src/servers/api/chat/opencode-sessions.test.ts @@ -0,0 +1,27 @@ +import { describe, expect, it } from 'bun:test'; +import { isPlaceholderOpenCodeTitle } from './opencode-sessions'; + +// OpenCode names a session `New session - ` until it has titled it from the conversation, which it +// does asynchronously. The strings below are real ones observed on 1.18.16 — both the placeholder and +// the good title it was replaced by. + +describe('isPlaceholderOpenCodeTitle', () => { + it('recognises the placeholder OpenCode uses before it has titled a session', () => { + expect(isPlaceholderOpenCodeTitle('New session - 2026-08-10T15:44:17.178Z')).toBe(true); + expect(isPlaceholderOpenCodeTitle('New session - 2026-08-10T12:41:02.757Z')).toBe(true); + }); + + it('treats an absent or empty title as unnamed too', () => { + expect(isPlaceholderOpenCodeTitle(undefined)).toBe(true); + expect(isPlaceholderOpenCodeTitle('')).toBe(true); + expect(isPlaceholderOpenCodeTitle(' ')).toBe(true); + }); + + it('keeps a real title, including one that merely mentions a session', () => { + // The title OpenCode actually gave a turn of ours, and the reason this is a pattern match rather + // than a `startsWith('New session')`: a genuine title is allowed to begin with those words. + expect(isPlaceholderOpenCodeTitle('Single color in oc-red2.png')).toBe(false); + expect(isPlaceholderOpenCodeTitle('New session handling in the runner')).toBe(false); + expect(isPlaceholderOpenCodeTitle('New session - what should it be called?')).toBe(false); + }); +}); diff --git a/src/servers/api/chat/opencode-sessions.ts b/src/servers/api/chat/opencode-sessions.ts index 277c19dc..229dd1e9 100644 --- a/src/servers/api/chat/opencode-sessions.ts +++ b/src/servers/api/chat/opencode-sessions.ts @@ -7,23 +7,39 @@ import { logger } from './logger'; // fixed pm2-managed server's HTTP API (never the DB directly). Returns the same shapes as the Claude // reader, tagged harness:'opencode', so chat.ts can merge both harnesses transparently. // -// Sessions do NOT all live in one project. That claim was true when turns went through the serve and -// inherited its directory; turns are now `opencode run --dir ` subprocesses, so each session -// records the directory it ran in and one serve happily lists sessions across many. Verified: a single -// serve returned 7 sessions spread over several directories, which is also why the cwd filter has to -// read `directory` rather than assume. +// Sessions do NOT all live in one project. Each records the directory its turn ran in — the serve takes +// it per request as `x-opencode-directory` — and one serve holds sessions for many. That is why the cwd +// filter has to read `directory` rather than assume, and why the list has to come from a read that spans +// projects (`client.listSessions`, which is where the project-scoping trap is written up). /** * List OpenCode sessions for a working directory, so each context (/chat pwd, email account, project) * sees only its own. With no cwd, returns all. Never throws — returns [] if the server is unavailable. * - * Filters on the session's own `directory`, which is what OpenCode records when the runner starts it - * with `--dir`. It used to filter on `metadata.officer.cwd`, a tag whose only writer + * Filters on the session's own `directory`, which is what OpenCode records from the directory the turn + * ran in. It used to filter on `metadata.officer.cwd`, a tag whose only writer * (`client.createSession`) has no callers — so the comparison was against `undefined` for every session * and the list was ALWAYS empty. `cwdOf` in chat.ts substitutes a default when no `?cwd=` is given, so * the `!cwd` escape never fired either and there was no configuration in which an OpenCode session * appeared in /chat. Verified against the live server: 7 sessions present, 0 returned. */ +/** + * OpenCode's own placeholder name for a session it has not titled yet. + * + * It titles a session from the conversation, but ASYNCHRONOUSLY and after there is something to read — + * a finished turn ends up called "Single color in oc-red2.png", which is better than anything we would + * generate. Until then the session is literally named `New session - 2026-08-10T15:44:17.178Z`. + * + * That window is exactly when a session is most visible: `/chat/live` shows turns that are RUNNING, so + * the placeholder is what the panel catches. Treating it as "not yet titled" lets the good name arrive + * on its own. Forcing `--title` on the run instead would fix the transient case by permanently + * replacing opencode's title with a truncated prompt — worse where it lasts longest. + */ +export const isPlaceholderOpenCodeTitle = (title: string | undefined): boolean => { + const trimmed = title?.trim(); + return !trimmed || /^New session - \d{4}-\d{2}-\d{2}T[\d:.]+Z?$/.test(trimmed); +}; + export async function listOpenCodeSessions(cwd?: string): Promise { try { const { baseUrl } = await ensureServer(); @@ -35,7 +51,7 @@ export async function listOpenCodeSessions(cwd?: string): Promise { - const res = await fetch(`${this.baseUrl}/session`); - if (!res.ok) throw new Error(`opencode GET /session → ${res.status}`); - return (await res.json()) as OpenCodeSessionInfo[]; + const res = await fetch(`${this.baseUrl}/api/session`); + if (!res.ok) throw new Error(`opencode GET /api/session → ${res.status}`); + const body = (await res.json()) as { data?: ApiSessionInfo[] } | ApiSessionInfo[]; + const sessions = Array.isArray(body) ? body : (body.data ?? []); + return sessions.map((s) => ({ ...s, directory: s.directory ?? s.location?.directory ?? null })); } /** One session's own record. The only place its working directory can be read on resume. */ @@ -72,6 +94,9 @@ export type OpenCodeSessionInfo = { directory?: string | null; }; +/** `/api/session`'s shape: the same record, with the directory one level down. Normalised by `listSessions`. */ +type ApiSessionInfo = OpenCodeSessionInfo & { location?: { directory?: string | null } }; + export type OpenCodeStoredPart = { type?: string; text?: string; diff --git a/src/servers/api/chat/opencode/state.ts b/src/servers/api/chat/opencode/state.ts index e7edf05c..8e852c0b 100644 --- a/src/servers/api/chat/opencode/state.ts +++ b/src/servers/api/chat/opencode/state.ts @@ -12,4 +12,25 @@ export const setOpenCodeSession = (sessionKey: string, opencodeSessionId: string export const clearOpenCodeSession = (sessionKey: string): void => { sessionKeyToOpenCode.delete(sessionKey); + sessionKeyToPrompt.delete(sessionKey); }; + +// What the user asked, kept only to name a session OpenCode has not named yet. +// +// OpenCode titles a session from the conversation, and does it well — but asynchronously, so a RUNNING +// turn is called `New session - `, which is precisely when `/chat/live` displays it. This is the +// stand-in for that window: shown while the real title is still the placeholder, and dropped the moment +// OpenCode publishes its own, which is always the better one. +// +// Set on the FIRST turn of a session only, so it stays the name of the conversation rather than +// following whatever was asked most recently. +const sessionKeyToPrompt = new Map(); + +export const rememberOpenCodePrompt = (sessionKey: string, prompt: string): void => { + if (sessionKeyToPrompt.has(sessionKey)) return; + const oneLine = prompt.replace(/\s+/g, ' ').trim(); + if (!oneLine) return; + sessionKeyToPrompt.set(sessionKey, oneLine.length > 60 ? `${oneLine.slice(0, 59)}…` : oneLine); +}; + +export const getOpenCodePrompt = (sessionKey: string): string | undefined => sessionKeyToPrompt.get(sessionKey); diff --git a/src/servers/api/chat/session-manager.ts b/src/servers/api/chat/session-manager.ts index c697b332..fcde0940 100644 --- a/src/servers/api/chat/session-manager.ts +++ b/src/servers/api/chat/session-manager.ts @@ -24,7 +24,7 @@ class SessionManager { cwd, model, piProcess: null, - ws: null, + sockets: new Set(), lastActivity: Date.now(), idleTimer: null, streamBuffer: '', @@ -145,7 +145,7 @@ class SessionManager { attachWs(sessionId: string, ws: any): void { const session = this.sessions.get(sessionId); if (session) { - session.ws = ws; + session.sockets.add(ws); session.lastActivity = Date.now(); if (session.idleTimer) { @@ -155,14 +155,24 @@ class SessionManager { } } - detachWs(sessionId: string): void { + /** + * Removes one socket. The caller must say WHICH — a bare `detachWs(sessionId)` used to null the + * session's only socket field, so a stale client's close event silenced whichever client had attached + * after it. A close is only the end of the conversation when nothing else is still watching. + */ + detachWs(sessionId: string, ws: any): void { const session = this.sessions.get(sessionId); if (session) { - session.ws = null; + session.sockets.delete(ws); session.lastActivity = Date.now(); } } + /** Whether anything is still watching — the idle GC must not start while another client is attached. */ + hasSockets(sessionId: string): boolean { + return (this.sessions.get(sessionId)?.sockets.size ?? 0) > 0; + } + setIdleTimeout(sessionId: string, timeoutMs: number): void { const session = this.sessions.get(sessionId); if (!session) return; diff --git a/src/servers/api/chat/types.ts b/src/servers/api/chat/types.ts index b428467d..d49ddc05 100644 --- a/src/servers/api/chat/types.ts +++ b/src/servers/api/chat/types.ts @@ -299,7 +299,13 @@ export type UserSession = { cwd: string; model: string; piProcess: any | null; - ws: any | null; + /** + * Every socket watching this conversation, not the most recent one. Two panes in one window, or a + * laptop and an iPad on the same chat, are both ordinary now that a tab holds several panes — and a + * single `ws` field meant the newest attach silently stole the turn from everyone else, while any one + * of them closing set it to null and killed delivery for the rest. + */ + sockets: Set; lastActivity: number; idleTimer: Timer | null; streamBuffer: string; diff --git a/src/servers/api/chat/websocket.test.ts b/src/servers/api/chat/websocket.test.ts index 94e1a2fd..a56b780a 100644 --- a/src/servers/api/chat/websocket.test.ts +++ b/src/servers/api/chat/websocket.test.ts @@ -28,17 +28,24 @@ describe('decideResume', () => { expect(decideResume(undefined, 'claude-code')).toEqual({ kind: 'adopt', model: 'claude-code' }); }); - it('stays unknown rather than defaulting to claude, which is the whole bug', () => { - // No record and no claim. Previously this became `claude-code` and was indistinguishable from a real - // Claude session; now it declines to adopt and declines to run the cut-off check. - expect(decideResume(undefined, undefined)).toEqual({ kind: 'replay-only' }); - expect(decideResume(undefined, '')).toEqual({ kind: 'replay-only' }); + it('marks a total unknown as an assumption instead of passing it off as knowledge', () => { + // No record and no claim. This still adopts — see below — but the caller can tell it apart, which is + // what keeps the durable cut-off row off a session nobody has identified. + expect(decideResume(undefined, undefined)).toEqual({ kind: 'assume', model: 'claude-code' }); + expect(decideResume(undefined, '')).toEqual({ kind: 'assume', model: 'claude-code' }); }); - it('never reports replay-only once anything is known, so a live turn is always re-bound', () => { - // The failure mode in the other direction: declining to adopt a session we could have identified - // would silently drop the reconnect that makes turn output resume. - expect(decideResume('claude-code', undefined).kind).toBe('known'); - expect(decideResume('', 'claude-code').kind).toBe('adopt'); + it('always yields a model to adopt with, because an unbound socket misses the whole turn', () => { + // Regression guard. Refusing to adopt when the harness was unknown looked principled and broke the + // app inside an hour: `useChat.ts` only sends `model` `if (modelRef.current)`, so a reconnect + // without one is routine. The socket never re-bound, the live turn's output went nowhere, and the + // transcript collapsed to "turn completed without output" until a refresh rebuilt it from the log. + for (const d of [ + decideResume('claude-code', undefined), + decideResume(undefined, 'opencode/x'), + decideResume(undefined, undefined), + ]) { + expect(d.model).toBeTruthy(); + } }); }); diff --git a/src/servers/api/chat/websocket.ts b/src/servers/api/chat/websocket.ts index 32236343..ee36abe4 100644 --- a/src/servers/api/chat/websocket.ts +++ b/src/servers/api/chat/websocket.ts @@ -10,6 +10,7 @@ import type { UserSession, } from './types'; import { sessionManager } from './session-manager'; +import { rememberOpenCodePrompt } from './opencode/state'; import { sendClaudeCodeStreaming } from '@@/channels/send-claude-code'; import { sendOpenCodeStreaming } from '@@/channels/send-opencode'; import { ensureGeneralChatSessionsCwd } from './claude-sessions'; @@ -157,8 +158,10 @@ export function close(ws: ServerWebSocket): void { const sessionId = wsToSessionMap.get(ws); if (sessionId) { - sessionManager.detachWs(sessionId); - sessionManager.setIdleTimeout(sessionId, IDLE_TIMEOUT_MS); + sessionManager.detachWs(sessionId, ws); + // Only once nothing is watching. Another pane or another device still attached means the + // conversation is live, and arming the idle GC here would collect it out from under them. + if (!sessionManager.hasSockets(sessionId)) sessionManager.setIdleTimeout(sessionId, IDLE_TIMEOUT_MS); } } @@ -243,7 +246,7 @@ function createMessageHandler(sessionId: string, model: string) { const session = sessionManager.getSession(sessionId); if (!session) return; foldIntoSession(session, msg, model); - sendToClient(session.ws as ServerWebSocket | null, msg, seq); + for (const socket of session.sockets) sendToClient(socket as ServerWebSocket, msg, seq); }; } @@ -396,6 +399,10 @@ async function handleOpenCodeChat( cwd?: string; cwdRoot?: string; resumeSessionId?: string; + // The whole of B4 lived in this omission. The browser sent images, the bubble rendered them, and + // they stopped at this signature — so they were never passed on and never reached the model, with + // nothing anywhere reporting a loss. + images?: PromptImage[]; }, effectivePrompt: string, ): Promise { @@ -403,6 +410,9 @@ async function handleOpenCodeChat( const cwd = await resolveChatCwd(msg, email, userId); + // Names this session in the Live panel until OpenCode gets round to titling it. First turn only. + rememberOpenCodePrompt(sessionId, msg.displayText || msg.prompt); + const groupSlug = msg.groupSlug || null; const session = sessionManager.getOrCreate(sessionId, email, cwd, model, groupSlug, msg.context, msg.contextId); @@ -457,6 +467,7 @@ async function handleOpenCodeChat( cwd, model, resumeSessionId: msg.resumeSessionId, + images: msg.images, onMessage, }); @@ -617,19 +628,17 @@ async function handleResumeCursor( const known = sessionManager.getSession(sessionId); const decision = decideResume(known?.model, msg.model); - const model = decision.kind === 'replay-only' ? null : decision.model; - if (decision.kind === 'adopt') { + if (decision.kind !== 'known') { + // Adopt on an assumption too: an unbound socket misses the turn entirely, which is worse than a + // harness guess that only ever costs us the cut-off check below. adoptOrphanedSession(ws, sessionId, decision.model, msg.cwd ?? ''); - } else if (decision.kind === 'replay-only') { - // Replay the durable log and stop there. Not adopting costs a live re-subscription; adopting on a - // guess cost correctness — see decideResume. - logger.warn('resume-cursor names a session this process does not know, and no model; replaying only', { - sessionId, - }); + if (decision.kind === 'assume') { + logger.warn('resume-cursor for an unknown session with no model; adopting on the default', { sessionId }); + } } - sessionManager.attachWs(sessionId, ws); // a no-op when adoption was skipped + sessionManager.attachWs(sessionId, ws); wsToSessionMap.set(ws as any, sessionId); try { const events = await getChatEventsSince(sessionId, cursor ?? 0); @@ -639,15 +648,19 @@ async function handleResumeCursor( } catch (err) { logger.error('Failed to replay chat events on resume-cursor', { sessionId, error: String(err) }); } - // Only ask when the harness is actually known. The check writes a DURABLE row, so a wrong answer here - // is permanent — silence is the safe failure. - if (msg.generating && model) await endTurnIfAgentIsGone(ws, sessionId, model); + // Only ask when the harness is actually known — never on an assumption. This check writes a DURABLE + // row, so a wrong answer is permanent: that was B7, where a defaulted `claude-code` made officer ask + // the Claude sidecar about an OpenCode session, hear "not generating", and write "the agent went + // away" into a turn that was running perfectly well. + if (msg.generating && decision.kind !== 'assume') { + await endTurnIfAgentIsGone([ws], sessionId, decision.model); + } } export type ResumeDecision = | { kind: 'known'; model: string } | { kind: 'adopt'; model: string } - | { kind: 'replay-only' }; + | { kind: 'assume'; model: string }; /** * Which harness a resuming session belongs to, and what that permits. @@ -668,13 +681,21 @@ export type ResumeDecision = * * Two rules. **The server's own record beats the client's claim** — a session in memory already knows its * harness, and letting a socket re-declare it is how the wrong sidecar gets a session in the first place. - * **An unknown harness stays unknown**: no adoption, no cut-off check, just the replay. Defaulting is - * what made a guess indistinguishable from knowledge. + * **A guess is still a guess**: when neither knows, the session is adopted on the default so delivery + * keeps working, but it is marked `assume` and nothing durable may be written from it. + * + * The middle position — refusing to adopt at all when the harness is unknown — was tried and was WRONG, + * visibly so within the hour. `useChat.ts` sends `model` only `if (modelRef.current)`, so a reconnect + * without one is ordinary, not exotic; declining to adopt left the socket unbound to a live turn, and + * the running turn's output went nowhere. On screen: the transcript collapsed to "turn completed + * without output" and only a refresh — which rebuilds from the durable log — brought it back. + * + * So adoption is about DELIVERY and must be generous. Only the durable write needs certainty. */ export function decideResume(knownModel: string | undefined, claimedModel: string | undefined): ResumeDecision { if (knownModel) return { kind: 'known', model: knownModel }; if (claimedModel) return { kind: 'adopt', model: claimedModel }; - return { kind: 'replay-only' }; + return { kind: 'assume', model: DEFAULT_MODEL }; } /** @@ -802,7 +823,7 @@ async function handleAttach(ws: ServerWebSocket, msg: { claudeSessionId: * so its sessions are left alone rather than guessed at. */ async function endTurnIfAgentIsGone( - ws: ServerWebSocket | null, + targets: Iterable | null>, sessionId: string, model: string, ): Promise { @@ -815,11 +836,11 @@ async function endTurnIfAgentIsGone( const event: ServerMessage = { type: 'cut-off' }; try { const seq = await appendChatEvent(sessionId, event); - sendToClient(ws, event, seq); + for (const target of targets) sendToClient(target, event, seq); } catch (err) { - // Still tell this client — an un-replayable explanation beats a spinner that never stops. + // Still tell every client — an un-replayable explanation beats a spinner that never stops. logger.error('Failed to persist cut-off notice', { sessionId, error: String(err) }); - sendToClient(ws, event); + for (const target of targets) sendToClient(target, event); } logger.info('Ended a turn whose agent had gone', { sessionId }); } @@ -831,7 +852,11 @@ async function endTurnIfAgentIsGone( sidecar.onClaudeSidecarStarted(() => { for (const session of sessionManager.getAllSessions()) { if (!session.isGenerating) continue; - void endTurnIfAgentIsGone(session.ws as ServerWebSocket | null, session.sessionId, session.model); + void endTurnIfAgentIsGone( + session.sockets as Set>, + session.sessionId, + session.model, + ); } }); diff --git a/src/servers/api/vault/public-router.ts b/src/servers/api/vault/public-router.ts new file mode 100644 index 00000000..a9c34b86 --- /dev/null +++ b/src/servers/api/vault/public-router.ts @@ -0,0 +1,110 @@ +import { createRouter } from '../../create-router'; +import { getVaultServerUrl } from './sidecar-server'; +import { stripHopByHop, redactPath } from './proxy-util'; + +// Vaultwarden, reachable at Officer's own hostname with NO Officer authentication. +// +// ── Why this exists ── +// +// The Bitwarden browser extension speaks Vaultwarden's protocol and nothing else: it obtains its own +// bearer token from `/identity/connect/token` and has nowhere to put a platform JWT. `/api/vault` cannot +// serve it — that router requires an Officer session and REPLACES the caller's Authorization header with +// a server-held Vaultwarden token, which is the right design for our own clients (the device then holds +// no vault credential at all) and an impossible one for a third-party client. +// +// So this mount exists to retire a second public hostname. Andre pointed the extension straight at +// Vaultwarden; that URL can now go away, and the extension can use the platform's. +// +// ── Why it is acceptable to leave unauthenticated ── +// +// It is not a new exposure, it is the same exposure through one door instead of two: everything here was +// already reachable at the Vaultwarden URL this replaces, guarded by the same thing that guards it now — +// the vault's own master password and Vaultwarden's own auth. Officer adds no security to a Bitwarden +// client today because it cannot: it has no credential to check. +// +// It is also going behind Tailscale, so the surface is a private network rather than the internet. +// +// ── TEMPORARY ── +// +// The intended end state is a browser extension of our own, reusing `@officer/vault` (already proven to +// run as a plain JS bundle outside React Native — the iOS autofill extension hosts it in JavaScriptCore) +// against the `/api/vault/session/login` broker. Then the client authenticates to OFFICER, Vaultwarden is +// never addressed directly by anything, and this mount is deleted rather than adjusted. +// +// ── The rule that keeps it safe ── +// +// DUMB PASS-THROUGH, and more so than `/api/vault`: that one rewrites the Authorization header, this one +// touches nothing. Method, path, query, headers, status and both body streams go through verbatim. The +// vault is end-to-end encrypted and no part of this may parse, decrypt, buffer, cache or log a body — +// paths are redacted in logs for the same reason. Every line of "cleverness" added here is reachable +// without an Officer session, which is the whole reason it must stay boring. + +export const publicVaultRouter = createRouter(); + +const PREFIX = '/vaultwarden'; + +/** + * Paths that belong to Vaultwarden and to nothing else in this platform. + * + * These are served at the ROOT so the extension can be configured with the bare Officer URL — no + * `/vaultwarden` suffix — which is the point: one hostname, one URL, nothing to explain. None of them + * collide with an Officer route, so no sniffing is involved; the path alone is the answer. + */ +export const VAULT_ONLY_PREFIXES = ['/identity', '/notifications', '/icons', '/events']; + +/** + * Is this a Bitwarden client talking to us? + * + * Needed only for `/api/*`, which is the one namespace both servers claim — Vaultwarden has + * `/api/sync`, `/api/ciphers`, `/api/settings/domains`; Officer has `/api/chat`, `/api/settings` and + * everything else. The path cannot decide it, so the client says who it is: every Bitwarden client + * stamps `Bitwarden-Client-Name` (`browser`, `desktop`, `cli`, `web`), and older ones `Device-Type`. + * + * This is ROUTING, not authentication, which is why trusting a client-supplied header is fine here: the + * worst a forged header achieves is reaching Vaultwarden, which then demands its own credential exactly + * as it would have. Nothing is authorised by this function. + */ +export const isBitwardenClient = (headers: Headers): boolean => + headers.has('bitwarden-client-name') || headers.has('bitwarden-client-version') || headers.has('device-type'); + +/** Everything Vaultwarden serves, whichever door it arrived through. */ +export const isVaultwardenPath = (path: string, headers: Headers): boolean => { + if (VAULT_ONLY_PREFIXES.some((p) => path === p || path.startsWith(`${p}/`))) return true; + return path.startsWith('/api/') && isBitwardenClient(headers); +}; + +publicVaultRouter.all('/*', async (ctx) => { + const base = getVaultServerUrl(); + if (!base) return ctx.text('Vault sidecar not available', 503); + + const url = new URL(ctx.req.url); + // Mounted at `/vaultwarden` AND at the root prefixes above. Strip the prefix only when it is there, + // so `/vaultwarden/identity/...` and a bare `/identity/...` both reach Vaultwarden's own path. + const subpath = (url.pathname.startsWith(PREFIX) ? url.pathname.slice(PREFIX.length) : url.pathname) || '/'; + const target = `${base}${subpath}${url.search}`; + const method = ctx.req.method; + const hasBody = method !== 'GET' && method !== 'HEAD'; + + // Authorization is FORWARDED, not replaced — the opposite of `/api/vault`. The client's Vaultwarden + // token is the only credential in play, and Vaultwarden is the only thing entitled to judge it. + const headers = stripHopByHop(ctx.req.raw.headers); + + const init: RequestInit & { duplex?: 'half' } = { method, headers, redirect: 'manual' }; + if (hasBody) { + init.body = ctx.req.raw.body; + init.duplex = 'half'; + } + + let upstream: Response; + try { + upstream = await fetch(target, init); + } catch { + console.error(`[vault:public] ${method} ${redactPath(subpath + url.search)} -> sidecar unreachable`); + return ctx.text('Vault sidecar unreachable', 502); + } + + // No 401-refresh-and-retry here, deliberately. That belongs to `/api/vault`, which owns the token it + // injected; this router holds no token and a 401 is Vaultwarden's answer to the CLIENT, which the + // client knows how to handle — it re-authenticates with the master password. + return new Response(upstream.body, { status: upstream.status, headers: stripHopByHop(upstream.headers) }); +}); diff --git a/src/servers/channels/send-opencode.ts b/src/servers/channels/send-opencode.ts index 4be73c23..4ca28230 100644 --- a/src/servers/channels/send-opencode.ts +++ b/src/servers/channels/send-opencode.ts @@ -1,4 +1,4 @@ -import type { TurnMessage } from '@@/api/chat/types'; +import type { PromptImage, TurnMessage } from '@@/api/chat/types'; import { logger } from '@@/api/chat/logger'; import * as sidecar from '@@/sidecar-registry'; import { getOpenCodeSession } from '@@/api/chat/opencode/state'; @@ -20,6 +20,7 @@ type OpenCodeStreamingParams = { role?: string; resumeSessionId?: string; durable?: boolean; + images?: PromptImage[]; // Finished turn messages, already committed by the sidecar; `seq` is the cursor id to deliver them under. onMessage: (msg: TurnMessage, seq?: number) => void; }; @@ -59,6 +60,7 @@ export async function sendOpenCodeStreaming(params: OpenCodeStreamingParams): Pr model: params.model, resumeSessionId, durable: params.durable, + images: params.images, }); } catch (err) { unsub(); diff --git a/src/servers/hono.ts b/src/servers/hono.ts index 9e699534..52cd9177 100644 --- a/src/servers/hono.ts +++ b/src/servers/hono.ts @@ -22,6 +22,7 @@ import { taskLogsRouter } from './api/task-logs/task-logs'; import { router as fileBrowserRouter } from './api/file-browser/router'; import { musicRouter } from './api/music/router'; import { vaultRouter } from './api/vault/router'; +import { publicVaultRouter, VAULT_ONLY_PREFIXES, isBitwardenClient } from './api/vault/public-router'; import { agentHandoffRouter } from './api/agent-handoff/router'; import { slskdRouter } from './api/slskd/router'; import { headscaleRouter } from './api/headscale/router'; @@ -103,6 +104,27 @@ honoServer.route('/api/waitlist', waitlistRouter); // gating still applies via originScopeMiddleware above (OFFICER_VAULT_ORIGIN → /api/vault). The // notifications WebSocket is upgraded at the serve level (server.tsx). honoServer.route('/api/vault', vaultRouter); + +// The same Vaultwarden, with NO Officer authentication, so the Bitwarden browser extension can point at +// this host instead of at a second public hostname for Vaultwarden. Deliberately its own mount rather +// than a mode of the router above: that one requires an Officer session and swaps the caller's +// Authorization header for a server-held token, and blending the two would put an unauthenticated branch +// inside the authenticated path. Temporary — see public-router.ts for what replaces it and why leaving it +// open is not a new exposure. +honoServer.route('/vaultwarden', publicVaultRouter); + +// …and at the ROOT, so the extension can be pointed at the bare Officer URL with no path at all. +// +// Registered BEFORE `/api` is mounted, because hono matches in registration order and this has to win +// for a Bitwarden client. It is deliberately narrow: the four prefixes below belong to Vaultwarden and +// to nothing else here, and `/api/*` is diverted ONLY when the request carries a Bitwarden client +// header. An ordinary Officer request never matches, so nothing that worked before changes. +for (const prefix of VAULT_ONLY_PREFIXES) honoServer.route(prefix, publicVaultRouter); + +honoServer.use('/api/*', async (ctx, next) => { + if (!isBitwardenClient(ctx.req.raw.headers)) return next(); + return publicVaultRouter.fetch(ctx.req.raw, ctx.env); +}); honoServer.get('/api/integrations/google/callback', googleCallbackHandler); // Agent-to-agent handoff — mounted TOP-LEVEL for the same reason the vault is: the caller is a Claude diff --git a/src/servers/sidecar/opencode/connect-credential.ts b/src/servers/sidecar/opencode/connect-credential.ts new file mode 100644 index 00000000..dbc2a3a3 --- /dev/null +++ b/src/servers/sidecar/opencode/connect-credential.ts @@ -0,0 +1,99 @@ +import { readFileSync } from 'node:fs'; +import { homedir } from 'node:os'; +import { join } from 'node:path'; + +// Hand the serve's NEW api surface the provider key it cannot find on its own. +// +// ── The problem this exists to prevent ── +// +// opencode keeps credentials in two unrelated places. The CLI, `opencode run` and the legacy +// `/session/*` surface read `~/.local/share/opencode/auth.json`. The newer `/api/session/*` surface — +// the one with `delivery: "steer" | "queue"`, `/interrupt` and a resumable per-session event stream — +// reads its own integration store instead (`/api/integration`, `/api/credential`), and knows nothing +// about that file. +// +// With no credential the new pipeline does not fail. It falls back to whatever needs none, which is the +// free tier, and a request for a paid model is simply never executed: prompt accepted, `prompt.admitted` +// and `prompted` emitted, no step, no error, no assistant message, forever. That silence cost most of an +// afternoon to diagnose (docs/opencode-fork-decision.md) and would cost it again on every new machine. +// +// So the sidecar connects it at start-up rather than relying on somebody having run a curl by hand. +// +// ── Deliberately best-effort ── +// +// Never throws and never blocks start-up. Turns run through `opencode run`, which uses `auth.json` and +// is unaffected by any of this; failing here costs the new pipeline only, and the sidecar is far more +// useful up than down. The connection persists in opencode's own store, so this is a no-op on every +// start after the first. + +const AUTH_PATH = join(process.env.XDG_DATA_HOME || join(homedir(), '.local', 'share'), 'opencode', 'auth.json'); + +const ATTEMPTS = 6; +const RETRY_DELAY_MS = 1_500; + +/** The provider key opencode already holds for itself, or null. Never logged, never returned to callers. */ +function readProviderKey(providerId: string): string | null { + try { + const auth = JSON.parse(readFileSync(AUTH_PATH, 'utf8')) as Record; + const entry = auth[providerId]; + return entry?.type === 'api' && typeof entry.key === 'string' && entry.key ? entry.key : null; + } catch { + return null; // no auth file, unreadable, or not JSON — nothing to connect + } +} + +/** + * Connect `auth.json`'s key for one provider to the serve's integration store. + * + * `providerId` doubles as the integration id: opencode names them the same, so the Zen key stored under + * `opencode` connects to integration `opencode`. + */ +export async function connectProviderCredential(baseUrl: string, providerId = 'opencode'): Promise { + const key = readProviderKey(providerId); + if (!key) { + console.log(`[opencode] no ${providerId} key in auth.json; the new API surface will only reach free models`); + return; + } + + // Retried, because `/api/health` answers before the integration store is ready: connecting immediately + // after the health check returns 500, and the identical request succeeds seconds later. Measured, not + // assumed — the first version of this shipped without the retry and failed on its first real boot. + // + // Only 5xx is retried. A 4xx means the request itself is wrong (bad key, unknown integration) and + // repeating it just prints the same complaint five times. + for (let attempt = 1; attempt <= ATTEMPTS; attempt++) { + try { + const res = await fetch(`${baseUrl}/api/integration/${providerId}/connect/key`, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ key, label: 'officer-opencode sidecar' }), + signal: AbortSignal.timeout(10_000), + }); + + if (res.ok) { + console.log(`[opencode] connected the ${providerId} credential to the api surface`); + return; + } + // The body is deliberately not logged: a credential endpoint's error may quote what it was given. + if (res.status < 500) { + console.error(`[opencode] could not connect the ${providerId} credential: HTTP ${res.status}`); + return; + } + if (attempt === ATTEMPTS) { + console.error( + `[opencode] could not connect the ${providerId} credential after ${ATTEMPTS} attempts: HTTP ${res.status}`, + ); + return; + } + } catch (err) { + if (attempt === ATTEMPTS) { + console.error( + `[opencode] could not connect the ${providerId} credential:`, + err instanceof Error ? err.message : err, + ); + return; + } + } + await Bun.sleep(RETRY_DELAY_MS); + } +} diff --git a/src/servers/sidecar/opencode/index.ts b/src/servers/sidecar/opencode/index.ts index bb2aee96..aac33861 100644 --- a/src/servers/sidecar/opencode/index.ts +++ b/src/servers/sidecar/opencode/index.ts @@ -4,17 +4,19 @@ import { join } from 'node:path'; import { DATA_PATH } from '../../data-path'; import { createSidecarConnector } from '../connect'; import { sweepRecordedServe } from './serve-sweep'; +import { connectProviderCredential } from './connect-credential'; import { createSessionLogStore } from '../claude/session-log'; import type { SidecarCommand, SidecarEvent } from '../protocol'; -import { runOpenCodeTurn, killOpenCodeTurn, listRunningOpenCodeTurns, stopAllOpenCodeTurns } from './runner'; +import type { RunnerMessage } from './serve-runner'; +import { runOpenCodeTurnOnServe, killServeTurn, listRunningServeTurns, stopAllServeTurns } from './serve-runner'; // The OpenCode sidecar (officer-opencode). Same philosophy as officer-claude: a singleton process that // OWNS its runtime — here, an `opencode serve` — registers with the API server, and answers commands. It // listens on a random port, reported to the API on connect so it can route there. // -// The serve's working directory is DATA_PATH/opencode_server, and that is now ALL it is: turns do not go -// through the serve, they are `opencode run --dir ` subprocesses (runner.ts). The serve is used for -// session CRUD and model enumeration only. +// The serve runs EVERYTHING: turns (serve-runner.ts), session CRUD and model enumeration. It used to be +// CRUD only, with turns spawned as `opencode run --dir ` subprocesses — that path was deleted on +// 2026-08-10 once the serve had streaming, mid-turn injection and interrupt working end to end. const API_URL = process.env.API_URL ?? `ws://127.0.0.1:${process.env.PORT ?? '5000'}`; const OPENCODE_BIN = process.env.OPENCODE_BIN || join(homedir(), '.opencode', 'bin', 'opencode'); @@ -132,11 +134,16 @@ if (!(await waitHealthy(baseUrl, HEALTH_TIMEOUT_MS))) { } console.log(`[opencode] serve healthy on port ${port}`); +// The new /api surface keeps credentials separately from auth.json and would otherwise reach free models +// only — silently. Best-effort and not awaited for correctness: turns go through `opencode run`, which +// reads auth.json directly and does not depend on this. +void connectProviderCredential(baseUrl); + // ── Command handlers ── type ReplyFn = (msg: SidecarEvent) => void; -const RUNNER_CONFIG = { bin: OPENCODE_BIN, fallbackCwd: SERVE_CWD }; +const SERVE_RUNNER_CONFIG = { baseUrl, fallbackCwd: SERVE_CWD }; function handleCommand(cmd: SidecarCommand, reply: ReplyFn) { switch (cmd.type) { @@ -148,23 +155,29 @@ function handleCommand(cmd: SidecarCommand, reply: ReplyFn) { // Turn output goes through the session log: translated to TurnMessages and committed to // chat_session_events here, in the process that produced it. Officer being down during a turn // no longer costs the transcript — the browser replays it from its cursor. - runOpenCodeTurn(cmd.params, RUNNER_CONFIG, (msg) => { + // Same emit contract either way, which is what makes the switch a switch: the durable commit and + // the routing fact behave identically whether a subprocess or the serve produced the event. + const onMessage = (msg: RunnerMessage) => { if (msg.type === 'opencode:event') { sessionLog.push(sessionKey, msg.event, durable); return; } // opencode:session is a routing fact (which `ses_…` to resume), not transcript — send it live. connection.send(msg); - }); + }; + + void runOpenCodeTurnOnServe(cmd.params, SERVE_RUNNER_CONFIG, onMessage); reply({ type: 'opencode:spawned', id: cmd.id, sessionKey }); break; } case 'opencode:list': - reply({ type: 'opencode:sessions', id: cmd.id, sessions: listRunningOpenCodeTurns() }); + reply({ type: 'opencode:sessions', id: cmd.id, sessions: listRunningServeTurns() }); break; case 'opencode:kill': - killOpenCodeTurn(cmd.sessionKey); + // An INTERRUPT, not a kill: the turn stops and the session survives, so the conversation can be + // continued rather than only re-opened. + void killServeTurn(cmd.sessionKey, SERVE_RUNNER_CONFIG); sessionLog.drop(cmd.sessionKey); break; default: @@ -215,7 +228,8 @@ async function shutdown(signal: string) { // Before the connection goes: killing a turn produces an event, and that event's durable write travels // over this socket. Tearing it down first would stop every turn silently — the exact outcome this is // here to prevent. - const stopped = stopAllOpenCodeTurns(`The OpenCode sidecar restarted (${signal}), so this turn stopped.`); + const message = `The OpenCode sidecar restarted (${signal}), so this turn stopped.`; + const stopped = stopAllServeTurns(message); if (stopped > 0) { console.log(`[opencode] ${signal} received, stopping ${stopped} in-flight turn(s)...`); await Promise.race([sessionLog.flush(), Bun.sleep(SHUTDOWN_FLUSH_MS)]); diff --git a/src/servers/sidecar/opencode/runner.test.ts b/src/servers/sidecar/opencode/runner.test.ts deleted file mode 100644 index 5932b238..00000000 --- a/src/servers/sidecar/opencode/runner.test.ts +++ /dev/null @@ -1,235 +0,0 @@ -import { afterAll, describe, expect, it } from 'bun:test'; -import { mkdtempSync, writeFileSync, chmodSync, rmSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import type { RunnerMessage } from './runner'; -import { - killOpenCodeTurn, - listRunningOpenCodeTurns, - mapRunLine, - runOpenCodeTurn, - stopAllOpenCodeTurns, -} from './runner'; - -// The first tests on the OpenCode path, which had none. -// -// This covers the NDJSON → ChatEvent mapping from `opencode run --format json`, which is the piece most -// likely to break against a new release: the event shape is not a documented contract, and this project -// already runs two different opencode versions across two machines (1.18.11 on the Mac, 1.17.9 on alpha -// — measured 2026-08-10; this file previously had them the wrong way round). -// Before this, a shape change would have surfaced as a silently empty or malformed turn. -// -// The fixtures below are the shapes the live 1.17.9 binary emits. If one of these tests fails after an -// upgrade, re-read the real NDJSON from the installed binary before editing the expectation — the test -// failing is the feature. - -describe('mapRunLine — what it forwards', () => { - it('forwards a text part as final text, because run emits whole blocks not deltas', () => { - const out = mapRunLine(JSON.stringify({ type: 'text', sessionID: 'ses_1', part: { type: 'text', text: 'hello' } })); - - expect(out?.events).toEqual([{ type: 'text', text: 'hello' }]); - expect(out?.sessionId).toBe('ses_1'); - }); - - it('drops an empty text part rather than emitting a blank message', () => { - expect(mapRunLine(JSON.stringify({ type: 'text', part: { type: 'text', text: '' } }))?.events).toEqual([]); - }); - - it('splits one resolved tool part into start then result', () => { - // `run` hands over the tool already finished, unlike the SSE path which saw it transition. Both - // events are still emitted so the UI renders a call and its output rather than output alone. - const out = mapRunLine( - JSON.stringify({ - type: 'tool_use', - sessionID: 'ses_1', - part: { - type: 'tool', - tool: 'bash', - callID: 'call_1', - state: { status: 'completed', input: { cmd: 'ls' }, output: 'a\nb' }, - }, - }), - ); - - expect(out?.events).toEqual([ - { type: 'tool:start', toolCallId: 'call_1', toolName: 'bash', toolInput: { cmd: 'ls' } }, - { type: 'tool:result', toolCallId: 'call_1', output: 'a\nb', isError: false }, - ]); - }); - - it('reports a failed tool with its error as the output', () => { - const out = mapRunLine( - JSON.stringify({ - type: 'tool_use', - part: { - type: 'tool', - tool: 'bash', - callID: 'call_2', - state: { status: 'error', error: 'boom', output: 'ignored' }, - }, - }), - ); - - // The error replaces the output rather than sitting beside it: a failed call has nothing useful in - // `output`, and showing both would put a stale value under an error. - expect(out?.events[1]).toEqual({ type: 'tool:result', toolCallId: 'call_2', output: 'boom', isError: true }); - }); - - it('ignores a tool part with no callID, which cannot be correlated to a result', () => { - const out = mapRunLine(JSON.stringify({ type: 'tool_use', part: { type: 'tool', tool: 'bash' } })); - expect(out?.events).toEqual([]); - }); - - it('says nothing about step_start and other unknown types', () => { - expect(mapRunLine(JSON.stringify({ type: 'step_start', sessionID: 'ses_1' }))?.events).toEqual([]); - expect(mapRunLine(JSON.stringify({ type: 'something_new_in_1_19' }))?.events).toEqual([]); - }); - - it('returns null for a non-JSON line, because opencode interleaves plain logs with the stream', () => { - expect(mapRunLine('Shell cwd was reset to /somewhere')).toBeNull(); - expect(mapRunLine('')).toBeNull(); - }); -}); - -describe('mapRunLine — cost', () => { - it('reports per-step tokens and cost as a delta for the caller to accumulate', () => { - const out = mapRunLine( - JSON.stringify({ type: 'step_finish', part: { tokens: { input: 10, output: 4 }, cost: 0.002 } }), - ); - - expect(out?.costDelta).toEqual({ inputTokens: 10, outputTokens: 4, totalUSD: 0.002 }); - }); - - it('is zero for every line that is not a step_finish, so accumulation is unconditional', () => { - const text = mapRunLine(JSON.stringify({ type: 'text', part: { type: 'text', text: 'x' } })); - expect(text?.costDelta).toEqual({ inputTokens: 0, outputTokens: 0, totalUSD: 0 }); - }); - - it('treats missing tokens and a missing cost as zero rather than NaN', () => { - // A NaN here would propagate into the turn total and render as an empty or broken cost in the UI. - const out = mapRunLine(JSON.stringify({ type: 'step_finish', part: {} })); - expect(out?.costDelta).toEqual({ inputTokens: 0, outputTokens: 0, totalUSD: 0 }); - }); -}); - -describe('mapRunLine — session id', () => { - it('reports the session id from any line that carries one', () => { - // The caller emits it only the first time; this function has no memory, which is what makes it - // testable line by line. - expect(mapRunLine(JSON.stringify({ type: 'step_start', sessionID: 'ses_abc' }))?.sessionId).toBe('ses_abc'); - expect(mapRunLine(JSON.stringify({ type: 'text', part: { text: 'hi' } }))?.sessionId).toBeUndefined(); - }); -}); - -// ── The turn lifecycle, which `mapRunLine`'s extraction deliberately left with the caller ── -// -// Emitting the session id once and accumulating cost were kept out of the pure mapper because they span -// lines. That was right, and it left them as the only untested logic on this path — which is exactly -// where the supersede defect lived: a replaced turn's `proc.exited` fired late and ran `finish()` against -// the turn that had replaced it. -// -// No real `opencode` needed. `RunnerConfig.bin` is the only injection point, so a shell script that -// outlives the test stands in for a turn that is still generating. - -const stubDir = mkdtempSync(join(tmpdir(), 'oc-runner-test-')); - -/** Stands in for a turn that is still generating. */ -const STUB_BIN = join(stubDir, 'fake-opencode'); -writeFileSync(STUB_BIN, '#!/bin/sh\nsleep 30\n'); -chmodSync(STUB_BIN, 0o755); - -/** Stands in for a turn that fails on its own. A script, not `/bin/false` — that is `/usr/bin/false` on macOS. */ -const FAILING_BIN = join(stubDir, 'failing-opencode'); -writeFileSync(FAILING_BIN, '#!/bin/sh\necho "boom" >&2\nexit 1\n'); -chmodSync(FAILING_BIN, 0o755); - -const CONFIG = { bin: STUB_BIN, fallbackCwd: stubDir }; - -afterAll(() => { - rmSync(stubDir, { recursive: true, force: true }); -}); - -/** - * Wait for something to BECOME true, rather than sleeping a guessed interval and hoping. - * - * These tests spawn real processes, so every "has it happened yet" is at the mercy of machine load — - * and a fixed `sleep(750)` duly failed once on a box that was busy running opencode probes. Polling - * makes a slow machine slow instead of red. Absence assertions still need a fixed wait, since there is - * no event to wait for; those are marked where they appear. - */ -async function waitFor(what: () => boolean, timeoutMs = 8000): Promise { - const deadline = Date.now() + timeoutMs; - while (Date.now() < deadline) { - if (what()) return; - await Bun.sleep(25); - } -} - -describe('runOpenCodeTurn — a second turn on a live session', () => { - it('lets the replacement keep the session: no error, still listed, still killable', async () => { - const sessionKey = 'sess-supersede'; - const messages: RunnerMessage[] = []; - const emit = (msg: RunnerMessage) => messages.push(msg); - - runOpenCodeTurn({ sessionKey, prompt: 'first', cwd: stubDir }, CONFIG, emit); - runOpenCodeTurn({ sessionKey, prompt: 'second', cwd: stubDir }, CONFIG, emit); - - // Wait for the superseded child to actually die — its exit handler is what used to reach across. - // Absence assertion, so there is no event to wait for: give it a generous fixed window instead, and - // wait on something observable (the kill landing) rather than purely on the clock. - await waitFor(() => listRunningOpenCodeTurns().length === 1); - await Bun.sleep(1500); - - // 1. Nothing is emitted for a turn the system replaced on purpose. This one mattered most: the emit - // is committed to chat_session_events by the sidecar, so a false "OpenCode exited" became history. - expect(messages.filter((m) => m.type === 'opencode:event' && m.event.type === 'error')).toEqual([]); - - // 2. The replacement is still enumerable — the Live panel's whole purpose. - expect(listRunningOpenCodeTurns()).toContainEqual({ sessionKey }); - - // 3. And still reachable by the stop button, rather than orphaned with no handle. - killOpenCodeTurn(sessionKey); - await waitFor(() => messages.some((m) => m.type === 'opencode:event' && m.event.type === 'stopped')); - expect(listRunningOpenCodeTurns()).not.toContainEqual({ sessionKey }); - expect(messages.some((m) => m.type === 'opencode:event' && m.event.type === 'stopped')).toBe(true); - }); - - it('stops every in-flight turn on shutdown, and says why in the transcript', async () => { - // `opencode run` is spawned, not supervised, so a sidecar restart used to leave these alive: - // reparented, still spending tokens and still writing files, with nothing reading their output. - const messages: RunnerMessage[] = []; - const emit = (m: RunnerMessage) => messages.push(m); - - runOpenCodeTurn({ sessionKey: 'sess-x', prompt: 'a', cwd: stubDir }, CONFIG, emit); - runOpenCodeTurn({ sessionKey: 'sess-y', prompt: 'b', cwd: stubDir }, CONFIG, emit); - expect(listRunningOpenCodeTurns()).toHaveLength(2); - - const stopped = stopAllOpenCodeTurns('sidecar restarted'); - - // Synchronous on purpose: the caller is about to call process.exit, so nothing that waits for - // `proc.exited` would ever run. A turn killed that way just trails off mid-tool-call. - expect(stopped).toBe(2); - expect(listRunningOpenCodeTurns()).toEqual([]); - const errors = messages.filter((m) => m.type === 'opencode:event' && m.event.type === 'error'); - expect(errors).toHaveLength(2); - - // And the late exits must not add a second, worse ending on top of the one just written. - await Bun.sleep(500); - expect(messages.filter((m) => m.type === 'opencode:event' && m.event.type === 'error')).toHaveLength(2); - expect(messages.some((m) => m.type === 'opencode:event' && m.event.type === 'stopped')).toBe(false); - }); - - it('still reports a turn that dies on its own, rather than swallowing every exit', async () => { - // The guard must not overreach: an ordinary failure is still an error the user needs to see. - const sessionKey = 'sess-solo'; - const messages: RunnerMessage[] = []; - - runOpenCodeTurn({ sessionKey, prompt: 'only', cwd: stubDir }, { ...CONFIG, bin: FAILING_BIN }, (m) => - messages.push(m), - ); - await waitFor(() => messages.some((m) => m.type === 'opencode:event' && m.event.type === 'error')); - - expect(messages.some((m) => m.type === 'opencode:event' && m.event.type === 'error')).toBe(true); - expect(listRunningOpenCodeTurns()).not.toContainEqual({ sessionKey }); - }); -}); diff --git a/src/servers/sidecar/opencode/runner.ts b/src/servers/sidecar/opencode/runner.ts deleted file mode 100644 index 6f836bca..00000000 --- a/src/servers/sidecar/opencode/runner.ts +++ /dev/null @@ -1,388 +0,0 @@ -import { existsSync } from 'node:fs'; -import type { Subprocess } from 'bun'; -import type { ChatEvent, MessageCost } from '../../api/chat/types'; -import type { OpenCodeRunParams } from '../protocol'; - -// Drives one chat turn by spawning `opencode run … --format json` and mapping its newline-delimited -// JSON events to the shared ChatEvent contract. This is the reliable path: `--dir ` hard-anchors -// every tool to the chat's working directory, and `run` (unlike serve + POST /message) reports tool -// completion faithfully and exits when the turn is done — no wedged "running" tools. `--dangerously- -// skip-permissions` auto-approves so tools never block on an approval we can't answer over this channel. -// -// A watchdog guards against a genuinely hung child (an interactive prompt, a `sleep`, a network stall): -// an inactivity timer (reset on every stdout chunk) and an absolute per-turn ceiling both kill the -// process and emit a clean `error`, so the UI never sits at "Working…" forever. - -const INACTIVITY_MS = 120_000; // no stdout for this long → assume wedged, kill -const HARD_CAP_MS = 10 * 60_000; // absolute per-turn ceiling - -export type RunnerConfig = { - bin: string; // absolute path to the opencode binary - fallbackCwd: string; // used when params.cwd is missing/nonexistent -}; - -// What a turn reports to the sidecar it runs in. `opencode:event` is deliberately not a wire event any -// more: the sidecar translates each one into a TurnMessage and commits it before officer sees anything, -// so the durable record does not depend on officer being up (see index.ts). -export type RunnerMessage = - | { type: 'opencode:event'; sessionKey: string; event: ChatEvent } - | { type: 'opencode:session'; sessionKey: string; sessionId: string }; - -type Emit = (msg: RunnerMessage) => void; - -type RunHandle = { - proc: Subprocess; - killedByUser: boolean; - /** - * Set when a newer turn has taken this sessionKey over. - * - * A killed process dies asynchronously, so a replaced turn's `proc.exited` fires LONG after its - * replacement is already running and registered under the same key. Without this flag that late - * handler ran the full completion path against the wrong turn: it emitted `OpenCode exited with code - * 143` — which the sidecar commits to `chat_session_events`, so a false failure became permanent - * history — and then deleted its replacement from `running`, which blinded the Live panel, made the - * stop button a no-op, and orphaned a process nothing could reach. - */ - superseded: boolean; - /** - * End this turn from outside the closure that owns it, with a reason. - * - * `killOpenCodeTurn` can kill a process and let `proc.exited` do the rest, because it has time. - * Shutdown does not: the sidecar is about to call `process.exit`, so nothing asynchronous will ever - * run again and a turn killed that way would simply stop mid-sentence, leaving a transcript that - * trails off. Settling synchronously is what puts the explanation in the log before we go. - */ - finish: (event: ChatEvent) => void; -}; - -// One turn per sessionKey; a new turn supersedes any stale process for that key. -const running = new Map(); - -// Shape of `opencode run --format json` events. -// -// Verified live against opencode 1.18.11 (this Mac) and 1.17.9 (alpha) — MEASURED on 2026-08-10, having -// previously been recorded the other way round here: the "this server" in the original note meant alpha, -// and the comment was copied to a machine where it was false. Nothing enforces a version anyway; the -// binary is whatever is installed, and the two machines in this project already differ. -// -// `runner.test.ts` pins the mapping so a shape change fails a test rather than a turn; if it starts -// failing, re-read the NDJSON from the installed binary before editing the test. -type RunPart = { - type?: string; - text?: string; - tool?: string; - callID?: string; - state?: { status?: string; input?: unknown; output?: unknown; error?: unknown }; - tokens?: { input?: number; output?: number }; - cost?: number; -}; -type RunEvent = { type?: string; sessionID?: string; part?: RunPart }; - -export function runOpenCodeTurn(params: OpenCodeRunParams, config: RunnerConfig, emit: Emit): void { - const { sessionKey } = params; - - // Supersede any lingering turn for this session. Mark it BEFORE killing: the flag is what tells its - // own exit handler that this death was intentional and belongs to nobody. - const stale = running.get(sessionKey); - if (stale) { - stale.superseded = true; - try { - stale.proc.kill(); - } catch { - /* already gone */ - } - running.delete(sessionKey); - } - - const args = ['run', '--format', 'json', '--dangerously-skip-permissions']; - if (params.cwd) args.push('--dir', params.cwd); - if (params.model) args.push('--model', params.model); - if (params.resumeSessionId) args.push('--session', params.resumeSessionId); - args.push(params.prompt); - - const cwd = params.cwd && existsSync(params.cwd) ? params.cwd : config.fallbackCwd; - - const proc = Bun.spawn([config.bin, ...args], { - cwd, - stdin: 'ignore', // == /dev/null: `run` hangs waiting on stdin otherwise - stdout: 'pipe', - stderr: 'pipe', - }); - - // `finish` is a placeholder for the few synchronous lines until the real one below exists — it closes - // over `handle`, so the two cannot both be defined first. Nothing can call it in between. - const handle: RunHandle = { proc, killedByUser: false, superseded: false, finish: () => {} }; - running.set(sessionKey, handle); - - let done = false; - let reportedSession = false; - let cost: MessageCost = { inputTokens: 0, outputTokens: 0, totalUSD: 0 }; - let stderrTail = ''; - - const emitEvent = (event: ChatEvent) => emit({ type: 'opencode:event', sessionKey, event }); - - let inactivityTimer: ReturnType | undefined; - - /** - * Retire this turn: stop its watchdogs, release its slot, and optionally say why it ended. - * - * The delete is identity-checked because `sessionKey` is not this turn's to own once it has been - * superseded — the map may already hold a live replacement under that key, and deleting by name alone - * removed it. `null` retires silently, which is what a superseded turn needs: it must still clear its - * timers (an armed 10-minute `hardTimer` would otherwise fire an error at whichever turn holds the key - * by then, reproducing the same cross-talk on a delay) while emitting nothing at all. - */ - const settle = (event: ChatEvent | null) => { - if (done) return; - done = true; - clearTimeout(hardTimer); - if (inactivityTimer) clearTimeout(inactivityTimer); - if (running.get(sessionKey) === handle) running.delete(sessionKey); - if (event) emitEvent(event); - }; - - const finish = (event: ChatEvent) => settle(event); - handle.finish = finish; - - // ── Watchdogs ── - const hardTimer = setTimeout(() => { - try { - proc.kill(); - } catch { - /* already gone */ - } - finish({ type: 'error', message: `OpenCode turn exceeded ${HARD_CAP_MS / 1000}s and was stopped` }); - }, HARD_CAP_MS); - - const bumpInactivity = () => { - if (done) return; - if (inactivityTimer) clearTimeout(inactivityTimer); - inactivityTimer = setTimeout(() => { - try { - proc.kill(); - } catch { - /* already gone */ - } - finish({ - type: 'error', - message: `OpenCode turn stalled (no output for ${INACTIVITY_MS / 1000}s) and was stopped`, - }); - }, INACTIVITY_MS); - }; - bumpInactivity(); - - // ── Capture a tail of stderr for error reporting ── - void (async () => { - const dec = new TextDecoder(); - try { - for await (const chunk of proc.stderr as unknown as AsyncIterable) { - stderrTail = (stderrTail + dec.decode(chunk, { stream: true })).slice(-2000); - } - } catch { - /* stream closed */ - } - })(); - - // ── Parse stdout: newline-delimited JSON events ── - void (async () => { - const dec = new TextDecoder(); - let buf = ''; - try { - for await (const chunk of proc.stdout as unknown as AsyncIterable) { - bumpInactivity(); - buf += dec.decode(chunk, { stream: true }); - let nl: number; - while ((nl = buf.indexOf('\n')) >= 0) { - const line = buf.slice(0, nl).trim(); - buf = buf.slice(nl + 1); - if (line) handleLine(line); - } - } - const last = buf.trim(); - if (last) handleLine(last); - } catch { - /* stream closed / process killed */ - } - })(); - - function handleLine(line: string): void { - // A retired turn says nothing more. Stdout is drained asynchronously, so a killed process can still - // have buffered lines in flight — and for a superseded turn those would be emitted under a - // sessionKey that now belongs to its replacement, interleaving one turn's output into another's. - if (done) return; - - const mapped = mapRunLine(line); - if (!mapped) return; - - // Report the OpenCode session id once, so the API can resume it (`--session`) next turn. - if (!reportedSession && mapped.sessionId) { - reportedSession = true; - emit({ type: 'opencode:session', sessionKey, sessionId: mapped.sessionId }); - } - - cost = { - inputTokens: cost.inputTokens + mapped.costDelta.inputTokens, - outputTokens: cost.outputTokens + mapped.costDelta.outputTokens, - totalUSD: cost.totalUSD + mapped.costDelta.totalUSD, - }; - - for (const event of mapped.events) emitEvent(event); - } - - // ── Completion: process exit is the authoritative turn-done signal ── - void proc.exited.then((code) => { - if (done) return; - // Replaced on purpose: not a result, not an error, and not this turn's session any more. - if (handle.superseded) { - settle(null); - return; - } - if (handle.killedByUser) { - finish({ type: 'stopped' }); - return; - } - if (code === 0) { - finish({ type: 'result', cost }); - return; - } - const tail = stderrTail.trim(); - finish({ - type: 'error', - message: tail ? `OpenCode exited (${code}): ${tail.slice(-500)}` : `OpenCode exited with code ${code}`, - }); - }); -} - -/** - * The turns this process is running right now. - * - * The OpenCode analog of `claude-manager.listSessions`, and deliberately thinner. Claude holds a warm - * session that outlives a turn, so it can report one that is merely open; OpenCode spawns a subprocess - * per turn and has nothing between them. So a session appears here only while it is generating — which - * is exactly the state the Live panel exists to show, and the state that was invisible for OpenCode. - * - * No `pendingTasks`: `opencode run` has no background-task concept, so reporting 0 would suggest a - * capability that does not exist rather than an empty one. - */ -export function listRunningOpenCodeTurns(): { sessionKey: string }[] { - return Array.from(running.keys()).map((sessionKey) => ({ sessionKey })); -} - -/** - * Kill every turn this process is running, because the process itself is going away. - * - * A turn is a child of this sidecar only in the bookkeeping sense: `opencode run` is spawned, not - * supervised, so `pm2 restart officer-opencode` used to leave every in-flight turn ALIVE — reparented, - * still spending tokens, and still writing files as the agent, while the only reader of its stdout had - * exited. The turn's output went nowhere and the transcript simply stopped mid-tool-call, which is - * indistinguishable from the agent hanging. - * - * Both halves matter. Killing the children stops the invisible work; settling them synchronously writes - * a reason into the transcript, so a reload after a restart explains itself instead of trailing off. - * Returns how many were stopped, so the caller can skip the flush wait when there were none. - */ -export function stopAllOpenCodeTurns(message: string): number { - const handles = [...running.values()]; - for (const handle of handles) { - // Suppress the exit handler's own error: this death is accounted for, and `finish` below is the - // account. Without it a late `proc.exited` would be a second, less accurate ending. - handle.killedByUser = true; - try { - handle.proc.kill(); - } catch { - /* already gone */ - } - handle.finish({ type: 'error', message }); - } - return handles.length; -} - -export function killOpenCodeTurn(sessionKey: string): void { - const handle = running.get(sessionKey); - if (!handle) return; - handle.killedByUser = true; - try { - handle.proc.kill(); - } catch { - /* already gone */ - } - // proc.exited fires → finish({ type: 'stopped' }). -} - -// ── The NDJSON → ChatEvent mapping, as a pure function ── -// -// Extracted from `handleLine` so it can be tested without spawning a binary. This is the piece most -// likely to break against a new OpenCode release — the event shape is not a stable contract and this -// project already runs two different versions across two machines — and it was the only untested part -// of the path. `runner.test.ts` pins it. -// -// Pure by construction: it takes a line and returns what should happen, holding no state. The caller -// owns the two stateful concerns, because they span lines rather than belonging to one: emitting the -// session id exactly once, and accumulating cost across steps. - -export type MappedRunLine = { - /** Present on any event that names a session; the caller emits it only the first time. */ - sessionId?: string; - /** Events to forward, in order. A tool part yields `tool:start` then `tool:result`. */ - events: ChatEvent[]; - /** Per-step tokens/cost to add to the turn total. Zeroes for every non-`step_finish` line. */ - costDelta: { inputTokens: number; outputTokens: number; totalUSD: number }; -}; - -const NO_COST = { inputTokens: 0, outputTokens: 0, totalUSD: 0 }; - -/** `null` for a line that is not JSON at all — `opencode` interleaves plain log lines with the stream. */ -export function mapRunLine(line: string): MappedRunLine | null { - let evt: RunEvent; - try { - evt = JSON.parse(line) as RunEvent; - } catch { - return null; // non-JSON log line - } - - const sessionId = evt.sessionID; - const events: ChatEvent[] = []; - - switch (evt.type) { - case 'text': { - // `run` emits complete text parts (not token deltas) — forward each as final text. - const text = evt.part?.text; - if (typeof text === 'string' && text.length > 0) events.push({ type: 'text', text }); - return { sessionId, events, costDelta: NO_COST }; - } - case 'tool_use': { - // In `run`, the tool part arrives already resolved (status + output). Emit start then result. - const part = evt.part; - if (!part || part.type !== 'tool' || !part.callID) return { sessionId, events, costDelta: NO_COST }; - const st = part.state ?? {}; - events.push({ - type: 'tool:start', - toolCallId: part.callID, - toolName: part.tool ?? 'tool', - toolInput: (st.input as Record) ?? {}, - }); - const isError = st.status === 'error'; - events.push({ - type: 'tool:result', - toolCallId: part.callID, - output: String((isError ? st.error : st.output) ?? ''), - isError, - }); - return { sessionId, events, costDelta: NO_COST }; - } - case 'step_finish': { - const t = evt.part?.tokens; - return { - sessionId, - events, - costDelta: { - inputTokens: t?.input ?? 0, - outputTokens: t?.output ?? 0, - totalUSD: typeof evt.part?.cost === 'number' ? evt.part.cost : 0, - }, - }; - } - default: - return { sessionId, events, costDelta: NO_COST }; // step_start etc. — nothing to forward - } -} diff --git a/src/servers/sidecar/opencode/serve-events.test.ts b/src/servers/sidecar/opencode/serve-events.test.ts new file mode 100644 index 00000000..5391ff25 --- /dev/null +++ b/src/servers/sidecar/opencode/serve-events.test.ts @@ -0,0 +1,196 @@ +import { describe, expect, it } from 'bun:test'; +import { isKnownServeEvent, mapServeEvent } from './serve-events'; + +// Fixtures are VERBATIM captures from opencode 1.18.16 — one real turn that ran `echo hello-from-tool` +// through the bash tool and then answered in prose. Ids and timestamps are as they arrived. +// +// They are real for the same reason `runner.test.ts`'s are: this mapping's only job is to match a shape +// nobody documents, so a hand-written fixture would test my imagination rather than the binary. If one +// of these fails after an upgrade, re-capture before editing the expectation. + +const SESSION = 'ses_01346bde5ffeB4hWdcQiq1BQUG'; +const ASSISTANT = 'msg_fecb96b3e0010o8smEyjxT2noD'; +const CALL = 'toolu_01TyLVFSSXus2eoyANdjieWW'; + +describe('mapServeEvent — text', () => { + it('turns a text delta into a delta, because that is what makes output appear as it is typed', () => { + const out = mapServeEvent({ + type: 'session.next.text.delta', + data: { timestamp: 1, sessionID: SESSION, assistantMessageID: ASSISTANT, textID: 'text-0', delta: 'It' }, + } as never); + + expect(out?.events).toEqual([{ type: 'delta', text: 'It' }]); + expect(out?.sessionId).toBe(SESSION); + }); + + it('turns the ended text into the committed block', () => { + const out = mapServeEvent({ + type: 'session.next.text.ended', + data: { sessionID: SESSION, textID: 'text-0', text: 'It printed **hello-from-tool**.' }, + } as never); + + expect(out?.events).toEqual([{ type: 'text', text: 'It printed **hello-from-tool**.' }]); + }); + + it('says nothing for text.started, which carries no text at all', () => { + // Verified against the capture: `text.started` has `{textID}` and nothing else. Emitting an empty + // text here would put a blank assistant bubble on screen before a single token arrived. + const out = mapServeEvent({ + type: 'session.next.text.started', + data: { sessionID: SESSION, textID: 'text-0' }, + } as never); + expect(out?.events).toEqual([]); + }); + + it('drops an empty delta rather than emitting nothing-shaped events', () => { + expect(mapServeEvent({ type: 'session.next.text.delta', data: { delta: '' } } as never)?.events).toEqual([]); + }); +}); + +describe('mapServeEvent — tools', () => { + it('starts the tool row on tool.called, where the input is finally a real object', () => { + const out = mapServeEvent({ + type: 'session.next.tool.called', + data: { + sessionID: SESSION, + assistantMessageID: ASSISTANT, + callID: CALL, + tool: 'bash', + input: { command: 'echo hello-from-tool' }, + }, + } as never); + + expect(out?.events).toEqual([ + { type: 'tool:start', toolCallId: CALL, toolName: 'bash', toolInput: { command: 'echo hello-from-tool' } }, + ]); + }); + + it('stays silent through the input stream, which is half-parsed JSON', () => { + // This is the reason `tool:start` waits for `tool.called`. The real delta below is a fragment of a + // JSON object; rendering a tool row with `{"comman` as its arguments would be worse than waiting. + const started = mapServeEvent({ + type: 'session.next.tool.input.started', + data: { sessionID: SESSION, callID: CALL, name: 'bash' }, + } as never); + const delta = mapServeEvent({ + type: 'session.next.tool.input.delta', + data: { sessionID: SESSION, callID: CALL, delta: '{"comman' }, + } as never); + + expect(started?.events).toEqual([]); + expect(delta?.events).toEqual([]); + }); + + it('flattens the success content blocks into one output string', () => { + const out = mapServeEvent({ + type: 'session.next.tool.success', + data: { + sessionID: SESSION, + callID: CALL, + structured: { exit: 0, truncated: false }, + content: [{ type: 'text', text: 'hello-from-tool' }], + }, + } as never); + + expect(out?.events).toEqual([{ type: 'tool:result', toolCallId: CALL, output: 'hello-from-tool', isError: false }]); + }); + + it('reports a failed tool with its error as the output', () => { + const out = mapServeEvent({ + type: 'session.next.tool.failed', + data: { sessionID: SESSION, callID: CALL, error: { message: 'command not found' } }, + } as never); + + expect(out?.events).toEqual([ + { type: 'tool:result', toolCallId: CALL, output: 'command not found', isError: true }, + ]); + }); + + it('ignores a tool event with no callID, which cannot be correlated', () => { + expect(mapServeEvent({ type: 'session.next.tool.called', data: { tool: 'bash' } } as never)?.events).toEqual([]); + expect(mapServeEvent({ type: 'session.next.tool.success', data: {} } as never)?.events).toEqual([]); + }); +}); + +describe('mapServeEvent — accounting', () => { + it('reports per-step tokens and cost for the caller to accumulate', () => { + const out = mapServeEvent({ + type: 'session.next.step.ended', + data: { + sessionID: SESSION, + finish: 'tool-calls', + cost: 0.0042, + tokens: { input: 3, output: 57, reasoning: 0, cache: { read: 3850, write: 0 } }, + }, + } as never); + + expect(out?.costDelta).toEqual({ inputTokens: 3, outputTokens: 57, totalUSD: 0.0042 }); + }); + + it('emits nothing terminal for a step that merely ended in tool calls', () => { + // `finish: "tool-calls"` is a step boundary MID-turn — the model stopped to run a tool and will + // continue. Treating it as the end of the turn would cut every tool-using conversation in half. + const out = mapServeEvent({ + type: 'session.next.step.ended', + data: { sessionID: SESSION, finish: 'tool-calls', cost: 0, tokens: { input: 3, output: 57 } }, + } as never); + + expect(out?.events).toEqual([]); + }); + + it('treats missing tokens and cost as zero rather than NaN', () => { + const out = mapServeEvent({ type: 'session.next.step.ended', data: { sessionID: SESSION } } as never); + expect(out?.costDelta).toEqual({ inputTokens: 0, outputTokens: 0, totalUSD: 0 }); + }); + + it('surfaces a failed step as an error carrying the provider message', () => { + // The real one, from a probe where the provider was down. + const out = mapServeEvent({ + type: 'session.next.step.failed', + data: { sessionID: SESSION, error: { type: 'unknown', message: 'Provider request failed with HTTP 503' } }, + } as never); + + expect(out?.events).toEqual([{ type: 'error', message: 'Provider request failed with HTTP 503' }]); + }); +}); + +describe('mapServeEvent — envelope', () => { + it('carries the durable cursor when reading the durable stream, and not otherwise', () => { + // The per-session stream stamps `durable.seq`; the global live stream does not. The caller needs + // that difference to know what it may commit and replay. + const durable = mapServeEvent({ + type: 'session.next.text.ended', + durable: { seq: 12 }, + data: { sessionID: SESSION, text: 'hi' }, + } as never); + const live = mapServeEvent({ type: 'session.next.text.ended', data: { sessionID: SESSION, text: 'hi' } } as never); + + expect(durable?.seq).toBe(12); + expect(live?.seq).toBeUndefined(); + }); + + it('always reports the session id, because the live stream is global', () => { + // Not a detail: `/api/event` carries EVERY session's events, so a consumer that forgets to filter + // splices one conversation into another. + for (const type of ['session.next.text.delta', 'session.next.tool.called', 'session.next.step.ended']) { + expect(mapServeEvent({ type, data: { sessionID: SESSION, delta: 'x', callID: CALL } } as never)?.sessionId).toBe( + SESSION, + ); + } + }); + + it('returns null for something that is not an event at all', () => { + expect(mapServeEvent(null)).toBeNull(); + expect(mapServeEvent(undefined)).toBeNull(); + expect(mapServeEvent({} as never)).toBeNull(); + }); + + it('is quiet about an unknown future type, but does not claim to know it', () => { + // Forward compatibility with a silent tell: an unrecognised type produces no events, and + // `isKnownServeEvent` is how a caller notices a release added something worth mapping. + expect(mapServeEvent({ type: 'session.next.something_new_in_1_19', data: {} } as never)?.events).toEqual([]); + expect(isKnownServeEvent('session.next.something_new_in_1_19')).toBe(false); + expect(isKnownServeEvent('session.next.text.delta')).toBe(true); + expect(isKnownServeEvent(undefined)).toBe(false); + }); +}); diff --git a/src/servers/sidecar/opencode/serve-events.ts b/src/servers/sidecar/opencode/serve-events.ts new file mode 100644 index 00000000..7b1162dc --- /dev/null +++ b/src/servers/sidecar/opencode/serve-events.ts @@ -0,0 +1,231 @@ +import type { ChatEvent } from '../../api/chat/types'; + +// `session.next.*` → ChatEvent. Phase A of the serve migration (docs/opencode-serve-migration-plan.md). +// +// NOTHING ROUTES THROUGH THIS YET. Turns still run as `opencode run` subprocesses via runner.ts; this is +// the mapping half of the serve path, written and pinned first so the switch-over is not also the moment +// the parsing is discovered to be wrong. +// +// ── The two streams, which is the part worth understanding ── +// +// The serve publishes the same turn twice, and the split maps exactly onto what officer already does for +// Claude: +// +// • `GET /api/session/{id}/event?after=` — DURABLE, per session, replayable. Every event carries +// `durable.seq`. Carries whole values (`text.ended` with the full text) and NO deltas. This is the +// transcript: what belongs in `chat_session_events`, and what a reconnecting browser replays. +// +// • `GET /api/event` — LIVE, GLOBAL, ephemeral. Carries the deltas +// (`text.delta`, `tool.input.delta`) and no cursor. This is what makes text appear as it is typed. +// +// Measured, not inferred: the same turn produced 13 events on the durable stream and 21 on the live one, +// the difference being 3 `text.delta` and 5 `tool.input.delta`. Reading only the durable stream — which +// is what I did first — makes it look like the serve cannot stream at all, and would have quietly killed +// the main reason for migrating. +// +// The live stream being GLOBAL is the real cost of this design: it carries every session's events, so a +// consumer must filter on `sessionID` and cannot assume it owns the socket. +// +// ── Fixture provenance ── +// +// Every shape here was captured from opencode 1.18.16 running a real turn that used the bash tool and +// then answered in prose. See `serve-events.test.ts`, whose fixtures are verbatim captures. + +/** The envelope every `session.next.*` event arrives in. `durable` is present only on the durable stream. */ +export type ServeEvent = { + type?: string; + durable?: { seq?: number }; + data?: { + sessionID?: string; + messageID?: string; + assistantMessageID?: string; + callID?: string; + textID?: string; + name?: string; + tool?: string; + input?: Record; + delta?: string; + text?: string; + content?: { type?: string; text?: string }[]; + structured?: Record; + error?: { message?: string; type?: string }; + finish?: string; + cost?: number; + tokens?: { input?: number; output?: number }; + model?: { id?: string; providerID?: string }; + }; +}; + +export type MappedServeEvent = { + /** The session this belongs to. The live stream is global, so a consumer MUST filter on it. */ + sessionId?: string; + /** The durable cursor, when this came from the durable stream. Absent on live events. */ + seq?: number; + events: ChatEvent[]; + costDelta: { inputTokens: number; outputTokens: number; totalUSD: number }; + /** + * Present only on `step.ended`, and the answer to "is the turn over". + * + * A subprocess turn ends by exiting; a serve turn has no such moment, so this is the only signal. + * `"tool-calls"` means the model paused to run a tool and WILL continue — treating it as terminal cuts + * every tool-using conversation in half. `"stop"` is the real end. Anything else (e.g. `"length"`) is + * also an ending, so the caller should test for "not tool-calls" rather than for "stop". + */ + stepFinish?: string; +}; + +const NO_COST = { inputTokens: 0, outputTokens: 0, totalUSD: 0 }; + +/** `tool.success` returns content blocks; officer's `tool:result` wants one string. */ +function textOfContent(content: { type?: string; text?: string }[] | undefined): string { + if (!Array.isArray(content)) return ''; + return content + .map((c) => (typeof c?.text === 'string' ? c.text : '')) + .filter(Boolean) + .join('\n'); +} + +/** + * One serve event in, whatever officer should do about it out. + * + * Pure by construction, exactly like `mapRunLine`: no accumulation, no memory, no I/O. The caller owns + * anything that spans events — summing cost across steps, and deciding whether it is reading the live or + * the durable stream. + * + * Returns `null` for an event that is not ours to interpret (another session's, or an envelope with no + * type). Everything recognised but deliberately ignored returns an empty `events` array instead, so + * "we saw it and had nothing to say" stays distinguishable from "we did not understand it". + */ +export function mapServeEvent(evt: ServeEvent | null | undefined): MappedServeEvent | null { + if (!evt || typeof evt.type !== 'string') return null; + + const d = evt.data ?? {}; + const base = { sessionId: d.sessionID, seq: evt.durable?.seq }; + const nothing = (): MappedServeEvent => ({ ...base, events: [], costDelta: NO_COST }); + + switch (evt.type) { + // ── Text ── + case 'session.next.text.delta': { + // Live stream only. `delta` is the increment, never the whole value. + const text = d.delta; + if (typeof text !== 'string' || text.length === 0) return nothing(); + return { ...base, events: [{ type: 'delta', text }], costDelta: NO_COST }; + } + case 'session.next.text.ended': { + // The committed block. Appears on BOTH streams, which is intended: officer's client treats `text` + // as the value that supersedes whatever the deltas built up, the same as the Claude path. + const text = d.text; + if (typeof text !== 'string' || text.length === 0) return nothing(); + return { ...base, events: [{ type: 'text', text }], costDelta: NO_COST }; + } + + // ── Tools ── + // + // `tool:start` is emitted on `tool.called`, NOT on `tool.input.started`, because only `tool.called` + // carries the resolved `input` object. `tool.input.started` knows the name but the arguments are + // still being streamed a few characters at a time (`tool.input.delta`: `{"comman`), and a tool row + // rendered with half-parsed JSON as its arguments is worse than one that appears a moment later. + case 'session.next.tool.called': { + const toolCallId = d.callID; + if (!toolCallId) return nothing(); + return { + ...base, + events: [ + { + type: 'tool:start', + toolCallId, + toolName: d.tool ?? d.name ?? 'tool', + toolInput: (d.input as Record) ?? {}, + }, + ], + costDelta: NO_COST, + }; + } + case 'session.next.tool.success': { + const toolCallId = d.callID; + if (!toolCallId) return nothing(); + return { + ...base, + events: [{ type: 'tool:result', toolCallId, output: textOfContent(d.content), isError: false }], + costDelta: NO_COST, + }; + } + case 'session.next.tool.failed': { + const toolCallId = d.callID; + if (!toolCallId) return nothing(); + // The error replaces the output rather than sitting beside it — same call the run path makes. + const message = d.error?.message ?? textOfContent(d.content) ?? ''; + return { + ...base, + events: [{ type: 'tool:result', toolCallId, output: String(message), isError: true }], + costDelta: NO_COST, + }; + } + + // ── Step accounting ── + case 'session.next.step.ended': { + // Per-step tokens and cost, to be summed by the caller. `finish: "tool-calls"` is a step boundary + // mid-turn, not the end of the turn — the turn has ended when the prompt is fully answered, which + // is a caller-level question, so nothing terminal is emitted here. + const t = d.tokens; + return { + ...base, + events: [], + stepFinish: typeof d.finish === 'string' ? d.finish : undefined, + costDelta: { + inputTokens: t?.input ?? 0, + outputTokens: t?.output ?? 0, + totalUSD: typeof d.cost === 'number' ? d.cost : 0, + }, + }; + } + case 'session.next.step.failed': { + const message = d.error?.message ?? 'OpenCode step failed'; + return { ...base, events: [{ type: 'error', message }], costDelta: NO_COST }; + } + + // ── Seen and deliberately silent ── + // + // Named rather than swept into `default` so that a genuinely NEW event type still lands in the + // unknown bucket, where it can be noticed. + case 'session.next.prompt.admitted': + case 'session.next.prompted': + case 'session.next.step.started': + case 'session.next.text.started': + case 'session.next.tool.input.started': + case 'session.next.tool.input.delta': + case 'session.next.tool.input.ended': + case 'session.next.tool.progress': + case 'session.next.model.switched': + case 'session.next.agent.switched': + case 'server.connected': + return nothing(); + + default: + return nothing(); + } +} + +/** Whether a type is one this mapper knows about — for logging what a new opencode release added. */ +export const isKnownServeEvent = (type: string | undefined): boolean => + typeof type === 'string' && (type === 'server.connected' || KNOWN.has(type)); + +const KNOWN = new Set([ + 'session.next.text.delta', + 'session.next.text.ended', + 'session.next.text.started', + 'session.next.tool.called', + 'session.next.tool.success', + 'session.next.tool.failed', + 'session.next.tool.input.started', + 'session.next.tool.input.delta', + 'session.next.tool.input.ended', + 'session.next.tool.progress', + 'session.next.step.started', + 'session.next.step.ended', + 'session.next.step.failed', + 'session.next.prompt.admitted', + 'session.next.prompted', + 'session.next.model.switched', + 'session.next.agent.switched', +]); diff --git a/src/servers/sidecar/opencode/serve-runner.ts b/src/servers/sidecar/opencode/serve-runner.ts new file mode 100644 index 00000000..3812ff79 --- /dev/null +++ b/src/servers/sidecar/opencode/serve-runner.ts @@ -0,0 +1,351 @@ +import type { ChatEvent, MessageCost } from '../../api/chat/types'; +import type { OpenCodeRunParams } from '../protocol'; +import { mapServeEvent } from './serve-events'; + +// How an OpenCode turn runs. The only way, since 2026-08-10. +// +// It used to be `opencode run --format json`, a subprocess per turn with `stdin: 'ignore'`. Everything +// that path could not do followed from that one closed pipe: no token streaming, no mid-turn injection, +// no queue, and a stop that could only kill the session rather than interrupt it. The serve offers all +// four as primitives, each verified end to end before the subprocess was deleted +// (docs/opencode-fork-decision.md, docs/opencode-serve-migration-plan.md). +// +// There is no fallback engine any more. If this path breaks, the recovery is git, not a config flag — +// a deliberate choice made while nothing depended on OpenCode. +// +// ── One global stream, demultiplexed ── +// +// The serve publishes each turn on two streams (docs/opencode-serve-migration-plan.md). This reads the +// LIVE one, `GET /api/event`, because it is a strict superset of the durable stream's content — same +// `tool.called`, `tool.success`, `step.ended`, `text.ended`, PLUS the deltas — and deltas are the point. +// +// It is GLOBAL: one socket carries every session on the box, so everything here filters on `sessionID`. +// Forgetting that would splice one conversation into another. There is exactly one subscription for the +// process, opened on the first turn and shared, because opening one per turn would multiply the same +// firehose by the number of turns. +// +// What this loses versus the durable stream is `durable.seq`, i.e. replay-after-the-fact. That matters +// for surviving an officer restart mid-turn and is deliberately NOT in Phase B: the sidecar commits +// every event to `chat_session_events` as it arrives (unchanged from the subprocess path), which is the +// same durability guarantee the subprocess had. Reading the durable stream to recover a turn this +// process never saw is its own change. + +type ServeConfig = { + /** The serve's base URL, e.g. http://127.0.0.1:53100 */ + baseUrl: string; + /** Used when a turn names no cwd — same fallback the subprocess runner applies. */ + fallbackCwd: string; +}; + +/** + * What a turn reports to the sidecar it runs in. + * + * Lived in `runner.ts` until the subprocess path was deleted (Phase D). `opencode:event` is not a wire + * event: the sidecar translates each one into a TurnMessage and commits it to `chat_session_events` + * before officer sees anything, so the durable record does not depend on officer being up. + * `opencode:session` is the routing fact — which `ses_…` to resume — and goes over the wire live. + */ +export type RunnerMessage = + | { type: 'opencode:event'; sessionKey: string; event: ChatEvent } + | { type: 'opencode:session'; sessionKey: string; sessionId: string }; + +type Emit = (msg: RunnerMessage) => void; + +type ServeTurn = { + sessionKey: string; + openCodeSessionId: string; + cost: MessageCost; + done: boolean; + emit: Emit; + finish: (event: ChatEvent) => void; +}; + +/** Live turns, by OpenCode session id — the id the event stream speaks. */ +const byOpenCodeId = new Map(); +/** The same turns by officer's key, which is what `kill` and the Live panel use. */ +const bySessionKey = new Map(); + +// ── The one shared subscription ── + +let streamStarted = false; + +/** + * Read `/api/event` forever, routing each event to the turn that owns it. + * + * Reconnects on drop with a fixed delay. A serve restart, a network blip or the stream simply ending + * must not permanently deafen the sidecar — every subsequent turn would hang with no output, which is + * the worst failure this path has, because it looks exactly like a slow model. + */ +function ensureEventStream(config: ServeConfig): void { + if (streamStarted) return; + streamStarted = true; + + void (async () => { + for (;;) { + try { + const res = await fetch(`${config.baseUrl}/api/event`, { headers: { accept: 'text/event-stream' } }); + if (!res.ok || !res.body) throw new Error(`event stream → ${res.status}`); + + const reader = res.body.getReader(); + const decoder = new TextDecoder(); + let buffer = ''; + + for (;;) { + const { done, value } = await reader.read(); + if (done) break; + buffer += decoder.decode(value, { stream: true }); + const lines = buffer.split('\n'); + buffer = lines.pop() ?? ''; + for (const line of lines) { + if (!line.startsWith('data:')) continue; + try { + handleServeEvent(JSON.parse(line.slice(5).trim())); + } catch { + /* a frame we could not parse is not worth killing the stream over */ + } + } + } + } catch (err) { + console.error('[opencode] event stream dropped, reconnecting:', err instanceof Error ? err.message : err); + } + await Bun.sleep(1_000); + } + })(); +} + +function handleServeEvent(raw: unknown): void { + const mapped = mapServeEvent(raw as never); + if (!mapped?.sessionId) return; + + const turn = byOpenCodeId.get(mapped.sessionId); + if (!turn || turn.done) return; // another session's, or one we have already finished + + turn.cost = { + inputTokens: turn.cost.inputTokens + mapped.costDelta.inputTokens, + outputTokens: turn.cost.outputTokens + mapped.costDelta.outputTokens, + totalUSD: turn.cost.totalUSD + mapped.costDelta.totalUSD, + }; + + for (const event of mapped.events) { + // An error from the harness ends the turn: nothing follows a failed step, and leaving the turn open + // would strand the UI on a spinner. + if (event.type === 'error') { + turn.finish(event); + return; + } + turn.emit({ type: 'opencode:event', sessionKey: turn.sessionKey, event }); + } + + // `tool-calls` means the model paused to run a tool and will continue. Anything else is the end. + if (mapped.stepFinish && mapped.stepFinish !== 'tool-calls') { + turn.finish({ type: 'result', cost: turn.cost }); + } +} + +// ── HTTP helpers ── + +async function serveJson(config: ServeConfig, path: string, init: RequestInit & { cwd: string }): Promise { + const { cwd, ...rest } = init; + const res = await fetch(`${config.baseUrl}${path}`, { + ...rest, + headers: { + 'content-type': 'application/json', + // The location is per REQUEST on this surface, not a property of the session. A call without it + // runs against the serve's own directory, which is not where the user's files are. + 'x-opencode-directory': cwd, + ...(rest.headers ?? {}), + }, + signal: AbortSignal.timeout(30_000), + }); + if (!res.ok) throw new Error(`${path} → ${res.status}`); + const text = await res.text(); + if (!text) return null; + // This surface wraps everything in `{data: …}`; the legacy one does not. Reading `.id` off the + // envelope silently yields undefined, which is how an entire afternoon disappeared once. + const body = JSON.parse(text) as { data?: T } | T; + return (body as { data?: T }).data ?? (body as T); +} + +// ── The turn ── + +export async function runOpenCodeTurnOnServe( + params: OpenCodeRunParams, + config: ServeConfig, + emit: Emit, +): Promise { + const { sessionKey } = params; + const cwd = params.cwd || config.fallbackCwd; + + ensureEventStream(config); + + // ── A message arriving while a turn is running is an INJECTION, not a new turn ── + // + // This is where the subprocess and the serve genuinely part company. `opencode run` had no input + // channel, so a second message could only supersede: kill the process, start again, lose the turn. + // The serve takes another prompt into the RUNNING turn, so the right move is to hand it over and keep + // the existing turn exactly as it is. + // + // Keeping the same turn object is the load-bearing part. Retiring it and registering a replacement — + // which is what this did at first — stops officer routing the events the serve is still producing, + // while the serve carries on regardless. The output goes nowhere and the turn looks hung. + // + // `steer` because the user typed it during the turn and means it now; officer's composer already + // treats a send-while-generating as "add this to what you are doing". A prompt sent when nothing is + // running takes `queue`, which is a no-op with an empty queue but never accidentally merges two + // messages into one turn. + const live = bySessionKey.get(sessionKey); + if (live && !live.done) { + try { + await serveJson(config, `/api/session/${live.openCodeSessionId}/prompt`, { + method: 'POST', + body: JSON.stringify({ + prompt: { text: params.prompt, files: promptFiles(params.images) }, + delivery: 'steer', + }), + cwd, + }); + } catch (err) { + // The turn itself is unharmed — only the injection failed — so say so and leave it running. + emit({ + type: 'opencode:event', + sessionKey, + event: { type: 'error', message: `OpenCode would not take that mid-turn: ${errText(err)}` }, + }); + } + return; + } + + let openCodeSessionId = params.resumeSessionId ?? ''; + + try { + if (!openCodeSessionId) { + const created = await serveJson<{ id: string }>(config, '/api/session', { + method: 'POST', + body: JSON.stringify({ location: { directory: cwd } }), + cwd, + }); + if (!created?.id) throw new Error('session create returned no id'); + openCodeSessionId = created.id; + } + + if (params.model) { + // `providerID/modelID`, the same string the subprocess passes to `--model`. + const slash = params.model.indexOf('/'); + const providerID = slash > 0 ? params.model.slice(0, slash) : 'opencode'; + const id = slash > 0 ? params.model.slice(slash + 1) : params.model; + await serveJson(config, `/api/session/${openCodeSessionId}/model`, { + method: 'POST', + body: JSON.stringify({ model: { providerID, id } }), + cwd, + }); + } + } catch (err) { + emit({ + type: 'opencode:event', + sessionKey, + event: { type: 'error', message: `Could not start an OpenCode session: ${errText(err)}` }, + }); + return; + } + + const turn: ServeTurn = { + sessionKey, + openCodeSessionId, + cost: { inputTokens: 0, outputTokens: 0, totalUSD: 0 }, + done: false, + emit, + finish: (event) => retire(turn, event), + }; + byOpenCodeId.set(openCodeSessionId, turn); + bySessionKey.set(sessionKey, turn); + + // Officer learns which `ses_…` to resume next time — a routing fact, not transcript. + emit({ type: 'opencode:session', sessionKey, sessionId: openCodeSessionId }); + + try { + await serveJson(config, `/api/session/${openCodeSessionId}/prompt`, { + method: 'POST', + // `delivery` is stated explicitly because it DEFAULTS to `"steer"`, which injects into a running + // turn. For an ordinary send that is the wrong default — two quick messages would merge into one + // turn instead of running in order. `steer` is Phase C's job, wired to the button that means it. + body: JSON.stringify({ + prompt: { text: params.prompt, files: promptFiles(params.images) }, + delivery: 'queue', + }), + cwd, + }); + } catch (err) { + retire(turn, { type: 'error', message: `OpenCode refused the prompt: ${errText(err)}` }); + } +} + +function retire(turn: ServeTurn, event: ChatEvent | null): void { + if (turn.done) return; + turn.done = true; + if (byOpenCodeId.get(turn.openCodeSessionId) === turn) byOpenCodeId.delete(turn.openCodeSessionId); + if (bySessionKey.get(turn.sessionKey) === turn) bySessionKey.delete(turn.sessionKey); + if (event) turn.emit({ type: 'opencode:event', sessionKey: turn.sessionKey, event }); +} + +const errText = (err: unknown): string => (err instanceof Error ? err.message : String(err)); + +/** + * Images as `prompt.files`, which this surface takes by URI. + * + * **`data:` URIs, not `file://`.** Measured, because the choice is not obvious and the wrong one fails + * at the provider rather than at the API: a `file://` attachment is accepted with a 200 and then dies + * inside the turn with `Anthropic Messages media must contain valid base64`. A `data:` URI round-trips + * and the model describes the image. + * + * This is strictly better than the subprocess path, which has to spill each image to a temp file for + * `--file` and delete it afterwards. Here the bytes go in the request and there is nothing to clean up. + * + * Silently dropping these is exactly defect B4 — the user sees their image in their own bubble and the + * model never receives it — so this exists before the serve path is switched on for anyone, not after. + */ +function promptFiles(images: OpenCodeRunParams['images']): { uri: string; name: string }[] | undefined { + if (!images?.length) return undefined; + return images.map((image, index) => ({ + uri: `data:${image.mediaType || 'image/png'};base64,${image.data}`, + name: `attachment-${index + 1}`, + })); +} + +/** The turns running right now, for `opencode:list` and the Live panel. */ +export function listRunningServeTurns(): { sessionKey: string }[] { + return [...bySessionKey.keys()].map((sessionKey) => ({ sessionKey })); +} + +/** + * Stop a turn without destroying its session — the thing the subprocess path cannot do. + * + * `POST /interrupt` leaves the conversation intact and resumable, where killing a subprocess ended it. + */ +export async function killServeTurn(sessionKey: string, config: ServeConfig): Promise { + const turn = bySessionKey.get(sessionKey); + if (!turn) return; + try { + await fetch(`${config.baseUrl}/api/session/${turn.openCodeSessionId}/interrupt`, { + method: 'POST', + headers: { 'x-opencode-directory': config.fallbackCwd }, + signal: AbortSignal.timeout(10_000), + }); + } catch { + /* interrupt is best-effort; the turn is retired either way so the UI is never stuck */ + } + retire(turn, { type: 'stopped' }); +} + +/** + * Retire every live turn, for shutdown. + * + * Note what this does NOT do, and why it is right: the turns keep running inside the serve, which is a + * separate process and survives us. Officer stops routing them and says so in the transcript. When the + * subprocess ran turns, shutdown had to kill children or they were orphaned; here the work is somebody + * else's and killing it would be the wrong call. + */ +export function stopAllServeTurns(message: string): number { + const turns = [...bySessionKey.values()]; + for (const turn of turns) retire(turn, { type: 'error', message }); + return turns.length; +} diff --git a/src/servers/sidecar/protocol.ts b/src/servers/sidecar/protocol.ts index dea40a5b..95bd97c7 100644 --- a/src/servers/sidecar/protocol.ts +++ b/src/servers/sidecar/protocol.ts @@ -26,8 +26,8 @@ export type LiveClaudeSession = { }; /** - * An OpenCode turn in flight. Only ever the generating ones — see `listRunningOpenCodeTurns` for why - * this carries neither `isGenerating` (it is always true) nor `pendingTasks` (no such concept). + * An OpenCode turn in flight. Only ever the generating ones, which is why it carries neither + * `isGenerating` (always true) nor `pendingTasks` (no such concept on this harness). * * It deliberately carries no id for OpenCode's own `ses_…` session, and does not need to: the sidecar * only ever knows its own `sessionKey`, while the `ses_…` is reported separately over `opencode:session` @@ -63,12 +63,13 @@ export type SidecarCommand = // records are in memory and die with `pm2 restart officer`, while the agent keeps running. Without it // a live session is invisible until a browser happens to reconnect to it by id. | { type: 'claude:list'; id: string } - // OpenCode — drive a turn via `opencode run … --format json` (tools re-anchored to cwd via --dir) + // OpenCode — drive a turn through the serve (POST /api/session/{id}/prompt), anchored to the chat cwd + // by a per-request location header. Was an `opencode run` subprocess until 2026-08-10. | { type: 'opencode:run-streaming'; id: string; params: OpenCodeRunParams } | { type: 'opencode:kill'; id: string; sessionKey: string } // Which OpenCode turns are running right now. The counterpart of `claude:list`, and thinner for a // reason: OpenCode has no warm session between turns, so there is nothing to report but the running - // ones. See `listRunningOpenCodeTurns`. + // ones — see `listRunningServeTurns`. | { type: 'opencode:list'; id: string } // VNC | { type: 'vnc:start'; id: string; params: VncStartParams } @@ -197,6 +198,14 @@ export type OpenCodeRunParams = { cwd?: string; // passed to `opencode run --dir` — hard-re-anchors tools to this directory model?: string; // `providerID/modelID` (e.g. opencode/claude-haiku-4-5); passed to --model verbatim resumeSessionId?: string; // OpenCode `ses_…` id to continue (`--session`) + /** + * Images for this turn, base64 as the browser sent them. + * + * `opencode run` takes attachments as PATHS (`-f`), not inline data, so the sidecar writes each one to + * a temp file and deletes it when the turn ends. They travel as data because that is what crosses the + * websocket from the browser, and because officer and the sidecar are not guaranteed to share a disk. + */ + images?: PromptImage[]; durable?: boolean; // commit turn output to chat_session_events (default true) — see ClaudeSpawnStreamingParams }; diff --git a/src/workspaces/hooks/src/useChatWebSocket.ts b/src/workspaces/hooks/src/useChatWebSocket.ts index 776da795..5ec7bb2e 100644 --- a/src/workspaces/hooks/src/useChatWebSocket.ts +++ b/src/workspaces/hooks/src/useChatWebSocket.ts @@ -15,14 +15,39 @@ export const useChatWebSocket = ({ url, onMessage, onOpen }: UseChatWebSocketPar const retryRef = useRef(0); const retryTimeoutRef = useRef(null); const isCleaningUpRef = useRef(false); + /** A deferred teardown, cancelled when the effect re-runs — see the cleanup below. */ + const closeTimerRef = useRef(null); const onMessageRef = useRef(onMessage); onMessageRef.current = onMessage; const onOpenRef = useRef(onOpen); onOpenRef.current = onOpen; + /** + * Messages typed before the socket was ready. + * + * `send` used to drop them: `readyState !== OPEN` returned, silently, with no error and no retry — so + * pressing enter did nothing and the turn never happened. That window is not rare. React's dev + * StrictMode double-invokes effects, so every socket is created, closed and recreated on mount, and a + * reconnect after a drop reopens it again; with several chat panes on screen there are several sockets + * doing this at once. One of them is always briefly not OPEN. + * + * Queued and flushed on open, in order. The mobile chat app does exactly this and for exactly this + * reason — the composer is allowed to fire before the transport is ready. + */ + const pendingRef = useRef([]); + const connect = () => { if (isCleaningUpRef.current) return; - if (socketRef.current && socketRef.current.readyState === WebSocket.OPEN) return; + // CONNECTING counts as ours, not just OPEN. The deferred teardown below keeps a remount's socket + // alive mid-handshake, and this is what reclaims it: checking only OPEN meant an effect re-run + // built a SECOND socket and orphaned the first, which then stayed open forever with its own `open` + // handler bailing on the socketRef mismatch. + // + // This is per-instance and does NOT explain the two sockets a /chat/new load opens — measured with + // a WebSocket-constructor counter, those come from two separate `useChat` instances mounting, each + // with its own refs. Unresolved, and tracked separately; both connect, so it reads as healthy. + const existing = socketRef.current; + if (existing && (existing.readyState === WebSocket.OPEN || existing.readyState === WebSocket.CONNECTING)) return; const socket = new WebSocket(url); socketRef.current = socket; @@ -31,7 +56,12 @@ export const useChatWebSocket = ({ url, onMessage, onOpen }: UseChatWebSocketPar if (socketRef.current !== socket) return; setIsConnected(true); retryRef.current = 0; + // BEFORE onOpen, deliberately: onOpen sends the resume/attach handshake, and anything the user + // typed while connecting belongs after that, not in front of it. + const queued = pendingRef.current; + pendingRef.current = []; onOpenRef.current?.(); + for (const message of queued) socket.send(message); }); socket.addEventListener('message', (ev) => { @@ -59,25 +89,56 @@ export const useChatWebSocket = ({ url, onMessage, onOpen }: UseChatWebSocketPar }; useEffect(() => { + // A pending teardown from a remount that is about to be undone — see below. + if (closeTimerRef.current !== null) { + clearTimeout(closeTimerRef.current); + closeTimerRef.current = null; + } isCleaningUpRef.current = false; connect(); + return () => { + /** + * Close LATER, not now. + * + * Closing here directly is correct for a real unmount and disastrous for a remount, and this hook + * cannot tell them apart at the moment it runs. React's dev StrictMode double-invokes every effect + * (mount → unmount → mount), and a subtree that is re-created — a resolved transcript, a parent key + * change — does the same. Each time, the socket was closed while still CONNECTING, the browser + * logged "closed before the connection is established", and the replacement was closed in turn, so + * the view could churn forever and never hold a connection while sitting on Disconnected. + * + * Deferring by a tick makes the two distinguishable. A remount re-runs the effect immediately and + * cancels this timer, so the live socket is kept and the handshake completes. A real unmount has + * nobody to cancel it and the socket closes a frame later, which costs nothing. + */ isCleaningUpRef.current = true; if (retryTimeoutRef.current !== null) { clearTimeout(retryTimeoutRef.current); retryTimeoutRef.current = null; } - if (socketRef.current) { - socketRef.current.close(); - socketRef.current = null; - } + const socket = socketRef.current; + closeTimerRef.current = window.setTimeout(() => { + closeTimerRef.current = null; + if (!isCleaningUpRef.current) return; // remounted: the effect above already reclaimed it + if (socket) socket.close(); + if (socketRef.current === socket) socketRef.current = null; + }, 0); }; }, [url]); const send = (data: Record) => { const socket = socketRef.current; - if (!socket || socket.readyState !== WebSocket.OPEN) return; - socket.send(JSON.stringify(data)); + const message = JSON.stringify(data); + if (socket && socket.readyState === WebSocket.OPEN) { + socket.send(message); + return; + } + // Not open yet, or reconnecting. Hold it rather than dropping it — see `pendingRef`. Bounded so a + // socket that never comes back cannot grow this without limit; the oldest go first, because the + // newest message is the one the user is still waiting on. + pendingRef.current.push(message); + if (pendingRef.current.length > 50) pendingRef.current.shift(); }; return { isConnected, send }; diff --git a/src/workspaces/officerdev/src/apps/ChatHistory/SessionList.tsx b/src/workspaces/officerdev/src/apps/ChatHistory/SessionList.tsx index 4c836465..8cf84826 100644 --- a/src/workspaces/officerdev/src/apps/ChatHistory/SessionList.tsx +++ b/src/workspaces/officerdev/src/apps/ChatHistory/SessionList.tsx @@ -10,6 +10,24 @@ import type { SelectedSession } from './ChatDetailPanel'; import { chatListPath, chatNewPath, chatSessionPath, cwdFromSplat } from './chat-routes'; import { PwdSelector } from './PwdSelector'; +/** + * Which agent owns a conversation, said on EVERY row. + * + * Only OpenCode rows carried a badge before, which marks one harness and leaves the other inferred from + * its absence — fine when you already know the list mixes two, useless when you don't. The list is + * merged from two stores (Claude transcripts, OpenCode's session db) and the harness decides where a + * reply is dispatched, so it is a property of the row, not a footnote on the odd one out. + * + * `harness` is absent on older Claude rows, so anything that is not OpenCode reads as Claude — the same + * default the server applies. + */ +const HarnessBadge = ({ harness }: { harness?: 'claude' | 'opencode' }) => + harness === 'opencode' ? ( + OpenCode + ) : ( + Claude + ); + // Reads the /chat conversation list from Claude's own transcript store (source of truth). // Clicking a session loads its transcript and continues the real Claude session via --resume. export const SessionList = () => { @@ -194,13 +212,14 @@ export const SessionList = () => { className="min-w-0 flex-1" meta={[ , - session.harness === 'opencode' ? ( - - OpenCode - - ) : ( - `${session.messageCount} msg${session.messageCount === 1 ? '' : 's'}` - ), + , + // Claude's transcripts give a real turn count. OpenCode's session list has no + // count field, so the number on those rows is a hardcoded 0 and costs an HTTP + // call per row to make real — the badge stands in its place rather than beside a + // zero that means "not asked". + session.harness === 'opencode' + ? null + : `${session.messageCount} msg${session.messageCount === 1 ? '' : 's'}`, // One row, several transcripts: this conversation was `/clear`ed and carried on, // and the server has spliced the parts back together. Worth saying out loud — // the message count above covers all of them, and opening the row shows all of