# Sidecar setup and bootstrapping

**Status: LIVE — this document is being written as the investigation runs.** Sections marked
`[unverified]` are read from code but not exercised; sections marked `[open]` are questions I have not
answered yet. Nothing here proposes a change yet.

Started 2026-08-10. Scope: how a sidecar comes into existence, how it finds officer, how officer finds
it, and what a new one has to do. Everything below was read from the tree or measured on the running
machine, not recalled.

---

## The shape, in one pass

A sidecar is a **PM2 peer of `officer`** — never a child. It dials _in_; officer never spawns it.

```
PM2 starts it  →  it binds its own ephemeral port (if it serves HTTP)
               →  it opens a WS to officer at /api/sidecar/register
               →  it sends { type:'register', name, capabilities[] }
               →  officer replies { type:'registered', id }
               →  it sends { type:'<name>:server', port }   (HTTP sidecars only)
               →  officer remembers the port and proxies <prefix>/* to it
```

Officer's side of that is `src/servers/sidecar-registry.ts`; the sidecar's side is
`src/servers/sidecar/connect.ts`.

**Nothing in this path is officer starting a process.** `waitForCapability` in the registry says so
explicitly — it replaced ~77 lines of spawn-and-poll (`ensureClaudeSidecar`,
`spawnAndWaitForRegistration`, and per-email process maps). The only startup problem left is _ordering_,
handled by waiting up to 15s for a capability to appear rather than failing the first request after boot.

---

## The two kinds

Counted across 18 sidecar directories:

| Kind                                                                                  | How it is reached    | Count               |
| ------------------------------------------------------------------------------------- | -------------------- | ------------------- |
| **HTTP-proxied** — binds a port, officer forwards `<prefix>/*`                        | `createSidecarProxy` | 16                  |
| **Command-vocabulary** — no HTTP, answers typed commands over the registration socket | `sendCommand`        | 2 (`claude`, `vnc`) |

The 16 that report a port each declare a `'<name>:server'` event in `protocol.ts` (verified: exactly 16
such declarations). `claude` and `vnc` report no port — they are driven entirely by commands.

`pty` is the odd one out and is worth stating plainly: it is `index.mjs`, run by **node** rather than
bun, because `node-pty` is a native addon. It does **not** use `connect.ts` and carries its own copy of
the reconnect loop. The ecosystem file says so in a comment, which is the right place for it.

---

## What officer requires of a new sidecar

Four things, and three of them fail loudly if missed.

1. **A PM2 entry** in `ecosystem.config.cjs` (`script: 'bun'`, `args: 'run src/servers/sidecar/<n>/index.ts'`).
2. **A registration** with a `name` and `capabilities[]`. Officer indexes by capability, not by name —
   `findSidecarByCapability` is how every caller reaches one.
3. **A `'<name>:server'` event in `protocol.ts`**, if it serves HTTP. Without it the type does not exist
   and `createSidecarProxy`'s listener never matches.
4. **A capability-registry entry**, if it mounts a router. `assertCapabilityTotality` runs in
   `server.tsx` _before_ `serve()` and **throws**, so a missing entry means the server refuses to boot,
   naming what is missing. Alternatively an `EXEMPT_API_PREFIXES` entry _with a stated reason_.

Item 4 is the one that is a deliberate wall rather than a convention, and the reasoning is recorded in
`totality.ts`: a Member could 403 on `GET /api/tasks` and open `/api/tasks/pipeline/ws` with a 101 in the
same minute, because Bun's route table matches the socket before the `/api/*` catch-all. Refusing to boot
survives the next door being added; a patch does not.

---

## Details worth knowing before changing any of this

**Ports are ephemeral and re-reported on every reconnect.** A sidecar binds `port: 0`, reads the port
back, then releases and rebinds (`getFreePort` — bind, read, `stop(true)`). Officer stores whatever was
last reported. Observed live: the photos sidecar moved 33891 → 34349 → 36337 → 41829 → 33637 across
restarts tonight, and officer followed each time. **A stale port is a 502, not a hang** — `getHttpUrl()`
returns null before first registration and the proxy answers 503.

`[unverified]` The bind-read-release in `getFreePort` is a classic TOCTOU: another process can take the
port between release and rebind. Never seen it happen here; noting it because it is a real race, not
because it has bitten.

**Re-registration replaces, it does not duplicate.** `registerSidecar` unregisters any existing sidecar
with the same _name_ first, so a sidecar that reconnects without officer noticing the old socket die
does not leave a ghost.

**Reconnect is the sidecar's job, with backoff** — `[200, 500, 1000, 2000, 4000, 8000, 15000]`ms in
`connect.ts`. Officer does nothing to bring a sidecar back; PM2 restarts the process, the process
re-dials.

**`API_URL` is derived, not configured.** Every sidecar computes
`process.env.API_URL ?? ws://127.0.0.1:${process.env.PORT ?? '5000'}`. Note the fallback port is **5000**
while this machine runs officer on **9010** — so the default is wrong here and it works only because
`.env` supplies `PORT`. `[open]` Is that fallback ever exercised, and should it be a hard failure
instead of a wrong guess?

**One directory, two processes — and this is the easiest thing here to get wrong.** `sidecar/claude/`
contains two entrypoints that register as _different sidecars_:

| File                      | PM2 entry                 | Registers as                         | What it is                                           |
| ------------------------- | ------------------------- | ------------------------------------ | ---------------------------------------------------- |
| `claude/index.ts`         | `officer-anthropic-proxy` | name `proxy`, capability `['proxy']` | Holds the Anthropic credential, forwards API traffic |
| `claude/user-instance.ts` | `officer-agent`           | capability `['claude']`              | The process that actually spawns `claude`            |

So **capability `proxy` is the Anthropic proxy, and capability `claude` is the agent.** Nothing named
"claude" registers the `claude` capability from `claude/index.ts`, which is exactly the sort of thing
that reads as a bug in a grep and is not one.

That resolves the special-casing: `isConnected()` returns "a sidecar with capability `proxy` exists" —
i.e. **the Anthropic proxy is up**, which is _not_ the same as "the agent is up", though the name reads
that way. `[verified]` It currently has **no callers** outside the registry itself, so nothing is
misreading it today. Worth either renaming or deleting before something starts trusting the name.

`registerSidecar` also fires a notification when a registration includes capability `claude`
(`sidecar-registry.ts:75`) — "a new agent process has come up". That one is correctly aimed at the agent.

---

## Duplication, measured

Each HTTP sidecar's `index.ts` independently contains: the `API_URL` line, `getFreePort`, a `Bun.serve`,
an `X-Officer-User` check, a `createSidecarConnector` call, and an `onConnected` that reports the port.
Sizes range 73–505 lines (`email` smallest at 73, `music` largest at 505).

`createSidecarProxy` factored out officer's side of this — the comment records that eight copies of the
port capture were byte-identical once the app name was normalised away. **The sidecar side has had no
equivalent factoring.** `[open]` Is a `createSidecarServer` worth it, or is the duplication load-bearing
because each sidecar's routes differ enough that a shared shell would grow options faster than it saved
lines?

---

## Open questions, in the order I would answer them

1. ~~What provides the `proxy` capability~~ — **answered above**: the Anthropic proxy, not the agent.
   `isConnected()` has no callers; rename or delete it before its name misleads someone.
2. **Is the sidecar-side boilerplate worth factoring**, given `create-proxy.ts` already proved the
   officer side was?
3. **What happens on a partial boot** — officer up, a sidecar permanently down. `waitForCapability`
   throws after 15s; who catches it, and what does the user see?
4. **Is the `PORT ?? '5000'` fallback reachable**, and should it fail loudly instead?
5. **`sweepStaleServes` is `/proc`-based and a no-op on macOS** (already noted in the OpenCode parity
   doc as B8). Does any other sidecar have a Linux-only assumption?

---

## Verified facts this document rests on

| Claim                                                          | How                                                                                               |
| -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| 20 PM2 entries, 18 sidecar dirs                                | `ecosystem.config.cjs`, `ls src/servers/sidecar/`                                                 |
| 16 sidecars report a port, 2 do not                            | `grep` for `':server'` in each `index.ts`, cross-checked against 16 declarations in `protocol.ts` |
| `pty` is node + `.mjs` + its own reconnect loop                | `ecosystem.config.cjs` comment and `ls sidecar/pty/`                                              |
| Officer spawns nothing                                         | `waitForCapability` comment; no spawn call in the registry                                        |
| Ports change across restarts and officer follows               | observed live tonight across five photos restarts                                                 |
| Boot fails on a missing capability entry                       | `assertCapabilityTotality` throws before `serve()`                                                |
| `sidecar/claude/` is two processes with different capabilities | `ecosystem.config.cjs` args + the two `createSidecarConnector` calls                              |
| `isConnected()` has no callers outside the registry            | grep across `src/servers`                                                                         |
