platform/docs/sidecar-bootstrapping.md

Sidecar setup and bootstrapping

Status: LIVE — this document is being written as the investigation runs. Sections marked [unverified] are read from code but not exercised; sections marked [open] are questions I have not answered yet. Nothing here proposes a change yet.

Started 2026-08-10. Scope: how a sidecar comes into existence, how it finds officer, how officer finds it, and what a new one has to do. Everything below was read from the tree or measured on the running machine, not recalled.

---

The shape, in one pass

A sidecar is a PM2 peer of officer — never a child. It dials in; officer never spawns it.

PM2 starts it  →  it binds its own ephemeral port (if it serves HTTP)
               →  it opens a WS to officer at /api/sidecar/register
               →  it sends { type:'register', name, capabilities[] }
               →  officer replies { type:'registered', id }
               →  it sends { type:'<name>:server', port }   (HTTP sidecars only)
               →  officer remembers the port and proxies <prefix>/* to it

Officer's side of that is src/servers/sidecar-registry.ts; the sidecar's side is src/servers/sidecar/connect.ts.

Nothing in this path is officer starting a process. waitForCapability in the registry says so explicitly — it replaced ~77 lines of spawn-and-poll (ensureClaudeSidecar, spawnAndWaitForRegistration, and per-email process maps). The only startup problem left is ordering, handled by waiting up to 15s for a capability to appear rather than failing the first request after boot.

---

The two kinds

Counted across 18 sidecar directories:

Kind	How it is reached	Count
HTTP-proxied — binds a port, officer forwards <prefix>/*	createSidecarProxy	16
Command-vocabulary — no HTTP, answers typed commands over the registration socket	sendCommand	2 (claude, vnc)

The 16 that report a port each declare a '<name>:server' event in protocol.ts (verified: exactly 16 such declarations). claude and vnc report no port — they are driven entirely by commands.

pty is the odd one out and is worth stating plainly: it is index.mjs, run by node rather than bun, because node-pty is a native addon. It does not use connect.ts and carries its own copy of the reconnect loop. The ecosystem file says so in a comment, which is the right place for it.

---

What officer requires of a new sidecar

Four things, and three of them fail loudly if missed.

1. A PM2 entry in ecosystem.config.cjs (script: 'bun', args: 'run src/servers/sidecar/<n>/index.ts').
2. A registration with a name and capabilities[]. Officer indexes by capability, not by name — findSidecarByCapability is how every caller reaches one.
3. A '<name>:server' event in protocol.ts, if it serves HTTP. Without it the type does not exist and createSidecarProxy's listener never matches.
4. A capability-registry entry, if it mounts a router. assertCapabilityTotality runs in server.tsx before serve() and throws, so a missing entry means the server refuses to boot, naming what is missing. Alternatively an EXEMPT_API_PREFIXES entry with a stated reason.

Item 4 is the one that is a deliberate wall rather than a convention, and the reasoning is recorded in totality.ts: a Member could 403 on GET /api/tasks and open /api/tasks/pipeline/ws with a 101 in the same minute, because Bun's route table matches the socket before the /api/* catch-all. Refusing to boot survives the next door being added; a patch does not.

---

Details worth knowing before changing any of this

Ports are ephemeral and re-reported on every reconnect. A sidecar binds port: 0, reads the port back, then releases and rebinds (getFreePort — bind, read, stop(true)). Officer stores whatever was last reported. Observed live: the photos sidecar moved 33891 → 34349 → 36337 → 41829 → 33637 across restarts tonight, and officer followed each time. A stale port is a 502, not a hang — getHttpUrl() returns null before first registration and the proxy answers 503.

[unverified] The bind-read-release in getFreePort is a classic TOCTOU: another process can take the port between release and rebind. Never seen it happen here; noting it because it is a real race, not because it has bitten.

Re-registration replaces, it does not duplicate. registerSidecar unregisters any existing sidecar with the same name first, so a sidecar that reconnects without officer noticing the old socket die does not leave a ghost.

Reconnect is the sidecar's job, with backoff — [200, 500, 1000, 2000, 4000, 8000, 15000]ms in connect.ts. Officer does nothing to bring a sidecar back; PM2 restarts the process, the process re-dials.

API_URL is derived, not configured. Every sidecar computes process.env.API_URL ?? ws://127.0.0.1:${process.env.PORT ?? '5000'}. Note the fallback port is 5000 while this machine runs officer on 9010 — so the default is wrong here and it works only because .env supplies PORT. [open] Is that fallback ever exercised, and should it be a hard failure instead of a wrong guess?

One directory, two processes — and this is the easiest thing here to get wrong. sidecar/claude/ contains two entrypoints that register as different sidecars:

File	PM2 entry	Registers as	What it is
claude/index.ts	officer-anthropic-proxy	name proxy, capability ['proxy']	Holds the Anthropic credential, forwards API traffic
claude/user-instance.ts	officer-agent	capability ['claude']	The process that actually spawns claude

So capability proxy is the Anthropic proxy, and capability claude is the agent. Nothing named "claude" registers the claude capability from claude/index.ts, which is exactly the sort of thing that reads as a bug in a grep and is not one.

That resolves the special-casing: isConnected() returns "a sidecar with capability proxy exists" — i.e. the Anthropic proxy is up, which is not the same as "the agent is up", though the name reads that way. [verified] It currently has no callers outside the registry itself, so nothing is misreading it today. Worth either renaming or deleting before something starts trusting the name.

registerSidecar also fires a notification when a registration includes capability claude (sidecar-registry.ts:75) — "a new agent process has come up". That one is correctly aimed at the agent.

---

Duplication, measured

Each HTTP sidecar's index.ts independently contains: the API_URL line, getFreePort, a Bun.serve, an X-Officer-User check, a createSidecarConnector call, and an onConnected that reports the port. Sizes range 73–505 lines (email smallest at 73, music largest at 505).

createSidecarProxy factored out officer's side of this — the comment records that eight copies of the port capture were byte-identical once the app name was normalised away. The sidecar side has had no equivalent factoring. [open] Is a createSidecarServer worth it, or is the duplication load-bearing because each sidecar's routes differ enough that a shared shell would grow options faster than it saved lines?

---

Open questions, in the order I would answer them

1. What provides the proxy capability — answered above: the Anthropic proxy, not the agent. isConnected() has no callers; rename or delete it before its name misleads someone.
2. Is the sidecar-side boilerplate worth factoring, given create-proxy.ts already proved the officer side was?
3. What happens on a partial boot — officer up, a sidecar permanently down. waitForCapability throws after 15s; who catches it, and what does the user see?
4. Is the PORT ?? '5000' fallback reachable, and should it fail loudly instead?
5. sweepStaleServes is /proc-based and a no-op on macOS (already noted in the OpenCode parity doc as B8). Does any other sidecar have a Linux-only assumption?

---

Verified facts this document rests on

Claim	How
20 PM2 entries, 18 sidecar dirs	ecosystem.config.cjs, ls src/servers/sidecar/
16 sidecars report a port, 2 do not	grep for ':server' in each index.ts, cross-checked against 16 declarations in protocol.ts
pty is node + .mjs + its own reconnect loop	ecosystem.config.cjs comment and ls sidecar/pty/
Officer spawns nothing	waitForCapability comment; no spawn call in the registry
Ports change across restarts and officer follows	observed live tonight across five photos restarts
Boot fails on a missing capability entry	assertCapabilityTotality throws before serve()
sidecar/claude/ is two processes with different capabilities	ecosystem.config.cjs args + the two createSidecarConnector calls
isConnected() has no callers outside the registry	grep across src/servers