drop inherited architecture docs that no longer match the code
AUTOMATION_CONTEXT.md and SIDECAR.md described the multi-tenant scope model, the seed/ tree, the marketplace, and a single sidecar owning the queue — all superseded. Fix the stale doc pointers in opencode.json and CLAUDE.md, and point at TODO.md as the source of truth on direction. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.8
@@ -1,366 +0,0 @@
|
||||
# Process Sidecar
|
||||
|
||||
Independent Bun process that owns all long-running work so the API server can restart without disrupting active sessions.
|
||||
|
||||
## Problem
|
||||
|
||||
The API server (port 5000) previously owned all spawned processes: Pi agents, Claude Code sessions, the Anthropic proxy, and the job queue. Restarting the API server would:
|
||||
|
||||
- Kill active Pi and Claude Code conversations mid-response
|
||||
- Regenerate the Anthropic proxy secret, breaking any Claude Code sessions using it
|
||||
- Lose in-flight job progress (queue engine ran in-process)
|
||||
|
||||
## Solution
|
||||
|
||||
A separate Bun process ("process sidecar") on port 5100 that owns all spawned processes. The API server communicates with it over a single WebSocket connection. The sidecar is managed by pm2 and starts before the API server.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────┐ ┌──────────────────────────────┐
|
||||
│ API Server (:5000) │ WS │ Process Sidecar (:5100) │
|
||||
│ │◄───────►│ │
|
||||
│ - Browser WS clients │ │ - Anthropic Proxy (:5051) │
|
||||
│ - REST API routes │ │ - Claude Code processes │
|
||||
│ - Channel bots │ │ - Pi agent processes │
|
||||
│ - sidecar-client.ts │ │ - Job queue engine │
|
||||
│ (auto-reconnect) │ │ - State persistence │
|
||||
└─────────────────────────┘ └──────────────────────────────┘
|
||||
```
|
||||
|
||||
### What the sidecar owns
|
||||
|
||||
| Concern | Previous location | Sidecar module |
|
||||
|---------|-------------------|----------------|
|
||||
| Anthropic proxy (port 5051) | `api/anthropic-proxy.ts` | `sidecar/proxy.ts` |
|
||||
| Claude Code spawn + session tracking | `channels/send-claude-code.ts` | `sidecar/claude-manager.ts` |
|
||||
| Pi agent spawn + RPC commands | `api/pi/pi-bridge.ts` | `sidecar/pi-manager.ts` |
|
||||
| Job queue engine + handlers | `queue/engine.ts` | `sidecar/queue-runner.ts` |
|
||||
|
||||
### What stays in the API server
|
||||
|
||||
- Browser WebSocket connections (ephemeral by nature)
|
||||
- REST API routes (now thin proxies to sidecar)
|
||||
- Channel bots (Discord/Telegram/WhatsApp — already reconnect gracefully)
|
||||
- Browser relay (CDP state is ephemeral)
|
||||
- PTY sidecar (already its own process, unchanged)
|
||||
|
||||
## Communication Protocol
|
||||
|
||||
Single WebSocket between API server and sidecar. JSON messages with `{ type, id?, ... }` envelopes.
|
||||
|
||||
**Request/response**: Commands include an `id` field. The sidecar responds with a message carrying the same `id`. The client correlates responses via this ID with configurable timeouts.
|
||||
|
||||
**Streaming events**: Pi and Claude Code output events are broadcast to all connected clients without a correlation ID. They carry a `sessionId` or `sessionKey` so the API server can route them to the correct browser WS.
|
||||
|
||||
### Command categories
|
||||
|
||||
```
|
||||
ping / pong — health check
|
||||
state:sync — full state snapshot on connect
|
||||
|
||||
proxy:secret — get persisted proxy secret
|
||||
|
||||
claude:spawn / claude:result — blocking Claude Code exec
|
||||
claude:spawn-streaming / claude:event — streaming Claude Code exec
|
||||
claude:kill / claude:clear-session — session management
|
||||
|
||||
pi:spawn / pi:prompt / pi:abort — Pi agent lifecycle
|
||||
pi:kill / pi:set-thinking — Pi session control
|
||||
|
||||
queue:enqueue / queue:cancel — job management
|
||||
queue:list / queue:get — job queries
|
||||
```
|
||||
|
||||
See `protocol.ts` for the full type definitions.
|
||||
|
||||
## State Persistence
|
||||
|
||||
File: `data/sidecar/state.json`
|
||||
|
||||
Written every 30 seconds (debounced) and on graceful shutdown (SIGTERM/SIGINT). Contains:
|
||||
|
||||
- **proxySecret** — generated once on first boot, reused forever. This is the key fix: the Anthropic proxy secret no longer changes on restart.
|
||||
- **claudeSessions** — map of `sessionKey → Claude Code session_id` for `--resume` support across restarts.
|
||||
- **piSessions** — session metadata with PIDs for liveness checking on restart.
|
||||
|
||||
### Lockfile
|
||||
|
||||
`data/sidecar/sidecar.lock` — contains the PID of the running sidecar. On startup, if the lock exists and the PID is alive, the sidecar exits. Stale locks (dead PID) are cleaned up automatically.
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
src/servers/sidecar/
|
||||
index.ts — entry point: Bun.serve on :5100, WS dispatch, shutdown
|
||||
protocol.ts — shared message types (imported by both sides)
|
||||
proxy.ts — Anthropic proxy server (moved from anthropic-proxy.ts)
|
||||
claude-manager.ts — Claude Code blocking + streaming spawn, session map
|
||||
pi-manager.ts — Pi agent spawn, RPC (prompt/abort/thinking), event parsing
|
||||
queue-runner.ts — Job queue engine (lanes, retries, notifications)
|
||||
state.ts — File-backed state persistence + lockfile
|
||||
|
||||
src/servers/sidecar-client.ts — API server's WebSocket client (singleton)
|
||||
```
|
||||
|
||||
## API Server Integration
|
||||
|
||||
The API server connects to the sidecar on startup via `initSidecarClient()` in `server.tsx`. The client auto-reconnects with exponential backoff (200ms → 15s).
|
||||
|
||||
On connect, it sends `state:sync` to get the current proxy secret and live session info.
|
||||
|
||||
### Modified API server files
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `server.tsx` | `startAnthropicProxy()` → `initSidecarClient()` |
|
||||
| `bootstrap.ts` | Removed `initQueue()` (sidecar owns it) |
|
||||
| `channels/send-claude-code.ts` | 550 lines → 73 lines thin client |
|
||||
| `api/pi/websocket.ts` | Spawn/prompt/abort go through sidecar |
|
||||
| `api/pi/session-manager.ts` | Cleanup sidecar subscriptions on delete |
|
||||
| `api/queue/queue.ts` | Routes use sidecar client |
|
||||
| `channels/discord/handler.ts` | `enqueue` → `enqueueJob` via sidecar |
|
||||
| `channels/telegram/handler.ts` | Same |
|
||||
| `channels/whatsapp/handler.ts` | Same |
|
||||
|
||||
## PM2 Configuration
|
||||
|
||||
```js
|
||||
// ecosystem.config.cjs
|
||||
module.exports = {
|
||||
apps: [
|
||||
{
|
||||
name: 'officer-sidecar', // starts first
|
||||
script: 'bun',
|
||||
args: 'run src/servers/sidecar/index.ts',
|
||||
watch: false,
|
||||
},
|
||||
{
|
||||
name: 'officer',
|
||||
script: 'bun',
|
||||
args: 'start',
|
||||
watch: false,
|
||||
},
|
||||
],
|
||||
};
|
||||
```
|
||||
|
||||
The sidecar is listed first so pm2 starts it before the API server. The API server's sidecar client handles the case where the sidecar isn't ready yet (auto-reconnect with backoff).
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `SIDECAR_PORT` | `5100` | Sidecar HTTP/WS port |
|
||||
| `ANTHROPIC_PROXY_PORT` | `5051` | Anthropic proxy port (owned by sidecar) |
|
||||
| `DATA_PATH` | `./data` | Shared data directory |
|
||||
|
||||
## Manual Testing
|
||||
|
||||
### 1. Start the sidecar standalone
|
||||
|
||||
```bash
|
||||
# From monorepo root
|
||||
bun run src/servers/sidecar/index.ts
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
[sidecar:proxy] listening on 127.0.0.1:5051
|
||||
[sidecar:queue] initialized
|
||||
[sidecar] listening on 127.0.0.1:5100
|
||||
```
|
||||
|
||||
### 2. Health check
|
||||
|
||||
```bash
|
||||
# Basic liveness
|
||||
curl http://127.0.0.1:5100/
|
||||
# → "process-sidecar"
|
||||
|
||||
# Detailed health
|
||||
curl http://127.0.0.1:5100/health
|
||||
# → {"status":"ok","uptime":12345,"piSessions":0,"claudeSessions":0}
|
||||
```
|
||||
|
||||
### 3. Verify Anthropic proxy is running
|
||||
|
||||
```bash
|
||||
# Should reject without valid secret
|
||||
curl -s http://127.0.0.1:5051/v1/messages
|
||||
# → {"error":"Unauthorized"}
|
||||
|
||||
# Check state file was created
|
||||
cat data/sidecar/state.json
|
||||
# → should show proxySecret, empty claudeSessions, empty piSessions
|
||||
```
|
||||
|
||||
### 4. Verify proxy secret persistence
|
||||
|
||||
```bash
|
||||
# Note the proxySecret from state.json
|
||||
cat data/sidecar/state.json | jq .proxySecret
|
||||
|
||||
# Stop and restart the sidecar
|
||||
# Kill the sidecar (Ctrl+C or kill)
|
||||
bun run src/servers/sidecar/index.ts
|
||||
|
||||
# Check the secret is the same
|
||||
cat data/sidecar/state.json | jq .proxySecret
|
||||
# → should be identical to before
|
||||
```
|
||||
|
||||
### 5. WebSocket communication test
|
||||
|
||||
```bash
|
||||
# In one terminal, start the sidecar
|
||||
bun run src/servers/sidecar/index.ts
|
||||
|
||||
# In another terminal, connect with websocat (or wscat)
|
||||
# Install: cargo install websocat OR npm install -g wscat
|
||||
websocat ws://127.0.0.1:5100
|
||||
|
||||
# Send a ping
|
||||
{"type":"ping","id":"test1"}
|
||||
# → should receive: {"type":"pong","id":"test1"}
|
||||
|
||||
# Send state sync
|
||||
{"type":"state:sync","id":"test2"}
|
||||
# → should receive: {"type":"state:sync","id":"test2","state":{...}}
|
||||
|
||||
# Get proxy secret
|
||||
{"type":"proxy:secret","id":"test3"}
|
||||
# → should receive: {"type":"proxy:secret","id":"test3","secret":"sk-ant-api03-..."}
|
||||
```
|
||||
|
||||
### 6. Test lockfile protection
|
||||
|
||||
```bash
|
||||
# Start sidecar in one terminal
|
||||
bun run src/servers/sidecar/index.ts
|
||||
|
||||
# Try starting another in a second terminal
|
||||
bun run src/servers/sidecar/index.ts
|
||||
# → should exit with: "[sidecar] another instance is already running"
|
||||
|
||||
# Check lockfile
|
||||
cat data/sidecar/sidecar.lock
|
||||
# → PID of the running sidecar
|
||||
```
|
||||
|
||||
### 7. Full integration test (sidecar + API server)
|
||||
|
||||
```bash
|
||||
# Start sidecar first
|
||||
bun run src/servers/sidecar/index.ts &
|
||||
|
||||
# Start API server
|
||||
bun start
|
||||
# → should see "[sidecar-client] connected" in logs
|
||||
|
||||
# Test via the dashboard:
|
||||
# 1. Open a chat panel, send a message with claude-code model
|
||||
# → should see streaming response (routed through sidecar)
|
||||
# 2. Open a chat with a Pi model
|
||||
# → should see streaming response (routed through sidecar)
|
||||
# 3. Restart the API server (kill + bun start)
|
||||
# → active Claude Code processes should NOT die
|
||||
# → sidecar should show "client disconnected" then "client connected"
|
||||
# → proxy secret should remain the same
|
||||
```
|
||||
|
||||
### 8. Test API server restart resilience
|
||||
|
||||
This is the key scenario that motivated the sidecar:
|
||||
|
||||
```bash
|
||||
# 1. Start sidecar + API server
|
||||
bun run src/servers/sidecar/index.ts &
|
||||
bun start &
|
||||
|
||||
# 2. Start a Claude Code streaming session in the dashboard
|
||||
|
||||
# 3. While Claude Code is running, kill the API server
|
||||
kill $(pgrep -f "bun start")
|
||||
|
||||
# 4. Restart the API server
|
||||
bun start
|
||||
|
||||
# 5. Check:
|
||||
# - The Claude Code process should still be running (check with ps)
|
||||
# - The proxy secret should be the same (check data/sidecar/state.json)
|
||||
# - The sidecar should show the reconnection in logs
|
||||
```
|
||||
|
||||
### 9. Test with pm2
|
||||
|
||||
```bash
|
||||
pm2 start ecosystem.config.cjs
|
||||
|
||||
# Check both are running
|
||||
pm2 status
|
||||
# → officer-sidecar: online
|
||||
# → officer: online
|
||||
|
||||
# Restart API server only
|
||||
pm2 restart officer
|
||||
|
||||
# Check sidecar is still running
|
||||
pm2 status
|
||||
curl http://127.0.0.1:5100/health
|
||||
|
||||
# Stop everything
|
||||
pm2 stop all
|
||||
```
|
||||
|
||||
### 10. Queue test
|
||||
|
||||
```bash
|
||||
# With sidecar running, queue a job via the API
|
||||
curl -X POST http://localhost:5000/api/queue/jobs \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer <your-token>" \
|
||||
-d '{"lane":"test","type":"gmail-sync"}'
|
||||
|
||||
# List jobs
|
||||
curl http://localhost:5000/api/queue/jobs \
|
||||
-H "Authorization: Bearer <your-token>"
|
||||
```
|
||||
|
||||
## Graceful Shutdown
|
||||
|
||||
On SIGTERM or SIGINT, the sidecar:
|
||||
1. Flushes pending state to `data/sidecar/state.json`
|
||||
2. Releases the lockfile
|
||||
3. Does **NOT** kill spawned processes — they are independent OS processes
|
||||
|
||||
On restart, the sidecar reads the persisted state and checks which PIDs are still alive.
|
||||
|
||||
## Risks & Mitigations
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Sidecar dies with live sessions | pm2 auto-restart + state.json + processes are independent OS processes |
|
||||
| API↔sidecar WebSocket drops | Auto-reconnect with exponential backoff (200ms → 15s) |
|
||||
| Two sidecar instances running | Lockfile with PID liveness check on startup |
|
||||
| Sidecar needs DB access | Imports DB modules directly (same user, same filesystem) |
|
||||
| Queue handlers need server context | Handlers are self-contained modules imported by the sidecar |
|
||||
|
||||
## Future Possibilities
|
||||
|
||||
The current architecture is one API server → one sidecar. But the protocol is sidecar-agnostic and could evolve in the opposite direction: one API server → many sidecars.
|
||||
|
||||
```
|
||||
┌── sidecar-A (machine 1, user X's files)
|
||||
API server ───┼── sidecar-B (machine 2, heavy compute)
|
||||
└── sidecar-C (localhost, default)
|
||||
```
|
||||
|
||||
In this model the API server becomes the orchestrator — it decides which sidecar to dispatch to based on the user, the task type, or available capacity. A user's Claude Code session runs on the machine where their project files live. A long-running Pi agent gets dispatched to a box with more resources. Quick jobs stay local.
|
||||
|
||||
What would need to change:
|
||||
|
||||
- **Sidecar identity**: Each sidecar needs a name/id. The current hardcoded `ws://127.0.0.1:5100` becomes a registry of endpoints (config file or DB table).
|
||||
- **Connection pool**: `sidecar-client.ts` becomes a pool of connections rather than a singleton.
|
||||
- **Routing**: The API server needs a session→sidecar mapping so it knows where to forward browser messages.
|
||||
- **The protocol itself**: Doesn't change. A `pi:spawn` command works the same whether it goes to a local or remote sidecar.
|
||||
Reference in New Issue
Block a user