Stage 2, and the end of the inversion. The two sync handlers (1,093 lines) ran in the
platform's queue, which meant the sidecar reached back over its registration socket to ask
the platform to enqueue work, and the credentials travelled through Postgres job metadata to
get there. Option (A) from the plan: they run here now, and the Jobs screen is left to the
things it actually describes.
The handlers moved almost unedited. Their bodies were already a list of steps taking a
context, so sync-runner.ts synthesizes that context and runs them; what went away is the
JobHandler wrapper and the registration. `job.userId` is the OWNER'S EMAIL rather than a
numeric id — the queue's naming — and it resolves the mail store path, so it is called out
in the type. That is the same field whose absence made the mailbox read as empty two commits
ago; it is set from user.email and checked this time.
Deliberately not a queue: one run per account, no persistence, no retry. A failure is picked
up by the ten-minute cron like any other, and a sync interrupted by a restart resumes from
the stored cursor rather than the beginning. PermanentError survives as a local class — it
signalled "do not retry" to the queue and now just carries its message to the sync state.
accounts.ts asks the runner whether an account is syncing instead of scanning job rows, and
the queue-over-WS shim in index.ts is gone: enqueueViaWs, listJobsViaWs, the pending-response
map and the queue branch in the command handler. Nothing but a port crosses that socket now.
The three chat channels stop opening the mail store directly. They each carried their own
copy of count-rows / enqueue / poll / count-again, coupling three chat bridges to the mail
schema — and they enqueued `gmail-sync` unconditionally, the OAuth path, for an
app-password account that syncs over IMAP, so the command was already broken. One shared
helper calls a new POST /sync-now on the sidecar, which syncs and reports what arrived.
queue/handlers/ is now empty; both handlers there were email. The queue is untouched and
still serves the Jobs screen.
Not moved, and fine where they are: scripts/migrate-emails-to-sqlite.ts and
scripts/seed-imap-uids.ts are one-off maintenance scripts that open the store directly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Email was the one sidecar built inside out. The platform held ~1,800 lines — the per-account
SQLite store, all 14 HTTP routes, account CRUD, resync, IMAP validation — while the 314-line
sidecar was a scheduler that reached BACK into the platform to do anything
(`import { performResync } from '../../api/email/resync'`).
The sidecar now serves its own HTTP listener and announces `email:server`, and
/api/email/* on the platform is createSidecarProxy like every other one: 1,801 lines down
to 22, with no mail knowledge left in it — not a message, not a folder, not a credential.
The routes moved verbatim, Hono and all. http.ts only reconstructs what the platform's
middleware used to provide: `user` on the context, from the X-Officer-User header the proxy
injects (trusted because this server binds loopback), and an error handler that turns
custom-errors into status codes.
The /email/events SSE stream went with them, which removes a whole round trip: the IDLE
watcher used to send `email:new` over the registration socket so the platform could push to
its SSE clients. Those clients are here now, so it calls broadcastEmailNew in-process and
`email:new` is gone from the wire protocol.
DELIBERATELY NOT DONE YET, and left backwards on purpose rather than half-moved:
- The two sync handlers (email-sync 381 lines, gmail-sync 712) still run in the platform's
queue and now import the store from its new home — a platform → sidecar import, which is
the wrong direction and is temporary. Moving them is option (A) from the plan: the sidecar
schedules its own syncs, independent of the platform Jobs list.
- accounts.ts still imports queue/init to enqueue a sync and to report sync status, and
index.ts still carries the queue-over-WS shim that inversion needs.
- The three channel handlers still open the mail store directly rather than asking over HTTP.
Two things worth knowing while testing: a from-scratch sync holds a proxied request open
well past the 60s idle default, hence timeoutSeconds on the proxy; and `gmail-sync` is
hardcoded in all three channel handlers even though the only account is provider=gmail with
auth_type=password, which routes to IMAP — so "sync emails" from a chat channel is
almost certainly already broken, and folds into the next stage.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reorganizes email storage: the DB moves from DATA_PATH/<user>/emails.db to
DATA_PATH/<user>/email_accounts/<accountEmail>/emails.db, with a shared
email_accounts/attachment_cache/ (was Gmail/emails/attachments). openEmailDb now
takes (owner, account); a new openUserEmailDb(owner, userId) resolves the user's
configured account (first enabled) for read paths. Threads the account through
email.ts, accounts, resync, queue sync, channel handlers, and the email_db MCP
tool path. Drops the dead getUserEmailDir helper.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Flips the connection model so sidecars register themselves with the API
server via WebSocket at /api/sidecar/register, enabling dynamic discovery,
location independence, and automatic reconnection from either side.
- Add registration protocol types and PTY command/event types
- Create sidecar-registry.ts (replaces sidecar-client.ts) as passive registry
- Create sidecar connector (connect.ts) with exponential backoff reconnect
- Convert process sidecar from WS server to WS client
- Convert PTY sidecar from WS server to multiplexed WS client
- Simplify terminal bridge to thin adapter using registry
- Add PTY sidecar as PM2-managed process
- Update all consumer imports
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces a separate Bun process (port 5100) that owns all spawned
processes and long-running work, so the API server can restart freely
without disrupting active sessions.
The sidecar owns:
- Anthropic proxy (port 5051) with persisted secret across restarts
- Claude Code process spawning and session tracking (--resume support)
- Pi agent spawning and RPC lifecycle (prompt/abort/thinking)
- Job queue engine (lane processing, retries, notifications)
The API server becomes a thin client that forwards commands over a
single WebSocket connection with auto-reconnect. send-claude-code.ts
goes from 550 lines of spawn logic to 73 lines of sidecar delegation.
State persisted to data/sidecar/state.json every 30s and on shutdown.
Lockfile prevents duplicate instances. See SIDECAR.md for full docs
and manual testing procedures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Channel session IDs now include userId (channel-{provider}-{userId}-{contextId})
to prevent cross-user contamination in multi-user setups. WhatsApp disconnect
properly logs out and clears cached auth. Browser relay uses server-derived token
directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
pastilhasandClaude Opus 5