email: the sidecar schedules its own syncs
Stage 2, and the end of the inversion. The two sync handlers (1,093 lines) ran in the platform's queue, which meant the sidecar reached back over its registration socket to ask the platform to enqueue work, and the credentials travelled through Postgres job metadata to get there. Option (A) from the plan: they run here now, and the Jobs screen is left to the things it actually describes. The handlers moved almost unedited. Their bodies were already a list of steps taking a context, so sync-runner.ts synthesizes that context and runs them; what went away is the JobHandler wrapper and the registration. `job.userId` is the OWNER'S EMAIL rather than a numeric id — the queue's naming — and it resolves the mail store path, so it is called out in the type. That is the same field whose absence made the mailbox read as empty two commits ago; it is set from user.email and checked this time. Deliberately not a queue: one run per account, no persistence, no retry. A failure is picked up by the ten-minute cron like any other, and a sync interrupted by a restart resumes from the stored cursor rather than the beginning. PermanentError survives as a local class — it signalled "do not retry" to the queue and now just carries its message to the sync state. accounts.ts asks the runner whether an account is syncing instead of scanning job rows, and the queue-over-WS shim in index.ts is gone: enqueueViaWs, listJobsViaWs, the pending-response map and the queue branch in the command handler. Nothing but a port crosses that socket now. The three chat channels stop opening the mail store directly. They each carried their own copy of count-rows / enqueue / poll / count-again, coupling three chat bridges to the mail schema — and they enqueued `gmail-sync` unconditionally, the OAuth path, for an app-password account that syncs over IMAP, so the command was already broken. One shared helper calls a new POST /sync-now on the sidecar, which syncs and reports what arrived. queue/handlers/ is now empty; both handlers there were email. The queue is untouched and still serves the Jobs screen. Not moved, and fine where they are: scripts/migrate-emails-to-sqlite.ts and scripts/seed-imap-uids.ts are one-off maintenance scripts that open the store directly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,5 +1,4 @@
|
||||
import type { SidecarEvent } from '../protocol';
|
||||
import type { Job, EnqueueParams } from '../../queue/types';
|
||||
import { initEmailCron, stopEmailCron } from './email-cron';
|
||||
import { initEmailIdle, stopEmailIdle } from './email-idle';
|
||||
import { broadcastEmailNew } from './routes';
|
||||
@@ -8,39 +7,9 @@ import { createSidecarConnector } from '../connect';
|
||||
|
||||
const API_URL = process.env.API_URL ?? `ws://127.0.0.1:${process.env.PORT ?? '5000'}`;
|
||||
|
||||
// ── Queue access via WS ──
|
||||
|
||||
let reqCounter = 0;
|
||||
const pendingQueue = new Map<string, { resolve: (v: unknown) => void; reject: (e: Error) => void; timer: Timer }>();
|
||||
|
||||
function nextQueueId(): string {
|
||||
return `eq_${Date.now()}_${++reqCounter}`;
|
||||
}
|
||||
|
||||
function sendQueueCommand(cmd: Record<string, unknown>): Promise<unknown> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const id = cmd.id as string;
|
||||
const timer = setTimeout(() => {
|
||||
pendingQueue.delete(id);
|
||||
reject(new Error(`Queue command ${cmd.type} timed out`));
|
||||
}, 30_000);
|
||||
pendingQueue.set(id, { resolve, reject, timer });
|
||||
connection.send(cmd as SidecarEvent);
|
||||
});
|
||||
}
|
||||
|
||||
async function enqueueViaWs(params: EnqueueParams): Promise<Job> {
|
||||
const res = (await sendQueueCommand({ type: 'queue:enqueue', id: nextQueueId(), params })) as Record<string, unknown>;
|
||||
if (res.type === 'queue:enqueued') return res.job as Job;
|
||||
if (res.type === 'queue:error') throw new Error(res.error as string);
|
||||
throw new Error('Unexpected response');
|
||||
}
|
||||
|
||||
async function listJobsViaWs(): Promise<Job[]> {
|
||||
const res = (await sendQueueCommand({ type: 'queue:list', id: nextQueueId() })) as Record<string, unknown>;
|
||||
if (res.type === 'queue:list') return res.jobs as Job[];
|
||||
throw new Error('Unexpected response');
|
||||
}
|
||||
// The sidecar used to reach BACK into the platform's queue over this socket to get a sync run —
|
||||
// enqueueViaWs / listJobsViaWs and a pending-response map. Syncs run in this process now
|
||||
// (sync-runner.ts), so the shim is gone and nothing but a port crosses the socket at startup.
|
||||
|
||||
// ── Command handlers ──
|
||||
|
||||
@@ -53,20 +22,6 @@ function handleCommand(cmd: Record<string, unknown>, reply: ReplyFn) {
|
||||
break;
|
||||
|
||||
default:
|
||||
// Check if this is a queue response (from API server responding to our queue commands)
|
||||
if (
|
||||
typeof cmd.type === 'string' &&
|
||||
cmd.type.startsWith('queue:') &&
|
||||
cmd.id &&
|
||||
pendingQueue.has(cmd.id as string)
|
||||
) {
|
||||
const pending = pendingQueue.get(cmd.id as string)!;
|
||||
pendingQueue.delete(cmd.id as string);
|
||||
clearTimeout(pending.timer);
|
||||
pending.resolve(cmd);
|
||||
return;
|
||||
}
|
||||
|
||||
reply({
|
||||
type: 'error',
|
||||
id: cmd.id as string,
|
||||
|
||||
Reference in New Issue
Block a user