pastilhasandClaude Opus 5
af56eb36ff
Email was the one sidecar built inside out. The platform held ~1,800 lines — the per-account
SQLite store, all 14 HTTP routes, account CRUD, resync, IMAP validation — while the 314-line
sidecar was a scheduler that reached BACK into the platform to do anything
(`import { performResync } from '../../api/email/resync'`).
The sidecar now serves its own HTTP listener and announces `email:server`, and
/api/email/* on the platform is createSidecarProxy like every other one: 1,801 lines down
to 22, with no mail knowledge left in it — not a message, not a folder, not a credential.
The routes moved verbatim, Hono and all. http.ts only reconstructs what the platform's
middleware used to provide: `user` on the context, from the X-Officer-User header the proxy
injects (trusted because this server binds loopback), and an error handler that turns
custom-errors into status codes.
The /email/events SSE stream went with them, which removes a whole round trip: the IDLE
watcher used to send `email:new` over the registration socket so the platform could push to
its SSE clients. Those clients are here now, so it calls broadcastEmailNew in-process and
`email:new` is gone from the wire protocol.
DELIBERATELY NOT DONE YET, and left backwards on purpose rather than half-moved:
- The two sync handlers (email-sync 381 lines, gmail-sync 712) still run in the platform's
queue and now import the store from its new home — a platform → sidecar import, which is
the wrong direction and is temporary. Moving them is option (A) from the plan: the sidecar
schedules its own syncs, independent of the platform Jobs list.
- accounts.ts still imports queue/init to enqueue a sync and to report sync status, and
index.ts still carries the queue-over-WS shim that inversion needs.
- The three channel handlers still open the mail store directly rather than asking over HTTP.
Two things worth knowing while testing: a from-scratch sync holds a proxied request open
well past the 60s idle default, hence timeoutSeconds on the proxy; and `gmail-sync` is
hardcoded in all three channel handlers even though the only account is provider=gmail with
auth_type=password, which routes to IMAP — so "sync emails" from a chat channel is
almost certainly already broken, and folds into the next stage.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
82 lines
2.5 KiB
TypeScript
82 lines
2.5 KiB
TypeScript
import { readdirSync, readFileSync } from 'node:fs';
|
|
import { join } from 'node:path';
|
|
import { openEmailDb, upsertFromRawEml, setSyncMeta } from '../src/servers/sidecar/email/store';
|
|
|
|
const DATA_PATH = process.env.DATA_PATH ?? join(process.cwd(), 'data');
|
|
|
|
// Find all user directories that have Gmail emails
|
|
const targetEmail = process.argv[2];
|
|
|
|
if (targetEmail) {
|
|
migrate(targetEmail);
|
|
} else {
|
|
const entries = readdirSync(DATA_PATH, { withFileTypes: true });
|
|
for (const entry of entries) {
|
|
if (!entry.isDirectory()) continue;
|
|
if (!entry.name.includes('@')) continue;
|
|
const emailDir = join(DATA_PATH, entry.name, 'Gmail', 'emails');
|
|
try {
|
|
const files = readdirSync(emailDir).filter((f) => f.endsWith('.eml'));
|
|
if (files.length > 0) migrate(entry.name);
|
|
} catch {
|
|
// no Gmail dir for this user
|
|
}
|
|
}
|
|
}
|
|
|
|
function migrate(userEmail: string): void {
|
|
console.log(`Migrating ${userEmail}...`);
|
|
const emailDir = join(DATA_PATH, userEmail, 'Gmail', 'emails');
|
|
// Obsolete one-off migration (old .eml-file store → SQLite); kept only to compile.
|
|
const db = openEmailDb(userEmail, userEmail);
|
|
|
|
let filenames: string[];
|
|
try {
|
|
filenames = readdirSync(emailDir).filter((f) => f.endsWith('.eml'));
|
|
} catch {
|
|
console.log(' No .eml files found');
|
|
db.close();
|
|
return;
|
|
}
|
|
|
|
const existingIds = new Set<string>();
|
|
const rows = db.query('SELECT id FROM emails').all() as Array<{ id: string }>;
|
|
for (const row of rows) existingIds.add(row.id);
|
|
|
|
let added = 0;
|
|
let skipped = 0;
|
|
let errors = 0;
|
|
|
|
db.exec('BEGIN');
|
|
try {
|
|
for (const filename of filenames) {
|
|
const id = filename.replace(/\.eml$/, '');
|
|
if (existingIds.has(id)) {
|
|
skipped++;
|
|
continue;
|
|
}
|
|
try {
|
|
const raw = readFileSync(join(emailDir, filename), 'utf-8');
|
|
upsertFromRawEml({ db, id, raw, integration: 'gmail', emailAccount: userEmail, labels: ['INBOX'] });
|
|
added++;
|
|
} catch {
|
|
errors++;
|
|
}
|
|
}
|
|
db.exec('COMMIT');
|
|
} catch (err) {
|
|
db.exec('ROLLBACK');
|
|
throw err;
|
|
}
|
|
|
|
console.log(` ${filenames.length} .eml files — ${added} added, ${skipped} skipped, ${errors} errors`);
|
|
|
|
// Store the latest email date so the next sync only fetches emails after it
|
|
const row = db.query('SELECT date FROM emails ORDER BY date DESC LIMIT 1').get() as { date: string } | null;
|
|
if (row?.date) {
|
|
setSyncMeta(db, 'last_sync_date', row.date);
|
|
console.log(` Stored last_sync_date: ${row.date}`);
|
|
}
|
|
db.close();
|
|
}
|