From c5ed6cc32c6a5d391f7918c630637852fd2289e0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9=20Padez?= Date: Mon, 17 Aug 2026 23:52:09 +0000 Subject: [PATCH] plan of record for extracting email into a plugin --- docs/email-plugin-extraction.md | 148 ++++++++++++++++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100644 docs/email-plugin-extraction.md diff --git a/docs/email-plugin-extraction.md b/docs/email-plugin-extraction.md new file mode 100644 index 00000000..6df0279c --- /dev/null +++ b/docs/email-plugin-extraction.md @@ -0,0 +1,148 @@ +# Extracting email into a plugin — plan of record + +Started 2026-08-17. The runbook is `plugins/EXTRACTING-A-PLUGIN.md` and it is the authority; this file only +records what is specific to email — the boundary, the decisions taken, and what they cost. + +**Status: mapped, decisions taken, not yet moved.** + +The owner asked for a full extraction with no interim step ("no temporary steps"), and was explicit that +speed is not the constraint. Email is therefore NOT being switched back on in place first, even though that +is a four-line change — doing so would create `email_accounts` as a core-owned table that the extraction +would immediately have to hand over. + +--- + +## What email actually is + +~5,500 lines, and the shape is better than its reputation. + +| part | where | lines | difficulty | +| ----------- | ------------------------------------------------- | ----- | -------------------------------------------------------------------------------------------------------- | +| sidecar | `src/servers/sidecar/email/` (14 files) | 3,697 | easy — self-contained, already announces `email:server` through the generalised `${string}:server` union | +| api router | `src/servers/api/email/router.ts` | 22 | trivial — a `createSidecarProxy` with a 1800s timeout that is load-bearing | +| database | `officer_db/src/email/` | 85 | easy — ONE table, `email_accounts`, already carrying the `email_` prefix a plugin requires | +| frontend | `officer-web/Screens/Dashboard/Email/` | 1,150 | easy — already a `WorkspaceView` with three panels | +| accounts UI | `Settings/IntegrationsSettings/EmailAccounts.tsx` | 509 | moves into the plugin | +| types | `workspaces/types/email.ts` | 29 | moves whole | + +Two findings that made this smaller than the survey suggested: + +- **The frontend is already panel-shaped.** `EmailScreen` renders `WorkspaceView` with a `components` map of + `email-list` / `email-reader` / `email-chat` and a `defaultLayout`. The plugin contract wants exactly that, + declared as `web/panels.ts` + `web/layout.ts`. This is a re-declaration, not a rewrite. +- **Theme debt is 17 palette classes**, all in the status/badge colours. The runbook requires paying this + down during the extraction; at this size that is minutes, not a project. + +**`src/workspaces/emailer/` is NOT part of this.** It is the outbound transactional mailer — React Email +templates, `sendMail`, `getTransport` — used by `forgot-password.ts`, `server-settings/smtp.ts` and the +queue engine, all core. It shares the word "email" and the `nodemailer` dependency and nothing else. + +--- + +## The boundary, and the one thing that decides it + +A plugin importing platform code is fine (`@@/…`). The reverse is not, and email has three of them. + +### `google-auth` is not a problem + +Five sidecar files import `getValidGoogleAccessToken` / `refreshGoogleAccessToken` from +`api/integrations/google-auth`, which core `integrations.ts` also uses. The survey called this the hardest +problem in the extraction; it is not one at all. That direction is plugin→platform, which the runbook +explicitly permits, and `google-auth` is generic Google OAuth that knows nothing about mail. **It stays in +core and the plugin imports it via `@@/`.** The only real work is that the import style is inconsistent +today (`../../` in three files, `@@/` in two), so a mechanical rewrite would miss cases. + +### The real problem: core reads email's database + +Three sites, and the first has already taken the platform down once (`fbb6d6c7`, 2026-08-13 — an unguarded +top-level await on `getEmailAccounts` rejected and put `officer-agent` into a PM2 restart loop, taking every +live chat session with it). + +1. `sidecar/claude/user-instance.ts` — queries `email_accounts`, picks the first enabled one, and bakes + `OFFICER_EMAIL_DB` into the agent's MCP config so the `email_db` tool can read it. +2. `api/chat/websocket.ts` — `resolveEmailCwd` queries the same table so a chat with `context: 'email'` + runs in that account's directory. +3. `api/integrations/integrations.ts` — the Google OAuth callback appends `/email` to the user's dock. + +**Note the `email_db` tool is not in this repository.** It lives in `PI_TOOLS_DIRS`, resolved at runtime, so +it is invisible to grep and will not follow the extraction wherever the boundary lands. + +--- + +## Decisions + +The owner was asked about the first two and had stepped away; both are taken on the lower-risk option and +both are reversible. **Revisit them before believing this document is settled.** + +### 1. Keep `email_db` working, by convention rather than by import — TAKEN, REVERSIBLE + +Core stops importing `getEmailAccounts` and instead finds the newest `DATA_PATH//email_accounts/*/emails.db` +on disk. Same for `resolveEmailCwd`. + +- **Removes the code dependency** that caused the crash — no core file imports anything email again. +- **Keeps the feature.** "Ask the agent about my mail" is a large part of why this product is worth running, + and silently deleting it during a refactor is not a trade the refactor gets to make on its own. +- **Costs** a vocabulary leak: core still knows the directory name `email_accounts`. The runbook says the + platform must not know a plugin's vocabulary, so this is a deliberate, recorded exception rather than an + oversight. The clean version is decision 1c below. + +Rejected for now: **clean cut** (purest, but removes a working feature by fiat) and **plugin ships its own +MCP tool** (correct long-term; needs a plugin→agent tool mechanism that does not exist, so it would block +this extraction on new platform work). + +### 2. Account management becomes a panel in the email workspace — TAKEN, REVERSIBLE + +`EmailAccounts.tsx` leaves core Settings and becomes a panel. The plugin contract allows panels and nothing +else, so this is the shape available; the alternative is a second plugin route. **User-visible change:** you +will add and remove mail accounts from `/email`, not from `/settings/integrations`. + +### 3. Encrypt the credentials — TAKEN + +`email_accounts.credentials` is jsonb holding an IMAP app password in the clear. It is encrypted through +`officerdb/crypto` under its own `email` secret-store purpose as part of the move. + +This is normally the kind of scope creep to refuse. It is free here for one reason: **the table does not +exist and has never held a row** — it was commented out of `schema.ts` on 2026-08-13 and `db:push` therefore +never created it. There is no migration, no backfill and no compatibility window. Doing it later would need +all three. + +### 4. `email_accounts` stays in `USER_DIRS` — TAKEN + +Core pre-creates the directory for every account at 0700. Follows from decision 1: core has to know the path +anyway, and having two places create it differently is the bug `USER_DIRS` exists to prevent. + +--- + +## Order + +Runbook order, with email's specifics: + +1. `plugins/email/` skeleton + `manifest.ts`. `appName` is the directory name. +2. **Sidecar** — 14 files, whole. Rewrite the five `google-auth` imports to `@@/`. +3. **Database** — `officer_db/src/email/` → `plugins/email/db/`. Schema takes `officerdb/auth/schema` for + the `users.id` FK, the one reference a plugin may make. Encrypt `credentials` (decision 3). +4. **API** — the 22-line proxy router → `plugins/email/api/router.ts`, exporting `router`. +5. **Frontend** — Screens/Email + EmailAccounts → `web/panels.ts` + `web/layout.ts`. Pay down the 17 palette + classes here. +6. **Core cleanup — delete, never comment.** `hono.ts` mount + import, `registry.ts` entry (both `api` AND + the still-live `routes: ['/email']`), `App.tsx` routes, `Screens/Dashboard/index.tsx`, `usePageTitle.ts`, + the Settings section, `types/index.ts`, **both** database barrels, and the three reverse imports above. +7. **Delete the `app-store/catalogue.ts` entry.** Not optional and not cosmetic: `permissionAvailability` + derives from `sidecar_installs`, a plugin only ever gets a `plugin_installs` row, so a leftover catalogue + entry makes the permission permanently `unavailable` and blanks the screen on a machine where the plugin + is installed and healthy. This has bitten twice already — headscale, and nearly music. +8. `bunx tsgo` until clean. It finds the wiring that greps miss. +9. Verify live through the runbook's table: install → API → screen → dock → permissions → disable → enable → + uninstall → **purge** → install again. Purge is the only honest test of a first install. +10. Its own public repository, a README, and a marketplace catalogue row — an extraction is not finished + when it installs from disk. + +## What is NOT known yet + +**Whether email works.** It has been off since 2026-08-13 and `email_accounts` has never existed on this +machine, so nothing has exercised IMAP sync, Gmail sync, sending or the UI in that time. The sidecar starts +cleanly and its only error is the missing table — which is evidence the code did not rot, and not evidence +that it works. + +So the first install is also the first functional test this feature has had in four days, and a failure then +is as likely to be pre-existing as extraction damage. Diff against git history before assuming which.