diff --git a/CLAUDE.md b/CLAUDE.md index 787016b2..21ed5b1f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -117,7 +117,8 @@ imported by their package name (`officerdev`, `hooks`, `state`, `types`, `helper - **Language**: TypeScript, strict. `bunx tsgo` is clean — keep it that way. - **Frontend**: React 19, React Router 7, React Query, Tailwind 4, shadcn/ui + custom components - **Backend**: Hono -- **Database**: one Postgres database via Drizzle +- **Database**: Postgres via Drizzle — `officer` for the platform, plus `officer_marketplace` on the + host that serves marketplace.officer.dev (see `docs/marketplace.md`) - **Agents**: Claude Code and opencode, driven through sidecars; tools exposed over MCP ## Data @@ -125,7 +126,11 @@ imported by their package name (`officerdev`, `hooks`, `state`, `types`, `helper Two stores, and the split matters: **Postgres** (`src/databases/officer_db`) holds the account, passkeys, settings, dashboards, -email accounts, queue and pipeline jobs. One directory per feature holding `schema.ts` and +email accounts, queue and pipeline jobs. Since 2026-08-16 there is a SECOND database, +`officer_marketplace` (`src/databases/marketplace_db`), and the split is deliberate: it is a public +catalogue served by a process with no login, no session and no user, so sharing a connection would put +an unauthenticated surface one query away from every private table. It runs only on the host serving +marketplace.officer.dev — `docs/marketplace.md`. One directory per feature holding `schema.ts` and `queries.ts` beside each other; `src/schema.ts` is what `db:push` reads, and it lists the core tables with the plugin ones commented out. Types inferred from the schema in `src/types.ts`. @@ -249,8 +254,10 @@ credentials, SSH and PEM keys, `.env` files and `.claude/settings.json`, then re through any npm token it found. It reached 434 further packages across 1,381 versions. This machine was unaffected only because nothing had installed since 2026-08-02. -**To change a dependency:** edit `package.json`, run `bun install --no-frozen-lockfile` deliberately, -**read the lockfile diff**, and commit it. The friction is the point — an unexplained lockfile change +**To change a dependency:** edit `package.json`, allow the lockfile to change deliberately, **read the +diff**, and commit it. Note that `--no-frozen-lockfile` — which this file and `bunfig.toml` both used to +name — is NOT a flag bun 1.3.14 has; it is silently ignored and the install still fails frozen. Comment +out `frozenLockfile` in `bunfig.toml` for the one command, then put it back. The friction is the point — an unexplained lockfile change in a diff is the signal this exists to produce. **Do not** add `--no-frozen-lockfile` to a script, a Dockerfile or CI to make an error go away. The diff --git a/TODO.md b/TODO.md index 88de76c6..3bff5523 100644 --- a/TODO.md +++ b/TODO.md @@ -322,6 +322,70 @@ pane belonging to the folder you left; and the breadcrumb opted out of that last restored. Deliberate as written — a dashboard can hold two browsers and one URL cannot serve both — but worth re-deciding rather than inheriting. +## Plugin repos — what writing their READMEs turned up + +Three READMEs were written on 2026-08-16 (`music`, `offscale`, `example`) and pushed to their own repos. +Writing them meant reading each plugin against its own documentation, which is the cheapest audit there +is. Everything below was found that way and none of it is fixed. + +**Offscale is still branded headscale in shipped, user-visible places.** + +- [ ] `web/panels.ts` registers three panels named "Headscale servers", "Headscale", "Headscale" — these + are what a user reads in the panel picker. +- [ ] `web/OffscaleServerPicker.tsx:26` renders the literal heading `Headscale`. +- [ ] `web/ConsoleView.tsx:22` mints terminal session ids as `headscale-console-`. That is persisted + state rather than a comment, so renaming it orphans live console sessions — cheap now, less so later. +- [ ] ~12 stale `/headscale` route paths in comments across `web/`, plus `App.tsx:78` in the platform. +- [ ] `PLUGIN.md` has ~11 stale self-references (`headscale_servers`, `listHeadscaleServers`, permission + key `headscale`, `/api/headscale`, panel keys, `useHeadscaleSection()`). + +**Documentation that actively lies.** + +- [ ] `plugins/offscale/PLUGIN.md:445-449` says the ACL assistant is deliberately unwired and must not be + wired up. It IS wired — `sidecar/assist.ts`, `claude-proxy.ts`, `web/PolicyAssistant.tsx` and the + manifest's `readOnlyWrites` all ship. +- [ ] `plugins/offscale/OFFSCALE_API.md` claims to publish the contract "in full" and omits six live + surfaces, including all of `/_officer/companion/*` and `/_officer/enroll/invites`. +- [ ] `plugins/music/MUSIC_API.md` says favourites and now-playing are served by the platform from + Postgres; the sidecar serves them itself, scoped by `X-Officer-User`. It also documents the playlist + item key as `/` while the schema and favourites use the home path — the same file + contradicts itself. +- [ ] `plugins/music/MUSIC_API.md` documents now-playing as one snapshot per user with no mention of + `?device=`, which the schema makes part of the primary key. A mobile client written to that doc is + correct only by accident. +- [ ] `plugins/example/manifest.ts:12` lists `web/Router.tsx` as the frontend convention; discovery + requires `web/panels.ts` AND `web/layout.ts`. The same file documents an `ownerOnly` field that does + not exist and never did. +- [ ] The Companion contract docs the offscale code cites (`COMMS/OFFSCALE_COMPANION_API.md`, + `COMMS/OFFSCALE_INVITE_ENROLLMENT.md`) exist nowhere — `COMMS/` was deleted. Two of the plugin's + eight sections depend on a service with no published spec. + +**Conventions a plugin author has to guess.** All verified in `discover.ts`/`mount.ts`, none documented +outside the example's new README: + +- [ ] Panel `key`s and layout `appType`s must agree, and a mismatch fails SILENTLY — `PluginScreen.tsx` + pins `appTypes.allowed` with `fallback: allowed[0]`, so a typo renders the first panel. +- [ ] The `_` table prefix is unenforced. The barrel is `export *` across every plugin into one + `public` schema, so two plugins choosing the same unprefixed name collide with no error until push. +- [ ] A manifest is module-cached for the process lifetime: editing it does nothing until + `pm2 restart officer`, and `outdated` cannot see a version bump either. +- [ ] Permission-key prefixing is convention, not validation. Only a collision with a CORE key is refused, + silently, at mount, with a console error. + +**Genuinely missing.** + +- [ ] `plugins/offscale` spawns the `ssh` binary (`sidecar/ssh.ts`) but declares no `osDependencies`, so + install never checks for it and the Console fails at runtime instead of refusing at install. + `PLUGIN.md:421` already lists it as a host dependency — the manifest is the file that is wrong. +- [ ] `scripts/reindex-music.ts` resolves its port file from `process.env.DATA_PATH ?? cwd/data`, and + `DATA_PATH` stopped being an env var on 2026-08-13. Run from the repo it looks in + `platform/data/music/.server` and never finds it. One line. +- [ ] `plugins/music/web/MusicPlayerHost.tsx` header says it is mounted once in the persistent + DashboardLayout; it moved into `MusicDetail`. That is exactly the comment that would mislead + someone about why playback stops off-route. +- [ ] A plugin's `assets/icon.png` is only published while installed, so its real icon is invisible in the + store before install. The catalogue's `iconUrl` works around it, and an author has to know to add one. + ## Known bugs - [ ] **Background job logs appeared INSIDE an open file in the code editor.** Reported 2026-08-16 by the diff --git a/bunfig.toml b/bunfig.toml index 30ea0dc1..0dab298f 100644 --- a/bunfig.toml +++ b/bunfig.toml @@ -9,8 +9,11 @@ # transitive range is all it takes; a frozen lockfile is the difference between "we were not affected" # and "we ran `bun install` on the wrong afternoon". # -# When a dependency genuinely needs to change: edit package.json, run `bun install --no-frozen-lockfile` -# deliberately, READ the lockfile diff, and commit it. The friction is the point. +# When a dependency genuinely needs to change: edit package.json, comment out the line below for that one +# command, READ the lockfile diff, and put it back. The friction is the point. +# +# It used to say `bun install --no-frozen-lockfile`. Bun 1.3.14 has no such flag — only `--frozen-lockfile` +# — so the documented escape hatch was ignored and the install failed anyway. Found 2026-08-16. [install] frozenLockfile = true diff --git a/docs/marketplace.md b/docs/marketplace.md new file mode 100644 index 00000000..a49ec792 --- /dev/null +++ b/docs/marketplace.md @@ -0,0 +1,121 @@ +# marketplace.officer.dev + +The public plugin catalogue, and the API the platform installs from. Built 2026-08-16. + +## What it is, in one paragraph + +A website listing plugins, backed by its own Postgres database, serving two read-only JSON routes. The +platform reads those routes to populate its Marketplace screen and to find out where to clone a plugin +from. **The website installs nothing** — installing means cloning a repository, pushing a schema and +starting a process on someone else's machine, so it happens entirely inside the owner's own Officer. + +``` +gitea.officer.dev/plugins/* the repositories (source of truth) + │ scripts/marketplace-sync.ts (manual, reads manifest.ts + README.md) + ▼ +officer_marketplace (postgres) plugins + plugin_metadata + │ src/servers/marketplace/server.tsx :9100 + ▼ +marketplace.officer.dev GET /api/plugins, /api/plugins/:appName + │ fetchCatalogue() live → cached → bundled + ▼ +an Officer install /marketplace → clone → push schema → mount +``` + +## The parts + +| Path | What it is | +| ------------------------------------ | ----------------------------------------------------------------------------------------------- | +| `src/databases/marketplace_db/` | Schema and queries for `officer_marketplace`. Deliberately NOT a workspace package — see below. | +| `scripts/marketplace-sync.ts` | Fills the database from the plugin repositories. Run by hand. | +| `src/servers/marketplace/server.tsx` | The public process. Two GET routes, a health route, and the SPA. | +| `src/apps/marketplace/` | The website. | +| `src/servers/plugins/marketplace.ts` | The PLATFORM's client. Fetch, cache, fall back. | + +## Why a second database + +`officer` is one owner's machine — their accounts, dashboards, mail, wallet. The marketplace is a public +catalogue that is identical for everyone and is served by a process with no login, no session and no +user. Sharing a connection would put an unauthenticated public surface one query away from every private +table on the box. Keeping it separate means the catalogue API structurally cannot read any of it. + +It is also the one database here that can be dropped and rebuilt from the repositories it indexes, which +is a very different operational posture from the platform's. + +`MARKETPLACE_POSTGRES_URL` overrides the connection; by default it is `POSTGRES_URL` with the database +name swapped, because in every deployment that exists they are on the same server. + +## Why the marketplace is not a core process + +`officer-setup`'s `CORE_PROCESSES` does not include it, on purpose. It is one website that officer.dev +hosts — not something every install should run. A copy on every machine would serve a stale catalogue on +a port nobody asked to open. + +On the host that serves it, the PM2 entry is added to that machine's `ecosystem.config.cjs` by hand: + +```js +{ name: 'officer-marketplace', script: 'bun', args: 'run src/servers/marketplace/server.tsx', + cwd: '', watch: false, env: { NODE_ENV: 'production', MARKETPLACE_PORT: '9100' } } +``` + +then `pm2 start ecosystem.config.cjs --only officer-marketplace && pm2 save`. + +## Two tables, split by owner + +`plugins` is the **registry**: app name, developer, clone URL, `listed`. Authoritative, hand-curated, +small. Nothing outside this database may write it. + +`plugin_metadata` is a **cache** of what each repository says about itself — label, summary, icon, +permissions, os dependencies, README. All of it already exists in the plugin's `manifest.ts`, which stays +the source of truth. + +Keeping them apart is what makes the cache safe to be wrong: a metadata row can be deleted, rebuilt or +found stale, and none of that can lose the fact that a plugin exists or where it lives. Merging them +would make a refresh a write to the registry — which is how a failed fetch turns into a plugin +disappearing from the store. + +**Nothing here is trusted by an install.** The catalogue is what the owner sees _before_ consenting; after +the clone, the plugin's own manifest is what the platform reads. A metadata row that disagrees with the +repository is a marketplace bug, never a second source of truth. + +## Sync + +```bash +bun --env-file=.env scripts/marketplace-sync.ts +``` + +Reads every directory under `plugins/`, imports its `manifest.ts`, derives `has` from the directory +rather than from a declaration, reads `README.md`, and upserts. Re-running is always safe: it overwrites, +never merges. + +It rewrites SSH remotes to HTTPS. `example` and `offscale` had `git@gitea…` origins, which would have +handed every installer a URL that only works on the machine that published it — the install fails at +authentication, which reads as "the plugin is broken" rather than "the catalogue is wrong". + +## The platform degrades, it does not break + +`fetchCatalogue()` answers **live → cached → bundled**, in that order, and never throws. The Marketplace +screen is also where an owner manages plugins they already have, so the store being unreachable must not +take that page with it. The cache is per-process with a 5-minute TTL; the bundled list is the old +constant, kept current, so a machine with no internet still shows something true. + +Responses are validated per entry. Not against a hostile marketplace — one that serves a malicious +`source` has already won, since the owner asked to install what it lists — but against a **wrong** one: a +proxy serving HTML as JSON, a half-deployed API, a field renamed on one side. A non-`https://` source is +refused outright, because that string reaches `git clone`. + +## Open items + +- **`[blocker]` Third-party submissions cannot use this sync.** It reads a manifest by importing it, which + is code execution. Fine for our own repositories; not a mechanism a stranger's plugin can go through. + Submissions need either a sandbox or a published `manifest.json` alongside the TypeScript one. +- **`[open]` Sync reads local clones, not the remotes.** The machine running it must have the plugins + checked out. The version that clones each repo into a temp directory is a small change and was left out + of the first pass. +- **`[open]` `rawUrl()` assumes Gitea and a `main` branch.** A GitHub-hosted plugin or one defaulting to + `master` gets a 404 for its icon. The forge should become part of the registry rather than inferred. +- **`[open]` No versions table.** `plugin_metadata` is one row per plugin — a snapshot, not a history. A + store that can offer "install 1.2 instead of 1.3" needs a different shape. +- **`[open]` Nothing refreshes automatically.** Publishing a plugin update does not change the catalogue + until somebody runs the sync. A cron or a Gitea webhook is the obvious next step. +- **`[open]` No search, no categories, no counts.** Three plugins do not need them.