plugins/atelier
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: PLUGIN.md as the current truth, not the extraction

Browse Source 
It described a separate atelier Postgres database, providers as a section in
Officer's settings, and a BLOCKERS.md that was never committed here — none of
which is true. It is now what the code does, and why: the three words that are
not interchangeable, where keys live and why getSecret rather than getKey, the
catalogue's four rules, why refresh is a button, what a refresh can and cannot
learn, and the two ceilings the 240s poll sits under.

Written so that someone arriving cold can carry on from here.
This commit is contained in:
pastilhas
2026-08-21 00:36:28 +01:00
parent 19e94129a0
commit b4c8cfad91
1 changed files with 168 additions and 184 deletions
Download Patch File Download Diff File Expand all files Collapse all files
PLUGIN.md
+168 -184
View File
@@ -1,204 +1,188 @@
# Atelier — extracted from a standalone service into a plugin # Atelier — how it is built, and why
**Status: extracted 2026-08-20, unattended overnight run.** Read `BLOCKERS.md` alongside this file — Current as of **2026-08-20**. Rewritten that evening, because the previous version described an
it has the decisions made without being able to ask, in the order they were found. This file is what extraction rather than the thing that now exists and was wrong in three ways that mattered: it claimed a
moved, what didn't, and why; that one is what was uncertain. separate `atelier` Postgres database, providers as a screen in Officer's settings, and a `BLOCKERS.md`
that was never committed to this repo.
Atelier is image generation as a service: a provider-shaped API over OpenAI's image endpoints, storing **Code is the truth.** Where this file and the code disagree, fix this file.
what it makes so every image has a durable reference. It came from
`~/officer-dev/dockers/officer-atelier`, a standalone Bun/Hono/Postgres server predating the plugin
system, still running in production as `officer-atelier` (PM2, port 3002, `atelier.officer.dev`) — see
"What did not move" below before assuming it is safe to touch.
--- ---
## What moved, and what changed on the way ## What it is
**Nothing pointed the other direction.** Unlike offscale and music, a grep of the whole platform for Image generation as an Officer plugin: a prompt goes in, a job is queued, images come out and stay —
`atelier` found no reverse dependency — no widget, no relay, nothing importing from the standalone with a gallery, collections, and an OpenAI-compatible API an agent can poll rather than block on.
service. It came out whole with no seam to design around, which is the exception rather than the rule
for this kind of extraction.
### The backend — `sidecar/` ```
manifest.ts the plugin's declaration; `sharp` is declared here, shared across plugins
Providers (adapter-shaped: `types.ts`, `openai.ts`, an `index.ts` that dispatches — OpenAI is the only api/router.ts a thin auth-gated proxy. No Atelier knowledge, and it must never grow any
adapter today, and the point of the shape is that a second one is a new file, not a new screen), the sidecar/ the whole contract: catalogue, providers, queue, database
generation queue (Postgres-backed, `MAX_CONCURRENT_GENERATIONS` workers), collections, and the database web/ three panels — Collections, Generate, Gallery
layer moved essentially unchanged. Three things did not: db/schema.ts found by the platform's schema generator; re-exports what lives in sidecar/db
```
1. **Images are served from `GET /files/:filename`, authenticated, not an unauthenticated `PUBLIC_URL`
route.** The standalone service served `/images/*` off Bun's own route table with no session,
because every image needed "a permanent URL" reachable by the mobile app and anything else on the
tailnet. Inside Officer that is not an option — see "What you may NOT do" in the kickoff prompt. The
platform's own `?token=` escape hatch (already used by `FileViewer/file-types.ts` and the music
plugin's `MusicPlayerHost` for exactly this — a browser cannot put an `Authorization` header on an
`<img src>`) covers it: the JSON API returns `/files/<name>` (relative, no host, no token — the
sidecar cannot know the platform's public origin any more and does not need to), and the frontend
appends its own token when rendering (`web/lib/api.ts`'s `fileUrl`).
2. **The OpenAI-key bootstrap fallback reads the secret store before `.env`.** `keyForProvider` still
prefers a provider row's own encrypted key; the fallback that used to be `process.env.OPENAI_API_KEY`
directly is now `getSecret('atelier-<adapter>-key')` first, `.env` second — `getSecret`, never
`getKey`, because a foreign credential must not be silently substituted with a random one. Set it
with `bun run secret:set atelier-openai-key`.
3. **`provider_settings` did not come along.** It is the one-row-per-adapter shape `providers` replaced
(see the standalone repo's own `providers/schema.ts` comment and `scripts/migrate-providers.ts`), and
grepping the server code found no caller anywhere except that migration script — which also did not
move. Its table and its one row stay in the `atelier` database, untouched. If a machine's migration
never ran, that is a real gap, but it is the standalone service's gap, not one this extraction
introduced.
`ping`/`echo` (smoke-test routes, no caller) did not move either — the sidecar's `/_health` covers
liveness, matching every other HTTP sidecar in this codebase.
### The database — stays separate, does not fold into officer_db
Decision 3 said to check whether `collections` and `provider_settings` were empty before folding them
into `officer_db`. Both had rows (2 and 1), so the `atelier` Postgres database stays separate — the
sidecar keeps its own connection (`sidecar/db/db.ts`), reading `OFFICERDEV_POSTGRES_URL` from the platform
`.env`. **This is new territory for the plugin system**: `db/schema.ts` at a plugin's root is the
`officer_db`-prefixed-tables convention (`pushPluginSchema`, `<appname>_` prefix), and this plugin has no
such file — `discoverPlugins()` correctly reports `schema: null` for it, and the install flow's schema
step is simply skipped. Nothing in `src/servers/plugins/` needed to change for this, because "no
`db/schema.ts`" is already a valid state the discovery contract handles — a plugin with an internal,
self-managed database is indistinguishable from a plugin with no tables at all, as far as the platform
is concerned. Worth knowing before assuming every plugin's data lives in `officer_db`.
**Not set automatically.** `OFFICERDEV_POSTGRES_URL` has no default — see `BLOCKERS.md` for why (mainly:
pointing it at the live production `atelier` database while that database's own standalone service is
still running risks two processes racing the same generation queue). The owner has to add it before
installing.
**Migrating into `officer_db` as `atelier_*`-prefixed tables is real follow-up work**, not done here —
five tables (`generations`, `collections`, `collection_items`, `providers`, and the dead
`provider_settings`, which should be dropped rather than migrated), real data, needs a human watching it.
### The frontend — three panels, plus a settings screen
`web/panels.ts` + `web/layout.ts`: Generate, Gallery (+ Lightbox), Collections — the ≈1,200 lines of
real work the brief identified, ported with the shadcn components Officer already ships
(`@/components/ui/*` — the standalone service's own `src/apps/frontend/components/ui/*` did not move,
per decision 1). Providers did not become a fourth panel:
**Providers is a platform Settings screen** (`Settings → Integrations → Atelier`,
`src/apps/officer-web/.../IntegrationsSettings/AtelierProviders.tsx`), per decision 2. Worth flagging
loudly: **`IntegrationsSettings/index.tsx` itself carries a comment, written five days earlier when email
was extracted, that says the opposite** — "a plugin contributes panels, not settings sections, so it
now lives inside the email workspace itself." Decision 2 was explicit and pre-made, so it was followed
anyway, and the contradiction is recorded in `BLOCKERS.md` rather than silently resolved either way.
`AtelierProviders.tsx` calls `/api/atelier/providers` over plain HTTP via `useClient()` — it imports
nothing from `plugins/atelier/`, so nothing here reaches back across the plugin boundary; it is exactly
the same relationship any authenticated frontend has with any authenticated backend.
Panel coordination is URL + one channel, never props (a panel component only ever receives `panelId`):
- Gallery's lightbox is `?selected=<id>`; Collections' detail is `?collection=<id>` — two different
names because both panels render on the same route at once and would otherwise collide.
- Generate bumps `useAtelierRefresh` (added to `src/workspaces/officerdev/src/channels.ts`, same
stable-bump shape as `useFilesRefresh`) when a job finishes; Gallery subscribes to it. This replaces
the standalone app's `onFinished` callback prop, which only worked because `App.tsx` lifted the
gallery's state above both tabs — not an option once they are independent panels.
The default layout is a fixed horizontal three-way split (Generate 26% / Gallery 48% / Collections
26%) rather than the standalone app's hash-based tabs — the `LayoutNode` type has no tab-group concept,
only `group` (split) and `panel`, so three panels means three panes. Users can resize or rearrange
freely; this is only the starting arrangement.
### `sharp` — promoted to a direct dependency
`generations/inspect.ts` needs it to measure width/height/alpha-channel/opacity of a written image —
load-bearing, not incidental: it is how a `background: transparent` request that silently produced an
opaque image gets caught, which the standalone repo's own comments call "the single most consequential
failure for an asset destined to be an app icon." It was already resolved in `bun.lock` as an optional
transitive dependency of `next`; adding it as a direct dependency pulled in no new package versions —
verified by reading the lockfile diff before committing it.
--- ---
## What did not move, and must not be touched ## Three words that are not interchangeable
**The standalone `officer-atelier` service keeps running, unchanged.** Its own `ecosystem.config.cjs` Getting these confused is what produced the UI we threw away on 2026-08-20, so they are fixed:
says plainly: "this is the instance the mobile app talks to, so it holds still." It serves
`https://atelier.officer.dev`, holds real data (87 generations, 2 collections, 5 providers on the | | |
machine this was extracted from), and this extraction did not stop it, did not touch its nginx config, |---|---|
and did not touch the mobile app. **Cutting the mobile app over to the plugin's route — or deciding | **provider** | A vendor. OpenAI. Curated by us, listed whether you have a key or not. |
whether to at all — is the owner's call, not something this run made unilaterally.** Until that happens, | **adapter** | Code that speaks a vendor's HTTP API. Written once, only developers say it. |
having both installed means two independent Atelier deployments existing side by side, which is | **catalogue** | The curated file: which providers exist, their models, and what each model can do. |
expected, not a bug.
There is deliberately **no fourth concept**. There used to be — a "provider" was a database row of
adapter + model + key, and choosing a model was therefore an act of configuration. That produced five
rows nobody had created (a migration minted one per model from a hard-coded list), a key icon and a
delete button inside a dropdown item, and no way to answer "which vendors could I use" without having
already set one up.
**A provider is a vendor with a key or without one. The model is chosen per generation.**
---
## Where things live
**Tables — `officer_db`, `atelier_` prefixed, like every other plugin.** Two of them:
`atelier_generations`, `atelier_collections` (+ `atelier_collection_items`). The extraction originally
kept a separate `atelier` database; that was reversed the same week. A plugin that behaves differently
from every other plugin is a plugin nobody can reason about.
**There is no providers table.** With the model off the row, all that was left of one was *the key for
OpenAI* — and Officer already stores keys.
**Keys — the platform secret store**, `secrets/officer-keys.db`, purpose `atelier-<provider>-key`. One
active value per purpose, `0600` inside a `0700` directory, deliberately outside `data/` so it does not
travel in the same tarball as a database dump. Purposes are created on first use, so nothing is
provisioned in advance and no plugin can read another's.
Read with **`getSecret`, never `getKey`**: `getKey` mints a random value for a purpose it cannot find,
which is right for our own encryption keys and catastrophic for a foreign credential — it means
authenticating with something invented. Falls back to `OPENAI_API_KEY` in the environment, which is how
an install supplies a key without a UI.
**Images — `$OFFICER_ROOT/data/atelier/images/`,** named by generation id, so a row and its file find
each other with no index between them. Not in the plugin directory: that is a git clone an update
replaces, and artefacts kept there vanish on upgrade with nothing logging it.
**Catalogue — two files.** `sidecar/catalogue/shipped.ts` is the curated baseline, in git, reviewed;
`$OFFICER_ROOT/data/atelier/catalogue.json` is what a refresh writes. The second merges over the first,
and a failed refresh therefore degrades to what we shipped rather than to nothing.
---
## The catalogue's four rules
1. **Advisory, never authoritative.** It shapes what the UI *offers*; it never decides what the server
*accepts*. `validate.ts` deliberately refuses to enforce per-model rules — a stale copy of them would
reject valid combinations, and the vendor should be the one to answer. A model the catalogue has
never heard of still generates.
2. **Allowed to be incomplete.** A field a refresh cannot establish stays absent, and absence degrades:
no price means a null cost, which is honest. An invented price sums into a total that is wrong.
3. **Fields merge, not objects.** `GET /v1/models` knows ids and nothing else, so an entry built from it
would otherwise wipe the sizes, qualities and prices that were curated by hand.
4. **Retire, never delete.** A model the vendor stopped listing is marked `deprecated`: hidden from the
dropdown, still resolving everywhere else. The gallery labels historical rows from this list, and 88
images name the model that made them.
### Refresh is a button
There was a daily cron and it was removed the day it was written. A refresh matters at exactly one
moment — you are choosing a provider and want to know its models are current — and a button answers that
question, where a nightly pass can only have already answered it, invisibly, or not. A timer also buys a
standing obligation to hit a vendor from every install forever, whether anyone opens Atelier or not.
`POST /providers/catalogue/refresh?provider=openai`, or without the parameter for all of them. A
provider with no refresher answers 400 rather than a silent success: from a button, "nothing happened"
and "nothing could happen" look identical.
### What a refresh can and cannot learn
`GET /v1/models` is authoritative and thin — ids, an owner, a timestamp. Which is why image models are
matched by pattern (`/^(gpt-image|dall-e)/`) rather than a list: `gpt-image-2` appeared this way, on the
day it shipped, with no release of this plugin.
**There is no pricing endpoint, on any plan.** Prices come off an HTML page, and `openai-refresh.ts`
parses it in a way that gives up rather than guesses — a guess gets written to disk and then believed.
As of 2026-08-20 it gives up: the page renders its prices client-side, and it says so in the log. **The
replacement is not a better regex** — it is handing the page to a model and asking for structured
output, which Officer can already do locally through its own Claude sidecar. That is the next piece of
work here.
---
## The panels
**Collections** (left, top) — a filterable list, one row per collection: icon, name, count, rename and
delete on hover. Create is a dialog. Deleting asks the one question that cannot be undone — keep the
images, or delete them too — unchecked by default, so the recoverable answer is what you get by pressing
Enter without reading. That needed `DELETE /collections/:id?images=delete`, which reads the members
*before* deleting the collection, since dropping it takes the memberships with it.
**Generate** (left, bottom) — provider dropdown (the catalogue, one entry per vendor, "no key" shown
inline), model dropdown (that provider's models), then Add/Change key, Update models, All providers, and
the form.
**Gallery** (right) — every image, or one collection's images when `?collection=` is set. The collection
view lives here rather than in the Collections panel: that panel is a narrow selector, and rendering a
grid inside it replaced the list you were selecting from with the thing you selected.
The panels coordinate through the URL and one refresh channel, never props: `?selected=` is the
lightbox, `?collection=` is what the gallery shows. Generate bumps `useAtelierRefresh` when a job
finishes; Gallery subscribes.
**Removing a panel key rewrites saved layouts** that name it — see `dashboards/queries.ts`, which is a
bug that cost an evening on offscale. The install flow clears this plugin's screen rows, so a panel
removed between versions is safe **as long as the plugin is reinstalled rather than merely pulled**.
---
## The job model
Generation answers **202 with a job id**, never the image. `n: 10` is ten HTTP requests and holding a
connection open for the length of them ties up a socket for minutes; the callers are agents that can
poll. `GET /images/generations/:id?wait=` long-polls up to **240s**, which is bounded by two ceilings
above it — Bun's `idleTimeout` at 255 (256 is rejected outright) and nginx at 320. The innermost has to
be the one that fires, or the caller gets a dropped connection instead of a status.
Each image is its own row with its own cost and duration, so one failure out of four does not discard
the three that worked.
The same model is what video will need when it arrives: a two-minute render is exactly the case 202 was
built for. Video needs a longer poll ceiling, per-second pricing (the catalogue's `Pricing` is already a
shape per unit for this reason), and duration/fps on the row — `has_alpha`/`opaque` go null for it, the
way pricing already goes null when we do not know it.
---
## Serving images
`GET /files/:filename`, authenticated. The standalone service served them from an unauthenticated
`PUBLIC_URL` route; inside Officer that is not an option. A browser cannot put an `Authorization` header
on an `<img src>`, so the JSON API returns a relative `/files/<name>` and the frontend appends its own
`?token=` — the same escape hatch `FileViewer/file-types.ts` and the music plugin already use.
--- ---
## Verification ## Verification
- `bunx tsgo`: clean. - `bunx tsgo` — clean, apart from `Cannot find module 'sharp'` in a checkout where the plugin's declared
- `bun test`: baseline was 1041 pass / 0 fail across 90 files; after this extraction, 1069 pass / 0 fail dependency has not been installed. That is expected outside a real install.
across 92 files — the two new files are `sidecar/serialize.test.ts` and - `bun test plugins/atelier` — 32 pass. `catalogue/merge.test.ts` covers the four merge rules above,
`sidecar/generations/response.test.ts`, both ported from the standalone repo (URLs updated for the new which are the part where a mistake silently rewrites curated data.
`/files/` scheme). No regressions, no file count drop. - The web bundle builds with the platform; the panels are typechecked against the real `@/components/ui`.
- `discoverPlugins()` finds the plugin, the manifest validates, `schema: null` as expected (separate
database).
- The platform-side proxy router (`api/router.ts`) mounts and correctly answers `503 atelier sidecar
not available` with no sidecar running — same behaviour as offscale's.
- The sidecar's internal Hono app was exercised directly (`buildHonoApp()`, real HTTP requests, no port
bound) against the `atelier_dev` Postgres database: `/providers`, `/providers/adapters`,
`/collections`, `/images` all answered `200` with real rows; `/files/<missing>` answered `404`; an
unknown route answered the OpenAI-shaped `404` envelope. Old provider rows correctly failed to
decrypt (a fresh `secret.key` was generated for this checkout, as intended — a key that will not
decrypt is broken, not absent, and `keyForProvider` reports `configured: false` rather than crashing).
- **The real production SPA bundle builds successfully with the plugin included.** Called
`generatePluginsModule` and `rebuildFrontend` directly (the same functions `refreshPluginMounts` calls
during a real install) against a synthetic "atelier is installed and enabled" state — no server
running, no port bound. `Bun.build`, with the real Tailwind plugin, compiled `Plugins.gen.tsx`, all
three panels, and the new `AtelierProviders` settings section in 2.2s with zero errors: 3 outputs, no
warnings. This is the same build step an install performs and exercises everything `bunx tsgo` cannot
— module resolution through Bun's own bundler, JSX compilation, Tailwind processing — without needing
a running server. What it does not prove is that the result **renders correctly in a browser**; see
below.
- **Not done**: a live PM2 install/enable/disable/uninstall/purge cycle through `/marketplace`, per the
runbook's own verification table. This machine's live `officer` process serves the owner's real
session and its own `plugins/` directory is the main checkout, not this worktree — installing through
it would write real rows into the shared `officer` Postgres database from code that only exists here.
That crosses into "affects shared systems beyond the local environment," so it was not done
unattended. The safer checks above cover the same ground read-only or against an isolated database.
--- ---
## Open, for whoever picks this up ## Open
1. **Publish to `gitea.officer.dev/plugins/atelier` and register it in the marketplace.** Not done — see 1. **The pricing extractor** — see above. Everything else in a refresh works; this is the one gap, and
`BLOCKERS.md` for why and the exact commands. it fails loudly rather than silently.
2. **`OFFICERDEV_POSTGRES_URL` has to be set before install.** No default, on purpose. 2. **Video.** Adapter, a longer poll, per-second pricing, duration/fps on the row.
3. **The `atelier` → `officer_db` migration**, described above, once someone can watch it happen. 3. **`bun db:push`** is required to drop `atelier_providers` and `generations.provider_id` on an install
4. ~~`assets/icon.png` and `assets/splash.png` are `[placeholder]` flat swatches.~~ Done that predates 2026-08-20.
2026-08-20: both come from the suite's own masters in the OfficerIcons repo — 4. **The 88 historical images** were generated by the standalone service and predate all of this. Their
`OffAtelierAppIcon` (trimmed, since that master is 1312x1199 rather than square, then centred on prompts are the source for `OfficerIcons/ICON_GENERATION_RECIPE.md`.
a transparent 256 canvas) and `OffAtelierSplashScreen` at 1024. Same shapes and same convention as
plugins/music.
5. **A real PM2 install/enable/disable/uninstall/purge cycle**, on a machine where doing so does not
touch the owner's live session or shared database.
6. **Deciding whether and when to cut the mobile app over** from `atelier.officer.dev` to the plugin's
route, and what happens to the standalone service afterward.
## Reversed 2026-08-20: officer_db, not a separate database
The extraction kept Atelier's own `atelier` Postgres database, because `BLOCKERS.md` decision 3 said to
when the tables already had rows. The owner reversed that afterwards, and the reasoning is worth keeping:
**A plugin that behaves differently from every other plugin is a plugin nobody can reason about.** The
separate database meant a second connection string to set before install, a second thing to back up, a
second thing to get wrong on a new machine, and failure modes no other plugin shares. None of that was
bought by anything except avoiding a migration.
So: four `atelier_`-prefixed tables in `officer_db` — `atelier_collections`, `atelier_collection_items`,
`atelier_providers`, `atelier_generations` — created empty by `bun db:push`, like every other plugin.
Index and check constraint names carry the prefix too; those are global in Postgres and would otherwise
be the one thing that collides.
`ATELIER_POSTGRES_URL` is gone. `sidecar/db/db.ts` reads `OFFICERDEV_POSTGRES_URL ?? POSTGRES_URL`, the
house convention, which the platform already sets.
**The standalone service's data is NOT here.** It is still in the old `atelier` database, and the old
service is still running against it, untouched. Migrating those rows — 2 collections, 73 items, 5
providers, 87 generations — is a deliberate, watched step, done after the plugin is installed and proven
on alpha. Nothing about this plugin depends on it happening.