# Nightly visual regression, on a second server **Plan, not built.** Written 2026-08-21. Capture every route of a dashboard behind a JWT login, every night, on a machine nobody is sitting at — and have an agent look only at what changed. The target is **not** Officer and **not** this machine: a separate box running Officer with the `iris` and `browser` plugins, pointed at a dashboard the owner runs elsewhere. Its login is a JWT, like Officer's. ## The shape ``` target server (Officer + iris + browser) │ ├─ cron, 03:00 ──▶ capture N routes with iris → runs//.png │ compare against runs/baseline/ → changed[] │ if changed is empty → record and stop, wake nobody │ else → open a chat turn with the changed pairs attached │ └─ agent reads only the changed images, judges each: intended, or regression → notification with the verdict, and a link to the run ``` The expensive, fuzzy step happens only where pixels actually moved. On a quiet night the run costs a few seconds of Chrome and nothing else. ## The hard part: a JWT lives in localStorage, and iris cannot put it there This is the thing to solve first, because everything else is plumbing. Cookie-authenticated sites work today: `iris-browser` copies `Cookies`, `Cookies-wal`, `Login Data` and `Local State` out of a signed-in profile, so a capture rides the same session. **A JWT dashboard is not cookie-authenticated.** The token sits in `localStorage`, which Chrome keeps in `Default/Local Storage/leveldb/` — a directory `iris-browser` does not copy. So a capture of a token-authenticated app renders the login screen, and does it silently: the screenshot is real, the page is real, and it is the wrong page. Three ways out, worth choosing between deliberately. ### A. Copy `Local Storage/leveldb` too — smallest change, biggest blast radius One more entry in `seed_sessions`. It works, and it copies **every** site's localStorage into the capture profile, not just the target's. That is someone's whole browser state in a temp directory, and the directory is a LevelDB — copying it under a live browser can catch a compaction mid-write, which Cookies-the-file mostly cannot. Viable, and the least attractive of the three. ### B. Log in during the capture — no profile at all Most JWT dashboards accept a token in the URL, or can be given one by a first-run page. If the target does, then: ``` iris 'https://dash.example.com/?token=' -o run/home.png ``` and there is no profile, no desktop, no signed-in browser anywhere. The token comes from a service account minted for this, stored in Officer's secret store, and never appears in a screenshot. **This is the recommended path** when the target supports it. It removes desktop from the picture entirely, which removes the only reason that plugin was in this plan. ### C. Seed via the relay, capture via iris Pair a browser once, use the relay to `evaluate` the token into `localStorage`, and let iris capture from that profile. Works without changing either plugin, and needs a browser paired at 3am — which is exactly the constraint iris exists to avoid. Keep as the fallback for a target that supports neither a token URL nor a clean profile copy. ## What to capture **Routes come from a list the repo owns**, not from crawling. A crawler finds what exists; a list says what is supposed to exist, and a route disappearing is the regression you most want caught. ```jsonc // routes.json, in the dashboard's own repository [ { "path": "/", "name": "home" }, { "path": "/reports", "name": "reports", "waitFor": "[data-ready]" }, { "path": "/reports/42", "name": "report-detail" }, ] ``` `waitFor` maps to iris's `--wait-for`, which is what stops a spinner being the thing under test. Two viewports — one desktop, one phone — doubles the run and catches the majority of layout breakage. `--jobs` renders them in parallel. ## Diffing, before any agent is involved Compare each capture with the same route's baseline. A pixel count, a threshold, nothing clever: - **identical** → nothing happens - **differs under threshold** (antialiasing, a clock, a random avatar) → nothing happens, but log it - **differs over threshold** → into `changed[]` The threshold exists because something on the page always moves. **Mask what is known to move** rather than raising the threshold until it passes — a timestamp region is a rectangle to blank before comparing, not a reason to accept 3% drift everywhere. Officer has no image diff today. `sharp` is already a plugin dependency (atelier) and can do it; a raw pixel loop over two decoded buffers is about thirty lines and needs nothing. **Baselines are promoted, never automatic.** A run does not overwrite the baseline because it succeeded — it does so when someone says "yes, that is the new correct". Otherwise a regression becomes the baseline on night two and disappears. ## What reaches the agent Only changed pairs, and each one as **two images plus the route name**: baseline and current. The question asked is narrow, because a broad one gets a broad answer: > This is `/reports` before and after. Describe what changed, then say whether it is a layout break, a > content change, or noise. If you cannot tell, say so. "If you cannot tell, say so" earns its place — a model asked to judge will judge, and a confident wrong verdict on night four is how the whole thing stops being trusted. The verdicts land in a notification (`officer-notify` is already core) with a count and a link, not the images. Nobody wants twenty pictures on a phone at 3am. ## What does not exist yet In rough order of how much they block: 1. **`--wait-for` against a JWT app** — solve auth first (§B), everything else is downstream. 2. **An image diff.** Thirty lines, or `sharp`. 3. **A scheduler.** Officer's queue has retries, not schedules — `scheduleRetry` is the only timed thing in it. So this is system cron calling a script, or a `setInterval` in a plugin sidecar the way the nudger does it. Cron is the honest answer for something that runs once a night. 4. **A place for runs and baselines.** `$OFFICER_ROOT/data/visual-regression/{baseline,runs}/` — outside any plugin directory, because a plugin update replaces that. 5. **Whatever holds this.** A plugin of its own on that server, with `routes.json` as its config and a panel showing the last run. It is not iris and it is not the relay; it is the thing that uses them. ## What this deliberately is not **Not a test runner.** It proves a page renders and looks like it did yesterday. It cannot tell you a button is wired to the wrong handler, and it should not be stretched to try — that is a real test suite, and the two answer different questions. **Not a crawler.** See routes, above. **Not the browser relay.** The relay drives a live browser someone is signed into; this runs when nobody is. They overlap only in that both produce screenshots. If §B holds, the relay has no role in this at all.