plan: nightly visual regression on a second server

Capture every route of a JWT-authenticated dashboard each night, diff against a promoted baseline, and
have an agent look only at what changed.

The blocker is auth, and it fails silently. iris seeds a capture with Cookies, Cookies-wal, Login Data
and Local State — a JWT lives in localStorage, which Chrome keeps in Default/Local Storage/leveldb and
iris does not copy. So an authenticated capture renders the login screen: a real screenshot of a real
page that is the wrong page.

Three ways out, with a recommendation. Copying leveldb works and drags every site's localStorage into
a temp profile. A token in the URL needs no profile at all and removes the desktop plugin from the
plan entirely — that is the one to try first. Seeding through the relay works and needs a browser
paired at 3am, which is the constraint iris exists to avoid.

The rest is deliberate about where the model is spent: pixel diff first, mask what is known to move
rather than raising the threshold, and only changed pairs reach an agent — with "if you cannot tell,
say so" in the prompt, because a confident wrong verdict on night four is how this stops being trusted.

Baselines are promoted by a person, never by a run succeeding, or a regression becomes the baseline on
night two.

Also records what Officer lacks: the queue has retries and no schedules, so this is cron.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-21 02:44:25 +00:00
co-authored by Claude Opus 5
parent d043e6650a
commit 3fb439fabf
2 changed files with 153 additions and 0 deletions
+4
View File
@@ -438,6 +438,10 @@ explaining why it was safe.
mounted and what it knows, the URL-vs-channel split for panel-to-panel communication, and the
persistence key families. Read before building a panel app. Its defect list is
`docs/workspace-panel-todo.md`.
- `docs/visual-regression-plan.md` — **plan, not built**: nightly screenshot regression against a JWT
dashboard on a second server. The blocker is that a JWT lives in `localStorage` and iris copies only
cookies, so an authenticated capture silently renders the login page — plus why diffing must happen
before any agent is asked to look
- `docs/plugin-dependency-scope.md` — a plugin declares WHO must be able to run each host binary. Why the
owner's login shell is the wrong question for a `confined` plugin, the two-part reachability check
(world-executable AND every parent traversable), and why members' binaries on alpha worked only because