a desktop audit was not a desktop audit

`formFactor: 'desktop'` changes which thresholds a metric is scored against and
NOTHING else. Throttling and the user agent keep their defaults, which are mobile
— so every desktop run was a desktop-sized window on a 4x-slowed CPU over slow 4G,
announced as a moto-g.

It read 86 where the same page reads 100. That gap was reported repeatedly as
network latency to the host. It was not latency; it was this. Measured after the
fix: desktop 100, LCP 0.4s against 2.2s.

Nine more from the same review:

  runtimeError was never checked, so a page that failed to load reported
  "nothing failing" — the most reassuring possible output for a site that is down

  --json exited 0 before the gate, so the documented non-zero exit was true for a
  person reading the table and false for the only caller that could act on it

  scoreDisplayMode 'error' was filtered out while still nulling its category, so
  a category could read 0 with an empty findings list

  a missing metric serialised as 0, which a script reads as a perfect measurement

  `source` is a string on some audits and an object on others; String() on the
  object printed [object Object]

  the watchdog's surrender was permanent — the health branch sat below the check,
  so once it stopped trying it also stopped noticing, and a server fixed by hand
  left it asleep until the platform restarted

  a failed restart leaves no process at all, because startServer deletes before
  it starts. status null then reads as "somebody stopped it", which the watchdog
  leaves alone — one bad restart would have turned it off and taken the server
  with it, silently

  the skill claimed the panel uses this engine. Nothing but the CLI does

  and gave `bun run build:site` with no cwd, which only works from the website
  checkout — api/static-site.ts documents that and the skill did not

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-05 18:42:45 +01:00
co-authored by Claude Opus 5
parent 05ad4f4d51
commit 74c6776c75
4 changed files with 147 additions and 27 deletions
+21 -8
View File
@@ -6,8 +6,8 @@ version: 1
---
`bin/lighthouse.ts` runs Lighthouse against a URL in a real Chrome and prints what failed, with the
resources each finding names. It is the same engine the panel uses (`api/lighthouse.ts`), so a number
you get here is the number the panel would show.
resources each finding names. The engine is `api/lighthouse.ts`, which is where the panel will read
from when it grows a screen for this — today the CLI is the only caller.
## Running it
@@ -34,8 +34,14 @@ static files under `dist/`, served by nginx behind a proxy, with different cachi
compression and a trailing-slash redirect. **They are not the same page** and their numbers do not
agree.
So: `bun run build:site` first, then audit `https://web.fofosfluffybastards.com/...`. A score from the
dev server is not wrong, it is about something nobody visits.
So: build first, then audit the public URL. The build has to run in the website checkout and with the
site's own environment — see `api/static-site.ts`, which documents both:
```bash
cd ~/officerdev/data/*/FofosWebsite/website && bun run build:site
```
A score from the dev server is not wrong, it is about something nobody visits.
## Reading the output
@@ -78,10 +84,17 @@ Performance is computed from these and nothing else. Insights are diagnostics; t
## Traps
**A desktop number from another machine includes that machine's latency.** Auditing from officer-dev
against fofos crosses a network the owner's browser does not. Compare like with like: two runs from
the same place, before and after a change. An absolute score from somewhere else is a different
question.
**A desktop audit is desktop in four ways, and `formFactor` is only one of them.** Setting
`formFactor: 'desktop'` changes which thresholds a metric is scored against and NOTHING else — the
throttling and the user agent keep their defaults, which are mobile. A run configured that way is a
desktop-sized window crawling on an emulated phone, and it reads about 86 where the same page reads
100. `api/lighthouse.ts` sets `screenEmulation`, `throttling` and `emulatedUserAgent` together for
this reason; if you write your own runner, set all four or the number is about nothing.
**A number from another machine includes that machine's network.** Compare like with like: two runs
from the same place, before and after a change. An absolute score from somewhere else is a different
question — though check the paragraph above first, because that explains far more of a gap than
latency usually does.
**Run-to-run variance is real** — a point or two either way on the same build. Chase a finding, not a
number. If a score moves and no finding changed, nothing happened.