The counters said how MUCH and never which file, so a run that was working normally and one that had
wedged produced the same screen: `done` stops moving either way. A photograph is around thirty encodes
and a video is three plus a poster ladder, and none of that was visible from outside.
So the run keeps a transcript and the button opens it.
Every rung reports itself by CONSTRUCTION. Ten `variants.push` call sites became one `add`, which is
the point — a rung added later is described without anybody remembering to describe it. The slow
stages that produce nothing until they finish say so BEFORE they start instead: the HEIC decode, each
video rung, the poster frame. Those are the pauses that read as a hang.
`?since=<n>` is what keeps it cheap. The transcript is thousands of lines and polls once a second, so
sending all of it every tick would be the same megabyte over and over; the client keeps what it has
and asks for the rest. No `since` means everything, which is what a modal opened mid-run wants.
The transcript lives in the shared slot, not the modal, so closing it does not throw away the run —
somebody who looks, closes, and looks again is watching one run rather than two. `startedAt` is what
identifies a run, because `n` restarts at 1 and appending across a boundary would interleave two.
Polling, not a socket. Each line transfers once and the events are seconds apart, so a second
transport to open, authenticate and reconnect would buy nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
publish-crop has emitted AVIF for cropped placements since it was written, and `render()` has returned
<picture> sources for them. The older derived ladder — which serves every UNCROPPED photograph, every
square thumbnail and every video poster — was WebP only. This closes that, so the site has one answer
about what formats it ships rather than two.
Measured on a 256px gallery thumbnail, from the master, no generation loss: 18,186 bytes as WebP at
quality 74, 14,381 as AVIF at matched quality. Around 20% on every photograph, which is the reason to
build a second file per rung — not the seven kilobytes it happens to save on one page.
The two quality numbers are publish-crop's own, so the site encodes one format pair rather than two
opinions about what AVIF is worth.
Not for a drawing. `lossless` is chosen by prefersLossless for line art, where WebP's lossless mode
beats any lossy encoder — the note there records a logo whose lossy rung came out two and a half times
the size of the original. AVIF is a second lossy answer to a question already settled, so it is not
asked, and those ladders carry no sources and render exactly as they did.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`WEBP_QUALITY` (82) is what a rung is worth when somebody opens it and looks at it. Nobody opens a
poster: it fills the frame for about a second and is then covered by the video it belongs to. Paying
photograph quality for that is paying for a second that ends.
Measured on the same pixels with the same encoder, at 768 wide: 22,906 bytes at 82, 17,738 at 74. That
is 23% off the Largest Contentful Paint for a difference nothing is on screen long enough to show.
Not lower than the small rungs use. It IS the LCP, so its first impression is the page's, and 74 is
what every thumbnail on the site already ships at — measured further down, 68 and 62 buy 1.2 and 2.7
KB more and start costing something a still frame will show.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A smooth kernel interpolates. On a photograph that is right; on a drawing it is destructive in a way
that shows up as SIZE rather than as blur — a logo with 174 flat colours downscaled with lanczos3
comes out with thousands of interpolated ones, and the lossless encoder that was winning a moment ago
then has to describe every one.
Measured on fofos-logo-saffron at 160 pixels, all lossless:
lanczos3 13,356 mitchell + nearLossless 7,982
mitchell 9,854 nearest 3,436
The whole 300-pixel original is 8,886. Every smooth option produced a 160-pixel file heavier than the
full-size one, which is what "the ladder made it worse" looks like from the inside.
The cost is real and accepted: nearest at a fractional scale drops rows rather than blending them, so
edges alias. These renditions are shown smaller than they are and the browser's own downscale smooths
what nearest left behind — a 160-pixel logo in an 80-pixel slot is where this lives.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
It probed at 96 pixels and got the answer wrong on exactly the file it was written for. Downscaling a
logo to 96 resamples its flat areas into gradients, so by the time the probe sees it, it already looks
like a photograph. Lossy wins that comparison, and the verdict is applied to every rung above — where
the flat areas are still flat and lossless would have won by a wide margin.
fofos-logo-saffron came out with a 13,074-byte 160-pixel rung and a 22,432-byte 256-pixel one, beside
a 9,144-byte 300-pixel original. Three files larger than the thing they were made from.
480 now, or the source's own width when it is smaller — which is the case that matters, because that
is what a logo is. Not the full size: a picture is still recognisably itself at 480, and probing a
4000x6000 photograph losslessly to learn something already obvious is minutes of encoding across a
library.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
160 was still not a floor. A logo renders at 80 CSS pixels, which is 140 device pixels on an ordinary
phone and 80 on a plain one, and the smallest rung being 160 meant the plain screen had nothing to
take but the retina file.
96 and 128 cost a few hundred bytes each and only exist for things this small — a logo, an icon, an
avatar. Every photograph in the library skips them, because nothing is ever upscaled and a rung wider
than the source is not written.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Lossy WebP is built for photographs — it discards detail the eye does not miss in something full of
texture. A logo is the opposite: flat areas, hard edges, a handful of colours. Encoding one lossily is
worse than useless twice over. It softens the edges, and it makes the file BIGGER, because flat
regions are exactly what lossless compression is good at and exactly what lossy encoding turns into
gradients it then has to describe.
Measured, not theorised: fofos-logo-saffron.png is 300x300 and produced a 22,432-byte 256x256 rung
beside a 9,144-byte full-size one. A smaller picture, two and a half times the file.
The decision is measured too. Every heuristic for this is wrong somewhere — a PNG can hold a
photograph, a JPEG can hold a diagram, plenty of logos are opaque — so it encodes the smallest rung
both ways and keeps the answer. One extra encode of a 160-pixel image per original, and what comes out
is not an opinion about the file: it is the file saying which encoder suits it. Applied to every rung,
because an image does not stop being a drawing at 640 pixels. Recorded in the manifest, because
somebody wondering why one directory is shaped differently should find the answer rather than
re-derive it.
Two other things the logos exposed:
The ladder had no floor. It started at 320 and a logo renders at 80, so fofos-logo-black got a single
12,350-byte rung against a 12,678-byte original — a ladder that saved 328 bytes. 160 and 256 are now
the bottom, and 128 and 192 on the square side.
And square crops of things already square. Both logos are 300x300, so their square rung cropped
nothing: the same image under a second name. The aspect ladder already produces squares for a square
source. 2% tolerance, because a 1000x1002 photograph is square for every purpose this serves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The hero declares a sizes well under its real width on phones so the browser stops multiplying by the
pixel ratio and asks for roughly one image pixel per screen pixel. 448 is the rung that lands on a
412-wide phone once that cap applies; without it the nearest was 512.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The still is the input for three things — the placeholder, poster.webp, and now the poster rungs — and
it is removed once they are done. I added the rung loop BELOW the removal, so sharp was handed a path
that no longer existed and every video failed to convert.
The comment warning that these lines are ordered deliberately was three lines above the mistake.
The removal now sits after everything that reads it, which is where it should have been all along
rather than in the middle of the sequence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A poster was one file at the video's full size — 1280x720 for a hero displayed 412 CSS pixels wide.
Three times the pixels a phone can show, on the element that IS the page's Largest Contentful Paint,
which is the most expensive place on the site to be careless.
384, 512, 768, 1024, plus poster.webp as the widest rung. Those are viewport widths times the density
behind them: 412 at 1.75 wants 768, a tablet wants 768, a desktop wants 1280.
Cut from the still rather than from poster.webp, so each rung is scaled once from the original frame
instead of being a re-encode of an encode. poster.webp keeps its name — the placeholder is cut from
it, the site falls back to it, and a directory built before this has only that one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The ladder was 320, 480, 640, and the middle of that was the problem. A gallery slot measures 193 CSS
pixels on a phone, so at 1.75x it needs 338 device pixels — which did not exist, so the browser
correctly took 480 and downloaded 1.4x the pixels it could use.
Now 256, 384, 480, 640, 960, which is 193 and 475 CSS pixels times the densities people actually hold.
960 is the other end of the same mistake: the ladder stopped at 640, so a retina desktop wanting 950
took 640 and showed it soft.
A rung is a few kilobytes of square WebP. A missing rung is paid for on every visit by everyone whose
screen falls in the gap.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A gallery grid is square slots and the ladder is the picture's own shape, so a 480x720 portrait
arrived, object-cover cropped it to 480x480 in the browser, and a third of every file was discarded
before anyone saw it. Measured here: 108,388 bytes shipped where a 480x480 crop is 70,070.
Cropping in the browser is the right default when the shape is unknown. It is the wrong one when the
shape is known at build time, and for a carousel of square thumbnails it always is.
Thumbnail widths only — 320, 480, 640. Anything larger is a picture being looked at, and a square crop
is a loss of information that only makes sense when the slot demands it.
Centre, never sharp's `attention`. Smart cropping decides differently for every picture, so a row of
thumbnails framed by an algorithm has no common logic to it and the one it gets wrong cannot be
predicted or explained. Centre is what a photographer framing for a square already assumes.
It is a default: a placement carrying its own crop keeps the aspect file and frames it in CSS, because
that crop is a person saying which part matters. Nothing here knows about placements, and should not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Quality 82 went on every rung, so a picture 173 CSS pixels wide in a gallery grid carried the same
fidelity as the copy somebody opens full screen. Compression artefacts scale with display size: at
thumbnail size 74 is indistinguishable from 82 and 15% smaller — 108,388 bytes to 92,568 on the worst
image in this gallery, measured rather than assumed.
640 is the line because it is the widest rung a phone picks for a grid slot. Above it something is
showing the picture large, and 82 is the right answer again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The ladder went 320 then 640, and a phone lands between them. Lighthouse measured the home gallery's
slots at 454 device pixels; with only those two rungs the browser correctly took 640, which is a
184 KB file for a 454-pixel slot — and the same on every image in that carousel.
A rung is one more WebP beside files that already exist, chosen only when it fits better than its
neighbours. Not having it is paid for on every visit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
It was reachable only when the SOURCE was smaller than 720, which is backwards: the rung a phone wants
is the one a 720p master never produces. The hero here is 1280x720 and the only file that existed for
it was 7.8 MB of 720p, served to every device.
`b:a` overrides the preset's 96k with 64k. The audio on a hero loop is ambience, and 64k AAC carries
it — the same argument as the preset's crf 26 for the picture. The rung exists for a phone on a mobile
connection and every kilobit is one it is paying for.
The override goes through av-club's documented contract: a preset supplies the options and anything
passed explicitly wins.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A photograph here is 189 KB of tufted yarn — high-frequency detail in every pixel, which is the worst
case for any lossy codec and not something a quality setting can fix. Re-encoding the same original
with ImageMagick at the same quality 82 gives 199 KB, so av-club is already slightly ahead of a
reference encoder. The 858 KB Lighthouse wants back is not available without softening the texture
the site exists to sell.
What was available was the wait. Each ladder now carries the picture at 16 pixels wide as a `data:`
URI inside its manifest — 300 to 500 bytes — and the site paints it as the image's own background. It
costs no request, which is the point: a placeholder you have to fetch is a smaller version of the
problem. When the real photograph arrives it paints over its own background and the placeholder is
gone, with no load handler, no class to remove and nothing to clean up if the script never ran. The
blur is the browser stretching 16 pixels across the box.
Videos take theirs from the poster still, before it is deleted — the same frame, so the two agree, and
sharp can read a JPEG where it cannot read an MP4.
── Existing directories are topped up, not rebuilt ──
The obvious move is to make `upToDate` insist on a placeholder, which turns all 62 ladders stale and
re-derives the library: minutes of CPU and 67 MB rewritten to add 400 bytes to each. Every rung in
them is already correct.
So `topUp` makes only the missing piece, and reads `320.webp` rather than the original — already
small, already browser-readable, already rotated. Going back to a 4000×6000 HEIC to produce a
16-pixel picture is the expensive half of a rebuild and precisely what this avoids. A directory with
no small rung is skipped rather than forced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`readdir` returns entries in the filesystem's own order — inode order on ext4 —
which is arbitrary and stable only until somebody moves a file.
Harmless for the sync, which does the same work whatever the order. Not harmless
for "Create Gallery": that list becomes the gallery's initial order, and nine
photographs arriving as 05, 08, 01, 07 reads as a bug even though nothing is
wrong.
`numeric` collation, so galeria2 sorts before galeria10 — which is what anybody
naming files that way means.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Moving a file into a folder made the sync look at a new path, find nothing, and
convert it again — minutes of work to produce bytes that already existed a
directory away — while the old folder stayed behind forever, because nothing ever
deleted one.
A move preserves size and mtime, and both are already in the manifest and in the
stat. So the sync indexes what is already derived on that pair, and a file that
has moved has its ladder RENAMED across.
An ambiguous match is refused rather than guessed. Two originals can share a size
and an mtime — a camera dump can write a whole card in one second — and
rebuilding is merely slow where attaching the wrong photographs to a name is
wrong.
Orphans are removed afterwards, and only on a WHOLE-TREE sync. A scoped run knows
about one folder, so everything outside it looks abandoned; deleting on that
basis would take out the library the first time somebody re-synced a subfolder.
Safe otherwise because derived/ is reproducible by definition, which is what its
name has meant since these two folders stopped being called `user` and `treated`.
Both are counted, recorded in the ledger and shown only when they happened, so an
ordinary run stays one short line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things the panel could not answer. A run's state lives in a Map, so a
restart erased it — and with it any record that a file FAILED. That is the gap
the manifests structurally cannot cover: a derived folder records the size,
mtime and variants it was built from, so provenance is already precise, but a
file that did not build has no folder and leaves no trace. Once the watcher is
running, a failure at three in the morning would never be discovered.
So one JSON line per run, appended to media/.sync-log.jsonl. A file rather than
a table because db/schema.ts is emphatic that the connection is the only thing
this plugin stores, and a ledger is losable — re-run and it rebuilds. Beside
derived/ rather than inside it, so wiping derived does not erase the history of
what happened before, which is the comparison it exists for. Written in a
finally, so a run that died still records that it did.
And the line above the button now says what is TRUE rather than what a run once
did: reconcile() walks originals, reads each manifest, and reports current,
missing and changed. It uses upToDate — the same predicate the sync uses to skip
a file — because two definitions of "current" is how a panel comes to say
everything is in step while the next run rebuilds half of it. Missing and
changed are separated because only the second means somebody edited something.
The ledger is not polled. It reads the whole tree, so it loads on arrival and
again when a run stops — the only two moments its answer can change.
Sync also takes an optional folder now, so a subtree can be re-synced on its own
rather than the whole library. The scope is recorded on each run, because "3
built" means different things for one folder and for everything.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`web-1080` was asked for whenever the source was at least 720 tall, so a 720p
video was upscaled to 1080p — the exact thing the image ladder is careful never
to do. The output was a 14.6 MB file that looks worse than the 7.8 MB one beside
it, and it cost 84 seconds of a single core on an idle box. Under a running sync
that overran av-club's 255 s idle timeout and homepage_video.mp4 failed
outright, which is the only reason the waste was noticed.
A rung is now made only when the source is at least that tall. `web-480` becomes
a floor rather than a rung, so a source smaller than 720 still gets something
instead of nothing or an enlargement. An unreadable height makes everything: a
probe that told us nothing is not evidence the video is small.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A HEIC was being decoded once per rung. av-club is stateless per request, so
seven rungs asked for from the original are seven ImageMagick conversions of it
— measured on fofos at 8.5 s each, against 1.1 s for the resize and encode that
follow. Six of the seven produced a pixel-identical intermediate and threw it
away: about 67 seconds per photograph where 16 would do, paid again on every
future upload.
So when av-club says a format needs converting, convert it ONCE into the staging
directory and cut every rung from that. `needsConversion` is asked rather than
guessed, because which formats are expensive is av-club's fact and a list of
them kept here would go on paying after its build stopped needing it. JPEG and
PNG are untouched — writing a full-resolution intermediate to save six cheap
reads would cost more than it saved.
Concurrency drops to one. Two was chosen to match av-club and is the right
number on a machine with room; fofos has 1.87 GiB of RAM, and two concurrent
12-megapixel decodes left 520 MiB free with 648 MB already paged out. The work
is CPU-bound on a box already at 99%, so the second worker was not buying
throughput — it was buying a second resident bitmap.
And the sharp calls now name their output format properly. `output` takes
`{ format, options }`; passing `{ webp: { … } }` was silently ignored, so
everything built before this encoded at sharp's default quality rather than the
82 asked for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The mapping: derived/ mirrors originals/ directory for directory, and where
originals holds a FILE, derived holds a DIRECTORY of the same name — extension
included — with the renditions inside it. Keeping the extension is what makes it
total: a.jpg and a.png are different originals and must not share a folder. The
inverse is pure string work, so the filesystem is the record and there is no
second copy of the truth.
Images get a WebP ladder at 320/640/960/1280/1920/2560 plus full.webp, and
nothing is ever upscaled — a rung wider than the original would be a soft file
that srcset then confidently prefers over the sharp one. full.webp always
exists because for a HEIC it is the only browser-readable copy of the picture
there is.
Video gets av-club's web-1080 and web-720 presets by name, plus a poster. Images
are named by width because srcset descriptors are widths; video by height
because that is av-club's vocabulary and nobody misreads 1080.mp4.
The ladder is fixed here and the SITE chooses from it. This never reads
website/src/media/slots.ts — a plugin that knows a website's layout is a plugin
for one website, and that boundary is the difference between a template and
something fofos-shaped.
Each derived folder carries a manifest naming the size and mtime it was built
from, so the sync is idempotent and resumable: a run that died halfway is
finished by running it again. That is what makes an in-memory job acceptable
rather than merely convenient. It builds into <name>.partial and renames, so an
interruption can never leave a folder that looks finished.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pastilhasandClaude Opus 5