ImageMagick sizes its pixel cache against the machine and will hold an entire decoded image in RAM. On a host with room that is right. On a small one it is how resizing a photograph kills whatever else is running — and "whatever else" includes the website this service exists to serve, which is not an acceptable way for an image to fail. So the decode runs under `-limit memory` / `-limit map` / `-limit disk`. Past the caps ImageMagick spills to a temporary file and carries on, which turns an out-of-memory kill of whatever the kernel picks into a conversion that is slower. Nobody minds a slow photograph; everybody minds a site that went down while one was being resized. Measured on fofos. A 12 MP HEIC peaks at 145 MB, so the 256 MB default does not bind and the output is byte-identical — it is a ceiling, not a tax. Forced to bind, on a 36 MP image with a 32 MiB cap, peak RSS is 12 MB and it still exits 0 with the file written. `-limit thread 1` for the same reason: one decode is one image, and each thread carries its own share of the cache the caps exist to bound. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
av-club
Turning whatever someone uploads into assets a website can actually serve.
People upload photos and videos straight off a phone or a camera: a 12 MB HEIC, a 4K clip in a codec Safari refuses to play, a JPEG carrying the GPS coordinates of the room it was shot in. None of that belongs on a page as-is. This is the thing in between.
It is an HTTP service that exposes as much of sharp, ffmpeg and ffprobe as can be exposed safely, keeping each tool's own vocabulary — sharp operations are named what sharp names them, ffprobe fields come back spelled the way ffprobe spells them. If you know the tool, you know the API.
What it will not let you do is name a file. That single restriction is what makes the rest safe to
open up; see docs/SECURITY.md.
Status
Assets, probing and the full sharp surface work. ffmpeg is designed but not built — that is the next piece.
docs/API.md |
The HTTP surface — start here if you are calling it |
docs/SHARP.md |
Every sharp operation and its bounds. Generated from the validator, so it cannot drift |
docs/SECURITY.md |
The threat model and the invariants. Read before adding an endpoint |
docs/TOOLING.md |
What is installed, and the two things that bite |
docs/PIPELINE.md |
What this is for, and what is still undecided |
Quick start
./scripts/install.sh # system packages + bun deps, then verifies
cp .env.example .env # then set AV_CLUB_TOKEN
bun dev
Verified on Ubuntu 24.04. The installer says so rather than guessing if it finds something else.
# upload something, then resize it
curl -X POST localhost:3020/v1/assets -H "Authorization: Bearer $AV_CLUB_TOKEN" -F file=@photo.jpg
curl -X POST localhost:3020/v1/sharp -H "Authorization: Bearer $AV_CLUB_TOKEN" \
-H 'Content-Type: application/json' -d '{
"input": "as_…",
"pipeline": [{ "rotate": {} }, { "resize": { "width": 1200, "withoutEnlargement": true } }],
"output": { "format": "avif", "options": { "quality": 55 } }
}'
bun run verify checks the tooling; bun run docs regenerates docs/SHARP.md from the validator.
Why this is its own repo
It has system dependencies a web app should not have to assume. ffmpeg, ImageMagick, libvips and
exiftool are a real installation, not an npm install. Keeping them here means a consuming project
can state that it needs this, rather than quietly failing on a machine that happens not to have
ffprobe on its PATH.
It changes on a different clock. Encoder settings, derivative sizes and format choices get revisited as browser support moves. Page templates and database schemas do not.
Nothing about it is specific to one site. Resizing an image and extracting a poster frame are the same problem everywhere, which is why this is not buried inside the project that first needed it.
Its first consumer
fofos/platform — a website with a public site, an admin, and a database. Admins upload media
through the admin; this processes it; the platform stores what comes back.
How that handoff works is not decided yet. The honest options are a library the platform imports,
a CLI it shells out to, or a service it talks to. Each has real consequences for deployment and for
whether processing blocks an admin's upload while they wait. That question is written down in
docs/PIPELINE.md as open, rather than being answered by whatever happens to get built first.
Working agreements
- Commit and push every turn that changes anything — working, broken, or half-finished. The history is the artifact; it is how decisions stay traceable and mistakes stay learnable.
- Docs are live documents, and the code is the source of truth. Docs move with the code in the same commit. A doc contradicting the code is a bug in the doc.
- Direction changes fast, and that is expected. Nothing written here is frozen.