The HTTP service: assets, probe, and the full sharp surface

Exposes sharp and ffprobe with their own vocabulary — operations are named what
sharp names them, ffprobe fields come back spelled the way ffprobe spells them —
so the tools' own documentation applies and there is no house dialect to learn.

Safety comes from one architectural invariant rather than from filtering: the
caller never supplies a path, a URL or a protocol. Input is an id we issued, output
is a name we chose. That removes the entire class of attack that makes "just pass
the flags through" a remote shell, which is precisely what makes it safe to open up
everything else. docs/SECURITY.md has the reasoning and the specific attacks.

The vocabulary is a data table, so the validator and docs/SHARP.md are generated
from one source and cannot drift. Two things are deliberately absent from it:
withMetadata, because that is how a client's GPS coordinates get published, and
limitInputPixels, because a caller who could raise it could turn a 1 KB PNG into a
40 GB allocation.

Verified against a running server, not asserted: traversal ids rejected, unknown
operations and typo'd parameters rejected by name, out-of-range values rejected
with the bound, a GPS-tagged photo coming out clean end to end, video probe, Range
requests, and a rejected upload leaving nothing on disk.

Found while building: config threw at import if AV_CLUB_TOKEN was unset, so
generating documentation required a production secret. The token is read at boot in
server.ts now — still a refusal to start, no longer a module-scope landmine.

ffmpeg is designed in docs/API.md and not built yet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Pastilhas
2026-08-20 02:37:57 +01:00
co-authored by Claude Opus 5
parent 81c8e17a8e
commit 1597fccd75
15 changed files with 2153 additions and 8 deletions
+31 -8
View File
@@ -6,27 +6,50 @@ People upload photos and videos straight off a phone or a camera: a 12 MB HEIC,
Safari refuses to play, a JPEG carrying the GPS coordinates of the room it was shot in. None of that
belongs on a page as-is. This is the thing in between.
It is an HTTP service that exposes as much of **sharp**, **ffmpeg** and **ffprobe** as can be exposed
safely, keeping each tool's own vocabulary — sharp operations are named what sharp names them,
ffprobe fields come back spelled the way ffprobe spells them. If you know the tool, you know the API.
What it will not let you do is name a file. That single restriction is what makes the rest safe to
open up; see [`docs/SECURITY.md`](docs/SECURITY.md).
## Status
**Early.** Right now this repo contains the tooling requirements, an installer, and the
documentation of what the pipeline is meant to become. The processing code is not written yet.
Assets, probing and the full sharp surface work. **ffmpeg is designed but not built** — that is the
next piece.
| | |
|---|---|
| `scripts/install.sh` | Brings a machine to the state this needs, then verifies it |
| `scripts/verify.sh` | Checks the tooling by running it — usable standalone |
| `docs/TOOLING.md` | What each tool is for, and the two things that bite |
| `docs/PIPELINE.md` | What the pipeline is supposed to do, and what is still undecided |
| [`docs/API.md`](docs/API.md) | The HTTP surface — start here if you are calling it |
| [`docs/SHARP.md`](docs/SHARP.md) | Every sharp operation and its bounds. Generated from the validator, so it cannot drift |
| [`docs/SECURITY.md`](docs/SECURITY.md) | The threat model and the invariants. Read before adding an endpoint |
| [`docs/TOOLING.md`](docs/TOOLING.md) | What is installed, and the two things that bite |
| [`docs/PIPELINE.md`](docs/PIPELINE.md) | What this is for, and what is still undecided |
## Quick start
```bash
./scripts/install.sh # system packages + bun deps, then verifies
./scripts/verify.sh # verification on its own
./scripts/install.sh # system packages + bun deps, then verifies
cp .env.example .env # then set AV_CLUB_TOKEN
bun dev
```
Verified on Ubuntu 24.04. The installer says so rather than guessing if it finds something else.
```bash
# upload something, then resize it
curl -X POST localhost:3020/v1/assets -H "Authorization: Bearer $AV_CLUB_TOKEN" -F file=@photo.jpg
curl -X POST localhost:3020/v1/sharp -H "Authorization: Bearer $AV_CLUB_TOKEN" \
-H 'Content-Type: application/json' -d '{
"input": "as_…",
"pipeline": [{ "rotate": {} }, { "resize": { "width": 1200, "withoutEnlargement": true } }],
"output": { "format": "avif", "options": { "quality": 55 } }
}'
```
`bun run verify` checks the tooling; `bun run docs` regenerates `docs/SHARP.md` from the validator.
## Why this is its own repo
**It has system dependencies a web app should not have to assume.** ffmpeg, ImageMagick, libvips and