Start av-club: what it is, and a verified installer
Media processing pulled out of the website that first needed it. ffmpeg, ImageMagick, libvips and exiftool are a real installation rather than an npm install, and encoder settings get revisited as browser support moves — neither of which belongs on the same clock as page templates and database schemas. No processing code yet. What is here is the tooling requirement, an installer, and docs/PIPELINE.md writing down what has NOT been decided — how a consumer talks to this, whether processing blocks an upload, where output goes — so those get chosen deliberately instead of by whatever gets written first. verify.sh checks by running: it encodes a video and probes the result against what ffmpeg was told to make, loads sharp under Bun specifically, and asserts a re-encode still strips GPS. That last one guards a real leak — if a future sharp changes its default, this fails a check rather than quietly publishing the coordinates of a client's living room. Confirmed non-vacuous by hiding ffmpeg and watching it exit 1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,101 @@
|
||||
# The pipeline
|
||||
|
||||
What av-club is meant to do, and — just as importantly — what has not been decided yet.
|
||||
|
||||
**None of this is built.** This document exists so the decisions get made deliberately rather than
|
||||
being settled by whatever gets written first. Where something is open, it says so.
|
||||
|
||||
## The problem
|
||||
|
||||
Someone uploads a file. It might be:
|
||||
|
||||
- a 12 MB photo straight off a phone, 4000px wide, in HEIC
|
||||
- a 4K video in a codec Safari will not play
|
||||
- a JPEG carrying the GPS coordinates of the room it was taken in
|
||||
- a file named `photo.jpg` that is actually a PNG, or a PDF, or nothing at all
|
||||
- something enormous, uploaded by accident
|
||||
|
||||
What a page needs is none of those: a few sizes in a modern format, a poster frame for video,
|
||||
dimensions known ahead of time so layout does not shift, and nothing embedded that the person who
|
||||
uploaded it did not mean to publish.
|
||||
|
||||
## What it should do
|
||||
|
||||
### 1. Identify, do not trust
|
||||
|
||||
Establish what a file **actually is** by inspecting it, never from its extension or the
|
||||
`Content-Type` a browser attached. A rejected upload should say something a non-technical person can
|
||||
act on, not a MIME type.
|
||||
|
||||
### 2. Strip what should not travel
|
||||
|
||||
GPS coordinates, camera serial numbers, timestamps, editing history. `sharp` drops these by default
|
||||
on re-encode — see `docs/TOOLING.md`. Video metadata needs the same treatment and is not free.
|
||||
|
||||
### 3. Probe, and keep the numbers
|
||||
|
||||
Width, height, duration, codec. These get stored, because knowing an image's aspect ratio before it
|
||||
loads is what lets a page reserve the space and not shift as it arrives.
|
||||
|
||||
### 4. Derive
|
||||
|
||||
Images: a small number of widths, in a modern format, never upscaled past the original.
|
||||
Video: something every browser plays, plus a poster frame.
|
||||
|
||||
### 5. Name outputs so they can be cached forever
|
||||
|
||||
Content-addressed or otherwise unique, so a derivative can be served `immutable` honestly and
|
||||
replacing an image never serves a stale one from a cache.
|
||||
|
||||
## What is NOT decided
|
||||
|
||||
These are real forks. Picking one by accident is how a project ends up with an answer nobody chose.
|
||||
|
||||
### How a consumer talks to this
|
||||
|
||||
| Option | Consequence |
|
||||
|---|---|
|
||||
| **A library** the consumer imports | Simplest. Processing happens in the web server's process — a big video transcode blocks a worker and competes for CPU with page rendering. |
|
||||
| **A CLI** the consumer shells out to | Clean isolation, no shared runtime. Every call pays process startup, and errors arrive as exit codes and stderr rather than as values. |
|
||||
| **A service** the consumer calls | Scales and isolates properly. It is another thing to deploy, monitor and secure, which is real work for a small site. |
|
||||
|
||||
### Whether processing is synchronous
|
||||
|
||||
Does an admin wait for a 4K video to transcode while staring at a spinner, or does the upload return
|
||||
immediately and the derivative appear later?
|
||||
|
||||
Asynchronous is better for them and means a job queue, a way to show "still processing", and a
|
||||
decision about what a page renders in the meantime. Synchronous is far simpler and is fine for
|
||||
images; it is not fine for video.
|
||||
|
||||
**These two questions are related** — a queue implies something long-lived, which pushes toward a
|
||||
service.
|
||||
|
||||
### Where output goes
|
||||
|
||||
Not decided, and it interacts with the consumer's deployment: whether a deploy replaces the working
|
||||
directory determines whether generated files can live inside it at all. This is currently an open
|
||||
question in `fofos/platform` too.
|
||||
|
||||
### The derivative ladder
|
||||
|
||||
Which widths, which formats, what quality. Deliberately unanswered — it should follow from a real
|
||||
design, and guessing produces sizes nobody uses and gaps where one was needed.
|
||||
|
||||
### Externally hosted media
|
||||
|
||||
A consumer may store a URL to a video hosted elsewhere rather than a file. Whether av-club has
|
||||
anything to say about those — probing them, fetching poster frames, or ignoring them entirely — is
|
||||
not decided.
|
||||
|
||||
## Principles worth keeping
|
||||
|
||||
**The original is never destroyed.** Derivatives are derivatives. A quality setting judged wrong in
|
||||
six months should be re-runnable, and it cannot be if the source was thrown away.
|
||||
|
||||
**Rejections explain themselves.** The person uploading is not a developer. "That file is 180 MB;
|
||||
the limit is 50 MB" is usable. "Unsupported media type" is not.
|
||||
|
||||
**Verify by running, not by asserting.** `scripts/verify.sh` is the existing example: it encodes a
|
||||
video and reads it back rather than checking a version string. The same standard applies to the
|
||||
processing code.
|
||||
@@ -0,0 +1,88 @@
|
||||
# Tooling
|
||||
|
||||
What is installed to process images and video, what each piece is for, and the failures that look
|
||||
like something else.
|
||||
|
||||
## Bringing a machine up
|
||||
|
||||
```bash
|
||||
./scripts/install.sh
|
||||
```
|
||||
|
||||
Verified on Ubuntu 24.04. What it does, if you would rather do it by hand:
|
||||
|
||||
```bash
|
||||
sudo apt-get install -y ffmpeg libimage-exiftool-perl imagemagick libvips-tools webp libavif-bin
|
||||
bun install
|
||||
./scripts/verify.sh
|
||||
```
|
||||
|
||||
## What each is for
|
||||
|
||||
| Tool | Verified at | Used for |
|
||||
|---|---|---|
|
||||
| `ffmpeg` / `ffprobe` | 6.1.1 | Video and audio. Transcoding, extracting poster frames, and probing a file for duration, codec and dimensions rather than trusting what a browser claimed about it. |
|
||||
| `sharp` (bun dep) | 0.35.3, libvips 8.18.3 | **The image path the code actually calls.** In-process resize and re-encode to WebP/AVIF/JPEG, libvips-backed, so no process spawn per image. |
|
||||
| `exiftool` | 12.76 | Reading and stripping metadata. Client photos routinely carry GPS coordinates and camera serial numbers. |
|
||||
| `imagemagick` | 6.9.12 (`convert`, `identify`) | Inspecting a file by hand when something looks wrong, and formats sharp does not cover. |
|
||||
| `libvips-tools` | 8.15.1 (`vipsthumbnail`) | The CLI side of the same library sharp uses — for comparing behaviour outside the app. |
|
||||
| `webp` | 1.3.2 (`cwebp`, `dwebp`) | WebP at the command line. |
|
||||
| `libavif-bin` | 1.0.4 (`avifenc`, `avifdec`) | AVIF at the command line. |
|
||||
|
||||
The command-line encoders are for inspection and one-off work. Processing goes through `sharp` and
|
||||
`ffmpeg`.
|
||||
|
||||
## Two things that bite
|
||||
|
||||
Both of these fail in a way that looks like a broken installation and is not.
|
||||
|
||||
### There is no `magick` command
|
||||
|
||||
Ubuntu 24.04 ships ImageMagick **6**, which provides `convert` and `identify`. The single `magick`
|
||||
entrypoint is ImageMagick **7**. A script written against `magick` fails with "command not found" on
|
||||
a machine that has ImageMagick fully installed and working.
|
||||
|
||||
Use `convert` and `identify`, or install IM7 deliberately and write down that you did.
|
||||
|
||||
### `sharp` must be resolved from the repo
|
||||
|
||||
Run from anywhere else — `bun -e` in `/tmp`, a script executed from another directory — Bun resolves
|
||||
`sharp` from its **global install cache** and lands on the `wasm32` build, which is missing
|
||||
`@emnapi/runtime` and dies with:
|
||||
|
||||
```
|
||||
TypeError: sharp.libvipsVersion is not a function
|
||||
```
|
||||
|
||||
The native `linux-x64` binary is in this repo's `node_modules`; the process just has to be able to
|
||||
see it. Anything using sharp runs with the repo as its working directory. `scripts/verify.sh` does
|
||||
exactly this, and the reason is commented at the call site.
|
||||
|
||||
## Metadata is stripped by default, and that is load-bearing
|
||||
|
||||
A `sharp` re-encode drops EXIF unless `.withMetadata()` is called explicitly.
|
||||
|
||||
Measured on a JPEG carrying injected GPS and camera tags: **34 tags before, 19 after**, and the 19
|
||||
survivors are filesystem and image-structure facts only — `ImageWidth`, `MIMEType`, `FileSize` and
|
||||
similar. GPS latitude and longitude, `Make` and `Model` were all gone.
|
||||
|
||||
So the default is the safe one, and **`.withMetadata()` is the call that publishes a client's
|
||||
location.** Do not add it without a reason. `scripts/verify.sh` asserts this on every run, so if a
|
||||
future sharp version changes the default, it fails a check rather than quietly publishing
|
||||
coordinates.
|
||||
|
||||
## Behaviour worth remembering
|
||||
|
||||
**`sharp(...).resize(400)` enlarges a smaller source.** A 300×300 input comes out 400×400 and soft.
|
||||
Anywhere a derivative size is requested will want `withoutEnlargement: true`, or small uploads get
|
||||
upscaled into blur that nobody asked for.
|
||||
|
||||
## Verifying
|
||||
|
||||
```bash
|
||||
./scripts/verify.sh
|
||||
```
|
||||
|
||||
Every check runs the tool and compares the result against what was asked for — a version string
|
||||
proves a binary exists, not that its encoders are present or that a native module loads. The script
|
||||
exits non-zero on failure, so it is usable in CI.
|
||||
Reference in New Issue
Block a user