Videos (all phone-compatible .mp4, plus common containers) that live in an artist or album folder are now indexed alongside audio: - Move 'mp4' out of AUDIO_EXT into a new VIDEO_EXT (mp4/m4v/mkv/mov/webm/avi) — it was wrongly treated as an audio track before. - ffprobeVideo captures file/title/durationSec/width/height per video. - meta.json gains an optional `videos: IndexVideo[]`; a folder with only videos now still gets a meta.json. Manifest entries gain optional `videos: N`. - Video files join the album version signature (changes bump `v` for resync). - New `videosIndexed` status counter + resync-log line. Location is inherent in the folder rel (always an artist/album dir), so no extra location field is needed. MUSIC_API.md documents the videos field + manifest count. No poster/thumbnail generation yet (folder cover is reused). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
11 KiB
Music API (/api/music/*)
Platform API the mobile app uses to stream music and sync a server-built library index, so the app no longer pre-downloads whole tracks or walks/ID3-parses the library on-device.
- Source of truth for the code:
src/servers/sidecar/music/index.ts(theofficer-musicsidecar owns all of this; the platform/api/music/*route is a transparent auth-ing proxy). - Music root:
~/Musicon the server. Allpathvalues are home-relative (e.g.Music/Albums/AC-DC/[1980] Back in Black/01 Hells Bells.mp3), identical to/api/file-browser/raw. <rel>: an album folder path relative to theMusicroot (e.g.Albums/AC-DC/[1980] Back in Black).
Auth
Every endpoint is behind the standard user auth. Two ways to pass the JWT:
- Header:
Authorization: Bearer <jwt>(normal fetches). - Query:
?token=<jwt>— for media elements / native players that can't set headers (audio, images).
401 = no token · 403 = invalid/expired token.
1. Playback — stream a track
GET /api/music/stream?path=<home-relative>&token=<jwt>
Byte-range streaming so the player can seek without downloading the whole file.
| Case | Status | Headers |
|---|---|---|
No Range |
200 |
Content-Type, Content-Length, Accept-Ranges: bytes, X-Audio-Duration |
With Range: bytes=… |
206 |
Content-Range, Content-Length, Accept-Ranges: bytes, Content-Type, X-Audio-Duration |
X-Audio-Duration: track duration in seconds (ffprobe-derived). Read this to set the player's duration up front — it's the fix for AVPlayer reporting an indefinite duration on progressively-streamed VBR MP3s. No need to scan the file.- Errors:
400invalid/missing path ·404not found ·416bad range.
curl -H "Authorization: Bearer $JWT" -H "Range: bytes=0-1023" \
"$BASE/api/music/stream?path=Music/Albums/AC-DC/[1980]%20Back%20in%20Black/01%20Hells%20Bells.mp3" -D -
2. Library index — the synced cache
The server maintains a cache tree that mirrors the library, one entry per album folder. The app syncs this instead of walking + ID3-parsing the library itself.
Each album has a version stamp v (hash of the album's source files' names/sizes/mtimes + its cover).
v changes iff the album's content changed → it's the whole basis of the diff: unchanged v ⇒ skip.
2.1 Manifest — one call, whole library
GET /api/music/manifest
{
"version": 1,
"generatedAt": 1785034701973, // ms; when the index was last built
"albums": {
"Albums/AC-DC/[1980] Back in Black": { "v": "50856380f1ca8f9", "cover": true, "tracks": 10 },
"DJ Sets/Dave Clarke": { "v": "a1b2c3d4e5f6a7b", "cover": false, "tracks": 3 },
"Albums/Metallica/[1989] Live Shit": { "v": "beefbeefbeefbee", "cover": true, "tracks": 0, "videos": 2 },
"Albums/AC-DC": { "v": "c0ffee1234567890", "cover": true, "tracks": 0, "disco": true }
// …
}
}
404 if the index has never been built (see §3). Entries with tracks: 0 are container folders (e.g. an
artist folder). An entry with disco: true is an artist folder that has a discography — fetch its
grouping via /discography (§2.4). videos: N (optional) counts video files (concerts, clips) that live
directly in that artist/album folder — their per-file metadata is in that folder's meta.json (§2.2). A folder
may have any mix of tracks, videos, and disco.
2.2 Album metadata
GET /api/music/meta?path=<rel>
Returns the album's meta.json. Sends ETag: <v>; a request with If-None-Match: <v> returns 304.
{
"path": "Albums/AC-DC/[1980] Back in Black",
"cover": "cover.jpg", // present only if a cover exists
"tracks": [
{
"file": "01 Hells Bells.mp3", // filename within the album folder
"title": "Hells Bells",
"artist": "AC/DC",
"albumArtist": "AC/DC",
"album": "Back in Black",
"track": "1",
"year": "1980",
"durationSec": 312
}
// …
],
"videos": [ // present only for folders that contain video files
{
"file": "1989 - Seattle.mp4", // filename within the folder
"title": "Live Shit: Seattle", // from the container title tag, if any
"durationSec": 8130,
"width": 1280,
"height": 720
}
// …
]
}
All track/video fields except file are optional (absent when the tag/stream info is missing). videos is
omitted entirely when the folder has none.
To stream a track or video: GET /api/music/stream?path=Music/<rel>/<file> (byte-range; works for .mp4).
2.3 Cover
GET /api/music/cover?path=<rel>
Compressed JPEG (≤600px on the long edge, ~30–80 KB). Sends ETag: <v>; If-None-Match: <v> → 304.
Only meaningful when the manifest entry has "cover": true.
2.4 Discography (artist album grouping)
For artist folders (manifest entry with "disco": true), this returns a map of album folder → release
type, so the player can split an artist's album list into sections (Studio, Live, Compilation, Single, EP…).
GET /api/music/discography?path=<artist rel> e.g. path=Albums/AC-DC
Sends ETag: <v>; If-None-Match: <v> → 304.
{
"artist": "Anthrax",
"albums": {
"[1984] Fistful Of Metal": "Studio",
"[1985] Armed And Dangerous": "EP",
"[1994] The Island Years": "Live",
"[1991] Attack Of The Killer B's": "Compilation"
// …
}
}
- Keys are album folder names (
[year] title) — they map 1:1 to the artist's album folders, i.e. the last path segment of that album's manifest<rel>. Group the artist's albums by looking each up here. - Types are a normalized set:
Studio,Live,Compilation,Single,EP,Soundtrack,Remix,DJ-Mix,Demo,Mixtape,Bootleg,Other(unknown values pass through as-is). The player defines section order. - An album folder not present here has no classification → put it in an "Other"/uncategorized section.
- Source of truth is each artist's
_discography.md(author-maintained); this JSON is derived from it and re-generated whenever that file changes (itsvbumps independently of the albums'meta/cover).
3. Building / refreshing the index
The index is built on demand (nothing is pre-built or scheduled). A build is incremental — albums
whose v is unchanged are skipped — and it prunes albums removed from the library.
3.1 Trigger
POST /api/music/reindex → returns IndexStatus (running: true)
GET /api/music/reindex/status → IndexStatus snapshot
IndexStatus:
{
"running": true,
"startedAt": 1785034701973, "finishedAt": null,
"foldersScanned": 45, "albumsBuilt": 12, "albumsSkipped": 3,
"tracksIndexed": 320, "videosIndexed": 4, "coversSaved": 12, "discographies": 3,
"currentPath": "Albums/AC-DC/[1980] Back in Black",
"error": null
}
3.2 Live progress — SSE
GET /api/music/reindex/stream
- Triggers a build if none is running. Pass
?trigger=0to watch only (subscribe without starting one). - Emits
event: progress(anIndexStatus) throttled to ~200 ms, then a singleevent: done(anIndexReport) and closes the stream.
event: progress
data: {"running":true,"foldersScanned":45,"tracksIndexed":320,"albumsBuilt":12,"albumsSkipped":3,"coversSaved":12,"currentPath":"Albums/AC-DC/[1980] Back in Black", …}
event: done
data: {"albums":15,"built":12,"skipped":3,"foldersScanned":45,"tracksIndexed":320,"coversSaved":12,"elapsedSec":37.2,"error":null}
IndexReport (the done payload):
{ "albums": 15, "built": 12, "skipped": 3, "foldersScanned": 45,
"tracksIndexed": 320, "coversSaved": 12, "discographies": 3, "elapsedSec": 37.2, "error": null }
First build of a large library takes a few minutes; re-runs are near-instant (unchanged albums skip via
v).
4. Recommended resync algorithm (app side)
Keep the last manifest.albums you synced. On resync:
GET /api/music/manifest.- For each
<rel>in the new manifest:- new, or
vdiffers from your stored copy → fetchGET /meta?path=<rel>(+GET /cover?path=<rel>ifcover:true, +GET /discography?path=<rel>ifdisco:true); store them under your local<rel>/. vunchanged → skip (no download).
- new, or
- For each
<rel>you have locally that's absent from the new manifest → delete it. - Save the new manifest as your baseline.
Optionally trigger a fresh server build first via GET /reindex/stream (and show progress from its
progress/done events) so the manifest reflects the latest library before you diff.
Result: a resync after adding one album = 1 manifest fetch + that one album's meta + cover. Nothing else moves.
Per-user state — Favorites & Currently-playing
Unlike everything above (library data served by the sidecar), these are per-user and served by the
platform straight from Postgres — same /api/music prefix and same auth. Keys are opaque paths the app
supplies; the server never interprets them:
| kind | key |
|---|---|
track |
home-path — Music/<rel>/<file> (also the /stream path & queue id) |
album |
music-rel — Albums/AC-DC/[1980] Back in Black |
artist |
music-rel — Albums/AC-DC |
Favorites
GET /api/music/favorites→ grouped keys, newest first:{ "tracks": ["Music/…/01 Hells Bells.mp3"], "albums": ["Albums/AC-DC/[1980] Back in Black"], "artists": ["Albums/AC-DC"] }POST /api/music/favorites{ "kind": "track|album|artist", "key": "…" }→{ ok: true }. Idempotent (a repeat add is a no-op).DELETE /api/music/favorites?kind=<kind>&key=<key>→{ ok: true }(no-op if not set). Key passed as a query param (URL-encode it).400 { error: "kind and key required" }on a bad/missing kind or empty key.
Currently-playing (resume)
One snapshot per user — persist while playing (throttled) and on pause / track-change / close; read it on launch to offer "resume".
GET /api/music/now-playing→ the snapshot ornull:{ "homePath": "Music/…/01 Hells Bells.mp3", "dir": "Music/Albums/AC-DC/[1980] Back in Black", "title": "Hells Bells", "artist": "AC/DC", "album": "Back in Black", "durationSec": 312.5, "positionSec": 140, "updatedAt": "2026-07-27T11:27:54.441Z" }diris the folder to rebuild the album queue from (empty for a cross-album queue → resume the single track).PUT /api/music/now-playing{ homePath (required), dir?, title?, artist?, album?, durationSec?, positionSec? }→{ ok: true }(upsert). Omitted fields default to""/0.DELETE /api/music/now-playing→{ ok: true }(clear, e.g. on stop).400 { error: "homePath required" }ifhomePathis missing/empty.
Notes
- Covers are server-compressed (≤600px / q5) — sync them as-is; no client-side resizing needed.
- Durations are exact (ffprobe) in both
X-Audio-Durationandmeta.json'sdurationSec(seconds). - Playback still goes through
/stream— the index is metadata + covers only. (Server-managed offline audio files is a separate, later feature.) - Errors are plain HTTP:
503if the music sidecar isn't connected,502if it's unreachable.