/email/messages and /email/stats built `labels LIKE '%${folder}%'` by string
interpolation, and `folder` comes straight off the query string. Five statements across
the two handlers were exposed. The fragment is bound now, and it carries its parameters
with it because each handler builds several statements from the same fragment and has
to spread them in order.
Checked against an in-memory table: inbox/INBOX/SENT/all return exactly what they
returned before, and `x' OR 1=1 --` now matches nothing instead of being SQL.
Also: page and limit reached the bindings as NaN for any non-numeric value, so a
mistyped query param was a 500. They fall back to their defaults now.
And deleted src/servers/sidecar/email-cron.ts — 92 lines imported by nothing. The live
cron is sidecar/email/email-cron.ts; this was an older copy that still reached into
queue-runner and google-auth directly, so leaving it there invites someone to fix the
wrong file.
This is the first commit on the email branch; the placement problems (the whole mail
store, both syncs, and the resync coalescing that cannot work across processes) are
untouched and much larger — see SIDECAR_WORK_LOG.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
it was the only sidecar living outside src/servers/sidecar/ — it sat in
api/terminal/ next to the bridge that talks to it, which is the one place a reader
looking for "the sidecars" would not check. now src/servers/sidecar/pty/index.mjs,
matching every peer, with a note on the pm2 entry about why this one is node and
.mjs (node-pty is a native addon) rather than bun and typescript like the rest.
the templates/ directory went to api/users/, next to provision.ts:seedShellConfigs,
which is now its only consumer — the sidecar's duplicate seeder went with the
sandbox branch in the previous commit. api/terminal/ is left holding exactly one
thing: the websocket bridge.
no behaviour change. the pm2 entry's script path changed, so `pm2 restart
officer-pty` is not enough — pm2 remembers the old path until the entry is deleted
and started again. commands are in SIDECAR_WORK_LOG.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
officer built the whole PtyInitConfig: it read the owner's SHELL (defaulting to
/bin/zsh), added `-i`, read their HOME, expanded `~` against it, and hardcoded
`host: true`. none of that is a proxy's business — the sidecar is the process that
calls pty.spawn, so it is the one that should know what to spawn and where.
the config now carries only what the bridge actually knows: sessionId, the folder
the panel was opened on, and the client's cols/rows. shell, args, home and cwd
resolution moved into the sidecar. home comes from HOME_DIR ?? HOME, mirroring
data-path.ts:getOwnerHomeDir — terminal was the one host-executing surface reading
process.env.HOME directly, which is identical here and divergent anywhere HOME_DIR
is set to something else.
deleted the bwrap sandbox branch rather than moving it. it was selected by
`config.host`, which officer hardcoded to true, so it never ran — and it expected
`shell` to contain a fully-built bwrap command that nothing on either side ever
built. it could not have worked. a terminal here is the owner's own shell on the
owner's own machine by design (platform/CLAUDE.md), so there is no jail to preserve.
its ensureUserFiles half duplicated api/users/provision.ts:seedShellConfigs, which
is the live seeder of those same templates and stays.
also deleted the 'cwd' handler that turned a message into `cd <path>\r` typed at
the shell. no frontend has ever sent that message — the browser composes its own cd
— so it was unreachable, and synthesizing keystrokes is not something a relay
should do.
the integration test pins SHELL and HOME_DIR now that the sidecar reads them, and
asserts the shell starts in the resolved `~` rather than officer having resolved it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
the pty sidecar registered `term.onData` with the socket that happened to be live
when the session was created. officer is a pm2 peer that restarts constantly, and
every restart hands this process a brand new socket, so every pre-existing session
went on writing to a closed one — where sendJson's readyState check dropped it
silently. the shell survived and still accepted input, because input arrives on the
new socket, but nothing ever came back. you typed and the terminal sat there. the
only way out was to close the panel, which orphaned the shell.
sendJson now reads the module-level socket at send time instead of taking one as an
argument, so there is no socket to capture and go stale. that is the whole fix.
the scrollback replay on re-attach becomes its own event, pty:replay -> 'replay'
on the browser socket. it used to arrive as ordinary output, which was fine for a
page load (fresh xterm) but not for a restart: the browser keeps its terminal, so
replaying blind printed a second copy of everything still on screen. marked as
history, Terminal.tsx resets and rebuilds from the sidecar's 50KB buffer instead.
it also stays out of the `output` branch so it cannot re-trigger the command /
initial-input logic that scrapes output for a sentinel.
added an integration test, because this is a reconnect bug and nothing short of an
actual reconnect proves it: it stands up a fake registration socket, runs the real
sidecar against it, echoes into a real shell, kills the socket, rebinds the same
port the way pm2 does, and asserts output still flows. verified it fails against
the old sendJson (times out after 15s waiting for the post-restart echo) and
passes in ~400ms with the fix. it never touches the running officer — the sidecar
dials API_URL, overridden per spawn.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
a browsed folder is a path, not a file list, and what clicking one means is
"everything under here" — so the expansion happens in the sidecar, off the
cache, rather than making the browser walk the tree a level at a time and
rebuild paths it only half knows.
browse reports file names as basenames, unlike search, so the peer's real
path is rejoined from the folder row. sizes come from the cache too: slskd
matches a queued download on filename AND size, so a number supplied by the
client would be a transfer that silently never starts.
a subtree can be the peer's whole share (284k files on one measured peer), so
an over-limit request is refused with its count rather than truncated into a
partial download nobody asked for.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
the tree search asked for no limit and clampInt read a missing param as 0
(Number(null) is 0, and 0 is finite), so it clamped to the minimum and ran
with LIMIT 1: one match came back, its siblings looked like non-matches, and
a folder with 25 matching albums showed none of them.
on top of that, a filtered row was a dead end — it only ever showed the
matches, with a note counting what it was hiding. now the note is the same
"show all N folders" button browse mode already had: clicking it drops that
row out of the filtered set, so from there down it is ordinary lazy
browsing, which is usually why you searched for the folder in the first
place. dropped the "don't match the filter" wording with it, since a
truncated result makes that sentence false.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
slskd's browse response is flat: every folder is a full backslash-delimited
path. Rendered as-is, a filter for "pogues" gave 26 rows that all began with
the same 31 characters, and the real hierarchy — which is the only way to tell
an artist folder from an album folder — was invisible.
The shape is now derived once, at ingest, in the sidecar: buildTree() links
each path to its parent, synthesizes any ancestor slskd omitted (measured:
exactly one missing across ~30k folders on two real peers, but a single gap
would strand a whole subtree), and rolls subtree file counts and sizes up
bottom-up. A parent's own files are usually just cover art, so the number
worth showing on a collapsed row is the subtree's.
Storing the shape rather than recomputing it is what lets the UI open one
level at a time. Levels are still paged, because fan-out is brutal — the
widest folder measured has 1,181 children.
Filtering keeps the tree instead of falling back to a list: the search route
returns matches plus every ancestor, and the UI renders that skeleton
pre-expanded, so you see where a hit lives. Matching runs against the whole
path, so a matched folder implies its descendants match too and a matched
subtree arrives complete. The match cap is reported in the payload and shown
in the UI rather than passed off as the whole answer.
The two existing snapshots were backfilled by scripts/rebuild-soulseek-tree.ts,
which runs the same buildTree + finishSoulseekBrowse the ingest path runs — no
second implementation to drift, and no peer contact needed. Kept for the next
time the tree shape changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Probing the live routes turned up that /browse/<peer>/dirs/<id>/files only checked
the owner, not the peer: dir ids are global, so asking for one peer's folder id
under a different peer's name returned the other peer's files with a 200. The UI
always sends a matching pair so nothing misbehaved, but the URL was asserting a
relationship the query never verified — a mismatched or stale request would show
the wrong peer's contents rather than a 404.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Browsing a peer inline could never work. slskd answers GET /users/{u}/browse with
the entire tree in one blocking response — measured at 59 MB / 18k folders / 284k
files for a single real peer — and it takes minutes because it round-trips to that
peer. The browser was made to wait for that, so navigating away threw the whole
thing out and the panel showed an error more often than a tree.
So the fetch moves into the sidecar and the result into Postgres. Clicking "fetch
shares" returns 202 and the job keeps running without the tab; the UI polls the
snapshot row and reads back pages. Folders are rows and files ride along as jsonb
on their folder, because folders are what you filter and page through while files
are only ever read for the one folder you opened — a row per file would be 284k
rows per peer for no gain.
A failed or in-flight refresh deliberately leaves the previous folders in place: a
peer going offline shouldn't cost you a good cache, so the panel drives off rows
existing rather than off status. Interrupted 'pending' snapshots are failed at
sidecar boot, since the job died with the process and would otherwise spin forever.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
slskd has no favorites or buddy-list concept — 0.26.0's UsersController exposes
only endpoint/browse/directory/info/status — so Officer owns that data itself:
a soulseek_favorites table served by the slskd sidecar under a /_officer/*
namespace, which can never collide with slskd's /api/v0/*. The main server
gains exactly one line, injecting X-Officer-User on the proxy hop, so it stays
a thin auth proxy and grows no Soulseek logic. The route is handled before the
upstream check, so favorites keep working with slskd down.
Usernames in search results and downloads become a dropdown (browse shares,
toggle favorite). Browsing publishes to a nonce-stamped, consumed-once channel
so the Users section looks the peer up without re-running the expensive browse
on every remount, and favorites get their own section at the top of that panel,
which doubles as its landing content. CardHeader had to split its toggle row to
host the dropdown, since a trigger can't live inside the collapse button.
The schema file is deliberately self-contained so it can move wholesale into
the sidecar directory when sidecars start owning their own schema. Its DDL was
applied by hand, matching drizzle's constraint naming, rather than running a
whole-schema push.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add officer-slskd, a singleton sidecar that reverse-proxies to a
self-hosted slskd (Soulseek) instance and reports its loopback port to
the API on connect. All slskd knowledge (URL + API key) lives in the
sidecar; the platform is a thin auth+forward proxy for /api/slskd/* and
holds no slskd credentials. Mirrors the officer-vault pattern.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Key music_now_playing on (user_id, device) instead of user_id alone so the
browser ('web') and phone ('' default) each keep their own resume snapshot
instead of sharing one row. Sidecar /now-playing threads device (?device=,
default '') through get/set/clearNowPlaying; the web client tags its calls
?device=web. Phone unchanged → '' bucket, inherits the existing row.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Transparent pass-through fronting a self-hosted Vaultwarden so the OffVault
(Bitwarden-SDK) app reaches it through the platform's per-app origin gate. True
out-of-process sidecar (officer-vault): it owns all Vaultwarden knowledge (URL,
paths, notifications WebSocket) on a random loopback port and registers via the
sidecar connector; the platform is a thin origin-gated forwarder that knows only
the sidecar's port. Never decrypts/parses/rewrites/logs bodies.
- sidecar/vault: HTTP + notifications-WS proxy to VAULTWARDEN_URL, /_health
- api/vault: sidecar-port discovery + thin forwarder + WS pipe + origin gate
- origin: OFFICER_VAULT_ORIGIN allow-listed, scoped to /api/vault
- mounted top-level (not protected) so the Bitwarden bearer token isn't 401'd
- protocol: vault:server event; ecosystem: officer-vault app
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two problems behind "deleted/changed the folder image but it still shows" (on
both web and app):
1. Client caching. Cover/meta/etc. were served with an ETag(=v) but NO
Cache-Control, so browsers served them straight from the heuristic cache at
the same URL — a changed cover kept showing the old image. And the platform
proxy never forwarded If-None-Match, so the ETag revalidation couldn't work
anyway. Now the sidecar sends `Cache-Control: no-cache` on every version-
stamped artifact (cover/meta/poster/lyrics/image) + the manifest, and the
proxy forwards If-None-Match → the client revalidates every time and gets a
cheap 304 when unchanged, a fresh 200 when v changed.
2. Removed covers lingered. On rebuild the indexer only (over)wrote cover.jpg
when a source cover existed — a deleted or now-undecodable source left the
old cover.jpg in the cache (still served, still cover:true). Now it clears
cover.jpg first and regenerates only if there's a valid source.
Verified live against a booted sidecar: cover carries no-cache + ETag, a
matching If-None-Match → 304, and deleting the source folder.jpg drops the
cached cover (404, meta.cover cleared, manifest cover:false).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Favorites / now-playing / playlists were being served by the platform router
straight from Postgres, which violated the intended split (officer = auth +
proxy; officer-music = the whole /api/music/* contract). Move them into the
sidecar so it owns ALL music endpoints — library AND user state.
- sidecar (index.ts): serves /favorites, /now-playing, /playlists[/:id[/items]]
backed by Postgres (the same officerdb queries other sidecars already use).
The authenticated user id arrives in X-Officer-User; the sidecar is loopback-
only, so it trusts the header (401 if absent). HTTP-contract comment updated.
- platform (router.ts): reduced to a pure auth+proxy catch-all — it now injects
X-Officer-User from the authenticated ctx user and forwards the request body
(favorites/now-playing/playlist writes carry JSON) in addition to Range/query.
No schema change — the tables are unchanged, only WHERE they're served moves.
Verified live: booted the real sidecar against the live DB and exercised the
endpoints with the X-Officer-User header — 401-without-header, favorites round-
trip, and full playlist CRUD with ownership scoping all pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The watcher's localized reindex was non-recursive and only ever touched the
exact folders it was handed, so renaming/moving a *container* dir (an artist
folder, or a whole subtree) left the manifest inconsistent: the renamed-in
album children were never indexed, and the old path's album keys lingered
forever (only the nightly full reindex healed it).
reindexFolder now takes { recursive } and, for any folder, also prunes any
manifest descendant whose top-level child dir has vanished from disk. The
watcher enqueues the containing folder SHALLOW (rebuild-this-album + prune a
renamed/removed-away child) and the event path itself RECURSIVE (index a
new/renamed-in container's album children). A recursive reindex of a plain
file or leaf album stays a cheap no-op / single rebuild, and a shallow reindex
of a big container (e.g. Albums) is just a readdir + key scan — no deep walk.
Verified in an isolated temp library: a case-only artist rename prunes the old
keys and indexes the 3 renamed albums while leaving unrelated albums alone; a
leaf file edit rebuilds just that album; deleting an album folder prunes it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The default POST /reindex is incremental (skips unchanged albums by version
stamp), so it can't backfill a meta-format change like the new track `disc`
field. Add ?full=1 to run reindexFull() instead — a from-scratch rebuild into a
fresh slot, atomically swapped in (safe, never disrupts the live index). Same
30-min per-request timeout applies. The nightly 3am run still does this
automatically; this is the on-demand trigger.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ffprobe surfaces ID3 TPOS as the `disc` tag ("n" or "n/total"); parse the leading
number → IndexTrack.disc?: number in meta.json (undefined when absent/unparseable),
alongside `track`. Verified on Pink Floyd - The Wall (Disc 1 → 1, Disc 2 → 2).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Analog of the videos feature — exposes per-folder images (excluding the album
cover files) under /api/music so the app can show an "Images" section.
Indexer: IMAGE_EXT + isImage; collect loose images (minus COVER_FILES); add them
to the version signature (img: parts, so v changes when one is added/removed —
no re-sync hack needed, source files); write meta.images: [{ file }] and a
manifest images count. Folders with only images now index too.
Serving: GET /image?path=<rel>&file=<img> streams the ORIGINAL image bytes from
the library folder (image/*, ETag=<v>, 304), basename + prefix-guarded against
traversal. Documented in the contract comment.
Verified: meta.images lists loose images with the cover excluded, manifest count
correct, /image path resolution + traversal guard. App side (music-api, Images
section) is the app's to add. No CACHE_VERSION bump — the sig change reindexes
exactly the folders that have images.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rework resolveTrackLyrics so a track's OWN embedded lyrics win, and SYNCED always
beats plain:
synced-embedded > synced-sidecar(.lrc) > plain-embedded > plain-sidecar(.txt)
So an mp3 whose synced lyrics are embedded (as LRC text in the USLT/`lyrics` tag)
becomes the canonical source over leftover .lrc/.txt sidecars — but a track with
only a *plain* embed still serves a synced .lrc until it's re-embedded (no silent
downgrade). Binary SYLT frames aren't readable here, so sync must be stored as
LRC text in the text lyrics tag (verified it round-trips + is detected as lrc).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract walk()'s per-folder logic into a reusable buildFolder(), so a single
album/artist can be (re)indexed on its own. Add:
- reindexFolder(rel): rebuild just one folder's live cache entry + patch the
manifest (prune if the folder vanished). Its mtime-based signature means any
change — add / re-tag / delete — is picked up, and irrelevant touches no-op.
- withIndexLock: serialize ALL index mutations (full / incremental / localized)
so a localized reindex can never race the full reindex's atomic swap.
- watcher.ts: fs.watch(~/Music, { recursive }) → log every change → debounce 3s →
reindexFolder the affected folder(s). Verified: Bun's recursive watch fires
through the ~/Music symlink and on new-dir creation (so a new album's contents
are read by reindexFolder); inotify max_user_watches (~483k) >> folder count.
Started at boot, stopped on shutdown. The nightly full reindex backstops any
change that lands after a new folder's debounce.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Self-scheduling timer in the sidecar (fresh setTimeout each night, so it always
fires at 3am local regardless of drift) runs reindexFull() — builds into a fresh
slot and swaps atomically only on success, never disrupting the live index.
Started at boot, cleared on shutdown. Logs the next scheduled time + each run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Foundation for a safe from-scratch reindex. The live cache path is now a SYMLINK
to a slot dir; readers + incremental writes follow it.
- ensureCacheSetup() (run at sidecar boot): makes `cache` a symlink to a slot,
migrating an existing real cache dir once (a fast rename, not a copy).
- runBuild(outRoot, prev): the build now writes to a given root and returns the
manifest without touching live serving state.
- reindexNow(): incremental, in-place live build (manual + localized updates).
- reindexFull(): builds a complete index into a FRESH slot without touching the
live one, then activateSlot() swaps the symlink atomically (rename-over) ONLY
on success — a failed rebuild leaves the live index untouched; old slots pruned.
walk() takes the output root. Verified end-to-end: fresh setup, full build +
swap + prune, incremental-through-symlink, and the one-time real-dir migration
(content preserved). Restart officer-music to apply (triggers the migration).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reading the manifest (which every app refresh hits) used to call
ensureIndexFresh(), kicking off a debounced rebuild — and after the CACHE_VERSION
bump that meant a plain refresh could launch a full library rebuild. Make reads
side-effect-free: /manifest now just returns the last completed index. Builds are
explicit only (POST /reindex or the SSE stream); pick up disk changes by
reindexing.
Removes the now-unused ensureIndexFresh + lastBuildFinishedAt, and drops
/manifest from the 30-min per-request timeout extension (both hops) since it no
longer blocks on a build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A from-scratch rebuild holds the triggering request open for many minutes with
no bytes flowing, so both server hops' idle timeouts would drop it. Bun caps the
server-level idleTimeout at 255s, but server.timeout(req, seconds) allows more
per-request:
- sidecar Bun.serve: extend /reindex, /manifest, /reindex/stream to 1800s.
- platform proxy (music router): same, via the Bun server exposed as Hono's env.
Baseline idleTimeouts unchanged (sidecar 255, main server 60). The build always
completed in the background regardless; this keeps the request itself alive.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A long full rebuild logged only "resync started" then nothing until "done".
Add a console heartbeat (throttled to 3s, piggybacked on emitProgress) showing
folders/built/skipped/tracks/videos/posters/lyrics counts + the current path, so
progress is visible in the pm2 logs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The one-time v1→v2 full rebuild (and long range/SSE reads) take far longer than
Bun.serve's default 10s request idle-timeout, which dropped the triggering
request mid-flight ("request timed out after 10 seconds"). Set idleTimeout: 255
(Bun's max). A build that still outruns it completes in the background regardless
— a closed socket doesn't cancel the in-flight build promise.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The fixed ~10% frame grab could land on a black/near-black frame for clips that
fade in from black (or on a title card). Keep the ~10% seek to skip intros, but
select the frame with `thumbnail=n=300` — ffmpeg picks the most representative
frame from the batch, which avoids uniform/black frames. Verified on a
fade-from-black video: luma ~122 (vs 0 at t=0), and it dodges the fade even when
seeking from the start.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Videos/tracks indexed before posters+lyrics existed were skipped by the per-album
`v` check (unchanged v → skip), so their posters/ and lyrics/ never generated —
a plain reindex couldn't fix it.
Add CACHE_VERSION (now 2). The `v` skip is only trusted when the on-disk
manifest is already at the current format; an older version forces a one-time
FULL rebuild that regenerates every album (incl. the new posters/lyrics), then
writes version:2 so subsequent builds skip normally. Verified: old-cache rebuild
regenerates the poster, next build skips (no loop).
Deploy = restart officer-music, then one reindex (a full rebuild, slower than an
incremental — one time only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each track's lyrics are resolved and cached at cache/<rel>/lyrics/<file>.<lrc|txt>,
with the format recorded as `lyrics: 'lrc'|'txt'` on the meta.tracks entry.
Precedence: external "<base>.lrc" > external "<base>.txt" > embedded tag
(lyrics / lyrics-<lang> / unsyncedlyrics — ffprobe now reads all format tags).
Content that contains [mm:ss] lines is stored as lrc even from a .txt/embedded
source. Only track-matching sidecars affect the version signature (a stray
notes.txt is ignored). Lyrics dir is wiped+regenerated per rebuild; new
`lyricsIndexed` counter.
Served by GET /api/music/lyrics?path=<rel>&file=<track> (text/plain +
X-Lyrics-Format header, ETag=<v>, 304, 404 when none).
Verified end-to-end: external .lrc wins over embedded; embedded → plain txt;
unmatched .txt ignored. MUSIC_API.md documents the field + endpoint.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each indexed video gets a compressed poster (a frame grab ~10% in, capped at
30s, scaled ≤600px q5 like covers), written to cache/<rel>/posters/<file>.jpg
and recorded as `poster` on the meta.videos entry. The posters dir is wiped and
regenerated on each rebuild so orphans (removed videos) don't linger. New
`postersSaved` status counter.
Served by a new sidecar route GET /api/music/poster?path=<rel>&file=<video>
(image/jpeg, ETag=<v>, 304, 404 when none) — path-safe via basename.
Verified end-to-end on a real .mp4: video-only album → manifest {tracks:0,
videos:1}, meta.poster set, 14 KB poster on disk. MUSIC_API.md documents the
poster field + endpoint + postersSaved.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Videos (all phone-compatible .mp4, plus common containers) that live in an
artist or album folder are now indexed alongside audio:
- Move 'mp4' out of AUDIO_EXT into a new VIDEO_EXT (mp4/m4v/mkv/mov/webm/avi) —
it was wrongly treated as an audio track before.
- ffprobeVideo captures file/title/durationSec/width/height per video.
- meta.json gains an optional `videos: IndexVideo[]`; a folder with only videos
now still gets a meta.json. Manifest entries gain optional `videos: N`.
- Video files join the album version signature (changes bump `v` for resync).
- New `videosIndexed` status counter + resync-log line.
Location is inherent in the folder rel (always an artist/album dir), so no
extra location field is needed. MUSIC_API.md documents the videos field +
manifest count. No poster/thumbnail generation yet (folder cover is reused).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 1 gap surfaced by a long overnight job: armIdle fired 30 min after the
last turn regardless of in-flight background work, so a silent run_in_background
job outliving the timeout got its persistent SDK session aborted — killing the
harness that delivers its task_notification (and any detached watcher's hook).
Fix = task-lifecycle heartbeat: track task:started → task:notification per
session; suppress/re-arm the idle timer while any task is pending. task:started
also clears a pending idle timer. So long run_in_background jobs keep their own
session alive and their completion is delivered; idle-GC resumes only once all
tasks finish and the session is truly idle. Pairs with the Activity path-tail
(35973a5) for the pure-setsid case.
Not deployed (no restart — long job still running); lands with the Activity
batch on next restart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root fix for orphaned background tasks: the platform drove Claude Code as a
one-shot `claude -p` per turn (stdin ignored, process exits at turn end), so
run_in_background/Monitor work — and its task_notification — had no live harness
to return to. Now each chat session runs ONE long-lived Agent SDK query() with
streaming input; turns are user messages pushed onto it, and the session stays
warm between turns.
- claude-manager: persistent `query({ prompt: AsyncIterable, options })` per
sessionKey (bypassPermissions, --resume, mcp via extraArgs, CLAUDECODE stripped).
Single consumer loop maps every SDK message → ChatEvent, incl. post-turn
task_started / task_notification. interrupt() = stop-turn; abort() = kill-session;
30-min idle GC.
- stream-parser: processMessage() (object-level, reused by the SDK loop) + task
message handling. ChatEvent/ServerMessage gain task:started / task:notification.
- API: the sidecar event subscription is now SESSION-scoped (no longer unsubscribes
on 'result'), so background events after turn-end still reach the client. First
turn opens the session; later turns push onto it. handleStop → interrupt (keeps
session warm); disconnect/deleteSession → kill.
- protocol/sidecar-registry/user-instance: claude:interrupt command + interruptClaude.
- client: render task:started / task:notification in the transcript.
Verified end-to-end through the real chat WS: a run_in_background task's completion
arrives ~6s AFTER the turn's result; multi-turn on one warm session works.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- POST /reindex now runs the build to completion before responding (via a
coalescing reindexNow), and GET /manifest ensures a fresh (debounced 3s)
index first — so on-disk changes show up on a plain app refresh, not only
via the explicit reindex sheet.
- Log each resync in the officer-music sidecar (start + one-line summary,
or a failure line).
- Fix albums whose cover file isn't a decodable image (junk .jpg): the
manifest cover flag and the skip check now reflect whether a cover was
actually cached, so they settle to cover:false instead of rebuilding every
run (and the app no longer 404s fetching a cover that was never there).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each Albums/<Artist>/_discography.md (author-maintained source of truth, never
modified) is compiled into a per-artist discography.json in the cache = album
folder → normalized release type (Studio/Live/Compilation/Single/EP/…), so the
player can split an artist's album list into sections.
- indexer.ts: parse the md table, normalize the Type (EP?→EP, Compilation (VA)→
Compilation, …), write discography.json. The artist folder's `v` now includes
_discography.md so regenerating it re-syncs just that small JSON (isolated from
the albums' meta/cover). Manifest gains `disco: true` on such entries. Also
fixed the skip check to require all expected outputs to exist, so artist/
cover-only folders no longer rebuild every run. New `discographies` counter.
- sidecar: GET /discography?path=<artist rel> (ETag/304), documented in the
contract header.
- MUSIC_API.md: §2.4 + manifest disco flag + resync algorithm updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
-scale 0.5 was hardcoded on the assumption that :0 is 4K. With no monitor
plugged in X falls back to something tiny — 800x480 on this box — and halving
that served an unreadable 400x240.
Read the framebuffer width from xrandr and scale only above 2560px. When the
width cannot be read, serve 1:1: too many pixels beats a thumbnail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The desktop page has never worked on a fresh install. Two faults, both fatal.
The password could never be created. DesktopView fetches /desktop/vnc-password
before opening the WebSocket, but ensureVncPassword ran only from startSession,
which only the WebSocket triggers — so the endpoint answered "not configured",
the UI stopped, and the socket that would have provisioned it was never opened.
A new vnc:ensure-password sidecar command provisions it directly; the endpoint
asks for it instead of returning 500.
The rfbauth file could never be written either. ensureVncPassword shelled out to
tigervnc's `vncpasswd -f`, which is not installed — and, contrary to the comment
in setup-desktop.sh, is not in tigervnc-common, which ships only tigervncconfig.
The failure was swallowed because only a zero exit wrote the file, so x11vnc got
-rfbauth pointing at nothing. x11vnc writes that format itself with -storepasswd,
so the dependency is gone and a failure now throws.
Verified on the box: the endpoint returns a password, .vnc/{passwd,password} are
written 0600, and the sidecar reports mirroring :0 on 5900 with x11vnc using the
generated rfbauth file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The proxy is an opaque catch-all, so the endpoint surface wasn't perceivable from
the platform side. Add a contract header (all routes + params + SSE/response
shapes) atop the sidecar fetch handler where the routes are defined, and point
the proxy router at it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a live progress channel for the library index:
- indexer.ts: progress subscribers (onIndexProgress) + throttled emit during the
walk, and buildReport() for a final summary.
- sidecar: GET /reindex/stream (SSE) — triggers a build if idle (?trigger=0 to
watch only), streams `progress` events, ends with a `done` event carrying the
report; auto-proxied at /api/music/reindex/stream for the app. Sidecar also
writes DATA_PATH/music/.server (its port) for local tooling.
- scripts/reindex-music.ts: CLI that reads the port file, follows the SSE, prints
live progress + a final report. Run: bun scripts/reindex-music.ts
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The officer-music sidecar now builds a cache tree mirroring the library (server
counterpart of the app's music-index.ts), and exposes an rsync-clean diff surface.
Indexer (indexer.ts): walks HOME_DIR/Music; per album computes a version `v` =
hash of the source signature (track name+size+mtime, cover size+mtime); ffprobe
→ meta.json (phone IndexMeta schema: file/title/artist/albumArtist/album/track/
year/durationSec); ffmpeg compresses the cover to <=600px q5 cover.jpg. Writes
DATA_PATH/music/cache/<rel>/. Incremental (skip albums whose `v` is unchanged),
prunes cache dirs for albums removed from the library, maintains manifest.json.
Endpoints (sidecar, auto-proxied by /api/music/*):
POST /reindex async build; GET /reindex/status polls progress
GET /manifest { version, albums: { "<rel>": { v, cover, tracks } } }
GET /meta?path=<rel> album meta.json (ETag: v, 304 on If-None-Match)
GET /cover?path=<rel> compressed cover (ETag: v, 304 on If-None-Match)
Phone resync: GET /manifest, diff `v` against last-stored → fetch only changed
albums' meta+cover; drop rels missing from the manifest. No re-download of
unchanged albums.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an officer-music sidecar that owns an audio-streaming HTTP server, and a
thin authenticating proxy on the platform. All processing (path resolution,
byte-range streaming, ffprobe duration) is in the sidecar; the platform only
authenticates and forwards.
App-facing contract (handoff):
GET /api/music/stream?path=<home-relative path>&token=<jwt>
- auth via userMiddleware (Bearer or ?token= for media elements)
- 200 full / 206 on Range, with Accept-Ranges, Content-Length,
Content-Range, Content-Type, and X-Audio-Duration (seconds, ffprobe)
- path resolved within HOME_DIR, traversal-guarded (400); 404 if missing
Purpose: stream + seek without pre-downloading the whole file — the app can
read X-Audio-Duration instead of scanning for VBR duration.
Pieces:
- sidecar/music/{index.ts,stream-audio.ts}: Bun.serve on a random port, /stream
+ /health, duration cached by path+mtime; reports its port via a new
music:server sidecar event on connect.
- api/music/{sidecar-server.ts,router.ts}: capture the port; reverse-proxy
/api/music/* → sidecar, streaming status + headers through.
- protocol.ts music:server event; hono.ts mounts /api/music; ecosystem adds
officer-music.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Officer is single-user: the server owner is the only account, created once by
/auth/bootstrap. Everything that existed to serve additional users was
unreachable, so it is gone rather than left looking like it does something.
Accounts: drop the invite / resend-invite / delete / list-users routes and the
Users settings screen, the inert /auth/signup handler, and the account
verification chain it fed (verify, resend-verification, VerifyScreen, the
UserInvite + VerifyAdmin + VerifyRegistration templates). /auth/verify-token
survives for password resets only, and now requires a reset-password token
rather than accepting any signed JWT.
Roles: drop the users.role column and the four-value USER_ROLES enum. The
permissions table granted every role identical methods, and every
role === 'Super Admin' check was permanently true. The JWT no longer carries a
role claim.
Sandbox: remove sidecar/sandbox.ts and its five call sites. bwrap was selected
only for non-Super-Admin users, so it never ran. It was also not a usable agent
jail as written — --share-net, the project root (with .env) bound read-only,
and runuser dropping to the server's own uid. Rebuilding it for agent
containment would be a different construction, and git history keeps this one.
getHomeDir keeps its DATA_PATH meaning; the new getOwnerHomeDir resolves the
owner's real login home, which is what terminals, chats and task runs use.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
setup-desktop.sh now installs ubuntu-desktop + gdm3 + x11vnc and forces the
Xorg session (WaylandEnable=false) with auto-login — x11vnc can only mirror an
Xorg :0, not Wayland. vnc-manager.ts resolves the X authority from the GDM
per-session path (/run/user/<uid>/gdm/Xauthority) with a ~/.Xauthority fallback.
setup.sh fixes:
- desktop step gates on `dpkg -s ubuntu-desktop` (was the decommissioned
officer-vnc service, which never matched so setup-desktop re-ran every time)
- remove Pi (install, --list-models validation, verification check)
- export GOPATH before the cliamp build so `go install` lands where it's checked
even when Go was already present this run
- write PUBLIC_BUILD_ENV=production and quote all .env values
- guard the interactive .env block behind a TTY check so non-interactive runs
skip cleanly instead of aborting on read EOF under set -e
- restart systemd-logind only when a key actually changed
- sed prefix-strip instead of `tr -d` (which deletes characters, not a prefix)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Turns now run in the sidecar via `opencode run --dir <cwd> --format json
--dangerously-skip-permissions [-s <ses_>]` instead of the serve's
`POST /session/{id}/message` path. That path was unreliable at reporting
tool completion — tools finished but the turn stayed status=running,
wedging the UI at "Working…". `run` re-anchors tools to the chat cwd via
--dir, reports completion faithfully, and exits when done.
- runner.ts (new): spawn `run`, map its JSON events (text/tool_use/
step_finish) to ChatEvent, report the `ses_` id for resume, accumulate
cost; inactivity (120s) + hard-cap (10min) watchdogs kill a hung turn
and emit a clean error instead of hanging forever.
- protocol.ts: opencode:run-streaming/kill commands; opencode:spawned/
event/session events; OpenCodeRunParams.
- sidecar index.ts: wire run/kill; sweepStaleServes() on startup kills
only an `opencode serve` whose resolved /proc/<pid>/cwd == SERVE_CWD,
so an unclean prior exit can't leave two.
- sidecar-registry.ts: spawnOpenCodeStreaming/killOpenCode/onOpenCodeEvent/
onOpenCodeSession helpers.
- send-opencode.ts: rewritten to mirror send-claude-code (subscribe →
resolve resume id → spawn → kill handle).
- sidecar-server.ts: persist reported ses_ id into state for resume.
- list-models/server-manager: route to the sidecar's reported serve URL.
The serve stays up only for read-only calls that never hung (model
listing, session history).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New officer-opencode sidecar (same philosophy as officer-claude): a singleton that owns
an `opencode serve` running from DATA_PATH/opencode-sidecar (created if missing) on a
random port, registers with the API as capability 'opencode', and reports its port via a
new `opencode:server` protocol event. The API stores it (sidecar-server.ts, wired in
server.tsx via getOpenCodeServerUrl). ecosystem.config.cjs runs the sidecar instead of a
bare pm2 serve. Turn-running + API rewiring come in later steps.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The default /chat working directory is used by both the Claude and OpenCode harnesses
now, so its Claude-specific name was misleading.
- Rename the dir + accessors: getClaudeSessionsCwd → getGeneralChatSessionsCwd,
ensureClaudeSessionsCwd → ensureGeneralChatSessionsCwd, path segment claude_sessions
→ general_chat_sessions (data-path on disk + code + UI labels/comments). No history
migration — the old Claude transcript slug is orphaned (intentionally).
- Remove the vestigial chat_sessions dir (leftover from the retired session store):
it only ever held empty claude/archived/ dirs, recreated by a signin hook. Drop that
hook (+ its dead imports) and the 4 unused data-path accessors (getUserSessionsDir,
getClaudeDir, getSessionDir, getArchivedSessionDir), and delete the dir.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
