officer's registration socket silently drops sends when it isn't OPEN
(sidecar/connect.ts:send — no queue, no error, no return value). the agent pushed
raw parser events over that socket and officer translated and persisted them, so
everything a turn produced while officer was restarting went nowhere: the turn kept
running, the output was gone, and a reconnecting client replayed a log that simply
had no rows for those seconds. stage 1 kept the agent alive across a restart; this
is what makes its output survive one too.
move the translation and the write into the sidecar:
- turn-stream.ts is the stateful ChatEvent -> browser-message translator lifted out
of websocket.ts (delta buffering, flush before tool:start and result). pure and
synchronous, so it is unit tested — 12 tests, 100% lines.
- session-log.ts commits each message to chat_session_events and only then hands it
to officer, with its cursor id attached. per-session promise chain: translation is
synchronous and therefore in arrival order, and only the commit is queued, so
cursor ids are assigned in the order events actually happened. a delta that
overtook the assistant:text in front of it would make the client commit its stream
buffer at the wrong point, so deltas go through the same queue even though they are
never written.
- claude:event on the wire becomes claude:message: a finished browser-facing message
plus its seq. officer relays it verbatim and folds it into the in-memory session
for sync:messages. it no longer builds or persists chat messages for this harness.
gap detection, which is what the durable log is for. chat_session_events.id is a
global bigserial, so two consecutive events of one session are not consecutive ids
and a client cannot tell a contiguous replay from one with a hole in it. each durable
message now carries prevSeq — the cursor of the previous message in the same session —
which is inside the persisted payload, so it survives replay. useChat compares it
against the cursor it holds before advancing, and surfaces a visible marker on a
mismatch: a conversation that silently skips a tool call or half an answer reads as
the assistant having done something inexplicable. only checked once a cursor exists,
because opening a session from history legitimately starts mid-chain (events are swept
after 7 days, the transcript is not).
a failed write delivers live with no seq, so the client sees the message but does not
advance past something it cannot replay, and the next successful write chains from the
cursor the client still holds.
pipeline steps pass durable: false. their sessionKey is a throwaway uuid no browser
will ever replay and the job's own event log is its record, so writing those rows only
grows the table.
opencode still goes through officer's createEventHandler, now labelled as such. that
is the sidecars-opencode branch.
this fixes R4 from CLAUDE_SIDECAR_ISOLATION.md. R3 and R5 already worked.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
move video/audio downloads off the dedicated ReClip download lane onto the
generic script-job path:
- delete execute-download.ts, reclip-client.ts and the POST /jobs/download
endpoint; drop the 'download' mode from the pipeline_jobs enum (legacy rows
tolerated)
- execute-script.ts: strip the @@officer:progress@@ sentinel from the log,
emit progress events, and isolate viewer/log writes (safeEmit/safeLog) so a
broadcast or log throw can't wedge the stdout pump
- pipeline-job-manager.ts: persist latest progress; guard sendToViewer sends
- ScriptJobDetail: render the two progress bars; DownloadJobDetail kept for
legacy history rows
- TaskRunnerModal: ScriptRunner descends into the triggered directory
- VideoDownloadPanel: rewire startJob to the script-job path
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A failed download has no media file, so its description was lost — no way to tell
which video was missed. Now a failure writes a `<title>.txt` (or `<videoId>.txt`
when untitled) holding the URL + description, so every missed item leaves a
recoverable reference. Always written (even with an empty description — the URL is
the reference); skipped only on a deliberate Stop, not a genuine error.
Verified: successful item → "<media base>.txt" (description); failed item →
"<title>.txt" (url + description); spaces preserved in both.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The metadata phase already fetches ReClip's description (ReClip now forwards it in
/api/info). Carry it into the download phase and write it to a text file with the
same base name as the media — "Song Name.mp3" → "Song Name.txt".
reclipDownloadOne gains an onFilename callback that fires the moment the final
filename is known (before the file transfers), so the executor writes the sidecar
in parallel with the download stream, and the exact name guarantees they pair up.
Empty descriptions write nothing; the write is best-effort (never fails a download).
Verified: correct base name + .txt, exact content, and no sidecar for an empty
description.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verified against live ReClip: POST /api/download with title:"" → a hash filename
(b5d04adc86.mp3); with title:"Me at the zoo" → "Me at the zoo.mp3". So ReClip
names the file from the title WE send (falling back to a hash) — it does not
self-name. The title is mandatory, which means a metadata pass is required.
Back to two phases:
1. metadata — fetch each item's /api/info (title + validity), keep survivors, skip
errors.
2. download — download each survivor passing its title, so files land with real
names; skip download errors.
Keeps the exact-urls[] input (Mix playlists can't drift) and the one-request-per-
item download. Progress is two counters again (Titles + Download); UI shows two
bars. ~2 requests/item is inherent to needing the title (per the user's call:
correctness over speed).
Verified two-phase filtering + title passthrough + skip-on-error with a mock.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A YouTube Mix/radio playlist (list=RD…) returns a different set of items on every
/api/playlist call (observed 779 / 1485 / 529 for the same URL). The job used to
re-expand the playlist server-side, so it would download a different list than the
count shown on the decision screen.
The panel now passes the already-expanded `urls[]` into the job, and the executor
uses them verbatim (falling back to expanding `url` only when no list is given).
The job downloads exactly what you decided on. Endpoint takes `urls[]` (stored as
inputs.urls) or `url`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The job's two phases were a misread — the "count" phase is the client-side
playlist expansion (for the inline-vs-job decision, already done in the panel).
The job itself is just one download request per item.
Dropped the in-job metadata pass entirely:
- reclip-client: reclipDownloadOne no longer prefetches /api/info for a title —
ReClip names the file from the video title itself, so it's a single request
per item.
- execute-download: one phase — expand the playlist, then /api/download each url,
skip failures. Progress is a single { done, failed, total, current } counter
(no meta/dl split); ~2× faster and downloads start right after expansion.
- UI (DownloadJobDetail + panel JobView): one "Downloaded" bar instead of two.
Verified: every item is attempted directly (no /api/info gate), skip-on-error
counts correct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Front half of the download-job feature.
Panel (VideoDownloadPanel): after Fetch expands a playlist and the count is known,
a decision screen — "Found N items" → pick Audio/Video + subfolder → "Download all
as a job" (POST /jobs/download), or "fetch inline to pick individually" (the
existing card grid). A single video still goes straight to the inline card. The
job phase shows live two-phase progress (polled from the job) + a "View in Jobs"
link; it notes the job runs server-side so closing the panel is fine, and it
refreshes the browser as each file lands.
/jobs (DownloadJobDetail + JobsPage dispatch): a `download` job renders a compact
two-phase readout — Metadata and Download bars (processed/total, found/skipped and
saved/failed) + the current item — polled from the job's progress, with a Stop.
Executor tweak: phase-1 meta.done now counts kept (not processed) so both phases
read the same `(done+failed)/total`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Turns the downloader into a server-side job on the existing jobs spine (Postgres
persistence, live WS viewers + replay, abort, /jobs UI) — but with its own
executor and its own lane, since it's deterministic scripting, not an agent, and
a multi-hour playlist mustn't block agentic jobs.
- reclip-client.ts (new, shared): reclipInfo / reclipPlaylist / reclipDownloadOne
(single download → streams the file to a dir, abort-aware). Extracted so both
the file-browser endpoints and the job executor use one client.
- execute-download.ts (new): the two-phase executor —
phase 1 metadata (expand playlist, fetch each info, keep survivors, skip
errors), phase 2 download (each survivor in the chosen format; skip download
errors). Emits a compact `download:progress` snapshot (counters, not per-item
events — playlists are thousands of items). Throws on abort / fatal.
- job manager: `download` mode dispatch → executeDownload; persists
download:progress; adds execution LANES (download vs default) so the two run
independently and each serializes on its own; promoteNext fills both lanes.
- POST /api/tasks/jobs/download { url, format, dir, root?, label? } — enqueues a
download job (own lane, no capability task needed; traversal-guarded target).
- schema: `download` added to the mode enum (drizzle text-enum — no DB migration);
getPendingJobs() query for lane filling.
Verified the executor with a mocked ReClip client: two-phase filtering, skip-on-
error counts, and abort-throws all correct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three changes that Transcribe Audio needs.
Whisper's translate mode only ever outputs English, so it cannot honour
"translate into <language>" for anything else — it answered Portuguese audio with
a rough English rendering instead of a transcript. The translate decision is gone
and transcription is always faithful to the detected language; spokenLanguages
now only breaks ties on clips Whisper is unsure about.
Script tasks get OFFICER_API_URL / OFFICER_API_HOST / OFFICER_AUTH_TOKEN so they
can call Officer's own endpoints rather than reimplementing server-side work.
Requests go to 127.0.0.1 so nothing depends on DNS or the proxy, but origin
validation matches Host against PUBLIC_URL, hence the separate host variable.
`inline` accepts "ask", which offers both affordances in the runner — Run here
streams into the modal, Run as job queues it. Useful when the same task can take
a second or an hour depending on whether it was pointed at a file or a library.
Existing true/false values behave exactly as before.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CATEGORY_ORDER hardcoded Video/Audio/Cleanup in the frontend, so adding a
category meant a code change. The order now lives in categories.yaml at the root
of the items store and reaches the client via GET /tasks/categories — the
platform no longer knows any category by name.
The endpoint is declared before /:name, which would otherwise match
"categories". Categories used by a task but absent from the file still work: they
sort alphabetically after the listed ones, and Other stays last.
Menus consume grouped tasks rather than grouping them per row. Groups are built
from the tasks and the file only ranks them, so a category listed with no
matching tasks cannot produce an empty submenu — locked in by tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every task declares a directory trigger, so right-clicking any folder listed all
fifteen at once — Tag Album offered on a folder of photos. TASK.md gains an
optional `category`, and the context menu nests by it: Run Task > Video > …
Nesting only kicks in when more than one category matches. A .mp4 matches eight
tasks that are all Video, so file menus stay flat rather than gaining a pointless
hop. Known categories lead (Video, Audio, Cleanup); anything else follows
alphabetically with Other last.
Applied to both menus — the right-click one and the ⋮ dropdown — which carried
identical blocks.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Officer is single-user: the server owner is the only account, created once by
/auth/bootstrap. Everything that existed to serve additional users was
unreachable, so it is gone rather than left looking like it does something.
Accounts: drop the invite / resend-invite / delete / list-users routes and the
Users settings screen, the inert /auth/signup handler, and the account
verification chain it fed (verify, resend-verification, VerifyScreen, the
UserInvite + VerifyAdmin + VerifyRegistration templates). /auth/verify-token
survives for password resets only, and now requires a reset-password token
rather than accepting any signed JWT.
Roles: drop the users.role column and the four-value USER_ROLES enum. The
permissions table granted every role identical methods, and every
role === 'Super Admin' check was permanently true. The JWT no longer carries a
role claim.
Sandbox: remove sidecar/sandbox.ts and its five call sites. bwrap was selected
only for non-Super-Admin users, so it never ran. It was also not a usable agent
jail as written — --share-net, the project root (with .env) bound read-only,
and runuser dropping to the server's own uid. Rebuilding it for agent
containment would be a different construction, and git history keeps this one.
getHomeDir keeps its DATA_PATH meaning; the new getOwnerHomeDir resolves the
owner's real login home, which is what terminals, chats and task runs use.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stage 1 of removing Pi (Claude-only). Cuts the non-Claude branches in the chat
WS handler, pipeline executor, and channel send-and-await; deletes the Pi
sidecar, its ecosystem entry, pi-bridge, and the Pi model-listing spawn (now a
static Claude tier list). Adds a guard coercing any legacy non-claude-code model
preference to the Claude default so old settings don't break chat or jobs.
Removes the dead no-op session-save REST route and stale Pi docs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
a script task that goes silent for a while (e.g. ffmpeg's faststart pass rewrites
a huge file for minutes with no output) would hit Bun's default 120s websocket
idle timeout → close(ws) → killTree killed the task mid-run, corrupting the
output. now the executor pings the socket every 30s while a task runs, resetting
the idle timer.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
proc.kill() only signalled the direct child (bash), leaving a running ffmpeg
grandchild orphaned and still encoding. now killTree() walks /proc, SIGTERMs the
whole subtree (bash + ffmpeg), then SIGKILLs any straggler after 2s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the marketplace service dependency and the native/global/user
scope tiers with a single external directory ($OFFICER_ITEMS_DIR) holding
skills, tools, tasks, processes and extensions as plain files.
- tasks move from Postgres to TASK.md files (new file-backed task layer);
task editing now works, which the DB path never supported
- skills/tools/processes collapse into one shared file router (single dir)
- remove the marketplace client (sync-marketplace/sync-version) and the
boot-time sync; pi-bridge/pi-manager/sandbox point at the flat store
- drop the dead tasks + vestigial skills/tools/processes/extensions +
item_chats tables (migration 0004)
- one-time migration script exports DB tasks and consolidates disk items
Migration verified: all 6 tasks round-trip through the runtime parser
identically to their DB rows (pipeline steps, triggers, script impls and
agentic bodies all intact).
NOTE: not yet functionally tested end-to-end — every item (each task mode,
tool, skill, extension) still needs to be run/exercised in the app before
this is trusted. To be done manually.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
the file browser sends msg.cwd relative to the user's home and it went straight to
Bun.spawn, which resolves it against the server's cwd. when that directory does not
exist posix_spawn reports ENOENT naming the binary rather than the directory, so
"Convert Audio" failed with the misleading "posix_spawn 'bash'".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Agentic steps now have a 5-minute inactivity timeout (non-Claude-Code
models only) so stalled local LLM requests fail with a clear error
instead of hanging forever. A "Waiting for model" indicator is emitted
every 10s and shown in the pipeline UI stats bar, clearing as soon as
the model starts responding.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix stop: abort poll now rejects the promise after killing the agent process
- Fix jobs link: close modal before navigating, use react-router navigate
- Fix seed parser: strip quotes from step input values (delete_source: "true")
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Pipeline jobs now persist to DB with progress tracking and cost accumulation
- Jobs survive WebSocket disconnects with in-memory event buffer replay
- New /jobs list page with search, status badges, and cost display
- New /jobs/:id detail page with live WebSocket attachment and REST fallback
- Two-column layout using WorkspaceLayout for resizable steps/output panels
- Streaming messages tagged with stepIndex/iterationLabel for per-step output grouping
- TaskRunnerModal links to job detail page once job is created
- Dock entry added for Jobs page
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Pipeline mode: new task mode that chains agentic tasks sequentially with
foreach/subdirectory iteration and skip_if conditions
- Pipeline executor backend (WebSocket at /api/tasks/pipeline/ws) with
support for both Pi and Claude Code models
- Frontend PipelineRunner component with step progress, streaming output,
and aggregate cost tracking
- New agentic tasks: prepare-discography, fetch-album-info, build-discography
(pipeline combining both)
- Seed parser extended to handle pipeline steps in frontmatter config
- CopyButton component added to assistant bubbles, error bubbles, and
tool input/output sections
- Removed obsolete SearXNG/Apify/browser relay code from pi-bridge
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tasks now live in the database (mode: script or agentic). Script-mode tasks
bypass the agent entirely — the implementation is materialized to a temp file
and executed directly, with stdout/stderr streamed to the UI via WebSocket.
Includes convert-to-mp3 as the first native script task.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- apify tool: TOOL.md definition, index.ts implementation with auto-auth via OFFICER_APIFY_TOKEN, output_path for large datasets
- tools API: /tools routes (list, detail, chat, create, delete) mirroring tasks pattern
- automation UI: tools tab in sidebar, NewTool component, tool detail view
- apify integration: settings page for enterprise API key config, pi-bridge passes env var to containers
- tiktok-trends task: rewritten as agent instructions using apify tool with output_path, scripted report generation for 50KB read limit
- restrict edit/delete of native/global capabilities to Super Admin only (backend + frontend)
- tools authoring guide: TOOLS.md with full spec for TOOL.md frontmatter, index.ts execute signature, patterns
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
