officerdev/platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
cca85a7524cf2b85c3bbb7dacf737951df461eee
platform/docs/jobs-unification.md
T
pastilhas 027b10bd6e step 4/4: the docs say permissions too, and capability means one thing again 
44 files of prose — CLAUDE.md, AGENTS.md, TODO.md, 20 docs, both plugin design
documents, and the comment surface the earlier steps could not reach.

Applied against an explicit keep-list, not swept, because the word turned out to
have SIX meanings in this repository rather than the three the offscale doc
recorded:

  permissions          renamed (steps 1–2)
  $OFFICER_ROOT/capabilities/  KEPT — the item store, and now the only thing
                               the word means that is ours
  sidecar routing keys renamed to `handles` (step 3)
  Lightning wallet     KEPT — a domain term, and on the wire to the mobile apps
  terminfo queries     KEPT — XTGETTCAP, in the pty sidecar
  InvoiceShelf         KEPT — per-resource { write, bulkDelete } flags

The sweep still falsified two things, both caught by checking rather than by
review, and both in prose that discusses more than one meaning at once:

CLAUDE.md began claiming the item store lives at `$OFFICER_ROOT/permissions`.
It does not; that directory is on disk and full of skills and tools.

And the offscale doc's own note about the collision became
"Named `permissions`, NOT `permissions`" — a sentence that had eaten the thing
it existed to warn about.

Both restored, and the note rewritten to say what is now true: capability means
one thing of ours, and three that belong to somebody else's vocabulary.

Verified live after restart: self and admin permission endpoints 200, gated
route 200, agent-status 200, 9 grants intact with 6 permissions offered.
tsgo clean, 797 tests, 787 pass, same 7.

The rename is done. Four steps, no data lost, no client break that survived
the step it was introduced in.
2026-08-15 16:31:11 +00:00

7.5 KiB
Raw Blame History

Jobs Unification — making every task a background job

Status: shipped, except push notifications. Phases 1–3 (data model, executor, queue, REST API and the master-detail /jobs screen) are done and live. Phase 4 is the only open item, and is the reason this file still exists. Email sync is deliberately NOT part of this any more — it moved into the email sidecar with its own scheduling, so it does not appear in the Jobs list.

Goal: every task run (script, pipeline, later agentic) becomes a persisted, background job — created over REST, streamed live over WebSocket, resumable/attachable, visible on desktop and phone, and ending in a push notification. Replaces today's ephemeral script-task WebSocket path.

Context: jobs belong to the owner. Not because the platform is single-user — it stopped being that on 2026-08-07 — but because tasks is an execution permission: running a job means running a script as the owner's OS user, so it can never be granted to a member. "Is anything running?" is therefore still a global check, and the conclusion below is unchanged even though the premise was rewritten. Favor power-user affordances over guardrails.

Current state (baseline)

  • Pipeline tasks already are jobs: pipeline_jobs (Postgres) + pipeline-job-manager (in-memory live registry, viewer attach/replay buffer, 3s progress flush, finalize, restart-reconciliation) + /jobs list (JobsScreen) + /jobs/:id detail (JobDetail, live-attach or persisted). WS at /api/tasks/pipeline/ws.
  • Script tasks (convert-video, edit-audio, …) run via task-executor.ts over /api/tasks/run/ws — ephemeral, zero persistence, dies with the tab. Modal TaskRunnerModal (ScriptRunner branch).
  • A separate file-backed queue engine exists at src/servers/queue/ (lane-serial, retries, crash-resume) wired only to email sync. NOT reused here — one Postgres jobs model instead.

Decisions (locked)

  • Concurrency: idle → single Run → /jobs/:id. Something running → Queue + a red Run (force concurrent). Run → start now → /jobs/:id. Queue → append pending → /jobs list. Backend (global): FIFO pending queue; on any finalize with 0 running, promote the oldest pending. Force-Run bypasses the queue. Queue backlog survives restart (promoteNext on startup).
  • Modal → page: replace TaskRunnerModal with a /jobs/new route. Right-click task action navigates there instead of mounting the modal. One code path, phone-friendly.
  • Deep-link: query params, not hash — /jobs/new?task=convert-video&root=home&path=<dir>&entry=<name>.
  • Phone: already has a file browser, so it can pick a target path and POST a job. No blocker.
  • inline tasks stay in the modal. A top-level inline: true in TASK.md marks quick/interactive tasks that run in the modal (ephemeral) instead of as a job. Default = job. Currently flagged: remove-jelly-meta, analyze-video, clean-playlist-files. So the modal is not retired — it's the inline path; the shared input form serves both the modal and /jobs/new. The flag is plumbed through the parser + task list/detail endpoints (task.inline), so the FileBrowser menu can branch.

Plan

Phase 1 — Unified jobs backend

  • 1a. Data model. Add mode (pipeline|script|agentic, default pipeline) + exit_code (int) to the jobs table. Log at DATA_PATH/jobs/<id>.log (derived from id). Table/symbol rename pipeline_jobs→jobs is deferred as a cosmetic cleanup — add columns first, keep it working.
  • 1b. Execution. Generalize the job manager: startJob takes mode and dispatches — pipeline → existing executePipeline; script → new executeScript (ports task-executor's materializeScript/buildInputEnv/bwrap sandbox/killTree/keepalive, but emits job events + appends stdout/stderr to the log file, finalizes with exit_code). Reuses the manager's viewer/broadcast/replay machinery.
  • 1c. Scheduler. runningCount() + FIFO pending promotion on finalize; promoteNext() on startup so a queued backlog resumes.

Phase 2 — REST job API (decouples creation from the socket; enables the phone)

  • POST /jobs {taskDirName, inputs, cwd, action} → {jobId} (create + start/queue, background).
  • GET /jobs (+?live=1), GET /jobs/:id, GET /jobs/:id/log?offset=, POST /jobs/:id/stop.
  • Consolidate the two WebSockets into one /api/tasks/jobs/ws doing only attach/stop/list.

Phase 3 — Frontend

  • /jobs/new → NewJobScreen: reads query params, renders the input UI lifted from TaskRunnerModal (TaskInputForm + per-group config + folder probing). Run/Queue per the concurrency UX. JobDetail gains a script branch (terminal output: live attach, or from log when idle). Retire TaskRunnerModal/TaskRunnerDialog/useTaskRunner. Header running-jobs indicator.

Phase 4 — Notifications (later)

  • One notifyJobDone(job) hook at finalize → push to the phone app.

Progress

  • 1a data model — mode + exit_code columns (schema + applied to DB)
  • 1b executeScript + manager dispatch — execute-script.ts (spawn/sandbox/killTree port, log file, abort poll, returns exitCode), process-tree.ts (shared killTree), pipeline-job-manager now dispatches by mode and finalizes script jobs by exit code. Compiles; runtime-untested until a REST caller + restart exist.
  • 1c scheduler / queue — enqueueJob(action) (start now / queue behind running), promoteNext() on finalize + startup, getOldestPendingJob, markInterruptedJobs now running-only (pending queue survives restart). startJob kept as a enqueueJob(...,'start') wrapper.
  • 2 REST job API — POST /jobs (create script|pipeline, action start/queue), GET /jobs (+?live=1, now returns mode/exitCode/isLive), GET /jobs/:id, GET /jobs/:id/log?offset=, POST /jobs/:id/stop. Router mounted at /jobs and /pipeline-jobs. Needs a restart to deploy; then curl/phone-testable. WS consolidation still pending (old /api/tasks/run/ws + /api/tasks/pipeline/ws still live).
  • 3 frontend — master-detail /jobs, modal-as-creator, split list, header badges. Done.
    • 3a jobs UI — master-detail JobsPage (like /chat): WorkspaceLayout with a list panel (left, polls GET /jobs, highlights active) + a detail panel (right) that branches by mode — ScriptJobDetail terminal (polls log + status, Stop) or PipelineJobDetail. One page serves both /jobs and /jobs/:id; list click navigates. Replaced the old separate list/detail pages.
    • inline flag plumbed (parser + list/detail endpoints); 3 quick tasks flagged.
    • 3b/3c task→job via the modal (pragmatic reuse). The task modal is now the job creator: on Run, an inline task runs ephemerally in-modal; a non-inline task POST /jobs (start) → navigates to /jobs/:id. When a job is already running, a red "Run now" + a "Queue" button (queue → /jobs). Reuses the modal's per-group input UI in place — no separate /jobs/new page or FileBrowser change needed. (A standalone deep-linkable /jobs/new is deferred; the phone creates jobs directly via POST /jobs.)
    • 3d header job indicators — JobsIndicator (two always-present badges next to RescanButton + UserMenu): running (→ running job's /jobs/:id) + queued (→ /jobs), polling GET /jobs/counts → { running, runningJobId, queued } every 3s; dim at 0.
  • 4 push notifications
Reference in New Issue View Git Blame Copy Permalink