pastilhas
027b10bd6e
44 files of prose — CLAUDE.md, AGENTS.md, TODO.md, 20 docs, both plugin design
documents, and the comment surface the earlier steps could not reach.
Applied against an explicit keep-list, not swept, because the word turned out to
have SIX meanings in this repository rather than the three the offscale doc
recorded:
permissions renamed (steps 1–2)
$OFFICER_ROOT/capabilities/ KEPT — the item store, and now the only thing
the word means that is ours
sidecar routing keys renamed to `handles` (step 3)
Lightning wallet KEPT — a domain term, and on the wire to the mobile apps
terminfo queries KEPT — XTGETTCAP, in the pty sidecar
InvoiceShelf KEPT — per-resource { write, bulkDelete } flags
The sweep still falsified two things, both caught by checking rather than by
review, and both in prose that discusses more than one meaning at once:
CLAUDE.md began claiming the item store lives at `$OFFICER_ROOT/permissions`.
It does not; that directory is on disk and full of skills and tools.
And the offscale doc's own note about the collision became
"Named `permissions`, NOT `permissions`" — a sentence that had eaten the thing
it existed to warn about.
Both restored, and the note rewritten to say what is now true: capability means
one thing of ours, and three that belong to somebody else's vocabulary.
Verified live after restart: self and admin permission endpoints 200, gated
route 200, agent-status 200, 9 grants intact with 6 permissions offered.
tsgo clean, 797 tests, 787 pass, same 7.
The rename is done. Four steps, no data lost, no client break that survived
the step it was introduced in.
107 lines
7.5 KiB
Markdown
107 lines
7.5 KiB
Markdown
# Jobs Unification — making every task a background job
|
||
|
||
**Status: shipped, except push notifications.** Phases 1–3 (data model, executor, queue, REST API and
|
||
the master-detail `/jobs` screen) are done and live. Phase 4 is the only open item, and is the reason
|
||
this file still exists. Email sync is deliberately NOT part of this any more — it moved into the email
|
||
sidecar with its own scheduling, so it does not appear in the Jobs list.
|
||
|
||
**Goal:** every task run (script, pipeline, later agentic) becomes a persisted, background **job** —
|
||
created over REST, streamed live over WebSocket, resumable/attachable, visible on desktop _and_ phone,
|
||
and ending in a push notification. Replaces today's ephemeral script-task WebSocket path.
|
||
|
||
**Context:** jobs belong to the owner. Not because the platform is single-user — it stopped being that
|
||
on 2026-08-07 — but because `tasks` is an `execution` permission: running a job means running a script
|
||
as the owner's OS user, so it can never be granted to a member. "Is anything running?" is therefore
|
||
still a global check, and the conclusion below is unchanged even though the premise was rewritten.
|
||
Favor power-user affordances over guardrails.
|
||
|
||
## Current state (baseline)
|
||
|
||
- **Pipeline tasks** already are jobs: `pipeline_jobs` (Postgres) + `pipeline-job-manager` (in-memory
|
||
live registry, viewer attach/replay buffer, 3s progress flush, finalize, restart-reconciliation) +
|
||
`/jobs` list (`JobsScreen`) + `/jobs/:id` detail (`JobDetail`, live-attach or persisted). WS at
|
||
`/api/tasks/pipeline/ws`.
|
||
- **Script tasks** (convert-video, edit-audio, …) run via `task-executor.ts` over `/api/tasks/run/ws`
|
||
— **ephemeral, zero persistence**, dies with the tab. Modal `TaskRunnerModal` (ScriptRunner branch).
|
||
- A **separate** file-backed queue engine exists at `src/servers/queue/` (lane-serial, retries,
|
||
crash-resume) wired only to email sync. NOT reused here — one Postgres jobs model instead.
|
||
|
||
## Decisions (locked)
|
||
|
||
- **Concurrency:** idle → single `Run` → `/jobs/:id`. Something running → `Queue` + a **red `Run`**
|
||
(force concurrent). Run → start now → `/jobs/:id`. Queue → append `pending` → `/jobs` list.
|
||
Backend (global): FIFO `pending` queue; on any finalize with 0 running, promote the oldest pending.
|
||
Force-Run bypasses the queue. Queue backlog survives restart (promoteNext on startup).
|
||
- **Modal → page:** replace `TaskRunnerModal` with a `/jobs/new` route. Right-click task action
|
||
navigates there instead of mounting the modal. One code path, phone-friendly.
|
||
- **Deep-link:** query params, not hash — `/jobs/new?task=convert-video&root=home&path=<dir>&entry=<name>`.
|
||
- **Phone:** already has a file browser, so it can pick a target path and POST a job. No blocker.
|
||
- **`inline` tasks stay in the modal.** A top-level `inline: true` in `TASK.md` marks quick/interactive
|
||
tasks that run in the modal (ephemeral) instead of as a job. Default = job. Currently flagged:
|
||
`remove-jelly-meta`, `analyze-video`, `clean-playlist-files`. So the modal is **not** retired — it's
|
||
the inline path; the shared input form serves both the modal and `/jobs/new`. The flag is plumbed
|
||
through the parser + task list/detail endpoints (`task.inline`), so the FileBrowser menu can branch.
|
||
|
||
## Plan
|
||
|
||
### Phase 1 — Unified jobs backend
|
||
|
||
- **1a. Data model.** Add `mode` (`pipeline|script|agentic`, default `pipeline`) + `exit_code` (int)
|
||
to the jobs table. Log at `DATA_PATH/jobs/<id>.log` (derived from id). _Table/symbol rename
|
||
`pipeline_jobs`→`jobs` is deferred as a cosmetic cleanup — add columns first, keep it working._
|
||
- **1b. Execution.** Generalize the job manager: `startJob` takes `mode` and dispatches — `pipeline`
|
||
→ existing `executePipeline`; `script` → new `executeScript` (ports task-executor's
|
||
`materializeScript`/`buildInputEnv`/bwrap sandbox/`killTree`/keepalive, but emits job events +
|
||
appends stdout/stderr to the log file, finalizes with `exit_code`). Reuses the manager's
|
||
viewer/broadcast/replay machinery.
|
||
- **1c. Scheduler.** `runningCount()` + FIFO `pending` promotion on finalize; `promoteNext()` on
|
||
startup so a queued backlog resumes.
|
||
|
||
### Phase 2 — REST job API (decouples creation from the socket; enables the phone)
|
||
|
||
- `POST /jobs {taskDirName, inputs, cwd, action}` → `{jobId}` (create + start/queue, background).
|
||
- `GET /jobs` (+`?live=1`), `GET /jobs/:id`, `GET /jobs/:id/log?offset=`, `POST /jobs/:id/stop`.
|
||
- Consolidate the two WebSockets into one `/api/tasks/jobs/ws` doing only attach/stop/list.
|
||
|
||
### Phase 3 — Frontend
|
||
|
||
- `/jobs/new` → `NewJobScreen`: reads query params, renders the input UI lifted from
|
||
`TaskRunnerModal` (`TaskInputForm` + per-group config + folder probing). Run/Queue per the
|
||
concurrency UX. `JobDetail` gains a script branch (terminal output: live attach, or from log when
|
||
idle). Retire `TaskRunnerModal`/`TaskRunnerDialog`/`useTaskRunner`. Header running-jobs indicator.
|
||
|
||
### Phase 4 — Notifications (later)
|
||
|
||
- One `notifyJobDone(job)` hook at finalize → push to the phone app.
|
||
|
||
## Progress
|
||
|
||
- [x] 1a data model — `mode` + `exit_code` columns (schema + applied to DB)
|
||
- [x] 1b executeScript + manager dispatch — `execute-script.ts` (spawn/sandbox/killTree port, log file,
|
||
abort poll, returns exitCode), `process-tree.ts` (shared killTree), `pipeline-job-manager` now
|
||
dispatches by `mode` and finalizes script jobs by exit code. _Compiles; runtime-untested until
|
||
a REST caller + restart exist._
|
||
- [x] 1c scheduler / queue — `enqueueJob(action)` (start now / queue behind running), `promoteNext()`
|
||
on finalize + startup, `getOldestPendingJob`, `markInterruptedJobs` now running-only (pending
|
||
queue survives restart). `startJob` kept as a `enqueueJob(...,'start')` wrapper.
|
||
- [x] 2 REST job API — `POST /jobs` (create script|pipeline, action start/queue), `GET /jobs` (+`?live=1`,
|
||
now returns mode/exitCode/isLive), `GET /jobs/:id`, `GET /jobs/:id/log?offset=`, `POST /jobs/:id/stop`.
|
||
Router mounted at `/jobs` and `/pipeline-jobs`. _Needs a restart to deploy; then curl/phone-testable._
|
||
WS consolidation still pending (old `/api/tasks/run/ws` + `/api/tasks/pipeline/ws` still live).
|
||
- [x] 3 frontend — master-detail `/jobs`, modal-as-creator, split list, header badges. Done.
|
||
- [x] 3a jobs UI — **master-detail** `JobsPage` (like `/chat`): `WorkspaceLayout` with a list panel
|
||
(left, polls `GET /jobs`, highlights active) + a detail panel (right) that branches by `mode`
|
||
— `ScriptJobDetail` terminal (polls log + status, Stop) or `PipelineJobDetail`. One page serves
|
||
both `/jobs` and `/jobs/:id`; list click navigates. Replaced the old separate list/detail pages.
|
||
- [x] `inline` flag plumbed (parser + list/detail endpoints); 3 quick tasks flagged.
|
||
- [x] 3b/3c task→job via the modal (pragmatic reuse). The task modal is now the job creator: on Run,
|
||
an **inline** task runs ephemerally in-modal; a **non-inline** task `POST /jobs` (start) →
|
||
navigates to `/jobs/:id`. When a job is already running, a red "Run now" + a "Queue" button
|
||
(queue → `/jobs`). Reuses the modal's per-group input UI in place — no separate `/jobs/new`
|
||
page or FileBrowser change needed. _(A standalone deep-linkable `/jobs/new` is deferred; the
|
||
phone creates jobs directly via `POST /jobs`.)_
|
||
- [x] 3d header job indicators — `JobsIndicator` (two always-present badges next to RescanButton +
|
||
UserMenu): **running** (→ running job's `/jobs/:id`) + **queued** (→ `/jobs`), polling
|
||
`GET /jobs/counts` → `{ running, runningJobId, queued }` every 3s; dim at 0.
|
||
- [ ] 4 push notifications
|