# Jobs Unification — making every task a background job

**Goal:** every task run (script, pipeline, later agentic) becomes a persisted, background **job** —
created over REST, streamed live over WebSocket, resumable/attachable, visible on desktop *and* phone,
and ending in a push notification. Replaces today's ephemeral script-task WebSocket path.

**Context:** single user, forever. No multi-tenant concerns — "is anything running?" is a global check.
Favor power-user affordances over guardrails. See memory `sole-user-assume-competence`.

## Current state (baseline)

- **Pipeline tasks** already are jobs: `pipeline_jobs` (Postgres) + `pipeline-job-manager` (in-memory
  live registry, viewer attach/replay buffer, 3s progress flush, finalize, restart-reconciliation) +
  `/jobs` list (`JobsScreen`) + `/jobs/:id` detail (`JobDetail`, live-attach or persisted). WS at
  `/api/tasks/pipeline/ws`.
- **Script tasks** (convert-video, edit-audio, …) run via `task-executor.ts` over `/api/tasks/run/ws`
  — **ephemeral, zero persistence**, dies with the tab. Modal `TaskRunnerModal` (ScriptRunner branch).
- A **separate** file-backed queue engine exists at `src/servers/queue/` (lane-serial, retries,
  crash-resume) wired only to email sync. NOT reused here — one Postgres jobs model instead.

## Decisions (locked)

- **Concurrency:** idle → single `Run` → `/jobs/:id`. Something running → `Queue` + a **red `Run`**
  (force concurrent). Run → start now → `/jobs/:id`. Queue → append `pending` → `/jobs` list.
  Backend (global): FIFO `pending` queue; on any finalize with 0 running, promote the oldest pending.
  Force-Run bypasses the queue. Queue backlog survives restart (promoteNext on startup).
- **Modal → page:** replace `TaskRunnerModal` with a `/jobs/new` route. Right-click task action
  navigates there instead of mounting the modal. One code path, phone-friendly.
- **Deep-link:** query params, not hash — `/jobs/new?task=convert-video&root=home&path=<dir>&entry=<name>`.
- **Phone:** already has a file browser, so it can pick a target path and POST a job. No blocker.

## Plan

### Phase 1 — Unified jobs backend
- **1a. Data model.** Add `mode` (`pipeline|script|agentic`, default `pipeline`) + `exit_code` (int)
  to the jobs table. Log at `DATA_PATH/jobs/<id>.log` (derived from id). *Table/symbol rename
  `pipeline_jobs`→`jobs` is deferred as a cosmetic cleanup — add columns first, keep it working.*
- **1b. Execution.** Generalize the job manager: `startJob` takes `mode` and dispatches — `pipeline`
  → existing `executePipeline`; `script` → new `executeScript` (ports task-executor's
  `materializeScript`/`buildInputEnv`/bwrap sandbox/`killTree`/keepalive, but emits job events +
  appends stdout/stderr to the log file, finalizes with `exit_code`). Reuses the manager's
  viewer/broadcast/replay machinery.
- **1c. Scheduler.** `runningCount()` + FIFO `pending` promotion on finalize; `promoteNext()` on
  startup so a queued backlog resumes.

### Phase 2 — REST job API (decouples creation from the socket; enables the phone)
- `POST /jobs {taskDirName, inputs, cwd, action}` → `{jobId}` (create + start/queue, background).
- `GET /jobs` (+`?live=1`), `GET /jobs/:id`, `GET /jobs/:id/log?offset=`, `POST /jobs/:id/stop`.
- Consolidate the two WebSockets into one `/api/tasks/jobs/ws` doing only attach/stop/list.

### Phase 3 — Frontend
- `/jobs/new` → `NewJobScreen`: reads query params, renders the input UI lifted from
  `TaskRunnerModal` (`TaskInputForm` + per-group config + folder probing). Run/Queue per the
  concurrency UX. `JobDetail` gains a script branch (terminal output: live attach, or from log when
  idle). Retire `TaskRunnerModal`/`TaskRunnerDialog`/`useTaskRunner`. Header running-jobs indicator.

### Phase 4 — Notifications (later)
- One `notifyJobDone(job)` hook at finalize → push to the phone app.

## Progress
- [x] 1a data model — `mode` + `exit_code` columns (schema + applied to DB)
- [x] 1b executeScript + manager dispatch — `execute-script.ts` (spawn/sandbox/killTree port, log file,
      abort poll, returns exitCode), `process-tree.ts` (shared killTree), `pipeline-job-manager` now
      dispatches by `mode` and finalizes script jobs by exit code. *Compiles; runtime-untested until
      a REST caller + restart exist.*
- [x] 1c scheduler / queue — `enqueueJob(action)` (start now / queue behind running), `promoteNext()`
      on finalize + startup, `getOldestPendingJob`, `markInterruptedJobs` now running-only (pending
      queue survives restart). `startJob` kept as a `enqueueJob(...,'start')` wrapper.
- [x] 2 REST job API — `POST /jobs` (create script|pipeline, action start/queue), `GET /jobs` (+`?live=1`,
      now returns mode/exitCode/isLive), `GET /jobs/:id`, `GET /jobs/:id/log?offset=`, `POST /jobs/:id/stop`.
      Router mounted at `/jobs` and `/pipeline-jobs`. *Needs a restart to deploy; then curl/phone-testable.*
      WS consolidation still pending (old `/api/tasks/run/ws` + `/api/tasks/pipeline/ws` still live).
- [~] 3 frontend
  - [x] 3a JobDetail script branch — `ScriptJobDetail` (terminal, polls `GET /jobs/:id/log` + status,
        Stop button) + `JobDetailView` wrapper routing by `mode` (pipeline detail renamed
        `PipelineJobDetail`). Script jobs are now viewable at `/jobs/:id`.
  - [ ] 3b `/jobs/new` page — extract the input UI (TaskInputForm + per-group config + folder probing)
        from `TaskRunnerModal` into a shared component; Run/Queue → `POST /jobs` → navigate.
  - [ ] 3c FileBrowser task action navigates to `/jobs/new?...`; retire modal/dialog/useTaskRunner.
  - [ ] 3d header running-jobs indicator (`GET /jobs?live=1`).
- [ ] 4 push notifications