jobs: video/audio download as a two-phase job (backend)
Turns the downloader into a server-side job on the existing jobs spine (Postgres
persistence, live WS viewers + replay, abort, /jobs UI) — but with its own
executor and its own lane, since it's deterministic scripting, not an agent, and
a multi-hour playlist mustn't block agentic jobs.
- reclip-client.ts (new, shared): reclipInfo / reclipPlaylist / reclipDownloadOne
(single download → streams the file to a dir, abort-aware). Extracted so both
the file-browser endpoints and the job executor use one client.
- execute-download.ts (new): the two-phase executor —
phase 1 metadata (expand playlist, fetch each info, keep survivors, skip
errors), phase 2 download (each survivor in the chosen format; skip download
errors). Emits a compact `download:progress` snapshot (counters, not per-item
events — playlists are thousands of items). Throws on abort / fatal.
- job manager: `download` mode dispatch → executeDownload; persists
download:progress; adds execution LANES (download vs default) so the two run
independently and each serializes on its own; promoteNext fills both lanes.
- POST /api/tasks/jobs/download { url, format, dir, root?, label? } — enqueues a
download job (own lane, no capability task needed; traversal-guarded target).
- schema: `download` added to the mode enum (drizzle text-enum — no DB migration);
getPendingJobs() query for lane filling.
Verified the executor with a mocked ReClip client: two-phase filtering, skip-on-
error counts, and abort-throws all correct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -5,6 +5,7 @@ import { getPipelineJobsForUser, getPipelineJob } from 'officerdb';
|
||||
import { getTaskByDirName } from './task-files';
|
||||
import * as jobManager from './pipeline-job-manager';
|
||||
import { jobLogPath } from './execute-script';
|
||||
import { getRootDir, resolveUserPath } from '../file-browser/router';
|
||||
|
||||
export const pipelineJobsRouter = createRouter();
|
||||
|
||||
@@ -70,6 +71,39 @@ pipelineJobsRouter.post('/', async (c) => {
|
||||
return c.json({ jobId, status });
|
||||
});
|
||||
|
||||
// POST /download — enqueue a video/audio download job (ReClip). Runs in its own lane, needs no capability
|
||||
// task. Body: { url (video or playlist), format:'audio'|'video', dir (target folder, home-relative),
|
||||
// root?, label? }. The job expands + fetches metadata (phase 1) then downloads survivors (phase 2).
|
||||
pipelineJobsRouter.post('/download', async (c) => {
|
||||
const user = c.get('user');
|
||||
const body = await c.req.json<{
|
||||
url: string;
|
||||
format?: 'audio' | 'video';
|
||||
dir?: string;
|
||||
root?: string;
|
||||
label?: string;
|
||||
}>();
|
||||
if (!body.url) throw errors.BAD_REQUEST('url is required');
|
||||
const rootDir = getRootDir(user, body.root);
|
||||
const absDir = resolveUserPath(rootDir, body.dir ?? '/'); // traversal-guarded
|
||||
const format = body.format === 'video' ? 'video' : 'audio';
|
||||
const { jobId, status } = await jobManager.enqueueJob(
|
||||
{
|
||||
userId: user.id,
|
||||
email: user.email,
|
||||
username: user.username ?? '',
|
||||
mode: 'download',
|
||||
taskDirName: 'video-download',
|
||||
taskName: body.label || (format === 'audio' ? 'Audio download' : 'Video download'),
|
||||
inputs: { url: body.url, format, absDir },
|
||||
cwd: body.dir ?? '/',
|
||||
config: {},
|
||||
},
|
||||
'queue',
|
||||
);
|
||||
return c.json({ jobId, status });
|
||||
});
|
||||
|
||||
// GET /counts — header-badge summary { running, runningJobId, queued }. Before /:id so it isn't
|
||||
// captured as an id.
|
||||
pipelineJobsRouter.get('/counts', async (c) => {
|
||||
|
||||
Reference in New Issue
Block a user