tools and skills, etc in the containers
This commit is contained in:
@@ -0,0 +1,293 @@
|
||||
# Automation System — Context Document
|
||||
|
||||
This document is written for AI assistants working on this codebase. It captures architectural decisions, the full vision, and the current state of the automation/tools system built during a long design and implementation session. Read this before touching anything in `src/servers/api/pi/`, `src/servers/api/skills/`, `src/servers/api/tasks/`, `src/servers/api/processes/`, `seed/tools/`, `seed/extensions/`, or `src/servers/sync-*.ts`.
|
||||
|
||||
---
|
||||
|
||||
## What Officer Is
|
||||
|
||||
Officer is a self-hosted AI-native intranet for small and medium businesses. Every user gets a personal AI assistant, terminal, file browser, code editor, workspaces, and automation tools. The platform is multi-user with role-based access: `Member → Admin → Owner → Super Admin`.
|
||||
|
||||
The AI backend was originally Claude SDK / OpenCode. It has been migrated to **Pi** (`@mariozechner/pi-coding-agent`) as the primary agent harness, running in RPC mode as a subprocess managed by `src/servers/api/pi/pi-bridge.ts`.
|
||||
|
||||
---
|
||||
|
||||
## The Automation Ladder
|
||||
|
||||
The owner has a clear conceptual model for automation capabilities, ordered from atomic to orchestrated:
|
||||
|
||||
| Rung | Description | Status |
|
||||
|------|-------------|--------|
|
||||
| **Skills** | Passive markdown documentation the agent reads as context. Reference for CLIs, APIs, services. | ✅ Implemented |
|
||||
| **Tools** | Executable TypeScript functions Pi can call directly during agent execution. Registered via Pi extensions. | ✅ Implemented |
|
||||
| **Tasks** | Structured instructions (TASK.md) to accomplish an atomic goal. Uses skills + tools. Has inputs, triggers, steps. | ✅ Skeleton exists |
|
||||
| **Pipelines** | A linear sequence of tasks, run one after the other (think pipe). | 🔲 Phase 2 |
|
||||
| **Processes** | A sequence of tasks that forks based on the result of the previous one (conditional branching). | 🔲 Phase 2 |
|
||||
| **Workflows** | Arbitrary task graphs, like n8n. | 🔲 Phase 3 |
|
||||
| **Services** | Event-driven execution — watches directories or events, triggers tasks based on configuration. | 🔲 Phase 3 |
|
||||
| **Crons** | Scheduled execution of a single task, pipeline, process, or workflow. | 🔲 Phase 3 |
|
||||
|
||||
**When the owner says "tool", they mean any rung of this ladder**, not just the Pi tool concept.
|
||||
|
||||
### Key distinction: agent-interpreted vs headless execution
|
||||
|
||||
Currently all tasks are **agent-interpreted** — the agent reads the TASK.md and reasons about how to execute it. In phase 2, well-defined tasks (deterministic inputs → tool call → result) should be executable **headlessly** without an LLM, directly by a task runner. The tool implementations are already standalone TypeScript functions that support this future — they have no dependency on Pi or the agent.
|
||||
|
||||
---
|
||||
|
||||
## How Pi Is Integrated
|
||||
|
||||
Pi runs as a subprocess in **RPC mode** (`pi --mode rpc`). The server communicates via stdin/stdout JSON. See `src/servers/api/pi/pi-bridge.ts` and `src/servers/api/pi/websocket.ts`.
|
||||
|
||||
### Spawning Pi (host, non-sandboxed)
|
||||
|
||||
```
|
||||
pi --mode rpc
|
||||
--no-skills --no-prompt-templates --no-themes
|
||||
--skill /data/skills/{name} (one per global skill)
|
||||
--skill /data/{email}/skills/{name} (one per user skill)
|
||||
--extension /data/extensions/{name}/index.ts
|
||||
--model {model}
|
||||
|
||||
env:
|
||||
PI_CODING_AGENT_DIR = DATA_PATH/pi-config
|
||||
PI_TOOLS_DIRS = DATA_PATH/tools:DATA_PATH/{email}/tools
|
||||
PI_SEARXNG_URL = https://searxng.home.pastilhas.eu
|
||||
+ all stored API keys
|
||||
```
|
||||
|
||||
Note: `--no-extensions` was intentionally removed to allow our tool-loader extension to work. `--no-skills` is kept because we pass skills explicitly via `--skill` flags to control scope correctly.
|
||||
|
||||
### Spawning Pi (sandboxed, Docker container)
|
||||
|
||||
Same pattern but via `docker exec`, using container-side paths:
|
||||
|
||||
```
|
||||
docker exec -i -w {workdir} \
|
||||
-e PI_CODING_AGENT_DIR=/home/{username}/.pi/agent \
|
||||
-e PI_TOOLS_DIRS=/officer/tools:/officer/user/tools \
|
||||
-e PI_SEARXNG_URL=... \
|
||||
-e {API_KEYS} \
|
||||
{containerId} \
|
||||
pi --mode rpc --no-skills --no-prompt-templates --no-themes \
|
||||
--skill /officer/skills/{name} \
|
||||
--skill /officer/user/skills/{name} \
|
||||
--extension /officer/extensions/tool-loader/index.ts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
DATA_PATH/ # Default: ./data, override with DATA_PATH env
|
||||
├── pi-config/ # Pi's global config (models.json, settings.json)
|
||||
├── skills/ # Global skills (org-wide)
|
||||
├── tools/ # Global tools (org-wide)
|
||||
├── extensions/ # Global extensions (managed by server, not users)
|
||||
├── {email}/
|
||||
│ ├── home/ # Mounted as /home/{username} in Docker container
|
||||
│ ├── skills/ # User-specific skills
|
||||
│ ├── tools/ # User-specific tools
|
||||
│ └── extensions/ # User-specific extensions (future)
|
||||
└── searxng.json # SearXNG instance URL config
|
||||
|
||||
seed/ # Bundled with the app (read-only source of truth)
|
||||
├── skills/ # Native/officerdev skills
|
||||
├── tools/ # Native/officerdev tools
|
||||
├── extensions/ # Native/officerdev extensions
|
||||
│ └── tool-loader/index.ts # The Pi extension that registers tools
|
||||
└── tasks/ # Native/officerdev tasks
|
||||
```
|
||||
|
||||
### Sync on server start (`src/servers/bootstrap.ts`)
|
||||
|
||||
- `syncSeedSkills()` → copies `seed/skills/*` → `DATA_PATH/skills/` (skip if exists, preserves user edits)
|
||||
- `syncSeedTools()` → copies `seed/tools/*` → `DATA_PATH/tools/` (skip if exists)
|
||||
- `syncSeedExtensions()` → copies `seed/extensions/*` → `DATA_PATH/extensions/` (**always overwrites** — extensions are server-managed code, not user-editable)
|
||||
|
||||
---
|
||||
|
||||
## The Tool System
|
||||
|
||||
### How tools are defined
|
||||
|
||||
Each tool lives in a directory with two files:
|
||||
|
||||
```
|
||||
seed/tools/web-fetch/
|
||||
├── TOOL.md # Metadata + input schema (read by tool-loader, registered with Pi)
|
||||
└── index.ts # Implementation (lazy-loaded when the tool is actually called)
|
||||
```
|
||||
|
||||
**TOOL.md frontmatter format:**
|
||||
```yaml
|
||||
---
|
||||
name: tool_name # Pi tool name (snake_case)
|
||||
label: Tool Label # Human-readable label
|
||||
description: ... # What Pi sees in its context (keep concise — this is in every session prompt)
|
||||
language: typescript # typescript | bash | python
|
||||
inputs:
|
||||
param_name:
|
||||
type: string # string | number | boolean | enum
|
||||
description: ...
|
||||
optional: true # omit if required
|
||||
mode:
|
||||
type: enum
|
||||
values: single,batch # comma-separated for enum (parser limitation)
|
||||
description: ...
|
||||
---
|
||||
```
|
||||
|
||||
**index.ts export signature:**
|
||||
```typescript
|
||||
export async function execute(
|
||||
toolCallId: string,
|
||||
params: { [key: string]: any },
|
||||
signal: AbortSignal | undefined,
|
||||
onUpdate?: (partial: { content: Array<{ type: string; text: string }> }) => void,
|
||||
): Promise<{ content: Array<{ type: string; text: string }>; details?: object; isError?: boolean }>
|
||||
```
|
||||
|
||||
The `onUpdate` callback streams progress to the agent. Use it liberally — the frontend shows it in real time.
|
||||
|
||||
### How the tool-loader extension works (`seed/extensions/tool-loader/index.ts`)
|
||||
|
||||
- Loaded by Pi as an extension via `--extension` flag
|
||||
- Reads `PI_TOOLS_DIRS` env var (colon-separated list of directories)
|
||||
- Discovers `TOOL.md` in each directory, parses frontmatter, builds TypeBox schema
|
||||
- Registers each tool synchronously in the extension factory function (so tools appear in the system prompt)
|
||||
- Lazy-loads `index.ts` implementation only when the tool is actually called
|
||||
- User dirs come after global — later registration wins (user tools override global by name)
|
||||
|
||||
### Existing tools
|
||||
|
||||
| Tool | Location | Description |
|
||||
|------|----------|-------------|
|
||||
| `web_fetch` | `seed/tools/web-fetch/` | Fetch URL content. Tries `.md` suffix → `/llms.txt` → plain text → HTML strip |
|
||||
| `web_search` | `seed/tools/web-search/` | Search via SearXNG. Returns titles, URLs, snippets. Pair with web_fetch |
|
||||
| `convert_audio_to_mp3` | `seed/tools/convert-audio-to-mp3/` | Convert audio to MP3 via ffmpeg. Single file (% progress) or batch directory (per-file progress) |
|
||||
|
||||
### Adding a new tool
|
||||
|
||||
1. Create `seed/tools/{name}/TOOL.md` and `seed/tools/{name}/index.ts`
|
||||
2. Restart server — `syncSeedTools()` copies it to `DATA_PATH/tools/`
|
||||
3. New Pi sessions automatically get the tool registered
|
||||
|
||||
---
|
||||
|
||||
## Docker Container Setup
|
||||
|
||||
Each user has a persistent Docker container (`officer-terminal-{userId}`). Containers are managed by `src/servers/api/terminal/websocket.ts`.
|
||||
|
||||
### Read-only resource mounts
|
||||
|
||||
Added to every container at creation time:
|
||||
|
||||
```
|
||||
/officer/skills → DATA_PATH/skills (global skills, ro)
|
||||
/officer/tools → DATA_PATH/tools (global tools, ro)
|
||||
/officer/extensions → DATA_PATH/extensions (global extensions, ro)
|
||||
/officer/user/skills → DATA_PATH/{email}/skills (user skills, ro)
|
||||
/officer/user/tools → DATA_PATH/{email}/tools (user tools, ro)
|
||||
```
|
||||
|
||||
The user's home directory (`DATA_PATH/{email}/home`) is mounted read-write as `/home/{username}`.
|
||||
|
||||
### Mount migration
|
||||
|
||||
`ensureDockerContainer` checks `containerHasResourceMounts()` before reusing an existing container. If the mounts are missing (old container created before this feature), the container is removed and recreated automatically. This is a one-time migration.
|
||||
|
||||
---
|
||||
|
||||
## Scope & Permission Model
|
||||
|
||||
All automation entities (skills, tasks, processes, and future rungs) follow the same three-tier scope model:
|
||||
|
||||
| Scope | Location | Who creates | Who can see |
|
||||
|-------|----------|-------------|-------------|
|
||||
| `native` | `seed/` (read-only) | officerdev (us) | Everyone |
|
||||
| `global` | `DATA_PATH/{type}/` | Admins+ | Everyone |
|
||||
| `user` | `DATA_PATH/{email}/{type}/` | Anyone | Owner + SAs (future) |
|
||||
|
||||
### Current API behavior (skills, tasks, processes)
|
||||
|
||||
```typescript
|
||||
function isPrivileged(role: string) {
|
||||
return role === 'Admin' || role === 'Owner' || role === 'Super Admin';
|
||||
}
|
||||
```
|
||||
|
||||
- `POST /` (create): Members → user scope. Admins+ → global scope.
|
||||
- `DELETE /:name`: Members can only delete their own (user scope). Admins+ can delete global.
|
||||
- `PUT /:name/chat` / `DELETE /:name/chat`: Members blocked from writing to native/global entries.
|
||||
|
||||
### Full vision (phase 2/3 — not yet implemented)
|
||||
|
||||
- **Super Admins** can see all users' personal items (currently everyone only sees their own)
|
||||
- **Members can submit** their personal item for SA review (`status: draft | submitted | approved | rejected` — a flag in frontmatter, not a new scope)
|
||||
- **SAs can promote** any user's item to global — this is a **move** (not a copy), item leaves user scope
|
||||
- **SAs can see and use** everyone's tools
|
||||
- The `CapabilitySummary` type already has `scope` surfaced in the frontend list with a badge
|
||||
|
||||
---
|
||||
|
||||
## SearXNG Integration
|
||||
|
||||
- Config stored in `DATA_PATH/searxng.json`
|
||||
- Default URL: `https://searxng.home.pastilhas.eu` (hardcoded default in `src/servers/api/server-settings/searxng.ts`)
|
||||
- API: `GET /api/server-settings/searxng`, `PUT /api/server-settings/searxng`
|
||||
- Injected into Pi processes as `PI_SEARXNG_URL` env var (both host and container)
|
||||
- UI configuration is a future task — the API is already there
|
||||
|
||||
---
|
||||
|
||||
## Attribution & Marketplace Vision (future)
|
||||
|
||||
The owner has a clear attribution model:
|
||||
|
||||
- **Native/officerdev**: `officerdev/<name>` — built-in, shipped with the product
|
||||
- **User-created org tools**: attribution via `author: <email>` in frontmatter
|
||||
- **Marketplace — officerdev official**: `officerdev/<name>`
|
||||
- **Marketplace — third party**: `<dev>/<name>` or `<organization>/<name>`
|
||||
|
||||
Future marketplace will cover: Skills, Tools, Tasks, Pipelines, Processes, Workflows, Services, Crons, **Widgets**, **Apps**, and **Themes**. Widgets and Apps are already bootstrapped in the platform. Each self-hosted instance will connect to a central webstore.
|
||||
|
||||
The `native` scope covers both built-in and future marketplace-installed items. When marketplace is implemented, `native` may split into `native` (local seed) and `marketplace` (installed from store, with version + source metadata).
|
||||
|
||||
---
|
||||
|
||||
## Files Changed / Created
|
||||
|
||||
### New files
|
||||
- `src/servers/sync-tools.ts` — mirrors sync-skills pattern for tools
|
||||
- `src/servers/sync-extensions.ts` — same for extensions (always overwrites)
|
||||
- `src/servers/api/server-settings/searxng.ts` — SearXNG config router
|
||||
- `seed/tools/web-fetch/TOOL.md` + `index.ts`
|
||||
- `seed/tools/web-search/TOOL.md` + `index.ts`
|
||||
- `seed/tools/convert-audio-to-mp3/TOOL.md` + `index.ts`
|
||||
- `seed/extensions/tool-loader/index.ts`
|
||||
|
||||
### Modified files
|
||||
- `src/servers/data-path.ts` — added tool/extension dir helpers
|
||||
- `src/servers/bootstrap.ts` — calls syncSeedTools, syncSeedExtensions
|
||||
- `src/servers/api/pi/pi-bridge.ts` — removed `--no-extensions`, added extension/tool flags, PI_TOOLS_DIRS, PI_SEARXNG_URL, container-aware path support
|
||||
- `src/servers/api/terminal/websocket.ts` — added 5 read-only resource mounts to containers, mount migration check
|
||||
- `src/servers/api/skills/skills.ts` — fixed scope-aware POST/DELETE/chat endpoints
|
||||
- `src/servers/api/tasks/tasks.ts` — same fixes
|
||||
- `src/servers/api/processes/processes.ts` — same fixes
|
||||
- `src/servers/api/server-settings/server-settings.ts` — registered searxng router
|
||||
- `seed/skills/convert-audio-to-mp3/SKILL.md` — removed hardcoded paths, references tool instead
|
||||
- `seed/tasks/convert-to-mp3/TASK.md` — updated to use tool, added `tools:` frontmatter field
|
||||
- `src/apps/officer-web/Screens/Dashboard/CapabilityPage.tsx` — fixed `scope` type to include `'native'`
|
||||
|
||||
---
|
||||
|
||||
## What To Do Next (Rough Priority)
|
||||
|
||||
1. **Wire `tool_execution_update` events** through `pi-bridge.ts` → websocket → frontend so `onUpdate` progress actually shows in the chat UI
|
||||
2. **Container support testing** — verify tools and extensions work correctly inside Docker after the read-only mount changes
|
||||
3. **SA cross-user visibility** — API + UI for Super Admins to see all users' personal items
|
||||
4. **Submit/approve workflow** — `status` flag in frontmatter, submission UI for members, review UI for SAs
|
||||
5. **SearXNG UI** — settings panel to configure the URL (API is already done)
|
||||
6. **Headless task runner** — execute deterministic tasks without an agent (phase 2)
|
||||
@@ -114,7 +114,14 @@ function buildSchema(inputs: Record<string, ToolParam>): TSchema {
|
||||
|
||||
switch (param.type) {
|
||||
case 'enum': {
|
||||
const values = param.values ?? [];
|
||||
// values can be a string[] from deeper YAML parsing,
|
||||
// or a comma-separated string like "single,batch" from flat YAML
|
||||
const raw = param.values;
|
||||
const values = Array.isArray(raw)
|
||||
? raw
|
||||
: typeof raw === 'string'
|
||||
? raw.split(',').map((v) => v.trim())
|
||||
: [];
|
||||
schema = Type.Union(values.map((v) => Type.Literal(v)), {
|
||||
description: param.description,
|
||||
});
|
||||
|
||||
@@ -1,13 +1,15 @@
|
||||
---
|
||||
name: Convert To MP3
|
||||
description: Convert audio files to MP3 320kbps, preserving metadata.
|
||||
version: 1
|
||||
version: 2
|
||||
author: pastilhas
|
||||
tags:
|
||||
- audio
|
||||
- conversion
|
||||
skills:
|
||||
- convert-audio-to-mp3
|
||||
tools:
|
||||
- convert_audio_to_mp3
|
||||
trigger:
|
||||
- type: file
|
||||
extensions:
|
||||
@@ -39,6 +41,7 @@ Convert audio files to MP3 320kbps, preserving metadata.
|
||||
## Steps
|
||||
|
||||
1. Determine whether `file_path` points to a single audio file or a directory.
|
||||
2. If it is a single file, use the Convert Audio To MP3 skill's single-file script to convert it. The source file is deleted on success.
|
||||
3. If it is a directory, use the Convert Audio To MP3 skill's batch script with the directory name as the artist name. All audio files within subdirectories are converted recursively.
|
||||
4. Verify the conversion completed successfully and report the result to the user.
|
||||
2. Call `convert_audio_to_mp3` with the appropriate mode:
|
||||
- Single file: `mode="single"`, `path=file_path`
|
||||
- Directory: `mode="batch"`, `path=file_path`
|
||||
3. Report the result to the user.
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
name: convert_audio_to_mp3
|
||||
label: Convert Audio to MP3
|
||||
description: Convert audio files to MP3 320kbps using ffmpeg, preserving metadata. Supports single file conversion with real-time percentage progress, and batch conversion of an entire artist directory with per-file progress. Use when the user wants to convert FLAC, WAV, OGG, or other audio formats to MP3.
|
||||
language: typescript
|
||||
inputs:
|
||||
mode:
|
||||
type: enum
|
||||
values: single,batch
|
||||
description: "single: convert one file. batch: convert all audio files recursively in a directory"
|
||||
path:
|
||||
type: string
|
||||
description: "single mode: absolute path to the audio file. batch mode: absolute path to the artist directory"
|
||||
---
|
||||
|
||||
# Convert Audio to MP3
|
||||
|
||||
Converts audio to MP3 320kbps CBR via libmp3lame, preserving all metadata tags.
|
||||
|
||||
## Modes
|
||||
|
||||
### single
|
||||
Converts one file. Reports ffmpeg percentage progress in real time. Deletes source on success.
|
||||
|
||||
### batch
|
||||
Scans a directory recursively for all supported audio files. Reports per-file progress.
|
||||
- All succeed → deletes all source files
|
||||
- Any failure → deletes all successfully created MP3s for a clean retry
|
||||
|
||||
## Supported formats
|
||||
flac, wav, ogg, wma, aac, m4a, opus, aiff, aif, ape, wv, alac, dsf, dff
|
||||
@@ -0,0 +1,267 @@
|
||||
import { readdirSync, existsSync, unlinkSync, statSync } from 'node:fs';
|
||||
import { join, extname, basename } from 'node:path';
|
||||
|
||||
const AUDIO_EXTS = new Set([
|
||||
'flac', 'wav', 'ogg', 'wma', 'aac', 'm4a', 'opus',
|
||||
'aiff', 'aif', 'ape', 'wv', 'alac', 'dsf', 'dff',
|
||||
]);
|
||||
|
||||
type OnUpdate = (partial: { content: Array<{ type: string; text: string }> }) => void;
|
||||
|
||||
function update(onUpdate: OnUpdate | undefined, text: string): void {
|
||||
onUpdate?.({ content: [{ type: 'text', text }] });
|
||||
}
|
||||
|
||||
async function getDuration(filePath: string): Promise<number | null> {
|
||||
const proc = Bun.spawn(
|
||||
['ffprobe', '-v', 'quiet', '-show_entries', 'format=duration', '-of', 'csv=p=0', filePath],
|
||||
{ stdout: 'pipe', stderr: 'pipe' },
|
||||
);
|
||||
const text = await new Response(proc.stdout).text();
|
||||
await proc.exited;
|
||||
const n = parseFloat(text.trim());
|
||||
return isNaN(n) ? null : n;
|
||||
}
|
||||
|
||||
function scanAudioFiles(dir: string): string[] {
|
||||
const files: string[] = [];
|
||||
for (const entry of readdirSync(dir, { withFileTypes: true })) {
|
||||
const full = join(dir, entry.name);
|
||||
if (entry.isDirectory()) {
|
||||
files.push(...scanAudioFiles(full));
|
||||
} else if (entry.isFile()) {
|
||||
const ext = extname(entry.name).slice(1).toLowerCase();
|
||||
if (AUDIO_EXTS.has(ext)) files.push(full);
|
||||
}
|
||||
}
|
||||
return files;
|
||||
}
|
||||
|
||||
type ConvertResult = {
|
||||
outputFile: string;
|
||||
success: boolean;
|
||||
skipped?: boolean;
|
||||
error?: string;
|
||||
};
|
||||
|
||||
async function convertFile(
|
||||
inputFile: string,
|
||||
duration: number | null,
|
||||
label: string,
|
||||
onUpdate: OnUpdate | undefined,
|
||||
): Promise<ConvertResult> {
|
||||
const ext = extname(inputFile);
|
||||
const outputFile = inputFile.slice(0, -ext.length) + '.mp3';
|
||||
|
||||
if (existsSync(outputFile)) {
|
||||
return { outputFile, success: true, skipped: true };
|
||||
}
|
||||
|
||||
const proc = Bun.spawn(
|
||||
[
|
||||
'ffmpeg',
|
||||
'-i', inputFile,
|
||||
'-progress', 'pipe:1', // progress data → stdout
|
||||
'-nostats',
|
||||
'-loglevel', 'error', // only errors → stderr
|
||||
'-codec:a', 'libmp3lame',
|
||||
'-b:a', '320k',
|
||||
outputFile,
|
||||
'-y',
|
||||
],
|
||||
{ stdin: 'ignore', stdout: 'pipe', stderr: 'pipe' },
|
||||
);
|
||||
|
||||
// Parse -progress output from stdout for real-time percentage
|
||||
const reader = (proc.stdout as ReadableStream<Uint8Array>).getReader();
|
||||
const decoder = new TextDecoder();
|
||||
let buf = '';
|
||||
let lastPercent = -1;
|
||||
|
||||
(async () => {
|
||||
try {
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
buf += decoder.decode(value, { stream: true });
|
||||
const lines = buf.split('\n');
|
||||
buf = lines.pop() ?? '';
|
||||
for (const line of lines) {
|
||||
const m = line.match(/^out_time_us=(\d+)$/);
|
||||
if (m && duration) {
|
||||
const pct = Math.min(100, Math.round((parseInt(m[1]!, 10) / 1_000_000 / duration) * 100));
|
||||
if (pct >= lastPercent + 5) {
|
||||
lastPercent = pct;
|
||||
update(onUpdate, `${label} ${pct}%`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// stream closed
|
||||
}
|
||||
})();
|
||||
|
||||
const stderrText = await new Response(proc.stderr).text();
|
||||
await proc.exited;
|
||||
|
||||
if (proc.exitCode !== 0) {
|
||||
if (existsSync(outputFile)) unlinkSync(outputFile);
|
||||
return { outputFile, success: false, error: stderrText.trim() || 'ffmpeg failed' };
|
||||
}
|
||||
|
||||
return { outputFile, success: true };
|
||||
}
|
||||
|
||||
export async function execute(
|
||||
_toolCallId: string,
|
||||
params: { mode: 'single' | 'batch'; path: string },
|
||||
_signal: AbortSignal | undefined,
|
||||
onUpdate?: OnUpdate,
|
||||
) {
|
||||
const { mode, path } = params;
|
||||
|
||||
// ── Single file ──────────────────────────────────────────────────────────
|
||||
if (mode === 'single') {
|
||||
if (!existsSync(path)) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `File not found: ${path}` }],
|
||||
details: { error: 'file_not_found' },
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
const ext = extname(path).slice(1).toLowerCase();
|
||||
if (!AUDIO_EXTS.has(ext)) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Unsupported format: .${ext}\nSupported: ${[...AUDIO_EXTS].join(', ')}` }],
|
||||
details: { error: 'unsupported_format' },
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
update(onUpdate, `Getting duration of ${basename(path)}...`);
|
||||
const duration = await getDuration(path);
|
||||
|
||||
update(onUpdate, `Converting ${basename(path)}...`);
|
||||
const result = await convertFile(path, duration, 'Progress:', onUpdate);
|
||||
|
||||
if (result.skipped) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Skipped: ${result.outputFile} already exists` }],
|
||||
details: { skipped: true, outputFile: result.outputFile },
|
||||
};
|
||||
}
|
||||
|
||||
if (!result.success) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Failed to convert ${basename(path)}:\n${result.error}` }],
|
||||
details: { error: result.error },
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
unlinkSync(path);
|
||||
|
||||
return {
|
||||
content: [{ type: 'text', text: `Done.\nConverted: ${basename(result.outputFile)}\nDeleted source: ${basename(path)}` }],
|
||||
details: { outputFile: result.outputFile },
|
||||
};
|
||||
}
|
||||
|
||||
// ── Batch ─────────────────────────────────────────────────────────────────
|
||||
if (!existsSync(path)) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Directory not found: ${path}` }],
|
||||
details: { error: 'dir_not_found' },
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
if (!statSync(path).isDirectory()) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Not a directory: ${path}\nUse mode="single" for individual files.` }],
|
||||
details: { error: 'not_a_directory' },
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
update(onUpdate, `Scanning ${basename(path)} for audio files...`);
|
||||
const audioFiles = scanAudioFiles(path);
|
||||
|
||||
if (audioFiles.length === 0) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `No audio files found in: ${path}` }],
|
||||
details: { found: 0 },
|
||||
};
|
||||
}
|
||||
|
||||
update(onUpdate, `Found ${audioFiles.length} audio files. Starting conversion...`);
|
||||
|
||||
type BatchResult = ConvertResult & { input: string };
|
||||
const results: BatchResult[] = [];
|
||||
|
||||
for (let i = 0; i < audioFiles.length; i++) {
|
||||
const inputFile = audioFiles[i]!;
|
||||
const label = `[${i + 1}/${audioFiles.length}] ${basename(inputFile)}`;
|
||||
|
||||
update(onUpdate, `Converting ${label}...`);
|
||||
const duration = await getDuration(inputFile);
|
||||
const result = await convertFile(inputFile, duration, label, onUpdate);
|
||||
|
||||
results.push({ ...result, input: inputFile });
|
||||
|
||||
if (result.skipped) {
|
||||
update(onUpdate, `→ Skipped ${label} (MP3 already exists)`);
|
||||
} else if (result.success) {
|
||||
update(onUpdate, `✓ Done ${label}`);
|
||||
} else {
|
||||
update(onUpdate, `✗ Failed ${label}: ${result.error}`);
|
||||
}
|
||||
}
|
||||
|
||||
const converted = results.filter((r) => r.success && !r.skipped);
|
||||
const failed = results.filter((r) => !r.success);
|
||||
const skipped = results.filter((r) => r.skipped);
|
||||
|
||||
if (failed.length === 0) {
|
||||
// All succeeded — delete source files
|
||||
update(onUpdate, `All conversions succeeded. Deleting ${converted.length} source files...`);
|
||||
for (const r of converted) unlinkSync(r.input);
|
||||
|
||||
const lines = [
|
||||
`Conversion complete.`,
|
||||
`Converted: ${converted.length}`,
|
||||
skipped.length > 0 ? `Skipped (already existed): ${skipped.length}` : null,
|
||||
`Source files deleted: ${converted.length}`,
|
||||
].filter(Boolean);
|
||||
|
||||
return {
|
||||
content: [{ type: 'text', text: lines.join('\n') }],
|
||||
details: { converted: converted.length, failed: 0, skipped: skipped.length },
|
||||
};
|
||||
}
|
||||
|
||||
// Some failed — delete created MP3s for a clean retry
|
||||
update(onUpdate, `${failed.length} failure(s). Rolling back ${converted.length} created MP3(s) for clean retry...`);
|
||||
for (const r of converted) {
|
||||
if (existsSync(r.outputFile)) unlinkSync(r.outputFile);
|
||||
}
|
||||
|
||||
const failLines = failed.map((r) => ` - ${basename(r.input)}: ${r.error}`).join('\n');
|
||||
|
||||
return {
|
||||
content: [{
|
||||
type: 'text',
|
||||
text: [
|
||||
`Conversion failed. ${failed.length}/${audioFiles.length} file(s) could not be converted.`,
|
||||
`Successfully created MP3s have been removed — the directory is unchanged for a clean retry.`,
|
||||
``,
|
||||
`Failed files:`,
|
||||
failLines,
|
||||
].join('\n'),
|
||||
}],
|
||||
details: { converted: 0, failed: failed.length, rolledBack: converted.length },
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
---
|
||||
name: web_search
|
||||
label: Web Search
|
||||
description: Search the web using a private SearXNG instance and return a list of results with titles, URLs, and snippets. Use when you need to find information or URLs without already knowing where to look. Pair with web_fetch to read the full content of any result.
|
||||
language: typescript
|
||||
inputs:
|
||||
query:
|
||||
type: string
|
||||
description: The search query
|
||||
max_results:
|
||||
type: number
|
||||
description: Maximum number of results to return (default 10, max 20)
|
||||
optional: true
|
||||
---
|
||||
|
||||
# Web Search
|
||||
|
||||
Searches the web via a self-hosted SearXNG instance. Returns titles, URLs, and content snippets.
|
||||
|
||||
## Usage pattern
|
||||
|
||||
1. Call `web_search` with a query to get a list of results
|
||||
2. Call `web_fetch` on any result URL to read its full content
|
||||
|
||||
## Configuration
|
||||
|
||||
The SearXNG instance URL is read from the `PI_SEARXNG_URL` environment variable.
|
||||
@@ -0,0 +1,124 @@
|
||||
const TIMEOUT_MS = 10_000;
|
||||
const DEFAULT_MAX_RESULTS = 10;
|
||||
const HARD_MAX_RESULTS = 20;
|
||||
|
||||
type SearxngResult = {
|
||||
title: string;
|
||||
url: string;
|
||||
content?: string;
|
||||
engine?: string;
|
||||
score?: number;
|
||||
};
|
||||
|
||||
type SearxngResponse = {
|
||||
query: string;
|
||||
number_of_results: number;
|
||||
results: SearxngResult[];
|
||||
};
|
||||
|
||||
type OnUpdate = (partial: { content: Array<{ type: string; text: string }> }) => void;
|
||||
|
||||
function update(onUpdate: OnUpdate | undefined, text: string): void {
|
||||
onUpdate?.({ content: [{ type: 'text', text }] });
|
||||
}
|
||||
|
||||
function formatResults(results: SearxngResult[]): string {
|
||||
if (results.length === 0) return 'No results found.';
|
||||
|
||||
return results
|
||||
.map((r, i) => {
|
||||
const lines = [`${i + 1}. **${r.title}**`, ` ${r.url}`];
|
||||
if (r.content?.trim()) lines.push(` ${r.content.trim()}`);
|
||||
return lines.join('\n');
|
||||
})
|
||||
.join('\n\n');
|
||||
}
|
||||
|
||||
export async function execute(
|
||||
_toolCallId: string,
|
||||
params: { query: string; max_results?: number },
|
||||
_signal: AbortSignal | undefined,
|
||||
onUpdate?: OnUpdate,
|
||||
) {
|
||||
const searxngUrl = process.env.PI_SEARXNG_URL;
|
||||
|
||||
if (!searxngUrl) {
|
||||
return {
|
||||
content: [{ type: 'text', text: 'Web search is not configured. PI_SEARXNG_URL is not set.' }],
|
||||
details: { error: 'not_configured' },
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
const { query } = params;
|
||||
const maxResults = Math.min(params.max_results ?? DEFAULT_MAX_RESULTS, HARD_MAX_RESULTS);
|
||||
|
||||
if (!query?.trim()) {
|
||||
return {
|
||||
content: [{ type: 'text', text: 'Query cannot be empty.' }],
|
||||
details: { error: 'empty_query' },
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
update(onUpdate, `Searching for: ${query}`);
|
||||
|
||||
const searchUrl = new URL('/search', searxngUrl);
|
||||
searchUrl.searchParams.set('q', query);
|
||||
searchUrl.searchParams.set('format', 'json');
|
||||
searchUrl.searchParams.set('categories', 'general');
|
||||
|
||||
let response: Response;
|
||||
try {
|
||||
response = await fetch(searchUrl.toString(), {
|
||||
signal: AbortSignal.timeout(TIMEOUT_MS),
|
||||
headers: { Accept: 'application/json' },
|
||||
});
|
||||
} catch (err) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `Failed to reach SearXNG at ${searxngUrl}: ${String(err)}` }],
|
||||
details: { error: 'fetch_failed', url: searxngUrl },
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
if (!response.ok) {
|
||||
return {
|
||||
content: [{ type: 'text', text: `SearXNG returned HTTP ${response.status}` }],
|
||||
details: { error: 'http_error', status: response.status },
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
let data: SearxngResponse;
|
||||
try {
|
||||
data = (await response.json()) as SearxngResponse;
|
||||
} catch {
|
||||
return {
|
||||
content: [{ type: 'text', text: 'SearXNG returned an invalid response.' }],
|
||||
details: { error: 'invalid_json' },
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
const results = (data.results ?? []).slice(0, maxResults);
|
||||
|
||||
update(onUpdate, `Found ${data.number_of_results ?? results.length} results, returning top ${results.length}`);
|
||||
|
||||
const output = [
|
||||
`Search: "${query}"`,
|
||||
`Results: ${results.length}`,
|
||||
``,
|
||||
formatResults(results),
|
||||
].join('\n');
|
||||
|
||||
return {
|
||||
content: [{ type: 'text', text: output }],
|
||||
details: {
|
||||
query,
|
||||
total: data.number_of_results ?? results.length,
|
||||
returned: results.length,
|
||||
results: results.map((r) => ({ title: r.title, url: r.url })),
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -14,7 +14,7 @@ type CapabilitySummary = {
|
||||
dirName: string;
|
||||
name: string;
|
||||
description: string;
|
||||
scope: 'global' | 'user';
|
||||
scope: 'native' | 'global' | 'user';
|
||||
};
|
||||
|
||||
export type CapabilityDetail = CapabilitySummary & {
|
||||
|
||||
@@ -3,24 +3,28 @@ import { readdirSync, existsSync, mkdirSync } from "node:fs";
|
||||
import type { Subprocess } from "bun";
|
||||
import type { PiEvent, MessageCost } from "./types";
|
||||
import { readApiKeys } from "../server-settings/pi-mono";
|
||||
import { readSearxngConfig } from "../server-settings/searxng";
|
||||
import { PI_CONFIG_DIR, getGlobalSkillsDir, getUserSkillsDir, getGlobalExtensionsDir, getUserExtensionsDir, getGlobalToolsDir, getUserToolsDir } from "../../data-path";
|
||||
import { ensureDockerContainer } from "../terminal/websocket";
|
||||
import { logger } from "./logger";
|
||||
|
||||
export type PiEventHandler = (event: PiEvent) => void;
|
||||
|
||||
function collectSkillFlags(email: string): string[] {
|
||||
const flags: string[] = [];
|
||||
const dirs = [getGlobalSkillsDir(), getUserSkillsDir(email)];
|
||||
type PathOverrides = { global: string; user: string };
|
||||
|
||||
for (const dir of dirs) {
|
||||
if (!existsSync(dir)) continue;
|
||||
const entries = readdirSync(dir, { withFileTypes: true });
|
||||
for (const entry of entries) {
|
||||
function collectSkillFlags(email: string, containerPaths?: PathOverrides): string[] {
|
||||
const flags: string[] = [];
|
||||
const pairs: Array<[hostDir: string, outputDir: string]> = [
|
||||
[getGlobalSkillsDir(), containerPaths?.global ?? getGlobalSkillsDir()],
|
||||
[getUserSkillsDir(email), containerPaths?.user ?? getUserSkillsDir(email)],
|
||||
];
|
||||
|
||||
for (const [hostDir, outputDir] of pairs) {
|
||||
if (!existsSync(hostDir)) continue;
|
||||
for (const entry of readdirSync(hostDir, { withFileTypes: true })) {
|
||||
if (!entry.isDirectory()) continue;
|
||||
const skillFile = join(dir, entry.name, 'SKILL.md');
|
||||
if (existsSync(skillFile)) {
|
||||
flags.push('--skill', join(dir, entry.name));
|
||||
if (existsSync(join(hostDir, entry.name, 'SKILL.md'))) {
|
||||
flags.push('--skill', `${outputDir}/${entry.name}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -28,18 +32,19 @@ function collectSkillFlags(email: string): string[] {
|
||||
return flags;
|
||||
}
|
||||
|
||||
function collectExtensionFlags(email: string): string[] {
|
||||
function collectExtensionFlags(email: string, containerPaths?: PathOverrides): string[] {
|
||||
const flags: string[] = [];
|
||||
const dirs = [getGlobalExtensionsDir(), getUserExtensionsDir(email)];
|
||||
const pairs: Array<[hostDir: string, outputDir: string]> = [
|
||||
[getGlobalExtensionsDir(), containerPaths?.global ?? getGlobalExtensionsDir()],
|
||||
[getUserExtensionsDir(email), containerPaths?.user ?? getUserExtensionsDir(email)],
|
||||
];
|
||||
|
||||
for (const dir of dirs) {
|
||||
if (!existsSync(dir)) continue;
|
||||
const entries = readdirSync(dir, { withFileTypes: true });
|
||||
for (const entry of entries) {
|
||||
for (const [hostDir, outputDir] of pairs) {
|
||||
if (!existsSync(hostDir)) continue;
|
||||
for (const entry of readdirSync(hostDir, { withFileTypes: true })) {
|
||||
if (!entry.isDirectory()) continue;
|
||||
const entryFile = join(dir, entry.name, 'index.ts');
|
||||
if (existsSync(entryFile)) {
|
||||
flags.push('--extension', entryFile);
|
||||
if (existsSync(join(hostDir, entry.name, 'index.ts'))) {
|
||||
flags.push('--extension', `${outputDir}/${entry.name}/index.ts`);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -66,17 +71,35 @@ export async function spawnPi(
|
||||
if (sandbox) {
|
||||
const container = await ensureDockerContainer(sandbox.email, sandbox.userId, sandbox.homeDir, sandbox.username);
|
||||
const storedKeys = await readApiKeys();
|
||||
const searxng = await readSearxngConfig();
|
||||
const dockerPath = Bun.which('docker') ?? 'docker';
|
||||
const containerId = container.dockerId;
|
||||
const containerHome = `/home/${sandbox.username}`;
|
||||
const containerPiConfig = `${containerHome}/.pi/agent`;
|
||||
const piArgs = ['pi', '--mode', 'rpc', '--no-extensions', '--no-skills', '--no-prompt-templates', '--no-themes'];
|
||||
|
||||
// Collect skill/extension flags using container-side paths
|
||||
const skillFlags = collectSkillFlags(sandbox.email, {
|
||||
global: '/officer/skills',
|
||||
user: '/officer/user/skills',
|
||||
});
|
||||
const extensionFlags = collectExtensionFlags(sandbox.email, {
|
||||
global: '/officer/extensions',
|
||||
user: '/officer/user/extensions',
|
||||
});
|
||||
|
||||
const piArgs = [
|
||||
'pi', '--mode', 'rpc',
|
||||
'--no-skills', '--no-prompt-templates', '--no-themes',
|
||||
...skillFlags,
|
||||
...extensionFlags,
|
||||
];
|
||||
if (model) piArgs.push('--model', model);
|
||||
|
||||
// Build env flags: Pi config dir + all stored API keys
|
||||
const envFlags = [
|
||||
'-e', `PI_CODING_AGENT_DIR=${containerPiConfig}`,
|
||||
'-e', `HOME=${containerHome}`,
|
||||
'-e', `PI_TOOLS_DIRS=/officer/tools:/officer/user/tools`,
|
||||
'-e', `PI_SEARXNG_URL=${searxng.url}`,
|
||||
];
|
||||
for (const [key, value] of Object.entries(storedKeys)) {
|
||||
if (value?.trim()) envFlags.push('-e', `${key}=${value.trim()}`);
|
||||
@@ -96,9 +119,15 @@ export async function spawnPi(
|
||||
stderr: 'pipe',
|
||||
});
|
||||
|
||||
logger.info('Spawned Pi in container', { containerId, model });
|
||||
logger.info('Spawned Pi in container', {
|
||||
containerId,
|
||||
model,
|
||||
skills: skillFlags.filter((f) => f !== '--skill').length,
|
||||
extensions: extensionFlags.filter((f) => f !== '--extension').length,
|
||||
});
|
||||
} else {
|
||||
const storedKeys = await readApiKeys();
|
||||
const searxng = await readSearxngConfig();
|
||||
const skillFlags = collectSkillFlags(email);
|
||||
const extensionFlags = collectExtensionFlags(email);
|
||||
const args = ['pi', '--mode', 'rpc', '--no-skills', '--no-prompt-templates', '--no-themes', ...skillFlags, ...extensionFlags];
|
||||
@@ -115,7 +144,7 @@ export async function spawnPi(
|
||||
stdin: 'pipe',
|
||||
stdout: 'pipe',
|
||||
stderr: 'pipe',
|
||||
env: { ...process.env, ...storedKeys, PI_CODING_AGENT_DIR: PI_CONFIG_DIR, PI_TOOLS_DIRS: toolsDirs },
|
||||
env: { ...process.env, ...storedKeys, PI_CODING_AGENT_DIR: PI_CONFIG_DIR, PI_TOOLS_DIRS: toolsDirs, PI_SEARXNG_URL: searxng.url },
|
||||
});
|
||||
|
||||
logger.info('Spawned Pi locally', {
|
||||
|
||||
@@ -53,6 +53,10 @@ function resolveFile(name: string, native: Map<string, string>, global: Map<stri
|
||||
return null;
|
||||
}
|
||||
|
||||
function isPrivileged(role: string) {
|
||||
return role === 'Admin' || role === 'Owner' || role === 'Super Admin';
|
||||
}
|
||||
|
||||
export const processesRouter = createRouter();
|
||||
|
||||
processesRouter.get('/', async (ctx) => {
|
||||
@@ -134,6 +138,8 @@ processesRouter.put('/:name/chat', async (ctx) => {
|
||||
const resolved = resolveFile(name, nativeProcesses, globalProcesses, userProcesses);
|
||||
if (!resolved) return ctx.text('Not found', 404);
|
||||
|
||||
if (resolved.scope !== 'user' && !isPrivileged(user.role)) return ctx.text('Forbidden', 403);
|
||||
|
||||
const chatDir = join(dirname(resolved.filePath), 'chat');
|
||||
const { sessionId, messages } = await ctx.req.json<{ sessionId: string; messages: unknown[] }>();
|
||||
|
||||
@@ -155,6 +161,8 @@ processesRouter.delete('/:name/chat', async (ctx) => {
|
||||
const resolved = resolveFile(name, nativeProcesses, globalProcesses, userProcesses);
|
||||
if (!resolved) return ctx.text('Not found', 404);
|
||||
|
||||
if (resolved.scope !== 'user' && !isPrivileged(user.role)) return ctx.text('Forbidden', 403);
|
||||
|
||||
const chatDir = join(dirname(resolved.filePath), 'chat');
|
||||
await rm(chatDir, { recursive: true, force: true });
|
||||
|
||||
@@ -162,13 +170,17 @@ processesRouter.delete('/:name/chat', async (ctx) => {
|
||||
});
|
||||
|
||||
processesRouter.post('/', async (ctx) => {
|
||||
const user = ctx.get('user');
|
||||
const { name } = await ctx.req.json<{ name: string }>();
|
||||
if (!name?.trim()) return ctx.text('Name is required', 400);
|
||||
|
||||
const dirName = name.trim().toLowerCase().replace(/\s+/g, '-').replace(/[^a-z0-9-]/g, '');
|
||||
if (!dirName) return ctx.text('Invalid name', 400);
|
||||
|
||||
const dir = join(getGlobalProcessesDir(), dirName);
|
||||
const targetDir = isPrivileged(user.role) ? getGlobalProcessesDir() : getUserProcessesDir(user.email);
|
||||
const scope: Scope = isPrivileged(user.role) ? 'global' : 'user';
|
||||
|
||||
const dir = join(targetDir, dirName);
|
||||
const filePath = join(dir, 'PROCESS.md');
|
||||
|
||||
if (await Bun.file(filePath).exists()) {
|
||||
@@ -178,18 +190,22 @@ processesRouter.post('/', async (ctx) => {
|
||||
await mkdir(dir, { recursive: true });
|
||||
await Bun.write(filePath, `---\nname: ${name.trim()}\ndescription: \n---\n`);
|
||||
|
||||
return ctx.json({ name: name.trim(), dirName, filePath });
|
||||
return ctx.json({ name: name.trim(), dirName, filePath, scope });
|
||||
});
|
||||
|
||||
processesRouter.delete('/:name', async (ctx) => {
|
||||
const user = ctx.get('user');
|
||||
const name = ctx.req.param('name');
|
||||
const globalDir = join(getGlobalProcessesDir(), name);
|
||||
const globalFile = join(globalDir, 'PROCESS.md');
|
||||
|
||||
if (!(await Bun.file(globalFile).exists())) {
|
||||
return ctx.text('Not found', 404);
|
||||
}
|
||||
const nativeProcesses = await readProcessDirs(getNativeProcessesDir());
|
||||
const globalProcesses = await readProcessDirs(getGlobalProcessesDir());
|
||||
const userProcesses = await readProcessDirs(getUserProcessesDir(user.email));
|
||||
|
||||
await rm(globalDir, { recursive: true });
|
||||
const resolved = resolveFile(name, nativeProcesses, globalProcesses, userProcesses);
|
||||
if (!resolved) return ctx.text('Not found', 404);
|
||||
|
||||
if (resolved.scope !== 'user' && !isPrivileged(user.role)) return ctx.text('Forbidden', 403);
|
||||
|
||||
await rm(dirname(resolved.filePath), { recursive: true });
|
||||
return ctx.json({ ok: true });
|
||||
});
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
import { join } from 'node:path';
|
||||
import { createRouter } from '../../create-router';
|
||||
import { DATA_PATH } from '../../data-path';
|
||||
|
||||
export const searxngRouter = createRouter();
|
||||
|
||||
const SEARXNG_FILE = join(DATA_PATH, 'searxng.json');
|
||||
|
||||
const DEFAULT_URL = 'https://searxng.home.pastilhas.eu';
|
||||
|
||||
export type SearxngConfig = {
|
||||
url: string;
|
||||
};
|
||||
|
||||
export async function readSearxngConfig(): Promise<SearxngConfig> {
|
||||
try {
|
||||
const file = Bun.file(SEARXNG_FILE);
|
||||
if (!(await file.exists())) return { url: DEFAULT_URL };
|
||||
return (await file.json()) as SearxngConfig;
|
||||
} catch {
|
||||
return { url: DEFAULT_URL };
|
||||
}
|
||||
}
|
||||
|
||||
async function writeSearxngConfig(config: SearxngConfig) {
|
||||
await Bun.write(SEARXNG_FILE, JSON.stringify(config, null, 2));
|
||||
}
|
||||
|
||||
searxngRouter.get('/', async (ctx) => {
|
||||
return ctx.json(await readSearxngConfig());
|
||||
});
|
||||
|
||||
searxngRouter.put('/', async (ctx) => {
|
||||
const { url } = await ctx.req.json<{ url: string }>();
|
||||
if (!url?.trim()) return ctx.json({ error: 'URL is required' }, 400);
|
||||
const config: SearxngConfig = { url: url.trim().replace(/\/+$/, '') };
|
||||
await writeSearxngConfig(config);
|
||||
return ctx.json(config);
|
||||
});
|
||||
@@ -12,6 +12,7 @@ import { smtpRouter } from './smtp';
|
||||
import { ttsRouter } from './tts';
|
||||
import { sttRouter } from './stt';
|
||||
import { ocrRouter } from './ocr';
|
||||
import { searxngRouter } from './searxng';
|
||||
|
||||
const configDir = `${homedir()}/.config/officer.dev`;
|
||||
export const settingsPath = `${configDir}/server-settings.json`;
|
||||
@@ -33,6 +34,7 @@ serverSettingsRouter.route('/smtp', smtpRouter);
|
||||
serverSettingsRouter.route('/tts', ttsRouter);
|
||||
serverSettingsRouter.route('/stt', sttRouter);
|
||||
serverSettingsRouter.route('/ocr', ocrRouter);
|
||||
serverSettingsRouter.route('/searxng', searxngRouter);
|
||||
|
||||
const readSettings = async () => {
|
||||
try { return await Bun.file(settingsPath).json(); } catch { return {}; }
|
||||
|
||||
@@ -40,6 +40,10 @@ export async function readSkillDirs(dir: string): Promise<Map<string, string>> {
|
||||
|
||||
type Scope = 'native' | 'global' | 'user';
|
||||
|
||||
function isPrivileged(role: string) {
|
||||
return role === 'Admin' || role === 'Owner' || role === 'Super Admin';
|
||||
}
|
||||
|
||||
function resolveScope(dirName: string, native: Map<string, string>, global: Map<string, string>, user: Map<string, string>): Scope {
|
||||
if (user.has(dirName)) return 'user';
|
||||
if (global.has(dirName)) return 'global';
|
||||
@@ -134,6 +138,8 @@ skillsRouter.put('/:name/chat', async (ctx) => {
|
||||
const resolved = resolveFile(name, nativeSkills, globalSkills, userSkills);
|
||||
if (!resolved) return ctx.text('Not found', 404);
|
||||
|
||||
if (resolved.scope !== 'user' && !isPrivileged(user.role)) return ctx.text('Forbidden', 403);
|
||||
|
||||
const chatDir = join(dirname(resolved.filePath), 'chat');
|
||||
const { sessionId, messages } = await ctx.req.json<{ sessionId: string; messages: unknown[] }>();
|
||||
|
||||
@@ -155,6 +161,8 @@ skillsRouter.delete('/:name/chat', async (ctx) => {
|
||||
const resolved = resolveFile(name, nativeSkills, globalSkills, userSkills);
|
||||
if (!resolved) return ctx.text('Not found', 404);
|
||||
|
||||
if (resolved.scope !== 'user' && !isPrivileged(user.role)) return ctx.text('Forbidden', 403);
|
||||
|
||||
const chatDir = join(dirname(resolved.filePath), 'chat');
|
||||
await rm(chatDir, { recursive: true, force: true });
|
||||
|
||||
@@ -162,13 +170,18 @@ skillsRouter.delete('/:name/chat', async (ctx) => {
|
||||
});
|
||||
|
||||
skillsRouter.post('/', async (ctx) => {
|
||||
const user = ctx.get('user');
|
||||
const { name } = await ctx.req.json<{ name: string }>();
|
||||
if (!name?.trim()) return ctx.text('Name is required', 400);
|
||||
|
||||
const dirName = name.trim().toLowerCase().replace(/\s+/g, '-').replace(/[^a-z0-9-]/g, '');
|
||||
if (!dirName) return ctx.text('Invalid name', 400);
|
||||
|
||||
const dir = join(getGlobalSkillsDir(), dirName);
|
||||
// Members save to their own scope; Admins and above save to global
|
||||
const targetDir = isPrivileged(user.role) ? getGlobalSkillsDir() : getUserSkillsDir(user.email);
|
||||
const scope: Scope = isPrivileged(user.role) ? 'global' : 'user';
|
||||
|
||||
const dir = join(targetDir, dirName);
|
||||
const filePath = join(dir, 'SKILL.md');
|
||||
|
||||
if (await Bun.file(filePath).exists()) {
|
||||
@@ -178,18 +191,24 @@ skillsRouter.post('/', async (ctx) => {
|
||||
await mkdir(dir, { recursive: true });
|
||||
await Bun.write(filePath, `---\nname: ${name.trim()}\ndescription: \n---\n`);
|
||||
|
||||
return ctx.json({ name: name.trim(), dirName, filePath });
|
||||
return ctx.json({ name: name.trim(), dirName, filePath, scope });
|
||||
});
|
||||
|
||||
skillsRouter.delete('/:name', async (ctx) => {
|
||||
const user = ctx.get('user');
|
||||
const name = ctx.req.param('name');
|
||||
const globalDir = join(getGlobalSkillsDir(), name);
|
||||
const globalFile = join(globalDir, 'SKILL.md');
|
||||
|
||||
if (!(await Bun.file(globalFile).exists())) {
|
||||
return ctx.text('Not found', 404);
|
||||
}
|
||||
const nativeSkills = await readSkillDirs(getNativeSkillsDir());
|
||||
const globalSkills = await readSkillDirs(getGlobalSkillsDir());
|
||||
const userSkills = await readSkillDirs(getUserSkillsDir(user.email));
|
||||
|
||||
await rm(globalDir, { recursive: true });
|
||||
const resolved = resolveFile(name, nativeSkills, globalSkills, userSkills);
|
||||
if (!resolved) return ctx.text('Not found', 404);
|
||||
|
||||
// Members can only delete their own skills
|
||||
if (resolved.scope !== 'user' && !isPrivileged(user.role)) return ctx.text('Forbidden', 403);
|
||||
|
||||
const dir = dirname(resolved.filePath);
|
||||
await rm(dir, { recursive: true });
|
||||
return ctx.json({ ok: true });
|
||||
});
|
||||
|
||||
@@ -24,7 +24,6 @@ export function parseFrontmatter(raw: string): { frontmatter: Frontmatter; body:
|
||||
const triggers: TriggerConfig[] = [];
|
||||
const triggerMatch = yaml.match(/^trigger:\s*\n((?:[ \t]+.+\n?)*)/m);
|
||||
if (triggerMatch) {
|
||||
// Split on top-level list items (lines starting with " - type:")
|
||||
const items = triggerMatch[1]!.split(/(?=^\s+-\s*type:)/m);
|
||||
for (const item of items) {
|
||||
const type = item.match(/type:\s*(.+)/)?.[1]?.trim();
|
||||
@@ -73,6 +72,10 @@ function resolveFile(name: string, native: Map<string, string>, global: Map<stri
|
||||
return null;
|
||||
}
|
||||
|
||||
function isPrivileged(role: string) {
|
||||
return role === 'Admin' || role === 'Owner' || role === 'Super Admin';
|
||||
}
|
||||
|
||||
export const tasksRouter = createRouter();
|
||||
|
||||
tasksRouter.get('/', async (ctx) => {
|
||||
@@ -161,6 +164,8 @@ tasksRouter.put('/:name/chat', async (ctx) => {
|
||||
const resolved = resolveFile(name, nativeTasks, globalTasks, userTasks);
|
||||
if (!resolved) return ctx.text('Not found', 404);
|
||||
|
||||
if (resolved.scope !== 'user' && !isPrivileged(user.role)) return ctx.text('Forbidden', 403);
|
||||
|
||||
const chatDir = join(dirname(resolved.filePath), 'chat');
|
||||
const { sessionId, messages } = await ctx.req.json<{ sessionId: string; messages: unknown[] }>();
|
||||
|
||||
@@ -182,6 +187,8 @@ tasksRouter.delete('/:name/chat', async (ctx) => {
|
||||
const resolved = resolveFile(name, nativeTasks, globalTasks, userTasks);
|
||||
if (!resolved) return ctx.text('Not found', 404);
|
||||
|
||||
if (resolved.scope !== 'user' && !isPrivileged(user.role)) return ctx.text('Forbidden', 403);
|
||||
|
||||
const chatDir = join(dirname(resolved.filePath), 'chat');
|
||||
await rm(chatDir, { recursive: true, force: true });
|
||||
|
||||
@@ -189,13 +196,17 @@ tasksRouter.delete('/:name/chat', async (ctx) => {
|
||||
});
|
||||
|
||||
tasksRouter.post('/', async (ctx) => {
|
||||
const user = ctx.get('user');
|
||||
const { name } = await ctx.req.json<{ name: string }>();
|
||||
if (!name?.trim()) return ctx.text('Name is required', 400);
|
||||
|
||||
const dirName = name.trim().toLowerCase().replace(/\s+/g, '-').replace(/[^a-z0-9-]/g, '');
|
||||
if (!dirName) return ctx.text('Invalid name', 400);
|
||||
|
||||
const dir = join(getGlobalTasksDir(), dirName);
|
||||
const targetDir = isPrivileged(user.role) ? getGlobalTasksDir() : getUserTasksDir(user.email);
|
||||
const scope: Scope = isPrivileged(user.role) ? 'global' : 'user';
|
||||
|
||||
const dir = join(targetDir, dirName);
|
||||
const filePath = join(dir, 'TASK.md');
|
||||
|
||||
if (await Bun.file(filePath).exists()) {
|
||||
@@ -205,18 +216,22 @@ tasksRouter.post('/', async (ctx) => {
|
||||
await mkdir(dir, { recursive: true });
|
||||
await Bun.write(filePath, `---\nname: ${name.trim()}\ndescription: \n---\n`);
|
||||
|
||||
return ctx.json({ name: name.trim(), dirName, filePath });
|
||||
return ctx.json({ name: name.trim(), dirName, filePath, scope });
|
||||
});
|
||||
|
||||
tasksRouter.delete('/:name', async (ctx) => {
|
||||
const user = ctx.get('user');
|
||||
const name = ctx.req.param('name');
|
||||
const globalDir = join(getGlobalTasksDir(), name);
|
||||
const globalFile = join(globalDir, 'TASK.md');
|
||||
|
||||
if (!(await Bun.file(globalFile).exists())) {
|
||||
return ctx.text('Not found', 404);
|
||||
}
|
||||
const nativeTasks = await readTaskDirs(getNativeTasksDir());
|
||||
const globalTasks = await readTaskDirs(getGlobalTasksDir());
|
||||
const userTasks = await readTaskDirs(getUserTasksDir(user.email));
|
||||
|
||||
await rm(globalDir, { recursive: true });
|
||||
const resolved = resolveFile(name, nativeTasks, globalTasks, userTasks);
|
||||
if (!resolved) return ctx.text('Not found', 404);
|
||||
|
||||
if (resolved.scope !== 'user' && !isPrivileged(user.role)) return ctx.text('Forbidden', 403);
|
||||
|
||||
await rm(dirname(resolved.filePath), { recursive: true });
|
||||
return ctx.json({ ok: true });
|
||||
});
|
||||
|
||||
@@ -2,7 +2,7 @@ import type { ServerWebSocket } from 'bun';
|
||||
import { mkdirSync, statSync } from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { getHomeDir } from '@@/data-path';
|
||||
import { getHomeDir, getGlobalSkillsDir, getGlobalToolsDir, getGlobalExtensionsDir, getUserSkillsDir, getUserToolsDir } from '@@/data-path';
|
||||
import { syncUserPiConfig } from '@@/api/server-settings/sync-user-pi-config';
|
||||
import { getUsers } from 'officerdb';
|
||||
|
||||
@@ -101,7 +101,20 @@ const ensureDockerImage = () => {
|
||||
dockerImageReady = true;
|
||||
};
|
||||
|
||||
const startDockerSidecar = (port: number, homeDir: string, userId: number, username: string): { dockerId: string } => {
|
||||
// Check whether a container already has the officer resource mounts.
|
||||
// We test for the global skills dir as a proxy for all mounts being present.
|
||||
const containerHasResourceMounts = (dockerId: string): boolean => {
|
||||
const dockerPath = Bun.which('docker') ?? 'docker';
|
||||
const result = Bun.spawnSync({
|
||||
cmd: [dockerPath, 'inspect', '--format', '{{range .Mounts}}{{.Source}}\n{{end}}', dockerId],
|
||||
stdout: 'pipe',
|
||||
stderr: 'ignore',
|
||||
});
|
||||
if (result.exitCode !== 0) return false;
|
||||
return result.stdout.toString().includes(getGlobalSkillsDir());
|
||||
};
|
||||
|
||||
const startDockerSidecar = (port: number, homeDir: string, userId: number, username: string, email: string): { dockerId: string } => {
|
||||
ensureDockerImage();
|
||||
const dockerPath = Bun.which('docker') ?? 'docker';
|
||||
const dockerId = `officer-terminal-${userId}`;
|
||||
@@ -144,10 +157,13 @@ const startDockerSidecar = (port: number, homeDir: string, userId: number, usern
|
||||
`TERMINAL_UID=${uid}`,
|
||||
'-e',
|
||||
`TERMINAL_GID=${gid}`,
|
||||
'-v',
|
||||
`${homeDir}:${containerHome}`,
|
||||
'-w',
|
||||
containerHome,
|
||||
'-v', `${homeDir}:${containerHome}`,
|
||||
'-v', `${getGlobalSkillsDir()}:/officer/skills:ro`,
|
||||
'-v', `${getGlobalToolsDir()}:/officer/tools:ro`,
|
||||
'-v', `${getGlobalExtensionsDir()}:/officer/extensions:ro`,
|
||||
'-v', `${getUserSkillsDir(email)}:/officer/user/skills:ro`,
|
||||
'-v', `${getUserToolsDir(email)}:/officer/user/tools:ro`,
|
||||
'-w', containerHome,
|
||||
tag,
|
||||
],
|
||||
stdout: 'inherit',
|
||||
@@ -211,17 +227,35 @@ const dockerStart = (dockerId: string) => {
|
||||
};
|
||||
|
||||
export const ensureDockerContainer = async (email: string, userId: number, homeDir: string, username: string) => {
|
||||
// Ensure user-specific resource dirs exist before mounting (Docker creates them as root if missing)
|
||||
mkdirSync(getUserSkillsDir(email), { recursive: true });
|
||||
mkdirSync(getUserToolsDir(email), { recursive: true });
|
||||
|
||||
const map = await loadContainerMap();
|
||||
const existing = map[email];
|
||||
if (existing && dockerContainerRunning(existing.dockerId)) return existing;
|
||||
|
||||
if (existing && dockerContainerRunning(existing.dockerId)) {
|
||||
// Recreate if resource mounts are missing (e.g. first run after feature was added)
|
||||
if (!containerHasResourceMounts(existing.dockerId)) {
|
||||
console.log(`[terminal] recreating container for ${email} — resource mounts missing`);
|
||||
stopDockerSidecar(existing.dockerId);
|
||||
} else {
|
||||
return existing;
|
||||
}
|
||||
}
|
||||
|
||||
if (existing && dockerContainerExists(existing.dockerId)) {
|
||||
if (dockerStart(existing.dockerId)) return existing;
|
||||
if (!containerHasResourceMounts(existing.dockerId)) {
|
||||
stopDockerSidecar(existing.dockerId);
|
||||
} else if (dockerStart(existing.dockerId)) {
|
||||
return existing;
|
||||
} else {
|
||||
stopDockerSidecar(existing.dockerId);
|
||||
}
|
||||
}
|
||||
|
||||
const port = existing?.port ?? getAvailablePort(map, userId);
|
||||
const docker = startDockerSidecar(port, homeDir, userId, username);
|
||||
const docker = startDockerSidecar(port, homeDir, userId, username, email);
|
||||
const next = { userId, email, dockerId: docker.dockerId, port };
|
||||
map[email] = next;
|
||||
await saveContainerMap(map);
|
||||
|
||||
@@ -4,6 +4,30 @@ import rehypeRaw from 'rehype-raw';
|
||||
import type { ChatMessage } from '../types';
|
||||
import { ToolActivity } from './ToolActivity';
|
||||
import { QuestionActivity } from './QuestionActivity';
|
||||
import { getRawUrl } from '../../FileViewer/file-types';
|
||||
|
||||
// Matches absolute image file paths, e.g. /home/user/pic.png or /tmp/photo.jpg
|
||||
const IMAGE_PATH_RE = /(\/(?:home\/[^/\s]+\/)?[^\s`"'<>\n\r[\]()]+\.(?:png|jpg|jpeg|gif|webp|svg|bmp|ico))/gi;
|
||||
|
||||
function pathToImageUrl(filePath: string): string {
|
||||
// Strip /home/{username} prefix — the file browser root=home serves from the user's home dir
|
||||
const relative = filePath.replace(/^\/home\/[^/]+/, '') || '/';
|
||||
return getRawUrl(relative, 'home');
|
||||
}
|
||||
|
||||
function injectImages(text: string): string {
|
||||
// Split on fenced code blocks and inline code so we don't touch paths inside backticks
|
||||
const parts = text.split(/(```[\s\S]*?```|`[^`\n]+`)/g);
|
||||
return parts
|
||||
.map((part, i) => {
|
||||
if (i % 2 === 1) return part; // inside code — leave untouched
|
||||
return part.replace(IMAGE_PATH_RE, (match) => {
|
||||
const filename = match.split('/').pop() ?? 'image';
|
||||
return `})`;
|
||||
});
|
||||
})
|
||||
.join('');
|
||||
}
|
||||
|
||||
const FRONTMATTER_RE = /^<frontmatter>([\s\S]*?)<\/frontmatter>\s*/;
|
||||
|
||||
@@ -56,7 +80,7 @@ export const MessageBubble = ({ message, onAnswer }: MessageBubbleProps) => {
|
||||
<div className="max-w-[85%] rounded-2xl rounded-tl-sm bg-muted/50 border border-border/50 px-4 py-2.5">
|
||||
<div className="chat-md">
|
||||
<ReactMarkdown remarkPlugins={[remarkGfm]} rehypePlugins={[rehypeRaw]}>
|
||||
{message.text}
|
||||
{injectImages(message.text)}
|
||||
</ReactMarkdown>
|
||||
</div>
|
||||
</div>
|
||||
@@ -100,7 +124,7 @@ export const StreamingBubble = ({ text }: StreamingBubbleProps) => {
|
||||
<div className="max-w-[85%] rounded-2xl rounded-tl-sm bg-muted/50 border border-border/50 px-4 py-2.5">
|
||||
<div className="chat-md">
|
||||
<ReactMarkdown remarkPlugins={[remarkGfm]} rehypePlugins={[rehypeRaw]}>
|
||||
{text}
|
||||
{injectImages(text)}
|
||||
</ReactMarkdown>
|
||||
<span className="inline-block w-2 h-4 bg-duck-teal/60 animate-pulse ml-0.5 align-middle" />
|
||||
</div>
|
||||
|
||||
Reference in New Issue
Block a user