tools and skills, etc in the containers

2026-02-24 01:24:28 +00:00
parent 071e2decc3
commit d6ffe43a11
16 changed files with 996 additions and 66 deletions
@@ -0,0 +1,293 @@
+# Automation System — Context Document
+
+This document is written for AI assistants working on this codebase. It captures architectural decisions, the full vision, and the current state of the automation/tools system built during a long design and implementation session. Read this before touching anything in `src/servers/api/pi/`, `src/servers/api/skills/`, `src/servers/api/tasks/`, `src/servers/api/processes/`, `seed/tools/`, `seed/extensions/`, or `src/servers/sync-*.ts`.
+
+---
+
+## What Officer Is
+
+Officer is a self-hosted AI-native intranet for small and medium businesses. Every user gets a personal AI assistant, terminal, file browser, code editor, workspaces, and automation tools. The platform is multi-user with role-based access: `Member → Admin → Owner → Super Admin`.
+
+The AI backend was originally Claude SDK / OpenCode. It has been migrated to **Pi** (`@mariozechner/pi-coding-agent`) as the primary agent harness, running in RPC mode as a subprocess managed by `src/servers/api/pi/pi-bridge.ts`.
+
+---
+
+## The Automation Ladder
+
+The owner has a clear conceptual model for automation capabilities, ordered from atomic to orchestrated:
+
+| Rung | Description | Status |
+|------|-------------|--------|
+| **Skills** | Passive markdown documentation the agent reads as context. Reference for CLIs, APIs, services. | ✅ Implemented |
+| **Tools** | Executable TypeScript functions Pi can call directly during agent execution. Registered via Pi extensions. | ✅ Implemented |
+| **Tasks** | Structured instructions (TASK.md) to accomplish an atomic goal. Uses skills + tools. Has inputs, triggers, steps. | ✅ Skeleton exists |
+| **Pipelines** | A linear sequence of tasks, run one after the other (think pipe). | 🔲 Phase 2 |
+| **Processes** | A sequence of tasks that forks based on the result of the previous one (conditional branching). | 🔲 Phase 2 |
+| **Workflows** | Arbitrary task graphs, like n8n. | 🔲 Phase 3 |
+| **Services** | Event-driven execution — watches directories or events, triggers tasks based on configuration. | 🔲 Phase 3 |
+| **Crons** | Scheduled execution of a single task, pipeline, process, or workflow. | 🔲 Phase 3 |
+
+**When the owner says "tool", they mean any rung of this ladder**, not just the Pi tool concept.
+
+### Key distinction: agent-interpreted vs headless execution
+
+Currently all tasks are **agent-interpreted** — the agent reads the TASK.md and reasons about how to execute it. In phase 2, well-defined tasks (deterministic inputs → tool call → result) should be executable **headlessly** without an LLM, directly by a task runner. The tool implementations are already standalone TypeScript functions that support this future — they have no dependency on Pi or the agent.
+
+---
+
+## How Pi Is Integrated
+
+Pi runs as a subprocess in **RPC mode** (`pi --mode rpc`). The server communicates via stdin/stdout JSON. See `src/servers/api/pi/pi-bridge.ts` and `src/servers/api/pi/websocket.ts`.
+
+### Spawning Pi (host, non-sandboxed)
+
+```
+pi --mode rpc
+   --no-skills --no-prompt-templates --no-themes
+   --skill /data/skills/{name}           (one per global skill)
+   --skill /data/{email}/skills/{name}   (one per user skill)
+   --extension /data/extensions/{name}/index.ts
+   --model {model}
+
+env:
+  PI_CODING_AGENT_DIR = DATA_PATH/pi-config
+  PI_TOOLS_DIRS       = DATA_PATH/tools:DATA_PATH/{email}/tools
+  PI_SEARXNG_URL      = https://searxng.home.pastilhas.eu
+  + all stored API keys
+```
+
+Note: `--no-extensions` was intentionally removed to allow our tool-loader extension to work. `--no-skills` is kept because we pass skills explicitly via `--skill` flags to control scope correctly.
+
+### Spawning Pi (sandboxed, Docker container)
+
+Same pattern but via `docker exec`, using container-side paths:
+
+```
+docker exec -i -w {workdir} \
+  -e PI_CODING_AGENT_DIR=/home/{username}/.pi/agent \
+  -e PI_TOOLS_DIRS=/officer/tools:/officer/user/tools \
+  -e PI_SEARXNG_URL=... \
+  -e {API_KEYS} \
+  {containerId} \
+  pi --mode rpc --no-skills --no-prompt-templates --no-themes \
+     --skill /officer/skills/{name} \
+     --skill /officer/user/skills/{name} \
+     --extension /officer/extensions/tool-loader/index.ts
+```
+
+---
+
+## Directory Structure
+
+```
+DATA_PATH/                          # Default: ./data, override with DATA_PATH env
+├── pi-config/                      # Pi's global config (models.json, settings.json)
+├── skills/                         # Global skills (org-wide)
+├── tools/                          # Global tools (org-wide)
+├── extensions/                     # Global extensions (managed by server, not users)
+├── {email}/
+│   ├── home/                       # Mounted as /home/{username} in Docker container
+│   ├── skills/                     # User-specific skills
+│   ├── tools/                      # User-specific tools
+│   └── extensions/                 # User-specific extensions (future)
+└── searxng.json                    # SearXNG instance URL config
+
+seed/                               # Bundled with the app (read-only source of truth)
+├── skills/                         # Native/officerdev skills
+├── tools/                          # Native/officerdev tools
+├── extensions/                     # Native/officerdev extensions
+│   └── tool-loader/index.ts        # The Pi extension that registers tools
+└── tasks/                          # Native/officerdev tasks
+```
+
+### Sync on server start (`src/servers/bootstrap.ts`)
+
+- `syncSeedSkills()` → copies `seed/skills/*` → `DATA_PATH/skills/` (skip if exists, preserves user edits)
+- `syncSeedTools()` → copies `seed/tools/*` → `DATA_PATH/tools/` (skip if exists)
+- `syncSeedExtensions()` → copies `seed/extensions/*` → `DATA_PATH/extensions/` (**always overwrites** — extensions are server-managed code, not user-editable)
+
+---
+
+## The Tool System
+
+### How tools are defined
+
+Each tool lives in a directory with two files:
+
+```
+seed/tools/web-fetch/
+├── TOOL.md      # Metadata + input schema (read by tool-loader, registered with Pi)
+└── index.ts     # Implementation (lazy-loaded when the tool is actually called)
+```
+
+**TOOL.md frontmatter format:**
+```yaml
+---
+name: tool_name          # Pi tool name (snake_case)
+label: Tool Label        # Human-readable label
+description: ...         # What Pi sees in its context (keep concise — this is in every session prompt)
+language: typescript     # typescript | bash | python
+inputs:
+  param_name:
+    type: string         # string | number | boolean | enum
+    description: ...
+    optional: true       # omit if required
+  mode:
+    type: enum
+    values: single,batch # comma-separated for enum (parser limitation)
+    description: ...
+---
+```
+
+**index.ts export signature:**
+```typescript
+export async function execute(
+  toolCallId: string,
+  params: { [key: string]: any },
+  signal: AbortSignal | undefined,
+  onUpdate?: (partial: { content: Array<{ type: string; text: string }> }) => void,
+): Promise<{ content: Array<{ type: string; text: string }>; details?: object; isError?: boolean }>
+```
+
+The `onUpdate` callback streams progress to the agent. Use it liberally — the frontend shows it in real time.
+
+### How the tool-loader extension works (`seed/extensions/tool-loader/index.ts`)
+
+- Loaded by Pi as an extension via `--extension` flag
+- Reads `PI_TOOLS_DIRS` env var (colon-separated list of directories)
+- Discovers `TOOL.md` in each directory, parses frontmatter, builds TypeBox schema
+- Registers each tool synchronously in the extension factory function (so tools appear in the system prompt)
+- Lazy-loads `index.ts` implementation only when the tool is actually called
+- User dirs come after global — later registration wins (user tools override global by name)
+
+### Existing tools
+
+| Tool | Location | Description |
+|------|----------|-------------|
+| `web_fetch` | `seed/tools/web-fetch/` | Fetch URL content. Tries `.md` suffix → `/llms.txt` → plain text → HTML strip |
+| `web_search` | `seed/tools/web-search/` | Search via SearXNG. Returns titles, URLs, snippets. Pair with web_fetch |
+| `convert_audio_to_mp3` | `seed/tools/convert-audio-to-mp3/` | Convert audio to MP3 via ffmpeg. Single file (% progress) or batch directory (per-file progress) |
+
+### Adding a new tool
+
+1. Create `seed/tools/{name}/TOOL.md` and `seed/tools/{name}/index.ts`
+2. Restart server — `syncSeedTools()` copies it to `DATA_PATH/tools/`
+3. New Pi sessions automatically get the tool registered
+
+---
+
+## Docker Container Setup
+
+Each user has a persistent Docker container (`officer-terminal-{userId}`). Containers are managed by `src/servers/api/terminal/websocket.ts`.
+
+### Read-only resource mounts
+
+Added to every container at creation time:
+
+```
+/officer/skills          → DATA_PATH/skills           (global skills, ro)
+/officer/tools           → DATA_PATH/tools            (global tools, ro)
+/officer/extensions      → DATA_PATH/extensions       (global extensions, ro)
+/officer/user/skills     → DATA_PATH/{email}/skills   (user skills, ro)
+/officer/user/tools      → DATA_PATH/{email}/tools    (user tools, ro)
+```
+
+The user's home directory (`DATA_PATH/{email}/home`) is mounted read-write as `/home/{username}`.
+
+### Mount migration
+
+`ensureDockerContainer` checks `containerHasResourceMounts()` before reusing an existing container. If the mounts are missing (old container created before this feature), the container is removed and recreated automatically. This is a one-time migration.
+
+---
+
+## Scope & Permission Model
+
+All automation entities (skills, tasks, processes, and future rungs) follow the same three-tier scope model:
+
+| Scope | Location | Who creates | Who can see |
+|-------|----------|-------------|-------------|
+| `native` | `seed/` (read-only) | officerdev (us) | Everyone |
+| `global` | `DATA_PATH/{type}/` | Admins+ | Everyone |
+| `user` | `DATA_PATH/{email}/{type}/` | Anyone | Owner + SAs (future) |
+
+### Current API behavior (skills, tasks, processes)
+
+```typescript
+function isPrivileged(role: string) {
+  return role === 'Admin' || role === 'Owner' || role === 'Super Admin';
+}
+```
+
+- `POST /` (create): Members → user scope. Admins+ → global scope.
+- `DELETE /:name`: Members can only delete their own (user scope). Admins+ can delete global.
+- `PUT /:name/chat` / `DELETE /:name/chat`: Members blocked from writing to native/global entries.
+
+### Full vision (phase 2/3 — not yet implemented)
+
+- **Super Admins** can see all users' personal items (currently everyone only sees their own)
+- **Members can submit** their personal item for SA review (`status: draft | submitted | approved | rejected` — a flag in frontmatter, not a new scope)
+- **SAs can promote** any user's item to global — this is a **move** (not a copy), item leaves user scope
+- **SAs can see and use** everyone's tools
+- The `CapabilitySummary` type already has `scope` surfaced in the frontend list with a badge
+
+---
+
+## SearXNG Integration
+
+- Config stored in `DATA_PATH/searxng.json`
+- Default URL: `https://searxng.home.pastilhas.eu` (hardcoded default in `src/servers/api/server-settings/searxng.ts`)
+- API: `GET /api/server-settings/searxng`, `PUT /api/server-settings/searxng`
+- Injected into Pi processes as `PI_SEARXNG_URL` env var (both host and container)
+- UI configuration is a future task — the API is already there
+
+---
+
+## Attribution & Marketplace Vision (future)
+
+The owner has a clear attribution model:
+
+- **Native/officerdev**: `officerdev/<name>` — built-in, shipped with the product
+- **User-created org tools**: attribution via `author: <email>` in frontmatter
+- **Marketplace — officerdev official**: `officerdev/<name>`
+- **Marketplace — third party**: `<dev>/<name>` or `<organization>/<name>`
+
+Future marketplace will cover: Skills, Tools, Tasks, Pipelines, Processes, Workflows, Services, Crons, **Widgets**, **Apps**, and **Themes**. Widgets and Apps are already bootstrapped in the platform. Each self-hosted instance will connect to a central webstore.
+
+The `native` scope covers both built-in and future marketplace-installed items. When marketplace is implemented, `native` may split into `native` (local seed) and `marketplace` (installed from store, with version + source metadata).
+
+---
+
+## Files Changed / Created
+
+### New files
+- `src/servers/sync-tools.ts` — mirrors sync-skills pattern for tools
+- `src/servers/sync-extensions.ts` — same for extensions (always overwrites)
+- `src/servers/api/server-settings/searxng.ts` — SearXNG config router
+- `seed/tools/web-fetch/TOOL.md` + `index.ts`
+- `seed/tools/web-search/TOOL.md` + `index.ts`
+- `seed/tools/convert-audio-to-mp3/TOOL.md` + `index.ts`
+- `seed/extensions/tool-loader/index.ts`
+
+### Modified files
+- `src/servers/data-path.ts` — added tool/extension dir helpers
+- `src/servers/bootstrap.ts` — calls syncSeedTools, syncSeedExtensions
+- `src/servers/api/pi/pi-bridge.ts` — removed `--no-extensions`, added extension/tool flags, PI_TOOLS_DIRS, PI_SEARXNG_URL, container-aware path support
+- `src/servers/api/terminal/websocket.ts` — added 5 read-only resource mounts to containers, mount migration check
+- `src/servers/api/skills/skills.ts` — fixed scope-aware POST/DELETE/chat endpoints
+- `src/servers/api/tasks/tasks.ts` — same fixes
+- `src/servers/api/processes/processes.ts` — same fixes
+- `src/servers/api/server-settings/server-settings.ts` — registered searxng router
+- `seed/skills/convert-audio-to-mp3/SKILL.md` — removed hardcoded paths, references tool instead
+- `seed/tasks/convert-to-mp3/TASK.md` — updated to use tool, added `tools:` frontmatter field
+- `src/apps/officer-web/Screens/Dashboard/CapabilityPage.tsx` — fixed `scope` type to include `'native'`
+
+---
+
+## What To Do Next (Rough Priority)
+
+1. **Wire `tool_execution_update` events** through `pi-bridge.ts` → websocket → frontend so `onUpdate` progress actually shows in the chat UI
+2. **Container support testing** — verify tools and extensions work correctly inside Docker after the read-only mount changes
+3. **SA cross-user visibility** — API + UI for Super Admins to see all users' personal items
+4. **Submit/approve workflow** — `status` flag in frontmatter, submission UI for members, review UI for SAs
+5. **SearXNG UI** — settings panel to configure the URL (API is already done)
+6. **Headless task runner** — execute deterministic tasks without an agent (phase 2)