apify tool, tools API + automation UI, integrations config, super admin restrictions

- apify tool: TOOL.md definition, index.ts implementation with auto-auth via OFFICER_APIFY_TOKEN, output_path for large datasets - tools API: /tools routes (list, detail, chat, create, delete) mirroring tasks pattern - automation UI: tools tab in sidebar, NewTool component, tool detail view - apify integration: settings page for enterprise API key config, pi-bridge passes env var to containers - tiktok-trends task: rewritten as agent instructions using apify tool with output_path, scripted report generation for 50KB read limit - restrict edit/delete of native/global capabilities to Super Admin only (backend + frontend) - tools authoring guide: TOOLS.md with full spec for TOOL.md frontmatter, index.ts execute signature, patterns Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 20:00:45 +00:00
parent 776738a983
commit bd362dc586
19 changed files with 1059 additions and 149 deletions
@@ -0,0 +1,179 @@
+# Tools
+
+A tool is a callable capability that agents can use during task execution. Each tool lives in its own directory under `tools/` and is defined by a `TOOL.md` file and an `index.ts` (or `index.js`) entry point.
+
+Tools are loaded by the `tool-loader` extension at startup and registered as callable functions. They appear in the agent's system prompt and can be invoked by name.
+
+## File Structure
+
+```
+tools/
+  <tool-name>/
+    TOOL.md       # Metadata and documentation
+    index.ts      # Implementation (required)
+```
+
+Both files are required. The loader skips directories missing either `TOOL.md` or an entry file.
+
+## TOOL.md Format
+
+A tool file has two parts: **frontmatter** (YAML metadata) and **body** (Markdown documentation).
+
+### Frontmatter
+
+```yaml
+---
+name: tool_name
+label: Tool Name
+description: What the tool does and when the agent should use it.
+version: 1
+language: typescript
+inputs:
+  param_name:
+    type: string
+    description: What this parameter is for.
+  optional_param:
+    type: number
+    description: An optional parameter.
+    optional: true
+  secret_param:
+    type: string
+    description: A sensitive value (e.g., API token).
+    optional: true
+    sensitive: true
+  mode:
+    type: enum
+    values: single,batch
+    description: Choose between modes.
+---
+```
+
+#### Fields
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `name` | string | yes | Identifier for the tool (used in tool calls). Use `snake_case`. |
+| `label` | string | no | Human-readable display name. Defaults to `name` if omitted. |
+| `description` | string | yes | What the tool does. This appears in the agent's system prompt — make it specific enough for the agent to know when to use it. |
+| `version` | integer | no | Version number. Used by `sync-tools` to detect updates — bump when changing the tool. |
+| `language` | string | no | Implementation language: `typescript`, `bash`, or `python`. Defaults to `typescript`. |
+| `inputs` | object | no | Input parameters the tool accepts. Keys are parameter names. |
+
+#### Input Fields
+
+Each input is a key under `inputs:` with these properties:
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `type` | string | yes | Parameter type: `string`, `number`, `boolean`, or `enum`. |
+| `description` | string | yes | What this parameter is for. Shown to the agent. |
+| `optional` | boolean | no | Whether the parameter is optional. Defaults to required. |
+| `sensitive` | boolean | no | Mark sensitive values (tokens, passwords). Prevents logging. |
+| `values` | string | no | Comma-separated allowed values when `type` is `enum`. |
+| `default` | any | no | Default value if not provided. |
+
+### Body
+
+The body is Markdown documentation that the agent sees when the tool is loaded. Include:
+
+- **Title** — `# Tool Name`
+- **Authentication** — How credentials are resolved (env vars, integrations, etc.)
+- **Usage** — How to call the tool and what parameters to pass.
+- **Examples** — Common usage patterns.
+- **Error Handling** — What errors can occur and what they mean.
+- **Notes** — Limits, billing, external links.
+
+## index.ts Format
+
+The entry file must export an `execute` function:
+
+```typescript
+type ToolResult = {
+  content: Array<{ type: string; text: string }>;
+  isError?: boolean;
+};
+
+type OnUpdate = (partial: { content: Array<{ type: string; text: string }> }) => void;
+
+export async function execute(
+  toolCallId: string,
+  params: Record<string, unknown>,
+  signal: AbortSignal | undefined,
+  onUpdate?: OnUpdate,
+): Promise<ToolResult> {
+  // Implementation here
+}
+```
+
+### Parameters
+
+| Parameter | Description |
+|-----------|-------------|
+| `toolCallId` | Unique ID for this tool call. |
+| `params` | Input values from the agent, matching the `inputs` defined in TOOL.md. |
+| `signal` | AbortSignal for cancellation. |
+| `onUpdate` | Callback for streaming progress updates to the agent during long operations. |
+
+### Return Value
+
+Return a `ToolResult` object:
+- `content` — Array of content blocks. Usually one `{ type: 'text', text: '...' }`.
+- `isError` — Set `true` to indicate failure. The agent sees the error and can react.
+
+### Progress Updates
+
+Use `onUpdate` to stream status during long-running operations:
+
+```typescript
+onUpdate?.({ content: [{ type: 'text', text: 'Processing step 2 of 5...' }] });
+```
+
+### Authentication
+
+Tools should resolve credentials internally, not require the agent to pass them. Pattern:
+
+1. Check for an explicit parameter override (e.g., `params.api_token`)
+2. Fall back to an environment variable (e.g., `process.env.OFFICER_APIFY_TOKEN`)
+3. Return a helpful error if neither is available
+
+Environment variables are set by `pi-bridge.ts` from the integration config stored in the database (Settings → Integrations).
+
+### Runtime Environment
+
+Tools run inside sandboxed containers using **Node.js** (not Bun). Do not use Bun-specific APIs like `Bun.sleep`, `Bun.file`, etc. Use Node.js equivalents:
+
+- `setTimeout` / `setInterval` for delays
+- `fs.readFileSync` / `fs.writeFileSync` for file I/O
+- `fetch` (available in Node 18+) for HTTP requests
+
+### Large Output
+
+If a tool may return large data (e.g., API responses with many items), provide an `output_path` parameter. When set, save the data to the file and return a summary instead:
+
+```typescript
+if (params.output_path) {
+  writeFileSync(params.output_path, JSON.stringify(items, null, 2));
+  return { content: [{ type: 'text', text: `${items.length} items saved to ${params.output_path}` }] };
+}
+```
+
+This prevents flooding the agent's context window with raw data.
+
+## Sync and Discovery
+
+Tools are synced from `seed/tools/` to `DATA_PATH/tools/` at server startup by `sync-tools.ts`. The sync is version-based — it only overwrites when the seed version is higher than the target version. Always bump `version` in the frontmatter when updating a tool.
+
+The `tool-loader` extension discovers tools from directories listed in the `PI_TOOLS_DIRS` environment variable (colon-separated). Only `DATA_PATH/tools/` is mounted into containers — `seed/tools/` is not directly accessible at runtime.
+
+## Existing Tools
+
+| Tool | Description |
+|------|-------------|
+| `gmail` | Read Gmail messages, threads, labels via Google API |
+| `web_search` | Search the web via SearXNG |
+| `web_fetch` | Fetch and extract content from URLs |
+| `browser` | Control a Chrome browser via Browser Relay |
+| `apify` | Run any Apify actor (web scraping, social media data) |
+| `convert_audio_to_mp3` | Convert audio files to MP3 via ffmpeg |
+| `ocr` | Optical character recognition on images |
+| `email_db` | Query the synced email database |