apify tool, tools API + automation UI, integrations config, super admin restrictions
- apify tool: TOOL.md definition, index.ts implementation with auto-auth via OFFICER_APIFY_TOKEN, output_path for large datasets - tools API: /tools routes (list, detail, chat, create, delete) mirroring tasks pattern - automation UI: tools tab in sidebar, NewTool component, tool detail view - apify integration: settings page for enterprise API key config, pi-bridge passes env var to containers - tiktok-trends task: rewritten as agent instructions using apify tool with output_path, scripted report generation for 50KB read limit - restrict edit/delete of native/global capabilities to Super Admin only (backend + frontend) - tools authoring guide: TOOLS.md with full spec for TOOL.md frontmatter, index.ts execute signature, patterns Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.6
+142
-129
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
name: TikTok Trends
|
name: TikTok Trends
|
||||||
description: Fetch top trending TikTok videos for a given country and generate an engagement report with optional video downloads.
|
description: Fetch top trending TikTok videos for a given country and generate an engagement report with optional video downloads.
|
||||||
version: 2
|
version: 3
|
||||||
author: pastilhas
|
author: pastilhas
|
||||||
tags:
|
tags:
|
||||||
- social-media
|
- social-media
|
||||||
@@ -9,17 +9,9 @@ tags:
|
|||||||
- trends
|
- trends
|
||||||
- apify
|
- apify
|
||||||
- content-analysis
|
- content-analysis
|
||||||
skills:
|
tools:
|
||||||
- Apify
|
- apify
|
||||||
dependencies:
|
dependencies:
|
||||||
- name: tiktok-trends-script
|
|
||||||
description: The main script at $OFFICER_USER_ROOT/../resources/scripts/apify/tiktok-trends.ts
|
|
||||||
check_command: test -f "$OFFICER_USER_ROOT/../resources/scripts/apify/tiktok-trends.ts"
|
|
||||||
optional: false
|
|
||||||
- name: bun
|
|
||||||
description: Required to run the TypeScript script
|
|
||||||
check_command: bun --version
|
|
||||||
optional: false
|
|
||||||
- name: yt-dlp
|
- name: yt-dlp
|
||||||
description: Required only when download option is enabled. Downloads TikTok videos.
|
description: Required only when download option is enabled. Downloads TikTok videos.
|
||||||
check_command: yt-dlp --version
|
check_command: yt-dlp --version
|
||||||
@@ -103,26 +95,15 @@ inputs:
|
|||||||
type: boolean
|
type: boolean
|
||||||
default: false
|
default: false
|
||||||
required: false
|
required: false
|
||||||
- name: api_token
|
|
||||||
description: Apify API token with access to actors. Get yours at https://console.apify.com/account/integrations
|
|
||||||
type: string
|
|
||||||
required: true
|
|
||||||
sensitive: true
|
|
||||||
outputs:
|
outputs:
|
||||||
- name: engagement_report
|
- name: engagement_report
|
||||||
description: Markdown report with trending videos analysis, engagement metrics, top hashtags, sounds, and creators.
|
description: Markdown report with trending videos analysis, engagement metrics, top hashtags, sounds, and creators.
|
||||||
path: tiktok_trends_<country>_<timestamp>/report-YYYY-MM-DD.md
|
path: tiktok_trends_<country>_<timestamp>/report.md
|
||||||
- name: raw_data
|
- name: raw_data
|
||||||
description: Raw JSON data from Apify actor containing all video metadata.
|
description: Raw JSON data from Apify actor containing all video metadata.
|
||||||
path: tiktok_trends_<country>_<timestamp>/raw-YYYY-MM-DD.json
|
path: tiktok_trends_<country>_<timestamp>/raw.json
|
||||||
- name: execution_log
|
|
||||||
description: Complete script execution log with stdout and stderr for debugging.
|
|
||||||
path: tiktok_trends_<country>_<timestamp>/run.log
|
|
||||||
- name: summary
|
|
||||||
description: Human-readable summary of the run including cost analysis and file listing.
|
|
||||||
path: tiktok_trends_<country>_<timestamp>/report.md
|
|
||||||
- name: videos
|
- name: videos
|
||||||
description: Downloaded video files with metadata (only present if download option was enabled).
|
description: Downloaded video files (only present if download option was enabled).
|
||||||
path: tiktok_trends_<country>_<timestamp>/videos/
|
path: tiktok_trends_<country>_<timestamp>/videos/
|
||||||
optional: true
|
optional: true
|
||||||
config:
|
config:
|
||||||
@@ -134,125 +115,157 @@ config:
|
|||||||
|
|
||||||
Fetch top trending TikTok videos for a given country and generate a comprehensive engagement report.
|
Fetch top trending TikTok videos for a given country and generate a comprehensive engagement report.
|
||||||
|
|
||||||
## Overview
|
## Important
|
||||||
|
|
||||||
This task uses the Apify `novi~fast-tiktok-scraper` actor to fetch trending videos from TikTok for a specified country. It generates:
|
- Use the `apify` tool to fetch data. Do NOT call the Apify REST API directly via curl or fetch.
|
||||||
- **Engagement metrics** (total/average views, likes, shares, comments)
|
- Do NOT explore or list files before starting. Create the output directory, call the tool, process results.
|
||||||
- **Top hashtags** found in trending content
|
- If `download` is false (default), do NOT attempt to download any videos.
|
||||||
- **Popular sounds** being used
|
- Large data files (like `raw.json`) exceed the 50KB read limit. Never try to read them directly. Instead, write a Node.js script to process and transform the data, execute it, then delete the script.
|
||||||
- **Creator analysis** (who appears multiple times in trending)
|
|
||||||
- **Complete video list** with links and stats
|
|
||||||
|
|
||||||
Optionally downloads videos using yt-dlp for offline analysis.
|
## Steps
|
||||||
|
|
||||||
## Pre-execution Checks
|
### 1. Setup output directory
|
||||||
|
|
||||||
1. **Validate inputs**:
|
Create a timestamped output directory:
|
||||||
- `api_token` must be provided (required)
|
```
|
||||||
- `country` must be a valid 2-letter country code
|
$HOME/tiktok-trends/tiktok_trends_<country>_<YYYYMMDD_HHMMSS>/
|
||||||
- `limit` must be between 1-100
|
|
||||||
|
|
||||||
2. **Check dependencies**:
|
|
||||||
- Verify script exists at `$OFFICER_USER_ROOT/../resources/scripts/apify/tiktok-trends.ts`
|
|
||||||
- Verify bun is installed
|
|
||||||
- If `download=true`, verify yt-dlp is installed
|
|
||||||
|
|
||||||
3. **Test Apify token**:
|
|
||||||
- Make a test call to `GET https://api.apify.com/v2/users/me` to validate the token
|
|
||||||
- Abort with clear error if token is invalid or account has no credits
|
|
||||||
|
|
||||||
## Execution
|
|
||||||
|
|
||||||
Run the script with the following command:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bun "$OFFICER_USER_ROOT/../resources/scripts/apify/tiktok-trends.ts" \
|
|
||||||
--apikey "<api_token>" \
|
|
||||||
--country "<country>" \
|
|
||||||
--limit <limit> \
|
|
||||||
--output "<output_dir>" \
|
|
||||||
<download_flag>
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Where:
|
### 2. Fetch trending videos
|
||||||
- `<download_flag>` is `--download` if `download=true`, otherwise omitted
|
|
||||||
- `<output_dir>` is a timestamped directory created inside `$HOME/tiktok-trends/` (the logged-in user's home directory)
|
|
||||||
|
|
||||||
**Pipe output to log**:
|
Call the `apify` tool with `output_path` pointing to `raw.json` in the output directory:
|
||||||
```bash
|
```
|
||||||
<command_above> 2>&1 | tee "<output_dir>/run.log"
|
apify(
|
||||||
|
actor_id: "novi~fast-tiktok-scraper",
|
||||||
|
input: { "type": "TREND", "region": "<country>", "maxItems": <limit> },
|
||||||
|
output_path: "<output_dir>/raw.json"
|
||||||
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Success Criteria
|
The tool saves the full dataset to `raw.json` and returns a summary (item count). If it returns an error or 0 items, report the error and stop.
|
||||||
|
|
||||||
The task is considered successful when:
|
### 3. Generate engagement report
|
||||||
- Script exits with code 0
|
|
||||||
- `report-YYYY-MM-DD.md` exists and is non-empty
|
**Important:** The raw JSON file is too large to read directly (exceeds the 50KB read limit). Instead, write a Node.js script (e.g. `generate-report.js`) in the output directory that reads `raw.json`, processes the data, and writes `report.md`. Then execute it with `node generate-report.js`. Delete the script after it runs successfully.
|
||||||
- `raw-YYYY-MM-DD.json` exists and contains valid JSON array
|
|
||||||
- (If download enabled) `videos/` directory exists with at least one file
|
The script should read `raw.json`, parse it as a JSON array of video items, and write `report.md` with the following sections:
|
||||||
|
|
||||||
|
#### Header
|
||||||
|
```markdown
|
||||||
|
# TikTok Trending Report — <COUNTRY> — YYYY-MM-DD
|
||||||
|
|
||||||
|
Total videos analyzed: <count>
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Engagement Summary
|
||||||
|
|
||||||
|
Build a table from each video's `statistics` object (`play_count`, `digg_count`, `share_count`, `comment_count`):
|
||||||
|
|
||||||
|
| Metric | Total | Avg per video |
|
||||||
|
|--------|------:|-------------:|
|
||||||
|
| Views | ... | ... |
|
||||||
|
| Likes | ... | ... |
|
||||||
|
| Shares | ... | ... |
|
||||||
|
| Comments | ... | ... |
|
||||||
|
|
||||||
|
#### Top Hashtags (up to 20)
|
||||||
|
|
||||||
|
Extract hashtags from each video's `text_extra` array (entries where `hashtag_name` is set). If `text_extra` is empty, fall back to parsing `#tags` from the `desc` field. Count occurrences, sort descending.
|
||||||
|
|
||||||
|
| Hashtag | Count |
|
||||||
|
|---------|------:|
|
||||||
|
|
||||||
|
#### Top Sounds (up to 10)
|
||||||
|
|
||||||
|
From each video's `music` object, format as `title — author`. Count occurrences, sort descending.
|
||||||
|
|
||||||
|
| Sound | Count |
|
||||||
|
|-------|------:|
|
||||||
|
|
||||||
|
#### Creators Appearing in Trending (up to 10)
|
||||||
|
|
||||||
|
From each video's `author.unique_id`. Count occurrences, sort descending.
|
||||||
|
|
||||||
|
| Creator | Videos |
|
||||||
|
|---------|-------:|
|
||||||
|
|
||||||
|
#### Video List
|
||||||
|
|
||||||
|
Full table of all videos, sorted by position:
|
||||||
|
|
||||||
|
| # | Creator | Description | Views | Likes | URL |
|
||||||
|
|--:|---------|-------------|------:|------:|-----|
|
||||||
|
|
||||||
|
- Creator: `@author.unique_id`
|
||||||
|
- Description: first 60 chars of `desc`, pipe and newline characters replaced, with `...` if truncated
|
||||||
|
- URL: `share_url`
|
||||||
|
- Format numbers with locale separators (e.g. `1,234,567`)
|
||||||
|
|
||||||
|
### 4. Download videos (only if `download` is true)
|
||||||
|
|
||||||
|
If `download` is false, skip this step entirely.
|
||||||
|
|
||||||
|
If `download` is true:
|
||||||
|
1. Check that `yt-dlp` is installed
|
||||||
|
2. Create a `videos/` subdirectory in the output directory
|
||||||
|
3. Write all `share_url` values to a `urls.txt` file
|
||||||
|
4. Run: `yt-dlp -a urls.txt -o "videos/%(id)s.%(ext)s" --write-info-json --no-overwrites`
|
||||||
|
5. Report how many videos were downloaded. Partial failures are acceptable — do not fail the task if some downloads fail.
|
||||||
|
|
||||||
|
### 5. Report results
|
||||||
|
|
||||||
|
Print a summary to the user:
|
||||||
|
- Country and date
|
||||||
|
- Number of videos fetched
|
||||||
|
- Top video: `@creator` — views count — URL
|
||||||
|
- Top hashtag and count
|
||||||
|
- Output directory path
|
||||||
|
- Files created and their sizes
|
||||||
|
|
||||||
|
## Data Shape Reference
|
||||||
|
|
||||||
|
Each video item from the actor has this structure:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"aweme_id": "string",
|
||||||
|
"desc": "video description with #hashtags",
|
||||||
|
"create_time": 1234567890,
|
||||||
|
"share_url": "https://www.tiktok.com/@user/video/123",
|
||||||
|
"author": {
|
||||||
|
"unique_id": "username",
|
||||||
|
"nickname": "Display Name",
|
||||||
|
"uid": "123"
|
||||||
|
},
|
||||||
|
"statistics": {
|
||||||
|
"play_count": 1000000,
|
||||||
|
"digg_count": 50000,
|
||||||
|
"share_count": 5000,
|
||||||
|
"comment_count": 2000,
|
||||||
|
"collect_count": 1000,
|
||||||
|
"download_count": 500
|
||||||
|
},
|
||||||
|
"music": {
|
||||||
|
"title": "Sound Name",
|
||||||
|
"author": "Sound Author"
|
||||||
|
},
|
||||||
|
"text_extra": [
|
||||||
|
{ "hashtag_name": "trending", "type": 1 }
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
## Error Handling
|
## Error Handling
|
||||||
|
|
||||||
| Scenario | Action |
|
| Scenario | Action |
|
||||||
|----------|--------|
|
|----------|--------|
|
||||||
| Script exits non-zero | Check `run.log` for Apify/API errors. Common causes: invalid token, no credits, rate limits |
|
| Apify tool returns error | Report the error message. Common causes: invalid token, no credits, rate limits |
|
||||||
| Empty results | Report "No trending videos found" but mark as success if files were created |
|
| Empty results | Report "No trending videos found for <country>" and stop |
|
||||||
| Partial results | Report success if at least 1 video returned, note the discrepancy |
|
| Partial results (fewer than requested) | Proceed normally, note the discrepancy in the summary |
|
||||||
| Timeout (>10 min) | Kill process, report timeout. Check Apify actor status in console |
|
| yt-dlp not installed when download=true | Report that yt-dlp is required and skip downloads |
|
||||||
| Download failures | Report which videos failed but mark task as success if main report generated |
|
| Download failures | Report which videos failed but do not fail the task |
|
||||||
|
|
||||||
## Post-execution
|
|
||||||
|
|
||||||
After successful execution:
|
|
||||||
|
|
||||||
1. **Verify outputs**:
|
|
||||||
- Check all expected files exist
|
|
||||||
- Validate JSON is parseable
|
|
||||||
- If download enabled, count video files
|
|
||||||
|
|
||||||
2. **Generate summary** (`report.md`):
|
|
||||||
```markdown
|
|
||||||
# TikTok Trends Run Summary
|
|
||||||
|
|
||||||
## Run Details
|
|
||||||
- **Date**: YYYY-MM-DD
|
|
||||||
- **Timestamp**: HH:MM:SS
|
|
||||||
- **Apify Actor**: novi~fast-tiktok-scraper
|
|
||||||
- **Country**: <country_name> (<country_code>)
|
|
||||||
- **Requested**: <limit> videos
|
|
||||||
- **Received**: <actual_count> videos
|
|
||||||
- **Duration**: <X> minutes <Y> seconds
|
|
||||||
|
|
||||||
## Cost Analysis
|
|
||||||
- **Apify Credits Used**: <cost> USD
|
|
||||||
- **Cost per Video**: $<cost/actual_count>
|
|
||||||
|
|
||||||
## Key Findings
|
|
||||||
- **Top Video**: @<creator> - <views> views
|
|
||||||
<description_preview>
|
|
||||||
URL: <url>
|
|
||||||
|
|
||||||
- **Top Hashtag**: #<hashtag> (<count> occurrences)
|
|
||||||
- **Top Sound**: <sound_name> (<count> uses)
|
|
||||||
|
|
||||||
## Output Files
|
|
||||||
- `report-YYYY-MM-DD.md` (<size>) - Engagement report
|
|
||||||
- `raw-YYYY-MM-DD.json` (<size>) - Raw API data
|
|
||||||
- `run.log` (<size>) - Execution log
|
|
||||||
- `videos/` (<count> files, <total_size>) - Downloaded videos (if enabled)
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
- <any_warnings_or_errors_from_log>
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Cleanup**:
|
|
||||||
- Remove temporary files if any
|
|
||||||
- Report final status to user
|
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
- **Rate limiting**: The Apify actor may take 1-3 minutes depending on the limit
|
- The Apify actor may take 1-3 minutes depending on the limit
|
||||||
- **Costs**: Each run uses Apify compute units. Monitor usage at https://console.apify.com/billing
|
- Each run uses Apify compute units — monitor at https://console.apify.com/billing
|
||||||
- **Data freshness**: Trends data is near real-time but may have slight delays
|
- Not all countries have sufficient trending data; some may return fewer results than requested
|
||||||
- **Video downloads**: Downloading many videos can take significant time and storage
|
|
||||||
- **Country availability**: Not all countries have sufficient trending data; some may return fewer results than requested
|
|
||||||
|
|||||||
@@ -0,0 +1,179 @@
|
|||||||
|
# Tools
|
||||||
|
|
||||||
|
A tool is a callable capability that agents can use during task execution. Each tool lives in its own directory under `tools/` and is defined by a `TOOL.md` file and an `index.ts` (or `index.js`) entry point.
|
||||||
|
|
||||||
|
Tools are loaded by the `tool-loader` extension at startup and registered as callable functions. They appear in the agent's system prompt and can be invoked by name.
|
||||||
|
|
||||||
|
## File Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
tools/
|
||||||
|
<tool-name>/
|
||||||
|
TOOL.md # Metadata and documentation
|
||||||
|
index.ts # Implementation (required)
|
||||||
|
```
|
||||||
|
|
||||||
|
Both files are required. The loader skips directories missing either `TOOL.md` or an entry file.
|
||||||
|
|
||||||
|
## TOOL.md Format
|
||||||
|
|
||||||
|
A tool file has two parts: **frontmatter** (YAML metadata) and **body** (Markdown documentation).
|
||||||
|
|
||||||
|
### Frontmatter
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
name: tool_name
|
||||||
|
label: Tool Name
|
||||||
|
description: What the tool does and when the agent should use it.
|
||||||
|
version: 1
|
||||||
|
language: typescript
|
||||||
|
inputs:
|
||||||
|
param_name:
|
||||||
|
type: string
|
||||||
|
description: What this parameter is for.
|
||||||
|
optional_param:
|
||||||
|
type: number
|
||||||
|
description: An optional parameter.
|
||||||
|
optional: true
|
||||||
|
secret_param:
|
||||||
|
type: string
|
||||||
|
description: A sensitive value (e.g., API token).
|
||||||
|
optional: true
|
||||||
|
sensitive: true
|
||||||
|
mode:
|
||||||
|
type: enum
|
||||||
|
values: single,batch
|
||||||
|
description: Choose between modes.
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Fields
|
||||||
|
|
||||||
|
| Field | Type | Required | Description |
|
||||||
|
|-------|------|----------|-------------|
|
||||||
|
| `name` | string | yes | Identifier for the tool (used in tool calls). Use `snake_case`. |
|
||||||
|
| `label` | string | no | Human-readable display name. Defaults to `name` if omitted. |
|
||||||
|
| `description` | string | yes | What the tool does. This appears in the agent's system prompt — make it specific enough for the agent to know when to use it. |
|
||||||
|
| `version` | integer | no | Version number. Used by `sync-tools` to detect updates — bump when changing the tool. |
|
||||||
|
| `language` | string | no | Implementation language: `typescript`, `bash`, or `python`. Defaults to `typescript`. |
|
||||||
|
| `inputs` | object | no | Input parameters the tool accepts. Keys are parameter names. |
|
||||||
|
|
||||||
|
#### Input Fields
|
||||||
|
|
||||||
|
Each input is a key under `inputs:` with these properties:
|
||||||
|
|
||||||
|
| Field | Type | Required | Description |
|
||||||
|
|-------|------|----------|-------------|
|
||||||
|
| `type` | string | yes | Parameter type: `string`, `number`, `boolean`, or `enum`. |
|
||||||
|
| `description` | string | yes | What this parameter is for. Shown to the agent. |
|
||||||
|
| `optional` | boolean | no | Whether the parameter is optional. Defaults to required. |
|
||||||
|
| `sensitive` | boolean | no | Mark sensitive values (tokens, passwords). Prevents logging. |
|
||||||
|
| `values` | string | no | Comma-separated allowed values when `type` is `enum`. |
|
||||||
|
| `default` | any | no | Default value if not provided. |
|
||||||
|
|
||||||
|
### Body
|
||||||
|
|
||||||
|
The body is Markdown documentation that the agent sees when the tool is loaded. Include:
|
||||||
|
|
||||||
|
- **Title** — `# Tool Name`
|
||||||
|
- **Authentication** — How credentials are resolved (env vars, integrations, etc.)
|
||||||
|
- **Usage** — How to call the tool and what parameters to pass.
|
||||||
|
- **Examples** — Common usage patterns.
|
||||||
|
- **Error Handling** — What errors can occur and what they mean.
|
||||||
|
- **Notes** — Limits, billing, external links.
|
||||||
|
|
||||||
|
## index.ts Format
|
||||||
|
|
||||||
|
The entry file must export an `execute` function:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
type ToolResult = {
|
||||||
|
content: Array<{ type: string; text: string }>;
|
||||||
|
isError?: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
type OnUpdate = (partial: { content: Array<{ type: string; text: string }> }) => void;
|
||||||
|
|
||||||
|
export async function execute(
|
||||||
|
toolCallId: string,
|
||||||
|
params: Record<string, unknown>,
|
||||||
|
signal: AbortSignal | undefined,
|
||||||
|
onUpdate?: OnUpdate,
|
||||||
|
): Promise<ToolResult> {
|
||||||
|
// Implementation here
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Parameters
|
||||||
|
|
||||||
|
| Parameter | Description |
|
||||||
|
|-----------|-------------|
|
||||||
|
| `toolCallId` | Unique ID for this tool call. |
|
||||||
|
| `params` | Input values from the agent, matching the `inputs` defined in TOOL.md. |
|
||||||
|
| `signal` | AbortSignal for cancellation. |
|
||||||
|
| `onUpdate` | Callback for streaming progress updates to the agent during long operations. |
|
||||||
|
|
||||||
|
### Return Value
|
||||||
|
|
||||||
|
Return a `ToolResult` object:
|
||||||
|
- `content` — Array of content blocks. Usually one `{ type: 'text', text: '...' }`.
|
||||||
|
- `isError` — Set `true` to indicate failure. The agent sees the error and can react.
|
||||||
|
|
||||||
|
### Progress Updates
|
||||||
|
|
||||||
|
Use `onUpdate` to stream status during long-running operations:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
onUpdate?.({ content: [{ type: 'text', text: 'Processing step 2 of 5...' }] });
|
||||||
|
```
|
||||||
|
|
||||||
|
### Authentication
|
||||||
|
|
||||||
|
Tools should resolve credentials internally, not require the agent to pass them. Pattern:
|
||||||
|
|
||||||
|
1. Check for an explicit parameter override (e.g., `params.api_token`)
|
||||||
|
2. Fall back to an environment variable (e.g., `process.env.OFFICER_APIFY_TOKEN`)
|
||||||
|
3. Return a helpful error if neither is available
|
||||||
|
|
||||||
|
Environment variables are set by `pi-bridge.ts` from the integration config stored in the database (Settings → Integrations).
|
||||||
|
|
||||||
|
### Runtime Environment
|
||||||
|
|
||||||
|
Tools run inside sandboxed containers using **Node.js** (not Bun). Do not use Bun-specific APIs like `Bun.sleep`, `Bun.file`, etc. Use Node.js equivalents:
|
||||||
|
|
||||||
|
- `setTimeout` / `setInterval` for delays
|
||||||
|
- `fs.readFileSync` / `fs.writeFileSync` for file I/O
|
||||||
|
- `fetch` (available in Node 18+) for HTTP requests
|
||||||
|
|
||||||
|
### Large Output
|
||||||
|
|
||||||
|
If a tool may return large data (e.g., API responses with many items), provide an `output_path` parameter. When set, save the data to the file and return a summary instead:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
if (params.output_path) {
|
||||||
|
writeFileSync(params.output_path, JSON.stringify(items, null, 2));
|
||||||
|
return { content: [{ type: 'text', text: `${items.length} items saved to ${params.output_path}` }] };
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
This prevents flooding the agent's context window with raw data.
|
||||||
|
|
||||||
|
## Sync and Discovery
|
||||||
|
|
||||||
|
Tools are synced from `seed/tools/` to `DATA_PATH/tools/` at server startup by `sync-tools.ts`. The sync is version-based — it only overwrites when the seed version is higher than the target version. Always bump `version` in the frontmatter when updating a tool.
|
||||||
|
|
||||||
|
The `tool-loader` extension discovers tools from directories listed in the `PI_TOOLS_DIRS` environment variable (colon-separated). Only `DATA_PATH/tools/` is mounted into containers — `seed/tools/` is not directly accessible at runtime.
|
||||||
|
|
||||||
|
## Existing Tools
|
||||||
|
|
||||||
|
| Tool | Description |
|
||||||
|
|------|-------------|
|
||||||
|
| `gmail` | Read Gmail messages, threads, labels via Google API |
|
||||||
|
| `web_search` | Search the web via SearXNG |
|
||||||
|
| `web_fetch` | Fetch and extract content from URLs |
|
||||||
|
| `browser` | Control a Chrome browser via Browser Relay |
|
||||||
|
| `apify` | Run any Apify actor (web scraping, social media data) |
|
||||||
|
| `convert_audio_to_mp3` | Convert audio files to MP3 via ffmpeg |
|
||||||
|
| `ocr` | Optical character recognition on images |
|
||||||
|
| `email_db` | Query the synced email database |
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
---
|
||||||
|
name: apify
|
||||||
|
label: Apify
|
||||||
|
version: 4
|
||||||
|
description: Run any Apify actor and return its dataset results. Use for web scraping, data extraction, and automation — TikTok, Twitter, Facebook, Instagram, YouTube, Google, and hundreds more. Authentication is handled automatically when configured in Settings → Integrations.
|
||||||
|
language: typescript
|
||||||
|
inputs:
|
||||||
|
actor_id:
|
||||||
|
type: string
|
||||||
|
description: "Actor ID to run (format: owner~actor-name or owner/actor-name, e.g. 'novi~fast-tiktok-scraper', 'apify/twitter-scraper')"
|
||||||
|
input:
|
||||||
|
type: object
|
||||||
|
description: Actor-specific input parameters as a JSON object (varies per actor)
|
||||||
|
optional: true
|
||||||
|
output_path:
|
||||||
|
type: string
|
||||||
|
description: File path to save JSON results to. When provided, the tool writes the dataset to this file and returns a summary instead of the raw JSON. Recommended for large datasets to avoid flooding the context.
|
||||||
|
optional: true
|
||||||
|
api_token:
|
||||||
|
type: string
|
||||||
|
description: Override the configured API token. Usually not needed — the token is provided automatically from Settings → Integrations → Apify.
|
||||||
|
optional: true
|
||||||
|
sensitive: true
|
||||||
|
timeout_ms:
|
||||||
|
type: number
|
||||||
|
description: Max time to wait for actor completion in milliseconds (default 300000 = 5 min)
|
||||||
|
optional: true
|
||||||
|
poll_interval_ms:
|
||||||
|
type: number
|
||||||
|
description: How often to check run status in milliseconds (default 3000)
|
||||||
|
optional: true
|
||||||
|
---
|
||||||
|
|
||||||
|
# Apify
|
||||||
|
|
||||||
|
Run any actor from the Apify Store, wait for completion, and return the dataset items as JSON.
|
||||||
|
|
||||||
|
## Authentication
|
||||||
|
|
||||||
|
The API token is resolved automatically:
|
||||||
|
1. `api_token` input parameter (explicit override)
|
||||||
|
2. `OFFICER_APIFY_TOKEN` environment variable (set automatically when configured in Settings → Integrations → Apify)
|
||||||
|
|
||||||
|
If neither is available, the tool returns an error prompting the user to configure the integration.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
Just provide the `actor_id` and optional `input`:
|
||||||
|
```
|
||||||
|
apify(actor_id: "novi~fast-tiktok-scraper", input: { type: "TREND", region: "PT", maxItems: 20 })
|
||||||
|
```
|
||||||
|
|
||||||
|
The tool starts the actor, polls until completion, fetches the dataset, and returns all items as JSON.
|
||||||
|
|
||||||
|
## Common Actors
|
||||||
|
|
||||||
|
| Actor | ID | Input example |
|
||||||
|
|-------|----|---------------|
|
||||||
|
| TikTok Scraper | `novi~fast-tiktok-scraper` | `{ type: 'TREND', region: 'US', maxItems: 20 }` |
|
||||||
|
| Twitter Scraper | `apify/twitter-scraper` | `{ searchTerms: ['#ai'], tweetsCount: 100 }` |
|
||||||
|
| Twitter User | `jupri/twitter-user-scraper` | `{ twitterUser: 'username', maxPosts: 50 }` |
|
||||||
|
| Facebook Scraper | `apify/facebook-scraper` | `{ startUrls: ['https://facebook.com/Page'], maxPostsPerPage: 50 }` |
|
||||||
|
| Facebook Search | `jupri/facebook-search-scraper` | `{ searchTerm: 'keyword', maxPosts: 30 }` |
|
||||||
|
| Instagram Hashtag | `apify/instagram-hashtag-scraper` | `{ hashtags: ['travel'], resultsLimit: 50 }` |
|
||||||
|
| Instagram User | `apify/instagram-user-scraper` | `{ usernames: ['natgeo'], resultsLimit: 50 }` |
|
||||||
|
| YouTube Scraper | `apify/youtube-scraper` | `{ searchTerms: ['tutorial'], maxResults: 20 }` |
|
||||||
|
| Google Search | `apify/google-search-scraper` | `{ queries: ['best restaurants lisbon'] }` |
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
The tool returns clear error messages for:
|
||||||
|
- Missing API token → "Configure it in Settings → Integrations → Apify"
|
||||||
|
- API errors (401, 403, etc.) → includes the HTTP status and response body
|
||||||
|
- Actor failures → includes the actor's `statusMessage`
|
||||||
|
- Timeouts → reports the run ID and last known status
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Actor IDs use `~` (Apify URL format) or `/` — both work
|
||||||
|
- Browse actors at https://apify.com/store
|
||||||
|
- Monitor usage and credits at https://console.apify.com/billing
|
||||||
@@ -0,0 +1,167 @@
|
|||||||
|
import { mkdirSync, writeFileSync } from 'node:fs';
|
||||||
|
import { dirname } from 'node:path';
|
||||||
|
|
||||||
|
const BASE = 'https://api.apify.com/v2';
|
||||||
|
|
||||||
|
type RunStatus = 'READY' | 'RUNNING' | 'SUCCEEDED' | 'FAILED' | 'ABORTING' | 'ABORTED' | 'TIMING-OUT' | 'TIMED-OUT';
|
||||||
|
|
||||||
|
type RunData = {
|
||||||
|
id: string;
|
||||||
|
actId: string;
|
||||||
|
status: RunStatus;
|
||||||
|
statusMessage?: string;
|
||||||
|
defaultDatasetId: string;
|
||||||
|
defaultKeyValueStoreId: string;
|
||||||
|
startedAt?: string;
|
||||||
|
finishedAt?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
type ToolResult = {
|
||||||
|
content: Array<{ type: string; text: string }>;
|
||||||
|
isError?: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
type OnUpdate = (partial: { content: Array<{ type: string; text: string }> }) => void;
|
||||||
|
|
||||||
|
type Params = {
|
||||||
|
actor_id: string;
|
||||||
|
input?: Record<string, unknown> | string;
|
||||||
|
api_token?: string;
|
||||||
|
output_path?: string;
|
||||||
|
timeout_ms?: number;
|
||||||
|
poll_interval_ms?: number;
|
||||||
|
};
|
||||||
|
|
||||||
|
function update(onUpdate: OnUpdate | undefined, text: string): void {
|
||||||
|
onUpdate?.({ content: [{ type: 'text', text }] });
|
||||||
|
}
|
||||||
|
|
||||||
|
function resolveToken(params: Params): string | null {
|
||||||
|
if (params.api_token) return params.api_token;
|
||||||
|
return process.env.OFFICER_APIFY_TOKEN ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function apiUrl(path: string, token: string, extra?: Record<string, string>): string {
|
||||||
|
const params = new URLSearchParams({ token, ...extra });
|
||||||
|
return `${BASE}${path}?${params}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function apiRequest<T>(url: string, init?: RequestInit): Promise<T> {
|
||||||
|
const res = await fetch(url, init);
|
||||||
|
if (!res.ok) {
|
||||||
|
const body = await res.text();
|
||||||
|
throw new Error(`Apify API error ${res.status}: ${body}`);
|
||||||
|
}
|
||||||
|
return res.json() as Promise<T>;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function startRun(token: string, actorId: string, input: Record<string, unknown>): Promise<RunData> {
|
||||||
|
const { data } = await apiRequest<{ data: RunData }>(apiUrl(`/acts/${actorId}/runs`, token), {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify(input),
|
||||||
|
});
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function getRun(token: string, runId: string): Promise<RunData> {
|
||||||
|
const { data } = await apiRequest<{ data: RunData }>(apiUrl(`/actor-runs/${runId}`, token));
|
||||||
|
return data;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function getDatasetItems<T>(token: string, datasetId: string): Promise<T[]> {
|
||||||
|
return apiRequest<T[]>(apiUrl(`/datasets/${datasetId}/items`, token, { format: 'json' }));
|
||||||
|
}
|
||||||
|
|
||||||
|
const TERMINAL_STATUSES = new Set<RunStatus>(['SUCCEEDED', 'FAILED', 'ABORTED', 'TIMED-OUT']);
|
||||||
|
|
||||||
|
export async function execute(
|
||||||
|
_toolCallId: string,
|
||||||
|
params: Params,
|
||||||
|
_signal: AbortSignal | undefined,
|
||||||
|
onUpdate?: OnUpdate,
|
||||||
|
): Promise<ToolResult> {
|
||||||
|
const token = resolveToken(params);
|
||||||
|
if (!token) {
|
||||||
|
return {
|
||||||
|
content: [{ type: 'text', text: 'Apify API token not available. Configure it in Settings → Integrations → Apify, or pass api_token explicitly.' }],
|
||||||
|
isError: true,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const { actor_id } = params;
|
||||||
|
if (!actor_id) {
|
||||||
|
return {
|
||||||
|
content: [{ type: 'text', text: 'actor_id is required.' }],
|
||||||
|
isError: true,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
let input: Record<string, unknown> = {};
|
||||||
|
if (params.input) {
|
||||||
|
if (typeof params.input === 'string') {
|
||||||
|
try {
|
||||||
|
input = JSON.parse(params.input);
|
||||||
|
} catch {
|
||||||
|
return {
|
||||||
|
content: [{ type: 'text', text: 'Invalid JSON in input parameter.' }],
|
||||||
|
isError: true,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
input = params.input;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const timeoutMs = params.timeout_ms ?? 300_000;
|
||||||
|
const pollIntervalMs = params.poll_interval_ms ?? 3_000;
|
||||||
|
|
||||||
|
try {
|
||||||
|
update(onUpdate, `Starting actor ${actor_id}...`);
|
||||||
|
const run = await startRun(token, actor_id, input);
|
||||||
|
update(onUpdate, `Run ${run.id} started. Waiting for completion...`);
|
||||||
|
|
||||||
|
const start = Date.now();
|
||||||
|
let finished = run;
|
||||||
|
|
||||||
|
while (!TERMINAL_STATUSES.has(finished.status)) {
|
||||||
|
if (Date.now() - start > timeoutMs) {
|
||||||
|
return {
|
||||||
|
content: [{ type: 'text', text: `Timeout after ${timeoutMs}ms waiting for run ${run.id}. Status: ${finished.status}` }],
|
||||||
|
isError: true,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
await new Promise((r) => setTimeout(r, pollIntervalMs));
|
||||||
|
finished = await getRun(token, run.id);
|
||||||
|
update(onUpdate, `Status: ${finished.status}...`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (finished.status !== 'SUCCEEDED') {
|
||||||
|
return {
|
||||||
|
content: [{ type: 'text', text: `Actor run ${finished.status}: ${finished.statusMessage ?? 'unknown error'}` }],
|
||||||
|
isError: true,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
update(onUpdate, `Run succeeded. Fetching dataset items...`);
|
||||||
|
const items = await getDatasetItems(token, finished.defaultDatasetId);
|
||||||
|
|
||||||
|
if (params.output_path) {
|
||||||
|
mkdirSync(dirname(params.output_path), { recursive: true });
|
||||||
|
writeFileSync(params.output_path, JSON.stringify(items, null, 2));
|
||||||
|
return {
|
||||||
|
content: [{ type: 'text', text: `${items.length} items saved to ${params.output_path}` }],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
content: [{ type: 'text', text: JSON.stringify(items) }],
|
||||||
|
};
|
||||||
|
} catch (err) {
|
||||||
|
const message = err instanceof Error ? err.message : String(err);
|
||||||
|
return {
|
||||||
|
content: [{ type: 'text', text: `Apify error: ${message}` }],
|
||||||
|
isError: true,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -8,6 +8,7 @@ import { NewPipeline } from './NewPipeline';
|
|||||||
import { NewCron } from './NewCron';
|
import { NewCron } from './NewCron';
|
||||||
import { NewService } from './NewService';
|
import { NewService } from './NewService';
|
||||||
import { NewWorkflow } from './NewWorkflow';
|
import { NewWorkflow } from './NewWorkflow';
|
||||||
|
import { NewTool } from './NewTool';
|
||||||
|
|
||||||
export type AutomationSelection = {
|
export type AutomationSelection = {
|
||||||
kind: string;
|
kind: string;
|
||||||
@@ -28,6 +29,7 @@ const newComponentMap: Record<string, React.ComponentType<{ selection: NonNullab
|
|||||||
Cron: NewCron,
|
Cron: NewCron,
|
||||||
Service: NewService,
|
Service: NewService,
|
||||||
Workflow: NewWorkflow,
|
Workflow: NewWorkflow,
|
||||||
|
Tool: NewTool,
|
||||||
};
|
};
|
||||||
|
|
||||||
export const AutomationRightPanel = () => {
|
export const AutomationRightPanel = () => {
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import { Box, Clock, Cpu, GitBranch, ListTodo, Server, Sparkles, Workflow } from 'lucide-react';
|
import { Box, Clock, Cpu, GitBranch, ListTodo, Server, Sparkles, Workflow, Wrench } from 'lucide-react';
|
||||||
import { usePanelChannel } from 'hooks/usePanelChannel';
|
import { usePanelChannel } from 'hooks/usePanelChannel';
|
||||||
import type { AutomationSelection } from './AutomationRightPanel';
|
import type { AutomationSelection } from './AutomationRightPanel';
|
||||||
|
|
||||||
@@ -6,6 +6,7 @@ const capabilityItems = [
|
|||||||
{ id: 'processes', label: 'Processes', icon: Cpu, kind: 'Process', endpoint: '/processes', queryKey: 'processes' },
|
{ id: 'processes', label: 'Processes', icon: Cpu, kind: 'Process', endpoint: '/processes', queryKey: 'processes' },
|
||||||
{ id: 'tasks', label: 'Tasks', icon: ListTodo, kind: 'Task', endpoint: '/tasks', queryKey: 'tasks' },
|
{ id: 'tasks', label: 'Tasks', icon: ListTodo, kind: 'Task', endpoint: '/tasks', queryKey: 'tasks' },
|
||||||
{ id: 'skills', label: 'Skills', icon: Sparkles, kind: 'Skill', endpoint: '/skills', queryKey: 'skills' },
|
{ id: 'skills', label: 'Skills', icon: Sparkles, kind: 'Skill', endpoint: '/skills', queryKey: 'skills' },
|
||||||
|
{ id: 'tools', label: 'Tools', icon: Wrench, kind: 'Tool', endpoint: '/tools', queryKey: 'tools' },
|
||||||
{ id: 'pipelines', label: 'Pipelines', icon: Workflow, kind: 'Pipeline', endpoint: '/pipelines', queryKey: 'pipelines' },
|
{ id: 'pipelines', label: 'Pipelines', icon: Workflow, kind: 'Pipeline', endpoint: '/pipelines', queryKey: 'pipelines' },
|
||||||
{ id: 'crons', label: 'Crons', icon: Clock, kind: 'Cron', endpoint: '/crons', queryKey: 'crons' },
|
{ id: 'crons', label: 'Crons', icon: Clock, kind: 'Cron', endpoint: '/crons', queryKey: 'crons' },
|
||||||
{ id: 'services', label: 'Services', icon: Server, kind: 'Service', endpoint: '/services', queryKey: 'services' },
|
{ id: 'services', label: 'Services', icon: Server, kind: 'Service', endpoint: '/services', queryKey: 'services' },
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import { ArrowLeft, Pencil, Play, Trash2 } from 'lucide-react';
|
|||||||
import { Button } from '@/components/ui/button';
|
import { Button } from '@/components/ui/button';
|
||||||
import { Dialog, DialogContent, DialogHeader, DialogTitle, DialogDescription } from '@/components/ui/dialog';
|
import { Dialog, DialogContent, DialogHeader, DialogTitle, DialogDescription } from '@/components/ui/dialog';
|
||||||
import { useClient } from 'hooks/useClient';
|
import { useClient } from 'hooks/useClient';
|
||||||
|
import { useAuth } from 'hooks/useAuth';
|
||||||
import { usePanelChannel } from 'hooks/usePanelChannel';
|
import { usePanelChannel } from 'hooks/usePanelChannel';
|
||||||
import { Card } from '@/components/Card';
|
import { Card } from '@/components/Card';
|
||||||
import { FrontmatterBlock } from '../CapabilityPage';
|
import { FrontmatterBlock } from '../CapabilityPage';
|
||||||
@@ -73,6 +74,7 @@ type CapabilityDetailViewProps = {
|
|||||||
export const CapabilityDetailView = ({ kind, endpoint, queryKey, dirName, editing }: CapabilityDetailViewProps) => {
|
export const CapabilityDetailView = ({ kind, endpoint, queryKey, dirName, editing }: CapabilityDetailViewProps) => {
|
||||||
const client = useClient();
|
const client = useClient();
|
||||||
const qc = useQueryClient();
|
const qc = useQueryClient();
|
||||||
|
const { user } = useAuth();
|
||||||
const [selection, setSelection] = usePanelChannel<AutomationSelection>('automation:selected-capability', null);
|
const [selection, setSelection] = usePanelChannel<AutomationSelection>('automation:selected-capability', null);
|
||||||
const [deleteConfirm, setDeleteConfirm] = useState(false);
|
const [deleteConfirm, setDeleteConfirm] = useState(false);
|
||||||
const [showInputForm, setShowInputForm] = useState(false);
|
const [showInputForm, setShowInputForm] = useState(false);
|
||||||
@@ -149,18 +151,22 @@ export const CapabilityDetailView = ({ kind, endpoint, queryKey, dirName, editin
|
|||||||
<Play className="h-3.5 w-3.5 text-duck-teal" />
|
<Play className="h-3.5 w-3.5 text-duck-teal" />
|
||||||
</button>
|
</button>
|
||||||
)}
|
)}
|
||||||
<button
|
{(detail.scope === 'user' || user?.role === 'Super Admin') && (
|
||||||
onClick={toggleEditing}
|
<>
|
||||||
className={`p-1 rounded hover:bg-duck-dark/10 dark:hover:bg-foreground/10 cursor-pointer transition-colors ${editing ? 'bg-duck-teal/10' : ''}`}
|
<button
|
||||||
>
|
onClick={toggleEditing}
|
||||||
<Pencil className={`h-3.5 w-3.5 ${editing ? 'text-duck-teal' : 'text-duck-dark/50 dark:text-foreground/50'}`} />
|
className={`p-1 rounded hover:bg-duck-dark/10 dark:hover:bg-foreground/10 cursor-pointer transition-colors ${editing ? 'bg-duck-teal/10' : ''}`}
|
||||||
</button>
|
>
|
||||||
<button
|
<Pencil className={`h-3.5 w-3.5 ${editing ? 'text-duck-teal' : 'text-duck-dark/50 dark:text-foreground/50'}`} />
|
||||||
onClick={() => setDeleteConfirm(true)}
|
</button>
|
||||||
className="p-1 rounded hover:bg-red-50 cursor-pointer transition-colors"
|
<button
|
||||||
>
|
onClick={() => setDeleteConfirm(true)}
|
||||||
<Trash2 className="h-3.5 w-3.5 text-duck-dark/50 dark:text-foreground/50 hover:text-red-500" />
|
className="p-1 rounded hover:bg-red-50 cursor-pointer transition-colors"
|
||||||
</button>
|
>
|
||||||
|
<Trash2 className="h-3.5 w-3.5 text-duck-dark/50 dark:text-foreground/50 hover:text-red-500" />
|
||||||
|
</button>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
</>
|
</>
|
||||||
)}
|
)}
|
||||||
</div>
|
</div>
|
||||||
@@ -275,7 +281,7 @@ export const CapabilityDetailView = ({ kind, endpoint, queryKey, dirName, editin
|
|||||||
onOpenChange={(open) => {
|
onOpenChange={(open) => {
|
||||||
if (!open) setRunPrompt(null);
|
if (!open) setRunPrompt(null);
|
||||||
}}
|
}}
|
||||||
task={{ dirName, name: detail.name, description: detail.description ?? '', scope: detail.scope ?? 'user', triggers: [], filePath: detail.filePath }}
|
task={{ dirName, name: detail.name, description: detail.description ?? '', scope: detail.scope === 'user' ? 'user' : 'global', triggers: [], filePath: detail.filePath }}
|
||||||
promptOverride={runPrompt}
|
promptOverride={runPrompt}
|
||||||
/>
|
/>
|
||||||
)}
|
)}
|
||||||
|
|||||||
@@ -0,0 +1,94 @@
|
|||||||
|
import { useState } from 'react';
|
||||||
|
import { useQueryClient } from '@tanstack/react-query';
|
||||||
|
import { X } from 'lucide-react';
|
||||||
|
import { toast } from 'sonner';
|
||||||
|
import { Button } from '@/components/ui/button';
|
||||||
|
import { useClient } from 'hooks/useClient';
|
||||||
|
import { usePanelChannel } from 'hooks/usePanelChannel';
|
||||||
|
import type { AutomationSelection } from './AutomationRightPanel';
|
||||||
|
|
||||||
|
type NewToolProps = {
|
||||||
|
selection: NonNullable<AutomationSelection>;
|
||||||
|
};
|
||||||
|
|
||||||
|
export const NewTool = ({ selection }: NewToolProps) => {
|
||||||
|
const client = useClient();
|
||||||
|
const qc = useQueryClient();
|
||||||
|
const [, setSelection] = usePanelChannel<AutomationSelection>('automation:selected-capability', null);
|
||||||
|
const [name, setName] = useState('');
|
||||||
|
const [description, setDescription] = useState('');
|
||||||
|
const [submitting, setSubmitting] = useState(false);
|
||||||
|
|
||||||
|
const handleCreate = async () => {
|
||||||
|
const trimmed = name.trim();
|
||||||
|
if (!trimmed) return;
|
||||||
|
setSubmitting(true);
|
||||||
|
try {
|
||||||
|
const res = await client.post<{ name: string; dirName: string }>(selection.endpoint, { name: trimmed });
|
||||||
|
await qc.invalidateQueries({ queryKey: [selection.queryKey] });
|
||||||
|
setSelection({
|
||||||
|
kind: selection.kind,
|
||||||
|
endpoint: selection.endpoint,
|
||||||
|
queryKey: selection.queryKey,
|
||||||
|
dirName: res.dirName,
|
||||||
|
isNew: true,
|
||||||
|
editing: true,
|
||||||
|
description: description.trim() || undefined,
|
||||||
|
});
|
||||||
|
} catch {
|
||||||
|
toast.error('Failed to create tool');
|
||||||
|
setSubmitting(false);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="h-full flex flex-col">
|
||||||
|
<div className="shrink-0 px-4 py-2 border-b border-duck-dark/10 dark:border-foreground/10 bg-background/60 flex items-center gap-2">
|
||||||
|
<span className="text-sm font-medium text-duck-dark/70 dark:text-foreground/70 flex-1">New Tool</span>
|
||||||
|
<button
|
||||||
|
onClick={() => setSelection(null)}
|
||||||
|
className="p-1 rounded hover:bg-duck-dark/10 dark:hover:bg-foreground/10 cursor-pointer transition-colors"
|
||||||
|
>
|
||||||
|
<X className="h-3.5 w-3.5 text-duck-dark/50 dark:text-foreground/50" />
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
<div className="flex-1 overflow-y-auto p-6 flex flex-col gap-4">
|
||||||
|
<div className="flex flex-col gap-1.5">
|
||||||
|
<label className="text-sm font-medium text-duck-dark/70 dark:text-foreground/70">Name</label>
|
||||||
|
<input
|
||||||
|
value={name}
|
||||||
|
onChange={(ev) => setName(ev.target.value)}
|
||||||
|
onKeyDown={(ev) => {
|
||||||
|
if (ev.key === 'Enter') {
|
||||||
|
ev.preventDefault();
|
||||||
|
handleCreate();
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
placeholder="Tool name..."
|
||||||
|
autoFocus
|
||||||
|
className="rounded border border-duck-dark/20 dark:border-foreground/20 bg-background px-3 py-2 text-sm text-duck-dark dark:text-foreground placeholder:text-duck-dark/30 dark:placeholder:text-foreground/30 focus:outline-none focus:ring-1 focus:ring-duck-teal/30"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<div className="flex flex-col gap-1.5">
|
||||||
|
<label className="text-sm font-medium text-duck-dark/70 dark:text-foreground/70">Description</label>
|
||||||
|
<textarea
|
||||||
|
value={description}
|
||||||
|
onChange={(ev) => setDescription(ev.target.value)}
|
||||||
|
placeholder="Describe what this tool does..."
|
||||||
|
rows={4}
|
||||||
|
className="rounded border border-duck-dark/20 dark:border-foreground/20 bg-background px-3 py-2 text-sm text-duck-dark dark:text-foreground placeholder:text-duck-dark/30 dark:placeholder:text-foreground/30 focus:outline-none focus:ring-1 focus:ring-duck-teal/30 resize-none"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<div className="flex justify-end">
|
||||||
|
<Button
|
||||||
|
onClick={handleCreate}
|
||||||
|
disabled={!name.trim() || submitting}
|
||||||
|
className="bg-duck-teal text-duck-yellow hover:bg-duck-teal/90 cursor-pointer"
|
||||||
|
>
|
||||||
|
{submitting ? 'Creating...' : 'Create'}
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
};
|
||||||
@@ -8,6 +8,7 @@ import { ArrowLeft, Pencil, Plus, Check, X, Trash2, Search, ChevronRight } from
|
|||||||
import { Button } from '@/components/ui/button';
|
import { Button } from '@/components/ui/button';
|
||||||
import { Dialog, DialogContent, DialogHeader, DialogTitle, DialogDescription } from '@/components/ui/dialog';
|
import { Dialog, DialogContent, DialogHeader, DialogTitle, DialogDescription } from '@/components/ui/dialog';
|
||||||
import { useClient } from 'hooks/useClient';
|
import { useClient } from 'hooks/useClient';
|
||||||
|
import { useAuth } from 'hooks/useAuth';
|
||||||
import { Card } from '@/components/Card';
|
import { Card } from '@/components/Card';
|
||||||
import { usePiChat, EmbeddableChat } from 'officerdev';
|
import { usePiChat, EmbeddableChat } from 'officerdev';
|
||||||
type CapabilitySummary = {
|
type CapabilitySummary = {
|
||||||
@@ -249,6 +250,7 @@ export const CapabilityList = ({ kind, endpoint, queryKey, selected, onSelect, o
|
|||||||
export const CapabilityPage = ({ kind, endpoint, queryKey }: CapabilityPageProps) => {
|
export const CapabilityPage = ({ kind, endpoint, queryKey }: CapabilityPageProps) => {
|
||||||
const client = useClient();
|
const client = useClient();
|
||||||
const qc = useQueryClient();
|
const qc = useQueryClient();
|
||||||
|
const { user } = useAuth();
|
||||||
const [selected, setSelected] = useState<string | null>(null);
|
const [selected, setSelected] = useState<string | null>(null);
|
||||||
const [editing, setEditing] = useState(false);
|
const [editing, setEditing] = useState(false);
|
||||||
const [isNew, setIsNew] = useState(false);
|
const [isNew, setIsNew] = useState(false);
|
||||||
@@ -330,7 +332,7 @@ export const CapabilityPage = ({ kind, endpoint, queryKey }: CapabilityPageProps
|
|||||||
<span className="text-sm font-medium text-duck-dark/70 flex-1">
|
<span className="text-sm font-medium text-duck-dark/70 flex-1">
|
||||||
{detail?.name ?? `Select a ${kind.toLowerCase()}`}
|
{detail?.name ?? `Select a ${kind.toLowerCase()}`}
|
||||||
</span>
|
</span>
|
||||||
{detail && (
|
{detail && (detail.scope === 'user' || user?.role === 'Super Admin') && (
|
||||||
<>
|
<>
|
||||||
<button
|
<button
|
||||||
onClick={() => setEditing((e) => !e)}
|
onClick={() => setEditing((e) => !e)}
|
||||||
|
|||||||
@@ -0,0 +1,90 @@
|
|||||||
|
import { useState, useEffect } from 'react';
|
||||||
|
import { toast } from 'sonner';
|
||||||
|
import { Input } from '@/components/ui/input';
|
||||||
|
import { Label } from '@/components/ui/label';
|
||||||
|
import { Button } from '@/components/ui/button';
|
||||||
|
import { useClient } from 'hooks/useClient';
|
||||||
|
|
||||||
|
type ApifyConfigData = {
|
||||||
|
apiToken: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
type ApifyStatus = {
|
||||||
|
configured: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
export const ApifyConfig = () => {
|
||||||
|
const client = useClient();
|
||||||
|
const [isLoading, setIsLoading] = useState(true);
|
||||||
|
const [isSaving, setIsSaving] = useState(false);
|
||||||
|
const [apiToken, setApiToken] = useState('');
|
||||||
|
const [status, setStatus] = useState<ApifyStatus | null>(null);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
Promise.all([
|
||||||
|
client.get<ApifyConfigData>('/integrations/apify/config').then((data) => {
|
||||||
|
if (data?.apiToken) setApiToken(data.apiToken);
|
||||||
|
}),
|
||||||
|
client.get<ApifyStatus>('/integrations/apify/status').then(setStatus),
|
||||||
|
])
|
||||||
|
.catch(() => {})
|
||||||
|
.finally(() => setIsLoading(false));
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
const handleSave = async () => {
|
||||||
|
if (isSaving) return;
|
||||||
|
setIsSaving(true);
|
||||||
|
try {
|
||||||
|
await client.put('/integrations/apify/config', { apiToken: apiToken.trim() });
|
||||||
|
toast.success('Apify API token saved');
|
||||||
|
setStatus({ configured: !!apiToken.trim() });
|
||||||
|
} catch {
|
||||||
|
toast.error('Failed to save Apify API token');
|
||||||
|
} finally {
|
||||||
|
setIsSaving(false);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
if (isLoading) return null;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="grid gap-5">
|
||||||
|
{status && (
|
||||||
|
<div className="flex items-center gap-2 rounded-lg border border-duck-dark/10 dark:border-foreground/10 p-3">
|
||||||
|
<div className={`h-2.5 w-2.5 rounded-full shrink-0 ${status.configured ? 'bg-green-500' : 'bg-duck-dark/20 dark:bg-foreground/20'}`} />
|
||||||
|
<span className="text-sm text-duck-dark dark:text-foreground">
|
||||||
|
{status.configured ? 'API token configured' : 'Not configured'}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<div className="grid gap-5">
|
||||||
|
<Label className="grid gap-2">
|
||||||
|
<span className="text-duck-dark/70 dark:text-foreground/70">API Token</span>
|
||||||
|
<Input
|
||||||
|
className="h-11 bg-background/60 border-duck-dark/20 text-duck-dark placeholder:text-duck-dark/40"
|
||||||
|
type="password"
|
||||||
|
value={apiToken}
|
||||||
|
onChange={(ev) => setApiToken(ev.target.value)}
|
||||||
|
placeholder="apify_api_..."
|
||||||
|
/>
|
||||||
|
<span className="text-xs text-duck-dark/40 dark:text-foreground/40">
|
||||||
|
Get your token at{' '}
|
||||||
|
<a href="https://console.apify.com/account/integrations" target="_blank" rel="noopener noreferrer" className="text-duck-teal underline">
|
||||||
|
console.apify.com/account/integrations
|
||||||
|
</a>
|
||||||
|
</span>
|
||||||
|
</Label>
|
||||||
|
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
onClick={handleSave}
|
||||||
|
disabled={isSaving || !apiToken.trim()}
|
||||||
|
className="w-full h-11 bg-duck-yellow hover:bg-duck-yellow/90 text-duck-teal font-bold transition-all duration-200 hover:scale-105 cursor-pointer disabled:opacity-50 disabled:hover:scale-100"
|
||||||
|
>
|
||||||
|
{isSaving ? 'Saving...' : 'Save'}
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
};
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
import { useMemo } from 'react';
|
import { useMemo } from 'react';
|
||||||
import { Puzzle, KeyRound, UserCircle, MessageCircle, Globe } from 'lucide-react';
|
import { Puzzle, KeyRound, UserCircle, MessageCircle, Globe, Wrench } from 'lucide-react';
|
||||||
import type { LayoutNode, PanelComponents } from 'officerdev';
|
import type { LayoutNode, PanelComponents } from 'officerdev';
|
||||||
import { WorkspaceLayout } from 'officerdev';
|
import { WorkspaceLayout } from 'officerdev';
|
||||||
import { useAuth } from 'hooks/useAuth';
|
import { useAuth } from 'hooks/useAuth';
|
||||||
@@ -16,6 +16,7 @@ import { TelegramAccount } from './TelegramAccount';
|
|||||||
import { WhatsAppBotConfig } from './WhatsAppBotConfig';
|
import { WhatsAppBotConfig } from './WhatsAppBotConfig';
|
||||||
import { WhatsAppAccount } from './WhatsAppAccount';
|
import { WhatsAppAccount } from './WhatsAppAccount';
|
||||||
import { BrowserRelay } from './BrowserRelay';
|
import { BrowserRelay } from './BrowserRelay';
|
||||||
|
import { ApifyConfig } from './ApifyConfig';
|
||||||
|
|
||||||
const GLOBAL_KEY = 'INTEGRATIONS_SETTINGS_SELECTED';
|
const GLOBAL_KEY = 'INTEGRATIONS_SETTINGS_SELECTED';
|
||||||
const TAB_KEY = 'INTEGRATIONS_SETTINGS_TAB';
|
const TAB_KEY = 'INTEGRATIONS_SETTINGS_TAB';
|
||||||
@@ -49,6 +50,13 @@ const enterpriseSections: SettingsSection[] = [
|
|||||||
description: 'WhatsApp Web connection',
|
description: 'WhatsApp Web connection',
|
||||||
content: <WhatsAppBotConfig />,
|
content: <WhatsAppBotConfig />,
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
key: 'apify',
|
||||||
|
icon: Wrench,
|
||||||
|
title: 'Apify',
|
||||||
|
description: 'API token for web scraping actors',
|
||||||
|
content: <ApifyConfig />,
|
||||||
|
},
|
||||||
];
|
];
|
||||||
|
|
||||||
const personalSections: SettingsSection[] = [
|
const personalSections: SettingsSection[] = [
|
||||||
|
|||||||
@@ -33,6 +33,40 @@ integrationsRouter.get('/', async (ctx) => {
|
|||||||
return ctx.json([]);
|
return ctx.json([]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// --- Enterprise: Apify config (Super Admin only) ---
|
||||||
|
|
||||||
|
type ApifyConfig = { apiToken: string };
|
||||||
|
|
||||||
|
export const readApifyConfig = async (): Promise<ApifyConfig | null> => {
|
||||||
|
const integration = await getServerIntegration('apify');
|
||||||
|
if (!integration) return null;
|
||||||
|
const config = integration.config as Record<string, unknown>;
|
||||||
|
if (!config.apiToken) return null;
|
||||||
|
return config as unknown as ApifyConfig;
|
||||||
|
};
|
||||||
|
|
||||||
|
integrationsRouter.get('/apify/config', async (ctx) => {
|
||||||
|
const user = ctx.get('user');
|
||||||
|
if (user.role !== 'Super Admin') throw FORBIDDEN();
|
||||||
|
return ctx.json(await readApifyConfig());
|
||||||
|
});
|
||||||
|
|
||||||
|
integrationsRouter.put('/apify/config', async (ctx) => {
|
||||||
|
const user = ctx.get('user');
|
||||||
|
if (user.role !== 'Super Admin') throw FORBIDDEN();
|
||||||
|
|
||||||
|
const body = ctx.get('body') as { apiToken?: string };
|
||||||
|
const config = { apiToken: body.apiToken ?? '' };
|
||||||
|
|
||||||
|
await upsertServerIntegration('apify', config);
|
||||||
|
return ctx.json(config);
|
||||||
|
});
|
||||||
|
|
||||||
|
integrationsRouter.get('/apify/status', async (ctx) => {
|
||||||
|
const config = await readApifyConfig();
|
||||||
|
return ctx.json({ configured: !!config?.apiToken });
|
||||||
|
});
|
||||||
|
|
||||||
// --- Enterprise: Google OAuth config (Super Admin only) ---
|
// --- Enterprise: Google OAuth config (Super Admin only) ---
|
||||||
|
|
||||||
integrationsRouter.get('/google/config', async (ctx) => {
|
integrationsRouter.get('/google/config', async (ctx) => {
|
||||||
|
|||||||
@@ -190,6 +190,16 @@ async function ensureGoogleTokenFile(userId: number, email: string): Promise<str
|
|||||||
return filePath;
|
return filePath;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function getApifyToken(): Promise<string> {
|
||||||
|
try {
|
||||||
|
const integration = await getServerIntegration('apify');
|
||||||
|
const config = integration?.config as Record<string, string> | undefined;
|
||||||
|
return config?.apiToken ?? '';
|
||||||
|
} catch {
|
||||||
|
return '';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
async function getBrowserRelayEnv(userId: number): Promise<Record<string, string>> {
|
async function getBrowserRelayEnv(userId: number): Promise<Record<string, string>> {
|
||||||
const port = getRelayPort();
|
const port = getRelayPort();
|
||||||
if (!port) return {};
|
if (!port) return {};
|
||||||
@@ -265,6 +275,7 @@ export async function spawnPi(
|
|||||||
const googleConfigHost = await ensureGoogleConfigFile();
|
const googleConfigHost = await ensureGoogleConfigFile();
|
||||||
await ensureGoogleTokenFile(sandbox.userId, sandbox.email);
|
await ensureGoogleTokenFile(sandbox.userId, sandbox.email);
|
||||||
const browserRelayEnv = await getBrowserRelayEnv(sandbox.userId);
|
const browserRelayEnv = await getBrowserRelayEnv(sandbox.userId);
|
||||||
|
const apifyToken = await getApifyToken();
|
||||||
|
|
||||||
const envFlags = [
|
const envFlags = [
|
||||||
'-e', `HOME=${containerHome}`,
|
'-e', `HOME=${containerHome}`,
|
||||||
@@ -276,6 +287,7 @@ export async function spawnPi(
|
|||||||
'-e', `OFFICER_GOOGLE_CONFIG_PATH=/officer/google-oauth.json`,
|
'-e', `OFFICER_GOOGLE_CONFIG_PATH=/officer/google-oauth.json`,
|
||||||
'-e', `OFFICER_GOOGLE_TOKEN_PATH=/officer/user/integrations/google.json`,
|
'-e', `OFFICER_GOOGLE_TOKEN_PATH=/officer/user/integrations/google.json`,
|
||||||
'-e', `OFFICER_EMAIL_DB=/officer/emails.db`,
|
'-e', `OFFICER_EMAIL_DB=/officer/emails.db`,
|
||||||
|
...(apifyToken ? ['-e', `OFFICER_APIFY_TOKEN=${apifyToken}`] : []),
|
||||||
...Object.entries(browserRelayEnv).flatMap(([k, v]) => ['-e', `${k}=${v}`]),
|
...Object.entries(browserRelayEnv).flatMap(([k, v]) => ['-e', `${k}=${v}`]),
|
||||||
];
|
];
|
||||||
|
|
||||||
@@ -321,6 +333,7 @@ export async function spawnPi(
|
|||||||
const googleConfigPath = await ensureGoogleConfigFile();
|
const googleConfigPath = await ensureGoogleConfigFile();
|
||||||
const googleTokenPath = await ensureGoogleTokenFile(userId, email);
|
const googleTokenPath = await ensureGoogleTokenFile(userId, email);
|
||||||
const browserRelayEnv = await getBrowserRelayEnv(userId);
|
const browserRelayEnv = await getBrowserRelayEnv(userId);
|
||||||
|
const apifyTokenLocal = await getApifyToken();
|
||||||
|
|
||||||
proc = Bun.spawn(args, {
|
proc = Bun.spawn(args, {
|
||||||
cwd,
|
cwd,
|
||||||
@@ -339,6 +352,7 @@ export async function spawnPi(
|
|||||||
OFFICER_GOOGLE_CONFIG_PATH: googleConfigPath,
|
OFFICER_GOOGLE_CONFIG_PATH: googleConfigPath,
|
||||||
OFFICER_GOOGLE_TOKEN_PATH: googleTokenPath,
|
OFFICER_GOOGLE_TOKEN_PATH: googleTokenPath,
|
||||||
OFFICER_EMAIL_DB: join(DATA_PATH, email, 'emails.db'),
|
OFFICER_EMAIL_DB: join(DATA_PATH, email, 'emails.db'),
|
||||||
|
...(apifyTokenLocal ? { OFFICER_APIFY_TOKEN: apifyTokenLocal } : {}),
|
||||||
...browserRelayEnv,
|
...browserRelayEnv,
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -54,7 +54,7 @@ function resolveFile(name: string, native: Map<string, string>, global: Map<stri
|
|||||||
}
|
}
|
||||||
|
|
||||||
function isPrivileged(role: string) {
|
function isPrivileged(role: string) {
|
||||||
return role === 'Admin' || role === 'Owner' || role === 'Super Admin';
|
return role === 'Super Admin';
|
||||||
}
|
}
|
||||||
|
|
||||||
export const processesRouter = createRouter();
|
export const processesRouter = createRouter();
|
||||||
|
|||||||
@@ -41,7 +41,7 @@ function mergeConfig(native: Record<string, string>, global: Record<string, stri
|
|||||||
}
|
}
|
||||||
|
|
||||||
function isPrivileged(role: string) {
|
function isPrivileged(role: string) {
|
||||||
return role === 'Admin' || role === 'Owner' || role === 'Super Admin';
|
return role === 'Super Admin';
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function readResourceConfig(name: string): Promise<Record<string, string>> {
|
export async function readResourceConfig(name: string): Promise<Record<string, string>> {
|
||||||
|
|||||||
@@ -41,7 +41,7 @@ export async function readSkillDirs(dir: string): Promise<Map<string, string>> {
|
|||||||
type Scope = 'native' | 'global' | 'user';
|
type Scope = 'native' | 'global' | 'user';
|
||||||
|
|
||||||
function isPrivileged(role: string) {
|
function isPrivileged(role: string) {
|
||||||
return role === 'Admin' || role === 'Owner' || role === 'Super Admin';
|
return role === 'Super Admin';
|
||||||
}
|
}
|
||||||
|
|
||||||
function resolveScope(dirName: string, native: Map<string, string>, global: Map<string, string>, user: Map<string, string>): Scope {
|
function resolveScope(dirName: string, native: Map<string, string>, global: Map<string, string>, user: Map<string, string>): Scope {
|
||||||
|
|||||||
@@ -73,7 +73,7 @@ function resolveFile(name: string, native: Map<string, string>, global: Map<stri
|
|||||||
}
|
}
|
||||||
|
|
||||||
function isPrivileged(role: string) {
|
function isPrivileged(role: string) {
|
||||||
return role === 'Admin' || role === 'Owner' || role === 'Super Admin';
|
return role === 'Super Admin';
|
||||||
}
|
}
|
||||||
|
|
||||||
export const tasksRouter = createRouter();
|
export const tasksRouter = createRouter();
|
||||||
|
|||||||
@@ -0,0 +1,217 @@
|
|||||||
|
import { createRouter } from '../../create-router';
|
||||||
|
import { readdir, mkdir, rm } from 'node:fs/promises';
|
||||||
|
import { join, dirname } from 'node:path';
|
||||||
|
import { getNativeToolsDir, getGlobalToolsDir, getUserToolsDir } from '../../data-path';
|
||||||
|
|
||||||
|
type Frontmatter = {
|
||||||
|
name: string;
|
||||||
|
description: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
export function parseFrontmatter(raw: string): { frontmatter: Frontmatter; body: string; rawYaml: string } {
|
||||||
|
const match = raw.match(/^---\n([\s\S]*?)\n---\n?([\s\S]*)$/);
|
||||||
|
if (!match) return { frontmatter: { name: '', description: '' }, body: raw, rawYaml: '' };
|
||||||
|
|
||||||
|
const yaml = match[1]!;
|
||||||
|
const body = match[2]!;
|
||||||
|
|
||||||
|
const name = yaml.match(/^name:\s*(.+)$/m)?.[1]?.trim() ?? '';
|
||||||
|
const description = yaml.match(/^description:\s*(.+)$/m)?.[1]?.trim() ?? '';
|
||||||
|
|
||||||
|
return { frontmatter: { name, description }, body, rawYaml: yaml };
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function readToolDirs(dir: string): Promise<Map<string, string>> {
|
||||||
|
const result = new Map<string, string>();
|
||||||
|
try {
|
||||||
|
const entries = await readdir(dir, { withFileTypes: true });
|
||||||
|
for (const entry of entries) {
|
||||||
|
if (!entry.isDirectory()) continue;
|
||||||
|
const toolFile = join(dir, entry.name, 'TOOL.md');
|
||||||
|
if (await Bun.file(toolFile).exists()) {
|
||||||
|
result.set(entry.name, toolFile);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
// directory doesn't exist yet
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
type Scope = 'native' | 'global' | 'user';
|
||||||
|
|
||||||
|
function resolveScope(dirName: string, native: Map<string, string>, global: Map<string, string>, user: Map<string, string>): Scope {
|
||||||
|
if (user.has(dirName)) return 'user';
|
||||||
|
if (global.has(dirName)) return 'global';
|
||||||
|
return 'native';
|
||||||
|
}
|
||||||
|
|
||||||
|
function resolveFile(name: string, native: Map<string, string>, global: Map<string, string>, user: Map<string, string>): { filePath: string; scope: Scope } | null {
|
||||||
|
if (user.has(name)) return { filePath: user.get(name)!, scope: 'user' };
|
||||||
|
if (global.has(name)) return { filePath: global.get(name)!, scope: 'global' };
|
||||||
|
if (native.has(name)) return { filePath: native.get(name)!, scope: 'native' };
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function isPrivileged(role: string) {
|
||||||
|
return role === 'Super Admin';
|
||||||
|
}
|
||||||
|
|
||||||
|
export const toolsRouter = createRouter();
|
||||||
|
|
||||||
|
toolsRouter.get('/', async (ctx) => {
|
||||||
|
const user = ctx.get('user');
|
||||||
|
const nativeTools = await readToolDirs(getNativeToolsDir());
|
||||||
|
const globalTools = await readToolDirs(getGlobalToolsDir());
|
||||||
|
const userTools = await readToolDirs(getUserToolsDir(user.email));
|
||||||
|
|
||||||
|
const merged = new Map(nativeTools);
|
||||||
|
for (const [name, path] of globalTools) merged.set(name, path);
|
||||||
|
for (const [name, path] of userTools) merged.set(name, path);
|
||||||
|
|
||||||
|
const tools = await Promise.all(
|
||||||
|
Array.from(merged.entries()).map(async ([dirName, filePath]) => {
|
||||||
|
const raw = await Bun.file(filePath).text();
|
||||||
|
const { frontmatter } = parseFrontmatter(raw);
|
||||||
|
const scope = resolveScope(dirName, nativeTools, globalTools, userTools);
|
||||||
|
return {
|
||||||
|
dirName,
|
||||||
|
name: frontmatter.name || dirName,
|
||||||
|
description: frontmatter.description,
|
||||||
|
scope,
|
||||||
|
filePath,
|
||||||
|
};
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
return ctx.json(tools);
|
||||||
|
});
|
||||||
|
|
||||||
|
toolsRouter.get('/:name', async (ctx) => {
|
||||||
|
const user = ctx.get('user');
|
||||||
|
const name = ctx.req.param('name');
|
||||||
|
|
||||||
|
const nativeTools = await readToolDirs(getNativeToolsDir());
|
||||||
|
const globalTools = await readToolDirs(getGlobalToolsDir());
|
||||||
|
const userTools = await readToolDirs(getUserToolsDir(user.email));
|
||||||
|
|
||||||
|
const resolved = resolveFile(name, nativeTools, globalTools, userTools);
|
||||||
|
if (!resolved) return ctx.text('Not found', 404);
|
||||||
|
|
||||||
|
const raw = await Bun.file(resolved.filePath).text();
|
||||||
|
const { frontmatter, body, rawYaml } = parseFrontmatter(raw);
|
||||||
|
|
||||||
|
const chatMeta = join(dirname(resolved.filePath), 'chat', 'meta.json');
|
||||||
|
const chatSessionId = await Bun.file(chatMeta).json().then((m: { id: string }) => m.id).catch(() => null);
|
||||||
|
|
||||||
|
return ctx.json({
|
||||||
|
name: frontmatter.name || name,
|
||||||
|
description: frontmatter.description,
|
||||||
|
scope: resolved.scope,
|
||||||
|
body,
|
||||||
|
rawFrontmatter: rawYaml,
|
||||||
|
filePath: resolved.filePath,
|
||||||
|
chatSessionId,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
toolsRouter.get('/:name/chat', async (ctx) => {
|
||||||
|
const user = ctx.get('user');
|
||||||
|
const name = ctx.req.param('name');
|
||||||
|
|
||||||
|
const nativeTools = await readToolDirs(getNativeToolsDir());
|
||||||
|
const globalTools = await readToolDirs(getGlobalToolsDir());
|
||||||
|
const userTools = await readToolDirs(getUserToolsDir(user.email));
|
||||||
|
|
||||||
|
const resolved = resolveFile(name, nativeTools, globalTools, userTools);
|
||||||
|
if (!resolved) return ctx.text('Not found', 404);
|
||||||
|
|
||||||
|
const chatDir = join(dirname(resolved.filePath), 'chat');
|
||||||
|
const sessionId = await Bun.file(join(chatDir, 'meta.json')).json().then((m: { id: string }) => m.id).catch(() => null);
|
||||||
|
const messages = await Bun.file(join(chatDir, 'messages.json')).json().catch(() => []);
|
||||||
|
|
||||||
|
return ctx.json({ sessionId, messages });
|
||||||
|
});
|
||||||
|
|
||||||
|
toolsRouter.put('/:name/chat', async (ctx) => {
|
||||||
|
const user = ctx.get('user');
|
||||||
|
const name = ctx.req.param('name');
|
||||||
|
|
||||||
|
const nativeTools = await readToolDirs(getNativeToolsDir());
|
||||||
|
const globalTools = await readToolDirs(getGlobalToolsDir());
|
||||||
|
const userTools = await readToolDirs(getUserToolsDir(user.email));
|
||||||
|
|
||||||
|
const resolved = resolveFile(name, nativeTools, globalTools, userTools);
|
||||||
|
if (!resolved) return ctx.text('Not found', 404);
|
||||||
|
|
||||||
|
if (resolved.scope !== 'user' && !isPrivileged(user.role)) return ctx.text('Forbidden', 403);
|
||||||
|
|
||||||
|
const chatDir = join(dirname(resolved.filePath), 'chat');
|
||||||
|
const { sessionId, messages } = await ctx.req.json<{ sessionId: string; messages: unknown[] }>();
|
||||||
|
|
||||||
|
await mkdir(chatDir, { recursive: true });
|
||||||
|
await Bun.write(join(chatDir, 'messages.json'), JSON.stringify(messages));
|
||||||
|
if (sessionId) await Bun.write(join(chatDir, 'meta.json'), JSON.stringify({ id: sessionId }));
|
||||||
|
|
||||||
|
return ctx.json({ ok: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
toolsRouter.delete('/:name/chat', async (ctx) => {
|
||||||
|
const user = ctx.get('user');
|
||||||
|
const name = ctx.req.param('name');
|
||||||
|
|
||||||
|
const nativeTools = await readToolDirs(getNativeToolsDir());
|
||||||
|
const globalTools = await readToolDirs(getGlobalToolsDir());
|
||||||
|
const userTools = await readToolDirs(getUserToolsDir(user.email));
|
||||||
|
|
||||||
|
const resolved = resolveFile(name, nativeTools, globalTools, userTools);
|
||||||
|
if (!resolved) return ctx.text('Not found', 404);
|
||||||
|
|
||||||
|
if (resolved.scope !== 'user' && !isPrivileged(user.role)) return ctx.text('Forbidden', 403);
|
||||||
|
|
||||||
|
const chatDir = join(dirname(resolved.filePath), 'chat');
|
||||||
|
await rm(chatDir, { recursive: true, force: true });
|
||||||
|
|
||||||
|
return ctx.json({ ok: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
toolsRouter.post('/', async (ctx) => {
|
||||||
|
const user = ctx.get('user');
|
||||||
|
const { name } = await ctx.req.json<{ name: string }>();
|
||||||
|
if (!name?.trim()) return ctx.text('Name is required', 400);
|
||||||
|
|
||||||
|
const dirName = name.trim().toLowerCase().replace(/\s+/g, '-').replace(/[^a-z0-9-]/g, '');
|
||||||
|
if (!dirName) return ctx.text('Invalid name', 400);
|
||||||
|
|
||||||
|
const targetDir = isPrivileged(user.role) ? getGlobalToolsDir() : getUserToolsDir(user.email);
|
||||||
|
const scope: Scope = isPrivileged(user.role) ? 'global' : 'user';
|
||||||
|
|
||||||
|
const dir = join(targetDir, dirName);
|
||||||
|
const filePath = join(dir, 'TOOL.md');
|
||||||
|
|
||||||
|
if (await Bun.file(filePath).exists()) {
|
||||||
|
return ctx.text('Tool already exists', 409);
|
||||||
|
}
|
||||||
|
|
||||||
|
await mkdir(dir, { recursive: true });
|
||||||
|
await Bun.write(filePath, `---\nname: ${name.trim()}\ndescription: \n---\n`);
|
||||||
|
|
||||||
|
return ctx.json({ name: name.trim(), dirName, filePath, scope });
|
||||||
|
});
|
||||||
|
|
||||||
|
toolsRouter.delete('/:name', async (ctx) => {
|
||||||
|
const user = ctx.get('user');
|
||||||
|
const name = ctx.req.param('name');
|
||||||
|
|
||||||
|
const nativeTools = await readToolDirs(getNativeToolsDir());
|
||||||
|
const globalTools = await readToolDirs(getGlobalToolsDir());
|
||||||
|
const userTools = await readToolDirs(getUserToolsDir(user.email));
|
||||||
|
|
||||||
|
const resolved = resolveFile(name, nativeTools, globalTools, userTools);
|
||||||
|
if (!resolved) return ctx.text('Not found', 404);
|
||||||
|
|
||||||
|
if (resolved.scope !== 'user' && !isPrivileged(user.role)) return ctx.text('Forbidden', 403);
|
||||||
|
|
||||||
|
await rm(dirname(resolved.filePath), { recursive: true });
|
||||||
|
return ctx.json({ ok: true });
|
||||||
|
});
|
||||||
@@ -10,6 +10,7 @@ import { usersRouter } from './api/users/users-router';
|
|||||||
import { plansRouter } from './api/plans/plans';
|
import { plansRouter } from './api/plans/plans';
|
||||||
import { skillsRouter } from './api/skills/skills';
|
import { skillsRouter } from './api/skills/skills';
|
||||||
import { tasksRouter } from './api/tasks/tasks';
|
import { tasksRouter } from './api/tasks/tasks';
|
||||||
|
import { toolsRouter } from './api/tools/tools';
|
||||||
import { processesRouter } from './api/processes/processes';
|
import { processesRouter } from './api/processes/processes';
|
||||||
import { sessionsRouter } from './api/sessions/sessions';
|
import { sessionsRouter } from './api/sessions/sessions';
|
||||||
import { scrapeRouter } from './api/scrape/scrape';
|
import { scrapeRouter } from './api/scrape/scrape';
|
||||||
@@ -71,6 +72,7 @@ protectedRouter.route('/users', usersRouter);
|
|||||||
protectedRouter.route('/plans', plansRouter);
|
protectedRouter.route('/plans', plansRouter);
|
||||||
protectedRouter.route('/skills', skillsRouter);
|
protectedRouter.route('/skills', skillsRouter);
|
||||||
protectedRouter.route('/tasks', tasksRouter);
|
protectedRouter.route('/tasks', tasksRouter);
|
||||||
|
protectedRouter.route('/tools', toolsRouter);
|
||||||
protectedRouter.route('/processes', processesRouter);
|
protectedRouter.route('/processes', processesRouter);
|
||||||
protectedRouter.route('/', sessionsRouter);
|
protectedRouter.route('/', sessionsRouter);
|
||||||
protectedRouter.route('/scrape', scrapeRouter);
|
protectedRouter.route('/scrape', scrapeRouter);
|
||||||
|
|||||||
Reference in New Issue
Block a user