26 lines
1.0 KiB
Markdown
26 lines
1.0 KiB
Markdown
---
|
|
name: web_fetch
|
|
label: Web Fetch
|
|
description: Fetch and return the text content of a URL. Tries to get clean readable content using a cascade of strategies in order: appending .md to the URL, fetching a /llms.txt discovery file, requesting plain text via Accept header, then falling back to stripping HTML. Use when the user provides a URL and wants to read, summarize, or extract information from web content.
|
|
language: typescript
|
|
inputs:
|
|
url:
|
|
type: string
|
|
description: The URL to fetch content from
|
|
---
|
|
|
|
# Web Fetch
|
|
|
|
Fetches web content with cascading fallback strategies to get the cleanest possible text.
|
|
|
|
## Strategies (in order)
|
|
|
|
1. **Markdown version** — Appends `.md` to the URL (works on GitHub, many docs sites)
|
|
2. **llms.txt discovery** — Checks `/llms.txt` at the root (sites that publish LLM-friendly content)
|
|
3. **Plain text request** — Sends `Accept: text/plain` header
|
|
4. **HTML strip** — Fetches HTML and strips tags, scripts, and styles
|
|
|
|
## Output
|
|
|
|
Returns the text content with a note indicating which strategy succeeded.
|