Content
# Tool List
**Rust MCP server replacing Claude Code's built-in `WebFetch`, 60x faster for simple URL lookups.**
HTTP fetch + HTML→markdown conversion + SQLite TTL cache performed natively in Rust. Skips LLM summarization step, allowing Claude to directly read markdown.
## Benchmark
10 URLs × 5 runs (Windows 10, typical home internet connection).
| Stack | Average cold | Compared to |
|---|---:|---:|
| **claude-fetch-mcp** (Rust) | **147 ms** | 1× (baseline) |
| Python (`requests` + `BeautifulSoup` + `html2text`) | 1,210 ms | 8.2× slower |
| Claude Code's built-in `WebFetch` (including LLM summarization) | ~9,000 ms | **~60× slower** |
With cache hit, Rust MCP averages **37 ms** (warm). Detailed measurement methods, URL-specific numbers, and limitations are in [`BENCHMARK_REPORT.md`](./BENCHMARK_REPORT.md).
## Why is it faster?
1. **LLM summarization skipped** — Built-in `WebFetch` fetches and summarizes with a Haiku-level model, taking 80~90% of the time. This tool returns markdown directly, allowing Claude to read as needed.
2. **Native Rust** — `reqwest` + `rustls` + `http2` + `pool-keepalive`. TLS uses AES-NI hardware acceleration.
3. **Persistent process** — MCP stdio maintains the process during the shell session, performing TLS handshake and SQLite open only once.
4. **SQLite TTL cache** — Repeated fetches only read from disk (skipping bandwidth and network).
**No assembly optimization**. The actual bottleneck was the LLM summarization layer, not I/O or parsing.
## Installation
### 1. Prerequisites
Claude Code CLI installed and `claude mcp` command available.
### 2. Download binary
Pre-built binaries available for each OS in [Releases](https://github.com/Hostingglobal-Tech/claude-fetch-mcp/releases/latest).
**Linux / macOS (bash)**:
```bash
curl -fsSL https://raw.githubusercontent.com/Hostingglobal-Tech/claude-fetch-mcp/main/install.sh | bash
```
**Windows (PowerShell)**:
```powershell
iwr -useb https://raw.githubusercontent.com/Hostingglobal-Tech/claude-fetch-mcp/main/install.ps1 | iex
```
### 3. Build from source (optional)
```bash
git clone https://github.com/Hostingglobal-Tech/claude-fetch-mcp.git
cd claude-fetch-mcp
cargo build --release
# Result: target/release/claude-fetch-mcp(.exe)
```
Required Rust version: **1.75+** (edition 2021).
### 4. Register with Claude Code
```bash
claude mcp add --scope user fetch-rs -- /full/path/to/claude-fetch-mcp
# Verify
claude mcp list
# fetch-rs: /full/path/to/claude-fetch-mcp - ✓ Connected
```
## Provided Tools
| Tool | Purpose |
|---|---|
| `fetch_url` | HTTP metadata + short snippet + title. Verify page existence / simple lookup |
| `fetch_markdown` | Full HTML → markdown conversion. For Claude to read the body |
| `fetch_raw` | Raw body (JSON / plain text / crawler feed) |
| `cache_stats` | Cache hit/miss/entries statistics |
| `cache_purge` | Remove entries past TTL |
Common arguments:
- `url` (required) — http / https
- `ttl_secs` (default 3600) — Cache expiration time
- `bypass_cache` (default false) — Force refetch
- `max_bytes` — Return body size limit (default 1 MB)
## When to use and when not to
**Recommended**:
- Static, small pages (< 50 KB) like docs / API docs / README / RFC
- Repeated lookups (same URL multiple times) — cache hit
- JSON / plain text feeds
- When exact original text is needed (LLM summarization may distort)
**Not recommended / keep built-in `WebFetch`**:
- Large HTML (> 50 KB) — summarization helps save context tokens
- JS-rendered pages (SPAs)
- Login / cookie-required pages (Confluence, Notion, etc. — use dedicated MCP)
The two tools are **complementary**. Let Claude decide and choose automatically.
## Configuration
### Environment Variables
| Variable | Default value | Description |
|---|---|---|
| `CLAUDE_FETCH_CACHE` | `%LOCALAPPDATA%\claude-fetch-mcp\cache.sqlite` (Windows) / `~/.cache/claude-fetch-mcp/cache.sqlite` (Linux/macOS) | SQLite cache file path |
| `CLAUDE_FETCH_LOG` | `info` | Tracing log level (`trace`/`debug`/`info`/`warn`/`error`) |
### Structure
```
Claude Code
│ MCP JSON-RPC stdio
▼
claude-fetch-mcp (single binary, ~8.5 MB)
├─ reqwest + rustls-tls (HTTP/2, gzip, brotli, keep-alive pool)
├─ html2md (HTML → markdown)
├─ rusqlite (SQLite WAL, SHA256 key, TTL)
└─ 5 exposed tools (see table above)
```
## Transparency
- **No API keys / passwords / auth tokens included**. Source code grep confirmed.
- **No external service calls**. No direct calls to Anthropic / OpenAI / third-party APIs. Only pure HTTP GET.
- **No telemetry**. No data sent to any external server.
- **Cache stored locally**. No external synchronization.
- **User-Agent**: `claude-fetch-mcp/<version>` (no personal identifiable information).
Dependencies are all public crates from [crates.io](https://crates.io). Sensitive handling and network communication are done by `reqwest` + `rustls` (Rust ecosystem standards).
## Limitations
1. **No JS-rendered page support**. Headless browser territory (separate tool).
2. **No robots.txt / rate limit compliance**. Use caution when automating against commercial sites.
3. **No authentication-required pages**. Use dedicated MCP for Confluence / Notion / Google Docs, etc.
4. **HTML → markdown quality**. Pages with deep DOM (e.g., Wikipedia, Hacker News) may produce large markdown. Improvement using `readability-rs` is being considered.
## Reproduction
```bash
# Run benchmark
cd claude-fetch-mcp
cargo build --release
./target/release/fetch-bench --urls bench_urls.txt --runs 5 --out bench_rust.json
# Compare with Python baseline (optional)
pip install requests beautifulsoup4 html2text
python bench_python.py bench_urls.txt 5 bench_python.json
```
Input URL list is in [`bench_urls.txt`](./bench_urls.txt). Encouraged to re-measure in your environment.
## License
MIT. [`LICENSE`](./LICENSE).
## Contributions
Issues / PRs welcome. Language can be Korean or English.
