wiki-mcp

<h3 align="center">wiki-mcp 🔥</h3> <br /> <p align="center"> <img src="https://img.shields.io/badge/MCP-2025--06--18-D97757?logo=anthropic&logoColor=white" alt="MCP" /> <img src="https://img.shields.io/badge/Cloudflare-Workers-F38020?logo=cloudflare&logoColor=white" alt="Cloudflare Workers" /> <img src="https://img.shields.io/badge/TypeScript-5.4-3178C6?logo=typescript&logoColor=white" alt="TypeScript" /> <img src="https://img.shields.io/badge/Vitest-1.6-6E9F18?logo=vitest&logoColor=white" alt="Vitest" /> <img src="https://img.shields.io/badge/pnpm-10-F69220?logo=pnpm&logoColor=white" alt="pnpm" /> <img src="https://img.shields.io/badge/Node-20-339933?logo=node.js&logoColor=white" alt="Node" /> <img src="https://img.shields.io/badge/Biome-linter-60A5FA?logo=biome&logoColor=white" alt="Biome" /> </p> --- ## 🔥 About Model Context Protocol server that exposes an LLM-wiki vault (the [Karpathy pattern](https://karpathy.bearblog.dev)) to Claude clients. **Read** access for the four primary tools, plus a single **write** path (`wiki_upload`) that commits arbitrary files into the wiki repo. Runs free on Cloudflare Workers — point it at your own private GitHub repo and deploy your own worker. Companion to [`wikionfire`](https://github.com/IsaiaScope/wikionfire) but agnostic — it works with any wiki shaped like the Karpathy pattern. ## ⚠️ 1.0.0 — Breaking changes - `wiki_context` now returns Markdown text. Schema, indexes, and the recent log are no longer in the response — fetch them via the existing `wiki://schema`, `wiki://index/all`, `wiki://log/recent` resources. - `wiki_context` input: `include_log` removed; `expand_links` added (default `false` — wikilink expansion is now opt-in). - `wiki_search`, `wiki_list`, `wiki_fetch` use short JSON keys (`p`, `t`, `s`, `sn`, `c`, `fm`, `err`). `wiki_list` is now grouped by domain → type under `g`. - Per-domain overview lines (`wiki://overview/<domain>`) are bare paths instead of `[[path]] — Title`. - `WIKI_PRIME_VOCAB=full` instructions cap reduced from 50 to 20 trigger titles; trigger vocabulary no longer appears in tool descriptions in any mode. Migration: callers parsing JSON from `wiki_context` must switch to text consumption. Callers reading `items` / `total` / `frontmatter` / `error` / `path` / `title` / `score` / `snippet` keys from tabular tools must rewrite to the short keys above. Schema/indexes/log are still available — just from MCP resources rather than embedded in `wiki_context`. ## 🛠️ Six MCP tools | | Tool | Purpose | Mode | |-|------|---------|------| | 🎁 | **`wiki_context(question, domain?, budget_tokens?, expand_links?)`** | ranked hits + 1-hop link expansion (opt-in via `expand_links: true`). Returns Markdown text — schema, indexes, and recent log are no longer in the response; fetch them via `wiki://schema`, `wiki://index/all`, `wiki://log/recent` resources. Hits include `truncated: boolean` when bodies are clipped to fit `budget_tokens`. | read | | 🔍 | **`wiki_search(query, domain?, limit?)`** | two-stage ranked keyword search: path-token shortlist + body/frontmatter re-rank (title, aliases, tags, entities, concepts, headings). Stage 2 fetches up to `limit*2` bodies. Output is terse JSON: `{p,t,sn?,s}` per row (path, title, snippet, score). | read | | 📄 | **`wiki_fetch(paths[])`** | batch read pages by exact path (max 20). Paths must exist in current snapshot. Unknown paths return per-path `err` field (partial success). `SENSITIVE_FRONTMATTER_KEYS` env strips listed keys from output. Output is terse JSON: `{p,c,fm}` for success rows, `{p,err}` for per-path failures. | read | | 🗂️ | **`wiki_list(domain?, type?, tag?, entity?, concept?, limit?, offset?)`** | structured directory listing with optional metadata filters (case-insensitive). Returns `{g, tot, off, lim, tr}` where `g` is `domain → type → [{p,t}]`. Default limit 200, max 1000. | read | | 🧱 | **`wiki_read_raw(path)`** | read a binary or text file under `{domain}/raw/{subpath}` as base64. Path must be in snapshot and live under `raw/`. | read | | ⬆️ | **`wiki_upload(domain, subpath, content_base64, message?)`** | upload any file (PDF, image, text, binary) under `{domain}/raw/{subpath}` — 25 MB cap, requires `contents:write` | **write** | ## ✨ Three MCP prompts (slash commands) Auto-generated from snapshot — clients render as user-invokable shortcuts. Bodies steer the model toward the right tool sequence; they are not themselves answers. | Prompt | Args | Steers toward | |--------|------|---------------| | `wiki_summary` | `domain?` | `wiki_context` + `wiki://overview` | | `wiki_recent` | `domain?` | `wiki://log/recent` digest | | `wiki_related` | `path` | `wiki_fetch` → `wiki_search`/`wiki_list` chain | ## 📚 Six MCP resources | | Resource | Source | |-|----------|--------| | 📜 | `wiki://schema` | `CLAUDE.md` + `docs/llm-wiki.md` + per-domain `CLAUDE.md` concatenated | | 🗂️ | `wiki://index/all` | every discovered domain `index.md` | | 📅 | `wiki://log/recent` | last 50 log entries across all domains | | 📄 | `wiki://page/{domain}/{type}/{slug}` | individual page template | | 🌐 | `wiki://overview` | dynamic top-level inventory (built from snapshot) | | 🌐 | `wiki://overview/{domain}` | per-domain page list grouped by type | **Wiki layout is discovered at runtime** — any top-level dir with `index.md` + `log.md` + `wiki/` is a domain. Adding new domains or page types in your repo requires zero code change here. ## 🗂️ Source layout The worker is split into seven feature-scoped modules. Each has its own README. ``` src/ ├── index.ts worker entry — HTTP routing ├── server.ts createServer + buildDeps (state lives here) ├── env.ts config + helpers ├── types.ts shared types │ ├── auth/ 📖 README — bearer-token guard ├── github/ 📖 README — REST + raw fetch + write ├── wiki/ 📖 README — discover + frontmatter + wikilinks ├── search/ 📖 README — BM25 + budget + bundle ├── prime/ 📖 README — dynamic instructions + overview ├── mcp/ 📖 README — tool + resource registration └── upload/ 📖 README — write path + sanitizer ``` | | Module | Owns | |-|--------|------| | 🔐 | [`auth/`](src/auth/) | constant-time bearer check, 401 helper, overlap-token rotation | | 🐙 | [`github/`](src/github/) | tree fetch (TTL-cached), SHA-pinned raw URL builder, contents API write | | 📚 | [`wiki/`](src/wiki/) | runtime domain discovery, YAML frontmatter, `[[link\|alias#section]]` parsing | | 🔍 | [`search/`](src/search/) | BM25 ranking, token budget, full bundle assembly | | 🪄 | [`prime/`](src/prime/) | dynamic `instructions`, tool descriptions, `wiki://overview` resources | | 🔌 | [`mcp/`](src/mcp/) | tool + resource registration, zod schemas, error surface | | ⬆️ | [`upload/`](src/upload/) | write orchestrator, subpath sanitizer, size cap | ## ⚙️ Setup ```bash # 1. Clone git clone https://github.com/<your-gh-user>/wiki-mcp.git cd wiki-mcp pnpm install pnpm prepare # installs husky hooks # 2. Configure wrangler.toml [vars].GITHUB_REPO to point at your wiki repo. # Example: GITHUB_REPO = "alice/my-wiki" # WIKI_SERVER_NAME = "alices-wiki" # 3. Cloudflare login + secrets pnpm exec wrangler login # one-time browser OAuth openssl rand -hex 32 # copy this token pnpm exec wrangler secret put MCP_BEARER # paste the openssl hex pnpm exec wrangler secret put GITHUB_TOKEN # paste a fine-grained PAT # 4. Deploy pnpm deploy # => https://wiki-mcp.<your-subdomain>.workers.dev # 5. Smoke-test curl https://wiki-mcp.<your-subdomain>.workers.dev/health # => {"ok":true,"at":"..."} ``` Requires pnpm 10+ and Node 20+. The worker runs with `compatibility_flags = ["nodejs_compat"]` (see `wrangler.toml`) — needed because `gray-matter` uses Node's `Buffer`. `[env.X]` blocks do **not** inherit this flag, so it is duplicated under `[env.dev]`; mirror it in any new environment you add. ### 🔑 GitHub PAT (fine-grained) 1. github.com → Settings → Developer settings → Personal access tokens → Fine-grained tokens → Generate new. 2. Resource owner: your user. 3. Repository access: **only** the wiki repo. 4. Permissions → Repository permissions → **Contents: Read and write** (write is required by `wiki_upload`; leave at read-only if you do not plan to use that tool). 5. Expiration: 1 year. Copy the token (shown once). ## 🔌 Add the connector to Claude ### Claude Code Add to `~/.claude.json` or project-level `.mcp.json`: ```json { "mcpServers": { "wiki": { "type": "http", "url": "https://wiki-mcp.<your-subdomain>.workers.dev/mcp", "headers": { "Authorization": "Bearer <your-bearer-token>" } } } } ``` Restart Claude Code. Verify with `/mcp`. ### Claude Desktop Edit `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or the Windows equivalent, using the same JSON shape above. Restart the app. ### claude.ai web Settings → Connectors → Add Custom Connector: - **Name**: `wiki` - **URL**: `https://wiki-mcp.<your-subdomain>.workers.dev/mcp` - **Auth**: Bearer, value `<your-bearer-token>` Create a Project, attach the connector, and paste project-level instructions such as: > You have access to my personal knowledge wiki via the "wiki" MCP connector. Before answering questions that may involve my entities, concepts, or past sources, call `wiki_context`. Cite with `[[path]]` per the wiki convention. ## ⚙️ Configuration surface Everything is env-driven. Fork this repo and point it at your wiki — no code changes required. | | Var / Secret | Set via | Purpose | |-|--------------|---------|---------| | 🐙 | `GITHUB_REPO` | `wrangler.toml [vars]` | Source repo `owner/name` | | 🌿 | `GITHUB_BRANCH` | `wrangler.toml [vars]` | Branch to read (default `main`) | | 🏷️ | `WIKI_SERVER_NAME` | `wrangler.toml [vars]` | Server display name (in `instructions`) | | ⏱️ | `CACHE_TTL_SECONDS` | `wrangler.toml [vars]` | Snapshot cache TTL (default `60`) | | 📜 | `SCHEMA_GLOBS` | `wrangler.toml [vars]` | Comma-list of schema file globs | | 🧱 | `DOMAIN_REQUIRED_FILES` | `wrangler.toml [vars]` | Files that mark a dir as a domain | | 📏 | `MAX_UPLOAD_BYTES` | `wrangler.toml [vars]` | Max upload size in bytes (default `26214400` = 25 MB) | | 📁 | `RAW_FOLDER` | `wrangler.toml [vars]` | Subfolder under each domain for `wiki_upload` (default `raw`) | | 🪄 | `WIKI_PRIME_VOCAB` | `wrangler.toml [vars]` | Priming privacy: `structural` (default), `full`, or `off` | | 👋 | `WIKI_PRIME_GREETING` | `wrangler.toml [vars]` | Optional one-line greeting prepended to instructions and overview | | 🚫 | `SENSITIVE_FRONTMATTER_KEYS` | `wrangler.toml [vars]` | CSV of frontmatter keys stripped from `wiki_fetch` output (e.g. `password,api_key`) | | 🪟 | `SKIP_TOP_DIRS` | `wrangler.toml [vars]` | CSV override of skipped top-level dirs (default `.git,.github,docs,mcp,node_modules,.obsidian,.trash`) | | 💾 | `BODY_CACHE_MAX` | `wrangler.toml [vars]` | Max LRU entries for the per-page body cache (default `800`). Raise for vaults >1k pages. | | 🔐 | `MCP_BEARER` | `wrangler secret put` | Client auth bearer token | | 🔁 | `MCP_BEARER_NEXT` | `wrangler secret put` | Optional overlap token for rotation | | 🐙 | `GITHUB_TOKEN` | `wrangler secret put` | GitHub PAT — `contents:read` minimum, `contents:write` for `wiki_upload` | ## 🪄 Server priming On every `initialize`, the server emits a dynamic `instructions` field computed from your wiki's actual shape (domains, types, page counts). Privacy is controlled by `WIKI_PRIME_VOCAB`: | | Mode | Behavior | |-|------|----------| | 🛡️ | `structural` (default) | per-domain page counts and type breakdown — no titles in passive surfaces | | 🔥 | `full` | titles injected into instructions, capped at 20; trigger vocabulary no longer appears in tool descriptions | | 🔇 | `off` | minimal greeting only, no enumeration | Two overview resources are always exposed: - `wiki://overview` — domain map with per-domain slice URIs - `wiki://overview/{domain}` — page listing for one domain — bare paths, one per line See [`src/prime/`](src/prime/) for the full design. ### 🛡️ Leak-surface audit Privacy mode (`WIKI_PRIME_VOCAB`) controls **priming surfaces** — passive content the server emits without being explicitly called. It does **not** silence tool returns; those always carry the data the agent asked for. | Surface | `structural` (default) | `full` | `off` | |---------|-----------------------|--------|-------| | `serverInfo.instructions` | domain names + counts | + 20 prettified titles | minimal one-liner | | Tool descriptions | static + domain list + when-to-use | static (trigger vocab no longer injected) | static | | `wiki://overview` | domain names + counts | + per-domain title list | suppressed | | `wiki://overview/{d}` | full per-domain titles | full | suppressed | | Tool returns (hits, snippets, log, frontmatter) | **always full body** | full | full | To strip sensitive frontmatter keys at the tool layer regardless of vocab mode, set `SENSITIVE_FRONTMATTER_KEYS`. Activity-log inclusion in `wiki_context` has been removed — use the `wiki://log/recent` resource instead. ## 🧪 Development ```bash pnpm dev # wrangler dev, local server on :8787 pnpm test # vitest (219 tests) pnpm test:coverage # with coverage report pnpm typecheck # tsc --noEmit pnpm lint # ultracite check (biome under the hood) pnpm fix # ultracite fix ``` **219 tests** across unit, integration, and contract layers. Mocked GitHub fetch reads from `test/fixtures/vault/` — a synthetic mini-vault safe to be public. ## 🛠️ Tooling | | Tool | Role | |-|------|------| | 📦 | **pnpm** 10+ | package manager | | 🧹 | **biome** + **ultracite** | linter + formatter | | 🪝 | **husky** + **lint-staged** | pre-commit checks | | 🔢 | **post-commit hook** | auto-bumps `package.json` version (patch by default; `feat:` → minor; `!:` / `BREAKING CHANGE` → major) | ## 🚀 CI/CD + branch flow Two protected branches, no direct pushes: - **`dev`** — default branch, integration target. PRs must pass the `test` job (typecheck + vitest). - **`prod`** — release branch. PR merges from `dev` trigger deploy to Cloudflare Workers. ``` feature branch ──PR──► dev (CI: test + deploy-dev) ──PR──► prod (CI: test + deploy) ``` `.github/workflows/deploy.yml`: - `pull_request` on `dev` or `prod` → runs `pnpm typecheck` + `pnpm test` - `push` to `prod` (only via PR merge) → runs tests then `wrangler deploy` Required repo secret: `CLOUDFLARE_API_TOKEN` in Settings → Secrets → Actions (single token covers both workers — same Cloudflare account). Per-environment secrets are independent — set them once each before first deploy: ```bash # Production pnpm exec wrangler secret put MCP_BEARER pnpm exec wrangler secret put GITHUB_TOKEN # Dev (separate worker, separate secrets) pnpm exec wrangler secret put MCP_BEARER --env dev pnpm exec wrangler secret put GITHUB_TOKEN --env dev ``` Use a different `MCP_BEARER` per environment so a leaked dev token can't read prod and vice versa. Protection rules on both branches: require PR, require `test` check, no force push, no deletion, admins enforced. ## 🏛️ Architecture ``` Claude client (Code / Desktop / claude.ai) │ JSON-RPC 2.0 over Streamable HTTP ▼ Cloudflare Worker https://wiki-mcp.<subdomain>.workers.dev/mcp │ bearer-gated ▼ GitHub API + raw.githubusercontent.com (your private wiki repo) ``` Stateless. Fresh `McpServer` per request, 60 s in-memory snapshot cache keyed by commit SHA. Page bodies fetched via SHA-pinned raw URLs — Cloudflare edge cache handles the rest. ### 🌐 HTTP surface | Method + Path | Auth | Purpose | |---------------|------|---------| | `GET /health` | none | liveness ping — `{ ok, at }` | | `GET /.well-known/oauth-protected-resource` | none | RFC 9728 metadata. Advertises bearer-via-header; satisfies MCP clients that probe before connecting. | | `GET /.well-known/oauth-authorization-server` | none | Returns 404 (RFC 8414) — no OAuth AS, bearer-only. | | `POST /mcp` | `Authorization: Bearer …` | JSON-RPC 2.0 transport. 401 includes `WWW-Authenticate: Bearer resource_metadata="…/.well-known/oauth-protected-resource"`. | | `GET /mcp` | bearer | Returns `405 Allow: POST, DELETE`. The stateless transport cannot serve the optional GET-SSE channel; rejecting up-front prevents Workers from cancelling hung requests and forces clients to POST. | ## 📖 Reading - Pattern — [Karpathy gist](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) - Companion wiki — [wikionfire](https://github.com/IsaiaScope/wikionfire) - Per-module READMEs — `src/<module>/README.md` ## 📄 License MIT. See [`LICENSE`](LICENSE). --- <p align="center"> Made with 🧠 and a lot of TypeScript. </p>