Content
# Spellbook v1.0 🔮
VectorDB based AI Agent memory MCP Server
## Overview
Stores information accumulated during the AI agent personalization process in VectorDB and searches based on meaning.
**Canon (Main Storage)**:
- **Scribe**: Stores information in Canon (REST mode only)
- **Memorize**: Canon semantic search
- **Find**: Canon keyword based search
**Lore (Sub Storage)**:
- **Chronicle**: Stores information in Lore (REST mode only, Lore name specified)
- **Recall**: Lore semantic search
- **Recall Find**: Lore keyword based search
Canon and Lore APIs are completely separated. You cannot access Lore data with the Canon API, and vice versa.
**Spellbook is a simple repository.** The user decides what to store and when to search.
## Key Features
- ✅ **Canon + Lore**: Separate main storage and sub storage for each purpose
- ✅ **REST State Management**: Ensures chunking consistency (shared by scribe/chronicle)
- ✅ **Dual Search**: Semantic-based + Keyword-based (Canon/Lore each)
- ✅ **nomic-embed-text**: 768 dimensions, excellent for Korean/English
- ✅ **HTTP/SSE MCP**: Easy deployment with Docker
- ✅ **Bun Runtime**: Fast startup, native TypeScript execution
## Quick Start
> 📖 **Detailed Installation Guide**:
> - 👤 **Person**: [INSTALL.md](./INSTALL.md) - Use interactive installation script
> - 🤖 **AI Agent**: [INSTALL_AI.md](./INSTALL_AI.md) - Environment variable setting guide
### Method 1: Docker Compose (Recommended) ⭐
**Prerequisites**: [Docker](https://docs.docker.com/get-docker/), [Ollama](https://ollama.com)
```bash
git clone https://github.com/zilhak/spellbook.git
cd spellbook
./scripts/setup.sh # Check environment + download model + build image
docker compose up -d
```
setup.sh handles Docker, Ollama verification, embedding model download, and image building.
The system guide seed is automatically performed on the first run.
**Customize settings** (optional):
```bash
cp .env.example .env
nano .env
```
| Environment Variable | Default Value | Description |
|----------------------|---------------|-------------|
| `PORT` | `17950` | Spellbook server port |
| `QDRANT_PORT` | `17951` | Qdrant external port |
| `QDRANT_DATA_PATH` | `./data/qdrant` | Data storage path |
| `OLLAMA_HOST` | `http://host.docker.internal:11434` | Ollama address |
---
### Method 2: Run Bun Directly (For Development)
```bash
git clone https://github.com/zilhak/spellbook.git
cd spellbook
bun install
# Prepare Qdrant + Ollama
docker run -d -p 17951:6333 qdrant/qdrant
ollama pull nomic-embed-text
# Development mode (hot reload)
bun run dev
```
**Requirements**: [Bun](https://bun.sh) (`curl -fsSL https://bun.sh/install | bash`)
---
### Claude Code Configuration
**Method 1: CLI Command (Recommended)**
```bash
claude mcp add --transport http spellbook http://localhost:17950/mcp
```
**Method 2: Manual Configuration**
`~/.claude/mcp.json`:
```json
{
"mcpServers": {
"spellbook": {
"url": "http://localhost:17950/mcp"
}
}
}
```
### AI Agent System Prompt Configuration (Recommended)
To effectively utilize Spellbook, it is recommended to add the following instructions to the AI Agent's system prompt (CLAUDE.md, etc.):
```markdown
## Spellbook Usage Rules
- If you encounter unknown concepts, project conventions, or past decisions, search Spellbook first
- At the start of a session, check what information is stored with get_index()
- Canon Search (General Knowledge):
- When you are curious about context/concepts: memorize(query)
- When you know the exact terms: find(keywords)
- Lore Search (Project-Specific Information):
- When you are curious about context/concepts: recall(lore, query)
- When you know the exact terms: recall_find(lore, keywords)
- If important decisions, new patterns, or user preferences are identified, save them in REST mode
- General knowledge → scribe (Canon)
- Project-specific information → chronicle (Lore)
```
This allows the Agent to naturally utilize the knowledge accumulated from previous sessions.
---
## Usage
### Canon Workflow (Main Storage)
```typescript
// 1. Enter REST mode
const session = await rest();
// 2. Save chunk to Canon
await scribe({
chunk: {
text: "Docker Compose is...",
metadata: {
topic_id: "infrastructure",
category: "technology",
keywords: ["Docker", "Compose"],
questions: ["How to configure Docker Compose?"],
entities: [{name: "Docker", type: "technology"}],
importance: "high"
}
},
session_id: session.session_id
});
// 3. End REST
await rest_end(session.session_id);
// Canon Search
await memorize({query: "Docker container configuration"});
await find({keywords: ["Docker", "Qdrant"]});
```
### Lore Workflow (Sub Storage)
```typescript
// 1. Enter REST mode
const session = await rest();
// 2. Save chunk to Lore (Lore is automatically created if it does not exist)
await chronicle({
lore: "my-project",
lore_description: "My Project technical decisions",
chunk: {
text: "Authentication is JWT based...",
metadata: {
topic_id: "auth",
category: "architecture",
keywords: ["JWT", "authentication"],
questions: ["What is the project authentication method?"],
entities: [{name: "JWT", type: "technology"}],
importance: "high"
}
},
session_id: session.session_id
});
// 3. End REST
await rest_end(session.session_id);
// Lore Search
await recall({lore: "my-project", query: "authentication method"});
await recall_find({lore: "my-project", keywords: ["JWT"]});
// Lore Management
await list_lores(); // List of all Lores + description
await lore_stats("my-project"); // Specific Lore statistics
```
## Data Management
### Persistence
**When using Docker Compose**:
- Use host path directly (`QDRANT_DATA_PATH` in `.env`)
- Data is automatically maintained even when the container is restarted/recreated
```bash
# Set in .env file
QDRANT_DATA_PATH=/your/data/path
# Check data
ls -la /your/data/path
```
**✅ Safe Shutdown** (Data Retention):
```bash
docker-compose down # Only delete containers, keep data
docker-compose restart # Restore data as is
```
---
### Backup/Restore (export/import)
#### export Tool (Backup)
```bash
# Initialize MCP session and call export
curl -X POST http://localhost:17950/mcp \
-H "Content-Type: application/json" \
-d '{"jsonrpc":"2.0","method":"initialize","params":{"clientInfo":{"name":"test"},"protocolVersion":"2024-11-05"},"id":1}'
# Export with session ID
curl -X POST http://localhost:17950/mcp \
-H "Content-Type: application/json" \
-H "mcp-session-id: YOUR_SESSION_ID" \
-d '{"jsonrpc":"2.0","method":"tools/call","params":{"name":"export","arguments":{}},"id":2}'
```
#### import Tool (Restore)
```bash
curl -X POST http://localhost:17950/mcp \
-H "Content-Type: application/json" \
-H "mcp-session-id: YOUR_SESSION_ID" \
-d '{
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"name": "import",
"arguments": {
"data": {
"chunks": [
{"text": "Content to restore", "category": "knowledge"}
]
}
}
},
"id": 1
}'
```
**Features**:
- Can be restored directly without a REST session
- Automatically regenerates embeddings for each chunk
- Returns the number of successes/failures
---
## Development
### Local Execution
```bash
# Install dependencies
bun install
# Qdrant only with Docker
docker-compose up -d qdrant
# Development mode (hot reload)
bun run dev
# Type check
bun run typecheck
```
### System Guide Seed
```bash
# First time only
bun run seed
```
## MCP Tools
### Common
| Tool | Description | Constraints |
|------|-------------|-------------|
| `rest` | Start REST mode | - |
| `rest_end` | End REST mode | - |
| `get_index` | Meta index | - |
| `stats` | Canon statistics | - |
| `filter_guide` | Filter usage guide | - |
| `export` | Canon JSON backup | - |
| `import` | Canon JSON restore | - |
### Canon (Main Storage)
| Tool | Description | Constraints |
|------|-------------|-------------|
| `scribe` | Save chunk to Canon | **REST mode required** |
| `memorize` | Canon semantic search | - |
| `find` | Canon keyword search | - |
| `get_topic` | Topic lookup | - |
| `erase` | Delete Canon chunk | - |
| `revise` | Modify Canon chunk | - |
### Lore (Sub Storage)
| Tool | Description | Constraints |
|------|-------------|-------------|
| `chronicle` | Save chunk to Lore | **REST mode required** |
| `recall` | Lore semantic search | - |
| `recall_find` | Lore keyword search | - |
| `erase_lore` | Delete Lore chunk | - |
| `revise_lore` | Modify Lore chunk | - |
| `list_lores` | List of all Lores | - |
| `delete_lore` | Delete Lore (cannot be recovered) | - |
| `lore_stats` | Lore statistics | - |
| `update_lore` | Modify Lore description | - |
## Architecture
```
Claude Code
│ HTTP/SSE (MCP Protocol)
▼
Spellbook (Bun + MCP SDK)
│
├─ REST session management (shared by scribe/chronicle)
├─ Canon tools (scribe, memorize, find)
├─ Lore tools (chronicle, recall, recall_find)
├─ Lore management (list_lores, delete_lore, lore_stats, update_lore)
├─ Embedding (Ollama + nomic-embed-text)
└─ VectorDB (Qdrant)
├─ Canon: chunks + chunks_metadata
└─ Lore: lore_{name} + lore_{name}_metadata (each)
```
## Technology Stack
| Component | Selection |
|-----------|-----------|
| **Runtime** | Bun |
| **Language** | TypeScript (Native Execution) |
| **MCP** | @modelcontextprotocol/sdk |
| **VectorDB** | Qdrant |
| **Embedding** | Ollama + nomic-embed-text |
| **HTTP** | Express |
## Detailed Documentation
- [CLAUDE.md](./CLAUDE.md) - Project overall design document
- [src/data/system-guides.ts](./src/data/system-guides.ts) - Chunking guide
## License
MIT
