octobrain

# Octobrain > Persistent memory for AI assistants — store insights, decisions, and knowledge that survives across conversations. [](https://crates.io/crates/octobrain) [](LICENSE) [](https://www.rust-lang.org/) [](https://github.com/Muvon/octobrain/actions/workflows/ci.yml) [](https://github.com/Muvon/octobrain/actions/workflows/ci.yml) _MCP Registry: `mcp-name: io.github.Muvon/octobrain`_ ## Table of Contents - [Why Octobrain?](#why-octobrain) - [Quick Start](#quick-start) - [Installation](#installation) - [Usage](#usage) - [Features](#features) - [Benchmarks](#benchmarks) - [Configuration](#configuration) - [Memory Types](#memory-types) - [MCP Integration](#mcp-integration) - [Storage Locations](#storage-locations) - [Contributing](#contributing) - [License](#license) **Octobrain** gives your AI assistant a long-term memory. Store code insights, architecture decisions, bug fixes, and knowledge — then retrieve them with semantic search in future sessions. Works as a CLI tool or as an MCP server for integration with Claude Desktop and other AI tools. ## Why Octobrain? AI assistants start every conversation with zero context. You explain your project, your preferences, your decisions — every single time. Octobrain breaks that cycle: - **Persistent memory** — Insights survive across sessions, not just within them - **Semantic search** — Find memories by meaning, not exact keywords - **Auto-linking** — Related memories connect automatically (Zettelkasten-style) - **Knowledge indexing** — Ingest docs, articles, and files for retrieval - **MCP integration** — Works with Claude Desktop and other MCP-compatible tools ## Quick Start ```bash # Install from crates.io cargo install octobrain # Store your first memory octobrain memory memorize --title "API Design Pattern" \ --content "Use REST for CRUD, GraphQL for complex queries" \ --memory-type architecture --tags "api,design" # Search memories octobrain memory remember "how should I design APIs" # Start MCP server for Claude Desktop integration octobrain mcp ``` ## Installation ### From crates.io (Recommended) ```bash cargo install octobrain ``` ### From Source ```bash # Clone and build git clone https://github.com/muvon/octobrain.git cd octobrain cargo build --release # Binary location ./target/release/octobrain --help ``` ### Docker ```bash docker pull ghcr.io/muvon/octobrain:latest ``` ### Homebrew (macOS/Linux) ```bash brew install muvon/tap/octobrain ``` ### Feature Flags Octobrain supports multiple embedding providers: | Flag | Description | API Key Required | |------|-------------|------------------| | `fastembed` | Local embeddings via FastEmbed | No | | `huggingface` | Local embeddings via HuggingFace | No | | (default) | Both `fastembed` + `huggingface` | No | | (no features) | API-based: Voyage, OpenAI, Google, Jina | Yes | ```bash # Build with local embeddings (default, no API keys needed) cargo build --release # Build with API-based embeddings only cargo build --no-default-features --release ``` For API-based embeddings, set the appropriate environment variable: - `VOYAGE_API_KEY` for Voyage AI - `OPENAI_API_KEY` for OpenAI - `GOOGLE_API_KEY` for Google - `JINA_API_KEY` for Jina ## Usage ### Memory Management Store and retrieve insights, decisions, and context: All `memory` subcommands accept global flags: - `--scope <string>` — Override project scope (default: auto-detected from Git remote) - `--role <string>` — Filter by role (e.g. "developer", "reviewer") ```bash # Store a memory octobrain memory memorize --title "API Design" \ --content "Use REST for CRUD, GraphQL for complex queries" \ --memory-type architecture --tags "api,design" # Search memories (semantic search) octobrain memory remember "api design patterns" # Multi-query search for broader coverage octobrain memory remember "authentication" "security" "jwt" # Get a memory by ID octobrain memory get <id> # Get recent memories octobrain memory recent --limit 20 # Filter by type octobrain memory by-type architecture --limit 10 # Filter by tags octobrain memory by-tags "api,security" # Find memories related to files octobrain memory for-files "src/main.rs,src/lib.rs" # Update a memory octobrain memory update <id> --title "New Title" --add-tags "new-tag" # Delete a memory octobrain memory forget --memory-id <id> # Get memories relevant to the current Git commit octobrain memory current-commit # Show memory statistics octobrain memory stats # Clean up old/low-importance memories octobrain memory cleanup # Compact tables and refresh indices (run when searches slow down) octobrain memory maintenance # ⚠️ Delete ALL memory data octobrain memory clear-all --yes ``` ### Memory Consolidation Close a goal and fold all its contributing memories into a consolidated summary: ```bash # Consolidate a goal (Achieves-link sources → Consolidated state, importance dampened) octobrain memory consolidate <goal-id> --summary "Final summary" # Sleep consolidation: auto-cluster recent similar memories octobrain memory sleep-consolidate --threshold 0.85 --min-size 3 ``` ### Memory Relationships Connect related memories for context-rich retrieval: ```bash # Create a relationship between memories octobrain memory relate <source-id> <target-id> \ --relationship-type "depends_on" \ --description "Source requires target to function" # View relationships for a memory octobrain memory relationships <memory-id> # Find related memories through relationships octobrain memory related <memory-id> # Auto-link similar memories (Zettelkasten-style) octobrain memory auto-link <memory-id> # Explore memory graph octobrain memory graph <memory-id> --depth 2 ``` ### Knowledge Base Index and search web content, docs, and files: ```bash # Index a URL octobrain knowledge index https://docs.rs/tokio/latest/tokio/ # Index a local file (.txt, .md, .pdf, .docx, .html) octobrain knowledge index ./docs/architecture.md # Search knowledge base octobrain knowledge search "how to handle async tasks" # Search within a specific source (auto-indexes if outdated) octobrain knowledge search "spawn blocking" --source https://docs.rs/tokio/ # Read full content of a URL or local file octobrain knowledge read https://docs.rs/tokio/latest/tokio/ # Search indexed content by regex pattern octobrain knowledge match "spawn_blocking|block_in_place" # Store raw text content octobrain knowledge store "meeting-notes" --content "Discussion points..." # List indexed sources octobrain knowledge list --limit 20 # Show statistics octobrain knowledge stats # Delete a source octobrain knowledge delete https://example.com/docs # Delete stored content by key octobrain knowledge delete-stored "meeting-notes" ``` ### Knowledge Boxes Import and sync git-backed knowledge bundles scoped to your projects: ```bash # Import a remote git repo as a knowledge box octobrain box import https://github.com/org/docs-repo.git # Import at global scope (visible in every project) octobrain box import https://github.com/org/docs-repo.git --global # Pull and re-index all subscribed boxes + local .box/ directories octobrain box sync # List subscribed boxes octobrain box list # Remove a box octobrain box remove github.com/org/docs-repo ``` ### MCP Server Run as an MCP server for integration with Claude Desktop and other AI tools: ```bash # Start with stdio transport (for Claude Desktop) octobrain mcp # Start with HTTP transport (for web-based tools) octobrain mcp --bind 0.0.0.0:12345 ``` **Available MCP Tools:** | Tool | Description | |------|-------------| | `memorize` | Store memories with metadata; optional `related_to` for inline relationships | | `remember` | Semantic search with filters; returns 1-hop graph neighbors | | `forget` | Delete memories (requires confirmation) | | `knowledge` | Unified tool: `search`, `store`, `delete`, `read`, `match` via `command` field | See [MCP Integration](#mcp-integration) for Claude Desktop setup. ## Features - **Semantic Search** — Find memories by meaning using vector embeddings, not exact keyword matches - **Hybrid Search** — Combines BM25 full-text search with vector similarity for better results - **Reranking Support** — Pluggable cross-encoder reranking stage (provider-dependent) - **Auto-Linking** — Automatically connects semantically similar memories (Zettelkasten-style) - **Temporal Decay** — Ebbinghaus forgetting curve for importance management - **Knowledge Indexing** — Ingest URLs, PDFs, docs for retrieval - **Project Scoping** — Isolate memories per Git project or share across projects - **Role Filtering** — Tag memories by role (developer, reviewer, etc.) - **Query Expansion (HyDE-lite)** — Pseudo-relevance feedback for +10-30% recall on long-tail queries - **MCP Protocol** — Full MCP 2026-07-28 compliance for AI tool integration ## Benchmarks Retrieval quality of octobrain's knowledge system on standard [BEIR](https://github.com/beir-cellar/beir) datasets — **nDCG@10**, fully local, no LLM judge, using the harness's default embedder `bge-small-en-v1.5` (384-dim, 33M params). Each corpus passage is indexed through octobrain's real retrieval path and scored against the official qrels (metrics reproduce `pytrec_eval`). | Dataset | octobrain vector | octobrain hybrid | BM25¹ | bge-small-en-v1.5² | |---|---|---|---|---| | **SciFact** (5.2K docs, 300 q) | 0.722 | **0.742** | 0.665 | 0.713 | | **NFCorpus** (3.6K docs, 323 q) | 0.341 | **0.363** | 0.325 | 0.343 | - **vector** = dense-only retrieval; reproduces the embedder's published BEIR numbers (validates the harness). - **hybrid** = BM25 + vector fused with Reciprocal Rank Fusion (k=60) — octobrain's default. Adds **+2 nDCG@10** over the bare embedding and beats classic BM25 on both datasets. ¹ Canonical BM25 from the [BEIR paper](https://arxiv.org/abs/2104.08663) (Anserini/Lucene, k1=0.9 b=0.4). ² From the [bge-small-en-v1.5 model card](https://huggingface.co/BAAI/bge-small-en-v1.5) (MTEB). > Scope: this measures the **ranking** layer (embedding + BM25 fusion + reranking). BEIR passages are pre-chunked, so octobrain's chunking strategy is not exercised here. > ⚠️ **Reranker status:** The cross-encoder reranker is currently a no-op with fastembed models (byte-identical results to hybrid-only). Rerank is excluded from the numbers above pending investigation. See [`benches/README.md`](benches/README.md#findings) for details. Reproduce (downloads the datasets, builds a release binary, runs fully offline): ```bash cd benches && bash scripts/run_retrieval.sh ``` ## Configuration Configuration is stored in `~/.local/share/octobrain/config.toml`. The template written on first run defines every option with sensible defaults, but loading is strict — a field you delete by hand causes startup to fail rather than falling back to a default. Override the config location with `OCTOBRAIN_CONFIG_PATH=/path/to/config.toml`. ### Key Settings | Section | Option | Default | Description | |---------|--------|---------|-------------| | `[embedding]` | `model` | `fastembed:Qdrant/all-MiniLM-L6-v2-onnx` | Embedding model (provider:model format). Default is a local fastembed model — no API key, runs on CPU. | | `[search]` | `similarity_threshold` | `0.3` | Minimum relevance (0.0-1.0) | | `[search.hybrid]` | `enabled` | `true` | Enable BM25 + vector fusion | | `[search.reranker]` | `enabled` | `true` | Enable cross-encoder reranking | | `[search.hyde]` | `enabled` | `true` | Pseudo-relevance feedback query expansion | | `[memory]` | `max_memories` | `10000` | Maximum stored memories | | `[memory]` | `auto_linking_enabled` | `true` | Auto-connect similar memories | | `[knowledge]` | `chunk_size` | `1200` | Characters per chunk | | `[embedding]` | `batch_size` | `32` | Texts embedded per batch | | `[embedding]` | `timeout_secs` | `30` | Embedding call timeout (0 = no timeout) | | `[search]` | `max_results` | `50` | Hard ceiling on results from any search | | `[search.reranker]` | `model` | `fastembed:jina-reranker-v2-base-multilingual` | Cross-encoder model | | `[search.reranker]` | `top_k_candidates` | `50` | Candidates retrieved before reranking | | `[search.hyde]` | `top_k` | `3` | Neighbors averaged for the centroid | | `[search.hyde]` | `alpha` | `0.5` | Blend weight on the original query embedding | | `[memory]` | `max_search_results` | `50` | Default page size when no limit is given | | `[memory]` | `sleep_consolidation_enabled` | `true` | Lazy sleep consolidation on manager init | | `[memory]` | `sleep_consolidation_interval_hours` | `24` | Hours between sleep-consolidation passes | | `[knowledge]` | `outdating_days` | `15` | Days before indexed content is reindexed on search | | `[knowledge]` | `max_results` | `5` | Results returned from knowledge search | ### Embedding Providers ```toml [embedding] # Local models (no API key, runs on CPU, model auto-downloaded on first use) model = "fastembed:Qdrant/all-MiniLM-L6-v2-onnx" # Default: 384-dim quantized, ~125MB resident model = "fastembed:nomic-ai/nomic-embed-text-v1.5" # 768-dim, 8192-ctx, ~550MB weights, highest quality model = "fastembed:BAAI/bge-small-en-v1.5" # 384-dim, ~62 MTEB, fast + good quality model = "fastembed:sentence-transformers/all-MiniLM-L6-v2-quantized" # Smallest (~22MB), fastest model = "fastembed:BAAI/bge-base-en-v1.5" # Larger (~440MB), higher quality model = "fastembed:intfloat/multilingual-e5-small" # Multilingual # Cloud providers (require API keys, generally higher quality) model = "voyage:voyage-3.5-lite" # VOYAGE_API_KEY model = "openai:text-embedding-3-small" # OPENAI_API_KEY model = "google:text-embedding-004" # GOOGLE_API_KEY model = "jina:jina-embeddings-v3" # JINA_API_KEY ``` ### Full Configuration See [`config-templates/default.toml`](config-templates/default.toml) for all available options with documentation. ## Memory Types Organize memories by category for better filtering: | Type | Use For | |------|---------| | `code` | Code patterns, solutions, implementations | | `architecture` | System design, decisions, patterns | | `bug_fix` | Bug fixes, troubleshooting, solutions | | `feature` | Feature specs, implementations | | `documentation` | Docs, explanations, knowledge | | `user_preference` | Settings, preferences, workflows | | `decision` | Project decisions, trade-offs | | `learning` | Tutorials, notes, education | | `configuration` | Setup, config, deployment | | `testing` | Test strategies, QA insights | | `performance` | Optimizations, benchmarks | | `security` | Vulnerabilities, fixes, considerations | | `validation` | Idea/product validation, hypothesis testing | | `research` | Technical/market research, analysis | | `workflow` | SOPs, playbooks, process descriptions | | `requirement` | Business requirements, specs, constraints | | `design` | UI/UX decisions, wireframes, system design | | `integration` | API integrations, third-party services | | `communication` | Stakeholder updates, team decisions | | `process` | Deployment procedures, runbooks, operations | | `insight` | General insights, tips | | `goal` | Task/intent anchors for consolidation workflow | ## MCP Integration ### Claude Desktop Setup Add to your Claude Desktop config (`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS): ```json { "mcpServers": { "octobrain": { "command": "/path/to/octobrain", "args": ["mcp"] } } } ``` Restart Claude Desktop. Octobrain tools will be available in your conversations. ### HTTP Transport For web-based integrations: ```bash octobrain mcp --bind 0.0.0.0:12345 ``` The server exposes endpoints at `/mcp` for MCP protocol communication. ## Storage Locations Data is stored in platform-specific directories: | Platform | Location | |----------|----------| | macOS | `~/.local/share/octobrain/` | | Linux | `~/.local/share/octobrain/` or `$XDG_DATA_HOME/octobrain/` | | Windows | `%APPDATA%\octobrain\` | Project-specific memories are isolated by normalized Git remote URL (e.g., `github.com/org/repo`). ## Contributing Contributions are welcome! Please: 1. Fork the repository 2. Create a feature branch (`git checkout -b feature/amazing-feature`) 3. Run `cargo fmt --all` to format code 4. Run `cargo clippy` and fix all warnings 5. Run `cargo test --no-default-features` 6. Submit a pull request ### Development Setup ```bash # Clone and build git clone https://github.com/muvon/octobrain.git cd octobrain cargo build --no-default-features # Run tests cargo test --no-default-features # Run clippy cargo clippy --no-default-features ``` ## License Apache-2.0 — see [LICENSE](LICENSE) for details. ## Credits Developed by [Muvon Un Limited](https://muvon.io).