Content
# LAMU
**Local Agent Model Utility** — the model OS for one GPU. A single Rust
binary that discovers your models, schedules them on a budgeted card,
and serves them to every client you have: OpenAI, Anthropic, and Ollama
HTTP; MCP for agent harnesses; native ACP for Zed; A2A for agent-to-agent
fleets. Built to be the local backend other agent systems stand on.
Started from 2021 InferKit / GPT-2 nostalgia — the GPT-2 proxy is still
in the registry, not dead code. Now it runs a 27B uncensored model at
**106 t/s** with speculative decoding, agentic research, temporal
memory, and a training pipeline, all on one RTX 4090.
| Tier | Speed | Engine | Use |
|------|-------|--------|-----|
| **DFlash** (Lucebox DDTree) | **106 t/s** | matched-3.6 draft, 5.12 tok/step | one-shot, full GPU |
| **ngram-mod** (warm) | 49.5 t/s | hash-based speculation, no draft | always-on, 131K ctx |
| **megakernel** | 494 t/s | hand-written CUDA, Qwen3.5-0.8B | routing, agent tools |
## Why LAMU
- **Six serving protocols, one port + one stdio pair.** OpenAI
(`/v1/chat/completions`), Anthropic (`/v1/messages` with real
`thinking` blocks), Ollama (`/api/chat`), MCP (30+ tools for Claude
Code), **ACP** — `lamu acp` is a native Zed agent with streamed
thoughts, tool lifecycle, and permission prompts — and **A2A** —
`lamu a2a` exposes an Agent2Agent card + JSON-RPC/SSE surface so any
agent (or a katana fleet) can discover and task the local model as a
remote peer.
- **Research where fake citations are structurally impossible.**
`deep_research` / `answer` resolve every `[N]` to a real retrieved
source IN CODE — the model never fabricates a link, because links
never come from the model.
- **A real memory store, not a vibe.** Temporal facts (valid-time,
supersede, contradiction judging), hybrid recall (FTS5 + quantized
vector search with persistent indexes), local embeddings — zero API
keys required — and an owner-scoped HTTP memory API other agent
harnesses consume as a service.
- **A world model with structure, not just similarity.** A causal event
hypergraph in the same `lamu.db` — content-addressed nodes (BLAKE3
`b3:` hashes, dedup + cross-system CAS interop), n-ary cause/effect
hyperedges, and cycle-safe `trace_causal` traversal over MCP tools.
Vector search finds *similar* facts; the graph traces what *caused*
what — the relational layer a `WHERE` clause can't fake (ADR
[0039](lamu-rs/docs/decisions/0039-causal-event-hypergraph.md)).
- **Provider-grade serving for harness builders.** Per-call engine-true
token usage and context occupancy (un-fakeable: counted by the
engine's own tokenizer), structured reasoning on every surface,
prefix-cache control + honest cached-token reporting, capability
discovery on `/v1/models`. The embedding contract is one documented
page: [`docs/API.md`](lamu-rs/docs/API.md) § *Embedding LAMU as a provider*.
- **Three model formats.** GGUF via llama.cpp/BeeLlama, `.onnx`
embedding models via ort, and raw HuggingFace safetensors via candle
(Llama/Mistral/Qwen2 families) — scan finds them all, the scheduler
budgets them all.
- **GPU safety as architecture.** PDEATHSIG-hardened children that
cannot outlive lamu, verified kills anchored on ports (PID reuse
safe), a VRAM scheduler that never silently evicts on the HTTP path,
and a training lock so `lamu` and a fine-tune run share the card
without fighting.
- **Multi-model judgment.** `council` runs N models on one prompt and
judges headlessly; `parallel_query` fans out with per-provider caps.
- **Hardware-aware model picking.** `lamu cookbook` ranks every model
for YOUR card with roofline math (quality/speed/fit/context).
- **Sandboxed agents.** `lamu agent -- <cmd>` wraps mutating agents in
bubblewrap (cwd-rw only) with a filesystem journal — `lamu rollback`
reverses any session.
- **39 ADRs.** Every architectural decision is written down with its
alternatives and a falsifiable validation section
([`docs/decisions/`](lamu-rs/docs/decisions/)).
Two upstream merges along the way ([Lucebox #89](https://github.com/Luce-Org/lucebox-hub/pull/89), [#94](https://github.com/Luce-Org/lucebox-hub/pull/94)).
---
## Current State
*Snapshot: 2026-06-13. One `lamu` binary · 13-crate Rust workspace · 39 ADRs · ~860 tests. Release line `v0.30` on `main`; the agent-fabric + world-model waves (W8–W9, ADRs 0038/0039) land on `feat/jart-deep-research`.*
**Six serving surfaces.** OpenAI `/v1/chat/completions` · Anthropic `/v1/messages` (real `thinking`) · Ollama `/api/chat` · MCP stdio (30+ tools) · ACP (`lamu acp`, native Zed agent) · A2A (`lamu a2a`, Agent2Agent card + JSON-RPC/SSE). Structured reasoning on every surface; prefix-cache control + engine-true token/occupancy reporting (ADR 0037). ACP + A2A share one `run_prompt_turn` loop core; tool dispatch single-sourced from MCP.
**Three model formats, one scheduler.** GGUF via llama.cpp/BeeLlama (+ DFlash / megakernel speculative tiers), `.onnx` embeddings via ort (ADR 0034), HuggingFace safetensors via candle — Llama/Mistral/Qwen2 (ADR 0035). One NVML VRAM scheduler bin-packs and evicts LRU; the HTTP path never silently evicts.
**Unified store — `lamu.db`** (one SQLite, versioned migrations, legacy import; ADR 0028):
- **Temporal fact memory** — valid-time, supersede, contradiction judging, hybrid recall (FTS5 + persistent quantized turbovec index), local-first embeddings (ADRs 0030/0031).
- **Memory-as-a-service HTTP** — `POST /v1/memory/{remember,recall,forget,supersede}`, per-API-key owner isolation (ADR 0032).
- **World model** — causal event hypergraph: content-addressed nodes (BLAKE3 `b3:`), n-ary cause/effect hyperedges, cycle-safe `trace_causal`, MCP-only v1 (ADR 0039).
**Ops & safety.** PDEATHSIG children that can't outlive lamu · port-anchored verified kills (PID-reuse safe) · training-lock GPU sharing · bubblewrap-sandboxed `lamu agent` + rollback journal · systemd unit · `lamu clean` retention.
**Shipped waves.** D1–D11 (hardening) → W1 provider-grade serve → W2 lamu-memory/unified DB → W3 inproc + onnx + embedder chain + persistent turbovec → W4 memory HTTP → W5 candle runtime → W6 ACP → W7 README → W8 A2A → W9 causal hypergraph.
**Live-gated / deferred** (not yet exercised on metal — GPU has been training): real Zed ACP session · A2A client driving a card-discovered call · ONNX/candle CUDA first loads · turbovec event-semantic search (`Store::Events`) · hyperbolic/Poincaré geometry · HTTP `/v1/graph`.
---
## Quick Start
LAMU ships a single canonical binary, `lamu` (Rust). Install once and use it from anywhere on `$PATH`.
```bash
git clone https://github.com/Quitetall/lamu ~/local-llm
cd ~/local-llm
just install # cargo install --path lamu-rs/lamu-cli --locked
lamu pull qwen36-27b # ~16 GB GGUF from HuggingFace → ~/models/
lamu scan # discover GGUFs → ~/.local/share/lamu/models.yaml
lamu serve & # OpenAI HTTP on :8020 (background)
lamu run heretic # one-shot: resolve, drop into chat
```
Or just type `lamu` — opens a ratatui dashboard with model list (j/k navigate, Enter chats), live VRAM gauge, MCP/HTTP/Bifrost status. First-run-aware: empty registry triggers a `[Y/n]` to download Qwen3.6-27B; if `LAMU_GATEWAY_URL` set + Bifrost down, prompts to `just serve-bifrost`.
Full Ollama-shaped surface:
| Command | Effect |
|---------|--------|
| `lamu` | TUI dashboard |
| `lamu run <model>` | Drop into chat with a model (substring match) |
| `lamu pull <id> [--quant Q4_K_M]` | Download a GGUF from HuggingFace; auto re-scan |
| `lamu show <model>` | Print full registry entry as YAML |
| `lamu rm <model>` | Remove from registry + delete file on disk |
| `lamu use <model>` | Set the `main: true` default-alias target (substring match) |
| `lamu list` / `lamu scan` | Re-discover models |
| `lamu status` | VRAM + which port answers |
| `lamu start` | MCP daemon on stdio (Claude Code) |
| `lamu serve [port]` | OpenAI HTTP (default :8020) |
| `lamu repl [url]` | Chat REPL against a running serve |
| `lamu cookbook [--suggest]` | Rank models for YOUR hardware (roofline + composite score, per-device) |
| `lamu research "<question>"` | Deep research: decompose → multi-source search → cited synthesis → follow-up chat |
| `lamu clean --all [--yes]` | Retention for drafts/sessions/transcripts/media/logs — dry-run by default |
That's the whole onboarding. `lamu` is your interface; everything else is plumbing.
Run it as a service: [`lamu-rs/deploy/`](lamu-rs/deploy/) ships a user-level systemd unit + env template (`systemctl --user enable --now lamu-serve`).
---
## Architecture
```
┌────────────────────────────────────────────────────────────┐
│ lamu daemon (single process, single binary) │
│ │
│ ┌───────────┐ ┌────────────┐ ┌─────────────────────┐ │
│ │ MCP stdio │ │ OpenAI :* │ │ CLI REPL (lamu repl)│ │
│ │ (primary) │ │ (compat) │ │ → talks to daemon │ │
│ └─────┬─────┘ └─────┬──────┘ └──────────┬──────────┘ │
│ │ │ │ │
│ ┌─────▼──────────────▼────────────────────▼──────────┐ │
│ │ Router — capability routing (chat/code/...) │ │
│ │ Queue — FIFO/LIFO/Priority, bounded concurrency │ │
│ │ Reasoning extractor — per-family <think> handling │ │
│ │ Health + Supervisor — restart-with-backoff │ │
│ └────────────────────────┬───────────────────────────┘ │
│ │ │
│ ┌────────────────────────▼───────────────────────────┐ │
│ │ VRAM scheduler — bin-packing + LRU eviction │ │
│ │ pinned models honoured · NVML-driven │ │
│ └─┬──────────────┬──────────────┬─────────────┬──────┘ │
│ │ │ │ │ │
│ ┌─▼─────┐ ┌─────▼──────┐ ┌────▼────┐ ┌─────▼────┐ │
│ │llama │ │megakernel │ │ DFlash │ │HF / ONNX │ │
│ │.cpp │ │ (PyTorch) │ │ lucebox │ │ ort/cndl │ │
│ └───────┘ └────────────┘ └─────────┘ └──────────┘ │
└────────────────────────────────────────────────────────────┘
```
Invariants:
- **MCP first.** OpenAI HTTP is a compat shim. The CLI also targets the daemon, not the backends directly.
- **Capabilities are requirements, not preferences.** `capabilities=["code"]` will load a code model, evicting LRU if needed. The router never silently downgrades.
- **`plan_query` dry-run.** Returns `{would_route_to, reason, loaded, would_evict}` for debugging agent loops.
- **Per-model request queue.** Concurrent agents calling the same model serialise on a configurable strategy (FIFO default). Set `LAMU_QUEUE_STRATEGY=priority` and pass `priority` / `origin` per request when ordering matters.
- **Reasoning extractor lives in the model entry.** `<think>...</think>` is buffered and stripped (or annotated) per family — Qwen3.5/3.6, DeepSeek, o1.
- **Backend death is loud, not silent.** Health state machine (HEALTHY → DEGRADED → DEAD → QUARANTINED) plus Supervisor with 1s/2s/4s backoff, structured JSON events to stderr or `$LAMU_EVENT_LOG`.
---
## MCP — Claude Code Integration
```jsonc
// ~/.claude.json
{
"mcpServers": {
"local-llm": {
"type": "stdio",
"command": "lamu",
"args": ["start"]
}
}
}
```
Reload Claude Code, then `/mcp` should show `local-llm` connected. Tools exposed:
| Tool | Purpose |
|------|---------|
| `query` | Send prompt. `model=` overrides routing; `capabilities=[…]` enforces requirements; `priority`/`origin` flow through the queue; `include_reasoning=true` returns `<think>` as a structured field. |
| `plan_query` | Dry-run routing decision — no generation. |
| `list_models` | Registry + load status + capabilities. |
| `load_model` / `unload_model` | Manual VRAM control. |
| `vram_status` | Snapshot of allocation. |
| `scan_models` | Re-discover GGUFs on disk (merges — curated fields survive). |
| `queue_status` | Per-model queue depth + scheduling strategy. |
| `cloud_query` / `parallel_query` / `council` | Cloud models (MiMo/DeepSeek/Claude via `~/.config/lamu/cloud-models.yaml`); fan-out; multi-model judged comparison. |
| `review_commit` / `review_diff` | Code review by a cloud reviewer with bundled review policy. |
| `research` / `deep_research` / `answer` / `web_search` | Grounded research: scraper fan-out, cited synthesis, keyless SearXNG lookup. |
| `generate_image` / `text_to_speech` | Managed ComfyUI / fish-speech backends (modality-tiered eviction). |
| `cookbook` | Hardware-fit model ranking (same engine as `lamu cookbook`). |
…plus memory (remember/recall/consolidate), context occupancy (ADR 0021), routing mode, and warmup — `tools/list` is the full inventory.
---
## Serving Surfaces
`lamu serve` exposes the local model pool over **OpenAI-, Anthropic-, and Ollama-compatible** HTTP; `lamu start` speaks MCP on stdio; `lamu acp` is a native ACP agent for Zed; `lamu a2a` speaks Agent2Agent over HTTP. Point whatever client you already use at it — LAMU is the backend orchestrator; the frontend is your choice of harness (ADR [0016](lamu-rs/docs/decisions/0016-backend-orchestrator-byo-frontend.md)).
Register LAMU as a Zed agent (`settings.json`):
```jsonc
{
"agent_servers": {
"LAMU": { "command": "lamu", "args": ["acp"] }
}
}
```
Zed gets streamed answers AND thoughts (`<think>` blocks arrive as
`agent_thought_chunk`s), live tool-call status for LAMU's research/memory
tools, and permission prompts before any file write (ADR
[0036](lamu-rs/docs/decisions/0036-acp-agent-surface.md)).
Run LAMU as an Agent2Agent peer and discover it via its card:
```bash
lamu a2a # loopback :8022 (off-loopback needs LAMU_A2A_TOKEN)
curl -s localhost:8022/.well-known/agent-card.json | jq .name # "LAMU"
```
Any A2A client tasks the local model with `message/send` (run to
completion) or `message/stream` (SSE: working → message/thought events →
completed Task), with `tasks/get` and `tasks/cancel` for lifecycle. The
A2A tool set is read/research/memory only — `write_file` is excluded and
fails closed (no human present to approve). The inter-agent fabric for
katana fleets (ADR
[0038](lamu-rs/docs/decisions/0038-a2a-agent-surface.md)).
Reasoning is structured data on every HTTP surface (OpenAI
`reasoning_content`, Anthropic `thinking` blocks, Ollama
`message.thinking`), and llama-server's prefix cache is exposed via
`cache_prompt` with engine-true cached-token reporting (ADR
[0037](lamu-rs/docs/decisions/0037-provider-grade-serving.md)). The
memory store is served too: `POST /v1/memory/{remember,recall,forget,supersede}`
with per-API-key owner isolation (ADR
[0032](lamu-rs/docs/decisions/0032-memory-as-a-service.md)).
📖 **The authoritative API reference is [`lamu-rs/docs/API.md`](lamu-rs/docs/API.md)** — every endpoint with request/response shapes, streaming, auth, error envelopes, LAMU extensions, and per-frontend setup. This README is a summary; **API.md is the single source of truth.**
Three dialects on one port:
| Flavor | Routes | Clients |
|--------|--------|---------|
| OpenAI | `/v1/chat/completions`, `/v1/embeddings`, `/v1/models` | Codex, Cursor, Aider, Continue, Open WebUI |
| Anthropic | `/v1/messages` (SSE + `tool_use`) | Claude Code, Crush, Hermes |
| Ollama | `/api/chat`, `/api/tags` (NDJSON) | AnythingLLM, Open WebUI (Ollama mode) |
- **Default model** — the registry entry with `main: true`; aliases `default`/`main`/`lamu` resolve there, so harnesses need no model name. `lamu scan` auto-promotes the first model to `main` so a fresh registry is usable immediately; `lamu use <model>` re-points it (substring match) without hand-editing YAML.
- **Auth** — off on a loopback bind; off-loopback (`LAMU_BIND_HOST=0.0.0.0`) requires a token (`lamu auth init`), ADR [0012](lamu-rs/docs/decisions/0012-minimal-bearer-auth.md). See API.md § Authentication.
- **Extensions** — `enable_thinking: false` disables Qwen3.6 reasoning on all three surfaces; reasoning surfaces as `reasoning_content`. See API.md § LAMU extensions.
- **Bifrost passthrough (optional)** — `LAMU_GATEWAY_URL=http://localhost:8080/v1` forwards chat completions through Bifrost (`just serve-bifrost`) for a unified cloud+local surface (~1.67% latency). Default off.
- **Observability** — `GET /metrics` (Prometheus), `GET /health` (`{"status":"ok","models_loaded":N}`), W3C `traceparent` propagation, `LAMU_EVENT_LOG=<jsonl>`.
Registered harnesses live in `config/harnesses.yaml` (`just open [name]`, `just open list`); per-harness setup: [`wiki/pages/harness-setup.md`](wiki/pages/harness-setup.md).
---
## Model Registry
`lamu scan` walks `~/models/`, parses GGUF headers, and writes the **live
registry at `~/.local/share/lamu/models.yaml`** — outside the git work tree,
because scans and load-status flips mutate it at runtime (ADR
[0025](lamu-rs/docs/decisions/0025-registry-out-of-work-tree.md)). The repo
tracks a read-only seed (`config/models_default.yaml`) that bootstraps the
live file on first run; `$LAMU_REGISTRY` overrides the path for
tests/sandboxes. Re-scans merge: curated fields (`main`, `speculative`,
`sampling`, `notes`, `status`, `system_prompt`, `backend_kind`) survive.
```yaml
models:
qwen3.6-27b-uncensored-heretic-v2-q4_k_m: # map key = model name
path: ~/models/qwen3.6-27b-heretic/Qwen3.6-27B-uncensored-heretic-v2-Q4_K_M.gguf
format: gguf
backend: llama_cpp
arch: qwen35
params_b: 27.6
quant: Q4_K_M
vram_mb: 17358
context_max: 131072
capabilities: [chat, code, reasoning, long_context]
status: recommended
main: true # the default-alias target (see `lamu use`)
reasoning_marker: { open_tag: "<think>", close_tag: "</think>", family: qwen35 }
speculative:
draft_path: ~/models/qwen3.6-27b-dflash-spiritbuun/dflash-draft-3.6-q4_k_m.gguf
method: dflash
draft_max: 8
```
Optional per-entry extras: `system_prompt` (per-model default system prompt;
precedence request > model > global, blank = explicitly disable) and
`backend_kind` (string dispatch key, ADR
[0026](lamu-rs/docs/decisions/0026-backend-kind-string-dispatch.md)).
Backends: `llama_cpp`, `megakernel`, `dflash`, `dflash_lucebox` in core, plus
module-provided kinds (`comfyui`, `fish_speech`, and the feature-gated
`onnx` + `hf_candle` engines — build with `--features full`) registered at startup (ADR
[0023](lamu-rs/docs/decisions/0023-module-architecture.md)). Adding a module
backend is one crate + one `register_backend("kind", …)` call at the
composition root — core never names it; a drift test proves every kind
resolves.
---
## Three Speed Tiers (perf legacy, still authoritative)
RTX 4090, 24 GB VRAM, Qwen3.6-27B-uncensored-heretic-v2 Q4_K_M:
| Method | Speed | Acceptance | Notes |
|--------|-------|-----------|-------|
| **Lucebox DFlash + DDTree** | **106 t/s** | 32%, 5.12 tok/step | Matched 3.6 draft; PR [#94](https://github.com/Luce-Org/lucebox-hub/pull/94) |
| llama.cpp DFlash PR | 82 t/s | 77.9%, draft-max=8 | GGUF Q4_K_M draft |
| ngram-mod (warm) | 49.5 t/s | pattern matching | no draft model |
| ngram-mod (cold) | 9.8 t/s | first request | |
| 0.8B megakernel | 494 t/s | n/a | hand-written CUDA |
Q4_K_M draft outperforms F16 (77.6 vs 72.7 t/s) — bandwidth beats accuracy on the draft path.
For the **106 t/s DFlash one-shot** run via the legacy stack: `just serve-fast "Write quicksort in Python"` (requires the custom DFlash llama.cpp branch built — see [`wiki/pages/dflash-speculative.md`](wiki/pages/dflash-speculative.md)).
---
## Hacking on LAMU — the Python prototype
The Python package at `lamu/` is the iteration surface. Every Rust module in `lamu-rs/` started as a Python prototype that got translated mechanically once the design stabilised. The two run in lock-step — cross-language MCP contract tests in `tests/contract/` lock the wire format.
Use Python when:
- You're sketching a new module.
- You want a stack trace.
- You're debugging health / scheduler / queue behaviour interactively.
Use Rust (`lamu`) for everything else. Mirror surface is identical:
```bash
python -m lamu scan|status|start|serve|repl # prototype
lamu scan|status|start|serve|repl # canonical
```
---
## Build Requirements
- **GPU:** NVIDIA RTX 4090 (24 GB) or larger
- **OS:** Linux (Arch / CachyOS tested)
- **CUDA:** 13.2 with **gcc-14** as host compiler (`CUDAHOSTCXX=g++-14`). gcc-16 + nvcc 13.2 do not link.
- **Rust:** 1.85+ (edition 2024) — `cargo install` lands `lamu` at `~/.cargo/bin/lamu`.
- **Python:** 3.12+ (only needed for the prototype + agents)
- **Tools:** `just`, `cmake`, `git`, `uv`
- **`lamu pull` only:** the HuggingFace Hub CLI on `$PATH` — `uv tool install huggingface-hub` (provides `hf`). Not needed if you place GGUFs in `~/models/` yourself and run `lamu scan`.
---
## Testing
```bash
pytest tests/ -q # 288 unit + 14 integration, heavy deps stubbed
cargo test --workspace # ~860 Rust tests across 13 crates
just test-contract # Python ↔ Rust MCP wire-format parity
ruff check lamu # strict on lamu/, soft on legacy paths
```
Agentic flows (research/answer/chat) test against a scripted `FakeCtx`
seam double (`lamu-core` feature `test-support`) — no model, no GPU, the
one networked step served by an in-test TCP stub.
CI gates on coverage (`fail_under = 70`), strict ruff over `lamu/`, full Python + Rust suites, and the cross-language contract diff.
`tests/conftest.py` stubs `torch`, `transformers`, `llama_cpp`, `langchain*`, `chainlit`, etc. with `_StubModule` instances at import time — the unit layer never touches a GPU. `mock_nvidia_smi` simulates VRAM/PIDs/failures; `no_real_subprocess` is autouse-guarded so a stray Popen can't escape a test.
---
## Legacy v1 stack
The script-driven v1 workflow (swap a model into `:8020`, sidecar a small one into `:8001`, run Bifrost on `:8080`) lives under `legacy/`. See [`legacy/README.md`](legacy/README.md) for the full inventory and what each script does.
`just` exposes both:
```bash
just install # v3 — lamu binary
just start # v3 — MCP daemon
just serve # v3 — OpenAI HTTP
just repl # v3 — chat REPL
just start-v1 # v1 — full Qwen3.6 + megakernel + Bifrost stack
just swap 3.6 | 3.5 # v1 — rotate model on :8020
just sidecar fast|lobo # v1 — small sidecar on :8001
just serve-fast "..." # v1 — DFlash 106 t/s one-shot
```
Bifrost (`:8080`) is dead on the v3 request path — kept under `scripts/serve-bifrost.sh` only because [`wiki/pages/bifrost-bench.md`](wiki/pages/bifrost-bench.md) hasn't been run yet to settle whether it's worth keeping. Run `bash scripts/bench-bifrost.sh` to decide.
---
## Wiki
13 pages in `wiki/pages/`:
`dflash-speculative.md` · `build-requirements.md` · `262k-context.md` · `ngram-speculation.md` · `vram-budget.md` · `eagle-training.md` · `eagle-cpp-integration.md` · `mcp-setup.md` · `harness-setup.md` · `model-selection.md` · `serving-engine.md` · `token-efficiency.md` · `training-loop.md` · `vllm-limitations.md` · `bifrost-bench.md`.
Knowledge graph of the whole repo (~106,000 nodes, ~245,000 edges, ~5,600 communities), rebuilt on every commit by the graphify post-commit hook → `graphify-out/graph.json` + `GRAPH_REPORT.md` (HTML viz auto-skipped above 5,000 nodes). Query with `/graphify query "<question>"`.
---
## Open Source Contributions
| PR | Repo | Status | Impact |
|----|------|--------|--------|
| [#89](https://github.com/Luce-Org/lucebox-hub/pull/89) | Luce-Org/lucebox-hub | **Merged** | Fixed `conv_input_cache` crash on all 24 GB GPUs |
| [#94](https://github.com/Luce-Org/lucebox-hub/pull/94) | Luce-Org/lucebox-hub | Submitted | Qwen3.6 SWA draft support → 57% speedup |
---
## Philosophy
Wanted GPT-2 running locally like InferKit in 2021. Now running a 27B uncensored model at 106 t/s with speculative decoding, agent swarms, MCP integration, and a Rust port — all on one consumer GPU.
The config is opinionated. The architecture isn't. Every layer is swappable: model, backend, transport, framework. When something better lands, change one path.
---
## License
MIT
