Content
# LFM2.5 — On-Device AI Chat & MCP Server
Run [Liquid AI's LFM2.5-1.2B-Instruct](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-MLX-8bit) model locally on your Mac with a beautiful chat interface and MCP integration for tools like Claude Desktop, Cursor, and OpenClaw.
<div align="center">
**~120 tok/s on Apple Silicon** · **~480 tok/s across 4 Macs** · **1.5 GB RAM** · **100% Private & Offline**
</div>
## ✨ Features
- 🖥️ **Beautiful Chat UI** — Dark-themed, responsive chat interface with markdown rendering, code highlighting, and real-time performance metrics
- ⚡ **MLX Acceleration** — Runs natively on Apple Silicon via MLX with 8-bit quantization
- 🔌 **MCP Server** — Expose LFM2.5 as tools for Claude Desktop, Cursor, OpenClaw, and any MCP-compatible client
- 🔀 **Multi-Mac Cluster** — Distribute inference across multiple M4 Macs with automatic load balancing and failover
- 📊 **Live Stats** — Token count, generation speed (tok/s), and response time displayed in real-time
- 🎯 **Quick Prompts** — Pre-built prompts for common tasks (CRISPR explanation, code architecture, creative writing, data analysis)
## 🏗️ Architecture
```
Single-Mac Mode:
Client → MLX Server (:8080)
Cluster Mode (4× throughput):
Client → Cluster LB (:5200) → Mac 1 MLX (:8080)
→ Mac 2 MLX (:8080)
→ Mac 3 MLX (:8080)
→ Mac 4 MLX (:8080)
```
## 🚀 Quick Start
### Prerequisites
- macOS with Apple Silicon (M1/M2/M3/M4)
- Python 3.10+
- [MLX](https://github.com/ml-explore/mlx) and [mlx-lm](https://github.com/ml-explore/mlx-examples/tree/main/llms)
### 1. Install
```bash
# Clone the repo
git clone https://github.com/WispAyr/LFM2.5-local.git
cd LFM2.5-local
# Install MLX dependencies (if needed)
pip install mlx-lm
# Install MCP dependencies
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
### 2. Run
```bash
# Just the MLX server + chat UI
./start_server.sh
# MLX + MCP server (for Claude Desktop, Cursor, OpenClaw)
./start_server.sh --mcp
# Everything (MLX + MCP + web UI)
./start_server.sh --all
# Multi-Mac cluster (MLX + Load Balancer + MCP + web UI)
./start_server.sh --cluster
```
### 3. Open
| Service | URL | Description |
|---------|-----|-------------|
| Chat UI | http://localhost:3000 | Web chat interface |
| MLX API | http://localhost:8080/v1 | OpenAI-compatible API |
| Cluster LB | http://localhost:5200/v1 | Load-balanced API (cluster mode) |
| Cluster Status | http://localhost:5200/cluster/status | Live node health |
| MCP Server | http://localhost:5100/mcp | MCP protocol endpoint |
## 🔌 MCP Integration
The MCP server exposes LFM2.5 as tools that any MCP-compatible client can call.
### Tools
| Tool | Description |
|------|-------------|
| `chat` | General-purpose chat — questions, reasoning, writing |
| `summarize` | Summarize text with configurable length |
| `analyze_code` | Code review, bug detection, suggestions |
| `translate` | Translate text to any language |
### Resources
| URI | Description |
|-----|-------------|
| `lfm25://model/info` | Model architecture, specs, capabilities |
| `lfm25://server/status` | Live MLX server connection status |
### Prompts
| Prompt | Description |
|--------|-------------|
| `code_review` | Structured code review template |
| `explain_concept` | ELI5-style concept explanation |
### Connect to Claude Desktop
Add to `~/Library/Application Support/Claude/claude_desktop_config.json`:
```json
{
"mcpServers": {
"lfm25": {
"command": "/path/to/LFM2.5-local/.venv/bin/python3",
"args": ["/path/to/LFM2.5-local/mcp_server.py"]
}
}
}
```
### Connect to Cursor
Settings → MCP Servers → Add and paste the same config.
### Connect to OpenClaw
Point to `http://localhost:5100/mcp` as an MCP server endpoint.
## 📁 Project Structure
```
├── index.html # Chat web UI (single-file, no build step)
├── mcp_server.py # FastMCP server with tools, resources, prompts
├── cluster.py # Multi-Mac cluster load balancer
├── cluster_config.json # Cluster node configuration
├── server.py # Web UI proxy server (serves static + proxies API)
├── mcp_config.json # MCP config template for Claude Desktop / Cursor
├── start_server.sh # Launcher script (--mcp, --all, --cluster flags)
├── demo_lfm25.py # Python demo script
└── requirements.txt # Python dependencies
```
## 🔀 Multi-Mac Cluster
Distribute inference across multiple M4 Macs for **~480 tok/s combined throughput** and automatic failover.
### Setup
**1. On each Mac**, start the MLX server:
```bash
pip install mlx-lm
python3 -m mlx_lm.server --model LiquidAI/LFM2.5-1.2B-Instruct-MLX-8bit --port 8080 --host 0.0.0.0
```
**2. Edit `cluster_config.json`** on your primary Mac with each node's IP:
```json
{
"nodes": [
{"name": "Mac-1", "host": "192.168.1.10", "port": 8080, "weight": 1},
{"name": "Mac-2", "host": "192.168.1.11", "port": 8080, "weight": 1},
{"name": "Mac-3", "host": "192.168.1.12", "port": 8080, "weight": 1},
{"name": "Mac-4", "host": "192.168.1.13", "port": 8080, "weight": 1}
]
}
```
**3. Start the cluster:**
```bash
./start_server.sh --cluster
```
### Features
- **Least-loaded routing** for POST requests, round-robin for GETs
- **Health checking** every 5 seconds with automatic failover
- **SSE streaming** passthrough for real-time token generation
- **Node weights** for prioritizing faster Macs
- **Status dashboard** at `/cluster/status` with per-node metrics
## 🧠 About LFM2.5
LFM2.5 is **not a Transformer**. It's a hybrid architecture from [Liquid AI](https://liquid.ai) combining:
- **10× Gated Short Convolution** layers — fast sequential processing
- **6× Grouped Query Attention** layers — selective attention
This gives it a unique performance profile: extremely fast inference (~120+ tok/s on M-series Macs) with a small 1.17B parameter footprint.
| Spec | Value |
|------|-------|
| Parameters | 1.17B |
| Layers | 16 |
| Context | 32K tokens |
| Vocab | 65,536 |
| Training | 28T tokens |
| Quantization | 8-bit (MLX) |
## 📄 License
MIT
## 🔗 Links
- [Liquid AI](https://liquid.ai)
- [LFM2.5 on HuggingFace](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-MLX-8bit)
- [LFM2.5 Paper](https://arxiv.org/abs/2412.04332)
- [MLX](https://github.com/ml-explore/mlx)
- [FastMCP](https://gofastmcp.com)
