Content
# Comfy Pilot
[](https://github.com/ConstantineB6/Comfy-Pilot/stargazers)
[](https://opensource.org/licenses/MIT)
[](https://registry.comfy.org/publishers/constantine/nodes/comfy-pilot)
Talk to your ComfyUI workflows. Comfy Pilot gives Claude Code direct access to see, edit, and run your workflows — with an embedded terminal right inside ComfyUI.
## Why?
Building ComfyUI workflows means manually searching for nodes, dragging connections, and tweaking values one at a time. With Comfy Pilot, you just describe what you want:
- *"Build me an SDXL workflow with ControlNet"* — Claude creates all the nodes, connects them, and sets the parameters
- *"Look at the output and increase the detail"* — Claude sees your generated image and adjusts the workflow
- *"Download the FLUX schnell model and set up a workflow for it"* — Claude downloads the model and builds a workflow from scratch
No copy-pasting node names. No hunting through menus. Just say what you want.
## Installation
**CLI (Recommended):**
```bash
comfy node install comfy-pilot
```
**ComfyUI Manager:**
1. Open ComfyUI
2. Click **Manager** → **Install Custom Nodes**
3. Search for "Comfy Pilot"
4. Click **Install**
5. Restart ComfyUI
**Git Clone:**
```bash
cd ~/Documents/ComfyUI/custom_nodes && git clone https://github.com/ConstantineB6/comfy-pilot.git
```
Claude Code CLI will be installed automatically if not found.
## Requirements
- ComfyUI
- Python 3.8+
## Features
- **MCP Server** - Gives Claude Code direct access to view, edit, and run your ComfyUI workflows
- **Embedded Terminal** - Full xterm.js terminal running Claude Code right inside ComfyUI
- **Image Viewing** - Claude can see outputs from Preview Image and Save Image nodes
- **Graph Editing** - Create, delete, move, and connect nodes programmatically
## Demo
https://github.com/user-attachments/assets/325b1194-2334-48a1-94c3-86effd1fef02
## Usage
1. Restart ComfyUI after installation
2. The floating Claude Code terminal appears in the top-right corner
3. The MCP server is automatically configured for Claude Code
4. Ask Claude to help with your workflow:
- "What nodes are in my current workflow?"
- "Add a KSampler node connected to my checkpoint loader"
- "Look at the preview image and tell me what you see"
- "Run the workflow up to node 5"
## MCP Tools
The MCP server provides these tools to Claude Code:
| Tool | Description |
|------|-------------|
| `get_workflow` | Get the current workflow from the browser |
| `summarize_workflow` | Human-readable workflow summary |
| `get_node_types` | Search available node types with filtering |
| `get_node_info` | Get detailed info about a specific node type |
| `get_status` | Queue status, system stats, and execution history |
| `run` | Run workflow (optionally up to a specific node) or interrupt |
| `edit_graph` | Batch create, delete, move, connect, and configure nodes |
| `view_image` | View images from Preview Image / Save Image nodes |
| `search_custom_nodes` | Search ComfyUI Manager registry for custom nodes |
| `install_custom_node` | Install a custom node from the registry |
| `uninstall_custom_node` | Uninstall a custom node |
| `update_custom_node` | Update a custom node to latest version |
| `download_model` | Download models from Hugging Face, CivitAI, or direct URLs |
### Example: Creating Nodes
```
Create a KSampler and connect it to my checkpoint loader
```
Claude will use `edit_graph` to:
1. Create the KSampler node
2. Connect the MODEL output from CheckpointLoader to KSampler's model input
3. Position it appropriately in the graph
### Example: Viewing Images
```
Look at the preview image and describe what you see
```
Claude will use `view_image` to fetch and analyze the image output.
### Example: Downloading Models
```
Download the FLUX.1 schnell model for me
```
Claude will use `download_model` to download from Hugging Face to your ComfyUI models folder. Supports:
- Hugging Face (including gated models with token auth)
- CivitAI
- Direct download URLs
## Terminal Controls
- **Drag** title bar to move
- **Drag** bottom-right corner to resize
- **−** Minimize
- **×** Close
- **↻** Reconnect session
## Architecture
```
┌─────────────────────────────────────────────────────┐
│ Browser (ComfyUI) │
│ ┌─────────────────┐ ┌──────────────────────────┐ │
│ │ xterm.js │ │ Workflow State │ │
│ │ Terminal │ │ (synced to backend) │ │
│ └────────┬────────┘ └────────────┬─────────────┘ │
│ │ WebSocket │ REST API │
└───────────┼────────────────────────┼────────────────┘
│ │
▼ ▼
┌─────────────────────────────────────────────────────┐
│ ComfyUI Server │
│ ┌─────────────────┐ ┌──────────────────────────┐ │
│ │ PTY Process │ │ Plugin Endpoints │ │
│ │ (claude CLI) │ │ /claude-code/* │ │
│ └─────────────────┘ └──────────────────────────┘ │
└─────────────────────────────────────────────────────┘
│ │
│ ▼
│ ┌──────────────────────────┐
└──────────▶│ MCP Server │
│ (stdio transport) │
└──────────────────────────┘
```
## Files
- `__init__.py` - Plugin backend: WebSocket terminal, REST endpoints
- `js/claude-code.js` - Frontend: xterm.js terminal, workflow sync
- `mcp_server.py` - MCP server for Claude Code integration
- `CLAUDE.md` - Instructions for Claude when working with ComfyUI
## Troubleshooting
### "Command 'claude' not found"
Install Claude Code CLI:
**macOS / Linux / WSL:**
```bash
curl -fsSL https://claude.ai/install.sh | bash
```
**Windows (PowerShell):**
```powershell
irm https://claude.ai/install.ps1 | iex
```
**Windows (CMD):**
```cmd
curl -fsSL https://claude.ai/install.cmd -o install.cmd && install.cmd && del install.cmd
```
### MCP server not connecting
The plugin auto-configures MCP on startup. Check ComfyUI console for errors, or manually add to `~/.claude.json`:
```json
{
"mcpServers": {
"comfyui": {
"command": "python3",
"args": ["/path/to/comfy-pilot/mcp_server.py"]
}
}
}
```
### Terminal disconnected
Click the ↻ button to reconnect, or check ComfyUI console for errors.
## License
MIT
