Content
# sandbox-host
Self-hosted Firecracker microVM platform for AI agent sandboxes — the infra that powers `mcp__easybits__sandbox_*` tools.
Spin up an isolated Linux microVM in ~5 seconds, exec commands inside it via HTTP, destroy it. Each microVM has its own kernel (real KVM hardware isolation), private network, and an in-VM agent that handles exec/file ops over a bearer-authenticated REST API.
```
[MCP client] [EasyBits Fly] [KS-5 bare metal]
sandbox_create ──────► sandbox-host API ──────► fc.Manager.Spawn
│
▼
firecracker process
│ KVM
▼
microVM (Linux 6.1)
│
▼
sandbox-agent :9909
(HTTP exec/files)
```
Built as the backend for [EasyBits](https://easybits.cloud)'s `sandbox_*` MCP toolset. Inspired by E2B / Daytona / Modal but self-hosted on a single bare-metal box.
## Spec
- **Spawn**: ~5s cold start (template ext4 → fresh microVM with running agent)
- **Concurrency**: ~15-25 simultaneous microVMs on a 32GB host (1 vCPU / 512MB each)
- **Isolation**: KVM hardware (separate kernel per VM, not just namespaces)
- **Network**: private bridge (`fcbr0` 172.20.0.0/16) with NAT egress
- **Templates**: any squashfs/ext4 with systemd; `nanoclaw`, `ubuntu`, `python`, `node` baked
- **Cost**: ~$20/mo on OVH KS-5 bare metal (vs $300+/mo on equivalent managed)
## Architecture
### Components
| Component | Path | Role |
|---|---|---|
| `sandbox-host` (Go) | `cmd/sandbox-host/` | HTTP API on the bare-metal host. Manages microVM lifecycle. Listens on `127.0.0.1:8080` (Caddy fronts TLS) |
| `sandbox-agent` (Go) | `cmd/sandbox-agent/` | Tiny HTTP daemon baked into each rootfs. Listens inside the microVM on `:9909`. Bearer-auth'd. Exposes `/exec`, `/run-code`, `/files/*` |
| `internal/fc/` | Go package | Firecracker lifecycle (spawn process, configure via Unix socket API, start, destroy) + IP/tap allocator + agent client |
| `scripts/` | shell | Provisioning + template builders (run on the host) |
| `config/` | files | Caddyfile, systemd unit, templates manifest |
### REST API (host-side)
```
POST /v1/sandbox spawn (template, timeoutSeconds, name)
GET /v1/sandbox?owner=<id> list owner's sandboxes
GET /v1/sandbox/{id} status (incl. assigned IP)
DELETE /v1/sandbox/{id} destroy + cleanup
POST /v1/sandbox/{id}/exec shell command inside microVM
POST /v1/sandbox/{id}/run-code inline python/node/bash
POST /v1/sandbox/{id}/files/write
GET /v1/sandbox/{id}/files/read?path=...
GET /v1/sandbox/{id}/files/list?path=...
```
All authed via `Authorization: Bearer <SANDBOX_HOST_TOKEN>`. Owner identity propagated via `X-Easybits-Owner` header.
### MicroVM lifecycle
1. **Allocate** IP from 172.20.0.10–172.20.255.250, derive MAC + tap name
2. **Create tap** interface on `fcbr0` bridge
3. **Snapshot rootfs**: `cp` from `templates/<name>.ext4` to `snapshots/<sandbox-id>.ext4`
4. **Inject token**: mount snapshot, write `/etc/sandbox-agent/.env`, unmount
5. **Spawn** `firecracker --api-sock <path>` as child process
6. **Configure** via Unix socket: kernel, machine-config, drive, network-interface
7. **Start** via `PUT /actions {InstanceStart}`
8. **Wait** for sandbox-agent `/health` to return 200 (~3-5s)
9. **Update** store with status=running and assigned IP
Destroy reverses: SIGTERM firecracker → delete tap → release IP → rm rootfs snapshot.
## Deploy (cold start)
Pre-reqs on a fresh Debian 12 / Ubuntu 24 bare-metal:
```bash
# 1. Bootstrap (apt update, hardening, KVM verification)
bash scripts/bootstrap.sh
# 2. Firecracker binary + kernel + sample rootfs
bash scripts/install_firecracker.sh
# 3. Network bridge fcbr0 + NAT
bash scripts/setup_network.sh
# 4. Docker (used by template builder) + ship sandbox-agent binary
bash scripts/install_docker_and_agent.sh
# 5. Build a base template
bash scripts/build_template_squashfs.sh ubuntu /var/lib/firecracker/rootfs.squashfs 2048
# 6. Install + start the host service (assumes sandbox-host binary is in /usr/local/bin)
cp config/sandbox-host.service /etc/systemd/system/
mkdir -p /etc/sandbox-host
echo "SANDBOX_HOST_TOKEN=$(openssl rand -hex 32)" > /etc/sandbox-host/.env
echo "SANDBOX_AGENT_TOKEN=$(openssl rand -hex 32)" >> /etc/sandbox-host/.env
echo "SANDBOX_HOST_ADDR=127.0.0.1:8080" >> /etc/sandbox-host/.env
echo "SANDBOX_HOST_TEMPLATES=/etc/sandbox-host/templates.yaml" >> /etc/sandbox-host/.env
cp config/templates.yaml /etc/sandbox-host/
systemctl daemon-reload && systemctl enable --now sandbox-host
# 7. (optional) Caddy in front for TLS
apt install caddy
cp config/Caddyfile /etc/caddy/Caddyfile # edit hostname
systemctl reload caddy
```
Or run **`scripts/deploy.sh`** from a workstation to do steps 1-7 in one shot (cross-compiles Go for linux/amd64 and scps everything).
## Templates
A "template" is an ext4 image bootable as a Firecracker microVM. It must:
1. Have systemd as init (`/sbin/init` → systemd)
2. Include `/usr/local/bin/sandbox-agent` + the systemd unit + multi-user.target.wants symlink
3. Leave eth0 alone so the kernel `ip=…` cmdline IP persists (no DHCP needed)
Two builders are provided:
| Script | Source | Use when |
|---|---|---|
| `build_template_squashfs.sh` | Firecracker CI squashfs | Base templates (ubuntu) — already has working systemd |
| `build_template.sh` | Docker image | Custom templates (nanoclaw, claude-code) — exports docker container, injects systemd-related bits |
Both inject the agent binary, mask `fcnet.service` (CI's hardcoded-IP service), and produce `/var/lib/sandbox-host/templates/<name>.ext4`.
## Tested templates
| Template | Source | Memory | Notes |
|---|---|---|---|
| ubuntu | CI squashfs | 512 MB | Minimal Ubuntu 24.04. Works ✓ |
| python | docker.io/python:3.12-slim | 512 MB | Pending — needs systemd injection |
| node | docker.io/node:22-slim | 512 MB | Pending |
| bun | docker.io/oven/bun:1 | 512 MB | Pending |
| claude-code | docker.io/ubuntu:24.04 + claude-code | 1 GB | Pending |
| nanoclaw | github.com/blissito/nanoclaw | 2 GB | Pending — requires Chromium + agent-runner. See nanoclaw repo |
## Smoke test
```bash
SANDBOX_HOST=https://your-host.example.com \
SANDBOX_TOKEN=... \
bash scripts/smoke_test.sh
```
Validates: /health → create → status → exec → destroy.
## Lessons learned (from POC build)
OVH-specific gotchas that consumed time during initial provisioning are documented in `docs/ovh-gotchas.md`. Highlights:
- **`customizations.sshKey` SILENTLY DROPS ED25519 keys** — use RSA-4096 or ECDSA
- `authenticationSecret` API returns a one-time secret ID, not the password; URL must be opened by a human
- Boot mode `230242` = customer rescue mode (Debian-12 based) — used for disk recovery if SSH lock-out
- `/etc/resolv.conf` in production install is a dangling symlink — must `rm -f && echo "nameserver ..."` before chroot
## License
Apache 2.0 (see LICENSE).
## Contributing
Issues and PRs welcome. The code is intentionally minimal (~1500 lines Go total) — read it before opening a PR.
