gollm

# gollm OpenAI-compatible LLM gateway — Go implementation. Single static binary, no runtime deps. _OpenAI 호환 LLM proxy — Go 구현. 단일 정적 바이너리._ ## Highlights - **OpenAI-compatible** `/v1/chat/completions`, `/v1/embeddings`, `/v1/rerank`, `/v1/models` - **Anthropic Messages API** `/v1/messages` - **Providers**: OpenAI / Anthropic / Ollama (incl. Cloud) / OpenRouter / TEI / A2A / `execcli` (Codex CLI, Claude CLI as a provider) - **Router**: load-balancing (round-robin / random / weighted / least-busy), retries, multi-deployment fallbacks, model rename proxy with parameter injection (e.g. `no_reason=true` per backend) - **Advanced routing**: tag-based (free/paid tiers, team access), wildcard patterns (anthropic/*, groq/*) - **Request/response caching** with TTL, selective per-model caching - **Streaming + non-streaming** end-to-end (incl. tool-call streams) - **Web search / reader** subsystem — 10 providers (searxng/brave/tavily/exa/ firecrawl/serpapi/jina/zai/ollama/http_fetch), strategies auto/fallback/best/blend, vendor-format alias routes (`/tavily/search`, `/brave/res/v1/web/search`, `/reader/<url>`, …), auto-injection as `web_search`/`web_fetch` tools to chat models - **MCP** integration (HTTP / SSE), tools auto-injected into matching models - **Inject pipeline** — declarative request mutators (system-prompt prefix, parameter overrides, conditional `no_reason`, etc.) - **Passthrough endpoints** — proxy arbitrary external APIs through gollm - **Virtual keys** (auth + per-key TPM/RPM), Prometheus `/metrics`, Swagger-style admin UI - **Debug capture** — input/output dump on demand, raw-dump trigger word - **A2A protocol** — Agent-to-Agent endpoints ## Quick start ```bash # 1) grab a binary VERSION=v0.1.0 PLATFORM=darwin-arm64 # or darwin-amd64 / linux-amd64 / linux-arm64 curl -L -o gollm "https://github.com/yourusername/gollm/releases/download/${VERSION}/gollm-${PLATFORM}" chmod +x gollm # 2) run with a minimal config cat > config.yaml <<'YAML' server: { addr: ":4000" } model_list: - model_name: gpt-4o-mini litellm_params: { model: openai/gpt-4o-mini, api_key: ${OPENAI_API_KEY} } YAML OPENAI_API_KEY=sk-... ./gollm -config config.yaml # 3) call it (drop-in for OpenAI SDK) curl http://localhost:4000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}' ``` Full walkthrough: **[docs/en/quickstart.md](docs/en/quickstart.md)** · 한국어: **[docs/ko/quickstart.md](docs/ko/quickstart.md)**. ## Build from source ```bash git clone https://github.com/yourusername/gollm cd gollm go build ./cmd/gollm # → ./gollm go test ./... ``` Cross-compile (no CGO, fully static): ```bash GOOS=linux GOARCH=amd64 CGO_ENABLED=0 \ go build -trimpath -ldflags="-s -w" -o gollm-linux-amd64 ./cmd/gollm ``` ## Documentation | Topic | English | 한국어 | |---|---|---| | Quickstart | [quickstart.md](docs/en/quickstart.md) | [quickstart.md](docs/ko/quickstart.md) | | Configuration | [config.md](docs/en/config.md) | [config.md](docs/ko/config.md) | | Providers | [providers.md](docs/en/providers.md) | [providers.md](docs/ko/providers.md) | | HTTP API | [api.md](docs/en/api.md) | [api.md](docs/ko/api.md) | | Web search/reader | [web.md](docs/en/web.md) | [web.md](docs/ko/web.md) | | Header transforms | [headers.md](docs/en/headers.md) | [headers.md](docs/ko/headers.md) | | Inject pipeline | [inject.md](docs/en/inject.md) | — | | PII detection | [pii.md](docs/en/pii.md) | [pii.md](docs/ko/pii.md) | | Response sanitize | [sanitize.md](docs/en/sanitize.md) | [sanitize.md](docs/ko/sanitize.md) | | Budget management | [budget.md](docs/en/budget.md) | [budget.md](docs/ko/budget.md) | | Concurrency | [concurrency.md](docs/en/concurrency.md) | [concurrency.md](docs/ko/concurrency.md) | | Quota management | [quota.md](docs/en/quota.md) | [quota.md](docs/ko/quota.md) | | Logging | [logging.md](docs/en/logging.md) | [logging.md](docs/ko/logging.md) | | Caching | [caching.md](docs/en/caching.md) | [caching.md](docs/ko/caching.md) | | Advanced Routing | [routing.md](docs/en/routing.md) | [routing.md](docs/ko/routing.md) | | A2A protocol | [a2a.md](docs/en/a2a.md) | — | ## License Apache License 2.0 — See [LICENSE](LICENSE) for details.