gen

# Gen Service for launching and interacting with LLM. ## Capabilities - Session, history, and dialogue parameter management - Agent scenarios: tool-loop, tool invocation, repeated inference - Connecting MCP servers and external tool catalog - RAG: document loading, context search, rerank - LLM inference on CPU or GPU, streaming answers - Multiple inference nodes with different models and hardware - Connecting as a Go library to your own backend ([docs/library.md](docs/library.md)) ## Documentation - [docs/server.md](docs/server.md) — server - [docs/llm-runner.md](docs/llm-runner.md) — runner - [docs/library.md](docs/library.md) — library - [docs/models-help.md](docs/models-help.md) — models and VRAM --- ## Repositories - **[gen-app](https://github.com/magomedcoder/gen-app)** - client application for Linux, Android, iOS, macOS, and Windows - **[gen-mcp-servers](https://github.com/magomedcoder/gen-mcp-servers)** — set of MCP servers for external tools --- ## Using gen as a library For your own backend, you can include the module `github.com/magomedcoder/gen` and run gen-runner separately: inference in runner, orchestration of tools/MCP/RAG in `gen/pkg/*`. Details: **[docs/library.md](docs/library.md)**.