Huntable-CTI-Studio

# Huntable CTI Studio <img width="952" height="64" alt="image" src="https://github.com/user-attachments/assets/4b29bc70-b518-4559-af0c-caf23b86000d" /> ## ** SECURITY WARNING ** !! The DEFAULT mode is UNAUTHENTICATED (`AUTH_MODE=disabled`) -- intended for research, learning, and automation on a trusted/local network. DO NOT expose the default mode to a hostile network. For hardened deployments, an optional enterprise boundary is available: SSO through a trusted-header reverse proxy (Google / GitHub / Microsoft), role-based access control, CSRF protection, fail-closed production startup, and a redacted, actor-attributed audit trail. See [Authentication](docs/guides/authentication.md) and [Enterprise SSO Setup](docs/guides/enterprise-sso.md). Even when hardened, the phase-one audit log is database-backed and mutable by database administrators (forward exports to a SIEM for higher assurance), and the app is not intended to store classified or proprietary threat intelligence at this time. ## **Huntable CTI Studio v7.8.0 "Europa"** - A Cyber Threat Intelligence ML/AI workbench that automates collection, extraction, and detection rule generation from 38 seeded OSINT sources (see `config/sources.yaml`; runtime may add or replace rows after DB sync). ## Purpose Aggregates cybersecurity threat intelligence from RSS feeds and web scraping; uses regex and AI to score relevance and extract observables; generates SIGMA detection rules, and prevents duplicates through jaccard similarity matching against 3,000+ community rules. More details here: https://dfirtnt.wordpress.com/2026/02/04/introducing-huntable-cti-studio/ ## Architecture - **8 default services**: PostgreSQL (pgvector), Redis, FastAPI web app, maintenance runtime, Celery workers (default + workflow), scheduler, and a one-shot Codex auth initializer - **LangGraph**: Orchestrates the 7-step agentic workflow as a linear pipeline with conditional early-exit gates (state machine, checkpointing) - **Database-backed workflows**: Articles, workflow executions, Sigma rules, presets, settings, evals, and supporting metadata - **Source healing**: operator-invoked Claude Code skill diagnoses failing sources (RSS inspection, sitemap discovery, JS-rendering detection, WP JSON API probing) and proposes a config fix for approval — never auto-applies, never runs on a schedule - **Multi-model AI**: OpenAI and Anthropic, an optional local LM Studio provider, and an optional subscription-backed Codex provider for workflows ## Agentic Workflow The main engine is a LangGraph-based workflow executed by Celery workers: 1. **Platform Detection** — Platform classification for capability-based extractor routing 2. **Junk Filter** — Conservative content filtering 3. **LLM Rank** — Relevance scoring 4. **Extract Agent** — Extract observables (command-line, process trees, network indicators, hunt queries, and Windows-specific artifacts) 5. **Generate SIGMA** — Create detection rules with iterative validation 6. **Similarity Search** — Compare against indexed Sigma rules using behavioral similarity 7. **Promote to Queue** — Queue novel rules for human review and PR submission ## Quick Start **Requirements:** Docker + Docker Compose ```bash git clone https://github.com/dfirtnt/Huntable-CTI-Studio.git cd Huntable-CTI-Studio ./setup.sh --no-backups ./start.sh ``` **Access:** - Web UI: http://localhost:8001 - API Docs: http://localhost:8001/docs - Docs site: http://localhost:8000 - Health: `curl http://localhost:8001/health` - CLI: `./run_cli.sh <command>` ### MCP (optional) MCP server for agents (articles, sources, SIGMA, workflow tools), with read tools plus scoped, audited write tools. Requires app env/DB as for the web app. **Tool reference:** [docs/reference/mcp-tools.md](docs/reference/mcp-tools.md) (`get_article` uses **Article ID** from search output, not list position). **Connecting an MCP client (Claude Code, Claude Desktop, Cursor, …):** the repo ships a committed [`.mcp.json`](.mcp.json) that points at [`scripts/run_mcp_server.sh`](scripts/run_mcp_server.sh). Clients that read project `.mcp.json` (e.g. Claude Code in this directory) need **no setup** — approve the `huntable-cti-studio` server when prompted. For other clients, register `bash scripts/run_mcp_server.sh` as the command. The launcher runs the server inside the Docker `cli` service, so it works regardless of cwd or shell state — and semantic search works on every platform (including Intel Mac, whose host venv can't load the torch/sentence-transformers embedding model). **Docker must be running.** Run it by hand (debugging) — same launcher the `.mcp.json` uses: ```bash bash scripts/run_mcp_server.sh # runs the server in the Docker cli container ``` ## For Coding Agents And Contributors If you need to get oriented quickly, read these first: 1. [`AGENTS.md`](AGENTS.md) 2. [`docs/development/agent-orientation.md`](docs/development/agent-orientation.md) 3. [`docs/architecture/workflow-data-flow.md`](docs/architecture/workflow-data-flow.md) 4. [`docs/development/testing.md`](docs/development/testing.md) 5. [`config/presets/AgentConfigs/README.md`](config/presets/AgentConfigs/README.md) Runtime entry points worth opening early: - [`src/web/modern_main.py`](src/web/modern_main.py) - [`src/web/routes/__init__.py`](src/web/routes/__init__.py) - [`src/workflows/agentic_workflow.py`](src/workflows/agentic_workflow.py) - [`src/worker/celery_app.py`](src/worker/celery_app.py) - [`src/config/workflow_config_schema.py`](src/config/workflow_config_schema.py) - [`run_tests.py`](run_tests.py) ## Documentation The documentation is organized under `/docs` and is published with MkDocs Material. - **Start here**: `docs/index.md` - **Quickstart**: `docs/quickstart.md` - **Architecture**: `docs/architecture/overview.md`, `docs/architecture/workflow-data-flow.md` - **Development**: `docs/development/setup.md`, `docs/development/testing.md` - **Reference**: `docs/reference/api.md`, `docs/reference/schemas.md`, `docs/reference/mcp-tools.md` - **Security & Auth**: [`docs/guides/authentication.md`](docs/guides/authentication.md), [`docs/guides/enterprise-sso.md`](docs/guides/enterprise-sso.md) ## License MIT License — see [LICENSE](LICENSE) for details.