内容
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="assets/Opfor_white%20text.svg" />
<source media="(prefers-color-scheme: light)" srcset="assets/Opfor_black%20text.svg" />
<img src="assets/Opfor_black%20text.svg" alt="OPFOR" width="250" />
</picture>
</p>
<p align="center">
<strong>Open-source adversary emulation for AI agents, LLM apps, and MCP servers.</strong><br/>
Test your AI like a real attacker would — from your CLI, your IDE, or a browser extension that anyone on your team can use.
</p>
<p align="center">
<a href="LICENSE"><img src="https://img.shields.io/badge/license-Apache_2.0-444441?style=flat&labelColor=3d3d3a" alt="License: Apache 2.0"></a>
<a href="https://github.com/KeyValueSoftwareSystems/agent-opfor/stargazers"><img src="https://img.shields.io/github/stars/KeyValueSoftwareSystems/agent-opfor?style=flat&labelColor=3d3d3a&color=444441" alt="GitHub stars"></a>
<a href="https://discord.gg/Sb8xfmrjjW"><img src="https://img.shields.io/badge/Discord-join-5865F2?style=flat&logo=discord&logoColor=white&labelColor=3d3d3a" alt="Discord"></a>
<a href="#evaluator-coverage"><img src="https://img.shields.io/badge/OWASP-LLM%20%2B%20Agentic%20%2B%20MCP%20%2B%20API-185FA5?style=flat&labelColor=3d3d3a" alt="OWASP coverage"></a>
</p>
<p align="center">
<a href="https://trendshift.io/repositories/75655?utm_source=trendshift-badge&utm_medium=badge&utm_campaign=badge-trendshift-75655" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/75655/daily?language=TypeScript" alt="KeyValueSoftwareSystems%2Fagent-opfor | Trendshift" width="250" height="55"/></a>
</p>
<p align="center">
<a href="https://agentopfor.ai/">Website</a> ·
<a href="https://docs.agentopfor.ai">Docs</a> ·
<a href="https://github.com/KeyValueSoftwareSystems/agent-opfor">GitHub</a> ·
<a href="https://chromewebstore.google.com/detail/blkflfeicejfnbecanjaolbllmoobenp">Browser Extension</a> ·
<a href="https://discord.gg/Sb8xfmrjjW">Discord</a>
</p>
OPFOR is short for _Opposition Force_ — a military term for the unit that plays the enemy in training, so the rest of the army learns what real attacks feel like before they come. We named the tool after that idea: to defend AI agents better, you have to attack them first.
<p align="center">
<img src="assets/opfor-high-level.svg" alt="How OPFOR works" width="860" />
</p>
## Why KeyValue built this
We've shipped 130 products for 90 startups over the last ten years. In the last 18 months, almost every one of them had an AI agent in it — and every one of those teams hit the same wall when it came to testing.
So we built OPFOR. For ourselves first. Now open source.
Apache 2.0. Built from India.
## Quick Start
```bash
npm install -g @keyvaluesystems/agent-opfor-cli
export OPENAI_API_KEY=your-key # or GEMINI_API_KEY, ANTHROPIC_API_KEY, etc.
```
**One-shot** — runs the setup wizard and immediately starts the scan:
```bash
opfor run
```
**Two-step** — save a config you can reuse or commit to CI:
```bash
opfor setup # wizard saves a config to .opfor/configs/
opfor run --config .opfor/configs/<file> # run any time against the saved config
```
https://github.com/user-attachments/assets/a6a3cff2-2cf9-4486-944e-ac0163e7ea04
## What opfor does
Opfor red-teams the full AI agent surface — prompts, tools, MCP servers, memory, and multi-turn reasoning. It generates targeted attacks for OWASP LLM Top 10, OWASP Agentic AI Top 10, OWASP MCP Top 10, OWASP API Security, and EU AI Act bias suites, fires them at your target, and judges each response with an LLM.
Most red-team tooling in this space is excellent at one thing — a probe library, a developer evaluator, a programmatic framework. Opfor covers more ground in one tool:
- **Browser extension for non-developers** — anyone on your team can red-team a deployed chatbot, no code, no env vars, no YAML
- **Run opfor as an MCP server** — let your AI coding agent in Cursor or Claude Desktop red-team your other agents through natural language
- **Full OWASP coverage in one tool** — LLM Top 10, Agentic AI Top 10, MCP Top 10, API Security Top 10
- **No black box** — every attack prompt, request, response, and judge verdict is logged; reproducible, auditable, forkable
- **Built for agents, not just models** — designed for tool calls, MCP, memory, and multi-turn state from day one
- **Trace-aware** — integrates with Langfuse and Netra so the LLM judge sees what your agent did internally, not just what it said
## Five ways to run opfor
Different people on your team need different entry points. Opfor ships five.
| Mode | How | Best for |
| ------------------------ | --------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| 🖥️ **CLI** | `opfor setup` → `opfor run` | Engineers, CI/CD, terminal-first workflows |
| 🌐 **Browser extension** | Install the extension, click the icon on any chat interface | Product managers, designers, QA, security analysts — anyone who can't or won't write code |
| 🤖 **MCP server** | Register opfor in Cursor or Claude Desktop, then ask in chat | AI coding agents that test your other agents |
| ⚡ **Skills** | `/opfor-setup` · `/opfor-run` · `/opfor-mcp-setup` · `/opfor-mcp-run` | Developers who want one-command testing inside their IDE |
| 📦 **SDK** | `npm install @keyvaluesystems/agent-opfor-sdk`, then call `run` / `hunt` from your code | Programmatic red-teaming and custom workflows |
All five share the same evaluators, attack templates, and judge logic.
→ [CLI reference](docs/cli.md) · [Browser extension setup](docs/browser-extension.md) · [MCP setup](docs/mcp.md) · [Skills setup](docs/skills.md) · [SDK reference](docs/sdk.md) · [Session handling](docs/sessions.md)
## How it works
<p align="center">
<img src="assets/opfor-execution-flow.svg" alt="What happens during opfor run" width="860" />
</p>
When you run a scan, opfor:
1. **Fetches target info** — connects to your agent, detects available tools, MCP endpoints, capabilities
2. **Plans attacks per category** — generates targeted prompts for each evaluator in your selected suite
3. **Emulates the attack** — runs multi-turn adversarial conversations (real requests, real responses)
4. **Evaluates with a judge** — an LLM judge classifies each response with pass/fail + reasoning
5. **Generates a report** — HTML for browsing, JSON for CI/CD, all artifacts logged for reproducibility
Each run lands in its own subfolder under `.opfor/reports/run-report-<compactTs>-<slug>-<shortId>/` containing `<slug>-report.html` and `<slug>-report.json`. Autonomous `opfor hunt` runs use the same layout under `hunt-report-<compactTs>-<slug>-<shortId>/`.
### Testing cost
Every run reports what its instrumented LLM calls cost, broken down by model:
```text
Token usage: 51,323 input / 6,057 output (57,380 total)
Testing cost: $0.18
deepseek/deepseek-v4-pro [attacker]: $0.037
anthropic/claude-opus-5 [judge]: $0.14
```
This is **opfor's own spend** — the attacker and judge LLMs. It excludes your target's inference cost, which opfor cannot see from the outside. The per-model split is the useful part: the judge is often the bigger share, and pointing it at a cheaper model is usually the easiest saving.
Prices come from a snapshot of LiteLLM's public price map that ships with the package, so runs work offline and a report re-rendered later produces the same figure. Cached input is billed at the provider's cache rate — multi-turn attacks re-send the conversation each turn, and that repeated prefix is often ~100× cheaper than fresh text, so the figure tracks the real bill rather than a worst case. Caveats worth knowing:
- **Caching is only credited when it's reported.** A provider that doesn't break out cached tokens, or a model with no published cache rate, is charged at the full input rate — an over-estimate, chosen over quietly under-reporting.
- **Unknown models are never counted as free.** A model missing from the price table is reported as unpriced and the total is marked a lower bound, rather than silently reading as $0.
- **A few helper calls aren't metered yet.** Trace curation, session summarisation and one JSON helper don't report token usage, so their spend is missing from the total. Treat the figure as a floor.
→ [Token usage and testing cost](docs/cli.md#token-usage-and-testing-cost)
## Evaluator coverage
Opfor ships with curated suites that map to industry standards. Pick a suite or run individual evaluators.
| Suite ID | Standard | Focus |
| ------------------ | ------------------------- | ------------------------------------------------------------------------- |
| `owasp-llm-top10` | OWASP LLM Top 10 (2025) | Prompt injection, jailbreaks, sensitive disclosure, system prompt leakage |
| `owasp-agentic-ai` | OWASP Agentic AI Top 10 | Excessive agency, tool misuse, agent goal hijack, memory poisoning |
| `owasp-mcp-top10` | OWASP MCP Top 10 (2025) | Secret exposure, scope escalation, tool description injection, SSRF |
| `owasp-api` | OWASP API Security Top 10 | BOLA, BFLA, SQL injection |
| `eu-ai-act-bias` | EU AI Act — Bias | Age, gender, race, disability |
→ [Full evaluator reference and OWASP mapping](docs/evaluators.md)
## Trace-aware testing
Plug opfor into your observability stack and the LLM judge sees not just the final response — but every tool call, retrieval, and intermediate reasoning step. Out of the box, opfor integrates with **[Langfuse](https://langfuse.com)** and **[Netra](https://getnetra.ai)**.
```json
"telemetry": {
"provider": "langfuse",
"langfuse": { "baseUrl": "https://cloud.langfuse.com" }
}
```
This catches what input/output testing misses — PII that leaks into a tool call but never reaches the user, scope escalations in MCP that don't change the response text, agents that retrieve unauthorized data but render a clean reply.
→ [Trace-aware testing guide](docs/telemetry.md)
## Autonomous Red-Teaming
`opfor hunt` skips the config file entirely. Give it an endpoint and an objective, and a multi-agent system — commander, operators, scout — runs an adaptive attack campaign on its own: recon, strategy, multi-turn probing, report. Unlike `opfor run`, the agents run on Claude only (via a Claude API key, `claude setup-token`, or your local `claude login` session) — your target can be anything.
```bash
opfor hunt \
--endpoint "https://your-agent.com/v1/chat" \
--objective "Find jailbreaks, system-prompt leakage, and safety bypasses."
```
Add `--ui` to watch the attack tree unfold in a live dashboard.
→ [Full reference](docs/hunt.md)
## Browser extension — red-team a chatbot
The browser extension is opfor's no-code path. Install from the Chrome Web Store, open any chat interface, click the opfor icon, pick a suite, and watch it run.
https://github.com/user-attachments/assets/80c2692f-b18b-4899-99df-e7eb8d50b02a
It auto-detects the chat interface, sends attack prompts as if you were typing them, watches the responses, and downloads an HTML report when done. No CLI, no target setup, no YAML.
This is the path for the half of every product team that doesn't open a terminal.
→ [Install from the Chrome Web Store](https://chromewebstore.google.com/detail/blkflfeicejfnbecanjaolbllmoobenp) · [Setup guide](docs/browser-extension.md)
## SDK — embed red-teaming in your code
The SDK is opfor's programmatic path. Install `@keyvaluesystems/agent-opfor-sdk`, call `run` or `hunt`, and get structured results back — no CLI, no config files, no subprocess.
```typescript
import { Opfor } from "@keyvaluesystems/agent-opfor-sdk";
const opfor = new Opfor({ apiKey: process.env.ANTHROPIC_API_KEY });
const results = await opfor.run({
target: { url: "https://api.example.com/chat" },
suite: "owasp-llm-top10",
});
```
Use it in CI, in test suites, or anywhere you need red-teaming without leaving TypeScript.
→ [SDK reference](docs/sdk.md)
## Examples
| Example | Description |
| ----------------------------------------------------- | --------------------------------------------------------------- |
| [vanilla-chat](tests/e2e/agents/vanilla-chat) | Plain customer support chatbot — test LLM-level vulnerabilities |
| [customer-support](tests/e2e/agents/customer-support) | Tool-calling agent with PostgreSQL — test BOLA, BFLA, RBAC, PII |
| [vulnerable-server](tests/e2e/mcp/vulnerable-server) | Sample MCP server with intentional vulnerabilities |
## Supported LLM providers
| Provider | Env var | Default model |
| ----------------- | ------------------------------ | ---------------------------------- |
| Groq | `GROQ_API_KEY` | `llama-3.3-70b-versatile` |
| OpenAI | `OPENAI_API_KEY` | `gpt-4o-mini` |
| Anthropic | `ANTHROPIC_API_KEY` | `claude-3-5-haiku-20241022` |
| Google | `GOOGLE_GENERATIVE_AI_API_KEY` | `gemini-2.0-flash` |
| OpenAI-compatible | `OPFOR_API_KEY` + `baseURL` | LiteLLM, OpenRouter, Azure, Ollama |
## Contributing
Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct, and the process for submitting pull requests to us.
## Authors
Built by the team at [KeyValue Software Systems](https://keyvalue.systems). Contact [contact@agentopfor.ai](mailto:contact@agentopfor.ai) for all enquiries.
## Security
Use opfor only on systems you own or are authorized to test. To report a vulnerability in opfor itself, see [SECURITY.md](SECURITY.md) — do not open a public issue.
## License
Opfor is licensed under Apache 2.0 — see the [LICENSE](LICENSE) file for details.
---
<p align="center">
Built with ❤️ by <a href="https://keyvalue.systems">KeyValue</a>
</p>
连接信息
猜你喜欢
暂无推荐服务器