Content
# FTR Partner Self-Assessment MCP Server
An MCP server and interactive CLI that automates the AWS Foundational Technical Review (FTR) partner self-assessment process. It evaluates partner-submitted compliance documents (SOC 2 Type II reports and WAFR reports) against defined checks and returns structured PASS/FAIL decisions with reasoning.
## Architecture
The system connects partner-submitted PDF reports to Amazon Bedrock for LLM-powered evaluation. A Model Context Protocol (MCP) server exposes evaluation tools to AI assistants, while a standalone CLI provides a guided terminal workflow. Both paths share a common evaluation engine backed by Bedrock, SOC 2 and WAFR check registries, and calibration guides that shape scoring decisions.
## Overview
Partners seeking AWS validation must submit evidence for two distinct review tracks:
| Track | Document Required | Checks Evaluated |
|---|---|---|
| **SOC 2** | SOC 2 Type II Report | SOC-001 through SOC-005 |
| **WAFR** | AWS Well-Architected Framework Review Report | WAFR-FTR-001 through WAFR-FTR-006 |
This package provides three ways to evaluate submissions:
1. **MCP Server** — Exposes evaluation tools to AI assistants (Kiro, Claude, etc.) via the Model Context Protocol
2. **Interactive CLI** — A terminal-based evaluation workflow with guided prompts, progress spinners, and color-coded results
3. **Kiro Power** — A native Kiro IDE integration that loads calibration criteria directly into chat, no server or CLI required
## Installation
### Option 1 — Standalone Binary (no Node.js required)
Download the binary for your platform from the [GitHub Releases](../../releases) page:
| Platform | File |
|---|---|
| macOS (Apple Silicon) | `ftr-eval-mcp-macos-arm64` |
| macOS (Intel) | `ftr-eval-mcp-macos-x64` |
| Linux x64 | `ftr-eval-mcp-linux-x64` |
| Linux ARM64 | `ftr-eval-mcp-linux-arm64` |
| Windows x64 | `ftr-eval-mcp-win-x64.exe` |
**macOS / Linux** — make the binary executable and run it:
```bash
chmod +x ftr-eval-mcp-macos-arm64
./ftr-eval-mcp-macos-arm64 evaluate
```
**Windows** — run it directly:
```cmd
ftr-eval-mcp-win-x64.exe evaluate
```
### Option 2 — Build from Source (requires Node.js >= 18)
Clone the repository and build locally:
```bash
git clone https://github.com/aws-samples/sample-ftr-self-assessment-mcp.git
cd sample-ftr-self-assessment-mcp
npm install
npm run build
npm link # optional — registers 'ftr-eval-mcp' as a global command
```
Then run via Node directly:
```bash
node dist/server.js evaluate
node dist/server.js serve
```
## Prerequisites
- AWS credentials configured (for Bedrock access)
- Node.js >= 18.0.0 (build from source only — not required for standalone binaries)
Node.js 22 is recommended. If you use [nvm](https://github.com/nvm-sh/nvm), switch to it before building:
```bash
nvm use 22
```
If Node.js 22 is not yet installed:
```bash
nvm install 22
nvm use 22
```
## Configuration
The server uses sensible defaults out of the box. Override via environment variables or CLI flags:
| Environment Variable | CLI Flag | Default | Description |
|---|---|---|---|
| `FTR_AWS_REGION` | `--region` | `us-east-1` | AWS region for Bedrock API calls |
| `FTR_BEDROCK_MODEL` | `--model` | `global.anthropic.claude-opus-4-6-v1` | Bedrock model ID |
| `FTR_TRANSPORT` | `--transport` | `stdio` | MCP transport: `stdio` or `http` |
| `FTR_PORT` | `--port` | `3000` | Port for HTTP transport |
| `FTR_LOG_LEVEL` | N/A | `info` | Log level: `debug`, `info`, `warn`, `error` |
Resolution order (highest priority first): CLI flags → Environment variables → Defaults
Example with a custom region:
```bash
FTR_AWS_REGION=us-west-2 ftr-eval-mcp
```
Or in your MCP config:
```json
{
"mcpServers": {
"ftr-eval-mcp": {
"command": "node",
"args": ["dist/server.js", "serve"],
"env": {
"FTR_AWS_REGION": "eu-west-1"
}
}
}
}
```
### MCP Config Levels
You can register this MCP server at different levels depending on your needs:
| Level | Config Path | Scope |
|---|---|---|
| **Workspace** | `<project>/.[IDE]/settings/mcp.json` | Only available when this specific project is open |
| **User (global)** | `~/.[IDE]/settings/mcp.json` | Available across all workspaces for the current user |
**Precedence:** Workspace config overrides user config. If the same server is defined at both levels, the workspace-level definition wins when that project is open. Outside that workspace, the user-level config applies.
**When to use each level:**
- **Workspace** — Best when developing or testing the server locally. The config lives with the project and won't affect other workspaces.
- **User** — Best when the server is stable and you want it available everywhere without per-project setup.
Example workspace config (`.kiro/settings/mcp.json`):
```json
{
"mcpServers": {
"ftr-eval-mcp": {
"command": "node",
"args": ["/path/to/dist/server.js", "serve"],
"disabled": false,
"autoApprove": ["get_prompt_template", "parse_pdf", "evaluate_submission"]
}
}
}
```
The `autoApprove` array lists tool names that the AI assistant can invoke without prompting for confirmation. Tools not in this list require manual approval before each execution.
## Usage
### MCP Server Mode (default)
Start the MCP server for use with AI assistants:
```bash
ftr-eval-mcp
```
With options:
```bash
ftr-eval-mcp serve --transport stdio --region us-east-1 --model <bedrock-model-id>
```
### Interactive CLI Mode
Launch the guided evaluation workflow:
```bash
ftr-eval-mcp evaluate
```
This will prompt you to:
1. Select a report type (SOC 2 or WAFR)
2. Enter the path to your PDF report
3. Choose a specific check or evaluate all
### Non-Interactive Mode
For scripting and CI/CD pipelines:
```bash
ftr-eval-mcp evaluate --report-type wafr --file ./path/to/report.pdf
```
Evaluate a single check:
```bash
ftr-eval-mcp evaluate --report-type soc2 --file ./report.pdf --control-id SOC-001
```
### CLI Options
```
ftr-eval-mcp evaluate --help
Options:
--report-type <type> Report type: soc2 or wafr
--file <path> Path to the PDF report file
--control-id <id> Specific control ID to evaluate (optional)
--region <region> AWS region (default: us-east-1)
--model <modelId> Bedrock model ID
```
## MCP Tools
When running as an MCP server, the following tools are exposed:
| Tool | Description |
|---|---|
| `parse_pdf` | Parse a PDF file and extract text content |
| `get_controls` | Get check definitions for a report type |
| `get_calibration_guide` | Get the calibration guide for a report type |
| `evaluate_submission` | Evaluate a PDF submission against checks |
| `get_prompt_template` | Get the FTR evaluation prompt template |
## Development
### Build
```bash
npm run build
```
### Run Tests
```bash
npm test
```
### Run Locally (without installing globally)
```bash
npm run build
node dist/server.js evaluate
```
### Build Standalone Binaries
```bash
npm run build:binaries
```
This produces platform-specific executables in `binaries/` for macOS (ARM/x64), Linux (x64/ARM), and Windows (x64).
## Project Structure
```
src/
├── server.ts # Entry point: commander routing (serve/evaluate)
├── cli.ts # CLI orchestrator (evaluation workflow)
├── cli/
│ ├── input-collector.ts # Interactive prompts and flag validation
│ ├── credential-validator.ts # AWS credential check via STS
│ ├── progress-reporter.ts # Spinner and progress display (ora)
│ └── results-formatter.ts # Color-coded results output (chalk)
├── config.ts # Configuration resolution
├── engine/
│ ├── evaluation-engine.ts # Core evaluation orchestration
│ ├── bedrock-client.ts # Amazon Bedrock API client
│ ├── prompt-builder.ts # LLM prompt construction
│ └── decision-parser.ts # Parse LLM responses into decisions
├── parsers/
│ └── pdf-parser.ts # PDF text extraction
├── registries/
│ ├── control-registry.ts # Check definitions
│ └── calibration-guide-registry.ts # Calibration guides
├── tools/ # MCP tool registrations
│ ├── evaluate-submission.ts
│ ├── get-calibration-guide.ts
│ ├── get-controls.ts
│ ├── get-prompt-template.ts
│ └── parse-pdf.ts
├── types.ts # Shared TypeScript types
└── assets/
├── calibration-guides/ # SOC 2 and WAFR calibration guides
├── controls/ # Check definition files
└── prompts/ # LLM prompt templates
```
## Kiro Power Tool
This project also includes a Kiro power at `.kiro/powers/ftr-self-assessment/` for direct use within the Kiro IDE. The steering files load automatically and give Kiro full calibration criteria to evaluate FTR submissions in chat.
## Checks Reference
### SOC 2 Checks
| Check | Description |
|---|---|
| **SOC-001** | SOC 2 Type II report must be active (issued within the last 12 months) |
| **SOC-002** | Auditor opinion must be exactly "Unqualified" |
| **SOC-003** | AWS must be listed as an in-scope cloud provider |
| **SOC-004** | The partner's specific solution must appear in the audit scope |
| **SOC-005** | Both Security AND Availability Trust Service Categories must be present |
### WAFR Checks
| Check | Description |
|---|---|
| **WAFR-FTR-001** | Review must be completed within 12 months |
| **WAFR-FTR-002** | Zero active High-Risk Issues (HRIs) in the Security pillar |
| **WAFR-FTR-003** | Zero active High-Risk Issues (HRIs) in the Operational Excellence pillar |
| **WAFR-FTR-004** | Zero active High-Risk Issues (HRIs) in the Reliability pillar |
| **WAFR-FTR-005** | Partner's solution must be identifiable in the WAFR workload name or description |
| **WAFR-FTR-006** | Report must contain all required sections (Workload properties, Lens overview, Improvement plan, Lens details) |
## Key Rules
- Expired reports (SOC 2 or WAFR older than 12 months) always **FAIL**
- SOC 2 Type I does **not** qualify; must be Type II
- Submitting a WAFR report for a SOC 2 check (or vice versa) **FAILS** immediately
- Only **active** (open/unresolved) HRIs cause failure — resolved HRIs are ignored
- Medium-Risk Issues (MRIs) never cause failure regardless of count or status
## Exit Codes
| Code | Meaning |
|---|---|
| 0 | Evaluation completed (regardless of PASS/FAIL), or user cancelled |
| 1 | Error: AWS credentials not configured, invalid inputs, or system error |
## Contributing
See [CONTRIBUTING](CONTRIBUTING.md) for guidelines on bug reports, pull requests, and the code of conduct.
## Security
See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for information on reporting security issues.
## License
This library is licensed under the MIT-0 License. See the [LICENSE](LICENSE) file.
