evidiq-rubric-mcp

<p align="center"> <img src="https://raw.githubusercontent.com/evidiq/evidiq-rubric-mcp/main/assets/evidiq-rubric.png" width="200" alt="EVIDIQ Rubric" /> </p> <h1 align="center">EVIDIQ Rubric</h1> <p align="center"><strong>Does this deliverable meet the contract?</strong></p> <p align="center"> Specify &middot; Check &middot; Attest — a deterministic verdict on work an agent was paid for. </p> <p align="center"> <a href="https://evidiq.dev">evidiq.dev</a> &middot; <a href="https://evidiq.dev/docs/rubric">Rubric Docs</a> &middot; <a href="https://github.com/evidiq/evidiq-rubric-mcp">Rubric Repository</a> </p> <p align="center"> <a href="https://mcp.evidiq.dev/rubric/mcp"><img src="https://img.shields.io/badge/MCP%20Server-Live-6E56CF?style=flat-square" alt="MCP Server" /></a> <a href="https://0g.ai"><img src="https://img.shields.io/badge/0G-Storage%20Anchor-00C2A8?style=flat-square" alt="0G Storage anchor" /></a> <a href="https://www.oklink.com/xlayer"><img src="https://img.shields.io/badge/X%20Layer-USDT0-3CCF4E?style=flat-square" alt="X Layer USDT0" /></a> <a href="https://mcp.evidiq.dev/rubric/x402"><img src="https://img.shields.io/badge/x402-0.005%E2%80%930.03%20USDT0-2563EB?style=flat-square" alt="x402 pricing" /></a> <a href="https://web3.okx.com/onchainos/dev-docs/payments/service-seller-sdk"><img src="https://img.shields.io/badge/Payments-Official%20OKX%20SDK-121212?style=flat-square&logo=okx&logoColor=white" alt="Official OKX Payment SDK" /></a> <a href="https://www.okx.ai/agents/9848"><img src="https://img.shields.io/badge/OKX.AI-Agent%20%239848%20Listed-121212?style=flat-square&logo=okx&logoColor=white" alt="OKX.AI Agent 9848 listed" /></a> <a href="./LICENSE"><img src="https://img.shields.io/badge/License-MIT-3DA639?style=flat-square" alt="License: MIT" /></a> <a href="#development"><img src="https://img.shields.io/badge/tests-118%20passing-success?style=flat-square" alt="118 tests passing" /></a> </p> --- EVIDIQ Rubric turns *"is this deliverable acceptable?"* into a deterministic, signed verdict against a spec both sides agreed on **before** the work started. Notary proves an output happened; Rubric decides whether it is acceptable — with a verdict anyone holding the same bytes can recompute. Deterministic by construction: JSON Schema, normalised string and number matching, arithmetic re-checks, and a small declarative criteria language. No model in the hot path. Same input, same rule set, same spec → byte-identical verdict and digest. > **What a Rubric report proves, and what it does not.** Proves: this exact > deliverable, under this spec version and this rule set version, produced these > criterion results and this verdict; anyone with the same bytes recomputes the > same digest. Does **not** prove: that the deliverable is correct, useful, or > complete in a sense the spec did not encode. A `PASS` means the spec's > criteria were met — no more. ## Position in the family | Service | Question | Boundary against Rubric | | --- | --- | --- | | Core | Can I trust this agent? | Counterparty risk *before* work. Rubric judges the work. | | Notary | Can I prove this output happened? | Provenance of an output. Rubric judges its *conformance*. | | Operator | Can an agent act in a browser? | Produces work; does not grade it. | | Sentinel | Is this MCP safe? | Judges a service. Rubric judges a deliverable. | | Atlas | What does this dataset say? | Analyses data. Rubric checks an answer against a spec. | | Lineage | Is this dependency trustworthy? | Supply chain, not deliverables. | | Vault | What exactly did the agent do? | Records history. Rubric decides acceptance; the verdict belongs in Vault. | | Redact | Is this content safe to share? | Sensitive values. Rubric checks completeness and grounding. | | Warden | Is this code safe to commit? | Source code quality. Rubric checks any deliverable against its spec. | | Assay | What will this transaction do? | Pre-signature intent, not deliverables. | | **Rubric** | **Does this deliverable meet the contract?** | **The acceptance verdict, signed and 0G-anchored.** | Composition worth documenting: **Rubric decides, Vault records the verdict, Notary timestamps the deliverable, Core scored the counterparty.** ## Status | Field | Value | | --- | --- | | MCP slot | #11 | | Public endpoint | `https://mcp.evidiq.dev/rubric/mcp` | | Host port | 3011 (loopback only; the public surface is the Traefik reverse proxy) | | Repository | to be created at `github.com/evidiq/evidiq-rubric-mcp` (not yet pushed — pending operator authorisation) | | OKX.AI agent id | not yet registered (registration is a separate, explicit step the operator runs after a proven paid call) | | Engine | `EVIDIQ-Rubric/1.0` | | Spec DSL version | `1.0` | | Rule set version | `1.0.0` | | Tests | 118 passing (`npm test`) | | Deployment | image `evidiq-rubric:latest`, container `evidiq-rubric`, env `/root/evidiq-rubric.env` (0600, root-only), Traefik `PathPrefix(/rubric)` with a `rubric-strip` middleware | ## Endpoints | Path | Method | Purpose | | --- | --- | --- | | `/mcp` | `POST` | The MCP endpoint (Streamable HTTP). | | `/mcp` | `HEAD` | **402** — answered explicitly so a reachability prober never hangs. | | `/mcp` | empty `POST` | **402** — never 400, never 406. | | `/x402` | `GET` | Public pricing discovery listing all ten tools and the v2 challenge. | | `/skill.md` | `GET` | The agent-readable Skill. | | `/health` | `GET` | Liveness. | All paths are served after Traefik strips `/rubric`. The container never sees the prefix, so the x402 challenge advertises `PUBLIC_BASE_URL/mcp`, never `req.url` (runbook section 27-B). ## Tools — five paid, five free Prices are fixed atomic USD₮0 amounts and never read from an env var. ### Paid | Tool | Atomic | USDT0 | Returns | | --- | ---: | ---: | --- | | `check_schema` | 5000 | 0.005 | Validates a structured deliverable against a JSON Schema (draft 2020-12) or an inline shape. Per-path violations with JSON Pointers, expected vs actual **type** (never the value), and a `PASS`/`BLOCK` verdict. | | `verify_grounding` | 10000 | 0.010 | Every quoted span, number, date and citation in the deliverable is checked against the supplied sources. Reports unsupported spans with offsets and which source each supported span matched. | | `check_acceptance` | 15000 | 0.015 | Evaluates the deliverable against an acceptance spec. Returns `PASS` / `REVIEW` / `BLOCK`, per-criterion results, and the criteria that decided it. | | `diff_deliverable` | 20000 | 0.020 | Compares a revision against the previous version **and** the spec: which criteria flipped, what changed by section, and whether a previously failing criterion now passes. | | `attest_acceptance` | 30000 | 0.030 | Binds spec digest + deliverable digest + verdict, signs EIP-191, anchors the digest to 0G (best effort, never fatal). The artifact for a dispute. **Refuses without a signer.** | ### Free | Tool | Returns | | --- | --- | | `rubric_capabilities` | Spec DSL version, criterion catalog with severities, supported deliverable kinds, limits, full pricing, **and the names of all ten tools**. | | `validate_spec` | Parse-checks the spec and the deliverable; returns detected kinds, criterion count, and counts by severity — **no findings**, no charge. Must refuse exactly what the paid tools refuse. | | `estimate_cost` | One tool, or the whole table when none is named. Unknown name → says it is not a tool of this service and shows the table. | | `verify_rubric_report` | Recomputes the digest and verifies the EIP-191 signature offline. On an unsigned report: digest verdict plus "origin cannot be attributed". | | `get_artifact` | Report or attestation by id within its in-memory TTL. A miss is `found: false`, never an error. | ## The acceptance spec — a small declarative language A spec is JSON. No expressions, no code, no regex the caller cannot see. It is versioned (`specVersion: "1.0"`) and an unknown version is rejected rather than guessed at. ```json { "specVersion": "1.0", "deliverable": { "kind": "markdown" }, "criteria": [ { "id": "has-summary", "severity": "blocker", "type": "requiredSection", "section": "Summary", "minWords": 40 }, { "id": "cites-sources", "severity": "blocker", "type": "minCitations", "count": 2 }, { "id": "no-placeholder", "severity": "blocker", "type": "forbiddenPhrase", "phrases": ["TODO", "lorem ipsum", "as an AI"] }, { "id": "length", "severity": "high", "type": "wordCount", "min": 300, "max": 1200 }, { "id": "schema", "severity": "blocker", "type": "jsonSchema", "schemaRef": "#/schemas/report" }, { "id": "numbers-grounded", "severity": "high", "type": "numbersGrounded", "tolerance": 0.001 } ] } ``` Criterion types for v1, all deterministic: - `requiredSection` — markdown heading by name; optional `minWords` for the section body. - `forbiddenPhrase` — case-insensitive, smart-quote-folded; reports offsets and phrase indices, never text. - `wordCount` — `min` / `max` inclusive. - `jsonSchema` — bounded JSON Schema 2020-12 evaluation (see below). - `minCitations` — distinct citation markers (e.g. `[1]`, `[Smith 2020]`, `[unknown]`). - `citationsResolve` — each marker must resolve against a supplied source id. - `numbersGrounded` — every numeric claim matched against sources within `tolerance`. - `dateSanity` — dates fall within an optional `windowStart`/`windowEnd` window. - `requiredField` — JSON Pointer (RFC 6901) path present in a JSON deliverable. - `enumValue` — JSON Pointer field value is in an allowed set. - `uniqueBy` — a JSON array has distinct values at a JSON Pointer field. - `arithmeticConsistent` — `sumField` sums to `targetField` across rows. **Verdict rules, fixed and documented:** - any failing `blocker` → `BLOCK` - no blocker, any failing `high` → `REVIEW` - everything passes → `PASS` - a criterion that cannot be evaluated (unparseable input, missing source) is `INCONCLUSIVE`, reported, excluded from `PASS`, and forces at least `REVIEW`. - empty deliverable, empty criteria, every criterion INCONCLUSIVE → `REFUSED` (a verdict about nothing is the worst answer the service can give). ## Grounding without a model `verify_grounding` extracts quoted spans, numerals with units, percentages, dates, and citation markers from the deliverable, and looks for support in the supplied sources using, in order: exact match; whitespace- and case-normalised match; unicode-NFC and smart-quote-folded match; and for numbers, value equality within a stated tolerance after unit normalisation. What it reports: `supported` with the source id and offset, `unsupported` with the offset in the deliverable, or `inconclusive` with a reason. **What it never does: decide whether an unsupported sentence is *false*.** Absence of support is not evidence of falsehood, and the report says so in those words. Findings carry **offsets and kinds, never the matched text** for anything that could be sensitive — a report gets pasted into tickets. ## Bounded JSON Schema evaluation (section 8.4) `check_schema` and the `jsonSchema` criterion use AJV (draft 2020-12) under a strict wrapper: - No remote `$ref`. A `$ref` starting with `http(s)://` is rejected. - No `$ref` cycles. A visited set keyed by resolved ref catches cycles. - Depth cap (default 32) and node cap (default 10 000) prevent exponential blowup on pathological schemas. - A time budget (default 2 000 ms) bounds total evaluation. A timeout returns `INCONCLUSIVE`, never `PASS`. Per-path violations report **expected vs actual type, never the value**, so a rejected `{name: 123}` against `{name: {type: "string"}}` reports `expectedType: "string", actualType: "number"` without echoing `123`. ## Integrity — deterministic digest + optional EIP-191 Every report carries an `integrity` block: ```json { "integrity": { "algorithm": "SHA-256", "digest": "<sha256 of canonical envelope>", "signature": "0x…", // optional "signer": "0x…", // optional "signed": false, // present only when no signer is configured "note": "No signer configured; the digest matches but the origin cannot be attributed." } } ``` The canonical envelope is property-order-stable JSON over `engine`, `ruleSetVersion`, `deliverableDigest`, `specDigest`, `verdict`, `counts`, `criteria`, `limits`. **The same bytes produce the same digest, regardless of who signs.** `verify_rubric_report` recomputes the digest and verifies the EIP-191 signature offline; an unsigned report gets a digest verdict plus "origin cannot be attributed" — never `valid: true`. ## OKX Payment SDK — the official path Rubric uses the official OKX Onchain OS Payment SDK (`@okxweb3/x402-core` + `@okxweb3/x402-evm`) against the OKX facilitator, reused from Sentinel/Atlas/Warden (runbook section 23). The price reaches the SDK as an explicit USD₮0 atomic `AssetAmount`, so the quoted fee token always matches the token registered on the listing. Perbehaviours, each of which has its own past rejection (runbook 0.13, 29-A): - `HEAD /mcp` → **402**, immediately. - Empty unauthenticated `POST /mcp` → **402** with a challenge. Never 400. - The challenge `resource.url` comes from `PUBLIC_BASE_URL`, never `req.url`. - Prices reach the SDK as USD₮0 atomic amounts. - A facilitator success without a transaction hash is honoured with a `[x402] SETTLED WITHOUT TX` warning; `pending`/`timeout` without a hash fails. - A malformed payment envelope is refused **locally** (~75 ms), never forwarded. - Free tools answer without any payment header, always. ## Hard safety rules (section 8) 1. A credential in the input (private key, mnemonic, API key by issuer shape) → **refused**, and nothing is stored, attested, or anchored. 2. Never echo deliverable content or source text in findings. Offsets, kinds, counts. 3. No network fetching of sources in v1. Sources are supplied inline. A URL in a spec is rejected with a reason, not fetched. 4. `jsonSchema` evaluation is bounded: no remote `$ref`, no `$ref` cycles, depth and node caps, and a time budget. 5. Artifacts live in memory with a short TTL, addressed by digest. Nothing is written to disk. ## Limits | Limit | Default | Env | | --- | ---: | --- | | Maximum input bytes | 524288 | `RUBRIC_MAX_INPUT_BYTES` | | Maximum sources | 20 | `RUBRIC_MAX_SOURCES` | | Maximum criteria | 100 | `RUBRIC_MAX_CRITERIA` | | Artifact TTL (ms) | 600000 | `RUBRIC_ARTIFACT_TTL_MS` | ## Pricing - **Per paid tool call**: atomic USD₮0 amounts as listed above, on X Layer (`eip155:196`, chain `196`). - Token: `USDT0` (`0x779ded0c9e1022225f8e0630b35a9b54be713736`, 6 decimals). - EIP-712 token domain: `{ name: "USD₮0", version: "1" }`. - Free tools are always `HTTP 200` and never gated. ## Security boundaries - The **signer key** is read from `RUBRIC_SIGNER_PRIVATE_KEY` and is the ONLY way reports get signed. There is no fallback key in the source. With no signer configured, the service runs unsigned and `attest_acceptance` refuses — an unsigned attestation would attest to nothing. - The **0G anchor key** is `OG_PRIVATE_KEY`, used to anchor digests only, never payloads. 0G failure is best-effort: never fatal to a paid call (the caller already paid; the local content-addressed artifact remains verifiable). - The **OKX SDK credentials** (`OKX_API_KEY/SECRET_KEY/PASSPHRASE`) are the ONLY path to verification and settlement when present. A partial credential set throws at startup — never silently downgrades to a non-SDK path. - The **settle key** (`X402_SETTLE_KEY`) is the legacy self-hosted settlement path. It is bypassed when OKX SDK credentials are present. ## Quickstart ```bash # Free: capabilities and pricing discovery curl https://mcp.evidiq.dev/rubric/x402 | python3 -m json.tool # Free: validate a spec before paying curl -X POST https://mcp.evidiq.dev/rubric/mcp \ -H 'content-type: application/json' \ -H 'accept: application/json, text/event-stream' \ -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"validate_spec","arguments":{"spec":{"specVersion":"1.0","deliverable":{"kind":"markdown"},"criteria":[{"id":"x","severity":"blocker","type":"wordCount","min":10}]}}}}' # Paid: check_acceptance (402 without a PAYMENT-SIGNATURE envelope) curl -X POST https://mcp.evidiq.dev/rubric/mcp \ -H 'content-type: application/json' \ -H 'accept: application/json' \ -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"check_acceptance","arguments":{"deliverable":"# Summary\nThis deliverable meets the agreed spec.","spec":{"specVersion":"1.0","deliverable":{"kind":"markdown"},"criteria":[{"id":"has-summary","severity":"blocker","type":"requiredSection","section":"Summary","minWords":5}]}}}}' # Expected: 402 with amount:"15000", x402Version:2, scheme:"exact" ``` ## Self-host ```bash # 1. Install + build npm ci && npm run build # 2. Run with no payment env → all tools free (honest downgrade) PORT=3100 npm start # 3. Verify curl -s localhost:3100/health curl -sX POST localhost:3100/mcp -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' \ -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' # 4. Add OKX SDK credentials + RUBRIC_SIGNER_PRIVATE_KEY to enable the paid path PORT=3100 OKX_API_KEY=… OKX_SECRET_KEY=… OKX_PASSPHRASE=… \ X402_PAY_TO=0x2a8efe3093278bb4bd3b2d9c7b5ba992ca4fc9b0 \ PUBLIC_BASE_URL=http://localhost:3100 \ RUBRIC_SIGNER_PRIVATE_KEY=… \ npm start ``` ## Development ```bash npm test # vitest run — 118 tests npm run build # tsc -p tsconfig.json npm run dev # tsx watch start-server.ts ``` ## Configuration (`/root/evidiq-rubric.env` on the box; `.env.example` here) See [`.env.example`](./.env.example) for the full list with names only (no values). Note in particular: - **Variable names are named after THIS service.** A copy-pasted variable name is what put a public signing key in production once (`RUBRIC_SIGNER_PRIVATE_KEY`, never `VAULT_SIGNER_PRIVATE_KEY`). - Every key is accepted **with or without the `0x` prefix** — every deployment here stores bare hex. - The passphrase contains `#`; keep it unquoted for `docker --env-file`. ## Proven on-chain The pre-submit gate requires one real paid call per paid tool at least once across the suite, with the receipt mining `status 0x1`, the container logging `[x402] SETTLED` with a hash, and the report signature tracing to the project signer (`0x8a3c7524…ee7D`, the family OG signer = `RUBRIC_SIGNER_PRIVATE_KEY`). The broadcaster for every settled tx is the OKX official facilitator relayer (not our buyer, not our settle wallet) — which is what proves the SDK is the live settlement path (runbook section 27-F). | Tool | Atomic | Settle tx | Block | Receipt | Signer | Verified | | --- | ---: | --- | ---: | --- | --- | --- | | `check_schema` | 5000 | `0xa3a45c28e169977c12ed82a1b748e2a264886cdcad777ded4354889bdf2300d2` | 66395009 | `0x1` | `0x8a3c7524…ee7D` | `valid:true, digestMatch:true, signatureValid:true` | | `verify_grounding` | 10000 | `0xc74c10945fbd5639d19c311430fbd4ca73fcc4ea48545df2dbc5811e90717197` | 66395729 | `0x1` | `0x8a3c7524…ee7D` | `valid:true, digestMatch:true, signatureValid:true` | | `check_acceptance` | 15000 | `0xc6a5c5c2219dbd7f8333d62603bbc997ad9d892e412f78df10d688c206984439` | 66395081 | `0x1` | `0x8a3c7524…ee7D` | `valid:true, digestMatch:true, signatureValid:true` | | `diff_deliverable` | 20000 | `0x84b3e129d88493a76731e1920e251ddc6a8550b8ba09ec99b794634f87401947` | 66395738 | `0x1` | `0x8a3c7524…ee7D` | `valid:true, digestMatch:true, signatureValid:true` | | `attest_acceptance` | 30000 | `0xe39d63c3181f429e880612bcfb110ea1c037e4331fcbf58f7065f9065d062d86` | 66395093 | `0x1` | `0x8a3c7524…ee7D` | `valid:true, digestMatch:true, signatureValid:true` | Container logs show `[x402] SETTLED` for every paid tool name with its tx hash. The free `verify_rubric_report` tool was called immediately after each paid call on the report the service just produced; all five returned `valid:true, digestMatch:true, signatureValid:true` (gate item 10). ## Free tool valid-argument sweep (gate item 3) Every free tool called with genuinely valid arguments against the live endpoint returns a substantive, non-error result. Per-tool: | Free tool | Valid args | Result (vs description) | | --- | --- | --- | | `rubric_capabilities` | `{}` | `engine:EVIDIQ-Rubric/1.0`, `specDslVersion:1.0`, 10 tools (5 paid + 5 free), 12 criterion types, `signing.available:true` ✓ matches description | | `validate_spec` | valid spec + deliverable | `ok:true`, `criterionCount:2`, `bySeverity:{blocker:2}`, `detectedDeliverableKind:markdown` ✓ no findings, as advertised | | `estimate_cost` | `{tool:"check_acceptance"}` | `{atomic:"15000", usd:"0.015"}` ✓ correct ladder price | | `verify_rubric_report` | unsigned report shape | `valid:false, signed:false, error:"…origin cannot be attributed"` ✓ honest on unsigned (defect 0.1) | | `get_artifact` | `{artifactId:"art_testdoesnotexist123"}` | `found:false, reason:"No such artifact…"`, not `isError` ✓ miss is an answer (defect 0.5) | The bare-`{}` sweep (gate item 2) and wrong-but-plausible sweep (gate item 4) are covered by the test suite (118 passing). The capabilities diff (gate item 6) is green: every name in `tools/list` appears in `rubric_capabilities`. ## Hospitality for the OKX listing validator `onchainos payment quote https://mcp.evidiq.dev/rubric/mcp --method POST` returns `ok:true` (gate item 11). Quoted per-tool: `check_schema` 5000 (`amountHuman:"0.005"`, `tokenSymbol:"USD₮0"`, `recipient:` family `0x2a8efe30…`). The SDK-path liveness check (gate item 8) is also green: a malformed envelope is refused locally in ~85 ms; a well-formed envelope with a bad signature reaches the OKX facilitator and comes back rejected as `invalid_signature` in ~253 ms (runbook 29-A). Reproduce the live proofs with `node paid-call-proof.mjs` (default: `check_schema`; override with `TOOL=verify_grounding node paid-call-proof.mjs`, etc.). Full-suite spend across all five paid tools: about 0.08 USDT0 (the phase-3 budget). ## Architecture ``` HTTP (node:http) ── /health, /x402, /skill.md, /mcp (HEAD → 402) │ ▼ withX402Gate ──── decodePaymentHeader ── verifyPaymentLocal (local preflight) │ │ │ └── OkxSdkVerifier.verify + settle (official SDK) │ │ │ └── poll getSettleStatus on pending/timeout ▼ rawHandler (JSON-RPC dispatch) │ ▼ createServer() — 5 paid tools + 5 free tools │ ▼ lib/rubric/ — spec, criteria (one fn per type), verdict, grounding, schema (bounded AJV), report (canonical digest + EIP-191), attest (EIP-191 + 0G anchor), artifacts (TTL cache), secrets (credential refusal). ``` ## Links - OKX Payment SDK: https://web3.okx.com/onchainos/dev-docs/payments/service-seller-sdk - EVIDIQ runbook (local): `EVIDIQ-RUNBOOK.md` — section 23 is the source of truth for payments. - Skill document: served at `https://mcp.evidiq.dev/rubric/skill.md`. ## TypeScript SDK A typed client for the live endpoint lives in [`sdk/index.ts`](sdk/index.ts) — 10 tools (5 free, 5 paid). Free tools answer a bare call; paid tools run the x402 flow automatically (402 challenge → `pay` hook → replay with the `x-payment` header). No key lives in the file. ```ts import { RubricClient } from "./sdk/index.js"; const client = new RubricClient(); // endpoint defaults to https://mcp.evidiq.dev/rubric/mcp // free const caps = await client.callTool("rubric_capabilities", {}); // paid — settle the 402 challenge via the constructor's pay hook, or omit it // to receive a PaymentRequiredError carrying the full x402 v2 challenge const result = await client.callTool("some_paid_tool", { arg: "value" }); ``` The `pay` hook receives the decoded x402 v2 challenge (`{ x402Version, resource, accepts[] }` — payTo, asset, amount) and returns the value for the `x-payment` header, e.g. an EIP-3009 `transferWithAuthorization` settled via the official OKX SDK. Without a hook, paid calls throw `PaymentRequiredError` so the caller can settle however it wants. ## License MIT © 2026 EVIDIQ. See [LICENSE](./LICENSE).