Content
> ⚠️ **This marketplace is deprecated.**
> All skills have been migrated to [jeredblu-marketplace](https://github.com/JeredBlu/jeredblu-marketplace), which includes these evaluators plus new tools.
> This repo will remain available but won't receive updates.
# Eval Marketplace
Comprehensive security evaluation tools for agent skills and MCP servers, powered by GitHub and Bright Data integrations.
## Overview
This marketplace provides two specialized evaluation skills that utlize MCP servers:
- **agent-skill-evaluator**: Security and safety evaluation for agent skills (.skill files)
- **mcp-evaluator**: Security and privacy evaluation for MCP servers
Both skills automatically assess security vulnerabilities, privacy risks, community feedback, and provide actionable recommendations with detailed risk scoring.
## Installation Options
### Option 1: Install via Plugin Marketplace for Claude Code (Recommended)
#### 1. Add Marketplace
```bash
/plugin marketplace add /path/to/eval-marketplace
```
Or from GitHub:
```bash
/plugin marketplace add github:jeredblu/eval-marketplace
```
#### 2. Install Plugin
```bash
/plugin install evaluator-tools@eval-marketplace
```
### Option 2: Download Individual Skills (Claude Code or Claude Desktop)
Download skills individually for manual installation:
#### Agent Skill Evaluator
1. Download: [agent-skill-evaluator.zip](./agent-skill-evaluator.zip)
2. Extract the zip file
3. Move contents to `~/.claude/skills/agent-skill-evaluator/`
4. Restart Claude Code
#### MCP Evaluator
1. Download: [mcp-evaluator.zip](./mcp-evaluator.zip)
2. Extract the zip file
3. Move contents to `~/.claude/skills/mcp-evaluator/`
4. Restart Claude Code
**Note**: These skills function best with recommended MCP servers, you'll need to manually configure the MCP servers (see Configuration section below).
### Claude Desktop Installation
For Claude Desktop users:
1. Download: [agent-skill-evaluator.zip](./agent-skill-evaluator.zip) or [mcp-evaluator.zip](./mcp-evaluator.zip)
2. Open Claude Desktop
3. Go to **Settings > Capabilities > Upload Skill**
4. Select the downloaded zip file
5. Repeat for the second skill if desired
## Configuration
The evaluator skills work best with two MCP servers. Both are optional but highly recommended for full functionality.
### Recommended MCP Servers
**GitHub MCP Server** (Recommended)
- Enables direct GitHub repository access for analyzing skills and MCP servers
- Installation: [@modelcontextprotocol/server-github](https://github.com/modelcontextprotocol/servers-archived/tree/main/src/github)
- Requires: GitHub Personal Access Token
**Bright Data MCP Server** (Recommended)
- Enables web scraping and Reddit access for community feedback analysis
- Installation: [@brightdata/mcp](https://github.com/brightdata/brightdata-mcp)
- Requires: Bright Data API token
- **Note**: Enable Pro Mode for Reddit scraping
Install and configure these MCP servers following their official installation instructions.
## Usage
### Agent Skill Evaluator
Evaluate the security of agent skills from various sources:
```
Evaluate this skill: https://github.com/username/skill-repo
```
```
Is this skill safe? https://example.com/my-skill.skill
```
```
Security assessment for this skill please: [attach .skill file]
```
The evaluator will:
- Download and extract the skill
- Analyze SKILL.md for prompt injections
- Review scripts for malicious code
- Search community feedback
- Generate comprehensive security report with risk scoring
### MCP Server Evaluator
Evaluate the security of MCP servers:
```
Evaluate this MCP server: https://github.com/username/mcp-server
```
```
Is this MCP safe to use? https://github.com/org/mcp-repo
```
The evaluator will:
- Analyze repository metadata and activity
- Review code for security vulnerabilities
- Search for alternatives and comparisons
- Gather community feedback (including Reddit with Pro Mode)
- Generate detailed assessment with recommendations
## Features
### Agent Skill Evaluator
- Prompt injection detection
- Malicious code pattern matching
- Hidden instruction scanning
- Data exfiltration detection
- Community validation
- Risk scoring (0-100 scale)
- Actionable recommendations
### MCP Server Evaluator
- Security vulnerability analysis
- Privacy risk assessment
- Code quality review
- Alternative server discovery
- Community feedback research (Reddit, forums, GitHub)
- Multi-dimensional scoring
- Usability assessment
## Graceful Degradation
Skills work without MCP servers but with reduced functionality:
| Scenario | Behavior |
|----------|----------|
| No GitHub MCP | Uses web scraping for repository access |
| No Bright Data | Uses built-in web search (limited) |
| No Pro Mode | No Reddit scraping, basic search only |
### Example: Without MCPs
```
User: "Evaluate this MCP: https://github.com/example/server"
Claude: Uses basic web scraping, can't access private repos,
limited Reddit data, slower analysis
```
### Example: With MCPs
```
User: "Evaluate this MCP: https://github.com/example/server"
Claude: Direct repo access, full code review, Reddit community
feedback, comprehensive security scan
```
## Requirements
- Claude Code or Claude Desktop
- GitHub Personal Access Token (recommended)
- Bright Data API token (recommended, for Reddit scraping)
