Content
# OpenDiscourse MCP
Model Context Protocol (MCP) servers for accessing bulk data from govinfo.gov and congress.gov APIs with comprehensive AI-powered code review and data ingestion capabilities.
## Overview
This project provides MCP servers that enable AI assistants to access comprehensive legislative and governmental data from:
- **GovInfo API**: Congressional documents, Federal Register, Code of Federal Regulations, and more
- **Congress.gov API**: Bills, members, committees, nominations, and legislative activities
- **Bulk Data Ingestion**: Automated downloading and processing of govinfo.gov bulk data
- **AI Code Review**: Multi-agent AI crews for comprehensive code analysis
## Features
### GovInfo Server Tools
- `govinfo_list_collections` - List collections updated since a specific date
- `govinfo_get_package` - Get detailed package information
- `govinfo_list_granules` - List granules within a package
- `govinfo_get_granule` - Get detailed granule information
- `govinfo_download_content` - Download content (PDF, XML, HTML)
### Congress.gov Server Tools
- `congress_get_bill` - Get detailed bill information
- `congress_search_bills` - Search for bills by text query
- `congress_get_bill_actions` - Get legislative actions for a bill
- `congress_get_bill_text` - Get full text of a bill
- `congress_search_members` - Search for members of Congress
- `congress_get_member` - Get detailed member information
- `congress_get_committee_meetings` - Get committee meetings
- `congress_get_nominations` - Get presidential nominations
### Bulk Data Ingestion
- **Comprehensive Download**: Automated ingestion of all govinfo.gov bulk data
- **Congress Coverage**: 113th to 123rd Congress (2013-2024)
- **Document Types**: BILLS, BILLSTATUS, PLAW, STATUTE, FR, CREC
- **Parallel Processing**: 50 workers for high-performance downloads
- **XML Validation**: Schema-based validation for data integrity
- **Skip-on-Existing**: Resumable downloads with intelligent file skipping
- **Progress Monitoring**: Real-time progress tracking and comprehensive reporting
### AI Code Review Suite
- **Security Review**: Vulnerability assessment and security analysis
- **Performance Review**: Bottleneck identification and optimization suggestions
- **Quality Review**: Code style, testing, architecture, and accessibility analysis
- **Documentation Review**: Documentation quality and completeness assessment
- **Multi-Agent**: Specialized AI agents for different review aspects
- **Observability**: Langfuse integration for monitoring and tracing
## Installation
### Prerequisites
1. **Node.js and npm** (for MCP server)
2. **Python 3.10+** (for setup scripts)
3. **PostgreSQL** (for database storage)
4. **Bitwarden CLI** (for API key management)
### Quick Start
```bash
# Clone the repository
git clone https://github.com/cbwinslow/opendiscourse_mcp.git
cd opendiscourse_mcp
# Install Node.js dependencies
npm install
# Install Bitwarden CLI
npm install -g @bitwarden/cli
# Set up database
./scripts/setup_database.sh
# Set up API keys with Bitwarden
python3 scripts/setup_bitwarden.py
# Start MCP server
npm start
```
### Detailed Setup
#### 1. Python Environment
```bash
# Verify Python version (requires 3.10+)
python --version
# Using uv (recommended)
curl -LsSf https://astral.sh/uv/install.sh | sh
# Or using pip
python3.10 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```
#### 2. Database Setup
```bash
# Automated database setup
./scripts/setup_database.sh
# Manual setup (if script fails)
# Start PostgreSQL
brew services start postgresql # macOS
sudo systemctl start postgresql # Linux
# Create database and user
sudo -u postgres createdb opendiscourse
sudo -u postgres createuser -s opendiscourse
sudo -u postgres psql -c "ALTER USER opendiscourse PASSWORD 'opendiscourse123';"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE opendiscourse TO opendiscourse;"
```
#### 3. Bitwarden Setup
```bash
# Install Bitwarden CLI
npm install -g @bitwarden/cli
# Test Bitwarden integration
python3 scripts/test_bitwarden.py
# Add API keys to Bitwarden:
# - GovInfo API Key
# - Congress.gov API Key
```
#### 4. API Key Configuration
```bash
# Set environment variables (optional)
export BITWARDEN_EMAIL="your-email@example.com"
export BITWARDEN_PASSWORD="your-master-password"
# Run automated setup
python3 scripts/setup_bitwarden.py
```
#### 5. MCP Server Deployment
```bash
# Install dependencies
uv sync
# Start server
npm start
# Development mode
npm run dev
```
**Bitwarden Item Setup:**
Create these items in your Bitwarden vault:
- **GovInfo API Key**
- Name: `GovInfo API Key`
- Custom Field: `api_key` with your GovInfo API key
- **Congress.gov API Key**
- Name: `Congress.gov API Key`
- Custom Field: `api_key` with your Congress.gov API key
## Configuration
### Required API Keys
1. **GovInfo API Key**
- Sign up at: <https://api.data.gov/signup>
- Add to `.env`: `GOVINFO_API_KEY=your_key_here`
2. **Congress.gov API Key**
- Sign up at: <https://api.congress.gov/sign-up/>
- Add to `.env`: `CONGRESS_API_KEY=your_key_here`
### Optional Configuration
- `MCP_LOG_LEVEL` - Logging level (default: info)
- `GOVINFO_RATE_LIMIT` - GovInfo rate limit requests/hour (default: 1000)
- `CONGRESS_RATE_LIMIT` - Congress.gov rate limit requests/hour (default: 4000)
## Usage
### Bulk Data Ingestion
The project includes comprehensive bulk data ingestion capabilities for govinfo.gov:
```bash
# Run comprehensive ingestion for all available data
python3 scripts/run_comprehensive_ingestion.py
# Run specific congress and document type
python3 -m scripts.ingest_all_govinfo --congress 118 --doc-types BILLS --workers 50
# Run with validation enabled
python3 -m scripts.ingest_all_govinfo --congress 118 --doc-types STATUTE --validate-xml --workers 50
```
**Ingestion Features:**
- **Congress Coverage**: 113th-123rd Congress (2013-2024)
- **Document Types**: BILLS, BILLSTATUS, PLAW, STATUTE, FR, CREC
- **Parallel Downloads**: 50 concurrent workers
- **XML Validation**: Schema-based validation with XSD files
- **Resume Capability**: Skip-on-existing functionality
- **Progress Tracking**: Real-time monitoring and JSON results
**Configuration:**
```python
# Use enums for type-safe configuration
from scripts.ingestion.enums import DocumentType, CongressSession
# Available document types
DocumentType.BILLS.value # "BILLS"
DocumentType.STATUTE.value # "STATUTE"
# Available congress sessions
CongressSession.get_available_sessions() # [113, 114, ..., 123]
```
### AI Code Review
The project includes specialized AI crews for comprehensive code review:
```bash
# Run all review types (via GitHub Actions)
# Or manually trigger specific reviews:
# Security review
python3 scripts/security_review_crew.py
# Performance review
python3 scripts/performance_review_crew.py
# Quality review (style, testing, architecture, dependencies, accessibility)
python3 scripts/quality_review_crew.py
# Documentation review
python3 scripts/documentation_review_crew.py
# Original comprehensive review
python3 scripts/code_review_crew.py
```
**AI Review Features:**
- **Security Analysis**: Vulnerability assessment, security best practices
- **Performance Optimization**: Bottleneck identification, efficiency improvements
- **Quality Assessment**: Code style, testing coverage, architecture review
- **Documentation Review**: API docs, README, inline comments
- **Multi-Agent**: Specialized AI agents for different review aspects
- **Observability**: Langfuse integration for monitoring and tracing
**GitHub Actions Integration:**
- Automatic PR reviews
- Configurable review types (security, performance, quality, documentation)
- Python 3.10 enforcement
- Comprehensive reporting
### As MCP Server
```bash
# Start MCP server
npm start
# Or in development mode with auto-restart
npm run dev
```
### Integration with Claude Desktop
Add to your Claude Desktop configuration (`~/Library/Application Support/Claude/claude_desktop_config.json`):
```json
{
"mcpServers": {
"opendiscourse": {
"command": "node",
"args": ["/path/to/opendiscourse_mcp/src/index.js"],
"env": {
"GOVINFO_API_KEY": "your_govinfo_key",
"CONGRESS_API_KEY": "your_congress_key"
}
}
}
}
```
### API Key Management
The project includes automated API key management through Bitwarden:
```bash
# Test Bitwarden integration
python3 scripts/test_bitwarden.py
# Set up API keys from Bitwarden
python3 scripts/setup_bitwarden.py
```
**Features:**
- 🔍 Automatic API key discovery from Bitwarden vault
- 🔓 Secure vault unlocking
- 📝 Environment file updates
- 🗄️ Database storage with encryption
- 🔐 Security masking in console output
### Integration with Claude Desktop
Add to your Claude Desktop configuration (`~/Library/Application Support/Claude/claude_desktop_config.json`):
```json
{
"mcpServers": {
"opendiscourse": {
"command": "node",
"args": ["/path/to/opendiscourse_mcp/src/index.js"],
"env": {
"GOVINFO_API_KEY": "your_govinfo_key",
"CONGRESS_API_KEY": "your_congress_key"
}
}
}
}
```
## API Coverage
### GovInfo Collections
- **BILLS** - Congressional bills (all versions)
- **CREC** - Congressional Record
- **FR** - Federal Register
- **CFR** - Code of Federal Regulations
- **USCOURTS** - U.S. Courts opinions
- **CHRG** - Committee reports and prints
### Congress.gov Data
- Bills and amendments (93rd Congress to present)
- Member information and voting records
- Committee data and meetings
- Presidential nominations
- Roll call votes
- Legislative text and summaries
## Rate Limits
- **GovInfo API**: Follows api.data.gov limits (configurable, default 1000/hour)
- **Congress.gov API**: 5,000 requests/hour (configurable, default 4000/hour)
Both clients implement automatic rate limiting with delays between requests.
## Development
```bash
# Run tests
npm test
# Lint code
npm run lint
# Full build process
npm run build
```
## Documentation
Comprehensive documentation is available in the `/docs` folder:
- `docs/README.md` - Complete reference guide
- `docs/govinfo-api-summary.md` - GovInfo API quick reference
- `docs/congress-api-summary.md` - Congress.gov API quick reference
- Downloaded official API repositories and samples
## Contributing
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Run `npm run build`
6. Submit a pull request
## License
MIT License - see LICENSE file for details.
## Support
- Issues: <https://github.com/cbwinslow/opendiscourse_mcp/issues>
- GovInfo API: <https://github.com/usgpo/api/issues>
- Congress.gov API: <https://github.com/LibraryOfCongress/api.congress.gov/issues>
