Content
# DSPy IT Help Assistant
A comprehensive AI assistant project demonstrating how to build intelligent agents using DSPy (Declarative Self-improving Language Programs) with Model Context Protocol (MCP) servers. This project showcases modern AI assistant architecture with semantic search, knowledge management, and modular tool integration.
## Features
### Core AI Assistant (GAIa)
- **DSPy-Powered**: Uses DSPy's ReAct patterns for autonomous reasoning and tool selection
- **MCP Integration**: Connects to multiple MCP servers for different capabilities
- **Semantic Search**: ChromaDB-powered vector search through knowledge base
- **Conversation Memory**: Maintains context across conversations
- **Optimized Performance**: Uses DSPy's compilation and optimization features
### Knowledge Management System
- **Document Processing**: Converts HTML help documentation to Wiki.js compatible Markdown
- **Vector Storage**: ChromaDB integration for semantic search and retrieval
- **Smart Tagging**: Automatically tags content for better organization
- **Comprehensive Coverage**: 310+ IT help documents covering common support topics
### MCP Server Architecture
- **Knowledge Server**: Manages storage and retrieval of knowledge base content
- **Search Server**: Provides semantic search capabilities with ChromaDB integration
- **Tools Server**: General utilities, task management, and calculations
- **Web Server**: Web search and content fetching capabilities
**Note**: The tools in this project are dummy implementations designed for demonstration. Users are encouraged to replace them with their own custom tools and business logic.
## Architecture
The GAIa assistant is implemented in remarkably few lines of code (under 400 lines), demonstrating the power of DSPy's modular approach:
```
GAIa Assistant (gaia_assistant.py)
├── DSPy ReAct Module (autonomous tool selection)
├── MCP Server 1: Knowledge Base
├── MCP Server 2: Semantic Search
├── MCP Server 3: General Tools (dummy implementations)
└── MCP Server 4: Web Tools (dummy implementations)
```
## Why DSPy Modulation is Perfect
DSPy's modular architecture makes this project an ideal starting point because:
- **Minimal Code**: The entire assistant logic fits in a single, readable file
- **Declarative Approach**: Focus on what the agent should do, not how
- **Self-Improving**: Built-in optimization and compilation capabilities
- **Tool Agnostic**: Easy to swap out dummy tools for real implementations
- **Context Management**: Simple conversation memory and state handling
- **Extensible**: Add new capabilities by simply adding new MCP servers
## Quick Start
### Prerequisites
- Python 3.8+
- OpenAI API key (for LLM access)
### Installation
1. **Clone the repository**
```bash
git clone https://github.com/yourusername/dspy-ithelp.git
cd dspy-ithelp
```
2. **Install dependencies**
```bash
pip install -r requirements.txt
```
3. **Set up environment variables**
```bash
# Create .env file
echo "OPENAI_API_KEY=your_api_key_here" > .env
```
4. **Run the assistant**
```bash
python gaia_assistant.py
```
## Usage Examples
### Basic Queries
```
You: How do I set up MFA for my account?
GAIa: I'll search for MFA setup information in our knowledge base...
[Returns detailed multi-factor authentication setup instructions]
You: What's the VPN configuration for Android?
GAIa: Let me find the VPN setup guide for Android devices...
[Provides step-by-step VPN configuration instructions]
```
### Complex Multi-step Tasks
```
You: Search for information about password policies and create a task to review them
GAIa: I'll search for password policy information and create a review task...
[Searches knowledge base, provides policy info, creates task]
```
## Developing Your Custom Agent
This project serves as an excellent starting point for building domain-specific agents:
### 1. Replace Dummy Tools
The current tools are basic implementations. Replace them with:
- Your database connections
- External API integrations
- Custom business logic
- Domain-specific calculations
### 2. Add Context
Enhance the agent's capabilities by:
- Loading domain-specific knowledge into ChromaDB
- Adding conversation persistence
- Implementing user-specific memory
- Creating specialized MCP servers for your use case
### 3. Customize DSPy Signatures
Modify the ReAct signatures for your domain:
```python
# Example: Customer service agent
signature = "customer_query, user_history -> helpful_response, next_actions"
react_agent = dspy.ReAct(signature, tools=your_tools, max_iters=10)
```
### 4. Extend MCP Servers
Add new servers for your specific needs:
- File system operations
- Database connectivity
- Email integration
- Calendar management
- Custom APIs
## Project Structure
```
dspy-ithelp/
├── gaia_assistant.py # Main DSPy AI assistant (< 400 lines)
├── enhanced_gaia_usage.py # Demo script showing compilation benefits
├── mcp_servers/ # MCP server implementations
│ ├── knowledge_server.py # Knowledge base management
│ ├── search_server.py # Semantic search with ChromaDB
│ ├── tools_server.py # General tools (dummy implementations)
│ └── web_server.py # Web search (dummy implementations)
├── storage/ # Knowledge base and conversion tools
│ ├── html_to_markdown_converter.py # HTML to Markdown conversion
│ ├── vector_store.py # ChromaDB integration
│ ├── conversation_store.py # Conversation persistence
│ └── markdown_docs/ # Processed help documentation (310+ files)
├── chroma_db_markdown/ # ChromaDB vector database
├── logs/ # Detailed logging (timestamped sessions)
└── LOGBOOK.md # Development progress tracking
```
## Available Tools (Dummy Implementations)
### Knowledge Management
- `store_knowledge`: Add information to knowledge base
- `retrieve_knowledge`: Get information by topic or tags
- `search_knowledge`: Semantic search across all content
### Search & Retrieval
- `semantic_search`: Vector-based similarity search
- `get_search_stats`: Database statistics and health
### General Tools (Replace with your implementations)
- `calculate`: Safe mathematical expression evaluation
- `create_task`: Task management and scheduling
- `get_current_time`: Date and time information
### Web Tools (Replace with your implementations)
- `web_search`: Internet search capabilities
- `fetch_url`: Retrieve web content
## DSPy Integration Benefits
This project showcases advanced DSPy features:
- **ReAct Patterns**: Autonomous reasoning and tool selection
- **Compilation & Optimization**: Uses MIPROv2 optimizer for improved performance
- **Custom Signatures**: Tailored for IT help scenarios
- **Tool Integration**: Seamless MCP server tool usage
- **Context Management**: Maintains conversation state
### Performance Optimization
```python
# The assistant can be compiled for better performance
gaia = GAIa(mcp_servers)
await gaia.initialize_tools()
gaia.compile_and_optimize(enable_optimization=True)
```
## Security & Privacy
- **Environment Variables**: API keys stored securely in .env files
- **No Hardcoded Secrets**: All sensitive data externalized
- **Git Ignored**: Sensitive files excluded from version control
- **Local Processing**: Knowledge base processed and stored locally
## Requirements
See `requirements.txt` for full dependency list. Key dependencies:
- `dspy-ai>=2.5`: DSPy framework
- `mcp>=0.1.0`: Model Context Protocol
- `chromadb>=0.4.0`: Vector database
- `beautifulsoup4>=4.12.0`: HTML parsing
- `openai>=1.0.0`: LLM integration
## Use Cases
This project is ideal for:
- **Learning DSPy**: Understand modern AI agent architecture
- **Prototyping**: Quick start for domain-specific agents
- **IT Help Desks**: Automated first-line support (with real tool implementations)
- **Knowledge Management**: Organize and search documentation
- **Enterprise**: Scale to organization-specific knowledge bases
## Getting Started with Your Own Agent
1. **Fork this repository**
2. **Replace dummy tools** in `mcp_servers/tools_server.py` and `mcp_servers/web_server.py`
3. **Load your knowledge base** into ChromaDB via `storage/vector_store.py`
4. **Customize DSPy signatures** in `gaia_assistant.py` for your domain
5. **Add domain-specific MCP servers** as needed
6. **Test and optimize** using DSPy's compilation features
The modular architecture and minimal codebase make this an excellent foundation for building sophisticated, domain-specific AI agents.
## License
This project is open source. Please see the LICENSE file for details.
## Acknowledgments
- DSPy team for the powerful framework
- Model Context Protocol for standardized tool integration
- ChromaDB for vector search capabilities
- OpenAI for language model access
---
**Built with DSPy, MCP, and modern AI architecture patterns**
