Content
# MCP Chat API Service
This project integrates the MCP tool into an HTTP API service, allowing interaction with large models through API requests and utilizing various tools. Currently, two API formats are supported: Simplified API and OpenAI-compatible API.
## Features
- Convert command-line chat interface to HTTP API service
- Support for using MCP toolset
- Maintain context for multiple sessions
- Automatic retry mechanism and error handling
- Support for cross-domain requests (CORS)
- **New**: Support for OpenAI-compatible API format
- **New**: Support for streaming responses
## Installation
1. Clone the repository
2. Install dependencies:
```bash
pip install -r requirements.txt
```
3. Create a `.env` file and set the following environment variables:
```
OPENAI_API_KEY=Your OpenAI API Key
OPENAI_BASE_URL=https://api.openai.com/v1
DEFAULT_MODEL=gpt-3.5-turbo
PORT=8000
HOST=0.0.0.0
```
4. Ensure the `servers_config.json` file is correctly configured for the required MCP server
## Usage
### Start the Server
```bash
python main.py
```
The server runs by default at `http://localhost:8000`
### API Endpoints
#### Simplified API
##### GET /
Returns a simple welcome message.
##### POST /chat
Send a chat message and get a reply.
Request body format:
```json
{
"message": "Your question or message",
"session_id": "Optional session ID"
}
```
If no `session_id` is provided, the server will create a new session.
Response format:
```json
{
"response": "Large model's reply",
"session_id": "Session ID for subsequent requests"
}
```
#### OpenAI-compatible API
##### GET /v1/models
Get a list of available models.
Response format:
```json
{
"object": "list",
"data": [
{
"id": "gpt-3.5-turbo",
"object": "model",
"created": 1677610602,
"owned_by": "organization-owner"
},
{
"id": "gpt-4",
"object": "model",
"created": 1677610602,
"owned_by": "organization-owner"
}
]
}
```
##### POST /v1/chat/completions
Send a chat message and get a reply, fully compatible with OpenAI API format. Supports normal and streaming responses.
Request body format:
```json
{
"model": "gpt-3.5-turbo",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello, introduce yourself."}
],
"temperature": 0.7,
"max_tokens": 4096,
"stream": false // Set to true to enable streaming response
}
```
**Normal response format:**
```json
{
"id": "chatcmpl-123abc456def",
"object": "chat.completion",
"created": 1677610602,
"model": "gpt-3.5-turbo",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! I'm an AI assistant..."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 30,
"completion_tokens": 100,
"total_tokens": 130
}
}
```
**Streaming response format:**
When using `stream=true` parameter, the server returns a series of SSE (Server-Sent Events) events:
```
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-3.5-turbo","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-3.5-turbo","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-3.5-turbo","choices":[{"index":0,"delta":{"content":"!"},"finish_reason":null}]}
... [More content blocks]
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-3.5-turbo","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
```
### Client Examples
Two client examples are provided:
1. `client.py` - Command-line client using Simplified API
2. `test_openai_client.py` - Test client using OpenAI-compatible API, supporting normal and streaming responses
Run client examples:
```bash
# Simplified API client
python client.py
# OpenAI-compatible API client
python test_openai_client.py
```
## Using OpenAI SDK
Since this service is compatible with OpenAI's API format, you can directly use the official OpenAI SDK or other third-party libraries to call this service. Just set the `base_url` to the address of this service:
### Normal Response Example
```python
from openai import OpenAI
# Specify base_url when creating the client
client = OpenAI(
api_key="Any string, not actually used",
base_url="http://localhost:8000/v1"
)
# Usage is the same as calling OpenAI API
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello, how's the weather today?"}
]
)
print(response.choices[0].message.content)
```
### Streaming Response Example
```python
from openai import OpenAI
# Specify base_url when creating the client
client = OpenAI(
api_key="Any string, not actually used",
base_url="http://localhost:8000/v1"
)
# Streaming response call
stream = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell a story about AI"}
],
stream=True # Enable streaming response
)
# Process response block by block
print("AI reply: ", end="")
for chunk in stream:
if chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="", flush=True)
print()
```
## Customization
- Modify `servers_config.json` to add or remove MCP servers
- Change models or other configurations in `.env` file
- Adjust timeout settings and retry strategies in `main.py`
## Precautions
- Limit CORS `allow_origins` in production environment
- Consider adding API authentication mechanism
- Implement persistent storage for sessions as needed
- Current token counting is estimated and may not be exactly consistent with OpenAI's calculation
- Streaming response mode does not support tool calls; if detected, it will switch to normal response
## Limitations of Streaming Response
When using streaming response, there are the following limitations:
1. No support for MCP tool calls - If the model returns content that is a tool call (JSON format), the system will automatically switch to non-streaming mode for processing
2. Tool execution results will not be returned in real-time streaming; they will wait for the tool execution to complete and then return all at once
3. Streaming response cannot be interrupted in the middle; it must wait for the complete response to finish
## Environment Requirements
- Python 3.7+
- Dependencies:
- httpx
- python-dotenv
- mcp-sdk
- fastapi
- uvicorn
- pydantic
- requests
- sseclient-py
## Configuration
### 1. Environment Variable Configuration
Create a `.env` file and configure the following environment variables:
```env
# LLM API configuration
OPENAI_API_KEY=Your API Key
OPENAI_BASE_URL=https://api.openai.com/v1 # Optional, defaults to OpenAI official address
DEFAULT_MODEL=gpt-3.5-turbo # Optional, defaults to gpt-3.5-turbo
PORT=8000
HOST=0.0.0.0
# JianShu configuration (if needed)
JIANSHU_USER_ID=Your user ID
JIANSHU_COOKIES=Your cookie string
```
### 2. Server Configuration
Edit the `servers_config.json` file and configure the servers to connect:
```json
{
"mcpServers": {
"sqlite": {
"command": "sqlite-server",
"args": ["database.db"],
"env": {
"DB_PATH": "path/to/database.db"
}
},
"jianshu": {
"type": "sse",
"url": "http://your-sse-server/sse"
}
}
}
```
Two types of servers are supported:
- Standard input/output server: Specify `command` and `args`
- SSE server: Specify `type: "sse"` and `url`
## Usage
1. Ensure configuration files are correctly set
2. Run the chatbot:
```bash
python main.py
```
3. Start a conversation:
- Input questions or instructions
- The robot will automatically choose the appropriate tool to process the request
- Input "quit" or "exit" to exit the program
## Available Tools
### SQLite Tool
- `read_query`: Execute SELECT query
- `write_query`: Execute INSERT/UPDATE/DELETE query
- `create_table`: Create a new table
- `list_tables`: List all tables
- `describe_table`: Get table structure
- `append_insight`: Add business insights
## Log Level
Default INFO level logging is used; for debugging, you can modify the log level in `main.py`:
```python
logging.basicConfig(
level=logging.DEBUG, # Change to DEBUG for more detailed logs
format="%(asctime)s - %(levelname)s - %(message)s"
)
```
## Error Handling
- Tool execution failures will automatically retry (default 2 times)
- Empty responses will prompt re-questioning
- Server connection failures will record errors and exit
- Resources will be automatically cleaned up when the program exits
## Development Instructions
### Add New Tools
1. Implement tool functionality on the server side
2. Add server configuration in `servers_config.json`
3. Tools will be automatically discovered and integrated into the chatbot
### Custom Response Processing
You can customize response processing logic by modifying the `process_llm_response` method.
### Session Management
The `ChatSession` class manages the entire conversation process, including:
- Initializing server connections
- Processing user input
- Calling LLM to get responses
- Executing tool calls
- Cleaning up resources
## Precautions
1. Ensure API key security and do not submit to version control systems
2. SSE servers require long connections
3. Extensive debug logs may affect performance
4. Regularly check and update dependency package versions
## Frequently Asked Questions
1. If encountering connection errors, please check:
- Network connection
- API key is correct
- Server address is accessible
2. If tool execution fails, please check:
- Tool parameters are correct
- Server status
- Error information in logs
3. If receiving empty responses, you can:
- Rephrase the question
- Check API quota
- View detailed logs
## Contribution Guide
Feel free to submit issues and improvement suggestions! Please ensure:
1. Provide clear issue descriptions
2. Include necessary log information
3. Explain reproduction steps
## License
MIT License
