Content
# MCP-Diag: A Deterministic, Protocol-Driven Architecture for AI-Native Network Diagnostics
## Abstract
MCP-Diag represents a hybrid neuro-symbolic architecture built upon the Model Context Protocol (MCP). It addresses two fundamental challenges in the integration of Large Language Models (LLMs) with network operations (AIOps): the *stochastic grounding problem*, where LLMs fail to reliably parse unstructured CLI output, and the *security gap* inherent in granting autonomous agents shell access.
This repository contains the reference implementation of MCP-Diag, comprising a secure MCP Server capable of deterministic output serialization (via `jc`) and protocol-enforced Human-in-the-Loop (HITL) elicitation. It also includes an interactive MCP Client and a comprehensive benchmarking suite used to validate the system's performance, token efficiency, and reliability against baseline approaches.
## Repository Structure
The monorepo is organized into the following workspaces:
* **`packages/mcp-server`**: The core diagnostic server implementing the Model Context Protocol. It exposes tools (`dig`, `ping`, `traceroute`, `geoiplookup`) and handles the serialization of raw `stdout` into strict JSON schemas. It manages the dual-channel transport (Stdio for control, SSE for data streaming).
* **`packages/mcp-client`**: An interactive terminal-based client that orchestrates sessions between the user, the LLM (Google Gemini), and the MCP server. It manages session state and tool execution workflows.
* **`mcp_diag_experiments`**: The experimental framework used to conduct quantitative evaluations. It contains the logic for running comparative benchmarks ($N=500$ trials) against top global domains and generating performance plots.
## Prerequisites
* **Runtime**: [Bun](https://bun.sh) (v1.2 or higher).
* **Python Environment**: [uv](https://github.com/astral-sh/uv) (for reproducible plotting environments).
* **System Utilities**: The host machine must have standard network utilities installed (`iputils-ping`, `traceroute`, `dnsutils`/`bind9-host`).
* **API Credentials**: A valid Google Generative AI API key.
## Installation
1. **Clone the repository:**
```bash
git clone https://github.com/devansh-lodha/mcp-diag.git
cd mcp-diag
```
2. **Install dependencies:**
This project uses Bun workspaces. Installing dependencies at the root will bootstrap all packages.
```bash
bun install
```
3. **Build the packages:**
Compiles the TypeScript source code for both the client and server.
```bash
bun run build
```
## Configuration
The client requires an API key to communicate with the inference model.
1. Navigate to the client package:
```bash
cd packages/mcp-client
```
2. Create and Edit `.env` and populate the `GEMINI_API_KEY` variable.
## Interactive Usage
To run the full system in interactive mode, where the client spawns the server as a subprocess:
```bash
bun run start
```
This initiates the `ChatSession` orchestrator. The user may issue natural language queries (e.g., *"Trace the route to google.com and verify the latency of the last hop"*). The system will:
1. Plan the tool execution chain.
2. Request explicit user approval via the Elicitation protocol.
3. Execute the tools via the MCP Server.
4. Stream results back to the client.
5. Synthesize a final response.
## Reproduction of Experimental Results
This section details the steps required to reproduce the quantitative evaluation presented in the associated paper. The benchmark compares the MCP-Diag architecture against a baseline "ShellTool" approach across latency, memory footprint, and token consumption metrics.
### 1. Benchmark Execution
The benchmark runner executes a controlled trial against the top 500 global domains. It handles JIT warmup and path priming to ensure statistical validity.
Navigate to the experiments directory:
```bash
cd mcp_diag_experiments
```
Install experiment dependencies:
```bash
bun install
```
Configure the experiment environment:
Ensure a `.env` file exists in `mcp_diag_experiments/` containing a valid `GOOGLE_API_KEY`.
Run the benchmark suite:
```bash
# This will execute N=500 trials for both Baseline and MCP systems
# Results are saved to benchmark_results.csv
bun run start
```
### 2. Data Visualization
The visualization scripts use Python. We utilize `uv` to manage the Python environment and dependencies deterministically.
Ensure `uv` is installed:
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
Generate the performance plots:
```bash
# uv will automatically create the venv and install dependencies (pandas, matplotlib, seaborn)
uv run plot.py
```
This will generate the scientific 6-panel grid visualization at `mcp_diag_experiments/plots/mcp_final_grid.pdf`.
## License
MIT License. See the `LICENSE` file for full text.
