Content
# InBrowserMcp - (Under Development)
InBrowserMcp is an experimental project aimed at bringing the capabilities of the Model Context Protocol (MCP) into browser internals through a Chrome extension.
It allows AI models or external applications to send commands (e.g., navigation, clicks, inputs, fetch content) to the browser via the MCP standard interface and actually perform these actions in the browser through the Chrome extension.
## Project Structure
The project consists of three main parts:
1. **`mcp-server`**: Backend service (Node.js/Express)
* Implements the MCP server, handling JSON-RPC requests from clients (e.g., AI models).
* Communicates with the Chrome extension (`extension`) via WebSocket, forwarding MCP instructions and receiving execution results.
* Provides the `/mcp` endpoint for MCP communication (POST/GET/DELETE).
* Offers the `/api/ai-command` endpoint to receive natural language instructions from the frontend UI, converting them into MCP instructions (simulated).
* Provides the `/api/cancel-command` endpoint to cancel ongoing browser operations.
2. **`extension`**: Chrome browser extension
* Acts as an agent on the browser side, connecting to `mcp-server` via WebSocket.
* Receives instructions from `mcp-server`.
* Utilizes Chrome Extension API (e.g., `chrome.tabs`, `chrome.windows`, `chrome.debugger`) to execute instructions within the browser.
* Returns execution results or errors to `mcp-server` via WebSocket.
3. **`frontend`**: Frontend user interface (React/Vite)
* Offers a simple web interface where users can input natural language instructions.
* Sends instructions to `mcp-server`'s `/api/ai-command` endpoint via HTTP POST requests.
* Receives MCP server status updates and operation results from the `/mcp` endpoint via Server-Sent Events (SSE) and updates the UI.
## Tech Stack
* **Backend (`mcp-server`)**: Node.js, Express, TypeScript, `@modelcontextprotocol/sdk`, `ws` (WebSocket)
* **Extension (`extension`)**: JavaScript, Chrome Extension API, WebSocket
* **Frontend (`frontend`)**: React, Vite, TypeScript, Tailwind CSS, Server-Sent Events (EventSource)
## Workflow
1. User inputs an instruction (e.g., "Open google.com") in the `frontend` UI.
2. `frontend` POSTs the instruction to `mcp-server`'s `/api/ai-command`.
3. `mcp-server` (simulated AI) converts the natural language instruction into an MCP tool call request (e.g., `navigate` tool).
4. `mcp-server` sends the instruction and parameters to `extension` via WebSocket (`sendBrowserAction`).
5. `extension` (in `background.js`) receives the instruction, calls the corresponding Chrome API to execute the action (e.g., `chrome.tabs.update(...)`).
6. `extension` sends the operation result (success or failure information) back to `mcp-server` via WebSocket (`action_response`).
7. `mcp-server` receives `action_response`, parses the result.
8. `mcp-server` sends a `message` or `error` event to `frontend` via the established SSE connection (`/mcp` GET).
9. `frontend` receives the SSE event, updates the UI state (e.g., from "Running" to "Success" or "Failure").
## Installation and Running
**Prerequisites:**
* Node.js (Recommended v18 or higher)
* npm or pnpm
* Google Chrome browser
**Steps:**
1. **Clone the repository:**
```bash
git clone <repository-url>
cd InBrowserMcp
```
2. **Install backend dependencies:**
```bash
cd mcp-server
npm install
# or use pnpm
# pnpm install
cd ..
```
3. **Install frontend dependencies:**
```bash
cd frontend
pnpm install # Recommend using pnpm, if using npm, run npm install
cd ..
```
4. **Load Chrome extension:**
* Open Chrome browser, enter `chrome://extensions/` in the address bar.
* Enable "Developer mode" at the top right.
* Click "Load unpacked".
* Select the `InBrowserMcp/extension` folder in the project.
* Ensure the extension is enabled.
5. **Start backend service:**
```bash
cd mcp-server
npm run build
npm start
```
The backend service runs at `http://localhost:8080` by default, and the WebSocket server listens at `ws://localhost:8081`.
6. **Start frontend development server:**
```bash
cd frontend
pnpm run dev
```
The frontend application runs at `http://localhost:5173` by default.
7. **Usage:**
* Open the browser and visit `http://localhost:5173`.
* Input instructions in the input box, e.g., "Open google.com" or "Navigate to bilibili.com".
* Observe the status changes and browser behavior.
## Notes
* The Chrome extension requires necessary permissions to perform certain operations (e.g., access tabs, windows, debugger), which are configured in `extension/manifest.json`.
* The WebSocket connection between the backend and extension is the core communication channel.
* The frontend communicates with the backend via SSE to get real-time updates.
* The current `/api/ai-command` is a simple simulation and needs to be replaced with actual AI model calls or more complex instruction parsing logic in practical applications.
## Contribution
Feel free to raise issues, report bugs, or submit merge requests.
## License
This project is licensed under the [MIT License](LICENSE).
