webmcp-sdk

# OpenTiny NEXT-SDKs: Built-in WebMCP & Polyfill + WebSkills + WebAgent <p align="center"> <a href="https://opentiny.design" target="_blank" rel="noopener noreferrer"> <img alt="OpenTiny Logo" src="logo.svg" height="100" style="max-width:100%;"> </a> </p> English | [简体中文](README.zh-CN.md) [](https://deepwiki.com/opentiny/webmcp-sdk) <p align="center"> <strong>A front-end intelligent application development and browser automation toolkit. It turns existing apps intelligent via WebMCP + WebSkills, and provides webmcp-cli to perceive and control any webpage with zero refactoring, giving them out-of-the-box AI-Native capabilities.</strong> </p> <p align="center"> <a href="https://docs.opentiny.design/next-sdk/">📖 Docs</a> | <a href="#-quick-start">🚀 Quick Start</a> | <a href="#-webmcp--polyfill">🌐 WebMCP & Polyfill</a> | <a href="#-scenarios">💡 Scenarios</a> </p> > **Next-Gen AI Protocol**: OpenTiny NEXT-SDKs is built on the **WebMCP (Model Context Protocol for Web)**. It is fully compatible with the native `document.modelContext` API (currently in experimental stage in browsers like Chrome), allowing your web apps to be controlled by AI via a standardized protocol. > [!TIP] > **✨ Command-line Automation & AI Skills**: > We now offer **`webmcp-cli`** (browser control & polyfill auto-injection CLI) and **`webmcp-skill`** (standard instructions and sub-skills like Excalidraw drawing for AI agents). Together, they enable AI agents to perform complex, fine-grained tasks and "remote-drive" any webpage out of the box. --- **OpenTiny NEXT-SDKs** is a front-end intelligent application development and browser automation toolkit. Beyond enabling the "WebMCP + WebSkills" model to expose page operations as standardized tools in just a few lines of code, it features **`webmcp-cli`** (browser perception and control) and **`webmcp-skill`** (agent skills library). Combined with our **Polyfill**, not only can you build future-proof AI-Native apps in today's browsers, but you can also let AI agents automatically perceive and control any webpage with zero source code modifications. ## 📑 Table of Contents - [✨ Main Features](#-main-features) - [🌐 WebMCP & Polyfill](#-webmcp--polyfill) - [🚀 Quick Start](#-quick-start) - [📦 Core Packages](#-core-packages) - [💻 WebMCP CLI & Agent Skills](#-webmcp-cli--agent-skills) - [💡 Core Concepts](#-core-concepts) - [📖 Scenarios](#-scenarios) - [🛠️ Contributing](#️-contributing) - [📄 License](#-license) ## ✨ Main Features - 🔌 **Standard WebMCP Implementation**: Fully implements the browser version of the MCP protocol, making front-ends "AI-Callable" via a unified standard. - 📡 **Remote AI Control**: Connect seamlessly to **WebAgent services**, allowing AI to easily and stably call your front-end tools remotely. - 🛠️ **Built-in Polyfill Support**: Provides `document.modelContext` / `document.modelContext` patches for current browsers, ensuring today's code works seamlessly and transitions to native support in the future. - 🎯 **Zero-Refactor Intelligence**: Expose business logic and UI operations as tools without changing your app's core architecture. - 🧩 **WebSkills Abstraction**: Organizes tools into "Business Skills" for progressive disclosure. - 🤖 **AI Chat Components**: Provides out-of-the-box `@opentiny/next-remoter` for an instant AI remote controller. ## 🌐 WebMCP & Polyfill ### What is WebMCP? WebMCP is an extension of the Model Context Protocol specifically for web browsers. It defines how a web page provides "Tools" and "Resources" to AI agents. In the near future, browsers will provide a native `document.modelContext` object to manage these capabilities. ### Why Polyfill? Since native APIs are still experimental, **OpenTiny NEXT-SDKs provides a powerful Polyfill**. By calling `initializeBuiltinWebMCP()`, the SDK will: 1. **Injects `document.modelContext`**: Provides a standard-compliant API for tool registration (also aliases `document.modelContext` for backward compatibility). 2. **Automated Routing & Bridging**: Automatically handles message synchronization and tool invocation across different page paths and iframes. This means you can write standard WebMCP code today, and it will automatically switch to the native engine when the browser supports it. ### 📡 Cross-Page & Remote Control One of the core highlights of OpenTiny NEXT-SDKs is its support for "cross-environment controllability" of pages. With the help of the built-in WebMCP Polyfill and `@opentiny/next-remoter`, you can achieve: - **Automated Bridging**: The underlying layer automatically handles message synchronization and tool invocations across different page paths and iframes. - **Persistent Session ID**: Establish long-term connections with cloud AI orchestrators. - **Cross-time Collaboration**: Even if the user isn't actively interacting, AI can remotely trigger page tools based on instructions sent by the business system. - **High-level Business Automation**: Converts front-end DOM operations and business component APIs into execution units in automated flows, achieving app-level "remote driving". #### Connection Example Through `WebMcpClient`, we can remotely call and control the built-in WebMCP tools of the browser via a cloud-based web-agent: ```typescript import { WebMcpClient } from '@opentiny/next-sdk' const client = new WebMcpClient() // Connect to the WebAgent service const { sessionId } = await client.connect({ agent: true, builtin: true, // Enable built-in WebMCP proxy url: 'https://agent.opentiny.design/api/v1/webmcp-trial/mcp' }) console.log('Connected! Session ID:', sessionId) // Now your app can be controlled remotely via this sessionId ``` > [!TIP] > The URL above is a public test server provided by OpenTiny for testing purposes. For production use, you should deploy your own WebAgent instance. > Source code: [https://github.com/opentiny/web-agent](https://github.com/opentiny/web-agent) ## 🚀 Quick Start Turn your front-end application into an AI-capable one in just a few lines of code. ### Step 1: Install Dependencies ```bash npm install @opentiny/next-sdk ``` ### Step 2: Initialize WebMCP Polyfill (Recommended) Add this to your application's entry point (e.g., `main.ts` or `app.js`): ```typescript import { initializeBuiltinWebMCP } from '@opentiny/next-sdk' // Initialize Polyfill and Bridging Mechanism initializeBuiltinWebMCP() ``` Now, you can use the standard `document.modelContext` anywhere in your app to register tools: ```typescript // Register a tool that can be called by AI const abortController = new AbortController() document.modelContext.registerTool({ name: 'get_user_info', description: 'Get current user information', inputSchema: { type: 'object', properties: { userId: { type: 'string' } } }, execute: async (args) => { // Write your business logic here return { content: [{ type: 'text', text: `Info for user ${args.userId}...` }] } } }, { signal: abortController.signal }) // To unregister the tool when it's no longer needed, simply call abort() // abortController.abort() ``` ✅ **Done!** Your app is now an MCP Server. You can connect it to any MCP-compatible client or use our [TinyRemoter](#-cross-page--remote-control) to chat with the app directly. ## 📦 Core Packages ### @opentiny/next-sdk (Current Package) The core SDK package, providing: - **Built-in WebMCP Polyfill**: Injects `document.modelContext` and sets up the bridging mechanism for seamless AI-to-page communication. - **WebMcpServer**: Managed MCP server for full control over lifecycle and transport layers. - **WebMcpClient**: MCP client for connecting to WebAgent or other remote services. - **WebAgent**: High-level intelligent agent orchestration logic. - **Transport Layer**: Support for MessageChannel, SSE, HTTP, and Chrome Extension messaging. ### @opentiny/next-remoter Vue3 AI chat component based on TinyRobot, providing: - Integrated AI assistant UI. - MCP Plugin marketplace. - Dynamic WebSkills discovery and execution. ### @opentiny/webmcp-cli A CLI tool based on `puppeteer-core` used to control Chrome browsers and expose WebMCP interfaces: - **Browser Takeover**: Automatically starts or connects to a local Chrome with debugging ports via Chrome DevTools Protocol (CDP). - **Auto-Injection**: When browser tabs open, it detects and automatically injects the WebMCP polyfill and page manipulation tools (`page-agent-tool`). - **Unified Tool Protocol**: Maps page operations to standard MCP interfaces, allowing AI agents to call them directly. ### webmcp-skill Guidelines and domain-specific skills for third-party AI agents interacting with web pages: - **Prompt Optimization**: Built-in comprehensive system prompts and usage specifications to assist external LLM agents in accurately using the CLI to control pages. - **Domain-Specific Skills**: Provides sub-skills like Excalidraw canvas commands (`excalidraw_execute_command`) for high-difficulty granular web operations. --- ## 💻 WebMCP CLI & Agent Skills With the WebMCP CLI, you can treat the browser as an MCP server, directly exposing page manipulation interfaces to AI Agents. Running Puppeteer in the background to drive a real Chrome browser, it maps traditional page interactions (click, input, scroll, etc.) into standard MCP Tool calls. ### 🚀 Getting Started #### 1. Installation You can install it globally via NPM: ```bash npm install -g @opentiny/webmcp-cli ``` Or for local debugging within the current Monorepo: ```bash cd packages/webmcp-cli pnpm build npm install -g . ``` #### 2. Open a Webpage Navigate to a specific URL in Chrome and prepare the environment: ```bash webmcp-cli tabs open https://excalidraw.com ``` #### 3. Get Browser State View the current page state, including URL, title, open tabs, and **the list of automatically injected and native MCP tools**. It also returns an indexed DOM tree: ```bash webmcp-cli state ``` The output will include elements like `[18]<button>Search</button>` with their index and the corresponding MCP interface names. #### 4. Execute an MCP Tool Use JSON-formatted arguments to directly execute registered MCP tools on the page: ```bash # Click a button with DOM index 18 webmcp-cli run page-agent-tool '{"action": "click", "index": 18}' # Fill text into an input field with DOM index 13 webmcp-cli run page-agent-tool '{"action": "fill", "index": 13, "text": "Model Context Protocol"}' ``` ### 🧠 Agent Skill Guidelines In the `packages/webmcp-skill` directory, we have defined a set of standard **Agent Skills**. When LLMs (like Claude or Gemini) connect to a webpage as an agent, they read `SKILL.md` to learn how to properly issue CLI commands. For complex pages, we also pair them with domain-specific sub-skills: - **Excalidraw Canvas Tools (`domains/excalidraw.md`)**: Instructs the Agent to use `excalidraw_execute_command` to draw shapes and diagrams when the URL contains `excalidraw.com`. - **Baidu Search Tool**: Guides the Agent to automatically issue searches and pull page results. This gives NEXT-SDKs a complete ecosystem enabling external AI to autonomously "remote drive" any webpage. --- ## 💡 Core Concepts ### WebMCP Bridge Architecture Unlike traditional backend MCP, WebMCP focuses on the **Browser Context**. ```text ┌─────────────────────────────────────────────────────────────┐ │ Web Browser │ │ ┌──────────────────┐ ┌───────────────────┐ │ │ │ Front-end App │◄── Bridging ──►│ AI Assistant │ │ │ │ (WebMCP Server) │ Mechanism │ (MCP Client) │ │ │ └──────────────────┘ └───────────────────┘ │ │ ▲ │ │ └───────────┼──────────────────────────────────┼──────────────┘ │ (Standard Protocol) │ └──────────────────────────────────┘ ``` 1. **Register Tools**: Use `document.modelContext.registerTool` to declare your app's capabilities, with support for automatic unregistration via `AbortSignal`. 2. **Bridge Synchronization**: Our bridging mechanism automatically routes AI requests to the correct page or iframe, even if the user has switched pages. 3. **Direct Execution**: Tools run directly in the page environment, allowing access to the DOM, component state, and local APIs. ### Underlying Transport and Engine Layer Although you can simply use `document.modelContext.registerTool` for daily development, NEXT-SDKs retains the full core MCP protocol implementation internally for advanced scheduling: - **WebMcpServer / WebMcpClient**: Responsible for handling underlying JSON-RPC messages, Schema validation, and channel handshakes. It is recommended to only understand these when expanding infrastructure; business apps should avoid calling them directly. ## 📖 Scenarios - **🤝 Smart Customer Service**: Quickly build an AI customer service system that supports tool calls. - **📚 Doc Assistant**: Add intelligent Q&A functionality to documentation websites. - **🛠️ Dev Tools**: Build developer auxiliary tools that support code generation and analysis. - **🌐 Browser Extensions**: Develop browser plugins with AI capabilities. - **🏢 Enterprise Apps**: Add intelligent capabilities to enterprise applications. - **📊 Data Analysis**: Build intelligent data analysis and visualization applications. - **✍️ Content Creation**: Develop AI-assisted content creation tools. ## 🛠️ Contributing We welcome all forms of contribution! Whether it's reporting bugs, suggesting new features, improving documentation, or submitting code, we appreciate it. ### Prerequisites Before you start developing, please make sure your environment meets the following requirements: - **Node.js** >= 18.0.0 - **pnpm** >= 8.0.0 - **Git** Latest version ### Get the Code ```bash # Clone the repository git clone https://github.com/opentiny/next-sdk.git cd next-sdk # Install dependencies pnpm install ``` ### Project Structure ```text next-sdk/ ├── packages/ │ ├── next-sdk/ # Core SDK package │ │ ├── agent/ # WebAgent implementation │ │ ├── client/ # WebMCP client │ │ ├── server/ # WebMCP server │ │ ├── transport/ # Transport layer implementation │ │ ├── McpSdk.ts # MCP SDK encapsulation │ │ ├── index.ts # Main entry │ │ ├── package.json │ │ └── README.md │ ├── next-remoter/ # Vue3 AI Chat Component │ │ ├── src/ │ │ │ ├── components/ # Component implementation │ │ │ └── composable/ # Composables │ │ ├── package.json │ │ └── README.md │ ├── webmcp-cli/ # CLI tool for browser control via WebMCP │ │ ├── src/ # CLI main implementation │ │ ├── webmcp-tools/ # Injected page tools (e.g. Excalidraw, Baidu) │ │ ├── package.json │ │ └── README.md │ ├── webmcp-skill/ # Guidelines and domain-specific skills for AI agents │ │ ├── SKILL.md # Master instruction file for agents │ │ └── domains/ # Domain-specific instructions (e.g., Excalidraw) │ └── doc-ai/ # Doc AI example app ├── docs/ # Project docs ├── pnpm-workspace.yaml # pnpm workspace config ├── package.json └── README.md ``` ## 📚 Related Resources ### Official Docs - [OpenTiny NEXT-SDKs Official Docs](https://docs.opentiny.design/next-sdk/) - [TinyRobot Remoter Component Docs](https://docs.opentiny.design/next-sdk/guide/tiny-robot-remoter.html) - [API Reference](https://docs.opentiny.design/next-sdk/api/) ### Related Projects - [OpenTiny](https://github.com/opentiny) - OpenTiny Organization Homepage - [TinyVue](https://github.com/opentiny/tiny-vue) - Enterprise Vue Component Library - [TinyEngine](https://github.com/opentiny/tiny-engine) - Low-code Engine - [TinyRobot](https://github.com/opentiny/tiny-robot) - AI Chat Component ### WebMCP + WebSkills Best Practice Projects It is recommended to refer directly to the following example projects, clone or implement them according to your tech stack: | Tech Stack | Example Project | Description | | ----------- | ----------------------------------------- | --------------------------------------------------------------------------------------- | | **Vue** | [doc-ai](packages/doc-ai) | Vue3 + Vite, best practices for local WebMCP / Polyfill mode | | **Angular** | [doc-ai-angular](packages/doc-ai-angular) | Angular main app + iframe Remoter, connected to WebMCP via MessageChannel | | **React** | [doc-ai-react](packages/doc-ai-react) | React main app + iframe Remoter, similar WebMCP architecture to Vue version | Accompanying Documentation: - [Vue Best Practice with WebMCP](docs/guide/vue-webmcp-best-practice.md) - [Angular Best Practice with WebMCP](docs/guide/angular-webmcp-best-practice.md) --- ## 📄 License [MIT](https://github.com/opentiny/next-sdk/blob/main/LICENSE) Copyright (c) 2024-present OpenTiny Team ## 🙏 Acknowledgments Thanks to all contributors to the OpenTiny NEXT-SDKs project! [](https://github.com/opentiny/next-sdk/graphs/contributors)