Content
# Computer Use AI SDK
* We've built an MCP server that controls computer
* You've heard of OpenAI's operator, you've heard of Claude's computer use. Now the open source alternative: Computer Use SDK from screenpipe.
* It's native on macOS—no virtual machine bs, no guardrails. Use it with any app or website however you want.
* No pixel-based bs—it relies on underlying desktop-rendered elements, making it much faster and far more reliable than pixel-based vision models.
* You can now build your own agents getting started with our simple Hello World Template using our MCP server and client.
* There are tools that our MCP Server provides out of the box:
* Launch apps
* Read content
* Click
* Enter text
* Press keys
* These will be computational primitives to allow the AI to control your computer and do your tasks for you. What will you build? Come check us out at https://screenpi.pe
## Demos
agent sending a message
https://github.com/user-attachments/assets/f8687500-9a8c-4a96-81b6-77562feff093
get latest whatsapp messages
open arc browser
## Get started
```bash
git clone https://github.com/m13v/computer-use-ai-sdk.git
cd MCP-server-client-computer-use-ai-sdk
```
```bash
# Install Rust (if not already installed)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Install Node.js and npm (if not already installed)
# Visit https://nodejs.org/ or use nvm
```
```bash
# run backend server
cd mcp-server-os-level
cargo run --bin server
# keep it running
```
### Option 1: CLI Interface
```bash
# run CLI interface client in a new terminal (good for debugging)
cd mcp-client-cli-interface
npm install # install dependencies first
# Set your Anthropic API key as an environment variable
export ANTHROPIC_API_KEY=sk-ant-xxxx # Replace with your actual Anthropic API key
# For Windows, use: set ANTHROPIC_API_KEY=sk-ant-xxxx
# For permanent setup, add to your shell profile (.bashrc, .zshrc, etc.)
npx tsx main.ts
```
### Option 2: Web app Interface
```bash
# run CLI interface client in a new terminal (good for debugging)
cd mcp-client-nextjs
npm install # install dependencies first
# Set API key via command line
echo "ANTHROPIC_API_KEY=sk-ant-XXXXXXXX" > .env # replace XXXXXXXX with your actual key
# Or append if you want to keep other env variables
# echo "ANTHROPIC_API_KEY=sk-ant-XXXXXXXX" >> .env
npm run dev
# go to provided localhost web page
```
## What do I do with it?
- Build custom worfklows of agents to performs various actions
- Build custom UI to make it easy for users to automate their computer work
- Save workflow and run in cron
- Combine with other MCP servers to do something cool, e.g.: fill out a google sheet based on the history of people i talk to throughout the day
## Request features and endpoints in github issues
https://github.com/m13v/computer-use-ai-sdk/issues/new/choose
