Agent-Reach

<h1 align="center">👁️ Agent Reach</h1> <p align="center"> <strong>Equip Your AI Agent with One-Click Internet Access</strong> </p> <p align="center"> <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a> <a href="https://www.python.org/"><img src="https://img.shields.io/badge/Python-3.10+-green.svg?style=for-the-badge&logo=python&logoColor=white" alt="Python 3.8+"></a> <a href="https://github.com/Panniantong/agent-reach/stargazers"><img src="https://img.shields.io/github/stars/Panniantong/agent-reach?style=for-the-badge" alt="GitHub Stars"></a> </p> <p align="center"> <a href="#快速上手">Quick Start</a> · <a href="docs/README_en.md">English</a> · <a href="#支持的平台">Supported Platforms</a> · <a href="#设计理念">Design Philosophy</a> </p> --- ## Why Do We Need Agent Reach? AI Agents can already help you write code, modify documents, and manage projects - but when you ask them to find something online, they get stuck: - 📺 "Can you check out this YouTube tutorial?" → **Can't access it**, no subtitles available - 🐦 "Can you search for reviews of this product on Twitter?" → **Can't search**, Twitter API requires payment - 📖 "Check if anyone on Reddit has encountered a similar bug" → **403 blocked**, server IP rejected - 📕 "Check the reputation of this product on Xiaohongshu" → **Can't access**, requires login - 📺 "There's a technical video on Bilibili, can you summarize it?" → **Can't connect**, overseas/server IP blocked - 🔍 "Can you search for the latest LLM framework comparisons online?" → **No good search results**, either paid or poor quality - 🌐 "Can you check what's on this webpage?" → **Returns a bunch of HTML tags**, un-readable - 📦 "What's this GitHub repository for? What's in the Issues?" → Can be used, but authentication configuration is complicated - 📡 "Can you subscribe to these RSS feeds and notify me of updates?" → Requires custom library and code **These tasks are not difficult, but require manual configuration** Each platform has its own threshold - paid APIs, firewalls to bypass, required logins, and data to clean. You have to go through each one, install tools, and configure settings. Just enabling an Agent to read Twitter takes a lot of effort. **Agent Reach simplifies this to one line:** ``` Install Agent Reach for me: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md ``` Copy and give it to your Agent, and in a few minutes, it can read Twitter, search Reddit, watch YouTube, and browse Xiaohongshu. **Already installed? Updating is also one line:** ``` Update Agent Reach for me: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md ``` > ⭐ **Star this project**, and we'll continuously track changes to each platform and add new channels. You don't have to monitor - we'll fix it if a platform blocks it, and add new channels. ### ✅ Before You Use It | | | |---|---| | 💰 **Completely Free** | All tools are open-source, and all APIs are free. The only possible cost is server proxy ($1/month), not required for local computers | | 🔒 **Privacy and Security** | Cookies are stored locally and not uploaded or shared. Code is fully open-source and can be reviewed at any time | | 🔄 **Continuous Updates** | Underlying tools (yt-dlp, bird, Jina Reader, etc.) are regularly updated to the latest versions, no need for manual tracking | | 🤖 **Compatible with All Agents** | Claude Code, OpenClaw, Cursor, Windsurf... any Agent that can run commands can use it | | 🩺 **Built-in Diagnostics** | `agent-reach doctor` checks which channels are working and which are not, and provides repair guidance | --- ## Supported Platforms | Platform | Ready to Use | Configurable | How to Configure | |------|---------|-----------|-------| | 🌐 **Webpage** | Read any webpage | — | No configuration required | | 📺 **YouTube** | Subtitle extraction + video search | — | No configuration required | | 📡 **RSS** | Read any RSS/Atom feed | — | No configuration required | | 🔍 **Full-Text Search** | — | Full-text semantic search | Automatic configuration (MCP access, free without Key) | | 📦 **GitHub** | Read public repositories + search | Private repositories, Issues/PR, Fork | Tell Agent "Help me log in to GitHub" | | 🐦 **Twitter/X** | Read single tweet | Search tweets, browse timeline, post tweet | Tell Agent "Help me configure Twitter" | | 📺 **Bilibili** | Local: subtitle extraction + search | Server access | Tell Agent "Help me configure proxy" | | 📖 **Reddit** | Search (via Exa, free) | Read posts and comments | Tell Agent "Help me configure proxy" | | 📕 **Xiaohongshu** | — | Read, search, post, comment, like | Tell Agent "Help me configure Xiaohongshu" | | 🎵 **Douyin** | — | Video parsing, watermark-free download link | Tell Agent "Help me configure Douyin" | | 💼 **LinkedIn** | Jina Reader reads public pages | Profile details, company pages, job search | Tell Agent "Help me configure LinkedIn" | | 💬 **WeChat Official Accounts** | Search + read public account articles (full-text Markdown) | — | No configuration required | | 📰 **Weibo** | Hot search, search content/user/topic, user dynamics, comments | — | No configuration required | | 💻 **V2EX** | Popular posts, node posts, post details + replies, user information | — | No configuration required | | 📈 **Xueqiu** | Stock quotes, search stocks, popular posts, popular stock rankings | — | No configuration required | | 🎙️ **Xiaoyuzhou Podcast** | — | Podcast audio transcription (Whisper transcription, free Key) | Tell Agent "Help me configure Xiaoyuzhou Podcast" | > **Don't know how to configure? No need to check documentation.** Just tell the Agent "Help me configure XXX", and it will guide you through the process. > > 🍪 For platforms that require Cookies (Twitter, Xiaohongshu, etc.), **优先使用** Chrome plugin [Cookie-Editor](https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm) to export Cookies and send to Agent for configuration. The process is unified: browser login → Cookie-Editor export → send to Agent. Simpler and more reliable than scanning QR codes. > > 🔒 Cookies are stored locally and not uploaded or shared. Code is fully open-source and can be reviewed at any time. > 💻 No proxy required for local computers. Proxy only required for server deployment (~$1/month). --- ## Quick Start > ⚠️ **OpenClaw users, please ensure exec permission is enabled** > > Agent Reach relies on the Agent to execute shell commands (`pip install`, `mcporter`, `bird`, etc.). If your OpenClaw uses the default `messaging` tool configuration, the Agent will not be able to execute commands. **Enable exec permission before installation**: > > ```bash > openclaw config set tools.profile "coding" > ``` > or set `"tools": { "profile": "coding" }` in `~/.openclaw/openclaw.json`. > Restart Gateway (`openclaw gateway restart`) and start a new conversation. Copy this to your AI Agent (Claude Code, OpenClaw, Cursor, etc.): ``` Install Agent Reach for me: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md ``` That's it. The Agent will complete the rest. > 🔄 **Already installed?** Update is also one line: > ``` > Update Agent Reach for me: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md > ``` > 🛡️ **Concerned about security?** Use safe mode - won't automatically install system packages, just lists what's needed: > ``` > Install Agent Reach in safe mode: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md > Use --safe parameter during installation > ``` <details> <summary>What does it do? (Click to expand)</summary> 1. **Install CLI tool** — `pip install` installs `agent-reach` command line 2. **Install system dependencies** — Automatically detects and installs Node.js, gh CLI, mcporter, bird, etc. 3. **Configure search engine** — Access Exa via MCP (free, no API Key) 4. **Detect environment** — Determine if it's a local computer or server, and provide configuration suggestions 5. **Register SKILL.md** — Install usage guide in Agent's skills directory, so Agent knows which tool to use for "search Twitter", "watch video", etc. After installation, `agent-reach doctor` checks the status of each channel. </details> --- ## Works Out of the Box No configuration required, just tell the Agent: - "Can you check this link?" → `curl https://r.jina.ai/URL` read any webpage - "What's this GitHub repository for?" → `gh repo view owner/repo` - "What's in this video?" → `yt-dlp --dump-json URL` extract subtitles - "Can you check this tweet?" → `bird read URL` - "Subscribe to this RSS feed" → `feedparser` parse - "Search for LLM frameworks on GitHub" → `gh search repos "LLM framework"` **No need to remember commands.** After the Agent reads SKILL.md, it knows which tool to use. --- ## Design Philosophy **Agent Reach is a scaffolding, not a framework.** When setting up a new Agent environment, you spend time finding tools, installing dependencies, and configuring settings - Twitter reader, Reddit firewall bypass, YouTube subtitle extraction, etc. Each time you have to redo it. Agent Reach simplifies this: **It does the selection and configuration for you.** After installation, the Agent directly calls upstream tools (bird CLI, yt-dlp, mcporter, gh CLI, etc.), without going through Agent Reach's packaging layer. ### 🔌 Each Channel is Pluggable Each platform has an independent upstream tool. **If you're not satisfied, just replace it.** ``` channels/ ├── web.py → Jina Reader ← Can be replaced with Firecrawl, Crawl4AI…… ├── twitter.py → bird ← Can be replaced with Nitter, official API…… ├── youtube.py → yt-dlp ← Can be replaced with YouTube API, Whisper…… ├── github.py → gh CLI ← Can be replaced with REST API, PyGithub…… ├── bilibili.py → yt-dlp ← Can be replaced with bilibili-api…… ├── reddit.py → JSON API + Exa ← Can be replaced with PRAW, Pushshift…… ├── xiaohongshu.py → mcporter MCP ← Can be replaced with other XHS tools…… ├── douyin.py → mcporter MCP ← Can be replaced with other Douyin tools…… ├── linkedin.py → linkedin-mcp ← Can be replaced with LinkedIn API…… ├── wechat.py → camoufox+miku ← Search and read WeChat official account articles ├── rss.py → feedparser ← Can be replaced with atoma…… ├── exa_search.py → mcporter MCP ← Can be replaced with Tavily, SerpAPI…… └── __init__.py → Channel registration (for doctor detection) ``` Each channel file only checks if the upstream tool is available (`check()` method) and provides status information to `agent-reach doctor`. Actual reading and searching are done by the Agent directly calling the upstream tool. ### Current Selection | Scenario | Selection | Why | |------|------|------| | Read webpage | [Jina Reader](https://github.com/jina-ai/reader) | 9.8K Star, free, no API Key required | | Read Twitter | [bird](https://www.npmjs.com/package/@steipete/bird) | Cookie login, free. Official API charges per usage ($0.005 per read) | | Video subtitles + search | [yt-dlp](https://github.com/yt-dlp/yt-dlp) | 148K Star, supports YouTube + Bilibili + 1800 sites | | Full-text search | [Exa](https://exa.ai) via [mcporter](https://github.com/steipete/mcporter) | AI semantic search, MCP access, no Key required | | GitHub | [gh CLI](https://cli.github.com) | Official tool, complete API capabilities after authentication | | Read RSS | [feedparser](https://github.com/kurtmckee/feedparser) | Python ecosystem standard choice, 2.3K Star | | Xiaohongshu | [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) | ⭐9K+, Go language, Docker one-click deployment | | Douyin | [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) | MCP server, no login required, video parsing + watermark-free download | | LinkedIn | [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) | ⭐900+, MCP server, browser automation | | WeChat Official Accounts | [wechat-article-for-ai](https://github.com/Panniantong/wechat-article-for-ai) + [miku_ai](https://github.com/GobinFan/Miku_Spider) | Camoufox stealth browser reads full text + Sogou search | > 📌 These are the **current selections**. If you're not satisfied, just replace the corresponding file. That's what scaffolding is for. --- ## Security Agent Reach prioritizes security: | Measure | Description | |------|------| | 🔒 **Credential local storage** | Cookies and tokens are stored locally in `~/.agent-reach/config.yaml`, with file permissions 600 (owner-only read and write), not uploaded or shared | | 🛡️ **Safe mode** | `agent-reach install --safe` won't modify the system, just lists what's needed, and you decide whether to install | | 👀 **Fully open-source** | Code is transparent and can be reviewed at any time. All dependency tools are also open-source projects | | 🔍 **Dry Run** | `agent-reach install --dry-run` previews all operations without making any changes | | 🧩 **Pluggable architecture** | If you don't trust a component, just replace the corresponding channel file, without affecting others | ### 🍪 Cookie Security Suggestions > ⚠️ **Account risk reminder:** Using Cookies to log in to platforms (Twitter, Xiaohongshu, etc.) **may be detected by the platform and result in account restrictions**. Please use a **dedicated account**, not your main account. For platforms that require Cookies (Twitter, Xiaohongshu), **优先使用** a dedicated account, not your main account. There are two reasons: 1. **Account risk** - The platform may detect non-normal browser API calls, leading to account restrictions or bans 2. **Security risk** - Cookies are equivalent to complete login permissions. Using a dedicated account can limit the impact of credential leaks ### 📦 Installation Methods | Method | Command | Suitable scenario | |------|------|---------| | One-click automatic (default) | `agent-reach install --env=auto` | Personal computer, development environment | | Safe mode | `agent-reach install --env=auto --safe` | Production server, shared machine | | Preview only | `agent-reach install --env=auto --dry-run` | See what it does before installation | ### 🗑️ Uninstallation ```bash agent-reach uninstall ``` Removes: `~/.agent-reach/` (including all tokens/cookies), skill files for each Agent, and MCP configuration in mcporter. ```bash # Preview only, no actual deletion agent-reach uninstall --dry-run # Delete skill files only, keep token configuration (for reinstallation) agent-reach uninstall --keep-config ``` Uninstall Python package itself: `pip uninstall agent-reach` --- ## Contribution This project was created with a vibe coding style 🎸 and may have some imperfections. If you encounter issues, please be patient and report them. Bugs can be reported in [Issues](https://github.com/Panniantong/agent-reach/issues), and I'll fix them as soon as possible. **Want a new channel?** Just report an Issue or submit a PR. **Want to add locally?** Clone it and modify it, each channel is an independent file, easy to add. [PR](https://github.com/Panniantong/agent-reach/pulls) is also welcome! ## ⭐ Why Star I use this project daily, so I'll keep maintaining it. - I'll add new features and channels as requested - I'll ensure each channel is **usable, user-friendly, and free** - If platforms change their anti-scraping measures or APIs, I'll find a solution Contributing to Web 4.0 infrastructure. Star it, so you can find it next time you need it. ⭐ --- ## Frequently Asked Questions / FAQ <details> <summary><strong>How to search Twitter/X with AI agent for free (no API)?</strong></summary> Agent Reach uses the bird CLI with cookie auth — zero API fees. After installing, export your Twitter cookies with the Cookie-Editor extension, run `agent-reach configure twitter-cookies "your_cookies"`, then your agent can search with `bird search "query"`. </details> <details> <summary><strong>Reddit returns 403 / server IP is blocked, what to do?</strong></summary> Reddit blocks data center IPs. Configure a residential proxy to solve this: `agent-reach configure proxy http://user:pass@ip:port`. Recommended: Webshare ($1/month). This usually doesn't happen on local machines. </details> <details> <summary><strong>How to get YouTube video transcripts for AI?</strong></summary> `yt-dlp --dump-json "https://youtube.com/watch?v=xxx"` extracts video metadata; `yt-dlp --write-sub --skip-download "URL"` extracts subtitles. Uses yt-dlp under the hood, supports multiple languages. No API key needed. </details> <details> <summary><strong>How to let AI Agent read Xiaohongshu?</strong></summary> Xiaohongshu requires running an MCP service with Docker. After installing Docker, run `agent-reach install` to configure automatically. Then, the Agent can use `mcporter call 'xiaohongshu.get_feed_detail(...)'` to read notes or `mcporter call 'xiaohongshu.search_feeds(keyword: "keyword")'` to search. </details> <details> <summary><strong>How to let AI Agent parse Douyin videos?</strong></summary> Install douyin-mcp-server, and the Agent can use `mcporter call 'douyin.parse_douyin_video_info(share_link: "share link")'` to parse video information and get a watermark-free download link. No login required; just share the Douyin link with the Agent. See https://github.com/yzfly/douyin-mcp-server for details. </details> <details> <summary><strong>Compatible with Claude Code / Cursor / OpenClaw / Windsurf?</strong></summary> Yes! Agent Reach is an installer + configuration tool — any AI coding agent that can run shell commands can use it. Works with Claude Code, Cursor, OpenClaw, Windsurf, Codex, and more. Just `pip install agent-reach`, run `agent-reach install`, and the agent can start using the upstream tools immediately. **OpenClaw note:** If your OpenClaw is using the default `messaging` tool profile, the agent won't be able to run shell commands. Enable exec first: `openclaw config set tools.profile "coding"` (or set `"tools": { "profile": "coding" }` in `~/.openclaw/openclaw.json`), then restart the Gateway and start a new conversation before installing. </details> <details> <summary><strong>Is this free? Any API costs?</strong></summary> 100% free. All backends are open-source tools (bird CLI, yt-dlp, Jina Reader, Exa, etc.) that don't require paid API keys. The only optional cost is a residential proxy (~$1/month) if you need Reddit/Bilibili access from a server. </details> --- ## Acknowledgements [Jina Reader](https://github.com/jina-ai/reader) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [bird](https://www.npmjs.com/package/@steipete/bird) · [Exa](https://exa.ai) · [mcporter](https://github.com/steipete/mcporter) · [feedparser](https://github.com/kurtmckee/feedparser) · [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) · [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) ## Contact - 📧 **Email:** pnt01@foxmail.com - 🐦 **Twitter/X:** [@Neo_Reidlab](https://x.com/Neo_Reidlab) For discussions or collaborations, add WeChat to join the group: <p align="center"> <img src="docs/wechat-group-qr.jpg" width="280" alt="WeChat QR"> </p> > Bug reports and feature requests: [GitHub Issues](https://github.com/Panniantong/Agent-Reach/issues) for easier tracking. ## License [MIT](LICENSE) ## Friendly Links [FluxNode](https://fluxnode.org) — Affordable AI API relay station, official 1-fold, can be paid by volume or package. Can be used for OpenClaw, Claude Code, and other Agents. [OpenClaw for Enterprise](https://github.com/littleben/openclaw-for-enterprise) — Enterprise-level OpenClaw multi-user deployment solution, directly use AI in Feishu, container isolation, one-command management. [Tencent Cloud OpenClaw](https://www.tencentcloud.com/act/pro/intl-openclaw?referral_code=G76Y819A&lang=en&pg=) — Deploy OpenClaw all-in-one assistant on Tencent Cloud Lighthouse in seconds, access Agent Reach through dialogue, and install Internet capabilities on OpenClaw with one click. ## Star History [](https://star-history.com/#Panniantong/Agent-Reach&Date)