SaySth-2025

# SaySth-2025 Cap Stone Project in 2025 # You would better go to this repository. [New repo](https://github.com/chs991209/AgenticAIforPC) ## Project Structure - `main.py`: FastAPI server entry point - `agents/`: AI agent related code - `tools/`: External API tools (YouTube, etc.) - `client/`: TypeScript-based web client (Vite + React) - `frontend-server/`: Next.js-based API proxy server (ASR, Agentic AI, MCP server proxy) - `localMCPServer/`: Local MCP server ## Server Execution ### 1. Agentic AI Server Execution ```bash # Activate virtual environment source agenticai-saysth/bin/activate # Run server uvicorn main:app --port 8002 --reload ``` ### 2. Local MCP Server Execution ```bash # Move to localMCPServer directory cd localMCPServer # Activate virtual environment (if necessary) source local-mcp-server/bin/activate # Run MCP server uvicorn local-mcp-server:app --port 8003 --reload ``` ### 3. Frontend Server Execution (API Proxy) ```bash # Move to frontend-server directory cd frontend-server # Set environment variables (required!) # Create .env.local file and add: # ASR_SERVER_URL=http://your-asr-server-url:8004 # AGENTIC_AI_SERVER_URL=http://127.0.0.1:8002 # MCP_SERVER_URL=http://127.0.0.1:8003 # Install dependencies npm install # Run development server npm run dev ``` Frontend Server runs at `http://localhost:3000`. **Important:** - All server addresses must be set in `.env.local` file. If not set, the server will return an error. - This server provides only API endpoints, not web pages. ### 4. Client Execution (Existing Vite Client) ```bash # Move to client directory cd client # Install dependencies npm install # Run development server npm run dev ``` **Reference:** The following servers must be running for normal operation: - Agentic AI server: `http://127.0.0.1:8002` - Local MCP server: `http://127.0.0.1:8003` - Frontend Server (Next.js): `http://localhost:3000` (API proxy only, no web pages) - Client (Vite): `http://localhost:5173` (or other port, provides web pages) **Frontend Server Role:** - Proxy client's requests to ASR server, Agentic AI server, MCP server - Provides only API endpoints, not web pages ## API Endpoints ### POST /execute Execute text prompt to generate actions. **Request:** ```json { "prompt": "Play IVE music video" } ``` **Response:** ```json { "actions_list": [ { "open_webbrowser": ["https://www.youtube.com/watch?v=xxx"] } ] } ``` ### POST /execute-voice Execute voice command (text converted by ASR) to generate actions. **Request:** ```json { "prompt": "Play IVE music video" } ``` **Response:** ```json { "actions_list": [ { "open_webbrowser": ["https://www.youtube.com/watch?v=xxx"] } ] } ``` ## Operation Flow ### Text Input 1. User inputs text 2. Frontend Server → Agentic AI server (`/execute`) 3. Generate actions 4. Frontend Server → MCP server (`/pc_action_execute`) 5. Execute actions ### Voice Input 1. Client clicks microphone button 2. Records voice 3. Client → Frontend Server (`/api/asr`) → ASR server (voice to text conversion) 4. Client → Frontend Server (`/api/execute-voice`) → Agentic AI server (`/execute-voice`) 5. Generate actions 6. Client → Frontend Server (`/api/execute-actions`) → MCP server (`/pc_action_execute`) 7. Execute actions