220 lines
6.4 KiB
Markdown
220 lines
6.4 KiB
Markdown
# Multi-User Chrome MCP System
|
|
|
|
## 🎯 Overview
|
|
|
|
A complete multi-user system where multiple users can simultaneously use Chrome extensions with voice commands through LiveKit agents, with complete session isolation and manual agent management.
|
|
|
|
## ✨ Key Features
|
|
|
|
### 🔑 **Unique User ID Generation**
|
|
|
|
- Each Chrome extension generates a unique random user ID: `user_{timestamp}_{random}`
|
|
- User ID is consistent across all components (Chrome → Server → Agent → Back to Chrome)
|
|
- No authentication required - anonymous sessions with strong isolation
|
|
|
|
### 🤖 **Automatic LiveKit Agent Spawning**
|
|
|
|
- Remote server automatically starts a dedicated LiveKit agent for each Chrome extension
|
|
- Each agent runs in its own process with the user's unique ID
|
|
- Agents are automatically cleaned up when users disconnect
|
|
|
|
### 🏠 **Session Isolation**
|
|
|
|
- Each user gets their own LiveKit room: `mcp-chrome-user-{userId}`
|
|
- Voice commands are routed only to the correct user's Chrome extension
|
|
- Multiple users can work simultaneously without interference
|
|
|
|
### 🎤 **Voice Command Routing**
|
|
|
|
- LiveKit agents include user ID in MCP requests
|
|
- Remote server routes commands to the correct Chrome extension
|
|
- Complete isolation ensures commands never cross between users
|
|
|
|
## 🚀 Quick Start
|
|
|
|
### 1. Start the Remote Server
|
|
|
|
```bash
|
|
cd app/remote-server
|
|
npm install
|
|
npm start
|
|
```
|
|
|
|
### 2. Install Chrome Extension
|
|
|
|
1. Load the extension in Chrome
|
|
2. Open the popup and click "Connect to Remote Server"
|
|
3. The extension will generate a unique user ID and connect
|
|
|
|
### 3. LiveKit Agent (Manual)
|
|
|
|
- LiveKit agents are no longer started automatically when Chrome extensions connect
|
|
- Agents should be started manually when voice functionality is needed
|
|
- When started, agents use the same user ID as the Chrome extension for proper session isolation
|
|
|
|
## 🔄 User Flow
|
|
|
|
```
|
|
1. Chrome Extension generates unique user ID
|
|
2. Connects to remote server with user ID
|
|
3. Remote server automatically spawns LiveKit agent with same user ID
|
|
4. User speaks voice commands to LiveKit agent
|
|
5. Commands are routed to correct Chrome extension based on user ID
|
|
6. Chrome extension executes commands and returns results
|
|
```
|
|
|
|
## 🧪 Testing
|
|
|
|
### Complete Integration Test
|
|
|
|
```bash
|
|
node test-complete-integration.js
|
|
```
|
|
|
|
Tests the full flow with multiple users, voice commands, and session isolation.
|
|
|
|
### Voice Command Routing Test
|
|
|
|
```bash
|
|
node test-voice-command-routing.js
|
|
```
|
|
|
|
Verifies that voice commands are routed to the correct Chrome extension.
|
|
|
|
### Multi-User Connection Test
|
|
|
|
```bash
|
|
node test-multi-user-complete.js
|
|
```
|
|
|
|
Tests multiple Chrome extension connections and LiveKit agent spawning.
|
|
|
|
## 📊 Example Scenarios
|
|
|
|
### Scenario 1: Multiple Users Searching
|
|
|
|
- **User 1** says: "Open Google and search for pizza"
|
|
- LiveKit Agent 1 → Remote Server → Chrome Extension 1 → Google search for pizza
|
|
- **User 2** says: "Navigate to Facebook"
|
|
- LiveKit Agent 2 → Remote Server → Chrome Extension 2 → Navigate to Facebook
|
|
|
|
**Result**: Both users work independently without interference.
|
|
|
|
### Scenario 2: Session Isolation
|
|
|
|
- **User 1** has 5 tabs open
|
|
- **User 2** has 3 tabs open
|
|
- **User 1** says: "Close all tabs"
|
|
- Only User 1's tabs are closed
|
|
- User 2's tabs remain untouched
|
|
|
|
**Result**: Perfect session isolation maintained.
|
|
|
|
## 🏗️ Architecture Components
|
|
|
|
### Chrome Extension (`app/chrome-extension/`)
|
|
|
|
- Generates unique user ID
|
|
- Connects to remote server via WebSocket
|
|
- Executes voice commands received from LiveKit agent
|
|
|
|
### Remote Server (`app/remote-server/`)
|
|
|
|
- **SessionManager**: Tracks user sessions and connections
|
|
- **LiveKitAgentManager**: Automatically spawns/manages LiveKit agents
|
|
- **ConnectionRouter**: Routes commands to correct Chrome extension
|
|
- **ChromeTools**: Handles MCP tool execution with user context
|
|
|
|
### LiveKit Agent (`agent-livekit/`)
|
|
|
|
- Processes voice commands using OpenAI/Deepgram
|
|
- Includes user ID in all MCP requests for routing
|
|
- Connects to user-specific LiveKit room
|
|
|
|
## 🔧 Configuration
|
|
|
|
### Environment Variables
|
|
|
|
```bash
|
|
# LiveKit Configuration
|
|
LIVEKIT_URL=ws://localhost:7880
|
|
LIVEKIT_API_KEY=devkey
|
|
LIVEKIT_API_SECRET=secret
|
|
|
|
# Remote Server
|
|
PORT=3001
|
|
HOST=0.0.0.0
|
|
```
|
|
|
|
### User ID Format
|
|
|
|
```
|
|
user_{timestamp}_{random}
|
|
Example: user_1703123456_abc123def
|
|
```
|
|
|
|
### LiveKit Room Names
|
|
|
|
```
|
|
mcp-chrome-user-{userId}
|
|
Example: mcp-chrome-user-user_1703123456_abc123def
|
|
```
|
|
|
|
## 📈 Monitoring
|
|
|
|
### Session Status
|
|
|
|
- View active sessions in remote server logs
|
|
- Each session shows user ID, connection status, and LiveKit agent status
|
|
- Automatic cleanup of inactive sessions
|
|
|
|
### LiveKit Agent Status
|
|
|
|
- Each agent logs its user ID and room name
|
|
- Agents automatically restart if Chrome extension reconnects
|
|
- Process monitoring and cleanup
|
|
|
|
## 🔒 Security & Isolation
|
|
|
|
- **Anonymous Sessions**: No authentication required, temporary user IDs
|
|
- **Session Isolation**: Each user's commands only affect their own browser
|
|
- **Process Isolation**: Each user gets a dedicated LiveKit agent process
|
|
- **Network Isolation**: Commands routed by user ID, no cross-contamination
|
|
|
|
## 📚 Documentation
|
|
|
|
- [`MULTI_USER_SYSTEM_GUIDE.md`](MULTI_USER_SYSTEM_GUIDE.md) - Complete usage guide
|
|
- [`docs/MULTI_USER_CHROME_LIVEKIT_INTEGRATION.md`](docs/MULTI_USER_CHROME_LIVEKIT_INTEGRATION.md) - Technical architecture
|
|
- Test files demonstrate complete system functionality
|
|
|
|
## 🎉 Success Criteria
|
|
|
|
✅ **Multiple Chrome Extensions**: Each user gets unique ID and session
|
|
✅ **Automatic Agent Spawning**: LiveKit agents start automatically for each user
|
|
✅ **User ID Consistency**: Same ID flows through Chrome → Server → Agent → Chrome
|
|
✅ **Voice Command Routing**: Commands reach correct user's Chrome extension
|
|
✅ **Session Isolation**: Users work independently without interference
|
|
✅ **Comprehensive Testing**: Full test suite validates all functionality
|
|
|
|
## 🚨 Troubleshooting
|
|
|
|
### Issue: LiveKit Agent Not Starting
|
|
|
|
**Solution**: Check Python environment and dependencies in `agent-livekit/`
|
|
|
|
### Issue: Voice Commands Going to Wrong User
|
|
|
|
**Solution**: Verify user ID consistency in logs across all components
|
|
|
|
### Issue: Chrome Extension Not Connecting
|
|
|
|
**Solution**: Ensure remote server is running on `localhost:3001`
|
|
|
|
### Issue: Multiple Users Interfering
|
|
|
|
**Solution**: Check that each user has unique user ID and separate LiveKit room
|
|
|
|
---
|
|
|
|
**🎤 Ready to experience multi-user voice automation? Start the system and connect multiple Chrome extensions to see the magic happen!**
|