Major refactor: Multi-user Chrome MCP extension with remote server architecture
This commit is contained in:
nasir@endelospay.com
219
README_MULTI_USER.md
Normal file
219
README_MULTI_USER.md
Normal file
@@ -0,0 +1,219 @@
|
||||
# Multi-User Chrome MCP System
|
||||
|
||||
## 🎯 Overview
|
||||
|
||||
A complete multi-user system where multiple users can simultaneously use Chrome extensions with voice commands through LiveKit agents, with complete session isolation and manual agent management.
|
||||
|
||||
## ✨ Key Features
|
||||
|
||||
### 🔑 **Unique User ID Generation**
|
||||
|
||||
- Each Chrome extension generates a unique random user ID: `user_{timestamp}_{random}`
|
||||
- User ID is consistent across all components (Chrome → Server → Agent → Back to Chrome)
|
||||
- No authentication required - anonymous sessions with strong isolation
|
||||
|
||||
### 🤖 **Automatic LiveKit Agent Spawning**
|
||||
|
||||
- Remote server automatically starts a dedicated LiveKit agent for each Chrome extension
|
||||
- Each agent runs in its own process with the user's unique ID
|
||||
- Agents are automatically cleaned up when users disconnect
|
||||
|
||||
### 🏠 **Session Isolation**
|
||||
|
||||
- Each user gets their own LiveKit room: `mcp-chrome-user-{userId}`
|
||||
- Voice commands are routed only to the correct user's Chrome extension
|
||||
- Multiple users can work simultaneously without interference
|
||||
|
||||
### 🎤 **Voice Command Routing**
|
||||
|
||||
- LiveKit agents include user ID in MCP requests
|
||||
- Remote server routes commands to the correct Chrome extension
|
||||
- Complete isolation ensures commands never cross between users
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### 1. Start the Remote Server
|
||||
|
||||
```bash
|
||||
cd app/remote-server
|
||||
npm install
|
||||
npm start
|
||||
```
|
||||
|
||||
### 2. Install Chrome Extension
|
||||
|
||||
1. Load the extension in Chrome
|
||||
2. Open the popup and click "Connect to Remote Server"
|
||||
3. The extension will generate a unique user ID and connect
|
||||
|
||||
### 3. LiveKit Agent (Manual)
|
||||
|
||||
- LiveKit agents are no longer started automatically when Chrome extensions connect
|
||||
- Agents should be started manually when voice functionality is needed
|
||||
- When started, agents use the same user ID as the Chrome extension for proper session isolation
|
||||
|
||||
## 🔄 User Flow
|
||||
|
||||
```
|
||||
1. Chrome Extension generates unique user ID
|
||||
2. Connects to remote server with user ID
|
||||
3. Remote server automatically spawns LiveKit agent with same user ID
|
||||
4. User speaks voice commands to LiveKit agent
|
||||
5. Commands are routed to correct Chrome extension based on user ID
|
||||
6. Chrome extension executes commands and returns results
|
||||
```
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Complete Integration Test
|
||||
|
||||
```bash
|
||||
node test-complete-integration.js
|
||||
```
|
||||
|
||||
Tests the full flow with multiple users, voice commands, and session isolation.
|
||||
|
||||
### Voice Command Routing Test
|
||||
|
||||
```bash
|
||||
node test-voice-command-routing.js
|
||||
```
|
||||
|
||||
Verifies that voice commands are routed to the correct Chrome extension.
|
||||
|
||||
### Multi-User Connection Test
|
||||
|
||||
```bash
|
||||
node test-multi-user-complete.js
|
||||
```
|
||||
|
||||
Tests multiple Chrome extension connections and LiveKit agent spawning.
|
||||
|
||||
## 📊 Example Scenarios
|
||||
|
||||
### Scenario 1: Multiple Users Searching
|
||||
|
||||
- **User 1** says: "Open Google and search for pizza"
|
||||
- LiveKit Agent 1 → Remote Server → Chrome Extension 1 → Google search for pizza
|
||||
- **User 2** says: "Navigate to Facebook"
|
||||
- LiveKit Agent 2 → Remote Server → Chrome Extension 2 → Navigate to Facebook
|
||||
|
||||
**Result**: Both users work independently without interference.
|
||||
|
||||
### Scenario 2: Session Isolation
|
||||
|
||||
- **User 1** has 5 tabs open
|
||||
- **User 2** has 3 tabs open
|
||||
- **User 1** says: "Close all tabs"
|
||||
- Only User 1's tabs are closed
|
||||
- User 2's tabs remain untouched
|
||||
|
||||
**Result**: Perfect session isolation maintained.
|
||||
|
||||
## 🏗️ Architecture Components
|
||||
|
||||
### Chrome Extension (`app/chrome-extension/`)
|
||||
|
||||
- Generates unique user ID
|
||||
- Connects to remote server via WebSocket
|
||||
- Executes voice commands received from LiveKit agent
|
||||
|
||||
### Remote Server (`app/remote-server/`)
|
||||
|
||||
- **SessionManager**: Tracks user sessions and connections
|
||||
- **LiveKitAgentManager**: Automatically spawns/manages LiveKit agents
|
||||
- **ConnectionRouter**: Routes commands to correct Chrome extension
|
||||
- **ChromeTools**: Handles MCP tool execution with user context
|
||||
|
||||
### LiveKit Agent (`agent-livekit/`)
|
||||
|
||||
- Processes voice commands using OpenAI/Deepgram
|
||||
- Includes user ID in all MCP requests for routing
|
||||
- Connects to user-specific LiveKit room
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
# LiveKit Configuration
|
||||
LIVEKIT_URL=ws://localhost:7880
|
||||
LIVEKIT_API_KEY=devkey
|
||||
LIVEKIT_API_SECRET=secret
|
||||
|
||||
# Remote Server
|
||||
PORT=3001
|
||||
HOST=0.0.0.0
|
||||
```
|
||||
|
||||
### User ID Format
|
||||
|
||||
```
|
||||
user_{timestamp}_{random}
|
||||
Example: user_1703123456_abc123def
|
||||
```
|
||||
|
||||
### LiveKit Room Names
|
||||
|
||||
```
|
||||
mcp-chrome-user-{userId}
|
||||
Example: mcp-chrome-user-user_1703123456_abc123def
|
||||
```
|
||||
|
||||
## 📈 Monitoring
|
||||
|
||||
### Session Status
|
||||
|
||||
- View active sessions in remote server logs
|
||||
- Each session shows user ID, connection status, and LiveKit agent status
|
||||
- Automatic cleanup of inactive sessions
|
||||
|
||||
### LiveKit Agent Status
|
||||
|
||||
- Each agent logs its user ID and room name
|
||||
- Agents automatically restart if Chrome extension reconnects
|
||||
- Process monitoring and cleanup
|
||||
|
||||
## 🔒 Security & Isolation
|
||||
|
||||
- **Anonymous Sessions**: No authentication required, temporary user IDs
|
||||
- **Session Isolation**: Each user's commands only affect their own browser
|
||||
- **Process Isolation**: Each user gets a dedicated LiveKit agent process
|
||||
- **Network Isolation**: Commands routed by user ID, no cross-contamination
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
- [`MULTI_USER_SYSTEM_GUIDE.md`](MULTI_USER_SYSTEM_GUIDE.md) - Complete usage guide
|
||||
- [`docs/MULTI_USER_CHROME_LIVEKIT_INTEGRATION.md`](docs/MULTI_USER_CHROME_LIVEKIT_INTEGRATION.md) - Technical architecture
|
||||
- Test files demonstrate complete system functionality
|
||||
|
||||
## 🎉 Success Criteria
|
||||
|
||||
✅ **Multiple Chrome Extensions**: Each user gets unique ID and session
|
||||
✅ **Automatic Agent Spawning**: LiveKit agents start automatically for each user
|
||||
✅ **User ID Consistency**: Same ID flows through Chrome → Server → Agent → Chrome
|
||||
✅ **Voice Command Routing**: Commands reach correct user's Chrome extension
|
||||
✅ **Session Isolation**: Users work independently without interference
|
||||
✅ **Comprehensive Testing**: Full test suite validates all functionality
|
||||
|
||||
## 🚨 Troubleshooting
|
||||
|
||||
### Issue: LiveKit Agent Not Starting
|
||||
|
||||
**Solution**: Check Python environment and dependencies in `agent-livekit/`
|
||||
|
||||
### Issue: Voice Commands Going to Wrong User
|
||||
|
||||
**Solution**: Verify user ID consistency in logs across all components
|
||||
|
||||
### Issue: Chrome Extension Not Connecting
|
||||
|
||||
**Solution**: Ensure remote server is running on `localhost:3001`
|
||||
|
||||
### Issue: Multiple Users Interfering
|
||||
|
||||
**Solution**: Check that each user has unique user ID and separate LiveKit room
|
||||
|
||||
---
|
||||
|
||||
**🎤 Ready to experience multi-user voice automation? Start the system and connect multiple Chrome extensions to see the magic happen!**
|
||||
Reference in New Issue
Block a user