Major refactor: Multi-user Chrome MCP extension with remote server architecture

MULTI_USER_SYSTEM_GUIDE.md

@@ -0,0 +1,214 @@
+# Multi-User Chrome MCP System Guide
+
+## Overview
+
+This system enables multiple users to simultaneously use Chrome extensions with voice commands through LiveKit agents, with complete session isolation and user ID consistency.
+
+## Architecture
+
+```
+User 1: Chrome Extension → Remote Server → LiveKit Agent → Voice Commands → Chrome Extension
+User 2: Chrome Extension → Remote Server → LiveKit Agent → Voice Commands → Chrome Extension
+User 3: Chrome Extension → Remote Server → LiveKit Agent → Voice Commands → Chrome Extension
+```
+
+## Key Features
+
+### 1. **Unique User ID Generation**
+- Each Chrome extension generates a unique random user ID: `user_{timestamp}_{random}`
+- User ID is consistent across all components
+- No authentication required - anonymous sessions
+
+### 2. **Automatic LiveKit Agent Spawning**
+- Remote server automatically starts a dedicated LiveKit agent for each Chrome extension
+- Each agent runs in its own process with the user's unique ID
+- Agents are automatically cleaned up when users disconnect
+
+### 3. **Session Isolation**
+- Each user gets their own LiveKit room: `mcp-chrome-user-{userId}`
+- Voice commands are routed only to the correct user's Chrome extension
+- Multiple users can work simultaneously without interference
+
+### 4. **Voice Command Routing**
+- LiveKit agents include user ID in MCP requests
+- Remote server routes commands to the correct Chrome extension
+- Complete isolation ensures commands never cross between users
+
+## Setup Instructions
+
+### 1. Start the Remote Server
+```bash
+cd app/remote-server
+npm install
+npm start
+```
+
+### 2. Install Chrome Extension
+1. Load the extension in Chrome
+2. Open the popup and click "Connect to Remote Server"
+3. The extension will generate a unique user ID and connect
+
+### 3. LiveKit Agent (Automatic)
+- The remote server automatically starts a LiveKit agent when a Chrome extension connects
+- No manual intervention required
+- Agent uses the same user ID as the Chrome extension
+
+## User Flow
+
+### Step 1: Chrome Extension Connection
+```javascript
+// Chrome extension generates user ID
+const userId = `user_${Date.now()}_${Math.random().toString(36).substring(2, 15)}`;
+
+// Connects to remote server with user ID
+const connectionInfo = {
+  type: 'connection_info',
+  userId: userId,
+  userAgent: navigator.userAgent,
+  timestamp: Date.now(),
+  extensionId: chrome.runtime.id
+};
+```
+
+### Step 2: Remote Server Processing
+```typescript
+// Remote server receives connection
+const sessionInfo = mcpServer.registerChromeExtension(connection, userId, metadata);
+
+// Automatically starts LiveKit agent
+const roomName = `mcp-chrome-user-${userId}`;
+const agentProcess = spawn('python', ['livekit_agent.py', '--room', roomName], {
+  env: { CHROME_USER_ID: userId }
+});
+```
+
+### Step 3: Voice Command Processing
+```python
+# LiveKit agent processes voice command
+async def search_google(context: RunContext, query: str):
+    # Agent includes user ID in MCP request
+    result = await self.mcp_client._search_google_mcp(query)
+    return result
+```
+
+### Step 4: Command Routing
+```typescript
+// Remote server routes command to correct Chrome extension
+const result = await this.sendToExtensions(message, sessionId, userId);
+```
+
+## Testing
+
+### Test 1: Basic Multi-User Connection
+```bash
+node test-multi-user-complete.js
+```
+
+### Test 2: Voice Command Routing
+```bash
+node test-voice-command-routing.js
+```
+
+### Test 3: Session Isolation
+```bash
+node app/remote-server/test-multi-user-livekit.js
+```
+
+## Example Voice Commands
+
+### User 1 says: "Open Google and search for pizza"
+1. LiveKit Agent 1 processes voice
+2. Sends MCP request with User 1's ID
+3. Remote server routes to Chrome Extension 1
+4. Chrome Extension 1 opens Google and searches for pizza
+
+### User 2 says: "Navigate to Facebook"
+1. LiveKit Agent 2 processes voice
+2. Sends MCP request with User 2's ID
+3. Remote server routes to Chrome Extension 2
+4. Chrome Extension 2 navigates to Facebook
+
+**Result**: Both users work independently without interference.
+
+## Session Management
+
+### User Sessions
+```typescript
+interface UserSession {
+  userId: string;           // Unique user ID
+  sessionId: string;        // Session identifier
+  connectionId: string;     // Connection identifier
+  createdAt: number;        // Creation timestamp
+  lastActivity: number;     // Last activity timestamp
+}
+```
+
+### Connection Routing
+```typescript
+// Routes by user ID first, then session ID, then any active connection
+routeMessage(message: any, sessionId?: string, userId?: string): RouteResult
+```
+
+## Monitoring
+
+### Session Status
+- View active sessions in remote server logs
+- Each session shows user ID, connection status, and LiveKit agent status
+- Automatic cleanup of inactive sessions
+
+### LiveKit Agent Status
+- Each agent logs its user ID and room name
+- Agents automatically restart if Chrome extension reconnects
+- Process monitoring and cleanup
+
+## Troubleshooting
+
+### Issue: LiveKit Agent Not Starting
+**Solution**: Check that Python and required packages are installed in `agent-livekit/`
+
+### Issue: Voice Commands Going to Wrong User
+**Solution**: Verify user ID consistency in logs - should be the same across all components
+
+### Issue: Chrome Extension Not Connecting
+**Solution**: Ensure remote server is running on `localhost:3001`
+
+### Issue: Multiple Users Interfering
+**Solution**: Check that each user has a unique user ID and separate LiveKit room
+
+## Configuration
+
+### Environment Variables
+```bash
+# LiveKit Configuration
+LIVEKIT_URL=ws://localhost:7880
+LIVEKIT_API_KEY=devkey
+LIVEKIT_API_SECRET=secret
+
+# Remote Server
+PORT=3001
+HOST=0.0.0.0
+```
+
+### Chrome Extension
+- No configuration required
+- Automatically generates unique user IDs
+- Connects to `ws://localhost:3001/chrome`
+
+### LiveKit Agents
+- Automatically configured by remote server
+- Each agent gets unique room name
+- User ID passed via environment variable
+
+## Security Notes
+
+- System uses anonymous sessions (no authentication)
+- User IDs are randomly generated and temporary
+- Sessions are isolated but not encrypted
+- Suitable for development and testing environments
+
+## Scaling
+
+- System supports multiple concurrent users
+- Each user gets dedicated LiveKit agent process
+- Resource usage scales linearly with user count
+- Consider process limits for production use