broswer-automation/MULTI_USER_SYSTEM_GUIDE.md

# Multi-User Chrome MCP System Guide

## Overview

This system enables multiple users to simultaneously use Chrome extensions with voice commands through LiveKit agents, with complete session isolation and user ID consistency.

## Architecture

```
User 1: Chrome Extension → Remote Server → LiveKit Agent → Voice Commands → Chrome Extension
User 2: Chrome Extension → Remote Server → LiveKit Agent → Voice Commands → Chrome Extension
User 3: Chrome Extension → Remote Server → LiveKit Agent → Voice Commands → Chrome Extension
```

## Key Features

### 1. **Unique User ID Generation**
- Each Chrome extension generates a unique random user ID: `user_{timestamp}_{random}`
- User ID is consistent across all components
- No authentication required - anonymous sessions

### 2. **Automatic LiveKit Agent Spawning**
- Remote server automatically starts a dedicated LiveKit agent for each Chrome extension
- Each agent runs in its own process with the user's unique ID
- Agents are automatically cleaned up when users disconnect

### 3. **Session Isolation**
- Each user gets their own LiveKit room: `mcp-chrome-user-{userId}`
- Voice commands are routed only to the correct user's Chrome extension
- Multiple users can work simultaneously without interference

### 4. **Voice Command Routing**
- LiveKit agents include user ID in MCP requests
- Remote server routes commands to the correct Chrome extension
- Complete isolation ensures commands never cross between users

## Setup Instructions

### 1. Start the Remote Server
```bash
cd app/remote-server
npm install
npm start
```

### 2. Install Chrome Extension
1. Load the extension in Chrome
2. Open the popup and click "Connect to Remote Server"
3. The extension will generate a unique user ID and connect

### 3. LiveKit Agent (Automatic)
- The remote server automatically starts a LiveKit agent when a Chrome extension connects
- No manual intervention required
- Agent uses the same user ID as the Chrome extension

## User Flow

### Step 1: Chrome Extension Connection
```javascript
// Chrome extension generates user ID
const userId = `user_${Date.now()}_${Math.random().toString(36).substring(2, 15)}`;

// Connects to remote server with user ID
const connectionInfo = {
  type: 'connection_info',
  userId: userId,
  userAgent: navigator.userAgent,
  timestamp: Date.now(),
  extensionId: chrome.runtime.id
};
```

### Step 2: Remote Server Processing
```typescript
// Remote server receives connection
const sessionInfo = mcpServer.registerChromeExtension(connection, userId, metadata);

// Automatically starts LiveKit agent
const roomName = `mcp-chrome-user-${userId}`;
const agentProcess = spawn('python', ['livekit_agent.py', '--room', roomName], {
  env: { CHROME_USER_ID: userId }
});
```

### Step 3: Voice Command Processing
```python
# LiveKit agent processes voice command
async def search_google(context: RunContext, query: str):
    # Agent includes user ID in MCP request
    result = await self.mcp_client._search_google_mcp(query)
    return result
```

### Step 4: Command Routing
```typescript
// Remote server routes command to correct Chrome extension
const result = await this.sendToExtensions(message, sessionId, userId);
```

## Testing

### Test 1: Basic Multi-User Connection
```bash
node test-multi-user-complete.js
```

### Test 2: Voice Command Routing
```bash
node test-voice-command-routing.js
```

### Test 3: Session Isolation
```bash
node app/remote-server/test-multi-user-livekit.js
```

## Example Voice Commands

### User 1 says: "Open Google and search for pizza"
1. LiveKit Agent 1 processes voice
2. Sends MCP request with User 1's ID
3. Remote server routes to Chrome Extension 1
4. Chrome Extension 1 opens Google and searches for pizza

### User 2 says: "Navigate to Facebook"
1. LiveKit Agent 2 processes voice
2. Sends MCP request with User 2's ID
3. Remote server routes to Chrome Extension 2
4. Chrome Extension 2 navigates to Facebook

**Result**: Both users work independently without interference.

## Session Management

### User Sessions
```typescript
interface UserSession {
  userId: string;           // Unique user ID
  sessionId: string;        // Session identifier
  connectionId: string;     // Connection identifier
  createdAt: number;        // Creation timestamp
  lastActivity: number;     // Last activity timestamp
}
```

### Connection Routing
```typescript
// Routes by user ID first, then session ID, then any active connection
routeMessage(message: any, sessionId?: string, userId?: string): RouteResult
```

## Monitoring

### Session Status
- View active sessions in remote server logs
- Each session shows user ID, connection status, and LiveKit agent status
- Automatic cleanup of inactive sessions

### LiveKit Agent Status
- Each agent logs its user ID and room name
- Agents automatically restart if Chrome extension reconnects
- Process monitoring and cleanup

## Troubleshooting

### Issue: LiveKit Agent Not Starting
**Solution**: Check that Python and required packages are installed in `agent-livekit/`

### Issue: Voice Commands Going to Wrong User
**Solution**: Verify user ID consistency in logs - should be the same across all components

### Issue: Chrome Extension Not Connecting
**Solution**: Ensure remote server is running on `localhost:3001`

### Issue: Multiple Users Interfering
**Solution**: Check that each user has a unique user ID and separate LiveKit room

## Configuration

### Environment Variables
```bash
# LiveKit Configuration
LIVEKIT_URL=ws://localhost:7880
LIVEKIT_API_KEY=devkey
LIVEKIT_API_SECRET=secret

# Remote Server
PORT=3001
HOST=0.0.0.0
```

### Chrome Extension
- No configuration required
- Automatically generates unique user IDs
- Connects to `ws://localhost:3001/chrome`

### LiveKit Agents
- Automatically configured by remote server
- Each agent gets unique room name
- User ID passed via environment variable

## Security Notes

- System uses anonymous sessions (no authentication)
- User IDs are randomly generated and temporary
- Sessions are isolated but not encrypted
- Suitable for development and testing environments

## Scaling

- System supports multiple concurrent users
- Each user gets dedicated LiveKit agent process
- Resource usage scales linearly with user count
- Consider process limits for production use