broswer-automation/MULTI_USER_SYSTEM_GUIDE.md

Multi-User Chrome MCP System Guide

Overview

This system enables multiple users to simultaneously use Chrome extensions with voice commands through LiveKit agents, with complete session isolation and user ID consistency.

Architecture

User 1: Chrome Extension → Remote Server → LiveKit Agent → Voice Commands → Chrome Extension
User 2: Chrome Extension → Remote Server → LiveKit Agent → Voice Commands → Chrome Extension
User 3: Chrome Extension → Remote Server → LiveKit Agent → Voice Commands → Chrome Extension

Key Features

1. Unique User ID Generation

• Each Chrome extension generates a unique random user ID: user_{timestamp}_{random}
• User ID is consistent across all components
• No authentication required - anonymous sessions

2. Automatic LiveKit Agent Spawning

• Remote server automatically starts a dedicated LiveKit agent for each Chrome extension
• Each agent runs in its own process with the user's unique ID
• Agents are automatically cleaned up when users disconnect

3. Session Isolation

• Each user gets their own LiveKit room: mcp-chrome-user-{userId}
• Voice commands are routed only to the correct user's Chrome extension
• Multiple users can work simultaneously without interference

4. Voice Command Routing

• LiveKit agents include user ID in MCP requests
• Remote server routes commands to the correct Chrome extension
• Complete isolation ensures commands never cross between users

Setup Instructions

1. Start the Remote Server

cd app/remote-server
npm install
npm start

2. Install Chrome Extension

1. Load the extension in Chrome
2. Open the popup and click "Connect to Remote Server"
3. The extension will generate a unique user ID and connect

3. LiveKit Agent (Automatic)

• The remote server automatically starts a LiveKit agent when a Chrome extension connects
• No manual intervention required
• Agent uses the same user ID as the Chrome extension

User Flow

Step 1: Chrome Extension Connection

// Chrome extension generates user ID
const userId = `user_${Date.now()}_${Math.random().toString(36).substring(2, 15)}`;

// Connects to remote server with user ID
const connectionInfo = {
  type: 'connection_info',
  userId: userId,
  userAgent: navigator.userAgent,
  timestamp: Date.now(),
  extensionId: chrome.runtime.id
};

Step 2: Remote Server Processing

// Remote server receives connection
const sessionInfo = mcpServer.registerChromeExtension(connection, userId, metadata);

// Automatically starts LiveKit agent
const roomName = `mcp-chrome-user-${userId}`;
const agentProcess = spawn('python', ['livekit_agent.py', '--room', roomName], {
  env: { CHROME_USER_ID: userId }
});

Step 3: Voice Command Processing

# LiveKit agent processes voice command
async def search_google(context: RunContext, query: str):
    # Agent includes user ID in MCP request
    result = await self.mcp_client._search_google_mcp(query)
    return result

Step 4: Command Routing

// Remote server routes command to correct Chrome extension
const result = await this.sendToExtensions(message, sessionId, userId);

Testing

Test 1: Basic Multi-User Connection

node test-multi-user-complete.js

Test 2: Voice Command Routing

node test-voice-command-routing.js

Test 3: Session Isolation

node app/remote-server/test-multi-user-livekit.js

Example Voice Commands

User 1 says: "Open Google and search for pizza"

1. LiveKit Agent 1 processes voice
2. Sends MCP request with User 1's ID
3. Remote server routes to Chrome Extension 1
4. Chrome Extension 1 opens Google and searches for pizza

User 2 says: "Navigate to Facebook"

1. LiveKit Agent 2 processes voice
2. Sends MCP request with User 2's ID
3. Remote server routes to Chrome Extension 2
4. Chrome Extension 2 navigates to Facebook

Result: Both users work independently without interference.

Session Management

User Sessions

interface UserSession {
  userId: string;           // Unique user ID
  sessionId: string;        // Session identifier
  connectionId: string;     // Connection identifier
  createdAt: number;        // Creation timestamp
  lastActivity: number;     // Last activity timestamp
}

Connection Routing

// Routes by user ID first, then session ID, then any active connection
routeMessage(message: any, sessionId?: string, userId?: string): RouteResult

Monitoring

Session Status

• View active sessions in remote server logs
• Each session shows user ID, connection status, and LiveKit agent status
• Automatic cleanup of inactive sessions

LiveKit Agent Status

• Each agent logs its user ID and room name
• Agents automatically restart if Chrome extension reconnects
• Process monitoring and cleanup

Troubleshooting

Issue: LiveKit Agent Not Starting

Solution: Check that Python and required packages are installed in agent-livekit/

Issue: Voice Commands Going to Wrong User

Solution: Verify user ID consistency in logs - should be the same across all components

Issue: Chrome Extension Not Connecting

Solution: Ensure remote server is running on localhost:3001

Issue: Multiple Users Interfering

Solution: Check that each user has a unique user ID and separate LiveKit room

Configuration

Environment Variables

# LiveKit Configuration
LIVEKIT_URL=ws://localhost:7880
LIVEKIT_API_KEY=devkey
LIVEKIT_API_SECRET=secret

# Remote Server
PORT=3001
HOST=0.0.0.0

Chrome Extension

• No configuration required
• Automatically generates unique user IDs
• Connects to ws://localhost:3001/chrome

LiveKit Agents

• Automatically configured by remote server
• Each agent gets unique room name
• User ID passed via environment variable

Security Notes

• System uses anonymous sessions (no authentication)
• User IDs are randomly generated and temporary
• Sessions are isolated but not encrypted
• Suitable for development and testing environments

Scaling

• System supports multiple concurrent users
• Each user gets dedicated LiveKit agent process
• Resource usage scales linearly with user count
• Consider process limits for production use