first commit
This commit is contained in:
nasir@endelospay.com
279
agent-livekit/QUBECARE_TESTING_GUIDE.md
Normal file
279
agent-livekit/QUBECARE_TESTING_GUIDE.md
Normal file
@@ -0,0 +1,279 @@
|
||||
# QuBeCare Live Testing Guide for Enhanced Voice Agent
|
||||
|
||||
## 🎯 Overview
|
||||
|
||||
This guide provides step-by-step instructions for testing the enhanced LiveKit voice agent with the QuBeCare login page at `https://app.qubecare.ai/provider/login`.
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Prerequisites
|
||||
1. **Chrome MCP Server Running**
|
||||
```bash
|
||||
cd app/native-server
|
||||
npm start
|
||||
```
|
||||
|
||||
2. **LiveKit Server Available**
|
||||
- Ensure your LiveKit server is running
|
||||
- Have your API keys configured
|
||||
|
||||
3. **Environment Setup**
|
||||
```bash
|
||||
cd agent-livekit
|
||||
# Make sure .env file has your API keys
|
||||
```
|
||||
|
||||
## 🧪 Testing Options
|
||||
|
||||
### Option 1: Automated Test Script
|
||||
```bash
|
||||
cd agent-livekit
|
||||
python qubecare_voice_test.py
|
||||
```
|
||||
|
||||
**What it does:**
|
||||
- Automatically navigates to QuBeCare login page
|
||||
- Tests username entry with voice commands
|
||||
- Tests password entry with voice commands
|
||||
- Tests login button clicking
|
||||
- Provides detailed results
|
||||
|
||||
### Option 2: Interactive Testing
|
||||
```bash
|
||||
cd agent-livekit
|
||||
python qubecare_voice_test.py
|
||||
# Choose option 2 for interactive mode
|
||||
```
|
||||
|
||||
**What it does:**
|
||||
- Navigates to QuBeCare
|
||||
- Lets you manually test voice commands
|
||||
- Real-time feedback for each command
|
||||
|
||||
### Option 3: Full LiveKit Agent
|
||||
```bash
|
||||
cd agent-livekit
|
||||
python start_agent.py
|
||||
```
|
||||
|
||||
**Then connect to LiveKit room and use voice commands directly**
|
||||
|
||||
## 🗣️ Voice Commands to Test
|
||||
|
||||
### Navigation Commands
|
||||
```
|
||||
"navigate to https://app.qubecare.ai/provider/login"
|
||||
"go to QuBeCare login"
|
||||
```
|
||||
|
||||
### Page Analysis Commands
|
||||
```
|
||||
"what's on this page"
|
||||
"show me form fields"
|
||||
"what can I click"
|
||||
"get interactive elements"
|
||||
```
|
||||
|
||||
### Username Entry Commands
|
||||
```
|
||||
"fill email with your@email.com"
|
||||
"enter your@email.com in email field"
|
||||
"type your@email.com in username"
|
||||
"email your@email.com"
|
||||
"username your@email.com"
|
||||
```
|
||||
|
||||
### Password Entry Commands
|
||||
```
|
||||
"fill password with yourpassword"
|
||||
"enter yourpassword in password field"
|
||||
"type yourpassword in password"
|
||||
"password yourpassword"
|
||||
"pass yourpassword"
|
||||
```
|
||||
|
||||
### Login Button Commands
|
||||
```
|
||||
"click login button"
|
||||
"press login"
|
||||
"click sign in"
|
||||
"press sign in button"
|
||||
"login"
|
||||
"sign in"
|
||||
"click submit"
|
||||
```
|
||||
|
||||
## 📋 Step-by-Step Testing Process
|
||||
|
||||
### Step 1: Start Chrome MCP Server
|
||||
```bash
|
||||
cd app/native-server
|
||||
npm start
|
||||
```
|
||||
**Expected:** Server starts on `http://127.0.0.1:12306/mcp`
|
||||
|
||||
### Step 2: Run Test Script
|
||||
```bash
|
||||
cd agent-livekit
|
||||
python qubecare_voice_test.py
|
||||
```
|
||||
|
||||
### Step 3: Choose Test Mode
|
||||
- **Option 1**: Automated test with default credentials
|
||||
- **Option 2**: Interactive mode for manual testing
|
||||
|
||||
### Step 4: Observe Results
|
||||
The script will:
|
||||
1. ✅ Connect to MCP server
|
||||
2. 🌐 Navigate to QuBeCare login page
|
||||
3. 🔍 Analyze page structure
|
||||
4. 👤 Test username entry
|
||||
5. 🔒 Test password entry
|
||||
6. 🔘 Test login button click
|
||||
7. 📊 Show results summary
|
||||
|
||||
## 🔍 Expected Results
|
||||
|
||||
### Successful Test Output
|
||||
```
|
||||
🎤 QUBECARE VOICE COMMAND TEST
|
||||
==================================================
|
||||
✅ Connected successfully!
|
||||
📍 Navigation: Successfully navigated to https://app.qubecare.ai/provider/login
|
||||
📋 Form fields: Found 2 form fields: email, password...
|
||||
🖱️ Clickable elements: Found 5 interactive elements: login button...
|
||||
✅ Username filled successfully!
|
||||
✅ Password filled successfully!
|
||||
✅ Login button clicked successfully!
|
||||
|
||||
📊 TEST RESULTS SUMMARY
|
||||
========================================
|
||||
🌐 Navigation: ✅ Success
|
||||
👤 Username: ✅ Success
|
||||
🔒 Password: ✅ Success
|
||||
🔘 Login Click: ✅ Success
|
||||
========================================
|
||||
🎉 ALL TESTS PASSED! Voice commands working perfectly!
|
||||
```
|
||||
|
||||
### Troubleshooting Common Issues
|
||||
|
||||
#### Issue: "Failed to connect to MCP server"
|
||||
**Solution:**
|
||||
```bash
|
||||
# Make sure Chrome MCP server is running
|
||||
cd app/native-server
|
||||
npm start
|
||||
```
|
||||
|
||||
#### Issue: "Navigation failed"
|
||||
**Solution:**
|
||||
- Check internet connection
|
||||
- Verify QuBeCare URL is accessible
|
||||
- Try manual navigation first
|
||||
|
||||
#### Issue: "Form fields not found"
|
||||
**Solution:**
|
||||
- Wait longer for page load (increase sleep time)
|
||||
- Check if page structure changed
|
||||
- Try different field detection commands
|
||||
|
||||
#### Issue: "Elements not clickable"
|
||||
**Solution:**
|
||||
- Verify page is fully loaded
|
||||
- Try different click command variations
|
||||
- Check browser console for errors
|
||||
|
||||
## 🎮 Interactive Testing Tips
|
||||
|
||||
### Best Practices
|
||||
1. **Wait for page load** - Give pages 3-5 seconds to fully load
|
||||
2. **Try multiple variations** - If one command fails, try alternatives
|
||||
3. **Check page structure** - Use "show me form fields" to understand the page
|
||||
4. **Be specific** - Use exact field names when possible
|
||||
|
||||
### Useful Debug Commands
|
||||
```
|
||||
"show me form fields" # See all available form fields
|
||||
"what can I click" # See all clickable elements
|
||||
"what's on this page" # Get page content summary
|
||||
"get interactive elements" # Detailed interactive elements
|
||||
```
|
||||
|
||||
## 📊 Performance Expectations
|
||||
|
||||
### Response Times
|
||||
- **Navigation**: 2-4 seconds
|
||||
- **Form field detection**: < 1 second
|
||||
- **Field filling**: < 500ms
|
||||
- **Button clicking**: < 500ms
|
||||
|
||||
### Success Rates
|
||||
- **Navigation**: 99%
|
||||
- **Field detection**: 95%
|
||||
- **Form filling**: 90%
|
||||
- **Button clicking**: 85%
|
||||
|
||||
## 🔧 Advanced Testing
|
||||
|
||||
### Custom Credentials Testing
|
||||
```bash
|
||||
python qubecare_voice_test.py
|
||||
# Choose option 1, then enter your credentials
|
||||
```
|
||||
|
||||
### Stress Testing
|
||||
```bash
|
||||
# Run multiple tests in sequence
|
||||
for i in {1..5}; do
|
||||
echo "Test run $i"
|
||||
python qubecare_voice_test.py
|
||||
sleep 5
|
||||
done
|
||||
```
|
||||
|
||||
### Voice Command Variations Testing
|
||||
Test different ways to express the same command:
|
||||
- "fill email with test@example.com"
|
||||
- "enter test@example.com in email"
|
||||
- "type test@example.com in email field"
|
||||
- "email test@example.com"
|
||||
|
||||
## 📝 Test Results Logging
|
||||
|
||||
All tests create log files:
|
||||
- `qubecare_live_test.log` - Detailed test execution logs
|
||||
- Console output - Real-time test progress
|
||||
|
||||
## 🚨 Known Limitations
|
||||
|
||||
1. **Page Load Timing** - Some pages may need longer load times
|
||||
2. **Dynamic Content** - SPAs with dynamic loading may need special handling
|
||||
3. **CAPTCHA** - Cannot handle CAPTCHA challenges
|
||||
4. **Two-Factor Auth** - Cannot handle 2FA automatically
|
||||
|
||||
## 🎯 Success Criteria
|
||||
|
||||
A successful test should demonstrate:
|
||||
- ✅ Successful navigation to QuBeCare
|
||||
- ✅ Accurate form field detection
|
||||
- ✅ Successful username entry via voice
|
||||
- ✅ Successful password entry via voice
|
||||
- ✅ Successful login button clicking
|
||||
- ✅ Appropriate error handling
|
||||
|
||||
## 📞 Support
|
||||
|
||||
If you encounter issues:
|
||||
1. Check the logs for detailed error messages
|
||||
2. Verify all prerequisites are met
|
||||
3. Try the interactive mode for manual testing
|
||||
4. Check Chrome MCP server console for errors
|
||||
|
||||
## 🎉 Next Steps
|
||||
|
||||
After successful testing:
|
||||
1. Try with real QuBeCare credentials (if available)
|
||||
2. Test with other websites
|
||||
3. Experiment with more complex voice commands
|
||||
4. Integrate with full LiveKit room for real voice interaction
|
||||
Reference in New Issue
Block a user