mirror of
https://github.com/leonvanzyl/autocoder.git
synced 2026-03-17 02:43:09 +00:00
feat: migrate browser automation from Playwright MCP to CLI, fix headless setting
Major changes across 21 files (755 additions, 196 deletions): Browser Automation Migration: - Add versioned project migration system (prompts.py) with content-based detection and section-level regex replacement for coding/testing prompts - Migrate STEP 5 (browser verification) and BROWSER AUTOMATION sections in coding prompt template to use playwright-cli commands - Migrate STEP 2 and AVAILABLE TOOLS sections in testing prompt template - Migration auto-runs at agent startup (autonomous_agent_demo.py), copies playwright-cli skill, scaffolds .playwright/cli.config.json, updates .gitignore, and stamps .migration_version file - Add playwright-cli command validation to security allowlist (security.py) with tests for allowed subcommands and blocked eval/run-code Headless Browser Setting Fix: - Add _apply_playwright_headless() to process_manager.py that reads/updates .playwright/cli.config.json before agent subprocess launch - Remove dead PLAYWRIGHT_HEADLESS env var that was never consumed - Settings UI toggle now correctly controls visible browser window Playwright CLI Auto-Install: - Add ensurePlaywrightCli() to lib/cli.js for npm global entry point - Add playwright-cli detection + npm install to start.bat, start.sh, start_ui.bat, start_ui.sh for all startup paths Other Improvements: - Add project folder path tooltip to ProjectSelector.tsx dropdown items - Remove legacy Playwright MCP server configuration from client.py - Update CLAUDE.md with playwright-cli skill documentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -86,24 +86,33 @@ Implement the chosen feature thoroughly:
|
||||
|
||||
**CRITICAL:** You MUST verify features through the actual UI.
|
||||
|
||||
Use browser automation tools:
|
||||
Use `playwright-cli` for browser automation:
|
||||
|
||||
- Navigate to the app in a real browser
|
||||
- Interact like a human user (click, type, scroll)
|
||||
- Take screenshots at each step (use inline screenshots only -- do NOT save screenshot files to disk)
|
||||
- Verify both functionality AND visual appearance
|
||||
- Open the browser: `playwright-cli open http://localhost:PORT`
|
||||
- Take a snapshot to see page elements: `playwright-cli snapshot`
|
||||
- Read the snapshot YAML file to see element refs
|
||||
- Click elements by ref: `playwright-cli click e5`
|
||||
- Type text: `playwright-cli type "search query"`
|
||||
- Fill form fields: `playwright-cli fill e3 "value"`
|
||||
- Take screenshots: `playwright-cli screenshot`
|
||||
- Read the screenshot file to verify visual appearance
|
||||
- Check console errors: `playwright-cli console`
|
||||
- Close browser when done: `playwright-cli close`
|
||||
|
||||
**Token-efficient workflow:** `playwright-cli screenshot` and `snapshot` save files
|
||||
to `.playwright-cli/`. You will see a file link in the output. Read the file only
|
||||
when you need to verify visual appearance or find element refs.
|
||||
|
||||
**DO:**
|
||||
|
||||
- Test through the UI with clicks and keyboard input
|
||||
- Take screenshots to verify visual appearance (inline only, never save to disk)
|
||||
- Check for console errors in browser
|
||||
- Take screenshots and read them to verify visual appearance
|
||||
- Check for console errors with `playwright-cli console`
|
||||
- Verify complete user workflows end-to-end
|
||||
- Always run `playwright-cli close` when finished testing
|
||||
|
||||
**DON'T:**
|
||||
|
||||
- Only test with curl commands (backend testing alone is insufficient)
|
||||
- Use JavaScript evaluation to bypass UI (no shortcuts)
|
||||
- Only test with curl commands
|
||||
- Use JavaScript evaluation to bypass UI (`eval` and `run-code` are blocked)
|
||||
- Skip visual verification
|
||||
- Mark tests passing without thorough verification
|
||||
|
||||
@@ -145,7 +154,7 @@ Use the feature_mark_passing tool with feature_id=42
|
||||
- Combine or consolidate features
|
||||
- Reorder features
|
||||
|
||||
**ONLY MARK A FEATURE AS PASSING AFTER VERIFICATION WITH SCREENSHOTS.**
|
||||
**ONLY MARK A FEATURE AS PASSING AFTER VERIFICATION WITH BROWSER AUTOMATION.**
|
||||
|
||||
### STEP 7: COMMIT YOUR PROGRESS
|
||||
|
||||
@@ -192,11 +201,15 @@ Before context fills up:
|
||||
|
||||
## BROWSER AUTOMATION
|
||||
|
||||
Use Playwright MCP tools (`browser_*`) for UI verification. Key tools: `navigate`, `click`, `type`, `fill_form`, `take_screenshot`, `console_messages`, `network_requests`. All tools have auto-wait built in.
|
||||
Use `playwright-cli` commands for UI verification. Key commands: `open`, `goto`,
|
||||
`snapshot`, `click`, `type`, `fill`, `screenshot`, `console`, `close`.
|
||||
|
||||
**Screenshot rule:** Always use inline mode (base64). NEVER save screenshots as files to disk.
|
||||
**How it works:** `playwright-cli` uses a persistent browser daemon. `open` starts it,
|
||||
subsequent commands interact via socket, `close` shuts it down. Screenshots and snapshots
|
||||
save to `.playwright-cli/` -- read the files when you need to verify content.
|
||||
|
||||
Test like a human user with mouse and keyboard. Use `browser_console_messages` to detect errors. Don't bypass UI with JavaScript evaluation.
|
||||
Test like a human user with mouse and keyboard. Use `playwright-cli console` to detect
|
||||
JS errors. Don't bypass UI with JavaScript evaluation.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -31,26 +31,32 @@ For the feature returned:
|
||||
1. Read and understand the feature's verification steps
|
||||
2. Navigate to the relevant part of the application
|
||||
3. Execute each verification step using browser automation
|
||||
4. Take screenshots to document the verification (inline only -- do NOT save to disk)
|
||||
4. Take screenshots and read them to verify visual appearance
|
||||
5. Check for console errors
|
||||
|
||||
Use browser automation tools:
|
||||
### Browser Automation (Playwright CLI)
|
||||
|
||||
**Navigation & Screenshots:**
|
||||
- browser_navigate - Navigate to a URL
|
||||
- browser_take_screenshot - Capture screenshot (inline mode only -- never save to disk)
|
||||
- browser_snapshot - Get accessibility tree snapshot
|
||||
- `playwright-cli open <url>` - Open browser and navigate
|
||||
- `playwright-cli goto <url>` - Navigate to URL
|
||||
- `playwright-cli screenshot` - Save screenshot to `.playwright-cli/`
|
||||
- `playwright-cli snapshot` - Save page snapshot with element refs to `.playwright-cli/`
|
||||
|
||||
**Element Interaction:**
|
||||
- browser_click - Click elements
|
||||
- browser_type - Type text into editable elements
|
||||
- browser_fill_form - Fill multiple form fields
|
||||
- browser_select_option - Select dropdown options
|
||||
- browser_press_key - Press keyboard keys
|
||||
- `playwright-cli click <ref>` - Click elements (ref from snapshot)
|
||||
- `playwright-cli type <text>` - Type text
|
||||
- `playwright-cli fill <ref> <text>` - Fill form fields
|
||||
- `playwright-cli select <ref> <val>` - Select dropdown
|
||||
- `playwright-cli press <key>` - Keyboard input
|
||||
|
||||
**Debugging:**
|
||||
- browser_console_messages - Get browser console output (check for errors)
|
||||
- browser_network_requests - Monitor API calls
|
||||
- `playwright-cli console` - Check for JS errors
|
||||
- `playwright-cli network` - Monitor API calls
|
||||
|
||||
**Cleanup:**
|
||||
- `playwright-cli close` - Close browser when done (ALWAYS do this)
|
||||
|
||||
**Note:** Screenshots and snapshots save to files. Read the file to see the content.
|
||||
|
||||
### STEP 3: HANDLE RESULTS
|
||||
|
||||
@@ -79,7 +85,7 @@ A regression has been introduced. You MUST fix it:
|
||||
|
||||
4. **Verify the fix:**
|
||||
- Run through all verification steps again
|
||||
- Take screenshots confirming the fix (inline only, never save to disk)
|
||||
- Take screenshots and read them to confirm the fix
|
||||
|
||||
5. **Mark as passing after fix:**
|
||||
```
|
||||
@@ -98,7 +104,7 @@ A regression has been introduced. You MUST fix it:
|
||||
|
||||
---
|
||||
|
||||
## AVAILABLE MCP TOOLS
|
||||
## AVAILABLE TOOLS
|
||||
|
||||
### Feature Management
|
||||
- `feature_get_stats` - Get progress overview (passing/in_progress/total counts)
|
||||
@@ -106,19 +112,17 @@ A regression has been introduced. You MUST fix it:
|
||||
- `feature_mark_failing` - Mark a feature as failing (when you find a regression)
|
||||
- `feature_mark_passing` - Mark a feature as passing (after fixing a regression)
|
||||
|
||||
### Browser Automation (Playwright)
|
||||
All interaction tools have **built-in auto-wait** -- no manual timeouts needed.
|
||||
|
||||
- `browser_navigate` - Navigate to URL
|
||||
- `browser_take_screenshot` - Capture screenshot (inline only, never save to disk)
|
||||
- `browser_snapshot` - Get accessibility tree
|
||||
- `browser_click` - Click elements
|
||||
- `browser_type` - Type text
|
||||
- `browser_fill_form` - Fill form fields
|
||||
- `browser_select_option` - Select dropdown
|
||||
- `browser_press_key` - Keyboard input
|
||||
- `browser_console_messages` - Check for JS errors
|
||||
- `browser_network_requests` - Monitor API calls
|
||||
### Browser Automation (Playwright CLI)
|
||||
Use `playwright-cli` commands for browser interaction. Key commands:
|
||||
- `playwright-cli open <url>` - Open browser
|
||||
- `playwright-cli goto <url>` - Navigate to URL
|
||||
- `playwright-cli screenshot` - Take screenshot (saved to `.playwright-cli/`)
|
||||
- `playwright-cli snapshot` - Get page snapshot with element refs
|
||||
- `playwright-cli click <ref>` - Click element
|
||||
- `playwright-cli type <text>` - Type text
|
||||
- `playwright-cli fill <ref> <text>` - Fill form field
|
||||
- `playwright-cli console` - Check for JS errors
|
||||
- `playwright-cli close` - Close browser (always do this when done)
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user