feat: migrate browser automation from Playwright MCP to CLI, fix headless setting

Major changes across 21 files (755 additions, 196 deletions):

Browser Automation Migration:
- Add versioned project migration system (prompts.py) with content-based
  detection and section-level regex replacement for coding/testing prompts
- Migrate STEP 5 (browser verification) and BROWSER AUTOMATION sections
  in coding prompt template to use playwright-cli commands
- Migrate STEP 2 and AVAILABLE TOOLS sections in testing prompt template
- Migration auto-runs at agent startup (autonomous_agent_demo.py), copies
  playwright-cli skill, scaffolds .playwright/cli.config.json, updates
  .gitignore, and stamps .migration_version file
- Add playwright-cli command validation to security allowlist (security.py)
  with tests for allowed subcommands and blocked eval/run-code

Headless Browser Setting Fix:
- Add _apply_playwright_headless() to process_manager.py that reads/updates
  .playwright/cli.config.json before agent subprocess launch
- Remove dead PLAYWRIGHT_HEADLESS env var that was never consumed
- Settings UI toggle now correctly controls visible browser window

Playwright CLI Auto-Install:
- Add ensurePlaywrightCli() to lib/cli.js for npm global entry point
- Add playwright-cli detection + npm install to start.bat, start.sh,
  start_ui.bat, start_ui.sh for all startup paths

Other Improvements:
- Add project folder path tooltip to ProjectSelector.tsx dropdown items
- Remove legacy Playwright MCP server configuration from client.py
- Update CLAUDE.md with playwright-cli skill documentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Auto
2026-02-11 13:37:03 +02:00
parent f285db1ad3
commit e9873a2642
21 changed files with 754 additions and 195 deletions

View File

@@ -86,24 +86,33 @@ Implement the chosen feature thoroughly:
**CRITICAL:** You MUST verify features through the actual UI.
Use browser automation tools:
Use `playwright-cli` for browser automation:
- Navigate to the app in a real browser
- Interact like a human user (click, type, scroll)
- Take screenshots at each step (use inline screenshots only -- do NOT save screenshot files to disk)
- Verify both functionality AND visual appearance
- Open the browser: `playwright-cli open http://localhost:PORT`
- Take a snapshot to see page elements: `playwright-cli snapshot`
- Read the snapshot YAML file to see element refs
- Click elements by ref: `playwright-cli click e5`
- Type text: `playwright-cli type "search query"`
- Fill form fields: `playwright-cli fill e3 "value"`
- Take screenshots: `playwright-cli screenshot`
- Read the screenshot file to verify visual appearance
- Check console errors: `playwright-cli console`
- Close browser when done: `playwright-cli close`
**Token-efficient workflow:** `playwright-cli screenshot` and `snapshot` save files
to `.playwright-cli/`. You will see a file link in the output. Read the file only
when you need to verify visual appearance or find element refs.
**DO:**
- Test through the UI with clicks and keyboard input
- Take screenshots to verify visual appearance (inline only, never save to disk)
- Check for console errors in browser
- Take screenshots and read them to verify visual appearance
- Check for console errors with `playwright-cli console`
- Verify complete user workflows end-to-end
- Always run `playwright-cli close` when finished testing
**DON'T:**
- Only test with curl commands (backend testing alone is insufficient)
- Use JavaScript evaluation to bypass UI (no shortcuts)
- Only test with curl commands
- Use JavaScript evaluation to bypass UI (`eval` and `run-code` are blocked)
- Skip visual verification
- Mark tests passing without thorough verification
@@ -145,7 +154,7 @@ Use the feature_mark_passing tool with feature_id=42
- Combine or consolidate features
- Reorder features
**ONLY MARK A FEATURE AS PASSING AFTER VERIFICATION WITH SCREENSHOTS.**
**ONLY MARK A FEATURE AS PASSING AFTER VERIFICATION WITH BROWSER AUTOMATION.**
### STEP 7: COMMIT YOUR PROGRESS
@@ -192,11 +201,15 @@ Before context fills up:
## BROWSER AUTOMATION
Use Playwright MCP tools (`browser_*`) for UI verification. Key tools: `navigate`, `click`, `type`, `fill_form`, `take_screenshot`, `console_messages`, `network_requests`. All tools have auto-wait built in.
Use `playwright-cli` commands for UI verification. Key commands: `open`, `goto`,
`snapshot`, `click`, `type`, `fill`, `screenshot`, `console`, `close`.
**Screenshot rule:** Always use inline mode (base64). NEVER save screenshots as files to disk.
**How it works:** `playwright-cli` uses a persistent browser daemon. `open` starts it,
subsequent commands interact via socket, `close` shuts it down. Screenshots and snapshots
save to `.playwright-cli/` -- read the files when you need to verify content.
Test like a human user with mouse and keyboard. Use `browser_console_messages` to detect errors. Don't bypass UI with JavaScript evaluation.
Test like a human user with mouse and keyboard. Use `playwright-cli console` to detect
JS errors. Don't bypass UI with JavaScript evaluation.
---

View File

@@ -31,26 +31,32 @@ For the feature returned:
1. Read and understand the feature's verification steps
2. Navigate to the relevant part of the application
3. Execute each verification step using browser automation
4. Take screenshots to document the verification (inline only -- do NOT save to disk)
4. Take screenshots and read them to verify visual appearance
5. Check for console errors
Use browser automation tools:
### Browser Automation (Playwright CLI)
**Navigation & Screenshots:**
- browser_navigate - Navigate to a URL
- browser_take_screenshot - Capture screenshot (inline mode only -- never save to disk)
- browser_snapshot - Get accessibility tree snapshot
- `playwright-cli open <url>` - Open browser and navigate
- `playwright-cli goto <url>` - Navigate to URL
- `playwright-cli screenshot` - Save screenshot to `.playwright-cli/`
- `playwright-cli snapshot` - Save page snapshot with element refs to `.playwright-cli/`
**Element Interaction:**
- browser_click - Click elements
- browser_type - Type text into editable elements
- browser_fill_form - Fill multiple form fields
- browser_select_option - Select dropdown options
- browser_press_key - Press keyboard keys
- `playwright-cli click <ref>` - Click elements (ref from snapshot)
- `playwright-cli type <text>` - Type text
- `playwright-cli fill <ref> <text>` - Fill form fields
- `playwright-cli select <ref> <val>` - Select dropdown
- `playwright-cli press <key>` - Keyboard input
**Debugging:**
- browser_console_messages - Get browser console output (check for errors)
- browser_network_requests - Monitor API calls
- `playwright-cli console` - Check for JS errors
- `playwright-cli network` - Monitor API calls
**Cleanup:**
- `playwright-cli close` - Close browser when done (ALWAYS do this)
**Note:** Screenshots and snapshots save to files. Read the file to see the content.
### STEP 3: HANDLE RESULTS
@@ -79,7 +85,7 @@ A regression has been introduced. You MUST fix it:
4. **Verify the fix:**
- Run through all verification steps again
- Take screenshots confirming the fix (inline only, never save to disk)
- Take screenshots and read them to confirm the fix
5. **Mark as passing after fix:**
```
@@ -98,7 +104,7 @@ A regression has been introduced. You MUST fix it:
---
## AVAILABLE MCP TOOLS
## AVAILABLE TOOLS
### Feature Management
- `feature_get_stats` - Get progress overview (passing/in_progress/total counts)
@@ -106,19 +112,17 @@ A regression has been introduced. You MUST fix it:
- `feature_mark_failing` - Mark a feature as failing (when you find a regression)
- `feature_mark_passing` - Mark a feature as passing (after fixing a regression)
### Browser Automation (Playwright)
All interaction tools have **built-in auto-wait** -- no manual timeouts needed.
- `browser_navigate` - Navigate to URL
- `browser_take_screenshot` - Capture screenshot (inline only, never save to disk)
- `browser_snapshot` - Get accessibility tree
- `browser_click` - Click elements
- `browser_type` - Type text
- `browser_fill_form` - Fill form fields
- `browser_select_option` - Select dropdown
- `browser_press_key` - Keyboard input
- `browser_console_messages` - Check for JS errors
- `browser_network_requests` - Monitor API calls
### Browser Automation (Playwright CLI)
Use `playwright-cli` commands for browser interaction. Key commands:
- `playwright-cli open <url>` - Open browser
- `playwright-cli goto <url>` - Navigate to URL
- `playwright-cli screenshot` - Take screenshot (saved to `.playwright-cli/`)
- `playwright-cli snapshot` - Get page snapshot with element refs
- `playwright-cli click <ref>` - Click element
- `playwright-cli type <text>` - Type text
- `playwright-cli fill <ref> <text>` - Fill form field
- `playwright-cli console` - Check for JS errors
- `playwright-cli close` - Close browser (always do this when done)
---