diff --git a/docs/explanation/features/tea-overview.md b/docs/explanation/features/tea-overview.md
index 11e4c8f3..cacd25e3 100644
--- a/docs/explanation/features/tea-overview.md
+++ b/docs/explanation/features/tea-overview.md
@@ -195,20 +195,295 @@ Epic/Release Gate β TEA: *nfr-assess, *trace Phase 2 (release decision)
**Note**: `*trace` is a two-phase workflow: Phase 1 (traceability) + Phase 2 (gate decision). This reduces cognitive load while maintaining natural workflow.
+### Why TEA Requires Its Own Knowledge Base
+
+TEA uniquely requires:
+
+- **Extensive domain knowledge**: 30+ fragments covering test patterns, CI/CD, fixtures, quality practices, and optional playwright-utils integration
+- **Cross-cutting concerns**: Domain-specific testing patterns that apply across all BMad projects (vs project-specific artifacts like PRDs/stories)
+- **Optional integrations**: MCP capabilities (exploratory, verification) and playwright-utils support
+
+This architecture enables TEA to maintain consistent, production-ready testing patterns across all BMad projects while operating across multiple development phases.
+
+---
+
+
+## High-Level Cheat Sheets
+
+These cheat sheets map TEA workflows to the **BMad Method and Enterprise tracks** across the **4-Phase Methodology** (Phase 1: Analysis, Phase 2: Planning, Phase 3: Solutioning, Phase 4: Implementation).
+
+**Note:** Quick Flow projects typically don't require TEA (covered in Overview). These cheat sheets focus on BMad Method and Enterprise tracks where TEA adds value.
+
+**Legend for Track Deltas:**
+
+- β = New workflow or phase added (doesn't exist in baseline)
+- π = Modified focus (same workflow, different emphasis or purpose)
+- π¦ = Additional output or archival requirement
+
+### Greenfield - BMad Method (Simple/Standard Work)
+
+**Planning Track:** BMad Method (PRD + Architecture)
+**Use Case:** New projects with standard complexity
+
+| Workflow Stage | Test Architect | Dev / Team | Outputs |
+| -------------------------- | ----------------------------------------------------------------- | ----------------------------------------------------------------------------------- | ---------------------------------------------------------- |
+| **Phase 1**: Discovery | - | Analyst `*product-brief` (optional) | `product-brief.md` |
+| **Phase 2**: Planning | - | PM `*prd` (creates PRD with FRs/NFRs) | PRD with functional/non-functional requirements |
+| **Phase 3**: Solutioning | Run `*framework`, `*ci` AFTER architecture and epic creation | Architect `*architecture`, `*create-epics-and-stories`, `*implementation-readiness` | Architecture, epics/stories, test scaffold, CI pipeline |
+| **Phase 4**: Sprint Start | - | SM `*sprint-planning` | Sprint status file with all epics and stories |
+| **Phase 4**: Epic Planning | Run `*test-design` for THIS epic (per-epic test plan) | Review epic scope | `test-design-epic-N.md` with risk assessment and test plan |
+| **Phase 4**: Story Dev | (Optional) `*atdd` before dev, then `*automate` after | SM `*create-story`, DEV implements | Tests, story implementation |
+| **Phase 4**: Story Review | Execute `*test-review` (optional), re-run `*trace` | Address recommendations, update code/tests | Quality report, refreshed coverage matrix |
+| **Phase 4**: Release Gate | (Optional) `*test-review` for final audit, Run `*trace` (Phase 2) | Confirm Definition of Done, share release notes | Quality audit, Gate YAML + release summary |
+
+
+Execution Notes
+
+- Run `*framework` only once per repo or when modern harness support is missing.
+- **Phase 3 (Solutioning)**: After architecture is complete, run `*framework` and `*ci` to setup test infrastructure based on architectural decisions.
+- **Phase 4 starts**: After solutioning is complete, sprint planning loads all epics.
+- **`*test-design` runs per-epic**: At the beginning of working on each epic, run `*test-design` to create a test plan for THAT specific epic/feature. Output: `test-design-epic-N.md`.
+- Use `*atdd` before coding when the team can adopt ATDD; share its checklist with the dev agent.
+- Post-implementation, keep `*trace` current, expand coverage with `*automate`, optionally review test quality with `*test-review`. For release gate, run `*trace` with Phase 2 enabled to get deployment decision.
+- Use `*test-review` after `*atdd` to validate generated tests, after `*automate` to ensure regression quality, or before gate for final audit.
+- Clarification: `*test-review` is optional and only audits existing tests; run it after `*atdd` or `*automate` when you want a quality review, not as a required step.
+- Clarification: `*atdd` outputs are not auto-consumed; share the ATDD doc/tests with the dev workflow. `*trace` does not run `*atdd`βit evaluates existing artifacts for coverage and gate readiness.
+- Clarification: `*ci` is a one-time setup; recommended early (Phase 3 or before feature work), but it can be done later if it was skipped.
+
+
+
+
+Worked Example β βNova CRMβ Greenfield Feature
+
+1. **Planning (Phase 2):** Analyst runs `*product-brief`; PM executes `*prd` to produce PRD with FRs/NFRs.
+2. **Solutioning (Phase 3):** Architect completes `*architecture` for the new module; `*create-epics-and-stories` generates epics/stories based on architecture; TEA sets up test infrastructure via `*framework` and `*ci` based on architectural decisions; gate check validates planning completeness.
+3. **Sprint Start (Phase 4):** Scrum Master runs `*sprint-planning` to load all epics into sprint status.
+4. **Epic 1 Planning (Phase 4):** TEA runs `*test-design` to create test plan for Epic 1, producing `test-design-epic-1.md` with risk assessment.
+5. **Story Implementation (Phase 4):** For each story in Epic 1, SM generates story via `*create-story`; TEA optionally runs `*atdd`; Dev implements with guidance from failing tests.
+6. **Post-Dev (Phase 4):** TEA runs `*automate`, optionally `*test-review` to audit test quality, re-runs `*trace` to refresh coverage.
+7. **Release Gate:** TEA runs `*trace` with Phase 2 enabled to generate gate decision.
+
+
+
+### Brownfield - BMad Method or Enterprise (Simple or Complex)
+
+**Planning Tracks:** BMad Method or Enterprise Method
+**Use Case:** Existing codebases - simple additions (BMad Method) or complex enterprise requirements (Enterprise Method)
+
+**π Brownfield Deltas from Greenfield:**
+
+- β Documentation (Prerequisite) - Document existing codebase if undocumented
+- β Phase 2: `*trace` - Baseline existing test coverage before planning
+- π Phase 4: `*test-design` - Focus on regression hotspots and brownfield risks
+- π Phase 4: Story Review - May include `*nfr-assess` if not done earlier
+
+| Workflow Stage | Test Architect | Dev / Team | Outputs |
+| --------------------------------- | --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
+| **Documentation**: Prerequisite β | - | Analyst `*document-project` (if undocumented) | Comprehensive project documentation |
+| **Phase 1**: Discovery | - | Analyst/PM/Architect rerun planning workflows | Updated planning artifacts in `{output_folder}` |
+| **Phase 2**: Planning | Run β `*trace` (baseline coverage) | PM `*prd` (creates PRD with FRs/NFRs) | PRD with FRs/NFRs, β coverage baseline |
+| **Phase 3**: Solutioning | Run `*framework`, `*ci` AFTER architecture and epic creation | Architect `*architecture`, `*create-epics-and-stories`, `*implementation-readiness` | Architecture, epics/stories, test framework, CI pipeline |
+| **Phase 4**: Sprint Start | - | SM `*sprint-planning` | Sprint status file with all epics and stories |
+| **Phase 4**: Epic Planning | Run `*test-design` for THIS epic π (regression hotspots) | Review epic scope and brownfield risks | `test-design-epic-N.md` with brownfield risk assessment and mitigation |
+| **Phase 4**: Story Dev | (Optional) `*atdd` before dev, then `*automate` after | SM `*create-story`, DEV implements | Tests, story implementation |
+| **Phase 4**: Story Review | Apply `*test-review` (optional), re-run `*trace`, β `*nfr-assess` if needed | Resolve gaps, update docs/tests | Quality report, refreshed coverage matrix, NFR report |
+| **Phase 4**: Release Gate | (Optional) `*test-review` for final audit, Run `*trace` (Phase 2) | Capture sign-offs, share release notes | Quality audit, Gate YAML + release summary |
+
+
+Execution Notes
+
+- Lead with `*trace` during Planning (Phase 2) to baseline existing test coverage before architecture work begins.
+- **Phase 3 (Solutioning)**: After architecture is complete, run `*framework` and `*ci` to modernize test infrastructure. For brownfield, framework may need to integrate with or replace existing test setup.
+- **Phase 4 starts**: After solutioning is complete and sprint planning loads all epics.
+- **`*test-design` runs per-epic**: At the beginning of working on each epic, run `*test-design` to identify regression hotspots, integration risks, and mitigation strategies for THAT specific epic/feature. Output: `test-design-epic-N.md`.
+- Use `*atdd` when stories benefit from ATDD; otherwise proceed to implementation and rely on post-dev automation.
+- After development, expand coverage with `*automate`, optionally review test quality with `*test-review`, re-run `*trace` (Phase 2 for gate decision). Run `*nfr-assess` now if non-functional risks weren't addressed earlier.
+- Use `*test-review` to validate existing brownfield tests or audit new tests before gate.
+
+
+
+
+Worked Example β βAtlas Paymentsβ Brownfield Story
+
+1. **Planning (Phase 2):** PM executes `*prd` to create PRD with FRs/NFRs; TEA runs `*trace` to baseline existing coverage.
+2. **Solutioning (Phase 3):** Architect triggers `*architecture` capturing legacy payment flows and integration architecture; `*create-epics-and-stories` generates Epic 1 (Payment Processing) based on architecture; TEA sets up `*framework` and `*ci` based on architectural decisions; gate check validates planning.
+3. **Sprint Start (Phase 4):** Scrum Master runs `*sprint-planning` to load Epic 1 into sprint status.
+4. **Epic 1 Planning (Phase 4):** TEA runs `*test-design` for Epic 1 (Payment Processing), producing `test-design-epic-1.md` that flags settlement edge cases, regression hotspots, and mitigation plans.
+5. **Story Implementation (Phase 4):** For each story in Epic 1, SM generates story via `*create-story`; TEA runs `*atdd` producing failing Playwright specs; Dev implements with guidance from tests and checklist.
+6. **Post-Dev (Phase 4):** TEA applies `*automate`, optionally `*test-review` to audit test quality, re-runs `*trace` to refresh coverage.
+7. **Release Gate:** TEA performs `*nfr-assess` to validate SLAs, runs `*trace` with Phase 2 enabled to generate gate decision (PASS/CONCERNS/FAIL).
+
+
+
+### Greenfield - Enterprise Method (Enterprise/Compliance Work)
+
+**Planning Track:** Enterprise Method (BMad Method + extended security/devops/test strategies)
+**Use Case:** New enterprise projects with compliance, security, or complex regulatory requirements
+
+**π’ Enterprise Deltas from BMad Method:**
+
+- β Phase 1: `*research` - Domain and compliance research (recommended)
+- β Phase 2: `*nfr-assess` - Capture NFR requirements early (security/performance/reliability)
+- π Phase 4: `*test-design` - Enterprise focus (compliance, security architecture alignment)
+- π¦ Release Gate - Archive artifacts and compliance evidence for audits
+
+| Workflow Stage | Test Architect | Dev / Team | Outputs |
+| -------------------------- | ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | ------------------------------------------------------------------ |
+| **Phase 1**: Discovery | - | Analyst β `*research`, `*product-brief` | Domain research, compliance analysis, product brief |
+| **Phase 2**: Planning | Run β `*nfr-assess` | PM `*prd` (creates PRD with FRs/NFRs), UX `*create-ux-design` | Enterprise PRD with FRs/NFRs, UX design, β NFR documentation |
+| **Phase 3**: Solutioning | Run `*framework`, `*ci` AFTER architecture and epic creation | Architect `*architecture`, `*create-epics-and-stories`, `*implementation-readiness` | Architecture, epics/stories, test framework, CI pipeline |
+| **Phase 4**: Sprint Start | - | SM `*sprint-planning` | Sprint plan with all epics |
+| **Phase 4**: Epic Planning | Run `*test-design` for THIS epic π (compliance focus) | Review epic scope and compliance requirements | `test-design-epic-N.md` with security/performance/compliance focus |
+| **Phase 4**: Story Dev | (Optional) `*atdd`, `*automate`, `*test-review`, `*trace` per story | SM `*create-story`, DEV implements | Tests, fixtures, quality reports, coverage matrices |
+| **Phase 4**: Release Gate | Final `*test-review` audit, Run `*trace` (Phase 2), π¦ archive artifacts | Capture sign-offs, π¦ compliance evidence | Quality audit, updated assessments, gate YAML, π¦ audit trail |
+
+
+Execution Notes
+
+- `*nfr-assess` runs early in Planning (Phase 2) to capture compliance, security, and performance requirements upfront.
+- **Phase 3 (Solutioning)**: After architecture is complete, run `*framework` and `*ci` with enterprise-grade configurations (selective testing, burn-in jobs, caching, notifications).
+- **Phase 4 starts**: After solutioning is complete and sprint planning loads all epics.
+- **`*test-design` runs per-epic**: At the beginning of working on each epic, run `*test-design` to create an enterprise-focused test plan for THAT specific epic, ensuring alignment with security architecture, performance targets, and compliance requirements. Output: `test-design-epic-N.md`.
+- Use `*atdd` for stories when feasible so acceptance tests can lead implementation.
+- Use `*test-review` per story or sprint to maintain quality standards and ensure compliance with testing best practices.
+- Prior to release, rerun coverage (`*trace`, `*automate`), perform final quality audit with `*test-review`, and formalize the decision with `*trace` Phase 2 (gate decision); archive artifacts for compliance audits.
+
+
+
+
+Worked Example β βHelios Ledgerβ Enterprise Release
+
+1. **Planning (Phase 2):** Analyst runs `*research` and `*product-brief`; PM completes `*prd` creating PRD with FRs/NFRs; TEA runs `*nfr-assess` to establish NFR targets.
+2. **Solutioning (Phase 3):** Architect completes `*architecture` with enterprise considerations; `*create-epics-and-stories` generates epics/stories based on architecture; TEA sets up `*framework` and `*ci` with enterprise-grade configurations based on architectural decisions; gate check validates planning completeness.
+3. **Sprint Start (Phase 4):** Scrum Master runs `*sprint-planning` to load all epics into sprint status.
+4. **Per-Epic (Phase 4):** For each epic, TEA runs `*test-design` to create epic-specific test plan (e.g., `test-design-epic-1.md`, `test-design-epic-2.md`) with compliance-focused risk assessment.
+5. **Per-Story (Phase 4):** For each story, TEA uses `*atdd`, `*automate`, `*test-review`, and `*trace`; Dev teams iterate on the findings.
+6. **Release Gate:** TEA re-checks coverage, performs final quality audit with `*test-review`, and logs the final gate decision via `*trace` Phase 2, archiving artifacts for compliance.
+
+
+
---
## TEA Command Catalog
-| Command | Primary Outputs | Notes |
-| -------------- | --------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
-| `*framework` | Playwright/Cypress scaffold, `.env.example`, `.nvmrc`, sample specs | Use when no production-ready harness exists |
-| `*ci` | CI workflow, selective test scripts, secrets checklist | Platform-aware (GitHub Actions default) |
-| `*test-design` | Combined risk assessment, mitigation plan, and coverage strategy | Risk scoring + optional exploratory mode |
-| `*atdd` | Failing acceptance tests + implementation checklist | TDD red phase + optional recording mode |
-| `*automate` | Prioritized specs, fixtures, README/script updates, DoD summary | Optional healing/recording, avoid duplicate coverage |
-| `*test-review` | Test quality review report with 0-100 score, violations, fixes | Reviews tests against knowledge base patterns |
-| `*nfr-assess` | NFR assessment report with actions | Focus on security/performance/reliability |
-| `*trace` | Phase 1: Coverage matrix, recommendations. Phase 2: Gate decision (PASS/CONCERNS/FAIL/WAIVED) | Two-phase workflow: traceability + gate decision |
+| Command | Primary Outputs | Notes | With Playwright MCP Enhancements |
+| -------------- | --------------------------------------------------------------------------------------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
+| `*framework` | Playwright/Cypress scaffold, `.env.example`, `.nvmrc`, sample specs | Use when no production-ready harness exists | - |
+| `*ci` | CI workflow, selective test scripts, secrets checklist | Platform-aware (GitHub Actions default) | - |
+| `*test-design` | Combined risk assessment, mitigation plan, and coverage strategy | Risk scoring + optional exploratory mode | **+ Exploratory**: Interactive UI discovery with browser automation (uncover actual functionality) |
+| `*atdd` | Failing acceptance tests + implementation checklist | TDD red phase + optional recording mode | **+ Recording**: AI generation verified with live browser (accurate selectors from real DOM) |
+| `*automate` | Prioritized specs, fixtures, README/script updates, DoD summary | Optional healing/recording, avoid duplicate coverage | **+ Healing**: Pattern fixes enhanced with visual debugging + **+ Recording**: AI verified with live browser |
+| `*test-review` | Test quality review report with 0-100 score, violations, fixes | Reviews tests against knowledge base patterns | - |
+| `*nfr-assess` | NFR assessment report with actions | Focus on security/performance/reliability | - |
+| `*trace` | Phase 1: Coverage matrix, recommendations. Phase 2: Gate decision (PASS/CONCERNS/FAIL/WAIVED) | Two-phase workflow: traceability + gate decision | - |
+
+---
+
+## Playwright Utils Integration
+
+TEA optionally integrates with `@seontechnologies/playwright-utils`, an open-source library providing fixture-based utilities for Playwright tests. This integration enhances TEA's test generation and review workflows with production-ready patterns.
+
+
+Installation & Configuration
+
+**Package**: `@seontechnologies/playwright-utils` ([npm](https://www.npmjs.com/package/@seontechnologies/playwright-utils) | [GitHub](https://github.com/seontechnologies/playwright-utils))
+
+**Install**: `npm install -D @seontechnologies/playwright-utils`
+
+**Enable during BMAD installation** by answering "Yes" when prompted, or manually set `tea_use_playwright_utils: true` in `_bmad/bmm/config.yaml`.
+
+**To disable**: Set `tea_use_playwright_utils: false` in `_bmad/bmm/config.yaml`.
+
+
+
+
+How Playwright Utils Enhances TEA Workflows
+
+1. `*framework`:
+ - Default: Basic Playwright scaffold
+ - **+ playwright-utils**: Scaffold with api-request, network-recorder, auth-session, burn-in, network-error-monitor fixtures pre-configured
+
+ Benefit: Production-ready patterns from day one
+
+2. `*automate`, `*atdd`:
+ - Default: Standard test patterns
+ - **+ playwright-utils**: Tests using api-request (schema validation), intercept-network-call (mocking), recurse (polling), log (structured logging), file-utils (CSV/PDF)
+
+ Benefit: Advanced patterns without boilerplate
+
+3. `*test-review`:
+ - Default: Reviews against core knowledge base (22 fragments)
+ - **+ playwright-utils**: Reviews against expanded knowledge base (33 fragments: 22 core + 11 playwright-utils)
+
+ Benefit: Reviews include fixture composition, auth patterns, network recording best practices
+
+4. `*ci`:
+ - Default: Standard CI workflow
+ - **+ playwright-utils**: CI workflow with burn-in script (smart test selection) and network-error-monitor integration
+
+ Benefit: Faster CI feedback, HTTP error detection
+
+**Utilities available** (10 total): api-request, network-recorder, auth-session, intercept-network-call, recurse, log, file-utils, burn-in, network-error-monitor, fixtures-composition
+
+
+
+---
+
+## Playwright MCP Enhancements
+
+TEA can leverage Playwright MCP servers to enhance test generation with live browser verification. MCP provides interactive capabilities on top of TEA's default AI-based approach.
+
+
+MCP Server Configuration
+
+**Two Playwright MCP servers** (actively maintained, continuously updated):
+
+- `playwright` - Browser automation (`npx @playwright/mcp@latest`)
+- `playwright-test` - Test runner with failure analysis (`npx playwright run-test-mcp-server`)
+
+**Config example**:
+
+```json
+{
+ "mcpServers": {
+ "playwright": {
+ "command": "npx",
+ "args": ["@playwright/mcp@latest"]
+ },
+ "playwright-test": {
+ "command": "npx",
+ "args": ["playwright", "run-test-mcp-server"]
+ }
+ }
+}
+```
+
+**To disable**: Set `tea_use_mcp_enhancements: false` in `_bmad/bmm/config.yaml` OR remove MCPs from IDE config.
+
+
+
+
+How MCP Enhances TEA Workflows
+
+1. `*test-design`:
+ - Default: Analysis + documentation
+ - **+ MCP**: Interactive UI discovery with `browser_navigate`, `browser_click`, `browser_snapshot`, behavior observation
+
+ Benefit: Discover actual functionality, edge cases, undocumented features
+
+2. `*atdd`, `*automate`:
+ - Default: Infers selectors and interactions from requirements and knowledge fragments
+ - **+ MCP**: Generates tests **then** verifies with `generator_setup_page`, `browser_*` tools, validates against live app
+
+ Benefit: Accurate selectors from real DOM, verified behavior, refined test code
+
+3. `*automate` (healing mode):
+ - Default: Pattern-based fixes from error messages + knowledge fragments
+ - **+ MCP**: Pattern fixes **enhanced with** `browser_snapshot`, `browser_console_messages`, `browser_network_requests`, `browser_generate_locator`
+
+ Benefit: Visual failure context, live DOM inspection, root cause discovery
+
+
---
diff --git a/src/modules/bmm/agents/tea.agent.yaml b/src/modules/bmm/agents/tea.agent.yaml
index 2ff25637..fa42b41b 100644
--- a/src/modules/bmm/agents/tea.agent.yaml
+++ b/src/modules/bmm/agents/tea.agent.yaml
@@ -12,15 +12,17 @@ agent:
persona:
role: Master Test Architect
- identity: Test architect specializing in CI/CD, automated frameworks, and scalable quality gates.
+ identity: Test architect specializing in API testing, backend services, UI automation, CI/CD pipelines, and scalable quality gates. Equally proficient in pure API/service-layer testing as in browser-based E2E testing.
communication_style: "Blends data with gut instinct. 'Strong opinions, weakly held' is their mantra. Speaks in risk calculations and impact assessments."
principles: |
- Risk-based testing - depth scales with impact
- Quality gates backed by data
- - Tests mirror usage patterns
+ - Tests mirror usage patterns (API, UI, or both)
- Flakiness is critical technical debt
- Tests first AI implements suite validates
- Calculate risk vs value for every testing decision
+ - Prefer lower test levels (unit > integration > E2E) when possible
+ - API tests are first-class citizens, not just UI support
critical_actions:
- "Consult {project-root}/_bmad/bmm/testarch/tea-index.csv to select knowledge fragments under knowledge/ and load only the files needed for the current task"
@@ -39,7 +41,7 @@ agent:
- trigger: AT or fuzzy match on atdd
workflow: "{project-root}/_bmad/bmm/workflows/testarch/atdd/workflow.yaml"
- description: "[AT] Generate E2E tests first, before starting implementation"
+ description: "[AT] Generate API and/or E2E tests first, before starting implementation"
- trigger: TA or fuzzy match on test-automate
workflow: "{project-root}/_bmad/bmm/workflows/testarch/automate/workflow.yaml"
diff --git a/src/modules/bmm/module.yaml b/src/modules/bmm/module.yaml
index e6b8dee1..b5aaf889 100644
--- a/src/modules/bmm/module.yaml
+++ b/src/modules/bmm/module.yaml
@@ -45,7 +45,7 @@ project_knowledge: # Artifacts from research, document-project output, other lon
result: "{project-root}/{value}"
tea_use_mcp_enhancements:
- prompt: "Test Architect Playwright MCP capabilities (healing, exploratory, verification) are optionally available.\nYou will have to setup your MCPs yourself; refer to test-architecture.md for hints.\nWould you like to enable MCP enhancements in Test Architect?"
+ prompt: "Test Architect Playwright MCP capabilities (healing, exploratory, verification) are optionally available.\nYou will have to setup your MCPs yourself; refer to https://docs.bmad-method.org/explanation/features/tea-overview for configuration examples.\nWould you like to enable MCP enhancements in Test Architect?"
default: false
result: "{value}"
diff --git a/src/modules/bmm/testarch/knowledge/api-request.md b/src/modules/bmm/testarch/knowledge/api-request.md
index b47bfc4f..d2b36cde 100644
--- a/src/modules/bmm/testarch/knowledge/api-request.md
+++ b/src/modules/bmm/testarch/knowledge/api-request.md
@@ -2,7 +2,7 @@
## Principle
-Use typed HTTP client with built-in schema validation and automatic retry for server errors. The utility handles URL resolution, header management, response parsing, and single-line response validation with proper TypeScript support.
+Use typed HTTP client with built-in schema validation and automatic retry for server errors. The utility handles URL resolution, header management, response parsing, and single-line response validation with proper TypeScript support. **Works without a browser** - ideal for pure API/service testing.
## Rationale
@@ -21,6 +21,7 @@ The `apiRequest` utility provides:
- **Schema validation**: Single-line validation (JSON Schema, Zod, OpenAPI)
- **URL resolution**: Four-tier strategy (explicit > config > Playwright > direct)
- **TypeScript generics**: Type-safe response bodies
+- **No browser required**: Pure API testing without browser overhead
## Pattern Examples
@@ -60,10 +61,11 @@ test('should fetch user data', async ({ apiRequest }) => {
```typescript
import { test } from '@seontechnologies/playwright-utils/api-request/fixtures';
+import { z } from 'zod';
-test('should validate response schema', async ({ apiRequest }) => {
- // JSON Schema validation
- const response = await apiRequest({
+// JSON Schema validation
+test('should validate response schema (JSON Schema)', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest({
method: 'GET',
path: '/api/users/123',
validateSchema: {
@@ -77,22 +79,25 @@ test('should validate response schema', async ({ apiRequest }) => {
},
});
// Throws if schema validation fails
+ expect(status).toBe(200);
+});
- // Zod schema validation
- import { z } from 'zod';
+// Zod schema validation
+const UserSchema = z.object({
+ id: z.string(),
+ name: z.string(),
+ email: z.string().email(),
+});
- const UserSchema = z.object({
- id: z.string(),
- name: z.string(),
- email: z.string().email(),
- });
-
- const response = await apiRequest({
+test('should validate response schema (Zod)', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest({
method: 'GET',
path: '/api/users/123',
validateSchema: UserSchema,
});
// Response body is type-safe AND validated
+ expect(status).toBe(200);
+ expect(body.email).toContain('@');
});
```
@@ -236,6 +241,136 @@ test('should poll until job completes', async ({ apiRequest, recurse }) => {
- `recurse` polls until predicate returns true
- Composable utilities work together seamlessly
+### Example 6: Microservice Testing (Multiple Services)
+
+**Context**: Test interactions between microservices without a browser.
+
+**Implementation**:
+
+```typescript
+import { test, expect } from '@seontechnologies/playwright-utils/fixtures';
+
+const USER_SERVICE = process.env.USER_SERVICE_URL || 'http://localhost:3001';
+const ORDER_SERVICE = process.env.ORDER_SERVICE_URL || 'http://localhost:3002';
+
+test.describe('Microservice Integration', () => {
+ test('should validate cross-service user lookup', async ({ apiRequest }) => {
+ // Create user in user-service
+ const { body: user } = await apiRequest({
+ method: 'POST',
+ path: '/api/users',
+ baseUrl: USER_SERVICE,
+ body: { name: 'Test User', email: 'test@example.com' },
+ });
+
+ // Create order in order-service (validates user via user-service)
+ const { status, body: order } = await apiRequest({
+ method: 'POST',
+ path: '/api/orders',
+ baseUrl: ORDER_SERVICE,
+ body: {
+ userId: user.id,
+ items: [{ productId: 'prod-1', quantity: 2 }],
+ },
+ });
+
+ expect(status).toBe(201);
+ expect(order.userId).toBe(user.id);
+ });
+
+ test('should reject order for invalid user', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: '/api/orders',
+ baseUrl: ORDER_SERVICE,
+ body: {
+ userId: 'non-existent-user',
+ items: [{ productId: 'prod-1', quantity: 1 }],
+ },
+ });
+
+ expect(status).toBe(400);
+ expect(body.code).toBe('INVALID_USER');
+ });
+});
+```
+
+**Key Points**:
+
+- Test multiple services without browser
+- Use `baseUrl` to target different services
+- Validate cross-service communication
+- Pure API testing - fast and reliable
+
+### Example 7: GraphQL API Testing
+
+**Context**: Test GraphQL endpoints with queries and mutations.
+
+**Implementation**:
+
+```typescript
+test.describe('GraphQL API', () => {
+ const GRAPHQL_ENDPOINT = '/graphql';
+
+ test('should query users via GraphQL', async ({ apiRequest }) => {
+ const query = `
+ query GetUsers($limit: Int) {
+ users(limit: $limit) {
+ id
+ name
+ email
+ }
+ }
+ `;
+
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: GRAPHQL_ENDPOINT,
+ body: {
+ query,
+ variables: { limit: 10 },
+ },
+ });
+
+ expect(status).toBe(200);
+ expect(body.errors).toBeUndefined();
+ expect(body.data.users).toHaveLength(10);
+ });
+
+ test('should create user via mutation', async ({ apiRequest }) => {
+ const mutation = `
+ mutation CreateUser($input: CreateUserInput!) {
+ createUser(input: $input) {
+ id
+ name
+ }
+ }
+ `;
+
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: GRAPHQL_ENDPOINT,
+ body: {
+ query: mutation,
+ variables: {
+ input: { name: 'GraphQL User', email: 'gql@example.com' },
+ },
+ },
+ });
+
+ expect(status).toBe(200);
+ expect(body.data.createUser.id).toBeDefined();
+ });
+});
+```
+
+**Key Points**:
+
+- GraphQL via POST request
+- Variables in request body
+- Check `body.errors` for GraphQL errors (not status code)
+- Works for queries and mutations
+
## Comparison with Vanilla Playwright
| Vanilla Playwright | playwright-utils apiRequest |
@@ -251,11 +386,13 @@ test('should poll until job completes', async ({ apiRequest, recurse }) => {
**Use apiRequest for:**
-- β
API endpoint testing
-- β
Background API calls in UI tests
+- β
Pure API/service testing (no browser needed)
+- β
Microservice integration testing
+- β
GraphQL API testing
- β
Schema validation needs
- β
Tests requiring retry logic
-- β
Typed API responses
+- β
Background API calls in UI tests
+- β
Contract testing support
**Stick with vanilla Playwright for:**
@@ -265,11 +402,13 @@ test('should poll until job completes', async ({ apiRequest, recurse }) => {
## Related Fragments
+- `api-testing-patterns.md` - Comprehensive pure API testing patterns
- `overview.md` - Installation and design principles
- `auth-session.md` - Authentication token management
- `recurse.md` - Polling for async operations
- `fixtures-composition.md` - Combining utilities with mergeTests
- `log.md` - Logging API requests
+- `contract-testing.md` - Pact contract testing
## Anti-Patterns
diff --git a/src/modules/bmm/testarch/knowledge/api-testing-patterns.md b/src/modules/bmm/testarch/knowledge/api-testing-patterns.md
new file mode 100644
index 00000000..65c81d7a
--- /dev/null
+++ b/src/modules/bmm/testarch/knowledge/api-testing-patterns.md
@@ -0,0 +1,843 @@
+# API Testing Patterns
+
+## Principle
+
+Test APIs and backend services directly without browser overhead. Use Playwright's `request` context for HTTP operations, `apiRequest` utility for enhanced features, and `recurse` for async operations. Pure API tests run faster, are more stable, and provide better coverage for service-layer logic.
+
+## Rationale
+
+Many teams over-rely on E2E/browser tests when API tests would be more appropriate:
+
+- **Slower feedback**: Browser tests take seconds, API tests take milliseconds
+- **More brittle**: UI changes break tests even when API works correctly
+- **Wrong abstraction**: Testing business logic through UI layers adds noise
+- **Resource heavy**: Browsers consume memory and CPU
+
+API-first testing provides:
+
+- **Fast execution**: No browser startup, no rendering, no JavaScript execution
+- **Direct validation**: Test exactly what the service returns
+- **Better isolation**: Test service logic independent of UI
+- **Easier debugging**: Clear request/response without DOM noise
+- **Contract validation**: Verify API contracts explicitly
+
+## When to Use API Tests vs E2E Tests
+
+| Scenario | API Test | E2E Test |
+|----------|----------|----------|
+| CRUD operations | β
Primary | β Overkill |
+| Business logic validation | β
Primary | β Overkill |
+| Error handling (4xx, 5xx) | β
Primary | β οΈ Supplement |
+| Authentication flows | β
Primary | β οΈ Supplement |
+| Data transformation | β
Primary | β Overkill |
+| User journeys | β Can't test | β
Primary |
+| Visual regression | β Can't test | β
Primary |
+| Cross-browser issues | β Can't test | β
Primary |
+
+**Rule of thumb**: If you're testing what the server returns (not how it looks), use API tests.
+
+## Pattern Examples
+
+### Example 1: Pure API Test (No Browser)
+
+**Context**: Test REST API endpoints directly without any browser context.
+
+**Implementation**:
+
+```typescript
+// tests/api/users.spec.ts
+import { test, expect } from '@playwright/test';
+
+// No page, no browser - just API
+test.describe('Users API', () => {
+ test('should create user', async ({ request }) => {
+ const response = await request.post('/api/users', {
+ data: {
+ name: 'John Doe',
+ email: 'john@example.com',
+ role: 'user',
+ },
+ });
+
+ expect(response.status()).toBe(201);
+
+ const user = await response.json();
+ expect(user.id).toBeDefined();
+ expect(user.name).toBe('John Doe');
+ expect(user.email).toBe('john@example.com');
+ });
+
+ test('should get user by ID', async ({ request }) => {
+ // Create user first
+ const createResponse = await request.post('/api/users', {
+ data: { name: 'Jane Doe', email: 'jane@example.com' },
+ });
+ const { id } = await createResponse.json();
+
+ // Get user
+ const getResponse = await request.get(`/api/users/${id}`);
+ expect(getResponse.status()).toBe(200);
+
+ const user = await getResponse.json();
+ expect(user.id).toBe(id);
+ expect(user.name).toBe('Jane Doe');
+ });
+
+ test('should return 404 for non-existent user', async ({ request }) => {
+ const response = await request.get('/api/users/non-existent-id');
+ expect(response.status()).toBe(404);
+
+ const error = await response.json();
+ expect(error.code).toBe('USER_NOT_FOUND');
+ });
+
+ test('should validate required fields', async ({ request }) => {
+ const response = await request.post('/api/users', {
+ data: { name: 'Missing Email' }, // email is required
+ });
+
+ expect(response.status()).toBe(400);
+
+ const error = await response.json();
+ expect(error.code).toBe('VALIDATION_ERROR');
+ expect(error.details).toContainEqual(
+ expect.objectContaining({ field: 'email', message: expect.any(String) })
+ );
+ });
+});
+```
+
+**Key Points**:
+
+- No `page` fixture needed - only `request`
+- Tests run without browser overhead
+- Direct HTTP assertions
+- Clear error handling tests
+
+### Example 2: API Test with apiRequest Utility
+
+**Context**: Use enhanced apiRequest for schema validation, retry, and type safety.
+
+**Implementation**:
+
+```typescript
+// tests/api/orders.spec.ts
+import { test, expect } from '@seontechnologies/playwright-utils/api-request/fixtures';
+import { z } from 'zod';
+
+// Define schema for type safety and validation
+const OrderSchema = z.object({
+ id: z.string().uuid(),
+ userId: z.string(),
+ items: z.array(
+ z.object({
+ productId: z.string(),
+ quantity: z.number().positive(),
+ price: z.number().positive(),
+ })
+ ),
+ total: z.number().positive(),
+ status: z.enum(['pending', 'processing', 'shipped', 'delivered']),
+ createdAt: z.string().datetime(),
+});
+
+type Order = z.infer;
+
+test.describe('Orders API', () => {
+ test('should create order with schema validation', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: '/api/orders',
+ body: {
+ userId: 'user-123',
+ items: [
+ { productId: 'prod-1', quantity: 2, price: 29.99 },
+ { productId: 'prod-2', quantity: 1, price: 49.99 },
+ ],
+ },
+ validateSchema: OrderSchema, // Validates response matches schema
+ });
+
+ expect(status).toBe(201);
+ expect(body.id).toBeDefined();
+ expect(body.status).toBe('pending');
+ expect(body.total).toBe(109.97); // 2*29.99 + 49.99
+ });
+
+ test('should handle server errors with retry', async ({ apiRequest }) => {
+ // apiRequest retries 5xx errors by default
+ const { status, body } = await apiRequest({
+ method: 'GET',
+ path: '/api/orders/order-123',
+ retryConfig: {
+ maxRetries: 3,
+ retryDelay: 1000,
+ },
+ });
+
+ expect(status).toBe(200);
+ });
+
+ test('should list orders with pagination', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest<{ orders: Order[]; total: number; page: number }>({
+ method: 'GET',
+ path: '/api/orders',
+ params: { page: 1, limit: 10, status: 'pending' },
+ });
+
+ expect(status).toBe(200);
+ expect(body.orders).toHaveLength(10);
+ expect(body.total).toBeGreaterThan(10);
+ expect(body.page).toBe(1);
+ });
+});
+```
+
+**Key Points**:
+
+- Zod schema for runtime validation AND TypeScript types
+- `validateSchema` throws if response doesn't match
+- Built-in retry for transient failures
+- Type-safe `body` access
+
+### Example 3: Microservice-to-Microservice Testing
+
+**Context**: Test service interactions without browser - validate API contracts between services.
+
+**Implementation**:
+
+```typescript
+// tests/api/service-integration.spec.ts
+import { test, expect } from '@seontechnologies/playwright-utils/fixtures';
+
+test.describe('Service Integration', () => {
+ const USER_SERVICE_URL = process.env.USER_SERVICE_URL || 'http://localhost:3001';
+ const ORDER_SERVICE_URL = process.env.ORDER_SERVICE_URL || 'http://localhost:3002';
+ const INVENTORY_SERVICE_URL = process.env.INVENTORY_SERVICE_URL || 'http://localhost:3003';
+
+ test('order service should validate user exists', async ({ apiRequest }) => {
+ // Create user in user-service
+ const { body: user } = await apiRequest({
+ method: 'POST',
+ path: '/api/users',
+ baseUrl: USER_SERVICE_URL,
+ body: { name: 'Test User', email: 'test@example.com' },
+ });
+
+ // Create order in order-service (should validate user via user-service)
+ const { status, body: order } = await apiRequest({
+ method: 'POST',
+ path: '/api/orders',
+ baseUrl: ORDER_SERVICE_URL,
+ body: {
+ userId: user.id,
+ items: [{ productId: 'prod-1', quantity: 1 }],
+ },
+ });
+
+ expect(status).toBe(201);
+ expect(order.userId).toBe(user.id);
+ });
+
+ test('order service should reject invalid user', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: '/api/orders',
+ baseUrl: ORDER_SERVICE_URL,
+ body: {
+ userId: 'non-existent-user',
+ items: [{ productId: 'prod-1', quantity: 1 }],
+ },
+ });
+
+ expect(status).toBe(400);
+ expect(body.code).toBe('INVALID_USER');
+ });
+
+ test('order should decrease inventory', async ({ apiRequest, recurse }) => {
+ // Get initial inventory
+ const { body: initialInventory } = await apiRequest({
+ method: 'GET',
+ path: '/api/inventory/prod-1',
+ baseUrl: INVENTORY_SERVICE_URL,
+ });
+
+ // Create order
+ await apiRequest({
+ method: 'POST',
+ path: '/api/orders',
+ baseUrl: ORDER_SERVICE_URL,
+ body: {
+ userId: 'user-123',
+ items: [{ productId: 'prod-1', quantity: 2 }],
+ },
+ });
+
+ // Poll for inventory update (eventual consistency)
+ const { body: updatedInventory } = await recurse(
+ () =>
+ apiRequest({
+ method: 'GET',
+ path: '/api/inventory/prod-1',
+ baseUrl: INVENTORY_SERVICE_URL,
+ }),
+ (response) => response.body.quantity === initialInventory.quantity - 2,
+ { timeout: 10000, interval: 500 }
+ );
+
+ expect(updatedInventory.quantity).toBe(initialInventory.quantity - 2);
+ });
+});
+```
+
+**Key Points**:
+
+- Multiple service URLs for microservice testing
+- Tests service-to-service communication
+- Uses `recurse` for eventual consistency
+- No browser needed for full integration testing
+
+### Example 4: GraphQL API Testing
+
+**Context**: Test GraphQL endpoints with queries and mutations.
+
+**Implementation**:
+
+```typescript
+// tests/api/graphql.spec.ts
+import { test, expect } from '@seontechnologies/playwright-utils/api-request/fixtures';
+
+const GRAPHQL_ENDPOINT = '/graphql';
+
+test.describe('GraphQL API', () => {
+ test('should query users', async ({ apiRequest }) => {
+ const query = `
+ query GetUsers($limit: Int) {
+ users(limit: $limit) {
+ id
+ name
+ email
+ role
+ }
+ }
+ `;
+
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: GRAPHQL_ENDPOINT,
+ body: {
+ query,
+ variables: { limit: 10 },
+ },
+ });
+
+ expect(status).toBe(200);
+ expect(body.errors).toBeUndefined();
+ expect(body.data.users).toHaveLength(10);
+ expect(body.data.users[0]).toHaveProperty('id');
+ expect(body.data.users[0]).toHaveProperty('name');
+ });
+
+ test('should create user via mutation', async ({ apiRequest }) => {
+ const mutation = `
+ mutation CreateUser($input: CreateUserInput!) {
+ createUser(input: $input) {
+ id
+ name
+ email
+ }
+ }
+ `;
+
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: GRAPHQL_ENDPOINT,
+ body: {
+ query: mutation,
+ variables: {
+ input: {
+ name: 'GraphQL User',
+ email: 'graphql@example.com',
+ },
+ },
+ },
+ });
+
+ expect(status).toBe(200);
+ expect(body.errors).toBeUndefined();
+ expect(body.data.createUser.id).toBeDefined();
+ expect(body.data.createUser.name).toBe('GraphQL User');
+ });
+
+ test('should handle GraphQL errors', async ({ apiRequest }) => {
+ const query = `
+ query GetUser($id: ID!) {
+ user(id: $id) {
+ id
+ name
+ }
+ }
+ `;
+
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: GRAPHQL_ENDPOINT,
+ body: {
+ query,
+ variables: { id: 'non-existent' },
+ },
+ });
+
+ expect(status).toBe(200); // GraphQL returns 200 even for errors
+ expect(body.errors).toBeDefined();
+ expect(body.errors[0].message).toContain('not found');
+ expect(body.data.user).toBeNull();
+ });
+
+ test('should handle validation errors', async ({ apiRequest }) => {
+ const mutation = `
+ mutation CreateUser($input: CreateUserInput!) {
+ createUser(input: $input) {
+ id
+ }
+ }
+ `;
+
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: GRAPHQL_ENDPOINT,
+ body: {
+ query: mutation,
+ variables: {
+ input: {
+ name: '', // Invalid: empty name
+ email: 'invalid-email', // Invalid: bad format
+ },
+ },
+ },
+ });
+
+ expect(status).toBe(200);
+ expect(body.errors).toBeDefined();
+ expect(body.errors[0].extensions.code).toBe('BAD_USER_INPUT');
+ });
+});
+```
+
+**Key Points**:
+
+- GraphQL queries and mutations via POST
+- Variables passed in request body
+- GraphQL returns 200 even for errors (check `body.errors`)
+- Test validation and business logic errors
+
+### Example 5: Database Seeding and Cleanup via API
+
+**Context**: Use API calls to set up and tear down test data without direct database access.
+
+**Implementation**:
+
+```typescript
+// tests/api/with-data-setup.spec.ts
+import { test, expect } from '@seontechnologies/playwright-utils/fixtures';
+
+test.describe('Orders with Data Setup', () => {
+ let testUser: { id: string; email: string };
+ let testProducts: Array<{ id: string; name: string; price: number }>;
+
+ test.beforeAll(async ({ request }) => {
+ // Seed user via API
+ const userResponse = await request.post('/api/users', {
+ data: {
+ name: 'Test User',
+ email: `test-${Date.now()}@example.com`,
+ },
+ });
+ testUser = await userResponse.json();
+
+ // Seed products via API
+ testProducts = [];
+ for (const product of [
+ { name: 'Widget A', price: 29.99 },
+ { name: 'Widget B', price: 49.99 },
+ { name: 'Widget C', price: 99.99 },
+ ]) {
+ const productResponse = await request.post('/api/products', {
+ data: product,
+ });
+ testProducts.push(await productResponse.json());
+ }
+ });
+
+ test.afterAll(async ({ request }) => {
+ // Cleanup via API
+ if (testUser?.id) {
+ await request.delete(`/api/users/${testUser.id}`);
+ }
+ for (const product of testProducts) {
+ await request.delete(`/api/products/${product.id}`);
+ }
+ });
+
+ test('should create order with seeded data', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: '/api/orders',
+ body: {
+ userId: testUser.id,
+ items: [
+ { productId: testProducts[0].id, quantity: 2 },
+ { productId: testProducts[1].id, quantity: 1 },
+ ],
+ },
+ });
+
+ expect(status).toBe(201);
+ expect(body.userId).toBe(testUser.id);
+ expect(body.items).toHaveLength(2);
+ expect(body.total).toBe(2 * 29.99 + 49.99);
+ });
+
+ test('should list user orders', async ({ apiRequest }) => {
+ // Create an order first
+ await apiRequest({
+ method: 'POST',
+ path: '/api/orders',
+ body: {
+ userId: testUser.id,
+ items: [{ productId: testProducts[2].id, quantity: 1 }],
+ },
+ });
+
+ // List orders for user
+ const { status, body } = await apiRequest({
+ method: 'GET',
+ path: '/api/orders',
+ params: { userId: testUser.id },
+ });
+
+ expect(status).toBe(200);
+ expect(body.orders.length).toBeGreaterThanOrEqual(1);
+ expect(body.orders.every((o: any) => o.userId === testUser.id)).toBe(true);
+ });
+});
+```
+
+**Key Points**:
+
+- `beforeAll`/`afterAll` for test data setup/cleanup
+- API-based seeding (no direct DB access needed)
+- Unique emails to prevent conflicts in parallel runs
+- Cleanup after all tests complete
+
+### Example 6: Background Job Testing with Recurse
+
+**Context**: Test async operations like background jobs, webhooks, and eventual consistency.
+
+**Implementation**:
+
+```typescript
+// tests/api/background-jobs.spec.ts
+import { test, expect } from '@seontechnologies/playwright-utils/fixtures';
+
+test.describe('Background Jobs', () => {
+ test('should process export job', async ({ apiRequest, recurse }) => {
+ // Trigger export job
+ const { body: job } = await apiRequest({
+ method: 'POST',
+ path: '/api/exports',
+ body: {
+ type: 'users',
+ format: 'csv',
+ filters: { createdAfter: '2024-01-01' },
+ },
+ });
+
+ expect(job.id).toBeDefined();
+ expect(job.status).toBe('pending');
+
+ // Poll until job completes
+ const { body: completedJob } = await recurse(
+ () => apiRequest({ method: 'GET', path: `/api/exports/${job.id}` }),
+ (response) => response.body.status === 'completed',
+ {
+ timeout: 60000,
+ interval: 2000,
+ log: `Waiting for export job ${job.id} to complete`,
+ }
+ );
+
+ expect(completedJob.status).toBe('completed');
+ expect(completedJob.downloadUrl).toBeDefined();
+ expect(completedJob.recordCount).toBeGreaterThan(0);
+ });
+
+ test('should handle job failure gracefully', async ({ apiRequest, recurse }) => {
+ // Trigger job that will fail
+ const { body: job } = await apiRequest({
+ method: 'POST',
+ path: '/api/exports',
+ body: {
+ type: 'invalid-type', // This will cause failure
+ format: 'csv',
+ },
+ });
+
+ // Poll until job fails
+ const { body: failedJob } = await recurse(
+ () => apiRequest({ method: 'GET', path: `/api/exports/${job.id}` }),
+ (response) => ['completed', 'failed'].includes(response.body.status),
+ { timeout: 30000 }
+ );
+
+ expect(failedJob.status).toBe('failed');
+ expect(failedJob.error).toBeDefined();
+ expect(failedJob.error.code).toBe('INVALID_EXPORT_TYPE');
+ });
+
+ test('should process webhook delivery', async ({ apiRequest, recurse }) => {
+ // Trigger action that sends webhook
+ const { body: order } = await apiRequest({
+ method: 'POST',
+ path: '/api/orders',
+ body: {
+ userId: 'user-123',
+ items: [{ productId: 'prod-1', quantity: 1 }],
+ webhookUrl: 'https://webhook.site/test-endpoint',
+ },
+ });
+
+ // Poll for webhook delivery status
+ const { body: webhookStatus } = await recurse(
+ () => apiRequest({ method: 'GET', path: `/api/webhooks/order/${order.id}` }),
+ (response) => response.body.delivered === true,
+ { timeout: 30000, interval: 1000 }
+ );
+
+ expect(webhookStatus.delivered).toBe(true);
+ expect(webhookStatus.deliveredAt).toBeDefined();
+ expect(webhookStatus.responseStatus).toBe(200);
+ });
+});
+```
+
+**Key Points**:
+
+- `recurse` for polling async operations
+- Test both success and failure scenarios
+- Configurable timeout and interval
+- Log messages for debugging
+
+### Example 7: Service Authentication (No Browser)
+
+**Context**: Test authenticated API endpoints using tokens directly - no browser login needed.
+
+**Implementation**:
+
+```typescript
+// tests/api/authenticated.spec.ts
+import { test, expect } from '@seontechnologies/playwright-utils/fixtures';
+
+test.describe('Authenticated API Tests', () => {
+ let authToken: string;
+
+ test.beforeAll(async ({ request }) => {
+ // Get token via API (no browser!)
+ const response = await request.post('/api/auth/login', {
+ data: {
+ email: process.env.TEST_USER_EMAIL,
+ password: process.env.TEST_USER_PASSWORD,
+ },
+ });
+
+ const { token } = await response.json();
+ authToken = token;
+ });
+
+ test('should access protected endpoint with token', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest({
+ method: 'GET',
+ path: '/api/me',
+ headers: {
+ Authorization: `Bearer ${authToken}`,
+ },
+ });
+
+ expect(status).toBe(200);
+ expect(body.email).toBe(process.env.TEST_USER_EMAIL);
+ });
+
+ test('should reject request without token', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest({
+ method: 'GET',
+ path: '/api/me',
+ // No Authorization header
+ });
+
+ expect(status).toBe(401);
+ expect(body.code).toBe('UNAUTHORIZED');
+ });
+
+ test('should reject expired token', async ({ apiRequest }) => {
+ const expiredToken = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...'; // Expired token
+
+ const { status, body } = await apiRequest({
+ method: 'GET',
+ path: '/api/me',
+ headers: {
+ Authorization: `Bearer ${expiredToken}`,
+ },
+ });
+
+ expect(status).toBe(401);
+ expect(body.code).toBe('TOKEN_EXPIRED');
+ });
+
+ test('should handle role-based access', async ({ apiRequest }) => {
+ // User token (non-admin)
+ const { status } = await apiRequest({
+ method: 'GET',
+ path: '/api/admin/users',
+ headers: {
+ Authorization: `Bearer ${authToken}`,
+ },
+ });
+
+ expect(status).toBe(403); // Forbidden for non-admin
+ });
+});
+```
+
+**Key Points**:
+
+- Token obtained via API login (no browser)
+- Token reused across all tests in describe block
+- Test auth, expired tokens, and RBAC
+- Pure API testing without UI
+
+## API Test Configuration
+
+### Playwright Config for API-Only Tests
+
+```typescript
+// playwright.config.ts
+import { defineConfig } from '@playwright/test';
+
+export default defineConfig({
+ testDir: './tests/api',
+
+ // No browser needed for API tests
+ use: {
+ baseURL: process.env.API_URL || 'http://localhost:3000',
+ extraHTTPHeaders: {
+ 'Accept': 'application/json',
+ 'Content-Type': 'application/json',
+ },
+ },
+
+ // Faster without browser overhead
+ timeout: 30000,
+
+ // Run API tests in parallel
+ workers: 4,
+ fullyParallel: true,
+
+ // No screenshots/traces needed for API tests
+ reporter: [['html'], ['json', { outputFile: 'api-test-results.json' }]],
+});
+```
+
+### Separate API Test Project
+
+```typescript
+// playwright.config.ts
+export default defineConfig({
+ projects: [
+ {
+ name: 'api',
+ testDir: './tests/api',
+ use: {
+ baseURL: process.env.API_URL,
+ },
+ },
+ {
+ name: 'e2e',
+ testDir: './tests/e2e',
+ use: {
+ baseURL: process.env.APP_URL,
+ ...devices['Desktop Chrome'],
+ },
+ },
+ ],
+});
+```
+
+## Comparison: API Tests vs E2E Tests
+
+| Aspect | API Test | E2E Test |
+|--------|----------|----------|
+| **Speed** | ~50-100ms per test | ~2-10s per test |
+| **Stability** | Very stable | More flaky (UI timing) |
+| **Setup** | Minimal | Browser, context, page |
+| **Debugging** | Clear request/response | DOM, screenshots, traces |
+| **Coverage** | Service logic | User experience |
+| **Parallelization** | Easy (stateless) | Complex (browser resources) |
+| **CI Cost** | Low (no browser) | High (browser containers) |
+
+## Related Fragments
+
+- `api-request.md` - apiRequest utility details
+- `recurse.md` - Polling patterns for async operations
+- `auth-session.md` - Token management
+- `contract-testing.md` - Pact contract testing
+- `test-levels-framework.md` - When to use which test level
+- `data-factories.md` - Test data setup patterns
+
+## Anti-Patterns
+
+**DON'T use E2E for API validation:**
+
+```typescript
+// Bad: Testing API through UI
+test('validate user creation', async ({ page }) => {
+ await page.goto('/admin/users');
+ await page.fill('#name', 'John');
+ await page.click('#submit');
+ await expect(page.getByText('User created')).toBeVisible();
+});
+```
+
+**DO test APIs directly:**
+
+```typescript
+// Good: Direct API test
+test('validate user creation', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: '/api/users',
+ body: { name: 'John' },
+ });
+ expect(status).toBe(201);
+ expect(body.id).toBeDefined();
+});
+```
+
+**DON'T ignore API tests because "E2E covers it":**
+
+```typescript
+// Bad thinking: "Our E2E tests create users, so API is tested"
+// Reality: E2E tests one happy path; API tests cover edge cases
+```
+
+**DO have dedicated API test coverage:**
+
+```typescript
+// Good: Explicit API test suite
+test.describe('Users API', () => {
+ test('creates user', async ({ apiRequest }) => { /* ... */ });
+ test('handles duplicate email', async ({ apiRequest }) => { /* ... */ });
+ test('validates required fields', async ({ apiRequest }) => { /* ... */ });
+ test('handles malformed JSON', async ({ apiRequest }) => { /* ... */ });
+ test('rate limits requests', async ({ apiRequest }) => { /* ... */ });
+});
+```
diff --git a/src/modules/bmm/testarch/knowledge/auth-session.md b/src/modules/bmm/testarch/knowledge/auth-session.md
index 3aa456af..e290476b 100644
--- a/src/modules/bmm/testarch/knowledge/auth-session.md
+++ b/src/modules/bmm/testarch/knowledge/auth-session.md
@@ -2,7 +2,7 @@
## Principle
-Persist authentication tokens to disk and reuse across test runs. Support multiple user identifiers, ephemeral authentication, and worker-specific accounts for parallel execution. Fetch tokens once, use everywhere.
+Persist authentication tokens to disk and reuse across test runs. Support multiple user identifiers, ephemeral authentication, and worker-specific accounts for parallel execution. Fetch tokens once, use everywhere. **Works for both API-only tests and browser tests.**
## Rationale
@@ -22,6 +22,7 @@ The `auth-session` utility provides:
- **Worker-specific accounts**: Parallel execution with isolated user accounts
- **Automatic token management**: Checks validity, renews if expired
- **Flexible provider pattern**: Adapt to any auth system (OAuth2, JWT, custom)
+- **API-first design**: Get tokens for API tests without browser overhead
## Pattern Examples
@@ -244,6 +245,200 @@ test('parallel test 2', async ({ page }) => {
- Token management automatic per worker
- Scales to any number of workers
+### Example 6: Pure API Authentication (No Browser)
+
+**Context**: Get auth tokens for API-only tests using auth-session disk persistence.
+
+**Implementation**:
+
+```typescript
+// Step 1: Create API-only auth provider (no browser needed)
+// playwright/support/api-auth-provider.ts
+import { type AuthProvider } from '@seontechnologies/playwright-utils/auth-session';
+
+const apiAuthProvider: AuthProvider = {
+ getEnvironment: (options) => options.environment || 'local',
+ getUserIdentifier: (options) => options.userIdentifier || 'api-user',
+
+ extractToken: (storageState) => {
+ // Token stored in localStorage format for disk persistence
+ const tokenEntry = storageState.origins?.[0]?.localStorage?.find(
+ (item) => item.name === 'auth_token'
+ );
+ return tokenEntry?.value;
+ },
+
+ isTokenExpired: (storageState) => {
+ const expiryEntry = storageState.origins?.[0]?.localStorage?.find(
+ (item) => item.name === 'token_expiry'
+ );
+ if (!expiryEntry) return true;
+ return Date.now() > parseInt(expiryEntry.value, 10);
+ },
+
+ manageAuthToken: async (request, options) => {
+ const email = process.env.TEST_USER_EMAIL;
+ const password = process.env.TEST_USER_PASSWORD;
+
+ if (!email || !password) {
+ throw new Error('TEST_USER_EMAIL and TEST_USER_PASSWORD must be set');
+ }
+
+ // Pure API login - no browser!
+ const response = await request.post('/api/auth/login', {
+ data: { email, password },
+ });
+
+ if (!response.ok()) {
+ throw new Error(`Auth failed: ${response.status()}`);
+ }
+
+ const { token, expiresIn } = await response.json();
+ const expiryTime = Date.now() + expiresIn * 1000;
+
+ // Return storage state format for disk persistence
+ return {
+ cookies: [],
+ origins: [
+ {
+ origin: process.env.API_BASE_URL || 'http://localhost:3000',
+ localStorage: [
+ { name: 'auth_token', value: token },
+ { name: 'token_expiry', value: String(expiryTime) },
+ ],
+ },
+ ],
+ };
+ },
+};
+
+export default apiAuthProvider;
+
+// Step 2: Create auth fixture
+// playwright/support/fixtures.ts
+import { test as base } from '@playwright/test';
+import { createAuthFixtures, setAuthProvider } from '@seontechnologies/playwright-utils/auth-session';
+import apiAuthProvider from './api-auth-provider';
+
+setAuthProvider(apiAuthProvider);
+
+export const test = base.extend(createAuthFixtures());
+
+// Step 3: Use in tests - token persisted to disk!
+// tests/api/authenticated-api.spec.ts
+import { test } from '../support/fixtures';
+import { expect } from '@playwright/test';
+
+test('should access protected endpoint', async ({ authToken, apiRequest }) => {
+ // authToken is automatically loaded from disk or fetched if expired
+ const { status, body } = await apiRequest({
+ method: 'GET',
+ path: '/api/me',
+ headers: { Authorization: `Bearer ${authToken}` },
+ });
+
+ expect(status).toBe(200);
+});
+
+test('should create resource with auth', async ({ authToken, apiRequest }) => {
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: '/api/orders',
+ headers: { Authorization: `Bearer ${authToken}` },
+ body: { items: [{ productId: 'prod-1', quantity: 2 }] },
+ });
+
+ expect(status).toBe(201);
+ expect(body.id).toBeDefined();
+});
+```
+
+**Key Points**:
+
+- Token persisted to disk (not in-memory) - survives test reruns
+- Provider fetches token once, reuses until expired
+- Pure API authentication - no browser context needed
+- `authToken` fixture handles disk read/write automatically
+- Environment variables validated with clear error message
+
+### Example 7: Service-to-Service Authentication
+
+**Context**: Test microservice authentication patterns (API keys, service tokens) with proper environment validation.
+
+**Implementation**:
+
+```typescript
+// tests/api/service-auth.spec.ts
+import { test as base, expect } from '@playwright/test';
+import { test as apiFixture } from '@seontechnologies/playwright-utils/api-request/fixtures';
+import { mergeTests } from '@playwright/test';
+
+// Validate environment variables at module load
+const SERVICE_API_KEY = process.env.SERVICE_API_KEY;
+const INTERNAL_SERVICE_URL = process.env.INTERNAL_SERVICE_URL;
+
+if (!SERVICE_API_KEY) {
+ throw new Error('SERVICE_API_KEY environment variable is required');
+}
+if (!INTERNAL_SERVICE_URL) {
+ throw new Error('INTERNAL_SERVICE_URL environment variable is required');
+}
+
+const test = mergeTests(base, apiFixture);
+
+test.describe('Service-to-Service Auth', () => {
+ test('should authenticate with API key', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest({
+ method: 'GET',
+ path: '/internal/health',
+ baseUrl: INTERNAL_SERVICE_URL,
+ headers: { 'X-API-Key': SERVICE_API_KEY },
+ });
+
+ expect(status).toBe(200);
+ expect(body.status).toBe('healthy');
+ });
+
+ test('should reject invalid API key', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest({
+ method: 'GET',
+ path: '/internal/health',
+ baseUrl: INTERNAL_SERVICE_URL,
+ headers: { 'X-API-Key': 'invalid-key' },
+ });
+
+ expect(status).toBe(401);
+ expect(body.code).toBe('INVALID_API_KEY');
+ });
+
+ test('should call downstream service with propagated auth', async ({ apiRequest }) => {
+ const { status, body } = await apiRequest({
+ method: 'POST',
+ path: '/internal/aggregate-data',
+ baseUrl: INTERNAL_SERVICE_URL,
+ headers: {
+ 'X-API-Key': SERVICE_API_KEY,
+ 'X-Request-ID': `test-${Date.now()}`,
+ },
+ body: { sources: ['users', 'orders', 'inventory'] },
+ });
+
+ expect(status).toBe(200);
+ expect(body.aggregatedFrom).toHaveLength(3);
+ });
+});
+```
+
+**Key Points**:
+
+- Environment variables validated at module load with clear errors
+- API key authentication (simpler than OAuth - no disk persistence needed)
+- Test internal/service endpoints
+- Validate auth rejection scenarios
+- Correlation ID for request tracing
+
+> **Note**: API keys are typically static secrets that don't expire, so disk persistence (auth-session) isn't needed. For rotating service tokens, use the auth-session provider pattern from Example 6.
+
## Custom Auth Provider Pattern
**Context**: Adapt auth-session to your authentication system (OAuth2, JWT, SAML, custom).
@@ -310,6 +505,7 @@ test('authenticated API call', async ({ apiRequest, authToken }) => {
## Related Fragments
+- `api-testing-patterns.md` - Pure API testing patterns (no browser)
- `overview.md` - Installation and fixture composition
- `api-request.md` - Authenticated API requests
- `fixtures-composition.md` - Merging auth with other utilities
diff --git a/src/modules/bmm/testarch/knowledge/file-utils.md b/src/modules/bmm/testarch/knowledge/file-utils.md
index 1fa02397..014ac81c 100644
--- a/src/modules/bmm/testarch/knowledge/file-utils.md
+++ b/src/modules/bmm/testarch/knowledge/file-utils.md
@@ -22,6 +22,16 @@ The `file-utils` module provides:
- **Validation helpers**: Row count, header checks, content validation
- **Format support**: Multiple sheet support (XLSX), text extraction (PDF), archive extraction (ZIP)
+## Why Use This Instead of Vanilla Playwright?
+
+| Vanilla Playwright | File Utils |
+| ------------------------------------------- | ------------------------------------------------ |
+| ~80 lines per CSV flow (download + parse) | ~10 lines end-to-end |
+| Manual event orchestration for downloads | Encapsulated in `handleDownload()` |
+| Manual path handling and `saveAs` | Returns a ready-to-use file path |
+| Manual existence checks and error handling | Centralized in one place via utility patterns |
+| Manual CSV parsing config (headers, typing) | `readCSV()` returns `{ data, headers }` directly |
+
## Pattern Examples
### Example 1: UI-Triggered CSV Download
@@ -40,20 +50,18 @@ test('should download and validate CSV', async ({ page }) => {
const downloadPath = await handleDownload({
page,
downloadDir: DOWNLOAD_DIR,
- trigger: () => page.click('[data-testid="export-csv"]'),
+ trigger: () => page.getByTestId('download-button-text/csv').click(),
});
- const { content } = await readCSV({ filePath: downloadPath });
+ const csvResult = await readCSV({ filePath: downloadPath });
- // Validate headers
- expect(content.headers).toEqual(['ID', 'Name', 'Email', 'Role']);
-
- // Validate data
- expect(content.data).toHaveLength(10);
- expect(content.data[0]).toMatchObject({
+ // Access parsed data and headers
+ const { data, headers } = csvResult.content;
+ expect(headers).toEqual(['ID', 'Name', 'Email']);
+ expect(data[0]).toMatchObject({
ID: expect.any(String),
Name: expect.any(String),
- Email: expect.stringMatching(/@/),
+ Email: expect.any(String),
});
});
```
@@ -81,25 +89,27 @@ test('should read multi-sheet XLSX', async () => {
trigger: () => page.click('[data-testid="export-xlsx"]'),
});
- const { content } = await readXLSX({ filePath: downloadPath });
+ const xlsxResult = await readXLSX({ filePath: downloadPath });
- // Access specific sheets
- const summarySheet = content.sheets.find((s) => s.name === 'Summary');
- const detailsSheet = content.sheets.find((s) => s.name === 'Details');
+ // Verify worksheet structure
+ expect(xlsxResult.content.worksheets.length).toBeGreaterThan(0);
+ const worksheet = xlsxResult.content.worksheets[0];
+ expect(worksheet).toBeDefined();
+ expect(worksheet).toHaveProperty('name');
- // Validate summary
- expect(summarySheet.data).toHaveLength(1);
- expect(summarySheet.data[0].TotalRecords).toBe('150');
+ // Access sheet data
+ const sheetData = worksheet?.data;
+ expect(Array.isArray(sheetData)).toBe(true);
- // Validate details
- expect(detailsSheet.data).toHaveLength(150);
- expect(detailsSheet.headers).toContain('TransactionID');
+ // Use type assertion for type safety
+ const firstRow = sheetData![0] as Record;
+ expect(firstRow).toHaveProperty('id');
});
```
**Key Points**:
-- `sheets` array with `name` and `data` properties
+- `worksheets` array with `name` and `data` properties
- Access sheets by name
- Each sheet has its own headers and data
- Type-safe sheet iteration
@@ -117,26 +127,48 @@ test('should validate PDF report', async () => {
const downloadPath = await handleDownload({
page,
downloadDir: DOWNLOAD_DIR,
- trigger: () => page.click('[data-testid="download-report"]'),
+ trigger: () => page.getByTestId('download-button-Text-based PDF Document').click(),
});
- const { content } = await readPDF({ filePath: downloadPath });
+ const pdfResult = await readPDF({ filePath: downloadPath });
- // content.text is extracted text from all pages
- expect(content.text).toContain('Financial Report Q4 2024');
- expect(content.text).toContain('Total Revenue:');
-
- // Validate page count
- expect(content.numpages).toBeGreaterThan(10);
+ // content is extracted text from all pages
+ expect(pdfResult.pagesCount).toBe(1);
+ expect(pdfResult.fileName).toContain('.pdf');
+ expect(pdfResult.content).toContain('All you need is the free Adobe Acrobat Reader');
});
```
-**Key Points**:
+**PDF Reader Options:**
-- `content.text` contains all extracted text
-- `content.numpages` for page count
-- PDF parsing handles multi-page documents
-- Search for specific phrases
+```typescript
+const result = await readPDF({
+ filePath: '/path/to/document.pdf',
+ mergePages: false, // Keep pages separate (default: true)
+ debug: true, // Enable debug logging
+ maxPages: 10, // Limit processing to first 10 pages
+});
+```
+
+**Important Limitation - Vector-based PDFs:**
+
+Text extraction may fail for PDFs that store text as vector graphics (e.g., those generated by jsPDF):
+
+```typescript
+// Vector-based PDF example (extraction fails gracefully)
+const pdfResult = await readPDF({ filePath: downloadPath });
+
+expect(pdfResult.pagesCount).toBe(1);
+expect(pdfResult.info.extractionNotes).toContain(
+ 'Text extraction from vector-based PDFs is not supported.'
+);
+```
+
+Such PDFs will have:
+
+- `textExtractionSuccess: false`
+- `isVectorBased: true`
+- Explanatory message in `extractionNotes`
### Example 4: ZIP Archive Validation
@@ -154,25 +186,33 @@ test('should validate ZIP archive', async () => {
trigger: () => page.click('[data-testid="download-backup"]'),
});
- const { content } = await readZIP({ filePath: downloadPath });
+ const zipResult = await readZIP({ filePath: downloadPath });
// Check file list
- expect(content.files).toContain('data.csv');
- expect(content.files).toContain('config.json');
- expect(content.files).toContain('readme.txt');
+ expect(Array.isArray(zipResult.content.entries)).toBe(true);
+ expect(zipResult.content.entries).toContain(
+ 'Case_53125_10-19-22_AM/Case_53125_10-19-22_AM_case_data.csv'
+ );
- // Read specific file from archive
- const configContent = content.zip.readAsText('config.json');
- const config = JSON.parse(configContent);
+ // Extract specific file
+ const targetFile = 'Case_53125_10-19-22_AM/Case_53125_10-19-22_AM_case_data.csv';
+ const zipWithExtraction = await readZIP({
+ filePath: downloadPath,
+ fileToExtract: targetFile,
+ });
- expect(config.version).toBe('2.0');
+ // Access extracted file buffer
+ const extractedFiles = zipWithExtraction.content.extractedFiles || {};
+ const fileBuffer = extractedFiles[targetFile];
+ expect(fileBuffer).toBeInstanceOf(Buffer);
+ expect(fileBuffer?.length).toBeGreaterThan(0);
});
```
**Key Points**:
-- `content.files` lists all files in archive
-- `content.zip.readAsText()` extracts specific files
+- `content.entries` lists all files in archive
+- `fileToExtract` extracts specific files to Buffer
- Validate archive structure
- Read and parse individual files from ZIP
@@ -185,7 +225,7 @@ test('should validate ZIP archive', async () => {
```typescript
test('should download via API', async ({ page, request }) => {
const downloadPath = await handleDownload({
- page,
+ page, // Still need page for download events
downloadDir: DOWNLOAD_DIR,
trigger: async () => {
const response = await request.get('/api/export/csv', {
@@ -211,20 +251,123 @@ test('should download via API', async ({ page, request }) => {
- Still need `page` for download events
- Works with authenticated endpoints
-## Validation Helpers
+### Example 6: Reading CSV from Buffer (ZIP extraction)
+
+**Context**: Read CSV content directly from a Buffer (e.g., extracted from ZIP).
+
+**Implementation**:
```typescript
-// CSV validation
-const { isValid, errors } = await validateCSV({
- filePath: downloadPath,
- expectedRowCount: 10,
- requiredHeaders: ['ID', 'Name', 'Email'],
+// Read from a Buffer (e.g., extracted from a ZIP)
+const zipResult = await readZIP({
+ filePath: 'archive.zip',
+ fileToExtract: 'data.csv',
});
+const fileBuffer = zipResult.content.extractedFiles?.['data.csv'];
+const csvFromBuffer = await readCSV({ content: fileBuffer });
-expect(isValid).toBe(true);
-expect(errors).toHaveLength(0);
+// Read from a string
+const csvString = 'name,age\nJohn,30\nJane,25';
+const csvFromString = await readCSV({ content: csvString });
+
+const { data, headers } = csvFromString.content;
+expect(headers).toContain('name');
+expect(headers).toContain('age');
```
+## API Reference
+
+### CSV Reader Options
+
+| Option | Type | Default | Description |
+| -------------- | ------------------ | -------- | -------------------------------------- |
+| `filePath` | `string` | - | Path to CSV file (mutually exclusive) |
+| `content` | `string \| Buffer` | - | Direct content (mutually exclusive) |
+| `delimiter` | `string \| 'auto'` | `','` | Value separator, auto-detect if 'auto' |
+| `encoding` | `string` | `'utf8'` | File encoding |
+| `parseHeaders` | `boolean` | `true` | Use first row as headers |
+| `trim` | `boolean` | `true` | Trim whitespace from values |
+
+### XLSX Reader Options
+
+| Option | Type | Description |
+| ----------- | -------- | ------------------------------ |
+| `filePath` | `string` | Path to XLSX file |
+| `sheetName` | `string` | Name of sheet to set as active |
+
+### PDF Reader Options
+
+| Option | Type | Default | Description |
+| ------------ | --------- | ------- | --------------------------- |
+| `filePath` | `string` | - | Path to PDF file (required) |
+| `mergePages` | `boolean` | `true` | Merge text from all pages |
+| `maxPages` | `number` | - | Maximum pages to extract |
+| `debug` | `boolean` | `false` | Enable debug logging |
+
+### ZIP Reader Options
+
+| Option | Type | Description |
+| --------------- | -------- | ---------------------------------- |
+| `filePath` | `string` | Path to ZIP file |
+| `fileToExtract` | `string` | Specific file to extract to Buffer |
+
+### Return Values
+
+#### CSV Reader Return Value
+
+```typescript
+{
+ content: {
+ data: Array>, // Parsed rows (excludes header row if parseHeaders: true)
+ headers: string[] | null // Column headers (null if parseHeaders: false)
+ }
+}
+```
+
+#### XLSX Reader Return Value
+
+```typescript
+{
+ content: {
+ worksheets: Array<{
+ name: string, // Sheet name
+ rows: Array>, // All rows including headers
+ headers?: string[] // First row as headers (if present)
+ }>
+ }
+}
+```
+
+#### PDF Reader Return Value
+
+```typescript
+{
+ content: string, // Extracted text (merged or per-page based on mergePages)
+ pagesCount: number, // Total pages in PDF
+ fileName?: string, // Original filename if available
+ info?: Record // PDF metadata (author, title, etc.)
+}
+```
+
+> **Note**: When `mergePages: false`, `content` is an array of strings (one per page). When `maxPages` is set, only that many pages are extracted.
+
+#### ZIP Reader Return Value
+
+```typescript
+{
+ content: {
+ entries: Array<{
+ name: string, // File/directory path within ZIP
+ size: number, // Uncompressed size in bytes
+ isDirectory: boolean // True for directories
+ }>,
+ extractedFiles: Record // Extracted file contents by path
+ }
+}
+```
+
+> **Note**: When `fileToExtract` is specified, only that file appears in `extractedFiles`.
+
## Download Cleanup Pattern
```typescript
@@ -234,6 +377,66 @@ test.afterEach(async () => {
});
```
+## Comparison with Vanilla Playwright
+
+Vanilla Playwright (real test) snippet:
+
+```typescript
+// ~80 lines of boilerplate!
+const [download] = await Promise.all([
+ page.waitForEvent('download'),
+ page.getByTestId('download-button-CSV Export').click(),
+]);
+
+const failure = await download.failure();
+expect(failure).toBeNull();
+
+const filePath = testInfo.outputPath(download.suggestedFilename());
+await download.saveAs(filePath);
+
+await expect
+ .poll(
+ async () => {
+ try {
+ await fs.access(filePath);
+ return true;
+ } catch {
+ return false;
+ }
+ },
+ { timeout: 5000, intervals: [100, 200, 500] }
+ )
+ .toBe(true);
+
+const csvContent = await fs.readFile(filePath, 'utf-8');
+
+const parseResult = parse(csvContent, {
+ header: true,
+ skipEmptyLines: true,
+ dynamicTyping: true,
+ transformHeader: (header: string) => header.trim(),
+});
+
+if (parseResult.errors.length > 0) {
+ throw new Error(`CSV parsing errors: ${JSON.stringify(parseResult.errors)}`);
+}
+
+const data = parseResult.data as Array>;
+const headers = parseResult.meta.fields || [];
+```
+
+With File Utils, the same flow becomes:
+
+```typescript
+const downloadPath = await handleDownload({
+ page,
+ downloadDir: DOWNLOAD_DIR,
+ trigger: () => page.getByTestId('download-button-text/csv').click(),
+});
+
+const { data, headers } = (await readCSV({ filePath: downloadPath })).content;
+```
+
## Related Fragments
- `overview.md` - Installation and imports
@@ -242,7 +445,7 @@ test.afterEach(async () => {
## Anti-Patterns
-**β Not cleaning up downloads:**
+**DON'T leave downloads in place:**
```typescript
test('creates file', async () => {
@@ -251,7 +454,7 @@ test('creates file', async () => {
})
```
-**β
Clean up after tests:**
+**DO clean up after tests:**
```typescript
test.afterEach(async () => {
diff --git a/src/modules/bmm/testarch/knowledge/intercept-network-call.md b/src/modules/bmm/testarch/knowledge/intercept-network-call.md
index a175d559..e47419da 100644
--- a/src/modules/bmm/testarch/knowledge/intercept-network-call.md
+++ b/src/modules/bmm/testarch/knowledge/intercept-network-call.md
@@ -183,7 +183,31 @@ test('should handle timeout', async ({ page, interceptNetworkCall }) => {
- Validate error UI states
- No real failures needed
-### Example 5: Multiple Intercepts (Order Matters!)
+### Example 5: Order Matters - Intercept Before Navigate
+
+**Context**: The interceptor must be set up before the network request occurs.
+
+**Implementation**:
+
+```typescript
+// INCORRECT - interceptor set up too late
+await page.goto('https://example.com'); // Request already happened
+const networkCall = interceptNetworkCall({ url: '**/api/data' });
+await networkCall; // Will hang indefinitely!
+
+// CORRECT - Set up interception first
+const networkCall = interceptNetworkCall({ url: '**/api/data' });
+await page.goto('https://example.com');
+const result = await networkCall;
+```
+
+This pattern follows the classic test spy/stub pattern:
+
+1. Define the spy/stub (set up interception)
+2. Perform the action (trigger the network request)
+3. Assert on the spy/stub (await and verify the response)
+
+### Example 6: Multiple Intercepts
**Context**: Intercepting different endpoints in same test - setup order is critical.
@@ -191,7 +215,7 @@ test('should handle timeout', async ({ page, interceptNetworkCall }) => {
```typescript
test('multiple intercepts', async ({ page, interceptNetworkCall }) => {
- // β
CORRECT: Setup all intercepts BEFORE navigation
+ // Setup all intercepts BEFORE navigation
const usersCall = interceptNetworkCall({ url: '**/api/users' });
const productsCall = interceptNetworkCall({ url: '**/api/products' });
const ordersCall = interceptNetworkCall({ url: '**/api/orders' });
@@ -211,11 +235,85 @@ test('multiple intercepts', async ({ page, interceptNetworkCall }) => {
- Setup all intercepts before triggering actions
- Use `Promise.all()` to wait for multiple calls
-- Order: intercept β navigate β await
+- Order: intercept -> navigate -> await
- Prevents race conditions
+### Example 7: Capturing Multiple Requests to the Same Endpoint
+
+**Context**: Each `interceptNetworkCall` captures only the first matching request.
+
+**Implementation**:
+
+```typescript
+// Capturing a known number of requests
+const firstRequest = interceptNetworkCall({ url: '/api/data' });
+const secondRequest = interceptNetworkCall({ url: '/api/data' });
+
+await page.click('#load-data-button');
+
+const firstResponse = await firstRequest;
+const secondResponse = await secondRequest;
+
+expect(firstResponse.status).toBe(200);
+expect(secondResponse.status).toBe(200);
+
+// Handling an unknown number of requests
+const getDataRequestInterceptor = () =>
+ interceptNetworkCall({
+ url: '/api/data',
+ timeout: 1000, // Short timeout to detect when no more requests are coming
+ });
+
+let currentInterceptor = getDataRequestInterceptor();
+const allResponses = [];
+
+await page.click('#load-multiple-data-button');
+
+while (true) {
+ try {
+ const response = await currentInterceptor;
+ allResponses.push(response);
+ currentInterceptor = getDataRequestInterceptor();
+ } catch (error) {
+ // No more requests (timeout)
+ break;
+ }
+}
+
+console.log(`Captured ${allResponses.length} requests to /api/data`);
+```
+
+### Example 8: Using Timeout
+
+**Context**: Set a timeout for waiting on a network request.
+
+**Implementation**:
+
+```typescript
+const dataCall = interceptNetworkCall({
+ method: 'GET',
+ url: '/api/data-that-might-be-slow',
+ timeout: 5000, // 5 seconds timeout
+});
+
+await page.goto('/data-page');
+
+try {
+ const { responseJson } = await dataCall;
+ console.log('Data loaded successfully:', responseJson);
+} catch (error) {
+ if (error.message.includes('timeout')) {
+ console.log('Request timed out as expected');
+ } else {
+ throw error;
+ }
+}
+```
+
## URL Pattern Matching
+The utility uses [picomatch](https://github.com/micromatch/picomatch) for powerful glob pattern matching, dramatically simplifying URL targeting:
+
**Supported glob patterns:**
```typescript
@@ -226,7 +324,59 @@ test('multiple intercepts', async ({ page, interceptNetworkCall }) => {
'**/api/users?id=*'; // With query params
```
-**Uses picomatch library** - same pattern syntax as Playwright's `page.route()` but cleaner API.
+**Comparison with vanilla Playwright:**
+
+```typescript
+// Vanilla Playwright - complex predicate
+const predicate = (response) => {
+ const url = response.url();
+ return (
+ url.endsWith('/api/users') ||
+ url.match(/\/api\/users\/\d+/) ||
+ (url.includes('/api/users/') && url.includes('/profile'))
+ );
+};
+page.waitForResponse(predicate);
+
+// With interceptNetworkCall - simple glob patterns
+interceptNetworkCall({ url: '/api/users' }); // Exact endpoint
+interceptNetworkCall({ url: '/api/users/*' }); // User by ID pattern
+interceptNetworkCall({ url: '/api/users/*/profile' }); // Specific sub-paths
+interceptNetworkCall({ url: '/api/users/**' }); // Match all
+```
+
+## API Reference
+
+### `interceptNetworkCall(options)`
+
+| Parameter | Type | Description |
+| ----------------- | ---------- | --------------------------------------------------------------------- |
+| `page` | `Page` | Required when using direct import (not needed with fixture) |
+| `method` | `string` | Optional: HTTP method to match (e.g., 'GET', 'POST') |
+| `url` | `string` | Optional: URL pattern to match (supports glob patterns via picomatch) |
+| `fulfillResponse` | `object` | Optional: Response to use when mocking |
+| `handler` | `function` | Optional: Custom handler function for the route |
+| `timeout` | `number` | Optional: Timeout in milliseconds for the network request |
+
+### `fulfillResponse` Object
+
+| Property | Type | Description |
+| --------- | ------------------------ | ----------------------------------------------------- |
+| `status` | `number` | HTTP status code (default: 200) |
+| `headers` | `Record` | Response headers |
+| `body` | `any` | Response body (will be JSON.stringified if an object) |
+
+### Return Value
+
+Returns a `Promise` with:
+
+| Property | Type | Description |
+| -------------- | ---------- | --------------------------------------- |
+| `request` | `Request` | The intercepted request |
+| `response` | `Response` | The response (null if mocked) |
+| `responseJson` | `any` | Parsed JSON response (if available) |
+| `status` | `number` | HTTP status code |
+| `requestJson` | `any` | Parsed JSON request body (if available) |
## Comparison with Vanilla Playwright
@@ -238,7 +388,7 @@ test('multiple intercepts', async ({ page, interceptNetworkCall }) => {
| `const status = resp.status()` | `const { status } = await call` |
| Complex filter predicates | Simple glob patterns |
-**Reduction:** ~5-7 lines β ~2-3 lines per interception
+**Reduction:** ~5-7 lines -> ~2-3 lines per interception
## Related Fragments
@@ -248,14 +398,14 @@ test('multiple intercepts', async ({ page, interceptNetworkCall }) => {
## Anti-Patterns
-**β Intercepting after navigation:**
+**DON'T intercept after navigation:**
```typescript
await page.goto('/dashboard'); // Navigation starts
const usersCall = interceptNetworkCall({ url: '**/api/users' }); // Too late!
```
-**β
Intercept before navigate:**
+**DO intercept before navigate:**
```typescript
const usersCall = interceptNetworkCall({ url: '**/api/users' }); // First
@@ -263,7 +413,7 @@ await page.goto('/dashboard'); // Then navigate
const { responseJson } = await usersCall; // Then await
```
-**β Ignoring the returned Promise:**
+**DON'T ignore the returned Promise:**
```typescript
interceptNetworkCall({ url: '**/api/users' }); // Not awaited!
@@ -271,7 +421,7 @@ await page.goto('/dashboard');
// No deterministic wait - race condition
```
-**β
Always await the intercept:**
+**DO always await the intercept:**
```typescript
const usersCall = interceptNetworkCall({ url: '**/api/users' });
diff --git a/src/modules/bmm/testarch/knowledge/log.md b/src/modules/bmm/testarch/knowledge/log.md
index 42ddc228..c222a0c6 100644
--- a/src/modules/bmm/testarch/knowledge/log.md
+++ b/src/modules/bmm/testarch/knowledge/log.md
@@ -21,6 +21,20 @@ The `log` utility provides:
- **Multiple levels**: info, step, success, warning, error, debug
- **Optional console**: Can disable console output but keep report logs
+## Quick Start
+
+```typescript
+import { log } from '@seontechnologies/playwright-utils';
+
+// Basic logging
+await log.info('Starting test');
+await log.step('Test step shown in Playwright UI');
+await log.success('Operation completed');
+await log.warning('Something to note');
+await log.error('Something went wrong');
+await log.debug('Debug information');
+```
+
## Pattern Examples
### Example 1: Basic Logging Levels
@@ -143,41 +157,105 @@ test('organized with steps', async ({ page, apiRequest }) => {
- Steps visible in Playwright trace viewer
- Better debugging when tests fail
-### Example 4: Conditional Logging
+### Example 4: Test Step Decorators
-**Context**: Log different messages based on environment or test conditions.
+**Context**: Create collapsible test steps in Playwright UI using decorators.
+
+**Page Object Methods with @methodTestStep:**
+
+```typescript
+import { methodTestStep } from '@seontechnologies/playwright-utils';
+
+class TodoPage {
+ constructor(private page: Page) {
+ this.name = 'TodoPage';
+ }
+
+ readonly name: string;
+
+ @methodTestStep('Add todo item')
+ async addTodo(text: string) {
+ await log.info(`Adding todo: ${text}`);
+ const newTodo = this.page.getByPlaceholder('What needs to be done?');
+ await newTodo.fill(text);
+ await newTodo.press('Enter');
+ await log.step('step within a decorator');
+ await log.success(`Added todo: ${text}`);
+ }
+
+ @methodTestStep('Get all todos')
+ async getTodos() {
+ await log.info('Getting all todos');
+ return this.page.getByTestId('todo-title');
+ }
+}
+```
+
+**Function Helpers with functionTestStep:**
+
+```typescript
+import { functionTestStep } from '@seontechnologies/playwright-utils';
+
+// Define todo items for the test
+const TODO_ITEMS = ['buy groceries', 'pay bills', 'schedule meeting'];
+
+const createDefaultTodos = functionTestStep('Create default todos', async (page: Page) => {
+ await log.info('Creating default todos');
+ await log.step('step within a functionWrapper');
+ const todoPage = new TodoPage(page);
+
+ for (const item of TODO_ITEMS) {
+ await todoPage.addTodo(item);
+ }
+
+ await log.success('Created all default todos');
+});
+
+const checkNumberOfTodosInLocalStorage = functionTestStep(
+ 'Check total todos count fn-step',
+ async (page: Page, expected: number) => {
+ await log.info(`Verifying todo count: ${expected}`);
+ const result = await page.waitForFunction(
+ (e) => JSON.parse(localStorage['react-todos']).length === e,
+ expected
+ );
+ await log.success(`Verified todo count: ${expected}`);
+ return result;
+ }
+);
+```
+
+### Example 5: File Logging
+
+**Context**: Enable file logging for persistent logs.
**Implementation**:
```typescript
-test('conditional logging', async ({ page }) => {
- const isCI = process.env.CI === 'true';
+// playwright/support/fixtures.ts
+import { test as base } from '@playwright/test';
+import { log, captureTestContext } from '@seontechnologies/playwright-utils';
- if (isCI) {
- await log.info('Running in CI environment');
- } else {
- await log.debug('Running locally');
- }
+// Configure file logging globally
+log.configure({
+ fileLogging: {
+ enabled: true,
+ outputDir: 'playwright-logs/organized-logs',
+ forceConsolidated: false, // One file per test
+ },
+});
- const isKafkaWorking = await checkKafkaHealth();
-
- if (!isKafkaWorking) {
- await log.warning('Kafka unavailable - skipping event checks');
- } else {
- await log.step('Verifying Kafka events');
- // ... event verification
- }
+// Extend base test with file logging context capture
+export const test = base.extend({
+ // Auto-capture test context for file logging
+ autoTestContext: [async ({}, use, testInfo) => {
+ captureTestContext(testInfo);
+ await use(undefined);
+ }, { auto: true }],
});
```
-**Key Points**:
-
-- Log based on environment
-- Skip logging with conditionals
-- Use appropriate log levels
-- Debug info for local, minimal for CI
-
-### Example 5: Integration with Auth and API
+### Example 6: Integration with Auth and API
**Context**: Log authenticated API requests with tokens (safely).
@@ -221,16 +299,73 @@ test('should log auth flow', async ({ authToken, apiRequest }) => {
- Combine with auth and API utilities
- Log at appropriate detail level
+## Configuration
+
+**Defaults:** console logging enabled, file logging disabled.
+
+```typescript
+// Enable file logging in config
+log.configure({
+ console: true, // default
+ fileLogging: {
+ enabled: true,
+ outputDir: 'playwright-logs',
+ forceConsolidated: false, // One file per test
+ },
+});
+
+// Per-test override
+await log.info('Message', {
+ console: { enabled: false },
+ fileLogging: { enabled: true },
+});
+```
+
+### Environment Variables
+
+```bash
+# Disable all logging
+SILENT=true
+
+# Disable only file logging
+DISABLE_FILE_LOGS=true
+
+# Disable only console logging
+DISABLE_CONSOLE_LOGS=true
+```
+
+### Level Filtering
+
+```typescript
+log.configure({
+ level: 'warning', // Only warning, error levels will show
+});
+
+// Available levels (in priority order):
+// debug < info < step < success < warning < error
+```
+
+### Sync Methods
+
+For non-test contexts (global setup, utility functions):
+
+```typescript
+// Use sync methods when async/await isn't available
+log.infoSync('Initializing configuration');
+log.successSync('Environment configured');
+log.errorSync('Setup failed');
+```
+
## Log Levels Guide
-| Level | When to Use | Shows in Report | Shows in Console |
-| --------- | ----------------------------------- | -------------------- | ---------------- |
-| `step` | Test organization, major actions | β
Collapsible steps | β
Yes |
-| `info` | General information, state changes | β
Yes | β
Yes |
-| `success` | Successful operations | β
Yes | β
Yes |
-| `warning` | Non-critical issues, skipped checks | β
Yes | β
Yes |
-| `error` | Failures, exceptions | β
Yes | β
Configurable |
-| `debug` | Detailed data, objects | β
Yes (attached) | β
Configurable |
+| Level | When to Use | Shows in Report | Shows in Console |
+| --------- | ----------------------------------- | ----------------- | ---------------- |
+| `step` | Test organization, major actions | Collapsible steps | Yes |
+| `info` | General information, state changes | Yes | Yes |
+| `success` | Successful operations | Yes | Yes |
+| `warning` | Non-critical issues, skipped checks | Yes | Yes |
+| `error` | Failures, exceptions | Yes | Configurable |
+| `debug` | Detailed data, objects | Yes (attached) | Configurable |
## Comparison with console.log
@@ -251,34 +386,34 @@ test('should log auth flow', async ({ authToken, apiRequest }) => {
## Anti-Patterns
-**β Logging objects in steps:**
+**DON'T log objects in steps:**
```typescript
await log.step({ user: 'test', action: 'create' }); // Shows empty in UI
```
-**β
Use strings for steps, objects for debug:**
+**DO use strings for steps, objects for debug:**
```typescript
await log.step('Creating user: test'); // Readable in UI
await log.debug({ user: 'test', action: 'create' }); // Detailed data
```
-**β Logging sensitive data:**
+**DON'T log sensitive data:**
```typescript
await log.info(`Password: ${password}`); // Security risk!
await log.info(`Token: ${authToken}`); // Full token exposed!
```
-**β
Use previews or omit sensitive data:**
+**DO use previews or omit sensitive data:**
```typescript
await log.info('User authenticated successfully'); // No sensitive data
await log.debug({ tokenPreview: token.slice(0, 6) + '...' });
```
-**β Excessive logging in loops:**
+**DON'T log excessively in loops:**
```typescript
for (const item of items) {
@@ -286,7 +421,7 @@ for (const item of items) {
}
```
-**β
Log summary or use debug level:**
+**DO log summary or use debug level:**
```typescript
await log.step(`Processing ${items.length} items`);
diff --git a/src/modules/bmm/testarch/knowledge/network-error-monitor.md b/src/modules/bmm/testarch/knowledge/network-error-monitor.md
index 0a2321bd..a5ee9069 100644
--- a/src/modules/bmm/testarch/knowledge/network-error-monitor.md
+++ b/src/modules/bmm/testarch/knowledge/network-error-monitor.md
@@ -21,6 +21,19 @@ The `network-error-monitor` provides:
- **Smart opt-out**: Disable for validation tests expecting errors
- **Deduplication**: Group repeated errors by pattern
- **Domino effect prevention**: Limit test failures per error pattern
+- **Respects test status**: Won't suppress actual test failures
+
+## Quick Start
+
+```typescript
+import { test } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
+
+// That's it! Network monitoring is automatically enabled
+test('my test', async ({ page }) => {
+ await page.goto('/dashboard');
+ // If any HTTP 4xx/5xx errors occur, the test will fail
+});
+```
## Pattern Examples
@@ -38,8 +51,8 @@ test('should load dashboard', async ({ page }) => {
await page.goto('/dashboard');
await expect(page.locator('h1')).toContainText('Dashboard');
- // β
Passes if no HTTP errors
- // β Fails if any 4xx/5xx errors detected with clear message:
+ // Passes if no HTTP errors
+ // Fails if any 4xx/5xx errors detected with clear message:
// "Network errors detected: 2 request(s) failed"
// Failed requests:
// GET 500 https://api.example.com/users
@@ -64,13 +77,17 @@ test('should load dashboard', async ({ page }) => {
import { test } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
// Opt-out with annotation
-test('should show error on invalid input', { annotation: [{ type: 'skipNetworkMonitoring' }] }, async ({ page }) => {
- await page.goto('/form');
- await page.click('#submit'); // Triggers 400 error
+test(
+ 'should show error on invalid input',
+ { annotation: [{ type: 'skipNetworkMonitoring' }] },
+ async ({ page }) => {
+ await page.goto('/form');
+ await page.click('#submit'); // Triggers 400 error
- // Monitoring disabled - test won't fail on 400
- await expect(page.getByText('Invalid input')).toBeVisible();
-});
+ // Monitoring disabled - test won't fail on 400
+ await expect(page.getByText('Invalid input')).toBeVisible();
+ }
+);
// Or opt-out entire describe block
test.describe('error handling', { annotation: [{ type: 'skipNetworkMonitoring' }] }, () => {
@@ -91,7 +108,139 @@ test.describe('error handling', { annotation: [{ type: 'skipNetworkMonitoring' }
- Monitoring still active for other tests
- Perfect for intentional error scenarios
-### Example 3: Integration with Merged Fixtures
+### Example 3: Respects Test Status
+
+**Context**: The monitor respects final test statuses to avoid suppressing important test outcomes.
+
+**Behavior by test status:**
+
+- **`failed`**: Network errors logged as additional context, not thrown
+- **`timedOut`**: Network errors logged as additional context
+- **`skipped`**: Network errors logged, skip status preserved
+- **`interrupted`**: Network errors logged, interrupted status preserved
+- **`passed`**: Network errors throw and fail the test
+
+**Example with test.skip():**
+
+```typescript
+test('feature gated test', async ({ page }) => {
+ const featureEnabled = await checkFeatureFlag();
+ test.skip(!featureEnabled, 'Feature not enabled');
+ // If skipped, network errors won't turn this into a failure
+ await page.goto('/new-feature');
+});
+```
+
+### Example 4: Excluding Legitimate Errors
+
+**Context**: Some endpoints legitimately return 4xx/5xx responses.
+
+**Implementation**:
+
+```typescript
+import { test as base } from '@playwright/test';
+import { createNetworkErrorMonitorFixture } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
+
+export const test = base.extend(
+ createNetworkErrorMonitorFixture({
+ excludePatterns: [
+ /email-cluster\/ml-app\/has-active-run/, // ML service returns 404 when no active run
+ /idv\/session-templates\/list/, // IDV service returns 404 when not configured
+ /sentry\.io\/api/, // External Sentry errors should not fail tests
+ ],
+ })
+);
+```
+
+**For merged fixtures:**
+
+```typescript
+import { test as base, mergeTests } from '@playwright/test';
+import { createNetworkErrorMonitorFixture } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
+
+const networkErrorMonitor = base.extend(
+ createNetworkErrorMonitorFixture({
+ excludePatterns: [/analytics\.google\.com/, /cdn\.example\.com/],
+ })
+);
+
+export const test = mergeTests(authFixture, networkErrorMonitor);
+```
+
+### Example 5: Preventing Domino Effect
+
+**Context**: One failing endpoint shouldn't fail all tests.
+
+**Implementation**:
+
+```typescript
+import { test as base } from '@playwright/test';
+import { createNetworkErrorMonitorFixture } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
+
+const networkErrorMonitor = base.extend(
+ createNetworkErrorMonitorFixture({
+ excludePatterns: [], // Required when using maxTestsPerError
+ maxTestsPerError: 1, // Only first test fails per error pattern, rest just log
+ })
+);
+```
+
+**How it works:**
+
+When `/api/v2/case-management/cases` returns 500:
+
+- **First test** encountering this error: **FAILS** with clear error message
+- **Subsequent tests** encountering same error: **PASSES** but logs warning
+
+Error patterns are grouped by `method + status + base path`:
+
+- `GET /api/v2/case-management/cases/123` -> Pattern: `GET:500:/api/v2/case-management`
+- `GET /api/v2/case-management/quota` -> Pattern: `GET:500:/api/v2/case-management` (same group!)
+- `POST /api/v2/case-management/cases` -> Pattern: `POST:500:/api/v2/case-management` (different group!)
+
+**Why include HTTP method?** A GET 404 vs POST 404 might represent different issues:
+
+- `GET 404 /api/users/123` -> User not found (expected in some tests)
+- `POST 404 /api/users` -> Endpoint doesn't exist (critical error)
+
+**Output for subsequent tests:**
+
+```
+Warning: Network errors detected but not failing test (maxTestsPerError limit reached):
+ GET 500 https://api.example.com/api/v2/case-management/cases
+```
+
+**Recommended configuration:**
+
+```typescript
+createNetworkErrorMonitorFixture({
+ excludePatterns: [...], // Required - known broken endpoints (can be empty [])
+ maxTestsPerError: 1 // Stop domino effect (requires excludePatterns)
+})
+```
+
+**Understanding worker-level state:**
+
+Error pattern counts are stored in worker-level global state:
+
+```typescript
+// test-file-1.spec.ts (runs in Worker 1)
+test('test A', () => {
+ /* triggers GET:500:/api/v2/cases */
+}); // FAILS
+
+// test-file-2.spec.ts (runs later in Worker 1)
+test('test B', () => {
+ /* triggers GET:500:/api/v2/cases */
+}); // PASSES (limit reached)
+
+// test-file-3.spec.ts (runs in Worker 2 - different worker)
+test('test C', () => {
+ /* triggers GET:500:/api/v2/cases */
+}); // FAILS (fresh worker)
+```
+
+### Example 6: Integration with Merged Fixtures
**Context**: Combine network-error-monitor with other utilities.
@@ -105,7 +254,7 @@ import { test as networkErrorMonitorFixture } from '@seontechnologies/playwright
export const test = mergeTests(
authFixture,
- networkErrorMonitorFixture,
+ networkErrorMonitorFixture
// Add other fixtures
);
@@ -127,110 +276,94 @@ test('authenticated with monitoring', async ({ page, authToken }) => {
- Monitoring active automatically
- No extra setup needed
-### Example 4: Domino Effect Prevention
-
-**Context**: One failing endpoint shouldn't fail all tests.
-
-**Implementation**:
-
-```typescript
-// Configuration (internal to utility)
-const config = {
- maxTestsPerError: 3, // Max 3 tests fail per unique error pattern
-};
-
-// Scenario:
-// Test 1: GET /api/broken β 500 error β Test fails β
-// Test 2: GET /api/broken β 500 error β Test fails β
-// Test 3: GET /api/broken β 500 error β Test fails β
-// Test 4: GET /api/broken β 500 error β Test passes β οΈ (limit reached, warning logged)
-// Test 5: Different error pattern β Test fails β (new pattern, counter resets)
-```
-
-**Key Points**:
-
-- Limits cascading failures
-- Groups errors by URL + status code pattern
-- Warns when limit reached
-- Prevents flaky backend from failing entire suite
-
-### Example 5: Artifact Structure
+### Example 7: Artifact Structure
**Context**: Debugging failed tests with network error artifacts.
-**Implementation**:
-
When test fails due to network errors, artifact attached:
```json
-// test-results/my-test/network-errors.json
-{
- "errors": [
- {
- "url": "https://api.example.com/users",
- "method": "GET",
- "status": 500,
- "statusText": "Internal Server Error",
- "timestamp": "2024-08-13T10:30:45.123Z"
- },
- {
- "url": "https://api.example.com/metrics",
- "method": "POST",
- "status": 503,
- "statusText": "Service Unavailable",
- "timestamp": "2024-08-13T10:30:46.456Z"
- }
- ],
- "summary": {
- "totalErrors": 2,
- "uniquePatterns": 2
+[
+ {
+ "url": "https://api.example.com/users",
+ "status": 500,
+ "method": "GET",
+ "timestamp": "2025-11-10T12:34:56.789Z"
+ },
+ {
+ "url": "https://api.example.com/metrics",
+ "status": 503,
+ "method": "POST",
+ "timestamp": "2025-11-10T12:34:57.123Z"
}
-}
+]
```
-**Key Points**:
+## Implementation Details
-- JSON artifact per failed test
-- Full error details (URL, method, status, timestamp)
-- Summary statistics
-- Easy debugging with structured data
+### How It Works
-## Comparison with Manual Error Checks
+1. **Fixture Extension**: Uses Playwright's `base.extend()` with `auto: true`
+2. **Response Listener**: Attaches `page.on('response')` listener at test start
+3. **Multi-Page Monitoring**: Automatically monitors popups and new tabs via `context.on('page')`
+4. **Error Collection**: Captures 4xx/5xx responses, checking exclusion patterns
+5. **Try/Finally**: Ensures error processing runs even if test fails early
+6. **Status Check**: Only throws errors if test hasn't already reached final status
+7. **Artifact**: Attaches JSON file to test report for debugging
-| Manual Approach | network-error-monitor |
-| ------------------------------------------------------ | -------------------------- |
-| `page.on('response', resp => { if (!resp.ok()) ... })` | Auto-enabled, zero setup |
-| Check each response manually | Automatic for all requests |
-| Custom error tracking logic | Built-in deduplication |
-| No structured artifacts | JSON artifacts attached |
-| Easy to forget | Never miss a backend error |
+### Performance
+
+The monitor has minimal performance impact:
+
+- Event listener overhead: ~0.1ms per response
+- Memory: ~200 bytes per unique error
+- No network delay (observes responses, doesn't intercept them)
+
+## Comparison with Alternatives
+
+| Approach | Network Error Monitor | Manual afterEach |
+| --------------------------- | --------------------- | --------------------- |
+| **Setup Required** | Zero (auto-enabled) | Every test file |
+| **Catches Silent Failures** | Yes | Yes (if configured) |
+| **Structured Artifacts** | JSON attached | Custom impl |
+| **Test Failure Safety** | Try/finally | afterEach may not run |
+| **Opt-Out Mechanism** | Annotation | Custom logic |
+| **Status Aware** | Respects skip/failed | No |
## When to Use
**Auto-enabled for:**
-- β
All E2E tests
-- β
Integration tests
-- β
Any test hitting real APIs
+- All E2E tests
+- Integration tests
+- Any test hitting real APIs
**Opt-out for:**
-- β Validation tests (expecting 4xx)
-- β Error handling tests (expecting 5xx)
-- β Offline tests (network-recorder playback)
+- Validation tests (expecting 4xx)
+- Error handling tests (expecting 5xx)
+- Offline tests (network-recorder playback)
-## Integration with Framework Setup
+## Troubleshooting
-In `*framework` workflow, mention network-error-monitor:
+### Test fails with network errors but I don't see them in my app
+
+The errors might be happening during page load or in background polling. Check the `network-errors.json` artifact in your test report for full details including timestamps.
+
+### False positives from external services
+
+Configure exclusion patterns as shown in the "Excluding Legitimate Errors" section above.
+
+### Network errors not being caught
+
+Ensure you're importing the test from the correct fixture:
```typescript
-// Add to merged-fixtures.ts
-import { test as networkErrorMonitorFixture } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
+// Correct
+import { test } from '@seontechnologies/playwright-utils/network-error-monitor/fixtures';
-export const test = mergeTests(
- // ... other fixtures
- networkErrorMonitorFixture,
-);
+// Wrong - this won't have network monitoring
+import { test } from '@playwright/test';
```
## Related Fragments
@@ -241,14 +374,14 @@ export const test = mergeTests(
## Anti-Patterns
-**β Opting out of monitoring globally:**
+**DON'T opt out of monitoring globally:**
```typescript
// Every test skips monitoring
test.use({ annotation: [{ type: 'skipNetworkMonitoring' }] });
```
-**β
Opt-out only for specific error tests:**
+**DO opt-out only for specific error tests:**
```typescript
test.describe('error scenarios', { annotation: [{ type: 'skipNetworkMonitoring' }] }, () => {
@@ -256,17 +389,17 @@ test.describe('error scenarios', { annotation: [{ type: 'skipNetworkMonitoring'
});
```
-**β Ignoring network error artifacts:**
+**DON'T ignore network error artifacts:**
```typescript
// Test fails, artifact shows 500 errors
// Developer: "Works on my machine" Β―\_(γ)_/Β―
```
-**β
Check artifacts for root cause:**
+**DO check artifacts for root cause:**
```typescript
// Read network-errors.json artifact
-// Identify failing endpoint: GET /api/users β 500
+// Identify failing endpoint: GET /api/users -> 500
// Fix backend issue before merging
```
diff --git a/src/modules/bmm/testarch/knowledge/network-recorder.md b/src/modules/bmm/testarch/knowledge/network-recorder.md
index ff24cb4e..7b9c6e26 100644
--- a/src/modules/bmm/testarch/knowledge/network-recorder.md
+++ b/src/modules/bmm/testarch/knowledge/network-recorder.md
@@ -21,6 +21,46 @@ HAR-based recording/playback provides:
- **Stateful mocking**: CRUD operations work naturally (not just read-only)
- **Environment flexibility**: Map URLs for any environment
+## Quick Start
+
+### 1. Record Network Traffic
+
+```typescript
+// Set mode to 'record' to capture network traffic
+process.env.PW_NET_MODE = 'record';
+
+test('should add, edit and delete a movie', async ({ page, context, networkRecorder }) => {
+ // Setup network recorder - it will record all network traffic
+ await networkRecorder.setup(context);
+
+ // Your normal test code
+ await page.goto('/');
+ await page.fill('#movie-name', 'Inception');
+ await page.click('#add-movie');
+
+ // Network traffic is automatically saved to HAR file
+});
+```
+
+### 2. Playback Network Traffic
+
+```typescript
+// Set mode to 'playback' to use recorded traffic
+process.env.PW_NET_MODE = 'playback';
+
+test('should add, edit and delete a movie', async ({ page, context, networkRecorder }) => {
+ // Setup network recorder - it will replay from HAR file
+ await networkRecorder.setup(context);
+
+ // Same test code runs without hitting real backend!
+ await page.goto('/');
+ await page.fill('#movie-name', 'Inception');
+ await page.click('#add-movie');
+});
+```
+
+That's it! Your tests now run completely offline using recorded network traffic.
+
## Pattern Examples
### Example 1: Basic Record and Playback
@@ -115,74 +155,173 @@ test.describe('Movie CRUD - offline with network recorder', () => {
- Combine with `interceptNetworkCall` for deterministic waits
- First run records, subsequent runs replay
-### Example 3: Environment Switching
+### Example 3: Common Patterns
+
+**Recording Only API Calls**:
+
+```typescript
+await networkRecorder.setup(context, {
+ recording: {
+ urlFilter: /\/api\// // Only record API calls, ignore static assets
+ }
+});
+```
+
+**Playback with Fallback**:
+
+```typescript
+await networkRecorder.setup(context, {
+ playback: {
+ fallback: true // Fall back to live requests if HAR entry missing
+ }
+});
+```
+
+**Custom HAR File Location**:
+
+```typescript
+await networkRecorder.setup(context, {
+ harFile: {
+ harDir: 'recordings/api-calls',
+ baseName: 'user-journey',
+ organizeByTestFile: false // Optional: flatten directory structure
+ }
+});
+```
+
+**Directory Organization:**
+
+- `organizeByTestFile: true` (default): `har-files/test-file-name/baseName-test-title.har`
+- `organizeByTestFile: false`: `har-files/baseName-test-title.har`
+
+### Example 4: Response Content Storage - Embed vs Attach
+
+**Context**: Choose how response content is stored in HAR files.
+
+**`embed` (Default - Recommended):**
+
+```typescript
+await networkRecorder.setup(context, {
+ recording: {
+ content: 'embed' // Store content inline (default)
+ }
+});
+```
+
+**Pros:**
+
+- Single self-contained file - Easy to share, version control
+- Better for small-medium responses (API JSON, HTML pages)
+- HAR specification compliant
+
+**Cons:**
+
+- Larger HAR files
+- Not ideal for large binary content (images, videos)
+
+**`attach` (Alternative):**
+
+```typescript
+await networkRecorder.setup(context, {
+ recording: {
+ content: 'attach' // Store content separately
+ }
+});
+```
+
+**Pros:**
+
+- Smaller HAR files
+- Better for large responses (images, videos, documents)
+
+**Cons:**
+
+- Multiple files to manage
+- Harder to share
+
+**When to Use Each:**
+
+| Use `embed` (default) when | Use `attach` when |
+|---------------------------|-------------------|
+| Recording API responses (JSON, XML) | Recording large images, videos |
+| Small to medium HTML pages | HAR file size >50MB |
+| You want a single, portable file | Maximum disk efficiency needed |
+| Sharing HAR files with team | Working with ZIP archive output |
+
+### Example 5: Cross-Environment Compatibility (URL Mapping)
**Context**: Record in dev environment, play back in CI with different base URLs.
-**Implementation**:
+**The Problem**: HAR files contain URLs for the recording environment (e.g., `dev.example.com`). Playing back on a different environment fails.
+
+**Simple Hostname Mapping:**
```typescript
-// playwright.config.ts - Map URLs for different environments
-export default defineConfig({
- use: {
- baseURL: process.env.CI ? 'https://app.ci.example.com' : 'http://localhost:3000',
- },
-});
-
-// Test works in both environments
-test('cross-environment playback', async ({ page, context, networkRecorder }) => {
- await networkRecorder.setup(context);
-
- // In dev: hits http://localhost:3000/api/movies
- // In CI: HAR replays with https://app.ci.example.com/api/movies
- await page.goto('/movies');
-
- // Network recorder auto-maps URLs
- await expect(page.getByTestId('movie-list')).toBeVisible();
+await networkRecorder.setup(context, {
+ playback: {
+ urlMapping: {
+ hostMapping: {
+ 'preview.example.com': 'dev.example.com',
+ 'staging.example.com': 'dev.example.com',
+ 'localhost:3000': 'dev.example.com'
+ }
+ }
+ }
});
```
-**Key Points**:
-
-- HAR files record absolute URLs
-- Playback maps to current baseURL
-- Same HAR works across environments
-- No manual URL rewriting needed
-
-### Example 4: Automatic vs Manual Mode Control
-
-**Context**: Choose between environment-based switching or in-test mode control.
-
-**Implementation**:
+**Pattern-Based Mapping (Recommended):**
```typescript
-// Option 1: Environment variable (recommended for CI)
-PW_NET_MODE=record npm run test:pw # Record traffic
-PW_NET_MODE=playback npm run test:pw # Playback traffic
-
-// Option 2: In-test control (recommended for development)
-process.env.PW_NET_MODE = 'record' // Set at top of test file
-
-test('my test', async ({ page, context, networkRecorder }) => {
- await networkRecorder.setup(context)
- // ...
-})
-
-// Option 3: Auto-fallback (record if HAR missing, else playback)
-// This is the default behavior when PW_NET_MODE not set
-test('auto mode', async ({ page, context, networkRecorder }) => {
- await networkRecorder.setup(context)
- // First run: auto-records
- // Subsequent runs: auto-plays back
-})
+await networkRecorder.setup(context, {
+ playback: {
+ urlMapping: {
+ patterns: [
+ // Map any preview-XXXX subdomain to dev
+ { match: /preview-\d+\.example\.com/, replace: 'dev.example.com' }
+ ]
+ }
+ }
+});
```
-**Key Points**:
+**Custom Function:**
-- Three mode options: record, playback, auto
-- `PW_NET_MODE` environment variable
-- In-test `process.env.PW_NET_MODE` assignment
-- Auto-fallback when no mode specified
+```typescript
+await networkRecorder.setup(context, {
+ playback: {
+ urlMapping: {
+ mapUrl: (url) => url.replace('staging.example.com', 'dev.example.com')
+ }
+ }
+});
+```
+
+**Complex Multi-Environment Example:**
+
+```typescript
+await networkRecorder.setup(context, {
+ playback: {
+ urlMapping: {
+ hostMapping: {
+ 'localhost:3000': 'admin.seondev.space',
+ 'admin-staging.seon.io': 'admin.seondev.space',
+ 'admin.seon.io': 'admin.seondev.space',
+ },
+ patterns: [
+ { match: /admin-\d+\.seondev\.space/, replace: 'admin.seondev.space' },
+ { match: /admin-staging-pr-\w+-\d\.seon\.io/, replace: 'admin.seondev.space' }
+ ]
+ }
+ }
+});
+```
+
+**Benefits:**
+
+- Record once on dev, all environments map back to recordings
+- CORS headers automatically updated based on request origin
+- Debug with: `LOG_LEVEL=debug npm run test`
## Why Use This Instead of Native Playwright?
@@ -191,7 +330,7 @@ test('auto mode', async ({ page, context, networkRecorder }) => {
| ~80 lines setup boilerplate | ~5 lines total |
| Manual HAR file management | Automatic file organization |
| Complex setup/teardown | Automatic cleanup via fixtures |
-| **Read-only tests** | **Full CRUD support** |
+| **Read-only tests only** | **Full CRUD support** |
| **Stateless** | **Stateful mocking** |
| Manual URL mapping | Automatic environment mapping |
@@ -199,9 +338,132 @@ test('auto mode', async ({ page, context, networkRecorder }) => {
Native Playwright HAR playback is stateless - a POST create followed by GET list won't show the created item. This utility intelligently tracks CRUD operations in memory to reflect state changes, making offline tests behave like real APIs.
+## How Stateful CRUD Detection Works
+
+When in playback mode, the Network Recorder automatically analyzes your HAR file to detect CRUD patterns. If it finds:
+
+- Multiple GET requests to the same resource endpoint (e.g., `/movies`)
+- Mutation operations (POST, PUT, DELETE) to those resources
+- Evidence of state changes between identical requests
+
+It automatically switches from static HAR playback to an intelligent stateful mock that:
+
+- Maintains state across requests
+- Auto-generates IDs for new resources
+- Returns proper 404s for deleted resources
+- Supports polling scenarios where state changes over time
+
+**This happens automatically - no configuration needed!**
+
+## API Reference
+
+### NetworkRecorder Methods
+
+| Method | Return Type | Description |
+| -------------------- | ------------------------ | ----------------------------------------------------- |
+| `setup(context)` | `Promise` | Sets up recording/playback on browser context |
+| `cleanup()` | `Promise` | Flushes data to disk and cleans up memory |
+| `getContext()` | `NetworkRecorderContext` | Gets current recorder context information |
+| `getStatusMessage()` | `string` | Gets human-readable status message |
+| `getHarStats()` | `Promise` | Gets HAR file statistics and metadata |
+
+### Understanding `cleanup()`
+
+The `cleanup()` method performs memory and resource cleanup - **it does NOT delete HAR files**:
+
+**What it does:**
+
+- Flushes recorded data to disk (writes HAR file in recording mode)
+- Releases file locks
+- Clears in-memory data
+- Resets internal state
+
+**What it does NOT do:**
+
+- Delete HAR files from disk
+- Remove recorded network traffic
+- Clear browser context or cookies
+
+### Configuration Options
+
+```typescript
+type NetworkRecorderConfig = {
+ harFile?: {
+ harDir?: string // Directory for HAR files (default: 'har-files')
+ baseName?: string // Base name for HAR files (default: 'network-traffic')
+ organizeByTestFile?: boolean // Organize by test file (default: true)
+ }
+
+ recording?: {
+ content?: 'embed' | 'attach' // Response content handling (default: 'embed')
+ urlFilter?: string | RegExp // URL filter for recording
+ update?: boolean // Update existing HAR files (default: false)
+ }
+
+ playback?: {
+ fallback?: boolean // Fall back to live requests (default: false)
+ urlFilter?: string | RegExp // URL filter for playback
+ updateMode?: boolean // Update mode during playback (default: false)
+ }
+
+ forceMode?: 'record' | 'playback' | 'disabled'
+}
+```
+
+## Environment Configuration
+
+Control the recording mode using the `PW_NET_MODE` environment variable:
+
+```bash
+# Record mode - captures network traffic to HAR files
+PW_NET_MODE=record npm run test:pw
+
+# Playback mode - replays network traffic from HAR files
+PW_NET_MODE=playback npm run test:pw
+
+# Disabled mode - no network recording/playback
+PW_NET_MODE=disabled npm run test:pw
+
+# Default behavior (when PW_NET_MODE is empty/unset) - same as disabled
+npm run test:pw
+```
+
+**Tip**: We recommend setting `process.env.PW_NET_MODE` directly in your test file for better control.
+
+## Troubleshooting
+
+### HAR File Not Found
+
+If you see "HAR file not found" errors during playback:
+
+1. Ensure you've recorded the test first with `PW_NET_MODE=record`
+2. Check the HAR file exists in the expected location (usually `har-files/`)
+3. Enable fallback mode: `playback: { fallback: true }`
+
+### Authentication and Network Recording
+
+The network recorder works seamlessly with authentication:
+
+```typescript
+test('Authenticated recording', async ({ page, context, authSession, networkRecorder }) => {
+ // First authenticate
+ await authSession.login('testuser', 'password');
+
+ // Then setup network recording with authenticated context
+ await networkRecorder.setup(context);
+
+ // Test authenticated flows
+ await page.goto('/dashboard');
+});
+```
+
+### Concurrent Test Issues
+
+The recorder includes built-in file locking for safe parallel execution. Each test gets its own HAR file based on the test name.
+
## Integration with Other Utilities
-**With interceptNetworkCall** (deterministic waits):
+**With interceptNetworkCall (deterministic waits):**
```typescript
test('use both utilities', async ({ page, context, networkRecorder, interceptNetworkCall }) => {
@@ -228,7 +490,7 @@ test('use both utilities', async ({ page, context, networkRecorder, interceptNet
## Anti-Patterns
-**β Mixing record and playback in same test:**
+**DON'T mix record and playback in same test:**
```typescript
process.env.PW_NET_MODE = 'record';
@@ -236,7 +498,7 @@ process.env.PW_NET_MODE = 'record';
process.env.PW_NET_MODE = 'playback'; // Don't switch mid-test
```
-**β
One mode per test:**
+**DO use one mode per test:**
```typescript
process.env.PW_NET_MODE = 'playback'; // Set once at top
@@ -247,7 +509,7 @@ test('my test', async ({ page, context, networkRecorder }) => {
});
```
-**β Forgetting to call setup:**
+**DON'T forget to call setup:**
```typescript
test('broken', async ({ page, networkRecorder }) => {
@@ -255,7 +517,7 @@ test('broken', async ({ page, networkRecorder }) => {
});
```
-**β
Always call setup before navigation:**
+**DO always call setup before navigation:**
```typescript
test('correct', async ({ page, context, networkRecorder }) => {
diff --git a/src/modules/bmm/testarch/knowledge/overview.md b/src/modules/bmm/testarch/knowledge/overview.md
index 8155d55f..a539f03f 100644
--- a/src/modules/bmm/testarch/knowledge/overview.md
+++ b/src/modules/bmm/testarch/knowledge/overview.md
@@ -2,7 +2,7 @@
## Principle
-Use production-ready, fixture-based utilities from `@seontechnologies/playwright-utils` for common Playwright testing patterns. Build test helpers as pure functions first, then wrap in framework-specific fixtures for composability and reuse.
+Use production-ready, fixture-based utilities from `@seontechnologies/playwright-utils` for common Playwright testing patterns. Build test helpers as pure functions first, then wrap in framework-specific fixtures for composability and reuse. **Works equally well for pure API testing (no browser) and UI testing.**
## Rationale
@@ -20,6 +20,7 @@ Writing Playwright utilities from scratch for every project leads to:
- **Composable fixtures**: Use `mergeTests` to combine utilities
- **TypeScript support**: Full type safety with generic types
- **Comprehensive coverage**: API requests, auth, network, logging, file handling, burn-in
+- **Backend-first mentality**: Most utilities work without a browser - pure API/service testing is a first-class use case
## Installation
@@ -37,17 +38,19 @@ npm install -D @seontechnologies/playwright-utils
### Core Testing Utilities
-| Utility | Purpose | Test Context |
-| -------------------------- | ------------------------------------------ | ------------- |
-| **api-request** | Typed HTTP client with schema validation | API tests |
-| **network-recorder** | HAR record/playback for offline testing | UI tests |
-| **auth-session** | Token persistence, multi-user auth | Both UI & API |
-| **recurse** | Cypress-style polling for async conditions | Both UI & API |
-| **intercept-network-call** | Network spy/stub with auto JSON parsing | UI tests |
-| **log** | Playwright report-integrated logging | Both UI & API |
-| **file-utils** | CSV/XLSX/PDF/ZIP reading & validation | Both UI & API |
-| **burn-in** | Smart test selection with git diff | CI/CD |
-| **network-error-monitor** | Automatic HTTP 4xx/5xx detection | UI tests |
+| Utility | Purpose | Test Context |
+| -------------------------- | ---------------------------------------------------- | ------------------ |
+| **api-request** | Typed HTTP client with schema validation and retry | **API/Backend** |
+| **recurse** | Polling for async operations, background jobs | **API/Backend** |
+| **auth-session** | Token persistence, multi-user, service-to-service | **API/Backend/UI** |
+| **log** | Playwright report-integrated logging | **API/Backend/UI** |
+| **file-utils** | CSV/XLSX/PDF/ZIP reading & validation | **API/Backend/UI** |
+| **burn-in** | Smart test selection with git diff | **CI/CD** |
+| **network-recorder** | HAR record/playback for offline testing | UI only |
+| **intercept-network-call** | Network spy/stub with auto JSON parsing | UI only |
+| **network-error-monitor** | Automatic HTTP 4xx/5xx detection | UI only |
+
+**Note**: 6 of 9 utilities work without a browser. Only 3 are UI-specific (network-recorder, intercept-network-call, network-error-monitor).
## Design Patterns
diff --git a/src/modules/bmm/testarch/knowledge/recurse.md b/src/modules/bmm/testarch/knowledge/recurse.md
index aec553a1..d9536400 100644
--- a/src/modules/bmm/testarch/knowledge/recurse.md
+++ b/src/modules/bmm/testarch/knowledge/recurse.md
@@ -2,7 +2,7 @@
## Principle
-Use Cypress-style polling with Playwright's `expect.poll` to wait for asynchronous conditions. Provides configurable timeout, interval, logging, and post-polling callbacks with enhanced error categorization.
+Use Cypress-style polling with Playwright's `expect.poll` to wait for asynchronous conditions. Provides configurable timeout, interval, logging, and post-polling callbacks with enhanced error categorization. **Ideal for backend testing**: polling API endpoints for job completion, database eventual consistency, message queue processing, and cache propagation.
## Rationale
@@ -21,6 +21,29 @@ The `recurse` utility provides:
- **Post-poll callbacks**: Process results after success
- **Type-safe**: Full TypeScript generic support
+## Quick Start
+
+```typescript
+import { test } from '@seontechnologies/playwright-utils/recurse/fixtures';
+
+test('wait for job completion', async ({ recurse, apiRequest }) => {
+ const { body } = await apiRequest({
+ method: 'POST',
+ path: '/api/jobs',
+ body: { type: 'export' },
+ });
+
+ // Poll until job completes
+ const result = await recurse(
+ () => apiRequest({ method: 'GET', path: `/api/jobs/${body.id}` }),
+ (response) => response.body.status === 'completed',
+ { timeout: 60000 }
+ );
+
+ expect(result.body.downloadUrl).toBeDefined();
+});
+```
+
## Pattern Examples
### Example 1: Basic Polling
@@ -48,7 +71,7 @@ test('should wait for job completion', async ({ recurse, apiRequest }) => {
timeout: 60000, // 60 seconds max
interval: 2000, // Check every 2 seconds
log: 'Waiting for export job to complete',
- },
+ }
);
expect(result.body.downloadUrl).toBeDefined();
@@ -62,7 +85,7 @@ test('should wait for job completion', async ({ recurse, apiRequest }) => {
- Options: timeout, interval, log message
- Returns the value when predicate returns true
-### Example 2: Polling with Assertions
+### Example 2: Working with Assertions
**Context**: Use assertions directly in predicate for more expressive tests.
@@ -76,35 +99,76 @@ test('should poll with assertions', async ({ recurse, apiRequest }) => {
body: { type: 'user-created', userId: '123' },
});
- // Poll with assertions in predicate
+ // Poll with assertions in predicate - no return true needed!
await recurse(
async () => {
const { body } = await apiRequest({ method: 'GET', path: '/api/events/123' });
return body;
},
(event) => {
- // Use assertions instead of boolean returns
+ // If all assertions pass, predicate succeeds
expect(event.processed).toBe(true);
expect(event.timestamp).toBeDefined();
- // If assertions pass, predicate succeeds
+ // No need to return true - just let assertions pass
},
- { timeout: 30000 },
+ { timeout: 30000 }
);
});
```
-**Key Points**:
+**Why no `return true` needed?**
-- Predicate can use `expect()` assertions
-- If assertions throw, polling continues
-- If assertions pass, polling succeeds
-- More expressive than boolean returns
+The predicate checks for "truthiness" of the return value. But there's a catch - in JavaScript, an empty `return` (or no return) returns `undefined`, which is falsy!
-### Example 3: Custom Error Messages
+The utility handles this by checking if:
-**Context**: Provide context-specific error messages for timeout failures.
+1. The predicate didn't throw (assertions passed)
+2. The return value was either `undefined` (implicit return) or truthy
-**Implementation**:
+So you can:
+
+```typescript
+// Option 1: Use assertions only (recommended)
+(event) => {
+ expect(event.processed).toBe(true);
+};
+
+// Option 2: Return boolean (also works)
+(event) => event.processed === true;
+
+// Option 3: Mixed (assertions + explicit return)
+(event) => {
+ expect(event.processed).toBe(true);
+ return true;
+};
+```
+
+### Example 3: Error Handling
+
+**Context**: Understanding the different error types.
+
+**Error Types:**
+
+```typescript
+// RecurseTimeoutError - Predicate never returned true within timeout
+// Contains last command value and predicate error
+try {
+ await recurse(/* ... */);
+} catch (error) {
+ if (error instanceof RecurseTimeoutError) {
+ console.log('Timed out. Last value:', error.lastCommandValue);
+ console.log('Last predicate error:', error.lastPredicateError);
+ }
+}
+
+// RecurseCommandError - Command function threw an error
+// The command itself failed (e.g., network error, API error)
+
+// RecursePredicateError - Predicate function threw (not from assertions failing)
+// Logic error in your predicate code
+```
+
+**Custom Error Messages:**
```typescript
test('custom error on timeout', async ({ recurse, apiRequest }) => {
@@ -115,7 +179,7 @@ test('custom error on timeout', async ({ recurse, apiRequest }) => {
{
timeout: 10000,
error: 'System failed to become ready within 10 seconds - check background workers',
- },
+ }
);
} catch (error) {
// Error message includes custom context
@@ -125,13 +189,6 @@ test('custom error on timeout', async ({ recurse, apiRequest }) => {
});
```
-**Key Points**:
-
-- `error` option provides custom message
-- Replaces default "Timed out after X ms"
-- Include debugging hints in error message
-- Helps diagnose failures faster
-
### Example 4: Post-Polling Callback
**Context**: Process or log results after successful polling.
@@ -151,7 +208,7 @@ test('post-poll processing', async ({ recurse, apiRequest }) => {
console.log(`Processed ${result.body.itemsProcessed} items`);
return result.body;
},
- },
+ }
);
expect(finalResult.itemsProcessed).toBeGreaterThan(0);
@@ -165,7 +222,67 @@ test('post-poll processing', async ({ recurse, apiRequest }) => {
- Can transform or log results
- Return value becomes final `recurse` result
-### Example 5: Integration with API Request (Common Pattern)
+### Example 5: UI Testing Scenarios
+
+**Context**: Wait for UI elements to reach a specific state through polling.
+
+**Implementation**:
+
+```typescript
+test('table data loads', async ({ page, recurse }) => {
+ await page.goto('/reports');
+
+ // Poll for table rows to appear
+ await recurse(
+ async () => page.locator('table tbody tr').count(),
+ (count) => count >= 10, // Wait for at least 10 rows
+ {
+ timeout: 15000,
+ interval: 500,
+ log: 'Waiting for table data to load',
+ }
+ );
+
+ // Now safe to interact with table
+ await page.locator('table tbody tr').first().click();
+});
+```
+
+### Example 6: Event-Based Systems (Kafka/Message Queues)
+
+**Context**: Testing eventual consistency with message queue processing.
+
+**Implementation**:
+
+```typescript
+test('kafka event processed', async ({ recurse, apiRequest }) => {
+ // Trigger action that publishes Kafka event
+ await apiRequest({
+ method: 'POST',
+ path: '/api/orders',
+ body: { productId: 'ABC123', quantity: 2 },
+ });
+
+ // Poll for downstream effect of Kafka consumer processing
+ const inventoryResult = await recurse(
+ () => apiRequest({ method: 'GET', path: '/api/inventory/ABC123' }),
+ (res) => {
+ // Assumes test fixture seeds inventory at 100; in production tests,
+ // fetch baseline first and assert: expect(res.body.available).toBe(baseline - 2)
+ expect(res.body.available).toBeLessThanOrEqual(98);
+ },
+ {
+ timeout: 30000, // Kafka processing may take time
+ interval: 1000,
+ log: 'Waiting for Kafka event to be processed',
+ }
+ );
+
+ expect(inventoryResult.body.lastOrderId).toBeDefined();
+});
+```
+
+### Example 7: Integration with API Request (Common Pattern)
**Context**: Most common use case - polling API endpoints for state changes.
@@ -193,7 +310,7 @@ test('end-to-end polling', async ({ apiRequest, recurse }) => {
timeout: 120000, // 2 minutes for large imports
interval: 5000, // Check every 5 seconds
log: `Polling import ${createResp.importId}`,
- },
+ }
);
expect(importResult.body.rowsImported).toBeGreaterThan(1000);
@@ -208,20 +325,26 @@ test('end-to-end polling', async ({ apiRequest, recurse }) => {
- Complex predicates with multiple conditions
- Logging shows polling progress in test reports
-## Enhanced Error Types
+## API Reference
-The utility categorizes errors for easier debugging:
+### RecurseOptions
-```typescript
-// TimeoutError - Predicate never returned true
-Error: Polling timed out after 30000ms: Job never completed
+| Option | Type | Default | Description |
+| ---------- | ------------------ | ----------- | ------------------------------------ |
+| `timeout` | `number` | `30000` | Maximum time to wait (ms) |
+| `interval` | `number` | `1000` | Time between polls (ms) |
+| `log` | `string` | `undefined` | Message logged on each poll |
+| `error` | `string` | `undefined` | Custom error message for timeout |
+| `post` | `(result: T) => R` | `undefined` | Callback after successful poll |
+| `delay` | `number` | `0` | Initial delay before first poll (ms) |
-// CommandError - Command function threw
-Error: Command failed: Request failed with status 500
+### Error Types
-// PredicateError - Predicate function threw (not from assertions)
-Error: Predicate failed: Cannot read property 'status' of undefined
-```
+| Error Type | When Thrown | Properties |
+| ----------------------- | --------------------------------------- | ---------------------------------------- |
+| `RecurseTimeoutError` | Predicate never passed within timeout | `lastCommandValue`, `lastPredicateError` |
+| `RecurseCommandError` | Command function threw an error | `cause` (original error) |
+| `RecursePredicateError` | Predicate threw (not assertion failure) | `cause` (original error) |
## Comparison with Vanilla Playwright
@@ -236,11 +359,11 @@ Error: Predicate failed: Cannot read property 'status' of undefined
**Use recurse for:**
-- β
Background job completion
-- β
Webhook/event processing
-- β
Database eventual consistency
-- β
Cache propagation
-- β
State machine transitions
+- Background job completion
+- Webhook/event processing
+- Database eventual consistency
+- Cache propagation
+- State machine transitions
**Stick with vanilla expect.poll for:**
@@ -250,13 +373,15 @@ Error: Predicate failed: Cannot read property 'status' of undefined
## Related Fragments
+- `api-testing-patterns.md` - Comprehensive pure API testing patterns
- `api-request.md` - Combine for API endpoint polling
- `overview.md` - Fixture composition patterns
- `fixtures-composition.md` - Using with mergeTests
+- `contract-testing.md` - Contract testing with async verification
## Anti-Patterns
-**β Using hard waits instead of polling:**
+**DON'T use hard waits instead of polling:**
```typescript
await page.click('#export');
@@ -264,33 +389,33 @@ await page.waitForTimeout(5000); // Arbitrary wait
expect(await page.textContent('#status')).toBe('Ready');
```
-**β
Poll for actual condition:**
+**DO poll for actual condition:**
```typescript
await page.click('#export');
await recurse(
() => page.textContent('#status'),
(status) => status === 'Ready',
- { timeout: 10000 },
+ { timeout: 10000 }
);
```
-**β Polling too frequently:**
+**DON'T poll too frequently:**
```typescript
await recurse(
() => apiRequest({ method: 'GET', path: '/status' }),
(res) => res.body.ready,
- { interval: 100 }, // Hammers API every 100ms!
+ { interval: 100 } // Hammers API every 100ms!
);
```
-**β
Reasonable interval for API calls:**
+**DO use reasonable interval for API calls:**
```typescript
await recurse(
() => apiRequest({ method: 'GET', path: '/status' }),
(res) => res.body.ready,
- { interval: 2000 }, // Check every 2 seconds (reasonable)
+ { interval: 2000 } // Check every 2 seconds (reasonable)
);
```
diff --git a/src/modules/bmm/testarch/tea-index.csv b/src/modules/bmm/testarch/tea-index.csv
index cf1efd67..4546d351 100644
--- a/src/modules/bmm/testarch/tea-index.csv
+++ b/src/modules/bmm/testarch/tea-index.csv
@@ -1,33 +1,34 @@
id,name,description,tags,fragment_file
fixture-architecture,Fixture Architecture,"Composable fixture patterns (pure function β fixture β merge) and reuse rules","fixtures,architecture,playwright,cypress",knowledge/fixture-architecture.md
-network-first,Network-First Safeguards,"Intercept-before-navigate workflow, HAR capture, deterministic waits, edge mocking","network,stability,playwright,cypress",knowledge/network-first.md
-data-factories,Data Factories and API Setup,"Factories with overrides, API seeding, cleanup discipline","data,factories,setup,api",knowledge/data-factories.md
+network-first,Network-First Safeguards,"Intercept-before-navigate workflow, HAR capture, deterministic waits, edge mocking","network,stability,playwright,cypress,ui",knowledge/network-first.md
+data-factories,Data Factories and API Setup,"Factories with overrides, API seeding, cleanup discipline","data,factories,setup,api,backend,seeding",knowledge/data-factories.md
component-tdd,Component TDD Loop,"Redβgreenβrefactor workflow, provider isolation, accessibility assertions","component-testing,tdd,ui",knowledge/component-tdd.md
playwright-config,Playwright Config Guardrails,"Environment switching, timeout standards, artifact outputs","playwright,config,env",knowledge/playwright-config.md
ci-burn-in,CI and Burn-In Strategy,"Staged jobs, shard orchestration, burn-in loops, artifact policy","ci,automation,flakiness",knowledge/ci-burn-in.md
selective-testing,Selective Test Execution,"Tag/grep usage, spec filters, diff-based runs, promotion rules","risk-based,selection,strategy",knowledge/selective-testing.md
feature-flags,Feature Flag Governance,"Enum management, targeting helpers, cleanup, release checklists","feature-flags,governance,launchdarkly",knowledge/feature-flags.md
-contract-testing,Contract Testing Essentials,"Pact publishing, provider verification, resilience coverage","contract-testing,pact,api",knowledge/contract-testing.md
+contract-testing,Contract Testing Essentials,"Pact publishing, provider verification, resilience coverage","contract-testing,pact,api,backend,microservices,service-contract",knowledge/contract-testing.md
email-auth,Email Authentication Testing,"Magic link extraction, state preservation, caching, negative flows","email-authentication,security,workflow",knowledge/email-auth.md
-error-handling,Error Handling Checks,"Scoped exception handling, retry validation, telemetry logging","resilience,error-handling,stability",knowledge/error-handling.md
-visual-debugging,Visual Debugging Toolkit,"Trace viewer usage, artifact expectations, accessibility integration","debugging,dx,tooling",knowledge/visual-debugging.md
+error-handling,Error Handling Checks,"Scoped exception handling, retry validation, telemetry logging","resilience,error-handling,stability,api,backend",knowledge/error-handling.md
+visual-debugging,Visual Debugging Toolkit,"Trace viewer usage, artifact expectations, accessibility integration","debugging,dx,tooling,ui",knowledge/visual-debugging.md
risk-governance,Risk Governance,"Scoring matrix, category ownership, gate decision rules","risk,governance,gates",knowledge/risk-governance.md
probability-impact,Probability and Impact Scale,"Shared definitions for scoring matrix and gate thresholds","risk,scoring,scale",knowledge/probability-impact.md
test-quality,Test Quality Definition of Done,"Execution limits, isolation rules, green criteria","quality,definition-of-done,tests",knowledge/test-quality.md
nfr-criteria,NFR Review Criteria,"Security, performance, reliability, maintainability status definitions","nfr,assessment,quality",knowledge/nfr-criteria.md
-test-levels,Test Levels Framework,"Guidelines for choosing unit, integration, or end-to-end coverage","testing,levels,selection",knowledge/test-levels-framework.md
+test-levels,Test Levels Framework,"Guidelines for choosing unit, integration, or end-to-end coverage","testing,levels,selection,api,backend,ui",knowledge/test-levels-framework.md
test-priorities,Test Priorities Matrix,"P0βP3 criteria, coverage targets, execution ordering","testing,prioritization,risk",knowledge/test-priorities-matrix.md
test-healing-patterns,Test Healing Patterns,"Common failure patterns and automated fixes","healing,debugging,patterns",knowledge/test-healing-patterns.md
-selector-resilience,Selector Resilience,"Robust selector strategies and debugging techniques","selectors,locators,debugging",knowledge/selector-resilience.md
+selector-resilience,Selector Resilience,"Robust selector strategies and debugging techniques","selectors,locators,debugging,ui",knowledge/selector-resilience.md
timing-debugging,Timing Debugging,"Race condition identification and deterministic wait fixes","timing,async,debugging",knowledge/timing-debugging.md
-overview,Playwright Utils Overview,"Installation, design principles, fixture patterns","playwright-utils,fixtures",knowledge/overview.md
-api-request,API Request,"Typed HTTP client, schema validation","api,playwright-utils",knowledge/api-request.md
-network-recorder,Network Recorder,"HAR record/playback, CRUD detection","network,playwright-utils",knowledge/network-recorder.md
-auth-session,Auth Session,"Token persistence, multi-user","auth,playwright-utils",knowledge/auth-session.md
-intercept-network-call,Intercept Network Call,"Network spy/stub, JSON parsing","network,playwright-utils",knowledge/intercept-network-call.md
-recurse,Recurse Polling,"Async polling, condition waiting","polling,playwright-utils",knowledge/recurse.md
-log,Log Utility,"Report logging, structured output","logging,playwright-utils",knowledge/log.md
-file-utils,File Utilities,"CSV/XLSX/PDF/ZIP validation","files,playwright-utils",knowledge/file-utils.md
-burn-in,Burn-in Runner,"Smart test selection, git diff","ci,playwright-utils",knowledge/burn-in.md
-network-error-monitor,Network Error Monitor,"HTTP 4xx/5xx detection","monitoring,playwright-utils",knowledge/network-error-monitor.md
-fixtures-composition,Fixtures Composition,"mergeTests composition patterns","fixtures,playwright-utils",knowledge/fixtures-composition.md
+overview,Playwright Utils Overview,"Installation, design principles, fixture patterns for API and UI testing","playwright-utils,fixtures,api,backend,ui",knowledge/overview.md
+api-request,API Request,"Typed HTTP client, schema validation, retry logic for API and service testing","api,backend,service-testing,api-testing,playwright-utils",knowledge/api-request.md
+network-recorder,Network Recorder,"HAR record/playback, CRUD detection for offline UI testing","network,playwright-utils,ui,har",knowledge/network-recorder.md
+auth-session,Auth Session,"Token persistence, multi-user, API and browser authentication","auth,playwright-utils,api,backend,jwt,token",knowledge/auth-session.md
+intercept-network-call,Intercept Network Call,"Network spy/stub, JSON parsing for UI tests","network,playwright-utils,ui",knowledge/intercept-network-call.md
+recurse,Recurse Polling,"Async polling for API responses, background jobs, eventual consistency","polling,playwright-utils,api,backend,async,eventual-consistency",knowledge/recurse.md
+log,Log Utility,"Report logging, structured output for API and UI tests","logging,playwright-utils,api,ui",knowledge/log.md
+file-utils,File Utilities,"CSV/XLSX/PDF/ZIP validation for API exports and UI downloads","files,playwright-utils,api,backend,ui",knowledge/file-utils.md
+burn-in,Burn-in Runner,"Smart test selection, git diff for CI optimization","ci,playwright-utils",knowledge/burn-in.md
+network-error-monitor,Network Error Monitor,"HTTP 4xx/5xx detection for UI tests","monitoring,playwright-utils,ui",knowledge/network-error-monitor.md
+fixtures-composition,Fixtures Composition,"mergeTests composition patterns for combining utilities","fixtures,playwright-utils",knowledge/fixtures-composition.md
+api-testing-patterns,API Testing Patterns,"Pure API test patterns without browser: service testing, microservices, GraphQL","api,backend,service-testing,api-testing,microservices,graphql,no-browser",knowledge/api-testing-patterns.md