feat: transform QA agent into Test Architect with advanced quality ca… (#433)

* feat: transform QA agent into Test Architect with advanced quality capabilities - Add 6 specialized quality assessment commands - Implement risk-based testing with scoring - Create quality gate system with deterministic decisions - Add comprehensive test design and NFR validation - Update documentation with stage-based workflow integration * feat: transform QA agent into Test Architect with advanced quality capabilities - Add 6 specialized quality assessment commands - Implement risk-based testing with scoring - Create quality gate system with deterministic decisions - Add comprehensive test design and NFR validation - Update documentation with stage-based workflow integration * docs: refined the docs for test architect * fix: addressed review comments from manjaroblack, round 1 * fix: addressed review comments from manjaroblack, round 1 --------- Co-authored-by: Murat Ozcan <murat@mac.lan> Co-authored-by: Brian <bmadcode@gmail.com>
2025-08-15 21:02:37 -05:00
parent 33269c888d
commit 0b61175d98
76 changed files with 9245 additions and 1442 deletions
--- a/docs/user-guide.md
+++ b/docs/user-guide.md
@@ -1,6 +1,6 @@
-# BMad-Method BMAd Code User Guide
+# BMad Method — User Guide

-This guide will help you understand and effectively use the BMad Method for agile AI driven planning and development.
+This guide will help you understand and effectively use the BMad Method for agile AI-driven planning and development.

 ## The BMad Plan and Execute Workflow

@@ -8,7 +8,7 @@ First, here is the full standard Greenfield Planning + Execution Workflow. Brown

 If you are going to use the BMad Method with a Brownfield project (an existing project), review **[Working in the Brownfield](./working-in-the-brownfield.md)**.

-If you do not see the diagrams that following rendering, you can install Markdown All in One along with the Markdown Preview Mermaid Support plugins to VSCode (or one of the forked clones). With these plugin's, if you right click on the tab when open, there should be a Open Preview option, or check the IDE documentation.
+If the diagrams below don't render, install Markdown All in One along with the Markdown Preview Mermaid Support plugins to VSCode (or one of the forked clones). With these plugins, if you right click on the tab when open, there should be an Open Preview option, or check the IDE documentation.

 ### The Planning Workflow (Web UI or Powerful IDE Agents)

@@ -32,8 +32,11 @@ graph TD
    F2 -->|No| H["Architect: Create Architecture from PRD"]
    F3 --> F4["UX Expert: Generate UI Prompt for Lovable/V0 (Optional)"]
    F4 --> H2["Architect: Create Architecture from PRD + UX Spec"]
-    H --> I["PO: Run Master Checklist"]
-    H2 --> I
+    H --> Q{"Early Test Strategy? (Optional)"}
+    H2 --> Q
+    Q -->|Yes| R["QA: Early Test Architecture Input on High-Risk Areas"]
+    Q -->|No| I
+    R --> I["PO: Run Master Checklist"]
    I --> J{"Documents Aligned?"}
    J -->|Yes| K["Planning Complete"]
    J -->|No| L["PO: Update Epics & Stories"]
@@ -58,6 +61,8 @@ graph TD
    style G fill:#e3f2fd,color:#000
    style H fill:#f3e5f5,color:#000
    style H2 fill:#f3e5f5,color:#000
+    style Q fill:#e3f2fd,color:#000
+    style R fill:#ffd54f,color:#000
    style I fill:#f9ab00,color:#fff
    style J fill:#e3f2fd,color:#000
    style K fill:#34a853,color:#fff
@@ -77,6 +82,17 @@ graph TD
 3. **Document Sharding**: Use the PO agent to shard the PRD and then the Architecture
 4. **Begin Development**: Start the Core Development Cycle that follows

+#### Planning Artifacts (Standard Paths)
+
+```text
+PRD              → docs/prd.md
+Architecture     → docs/architecture.md
+Sharded Epics    → docs/epics/
+Sharded Stories  → docs/stories/
+QA Assessments   → docs/qa/assessments/
+QA Gates         → docs/qa/gates/
+```
+
 ### The Core Development Cycle (IDE)

 Once planning is complete and documents are sharded, BMad follows a structured development workflow:
@@ -85,35 +101,52 @@ Once planning is complete and documents are sharded, BMad follows a structured d
 graph TD
    A["Development Phase Start"] --> B["SM: Reviews Previous Story Dev/QA Notes"]
    B --> B2["SM: Drafts Next Story from Sharded Epic + Architecture"]
-    B2 --> B3{"PO: Validate Story Draft (Optional)"}
+    B2 --> S{"High-Risk Story? (Optional)"}
+    S -->|Yes| T["QA: *risk + *design on Draft Story"]
+    S -->|No| B3
+    T --> U["Test Strategy & Risk Profile Created"]
+    U --> B3{"PO: Validate Story Draft (Optional)"}
    B3 -->|Validation Requested| B4["PO: Validate Story Against Artifacts"]
    B3 -->|Skip Validation| C{"User Approval"}
    B4 --> C
    C -->|Approved| D["Dev: Sequential Task Execution"]
    C -->|Needs Changes| B2
    D --> E["Dev: Implement Tasks + Tests"]
-    E --> F["Dev: Run All Validations"]
+    E --> V{"Mid-Dev QA Check? (Optional)"}
+    V -->|Yes| W["QA: *trace or *nfr for Early Validation"]
+    V -->|No| F
+    W --> X["Dev: Address Coverage/NFR Gaps"]
+    X --> F["Dev: Run All Validations"]
    F --> G["Dev: Mark Ready for Review + Add Notes"]
    G --> H{"User Verification"}
-    H -->|Request QA Review| I["QA: Senior Dev Review + Active Refactoring"]
+    H -->|Request QA Review| I["QA: Test Architect Review + Quality Gate"]
    H -->|Approve Without QA| M["IMPORTANT: Verify All Regression Tests and Linting are Passing"]
-    I --> J["QA: Review, Refactor Code, Add Tests, Document Notes"]
+    I --> J["QA: Test Architecture Analysis + Active Refactoring"]
    J --> L{"QA Decision"}
    L -->|Needs Dev Work| D
    L -->|Approved| M
    H -->|Needs Fixes| D
    M --> N["IMPORTANT: COMMIT YOUR CHANGES BEFORE PROCEEDING!"]
-    N --> K["Mark Story as Done"]
+    N --> Y{"Gate Update Needed?"}
+    Y -->|Yes| Z["QA: *gate to Update Status"]
+    Y -->|No| K
+    Z --> K["Mark Story as Done"]
    K --> B

    style A fill:#f5f5f5,color:#000
    style B fill:#e8f5e9,color:#000
    style B2 fill:#e8f5e9,color:#000
+    style S fill:#e3f2fd,color:#000
+    style T fill:#ffd54f,color:#000
+    style U fill:#ffd54f,color:#000
    style B3 fill:#e3f2fd,color:#000
    style B4 fill:#fce4ec,color:#000
    style C fill:#e3f2fd,color:#000
    style D fill:#e3f2fd,color:#000
    style E fill:#e3f2fd,color:#000
+    style V fill:#e3f2fd,color:#000
+    style W fill:#ffd54f,color:#000
+    style X fill:#e3f2fd,color:#000
    style F fill:#e3f2fd,color:#000
    style G fill:#e3f2fd,color:#000
    style H fill:#e3f2fd,color:#000
@@ -123,13 +156,23 @@ graph TD
    style L fill:#e3f2fd,color:#000
    style M fill:#ff5722,color:#fff
    style N fill:#d32f2f,color:#fff
+    style Y fill:#e3f2fd,color:#000
+    style Z fill:#ffd54f,color:#000
 ```

+## Prerequisites
+
+Before installing BMad Method, ensure you have:
+
+- **Node.js** ≥ 18, **npm** ≥ 9
+- **Git** installed and configured
+- **(Optional)** VS Code with "Markdown All in One" + "Markdown Preview Mermaid Support" extensions
+
 ## Installation

 ### Optional

-If you want to do the planning in the Web with Claude (Sonnet 4 or Opus), Gemini Gem (2.5 Pro), or Custom GPT's:
+If you want to do the planning on the web with Claude (Sonnet 4 or Opus), Gemini Gem (2.5 Pro), or Custom GPTs:

 1. Navigate to `dist/teams/`
 2. Copy `team-fullstack.txt`
@@ -146,17 +189,17 @@ npx bmad-method install

 ## Special Agents

-There are two bmad agents - in the future they will be consolidated into the single bmad-master.
+There are two BMad agents — in the future they'll be consolidated into a single BMad-Master.

 ### BMad-Master

-This agent can do any task or command that all other agents can do, aside from actual story implementation. Additionally, this agent can help explain the BMad Method when in the web by accessing the knowledge base and explaining anything to you about the process.
+This agent can do any task or command that all other agents can do, aside from actual story implementation. Additionally, this agent can help explain the BMad Method when on the web by accessing the knowledge base and explaining anything to you about the process.

 If you don't want to bother switching between different agents aside from the dev, this is the agent for you. Just remember that as the context grows, the performance of the agent degrades, therefore it is important to instruct the agent to compact the conversation and start a new conversation with the compacted conversation as the initial message. Do this often, preferably after each story is implemented.

 ### BMad-Orchestrator

-This agent should NOT be used within the IDE, it is a heavy weight special purpose agent that utilizes a lot of context and can morph into any other agent. This exists solely to facilitate the team's within the web bundles. If you use a web bundle you will be greeted by the BMad Orchestrator.
+This agent should NOT be used within the IDE, it is a heavyweight, special-purpose agent that utilizes a lot of context and can morph into any other agent. This exists solely to facilitate the teams within the web bundles. If you use a web bundle you will be greeted by the BMad Orchestrator.

 ### How Agents Work

@@ -187,12 +230,12 @@ dependencies:
 **In IDE:**

 ```bash
-# Some Ide's, like Cursor or Windsurf for example, utilize manual rules so interaction is done with the '@' symbol
+# Some IDEs, like Cursor or Windsurf for example, utilize manual rules so interaction is done with the '@' symbol
@pm Create a PRD for a task management app
@architect Design the system architecture
@dev Implement the user authentication

-# Some, like Claude Code use slash commands instead
+# Some IDEs, like Claude Code, use slash commands instead
 /pm Create user stories
 /dev Fix the login bug
 ```
@@ -212,6 +255,216 @@ dependencies:
 - **File Organization**: Maintain clean project structure
 - **Commit Regularly**: Save your work frequently

+## The Test Architect (QA Agent)
+
+### Overview
+
+The QA agent in BMad is not just a "senior developer reviewer" - it's a **Test Architect** with deep expertise in test strategy, quality gates, and risk-based testing. Named Quinn, this agent provides advisory authority on quality matters while actively improving code when safe to do so.
+
+#### Quick Start (Essential Commands)
+
+```bash
+@qa *risk {story}       # Assess risks before development
+@qa *design {story}     # Create test strategy
+@qa *trace {story}      # Verify test coverage during dev
+@qa *nfr {story}        # Check quality attributes
+@qa *review {story}     # Full assessment → writes gate
+```
+
+#### Command Aliases (Test Architect)
+
+The documentation uses short forms for convenience. Both styles are valid:
+
+```text
+*risk    → *risk-profile
+*design  → *test-design  
+*nfr     → *nfr-assess
+*trace   → *trace-requirements (or just *trace)
+*review  → *review
+*gate    → *gate
+```
+
+### Core Capabilities
+
+#### 1. Risk Profiling (`*risk`)
+
+**When:** After story draft, before development begins (earliest intervention point)
+
+Identifies and assesses implementation risks:
+
+- **Categories**: Technical, Security, Performance, Data, Business, Operational
+- **Scoring**: Probability × Impact analysis (1-9 scale)
+- **Mitigation**: Specific strategies for each identified risk
+- **Gate Impact**: Risks ≥9 trigger FAIL, ≥6 trigger CONCERNS (see `tasks/risk-profile.md` for authoritative rules)
+
+#### 2. Test Design (`*design`)
+
+**When:** After story draft, before development begins (guides what tests to write)
+
+Creates comprehensive test strategies including:
+
+- Test scenarios for each acceptance criterion
+- Appropriate test level recommendations (unit vs integration vs E2E)
+- Risk-based prioritization (P0/P1/P2)
+- Test data requirements and mock strategies
+- Execution strategies for CI/CD integration
+
+**Example output:**
+
+```yaml
+test_summary:
+  total: 24
+  by_level:
+    unit: 15
+    integration: 7
+    e2e: 2
+  by_priority:
+    P0: 8 # Must have - linked to critical risks
+    P1: 10 # Should have - medium risks
+    P2: 6 # Nice to have - low risks
+```
+
+#### 3. Requirements Tracing (`*trace`)
+
+**When:** During development (mid-implementation checkpoint)
+
+Maps requirements to test coverage:
+
+- Documents which tests validate each acceptance criterion
+- Uses Given-When-Then for clarity (documentation only, not BDD code)
+- Identifies coverage gaps with severity ratings
+- Creates traceability matrix for audit purposes
+
+#### 4. NFR Assessment (`*nfr`)
+
+**When:** During development or early review (validate quality attributes)
+
+Validates non-functional requirements:
+
+- **Core Four**: Security, Performance, Reliability, Maintainability
+- **Evidence-Based**: Looks for actual implementation proof
+- **Gate Integration**: NFR failures directly impact quality gates
+
+#### 5. Comprehensive Test Architecture Review (`*review`)
+
+**When:** After development complete, story marked "Ready for Review"
+
+When you run `@qa *review {story}`, Quinn performs:
+
+- **Requirements Traceability**: Maps every acceptance criterion to its validating tests
+- **Test Level Analysis**: Ensures appropriate testing at unit, integration, and E2E levels
+- **Coverage Assessment**: Identifies gaps and redundant test coverage
+- **Active Refactoring**: Improves code quality directly when safe
+- **Quality Gate Decision**: Issues PASS/CONCERNS/FAIL status based on findings
+
+#### 6. Quality Gates (`*gate`)
+
+**When:** After review fixes or when gate status needs updating
+
+Manages quality gate decisions:
+
+- **Deterministic Rules**: Clear criteria for PASS/CONCERNS/FAIL
+- **Parallel Authority**: QA owns gate files in `docs/qa/gates/`
+- **Advisory Nature**: Provides recommendations, not blocks
+- **Waiver Support**: Documents accepted risks when needed
+
+**Note:** Gates are advisory; teams choose their quality bar. WAIVED requires reason, approver, and expiry date. See `templates/qa-gate-tmpl.yaml` for schema and `tasks/review-story.md` (gate rules) and `tasks/risk-profile.md` for scoring.
+
+### Working with the Test Architect
+
+#### Integration with BMad Workflow
+
+The Test Architect provides value throughout the entire development lifecycle. Here's when and how to leverage each capability:
+
+| **Stage** | **Command** | **When to Use** | **Value** | **Output** |
+|-----------|------------|-----------------|-----------|------------|
+| **Story Drafting** | `*risk` | After SM drafts story | Identify pitfalls early | `docs/qa/assessments/{epic}.{story}-risk-{YYYYMMDD}.md` |
+| | `*design` | After risk assessment | Guide dev on test strategy | `docs/qa/assessments/{epic}.{story}-test-design-{YYYYMMDD}.md` |
+| **Development** | `*trace` | Mid-implementation | Verify test coverage | `docs/qa/assessments/{epic}.{story}-trace-{YYYYMMDD}.md` |
+| | `*nfr` | While building features | Catch quality issues early | `docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md` |
+| **Review** | `*review` | Story marked complete | Full quality assessment | QA Results in story + gate file |
+| **Post-Review** | `*gate` | After fixing issues | Update quality decision | Updated `docs/qa/gates/{epic}.{story}-{slug}.yml` |
+
+#### Example Commands
+
+```bash
+# Planning Stage - Run these BEFORE development starts
+@qa *risk {draft-story}     # What could go wrong?
+@qa *design {draft-story}   # What tests should we write?
+
+# Development Stage - Run these DURING coding
+@qa *trace {story}          # Are we testing everything?
+@qa *nfr {story}            # Are we meeting quality standards?
+
+# Review Stage - Run when development complete
+@qa *review {story}         # Comprehensive assessment + refactoring
+
+# Post-Review - Run after addressing issues
+@qa *gate {story}           # Update gate status
+```
+
+### Quality Standards Enforced
+
+Quinn enforces these test quality principles:
+
+- **No Flaky Tests**: Ensures reliability through proper async handling
+- **No Hard Waits**: Dynamic waiting strategies only
+- **Stateless & Parallel-Safe**: Tests run independently
+- **Self-Cleaning**: Tests manage their own test data
+- **Appropriate Test Levels**: Unit for logic, integration for interactions, E2E for journeys
+- **Explicit Assertions**: Keep assertions in tests, not helpers
+
+### Gate Status Meanings
+
+- **PASS**: All critical requirements met, no blocking issues
+- **CONCERNS**: Non-critical issues found, team should review
+- **FAIL**: Critical issues that should be addressed (security risks, missing P0 tests)
+- **WAIVED**: Issues acknowledged but explicitly accepted by team
+
+### Special Situations
+
+**High-Risk Stories:**
+
+- Always run `*risk` and `*design` before development starts
+- Consider mid-development `*trace` and `*nfr` checkpoints
+
+**Complex Integrations:**
+
+- Run `*trace` during development to ensure all integration points tested
+- Follow up with `*nfr` to validate performance across integrations
+
+**Performance-Critical:**
+
+- Run `*nfr` early and often during development
+- Don't wait until review to discover performance issues
+
+**Brownfield/Legacy Code:**
+
+- Start with `*risk` to identify regression dangers
+- Use `*review` with extra focus on backward compatibility
+
+### Best Practices
+
+- **Early Engagement**: Run `*design` and `*risk` during story drafting
+- **Risk-Based Focus**: Let risk scores drive test prioritization
+- **Iterative Improvement**: Use QA feedback to improve future stories
+- **Gate Transparency**: Share gate decisions with the team
+- **Continuous Learning**: QA documents patterns for team knowledge sharing
+- **Brownfield Care**: Pay extra attention to regression risks in existing systems
+
+### Output Paths Reference
+
+Quick reference for where Test Architect outputs are stored:
+
+```text
+*risk-profile  → docs/qa/assessments/{epic}.{story}-risk-{YYYYMMDD}.md
+*test-design   → docs/qa/assessments/{epic}.{story}-test-design-{YYYYMMDD}.md
+*trace         → docs/qa/assessments/{epic}.{story}-trace-{YYYYMMDD}.md
+*nfr-assess    → docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md
+*review        → QA Results section in story + gate file reference
+*gate          → docs/qa/gates/{epic}.{story}-{slug}.yml
+```
+
 ## Technical Preferences System

 BMad includes a personalization system through the `technical-preferences.md` file located in `.bmad-core/data/` - this can help bias the PM and Architect to recommend your preferences for design patterns, technology selection, or anything else you would like to put in here.
@@ -235,9 +488,9 @@ devLoadAlwaysFiles:
  - docs/architecture/project-structure.md
 ```

-You will want to verify from sharding your architecture that these documents exist, that they are as lean as possible, and contain exactly the information you want your dev agent to ALWAYS load into it's context. These are the rules the agent will follow.
+You will want to verify from sharding your architecture that these documents exist, that they are as lean as possible, and contain exactly the information you want your dev agent to ALWAYS load into its context. These are the rules the agent will follow.

-As your project grows and the code starts to build consistent patterns, coding standards should be reduced to include only the standards that the agent still makes with. The agent will look at surrounding code in files to infer the coding standards that are relevant to the current task.
+As your project grows and the code starts to build consistent patterns, coding standards should be reduced to include only the standards the agent still needs enforced. The agent will look at surrounding code in files to infer the coding standards that are relevant to the current task.

 ## Getting Help