feat: transform QA agent into Test Architect with advanced quality ca… (#433)
* feat: transform QA agent into Test Architect with advanced quality capabilities - Add 6 specialized quality assessment commands - Implement risk-based testing with scoring - Create quality gate system with deterministic decisions - Add comprehensive test design and NFR validation - Update documentation with stage-based workflow integration * feat: transform QA agent into Test Architect with advanced quality capabilities - Add 6 specialized quality assessment commands - Implement risk-based testing with scoring - Create quality gate system with deterministic decisions - Add comprehensive test design and NFR validation - Update documentation with stage-based workflow integration * docs: refined the docs for test architect * fix: addressed review comments from manjaroblack, round 1 * fix: addressed review comments from manjaroblack, round 1 --------- Co-authored-by: Murat Ozcan <murat@mac.lan> Co-authored-by: Brian <bmadcode@gmail.com>
This commit is contained in:
Murat K Ozcan
@@ -128,7 +128,7 @@ Critical: For brownfield, ALWAYS include criteria about maintaining existing fun
|
||||
Standard structure:
|
||||
|
||||
1. New functionality works as specified
|
||||
2. Existing {{affected feature}} continues to work unchanged
|
||||
2. Existing {{affected feature}} continues to work unchanged
|
||||
3. Integration with {{existing system}} maintains current behavior
|
||||
4. No regression in {{related area}}
|
||||
5. Performance remains within acceptable bounds
|
||||
@@ -139,16 +139,19 @@ Critical: This is where you'll need to be interactive with the user if informati
|
||||
|
||||
Create Dev Technical Guidance section with available information:
|
||||
|
||||
```markdown
|
||||
````markdown
|
||||
## Dev Technical Guidance
|
||||
|
||||
### Existing System Context
|
||||
|
||||
[Extract from available documentation]
|
||||
|
||||
### Integration Approach
|
||||
|
||||
[Based on patterns found or ask user]
|
||||
|
||||
### Technical Constraints
|
||||
|
||||
[From documentation or user input]
|
||||
|
||||
### Missing Information
|
||||
@@ -191,6 +194,7 @@ Example task structure for brownfield:
|
||||
- [ ] Integration test for {{integration point}}
|
||||
- [ ] Update existing tests if needed
|
||||
```
|
||||
````
|
||||
|
||||
### 5. Risk Assessment and Mitigation
|
||||
|
||||
@@ -202,14 +206,17 @@ Add section for brownfield-specific risks:
|
||||
## Risk Assessment
|
||||
|
||||
### Implementation Risks
|
||||
|
||||
- **Primary Risk**: {{main risk to existing system}}
|
||||
- **Mitigation**: {{how to address}}
|
||||
- **Verification**: {{how to confirm safety}}
|
||||
|
||||
### Rollback Plan
|
||||
|
||||
- {{Simple steps to undo changes if needed}}
|
||||
|
||||
### Safety Checks
|
||||
|
||||
- [ ] Existing {{feature}} tested before changes
|
||||
- [ ] Changes can be feature-flagged or isolated
|
||||
- [ ] Rollback procedure documented
|
||||
@@ -252,6 +259,7 @@ Include header noting documentation context:
|
||||
<!-- Context: Brownfield enhancement to {{existing system}} -->
|
||||
|
||||
## Status: Draft
|
||||
|
||||
[Rest of story content...]
|
||||
```
|
||||
|
||||
@@ -272,7 +280,7 @@ Key Integration Points Identified:
|
||||
Risks Noted:
|
||||
- {{primary risk}}
|
||||
|
||||
{{If missing info}}:
|
||||
{{If missing info}}:
|
||||
Note: Some technical details were unclear. The story includes exploration tasks to gather needed information during implementation.
|
||||
|
||||
Next Steps:
|
||||
|
||||
@@ -21,63 +21,54 @@ CRITICAL: First, help the user select the most appropriate research focus based
|
||||
Present these numbered options to the user:
|
||||
|
||||
1. **Product Validation Research**
|
||||
|
||||
- Validate product hypotheses and market fit
|
||||
- Test assumptions about user needs and solutions
|
||||
- Assess technical and business feasibility
|
||||
- Identify risks and mitigation strategies
|
||||
|
||||
2. **Market Opportunity Research**
|
||||
|
||||
- Analyze market size and growth potential
|
||||
- Identify market segments and dynamics
|
||||
- Assess market entry strategies
|
||||
- Evaluate timing and market readiness
|
||||
|
||||
3. **User & Customer Research**
|
||||
|
||||
- Deep dive into user personas and behaviors
|
||||
- Understand jobs-to-be-done and pain points
|
||||
- Map customer journeys and touchpoints
|
||||
- Analyze willingness to pay and value perception
|
||||
|
||||
4. **Competitive Intelligence Research**
|
||||
|
||||
- Detailed competitor analysis and positioning
|
||||
- Feature and capability comparisons
|
||||
- Business model and strategy analysis
|
||||
- Identify competitive advantages and gaps
|
||||
|
||||
5. **Technology & Innovation Research**
|
||||
|
||||
- Assess technology trends and possibilities
|
||||
- Evaluate technical approaches and architectures
|
||||
- Identify emerging technologies and disruptions
|
||||
- Analyze build vs. buy vs. partner options
|
||||
|
||||
6. **Industry & Ecosystem Research**
|
||||
|
||||
- Map industry value chains and dynamics
|
||||
- Identify key players and relationships
|
||||
- Analyze regulatory and compliance factors
|
||||
- Understand partnership opportunities
|
||||
|
||||
7. **Strategic Options Research**
|
||||
|
||||
- Evaluate different strategic directions
|
||||
- Assess business model alternatives
|
||||
- Analyze go-to-market strategies
|
||||
- Consider expansion and scaling paths
|
||||
|
||||
8. **Risk & Feasibility Research**
|
||||
|
||||
- Identify and assess various risk factors
|
||||
- Evaluate implementation challenges
|
||||
- Analyze resource requirements
|
||||
- Consider regulatory and legal implications
|
||||
|
||||
9. **Custom Research Focus**
|
||||
|
||||
- User-defined research objectives
|
||||
- Specialized domain investigation
|
||||
- Cross-functional research needs
|
||||
@@ -246,13 +237,11 @@ CRITICAL: collaborate with the user to develop specific, actionable research que
|
||||
### 5. Review and Refinement
|
||||
|
||||
1. **Present Complete Prompt**
|
||||
|
||||
- Show the full research prompt
|
||||
- Explain key elements and rationale
|
||||
- Highlight any assumptions made
|
||||
|
||||
2. **Gather Feedback**
|
||||
|
||||
- Are the objectives clear and correct?
|
||||
- Do the questions address all concerns?
|
||||
- Is the scope appropriate?
|
||||
|
||||
@@ -111,9 +111,9 @@ This document captures the CURRENT STATE of the [Project Name] codebase, includi
|
||||
|
||||
### Change Log
|
||||
|
||||
| Date | Version | Description | Author |
|
||||
|------|---------|-------------|--------|
|
||||
| [Date] | 1.0 | Initial brownfield analysis | [Analyst] |
|
||||
| Date | Version | Description | Author |
|
||||
| ------ | ------- | --------------------------- | --------- |
|
||||
| [Date] | 1.0 | Initial brownfield analysis | [Analyst] |
|
||||
|
||||
## Quick Reference - Key Files and Entry Points
|
||||
|
||||
@@ -136,11 +136,11 @@ This document captures the CURRENT STATE of the [Project Name] codebase, includi
|
||||
|
||||
### Actual Tech Stack (from package.json/requirements.txt)
|
||||
|
||||
| Category | Technology | Version | Notes |
|
||||
|----------|------------|---------|--------|
|
||||
| Runtime | Node.js | 16.x | [Any constraints] |
|
||||
| Framework | Express | 4.18.2 | [Custom middleware?] |
|
||||
| Database | PostgreSQL | 13 | [Connection pooling setup] |
|
||||
| Category | Technology | Version | Notes |
|
||||
| --------- | ---------- | ------- | -------------------------- |
|
||||
| Runtime | Node.js | 16.x | [Any constraints] |
|
||||
| Framework | Express | 4.18.2 | [Custom middleware?] |
|
||||
| Database | PostgreSQL | 13 | [Connection pooling setup] |
|
||||
|
||||
etc...
|
||||
|
||||
@@ -179,6 +179,7 @@ project-root/
|
||||
### Data Models
|
||||
|
||||
Instead of duplicating, reference actual model files:
|
||||
|
||||
- **User Model**: See `src/models/User.js`
|
||||
- **Order Model**: See `src/models/Order.js`
|
||||
- **Related Types**: TypeScript definitions in `src/types/`
|
||||
@@ -208,10 +209,10 @@ Instead of duplicating, reference actual model files:
|
||||
|
||||
### External Services
|
||||
|
||||
| Service | Purpose | Integration Type | Key Files |
|
||||
|---------|---------|------------------|-----------|
|
||||
| Stripe | Payments | REST API | `src/integrations/stripe/` |
|
||||
| SendGrid | Emails | SDK | `src/services/emailService.js` |
|
||||
| Service | Purpose | Integration Type | Key Files |
|
||||
| -------- | -------- | ---------------- | ------------------------------ |
|
||||
| Stripe | Payments | REST API | `src/integrations/stripe/` |
|
||||
| SendGrid | Emails | SDK | `src/services/emailService.js` |
|
||||
|
||||
etc...
|
||||
|
||||
@@ -256,6 +257,7 @@ npm run test:integration # Runs integration tests (requires local DB)
|
||||
### Files That Will Need Modification
|
||||
|
||||
Based on the enhancement requirements, these files will be affected:
|
||||
|
||||
- `src/services/userService.js` - Add new user fields
|
||||
- `src/models/User.js` - Update schema
|
||||
- `src/routes/userRoutes.js` - New endpoints
|
||||
@@ -338,4 +340,4 @@ Apply the advanced elicitation task after major sections to refine based on user
|
||||
- References actual files rather than duplicating content when possible
|
||||
- Documents technical debt, workarounds, and constraints honestly
|
||||
- For brownfield projects with PRD: Provides clear enhancement impact analysis
|
||||
- The goal is PRACTICAL documentation for AI agents doing real work
|
||||
- The goal is PRACTICAL documentation for AI agents doing real work
|
||||
|
||||
@@ -43,7 +43,7 @@ If user selects Option 1, present numbered list of techniques from the brainstor
|
||||
1. Apply selected technique according to data file description
|
||||
2. Keep engaging with technique until user indicates they want to:
|
||||
- Choose a different technique
|
||||
- Apply current ideas to a new technique
|
||||
- Apply current ideas to a new technique
|
||||
- Move to convergent phase
|
||||
- End session
|
||||
|
||||
|
||||
@@ -11,14 +11,12 @@ You are now operating as a Documentation Indexer. Your goal is to ensure all doc
|
||||
### Required Steps
|
||||
|
||||
1. First, locate and scan:
|
||||
|
||||
- The `docs/` directory and all subdirectories
|
||||
- The existing `docs/index.md` file (create if absent)
|
||||
- All markdown (`.md`) and text (`.txt`) files in the documentation structure
|
||||
- Note the folder structure for hierarchical organization
|
||||
|
||||
2. For the existing `docs/index.md`:
|
||||
|
||||
- Parse current entries
|
||||
- Note existing file references and descriptions
|
||||
- Identify any broken links or missing files
|
||||
@@ -26,7 +24,6 @@ You are now operating as a Documentation Indexer. Your goal is to ensure all doc
|
||||
- Preserve existing folder sections
|
||||
|
||||
3. For each documentation file found:
|
||||
|
||||
- Extract the title (from first heading or filename)
|
||||
- Generate a brief description by analyzing the content
|
||||
- Create a relative markdown link to the file
|
||||
@@ -35,7 +32,6 @@ You are now operating as a Documentation Indexer. Your goal is to ensure all doc
|
||||
- If missing or outdated, prepare an update
|
||||
|
||||
4. For any missing or non-existent files found in index:
|
||||
|
||||
- Present a list of all entries that reference non-existent files
|
||||
- For each entry:
|
||||
- Show the full entry details (title, path, description)
|
||||
@@ -88,7 +84,6 @@ Documents within the `another-folder/` directory:
|
||||
### [Nested Document](./another-folder/document.md)
|
||||
|
||||
Description of nested document.
|
||||
|
||||
```
|
||||
|
||||
### Index Entry Format
|
||||
@@ -157,7 +152,6 @@ For each file referenced in the index but not found in the filesystem:
|
||||
### Special Cases
|
||||
|
||||
1. **Sharded Documents**: If a folder contains an `index.md` file, treat it as a sharded document:
|
||||
|
||||
- Use the folder's `index.md` title as the section title
|
||||
- List the folder's documents as subsections
|
||||
- Note in the description that this is a multi-part document
|
||||
|
||||
@@ -6,7 +6,7 @@ Provide a user-friendly interface to the BMad knowledge base without overwhelmin
|
||||
|
||||
## Instructions
|
||||
|
||||
When entering KB mode (*kb-mode), follow these steps:
|
||||
When entering KB mode (\*kb-mode), follow these steps:
|
||||
|
||||
### 1. Welcome and Guide
|
||||
|
||||
@@ -48,12 +48,12 @@ Or ask me about anything else related to BMad-Method!
|
||||
When user is done or wants to exit KB mode:
|
||||
|
||||
- Summarize key points discussed if helpful
|
||||
- Remind them they can return to KB mode anytime with *kb-mode
|
||||
- Remind them they can return to KB mode anytime with \*kb-mode
|
||||
- Suggest next steps based on what was discussed
|
||||
|
||||
## Example Interaction
|
||||
|
||||
**User**: *kb-mode
|
||||
**User**: \*kb-mode
|
||||
|
||||
**Assistant**: I've entered KB mode and have access to the full BMad knowledge base. I can help you with detailed information about any aspect of BMad-Method.
|
||||
|
||||
|
||||
315
bmad-core/tasks/nfr-assess.md
Normal file
315
bmad-core/tasks/nfr-assess.md
Normal file
@@ -0,0 +1,315 @@
|
||||
# nfr-assess
|
||||
|
||||
Quick NFR validation focused on the core four: security, performance, reliability, maintainability.
|
||||
|
||||
## Inputs
|
||||
|
||||
```yaml
|
||||
required:
|
||||
- story_id: "{epic}.{story}" # e.g., "1.3"
|
||||
- story_path: "docs/stories/{epic}.{story}.*.md"
|
||||
|
||||
optional:
|
||||
- architecture_refs: "docs/architecture/*.md"
|
||||
- technical_preferences: "docs/technical-preferences.md"
|
||||
- acceptance_criteria: From story file
|
||||
```
|
||||
|
||||
## Purpose
|
||||
|
||||
Assess non-functional requirements for a story and generate:
|
||||
1. YAML block for the gate file's `nfr_validation` section
|
||||
2. Brief markdown assessment saved to `docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md`
|
||||
|
||||
## Process
|
||||
|
||||
### 0. Fail-safe for Missing Inputs
|
||||
|
||||
If story_path or story file can't be found:
|
||||
- Still create assessment file with note: "Source story not found"
|
||||
- Set all selected NFRs to CONCERNS with notes: "Target unknown / evidence missing"
|
||||
- Continue with assessment to provide value
|
||||
|
||||
### 1. Elicit Scope
|
||||
|
||||
**Interactive mode:** Ask which NFRs to assess
|
||||
**Non-interactive mode:** Default to core four (security, performance, reliability, maintainability)
|
||||
|
||||
```text
|
||||
Which NFRs should I assess? (Enter numbers or press Enter for default)
|
||||
[1] Security (default)
|
||||
[2] Performance (default)
|
||||
[3] Reliability (default)
|
||||
[4] Maintainability (default)
|
||||
[5] Usability
|
||||
[6] Compatibility
|
||||
[7] Portability
|
||||
[8] Functional Suitability
|
||||
|
||||
> [Enter for 1-4]
|
||||
```
|
||||
|
||||
### 2. Check for Thresholds
|
||||
|
||||
Look for NFR requirements in:
|
||||
- Story acceptance criteria
|
||||
- `docs/architecture/*.md` files
|
||||
- `docs/technical-preferences.md`
|
||||
|
||||
**Interactive mode:** Ask for missing thresholds
|
||||
**Non-interactive mode:** Mark as CONCERNS with "Target unknown"
|
||||
|
||||
```text
|
||||
No performance requirements found. What's your target response time?
|
||||
> 200ms for API calls
|
||||
|
||||
No security requirements found. Required auth method?
|
||||
> JWT with refresh tokens
|
||||
```
|
||||
|
||||
**Unknown targets policy:** If a target is missing and not provided, mark status as CONCERNS with notes: "Target unknown"
|
||||
|
||||
### 3. Quick Assessment
|
||||
|
||||
For each selected NFR, check:
|
||||
- Is there evidence it's implemented?
|
||||
- Can we validate it?
|
||||
- Are there obvious gaps?
|
||||
|
||||
### 4. Generate Outputs
|
||||
|
||||
## Output 1: Gate YAML Block
|
||||
|
||||
Generate ONLY for NFRs actually assessed (no placeholders):
|
||||
|
||||
```yaml
|
||||
# Gate YAML (copy/paste):
|
||||
nfr_validation:
|
||||
_assessed: [security, performance, reliability, maintainability]
|
||||
security:
|
||||
status: CONCERNS
|
||||
notes: "No rate limiting on auth endpoints"
|
||||
performance:
|
||||
status: PASS
|
||||
notes: "Response times < 200ms verified"
|
||||
reliability:
|
||||
status: PASS
|
||||
notes: "Error handling and retries implemented"
|
||||
maintainability:
|
||||
status: CONCERNS
|
||||
notes: "Test coverage at 65%, target is 80%"
|
||||
```
|
||||
|
||||
## Deterministic Status Rules
|
||||
|
||||
- **FAIL**: Any selected NFR has critical gap or target clearly not met
|
||||
- **CONCERNS**: No FAILs, but any NFR is unknown/partial/missing evidence
|
||||
- **PASS**: All selected NFRs meet targets with evidence
|
||||
|
||||
## Quality Score Calculation
|
||||
|
||||
```
|
||||
quality_score = 100
|
||||
- 20 for each FAIL attribute
|
||||
- 10 for each CONCERNS attribute
|
||||
Floor at 0, ceiling at 100
|
||||
```
|
||||
|
||||
If `technical-preferences.md` defines custom weights, use those instead.
|
||||
|
||||
## Output 2: Brief Assessment Report
|
||||
|
||||
**ALWAYS save to:** `docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md`
|
||||
|
||||
```markdown
|
||||
# NFR Assessment: {epic}.{story}
|
||||
Date: {date}
|
||||
Reviewer: Quinn
|
||||
|
||||
<!-- Note: Source story not found (if applicable) -->
|
||||
|
||||
## Summary
|
||||
- Security: CONCERNS - Missing rate limiting
|
||||
- Performance: PASS - Meets <200ms requirement
|
||||
- Reliability: PASS - Proper error handling
|
||||
- Maintainability: CONCERNS - Test coverage below target
|
||||
|
||||
## Critical Issues
|
||||
1. **No rate limiting** (Security)
|
||||
- Risk: Brute force attacks possible
|
||||
- Fix: Add rate limiting middleware to auth endpoints
|
||||
|
||||
2. **Test coverage 65%** (Maintainability)
|
||||
- Risk: Untested code paths
|
||||
- Fix: Add tests for uncovered branches
|
||||
|
||||
## Quick Wins
|
||||
- Add rate limiting: ~2 hours
|
||||
- Increase test coverage: ~4 hours
|
||||
- Add performance monitoring: ~1 hour
|
||||
```
|
||||
|
||||
## Output 3: Story Update Line
|
||||
|
||||
**End with this line for the review task to quote:**
|
||||
```
|
||||
NFR assessment: docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md
|
||||
```
|
||||
|
||||
## Output 4: Gate Integration Line
|
||||
|
||||
**Always print at the end:**
|
||||
```
|
||||
Gate NFR block ready → paste into docs/qa/gates/{epic}.{story}-{slug}.yml under nfr_validation
|
||||
```
|
||||
|
||||
## Assessment Criteria
|
||||
|
||||
### Security
|
||||
**PASS if:**
|
||||
- Authentication implemented
|
||||
- Authorization enforced
|
||||
- Input validation present
|
||||
- No hardcoded secrets
|
||||
|
||||
**CONCERNS if:**
|
||||
- Missing rate limiting
|
||||
- Weak encryption
|
||||
- Incomplete authorization
|
||||
|
||||
**FAIL if:**
|
||||
- No authentication
|
||||
- Hardcoded credentials
|
||||
- SQL injection vulnerabilities
|
||||
|
||||
### Performance
|
||||
**PASS if:**
|
||||
- Meets response time targets
|
||||
- No obvious bottlenecks
|
||||
- Reasonable resource usage
|
||||
|
||||
**CONCERNS if:**
|
||||
- Close to limits
|
||||
- Missing indexes
|
||||
- No caching strategy
|
||||
|
||||
**FAIL if:**
|
||||
- Exceeds response time limits
|
||||
- Memory leaks
|
||||
- Unoptimized queries
|
||||
|
||||
### Reliability
|
||||
**PASS if:**
|
||||
- Error handling present
|
||||
- Graceful degradation
|
||||
- Retry logic where needed
|
||||
|
||||
**CONCERNS if:**
|
||||
- Some error cases unhandled
|
||||
- No circuit breakers
|
||||
- Missing health checks
|
||||
|
||||
**FAIL if:**
|
||||
- No error handling
|
||||
- Crashes on errors
|
||||
- No recovery mechanisms
|
||||
|
||||
### Maintainability
|
||||
**PASS if:**
|
||||
- Test coverage meets target
|
||||
- Code well-structured
|
||||
- Documentation present
|
||||
|
||||
**CONCERNS if:**
|
||||
- Test coverage below target
|
||||
- Some code duplication
|
||||
- Missing documentation
|
||||
|
||||
**FAIL if:**
|
||||
- No tests
|
||||
- Highly coupled code
|
||||
- No documentation
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### What to Check
|
||||
|
||||
```yaml
|
||||
security:
|
||||
- Authentication mechanism
|
||||
- Authorization checks
|
||||
- Input validation
|
||||
- Secret management
|
||||
- Rate limiting
|
||||
|
||||
performance:
|
||||
- Response times
|
||||
- Database queries
|
||||
- Caching usage
|
||||
- Resource consumption
|
||||
|
||||
reliability:
|
||||
- Error handling
|
||||
- Retry logic
|
||||
- Circuit breakers
|
||||
- Health checks
|
||||
- Logging
|
||||
|
||||
maintainability:
|
||||
- Test coverage
|
||||
- Code structure
|
||||
- Documentation
|
||||
- Dependencies
|
||||
```
|
||||
|
||||
## Key Principles
|
||||
|
||||
- Focus on the core four NFRs by default
|
||||
- Quick assessment, not deep analysis
|
||||
- Gate-ready output format
|
||||
- Brief, actionable findings
|
||||
- Skip what doesn't apply
|
||||
- Deterministic status rules for consistency
|
||||
- Unknown targets → CONCERNS, not guesses
|
||||
|
||||
---
|
||||
|
||||
## Appendix: ISO 25010 Reference
|
||||
|
||||
<details>
|
||||
<summary>Full ISO 25010 Quality Model (click to expand)</summary>
|
||||
|
||||
### All 8 Quality Characteristics
|
||||
|
||||
1. **Functional Suitability**: Completeness, correctness, appropriateness
|
||||
2. **Performance Efficiency**: Time behavior, resource use, capacity
|
||||
3. **Compatibility**: Co-existence, interoperability
|
||||
4. **Usability**: Learnability, operability, accessibility
|
||||
5. **Reliability**: Maturity, availability, fault tolerance
|
||||
6. **Security**: Confidentiality, integrity, authenticity
|
||||
7. **Maintainability**: Modularity, reusability, testability
|
||||
8. **Portability**: Adaptability, installability
|
||||
|
||||
Use these when assessing beyond the core four.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Example: Deep Performance Analysis (click to expand)</summary>
|
||||
|
||||
```yaml
|
||||
performance_deep_dive:
|
||||
response_times:
|
||||
p50: 45ms
|
||||
p95: 180ms
|
||||
p99: 350ms
|
||||
database:
|
||||
slow_queries: 2
|
||||
missing_indexes: ["users.email", "orders.user_id"]
|
||||
caching:
|
||||
hit_rate: 0%
|
||||
recommendation: "Add Redis for session data"
|
||||
load_test:
|
||||
max_rps: 150
|
||||
breaking_point: 200 rps
|
||||
```
|
||||
</details>
|
||||
159
bmad-core/tasks/qa-gate.md
Normal file
159
bmad-core/tasks/qa-gate.md
Normal file
@@ -0,0 +1,159 @@
|
||||
# qa-gate
|
||||
|
||||
Create or update a quality gate decision file for a story based on review findings.
|
||||
|
||||
## Purpose
|
||||
|
||||
Generate a standalone quality gate file that provides a clear pass/fail decision with actionable feedback. This gate serves as an advisory checkpoint for teams to understand quality status.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Story has been reviewed (manually or via review-story task)
|
||||
- Review findings are available
|
||||
- Understanding of story requirements and implementation
|
||||
|
||||
## Gate File Location
|
||||
|
||||
**ALWAYS** create file at: `docs/qa/gates/{epic}.{story}-{slug}.yml`
|
||||
|
||||
Slug rules:
|
||||
|
||||
- Convert to lowercase
|
||||
- Replace spaces with hyphens
|
||||
- Strip punctuation
|
||||
- Example: "User Auth - Login!" becomes "user-auth-login"
|
||||
|
||||
## Minimal Required Schema
|
||||
|
||||
```yaml
|
||||
schema: 1
|
||||
story: "{epic}.{story}"
|
||||
gate: PASS|CONCERNS|FAIL|WAIVED
|
||||
status_reason: "1-2 sentence explanation of gate decision"
|
||||
reviewer: "Quinn"
|
||||
updated: "{ISO-8601 timestamp}"
|
||||
top_issues: [] # Empty array if no issues
|
||||
waiver: { active: false } # Only set active: true if WAIVED
|
||||
```
|
||||
|
||||
## Schema with Issues
|
||||
|
||||
```yaml
|
||||
schema: 1
|
||||
story: "1.3"
|
||||
gate: CONCERNS
|
||||
status_reason: "Missing rate limiting on auth endpoints poses security risk."
|
||||
reviewer: "Quinn"
|
||||
updated: "2025-01-12T10:15:00Z"
|
||||
top_issues:
|
||||
- id: "SEC-001"
|
||||
severity: high # ONLY: low|medium|high
|
||||
finding: "No rate limiting on login endpoint"
|
||||
suggested_action: "Add rate limiting middleware before production"
|
||||
- id: "TEST-001"
|
||||
severity: medium
|
||||
finding: "No integration tests for auth flow"
|
||||
suggested_action: "Add integration test coverage"
|
||||
waiver: { active: false }
|
||||
```
|
||||
|
||||
## Schema when Waived
|
||||
|
||||
```yaml
|
||||
schema: 1
|
||||
story: "1.3"
|
||||
gate: WAIVED
|
||||
status_reason: "Known issues accepted for MVP release."
|
||||
reviewer: "Quinn"
|
||||
updated: "2025-01-12T10:15:00Z"
|
||||
top_issues:
|
||||
- id: "PERF-001"
|
||||
severity: low
|
||||
finding: "Dashboard loads slowly with 1000+ items"
|
||||
suggested_action: "Implement pagination in next sprint"
|
||||
waiver:
|
||||
active: true
|
||||
reason: "MVP release - performance optimization deferred"
|
||||
approved_by: "Product Owner"
|
||||
```
|
||||
|
||||
## Gate Decision Criteria
|
||||
|
||||
### PASS
|
||||
|
||||
- All acceptance criteria met
|
||||
- No high-severity issues
|
||||
- Test coverage meets project standards
|
||||
|
||||
### CONCERNS
|
||||
|
||||
- Non-blocking issues present
|
||||
- Should be tracked and scheduled
|
||||
- Can proceed with awareness
|
||||
|
||||
### FAIL
|
||||
|
||||
- Acceptance criteria not met
|
||||
- High-severity issues present
|
||||
- Recommend return to InProgress
|
||||
|
||||
### WAIVED
|
||||
|
||||
- Issues explicitly accepted
|
||||
- Requires approval and reason
|
||||
- Proceed despite known issues
|
||||
|
||||
## Severity Scale
|
||||
|
||||
**FIXED VALUES - NO VARIATIONS:**
|
||||
|
||||
- `low`: Minor issues, cosmetic problems
|
||||
- `medium`: Should fix soon, not blocking
|
||||
- `high`: Critical issues, should block release
|
||||
|
||||
## Issue ID Prefixes
|
||||
|
||||
- `SEC-`: Security issues
|
||||
- `PERF-`: Performance issues
|
||||
- `REL-`: Reliability issues
|
||||
- `TEST-`: Testing gaps
|
||||
- `MNT-`: Maintainability concerns
|
||||
- `ARCH-`: Architecture issues
|
||||
- `DOC-`: Documentation gaps
|
||||
- `REQ-`: Requirements issues
|
||||
|
||||
## Output Requirements
|
||||
|
||||
1. **ALWAYS** create gate file at: `docs/qa/gates/{epic}.{story}-{slug}.yml`
|
||||
2. **ALWAYS** append this exact format to story's QA Results section:
|
||||
```
|
||||
Gate: {STATUS} → docs/qa/gates/{epic}.{story}-{slug}.yml
|
||||
```
|
||||
3. Keep status_reason to 1-2 sentences maximum
|
||||
4. Use severity values exactly: `low`, `medium`, or `high`
|
||||
|
||||
## Example Story Update
|
||||
|
||||
After creating gate file, append to story's QA Results section:
|
||||
|
||||
```markdown
|
||||
## QA Results
|
||||
|
||||
### Review Date: 2025-01-12
|
||||
|
||||
### Reviewed By: Quinn (Test Architect)
|
||||
|
||||
[... existing review content ...]
|
||||
|
||||
### Gate Status
|
||||
|
||||
Gate: CONCERNS → docs/qa/gates/1.3-user-auth-login.yml
|
||||
```
|
||||
|
||||
## Key Principles
|
||||
|
||||
- Keep it minimal and predictable
|
||||
- Fixed severity scale (low/medium/high)
|
||||
- Always write to standard path
|
||||
- Always update story with gate reference
|
||||
- Clear, actionable findings
|
||||
@@ -1,6 +1,16 @@
|
||||
# review-story
|
||||
|
||||
When a developer agent marks a story as "Ready for Review", perform a comprehensive senior developer code review with the ability to refactor and improve code directly.
|
||||
Perform a comprehensive test architecture review with quality gate decision. This adaptive, risk-aware review creates both a story update and a detailed gate file.
|
||||
|
||||
## Inputs
|
||||
|
||||
```yaml
|
||||
required:
|
||||
- story_id: "{epic}.{story}" # e.g., "1.3"
|
||||
- story_path: "{devStoryLocation}/{epic}.{story}.*.md" # Path from core-config.yaml
|
||||
- story_title: "{title}" # If missing, derive from story file H1
|
||||
- story_slug: "{slug}" # If missing, derive from title (lowercase, hyphenated)
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
@@ -8,98 +18,133 @@ When a developer agent marks a story as "Ready for Review", perform a comprehens
|
||||
- Developer has completed all tasks and updated the File List
|
||||
- All automated tests are passing
|
||||
|
||||
## Review Process
|
||||
## Review Process - Adaptive Test Architecture
|
||||
|
||||
1. **Read the Complete Story**
|
||||
- Review all acceptance criteria
|
||||
- Understand the dev notes and requirements
|
||||
- Note any completion notes from the developer
|
||||
### 1. Risk Assessment (Determines Review Depth)
|
||||
|
||||
2. **Verify Implementation Against Dev Notes Guidance**
|
||||
- Review the "Dev Notes" section for specific technical guidance provided to the developer
|
||||
- Verify the developer's implementation follows the architectural patterns specified in Dev Notes
|
||||
- Check that file locations match the project structure guidance in Dev Notes
|
||||
- Confirm any specified libraries, frameworks, or technical approaches were used correctly
|
||||
- Validate that security considerations mentioned in Dev Notes were implemented
|
||||
**Auto-escalate to deep review when:**
|
||||
|
||||
3. **Focus on the File List**
|
||||
- Verify all files listed were actually created/modified
|
||||
- Check for any missing files that should have been updated
|
||||
- Ensure file locations align with the project structure guidance from Dev Notes
|
||||
- Auth/payment/security files touched
|
||||
- No tests added to story
|
||||
- Diff > 500 lines
|
||||
- Previous gate was FAIL/CONCERNS
|
||||
- Story has > 5 acceptance criteria
|
||||
|
||||
4. **Senior Developer Code Review**
|
||||
- Review code with the eye of a senior developer
|
||||
- If changes form a cohesive whole, review them together
|
||||
- If changes are independent, review incrementally file by file
|
||||
- Focus on:
|
||||
- Code architecture and design patterns
|
||||
- Refactoring opportunities
|
||||
- Code duplication or inefficiencies
|
||||
- Performance optimizations
|
||||
- Security concerns
|
||||
- Best practices and patterns
|
||||
### 2. Comprehensive Analysis
|
||||
|
||||
5. **Active Refactoring**
|
||||
- As a senior developer, you CAN and SHOULD refactor code where improvements are needed
|
||||
- When refactoring:
|
||||
- Make the changes directly in the files
|
||||
- Explain WHY you're making the change
|
||||
- Describe HOW the change improves the code
|
||||
- Ensure all tests still pass after refactoring
|
||||
- Update the File List if you modify additional files
|
||||
**A. Requirements Traceability**
|
||||
|
||||
6. **Standards Compliance Check**
|
||||
- Verify adherence to `docs/coding-standards.md`
|
||||
- Check compliance with `docs/unified-project-structure.md`
|
||||
- Validate testing approach against `docs/testing-strategy.md`
|
||||
- Ensure all guidelines mentioned in the story are followed
|
||||
- Map each acceptance criteria to its validating tests (document mapping with Given-When-Then, not test code)
|
||||
- Identify coverage gaps
|
||||
- Verify all requirements have corresponding test cases
|
||||
|
||||
7. **Acceptance Criteria Validation**
|
||||
- Verify each AC is fully implemented
|
||||
- Check for any missing functionality
|
||||
- Validate edge cases are handled
|
||||
**B. Code Quality Review**
|
||||
|
||||
8. **Test Coverage Review**
|
||||
- Ensure unit tests cover edge cases
|
||||
- Add missing tests if critical coverage is lacking
|
||||
- Verify integration tests (if required) are comprehensive
|
||||
- Check that test assertions are meaningful
|
||||
- Look for missing test scenarios
|
||||
- Architecture and design patterns
|
||||
- Refactoring opportunities (and perform them)
|
||||
- Code duplication or inefficiencies
|
||||
- Performance optimizations
|
||||
- Security vulnerabilities
|
||||
- Best practices adherence
|
||||
|
||||
9. **Documentation and Comments**
|
||||
- Verify code is self-documenting where possible
|
||||
- Add comments for complex logic if missing
|
||||
- Ensure any API changes are documented
|
||||
**C. Test Architecture Assessment**
|
||||
|
||||
## Update Story File - QA Results Section ONLY
|
||||
- Test coverage adequacy at appropriate levels
|
||||
- Test level appropriateness (what should be unit vs integration vs e2e)
|
||||
- Test design quality and maintainability
|
||||
- Test data management strategy
|
||||
- Mock/stub usage appropriateness
|
||||
- Edge case and error scenario coverage
|
||||
- Test execution time and reliability
|
||||
|
||||
**D. Non-Functional Requirements (NFRs)**
|
||||
|
||||
- Security: Authentication, authorization, data protection
|
||||
- Performance: Response times, resource usage
|
||||
- Reliability: Error handling, recovery mechanisms
|
||||
- Maintainability: Code clarity, documentation
|
||||
|
||||
**E. Testability Evaluation**
|
||||
|
||||
- Controllability: Can we control the inputs?
|
||||
- Observability: Can we observe the outputs?
|
||||
- Debuggability: Can we debug failures easily?
|
||||
|
||||
**F. Technical Debt Identification**
|
||||
|
||||
- Accumulated shortcuts
|
||||
- Missing tests
|
||||
- Outdated dependencies
|
||||
- Architecture violations
|
||||
|
||||
### 3. Active Refactoring
|
||||
|
||||
- Refactor code where safe and appropriate
|
||||
- Run tests to ensure changes don't break functionality
|
||||
- Document all changes in QA Results section with clear WHY and HOW
|
||||
- Do NOT alter story content beyond QA Results section
|
||||
- Do NOT change story Status or File List; recommend next status only
|
||||
|
||||
### 4. Standards Compliance Check
|
||||
|
||||
- Verify adherence to `docs/coding-standards.md`
|
||||
- Check compliance with `docs/unified-project-structure.md`
|
||||
- Validate testing approach against `docs/testing-strategy.md`
|
||||
- Ensure all guidelines mentioned in the story are followed
|
||||
|
||||
### 5. Acceptance Criteria Validation
|
||||
|
||||
- Verify each AC is fully implemented
|
||||
- Check for any missing functionality
|
||||
- Validate edge cases are handled
|
||||
|
||||
### 6. Documentation and Comments
|
||||
|
||||
- Verify code is self-documenting where possible
|
||||
- Add comments for complex logic if missing
|
||||
- Ensure any API changes are documented
|
||||
|
||||
## Output 1: Update Story File - QA Results Section ONLY
|
||||
|
||||
**CRITICAL**: You are ONLY authorized to update the "QA Results" section of the story file. DO NOT modify any other sections.
|
||||
|
||||
**QA Results Anchor Rule:**
|
||||
|
||||
- If `## QA Results` doesn't exist, append it at end of file
|
||||
- If it exists, append a new dated entry below existing entries
|
||||
- Never edit other sections
|
||||
|
||||
After review and any refactoring, append your results to the story file in the QA Results section:
|
||||
|
||||
```markdown
|
||||
## QA Results
|
||||
|
||||
### Review Date: [Date]
|
||||
### Reviewed By: Quinn (Senior Developer QA)
|
||||
|
||||
### Reviewed By: Quinn (Test Architect)
|
||||
|
||||
### Code Quality Assessment
|
||||
|
||||
[Overall assessment of implementation quality]
|
||||
|
||||
### Refactoring Performed
|
||||
|
||||
[List any refactoring you performed with explanations]
|
||||
|
||||
- **File**: [filename]
|
||||
- **Change**: [what was changed]
|
||||
- **Why**: [reason for change]
|
||||
- **How**: [how it improves the code]
|
||||
|
||||
### Compliance Check
|
||||
|
||||
- Coding Standards: [✓/✗] [notes if any]
|
||||
- Project Structure: [✓/✗] [notes if any]
|
||||
- Testing Strategy: [✓/✗] [notes if any]
|
||||
- All ACs Met: [✓/✗] [notes if any]
|
||||
|
||||
### Improvements Checklist
|
||||
|
||||
[Check off items you handled yourself, leave unchecked for dev to address]
|
||||
|
||||
- [x] Refactored user service for better error handling (services/user.service.ts)
|
||||
@@ -109,22 +154,144 @@ After review and any refactoring, append your results to the story file in the Q
|
||||
- [ ] Update API documentation for new error codes
|
||||
|
||||
### Security Review
|
||||
|
||||
[Any security concerns found and whether addressed]
|
||||
|
||||
### Performance Considerations
|
||||
|
||||
[Any performance issues found and whether addressed]
|
||||
|
||||
### Final Status
|
||||
[✓ Approved - Ready for Done] / [✗ Changes Required - See unchecked items above]
|
||||
### Files Modified During Review
|
||||
|
||||
[If you modified files, list them here - ask Dev to update File List]
|
||||
|
||||
### Gate Status
|
||||
|
||||
Gate: {STATUS} → docs/qa/gates/{epic}.{story}-{slug}.yml
|
||||
Risk profile: docs/qa/assessments/{epic}.{story}-risk-{YYYYMMDD}.md
|
||||
NFR assessment: docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md
|
||||
|
||||
# Note: Paths should reference core-config.yaml for custom configurations
|
||||
|
||||
### Recommended Status
|
||||
|
||||
[✓ Ready for Done] / [✗ Changes Required - See unchecked items above]
|
||||
(Story owner decides final status)
|
||||
```
|
||||
|
||||
## Output 2: Create Quality Gate File
|
||||
|
||||
**Template and Directory:**
|
||||
|
||||
- Render from `templates/qa-gate-tmpl.yaml`
|
||||
- Create `docs/qa/gates/` directory if missing (or configure in core-config.yaml)
|
||||
- Save to: `docs/qa/gates/{epic}.{story}-{slug}.yml`
|
||||
|
||||
Gate file structure:
|
||||
|
||||
```yaml
|
||||
schema: 1
|
||||
story: "{epic}.{story}"
|
||||
story_title: "{story title}"
|
||||
gate: PASS|CONCERNS|FAIL|WAIVED
|
||||
status_reason: "1-2 sentence explanation of gate decision"
|
||||
reviewer: "Quinn (Test Architect)"
|
||||
updated: "{ISO-8601 timestamp}"
|
||||
|
||||
top_issues: [] # Empty if no issues
|
||||
waiver: { active: false } # Set active: true only if WAIVED
|
||||
|
||||
# Extended fields (optional but recommended):
|
||||
quality_score: 0-100 # 100 - (20*FAILs) - (10*CONCERNS) or use technical-preferences.md weights
|
||||
expires: "{ISO-8601 timestamp}" # Typically 2 weeks from review
|
||||
|
||||
evidence:
|
||||
tests_reviewed: { count }
|
||||
risks_identified: { count }
|
||||
trace:
|
||||
ac_covered: [1, 2, 3] # AC numbers with test coverage
|
||||
ac_gaps: [4] # AC numbers lacking coverage
|
||||
|
||||
nfr_validation:
|
||||
security:
|
||||
status: PASS|CONCERNS|FAIL
|
||||
notes: "Specific findings"
|
||||
performance:
|
||||
status: PASS|CONCERNS|FAIL
|
||||
notes: "Specific findings"
|
||||
reliability:
|
||||
status: PASS|CONCERNS|FAIL
|
||||
notes: "Specific findings"
|
||||
maintainability:
|
||||
status: PASS|CONCERNS|FAIL
|
||||
notes: "Specific findings"
|
||||
|
||||
recommendations:
|
||||
immediate: # Must fix before production
|
||||
- action: "Add rate limiting"
|
||||
refs: ["api/auth/login.ts"]
|
||||
future: # Can be addressed later
|
||||
- action: "Consider caching"
|
||||
refs: ["services/data.ts"]
|
||||
```
|
||||
|
||||
### Gate Decision Criteria
|
||||
|
||||
**Deterministic rule (apply in order):**
|
||||
|
||||
If risk_summary exists, apply its thresholds first (≥9 → FAIL, ≥6 → CONCERNS), then NFR statuses, then top_issues severity.
|
||||
|
||||
1. **Risk thresholds (if risk_summary present):**
|
||||
- If any risk score ≥ 9 → Gate = FAIL (unless waived)
|
||||
- Else if any score ≥ 6 → Gate = CONCERNS
|
||||
|
||||
2. **Test coverage gaps (if trace available):**
|
||||
- If any P0 test from test-design is missing → Gate = CONCERNS
|
||||
- If security/data-loss P0 test missing → Gate = FAIL
|
||||
|
||||
3. **Issue severity:**
|
||||
- If any `top_issues.severity == high` → Gate = FAIL (unless waived)
|
||||
- Else if any `severity == medium` → Gate = CONCERNS
|
||||
|
||||
4. **NFR statuses:**
|
||||
- If any NFR status is FAIL → Gate = FAIL
|
||||
- Else if any NFR status is CONCERNS → Gate = CONCERNS
|
||||
- Else → Gate = PASS
|
||||
|
||||
- WAIVED only when waiver.active: true with reason/approver
|
||||
|
||||
Detailed criteria:
|
||||
|
||||
- **PASS**: All critical requirements met, no blocking issues
|
||||
- **CONCERNS**: Non-critical issues found, team should review
|
||||
- **FAIL**: Critical issues that should be addressed
|
||||
- **WAIVED**: Issues acknowledged but explicitly waived by team
|
||||
|
||||
### Quality Score Calculation
|
||||
|
||||
```text
|
||||
quality_score = 100 - (20 × number of FAILs) - (10 × number of CONCERNS)
|
||||
Bounded between 0 and 100
|
||||
```
|
||||
|
||||
If `technical-preferences.md` defines custom weights, use those instead.
|
||||
|
||||
### Suggested Owner Convention
|
||||
|
||||
For each issue in `top_issues`, include a `suggested_owner`:
|
||||
|
||||
- `dev`: Code changes needed
|
||||
- `sm`: Requirements clarification needed
|
||||
- `po`: Business decision needed
|
||||
|
||||
## Key Principles
|
||||
|
||||
- You are a SENIOR developer reviewing junior/mid-level work
|
||||
- You have the authority and responsibility to improve code directly
|
||||
- You are a Test Architect providing comprehensive quality assessment
|
||||
- You have the authority to improve code directly when appropriate
|
||||
- Always explain your changes for learning purposes
|
||||
- Balance between perfection and pragmatism
|
||||
- Focus on significant improvements, not nitpicks
|
||||
- Focus on risk-based prioritization
|
||||
- Provide actionable recommendations with clear ownership
|
||||
|
||||
## Blocking Conditions
|
||||
|
||||
@@ -140,6 +307,8 @@ Stop the review and request clarification if:
|
||||
|
||||
After review:
|
||||
|
||||
1. If all items are checked and approved: Update story status to "Done"
|
||||
2. If unchecked items remain: Keep status as "Review" for dev to address
|
||||
3. Always provide constructive feedback and explanations for learning
|
||||
1. Update the QA Results section in the story file
|
||||
2. Create the gate file in `docs/qa/gates/`
|
||||
3. Recommend status: "Ready for Done" or "Changes Required" (owner decides)
|
||||
4. If files were modified, list them in QA Results and ask Dev to update File List
|
||||
5. Always provide constructive feedback and actionable recommendations
|
||||
|
||||
353
bmad-core/tasks/risk-profile.md
Normal file
353
bmad-core/tasks/risk-profile.md
Normal file
@@ -0,0 +1,353 @@
|
||||
# risk-profile
|
||||
|
||||
Generate a comprehensive risk assessment matrix for a story implementation using probability × impact analysis.
|
||||
|
||||
## Inputs
|
||||
|
||||
```yaml
|
||||
required:
|
||||
- story_id: "{epic}.{story}" # e.g., "1.3"
|
||||
- story_path: "docs/stories/{epic}.{story}.*.md"
|
||||
- story_title: "{title}" # If missing, derive from story file H1
|
||||
- story_slug: "{slug}" # If missing, derive from title (lowercase, hyphenated)
|
||||
```
|
||||
|
||||
## Purpose
|
||||
|
||||
Identify, assess, and prioritize risks in the story implementation. Provide risk mitigation strategies and testing focus areas based on risk levels.
|
||||
|
||||
## Risk Assessment Framework
|
||||
|
||||
### Risk Categories
|
||||
|
||||
**Category Prefixes:**
|
||||
|
||||
- `TECH`: Technical Risks
|
||||
- `SEC`: Security Risks
|
||||
- `PERF`: Performance Risks
|
||||
- `DATA`: Data Risks
|
||||
- `BUS`: Business Risks
|
||||
- `OPS`: Operational Risks
|
||||
|
||||
1. **Technical Risks (TECH)**
|
||||
- Architecture complexity
|
||||
- Integration challenges
|
||||
- Technical debt
|
||||
- Scalability concerns
|
||||
- System dependencies
|
||||
|
||||
2. **Security Risks (SEC)**
|
||||
- Authentication/authorization flaws
|
||||
- Data exposure vulnerabilities
|
||||
- Injection attacks
|
||||
- Session management issues
|
||||
- Cryptographic weaknesses
|
||||
|
||||
3. **Performance Risks (PERF)**
|
||||
- Response time degradation
|
||||
- Throughput bottlenecks
|
||||
- Resource exhaustion
|
||||
- Database query optimization
|
||||
- Caching failures
|
||||
|
||||
4. **Data Risks (DATA)**
|
||||
- Data loss potential
|
||||
- Data corruption
|
||||
- Privacy violations
|
||||
- Compliance issues
|
||||
- Backup/recovery gaps
|
||||
|
||||
5. **Business Risks (BUS)**
|
||||
- Feature doesn't meet user needs
|
||||
- Revenue impact
|
||||
- Reputation damage
|
||||
- Regulatory non-compliance
|
||||
- Market timing
|
||||
|
||||
6. **Operational Risks (OPS)**
|
||||
- Deployment failures
|
||||
- Monitoring gaps
|
||||
- Incident response readiness
|
||||
- Documentation inadequacy
|
||||
- Knowledge transfer issues
|
||||
|
||||
## Risk Analysis Process
|
||||
|
||||
### 1. Risk Identification
|
||||
|
||||
For each category, identify specific risks:
|
||||
|
||||
```yaml
|
||||
risk:
|
||||
id: "SEC-001" # Use prefixes: SEC, PERF, DATA, BUS, OPS, TECH
|
||||
category: security
|
||||
title: "Insufficient input validation on user forms"
|
||||
description: "Form inputs not properly sanitized could lead to XSS attacks"
|
||||
affected_components:
|
||||
- "UserRegistrationForm"
|
||||
- "ProfileUpdateForm"
|
||||
detection_method: "Code review revealed missing validation"
|
||||
```
|
||||
|
||||
### 2. Risk Assessment
|
||||
|
||||
Evaluate each risk using probability × impact:
|
||||
|
||||
**Probability Levels:**
|
||||
|
||||
- `High (3)`: Likely to occur (>70% chance)
|
||||
- `Medium (2)`: Possible occurrence (30-70% chance)
|
||||
- `Low (1)`: Unlikely to occur (<30% chance)
|
||||
|
||||
**Impact Levels:**
|
||||
|
||||
- `High (3)`: Severe consequences (data breach, system down, major financial loss)
|
||||
- `Medium (2)`: Moderate consequences (degraded performance, minor data issues)
|
||||
- `Low (1)`: Minor consequences (cosmetic issues, slight inconvenience)
|
||||
|
||||
**Risk Score = Probability × Impact**
|
||||
|
||||
- 9: Critical Risk (Red)
|
||||
- 6: High Risk (Orange)
|
||||
- 4: Medium Risk (Yellow)
|
||||
- 2-3: Low Risk (Green)
|
||||
- 1: Minimal Risk (Blue)
|
||||
|
||||
### 3. Risk Prioritization
|
||||
|
||||
Create risk matrix:
|
||||
|
||||
```markdown
|
||||
## Risk Matrix
|
||||
|
||||
| Risk ID | Description | Probability | Impact | Score | Priority |
|
||||
| -------- | ----------------------- | ----------- | ---------- | ----- | -------- |
|
||||
| SEC-001 | XSS vulnerability | High (3) | High (3) | 9 | Critical |
|
||||
| PERF-001 | Slow query on dashboard | Medium (2) | Medium (2) | 4 | Medium |
|
||||
| DATA-001 | Backup failure | Low (1) | High (3) | 3 | Low |
|
||||
```
|
||||
|
||||
### 4. Risk Mitigation Strategies
|
||||
|
||||
For each identified risk, provide mitigation:
|
||||
|
||||
```yaml
|
||||
mitigation:
|
||||
risk_id: "SEC-001"
|
||||
strategy: "preventive" # preventive|detective|corrective
|
||||
actions:
|
||||
- "Implement input validation library (e.g., validator.js)"
|
||||
- "Add CSP headers to prevent XSS execution"
|
||||
- "Sanitize all user inputs before storage"
|
||||
- "Escape all outputs in templates"
|
||||
testing_requirements:
|
||||
- "Security testing with OWASP ZAP"
|
||||
- "Manual penetration testing of forms"
|
||||
- "Unit tests for validation functions"
|
||||
residual_risk: "Low - Some zero-day vulnerabilities may remain"
|
||||
owner: "dev"
|
||||
timeline: "Before deployment"
|
||||
```
|
||||
|
||||
## Outputs
|
||||
|
||||
### Output 1: Gate YAML Block
|
||||
|
||||
Generate for pasting into gate file under `risk_summary`:
|
||||
|
||||
**Output rules:**
|
||||
|
||||
- Only include assessed risks; do not emit placeholders
|
||||
- Sort risks by score (desc) when emitting highest and any tabular lists
|
||||
- If no risks: totals all zeros, omit highest, keep recommendations arrays empty
|
||||
|
||||
```yaml
|
||||
# risk_summary (paste into gate file):
|
||||
risk_summary:
|
||||
totals:
|
||||
critical: X # score 9
|
||||
high: Y # score 6
|
||||
medium: Z # score 4
|
||||
low: W # score 2-3
|
||||
highest:
|
||||
id: SEC-001
|
||||
score: 9
|
||||
title: "XSS on profile form"
|
||||
recommendations:
|
||||
must_fix:
|
||||
- "Add input sanitization & CSP"
|
||||
monitor:
|
||||
- "Add security alerts for auth endpoints"
|
||||
```
|
||||
|
||||
### Output 2: Markdown Report
|
||||
|
||||
**Save to:** `docs/qa/assessments/{epic}.{story}-risk-{YYYYMMDD}.md`
|
||||
|
||||
```markdown
|
||||
# Risk Profile: Story {epic}.{story}
|
||||
|
||||
Date: {date}
|
||||
Reviewer: Quinn (Test Architect)
|
||||
|
||||
## Executive Summary
|
||||
|
||||
- Total Risks Identified: X
|
||||
- Critical Risks: Y
|
||||
- High Risks: Z
|
||||
- Risk Score: XX/100 (calculated)
|
||||
|
||||
## Critical Risks Requiring Immediate Attention
|
||||
|
||||
### 1. [ID]: Risk Title
|
||||
|
||||
**Score: 9 (Critical)**
|
||||
**Probability**: High - Detailed reasoning
|
||||
**Impact**: High - Potential consequences
|
||||
**Mitigation**:
|
||||
|
||||
- Immediate action required
|
||||
- Specific steps to take
|
||||
**Testing Focus**: Specific test scenarios needed
|
||||
|
||||
## Risk Distribution
|
||||
|
||||
### By Category
|
||||
|
||||
- Security: X risks (Y critical)
|
||||
- Performance: X risks (Y critical)
|
||||
- Data: X risks (Y critical)
|
||||
- Business: X risks (Y critical)
|
||||
- Operational: X risks (Y critical)
|
||||
|
||||
### By Component
|
||||
|
||||
- Frontend: X risks
|
||||
- Backend: X risks
|
||||
- Database: X risks
|
||||
- Infrastructure: X risks
|
||||
|
||||
## Detailed Risk Register
|
||||
|
||||
[Full table of all risks with scores and mitigations]
|
||||
|
||||
## Risk-Based Testing Strategy
|
||||
|
||||
### Priority 1: Critical Risk Tests
|
||||
|
||||
- Test scenarios for critical risks
|
||||
- Required test types (security, load, chaos)
|
||||
- Test data requirements
|
||||
|
||||
### Priority 2: High Risk Tests
|
||||
|
||||
- Integration test scenarios
|
||||
- Edge case coverage
|
||||
|
||||
### Priority 3: Medium/Low Risk Tests
|
||||
|
||||
- Standard functional tests
|
||||
- Regression test suite
|
||||
|
||||
## Risk Acceptance Criteria
|
||||
|
||||
### Must Fix Before Production
|
||||
|
||||
- All critical risks (score 9)
|
||||
- High risks affecting security/data
|
||||
|
||||
### Can Deploy with Mitigation
|
||||
|
||||
- Medium risks with compensating controls
|
||||
- Low risks with monitoring in place
|
||||
|
||||
### Accepted Risks
|
||||
|
||||
- Document any risks team accepts
|
||||
- Include sign-off from appropriate authority
|
||||
|
||||
## Monitoring Requirements
|
||||
|
||||
Post-deployment monitoring for:
|
||||
|
||||
- Performance metrics for PERF risks
|
||||
- Security alerts for SEC risks
|
||||
- Error rates for operational risks
|
||||
- Business KPIs for business risks
|
||||
|
||||
## Risk Review Triggers
|
||||
|
||||
Review and update risk profile when:
|
||||
|
||||
- Architecture changes significantly
|
||||
- New integrations added
|
||||
- Security vulnerabilities discovered
|
||||
- Performance issues reported
|
||||
- Regulatory requirements change
|
||||
```
|
||||
|
||||
## Risk Scoring Algorithm
|
||||
|
||||
Calculate overall story risk score:
|
||||
|
||||
```
|
||||
Base Score = 100
|
||||
For each risk:
|
||||
- Critical (9): Deduct 20 points
|
||||
- High (6): Deduct 10 points
|
||||
- Medium (4): Deduct 5 points
|
||||
- Low (2-3): Deduct 2 points
|
||||
|
||||
Minimum score = 0 (extremely risky)
|
||||
Maximum score = 100 (minimal risk)
|
||||
```
|
||||
|
||||
## Risk-Based Recommendations
|
||||
|
||||
Based on risk profile, recommend:
|
||||
|
||||
1. **Testing Priority**
|
||||
- Which tests to run first
|
||||
- Additional test types needed
|
||||
- Test environment requirements
|
||||
|
||||
2. **Development Focus**
|
||||
- Code review emphasis areas
|
||||
- Additional validation needed
|
||||
- Security controls to implement
|
||||
|
||||
3. **Deployment Strategy**
|
||||
- Phased rollout for high-risk changes
|
||||
- Feature flags for risky features
|
||||
- Rollback procedures
|
||||
|
||||
4. **Monitoring Setup**
|
||||
- Metrics to track
|
||||
- Alerts to configure
|
||||
- Dashboard requirements
|
||||
|
||||
## Integration with Quality Gates
|
||||
|
||||
**Deterministic gate mapping:**
|
||||
|
||||
- Any risk with score ≥ 9 → Gate = FAIL (unless waived)
|
||||
- Else if any score ≥ 6 → Gate = CONCERNS
|
||||
- Else → Gate = PASS
|
||||
- Unmitigated risks → Document in gate
|
||||
|
||||
### Output 3: Story Hook Line
|
||||
|
||||
**Print this line for review task to quote:**
|
||||
|
||||
```
|
||||
Risk profile: docs/qa/assessments/{epic}.{story}-risk-{YYYYMMDD}.md
|
||||
```
|
||||
|
||||
## Key Principles
|
||||
|
||||
- Identify risks early and systematically
|
||||
- Use consistent probability × impact scoring
|
||||
- Provide actionable mitigation strategies
|
||||
- Link risks to specific test requirements
|
||||
- Track residual risk after mitigation
|
||||
- Update risk profile as story evolves
|
||||
@@ -91,13 +91,11 @@ CRITICAL: Use proper parsing that understands markdown context. A ## inside a co
|
||||
For each extracted section:
|
||||
|
||||
1. **Generate filename**: Convert the section heading to lowercase-dash-case
|
||||
|
||||
- Remove special characters
|
||||
- Replace spaces with dashes
|
||||
- Example: "## Tech Stack" → `tech-stack.md`
|
||||
|
||||
2. **Adjust heading levels**:
|
||||
|
||||
- The level 2 heading becomes level 1 (# instead of ##) in the sharded new document
|
||||
- All subsection levels decrease by 1:
|
||||
|
||||
|
||||
174
bmad-core/tasks/test-design.md
Normal file
174
bmad-core/tasks/test-design.md
Normal file
@@ -0,0 +1,174 @@
|
||||
# test-design
|
||||
|
||||
Create comprehensive test scenarios with appropriate test level recommendations for story implementation.
|
||||
|
||||
## Inputs
|
||||
|
||||
```yaml
|
||||
required:
|
||||
- story_id: "{epic}.{story}" # e.g., "1.3"
|
||||
- story_path: "{devStoryLocation}/{epic}.{story}.*.md" # Path from core-config.yaml
|
||||
- story_title: "{title}" # If missing, derive from story file H1
|
||||
- story_slug: "{slug}" # If missing, derive from title (lowercase, hyphenated)
|
||||
```
|
||||
|
||||
## Purpose
|
||||
|
||||
Design a complete test strategy that identifies what to test, at which level (unit/integration/e2e), and why. This ensures efficient test coverage without redundancy while maintaining appropriate test boundaries.
|
||||
|
||||
## Dependencies
|
||||
|
||||
```yaml
|
||||
data:
|
||||
- test-levels-framework.md # Unit/Integration/E2E decision criteria
|
||||
- test-priorities-matrix.md # P0/P1/P2/P3 classification system
|
||||
```
|
||||
|
||||
## Process
|
||||
|
||||
### 1. Analyze Story Requirements
|
||||
|
||||
Break down each acceptance criterion into testable scenarios. For each AC:
|
||||
|
||||
- Identify the core functionality to test
|
||||
- Determine data variations needed
|
||||
- Consider error conditions
|
||||
- Note edge cases
|
||||
|
||||
### 2. Apply Test Level Framework
|
||||
|
||||
**Reference:** Load `test-levels-framework.md` for detailed criteria
|
||||
|
||||
Quick rules:
|
||||
|
||||
- **Unit**: Pure logic, algorithms, calculations
|
||||
- **Integration**: Component interactions, DB operations
|
||||
- **E2E**: Critical user journeys, compliance
|
||||
|
||||
### 3. Assign Priorities
|
||||
|
||||
**Reference:** Load `test-priorities-matrix.md` for classification
|
||||
|
||||
Quick priority assignment:
|
||||
|
||||
- **P0**: Revenue-critical, security, compliance
|
||||
- **P1**: Core user journeys, frequently used
|
||||
- **P2**: Secondary features, admin functions
|
||||
- **P3**: Nice-to-have, rarely used
|
||||
|
||||
### 4. Design Test Scenarios
|
||||
|
||||
For each identified test need, create:
|
||||
|
||||
```yaml
|
||||
test_scenario:
|
||||
id: "{epic}.{story}-{LEVEL}-{SEQ}"
|
||||
requirement: "AC reference"
|
||||
priority: P0|P1|P2|P3
|
||||
level: unit|integration|e2e
|
||||
description: "What is being tested"
|
||||
justification: "Why this level was chosen"
|
||||
mitigates_risks: ["RISK-001"] # If risk profile exists
|
||||
```
|
||||
|
||||
### 5. Validate Coverage
|
||||
|
||||
Ensure:
|
||||
|
||||
- Every AC has at least one test
|
||||
- No duplicate coverage across levels
|
||||
- Critical paths have multiple levels
|
||||
- Risk mitigations are addressed
|
||||
|
||||
## Outputs
|
||||
|
||||
### Output 1: Test Design Document
|
||||
|
||||
**Save to:** `docs/qa/assessments/{epic}.{story}-test-design-{YYYYMMDD}.md`
|
||||
|
||||
```markdown
|
||||
# Test Design: Story {epic}.{story}
|
||||
|
||||
Date: {date}
|
||||
Designer: Quinn (Test Architect)
|
||||
|
||||
## Test Strategy Overview
|
||||
|
||||
- Total test scenarios: X
|
||||
- Unit tests: Y (A%)
|
||||
- Integration tests: Z (B%)
|
||||
- E2E tests: W (C%)
|
||||
- Priority distribution: P0: X, P1: Y, P2: Z
|
||||
|
||||
## Test Scenarios by Acceptance Criteria
|
||||
|
||||
### AC1: {description}
|
||||
|
||||
#### Scenarios
|
||||
|
||||
| ID | Level | Priority | Test | Justification |
|
||||
| ------------ | ----------- | -------- | ------------------------- | ------------------------ |
|
||||
| 1.3-UNIT-001 | Unit | P0 | Validate input format | Pure validation logic |
|
||||
| 1.3-INT-001 | Integration | P0 | Service processes request | Multi-component flow |
|
||||
| 1.3-E2E-001 | E2E | P1 | User completes journey | Critical path validation |
|
||||
|
||||
[Continue for all ACs...]
|
||||
|
||||
## Risk Coverage
|
||||
|
||||
[Map test scenarios to identified risks if risk profile exists]
|
||||
|
||||
## Recommended Execution Order
|
||||
|
||||
1. P0 Unit tests (fail fast)
|
||||
2. P0 Integration tests
|
||||
3. P0 E2E tests
|
||||
4. P1 tests in order
|
||||
5. P2+ as time permits
|
||||
```
|
||||
|
||||
### Output 2: Gate YAML Block
|
||||
|
||||
Generate for inclusion in quality gate:
|
||||
|
||||
```yaml
|
||||
test_design:
|
||||
scenarios_total: X
|
||||
by_level:
|
||||
unit: Y
|
||||
integration: Z
|
||||
e2e: W
|
||||
by_priority:
|
||||
p0: A
|
||||
p1: B
|
||||
p2: C
|
||||
coverage_gaps: [] # List any ACs without tests
|
||||
```
|
||||
|
||||
### Output 3: Trace References
|
||||
|
||||
Print for use by trace-requirements task:
|
||||
|
||||
```text
|
||||
Test design matrix: docs/qa/assessments/{epic}.{story}-test-design-{YYYYMMDD}.md
|
||||
P0 tests identified: {count}
|
||||
```
|
||||
|
||||
## Quality Checklist
|
||||
|
||||
Before finalizing, verify:
|
||||
|
||||
- [ ] Every AC has test coverage
|
||||
- [ ] Test levels are appropriate (not over-testing)
|
||||
- [ ] No duplicate coverage across levels
|
||||
- [ ] Priorities align with business risk
|
||||
- [ ] Test IDs follow naming convention
|
||||
- [ ] Scenarios are atomic and independent
|
||||
|
||||
## Key Principles
|
||||
|
||||
- **Shift left**: Prefer unit over integration, integration over E2E
|
||||
- **Risk-based**: Focus on what could go wrong
|
||||
- **Efficient coverage**: Test once at the right level
|
||||
- **Maintainability**: Consider long-term test maintenance
|
||||
- **Fast feedback**: Quick tests run first
|
||||
264
bmad-core/tasks/trace-requirements.md
Normal file
264
bmad-core/tasks/trace-requirements.md
Normal file
@@ -0,0 +1,264 @@
|
||||
# trace-requirements
|
||||
|
||||
Map story requirements to test cases using Given-When-Then patterns for comprehensive traceability.
|
||||
|
||||
## Purpose
|
||||
|
||||
Create a requirements traceability matrix that ensures every acceptance criterion has corresponding test coverage. This task helps identify gaps in testing and ensures all requirements are validated.
|
||||
|
||||
**IMPORTANT**: Given-When-Then is used here for documenting the mapping between requirements and tests, NOT for writing the actual test code. Tests should follow your project's testing standards (no BDD syntax in test code).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Story file with clear acceptance criteria
|
||||
- Access to test files or test specifications
|
||||
- Understanding of the implementation
|
||||
|
||||
## Traceability Process
|
||||
|
||||
### 1. Extract Requirements
|
||||
|
||||
Identify all testable requirements from:
|
||||
|
||||
- Acceptance Criteria (primary source)
|
||||
- User story statement
|
||||
- Tasks/subtasks with specific behaviors
|
||||
- Non-functional requirements mentioned
|
||||
- Edge cases documented
|
||||
|
||||
### 2. Map to Test Cases
|
||||
|
||||
For each requirement, document which tests validate it. Use Given-When-Then to describe what the test validates (not how it's written):
|
||||
|
||||
```yaml
|
||||
requirement: "AC1: User can login with valid credentials"
|
||||
test_mappings:
|
||||
- test_file: "auth/login.test.ts"
|
||||
test_case: "should successfully login with valid email and password"
|
||||
# Given-When-Then describes WHAT the test validates, not HOW it's coded
|
||||
given: "A registered user with valid credentials"
|
||||
when: "They submit the login form"
|
||||
then: "They are redirected to dashboard and session is created"
|
||||
coverage: full
|
||||
|
||||
- test_file: "e2e/auth-flow.test.ts"
|
||||
test_case: "complete login flow"
|
||||
given: "User on login page"
|
||||
when: "Entering valid credentials and submitting"
|
||||
then: "Dashboard loads with user data"
|
||||
coverage: integration
|
||||
```
|
||||
|
||||
### 3. Coverage Analysis
|
||||
|
||||
Evaluate coverage for each requirement:
|
||||
|
||||
**Coverage Levels:**
|
||||
|
||||
- `full`: Requirement completely tested
|
||||
- `partial`: Some aspects tested, gaps exist
|
||||
- `none`: No test coverage found
|
||||
- `integration`: Covered in integration/e2e tests only
|
||||
- `unit`: Covered in unit tests only
|
||||
|
||||
### 4. Gap Identification
|
||||
|
||||
Document any gaps found:
|
||||
|
||||
```yaml
|
||||
coverage_gaps:
|
||||
- requirement: "AC3: Password reset email sent within 60 seconds"
|
||||
gap: "No test for email delivery timing"
|
||||
severity: medium
|
||||
suggested_test:
|
||||
type: integration
|
||||
description: "Test email service SLA compliance"
|
||||
|
||||
- requirement: "AC5: Support 1000 concurrent users"
|
||||
gap: "No load testing implemented"
|
||||
severity: high
|
||||
suggested_test:
|
||||
type: performance
|
||||
description: "Load test with 1000 concurrent connections"
|
||||
```
|
||||
|
||||
## Outputs
|
||||
|
||||
### Output 1: Gate YAML Block
|
||||
|
||||
**Generate for pasting into gate file under `trace`:**
|
||||
|
||||
```yaml
|
||||
trace:
|
||||
totals:
|
||||
requirements: X
|
||||
full: Y
|
||||
partial: Z
|
||||
none: W
|
||||
planning_ref: "docs/qa/assessments/{epic}.{story}-test-design-{YYYYMMDD}.md"
|
||||
uncovered:
|
||||
- ac: "AC3"
|
||||
reason: "No test found for password reset timing"
|
||||
notes: "See docs/qa/assessments/{epic}.{story}-trace-{YYYYMMDD}.md"
|
||||
```
|
||||
|
||||
### Output 2: Traceability Report
|
||||
|
||||
**Save to:** `docs/qa/assessments/{epic}.{story}-trace-{YYYYMMDD}.md`
|
||||
|
||||
Create a traceability report with:
|
||||
|
||||
```markdown
|
||||
# Requirements Traceability Matrix
|
||||
|
||||
## Story: {epic}.{story} - {title}
|
||||
|
||||
### Coverage Summary
|
||||
|
||||
- Total Requirements: X
|
||||
- Fully Covered: Y (Z%)
|
||||
- Partially Covered: A (B%)
|
||||
- Not Covered: C (D%)
|
||||
|
||||
### Requirement Mappings
|
||||
|
||||
#### AC1: {Acceptance Criterion 1}
|
||||
|
||||
**Coverage: FULL**
|
||||
|
||||
Given-When-Then Mappings:
|
||||
|
||||
- **Unit Test**: `auth.service.test.ts::validateCredentials`
|
||||
- Given: Valid user credentials
|
||||
- When: Validation method called
|
||||
- Then: Returns true with user object
|
||||
|
||||
- **Integration Test**: `auth.integration.test.ts::loginFlow`
|
||||
- Given: User with valid account
|
||||
- When: Login API called
|
||||
- Then: JWT token returned and session created
|
||||
|
||||
#### AC2: {Acceptance Criterion 2}
|
||||
|
||||
**Coverage: PARTIAL**
|
||||
|
||||
[Continue for all ACs...]
|
||||
|
||||
### Critical Gaps
|
||||
|
||||
1. **Performance Requirements**
|
||||
- Gap: No load testing for concurrent users
|
||||
- Risk: High - Could fail under production load
|
||||
- Action: Implement load tests using k6 or similar
|
||||
|
||||
2. **Security Requirements**
|
||||
- Gap: Rate limiting not tested
|
||||
- Risk: Medium - Potential DoS vulnerability
|
||||
- Action: Add rate limit tests to integration suite
|
||||
|
||||
### Test Design Recommendations
|
||||
|
||||
Based on gaps identified, recommend:
|
||||
|
||||
1. Additional test scenarios needed
|
||||
2. Test types to implement (unit/integration/e2e/performance)
|
||||
3. Test data requirements
|
||||
4. Mock/stub strategies
|
||||
|
||||
### Risk Assessment
|
||||
|
||||
- **High Risk**: Requirements with no coverage
|
||||
- **Medium Risk**: Requirements with only partial coverage
|
||||
- **Low Risk**: Requirements with full unit + integration coverage
|
||||
```
|
||||
|
||||
## Traceability Best Practices
|
||||
|
||||
### Given-When-Then for Mapping (Not Test Code)
|
||||
|
||||
Use Given-When-Then to document what each test validates:
|
||||
|
||||
**Given**: The initial context the test sets up
|
||||
|
||||
- What state/data the test prepares
|
||||
- User context being simulated
|
||||
- System preconditions
|
||||
|
||||
**When**: The action the test performs
|
||||
|
||||
- What the test executes
|
||||
- API calls or user actions tested
|
||||
- Events triggered
|
||||
|
||||
**Then**: What the test asserts
|
||||
|
||||
- Expected outcomes verified
|
||||
- State changes checked
|
||||
- Values validated
|
||||
|
||||
**Note**: This is for documentation only. Actual test code follows your project's standards (e.g., describe/it blocks, no BDD syntax).
|
||||
|
||||
### Coverage Priority
|
||||
|
||||
Prioritize coverage based on:
|
||||
|
||||
1. Critical business flows
|
||||
2. Security-related requirements
|
||||
3. Data integrity requirements
|
||||
4. User-facing features
|
||||
5. Performance SLAs
|
||||
|
||||
### Test Granularity
|
||||
|
||||
Map at appropriate levels:
|
||||
|
||||
- Unit tests for business logic
|
||||
- Integration tests for component interaction
|
||||
- E2E tests for user journeys
|
||||
- Performance tests for NFRs
|
||||
|
||||
## Quality Indicators
|
||||
|
||||
Good traceability shows:
|
||||
|
||||
- Every AC has at least one test
|
||||
- Critical paths have multiple test levels
|
||||
- Edge cases are explicitly covered
|
||||
- NFRs have appropriate test types
|
||||
- Clear Given-When-Then for each test
|
||||
|
||||
## Red Flags
|
||||
|
||||
Watch for:
|
||||
|
||||
- ACs with no test coverage
|
||||
- Tests that don't map to requirements
|
||||
- Vague test descriptions
|
||||
- Missing edge case coverage
|
||||
- NFRs without specific tests
|
||||
|
||||
## Integration with Gates
|
||||
|
||||
This traceability feeds into quality gates:
|
||||
|
||||
- Critical gaps → FAIL
|
||||
- Minor gaps → CONCERNS
|
||||
- Missing P0 tests from test-design → CONCERNS
|
||||
|
||||
### Output 3: Story Hook Line
|
||||
|
||||
**Print this line for review task to quote:**
|
||||
|
||||
```text
|
||||
Trace matrix: docs/qa/assessments/{epic}.{story}-trace-{YYYYMMDD}.md
|
||||
```
|
||||
|
||||
- Full coverage → PASS contribution
|
||||
|
||||
## Key Principles
|
||||
|
||||
- Every requirement must be testable
|
||||
- Use Given-When-Then for clarity
|
||||
- Identify both presence and absence
|
||||
- Prioritize based on risk
|
||||
- Make recommendations actionable
|
||||
Reference in New Issue
Block a user