feat: transform QA agent into Test Architect with advanced quality capabilities
- Add 6 specialized quality assessment commands - Implement risk-based testing with scoring - Create quality gate system with deterministic decisions - Add comprehensive test design and NFR validation - Update documentation with stage-based workflow integration
This commit is contained in:
Murat Ozcan
@@ -128,7 +128,7 @@ Critical: For brownfield, ALWAYS include criteria about maintaining existing fun
|
||||
Standard structure:
|
||||
|
||||
1. New functionality works as specified
|
||||
2. Existing {{affected feature}} continues to work unchanged
|
||||
2. Existing {{affected feature}} continues to work unchanged
|
||||
3. Integration with {{existing system}} maintains current behavior
|
||||
4. No regression in {{related area}}
|
||||
5. Performance remains within acceptable bounds
|
||||
@@ -139,16 +139,19 @@ Critical: This is where you'll need to be interactive with the user if informati
|
||||
|
||||
Create Dev Technical Guidance section with available information:
|
||||
|
||||
```markdown
|
||||
````markdown
|
||||
## Dev Technical Guidance
|
||||
|
||||
### Existing System Context
|
||||
|
||||
[Extract from available documentation]
|
||||
|
||||
### Integration Approach
|
||||
|
||||
[Based on patterns found or ask user]
|
||||
|
||||
### Technical Constraints
|
||||
|
||||
[From documentation or user input]
|
||||
|
||||
### Missing Information
|
||||
@@ -191,6 +194,7 @@ Example task structure for brownfield:
|
||||
- [ ] Integration test for {{integration point}}
|
||||
- [ ] Update existing tests if needed
|
||||
```
|
||||
````
|
||||
|
||||
### 5. Risk Assessment and Mitigation
|
||||
|
||||
@@ -202,14 +206,17 @@ Add section for brownfield-specific risks:
|
||||
## Risk Assessment
|
||||
|
||||
### Implementation Risks
|
||||
|
||||
- **Primary Risk**: {{main risk to existing system}}
|
||||
- **Mitigation**: {{how to address}}
|
||||
- **Verification**: {{how to confirm safety}}
|
||||
|
||||
### Rollback Plan
|
||||
|
||||
- {{Simple steps to undo changes if needed}}
|
||||
|
||||
### Safety Checks
|
||||
|
||||
- [ ] Existing {{feature}} tested before changes
|
||||
- [ ] Changes can be feature-flagged or isolated
|
||||
- [ ] Rollback procedure documented
|
||||
@@ -252,6 +259,7 @@ Include header noting documentation context:
|
||||
<!-- Context: Brownfield enhancement to {{existing system}} -->
|
||||
|
||||
## Status: Draft
|
||||
|
||||
[Rest of story content...]
|
||||
```
|
||||
|
||||
@@ -272,7 +280,7 @@ Key Integration Points Identified:
|
||||
Risks Noted:
|
||||
- {{primary risk}}
|
||||
|
||||
{{If missing info}}:
|
||||
{{If missing info}}:
|
||||
Note: Some technical details were unclear. The story includes exploration tasks to gather needed information during implementation.
|
||||
|
||||
Next Steps:
|
||||
|
||||
@@ -21,63 +21,54 @@ CRITICAL: First, help the user select the most appropriate research focus based
|
||||
Present these numbered options to the user:
|
||||
|
||||
1. **Product Validation Research**
|
||||
|
||||
- Validate product hypotheses and market fit
|
||||
- Test assumptions about user needs and solutions
|
||||
- Assess technical and business feasibility
|
||||
- Identify risks and mitigation strategies
|
||||
|
||||
2. **Market Opportunity Research**
|
||||
|
||||
- Analyze market size and growth potential
|
||||
- Identify market segments and dynamics
|
||||
- Assess market entry strategies
|
||||
- Evaluate timing and market readiness
|
||||
|
||||
3. **User & Customer Research**
|
||||
|
||||
- Deep dive into user personas and behaviors
|
||||
- Understand jobs-to-be-done and pain points
|
||||
- Map customer journeys and touchpoints
|
||||
- Analyze willingness to pay and value perception
|
||||
|
||||
4. **Competitive Intelligence Research**
|
||||
|
||||
- Detailed competitor analysis and positioning
|
||||
- Feature and capability comparisons
|
||||
- Business model and strategy analysis
|
||||
- Identify competitive advantages and gaps
|
||||
|
||||
5. **Technology & Innovation Research**
|
||||
|
||||
- Assess technology trends and possibilities
|
||||
- Evaluate technical approaches and architectures
|
||||
- Identify emerging technologies and disruptions
|
||||
- Analyze build vs. buy vs. partner options
|
||||
|
||||
6. **Industry & Ecosystem Research**
|
||||
|
||||
- Map industry value chains and dynamics
|
||||
- Identify key players and relationships
|
||||
- Analyze regulatory and compliance factors
|
||||
- Understand partnership opportunities
|
||||
|
||||
7. **Strategic Options Research**
|
||||
|
||||
- Evaluate different strategic directions
|
||||
- Assess business model alternatives
|
||||
- Analyze go-to-market strategies
|
||||
- Consider expansion and scaling paths
|
||||
|
||||
8. **Risk & Feasibility Research**
|
||||
|
||||
- Identify and assess various risk factors
|
||||
- Evaluate implementation challenges
|
||||
- Analyze resource requirements
|
||||
- Consider regulatory and legal implications
|
||||
|
||||
9. **Custom Research Focus**
|
||||
|
||||
- User-defined research objectives
|
||||
- Specialized domain investigation
|
||||
- Cross-functional research needs
|
||||
@@ -246,13 +237,11 @@ CRITICAL: collaborate with the user to develop specific, actionable research que
|
||||
### 5. Review and Refinement
|
||||
|
||||
1. **Present Complete Prompt**
|
||||
|
||||
- Show the full research prompt
|
||||
- Explain key elements and rationale
|
||||
- Highlight any assumptions made
|
||||
|
||||
2. **Gather Feedback**
|
||||
|
||||
- Are the objectives clear and correct?
|
||||
- Do the questions address all concerns?
|
||||
- Is the scope appropriate?
|
||||
|
||||
@@ -111,9 +111,9 @@ This document captures the CURRENT STATE of the [Project Name] codebase, includi
|
||||
|
||||
### Change Log
|
||||
|
||||
| Date | Version | Description | Author |
|
||||
|------|---------|-------------|--------|
|
||||
| [Date] | 1.0 | Initial brownfield analysis | [Analyst] |
|
||||
| Date | Version | Description | Author |
|
||||
| ------ | ------- | --------------------------- | --------- |
|
||||
| [Date] | 1.0 | Initial brownfield analysis | [Analyst] |
|
||||
|
||||
## Quick Reference - Key Files and Entry Points
|
||||
|
||||
@@ -136,11 +136,11 @@ This document captures the CURRENT STATE of the [Project Name] codebase, includi
|
||||
|
||||
### Actual Tech Stack (from package.json/requirements.txt)
|
||||
|
||||
| Category | Technology | Version | Notes |
|
||||
|----------|------------|---------|--------|
|
||||
| Runtime | Node.js | 16.x | [Any constraints] |
|
||||
| Framework | Express | 4.18.2 | [Custom middleware?] |
|
||||
| Database | PostgreSQL | 13 | [Connection pooling setup] |
|
||||
| Category | Technology | Version | Notes |
|
||||
| --------- | ---------- | ------- | -------------------------- |
|
||||
| Runtime | Node.js | 16.x | [Any constraints] |
|
||||
| Framework | Express | 4.18.2 | [Custom middleware?] |
|
||||
| Database | PostgreSQL | 13 | [Connection pooling setup] |
|
||||
|
||||
etc...
|
||||
|
||||
@@ -179,6 +179,7 @@ project-root/
|
||||
### Data Models
|
||||
|
||||
Instead of duplicating, reference actual model files:
|
||||
|
||||
- **User Model**: See `src/models/User.js`
|
||||
- **Order Model**: See `src/models/Order.js`
|
||||
- **Related Types**: TypeScript definitions in `src/types/`
|
||||
@@ -208,10 +209,10 @@ Instead of duplicating, reference actual model files:
|
||||
|
||||
### External Services
|
||||
|
||||
| Service | Purpose | Integration Type | Key Files |
|
||||
|---------|---------|------------------|-----------|
|
||||
| Stripe | Payments | REST API | `src/integrations/stripe/` |
|
||||
| SendGrid | Emails | SDK | `src/services/emailService.js` |
|
||||
| Service | Purpose | Integration Type | Key Files |
|
||||
| -------- | -------- | ---------------- | ------------------------------ |
|
||||
| Stripe | Payments | REST API | `src/integrations/stripe/` |
|
||||
| SendGrid | Emails | SDK | `src/services/emailService.js` |
|
||||
|
||||
etc...
|
||||
|
||||
@@ -256,6 +257,7 @@ npm run test:integration # Runs integration tests (requires local DB)
|
||||
### Files That Will Need Modification
|
||||
|
||||
Based on the enhancement requirements, these files will be affected:
|
||||
|
||||
- `src/services/userService.js` - Add new user fields
|
||||
- `src/models/User.js` - Update schema
|
||||
- `src/routes/userRoutes.js` - New endpoints
|
||||
@@ -338,4 +340,4 @@ Apply the advanced elicitation task after major sections to refine based on user
|
||||
- References actual files rather than duplicating content when possible
|
||||
- Documents technical debt, workarounds, and constraints honestly
|
||||
- For brownfield projects with PRD: Provides clear enhancement impact analysis
|
||||
- The goal is PRACTICAL documentation for AI agents doing real work
|
||||
- The goal is PRACTICAL documentation for AI agents doing real work
|
||||
|
||||
@@ -43,7 +43,7 @@ If user selects Option 1, present numbered list of techniques from the brainstor
|
||||
1. Apply selected technique according to data file description
|
||||
2. Keep engaging with technique until user indicates they want to:
|
||||
- Choose a different technique
|
||||
- Apply current ideas to a new technique
|
||||
- Apply current ideas to a new technique
|
||||
- Move to convergent phase
|
||||
- End session
|
||||
|
||||
|
||||
@@ -11,14 +11,12 @@ You are now operating as a Documentation Indexer. Your goal is to ensure all doc
|
||||
### Required Steps
|
||||
|
||||
1. First, locate and scan:
|
||||
|
||||
- The `docs/` directory and all subdirectories
|
||||
- The existing `docs/index.md` file (create if absent)
|
||||
- All markdown (`.md`) and text (`.txt`) files in the documentation structure
|
||||
- Note the folder structure for hierarchical organization
|
||||
|
||||
2. For the existing `docs/index.md`:
|
||||
|
||||
- Parse current entries
|
||||
- Note existing file references and descriptions
|
||||
- Identify any broken links or missing files
|
||||
@@ -26,7 +24,6 @@ You are now operating as a Documentation Indexer. Your goal is to ensure all doc
|
||||
- Preserve existing folder sections
|
||||
|
||||
3. For each documentation file found:
|
||||
|
||||
- Extract the title (from first heading or filename)
|
||||
- Generate a brief description by analyzing the content
|
||||
- Create a relative markdown link to the file
|
||||
@@ -35,7 +32,6 @@ You are now operating as a Documentation Indexer. Your goal is to ensure all doc
|
||||
- If missing or outdated, prepare an update
|
||||
|
||||
4. For any missing or non-existent files found in index:
|
||||
|
||||
- Present a list of all entries that reference non-existent files
|
||||
- For each entry:
|
||||
- Show the full entry details (title, path, description)
|
||||
@@ -88,7 +84,6 @@ Documents within the `another-folder/` directory:
|
||||
### [Nested Document](./another-folder/document.md)
|
||||
|
||||
Description of nested document.
|
||||
|
||||
```
|
||||
|
||||
### Index Entry Format
|
||||
@@ -157,7 +152,6 @@ For each file referenced in the index but not found in the filesystem:
|
||||
### Special Cases
|
||||
|
||||
1. **Sharded Documents**: If a folder contains an `index.md` file, treat it as a sharded document:
|
||||
|
||||
- Use the folder's `index.md` title as the section title
|
||||
- List the folder's documents as subsections
|
||||
- Note in the description that this is a multi-part document
|
||||
|
||||
@@ -6,7 +6,7 @@ Provide a user-friendly interface to the BMad knowledge base without overwhelmin
|
||||
|
||||
## Instructions
|
||||
|
||||
When entering KB mode (*kb-mode), follow these steps:
|
||||
When entering KB mode (\*kb-mode), follow these steps:
|
||||
|
||||
### 1. Welcome and Guide
|
||||
|
||||
@@ -48,12 +48,12 @@ Or ask me about anything else related to BMad-Method!
|
||||
When user is done or wants to exit KB mode:
|
||||
|
||||
- Summarize key points discussed if helpful
|
||||
- Remind them they can return to KB mode anytime with *kb-mode
|
||||
- Remind them they can return to KB mode anytime with \*kb-mode
|
||||
- Suggest next steps based on what was discussed
|
||||
|
||||
## Example Interaction
|
||||
|
||||
**User**: *kb-mode
|
||||
**User**: \*kb-mode
|
||||
|
||||
**Assistant**: I've entered KB mode and have access to the full BMad knowledge base. I can help you with detailed information about any aspect of BMad-Method.
|
||||
|
||||
|
||||
@@ -6,9 +6,9 @@ Quick NFR validation focused on the core four: security, performance, reliabilit
|
||||
|
||||
```yaml
|
||||
required:
|
||||
- story_id: "{epic}.{story}" # e.g., "1.3"
|
||||
- story_id: "{epic}.{story}" # e.g., "1.3"
|
||||
- story_path: "docs/stories/{epic}.{story}.*.md"
|
||||
|
||||
|
||||
optional:
|
||||
- architecture_refs: "docs/architecture/*.md"
|
||||
- technical_preferences: "docs/technical-preferences.md"
|
||||
@@ -18,6 +18,7 @@ optional:
|
||||
## Purpose
|
||||
|
||||
Assess non-functional requirements for a story and generate:
|
||||
|
||||
1. YAML block for the gate file's `nfr_validation` section
|
||||
2. Brief markdown assessment saved to `docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md`
|
||||
|
||||
@@ -26,6 +27,7 @@ Assess non-functional requirements for a story and generate:
|
||||
### 0. Fail-safe for Missing Inputs
|
||||
|
||||
If story_path or story file can't be found:
|
||||
|
||||
- Still create assessment file with note: "Source story not found"
|
||||
- Set all selected NFRs to CONCERNS with notes: "Target unknown / evidence missing"
|
||||
- Continue with assessment to provide value
|
||||
@@ -38,7 +40,7 @@ If story_path or story file can't be found:
|
||||
```text
|
||||
Which NFRs should I assess? (Enter numbers or press Enter for default)
|
||||
[1] Security (default)
|
||||
[2] Performance (default)
|
||||
[2] Performance (default)
|
||||
[3] Reliability (default)
|
||||
[4] Maintainability (default)
|
||||
[5] Usability
|
||||
@@ -52,6 +54,7 @@ Which NFRs should I assess? (Enter numbers or press Enter for default)
|
||||
### 2. Check for Thresholds
|
||||
|
||||
Look for NFR requirements in:
|
||||
|
||||
- Story acceptance criteria
|
||||
- `docs/architecture/*.md` files
|
||||
- `docs/technical-preferences.md`
|
||||
@@ -72,6 +75,7 @@ No security requirements found. Required auth method?
|
||||
### 3. Quick Assessment
|
||||
|
||||
For each selected NFR, check:
|
||||
|
||||
- Is there evidence it's implemented?
|
||||
- Can we validate it?
|
||||
- Are there obvious gaps?
|
||||
@@ -86,7 +90,7 @@ Generate ONLY for NFRs actually assessed (no placeholders):
|
||||
# Gate YAML (copy/paste):
|
||||
nfr_validation:
|
||||
_assessed: [security, performance, reliability, maintainability]
|
||||
security:
|
||||
security:
|
||||
status: CONCERNS
|
||||
notes: "No rate limiting on auth endpoints"
|
||||
performance:
|
||||
@@ -103,7 +107,7 @@ nfr_validation:
|
||||
## Deterministic Status Rules
|
||||
|
||||
- **FAIL**: Any selected NFR has critical gap or target clearly not met
|
||||
- **CONCERNS**: No FAILs, but any NFR is unknown/partial/missing evidence
|
||||
- **CONCERNS**: No FAILs, but any NFR is unknown/partial/missing evidence
|
||||
- **PASS**: All selected NFRs meet targets with evidence
|
||||
|
||||
## Quality Score Calculation
|
||||
@@ -123,18 +127,21 @@ If `technical-preferences.md` defines custom weights, use those instead.
|
||||
|
||||
```markdown
|
||||
# NFR Assessment: {epic}.{story}
|
||||
|
||||
Date: {date}
|
||||
Reviewer: Quinn
|
||||
|
||||
<!-- Note: Source story not found (if applicable) -->
|
||||
|
||||
## Summary
|
||||
|
||||
- Security: CONCERNS - Missing rate limiting
|
||||
- Performance: PASS - Meets <200ms requirement
|
||||
- Reliability: PASS - Proper error handling
|
||||
- Maintainability: CONCERNS - Test coverage below target
|
||||
|
||||
## Critical Issues
|
||||
|
||||
1. **No rate limiting** (Security)
|
||||
- Risk: Brute force attacks possible
|
||||
- Fix: Add rate limiting middleware to auth endpoints
|
||||
@@ -144,6 +151,7 @@ Reviewer: Quinn
|
||||
- Fix: Add tests for uncovered branches
|
||||
|
||||
## Quick Wins
|
||||
|
||||
- Add rate limiting: ~2 hours
|
||||
- Increase test coverage: ~4 hours
|
||||
- Add performance monitoring: ~1 hour
|
||||
@@ -152,6 +160,7 @@ Reviewer: Quinn
|
||||
## Output 3: Story Update Line
|
||||
|
||||
**End with this line for the review task to quote:**
|
||||
|
||||
```
|
||||
NFR assessment: docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md
|
||||
```
|
||||
@@ -159,6 +168,7 @@ NFR assessment: docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md
|
||||
## Output 4: Gate Integration Line
|
||||
|
||||
**Always print at the end:**
|
||||
|
||||
```
|
||||
Gate NFR block ready → paste into docs/qa/gates/{epic}.{story}-{slug}.yml under nfr_validation
|
||||
```
|
||||
@@ -166,66 +176,82 @@ Gate NFR block ready → paste into docs/qa/gates/{epic}.{story}-{slug}.yml unde
|
||||
## Assessment Criteria
|
||||
|
||||
### Security
|
||||
|
||||
**PASS if:**
|
||||
|
||||
- Authentication implemented
|
||||
- Authorization enforced
|
||||
- Input validation present
|
||||
- No hardcoded secrets
|
||||
|
||||
**CONCERNS if:**
|
||||
|
||||
- Missing rate limiting
|
||||
- Weak encryption
|
||||
- Incomplete authorization
|
||||
|
||||
**FAIL if:**
|
||||
|
||||
- No authentication
|
||||
- Hardcoded credentials
|
||||
- SQL injection vulnerabilities
|
||||
|
||||
### Performance
|
||||
|
||||
**PASS if:**
|
||||
|
||||
- Meets response time targets
|
||||
- No obvious bottlenecks
|
||||
- Reasonable resource usage
|
||||
|
||||
**CONCERNS if:**
|
||||
|
||||
- Close to limits
|
||||
- Missing indexes
|
||||
- No caching strategy
|
||||
|
||||
**FAIL if:**
|
||||
|
||||
- Exceeds response time limits
|
||||
- Memory leaks
|
||||
- Unoptimized queries
|
||||
|
||||
### Reliability
|
||||
|
||||
**PASS if:**
|
||||
|
||||
- Error handling present
|
||||
- Graceful degradation
|
||||
- Retry logic where needed
|
||||
|
||||
**CONCERNS if:**
|
||||
|
||||
- Some error cases unhandled
|
||||
- No circuit breakers
|
||||
- Missing health checks
|
||||
|
||||
**FAIL if:**
|
||||
|
||||
- No error handling
|
||||
- Crashes on errors
|
||||
- No recovery mechanisms
|
||||
|
||||
### Maintainability
|
||||
|
||||
**PASS if:**
|
||||
|
||||
- Test coverage meets target
|
||||
- Code well-structured
|
||||
- Documentation present
|
||||
|
||||
**CONCERNS if:**
|
||||
|
||||
- Test coverage below target
|
||||
- Some code duplication
|
||||
- Missing documentation
|
||||
|
||||
**FAIL if:**
|
||||
|
||||
- No tests
|
||||
- Highly coupled code
|
||||
- No documentation
|
||||
@@ -283,7 +309,7 @@ maintainability:
|
||||
|
||||
1. **Functional Suitability**: Completeness, correctness, appropriateness
|
||||
2. **Performance Efficiency**: Time behavior, resource use, capacity
|
||||
3. **Compatibility**: Co-existence, interoperability
|
||||
3. **Compatibility**: Co-existence, interoperability
|
||||
4. **Usability**: Learnability, operability, accessibility
|
||||
5. **Reliability**: Maturity, availability, fault tolerance
|
||||
6. **Security**: Confidentiality, integrity, authenticity
|
||||
@@ -291,6 +317,7 @@ maintainability:
|
||||
8. **Portability**: Adaptability, installability
|
||||
|
||||
Use these when assessing beyond the core four.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
@@ -312,4 +339,5 @@ performance_deep_dive:
|
||||
max_rps: 150
|
||||
breaking_point: 200 rps
|
||||
```
|
||||
</details>
|
||||
|
||||
</details>
|
||||
|
||||
@@ -17,6 +17,7 @@ Generate a standalone quality gate file that provides a clear pass/fail decision
|
||||
**ALWAYS** create file at: `docs/qa/gates/{epic}.{story}-{slug}.yml`
|
||||
|
||||
Slug rules:
|
||||
|
||||
- Convert to lowercase
|
||||
- Replace spaces with hyphens
|
||||
- Strip punctuation
|
||||
@@ -31,8 +32,8 @@ gate: PASS|CONCERNS|FAIL|WAIVED
|
||||
status_reason: "1-2 sentence explanation of gate decision"
|
||||
reviewer: "Quinn"
|
||||
updated: "{ISO-8601 timestamp}"
|
||||
top_issues: [] # Empty array if no issues
|
||||
waiver: { active: false } # Only set active: true if WAIVED
|
||||
top_issues: [] # Empty array if no issues
|
||||
waiver: { active: false } # Only set active: true if WAIVED
|
||||
```
|
||||
|
||||
## Schema with Issues
|
||||
@@ -46,7 +47,7 @@ reviewer: "Quinn"
|
||||
updated: "2025-01-12T10:15:00Z"
|
||||
top_issues:
|
||||
- id: "SEC-001"
|
||||
severity: high # ONLY: low|medium|high
|
||||
severity: high # ONLY: low|medium|high
|
||||
finding: "No rate limiting on login endpoint"
|
||||
suggested_action: "Add rate limiting middleware before production"
|
||||
- id: "TEST-001"
|
||||
@@ -79,21 +80,25 @@ waiver:
|
||||
## Gate Decision Criteria
|
||||
|
||||
### PASS
|
||||
|
||||
- All acceptance criteria met
|
||||
- No high-severity issues
|
||||
- Test coverage meets project standards
|
||||
|
||||
### CONCERNS
|
||||
|
||||
- Non-blocking issues present
|
||||
- Should be tracked and scheduled
|
||||
- Can proceed with awareness
|
||||
|
||||
### FAIL
|
||||
|
||||
- Acceptance criteria not met
|
||||
- High-severity issues present
|
||||
- Recommend return to InProgress
|
||||
|
||||
### WAIVED
|
||||
|
||||
- Issues explicitly accepted
|
||||
- Requires approval and reason
|
||||
- Proceed despite known issues
|
||||
@@ -101,6 +106,7 @@ waiver:
|
||||
## Severity Scale
|
||||
|
||||
**FIXED VALUES - NO VARIATIONS:**
|
||||
|
||||
- `low`: Minor issues, cosmetic problems
|
||||
- `medium`: Should fix soon, not blocking
|
||||
- `high`: Critical issues, should block release
|
||||
@@ -134,11 +140,13 @@ After creating gate file, append to story's QA Results section:
|
||||
## QA Results
|
||||
|
||||
### Review Date: 2025-01-12
|
||||
|
||||
### Reviewed By: Quinn (Test Architect)
|
||||
|
||||
[... existing review content ...]
|
||||
|
||||
### Gate Status
|
||||
|
||||
Gate: CONCERNS → docs/qa/gates/1.3-user-auth-login.yml
|
||||
```
|
||||
|
||||
@@ -148,4 +156,4 @@ Gate: CONCERNS → docs/qa/gates/1.3-user-auth-login.yml
|
||||
- Fixed severity scale (low/medium/high)
|
||||
- Always write to standard path
|
||||
- Always update story with gate reference
|
||||
- Clear, actionable findings
|
||||
- Clear, actionable findings
|
||||
|
||||
@@ -6,10 +6,10 @@ Perform a comprehensive test architecture review with quality gate decision. Thi
|
||||
|
||||
```yaml
|
||||
required:
|
||||
- story_id: "{epic}.{story}" # e.g., "1.3"
|
||||
- story_id: "{epic}.{story}" # e.g., "1.3"
|
||||
- story_path: "docs/stories/{epic}.{story}.*.md"
|
||||
- story_title: "{title}" # If missing, derive from story file H1
|
||||
- story_slug: "{slug}" # If missing, derive from title (lowercase, hyphenated)
|
||||
- story_title: "{title}" # If missing, derive from story file H1
|
||||
- story_slug: "{slug}" # If missing, derive from title (lowercase, hyphenated)
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
@@ -23,6 +23,7 @@ required:
|
||||
### 1. Risk Assessment (Determines Review Depth)
|
||||
|
||||
**Auto-escalate to deep review when:**
|
||||
|
||||
- Auth/payment/security files touched
|
||||
- No tests added to story
|
||||
- Diff > 500 lines
|
||||
@@ -38,6 +39,7 @@ required:
|
||||
- Verify all requirements have corresponding test cases
|
||||
|
||||
**B. Code Quality Review**
|
||||
|
||||
- Architecture and design patterns
|
||||
- Refactoring opportunities (and perform them)
|
||||
- Code duplication or inefficiencies
|
||||
@@ -46,6 +48,7 @@ required:
|
||||
- Best practices adherence
|
||||
|
||||
**C. Test Architecture Assessment**
|
||||
|
||||
- Test coverage adequacy at appropriate levels
|
||||
- Test level appropriateness (what should be unit vs integration vs e2e)
|
||||
- Test design quality and maintainability
|
||||
@@ -55,17 +58,20 @@ required:
|
||||
- Test execution time and reliability
|
||||
|
||||
**D. Non-Functional Requirements (NFRs)**
|
||||
|
||||
- Security: Authentication, authorization, data protection
|
||||
- Performance: Response times, resource usage
|
||||
- Reliability: Error handling, recovery mechanisms
|
||||
- Maintainability: Code clarity, documentation
|
||||
|
||||
**E. Testability Evaluation**
|
||||
|
||||
- Controllability: Can we control the inputs?
|
||||
- Observability: Can we observe the outputs?
|
||||
- Debuggability: Can we debug failures easily?
|
||||
|
||||
**F. Technical Debt Identification**
|
||||
|
||||
- Accumulated shortcuts
|
||||
- Missing tests
|
||||
- Outdated dependencies
|
||||
@@ -103,6 +109,7 @@ required:
|
||||
**CRITICAL**: You are ONLY authorized to update the "QA Results" section of the story file. DO NOT modify any other sections.
|
||||
|
||||
**QA Results Anchor Rule:**
|
||||
|
||||
- If `## QA Results` doesn't exist, append it at end of file
|
||||
- If it exists, append a new dated entry below existing entries
|
||||
- Never edit other sections
|
||||
@@ -113,25 +120,31 @@ After review and any refactoring, append your results to the story file in the Q
|
||||
## QA Results
|
||||
|
||||
### Review Date: [Date]
|
||||
|
||||
### Reviewed By: Quinn (Test Architect)
|
||||
|
||||
### Code Quality Assessment
|
||||
|
||||
[Overall assessment of implementation quality]
|
||||
|
||||
### Refactoring Performed
|
||||
|
||||
[List any refactoring you performed with explanations]
|
||||
|
||||
- **File**: [filename]
|
||||
- **Change**: [what was changed]
|
||||
- **Why**: [reason for change]
|
||||
- **How**: [how it improves the code]
|
||||
|
||||
### Compliance Check
|
||||
|
||||
- Coding Standards: [✓/✗] [notes if any]
|
||||
- Project Structure: [✓/✗] [notes if any]
|
||||
- Testing Strategy: [✓/✗] [notes if any]
|
||||
- All ACs Met: [✓/✗] [notes if any]
|
||||
|
||||
### Improvements Checklist
|
||||
|
||||
[Check off items you handled yourself, leave unchecked for dev to address]
|
||||
|
||||
- [x] Refactored user service for better error handling (services/user.service.ts)
|
||||
@@ -141,20 +154,25 @@ After review and any refactoring, append your results to the story file in the Q
|
||||
- [ ] Update API documentation for new error codes
|
||||
|
||||
### Security Review
|
||||
|
||||
[Any security concerns found and whether addressed]
|
||||
|
||||
### Performance Considerations
|
||||
|
||||
[Any performance issues found and whether addressed]
|
||||
|
||||
### Files Modified During Review
|
||||
|
||||
[If you modified files, list them here - ask Dev to update File List]
|
||||
|
||||
### Gate Status
|
||||
|
||||
Gate: {STATUS} → docs/qa/gates/{epic}.{story}-{slug}.yml
|
||||
Risk profile: docs/qa/assessments/{epic}.{story}-risk-{YYYYMMDD}.md
|
||||
NFR assessment: docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md
|
||||
|
||||
### Recommended Status
|
||||
|
||||
[✓ Ready for Done] / [✗ Changes Required - See unchecked items above]
|
||||
(Story owner decides final status)
|
||||
```
|
||||
@@ -162,6 +180,7 @@ NFR assessment: docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md
|
||||
## Output 2: Create Quality Gate File
|
||||
|
||||
**Template and Directory:**
|
||||
|
||||
- Render from `templates/qa-gate-tmpl.yaml`
|
||||
- Create `docs/qa/gates/` directory if missing
|
||||
- Save to: `docs/qa/gates/{epic}.{story}-{slug}.yml`
|
||||
@@ -177,22 +196,22 @@ status_reason: "1-2 sentence explanation of gate decision"
|
||||
reviewer: "Quinn (Test Architect)"
|
||||
updated: "{ISO-8601 timestamp}"
|
||||
|
||||
top_issues: [] # Empty if no issues
|
||||
waiver: { active: false } # Set active: true only if WAIVED
|
||||
top_issues: [] # Empty if no issues
|
||||
waiver: { active: false } # Set active: true only if WAIVED
|
||||
|
||||
# Extended fields (optional but recommended):
|
||||
quality_score: 0-100 # 100 - (20*FAILs) - (10*CONCERNS) or use technical-preferences.md weights
|
||||
expires: "{ISO-8601 timestamp}" # Typically 2 weeks from review
|
||||
quality_score: 0-100 # 100 - (20*FAILs) - (10*CONCERNS) or use technical-preferences.md weights
|
||||
expires: "{ISO-8601 timestamp}" # Typically 2 weeks from review
|
||||
|
||||
evidence:
|
||||
tests_reviewed: {count}
|
||||
risks_identified: {count}
|
||||
tests_reviewed: { count }
|
||||
risks_identified: { count }
|
||||
trace:
|
||||
ac_covered: [1, 2, 3] # AC numbers with test coverage
|
||||
ac_gaps: [4] # AC numbers lacking coverage
|
||||
ac_covered: [1, 2, 3] # AC numbers with test coverage
|
||||
ac_gaps: [4] # AC numbers lacking coverage
|
||||
|
||||
nfr_validation:
|
||||
security:
|
||||
security:
|
||||
status: PASS|CONCERNS|FAIL
|
||||
notes: "Specific findings"
|
||||
performance:
|
||||
@@ -206,10 +225,10 @@ nfr_validation:
|
||||
notes: "Specific findings"
|
||||
|
||||
recommendations:
|
||||
immediate: # Must fix before production
|
||||
immediate: # Must fix before production
|
||||
- action: "Add rate limiting"
|
||||
refs: ["api/auth/login.ts"]
|
||||
future: # Can be addressed later
|
||||
future: # Can be addressed later
|
||||
- action: "Consider caching"
|
||||
refs: ["services/data.ts"]
|
||||
```
|
||||
@@ -240,6 +259,7 @@ If risk_summary exists, apply its thresholds first (≥9 → FAIL, ≥6 → CONC
|
||||
- WAIVED only when waiver.active: true with reason/approver
|
||||
|
||||
Detailed criteria:
|
||||
|
||||
- **PASS**: All critical requirements met, no blocking issues
|
||||
- **CONCERNS**: Non-critical issues found, team should review
|
||||
- **FAIL**: Critical issues that should be addressed
|
||||
|
||||
@@ -91,13 +91,11 @@ CRITICAL: Use proper parsing that understands markdown context. A ## inside a co
|
||||
For each extracted section:
|
||||
|
||||
1. **Generate filename**: Convert the section heading to lowercase-dash-case
|
||||
|
||||
- Remove special characters
|
||||
- Replace spaces with dashes
|
||||
- Example: "## Tech Stack" → `tech-stack.md`
|
||||
|
||||
2. **Adjust heading levels**:
|
||||
|
||||
- The level 2 heading becomes level 1 (# instead of ##) in the sharded new document
|
||||
- All subsection levels decrease by 1:
|
||||
|
||||
|
||||
@@ -93,7 +93,7 @@ trace:
|
||||
totals:
|
||||
requirements: X
|
||||
full: Y
|
||||
partial: Z
|
||||
partial: Z
|
||||
none: W
|
||||
planning_ref: "docs/qa/assessments/{epic}.{story}-test-design-{YYYYMMDD}.md"
|
||||
uncovered:
|
||||
|
||||
Reference in New Issue
Block a user