ros/BMAD-METHOD
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: addressed review comments from manjaroblack, round 1

Browse Source
This commit is contained in:
Murat Ozcan
2025-08-14 10:00:54 -05:00
parent 8d300dadf3
commit 147d444aeb
6 changed files with 488 additions and 541 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
bmad-core/tasks/nfr-assess.md
View File

@@ -6,9 +6,9 @@ Quick NFR validation focused on the core four: security, performance, reliabilit
```yaml
required:
- story_id: "{epic}.{story}" # e.g., "1.3"
- story_id: "{epic}.{story}" # e.g., "1.3"
- story_path: "docs/stories/{epic}.{story}.*.md"
optional:
- architecture_refs: "docs/architecture/*.md"
- technical_preferences: "docs/technical-preferences.md"
@@ -18,7 +18,6 @@ optional:
## Purpose
Assess non-functional requirements for a story and generate:
1. YAML block for the gate file's `nfr_validation` section
2. Brief markdown assessment saved to `docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md`
@@ -27,7 +26,6 @@ Assess non-functional requirements for a story and generate:
### 0. Fail-safe for Missing Inputs
If story_path or story file can't be found:
- Still create assessment file with note: "Source story not found"
- Set all selected NFRs to CONCERNS with notes: "Target unknown / evidence missing"
- Continue with assessment to provide value
@@ -40,7 +38,7 @@ If story_path or story file can't be found:
```text
Which NFRs should I assess? (Enter numbers or press Enter for default)
[1] Security (default)
[2] Performance (default)
[2] Performance (default)
[3] Reliability (default)
[4] Maintainability (default)
[5] Usability
@@ -54,7 +52,6 @@ Which NFRs should I assess? (Enter numbers or press Enter for default)
### 2. Check for Thresholds
Look for NFR requirements in:
- Story acceptance criteria
- `docs/architecture/*.md` files
- `docs/technical-preferences.md`
@@ -75,7 +72,6 @@ No security requirements found. Required auth method?
### 3. Quick Assessment
For each selected NFR, check:
- Is there evidence it's implemented?
- Can we validate it?
- Are there obvious gaps?
@@ -90,7 +86,7 @@ Generate ONLY for NFRs actually assessed (no placeholders):
# Gate YAML (copy/paste):
nfr_validation:
_assessed: [security, performance, reliability, maintainability]
security:
security:
status: CONCERNS
notes: "No rate limiting on auth endpoints"
performance:
@@ -107,7 +103,7 @@ nfr_validation:
## Deterministic Status Rules
- **FAIL**: Any selected NFR has critical gap or target clearly not met
- **CONCERNS**: No FAILs, but any NFR is unknown/partial/missing evidence
- **CONCERNS**: No FAILs, but any NFR is unknown/partial/missing evidence
- **PASS**: All selected NFRs meet targets with evidence
## Quality Score Calculation
@@ -127,21 +123,18 @@ If `technical-preferences.md` defines custom weights, use those instead.
```markdown
# NFR Assessment: {epic}.{story}
Date: {date}
Reviewer: Quinn
<!-- Note: Source story not found (if applicable) -->
## Summary
- Security: CONCERNS - Missing rate limiting
- Performance: PASS - Meets <200ms requirement
- Reliability: PASS - Proper error handling
- Maintainability: CONCERNS - Test coverage below target
## Critical Issues
1. **No rate limiting** (Security)
- Risk: Brute force attacks possible
- Fix: Add rate limiting middleware to auth endpoints
@@ -151,7 +144,6 @@ Reviewer: Quinn
- Fix: Add tests for uncovered branches
## Quick Wins
- Add rate limiting: ~2 hours
- Increase test coverage: ~4 hours
- Add performance monitoring: ~1 hour
@@ -160,7 +152,6 @@ Reviewer: Quinn
## Output 3: Story Update Line
**End with this line for the review task to quote:**
```
NFR assessment: docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md
```
@@ -168,7 +159,6 @@ NFR assessment: docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md
## Output 4: Gate Integration Line
**Always print at the end:**
```
Gate NFR block ready → paste into docs/qa/gates/{epic}.{story}-{slug}.yml under nfr_validation
```
@@ -176,82 +166,66 @@ Gate NFR block ready → paste into docs/qa/gates/{epic}.{story}-{slug}.yml unde
## Assessment Criteria
### Security
**PASS if:**
- Authentication implemented
- Authorization enforced
- Input validation present
- No hardcoded secrets
**CONCERNS if:**
- Missing rate limiting
- Weak encryption
- Incomplete authorization
**FAIL if:**
- No authentication
- Hardcoded credentials
- SQL injection vulnerabilities
### Performance
**PASS if:**
- Meets response time targets
- No obvious bottlenecks
- Reasonable resource usage
**CONCERNS if:**
- Close to limits
- Missing indexes
- No caching strategy
**FAIL if:**
- Exceeds response time limits
- Memory leaks
- Unoptimized queries
### Reliability
**PASS if:**
- Error handling present
- Graceful degradation
- Retry logic where needed
**CONCERNS if:**
- Some error cases unhandled
- No circuit breakers
- Missing health checks
**FAIL if:**
- No error handling
- Crashes on errors
- No recovery mechanisms
### Maintainability
**PASS if:**
- Test coverage meets target
- Code well-structured
- Documentation present
**CONCERNS if:**
- Test coverage below target
- Some code duplication
- Missing documentation
**FAIL if:**
- No tests
- Highly coupled code
- No documentation
@@ -309,7 +283,7 @@ maintainability:
1. **Functional Suitability**: Completeness, correctness, appropriateness
2. **Performance Efficiency**: Time behavior, resource use, capacity
3. **Compatibility**: Co-existence, interoperability
3. **Compatibility**: Co-existence, interoperability
4. **Usability**: Learnability, operability, accessibility
5. **Reliability**: Maturity, availability, fault tolerance
6. **Security**: Confidentiality, integrity, authenticity
@@ -317,7 +291,6 @@ maintainability:
8. **Portability**: Adaptability, installability
Use these when assessing beyond the core four.
</details>
<details>
@@ -339,5 +312,4 @@ performance_deep_dive:
max_rps: 150
breaking_point: 200 rps
```
</details>
</details>

6
bmad-core/tasks/review-story.md
View File

@@ -7,7 +7,7 @@ Perform a comprehensive test architecture review with quality gate decision. Thi
```yaml
required:
- story_id: "{epic}.{story}" # e.g., "1.3"
- story_path: "docs/stories/{epic}.{story}.*.md"
- story_path: "{devStoryLocation}/{epic}.{story}.*.md" # Path from core-config.yaml
- story_title: "{title}" # If missing, derive from story file H1
- story_slug: "{slug}" # If missing, derive from title (lowercase, hyphenated)
```
@@ -171,6 +171,8 @@ Gate: {STATUS} → docs/qa/gates/{epic}.{story}-{slug}.yml
Risk profile: docs/qa/assessments/{epic}.{story}-risk-{YYYYMMDD}.md
NFR assessment: docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md
# Note: Paths should reference core-config.yaml for custom configurations
### Recommended Status
[✓ Ready for Done] / [✗ Changes Required - See unchecked items above]
@@ -182,7 +184,7 @@ NFR assessment: docs/qa/assessments/{epic}.{story}-nfr-{YYYYMMDD}.md
**Template and Directory:**
- Render from `templates/qa-gate-tmpl.yaml`
- Create `docs/qa/gates/` directory if missing
- Create `docs/qa/gates/` directory if missing (or configure in core-config.yaml)
- Save to: `docs/qa/gates/{epic}.{story}-{slug}.yml`
Gate file structure:

538
bmad-core/tasks/test-design.md
View File

@@ -7,7 +7,7 @@ Create comprehensive test scenarios with appropriate test level recommendations
```yaml
required:
- story_id: "{epic}.{story}" # e.g., "1.3"
- story_path: "docs/stories/{epic}.{story}.*.md"
- story_path: "{devStoryLocation}/{epic}.{story}.*.md" # Path from core-config.yaml
- story_title: "{title}" # If missing, derive from story file H1
- story_slug: "{slug}" # If missing, derive from title (lowercase, hyphenated)
```
@@ -16,289 +16,69 @@ required:
Design a complete test strategy that identifies what to test, at which level (unit/integration/e2e), and why. This ensures efficient test coverage without redundancy while maintaining appropriate test boundaries.
## Test Level Decision Framework
### Unit Tests
**When to use:**
- Testing pure functions and business logic
- Algorithm correctness
- Input validation and data transformation
- Error handling in isolated components
- Complex calculations or state machines
**Characteristics:**
- Fast execution (immediate feedback)
- No external dependencies (DB, API, file system)
- Highly maintainable and stable
- Easy to debug failures
**Example scenarios:**
## Dependencies
```yaml
unit_test:
component: "PriceCalculator"
scenario: "Calculate discount with multiple rules"
justification: "Complex business logic with multiple branches"
mock_requirements: "None - pure function"
data:
- test-levels-framework.md # Unit/Integration/E2E decision criteria
- test-priorities-matrix.md # P0/P1/P2/P3 classification system
```
### Integration Tests
**When to use:**
- Testing component interactions
- Database operations and queries
- API endpoint behavior
- Service layer orchestration
- External service integration (with test doubles)
**Characteristics:**
- Moderate execution time
- May use test databases or containers
- Tests multiple components together
- Validates contracts between components
**Example scenarios:**
```yaml
integration_test:
components: ["UserService", "UserRepository", "Database"]
scenario: "Create user with duplicate email check"
justification: "Tests transaction boundaries and constraint handling"
test_doubles: "Mock email service, real test database"
```
### End-to-End Tests
**When to use:**
- Critical user journeys
- Cross-system workflows
- UI interaction flows
- Full stack validation
- Production-like scenario testing
**Characteristics:**
- Keep under 90 seconds per test
- Tests complete user scenarios
- Uses real or production-like environment
- Higher maintenance cost
- More prone to flakiness
**Example scenarios:**
```yaml
e2e_test:
flow: "Complete purchase flow"
scenario: "User browses, adds to cart, and completes checkout"
justification: "Critical business flow requiring full stack validation"
environment: "Staging with test payment gateway"
```
## Test Design Process
## Process
### 1. Analyze Story Requirements
Break down each acceptance criterion into testable scenarios:
Break down each acceptance criterion into testable scenarios. For each AC:
```yaml
acceptance_criterion: "User can reset password via email"
test_scenarios:
- level: unit
what: "Password validation rules"
why: "Complex regex and business rules"
- Identify the core functionality to test
- Determine data variations needed
- Consider error conditions
- Note edge cases
- level: integration
what: "Password reset token generation and storage"
why: "Database interaction with expiry logic"
### 2. Apply Test Level Framework
- level: integration
what: "Email service integration"
why: "External service with retry logic"
**Reference:** Load `test-levels-framework.md` for detailed criteria
- level: e2e
what: "Complete password reset flow"
why: "Critical security flow needing full validation"
```
Quick rules:
### 2. Apply Test Level Heuristics
- **Unit**: Pure logic, algorithms, calculations
- **Integration**: Component interactions, DB operations
- **E2E**: Critical user journeys, compliance
Use these rules to determine appropriate test levels:
### 3. Assign Priorities
```markdown
## Test Level Selection Rules
**Reference:** Load `test-priorities-matrix.md` for classification
### Favor Unit Tests When:
Quick priority assignment:
- Logic can be isolated
- No side effects involved
- Fast feedback needed
- High cyclomatic complexity
- **P0**: Revenue-critical, security, compliance
- **P1**: Core user journeys, frequently used
- **P2**: Secondary features, admin functions
- **P3**: Nice-to-have, rarely used
### Favor Integration Tests When:
### 4. Design Test Scenarios
- Testing persistence layer
- Validating service contracts
- Testing middleware/interceptors
- Component boundaries critical
### Favor E2E Tests When:
- User-facing critical paths
- Multi-system interactions
- Regulatory compliance scenarios
- Visual regression important
### Anti-patterns to Avoid:
- E2E testing for business logic validation
- Unit testing framework behavior
- Integration testing third-party libraries
- Duplicate coverage across levels
### Duplicate Coverage Guard
**Before adding any test, check:**
1. Is this already tested at a lower level?
2. Can a unit test cover this instead of integration?
3. Can an integration test cover this instead of E2E?
**Coverage overlap is only acceptable when:**
- Testing different aspects (unit: logic, integration: interaction, e2e: user experience)
- Critical paths requiring defense in depth
- Regression prevention for previously broken functionality
```
### 3. Design Test Scenarios
**Test ID Format:** `{EPIC}.{STORY}-{LEVEL}-{SEQ}`
- Example: `1.3-UNIT-001`, `1.3-INT-002`, `1.3-E2E-001`
- Ensures traceability across all artifacts
**Naming Convention:**
- Unit: `test_{component}_{scenario}`
- Integration: `test_{flow}_{interaction}`
- E2E: `test_{journey}_{outcome}`
**Risk Linkage:**
- Tag tests with risk IDs they mitigate
- Prioritize tests for high-risk areas (P0)
- Link to risk profile when available
For each identified test need:
For each identified test need, create:
```yaml
test_scenario:
id: "1.3-INT-002"
requirement: "AC2: Rate limiting on login attempts"
mitigates_risks: ["SEC-001", "PERF-003"] # Links to risk profile
priority: P0 # Based on risk score
unit_tests:
- name: "RateLimiter calculates window correctly"
input: "Timestamp array"
expected: "Correct window calculation"
integration_tests:
- name: "Login endpoint enforces rate limit"
setup: "5 failed attempts"
action: "6th attempt"
expected: "429 response with retry-after header"
e2e_tests:
- name: "User sees rate limit message"
setup: "Trigger rate limit"
validation: "Error message displayed, retry timer shown"
id: "{epic}.{story}-{LEVEL}-{SEQ}"
requirement: "AC reference"
priority: P0|P1|P2|P3
level: unit|integration|e2e
description: "What is being tested"
justification: "Why this level was chosen"
mitigates_risks: ["RISK-001"] # If risk profile exists
```
## Deterministic Test Level Minimums
### 5. Validate Coverage
**Per Acceptance Criterion:**
Ensure:
- At least 1 unit test for business logic
- At least 1 integration test if multiple components interact
- At least 1 E2E test if it's a user-facing feature
**Exceptions:**
- Pure UI changes: May skip unit tests
- Pure logic changes: May skip E2E tests
- Infrastructure changes: May focus on integration tests
**When in doubt:** Start with unit tests, add integration for interactions, E2E for critical paths only.
## Test Quality Standards
### Core Testing Principles
**No Flaky Tests:** Ensure reliability through proper async handling, explicit waits, and atomic test design.
**No Hard Waits/Sleeps:** Use dynamic waiting strategies (e.g., polling, event-based triggers).
**Stateless & Parallel-Safe:** Tests run independently; use cron jobs or semaphores only if unavoidable.
**No Order Dependency:** Every it/describe/context block works in isolation (supports .only execution).
**Self-Cleaning Tests:** Test sets up its own data and automatically deletes/deactivates entities created during testing.
**Tests Live Near Source Code:** Co-locate test files with the code they validate (e.g., `*.spec.js` alongside components).
### Execution Strategy
**Shifted Left:**
- Start with local environments or ephemeral stacks
- Validate functionality across all deployment stages (local → dev → stage)
**Low Maintenance:** Minimize manual upkeep (avoid brittle selectors, do not repeat UI actions, leverage APIs).
**CI Execution Evidence:** Integrate into pipelines with clear logs/artifacts.
**Visibility:** Generate test reports (e.g., JUnit XML, HTML) for failures and trends.
### Coverage Requirements
**Release Confidence:**
- Happy Path: Core user journeys are prioritized
- Edge Cases: Critical error/validation scenarios are covered
- Feature Flags: Test both enabled and disabled states where applicable
### Test Design Rules
**Assertions:** Keep them explicit in tests; avoid abstraction into helpers. Use parametrized tests for soft assertions.
**Naming:** Follow conventions (e.g., `describe('Component')`, `it('should do X when Y')`).
**Size:** Aim for files ≤200 lines; split/chunk large tests logically.
**Speed:** Target individual tests ≤90 seconds; optimize slow setups (e.g., shared fixtures).
**Careful Abstractions:** Favor readability over DRY when balancing helper reuse (page objects are okay, assertion logic is not).
**Test Cleanup:** Ensure tests clean up resources they create (e.g., closing browser, deleting test data).
**Deterministic Flow:** Tests should refrain from using conditionals (e.g., if/else) to control flow or try/catch blocks where possible.
### API Testing Standards
- Tests must not depend on hardcoded data → use factories and per-test setup
- Always test both happy path and negative/error cases
- API tests should run parallel safely (no global state shared)
- Test idempotency where applicable (e.g., duplicate requests)
- Tests should clean up their data
- Response logs should only be printed in case of failure
- Auth tests must validate token expiration and renewal
- Every AC has at least one test
- No duplicate coverage across levels
- Critical paths have multiple levels
- Risk mitigations are addressed
## Outputs
@@ -306,13 +86,11 @@ test_scenario:
**Save to:** `docs/qa/assessments/{epic}.{story}-test-design-{YYYYMMDD}.md`
Generate a comprehensive test design document:
```markdown
# Test Design: Story {epic}.{story}
Date: {date}
Reviewer: Quinn (Test Architect)
Designer: Quinn (Test Architect)
## Test Strategy Overview
@@ -320,209 +98,77 @@ Reviewer: Quinn (Test Architect)
- Unit tests: Y (A%)
- Integration tests: Z (B%)
- E2E tests: W (C%)
- Priority distribution: P0: X, P1: Y, P2: Z
## Test Level Rationale
## Test Scenarios by Acceptance Criteria
[Explain why this distribution was chosen]
### AC1: {description}
## Detailed Test Scenarios
#### Scenarios
### Requirement: AC1 - {description}
| ID | Level | Priority | Test | Justification |
| ------------ | ----------- | -------- | ------------------------- | ------------------------ |
| 1.3-UNIT-001 | Unit | P0 | Validate input format | Pure validation logic |
| 1.3-INT-001 | Integration | P0 | Service processes request | Multi-component flow |
| 1.3-E2E-001 | E2E | P1 | User completes journey | Critical path validation |
#### Unit Tests (3 scenarios)
[Continue for all ACs...]
1. **ID**: 1.3-UNIT-001
**Test**: Validate input format
- **Why Unit**: Pure validation logic
- **Coverage**: Input edge cases
- **Mocks**: None needed
- **Mitigates**: DATA-001 (if applicable)
## Risk Coverage
#### Integration Tests (2 scenarios)
[Map test scenarios to identified risks if risk profile exists]
1. **ID**: 1.3-INT-001
**Test**: Service processes valid request
- **Why Integration**: Multiple components involved
- **Coverage**: Happy path + error handling
- **Test Doubles**: Mock external API
- **Mitigates**: TECH-002
## Recommended Execution Order
#### E2E Tests (1 scenario)
1. **ID**: 1.3-E2E-001
**Test**: Complete user workflow
- **Why E2E**: Critical user journey
- **Coverage**: Full stack validation
- **Environment**: Staging
- **Max Duration**: 90 seconds
- **Mitigates**: BUS-001
[Continue for all requirements...]
## Test Data Requirements
### Unit Test Data
- Static fixtures for calculations
- Edge case values arrays
### Integration Test Data
- Test database seeds
- API response fixtures
### E2E Test Data
- Test user accounts
- Sandbox environment data
## Mock/Stub Strategy
### What to Mock
- External services (payment, email)
- Time-dependent functions
- Random number generators
### What NOT to Mock
- Core business logic
- Database in integration tests
- Critical security functions
## Test Execution Implementation
### Parallel Execution
- All unit tests: Fully parallel (stateless requirement)
- Integration tests: Parallel with isolated databases
- E2E tests: Sequential or limited parallelism
### Execution Order
1. Unit tests first (fail fast)
2. Integration tests second
3. E2E tests last (expensive, max 90 seconds each)
## Risk-Based Test Priority
### P0 - Must Have (Linked to Critical/High Risks)
- Security-related tests (SEC-\* risks)
- Data integrity tests (DATA-\* risks)
- Critical business flow tests (BUS-\* risks)
- Tests for risks scored ≥6 in risk profile
### P1 - Should Have (Medium Risks)
- Edge case coverage
- Performance tests (PERF-\* risks)
- Error recovery tests
- Tests for risks scored 4-5
### P2 - Nice to Have (Low Risks)
- UI polish tests
- Minor validation tests
- Tests for risks scored ≤3
## Test Maintenance Considerations
### High Maintenance Tests
[List tests that may need frequent updates]
### Stability Measures
- No retry strategies (tests must be deterministic)
- Dynamic waits only (no hard sleeps)
- Environment isolation
- Self-cleaning test data
## Coverage Goals
### Unit Test Coverage
- Target: 80% line coverage
- Focus: Business logic, calculations
### Integration Coverage
- Target: All API endpoints
- Focus: Contract validation
### E2E Coverage
- Target: Critical paths only
- Focus: User value delivery
1. P0 Unit tests (fail fast)
2. P0 Integration tests
3. P0 E2E tests
4. P1 tests in order
5. P2+ as time permits
```
## Test Level Smells to Flag
### Output 2: Gate YAML Block
### Over-testing Smells
Generate for inclusion in quality gate:
- Same logic tested at multiple levels
- E2E tests for calculations
- Integration tests for framework features
```yaml
test_design:
scenarios_total: X
by_level:
unit: Y
integration: Z
e2e: W
by_priority:
p0: A
p1: B
p2: C
coverage_gaps: [] # List any ACs without tests
```
### Under-testing Smells
### Output 3: Trace References
- No unit tests for complex logic
- Missing integration tests for data operations
- No E2E tests for critical user paths
Print for use by trace-requirements task:
### Wrong Level Smells
```text
Test design matrix: docs/qa/assessments/{epic}.{story}-test-design-{YYYYMMDD}.md
P0 tests identified: {count}
```
- Unit tests with real database
- E2E tests checking calculation results
- Integration tests mocking everything
## Quality Checklist
## Quality Indicators
Before finalizing, verify:
Good test design shows:
- Clear level separation
- No redundant coverage
- Fast feedback from unit tests
- Reliable integration tests
- Focused e2e tests
- [ ] Every AC has test coverage
- [ ] Test levels are appropriate (not over-testing)
- [ ] No duplicate coverage across levels
- [ ] Priorities align with business risk
- [ ] Test IDs follow naming convention
- [ ] Scenarios are atomic and independent
## Key Principles
- Test at the lowest appropriate level
- One clear owner per test
- Fast tests run first
- Mock at boundaries, not internals
- E2E for user value, not implementation
- Maintain test/production parity where critical
- Tests must be atomic and self-contained
- No shared state between tests
- Explicit assertions in test files (not helpers)
### Output 2: Story Hook Line
**Print this line for review task to quote:**
```text
Test design: docs/qa/assessments/{epic}.{story}-test-design-{YYYYMMDD}.md
```
**For traceability:** This planning document will be referenced by trace-requirements task.
### Output 3: Test Count Summary
**Print summary for quick reference:**
```yaml
test_summary:
total: { total_count }
by_level:
unit: { unit_count }
integration: { int_count }
e2e: { e2e_count }
by_priority:
P0: { p0_count }
P1: { p1_count }
P2: { p2_count }
coverage_gaps: [] # List any ACs without tests
```
- **Shift left**: Prefer unit over integration, integration over E2E
- **Risk-based**: Focus on what could go wrong
- **Efficient coverage**: Test once at the right level
- **Maintainability**: Consider long-term test maintenance
- **Fast feedback**: Quick tests run first