fix: addressed review comments from manjaroblack, round 1
This commit is contained in:
Murat Ozcan
146
bmad-core/data/test-levels-framework.md
Normal file
146
bmad-core/data/test-levels-framework.md
Normal file
@@ -0,0 +1,146 @@
|
||||
# Test Levels Framework
|
||||
|
||||
Comprehensive guide for determining appropriate test levels (unit, integration, E2E) for different scenarios.
|
||||
|
||||
## Test Level Decision Matrix
|
||||
|
||||
### Unit Tests
|
||||
|
||||
**When to use:**
|
||||
|
||||
- Testing pure functions and business logic
|
||||
- Algorithm correctness
|
||||
- Input validation and data transformation
|
||||
- Error handling in isolated components
|
||||
- Complex calculations or state machines
|
||||
|
||||
**Characteristics:**
|
||||
|
||||
- Fast execution (immediate feedback)
|
||||
- No external dependencies (DB, API, file system)
|
||||
- Highly maintainable and stable
|
||||
- Easy to debug failures
|
||||
|
||||
**Example scenarios:**
|
||||
|
||||
```yaml
|
||||
unit_test:
|
||||
component: "PriceCalculator"
|
||||
scenario: "Calculate discount with multiple rules"
|
||||
justification: "Complex business logic with multiple branches"
|
||||
mock_requirements: "None - pure function"
|
||||
```
|
||||
|
||||
### Integration Tests
|
||||
|
||||
**When to use:**
|
||||
|
||||
- Component interaction verification
|
||||
- Database operations and transactions
|
||||
- API endpoint contracts
|
||||
- Service-to-service communication
|
||||
- Middleware and interceptor behavior
|
||||
|
||||
**Characteristics:**
|
||||
|
||||
- Moderate execution time
|
||||
- Tests component boundaries
|
||||
- May use test databases or containers
|
||||
- Validates system integration points
|
||||
|
||||
**Example scenarios:**
|
||||
|
||||
```yaml
|
||||
integration_test:
|
||||
components: ["UserService", "AuthRepository"]
|
||||
scenario: "Create user with role assignment"
|
||||
justification: "Critical data flow between service and persistence"
|
||||
test_environment: "In-memory database"
|
||||
```
|
||||
|
||||
### End-to-End Tests
|
||||
|
||||
**When to use:**
|
||||
|
||||
- Critical user journeys
|
||||
- Cross-system workflows
|
||||
- Visual regression testing
|
||||
- Compliance and regulatory requirements
|
||||
- Final validation before release
|
||||
|
||||
**Characteristics:**
|
||||
|
||||
- Slower execution
|
||||
- Tests complete workflows
|
||||
- Requires full environment setup
|
||||
- Most realistic but most brittle
|
||||
|
||||
**Example scenarios:**
|
||||
|
||||
```yaml
|
||||
e2e_test:
|
||||
journey: "Complete checkout process"
|
||||
scenario: "User purchases with saved payment method"
|
||||
justification: "Revenue-critical path requiring full validation"
|
||||
environment: "Staging with test payment gateway"
|
||||
```
|
||||
|
||||
## Test Level Selection Rules
|
||||
|
||||
### Favor Unit Tests When:
|
||||
|
||||
- Logic can be isolated
|
||||
- No side effects involved
|
||||
- Fast feedback needed
|
||||
- High cyclomatic complexity
|
||||
|
||||
### Favor Integration Tests When:
|
||||
|
||||
- Testing persistence layer
|
||||
- Validating service contracts
|
||||
- Testing middleware/interceptors
|
||||
- Component boundaries critical
|
||||
|
||||
### Favor E2E Tests When:
|
||||
|
||||
- User-facing critical paths
|
||||
- Multi-system interactions
|
||||
- Regulatory compliance scenarios
|
||||
- Visual regression important
|
||||
|
||||
## Anti-patterns to Avoid
|
||||
|
||||
- E2E testing for business logic validation
|
||||
- Unit testing framework behavior
|
||||
- Integration testing third-party libraries
|
||||
- Duplicate coverage across levels
|
||||
|
||||
## Duplicate Coverage Guard
|
||||
|
||||
**Before adding any test, check:**
|
||||
|
||||
1. Is this already tested at a lower level?
|
||||
2. Can a unit test cover this instead of integration?
|
||||
3. Can an integration test cover this instead of E2E?
|
||||
|
||||
**Coverage overlap is only acceptable when:**
|
||||
|
||||
- Testing different aspects (unit: logic, integration: interaction, e2e: user experience)
|
||||
- Critical paths requiring defense in depth
|
||||
- Regression prevention for previously broken functionality
|
||||
|
||||
## Test Naming Conventions
|
||||
|
||||
- Unit: `test_{component}_{scenario}`
|
||||
- Integration: `test_{flow}_{interaction}`
|
||||
- E2E: `test_{journey}_{outcome}`
|
||||
|
||||
## Test ID Format
|
||||
|
||||
`{EPIC}.{STORY}-{LEVEL}-{SEQ}`
|
||||
|
||||
Examples:
|
||||
|
||||
- `1.3-UNIT-001`
|
||||
- `1.3-INT-002`
|
||||
- `1.3-E2E-001`
|
||||
172
bmad-core/data/test-priorities-matrix.md
Normal file
172
bmad-core/data/test-priorities-matrix.md
Normal file
@@ -0,0 +1,172 @@
|
||||
# Test Priorities Matrix
|
||||
|
||||
Guide for prioritizing test scenarios based on risk, criticality, and business impact.
|
||||
|
||||
## Priority Levels
|
||||
|
||||
### P0 - Critical (Must Test)
|
||||
|
||||
**Criteria:**
|
||||
|
||||
- Revenue-impacting functionality
|
||||
- Security-critical paths
|
||||
- Data integrity operations
|
||||
- Regulatory compliance requirements
|
||||
- Previously broken functionality (regression prevention)
|
||||
|
||||
**Examples:**
|
||||
|
||||
- Payment processing
|
||||
- Authentication/authorization
|
||||
- User data creation/deletion
|
||||
- Financial calculations
|
||||
- GDPR/privacy compliance
|
||||
|
||||
**Testing Requirements:**
|
||||
|
||||
- Comprehensive coverage at all levels
|
||||
- Both happy and unhappy paths
|
||||
- Edge cases and error scenarios
|
||||
- Performance under load
|
||||
|
||||
### P1 - High (Should Test)
|
||||
|
||||
**Criteria:**
|
||||
|
||||
- Core user journeys
|
||||
- Frequently used features
|
||||
- Features with complex logic
|
||||
- Integration points between systems
|
||||
- Features affecting user experience
|
||||
|
||||
**Examples:**
|
||||
|
||||
- User registration flow
|
||||
- Search functionality
|
||||
- Data import/export
|
||||
- Notification systems
|
||||
- Dashboard displays
|
||||
|
||||
**Testing Requirements:**
|
||||
|
||||
- Primary happy paths required
|
||||
- Key error scenarios
|
||||
- Critical edge cases
|
||||
- Basic performance validation
|
||||
|
||||
### P2 - Medium (Nice to Test)
|
||||
|
||||
**Criteria:**
|
||||
|
||||
- Secondary features
|
||||
- Admin functionality
|
||||
- Reporting features
|
||||
- Configuration options
|
||||
- UI polish and aesthetics
|
||||
|
||||
**Examples:**
|
||||
|
||||
- Admin settings panels
|
||||
- Report generation
|
||||
- Theme customization
|
||||
- Help documentation
|
||||
- Analytics tracking
|
||||
|
||||
**Testing Requirements:**
|
||||
|
||||
- Happy path coverage
|
||||
- Basic error handling
|
||||
- Can defer edge cases
|
||||
|
||||
### P3 - Low (Test if Time Permits)
|
||||
|
||||
**Criteria:**
|
||||
|
||||
- Rarely used features
|
||||
- Nice-to-have functionality
|
||||
- Cosmetic issues
|
||||
- Non-critical optimizations
|
||||
|
||||
**Examples:**
|
||||
|
||||
- Advanced preferences
|
||||
- Legacy feature support
|
||||
- Experimental features
|
||||
- Debug utilities
|
||||
|
||||
**Testing Requirements:**
|
||||
|
||||
- Smoke tests only
|
||||
- Can rely on manual testing
|
||||
- Document known limitations
|
||||
|
||||
## Risk-Based Priority Adjustments
|
||||
|
||||
### Increase Priority When:
|
||||
|
||||
- High user impact (affects >50% of users)
|
||||
- High financial impact (>$10K potential loss)
|
||||
- Security vulnerability potential
|
||||
- Compliance/legal requirements
|
||||
- Customer-reported issues
|
||||
- Complex implementation (>500 LOC)
|
||||
- Multiple system dependencies
|
||||
|
||||
### Decrease Priority When:
|
||||
|
||||
- Feature flag protected
|
||||
- Gradual rollout planned
|
||||
- Strong monitoring in place
|
||||
- Easy rollback capability
|
||||
- Low usage metrics
|
||||
- Simple implementation
|
||||
- Well-isolated component
|
||||
|
||||
## Test Coverage by Priority
|
||||
|
||||
| Priority | Unit Coverage | Integration Coverage | E2E Coverage |
|
||||
| -------- | ------------- | -------------------- | ------------------ |
|
||||
| P0 | >90% | >80% | All critical paths |
|
||||
| P1 | >80% | >60% | Main happy paths |
|
||||
| P2 | >60% | >40% | Smoke tests |
|
||||
| P3 | Best effort | Best effort | Manual only |
|
||||
|
||||
## Priority Assignment Rules
|
||||
|
||||
1. **Start with business impact** - What happens if this fails?
|
||||
2. **Consider probability** - How likely is failure?
|
||||
3. **Factor in detectability** - Would we know if it failed?
|
||||
4. **Account for recoverability** - Can we fix it quickly?
|
||||
|
||||
## Priority Decision Tree
|
||||
|
||||
```
|
||||
Is it revenue-critical?
|
||||
├─ YES → P0
|
||||
└─ NO → Does it affect core user journey?
|
||||
├─ YES → Is it high-risk?
|
||||
│ ├─ YES → P0
|
||||
│ └─ NO → P1
|
||||
└─ NO → Is it frequently used?
|
||||
├─ YES → P1
|
||||
└─ NO → Is it customer-facing?
|
||||
├─ YES → P2
|
||||
└─ NO → P3
|
||||
```
|
||||
|
||||
## Test Execution Order
|
||||
|
||||
1. Execute P0 tests first (fail fast on critical issues)
|
||||
2. Execute P1 tests second (core functionality)
|
||||
3. Execute P2 tests if time permits
|
||||
4. P3 tests only in full regression cycles
|
||||
|
||||
## Continuous Adjustment
|
||||
|
||||
Review and adjust priorities based on:
|
||||
|
||||
- Production incident patterns
|
||||
- User feedback and complaints
|
||||
- Usage analytics
|
||||
- Test failure history
|
||||
- Business priority changes
|
||||
Reference in New Issue
Block a user