Files
claude-task-master/tasks/task_084.txt

640 lines
23 KiB
Plaintext

# Task ID: 84
# Title: Enhance Parse-PRD with Intelligent Task Expansion and Detail Preservation
# Status: pending
# Dependencies: None
# Priority: high
# Description: Transform parse-prd from a simple task generator into an intelligent system that preserves PRD detail resolution through context-aware task expansion. This addresses the critical issue where highly detailed PRDs lose their specificity when parsed into too few top-level tasks, and ensures that task expansions are grounded in actual PRD content rather than generic AI assumptions.
# Details:
## Core Problem Statement
The current parse-prd implementation suffers from a fundamental resolution loss problem:
1. **Detail Compression**: Complex, detailed PRDs get compressed into a fixed number of top-level tasks (default 10), losing critical specificity
2. **Orphaned Expansions**: When tasks are later expanded via expand-task, the AI lacks the original PRD context, resulting in generic subtasks that don't reflect the PRD's specific requirements
3. **Binary Approach**: The system either creates too few high-level tasks OR requires manual expansion that loses PRD context
## Solution Architecture
### Phase 1: Enhanced PRD Analysis Engine
- Implement intelligent PRD segmentation that identifies natural task boundaries based on content structure
- Create a PRD context preservation system that maintains detailed mappings between PRD sections and generated tasks
- Develop adaptive task count determination based on PRD complexity metrics (length, technical depth, feature count)
### Phase 2: Context-Aware Task Generation
- Modify generateTasksFromPRD to create tasks with embedded PRD context references
- Implement a PRD section mapping system that links each task to its source PRD content
- Add metadata fields to tasks that preserve original PRD language and specifications
### Phase 3: Intelligent In-Flight Expansion
- Add optional `--expand-tasks` flag to parse-prd that triggers immediate expansion after initial task generation
- Implement context-aware expansion that uses the original PRD content for each task's expansion
- Create a two-pass system: first pass generates tasks with PRD context, second pass expands using that context
### Phase 4: PRD-Grounded Expansion Logic
- Enhance the expansion prompt generation to include relevant PRD excerpts for each task being expanded
- Implement smart context windowing that includes related PRD sections when expanding tasks
- Add validation to ensure expanded subtasks maintain fidelity to original PRD specifications
## Technical Implementation Details
### File Modifications Required:
1. **scripts/modules/task-manager/parse-prd.js**
- Add PRD analysis functions for intelligent segmentation
- Implement context preservation during task generation
- Add optional expansion pipeline integration
- Create PRD-to-task mapping system
2. **scripts/modules/task-manager/expand-task.js**
- Enhance to accept PRD context as additional input
- Modify expansion prompts to include relevant PRD excerpts
- Add PRD-grounded validation for generated subtasks
3. **scripts/modules/ai-services-unified.js**
- Add support for context-aware prompting with PRD excerpts
- Implement intelligent context windowing for large PRDs
- Add PRD analysis capabilities for complexity assessment
### New Data Structures:
```javascript
// Enhanced task structure with PRD context
{
id: "1",
title: "User Authentication System",
description: "...",
prdContext: {
sourceSection: "Authentication Requirements (Lines 45-78)",
originalText: "The system must implement OAuth 2.0...",
relatedSections: ["Security Requirements", "User Management"],
contextWindow: "Full PRD excerpt relevant to this task"
},
// ... existing fields
}
// PRD analysis metadata
{
prdAnalysis: {
totalComplexity: 8.5,
naturalTaskBoundaries: [...],
recommendedTaskCount: 15,
sectionMappings: {...}
}
}
```
### New CLI Options:
- `--expand-tasks`: Automatically expand generated tasks using PRD context
- `--preserve-detail`: Maximum detail preservation mode
- `--adaptive-count`: Let AI determine optimal task count based on PRD complexity
- `--context-window-size`: Control how much PRD context to include in expansions
## Implementation Strategy
### Step 1: PRD Analysis Enhancement
- Create PRD parsing utilities that identify natural section boundaries
- Implement complexity scoring for different PRD sections
- Build context extraction functions that preserve relevant details
### Step 2: Context-Aware Task Generation
- Modify the task generation prompt to include section-specific context
- Implement task-to-PRD mapping during generation
- Add metadata fields to preserve PRD relationships
### Step 3: Intelligent Expansion Pipeline
- Create expansion logic that uses preserved PRD context
- Implement smart prompt engineering that includes relevant PRD excerpts
- Add validation to ensure subtask fidelity to original requirements
### Step 4: Integration and Testing
- Integrate new functionality with existing parse-prd workflow
- Add comprehensive testing with various PRD types and complexities
- Implement telemetry for tracking detail preservation effectiveness
## Success Metrics
- PRD detail preservation rate (measured by semantic similarity between PRD and generated tasks)
- Reduction in manual task refinement needed post-parsing
- Improved accuracy of expanded subtasks compared to PRD specifications
- User satisfaction with task granularity and detail accuracy
## Edge Cases and Considerations
- Very large PRDs that exceed context windows
- PRDs with conflicting or ambiguous requirements
- Integration with existing task expansion workflows
- Performance impact of enhanced analysis
- Backward compatibility with existing parse-prd usage
# Test Strategy:
# Subtasks:
## 1. Implement PRD Analysis and Segmentation Engine [pending]
### Dependencies: None
### Description: Create intelligent PRD parsing that identifies natural task boundaries and complexity metrics
### Details:
## Implementation Requirements
### Core Functions to Implement:
1. **analyzePRDStructure(prdContent)**
- Parse PRD into logical sections using headers, bullet points, and semantic breaks
- Identify feature boundaries, technical requirements, and implementation sections
- Return structured analysis with section metadata
2. **calculatePRDComplexity(prdContent)**
- Analyze technical depth, feature count, integration requirements
- Score complexity on 1-10 scale for different aspects
- Return recommended task count based on complexity
3. **extractTaskBoundaries(prdAnalysis)**
- Identify natural breaking points for task creation
- Group related requirements into logical task units
- Preserve context relationships between sections
### Technical Approach:
- Use regex patterns and NLP techniques to identify section headers
- Implement keyword analysis for technical complexity assessment
- Create semantic grouping algorithms for related requirements
- Build context preservation mappings
### Output Structure:
```javascript
{
sections: [
{
title: "User Authentication",
content: "...",
startLine: 45,
endLine: 78,
complexity: 7,
relatedSections: ["Security", "User Management"]
}
],
overallComplexity: 8.5,
recommendedTaskCount: 15,
naturalBoundaries: [...],
contextMappings: {...}
}
```
### Integration Points:
- Called at the beginning of parse-prd process
- Results used to inform task generation strategy
- Analysis stored for later use in expansion phase
## 2. Enhance Task Generation with PRD Context Preservation [pending]
### Dependencies: None
### Description: Modify generateTasksFromPRD to embed PRD context and maintain source mappings
### Details:
## Implementation Requirements
### Core Modifications to generateTasksFromPRD:
1. **Add PRD Context Embedding**
- Modify task generation prompt to include relevant PRD excerpts
- Ensure each generated task includes source section references
- Preserve original PRD language and specifications in task metadata
2. **Implement Context Windowing**
- For large PRDs, implement intelligent context windowing
- Include relevant sections for each task being generated
- Maintain context relationships between related tasks
3. **Enhanced Task Structure**
- Add prdContext field to task objects
- Include sourceSection, originalText, and relatedSections
- Store contextWindow for later use in expansions
### Technical Implementation:
```javascript
// Enhanced task generation with context
const generateTaskWithContext = async (prdSection, relatedSections, fullPRD) => {
const contextWindow = buildContextWindow(prdSection, relatedSections, fullPRD);
const prompt = `
Generate a task based on this PRD section:
PRIMARY SECTION:
${prdSection.content}
RELATED CONTEXT:
${contextWindow}
Ensure the task preserves all specific requirements and technical details.
`;
// Generate task with embedded context
const task = await generateTask(prompt);
task.prdContext = {
sourceSection: prdSection.title,
originalText: prdSection.content,
relatedSections: relatedSections.map(s => s.title),
contextWindow: contextWindow
};
return task;
};
```
### Context Preservation Strategy:
- Map each task to its source PRD sections
- Preserve technical specifications and requirements language
- Maintain relationships between interdependent features
- Store context for later use in expansion phase
### Integration with Existing Flow:
- Modify existing generateTasksFromPRD function
- Maintain backward compatibility with simple PRDs
- Add new metadata fields without breaking existing structure
- Ensure context is available for subsequent operations
## 3. Implement In-Flight Task Expansion Pipeline [pending]
### Dependencies: None
### Description: Add optional --expand-tasks flag and intelligent expansion using preserved PRD context
### Details:
## Implementation Requirements
### Core Features:
1. **Add --expand-tasks CLI Flag**
- Optional flag for parse-prd command
- Triggers automatic expansion after initial task generation
- Configurable expansion depth and strategy
2. **Two-Pass Processing System**
- First pass: Generate tasks with PRD context preservation
- Second pass: Expand tasks using their embedded PRD context
- Maintain context fidelity throughout the process
3. **Context-Aware Expansion Logic**
- Use preserved PRD context for each task's expansion
- Include relevant PRD excerpts in expansion prompts
- Ensure subtasks maintain fidelity to original specifications
### Technical Implementation:
```javascript
// Enhanced parse-prd with expansion pipeline
const parsePRDWithExpansion = async (prdContent, options) => {
// Phase 1: Analyze and generate tasks with context
const prdAnalysis = await analyzePRDStructure(prdContent);
const tasksWithContext = await generateTasksWithContext(prdAnalysis);
// Phase 2: Expand tasks if requested
if (options.expandTasks) {
for (const task of tasksWithContext) {
if (shouldExpandTask(task, prdAnalysis)) {
const expandedSubtasks = await expandTaskWithPRDContext(task);
task.subtasks = expandedSubtasks;
}
}
}
return tasksWithContext;
};
// Context-aware task expansion
const expandTaskWithPRDContext = async (task) => {
const { prdContext } = task;
const expansionPrompt = `
Expand this task into detailed subtasks using the original PRD context:
TASK: ${task.title}
DESCRIPTION: ${task.description}
ORIGINAL PRD CONTEXT:
${prdContext.originalText}
RELATED SECTIONS:
${prdContext.contextWindow}
Generate subtasks that preserve all technical details and requirements from the PRD.
`;
return await generateSubtasks(expansionPrompt);
};
```
### CLI Integration:
- Add --expand-tasks flag to parse-prd command
- Add --expansion-depth option for controlling subtask levels
- Add --preserve-detail flag for maximum context preservation
- Maintain backward compatibility with existing parse-prd usage
### Expansion Strategy:
- Determine which tasks should be expanded based on complexity
- Use PRD context to generate accurate, detailed subtasks
- Preserve technical specifications and implementation details
- Validate subtask accuracy against original PRD content
### Performance Considerations:
- Implement batching for large numbers of tasks
- Add progress indicators for long-running expansions
- Optimize context window sizes for efficiency
- Cache PRD analysis results for reuse
## 4. Enhance Expand-Task with PRD Context Integration [pending]
### Dependencies: None
### Description: Modify existing expand-task functionality to leverage preserved PRD context for more accurate expansions
### Details:
## Implementation Requirements
### Core Enhancements to expand-task.js:
1. **PRD Context Detection**
- Check if task has embedded prdContext metadata
- Extract relevant PRD sections for expansion
- Fall back to existing expansion logic if no PRD context
2. **Context-Enhanced Expansion Prompts**
- Include original PRD excerpts in expansion prompts
- Add related section context for comprehensive understanding
- Preserve technical specifications and requirements language
3. **Validation and Quality Assurance**
- Validate generated subtasks against original PRD content
- Ensure technical accuracy and requirement compliance
- Flag potential discrepancies for review
### Technical Implementation:
```javascript
// Enhanced expand-task with PRD context
const expandTaskWithContext = async (taskId, options, context) => {
const task = await getTask(taskId);
// Check for PRD context
if (task.prdContext) {
return await expandWithPRDContext(task, options);
} else {
// Fall back to existing expansion logic
return await expandTaskStandard(task, options);
}
};
const expandWithPRDContext = async (task, options) => {
const { prdContext } = task;
const enhancedPrompt = `
Expand this task into detailed subtasks using the original PRD context:
TASK DETAILS:
Title: ${task.title}
Description: ${task.description}
Current Details: ${task.details}
ORIGINAL PRD CONTEXT:
Source Section: ${prdContext.sourceSection}
Original Requirements:
${prdContext.originalText}
RELATED CONTEXT:
${prdContext.contextWindow}
EXPANSION REQUIREMENTS:
- Preserve all technical specifications from the PRD
- Maintain requirement accuracy and completeness
- Generate ${options.num || 'appropriate number of'} subtasks
- Include implementation details that reflect PRD specifics
Generate subtasks that are grounded in the original PRD content.
`;
const subtasks = await generateSubtasks(enhancedPrompt, options);
// Add PRD context inheritance to subtasks
subtasks.forEach(subtask => {
subtask.prdContext = {
inheritedFrom: task.id,
sourceSection: prdContext.sourceSection,
relevantExcerpt: extractRelevantExcerpt(prdContext, subtask)
};
});
return subtasks;
};
```
### Integration Points:
1. **Modify existing expand-task.js**
- Add PRD context detection logic
- Enhance prompt generation with context
- Maintain backward compatibility
2. **Update expansion validation**
- Add PRD compliance checking
- Implement quality scoring for context fidelity
- Flag potential accuracy issues
3. **CLI and MCP Integration**
- Update expand-task command to leverage PRD context
- Add options for context-aware expansion
- Maintain existing command interface
### Context Inheritance Strategy:
- Pass relevant PRD context to generated subtasks
- Create context inheritance chain for nested expansions
- Preserve source traceability throughout expansion tree
- Enable future re-expansion with maintained context
### Quality Assurance Features:
- Semantic similarity checking between subtasks and PRD
- Technical requirement compliance validation
- Automated flagging of potential context drift
- User feedback integration for continuous improvement
## 5. Add New CLI Options and MCP Parameters [pending]
### Dependencies: None
### Description: Implement new command-line flags and MCP tool parameters for enhanced PRD parsing
### Details:
## Implementation Requirements
### New CLI Options for parse-prd:
1. **--expand-tasks**
- Automatically expand generated tasks using PRD context
- Boolean flag, default false
- Triggers in-flight expansion pipeline
2. **--preserve-detail**
- Maximum detail preservation mode
- Boolean flag, default false
- Ensures highest fidelity to PRD content
3. **--adaptive-count**
- Let AI determine optimal task count based on PRD complexity
- Boolean flag, default false
- Overrides --num-tasks when enabled
4. **--context-window-size**
- Control how much PRD context to include in expansions
- Integer value, default 2000 characters
- Balances context richness with performance
5. **--expansion-depth**
- Control how many levels deep to expand tasks
- Integer value, default 1
- Prevents excessive nesting
### MCP Tool Parameter Updates:
```javascript
// Enhanced parse_prd MCP tool parameters
{
input: "Path to PRD file",
output: "Output path for tasks.json",
numTasks: "Number of top-level tasks (overridden by adaptiveCount)",
expandTasks: "Boolean - automatically expand tasks with PRD context",
preserveDetail: "Boolean - maximum detail preservation mode",
adaptiveCount: "Boolean - AI determines optimal task count",
contextWindowSize: "Integer - context size for expansions",
expansionDepth: "Integer - levels of expansion to perform",
research: "Boolean - use research model for enhanced analysis",
force: "Boolean - overwrite existing files"
}
```
### CLI Command Updates:
```bash
# Enhanced parse-prd command examples
task-master parse-prd prd.txt --expand-tasks --preserve-detail
task-master parse-prd prd.txt --adaptive-count --expansion-depth=2
task-master parse-prd prd.txt --context-window-size=3000 --research
```
### Implementation Details:
1. **Update commands.js**
- Add new option definitions
- Update parse-prd command handler
- Maintain backward compatibility
2. **Update MCP tool definition**
- Add new parameter schemas
- Update tool description and examples
- Ensure parameter validation
3. **Parameter Processing Logic**
- Validate parameter combinations
- Set appropriate defaults
- Handle conflicting options gracefully
### Validation Rules:
- expansion-depth must be positive integer ≤ 3
- context-window-size must be between 500-5000 characters
- adaptive-count overrides num-tasks when both specified
- expand-tasks requires either adaptive-count or num-tasks > 5
### Help Documentation Updates:
- Update command help text with new options
- Add usage examples for different scenarios
- Document parameter interactions and constraints
- Include performance considerations for large PRDs
## 6. Implement Comprehensive Testing and Validation [pending]
### Dependencies: None
### Description: Create test suite for PRD analysis, context preservation, and expansion accuracy
### Details:
## Implementation Requirements
### Test Categories:
1. **PRD Analysis Testing**
- Test section identification with various PRD formats
- Validate complexity scoring accuracy
- Test boundary detection for different document structures
- Verify context mapping correctness
2. **Context Preservation Testing**
- Validate PRD context embedding in generated tasks
- Test context window generation and sizing
- Verify source section mapping accuracy
- Test context inheritance in subtasks
3. **Expansion Accuracy Testing**
- Compare PRD-grounded vs standard expansions
- Measure semantic similarity between PRD and subtasks
- Test technical requirement preservation
- Validate expansion depth and quality
4. **Integration Testing**
- Test full parse-prd pipeline with expansion
- Validate CLI option combinations
- Test MCP tool parameter handling
- Verify backward compatibility
### Test Data Requirements:
```javascript
// Test PRD samples
const testPRDs = {
simple: "Basic PRD with minimal technical details",
complex: "Detailed PRD with extensive technical specifications",
structured: "Well-organized PRD with clear sections",
unstructured: "Free-form PRD with mixed content",
technical: "Highly technical PRD with specific requirements",
large: "Very large PRD testing context window limits"
};
```
### Validation Metrics:
1. **Detail Preservation Score**
- Semantic similarity between PRD and generated tasks
- Technical requirement coverage percentage
- Specification accuracy rating
2. **Context Fidelity Score**
- Accuracy of source section mapping
- Relevance of included context windows
- Quality of context inheritance
3. **Expansion Quality Score**
- Subtask relevance to parent task and PRD
- Technical accuracy of implementation details
- Completeness of requirement coverage
### Test Implementation:
```javascript
// Example test structure
describe('Enhanced Parse-PRD', () => {
describe('PRD Analysis', () => {
test('should identify sections correctly', async () => {
const analysis = await analyzePRDStructure(testPRDs.structured);
expect(analysis.sections).toHaveLength(expectedSectionCount);
expect(analysis.overallComplexity).toBeGreaterThan(0);
});
test('should calculate appropriate task count', async () => {
const analysis = await analyzePRDStructure(testPRDs.complex);
expect(analysis.recommendedTaskCount).toBeGreaterThan(10);
});
});
describe('Context Preservation', () => {
test('should embed PRD context in tasks', async () => {
const tasks = await generateTasksWithContext(testPRDs.technical);
tasks.forEach(task => {
expect(task.prdContext).toBeDefined();
expect(task.prdContext.sourceSection).toBeTruthy();
expect(task.prdContext.originalText).toBeTruthy();
});
});
});
describe('Expansion Accuracy', () => {
test('should generate relevant subtasks from PRD context', async () => {
const task = createTestTaskWithPRDContext();
const subtasks = await expandTaskWithPRDContext(task);
const relevanceScore = calculateRelevanceScore(subtasks, task.prdContext);
expect(relevanceScore).toBeGreaterThan(0.8);
});
});
});
```
### Performance Testing:
- Test with large PRDs (>10,000 words)
- Measure processing time for different complexity levels
- Test memory usage with extensive context preservation
- Validate timeout handling for long-running operations
### Quality Assurance Tools:
- Automated semantic similarity checking
- Technical requirement compliance validation
- Context drift detection algorithms
- User acceptance testing framework
### Continuous Integration:
- Add tests to existing CI pipeline
- Set up performance benchmarking
- Implement quality gates for PRD processing
- Create regression testing for context preservation