fix(move): adjusts logic to prevent an issue when moving from parent to subtask if the target parent has no subtasks.
This commit is contained in:
Eyal Toledano
@@ -1,93 +1,639 @@
|
||||
# Task ID: 84
|
||||
# Title: Implement token counting utility
|
||||
# Title: Enhance Parse-PRD with Intelligent Task Expansion and Detail Preservation
|
||||
# Status: pending
|
||||
# Dependencies: 82
|
||||
# Dependencies: None
|
||||
# Priority: high
|
||||
# Description: Create a utility function to count tokens for prompts based on the model being used, primarily using tiktoken for OpenAI and Anthropic models with character-based fallbacks for other providers.
|
||||
# Description: Transform parse-prd from a simple task generator into an intelligent system that preserves PRD detail resolution through context-aware task expansion. This addresses the critical issue where highly detailed PRDs lose their specificity when parsed into too few top-level tasks, and ensures that task expansions are grounded in actual PRD content rather than generic AI assumptions.
|
||||
# Details:
|
||||
1. Install the tiktoken package:
|
||||
```bash
|
||||
npm install tiktoken
|
||||
```
|
||||
## Core Problem Statement
|
||||
|
||||
2. Create a new file `scripts/modules/token-counter.js`:
|
||||
The current parse-prd implementation suffers from a fundamental resolution loss problem:
|
||||
|
||||
1. **Detail Compression**: Complex, detailed PRDs get compressed into a fixed number of top-level tasks (default 10), losing critical specificity
|
||||
2. **Orphaned Expansions**: When tasks are later expanded via expand-task, the AI lacks the original PRD context, resulting in generic subtasks that don't reflect the PRD's specific requirements
|
||||
3. **Binary Approach**: The system either creates too few high-level tasks OR requires manual expansion that loses PRD context
|
||||
|
||||
## Solution Architecture
|
||||
|
||||
### Phase 1: Enhanced PRD Analysis Engine
|
||||
- Implement intelligent PRD segmentation that identifies natural task boundaries based on content structure
|
||||
- Create a PRD context preservation system that maintains detailed mappings between PRD sections and generated tasks
|
||||
- Develop adaptive task count determination based on PRD complexity metrics (length, technical depth, feature count)
|
||||
|
||||
### Phase 2: Context-Aware Task Generation
|
||||
- Modify generateTasksFromPRD to create tasks with embedded PRD context references
|
||||
- Implement a PRD section mapping system that links each task to its source PRD content
|
||||
- Add metadata fields to tasks that preserve original PRD language and specifications
|
||||
|
||||
### Phase 3: Intelligent In-Flight Expansion
|
||||
- Add optional `--expand-tasks` flag to parse-prd that triggers immediate expansion after initial task generation
|
||||
- Implement context-aware expansion that uses the original PRD content for each task's expansion
|
||||
- Create a two-pass system: first pass generates tasks with PRD context, second pass expands using that context
|
||||
|
||||
### Phase 4: PRD-Grounded Expansion Logic
|
||||
- Enhance the expansion prompt generation to include relevant PRD excerpts for each task being expanded
|
||||
- Implement smart context windowing that includes related PRD sections when expanding tasks
|
||||
- Add validation to ensure expanded subtasks maintain fidelity to original PRD specifications
|
||||
|
||||
## Technical Implementation Details
|
||||
|
||||
### File Modifications Required:
|
||||
1. **scripts/modules/task-manager/parse-prd.js**
|
||||
- Add PRD analysis functions for intelligent segmentation
|
||||
- Implement context preservation during task generation
|
||||
- Add optional expansion pipeline integration
|
||||
- Create PRD-to-task mapping system
|
||||
|
||||
2. **scripts/modules/task-manager/expand-task.js**
|
||||
- Enhance to accept PRD context as additional input
|
||||
- Modify expansion prompts to include relevant PRD excerpts
|
||||
- Add PRD-grounded validation for generated subtasks
|
||||
|
||||
3. **scripts/modules/ai-services-unified.js**
|
||||
- Add support for context-aware prompting with PRD excerpts
|
||||
- Implement intelligent context windowing for large PRDs
|
||||
- Add PRD analysis capabilities for complexity assessment
|
||||
|
||||
### New Data Structures:
|
||||
```javascript
|
||||
const tiktoken = require('tiktoken');
|
||||
|
||||
/**
|
||||
* Count tokens for a given text and model
|
||||
* @param {string} text - The text to count tokens for
|
||||
* @param {string} provider - The AI provider (e.g., 'openai', 'anthropic')
|
||||
* @param {string} modelId - The model ID
|
||||
* @returns {number} - Estimated token count
|
||||
*/
|
||||
function countTokens(text, provider, modelId) {
|
||||
if (!text) return 0;
|
||||
|
||||
// Convert to lowercase for case-insensitive matching
|
||||
const providerLower = provider?.toLowerCase();
|
||||
|
||||
try {
|
||||
// OpenAI models
|
||||
if (providerLower === 'openai') {
|
||||
// Most OpenAI chat models use cl100k_base encoding
|
||||
const encoding = tiktoken.encoding_for_model(modelId) || tiktoken.get_encoding('cl100k_base');
|
||||
return encoding.encode(text).length;
|
||||
}
|
||||
|
||||
// Anthropic models - can use cl100k_base as an approximation
|
||||
// or follow Anthropic's guidance
|
||||
if (providerLower === 'anthropic') {
|
||||
try {
|
||||
// Try to use cl100k_base as a reasonable approximation
|
||||
const encoding = tiktoken.get_encoding('cl100k_base');
|
||||
return encoding.encode(text).length;
|
||||
} catch (e) {
|
||||
// Fallback to Anthropic's character-based estimation
|
||||
return Math.ceil(text.length / 3.5); // ~3.5 chars per token for English
|
||||
}
|
||||
}
|
||||
|
||||
// For other providers, use character-based estimation as fallback
|
||||
// Different providers may have different tokenization schemes
|
||||
return Math.ceil(text.length / 4); // General fallback estimate
|
||||
} catch (error) {
|
||||
console.warn(`Token counting error: ${error.message}. Using character-based estimate.`);
|
||||
return Math.ceil(text.length / 4); // Fallback if tiktoken fails
|
||||
}
|
||||
// Enhanced task structure with PRD context
|
||||
{
|
||||
id: "1",
|
||||
title: "User Authentication System",
|
||||
description: "...",
|
||||
prdContext: {
|
||||
sourceSection: "Authentication Requirements (Lines 45-78)",
|
||||
originalText: "The system must implement OAuth 2.0...",
|
||||
relatedSections: ["Security Requirements", "User Management"],
|
||||
contextWindow: "Full PRD excerpt relevant to this task"
|
||||
},
|
||||
// ... existing fields
|
||||
}
|
||||
|
||||
module.exports = { countTokens };
|
||||
// PRD analysis metadata
|
||||
{
|
||||
prdAnalysis: {
|
||||
totalComplexity: 8.5,
|
||||
naturalTaskBoundaries: [...],
|
||||
recommendedTaskCount: 15,
|
||||
sectionMappings: {...}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. Add tests for the token counter in `tests/token-counter.test.js`:
|
||||
```javascript
|
||||
const { countTokens } = require('../scripts/modules/token-counter');
|
||||
### New CLI Options:
|
||||
- `--expand-tasks`: Automatically expand generated tasks using PRD context
|
||||
- `--preserve-detail`: Maximum detail preservation mode
|
||||
- `--adaptive-count`: Let AI determine optimal task count based on PRD complexity
|
||||
- `--context-window-size`: Control how much PRD context to include in expansions
|
||||
|
||||
describe('Token Counter', () => {
|
||||
test('counts tokens for OpenAI models', () => {
|
||||
const text = 'Hello, world! This is a test.';
|
||||
const count = countTokens(text, 'openai', 'gpt-4');
|
||||
expect(count).toBeGreaterThan(0);
|
||||
expect(typeof count).toBe('number');
|
||||
## Implementation Strategy
|
||||
|
||||
### Step 1: PRD Analysis Enhancement
|
||||
- Create PRD parsing utilities that identify natural section boundaries
|
||||
- Implement complexity scoring for different PRD sections
|
||||
- Build context extraction functions that preserve relevant details
|
||||
|
||||
### Step 2: Context-Aware Task Generation
|
||||
- Modify the task generation prompt to include section-specific context
|
||||
- Implement task-to-PRD mapping during generation
|
||||
- Add metadata fields to preserve PRD relationships
|
||||
|
||||
### Step 3: Intelligent Expansion Pipeline
|
||||
- Create expansion logic that uses preserved PRD context
|
||||
- Implement smart prompt engineering that includes relevant PRD excerpts
|
||||
- Add validation to ensure subtask fidelity to original requirements
|
||||
|
||||
### Step 4: Integration and Testing
|
||||
- Integrate new functionality with existing parse-prd workflow
|
||||
- Add comprehensive testing with various PRD types and complexities
|
||||
- Implement telemetry for tracking detail preservation effectiveness
|
||||
|
||||
## Success Metrics
|
||||
- PRD detail preservation rate (measured by semantic similarity between PRD and generated tasks)
|
||||
- Reduction in manual task refinement needed post-parsing
|
||||
- Improved accuracy of expanded subtasks compared to PRD specifications
|
||||
- User satisfaction with task granularity and detail accuracy
|
||||
|
||||
## Edge Cases and Considerations
|
||||
- Very large PRDs that exceed context windows
|
||||
- PRDs with conflicting or ambiguous requirements
|
||||
- Integration with existing task expansion workflows
|
||||
- Performance impact of enhanced analysis
|
||||
- Backward compatibility with existing parse-prd usage
|
||||
|
||||
# Test Strategy:
|
||||
|
||||
|
||||
# Subtasks:
|
||||
## 1. Implement PRD Analysis and Segmentation Engine [pending]
|
||||
### Dependencies: None
|
||||
### Description: Create intelligent PRD parsing that identifies natural task boundaries and complexity metrics
|
||||
### Details:
|
||||
## Implementation Requirements
|
||||
|
||||
### Core Functions to Implement:
|
||||
1. **analyzePRDStructure(prdContent)**
|
||||
- Parse PRD into logical sections using headers, bullet points, and semantic breaks
|
||||
- Identify feature boundaries, technical requirements, and implementation sections
|
||||
- Return structured analysis with section metadata
|
||||
|
||||
2. **calculatePRDComplexity(prdContent)**
|
||||
- Analyze technical depth, feature count, integration requirements
|
||||
- Score complexity on 1-10 scale for different aspects
|
||||
- Return recommended task count based on complexity
|
||||
|
||||
3. **extractTaskBoundaries(prdAnalysis)**
|
||||
- Identify natural breaking points for task creation
|
||||
- Group related requirements into logical task units
|
||||
- Preserve context relationships between sections
|
||||
|
||||
### Technical Approach:
|
||||
- Use regex patterns and NLP techniques to identify section headers
|
||||
- Implement keyword analysis for technical complexity assessment
|
||||
- Create semantic grouping algorithms for related requirements
|
||||
- Build context preservation mappings
|
||||
|
||||
### Output Structure:
|
||||
```javascript
|
||||
{
|
||||
sections: [
|
||||
{
|
||||
title: "User Authentication",
|
||||
content: "...",
|
||||
startLine: 45,
|
||||
endLine: 78,
|
||||
complexity: 7,
|
||||
relatedSections: ["Security", "User Management"]
|
||||
}
|
||||
],
|
||||
overallComplexity: 8.5,
|
||||
recommendedTaskCount: 15,
|
||||
naturalBoundaries: [...],
|
||||
contextMappings: {...}
|
||||
}
|
||||
```
|
||||
|
||||
### Integration Points:
|
||||
- Called at the beginning of parse-prd process
|
||||
- Results used to inform task generation strategy
|
||||
- Analysis stored for later use in expansion phase
|
||||
|
||||
## 2. Enhance Task Generation with PRD Context Preservation [pending]
|
||||
### Dependencies: None
|
||||
### Description: Modify generateTasksFromPRD to embed PRD context and maintain source mappings
|
||||
### Details:
|
||||
## Implementation Requirements
|
||||
|
||||
### Core Modifications to generateTasksFromPRD:
|
||||
1. **Add PRD Context Embedding**
|
||||
- Modify task generation prompt to include relevant PRD excerpts
|
||||
- Ensure each generated task includes source section references
|
||||
- Preserve original PRD language and specifications in task metadata
|
||||
|
||||
2. **Implement Context Windowing**
|
||||
- For large PRDs, implement intelligent context windowing
|
||||
- Include relevant sections for each task being generated
|
||||
- Maintain context relationships between related tasks
|
||||
|
||||
3. **Enhanced Task Structure**
|
||||
- Add prdContext field to task objects
|
||||
- Include sourceSection, originalText, and relatedSections
|
||||
- Store contextWindow for later use in expansions
|
||||
|
||||
### Technical Implementation:
|
||||
```javascript
|
||||
// Enhanced task generation with context
|
||||
const generateTaskWithContext = async (prdSection, relatedSections, fullPRD) => {
|
||||
const contextWindow = buildContextWindow(prdSection, relatedSections, fullPRD);
|
||||
const prompt = `
|
||||
Generate a task based on this PRD section:
|
||||
|
||||
PRIMARY SECTION:
|
||||
${prdSection.content}
|
||||
|
||||
RELATED CONTEXT:
|
||||
${contextWindow}
|
||||
|
||||
Ensure the task preserves all specific requirements and technical details.
|
||||
`;
|
||||
|
||||
// Generate task with embedded context
|
||||
const task = await generateTask(prompt);
|
||||
task.prdContext = {
|
||||
sourceSection: prdSection.title,
|
||||
originalText: prdSection.content,
|
||||
relatedSections: relatedSections.map(s => s.title),
|
||||
contextWindow: contextWindow
|
||||
};
|
||||
|
||||
return task;
|
||||
};
|
||||
```
|
||||
|
||||
### Context Preservation Strategy:
|
||||
- Map each task to its source PRD sections
|
||||
- Preserve technical specifications and requirements language
|
||||
- Maintain relationships between interdependent features
|
||||
- Store context for later use in expansion phase
|
||||
|
||||
### Integration with Existing Flow:
|
||||
- Modify existing generateTasksFromPRD function
|
||||
- Maintain backward compatibility with simple PRDs
|
||||
- Add new metadata fields without breaking existing structure
|
||||
- Ensure context is available for subsequent operations
|
||||
|
||||
## 3. Implement In-Flight Task Expansion Pipeline [pending]
|
||||
### Dependencies: None
|
||||
### Description: Add optional --expand-tasks flag and intelligent expansion using preserved PRD context
|
||||
### Details:
|
||||
## Implementation Requirements
|
||||
|
||||
### Core Features:
|
||||
1. **Add --expand-tasks CLI Flag**
|
||||
- Optional flag for parse-prd command
|
||||
- Triggers automatic expansion after initial task generation
|
||||
- Configurable expansion depth and strategy
|
||||
|
||||
2. **Two-Pass Processing System**
|
||||
- First pass: Generate tasks with PRD context preservation
|
||||
- Second pass: Expand tasks using their embedded PRD context
|
||||
- Maintain context fidelity throughout the process
|
||||
|
||||
3. **Context-Aware Expansion Logic**
|
||||
- Use preserved PRD context for each task's expansion
|
||||
- Include relevant PRD excerpts in expansion prompts
|
||||
- Ensure subtasks maintain fidelity to original specifications
|
||||
|
||||
### Technical Implementation:
|
||||
```javascript
|
||||
// Enhanced parse-prd with expansion pipeline
|
||||
const parsePRDWithExpansion = async (prdContent, options) => {
|
||||
// Phase 1: Analyze and generate tasks with context
|
||||
const prdAnalysis = await analyzePRDStructure(prdContent);
|
||||
const tasksWithContext = await generateTasksWithContext(prdAnalysis);
|
||||
|
||||
// Phase 2: Expand tasks if requested
|
||||
if (options.expandTasks) {
|
||||
for (const task of tasksWithContext) {
|
||||
if (shouldExpandTask(task, prdAnalysis)) {
|
||||
const expandedSubtasks = await expandTaskWithPRDContext(task);
|
||||
task.subtasks = expandedSubtasks;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return tasksWithContext;
|
||||
};
|
||||
|
||||
// Context-aware task expansion
|
||||
const expandTaskWithPRDContext = async (task) => {
|
||||
const { prdContext } = task;
|
||||
const expansionPrompt = `
|
||||
Expand this task into detailed subtasks using the original PRD context:
|
||||
|
||||
TASK: ${task.title}
|
||||
DESCRIPTION: ${task.description}
|
||||
|
||||
ORIGINAL PRD CONTEXT:
|
||||
${prdContext.originalText}
|
||||
|
||||
RELATED SECTIONS:
|
||||
${prdContext.contextWindow}
|
||||
|
||||
Generate subtasks that preserve all technical details and requirements from the PRD.
|
||||
`;
|
||||
|
||||
return await generateSubtasks(expansionPrompt);
|
||||
};
|
||||
```
|
||||
|
||||
### CLI Integration:
|
||||
- Add --expand-tasks flag to parse-prd command
|
||||
- Add --expansion-depth option for controlling subtask levels
|
||||
- Add --preserve-detail flag for maximum context preservation
|
||||
- Maintain backward compatibility with existing parse-prd usage
|
||||
|
||||
### Expansion Strategy:
|
||||
- Determine which tasks should be expanded based on complexity
|
||||
- Use PRD context to generate accurate, detailed subtasks
|
||||
- Preserve technical specifications and implementation details
|
||||
- Validate subtask accuracy against original PRD content
|
||||
|
||||
### Performance Considerations:
|
||||
- Implement batching for large numbers of tasks
|
||||
- Add progress indicators for long-running expansions
|
||||
- Optimize context window sizes for efficiency
|
||||
- Cache PRD analysis results for reuse
|
||||
|
||||
## 4. Enhance Expand-Task with PRD Context Integration [pending]
|
||||
### Dependencies: None
|
||||
### Description: Modify existing expand-task functionality to leverage preserved PRD context for more accurate expansions
|
||||
### Details:
|
||||
## Implementation Requirements
|
||||
|
||||
### Core Enhancements to expand-task.js:
|
||||
1. **PRD Context Detection**
|
||||
- Check if task has embedded prdContext metadata
|
||||
- Extract relevant PRD sections for expansion
|
||||
- Fall back to existing expansion logic if no PRD context
|
||||
|
||||
2. **Context-Enhanced Expansion Prompts**
|
||||
- Include original PRD excerpts in expansion prompts
|
||||
- Add related section context for comprehensive understanding
|
||||
- Preserve technical specifications and requirements language
|
||||
|
||||
3. **Validation and Quality Assurance**
|
||||
- Validate generated subtasks against original PRD content
|
||||
- Ensure technical accuracy and requirement compliance
|
||||
- Flag potential discrepancies for review
|
||||
|
||||
### Technical Implementation:
|
||||
```javascript
|
||||
// Enhanced expand-task with PRD context
|
||||
const expandTaskWithContext = async (taskId, options, context) => {
|
||||
const task = await getTask(taskId);
|
||||
|
||||
// Check for PRD context
|
||||
if (task.prdContext) {
|
||||
return await expandWithPRDContext(task, options);
|
||||
} else {
|
||||
// Fall back to existing expansion logic
|
||||
return await expandTaskStandard(task, options);
|
||||
}
|
||||
};
|
||||
|
||||
const expandWithPRDContext = async (task, options) => {
|
||||
const { prdContext } = task;
|
||||
|
||||
const enhancedPrompt = `
|
||||
Expand this task into detailed subtasks using the original PRD context:
|
||||
|
||||
TASK DETAILS:
|
||||
Title: ${task.title}
|
||||
Description: ${task.description}
|
||||
Current Details: ${task.details}
|
||||
|
||||
ORIGINAL PRD CONTEXT:
|
||||
Source Section: ${prdContext.sourceSection}
|
||||
Original Requirements:
|
||||
${prdContext.originalText}
|
||||
|
||||
RELATED CONTEXT:
|
||||
${prdContext.contextWindow}
|
||||
|
||||
EXPANSION REQUIREMENTS:
|
||||
- Preserve all technical specifications from the PRD
|
||||
- Maintain requirement accuracy and completeness
|
||||
- Generate ${options.num || 'appropriate number of'} subtasks
|
||||
- Include implementation details that reflect PRD specifics
|
||||
|
||||
Generate subtasks that are grounded in the original PRD content.
|
||||
`;
|
||||
|
||||
const subtasks = await generateSubtasks(enhancedPrompt, options);
|
||||
|
||||
// Add PRD context inheritance to subtasks
|
||||
subtasks.forEach(subtask => {
|
||||
subtask.prdContext = {
|
||||
inheritedFrom: task.id,
|
||||
sourceSection: prdContext.sourceSection,
|
||||
relevantExcerpt: extractRelevantExcerpt(prdContext, subtask)
|
||||
};
|
||||
});
|
||||
|
||||
test('counts tokens for Anthropic models', () => {
|
||||
const text = 'Hello, world! This is a test.';
|
||||
const count = countTokens(text, 'anthropic', 'claude-3-7-sonnet-20250219');
|
||||
expect(count).toBeGreaterThan(0);
|
||||
expect(typeof count).toBe('number');
|
||||
return subtasks;
|
||||
};
|
||||
```
|
||||
|
||||
### Integration Points:
|
||||
1. **Modify existing expand-task.js**
|
||||
- Add PRD context detection logic
|
||||
- Enhance prompt generation with context
|
||||
- Maintain backward compatibility
|
||||
|
||||
2. **Update expansion validation**
|
||||
- Add PRD compliance checking
|
||||
- Implement quality scoring for context fidelity
|
||||
- Flag potential accuracy issues
|
||||
|
||||
3. **CLI and MCP Integration**
|
||||
- Update expand-task command to leverage PRD context
|
||||
- Add options for context-aware expansion
|
||||
- Maintain existing command interface
|
||||
|
||||
### Context Inheritance Strategy:
|
||||
- Pass relevant PRD context to generated subtasks
|
||||
- Create context inheritance chain for nested expansions
|
||||
- Preserve source traceability throughout expansion tree
|
||||
- Enable future re-expansion with maintained context
|
||||
|
||||
### Quality Assurance Features:
|
||||
- Semantic similarity checking between subtasks and PRD
|
||||
- Technical requirement compliance validation
|
||||
- Automated flagging of potential context drift
|
||||
- User feedback integration for continuous improvement
|
||||
|
||||
## 5. Add New CLI Options and MCP Parameters [pending]
|
||||
### Dependencies: None
|
||||
### Description: Implement new command-line flags and MCP tool parameters for enhanced PRD parsing
|
||||
### Details:
|
||||
## Implementation Requirements
|
||||
|
||||
### New CLI Options for parse-prd:
|
||||
1. **--expand-tasks**
|
||||
- Automatically expand generated tasks using PRD context
|
||||
- Boolean flag, default false
|
||||
- Triggers in-flight expansion pipeline
|
||||
|
||||
2. **--preserve-detail**
|
||||
- Maximum detail preservation mode
|
||||
- Boolean flag, default false
|
||||
- Ensures highest fidelity to PRD content
|
||||
|
||||
3. **--adaptive-count**
|
||||
- Let AI determine optimal task count based on PRD complexity
|
||||
- Boolean flag, default false
|
||||
- Overrides --num-tasks when enabled
|
||||
|
||||
4. **--context-window-size**
|
||||
- Control how much PRD context to include in expansions
|
||||
- Integer value, default 2000 characters
|
||||
- Balances context richness with performance
|
||||
|
||||
5. **--expansion-depth**
|
||||
- Control how many levels deep to expand tasks
|
||||
- Integer value, default 1
|
||||
- Prevents excessive nesting
|
||||
|
||||
### MCP Tool Parameter Updates:
|
||||
```javascript
|
||||
// Enhanced parse_prd MCP tool parameters
|
||||
{
|
||||
input: "Path to PRD file",
|
||||
output: "Output path for tasks.json",
|
||||
numTasks: "Number of top-level tasks (overridden by adaptiveCount)",
|
||||
expandTasks: "Boolean - automatically expand tasks with PRD context",
|
||||
preserveDetail: "Boolean - maximum detail preservation mode",
|
||||
adaptiveCount: "Boolean - AI determines optimal task count",
|
||||
contextWindowSize: "Integer - context size for expansions",
|
||||
expansionDepth: "Integer - levels of expansion to perform",
|
||||
research: "Boolean - use research model for enhanced analysis",
|
||||
force: "Boolean - overwrite existing files"
|
||||
}
|
||||
```
|
||||
|
||||
### CLI Command Updates:
|
||||
```bash
|
||||
# Enhanced parse-prd command examples
|
||||
task-master parse-prd prd.txt --expand-tasks --preserve-detail
|
||||
task-master parse-prd prd.txt --adaptive-count --expansion-depth=2
|
||||
task-master parse-prd prd.txt --context-window-size=3000 --research
|
||||
```
|
||||
|
||||
### Implementation Details:
|
||||
1. **Update commands.js**
|
||||
- Add new option definitions
|
||||
- Update parse-prd command handler
|
||||
- Maintain backward compatibility
|
||||
|
||||
2. **Update MCP tool definition**
|
||||
- Add new parameter schemas
|
||||
- Update tool description and examples
|
||||
- Ensure parameter validation
|
||||
|
||||
3. **Parameter Processing Logic**
|
||||
- Validate parameter combinations
|
||||
- Set appropriate defaults
|
||||
- Handle conflicting options gracefully
|
||||
|
||||
### Validation Rules:
|
||||
- expansion-depth must be positive integer ≤ 3
|
||||
- context-window-size must be between 500-5000 characters
|
||||
- adaptive-count overrides num-tasks when both specified
|
||||
- expand-tasks requires either adaptive-count or num-tasks > 5
|
||||
|
||||
### Help Documentation Updates:
|
||||
- Update command help text with new options
|
||||
- Add usage examples for different scenarios
|
||||
- Document parameter interactions and constraints
|
||||
- Include performance considerations for large PRDs
|
||||
|
||||
## 6. Implement Comprehensive Testing and Validation [pending]
|
||||
### Dependencies: None
|
||||
### Description: Create test suite for PRD analysis, context preservation, and expansion accuracy
|
||||
### Details:
|
||||
## Implementation Requirements
|
||||
|
||||
### Test Categories:
|
||||
1. **PRD Analysis Testing**
|
||||
- Test section identification with various PRD formats
|
||||
- Validate complexity scoring accuracy
|
||||
- Test boundary detection for different document structures
|
||||
- Verify context mapping correctness
|
||||
|
||||
2. **Context Preservation Testing**
|
||||
- Validate PRD context embedding in generated tasks
|
||||
- Test context window generation and sizing
|
||||
- Verify source section mapping accuracy
|
||||
- Test context inheritance in subtasks
|
||||
|
||||
3. **Expansion Accuracy Testing**
|
||||
- Compare PRD-grounded vs standard expansions
|
||||
- Measure semantic similarity between PRD and subtasks
|
||||
- Test technical requirement preservation
|
||||
- Validate expansion depth and quality
|
||||
|
||||
4. **Integration Testing**
|
||||
- Test full parse-prd pipeline with expansion
|
||||
- Validate CLI option combinations
|
||||
- Test MCP tool parameter handling
|
||||
- Verify backward compatibility
|
||||
|
||||
### Test Data Requirements:
|
||||
```javascript
|
||||
// Test PRD samples
|
||||
const testPRDs = {
|
||||
simple: "Basic PRD with minimal technical details",
|
||||
complex: "Detailed PRD with extensive technical specifications",
|
||||
structured: "Well-organized PRD with clear sections",
|
||||
unstructured: "Free-form PRD with mixed content",
|
||||
technical: "Highly technical PRD with specific requirements",
|
||||
large: "Very large PRD testing context window limits"
|
||||
};
|
||||
```
|
||||
|
||||
### Validation Metrics:
|
||||
1. **Detail Preservation Score**
|
||||
- Semantic similarity between PRD and generated tasks
|
||||
- Technical requirement coverage percentage
|
||||
- Specification accuracy rating
|
||||
|
||||
2. **Context Fidelity Score**
|
||||
- Accuracy of source section mapping
|
||||
- Relevance of included context windows
|
||||
- Quality of context inheritance
|
||||
|
||||
3. **Expansion Quality Score**
|
||||
- Subtask relevance to parent task and PRD
|
||||
- Technical accuracy of implementation details
|
||||
- Completeness of requirement coverage
|
||||
|
||||
### Test Implementation:
|
||||
```javascript
|
||||
// Example test structure
|
||||
describe('Enhanced Parse-PRD', () => {
|
||||
describe('PRD Analysis', () => {
|
||||
test('should identify sections correctly', async () => {
|
||||
const analysis = await analyzePRDStructure(testPRDs.structured);
|
||||
expect(analysis.sections).toHaveLength(expectedSectionCount);
|
||||
expect(analysis.overallComplexity).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
test('should calculate appropriate task count', async () => {
|
||||
const analysis = await analyzePRDStructure(testPRDs.complex);
|
||||
expect(analysis.recommendedTaskCount).toBeGreaterThan(10);
|
||||
});
|
||||
});
|
||||
|
||||
test('handles empty text', () => {
|
||||
expect(countTokens('', 'openai', 'gpt-4')).toBe(0);
|
||||
expect(countTokens(null, 'openai', 'gpt-4')).toBe(0);
|
||||
describe('Context Preservation', () => {
|
||||
test('should embed PRD context in tasks', async () => {
|
||||
const tasks = await generateTasksWithContext(testPRDs.technical);
|
||||
tasks.forEach(task => {
|
||||
expect(task.prdContext).toBeDefined();
|
||||
expect(task.prdContext.sourceSection).toBeTruthy();
|
||||
expect(task.prdContext.originalText).toBeTruthy();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('Expansion Accuracy', () => {
|
||||
test('should generate relevant subtasks from PRD context', async () => {
|
||||
const task = createTestTaskWithPRDContext();
|
||||
const subtasks = await expandTaskWithPRDContext(task);
|
||||
|
||||
const relevanceScore = calculateRelevanceScore(subtasks, task.prdContext);
|
||||
expect(relevanceScore).toBeGreaterThan(0.8);
|
||||
});
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
# Test Strategy:
|
||||
1. Unit test the countTokens function with various inputs and models
|
||||
2. Compare token counts with known examples from OpenAI and Anthropic documentation
|
||||
3. Test edge cases: empty strings, very long texts, non-English texts
|
||||
4. Test fallback behavior when tiktoken fails or is not applicable
|
||||
### Performance Testing:
|
||||
- Test with large PRDs (>10,000 words)
|
||||
- Measure processing time for different complexity levels
|
||||
- Test memory usage with extensive context preservation
|
||||
- Validate timeout handling for long-running operations
|
||||
|
||||
### Quality Assurance Tools:
|
||||
- Automated semantic similarity checking
|
||||
- Technical requirement compliance validation
|
||||
- Context drift detection algorithms
|
||||
- User acceptance testing framework
|
||||
|
||||
### Continuous Integration:
|
||||
- Add tests to existing CI pipeline
|
||||
- Set up performance benchmarking
|
||||
- Implement quality gates for PRD processing
|
||||
- Create regression testing for context preservation
|
||||
|
||||
|
||||
Reference in New Issue
Block a user