docs: add context gathering rule and update existing rules

- Create comprehensive context_gathering.mdc rule documenting ContextGatherer utility patterns, FuzzyTaskSearch integration, token breakdown display, code block syntax highlighting, and enhanced result display patterns
- Update new_features.mdc to include context gathering step
- Update commands.mdc with context-aware command pattern
- Update ui.mdc with enhanced display patterns and syntax highlighting
- Update utilities.mdc to document new context gathering utilities
- Update glossary.mdc to include new context_gathering rule
- Establishes standardized patterns for building intelligent, context-aware commands that can leverage project knowledge for better AI assistance
This commit is contained in:
Eyal Toledano
2025-05-25 17:19:28 -04:00
parent 275cd55da7
commit 947541e4ee
5 changed files with 476 additions and 10 deletions

View File

@@ -329,6 +329,60 @@ When implementing commands that delete or remove data (like `remove-task` or `re
};
```
## Context-Aware Command Pattern
For AI-powered commands that benefit from project context, follow the research command pattern:
- **Context Integration**:
- ✅ DO: Use `ContextGatherer` utility for multi-source context extraction
- ✅ DO: Support task IDs, file paths, custom context, and project tree
- ✅ DO: Implement fuzzy search for automatic task discovery
- ✅ DO: Display detailed token breakdown for transparency
```javascript
// ✅ DO: Follow this pattern for context-aware commands
programInstance
.command('research')
.description('Perform AI-powered research queries with project context')
.argument('<prompt>', 'Research prompt to investigate')
.option('-i, --id <ids>', 'Comma-separated task/subtask IDs to include as context')
.option('-f, --files <paths>', 'Comma-separated file paths to include as context')
.option('-c, --context <text>', 'Additional custom context')
.option('--project-tree', 'Include project file tree structure')
.option('-d, --detail <level>', 'Output detail level: low, medium, high', 'medium')
.action(async (prompt, options) => {
// 1. Parameter validation and parsing
const taskIds = options.id ? parseTaskIds(options.id) : [];
const filePaths = options.files ? parseFilePaths(options.files) : [];
// 2. Initialize context gatherer
const projectRoot = findProjectRoot() || '.';
const gatherer = new ContextGatherer(projectRoot, tasksPath);
// 3. Auto-discover relevant tasks if none specified
if (taskIds.length === 0) {
const fuzzySearch = new FuzzyTaskSearch(tasksData.tasks, 'research');
const discoveredIds = fuzzySearch.getTaskIds(
fuzzySearch.findRelevantTasks(prompt)
);
taskIds.push(...discoveredIds);
}
// 4. Gather context with token breakdown
const contextResult = await gatherer.gather({
tasks: taskIds,
files: filePaths,
customContext: options.context,
includeProjectTree: options.projectTree,
format: 'research',
includeTokenCounts: true
});
// 5. Display token breakdown and execute AI call
// Implementation continues...
});
```
## Error Handling
- **Exception Management**:

View File

@@ -0,0 +1,268 @@
---
description: Standardized patterns for gathering and processing context from multiple sources in Task Master commands, particularly for AI-powered features.
globs:
alwaysApply: false
---
# Context Gathering Patterns and Utilities
This document outlines the standardized patterns for gathering and processing context from multiple sources in Task Master commands, particularly for AI-powered features.
## Core Context Gathering Utility
The `ContextGatherer` class (`scripts/modules/utils/contextGatherer.js`) provides a centralized, reusable utility for extracting context from multiple sources:
### **Key Features**
- **Multi-source Context**: Tasks, files, custom text, project file tree
- **Token Counting**: Detailed breakdown using `gpt-tokens` library
- **Format Support**: Different output formats (research, chat, system-prompt)
- **Error Handling**: Graceful handling of missing files, invalid task IDs
- **Performance**: File size limits, depth limits for tree generation
### **Usage Pattern**
```javascript
import { ContextGatherer } from '../utils/contextGatherer.js';
// Initialize with project paths
const gatherer = new ContextGatherer(projectRoot, tasksPath);
// Gather context with detailed token breakdown
const result = await gatherer.gather({
tasks: ['15', '16.2'], // Task and subtask IDs
files: ['src/api.js', 'README.md'], // File paths
customContext: 'Additional context text',
includeProjectTree: true, // Include file tree
format: 'research', // Output format
includeTokenCounts: true // Get detailed token breakdown
});
// Access results
const contextString = result.context;
const tokenBreakdown = result.tokenBreakdown;
```
### **Token Breakdown Structure**
```javascript
{
customContext: { tokens: 150, characters: 800 },
tasks: [
{ id: '15', type: 'task', title: 'Task Title', tokens: 245, characters: 1200 },
{ id: '16.2', type: 'subtask', title: 'Subtask Title', tokens: 180, characters: 900 }
],
files: [
{ path: 'src/api.js', tokens: 890, characters: 4500, size: '4.5 KB' }
],
projectTree: { tokens: 320, characters: 1600 },
total: { tokens: 1785, characters: 8000 }
}
```
## Fuzzy Search Integration
The `FuzzyTaskSearch` class (`scripts/modules/utils/fuzzyTaskSearch.js`) provides intelligent task discovery:
### **Key Features**
- **Semantic Matching**: Uses Fuse.js for similarity scoring
- **Purpose Categories**: Pattern-based task categorization
- **Relevance Scoring**: High/medium/low relevance thresholds
- **Context-Aware**: Different search configurations for different use cases
### **Usage Pattern**
```javascript
import { FuzzyTaskSearch } from '../utils/fuzzyTaskSearch.js';
// Initialize with tasks data and context
const fuzzySearch = new FuzzyTaskSearch(tasksData.tasks, 'research');
// Find relevant tasks
const searchResults = fuzzySearch.findRelevantTasks(query, {
maxResults: 8,
includeRecent: true,
includeCategoryMatches: true
});
// Get task IDs for context gathering
const taskIds = fuzzySearch.getTaskIds(searchResults);
```
## Implementation Patterns for Commands
### **1. Context-Aware Command Structure**
```javascript
// In command action handler
async function commandAction(prompt, options) {
// 1. Parameter validation and parsing
const taskIds = options.id ? parseTaskIds(options.id) : [];
const filePaths = options.files ? parseFilePaths(options.files) : [];
// 2. Initialize context gatherer
const projectRoot = findProjectRoot() || '.';
const tasksPath = path.join(projectRoot, 'tasks', 'tasks.json');
const gatherer = new ContextGatherer(projectRoot, tasksPath);
// 3. Auto-discover relevant tasks if none specified
if (taskIds.length === 0) {
const fuzzySearch = new FuzzyTaskSearch(tasksData.tasks, 'research');
const discoveredIds = fuzzySearch.getTaskIds(
fuzzySearch.findRelevantTasks(prompt)
);
taskIds.push(...discoveredIds);
}
// 4. Gather context with token breakdown
const contextResult = await gatherer.gather({
tasks: taskIds,
files: filePaths,
customContext: options.context,
includeProjectTree: options.projectTree,
format: 'research',
includeTokenCounts: true
});
// 5. Display token breakdown (for CLI)
if (outputFormat === 'text') {
displayDetailedTokenBreakdown(contextResult.tokenBreakdown);
}
// 6. Use context in AI call
const aiResult = await generateTextService(role, session, systemPrompt, userPrompt);
// 7. Display results with enhanced formatting
displayResults(aiResult, contextResult.tokenBreakdown);
}
```
### **2. Token Display Pattern**
```javascript
function displayDetailedTokenBreakdown(tokenBreakdown, systemTokens, userTokens) {
const sections = [];
// Build context breakdown
if (tokenBreakdown.tasks?.length > 0) {
const taskDetails = tokenBreakdown.tasks.map(task =>
`${task.type === 'subtask' ? ' ' : ''}${task.id}: ${task.tokens.toLocaleString()}`
).join('\n');
sections.push(`Tasks (${tokenBreakdown.tasks.reduce((sum, t) => sum + t.tokens, 0).toLocaleString()}):\n${taskDetails}`);
}
if (tokenBreakdown.files?.length > 0) {
const fileDetails = tokenBreakdown.files.map(file =>
` ${file.path}: ${file.tokens.toLocaleString()} (${file.size})`
).join('\n');
sections.push(`Files (${tokenBreakdown.files.reduce((sum, f) => sum + f.tokens, 0).toLocaleString()}):\n${fileDetails}`);
}
// Add prompts breakdown
sections.push(`Prompts: system ${systemTokens.toLocaleString()}, user ${userTokens.toLocaleString()}`);
// Display in clean box
const content = sections.join('\n\n');
console.log(boxen(content, {
title: chalk.cyan('Token Usage'),
padding: { top: 1, bottom: 1, left: 2, right: 2 },
borderStyle: 'round',
borderColor: 'cyan'
}));
}
```
### **3. Enhanced Result Display Pattern**
```javascript
function displayResults(result, query, detailLevel, tokenBreakdown) {
// Header with query info
const header = boxen(
chalk.green.bold('Research Results') + '\n\n' +
chalk.gray('Query: ') + chalk.white(query) + '\n' +
chalk.gray('Detail Level: ') + chalk.cyan(detailLevel),
{
padding: { top: 1, bottom: 1, left: 2, right: 2 },
margin: { top: 1, bottom: 0 },
borderStyle: 'round',
borderColor: 'green'
}
);
console.log(header);
// Process and highlight code blocks
const processedResult = processCodeBlocks(result);
// Main content in clean box
const contentBox = boxen(processedResult, {
padding: { top: 1, bottom: 1, left: 2, right: 2 },
margin: { top: 0, bottom: 1 },
borderStyle: 'single',
borderColor: 'gray'
});
console.log(contentBox);
console.log(chalk.green('✓ Research complete'));
}
```
## Code Block Enhancement
### **Syntax Highlighting Pattern**
```javascript
import { highlight } from 'cli-highlight';
function processCodeBlocks(text) {
return text.replace(/```(\w+)?\n([\s\S]*?)```/g, (match, language, code) => {
try {
const highlighted = highlight(code.trim(), {
language: language || 'javascript',
theme: 'default'
});
return `\n${highlighted}\n`;
} catch (error) {
return `\n${code.trim()}\n`;
}
});
}
```
## Integration Guidelines
### **When to Use Context Gathering**
- ✅ **DO**: Use for AI-powered commands that benefit from project context
- ✅ **DO**: Use when users might want to reference specific tasks or files
- ✅ **DO**: Use for research, analysis, or generation commands
- ❌ **DON'T**: Use for simple CRUD operations that don't need AI context
### **Performance Considerations**
- ✅ **DO**: Set reasonable file size limits (50KB default)
- ✅ **DO**: Limit project tree depth (3-5 levels)
- ✅ **DO**: Provide token counts to help users understand context size
- ✅ **DO**: Allow users to control what context is included
### **Error Handling**
- ✅ **DO**: Gracefully handle missing files with warnings
- ✅ **DO**: Validate task IDs and provide helpful error messages
- ✅ **DO**: Continue processing even if some context sources fail
- ✅ **DO**: Provide fallback behavior when context gathering fails
### **Future Command Integration**
Commands that should consider adopting this pattern:
- `analyze-complexity` - Could benefit from file context
- `expand-task` - Could use related task context
- `update-task` - Could reference similar tasks for consistency
- `add-task` - Could use project context for better task generation
## Export Patterns
### **Context Gatherer Module**
```javascript
export {
ContextGatherer,
createContextGatherer // Factory function
};
```
### **Fuzzy Search Module**
```javascript
export {
FuzzyTaskSearch,
PURPOSE_CATEGORIES,
RELEVANCE_THRESHOLDS
};
```
This context gathering system provides a foundation for building more intelligent, context-aware commands that can leverage project knowledge to provide better AI-powered assistance.

View File

@@ -24,17 +24,22 @@ alwaysApply: false
The standard pattern for adding a feature follows this workflow:
1. **Core Logic**: Implement the business logic in the appropriate module (e.g., [`task-manager.js`](mdc:scripts/modules/task-manager.js)).
2. **AI Integration (If Applicable)**:
2. **Context Gathering (If Applicable)**:
- For AI-powered commands that benefit from project context, use the standardized context gathering patterns from [`context_gathering.mdc`](mdc:.cursor/rules/context_gathering.mdc).
- Import `ContextGatherer` and `FuzzyTaskSearch` utilities for reusable context extraction.
- Support multiple context types: tasks, files, custom text, project tree.
- Implement detailed token breakdown display for transparency.
3. **AI Integration (If Applicable)**:
- Import necessary service functions (e.g., `generateTextService`, `streamTextService`) from [`ai-services-unified.js`](mdc:scripts/modules/ai-services-unified.js).
- Prepare parameters (`role`, `session`, `systemPrompt`, `prompt`).
- Call the service function.
- Handle the response (direct text or stream object).
- **Important**: Prefer `generateTextService` for calls sending large context (like stringified JSON) where incremental display is not needed. See [`ai_services.mdc`](mdc:.cursor/rules/ai_services.mdc) for detailed usage patterns and cautions.
3. **UI Components**: Add any display functions to [`ui.js`](mdc:scripts/modules/ui.js) following [`ui.mdc`](mdc:.cursor/rules/ui.mdc).
4. **Command Integration**: Add the CLI command to [`commands.js`](mdc:scripts/modules/commands.js) following [`commands.mdc`](mdc:.cursor/rules/commands.mdc).
5. **Testing**: Write tests for all components of the feature (following [`tests.mdc`](mdc:.cursor/rules/tests.mdc))
6. **Configuration**: Update configuration settings or add new ones in [`config-manager.js`](mdc:scripts/modules/config-manager.js) and ensure getters/setters are appropriate. Update documentation in [`utilities.mdc`](mdc:.cursor/rules/utilities.mdc) and [`taskmaster.mdc`](mdc:.cursor/rules/taskmaster.mdc). Update the `.taskmasterconfig` structure if needed.
7. **Documentation**: Update help text and documentation in [`dev_workflow.mdc`](mdc:.cursor/rules/dev_workflow.mdc) and [`taskmaster.mdc`](mdc:.cursor/rules/taskmaster.mdc).
4. **UI Components**: Add any display functions to [`ui.js`](mdc:scripts/modules/ui.js) following [`ui.mdc`](mdc:.cursor/rules/ui.mdc). Consider enhanced formatting with syntax highlighting for code blocks.
5. **Command Integration**: Add the CLI command to [`commands.js`](mdc:scripts/modules/commands.js) following [`commands.mdc`](mdc:.cursor/rules/commands.mdc).
6. **Testing**: Write tests for all components of the feature (following [`tests.mdc`](mdc:.cursor/rules/tests.mdc))
7. **Configuration**: Update configuration settings or add new ones in [`config-manager.js`](mdc:scripts/modules/config-manager.js) and ensure getters/setters are appropriate. Update documentation in [`utilities.mdc`](mdc:.cursor/rules/utilities.mdc) and [`taskmaster.mdc`](mdc:.cursor/rules/taskmaster.mdc). Update the `.taskmasterconfig` structure if needed.
8. **Documentation**: Update help text and documentation in [`dev_workflow.mdc`](mdc:.cursor/rules/dev_workflow.mdc) and [`taskmaster.mdc`](mdc:.cursor/rules/taskmaster.mdc).
## Critical Checklist for New Features

View File

@@ -150,4 +150,91 @@ alwaysApply: false
));
```
Refer to [`ui.js`](mdc:scripts/modules/ui.js) for implementation examples and [`new_features.mdc`](mdc:.cursor/rules/new_features.mdc) for integration guidelines.
## Enhanced Display Patterns
### **Token Breakdown Display**
- Use detailed, granular token breakdowns for AI-powered commands
- Display context sources with individual token counts
- Show both token count and character count for transparency
```javascript
// ✅ DO: Display detailed token breakdown
function displayDetailedTokenBreakdown(tokenBreakdown, systemTokens, userTokens) {
const sections = [];
if (tokenBreakdown.tasks?.length > 0) {
const taskDetails = tokenBreakdown.tasks.map(task =>
`${task.type === 'subtask' ? ' ' : ''}${task.id}: ${task.tokens.toLocaleString()}`
).join('\n');
sections.push(`Tasks (${tokenBreakdown.tasks.reduce((sum, t) => sum + t.tokens, 0).toLocaleString()}):\n${taskDetails}`);
}
const content = sections.join('\n\n');
console.log(boxen(content, {
title: chalk.cyan('Token Usage'),
padding: { top: 1, bottom: 1, left: 2, right: 2 },
borderStyle: 'round',
borderColor: 'cyan'
}));
}
```
### **Code Block Syntax Highlighting**
- Use `cli-highlight` library for syntax highlighting in terminal output
- Process code blocks in AI responses for better readability
```javascript
// ✅ DO: Enhance code blocks with syntax highlighting
import { highlight } from 'cli-highlight';
function processCodeBlocks(text) {
return text.replace(/```(\w+)?\n([\s\S]*?)```/g, (match, language, code) => {
try {
const highlighted = highlight(code.trim(), {
language: language || 'javascript',
theme: 'default'
});
return `\n${highlighted}\n`;
} catch (error) {
return `\n${code.trim()}\n`;
}
});
}
```
### **Multi-Section Result Display**
- Use separate boxes for headers, content, and metadata
- Maintain consistent styling across different result types
```javascript
// ✅ DO: Use structured result display
function displayResults(result, query, detailLevel) {
// Header with query info
const header = boxen(
chalk.green.bold('Research Results') + '\n\n' +
chalk.gray('Query: ') + chalk.white(query) + '\n' +
chalk.gray('Detail Level: ') + chalk.cyan(detailLevel),
{
padding: { top: 1, bottom: 1, left: 2, right: 2 },
margin: { top: 1, bottom: 0 },
borderStyle: 'round',
borderColor: 'green'
}
);
console.log(header);
// Process and display main content
const processedResult = processCodeBlocks(result);
const contentBox = boxen(processedResult, {
padding: { top: 1, bottom: 1, left: 2, right: 2 },
margin: { top: 0, bottom: 1 },
borderStyle: 'single',
borderColor: 'gray'
});
console.log(contentBox);
console.log(chalk.green('✓ Operation complete'));
}
```
Refer to [`ui.js`](mdc:scripts/modules/ui.js) for implementation examples, [`context_gathering.mdc`](mdc:.cursor/rules/context_gathering.mdc) for context display patterns, and [`new_features.mdc`](mdc:.cursor/rules/new_features.mdc) for integration guidelines.

View File

@@ -46,7 +46,7 @@ alwaysApply: false
- **Location**:
- **Core CLI Utilities**: Place utilities used primarily by the core `task-master` CLI logic and command modules (`scripts/modules/*`) into [`scripts/modules/utils.js`](mdc:scripts/modules/utils.js).
- **MCP Server Utilities**: Place utilities specifically designed to support the MCP server implementation into the appropriate subdirectories within `mcp-server/src/`.
- Path/Core Logic Helpers: [`mcp-server/src/core/utils/`](mdc:mcp-server/src/core/utils/) (e.g., `path-utils.js`).
- Path/Core Logic Helpers: [`mcp-server/src/core/utils/`](mdc:mcp-server/src/core/utils) (e.g., `path-utils.js`).
- Tool Execution/Response Helpers: [`mcp-server/src/tools/utils.js`](mdc:mcp-server/src/tools/utils.js).
## Documentation Standards
@@ -110,7 +110,7 @@ Taskmaster configuration (excluding API keys) is primarily managed through the `
- ✅ DO: Use appropriate icons for different log levels
- ✅ DO: Respect the configured log level
- ❌ DON'T: Add direct console.log calls outside the logging utility
- **Note on Passed Loggers**: When a logger object (like the FastMCP `log` object) is passed *as a parameter* (e.g., as `mcpLog`) into core Task Master functions, the receiving function often expects specific methods (`.info`, `.warn`, `.error`, etc.) to be directly callable on that object (e.g., `mcpLog[level](...)`). If the passed logger doesn't have this exact structure, a wrapper object may be needed. See the **Handling Logging Context (`mcpLog`)** section in [`mcp.mdc`](mdc:.cursor/rules/mcp.mdc) for the standard pattern used in direct functions.
- **Note on Passed Loggers**: When a logger object (like the FastMCP `log` object) is passed *as a parameter* (e.g., as `mcpLog`) into core Task Master functions, the receiving function often expects specific methods (`.info`, `.warn`, `.error`, etc.) to be directly callable on that object (e.g., `mcpLog[level](mdc:...)`). If the passed logger doesn't have this exact structure, a wrapper object may be needed. See the **Handling Logging Context (`mcpLog`)** section in [`mcp.mdc`](mdc:.cursor/rules/mcp.mdc) for the standard pattern used in direct functions.
- **Logger Wrapper Pattern**:
- ✅ DO: Use the logger wrapper pattern when passing loggers to prevent `mcpLog[level] is not a function` errors:
@@ -548,4 +548,56 @@ export {
};
```
Refer to [`mcp.mdc`](mdc:.cursor/rules/mcp.mdc) and [`architecture.mdc`](mdc:.cursor/rules/architecture.mdc) for more context on MCP server architecture and integration.
## Context Gathering Utilities
### **ContextGatherer** (`scripts/modules/utils/contextGatherer.js`)
- **Multi-Source Context Extraction**:
- ✅ DO: Use for AI-powered commands that need project context
- ✅ DO: Support tasks, files, custom text, and project tree context
- ✅ DO: Implement detailed token counting with `gpt-tokens` library
- ✅ DO: Provide multiple output formats (research, chat, system-prompt)
```javascript
// ✅ DO: Use ContextGatherer for consistent context extraction
import { ContextGatherer } from '../utils/contextGatherer.js';
const gatherer = new ContextGatherer(projectRoot, tasksPath);
const result = await gatherer.gather({
tasks: ['15', '16.2'],
files: ['src/api.js'],
customContext: 'Additional context',
includeProjectTree: true,
format: 'research',
includeTokenCounts: true
});
```
### **FuzzyTaskSearch** (`scripts/modules/utils/fuzzyTaskSearch.js`)
- **Intelligent Task Discovery**:
- ✅ DO: Use for automatic task relevance detection
- ✅ DO: Configure search parameters based on use case context
- ✅ DO: Implement purpose-based categorization for better matching
- ✅ DO: Sort results by relevance score and task ID
```javascript
// ✅ DO: Use FuzzyTaskSearch for intelligent task discovery
import { FuzzyTaskSearch } from '../utils/fuzzyTaskSearch.js';
const fuzzySearch = new FuzzyTaskSearch(tasksData.tasks, 'research');
const searchResults = fuzzySearch.findRelevantTasks(query, {
maxResults: 8,
includeRecent: true,
includeCategoryMatches: true
});
const taskIds = fuzzySearch.getTaskIds(searchResults);
```
- **Integration Guidelines**:
- ✅ DO: Use fuzzy search to supplement user-provided task IDs
- ✅ DO: Display discovered task IDs to users for transparency
- ✅ DO: Sort discovered task IDs numerically for better readability
- ❌ DON'T: Replace explicit user task selections with fuzzy results
Refer to [`context_gathering.mdc`](mdc:.cursor/rules/context_gathering.mdc) for detailed implementation patterns, [`mcp.mdc`](mdc:.cursor/rules/mcp.mdc) and [`architecture.mdc`](mdc:.cursor/rules/architecture.mdc) for more context on MCP server architecture and integration.