Task management, research, improvements for 24, 41 and 51

This commit is contained in:
Eyal Toledano
2025-05-23 17:30:25 -04:00
parent 43ad75c7fa
commit 8c69c0aafd
4 changed files with 1124 additions and 50 deletions

View File

@@ -3,56 +3,89 @@
# Status: pending
# Dependencies: None
# Priority: medium
# Description: Create a command that allows users to quickly research topics using Perplexity AI, with options to include task context or custom prompts.
# Description: Create an interactive REPL-style chat interface for AI-powered research that maintains conversation context, integrates project information, and provides session management capabilities.
# Details:
Develop a new command called 'research' that integrates with Perplexity AI's API to fetch information on specified topics. The command should:
Develop an interactive REPL-style chat interface for AI-powered research that allows users to have ongoing research conversations with context awareness. The system should:
1. Accept the following parameters:
- A search query string (required)
- A task or subtask ID for context (optional)
- A custom prompt to guide the research (optional)
1. Create an interactive REPL using inquirer that:
- Maintains conversation history and context
- Provides a natural chat-like experience
- Supports special commands with the '/' prefix
2. When a task/subtask ID is provided, extract relevant information from it to enrich the research query with context.
2. Integrate with the existing ai-services-unified.js using research mode:
- Leverage our unified AI service architecture
- Configure appropriate system prompts for research context
- Handle streaming responses for real-time feedback
3. Implement proper API integration with Perplexity, including authentication and rate limiting handling.
3. Support multiple context sources:
- Task/subtask IDs for project context
- File paths for code or document context
- Custom prompts for specific research directions
- Project file tree for system context
4. Format and display the research results in a readable format in the terminal, with options to:
- Save the results to a file
- Copy results to clipboard
- Generate a summary of key points
4. Implement chat commands including:
- `/save` - Save conversation to file
- `/task` - Associate with or load context from a task
- `/help` - Show available commands and usage
- `/exit` - End the research session
- `/copy` - Copy last response to clipboard
- `/summary` - Generate summary of conversation
- `/detail` - Adjust research depth level
5. Cache research results to avoid redundant API calls for the same queries.
5. Create session management capabilities:
- Generate and track unique session IDs
- Save/load sessions automatically
- Browse and switch between previous sessions
- Export sessions to portable formats
6. Provide a configuration option to set the depth/detail level of research (quick overview vs. comprehensive).
6. Design a consistent UI using ui.js patterns:
- Color-coded messages for user/AI distinction
- Support for markdown rendering in terminal
- Progressive display of AI responses
- Clear visual hierarchy and readability
7. Handle errors gracefully, especially network issues or API limitations.
7. Follow the "taskmaster way":
- Create something new and exciting
- Focus on usefulness and practicality
- Avoid over-engineering
- Maintain consistency with existing patterns
The command should follow the existing CLI structure and maintain consistency with other commands in the system.
The REPL should feel like a natural conversation while providing powerful research capabilities that integrate seamlessly with the rest of the system.
# Test Strategy:
1. Unit tests:
- Test the command with various combinations of parameters (query only, query+task, query+custom prompt, all parameters)
- Mock the Perplexity API responses to test different scenarios (successful response, error response, rate limiting)
- Verify that task context is correctly extracted and incorporated into the research query
- Test the REPL command parsing and execution
- Mock AI service responses to test different scenarios
- Verify context extraction and integration from various sources
- Test session serialization and deserialization
2. Integration tests:
- Test actual API calls to Perplexity with valid credentials (using a test account)
- Verify the caching mechanism works correctly for repeated queries
- Test error handling with intentionally invalid requests
- Test actual AI service integration with the REPL
- Verify session persistence across application restarts
- Test conversation state management with long interactions
- Verify context switching between different tasks and files
3. User acceptance testing:
- Have team members use the command for real research needs and provide feedback
- Verify the command works in different network environments
- Test the command with very long queries and responses
- Have team members use the REPL for real research needs
- Test the conversation flow and command usability
- Verify the UI is intuitive and responsive
- Test with various terminal sizes and environments
4. Performance testing:
- Measure and optimize response time for queries
- Test behavior under poor network conditions
- Test behavior with large conversation histories
- Verify performance with complex context sources
- Test under poor network conditions
Validate that the research results are properly formatted, readable, and that all output options (save, copy) function correctly.
5. Specific test scenarios:
- Verify markdown rendering for complex formatting
- Test streaming display with various response lengths
- Verify export features create properly formatted files
- Test session recovery from simulated crashes
- Validate handling of special characters and unicode
# Subtasks:
## 1. Create Perplexity API Client Service [pending]
## 1. Create Perplexity API Client Service [cancelled]
### Dependencies: None
### Description: Develop a service module that handles all interactions with the Perplexity AI API, including authentication, request formatting, and response handling.
### Details:
@@ -72,6 +105,9 @@ Testing approach:
- Test error handling with simulated network failures
- Verify caching mechanism works correctly
- Test with various query types and options
<info added on 2025-05-23T21:06:45.726Z>
DEPRECATION NOTICE: This subtask is no longer needed and has been marked for removal. Instead of creating a new Perplexity service, we will leverage the existing ai-services-unified.js with research mode. This approach allows us to maintain a unified architecture for AI services rather than implementing a separate service specifically for Perplexity.
</info added on 2025-05-23T21:06:45.726Z>
## 2. Implement Task Context Extraction Logic [pending]
### Dependencies: None
@@ -94,6 +130,37 @@ Testing approach:
- Test with various task structures and content types
- Verify error handling for missing or invalid tasks
- Test the quality of extracted context with sample queries
<info added on 2025-05-23T21:11:44.560Z>
Updated Implementation Approach:
REFACTORED IMPLEMENTATION:
1. Extract the fuzzy search logic from add-task.js (lines ~240-400) into `utils/contextExtractor.js`
2. Implement a reusable `TaskContextExtractor` class with the following methods:
- `extractTaskContext(taskId)` - Base context extraction
- `performFuzzySearch(query, options)` - Enhanced Fuse.js implementation
- `getRelevanceScore(task, query)` - Scoring mechanism from add-task.js
- `detectPurposeCategories(task)` - Category classification logic
- `findRelatedTasks(taskId)` - Identify dependencies and relationships
- `aggregateMultiQueryContext(queries)` - Support for multiple search terms
3. Add configurable context depth levels:
- Minimal: Just task title and description
- Standard: Include details and immediate relationships
- Comprehensive: Full context with all dependencies and related tasks
4. Implement context formatters:
- `formatForSystemPrompt(context)` - Structured for AI system instructions
- `formatForChatContext(context)` - Conversational format for chat
- `formatForResearchQuery(context, query)` - Optimized for research commands
5. Add caching layer for performance optimization:
- Implement LRU cache for expensive fuzzy search results
- Cache invalidation on task updates
6. Ensure backward compatibility with existing context extraction requirements
This approach leverages our existing sophisticated search logic rather than rebuilding from scratch, while making it more flexible and reusable across the application.
</info added on 2025-05-23T21:11:44.560Z>
## 3. Build Research Command CLI Interface [pending]
### Dependencies: 51.1, 51.2
@@ -120,6 +187,40 @@ Testing approach:
- Verify command validation logic works correctly
- Test with various combinations of options
- Ensure proper error messages for invalid inputs
<info added on 2025-05-23T21:09:08.478Z>
Implementation details:
1. Create a new module `repl/research-chat.js` for the interactive research experience
2. Implement REPL-style chat interface using inquirer with:
- Persistent conversation history management
- Context-aware prompting system
- Command parsing for special instructions
3. Implement REPL commands:
- `/save` - Save conversation to file
- `/task` - Associate with or load context from a task
- `/help` - Show available commands and usage
- `/exit` - End the research session
- `/copy` - Copy last response to clipboard
- `/summary` - Generate summary of conversation
- `/detail` - Adjust research depth level
4. Create context initialization system:
- Task/subtask context loading
- File content integration
- System prompt configuration
5. Integrate with ai-services-unified.js research mode
6. Implement conversation state management:
- Track message history
- Maintain context window
- Handle context pruning for long conversations
7. Design consistent UI patterns using ui.js library
8. Add entry point in main CLI application
Testing approach:
- Test REPL command parsing and execution
- Verify context initialization with various inputs
- Test conversation state management
- Ensure proper error handling and recovery
- Validate UI consistency across different terminal environments
</info added on 2025-05-23T21:09:08.478Z>
## 4. Implement Results Processing and Output Formatting [pending]
### Dependencies: 51.1, 51.3
@@ -145,8 +246,45 @@ Testing approach:
- Verify file saving functionality creates proper files with correct content
- Test clipboard functionality
- Verify summarization produces useful results
<info added on 2025-05-23T21:10:00.181Z>
Implementation details:
1. Create a new module `utils/chatFormatter.js` for REPL interface formatting
2. Implement terminal output formatting for conversational display:
- Color-coded messages distinguishing user inputs and AI responses
- Proper text wrapping and indentation for readability
- Support for markdown rendering in terminal
- Visual indicators for system messages and status updates
3. Implement streaming/progressive display of AI responses:
- Character-by-character or chunk-by-chunk display
- Cursor animations during response generation
- Ability to interrupt long responses
4. Design chat history visualization:
- Scrollable history with clear message boundaries
- Timestamp display options
- Session identification
5. Create specialized formatters for different content types:
- Code blocks with syntax highlighting
- Bulleted and numbered lists
- Tables and structured data
- Citations and references
6. Implement export functionality:
- Save conversations to markdown or text files
- Export individual responses
- Copy responses to clipboard
7. Adapt existing ui.js patterns for conversational context:
- Maintain consistent styling while supporting chat flow
- Handle multi-turn context appropriately
## 5. Implement Caching and Results Management System [pending]
Testing approach:
- Test streaming display with various response lengths and speeds
- Verify markdown rendering accuracy for complex formatting
- Test history navigation and scrolling functionality
- Verify export features create properly formatted files
- Test display on various terminal sizes and configurations
- Verify handling of special characters and unicode
</info added on 2025-05-23T21:10:00.181Z>
## 5. Implement Caching and Results Management System [cancelled]
### Dependencies: 51.1, 51.4
### Description: Create a persistent caching system for research results and implement functionality to manage, retrieve, and reference previous research.
### Details:
@@ -173,4 +311,142 @@ Testing approach:
- Test history management commands
- Verify task association functionality
- Test with large cache sizes to ensure performance
<info added on 2025-05-23T21:10:28.544Z>
Implementation details:
1. Create a session management system for the REPL experience:
- Generate and track unique session IDs
- Store conversation history with timestamps
- Maintain context and state between interactions
2. Implement session persistence:
- Save sessions to disk automatically
- Load previous sessions on startup
- Handle graceful recovery from crashes
3. Build session browser and selector:
- List available sessions with preview
- Filter sessions by date, topic, or content
- Enable quick switching between sessions
4. Implement conversation state serialization:
- Capture full conversation context
- Preserve user preferences per session
- Handle state migration during updates
5. Add session sharing capabilities:
- Export sessions to portable formats
- Import sessions from files
- Generate shareable links (if applicable)
6. Create session management commands:
- Create new sessions
- Clone existing sessions
- Archive or delete old sessions
Testing approach:
- Verify session persistence across application restarts
- Test session recovery from simulated crashes
- Validate state serialization with complex conversations
- Ensure session switching maintains proper context
- Test session import/export functionality
- Verify performance with large conversation histories
</info added on 2025-05-23T21:10:28.544Z>
## 6. Implement Project Context Generation [pending]
### Dependencies: 51.2
### Description: Create functionality to generate and include project-level context such as file trees, repository structure, and codebase insights for more informed research.
### Details:
Implementation details:
1. Create a new module `utils/projectContextGenerator.js` for project-level context extraction
2. Implement file tree generation functionality:
- Scan project directory structure recursively
- Filter out irrelevant files (node_modules, .git, etc.)
- Format file tree for AI consumption
- Include file counts and structure statistics
3. Add code analysis capabilities:
- Extract key imports and dependencies
- Identify main modules and their relationships
- Generate high-level architecture overview
4. Implement context summarization:
- Create concise project overview
- Identify key technologies and patterns
- Summarize project purpose and structure
5. Add caching for expensive operations:
- Cache file tree with invalidation on changes
- Store analysis results with TTL
6. Create integration with research REPL:
- Add project context to system prompts
- Support `/project` command to refresh context
- Allow selective inclusion of project components
Testing approach:
- Test file tree generation with various project structures
- Verify filtering logic works correctly
- Test context summarization quality
- Measure performance impact of context generation
- Verify caching mechanism effectiveness
## 7. Create REPL Command System [pending]
### Dependencies: 51.3
### Description: Implement a flexible command system for the research REPL that allows users to control the conversation flow, manage sessions, and access additional functionality.
### Details:
Implementation details:
1. Create a new module `repl/commands.js` for REPL command handling
2. Implement a command parser that:
- Detects commands starting with `/`
- Parses arguments and options
- Handles quoted strings and special characters
3. Create a command registry system:
- Register command handlers with descriptions
- Support command aliases
- Enable command discovery and help
4. Implement core commands:
- `/save [filename]` - Save conversation
- `/task <taskId>` - Load task context
- `/file <path>` - Include file content
- `/help [command]` - Show help
- `/exit` - End session
- `/copy [n]` - Copy nth response
- `/summary` - Generate conversation summary
- `/detail <level>` - Set detail level
- `/clear` - Clear conversation
- `/project` - Refresh project context
- `/session <id|new>` - Switch/create session
5. Add command completion and suggestions
6. Implement error handling for invalid commands
7. Create a help system with examples
Testing approach:
- Test command parsing with various inputs
- Verify command execution and error handling
- Test command completion functionality
- Verify help system provides useful information
- Test with complex command sequences
## 8. Integrate with AI Services Unified [pending]
### Dependencies: 51.3, 51.4
### Description: Integrate the research REPL with the existing ai-services-unified.js to leverage the unified AI service architecture with research mode.
### Details:
Implementation details:
1. Update `repl/research-chat.js` to integrate with ai-services-unified.js
2. Configure research mode in AI service:
- Set appropriate system prompts
- Configure temperature and other parameters
- Enable streaming responses
3. Implement context management:
- Format conversation history for AI context
- Include task and project context
- Handle context window limitations
4. Add support for different research styles:
- Exploratory research with broader context
- Focused research with specific questions
- Comparative analysis between concepts
5. Implement response handling:
- Process streaming chunks
- Format and display responses
- Handle errors and retries
6. Add configuration options for AI service selection
7. Implement fallback mechanisms for service unavailability
Testing approach:
- Test integration with mocked AI services
- Verify context formatting and management
- Test streaming response handling
- Verify error handling and recovery
- Test with various research styles and queries