claude-task-master/.taskmaster/tasks/task_051.txt

# Task ID: 51
# Title: Implement Perplexity Research Command
# Status: pending
# Dependencies: None
# Priority: medium
# Description: Create an interactive REPL-style chat interface for AI-powered research that maintains conversation context, integrates project information, and provides session management capabilities.
# Details:
Develop an interactive REPL-style chat interface for AI-powered research that allows users to have ongoing research conversations with context awareness. The system should:

1. Create an interactive REPL using inquirer that:
   - Maintains conversation history and context
   - Provides a natural chat-like experience
   - Supports special commands with the '/' prefix

2. Integrate with the existing ai-services-unified.js using research mode:
   - Leverage our unified AI service architecture
   - Configure appropriate system prompts for research context
   - Handle streaming responses for real-time feedback

3. Support multiple context sources:
   - Task/subtask IDs for project context
   - File paths for code or document context
   - Custom prompts for specific research directions
   - Project file tree for system context

4. Implement chat commands including:
   - `/save` - Save conversation to file
   - `/task` - Associate with or load context from a task
   - `/help` - Show available commands and usage
   - `/exit` - End the research session
   - `/copy` - Copy last response to clipboard
   - `/summary` - Generate summary of conversation
   - `/detail` - Adjust research depth level

5. Create session management capabilities:
   - Generate and track unique session IDs
   - Save/load sessions automatically
   - Browse and switch between previous sessions
   - Export sessions to portable formats

6. Design a consistent UI using ui.js patterns:
   - Color-coded messages for user/AI distinction
   - Support for markdown rendering in terminal
   - Progressive display of AI responses
   - Clear visual hierarchy and readability

7. Follow the "taskmaster way":
   - Create something new and exciting
   - Focus on usefulness and practicality
   - Avoid over-engineering
   - Maintain consistency with existing patterns

The REPL should feel like a natural conversation while providing powerful research capabilities that integrate seamlessly with the rest of the system.

# Test Strategy:
1. Unit tests:
   - Test the REPL command parsing and execution
   - Mock AI service responses to test different scenarios
   - Verify context extraction and integration from various sources
   - Test session serialization and deserialization

2. Integration tests:
   - Test actual AI service integration with the REPL
   - Verify session persistence across application restarts
   - Test conversation state management with long interactions
   - Verify context switching between different tasks and files

3. User acceptance testing:
   - Have team members use the REPL for real research needs
   - Test the conversation flow and command usability
   - Verify the UI is intuitive and responsive
   - Test with various terminal sizes and environments

4. Performance testing:
   - Measure and optimize response time for queries
   - Test behavior with large conversation histories
   - Verify performance with complex context sources
   - Test under poor network conditions

5. Specific test scenarios:
   - Verify markdown rendering for complex formatting
   - Test streaming display with various response lengths
   - Verify export features create properly formatted files
   - Test session recovery from simulated crashes
   - Validate handling of special characters and unicode

# Subtasks:
## 1. Create Perplexity API Client Service [cancelled]
### Dependencies: None
### Description: Develop a service module that handles all interactions with the Perplexity AI API, including authentication, request formatting, and response handling.
### Details:
Implementation details:
1. Create a new service file `services/perplexityService.js`
2. Implement authentication using the PERPLEXITY_API_KEY from environment variables
3. Create functions for making API requests to Perplexity with proper error handling:
   - `queryPerplexity(searchQuery, options)` - Main function to query the API
   - `handleRateLimiting(response)` - Logic to handle rate limits with exponential backoff
4. Implement response parsing and formatting functions
5. Add proper error handling for network issues, authentication problems, and API limitations
6. Create a simple caching mechanism using a Map or object to store recent query results
7. Add configuration options for different detail levels (quick vs comprehensive)

Testing approach:
- Write unit tests using Jest to verify API client functionality with mocked responses
- Test error handling with simulated network failures
- Verify caching mechanism works correctly
- Test with various query types and options
<info added on 2025-05-23T21:06:45.726Z>
DEPRECATION NOTICE: This subtask is no longer needed and has been marked for removal. Instead of creating a new Perplexity service, we will leverage the existing ai-services-unified.js with research mode. This approach allows us to maintain a unified architecture for AI services rather than implementing a separate service specifically for Perplexity.
</info added on 2025-05-23T21:06:45.726Z>

## 2. Implement Task Context Extraction Logic [pending]
### Dependencies: None
### Description: Create utility functions to extract relevant context from tasks and subtasks to enhance research queries with project-specific information.
### Details:
Implementation details:
1. Create a new utility file `utils/contextExtractor.js`
2. Implement a function `extractTaskContext(taskId)` that:
   - Loads the task/subtask data from tasks.json
   - Extracts relevant information (title, description, details)
   - Formats the extracted information into a context string for research
3. Add logic to handle both task and subtask IDs
4. Implement a function to combine extracted context with the user's search query
5. Create a function to identify and extract key terminology from tasks
6. Add functionality to include parent task context when a subtask ID is provided
7. Implement proper error handling for invalid task IDs

Testing approach:
- Write unit tests to verify context extraction from sample tasks
- Test with various task structures and content types
- Verify error handling for missing or invalid tasks
- Test the quality of extracted context with sample queries
<info added on 2025-05-23T21:11:44.560Z>
Updated Implementation Approach:

REFACTORED IMPLEMENTATION:
1. Extract the fuzzy search logic from add-task.js (lines ~240-400) into `utils/contextExtractor.js`
2. Implement a reusable `TaskContextExtractor` class with the following methods:
   - `extractTaskContext(taskId)` - Base context extraction
   - `performFuzzySearch(query, options)` - Enhanced Fuse.js implementation
   - `getRelevanceScore(task, query)` - Scoring mechanism from add-task.js
   - `detectPurposeCategories(task)` - Category classification logic
   - `findRelatedTasks(taskId)` - Identify dependencies and relationships
   - `aggregateMultiQueryContext(queries)` - Support for multiple search terms

3. Add configurable context depth levels:
   - Minimal: Just task title and description
   - Standard: Include details and immediate relationships
   - Comprehensive: Full context with all dependencies and related tasks

4. Implement context formatters:
   - `formatForSystemPrompt(context)` - Structured for AI system instructions
   - `formatForChatContext(context)` - Conversational format for chat
   - `formatForResearchQuery(context, query)` - Optimized for research commands

5. Add caching layer for performance optimization:
   - Implement LRU cache for expensive fuzzy search results
   - Cache invalidation on task updates

6. Ensure backward compatibility with existing context extraction requirements

This approach leverages our existing sophisticated search logic rather than rebuilding from scratch, while making it more flexible and reusable across the application.
</info added on 2025-05-23T21:11:44.560Z>

## 3. Build Research Command CLI Interface [pending]
### Dependencies: 51.1, 51.2
### Description: Implement the Commander.js command structure for the 'research' command with all required options and parameters.
### Details:
Implementation details:
1. Create a new command file `commands/research.js`
2. Set up the Commander.js command structure with the following options:
   - Required search query parameter
   - `--task` or `-t` option for task/subtask ID
   - `--prompt` or `-p` option for custom research prompt
   - `--save` or `-s` option to save results to a file
   - `--copy` or `-c` option to copy results to clipboard
   - `--summary` or `-m` option to generate a summary
   - `--detail` or `-d` option to set research depth (default: medium)
3. Implement command validation logic
4. Connect the command to the Perplexity service created in subtask 1
5. Integrate the context extraction logic from subtask 2
6. Register the command in the main CLI application
7. Add help text and examples

Testing approach:
- Test command registration and option parsing
- Verify command validation logic works correctly
- Test with various combinations of options
- Ensure proper error messages for invalid inputs
<info added on 2025-05-23T21:09:08.478Z>
Implementation details:
1. Create a new module `repl/research-chat.js` for the interactive research experience
2. Implement REPL-style chat interface using inquirer with:
   - Persistent conversation history management
   - Context-aware prompting system
   - Command parsing for special instructions
3. Implement REPL commands:
   - `/save` - Save conversation to file
   - `/task` - Associate with or load context from a task
   - `/help` - Show available commands and usage
   - `/exit` - End the research session
   - `/copy` - Copy last response to clipboard
   - `/summary` - Generate summary of conversation
   - `/detail` - Adjust research depth level
4. Create context initialization system:
   - Task/subtask context loading
   - File content integration
   - System prompt configuration
5. Integrate with ai-services-unified.js research mode
6. Implement conversation state management:
   - Track message history
   - Maintain context window
   - Handle context pruning for long conversations
7. Design consistent UI patterns using ui.js library
8. Add entry point in main CLI application

Testing approach:
- Test REPL command parsing and execution
- Verify context initialization with various inputs
- Test conversation state management
- Ensure proper error handling and recovery
- Validate UI consistency across different terminal environments
</info added on 2025-05-23T21:09:08.478Z>

## 4. Implement Results Processing and Output Formatting [pending]
### Dependencies: 51.1, 51.3
### Description: Create functionality to process, format, and display research results in the terminal with options for saving, copying, and summarizing.
### Details:
Implementation details:
1. Create a new module `utils/researchFormatter.js`
2. Implement terminal output formatting with:
   - Color-coded sections for better readability
   - Proper text wrapping for terminal width
   - Highlighting of key points
3. Add functionality to save results to a file:
   - Create a `research-results` directory if it doesn't exist
   - Save results with timestamp and query in filename
   - Support multiple formats (text, markdown, JSON)
4. Implement clipboard copying using a library like `clipboardy`
5. Create a summarization function that extracts key points from research results
6. Add progress indicators during API calls
7. Implement pagination for long results

Testing approach:
- Test output formatting with various result lengths and content types
- Verify file saving functionality creates proper files with correct content
- Test clipboard functionality
- Verify summarization produces useful results
<info added on 2025-05-23T21:10:00.181Z>
Implementation details:
1. Create a new module `utils/chatFormatter.js` for REPL interface formatting
2. Implement terminal output formatting for conversational display:
   - Color-coded messages distinguishing user inputs and AI responses
   - Proper text wrapping and indentation for readability
   - Support for markdown rendering in terminal
   - Visual indicators for system messages and status updates
3. Implement streaming/progressive display of AI responses:
   - Character-by-character or chunk-by-chunk display
   - Cursor animations during response generation
   - Ability to interrupt long responses
4. Design chat history visualization:
   - Scrollable history with clear message boundaries
   - Timestamp display options
   - Session identification
5. Create specialized formatters for different content types:
   - Code blocks with syntax highlighting
   - Bulleted and numbered lists
   - Tables and structured data
   - Citations and references
6. Implement export functionality:
   - Save conversations to markdown or text files
   - Export individual responses
   - Copy responses to clipboard
7. Adapt existing ui.js patterns for conversational context:
   - Maintain consistent styling while supporting chat flow
   - Handle multi-turn context appropriately

Testing approach:
- Test streaming display with various response lengths and speeds
- Verify markdown rendering accuracy for complex formatting
- Test history navigation and scrolling functionality
- Verify export features create properly formatted files
- Test display on various terminal sizes and configurations
- Verify handling of special characters and unicode
</info added on 2025-05-23T21:10:00.181Z>

## 5. Implement Caching and Results Management System [cancelled]
### Dependencies: 51.1, 51.4
### Description: Create a persistent caching system for research results and implement functionality to manage, retrieve, and reference previous research.
### Details:
Implementation details:
1. Create a research results database using a simple JSON file or SQLite:
   - Store queries, timestamps, and results
   - Index by query and related task IDs
2. Implement cache retrieval and validation:
   - Check for cached results before making API calls
   - Validate cache freshness with configurable TTL
3. Add commands to manage research history:
   - List recent research queries
   - Retrieve past research by ID or search term
   - Clear cache or delete specific entries
4. Create functionality to associate research results with tasks:
   - Add metadata linking research to specific tasks
   - Implement command to show all research related to a task
5. Add configuration options for cache behavior in user settings
6. Implement export/import functionality for research data

Testing approach:
- Test cache storage and retrieval with various queries
- Verify cache invalidation works correctly
- Test history management commands
- Verify task association functionality
- Test with large cache sizes to ensure performance
<info added on 2025-05-23T21:10:28.544Z>
Implementation details:
1. Create a session management system for the REPL experience:
   - Generate and track unique session IDs
   - Store conversation history with timestamps
   - Maintain context and state between interactions
2. Implement session persistence:
   - Save sessions to disk automatically
   - Load previous sessions on startup
   - Handle graceful recovery from crashes
3. Build session browser and selector:
   - List available sessions with preview
   - Filter sessions by date, topic, or content
   - Enable quick switching between sessions
4. Implement conversation state serialization:
   - Capture full conversation context
   - Preserve user preferences per session
   - Handle state migration during updates
5. Add session sharing capabilities:
   - Export sessions to portable formats
   - Import sessions from files
   - Generate shareable links (if applicable)
6. Create session management commands:
   - Create new sessions
   - Clone existing sessions
   - Archive or delete old sessions

Testing approach:
- Verify session persistence across application restarts
- Test session recovery from simulated crashes
- Validate state serialization with complex conversations
- Ensure session switching maintains proper context
- Test session import/export functionality
- Verify performance with large conversation histories
</info added on 2025-05-23T21:10:28.544Z>

## 6. Implement Project Context Generation [pending]
### Dependencies: 51.2
### Description: Create functionality to generate and include project-level context such as file trees, repository structure, and codebase insights for more informed research.
### Details:
Implementation details:
1. Create a new module `utils/projectContextGenerator.js` for project-level context extraction
2. Implement file tree generation functionality:
   - Scan project directory structure recursively
   - Filter out irrelevant files (node_modules, .git, etc.)
   - Format file tree for AI consumption
   - Include file counts and structure statistics
3. Add code analysis capabilities:
   - Extract key imports and dependencies
   - Identify main modules and their relationships
   - Generate high-level architecture overview
4. Implement context summarization:
   - Create concise project overview
   - Identify key technologies and patterns
   - Summarize project purpose and structure
5. Add caching for expensive operations:
   - Cache file tree with invalidation on changes
   - Store analysis results with TTL
6. Create integration with research REPL:
   - Add project context to system prompts
   - Support `/project` command to refresh context
   - Allow selective inclusion of project components

Testing approach:
- Test file tree generation with various project structures
- Verify filtering logic works correctly
- Test context summarization quality
- Measure performance impact of context generation
- Verify caching mechanism effectiveness

## 7. Create REPL Command System [pending]
### Dependencies: 51.3
### Description: Implement a flexible command system for the research REPL that allows users to control the conversation flow, manage sessions, and access additional functionality.
### Details:
Implementation details:
1. Create a new module `repl/commands.js` for REPL command handling
2. Implement a command parser that:
   - Detects commands starting with `/`
   - Parses arguments and options
   - Handles quoted strings and special characters
3. Create a command registry system:
   - Register command handlers with descriptions
   - Support command aliases
   - Enable command discovery and help
4. Implement core commands:
   - `/save [filename]` - Save conversation
   - `/task <taskId>` - Load task context
   - `/file <path>` - Include file content
   - `/help [command]` - Show help
   - `/exit` - End session
   - `/copy [n]` - Copy nth response
   - `/summary` - Generate conversation summary
   - `/detail <level>` - Set detail level
   - `/clear` - Clear conversation
   - `/project` - Refresh project context
   - `/session <id|new>` - Switch/create session
5. Add command completion and suggestions
6. Implement error handling for invalid commands
7. Create a help system with examples

Testing approach:
- Test command parsing with various inputs
- Verify command execution and error handling
- Test command completion functionality
- Verify help system provides useful information
- Test with complex command sequences

## 8. Integrate with AI Services Unified [pending]
### Dependencies: 51.3, 51.4
### Description: Integrate the research REPL with the existing ai-services-unified.js to leverage the unified AI service architecture with research mode.
### Details:
Implementation details:
1. Update `repl/research-chat.js` to integrate with ai-services-unified.js
2. Configure research mode in AI service:
   - Set appropriate system prompts
   - Configure temperature and other parameters
   - Enable streaming responses
3. Implement context management:
   - Format conversation history for AI context
   - Include task and project context
   - Handle context window limitations
4. Add support for different research styles:
   - Exploratory research with broader context
   - Focused research with specific questions
   - Comparative analysis between concepts
5. Implement response handling:
   - Process streaming chunks
   - Format and display responses
   - Handle errors and retries
6. Add configuration options for AI service selection
7. Implement fallback mechanisms for service unavailability

Testing approach:
- Test integration with mocked AI services
- Verify context formatting and management
- Test streaming response handling
- Verify error handling and recovery
- Test with various research styles and queries