claude-task-master/tasks/task_023.txt

# Task ID: 23
# Title: Complete MCP Server Implementation for Task Master using FastMCP
# Status: in-progress
# Dependencies: 22
# Priority: medium
# Description: Finalize the MCP server functionality for Task Master by leveraging FastMCP's capabilities, transitioning from CLI-based execution to direct function imports, and optimizing performance, authentication, and context management. Ensure the server integrates seamlessly with Cursor via `mcp.json` and supports proper tool registration, efficient context handling, and transport type handling (focusing on stdio). Additionally, ensure the server can be instantiated properly when installed via `npx` or `npm i -g`. Evaluate and address gaps in the current implementation, including function imports, context management, caching, tool registration, and adherence to FastMCP best practices.
# Details:
This task involves completing the Model Context Protocol (MCP) server implementation for Task Master using FastMCP. Key updates include:

1. Transition from CLI-based execution (currently using `child_process.spawnSync`) to direct Task Master function imports for improved performance and reliability.
2. Implement caching mechanisms for frequently accessed contexts to enhance performance, leveraging FastMCP's efficient transport mechanisms (e.g., stdio).
3. Refactor context management to align with best practices for handling large context windows, metadata, and tagging.
4. Refactor tool registration in `tools/index.js` to include clear descriptions and parameter definitions, leveraging FastMCP's decorator-based patterns for better integration.
5. Enhance transport type handling to ensure proper stdio communication and compatibility with FastMCP.
6. Ensure the MCP server can be instantiated and run correctly when installed globally via `npx` or `npm i -g`.
7. Integrate the ModelContextProtocol SDK directly to streamline resource and tool registration, ensuring compatibility with FastMCP's transport mechanisms.
8. Identify and address missing components or functionalities to meet FastMCP best practices, such as robust error handling, monitoring endpoints, and concurrency support.
9. Update documentation to include examples of using the MCP server with FastMCP, detailed setup instructions, and client integration guides.

The implementation must ensure compatibility with existing MCP clients and follow RESTful API design principles, while supporting concurrent requests and maintaining robust error handling.

# Test Strategy:
Testing for the MCP server implementation will follow a comprehensive approach based on our established testing guidelines:

## Test Organization

1. **Unit Tests** (`tests/unit/mcp-server/`):
   - Test individual MCP server components in isolation
   - Mock all external dependencies including FastMCP SDK
   - Test each tool implementation separately
   - Verify direct function imports work correctly
   - Test context management and caching mechanisms
   - Example files: `context-manager.test.js`, `tool-registration.test.js`, `direct-imports.test.js`

2. **Integration Tests** (`tests/integration/mcp-server/`):
   - Test interactions between MCP server components
   - Verify proper tool registration with FastMCP
   - Test context flow between components
   - Validate error handling across module boundaries
   - Example files: `server-tool-integration.test.js`, `context-flow.test.js`

3. **End-to-End Tests** (`tests/e2e/mcp-server/`):
   - Test complete MCP server workflows
   - Verify server instantiation via different methods (direct, npx, global install)
   - Test actual stdio communication with mock clients
   - Example files: `server-startup.e2e.test.js`, `client-communication.e2e.test.js`

4. **Test Fixtures** (`tests/fixtures/mcp-server/`):
   - Sample context data
   - Mock tool definitions
   - Sample MCP requests and responses

## Testing Approach

### Module Mocking Strategy
```javascript
// Mock the FastMCP SDK
jest.mock('@model-context-protocol/sdk', () => ({
  MCPServer: jest.fn().mockImplementation(() => ({
    registerTool: jest.fn(),
    registerResource: jest.fn(),
    start: jest.fn().mockResolvedValue(undefined),
    stop: jest.fn().mockResolvedValue(undefined)
  })),
  MCPError: jest.fn().mockImplementation(function(message, code) {
    this.message = message;
    this.code = code;
  })
}));

// Import modules after mocks
import { MCPServer, MCPError } from '@model-context-protocol/sdk';
import { initMCPServer } from '../../scripts/mcp-server.js';
```

### Context Management Testing
- Test context creation, retrieval, and manipulation
- Verify caching mechanisms work correctly
- Test context windowing and metadata handling
- Validate context persistence across server restarts

### Direct Function Import Testing
- Verify Task Master functions are imported correctly
- Test performance improvements compared to CLI execution
- Validate error handling with direct imports

### Tool Registration Testing
- Verify tools are registered with proper descriptions and parameters
- Test decorator-based registration patterns
- Validate tool execution with different input types

### Error Handling Testing
- Test all error paths with appropriate MCPError types
- Verify error propagation to clients
- Test recovery from various error conditions

### Performance Testing
- Benchmark response times with and without caching
- Test memory usage under load
- Verify concurrent request handling

## Test Quality Guidelines

- Follow TDD approach when possible
- Maintain test independence and isolation
- Use descriptive test names explaining expected behavior
- Aim for 80%+ code coverage, with critical paths at 100%
- Follow the mock-first-then-import pattern for all Jest mocks
- Avoid testing implementation details that might change
- Ensure tests don't depend on execution order

## Specific Test Cases

1. **Server Initialization**
   - Test server creation with various configuration options
   - Verify proper tool and resource registration
   - Test server startup and shutdown procedures

2. **Context Operations**
   - Test context creation, retrieval, update, and deletion
   - Verify context windowing and truncation
   - Test context metadata and tagging

3. **Tool Execution**
   - Test each tool with various input parameters
   - Verify proper error handling for invalid inputs
   - Test tool execution performance

4. **MCP.json Integration**
   - Test creation and updating of .cursor/mcp.json
   - Verify proper server registration in mcp.json
   - Test handling of existing mcp.json files

5. **Transport Handling**
   - Test stdio communication
   - Verify proper message formatting
   - Test error handling in transport layer

All tests will be automated and integrated into the CI/CD pipeline to ensure consistent quality.

# Subtasks:
## 1. Create Core MCP Server Module and Basic Structure [done]
### Dependencies: None
### Description: Create the foundation for the MCP server implementation by setting up the core module structure, configuration, and server initialization.
### Details:
Implementation steps:
1. Create a new module `mcp-server.js` with the basic server structure
2. Implement configuration options to enable/disable the MCP server
3. Set up Express.js routes for the required MCP endpoints (/context, /models, /execute)
4. Create middleware for request validation and response formatting
5. Implement basic error handling according to MCP specifications
6. Add logging infrastructure for MCP operations
7. Create initialization and shutdown procedures for the MCP server
8. Set up integration with the main Task Master application

Testing approach:
- Unit tests for configuration loading and validation
- Test server initialization and shutdown procedures
- Verify that routes are properly registered
- Test basic error handling with invalid requests

## 2. Implement Context Management System [done]
### Dependencies: 23.1
### Description: Develop a robust context management system that can efficiently store, retrieve, and manipulate context data according to the MCP specification.
### Details:
Implementation steps:
1. Design and implement data structures for context storage
2. Create methods for context creation, retrieval, updating, and deletion
3. Implement context windowing and truncation algorithms for handling size limits
4. Add support for context metadata and tagging
5. Create utilities for context serialization and deserialization
6. Implement efficient indexing for quick context lookups
7. Add support for context versioning and history
8. Develop mechanisms for context persistence (in-memory, disk-based, or database)

Testing approach:
- Unit tests for all context operations (CRUD)
- Performance tests for context retrieval with various sizes
- Test context windowing and truncation with edge cases
- Verify metadata handling and tagging functionality
- Test persistence mechanisms with simulated failures

## 3. Implement MCP Endpoints and API Handlers [done]
### Dependencies: 23.1, 23.2
### Description: Develop the complete API handlers for all required MCP endpoints, ensuring they follow the protocol specification and integrate with the context management system.
### Details:
Implementation steps:
1. Implement the `/context` endpoint for:
   - GET: retrieving existing context
   - POST: creating new context
   - PUT: updating existing context
   - DELETE: removing context
2. Implement the `/models` endpoint to list available models
3. Develop the `/execute` endpoint for performing operations with context
4. Create request validators for each endpoint
5. Implement response formatters according to MCP specifications
6. Add detailed error handling for each endpoint
7. Set up proper HTTP status codes for different scenarios
8. Implement pagination for endpoints that return lists

Testing approach:
- Unit tests for each endpoint handler
- Integration tests with mock context data
- Test various request formats and edge cases
- Verify response formats match MCP specifications
- Test error handling with invalid inputs
- Benchmark endpoint performance

## 6. Refactor MCP Server to Leverage ModelContextProtocol SDK [deferred]
### Dependencies: 23.1, 23.2, 23.3
### Description: Integrate the ModelContextProtocol SDK directly into the MCP server implementation to streamline tool registration and resource handling.
### Details:
Implementation steps:
1. Replace manual tool registration with ModelContextProtocol SDK methods.
2. Use SDK utilities to simplify resource and template management.
3. Ensure compatibility with FastMCP's transport mechanisms.
4. Update server initialization to include SDK-based configurations.

Testing approach:
- Verify SDK integration with all MCP endpoints.
- Test resource and template registration using SDK methods.
- Validate compatibility with existing MCP clients.
- Benchmark performance improvements from SDK integration.

## 8. Implement Direct Function Imports and Replace CLI-based Execution [in-progress]
### Dependencies: 23.13
### Description: Refactor the MCP server implementation to use direct Task Master function imports instead of the current CLI-based execution using child_process.spawnSync. This will improve performance, reliability, and enable better error handling.
### Details:
1. Create a new module to import and expose Task Master core functions directly
2. Modify tools/utils.js to remove executeTaskMasterCommand and replace with direct function calls
3. Update each tool implementation (listTasks.js, showTask.js, etc.) to use the direct function imports
4. Implement proper error handling with try/catch blocks and FastMCP's MCPError
5. Add unit tests to verify the function imports work correctly
6. Test performance improvements by comparing response times between CLI and function import approaches

## 9. Implement Context Management and Caching Mechanisms [deferred]
### Dependencies: 23.1
### Description: Enhance the MCP server with proper context management and caching to improve performance and user experience, especially for frequently accessed data and contexts.
### Details:
1. Implement a context manager class that leverages FastMCP's Context object
2. Add caching for frequently accessed task data with configurable TTL settings
3. Implement context tagging for better organization of context data
4. Add methods to efficiently handle large context windows
5. Create helper functions for storing and retrieving context data
6. Implement cache invalidation strategies for task updates
7. Add cache statistics for monitoring performance
8. Create unit tests for context management and caching functionality

## 10. Enhance Tool Registration and Resource Management [in-progress]
### Dependencies: 23.1, 23.8
### Description: Refactor tool registration to follow FastMCP best practices, using decorators and improving the overall structure. Implement proper resource management for task templates and other shared resources.
### Details:
1. Update registerTaskMasterTools function to use FastMCP's decorator pattern
2. Implement @mcp.tool() decorators for all existing tools
3. Add proper type annotations and documentation for all tools
4. Create resource handlers for task templates using @mcp.resource()
5. Implement resource templates for common task patterns
6. Update the server initialization to properly register all tools and resources
7. Add validation for tool inputs using FastMCP's built-in validation
8. Create comprehensive tests for tool registration and resource access

## 11. Implement Comprehensive Error Handling [pending]
### Dependencies: 23.1, 23.3
### Description: Implement robust error handling using FastMCP's MCPError, including custom error types for different categories and standardized error responses.
### Details:
1. Create custom error types extending MCPError for different categories (validation, auth, etc.)\n2. Implement standardized error responses following MCP protocol\n3. Add error handling middleware for all MCP endpoints\n4. Ensure proper error propagation from tools to client\n5. Add debug mode with detailed error information\n6. Document error types and handling patterns

## 12. Implement Structured Logging System [pending]
### Dependencies: 23.1, 23.3
### Description: Implement a comprehensive logging system for the MCP server with different log levels, structured logging format, and request/response tracking.
### Details:
1. Design structured log format for consistent parsing\n2. Implement different log levels (debug, info, warn, error)\n3. Add request/response logging middleware\n4. Implement correlation IDs for request tracking\n5. Add performance metrics logging\n6. Configure log output destinations (console, file)\n7. Document logging patterns and usage

## 13. Create Testing Framework and Test Suite [pending]
### Dependencies: 23.1, 23.3
### Description: Implement a comprehensive testing framework for the MCP server, including unit tests, integration tests, and end-to-end tests.
### Details:
1. Set up Jest testing framework with proper configuration\n2. Create MCPTestClient for testing FastMCP server interaction\n3. Implement unit tests for individual tool functions\n4. Create integration tests for end-to-end request/response cycles\n5. Set up test fixtures and mock data\n6. Implement test coverage reporting\n7. Document testing guidelines and examples

## 14. Add MCP.json to the Init Workflow [done]
### Dependencies: 23.1, 23.3
### Description: Implement functionality to create or update .cursor/mcp.json during project initialization, handling cases where: 1) If there's no mcp.json, create it with the appropriate configuration; 2) If there is an mcp.json, intelligently append to it without syntax errors like trailing commas
### Details:
1. Create functionality to detect if .cursor/mcp.json exists in the project\n2. Implement logic to create a new mcp.json file with proper structure if it doesn't exist\n3. Add functionality to read and parse existing mcp.json if it exists\n4. Create method to add a new taskmaster-ai server entry to the mcpServers object\n5. Implement intelligent JSON merging that avoids trailing commas and syntax errors\n6. Ensure proper formatting and indentation in the generated/updated JSON\n7. Add validation to verify the updated configuration is valid JSON\n8. Include this functionality in the init workflow\n9. Add error handling for file system operations and JSON parsing\n10. Document the mcp.json structure and integration process

## 15. Implement SSE Support for Real-time Updates [deferred]
### Dependencies: 23.1, 23.3, 23.11
### Description: Add Server-Sent Events (SSE) capabilities to the MCP server to enable real-time updates and streaming of task execution progress, logs, and status changes to clients
### Details:
1. Research and implement SSE protocol for the MCP server\n2. Create dedicated SSE endpoints for event streaming\n3. Implement event emitter pattern for internal event management\n4. Add support for different event types (task status, logs, errors)\n5. Implement client connection management with proper keep-alive handling\n6. Add filtering capabilities to allow subscribing to specific event types\n7. Create in-memory event buffer for clients reconnecting\n8. Document SSE endpoint usage and client implementation examples\n9. Add robust error handling for dropped connections\n10. Implement rate limiting and backpressure mechanisms\n11. Add authentication for SSE connections