c5738a2513
This commit introduces a comprehensive set of skipped tests to both and . These skipped tests serve as a blueprint for future test implementation, outlining the necessary test cases for currently untested functionalities.
- Ensures sync with bin/ folder by adding -r/--research to the command
- Fixes an issue that improperly parsed command line args
- Ensures confirmation card on dependency add/remove
- Properly formats some sub-task dependencies
**Potentially addressed issues:**
While primarily focused on adding test coverage, this commit also implicitly addresses potential issues by:
- **Improving error handling coverage:** The addition of skipped tests for error scenarios in functions like , , , and highlights areas where error handling needs to be robustly tested and potentially improved in the codebase.
- **Enhancing dependency validation:** Skipped tests for include validation of dependencies, prompting a review of the dependency validation logic and ensuring its correctness.
- **Standardizing test coverage:** By creating a clear roadmap for testing all functions, this commit contributes to a more standardized and complete test suite, reducing the likelihood of undiscovered bugs in the future.
**task-manager.test.js:**
- Added skipped test blocks for the following functions:
- : Includes tests for handling valid JSON responses, malformed JSON, missing tasks in responses, Perplexity AI research integration, Claude fallback, and parallel task processing.
- : Covers tests for updating tasks based on context, handling Claude streaming, Perplexity AI integration, scenarios with no tasks to update, and error handling during updates.
- : Includes tests for generating task files from , formatting dependencies with status indicators, handling tasks without subtasks, empty task arrays, and dependency validation before file generation.
- : Covers tests for updating task status, subtask status using dot notation, updating multiple tasks, automatic subtask status updates, parent task update suggestions, and handling non-existent task IDs.
- : Includes tests for updating regular and subtask statuses, handling parent tasks without subtasks, and non-existent subtask IDs.
- : Covers tests for displaying all tasks, filtering by status, displaying subtasks, showing completion statistics, identifying the next task, and handling empty task arrays.
- : Includes tests for generating subtasks, using complexity reports for subtask counts, Perplexity AI integration, appending subtasks, skipping completed tasks, and error handling during subtask generation.
- : Covers tests for expanding all pending tasks, sorting by complexity, skipping tasks with existing subtasks (unless forced), using task-specific parameters from complexity reports, handling empty task arrays, and error handling for individual tasks.
- : Includes tests for clearing subtasks from specific and multiple tasks, handling tasks without subtasks, non-existent task IDs, and regenerating task files after clearing subtasks.
- : Covers tests for adding new tasks using AI, handling Claude streaming, validating dependencies, handling malformed AI responses, and using existing task context for generation.
**utils.test.js:**
- Added skipped test blocks for the following functions:
- : Tests for logging messages according to log levels and filtering messages below configured levels.
- : Tests for reading and parsing valid JSON files, handling file not found errors, and invalid JSON formats.
- : Tests for writing JSON data to files and handling file write errors.
- : Tests for escaping double quotes in prompts and handling prompts without special characters.
- : Tests for reading and parsing complexity reports, handling missing report files, and custom report paths.
- : Tests for finding tasks in reports by ID, handling non-existent task IDs, and invalid report structures.
- : Tests for verifying existing task and subtask IDs, handling non-existent IDs, and invalid inputs.
- : Tests for formatting numeric and string task IDs and preserving dot notation for subtasks.
- : Tests for detecting simple and complex cycles in dependency graphs, handling acyclic graphs, and empty dependency maps.
These skipped tests provide a clear roadmap for future test development, ensuring comprehensive coverage for core functionalities in both modules. They document the intended behavior of each function and outline various scenarios, including happy paths, edge cases, and error conditions, thereby improving the overall test strategy and maintainability of the Task Master CLI.
51 lines
2.9 KiB
Plaintext
51 lines
2.9 KiB
Plaintext
# Task ID: 20
|
|
# Title: Create Token Usage Tracking and Cost Management
|
|
# Status: done
|
|
# Dependencies: 5, 9, 17
|
|
# Priority: medium
|
|
# Description: Implement system for tracking API token usage and managing costs.
|
|
# Details:
|
|
Implement token tracking including:
|
|
- Track token usage for all API calls
|
|
- Implement configurable usage limits
|
|
- Add reporting on token consumption
|
|
- Create cost estimation features
|
|
- Implement caching to reduce API calls
|
|
- Add token optimization for prompts
|
|
- Create usage alerts when approaching limits
|
|
|
|
# Test Strategy:
|
|
Track token usage across various operations and verify accuracy. Test that limits properly prevent excessive usage. Verify that caching reduces token consumption for repeated operations.
|
|
|
|
# Subtasks:
|
|
## 1. Implement Token Usage Tracking for API Calls [done]
|
|
### Dependencies: 20.5
|
|
### Description: Create a middleware or wrapper function that intercepts all API calls to OpenAI, Anthropic, and Perplexity. This function should count the number of tokens used in both the request and response, storing this information in a persistent data store (e.g., SQLite database). Implement a caching mechanism to reduce redundant API calls and token usage.
|
|
### Details:
|
|
|
|
|
|
## 2. Develop Configurable Usage Limits [done]
|
|
### Dependencies: None
|
|
### Description: Create a configuration system that allows setting token usage limits at the project, user, and API level. Implement a mechanism to enforce these limits by checking the current usage against the configured limits before making API calls. Add the ability to set different limit types (e.g., daily, weekly, monthly) and actions to take when limits are reached (e.g., block calls, send notifications).
|
|
### Details:
|
|
|
|
|
|
## 3. Implement Token Usage Reporting and Cost Estimation [done]
|
|
### Dependencies: 20.1, 20.2
|
|
### Description: Develop a reporting module that generates detailed token usage reports. Include breakdowns by API, user, and time period. Implement cost estimation features by integrating current pricing information for each API. Create both command-line and programmatic interfaces for generating reports and estimates.
|
|
### Details:
|
|
|
|
|
|
## 4. Optimize Token Usage in Prompts [done]
|
|
### Dependencies: None
|
|
### Description: Implement a prompt optimization system that analyzes and refines prompts to reduce token usage while maintaining effectiveness. Use techniques such as prompt compression, removing redundant information, and leveraging efficient prompting patterns. Integrate this system into the existing prompt generation and API call processes.
|
|
### Details:
|
|
|
|
|
|
## 5. Develop Token Usage Alert System [done]
|
|
### Dependencies: 20.2, 20.3
|
|
### Description: Create an alert system that monitors token usage in real-time and sends notifications when usage approaches or exceeds defined thresholds. Implement multiple notification channels (e.g., email, Slack, system logs) and allow for customizable alert rules. Integrate this system with the existing logging and reporting modules.
|
|
### Details:
|
|
|
|
|