Files
claude-task-master/tests/unit/ai-services-unified.test.js
Ben Vargas dd96f51179 feat: Add gemini-cli provider integration for Task Master (#897)
* feat: Add gemini-cli provider integration for Task Master

This commit adds comprehensive support for the Gemini CLI provider, enabling users
to leverage Google's Gemini models through OAuth authentication via the gemini CLI
tool. This integration provides a seamless experience for users who prefer using
their existing Google account authentication rather than managing API keys.

## Implementation Details

### Provider Class (`src/ai-providers/gemini-cli.js`)
- Created GeminiCliProvider extending BaseAIProvider
- Implements dual authentication support:
  - Primary: OAuth authentication via `gemini auth login` (authType: 'oauth-personal')
  - Secondary: API key authentication for compatibility (authType: 'api-key')
- Uses the npm package `ai-sdk-provider-gemini-cli` (v0.0.3) for SDK integration
- Properly handles authentication validation without console output

### Model Configuration (`scripts/modules/supported-models.json`)
- Added two Gemini models with accurate specifications:
  - gemini-2.5-pro: 72% SWE score, 65,536 max output tokens
  - gemini-2.5-flash: 71% SWE score, 65,536 max output tokens
- Both models support main, fallback, and research roles
- Configured with zero cost (free tier)

### System Integration
- Registered provider in PROVIDERS map (`scripts/modules/ai-services-unified.js`)
- Added to OPTIONAL_AUTH_PROVIDERS set for flexible authentication
- Added GEMINI_CLI constant to provider constants (`src/constants/providers.js`)
- Exported GeminiCliProvider from index (`src/ai-providers/index.js`)

### Command Line Support (`scripts/modules/commands.js`)
- Added --gemini-cli flag to models command for provider hint
- Integrated into model selection logic (setModel function)
- Updated error messages to include gemini-cli in provider list
- Removed unrelated azure/vertex changes to maintain PR focus

### Documentation (`docs/providers/gemini-cli.md`)
- Comprehensive provider documentation emphasizing OAuth-first approach
- Clear explanation of why users would choose gemini-cli over standard google provider
- Detailed installation, authentication, and configuration instructions
- Troubleshooting section with common issues and solutions

### Testing (`tests/unit/ai-providers/gemini-cli.test.js`)
- Complete test suite with 12 tests covering all functionality
- Tests for both OAuth and API key authentication paths
- Error handling and edge case coverage
- Updated mocks in ai-services-unified.test.js for integration testing

## Key Design Decisions

1. **OAuth-First Design**: The provider assumes users want to leverage their existing
   `gemini auth login` credentials, making this the default authentication method.

2. **Authentication Type Mapping**: Discovered through testing that the SDK expects:
   - 'oauth-personal' for OAuth/CLI authentication (not 'gemini-cli' or 'oauth')
   - 'api-key' for API key authentication (not 'gemini-api-key')

3. **Silent Operation**: Removed console.log statements from validateAuth to match
   the pattern used by other providers like claude-code.

4. **Limited Model Support**: Only gemini-2.5-pro and gemini-2.5-flash are available
   through the CLI, as confirmed by the package author.

## Usage

```bash
# Install gemini CLI globally
npm install -g @google/gemini-cli

# Authenticate with Google account
gemini auth login

# Configure Task Master to use gemini-cli
task-master models --set-main gemini-2.5-pro --gemini-cli

# Use Task Master normally
task-master new "Create a REST API endpoint"
```

## Dependencies
- Added `ai-sdk-provider-gemini-cli@^0.0.3` to package.json
- This package wraps the Google Gemini CLI Core functionality for Vercel AI SDK

## Testing
All tests pass (613 total), including the new gemini-cli provider tests.
Code has been formatted with biome to maintain consistency.

This implementation provides a clean, well-tested integration that follows Task Master's
existing patterns while offering users a convenient way to use Gemini models with their
existing Google authentication.

* feat: implement lazy loading for gemini-cli provider

- Move ai-sdk-provider-gemini-cli to optionalDependencies
- Implement dynamic import with loadGeminiCliModule() function
- Make getClient() async to support lazy loading
- Update base-provider to handle async getClient() calls
- Update tests to handle async getClient() method

This allows the application to start without the gemini-cli package
installed, only loading it when actually needed.

* feat(gemini-cli): replace regex-based JSON extraction with jsonc-parser

- Add jsonc-parser dependency for robust JSON parsing
- Replace simple regex approach with progressive parsing strategy:
  1. Direct parsing after cleanup
  2. Smart boundary detection with single-pass analysis
  3. Limited fallback for edge cases
- Optimize performance with early termination and strategic sampling
- Add comprehensive tests for variable declarations, trailing commas,
  escaped quotes, nested objects, and performance edge cases
- Improve reliability for complex JSON structures that Gemini commonly produces
- Fix code formatting with biome

This addresses JSON parsing failures in generateObject operations while
maintaining backward compatibility and significantly improving performance
for large responses.

* fix: update package-lock.json and fix formatting for CI/CD

- Add jsonc-parser to package-lock.json for proper npm ci compatibility
- Fix biome formatting issues in gemini-cli provider and tests
- Ensure all CI/CD checks pass

* feat(gemini-cli): implement comprehensive JSON output reliability system

- Add automatic JSON request detection via content analysis patterns
- Implement task-specific prompt simplification for improved AI compliance
- Add strict JSON enforcement through enhanced system prompts
- Implement response interception with intelligent JSON extraction fallback
- Add comprehensive test coverage for all new JSON handling methods
- Move debug logging to appropriate level for clean user experience

This multi-layered approach addresses gemini-cli's conversational response
tendencies, ensuring reliable structured JSON output for task expansion
operations. Achieves 100% success rate in end-to-end testing while
maintaining full backward compatibility with existing functionality.

Technical implementation includes:
• JSON detection via user message content analysis
• Expand-task prompt simplification with cleaner instructions
• System prompt enhancement with strict JSON enforcement
• Response processing with jsonc-parser-based extraction
• Comprehensive unit test coverage for edge cases
• Debug-level logging to prevent user interface clutter

Resolves: gemini-cli JSON formatting inconsistencies
Tested: All 46 test suites pass, formatting verified

* chore: add changeset for gemini-cli provider implementation

Adds minor version bump for comprehensive gemini-cli provider with:
- Lazy loading and optional dependency management
- Advanced JSON parsing with jsonc-parser
- Multi-layer reliability system for structured output
- Complete test coverage and CI/CD compliance

* refactor: consolidate optional auth provider logic

- Add gemini-cli to existing providersWithoutApiKeys array in config-manager
- Export providersWithoutApiKeys for reuse across modules
- Remove duplicate OPTIONAL_AUTH_PROVIDERS Set from ai-services-unified
- Update ai-services-unified to import and use centralized array
- Fix Jest mock to include new providersWithoutApiKeys export

This eliminates code duplication and provides a single source of truth
for which providers support optional authentication, addressing PR
reviewer feedback about existing similar functionality in src/constants.
2025-07-02 21:46:19 +02:00

711 lines
24 KiB
JavaScript

import { jest } from '@jest/globals';
// Mock config-manager
const mockGetMainProvider = jest.fn();
const mockGetMainModelId = jest.fn();
const mockGetResearchProvider = jest.fn();
const mockGetResearchModelId = jest.fn();
const mockGetFallbackProvider = jest.fn();
const mockGetFallbackModelId = jest.fn();
const mockGetParametersForRole = jest.fn();
const mockGetUserId = jest.fn();
const mockGetDebugFlag = jest.fn();
const mockIsApiKeySet = jest.fn();
// --- Mock MODEL_MAP Data ---
// Provide a simplified structure sufficient for cost calculation tests
const mockModelMap = {
anthropic: [
{
id: 'test-main-model',
cost_per_1m_tokens: { input: 3, output: 15, currency: 'USD' }
},
{
id: 'test-fallback-model',
cost_per_1m_tokens: { input: 3, output: 15, currency: 'USD' }
}
],
perplexity: [
{
id: 'test-research-model',
cost_per_1m_tokens: { input: 1, output: 1, currency: 'USD' }
}
],
openai: [
{
id: 'test-openai-model',
cost_per_1m_tokens: { input: 2, output: 6, currency: 'USD' }
}
]
// Add other providers/models if needed for specific tests
};
const mockGetBaseUrlForRole = jest.fn();
const mockGetAllProviders = jest.fn();
const mockGetOllamaBaseURL = jest.fn();
const mockGetAzureBaseURL = jest.fn();
const mockGetBedrockBaseURL = jest.fn();
const mockGetVertexProjectId = jest.fn();
const mockGetVertexLocation = jest.fn();
const mockGetAvailableModels = jest.fn();
const mockValidateProvider = jest.fn();
const mockValidateProviderModelCombination = jest.fn();
const mockGetConfig = jest.fn();
const mockWriteConfig = jest.fn();
const mockIsConfigFilePresent = jest.fn();
const mockGetMcpApiKeyStatus = jest.fn();
const mockGetMainMaxTokens = jest.fn();
const mockGetMainTemperature = jest.fn();
const mockGetResearchMaxTokens = jest.fn();
const mockGetResearchTemperature = jest.fn();
const mockGetFallbackMaxTokens = jest.fn();
const mockGetFallbackTemperature = jest.fn();
const mockGetLogLevel = jest.fn();
const mockGetDefaultNumTasks = jest.fn();
const mockGetDefaultSubtasks = jest.fn();
const mockGetDefaultPriority = jest.fn();
const mockGetProjectName = jest.fn();
jest.unstable_mockModule('../../scripts/modules/config-manager.js', () => ({
// Core config access
getConfig: mockGetConfig,
writeConfig: mockWriteConfig,
isConfigFilePresent: mockIsConfigFilePresent,
ConfigurationError: class ConfigurationError extends Error {
constructor(message) {
super(message);
this.name = 'ConfigurationError';
}
},
// Validation
validateProvider: mockValidateProvider,
validateProviderModelCombination: mockValidateProviderModelCombination,
VALID_PROVIDERS: ['anthropic', 'perplexity', 'openai', 'google'],
MODEL_MAP: mockModelMap,
getAvailableModels: mockGetAvailableModels,
// Role-specific getters
getMainProvider: mockGetMainProvider,
getMainModelId: mockGetMainModelId,
getMainMaxTokens: mockGetMainMaxTokens,
getMainTemperature: mockGetMainTemperature,
getResearchProvider: mockGetResearchProvider,
getResearchModelId: mockGetResearchModelId,
getResearchMaxTokens: mockGetResearchMaxTokens,
getResearchTemperature: mockGetResearchTemperature,
getFallbackProvider: mockGetFallbackProvider,
getFallbackModelId: mockGetFallbackModelId,
getFallbackMaxTokens: mockGetFallbackMaxTokens,
getFallbackTemperature: mockGetFallbackTemperature,
getParametersForRole: mockGetParametersForRole,
getUserId: mockGetUserId,
getDebugFlag: mockGetDebugFlag,
getBaseUrlForRole: mockGetBaseUrlForRole,
// Global settings
getLogLevel: mockGetLogLevel,
getDefaultNumTasks: mockGetDefaultNumTasks,
getDefaultSubtasks: mockGetDefaultSubtasks,
getDefaultPriority: mockGetDefaultPriority,
getProjectName: mockGetProjectName,
// API Key and provider functions
isApiKeySet: mockIsApiKeySet,
getAllProviders: mockGetAllProviders,
getOllamaBaseURL: mockGetOllamaBaseURL,
getAzureBaseURL: mockGetAzureBaseURL,
getBedrockBaseURL: mockGetBedrockBaseURL,
getVertexProjectId: mockGetVertexProjectId,
getVertexLocation: mockGetVertexLocation,
getMcpApiKeyStatus: mockGetMcpApiKeyStatus,
// Providers without API keys
providersWithoutApiKeys: ['ollama', 'bedrock', 'gemini-cli']
}));
// Mock AI Provider Classes with proper methods
const mockAnthropicProvider = {
generateText: jest.fn(),
streamText: jest.fn(),
generateObject: jest.fn()
};
const mockPerplexityProvider = {
generateText: jest.fn(),
streamText: jest.fn(),
generateObject: jest.fn()
};
const mockOpenAIProvider = {
generateText: jest.fn(),
streamText: jest.fn(),
generateObject: jest.fn()
};
const mockOllamaProvider = {
generateText: jest.fn(),
streamText: jest.fn(),
generateObject: jest.fn()
};
// Mock the provider classes to return our mock instances
jest.unstable_mockModule('../../src/ai-providers/index.js', () => ({
AnthropicAIProvider: jest.fn(() => mockAnthropicProvider),
PerplexityAIProvider: jest.fn(() => mockPerplexityProvider),
GoogleAIProvider: jest.fn(() => ({
generateText: jest.fn(),
streamText: jest.fn(),
generateObject: jest.fn()
})),
OpenAIProvider: jest.fn(() => mockOpenAIProvider),
XAIProvider: jest.fn(() => ({
generateText: jest.fn(),
streamText: jest.fn(),
generateObject: jest.fn()
})),
OpenRouterAIProvider: jest.fn(() => ({
generateText: jest.fn(),
streamText: jest.fn(),
generateObject: jest.fn()
})),
OllamaAIProvider: jest.fn(() => mockOllamaProvider),
BedrockAIProvider: jest.fn(() => ({
generateText: jest.fn(),
streamText: jest.fn(),
generateObject: jest.fn()
})),
AzureProvider: jest.fn(() => ({
generateText: jest.fn(),
streamText: jest.fn(),
generateObject: jest.fn()
})),
VertexAIProvider: jest.fn(() => ({
generateText: jest.fn(),
streamText: jest.fn(),
generateObject: jest.fn()
})),
ClaudeCodeProvider: jest.fn(() => ({
generateText: jest.fn(),
streamText: jest.fn(),
generateObject: jest.fn()
})),
GeminiCliProvider: jest.fn(() => ({
generateText: jest.fn(),
streamText: jest.fn(),
generateObject: jest.fn()
}))
}));
// Mock utils logger, API key resolver, AND findProjectRoot
const mockLog = jest.fn();
const mockResolveEnvVariable = jest.fn();
const mockFindProjectRoot = jest.fn();
const mockIsSilentMode = jest.fn();
const mockLogAiUsage = jest.fn();
const mockFindCycles = jest.fn();
const mockFormatTaskId = jest.fn();
const mockTaskExists = jest.fn();
const mockFindTaskById = jest.fn();
const mockTruncate = jest.fn();
const mockToKebabCase = jest.fn();
const mockDetectCamelCaseFlags = jest.fn();
const mockDisableSilentMode = jest.fn();
const mockEnableSilentMode = jest.fn();
const mockGetTaskManager = jest.fn();
const mockAddComplexityToTask = jest.fn();
const mockReadJSON = jest.fn();
const mockWriteJSON = jest.fn();
const mockSanitizePrompt = jest.fn();
const mockReadComplexityReport = jest.fn();
const mockFindTaskInComplexityReport = jest.fn();
const mockAggregateTelemetry = jest.fn();
const mockGetCurrentTag = jest.fn(() => 'master');
const mockResolveTag = jest.fn(() => 'master');
const mockGetTasksForTag = jest.fn(() => []);
jest.unstable_mockModule('../../scripts/modules/utils.js', () => ({
LOG_LEVELS: { error: 0, warn: 1, info: 2, debug: 3 },
log: mockLog,
resolveEnvVariable: mockResolveEnvVariable,
findProjectRoot: mockFindProjectRoot,
isSilentMode: mockIsSilentMode,
logAiUsage: mockLogAiUsage,
findCycles: mockFindCycles,
formatTaskId: mockFormatTaskId,
taskExists: mockTaskExists,
findTaskById: mockFindTaskById,
truncate: mockTruncate,
toKebabCase: mockToKebabCase,
detectCamelCaseFlags: mockDetectCamelCaseFlags,
disableSilentMode: mockDisableSilentMode,
enableSilentMode: mockEnableSilentMode,
getTaskManager: mockGetTaskManager,
addComplexityToTask: mockAddComplexityToTask,
readJSON: mockReadJSON,
writeJSON: mockWriteJSON,
sanitizePrompt: mockSanitizePrompt,
readComplexityReport: mockReadComplexityReport,
findTaskInComplexityReport: mockFindTaskInComplexityReport,
aggregateTelemetry: mockAggregateTelemetry,
getCurrentTag: mockGetCurrentTag,
resolveTag: mockResolveTag,
getTasksForTag: mockGetTasksForTag
}));
// Import the module to test (AFTER mocks)
const { generateTextService } = await import(
'../../scripts/modules/ai-services-unified.js'
);
describe('Unified AI Services', () => {
const fakeProjectRoot = '/fake/project/root'; // Define for reuse
beforeEach(() => {
// Clear mocks before each test
jest.clearAllMocks(); // Clears all mocks
// Set default mock behaviors
mockGetMainProvider.mockReturnValue('anthropic');
mockGetMainModelId.mockReturnValue('test-main-model');
mockGetResearchProvider.mockReturnValue('perplexity');
mockGetResearchModelId.mockReturnValue('test-research-model');
mockGetFallbackProvider.mockReturnValue('anthropic');
mockGetFallbackModelId.mockReturnValue('test-fallback-model');
mockGetParametersForRole.mockImplementation((role) => {
if (role === 'main') return { maxTokens: 100, temperature: 0.5 };
if (role === 'research') return { maxTokens: 200, temperature: 0.3 };
if (role === 'fallback') return { maxTokens: 150, temperature: 0.6 };
return { maxTokens: 100, temperature: 0.5 }; // Default
});
mockResolveEnvVariable.mockImplementation((key) => {
if (key === 'ANTHROPIC_API_KEY') return 'mock-anthropic-key';
if (key === 'PERPLEXITY_API_KEY') return 'mock-perplexity-key';
if (key === 'OPENAI_API_KEY') return 'mock-openai-key';
if (key === 'OLLAMA_API_KEY') return 'mock-ollama-key';
return null;
});
// Set a default behavior for the new mock
mockFindProjectRoot.mockReturnValue(fakeProjectRoot);
mockGetDebugFlag.mockReturnValue(false);
mockGetUserId.mockReturnValue('test-user-id'); // Add default mock for getUserId
mockIsApiKeySet.mockReturnValue(true); // Default to true for most tests
mockGetBaseUrlForRole.mockReturnValue(null); // Default to no base URL
});
describe('generateTextService', () => {
test('should use main provider/model and succeed', async () => {
mockAnthropicProvider.generateText.mockResolvedValue({
text: 'Main provider response',
usage: { inputTokens: 10, outputTokens: 20, totalTokens: 30 }
});
const params = {
role: 'main',
session: { env: {} },
systemPrompt: 'System',
prompt: 'Test'
};
const result = await generateTextService(params);
expect(result.mainResult).toBe('Main provider response');
expect(result).toHaveProperty('telemetryData');
expect(mockGetMainProvider).toHaveBeenCalledWith(fakeProjectRoot);
expect(mockGetMainModelId).toHaveBeenCalledWith(fakeProjectRoot);
expect(mockGetParametersForRole).toHaveBeenCalledWith(
'main',
fakeProjectRoot
);
expect(mockAnthropicProvider.generateText).toHaveBeenCalledTimes(1);
expect(mockPerplexityProvider.generateText).not.toHaveBeenCalled();
});
test('should fall back to fallback provider if main fails', async () => {
const mainError = new Error('Main provider failed');
mockAnthropicProvider.generateText
.mockRejectedValueOnce(mainError)
.mockResolvedValueOnce({
text: 'Fallback provider response',
usage: { inputTokens: 15, outputTokens: 25, totalTokens: 40 }
});
const explicitRoot = '/explicit/test/root';
const params = {
role: 'main',
prompt: 'Fallback test',
projectRoot: explicitRoot
};
const result = await generateTextService(params);
expect(result.mainResult).toBe('Fallback provider response');
expect(result).toHaveProperty('telemetryData');
expect(mockGetMainProvider).toHaveBeenCalledWith(explicitRoot);
expect(mockGetFallbackProvider).toHaveBeenCalledWith(explicitRoot);
expect(mockGetParametersForRole).toHaveBeenCalledWith(
'main',
explicitRoot
);
expect(mockGetParametersForRole).toHaveBeenCalledWith(
'fallback',
explicitRoot
);
expect(mockAnthropicProvider.generateText).toHaveBeenCalledTimes(2);
expect(mockPerplexityProvider.generateText).not.toHaveBeenCalled();
expect(mockLog).toHaveBeenCalledWith(
'error',
expect.stringContaining('Service call failed for role main')
);
expect(mockLog).toHaveBeenCalledWith(
'info',
expect.stringContaining('New AI service call with role: fallback')
);
});
test('should fall back to research provider if main and fallback fail', async () => {
const mainError = new Error('Main failed');
const fallbackError = new Error('Fallback failed');
mockAnthropicProvider.generateText
.mockRejectedValueOnce(mainError)
.mockRejectedValueOnce(fallbackError);
mockPerplexityProvider.generateText.mockResolvedValue({
text: 'Research provider response',
usage: { inputTokens: 20, outputTokens: 30, totalTokens: 50 }
});
const params = { role: 'main', prompt: 'Research fallback test' };
const result = await generateTextService(params);
expect(result.mainResult).toBe('Research provider response');
expect(result).toHaveProperty('telemetryData');
expect(mockGetMainProvider).toHaveBeenCalledWith(fakeProjectRoot);
expect(mockGetFallbackProvider).toHaveBeenCalledWith(fakeProjectRoot);
expect(mockGetResearchProvider).toHaveBeenCalledWith(fakeProjectRoot);
expect(mockGetParametersForRole).toHaveBeenCalledWith(
'main',
fakeProjectRoot
);
expect(mockGetParametersForRole).toHaveBeenCalledWith(
'fallback',
fakeProjectRoot
);
expect(mockGetParametersForRole).toHaveBeenCalledWith(
'research',
fakeProjectRoot
);
expect(mockAnthropicProvider.generateText).toHaveBeenCalledTimes(2);
expect(mockPerplexityProvider.generateText).toHaveBeenCalledTimes(1);
expect(mockLog).toHaveBeenCalledWith(
'error',
expect.stringContaining('Service call failed for role fallback')
);
expect(mockLog).toHaveBeenCalledWith(
'info',
expect.stringContaining('New AI service call with role: research')
);
});
test('should throw error if all providers in sequence fail', async () => {
mockAnthropicProvider.generateText.mockRejectedValue(
new Error('Anthropic failed')
);
mockPerplexityProvider.generateText.mockRejectedValue(
new Error('Perplexity failed')
);
const params = { role: 'main', prompt: 'All fail test' };
await expect(generateTextService(params)).rejects.toThrow(
'Perplexity failed' // Error from the last attempt (research)
);
expect(mockAnthropicProvider.generateText).toHaveBeenCalledTimes(2); // main, fallback
expect(mockPerplexityProvider.generateText).toHaveBeenCalledTimes(1); // research
});
test('should handle retryable errors correctly', async () => {
const retryableError = new Error('Rate limit');
mockAnthropicProvider.generateText
.mockRejectedValueOnce(retryableError) // Fails once
.mockResolvedValueOnce({
// Succeeds on retry
text: 'Success after retry',
usage: { inputTokens: 5, outputTokens: 10, totalTokens: 15 }
});
const params = { role: 'main', prompt: 'Retry success test' };
const result = await generateTextService(params);
expect(result.mainResult).toBe('Success after retry');
expect(result).toHaveProperty('telemetryData');
expect(mockAnthropicProvider.generateText).toHaveBeenCalledTimes(2); // Initial + 1 retry
expect(mockLog).toHaveBeenCalledWith(
'info',
expect.stringContaining(
'Something went wrong on the provider side. Retrying'
)
);
});
test('should use default project root or handle null if findProjectRoot returns null', async () => {
mockFindProjectRoot.mockReturnValue(null); // Simulate not finding root
mockAnthropicProvider.generateText.mockResolvedValue({
text: 'Response with no root',
usage: { inputTokens: 1, outputTokens: 1, totalTokens: 2 }
});
const params = { role: 'main', prompt: 'No root test' }; // No explicit root passed
await generateTextService(params);
expect(mockGetMainProvider).toHaveBeenCalledWith(null);
expect(mockGetParametersForRole).toHaveBeenCalledWith('main', null);
expect(mockAnthropicProvider.generateText).toHaveBeenCalledTimes(1);
});
test('should skip provider with missing API key and try next in fallback sequence', async () => {
// Setup isApiKeySet to return false for anthropic but true for perplexity
mockIsApiKeySet.mockImplementation((provider, session, root) => {
if (provider === 'anthropic') return false; // Main provider has no key
return true; // Other providers have keys
});
// Mock perplexity text response (since we'll skip anthropic)
mockPerplexityProvider.generateText.mockResolvedValue({
text: 'Perplexity response (skipped to research)',
usage: { inputTokens: 20, outputTokens: 30, totalTokens: 50 }
});
const params = {
role: 'main',
prompt: 'Skip main provider test',
session: { env: {} }
};
const result = await generateTextService(params);
// Should have gotten the perplexity response
expect(result.mainResult).toBe(
'Perplexity response (skipped to research)'
);
// Should check API keys
expect(mockIsApiKeySet).toHaveBeenCalledWith(
'anthropic',
params.session,
fakeProjectRoot
);
expect(mockIsApiKeySet).toHaveBeenCalledWith(
'perplexity',
params.session,
fakeProjectRoot
);
// Should log a warning
expect(mockLog).toHaveBeenCalledWith(
'warn',
expect.stringContaining(
`Skipping role 'main' (Provider: anthropic): API key not set or invalid.`
)
);
// Should NOT call anthropic provider
expect(mockAnthropicProvider.generateText).not.toHaveBeenCalled();
// Should call perplexity provider
expect(mockPerplexityProvider.generateText).toHaveBeenCalledTimes(1);
});
test('should skip multiple providers with missing API keys and use first available', async () => {
// Setup: Main and fallback providers have no keys, only research has a key
mockIsApiKeySet.mockImplementation((provider, session, root) => {
if (provider === 'anthropic') return false; // Main and fallback are both anthropic
if (provider === 'perplexity') return true; // Research has a key
return false;
});
// Define different providers for testing multiple skips
mockGetFallbackProvider.mockReturnValue('openai'); // Different from main
mockGetFallbackModelId.mockReturnValue('test-openai-model');
// Mock isApiKeySet to return false for both main and fallback
mockIsApiKeySet.mockImplementation((provider, session, root) => {
if (provider === 'anthropic') return false; // Main provider has no key
if (provider === 'openai') return false; // Fallback provider has no key
return true; // Research provider has a key
});
// Mock perplexity text response (since we'll skip to research)
mockPerplexityProvider.generateText.mockResolvedValue({
text: 'Research response after skipping main and fallback',
usage: { inputTokens: 20, outputTokens: 30, totalTokens: 50 }
});
const params = {
role: 'main',
prompt: 'Skip multiple providers test',
session: { env: {} }
};
const result = await generateTextService(params);
// Should have gotten the perplexity (research) response
expect(result.mainResult).toBe(
'Research response after skipping main and fallback'
);
// Should check API keys for all three roles
expect(mockIsApiKeySet).toHaveBeenCalledWith(
'anthropic',
params.session,
fakeProjectRoot
);
expect(mockIsApiKeySet).toHaveBeenCalledWith(
'openai',
params.session,
fakeProjectRoot
);
expect(mockIsApiKeySet).toHaveBeenCalledWith(
'perplexity',
params.session,
fakeProjectRoot
);
// Should log warnings for both skipped providers
expect(mockLog).toHaveBeenCalledWith(
'warn',
expect.stringContaining(
`Skipping role 'main' (Provider: anthropic): API key not set or invalid.`
)
);
expect(mockLog).toHaveBeenCalledWith(
'warn',
expect.stringContaining(
`Skipping role 'fallback' (Provider: openai): API key not set or invalid.`
)
);
// Should NOT call skipped providers
expect(mockAnthropicProvider.generateText).not.toHaveBeenCalled();
expect(mockOpenAIProvider.generateText).not.toHaveBeenCalled();
// Should call perplexity provider
expect(mockPerplexityProvider.generateText).toHaveBeenCalledTimes(1);
});
test('should throw error if all providers in sequence have missing API keys', async () => {
// Mock all providers to have missing API keys
mockIsApiKeySet.mockReturnValue(false);
const params = {
role: 'main',
prompt: 'All API keys missing test',
session: { env: {} }
};
// Should throw error since all providers would be skipped
await expect(generateTextService(params)).rejects.toThrow(
'AI service call failed for all configured roles'
);
// Should log warnings for all skipped providers
expect(mockLog).toHaveBeenCalledWith(
'warn',
expect.stringContaining(
`Skipping role 'main' (Provider: anthropic): API key not set or invalid.`
)
);
expect(mockLog).toHaveBeenCalledWith(
'warn',
expect.stringContaining(
`Skipping role 'fallback' (Provider: anthropic): API key not set or invalid.`
)
);
expect(mockLog).toHaveBeenCalledWith(
'warn',
expect.stringContaining(
`Skipping role 'research' (Provider: perplexity): API key not set or invalid.`
)
);
// Should log final error
expect(mockLog).toHaveBeenCalledWith(
'error',
expect.stringContaining(
'All roles in the sequence [main, fallback, research] failed.'
)
);
// Should NOT call any providers
expect(mockAnthropicProvider.generateText).not.toHaveBeenCalled();
expect(mockPerplexityProvider.generateText).not.toHaveBeenCalled();
});
test('should not check API key for Ollama provider and try to use it', async () => {
// Setup: Set main provider to ollama
mockGetMainProvider.mockReturnValue('ollama');
mockGetMainModelId.mockReturnValue('llama3');
// Mock Ollama text generation to succeed
mockOllamaProvider.generateText.mockResolvedValue({
text: 'Ollama response (no API key required)',
usage: { inputTokens: 10, outputTokens: 10, totalTokens: 20 }
});
const params = {
role: 'main',
prompt: 'Ollama special case test',
session: { env: {} }
};
const result = await generateTextService(params);
// Should have gotten the Ollama response
expect(result.mainResult).toBe('Ollama response (no API key required)');
// isApiKeySet shouldn't be called for Ollama
// Note: This is indirect - the code just doesn't check isApiKeySet for ollama
// so we're verifying ollama provider was called despite isApiKeySet being mocked to false
mockIsApiKeySet.mockReturnValue(false); // Should be ignored for Ollama
// Should call Ollama provider
expect(mockOllamaProvider.generateText).toHaveBeenCalledTimes(1);
});
test('should correctly use the provided session for API key check', async () => {
// Mock custom session object with env vars
const customSession = { env: { ANTHROPIC_API_KEY: 'session-api-key' } };
// Setup API key check to verify the session is passed correctly
mockIsApiKeySet.mockImplementation((provider, session, root) => {
// Only return true if the correct session was provided
return session === customSession;
});
// Mock the anthropic response
mockAnthropicProvider.generateText.mockResolvedValue({
text: 'Anthropic response with session key',
usage: { inputTokens: 10, outputTokens: 10, totalTokens: 20 }
});
const params = {
role: 'main',
prompt: 'Session API key test',
session: customSession
};
const result = await generateTextService(params);
// Should check API key with the custom session
expect(mockIsApiKeySet).toHaveBeenCalledWith(
'anthropic',
customSession,
fakeProjectRoot
);
// Should have gotten the anthropic response
expect(result.mainResult).toBe('Anthropic response with session key');
});
});
});