Files
automaker/apps/server/tests/unit/lib/model-resolver.test.ts
Stefan de Vogelaere a1f234c7e2 feat: Claude Compatible Providers System (#629)
* feat: refactor Claude API Profiles to Claude Compatible Providers

- Rename ClaudeApiProfile to ClaudeCompatibleProvider with models[] array
- Each ProviderModel has mapsToClaudeModel field for Claude tier mapping
- Add providerType field for provider-specific icons (glm, minimax, openrouter)
- Add thinking level support for provider models in phase selectors
- Show all mapped Claude models per provider model (e.g., "Maps to Haiku, Sonnet, Opus")
- Add Bulk Replace feature to switch all phases to a provider at once
- Hide Bulk Replace button when no providers are enabled
- Fix project-level phaseModelOverrides not persisting after refresh
- Fix deleting last provider not persisting (remove empty array guard)
- Add getProviderByModelId() helper for all SDK routes
- Update all routes to pass provider config for provider models
- Update terminology from "profiles" to "providers" throughout UI
- Update documentation to reflect new provider system

* fix: atomic writer race condition and bulk replace reset to defaults

1. AtomicWriter Race Condition Fix (libs/utils/src/atomic-writer.ts):
   - Changed temp file naming from Date.now() to Date.now() + random hex
   - Uses crypto.randomBytes(4).toString('hex') for uniqueness
   - Prevents ENOENT errors when multiple concurrent writes happen
     within the same millisecond

2. Bulk Replace "Anthropic Direct" Reset (both dialogs):
   - When selecting "Anthropic Direct", now uses DEFAULT_PHASE_MODELS
   - Properly resets thinking levels and other settings to defaults
   - Added thinkingLevel to the change detection comparison
   - Affects both global and project-level bulk replace dialogs

* fix: update tests for new model resolver passthrough behavior

1. model-resolver tests:
   - Unknown models now pass through unchanged (provider model support)
   - Removed expectations for warnings on unknown models
   - Updated case sensitivity and edge case tests accordingly
   - Added tests for provider-like model names (GLM-4.7, MiniMax-M2.1)

2. atomic-writer tests:
   - Updated regex to match new temp file format with random suffix
   - Format changed from .tmp.{timestamp} to .tmp.{timestamp}.{hex}

* refactor: simplify getPhaseModelWithOverrides calls per code review

Address code review feedback on PR #629:
- Make settingsService parameter optional in getPhaseModelWithOverrides
- Function now handles undefined settingsService gracefully by returning defaults
- Remove redundant ternary checks in 4 call sites:
  - apps/server/src/routes/context/routes/describe-file.ts
  - apps/server/src/routes/context/routes/describe-image.ts
  - apps/server/src/routes/worktree/routes/generate-commit-message.ts
  - apps/server/src/services/auto-mode-service.ts
- Remove unused DEFAULT_PHASE_MODELS imports where applicable

* test: fix server tests for provider model passthrough behavior

- Update model-resolver.test.ts to expect unknown models to pass through
  unchanged (supports ClaudeCompatibleProvider models like GLM-4.7)
- Remove warning expectations for unknown models (valid for providers)
- Add missing getCredentials and getGlobalSettings mocks to
  ideation-service.test.ts for settingsService

* fix: address code review feedback for model providers

- Honor thinkingLevel in generate-commit-message.ts
- Pass claudeCompatibleProvider in ideation-service.ts for provider models
- Resolve provider configuration for model overrides in generate-suggestions.ts
- Update "Active Profile" to "Active Provider" label in project-claude-section
- Use substring instead of deprecated substr in api-profiles-section
- Preserve provider enabled state when editing in api-profiles-section

* fix: address CodeRabbit review issues for Claude Compatible Providers

- Fix TypeScript TS2339 error in generate-suggestions.ts where
  settingsService was narrowed to 'never' type in else branch
- Use DEFAULT_PHASE_MODELS per-phase defaults instead of hardcoded
  'sonnet' in settings-helpers.ts
- Remove duplicate eventHooks key in use-settings-migration.ts
- Add claudeCompatibleProviders to localStorage migration parsing
  and merging functions
- Handle canonical claude-* model IDs (claude-haiku, claude-sonnet,
  claude-opus) in project-models-section display names

This resolves the CI build failures and addresses code review feedback.

* fix: skip broken list-view-priority E2E test and add Priority column label

- Skip list-view-priority.spec.ts with TODO explaining the infrastructure
  issue: setupRealProject only sets localStorage but server settings
  take precedence with localStorageMigrated: true
- Add 'Priority' label to list-header.tsx for the priority column
  (was empty string, now shows proper header text)
- Increase column width to accommodate the label

The E2E test issue is that tests create features in a temp directory,
but the server loads from the E2E Test Project fixture path set in
setup-e2e-fixtures.mjs. Needs infrastructure fix to properly switch
projects or create features through UI instead of on disk.
2026-01-20 20:57:23 +01:00

169 lines
5.9 KiB
TypeScript

import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import {
resolveModelString,
getEffectiveModel,
CLAUDE_MODEL_MAP,
CURSOR_MODEL_MAP,
DEFAULT_MODELS,
} from '@automaker/model-resolver';
describe('model-resolver.ts', () => {
let consoleSpy: any;
beforeEach(() => {
consoleSpy = {
log: vi.spyOn(console, 'log').mockImplementation(() => {}),
warn: vi.spyOn(console, 'warn').mockImplementation(() => {}),
};
});
afterEach(() => {
consoleSpy.log.mockRestore();
consoleSpy.warn.mockRestore();
});
describe('resolveModelString', () => {
it("should resolve 'haiku' alias to full model string", () => {
const result = resolveModelString('haiku');
expect(result).toBe(CLAUDE_MODEL_MAP.haiku);
});
it("should resolve 'sonnet' alias to full model string", () => {
const result = resolveModelString('sonnet');
expect(result).toBe(CLAUDE_MODEL_MAP.sonnet);
});
it("should resolve 'opus' alias to full model string", () => {
const result = resolveModelString('opus');
expect(result).toBe('claude-opus-4-5-20251101');
expect(consoleSpy.log).toHaveBeenCalledWith(
expect.stringContaining('Migrated legacy ID: "opus" -> "claude-opus"')
);
});
it('should pass through unknown models unchanged (may be provider models)', () => {
// Unknown models now pass through unchanged to support ClaudeCompatibleProvider models
// like GLM-4.7, MiniMax-M2.1, o1, etc.
const models = ['o1', 'o1-mini', 'o3', 'unknown-model', 'fake-model-123', 'GLM-4.7'];
models.forEach((model) => {
const result = resolveModelString(model);
// Should pass through unchanged (could be provider models)
expect(result).toBe(model);
});
});
it('should pass through full Claude model strings', () => {
const models = [CLAUDE_MODEL_MAP.opus, CLAUDE_MODEL_MAP.sonnet, CLAUDE_MODEL_MAP.haiku];
models.forEach((model) => {
const result = resolveModelString(model);
expect(result).toBe(model);
});
expect(consoleSpy.log).toHaveBeenCalledWith(
expect.stringContaining('Using full Claude model string')
);
});
it('should return default model when modelKey is undefined', () => {
const result = resolveModelString(undefined);
expect(result).toBe(DEFAULT_MODELS.claude);
});
it('should return custom default model when provided', () => {
const customDefault = 'custom-model';
const result = resolveModelString(undefined, customDefault);
expect(result).toBe(customDefault);
});
it('should pass through unknown model key unchanged (no warning)', () => {
const result = resolveModelString('unknown-model');
// Unknown models pass through unchanged (could be provider models)
expect(result).toBe('unknown-model');
// No warning - unknown models are valid for providers
expect(consoleSpy.warn).not.toHaveBeenCalled();
});
it('should handle empty string', () => {
const result = resolveModelString('');
expect(result).toBe(DEFAULT_MODELS.claude);
});
describe('Cursor models', () => {
it('should pass through cursor-prefixed models unchanged', () => {
const result = resolveModelString('cursor-composer-1');
expect(result).toBe('cursor-composer-1');
expect(consoleSpy.log).toHaveBeenCalledWith(expect.stringContaining('Using Cursor model'));
});
it('should add cursor- prefix to bare Cursor model IDs', () => {
const result = resolveModelString('composer-1');
expect(result).toBe('cursor-composer-1');
});
it('should handle cursor-auto model', () => {
const result = resolveModelString('cursor-auto');
expect(result).toBe('cursor-auto');
});
it('should handle all known Cursor model IDs with prefix', () => {
const cursorModelIds = Object.keys(CURSOR_MODEL_MAP);
cursorModelIds.forEach((modelId) => {
const result = resolveModelString(`cursor-${modelId}`);
expect(result).toBe(`cursor-${modelId}`);
});
});
});
});
describe('getEffectiveModel', () => {
it('should prioritize explicit model over session and default', () => {
const result = getEffectiveModel('opus', 'haiku', 'gpt-5.2');
expect(result).toBe('claude-opus-4-5-20251101');
});
it('should use session model when explicit is not provided', () => {
const result = getEffectiveModel(undefined, 'sonnet', 'gpt-5.2');
expect(result).toBe(CLAUDE_MODEL_MAP.sonnet);
});
it('should use default when neither explicit nor session is provided', () => {
const customDefault = CLAUDE_MODEL_MAP.haiku;
const result = getEffectiveModel(undefined, undefined, customDefault);
expect(result).toBe(customDefault);
});
it('should use Claude default when no arguments provided', () => {
const result = getEffectiveModel();
expect(result).toBe(DEFAULT_MODELS.claude);
});
it('should handle explicit empty strings as undefined', () => {
const result = getEffectiveModel('', 'haiku');
expect(result).toBe(CLAUDE_MODEL_MAP.haiku);
});
});
describe('CLAUDE_MODEL_MAP', () => {
it('should have haiku, sonnet, opus mappings', () => {
expect(CLAUDE_MODEL_MAP).toHaveProperty('haiku');
expect(CLAUDE_MODEL_MAP).toHaveProperty('sonnet');
expect(CLAUDE_MODEL_MAP).toHaveProperty('opus');
});
it('should have valid Claude model strings', () => {
expect(CLAUDE_MODEL_MAP.haiku).toContain('haiku');
expect(CLAUDE_MODEL_MAP.sonnet).toContain('sonnet');
expect(CLAUDE_MODEL_MAP.opus).toContain('opus');
});
});
describe('DEFAULT_MODELS', () => {
it('should have claude default', () => {
expect(DEFAULT_MODELS).toHaveProperty('claude');
});
it('should have valid default model', () => {
expect(DEFAULT_MODELS.claude).toContain('claude');
});
});
});