czlonkowski
caf9383ba1
test: Add comprehensive edge case coverage for telemetry quick wins
...
Added 20 edge case tests based on code review recommendations:
**Duplicate ID Validation (4 tests)**:
- Multiple duplicate IDs (3+ nodes with same ID)
- Duplicate IDs with same node type
- Duplicate IDs with empty/null node names
- Duplicate IDs with missing node properties
**AI Agent Validator (16 tests)**:
maxIterations edge cases (7 tests):
- Boundary values: 0 (reject), 1 (accept), 51 (warn), MAX_SAFE_INTEGER (warn)
- Invalid types: NaN (reject), negative decimal (reject)
- Threshold testing: 50 vs 51
promptType validation (4 tests):
- Whitespace-only text (reject)
- Very long text 3200+ chars (accept)
- undefined/null text (reject)
System message validation (5 tests):
- Empty/whitespace messages (suggest adding)
- Very long messages >1000 chars (accept)
- Special characters, emojis, unicode (accept)
- Multi-line formatting (accept)
- Boundary: 19 chars (warn), 20 chars (accept)
**Test Quality Improvements**:
- Fixed flaky system message test (changed from expect.stringContaining to .some())
- All tests are deterministic
- Comprehensive inline comments
- Follows existing test patterns
All 20 new tests passing. Zero regressions.
Concieved by Romuald Członkowski - www.aiadvisors.pl/en
2025-11-08 18:49:59 +01:00
czlonkowski
60ab66d64d
feat: telemetry-driven quick wins to reduce AI agent validation errors by 30-40%
...
Enhanced tools documentation, duplicate ID errors, and AI Agent validator based on telemetry analysis of 593 validation errors across 3 categories:
- 378 errors: Duplicate node IDs (64%)
- 179 errors: AI Agent configuration (30%)
- 36 errors: Other validations (6%)
Quick Win #1 : Enhanced tools documentation (src/mcp/tools-documentation.ts)
- Added prominent warnings to call get_node_essentials() FIRST before configuring nodes
- Emphasized 5KB vs 100KB+ size difference between essentials and full info
- Updated workflow patterns to prioritize essentials over get_node_info
Quick Win #2 : Improved duplicate ID error messages (src/services/workflow-validator.ts)
- Added crypto import for UUID generation examples
- Enhanced error messages with node indices, names, and types
- Included crypto.randomUUID() example in error messages
- Helps AI agents understand EXACTLY which nodes conflict and how to fix
Quick Win #3 : Added AI Agent node-specific validator (src/services/node-specific-validators.ts)
- Validates prompt configuration (promptType + text requirement)
- Checks maxIterations bounds (1-50 recommended)
- Suggests error handling (onError + retryOnFail)
- Warns about high iteration limits (cost/performance impact)
- Integrated into enhanced-config-validator.ts
Test Coverage:
- Added duplicate ID validation tests (workflow-validator.test.ts)
- Added AI Agent validator tests (node-specific-validators.test.ts:2312-2491)
- All new tests passing (3527 total passing)
Version: 2.22.12 → 2.22.13
Expected Impact: 30-40% reduction in AI agent validation errors
Technical Details:
- Telemetry analysis: 593 validation errors (Dec 2024 - Jan 2025)
- 100% error recovery rate maintained (validation working correctly)
- Root cause: Documentation/guidance gaps, not validation logic failures
- Solution: Proactive guidance at decision points
References:
- Telemetry analysis findings
- Issue #392 (helpful error messages pattern)
- Existing Slack validator pattern (node-specific-validators.ts:98-230)
Concieved by Romuald Członkowski - www.aiadvisors.pl/en
2025-11-08 18:07:26 +01:00
czlonkowski
6699a1d34c
test: implement comprehensive testing improvements from PR #104 review
...
Major improvements based on comprehensive test suite review:
Test Fixes:
- Fix all 78 failing tests across logger, MSW, and validator tests
- Fix console spy management in logger tests with proper DEBUG env handling
- Fix MSW test environment restoration in session-management.test.ts
- Fix workflow validator tests by adding proper node connections
- Fix mock setup issues in edge case tests
Test Organization:
- Split large config-validator.test.ts (1,075 lines) into 4 focused files
- Rename 63+ tests to follow "should X when Y" naming convention
- Add comprehensive edge case test files for all major validators
- Create tests/README.md with testing guidelines and best practices
New Features:
- Add ConfigValidator.validateBatch() method for bulk validation
- Add edge case coverage for null/undefined, boundaries, invalid data
- Add CI-aware performance test timeouts
- Add JSDoc comments to test utilities and factories
- Add workflow duplicate node name validation tests
Results:
- All tests passing: 1,356 passed, 19 skipped
- Test coverage: 85.34% statements, 85.3% branches
- From 78 failures to 0 failures
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-07-30 13:44:35 +02:00
czlonkowski
b49043171e
test: Phase 3 - Create comprehensive unit tests for services
...
- Add unit tests for ConfigValidator with 44 test cases (95.21% coverage)
- Create test templates for 7 major services:
- PropertyFilter (23 tests)
- ExampleGenerator (35 tests)
- TaskTemplates (36 tests)
- PropertyDependencies (21 tests)
- EnhancedConfigValidator (8 tests)
- ExpressionValidator (11 tests)
- WorkflowValidator (9 tests)
- Fix service implementations to handle edge cases discovered during testing
- Add comprehensive testing documentation:
- Phase 3 testing plan with priorities and timeline
- Context documentation for quick implementation
- Mocking strategy for complex dependencies
- All 262 tests now passing (up from 75)
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Claude <noreply@anthropic.com >
2025-07-28 14:15:09 +02:00
