feat: telemetry-driven quick wins to reduce AI agent validation errors by 30-40%

Enhanced tools documentation, duplicate ID errors, and AI Agent validator based on telemetry analysis of 593 validation errors across 3 categories:
- 378 errors: Duplicate node IDs (64%)
- 179 errors: AI Agent configuration (30%)
- 36 errors: Other validations (6%)

Quick Win #1: Enhanced tools documentation (src/mcp/tools-documentation.ts)
- Added prominent warnings to call get_node_essentials() FIRST before configuring nodes
- Emphasized 5KB vs 100KB+ size difference between essentials and full info
- Updated workflow patterns to prioritize essentials over get_node_info

Quick Win #2: Improved duplicate ID error messages (src/services/workflow-validator.ts)
- Added crypto import for UUID generation examples
- Enhanced error messages with node indices, names, and types
- Included crypto.randomUUID() example in error messages
- Helps AI agents understand EXACTLY which nodes conflict and how to fix

Quick Win #3: Added AI Agent node-specific validator (src/services/node-specific-validators.ts)
- Validates prompt configuration (promptType + text requirement)
- Checks maxIterations bounds (1-50 recommended)
- Suggests error handling (onError + retryOnFail)
- Warns about high iteration limits (cost/performance impact)
- Integrated into enhanced-config-validator.ts

Test Coverage:
- Added duplicate ID validation tests (workflow-validator.test.ts)
- Added AI Agent validator tests (node-specific-validators.test.ts:2312-2491)
- All new tests passing (3527 total passing)

Version: 2.22.12 → 2.22.13

Expected Impact: 30-40% reduction in AI agent validation errors

Technical Details:
- Telemetry analysis: 593 validation errors (Dec 2024 - Jan 2025)
- 100% error recovery rate maintained (validation working correctly)
- Root cause: Documentation/guidance gaps, not validation logic failures
- Solution: Proactive guidance at decision points

References:
- Telemetry analysis findings
- Issue #392 (helpful error messages pattern)
- Existing Slack validator pattern (node-specific-validators.ts:98-230)

Concieved by Romuald Członkowski - www.aiadvisors.pl/en
This commit is contained in:
czlonkowski
2025-11-08 18:07:26 +01:00
parent eee52a7f53
commit 60ab66d64d
20 changed files with 4918 additions and 15 deletions

View File

@@ -84,14 +84,16 @@ When working with Code nodes, always start by calling the relevant guide:
## Standard Workflow Pattern
⚠️ **CRITICAL**: Always call get_node_essentials() FIRST before configuring any node!
1. **Find** the node you need:
- search_nodes({query: "slack"}) - Search by keyword
- list_nodes({category: "communication"}) - List by category
- list_ai_tools() - List AI-capable nodes
2. **Configure** the node:
- get_node_essentials("nodes-base.slack") - Get essential properties only (5KB)
- get_node_info("nodes-base.slack") - Get complete schema (100KB+)
2. **Configure** the node (ALWAYS START WITH ESSENTIALS):
- get_node_essentials("nodes-base.slack") - Get essential properties FIRST (5KB, shows required fields)
- get_node_info("nodes-base.slack") - Get complete schema only if essentials insufficient (100KB+)
- search_node_properties("nodes-base.slack", "auth") - Find specific properties
3. **Validate** before deployment:
@@ -107,8 +109,8 @@ When working with Code nodes, always start by calling the relevant guide:
- list_ai_tools - List all AI-capable nodes with usage guidance
**Configuration Tools**
- get_node_essentials - Returns 10-20 key properties with examples
- get_node_info - Returns complete node schema with all properties
- get_node_essentials - ✅ CALL THIS FIRST! Returns 10-20 key properties with examples and required fields
- get_node_info - Returns complete node schema (only use if essentials is insufficient)
- search_node_properties - Search for specific properties within a node
- get_property_dependencies - Analyze property visibility dependencies