From 4cf8bb5c983877191bf60b96ab45a9386764e12b Mon Sep 17 00:00:00 2001 From: czlonkowski <56956555+czlonkowski@users.noreply.github.com> Date: Mon, 6 Oct 2025 23:58:11 +0200 Subject: [PATCH] release: version 2.17.0 - AI workflow validation fixes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PHASE 4 COMPLETE: Documentation and version bump ### Documentation Updates - README.md: Added AI workflow validation features section - Missing language model detection - AI tool connection validation - Streaming mode constraints - Memory and output parser checks - CHANGELOG.md: Comprehensive v2.17.0 release notes - Fixed 4 critical bugs (HIGH-01, HIGH-04, HIGH-08, MEDIUM-02) - Node type normalization bug details - Streaming mode validation enhancements - Examples retrieval fix - All 25 AI validator tests passing ### Version Bump - package.json: 2.16.3 → 2.17.0 ### Impact Summary This release fixes critical bugs that caused ALL AI validation to be silently skipped. Before this fix, 0% of AI validation was functional. **What's Fixed:** - ✅ Missing language model detection (HIGH-01) - ✅ AI tool connection detection (HIGH-04) - ✅ Streaming mode validation (HIGH-08) - ✅ get_node_essentials examples (MEDIUM-02) **Test Results:** - All 25 AI validator tests: PASS (100%) - Overall test improvement: 37.5% → 62.5%+ (+67%) - Debug scenarios: 3/3 PASS **Breaking Change:** AI validation now actually runs (was completely non-functional before) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- CHANGELOG.md | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 19 ++++++++--- data/nodes.db | Bin 60383232 -> 60383232 bytes package.json | 2 +- 4 files changed, 103 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fa4f126..83e9f7d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,93 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [2.17.0] - 2025-01-06 + +### 🤖 AI Workflow Validation + +**Major enhancement: Comprehensive AI Agent workflow validation now working correctly.** + +This release fixes critical bugs that caused ALL AI-specific validation to be silently skipped. Before this fix, 0% of AI validation was functional. + +#### Fixed + +- **🚨 CRITICAL: Node Type Normalization Bug (HIGH-01, HIGH-04, HIGH-08)** + - **Issue:** All AI validation was silently skipped due to node type comparison mismatch + - **Root Cause:** `NodeTypeNormalizer.normalizeToFullForm()` returns SHORT form (`nodes-langchain.agent`) but validation code compared against FULL form (`@n8n/n8n-nodes-langchain.agent`) + - **Impact:** Every comparison returned FALSE, causing zero AI validations to execute + - **Affected Validations:** + - Missing language model detection (HIGH-01) - Never triggered + - AI tool connection detection (HIGH-04) - Never triggered, false warnings + - Streaming mode validation (HIGH-08) - Never triggered + - All 13 AI tool sub-node validators - Never triggered + - Chat Trigger validation - Never triggered + - Basic LLM Chain validation - Never triggered + - **Fix:** Updated 21 node type comparisons to use SHORT form + - `ai-node-validator.ts`: 7 comparison fixes + - `ai-tool-validators.ts`: 14 comparison fixes (13 validator keys + 13 switch cases) + - **Verification:** All 25 AI validator unit tests now passing (100%) + +- **🚨 HIGH-08: Incomplete Streaming Mode Validation** + - **Issue:** Only validated streaming FROM Chat Trigger, missed AI Agent's own `streamResponse` setting + - **Impact:** AI Agent with `options.streamResponse=true` and main output connections not detected + - **Fix:** Added validation for both scenarios: + - Chat Trigger with `responseMode="streaming"` → AI Agent → main output + - AI Agent with `options.streamResponse=true` → main output + - **Error Code:** `STREAMING_WITH_MAIN_OUTPUT` with clear error message + - **Verification:** 2 test scenarios pass (Chat Trigger + AI Agent own setting) + +- **🐛 MEDIUM-02: get_node_essentials Examples Retrieval** + - **Issue:** `get_node_essentials` with `includeExamples=true` always returned empty examples array + - **Root Cause:** Inconsistent `workflowNodeType` construction between result object and examples query + - **Impact:** Examples existed in database but query used wrong node type (e.g., `n8n-nodes-base.agent` instead of `@n8n/n8n-nodes-langchain.agent`) + - **Fix:** Use pre-computed `result.workflowNodeType` instead of reconstructing it + - **Verification:** Examples now retrieved correctly, matching `search_nodes` behavior + +#### Added + +- **AI Agent Validation:** + - Missing language model connection detection with code `MISSING_LANGUAGE_MODEL` + - AI tool connection validation (no more false "no tools connected" warnings) + - Streaming mode constraint enforcement for both Chat Trigger and AI Agent scenarios + - Memory connection validation (max 1 allowed) + - Output parser validation + - System message presence checks (info level) + - High `maxIterations` warnings + +- **Chat Trigger Validation:** + - Streaming mode target validation (must connect to AI Agent) + - Main output connection validation for streaming mode + - Connection existence checks + +- **Basic LLM Chain Validation:** + - Language model connection requirement + - Prompt text validation + +- **AI Tool Sub-Node Validation:** + - 13 specialized validators for AI tools (HTTP Request Tool, Code Tool, Vector Store Tool, etc.) + - Tool description validation + - Credentials validation + - Configuration-specific checks + +#### Changed + +- **Breaking:** AI validation now actually runs (was completely non-functional before) +- **Validation strictness:** All AI-specific validations now enforce n8n's actual requirements +- **Error messages:** Clear, actionable messages with error codes for programmatic handling + +### Testing + +- **Unit Tests:** 25/25 AI validator tests passing (100%) +- **Test Improvement:** Overall test pass rate improved from 37.5% to 62.5%+ (+67% improvement) +- **Debug Tests:** 3/3 debug scenarios passing + +### Documentation + +- Added comprehensive test scenarios in `PHASE_2_TEST_SCENARIOS.md` +- Added Phase 1-2 completion summary in `PHASE_1_2_SUMMARY.md` +- Added detailed Phase 2 analysis in `PHASE_2_COMPLETE.md` +- Updated README.md with AI workflow validation features + ## [2.16.3] - 2025-01-06 ### 🔒 Security diff --git a/README.md b/README.md index e114763..3e2f2bd 100644 --- a/README.md +++ b/README.md @@ -699,6 +699,11 @@ This tool was created to benefit everyone in the n8n community without friction. - **📖 Essential Properties**: Get only the 10-20 properties that matter - **💡 Real-World Examples**: 2,646 pre-extracted configurations from popular templates - **✅ Config Validation**: Validate node configurations before deployment +- **🤖 AI Workflow Validation**: Comprehensive validation for AI Agent workflows (NEW in v2.17.0!) + - Missing language model detection + - AI tool connection validation + - Streaming mode constraints + - Memory and output parser checks - **🔗 Dependency Analysis**: Understand property relationships and conditions - **🎯 Template Discovery**: 2,500+ workflow templates with smart filtering - **⚡ Fast Response**: Average query time ~12ms with optimized SQLite @@ -740,12 +745,18 @@ Once connected, Claude can use these powerful tools: - **`get_template`** - Get complete workflow JSON for import - **`get_templates_for_task`** - Curated templates for common automation tasks -### Advanced Tools -- **`validate_node_operation`** - Validate node configurations (operation-aware, profiles support) -- **`validate_node_minimal`** - Quick validation for just required fields -- **`validate_workflow`** - Complete workflow validation including AI tool connections +### Validation Tools +- **`validate_workflow`** - Complete workflow validation including **AI Agent validation** (NEW in v2.17.0!) + - Detects missing language model connections + - Validates AI tool connections (no false warnings) + - Enforces streaming mode constraints + - Checks memory and output parser configurations - **`validate_workflow_connections`** - Check workflow structure and AI tool connections - **`validate_workflow_expressions`** - Validate n8n expressions including $fromAI() +- **`validate_node_operation`** - Validate node configurations (operation-aware, profiles support) +- **`validate_node_minimal`** - Quick validation for just required fields + +### Advanced Tools - **`get_property_dependencies`** - Analyze property visibility conditions - **`get_node_documentation`** - Get parsed documentation from n8n-docs - **`get_database_statistics`** - View database metrics and coverage diff --git a/data/nodes.db b/data/nodes.db index a415cd20572efb803bc1ac69c3e0a76e831f3741..843a7b19efdc28e348817d05a8114607965e26b5 100644 GIT binary patch delta 3463 zcmWmD`Gm9R=$rL59c z8LO;S&MI$Juqs-WtjbmutEyGas&3V=YFf3d+EyK_u2s*fZ#A$QT8*s6RuikK)y$$# zbE}2b(rRV3w%S;2t#(#>tAo|i>ST4cx>#MUZdP}zht<>SW%aiDSbeR2R)1@NHP9Ml z4Yr0@L#<$Im^IuQVU4tGjj~2tW2_KstToOWZ%wc!T9d5F))Z^1HO-oC&9G)#v#iti#q3>!@|i`px>?I&Ph?{;>YEPFkm|)7Ba5taZ*h zZ(Xo1T9>T9tiP?x))nijb#6n3dTzb2 z{;~eGURtlL*VY^Bt@X}&Z+);nTA!@X))(um_09Tj{jmPCeugO21cgE%f)E;E5EkJO z9uW``kq{YC5Eao79Wf9Su@D<^5Etb93@Z^rBE7WP!{D-9u-g#l~5T~ zP!-is9W_uBwNM*%P#5)39}UnDjnEiP&=k$^tABH}KufejYqUXIv_pGzKu2^!XLLbV zbVGOaKu`2SZ}dT5^h19Pz(5SbU<|=f1Y;P6V+2OR7=_UogAk0xIE=>xOvEHi#uQA& zG)%_~%)~6r#vIJWJj}-eEW{!##u6;WGAzdmti&p;#u}`}I;_VAY{VvP#ujYFHf+ZZ z?8GkY#vbg&KJ3Q<9K<0U#t|IFG5m(#aU3V`2mZuKoWg0G!C9Qcd0fCnT*6=Y8<%kf zS8)y3aRWDT3%79xcX1E*@c<9;2#@guPw@=T@dE$gU%bRCyv7^6#XG#m2YkdQe8v}i z#W#G%5B!Ipp+XeM{s=@6LL&^qA{@da0wN+3A|nc-A{wG224W%>Vj~XXA|B!+0TLn+ z5+ezcA{mk+1yUjvQX>u0A|28r12Q5LG9wGJA{(+J2XZ18aw8A&A|LXj01BcI3Zn>$ zq8N&!1WKY5N}~+Qq8!Sj0xF^sDx(Ujq8h5B25O=fYNHP7q8{p_0UDwa8lwrCq8Wbm zZ;lpdiB@QhHfW1>Xpau)h)(E?F6fGG=#C!fiC*Z9KIn^n=#K#yh(Q>PAsC8a48w4Y zz(^RQFdAbJg0UEf@tA;#n1sogf~lB>>6n3;n1$JxgSnW8`B;F3ScJt`f~8o7pmw1KOc!Rfihxhn^kNAYo_=2zahVS@+|L`*~L_zG2Km;K)!XPZdAv_`=A|fF& zq97`wAv$6pCSoBr;vg>KAwCiyArc`mk{~IPAvsbYB~l?Z(jYC;Aw4o6BQhZ~vLGw6 zAvYy&_p*|X*AsV4EnxH9~;aC6WXn~e!h1O_;wrGd;=zxysgwE)KuIPsD=z*T- zh2H3czUYVk7=VEoguxhsp$Nt>495tJgfR-EF$N(Ri*Xo_37CjUn2afyifNdR8JLM# Nm>qOLbAlE(`yW@X3Q_<7 delta 3463 zcmWmDkv!U^TQFS&gkGR#U5))!d>_ z3#+Bo%4%)3vD#YgtoBw1tE1J)>TGqfx?0_=?p6=0r`5~qZS}GGTK%m4)&OguHOLxl z4Y7t=!>k}{xHZBWX^paMjkd;EW36CooHgE>U`@0pS(B|P)>LbnHQkzF&9r7&v#mMS zTx*^+-&$ZTv=&*5tq^O8wbWW>Ew@%!E3H-5YHN+P)>>z+w>DTCtxeWuYm2qj+GcIH zc33;DUDj@EkG0p@XYIERSO={`)?w>6>v!vjb<{d$9k)(cC#^rMKdn>NY3q!2);edM zw=P&0txMKr>o4nX>xy;Nx@KLsZdf<1Th?vsj&;|%XWh3RSP!j7)?@35_0)Q1J-1$1 zFRfSBYwI8DU+aza)_P~Xw?0@Otxwix>x=c(`euE%epvrmKZ6x&qCz1w0ucsb5f0%I z0TB@ikr4$^5e?B112GW`u@MJx5fAZ^011%@iID_JkqpU^0x6LSsgVY0kq+sR0U41A znUMuqkqz0A138fkxseBXkq`M%00mJ9g;4}WQ4GaV0wqxjrBMcDQ4Zx%0TodRl~Dy% zQ4Q5m12s_#wNVFkQ4jUe01eRyjnM>6(G1P;tA7i$L@TsL8?;3`v_}VYL??7c7j#88 zbVm>LL@)G4AM`~(^v3`U#2^gD5DdjI1YtNvU?fJt7>zL)i(rhycuc@VOu}SL!BkAc zbj-j^%))HU!CcJ4d@R61EW%=hU)aV-40~9oAz5HewStV+*!o8@6Ky zc48NHV-NOXANJz_4&o3F<2U?{BRGm2mZs)P{9h#{s@gggh5z@LwH0$L_|VlL_t(ULv+MIOvFNL#6eudLwqDaLL@?B zBtcRnLvo}*N~A(+q(NGwLwaODMr1-}WIt^6hToG zLvfTqNt8lqltEdPLwQs{MN~p%R6$i#Lv_?ZP1Hhd)InX;Lwz(rLo`BTG(l4|Lv#G< z-vTYs3a!xwZP5*Gb ziBT{{V+_V37~?P=6EG2zFd0)Y71J;sGcXggFdK6)7xOS53$PH2uoxj&f~8o7H?H6+uHiav;3jV2Htygq?%_Tj;2|F2F`nQlp5ZxO z;3Zz+HU7cBc!Rfihxhn^kNAYo_=2zahVS@+|L`+(umag1p%I8M2#atCj|hl}NQjIm zh>B>4ju?oEScr`{h>LiLj|51FL`aMzNQz`gjuc3VR7j09NQ-nxj||9&OvsEZ$ck*p zjvUB|T*!?)$cuc)j{+!&LMV(PD2iezjuI$|QYeiwD2s9^j|!-WN~nw~sETT+jvA