n8n-mcp/tests/integration/ai-validation/README.md

# AI Validation Integration Tests

Comprehensive integration tests for AI workflow validation introduced in v2.17.0.

## Overview

These tests validate ALL AI validation operations against a REAL n8n instance. They verify:
- AI Agent validation rules
- Chat Trigger validation constraints
- Basic LLM Chain validation requirements
- AI Tool sub-node validation (HTTP Request, Code, Vector Store, Workflow, Calculator)
- End-to-end workflow validation
- Multi-error detection
- Node type normalization (bug fix validation)

## Test Files

### 1. `helpers.ts`
Utility functions for creating AI workflow components:
- `createAIAgentNode()` - AI Agent with configurable options
- `createChatTriggerNode()` - Chat Trigger with streaming modes
- `createBasicLLMChainNode()` - Basic LLM Chain
- `createLanguageModelNode()` - OpenAI/Anthropic models
- `createHTTPRequestToolNode()` - HTTP Request Tool
- `createCodeToolNode()` - Code Tool
- `createVectorStoreToolNode()` - Vector Store Tool
- `createWorkflowToolNode()` - Workflow Tool
- `createCalculatorToolNode()` - Calculator Tool
- `createMemoryNode()` - Buffer Window Memory
- `createRespondNode()` - Respond to Webhook
- `createAIConnection()` - AI connection helper (reversed for langchain)
- `createMainConnection()` - Standard n8n connection
- `mergeConnections()` - Merge multiple connection objects
- `createAIWorkflow()` - Complete workflow builder

### 2. `ai-agent-validation.test.ts` (7 tests)
Tests AI Agent validation:
- ✅ Detects missing language model (MISSING_LANGUAGE_MODEL error)
- ✅ Validates AI Agent with language model connected
- ✅ Detects tool connections correctly (no false warnings)
- ✅ Validates streaming mode constraints (Chat Trigger)
- ✅ Validates AI Agent own streamResponse setting
- ✅ Detects multiple memory connections (error)
- ✅ Validates complete AI workflow (all components)

### 3. `chat-trigger-validation.test.ts` (5 tests)
Tests Chat Trigger validation:
- ✅ Detects streaming to non-AI-Agent (STREAMING_WRONG_TARGET error)
- ✅ Detects missing connections (MISSING_CONNECTIONS error)
- ✅ Validates valid streaming setup
- ✅ Validates lastNode mode with AI Agent
- ✅ Detects streaming agent with output connection

### 4. `llm-chain-validation.test.ts` (6 tests)
Tests Basic LLM Chain validation:
- ✅ Detects missing language model (MISSING_LANGUAGE_MODEL error)
- ✅ Detects missing prompt text (MISSING_PROMPT_TEXT error)
- ✅ Validates complete LLM Chain
- ✅ Validates LLM Chain with memory
- ✅ Detects multiple language models (error - no fallback support)
- ✅ Detects tools connection (TOOLS_NOT_SUPPORTED error)

### 5. `ai-tool-validation.test.ts` (9 tests)
Tests AI Tool validation:

**HTTP Request Tool:**
- ✅ Detects missing toolDescription (MISSING_TOOL_DESCRIPTION)
- ✅ Detects missing URL (MISSING_URL)
- ✅ Validates valid HTTP Request Tool

**Code Tool:**
- ✅ Detects missing code (MISSING_CODE)
- ✅ Validates valid Code Tool

**Vector Store Tool:**
- ✅ Detects missing toolDescription
- ✅ Validates valid Vector Store Tool

**Workflow Tool:**
- ✅ Detects missing workflowId (MISSING_WORKFLOW_ID)
- ✅ Validates valid Workflow Tool

**Calculator Tool:**
- ✅ Validates Calculator Tool (no configuration needed)

### 6. `e2e-validation.test.ts` (5 tests)
End-to-end validation tests:
- ✅ Validates and creates complex AI workflow (7 nodes, all components)
- ✅ Detects multiple validation errors (5+ errors in one workflow)
- ✅ Validates streaming workflow without main output
- ✅ Validates non-streaming workflow with main output
- ✅ Tests node type normalization (v2.17.0 bug fix validation)

## Running Tests

### Run All AI Validation Tests
```bash
npm test -- tests/integration/ai-validation --run
```

### Run Specific Test Suite
```bash
npm test -- tests/integration/ai-validation/ai-agent-validation.test.ts --run
npm test -- tests/integration/ai-validation/chat-trigger-validation.test.ts --run
npm test -- tests/integration/ai-validation/llm-chain-validation.test.ts --run
npm test -- tests/integration/ai-validation/ai-tool-validation.test.ts --run
npm test -- tests/integration/ai-validation/e2e-validation.test.ts --run
```

### Prerequisites

1. **n8n Instance**: Real n8n instance required (not mocked)
2. **Environment Variables**:
   ```env
   N8N_API_URL=http://localhost:5678
   N8N_API_KEY=your-api-key
   TEST_CLEANUP=true  # Auto-cleanup test workflows (default: true)
   ```
3. **Build**: Run `npm run build` before testing

## Test Infrastructure

### Cleanup
- All tests use `TestContext` for automatic workflow cleanup
- Workflows are tagged with `mcp-integration-test` and `ai-validation`
- Cleanup runs in `afterEach` hooks
- Orphaned workflow cleanup runs in `afterAll` (non-CI only)

### Workflow Naming
- All test workflows use timestamps: `[MCP-TEST] Description 1696723200000`
- Prevents name collisions
- Easy identification in n8n UI

### Connection Patterns
- **Main connections**: Standard n8n flow (A → B)
- **AI connections**: Reversed flow (Language Model → AI Agent)
- Uses helper functions to ensure correct connection structure

## Key Validation Checks

### AI Agent
- Language model connections (1 or 2 for fallback)
- Output parser configuration
- Prompt type validation (auto vs define)
- System message recommendations
- Streaming mode constraints (CRITICAL)
- Memory connections (0-1 max)
- Tool connections
- maxIterations validation

### Chat Trigger
- responseMode validation (streaming vs lastNode)
- Streaming requires AI Agent target
- AI Agent in streaming mode: NO main output allowed

### Basic LLM Chain
- Exactly 1 language model (no fallback)
- Memory connections (0-1 max)
- No tools support (error if connected)
- Prompt configuration validation

### AI Tools
- HTTP Request Tool: requires toolDescription + URL
- Code Tool: requires jsCode
- Vector Store Tool: requires toolDescription + vector store connection
- Workflow Tool: requires workflowId
- Calculator Tool: no configuration required

## Validation Error Codes

Tests verify these error codes are correctly detected:

- `MISSING_LANGUAGE_MODEL` - No language model connected
- `MISSING_TOOL_DESCRIPTION` - Tool missing description
- `MISSING_URL` - HTTP tool missing URL
- `MISSING_CODE` - Code tool missing code
- `MISSING_WORKFLOW_ID` - Workflow tool missing ID
- `MISSING_PROMPT_TEXT` - Prompt type=define but no text
- `MISSING_CONNECTIONS` - Chat Trigger has no output
- `STREAMING_WITH_MAIN_OUTPUT` - AI Agent in streaming mode with main output
- `STREAMING_WRONG_TARGET` - Chat Trigger streaming to non-AI-Agent
- `STREAMING_AGENT_HAS_OUTPUT` - Streaming agent has output connection
- `MULTIPLE_LANGUAGE_MODELS` - LLM Chain with multiple models
- `MULTIPLE_MEMORY_CONNECTIONS` - Multiple memory connected
- `TOOLS_NOT_SUPPORTED` - Basic LLM Chain with tools
- `TOO_MANY_LANGUAGE_MODELS` - AI Agent with 3+ models
- `FALLBACK_MISSING_SECOND_MODEL` - needsFallback=true but 1 model
- `MULTIPLE_OUTPUT_PARSERS` - Multiple output parsers

## Bug Fix Validation

### v2.17.0 Node Type Normalization
Test 5 in `e2e-validation.test.ts` validates the fix for node type normalization:
- Creates AI Agent + OpenAI Model + HTTP Request Tool
- Connects tool via ai_tool connection
- Verifies NO false "no tools connected" warning
- Validates workflow is valid

This test would have caught the bug where:
```typescript
// BUG: Incorrect comparison
sourceNode.type === 'nodes-langchain.chatTrigger'  // ❌ Never matches

// FIX: Use normalizer
NodeTypeNormalizer.normalizeToFullForm(sourceNode.type) === 'nodes-langchain.chatTrigger'  // ✅ Works
```

## Success Criteria

All tests should:
- ✅ Create workflows in real n8n
- ✅ Validate using actual MCP tools (handleValidateWorkflow)
- ✅ Verify validation results match expected outcomes
- ✅ Clean up after themselves (no orphaned workflows)
- ✅ Run in under 30 seconds each
- ✅ Be deterministic (no flakiness)

## Test Coverage

Total: **32 tests** covering:
- **7 AI Agent tests** - Complete AI Agent validation logic
- **5 Chat Trigger tests** - Streaming mode and connection validation
- **6 Basic LLM Chain tests** - LLM Chain constraints and requirements
- **9 AI Tool tests** - All AI tool sub-node types
- **5 E2E tests** - Complex workflows and multi-error detection

## Coverage Summary

### Validation Features Tested
- ✅ Language model connections (required, fallback)
- ✅ Output parser configuration
- ✅ Prompt type validation
- ✅ System message checks
- ✅ Streaming mode constraints
- ✅ Memory connections (single)
- ✅ Tool connections
- ✅ maxIterations validation
- ✅ Chat Trigger modes (streaming, lastNode)
- ✅ Tool description requirements
- ✅ Tool-specific parameters (URL, code, workflowId)
- ✅ Multi-error detection
- ✅ Node type normalization
- ✅ Connection validation (missing, invalid)

### Edge Cases Tested
- ✅ Empty/missing required fields
- ✅ Invalid configurations
- ✅ Multiple connections (when not allowed)
- ✅ Streaming with main output (forbidden)
- ✅ Tool connections to non-agent nodes
- ✅ Fallback model configuration
- ✅ Complex workflows with all components

## Recommendations

### Additional Tests (Future)
1. **Performance tests** - Validate large AI workflows (20+ nodes)
2. **Credential validation** - Test with invalid/missing credentials
3. **Expression validation** - Test n8n expressions in AI node parameters
4. **Cross-version tests** - Test different node typeVersions
5. **Concurrent validation** - Test multiple workflows in parallel

### Test Maintenance
- Update tests when new AI nodes are added
- Add tests for new validation rules
- Keep helpers.ts updated with new node types
- Verify error codes match specification

## Notes

- Tests create real workflows in n8n (not mocked)
- Each test is independent (no shared state)
- Workflows are automatically cleaned up
- Tests use actual MCP validation handlers
- All AI connection types are tested
- Streaming mode validation is comprehensive
- Node type normalization is validated