mirror of
https://github.com/czlonkowski/n8n-mcp.git
synced 2026-01-30 06:22:04 +00:00
CRITICAL BUG FIX: Fixed array index corruption in multi-output nodes (Switch, IF with multiple handlers, Merge) when rewiring connections. Problem: - applyRemoveConnection() filtered out empty arrays after removing connections - This caused indices to shift in multi-output nodes - Example: Switch.main = [[H0], [H1], [H2]] -> remove H1 -> [[H0], [H2]] - H2 moved from index 2 to index 1, corrupting workflow structure Root Cause: ```typescript // Line 697 - BUGGY CODE: workflow.connections[node][output] = connections.filter(conns => conns.length > 0); ``` Solution: - Only remove trailing empty arrays - Preserve intermediate empty arrays to maintain index integrity - Example: [[H0], [], [H2]] stays [[H0], [], [H2]] not [[H0], [H2]] Impact: - Prevents production-breaking workflow corruption - Fixes rewireConnection operation for multi-output nodes - Critical for AI agents working with complex workflows Testing: - Added integration test for Switch node rewiring with array index verification - Test creates 4-output Switch node, rewires middle connection - Verifies indices 0, 2, 3 unchanged after rewiring index 1 - All 137 unit tests + 12 integration tests passing Discovered by: @agent-n8n-mcp-tester during comprehensive testing Issue: #272 (Connection Operations - Phase 1) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1307 lines
45 KiB
Markdown
1307 lines
45 KiB
Markdown
# Comprehensive Integration Testing Plan
|
||
|
||
## Status
|
||
|
||
**Phase 1: Foundation** ✅ **COMPLETE** (October 3, 2025)
|
||
- All utility files created and tested
|
||
- Webhook workflows created on `https://n8n-test.n8n-mcp.com`
|
||
- GitHub secrets configured
|
||
- Critical fix: Updated credentials to use webhook URLs instead of IDs
|
||
- Environment loading fixed to support real n8n API integration tests
|
||
|
||
**Phase 2: Workflow Creation Tests** ✅ **COMPLETE** (October 3, 2025)
|
||
- 15 test scenarios implemented and passing
|
||
- P0 bug verification confirmed (FULL node type format)
|
||
- All test categories covered: base nodes, advanced features, error scenarios, edge cases
|
||
- Documented actual n8n API behavior (validation at execution time, not creation time)
|
||
- Test file: `tests/integration/n8n-api/workflows/create-workflow.test.ts` (484 lines)
|
||
|
||
**Phase 3: Workflow Retrieval Tests** ✅ **COMPLETE** (October 3, 2025)
|
||
- 11 test scenarios implemented and passing
|
||
- All MCP retrieval handlers tested: handleGetWorkflow, handleGetWorkflowDetails, handleGetWorkflowStructure, handleGetWorkflowMinimal
|
||
- Test files:
|
||
- `get-workflow.test.ts` (3 scenarios)
|
||
- `get-workflow-details.test.ts` (4 scenarios)
|
||
- `get-workflow-structure.test.ts` (2 scenarios)
|
||
- `get-workflow-minimal.test.ts` (2 scenarios)
|
||
|
||
**Phase 4: Workflow Update Tests** ✅ **COMPLETE** (October 4, 2025)
|
||
- 42 test scenarios implemented and passing
|
||
- Enhanced settings filtering (whitelist approach) to enable updates while maintaining Issue #248 protection
|
||
- All update operations tested:
|
||
- Full workflow updates: 7 scenarios (update-workflow.test.ts)
|
||
- Partial/diff-based updates: 32 scenarios covering all 15 operations (update-partial-workflow.test.ts)
|
||
- Validation error scenarios: 3 scenarios
|
||
- Critical fixes:
|
||
- Settings filtering uses OpenAPI spec whitelist (filters callerPolicy, preserves safe properties)
|
||
- All tests comply with n8n API requirements (name, nodes, connections, settings fields)
|
||
- Removed invalid "Update Connections" test (empty connections invalid for multi-node workflows)
|
||
- Version 2.15.4 released with comprehensive CHANGELOG entry
|
||
|
||
**Phase 5: Workflow Management Tests** ✅ **COMPLETE** (October 4, 2025)
|
||
- 16 test scenarios implemented and passing
|
||
- All workflow management operations tested:
|
||
- Delete workflow: 3 scenarios (delete-workflow.test.ts)
|
||
- List workflows: 13 scenarios (list-workflows.test.ts)
|
||
- Critical API compliance fixes:
|
||
- handleDeleteWorkflow: Now returns deleted workflow data (per n8n API spec)
|
||
- handleListWorkflows: Fixed tags parameter format (array → CSV string conversion)
|
||
- N8nApiClient.deleteWorkflow: Return type corrected (void → Workflow)
|
||
- WorkflowListParams.tags: Type corrected (string[] → string per n8n OpenAPI spec)
|
||
- Unit test coverage: Added 9 unit tests for handler coverage (100% coverage achieved)
|
||
- n8n-mcp-tester validation: All tools tested and working correctly in production
|
||
- Version 2.15.5 released with comprehensive CHANGELOG entry
|
||
- Test results: 71/71 integration tests passing (Phase 1-5 complete)
|
||
|
||
**Phase 6A: Workflow Validation Tests** ✅ **COMPLETE** (October 5, 2025)
|
||
- 12 test scenarios implemented and passing
|
||
- NodeRepository utility created for tests requiring node validation
|
||
- All validation profiles tested: strict, runtime, ai-friendly, minimal
|
||
- Test coverage:
|
||
- Valid workflows across all 4 profiles (4 scenarios)
|
||
- Invalid workflow detection (2 scenarios - bad node types, missing connections)
|
||
- Selective validation (3 scenarios - nodes only, connections only, expressions only)
|
||
- Error handling (2 scenarios - non-existent workflow, invalid profile)
|
||
- Response format verification (1 scenario)
|
||
- Critical discoveries:
|
||
- Response only includes errors/warnings fields when they exist (not empty arrays)
|
||
- Field name is errorCount, not totalErrors
|
||
- Tests require NodeRepository instance (added singleton utility)
|
||
- Test file: validate-workflow.test.ts (431 lines)
|
||
- Test results: 83/83 integration tests passing (Phase 1-5, 6A complete)
|
||
|
||
**Phase 6B: Workflow Autofix Tests** ✅ **COMPLETE** (October 5, 2025)
|
||
- 16 test scenarios implemented and passing
|
||
- All autofix operations tested: preview mode, apply mode, fix types, confidence filtering
|
||
- Test coverage:
|
||
- Preview mode (2 scenarios - expression-format, multiple fix types)
|
||
- Apply mode (2 scenarios - expression-format, webhook-missing-path)
|
||
- Fix type filtering (2 scenarios - single type, multiple types)
|
||
- Confidence thresholds (3 scenarios - high, medium, low)
|
||
- Max fixes parameter (1 scenario)
|
||
- No fixes available (1 scenario)
|
||
- Error handling (3 scenarios - non-existent workflow, invalid parameters)
|
||
- Response format verification (2 scenarios - preview and apply modes)
|
||
- Fix types tested:
|
||
- expression-format (missing = prefix for resource locators)
|
||
- typeversion-correction (outdated typeVersion values)
|
||
- error-output-config (error output configuration issues)
|
||
- node-type-correction (incorrect node types)
|
||
- webhook-missing-path (missing webhook path parameters)
|
||
- Code quality improvements:
|
||
- Fixed database resource leak in NodeRepository utility
|
||
- Added TypeScript interfaces (ValidationResponse, AutofixResponse)
|
||
- Replaced unsafe `as any` casts with proper type definitions
|
||
- All lint and typecheck errors resolved
|
||
- Test file: autofix-workflow.test.ts (855 lines)
|
||
- Test results: 99/99 integration tests passing (Phase 1-6 complete)
|
||
|
||
**Phase 7: Execution Management Tests** ✅ **COMPLETE** (October 5, 2025)
|
||
- 54 test scenarios implemented and passing
|
||
- All 4 execution management handlers tested against real n8n instance
|
||
- Test coverage:
|
||
- handleTriggerWebhookWorkflow (20 tests): All HTTP methods (GET/POST/PUT/DELETE), query params, JSON body, custom headers, error handling
|
||
- handleGetExecution (16 tests): All 4 retrieval modes (preview/summary/filtered/full), node filtering, item limits, input data inclusion, legacy compatibility
|
||
- handleListExecutions (13 tests): Status filtering (success/error/waiting), pagination with cursor, various limits (1/10/50/100), data inclusion control
|
||
- handleDeleteExecution (5 tests): Successful deletion, verification via fetch attempt, error handling
|
||
- Critical fix: Corrected response structure expectations (executions/returned vs data/count)
|
||
- Test files:
|
||
- trigger-webhook.test.ts (375 lines, 20 tests)
|
||
- get-execution.test.ts (429 lines, 16 tests)
|
||
- list-executions.test.ts (264 lines, 13 tests)
|
||
- delete-execution.test.ts (149 lines, 5 tests)
|
||
- Code review: APPROVED (9.5/10 quality score)
|
||
- Test results: 153/153 integration tests passing (Phase 1-7 complete)
|
||
|
||
**Phase 8: System Tools Tests** ✅ **COMPLETE** (October 5, 2025)
|
||
- 19 test scenarios implemented and passing
|
||
- All 3 system tool handlers tested against real n8n instance
|
||
- Test coverage:
|
||
- handleHealthCheck (3 tests): API connectivity verification, version information, feature availability
|
||
- handleListAvailableTools (7 tests): Complete tool inventory by category, configuration status, API limitations
|
||
- handleDiagnostic (9 tests): Environment checks, API connectivity, tools availability, verbose mode with debug info
|
||
- TypeScript type safety improvements:
|
||
- Created response-types.ts with comprehensive interfaces for all response types
|
||
- Replaced all 'as any' casts with proper TypeScript interfaces
|
||
- Added null-safety checks and non-null assertions
|
||
- Full type safety and IDE autocomplete support
|
||
- Test files:
|
||
- health-check.test.ts (117 lines, 3 tests)
|
||
- list-tools.test.ts (181 lines, 7 tests)
|
||
- diagnostic.test.ts (243 lines, 9 tests)
|
||
- response-types.ts (241 lines, comprehensive type definitions)
|
||
- Code review: APPROVED
|
||
- Test results: 172/172 integration tests passing (Phase 1-8 complete)
|
||
|
||
**🎉 INTEGRATION TEST SUITE COMPLETE**: All 18 MCP handlers fully tested
|
||
|
||
**Next Phase**: Update documentation and finalize integration testing plan
|
||
|
||
---
|
||
|
||
## Overview
|
||
|
||
Transform the test suite to test all 17 **MCP handlers** against a **real n8n instance** instead of mocks. This plan ensures 100% coverage of every tool, operation, and parameter combination to prevent bugs like the P0 workflow creation issue from slipping through.
|
||
|
||
### What We Test: MCP Handlers (The Product Layer)
|
||
|
||
**IMPORTANT**: These integration tests validate the **MCP handler layer** (the actual product that AI assistants interact with), not just the raw n8n API client.
|
||
|
||
**Architecture:**
|
||
```
|
||
AI Assistant (Claude)
|
||
↓
|
||
MCP Tools (What AI sees)
|
||
↓
|
||
MCP Handlers (What we test) ← INTEGRATION TESTS TARGET THIS LAYER
|
||
↓
|
||
N8nApiClient (Low-level HTTP)
|
||
↓
|
||
n8n REST API
|
||
```
|
||
|
||
**Why This Matters:**
|
||
- **MCP handlers** wrap API responses in `McpToolResponse` format: `{ success: boolean, data?: any, error?: string }`
|
||
- **MCP handlers** transform and enrich API responses (e.g., `handleGetWorkflowDetails` adds execution stats)
|
||
- **MCP handlers** provide the exact interface that AI assistants consume
|
||
- Testing raw API client bypasses the product layer and misses MCP-specific logic
|
||
|
||
**Test Pattern:**
|
||
```typescript
|
||
// ❌ WRONG: Testing raw API client (low-level service)
|
||
const result = await client.createWorkflow(workflow);
|
||
|
||
// ✅ CORRECT: Testing MCP handler (product layer)
|
||
const response = await handleCreateWorkflow({ ...workflow }, mcpContext);
|
||
expect(response.success).toBe(true);
|
||
const result = response.data;
|
||
```
|
||
|
||
## Critical Requirements
|
||
|
||
1. **Credentials**:
|
||
- Local development: Read from `.env` file
|
||
- CI/GitHub Actions: Use GitHub secrets (`N8N_URL`, `N8N_API_KEY`)
|
||
|
||
2. **Pre-activated Webhook Workflows**:
|
||
- n8n API doesn't support workflow activation via API
|
||
- Need pre-created, activated workflows for webhook testing
|
||
- Store webhook URLs (not workflow IDs) in `.env`:
|
||
- `N8N_TEST_WEBHOOK_GET_URL` - GET method webhook URL
|
||
- `N8N_TEST_WEBHOOK_POST_URL` - POST method webhook URL
|
||
- `N8N_TEST_WEBHOOK_PUT_URL` - PUT method webhook URL
|
||
- `N8N_TEST_WEBHOOK_DELETE_URL` - DELETE method webhook URL
|
||
- **Rationale**: Webhook URLs are what the `n8n_trigger_webhook_workflow` tool needs. Workflow IDs are only for workflow management tests (which create workflows dynamically during test execution).
|
||
|
||
3. **100% Coverage Goal**: Test EVERY tool, EVERY operation, EVERY parameter combination
|
||
|
||
---
|
||
|
||
## Complete Test Coverage Matrix
|
||
|
||
### Total Test Scenarios: ~150+
|
||
|
||
#### Workflow Management (10 MCP handlers)
|
||
|
||
**1. `handleCreateWorkflow`** - 15+ scenarios (MCP handler testing)
|
||
- Create workflow with base nodes (webhook, httpRequest, set)
|
||
- Create workflow with langchain nodes (agent, aiChain)
|
||
- Invalid node types (error handling)
|
||
- Complex multi-node workflows
|
||
- Complex connection patterns
|
||
- **P0 Bug Verification**: SHORT vs FULL node type handling
|
||
- Missing required parameters
|
||
- Duplicate node names
|
||
- Invalid connection references
|
||
- Settings variations
|
||
|
||
**2. `handleGetWorkflow`** - 3 scenarios
|
||
- Successful retrieval
|
||
- Not found (invalid ID)
|
||
- Malformed ID
|
||
|
||
**3. `handleGetWorkflowDetails`** - 4 scenarios
|
||
- Basic workflow
|
||
- Workflow with metadata
|
||
- Workflow with version history
|
||
- Workflow with execution stats
|
||
|
||
**4. `handleGetWorkflowStructure`** - 2 scenarios
|
||
- Simple workflow
|
||
- Complex workflow (verify no parameter data)
|
||
|
||
**5. `handleGetWorkflowMinimal`** - 2 scenarios
|
||
- Active workflow
|
||
- Inactive workflow
|
||
|
||
**6. `handleUpdateWorkflow`** - 8+ scenarios
|
||
- Full workflow replacement
|
||
- Update nodes
|
||
- Update connections
|
||
- Update settings
|
||
- Update tags
|
||
- Validation errors
|
||
- Concurrent update conflicts
|
||
- Large workflow updates
|
||
|
||
**7. `handleUpdatePartialWorkflow`** - 30+ scenarios (15 operations × 2 paths)
|
||
|
||
**Node Operations (12 scenarios):**
|
||
- `addNode`: Success, duplicate name, invalid type, missing position
|
||
- `removeNode`: By ID, by name, not found, with connection cleanup
|
||
- `updateNode`: By ID, by name, invalid updates, nested parameter updates
|
||
- `moveNode`: Valid position, boundary positions
|
||
- `enableNode`: Success, already enabled
|
||
- `disableNode`: Success, already disabled
|
||
|
||
**Connection Operations (10 scenarios):**
|
||
- `addConnection`: Default ports, custom ports, invalid nodes
|
||
- `removeConnection`: Success, not found, with ignoreErrors
|
||
- `updateConnection`: Change ports, change indices
|
||
- `cleanStaleConnections`: Dry run, actual cleanup
|
||
- `replaceConnections`: Full replacement, validation
|
||
|
||
**Metadata Operations (8 scenarios):**
|
||
- `updateSettings`: Timezone, execution order, error workflow
|
||
- `updateName`: Valid, duplicate, empty
|
||
- `addTag`: New tag, existing tag
|
||
- `removeTag`: Existing, non-existing
|
||
|
||
**8. `handleDeleteWorkflow`** - 3 scenarios
|
||
- Successful deletion
|
||
- Not found
|
||
- Verify cleanup (workflow actually deleted)
|
||
|
||
**9. `handleListWorkflows`** - 12+ scenarios
|
||
- No filters (all workflows)
|
||
- Filter by active status (true/false)
|
||
- Filter by tags (single, multiple)
|
||
- Filter by projectId (enterprise feature)
|
||
- Pagination: first page, next page, last page
|
||
- Pagination: cursor handling
|
||
- Exclude pinned data
|
||
- Limit variations (1, 50, 100)
|
||
- Empty results
|
||
- Sort order verification
|
||
|
||
**10. `handleValidateWorkflow`** - 16 scenarios (4 profiles × 4 validation types)
|
||
|
||
**Validation Profiles:**
|
||
- `strict`: All validations enabled, strictest rules
|
||
- `runtime`: Production-ready validation
|
||
- `ai-friendly`: Relaxed rules for AI-generated workflows
|
||
- `minimal`: Basic structure validation only
|
||
|
||
**Validation Types (for each profile):**
|
||
- All validations enabled (default)
|
||
- Nodes only (`validateNodes: true`, others false)
|
||
- Connections only (`validateConnections: true`, others false)
|
||
- Expressions only (`validateExpressions: true`, others false)
|
||
|
||
**11. `handleAutofixWorkflow`** - 20+ scenarios
|
||
|
||
**Fix Types (5):**
|
||
- `expression-format`: Fix `{{}}` syntax issues
|
||
- `typeversion-correction`: Fix outdated typeVersion
|
||
- `error-output-config`: Fix error output configuration
|
||
- `node-type-correction`: Fix incorrect node types
|
||
- `webhook-missing-path`: Add missing webhook paths
|
||
|
||
**Confidence Levels (3):**
|
||
- `high`: Only apply high-confidence fixes
|
||
- `medium`: Apply high + medium confidence fixes
|
||
- `low`: Apply all fixes
|
||
|
||
**Test Matrix:**
|
||
- Each fix type with preview mode (`applyFixes: false`)
|
||
- Each fix type with apply mode (`applyFixes: true`)
|
||
- Confidence threshold filtering
|
||
- `maxFixes` parameter limiting
|
||
- Multiple fix types in single workflow
|
||
- No fixes available scenario
|
||
|
||
---
|
||
|
||
#### Execution Management (4 handlers)
|
||
|
||
**12. `handleTriggerWebhookWorkflow`** - 16+ scenarios
|
||
|
||
**HTTP Methods (4):**
|
||
- GET: Query parameters, no data
|
||
- POST: JSON body, form data, headers
|
||
- PUT: Update data, custom headers
|
||
- DELETE: Query parameters, headers
|
||
|
||
**Scenarios per method:**
|
||
- Basic trigger (no data)
|
||
- With request data
|
||
- With custom headers
|
||
- Wait for response (true/false)
|
||
- Workflow not found
|
||
- Invalid webhook URL
|
||
|
||
**13. `handleGetExecution`** - 20+ scenarios
|
||
|
||
**Execution Modes (4):**
|
||
- `preview`: Structure & counts only (no data)
|
||
- `summary`: 2 samples per node (default)
|
||
- `filtered`: Custom limits and node filters
|
||
- `full`: Complete execution data
|
||
|
||
**Scenarios per mode:**
|
||
- Successful execution
|
||
- Failed execution
|
||
- Running execution
|
||
- With input data (`includeInputData: true`)
|
||
- Node filters (`nodeNames: ['Node1', 'Node2']`)
|
||
- Item limits (`itemsLimit: 0, 2, 5, -1`)
|
||
- Not found
|
||
|
||
**14. `handleListExecutions`** - 10+ scenarios
|
||
- No filters (all executions)
|
||
- Filter by workflowId
|
||
- Filter by status (success, error, waiting)
|
||
- Filter by projectId
|
||
- Pagination: first page, next page, last page
|
||
- Include execution data (`includeData: true/false`)
|
||
- Limit variations (1, 50, 100)
|
||
- Empty results
|
||
|
||
**15. `handleDeleteExecution`** - 3 scenarios
|
||
- Successful deletion
|
||
- Not found
|
||
- Verify cleanup
|
||
|
||
---
|
||
|
||
#### System/Utility (3 handlers)
|
||
|
||
**16. `handleHealthCheck`** - 2 scenarios
|
||
- API available
|
||
- Feature availability check
|
||
|
||
**17. `handleListAvailableTools`** - 1 scenario
|
||
- List all tools
|
||
|
||
**18. `handleDiagnostic`** - 3 scenarios
|
||
- Basic diagnostic
|
||
- Verbose mode (`verbose: true`)
|
||
- Configuration display
|
||
|
||
---
|
||
|
||
## Implementation Phases
|
||
|
||
### Phase 1: Foundation (Branch: `feat/integration-tests-foundation`)
|
||
|
||
#### 1.1 Environment Configuration
|
||
|
||
**Update `.env.example`:**
|
||
```bash
|
||
# ========================================
|
||
# INTEGRATION TESTING CONFIGURATION
|
||
# ========================================
|
||
|
||
# n8n API Configuration for Integration Tests
|
||
N8N_API_URL=http://localhost:5678
|
||
N8N_API_KEY=your-api-key-here
|
||
|
||
# Pre-activated Webhook URLs for Testing
|
||
# Create these workflows manually in n8n and activate them
|
||
# Store the full webhook URLs (not workflow IDs)
|
||
N8N_TEST_WEBHOOK_GET_URL=https://n8n-test.n8n-mcp.com/webhook/mcp-test-get
|
||
N8N_TEST_WEBHOOK_POST_URL=https://n8n-test.n8n-mcp.com/webhook/mcp-test-post
|
||
N8N_TEST_WEBHOOK_PUT_URL=https://n8n-test.n8n-mcp.com/webhook/mcp-test-put
|
||
N8N_TEST_WEBHOOK_DELETE_URL=https://n8n-test.n8n-mcp.com/webhook/mcp-test-delete
|
||
|
||
# Test Configuration
|
||
N8N_TEST_CLEANUP_ENABLED=true # Enable automatic cleanup
|
||
N8N_TEST_TAG=mcp-integration-test # Tag for test workflows
|
||
N8N_TEST_NAME_PREFIX=[MCP-TEST] # Name prefix for test workflows
|
||
```
|
||
|
||
**GitHub Secrets (for CI):**
|
||
- `N8N_URL`: n8n instance URL (e.g., `https://n8n-test.n8n-mcp.com`)
|
||
- `N8N_API_KEY`: n8n API key (JWT token from n8n Settings > API)
|
||
- `N8N_TEST_WEBHOOK_GET_URL`: Pre-activated GET webhook URL
|
||
- `N8N_TEST_WEBHOOK_POST_URL`: Pre-activated POST webhook URL
|
||
- `N8N_TEST_WEBHOOK_PUT_URL`: Pre-activated PUT webhook URL
|
||
- `N8N_TEST_WEBHOOK_DELETE_URL`: Pre-activated DELETE webhook URL
|
||
|
||
**Note**: Webhook URLs can be stored as repository secrets (not environment secrets) since they don't grant API access. The real secret is `N8N_API_KEY`.
|
||
|
||
#### 1.2 Directory Structure
|
||
|
||
```
|
||
tests/integration/n8n-api/
|
||
├── workflows/
|
||
│ ├── create-workflow.test.ts (10+ scenarios)
|
||
│ ├── get-workflow.test.ts (3 scenarios)
|
||
│ ├── get-workflow-details.test.ts (4 scenarios)
|
||
│ ├── get-workflow-structure.test.ts (2 scenarios)
|
||
│ ├── get-workflow-minimal.test.ts (2 scenarios)
|
||
│ ├── update-workflow.test.ts (8+ scenarios)
|
||
│ ├── update-partial-workflow.test.ts (30+ scenarios - 15 operations)
|
||
│ ├── delete-workflow.test.ts (3 scenarios)
|
||
│ ├── list-workflows.test.ts (12+ scenarios)
|
||
│ ├── validate-workflow.test.ts (16 scenarios - 4 profiles × 4 types)
|
||
│ └── autofix-workflow.test.ts (20+ scenarios - 5 types × modes)
|
||
├── executions/
|
||
│ ├── trigger-webhook.test.ts (16+ scenarios - 4 methods)
|
||
│ ├── get-execution.test.ts (20+ scenarios - 4 modes)
|
||
│ ├── list-executions.test.ts (10+ scenarios)
|
||
│ └── delete-execution.test.ts (3 scenarios)
|
||
├── system/
|
||
│ ├── health-check.test.ts (2 scenarios)
|
||
│ ├── list-tools.test.ts (1 scenario)
|
||
│ └── diagnostic.test.ts (3 scenarios)
|
||
└── utils/
|
||
├── credentials.ts # Environment-aware credential loader
|
||
├── n8n-client.ts # Pre-configured API client
|
||
├── cleanup-helpers.ts # Multi-level cleanup
|
||
├── test-context.ts # Resource tracking
|
||
├── fixtures.ts # Reusable workflow templates
|
||
├── factories.ts # Test data generators
|
||
└── webhook-workflows.ts # Webhook workflow configurations
|
||
```
|
||
|
||
#### 1.3 Core Utilities
|
||
|
||
**mcp-context.ts** - MCP context configuration for handler testing:
|
||
```typescript
|
||
import { InstanceContext } from '../../../../src/types/instance-context';
|
||
import { getN8nCredentials } from './credentials';
|
||
|
||
/**
|
||
* Creates MCP context for testing MCP handlers against real n8n instance
|
||
* This is what gets passed to MCP handlers (handleCreateWorkflow, etc.)
|
||
*/
|
||
export function createMcpContext(): InstanceContext {
|
||
const creds = getN8nCredentials();
|
||
return {
|
||
n8nApiUrl: creds.url,
|
||
n8nApiKey: creds.apiKey
|
||
};
|
||
}
|
||
```
|
||
|
||
**credentials.ts** - Environment-aware credential loader:
|
||
```typescript
|
||
import dotenv from 'dotenv';
|
||
|
||
dotenv.config();
|
||
|
||
export interface N8nTestCredentials {
|
||
url: string;
|
||
apiKey: string;
|
||
webhookUrls: {
|
||
get: string;
|
||
post: string;
|
||
put: string;
|
||
delete: string;
|
||
};
|
||
cleanup: {
|
||
enabled: boolean;
|
||
tag: string;
|
||
namePrefix: string;
|
||
};
|
||
}
|
||
|
||
export function getN8nCredentials(): N8nTestCredentials {
|
||
if (process.env.CI) {
|
||
// CI: Use GitHub secrets
|
||
const url = process.env.N8N_URL;
|
||
const apiKey = process.env.N8N_API_KEY;
|
||
|
||
if (!url || !apiKey) {
|
||
throw new Error(
|
||
'Missing required CI credentials:\n' +
|
||
` N8N_URL: ${url ? 'set' : 'MISSING'}\n` +
|
||
` N8N_API_KEY: ${apiKey ? 'set' : 'MISSING'}\n` +
|
||
'Please configure GitHub secrets for integration tests.'
|
||
);
|
||
}
|
||
|
||
return {
|
||
url,
|
||
apiKey,
|
||
webhookUrls: {
|
||
get: process.env.N8N_TEST_WEBHOOK_GET_URL || '',
|
||
post: process.env.N8N_TEST_WEBHOOK_POST_URL || '',
|
||
put: process.env.N8N_TEST_WEBHOOK_PUT_URL || '',
|
||
delete: process.env.N8N_TEST_WEBHOOK_DELETE_URL || ''
|
||
},
|
||
cleanup: {
|
||
enabled: true,
|
||
tag: 'mcp-integration-test',
|
||
namePrefix: '[MCP-TEST]'
|
||
}
|
||
};
|
||
} else {
|
||
// Local: Use .env file
|
||
const url = process.env.N8N_API_URL;
|
||
const apiKey = process.env.N8N_API_KEY;
|
||
|
||
if (!url || !apiKey) {
|
||
throw new Error(
|
||
'Missing required credentials in .env:\n' +
|
||
` N8N_API_URL: ${url ? 'set' : 'MISSING'}\n` +
|
||
` N8N_API_KEY: ${apiKey ? 'set' : 'MISSING'}\n\n` +
|
||
'Please add these to your .env file.\n' +
|
||
'See .env.example for configuration details.'
|
||
);
|
||
}
|
||
|
||
return {
|
||
url,
|
||
apiKey,
|
||
webhookUrls: {
|
||
get: process.env.N8N_TEST_WEBHOOK_GET_URL || '',
|
||
post: process.env.N8N_TEST_WEBHOOK_POST_URL || '',
|
||
put: process.env.N8N_TEST_WEBHOOK_PUT_URL || '',
|
||
delete: process.env.N8N_TEST_WEBHOOK_DELETE_URL || ''
|
||
},
|
||
cleanup: {
|
||
enabled: process.env.N8N_TEST_CLEANUP_ENABLED !== 'false',
|
||
tag: process.env.N8N_TEST_TAG || 'mcp-integration-test',
|
||
namePrefix: process.env.N8N_TEST_NAME_PREFIX || '[MCP-TEST]'
|
||
}
|
||
};
|
||
}
|
||
}
|
||
|
||
export function validateCredentials(creds: N8nTestCredentials): void {
|
||
if (!creds.url) throw new Error('N8N_API_URL is required');
|
||
if (!creds.apiKey) throw new Error('N8N_API_KEY is required');
|
||
}
|
||
|
||
export function validateWebhookUrls(creds: N8nTestCredentials): void {
|
||
const missing: string[] = [];
|
||
if (!creds.webhookUrls.get) missing.push('GET');
|
||
if (!creds.webhookUrls.post) missing.push('POST');
|
||
if (!creds.webhookUrls.put) missing.push('PUT');
|
||
if (!creds.webhookUrls.delete) missing.push('DELETE');
|
||
|
||
if (missing.length > 0) {
|
||
throw new Error(
|
||
`Missing webhook URLs for HTTP methods: ${missing.join(', ')}\n` +
|
||
`Please create and activate webhook workflows, then set:\n` +
|
||
missing.map(m => ` N8N_TEST_WEBHOOK_${m}_URL`).join('\n')
|
||
);
|
||
}
|
||
}
|
||
```
|
||
|
||
**n8n-client.ts** - Pre-configured API client (for test utilities only):
|
||
```typescript
|
||
import { N8nApiClient } from '../../../src/services/n8n-api-client';
|
||
import { getN8nCredentials } from './credentials';
|
||
|
||
/**
|
||
* IMPORTANT: This client is ONLY used for test setup/cleanup utilities.
|
||
* DO NOT use this in actual test cases - use MCP handlers instead!
|
||
*
|
||
* Test utilities that need direct API access:
|
||
* - cleanupOrphanedWorkflows() - bulk cleanup
|
||
* - Fixture setup/teardown
|
||
* - Pre-test verification
|
||
*
|
||
* Actual tests MUST use MCP handlers:
|
||
* - handleCreateWorkflow()
|
||
* - handleGetWorkflow()
|
||
* - etc.
|
||
*/
|
||
let client: N8nApiClient | null = null;
|
||
|
||
export function getTestN8nClient(): N8nApiClient {
|
||
if (!client) {
|
||
const creds = getN8nCredentials();
|
||
client = new N8nApiClient(creds.url, creds.apiKey);
|
||
}
|
||
return client;
|
||
}
|
||
|
||
export function resetTestN8nClient(): void {
|
||
client = null;
|
||
}
|
||
```
|
||
|
||
**test-context.ts** - Resource tracking for cleanup:
|
||
```typescript
|
||
import { getN8nCredentials } from './credentials';
|
||
|
||
export interface TestContext {
|
||
workflowIds: string[];
|
||
executionIds: string[];
|
||
cleanup: () => Promise<void>;
|
||
}
|
||
|
||
export function createTestContext(): TestContext {
|
||
const context: TestContext = {
|
||
workflowIds: [],
|
||
executionIds: [],
|
||
cleanup: async () => {
|
||
const creds = getN8nCredentials();
|
||
if (!creds.cleanup.enabled) return;
|
||
|
||
const client = getTestN8nClient();
|
||
|
||
// Delete executions first
|
||
for (const id of context.executionIds) {
|
||
try {
|
||
await client.deleteExecution(id);
|
||
} catch (error) {
|
||
console.warn(`Failed to delete execution ${id}:`, error);
|
||
}
|
||
}
|
||
|
||
// Then delete workflows
|
||
for (const id of context.workflowIds) {
|
||
try {
|
||
await client.deleteWorkflow(id);
|
||
} catch (error) {
|
||
console.warn(`Failed to delete workflow ${id}:`, error);
|
||
}
|
||
}
|
||
|
||
context.workflowIds = [];
|
||
context.executionIds = [];
|
||
}
|
||
};
|
||
|
||
return context;
|
||
}
|
||
```
|
||
|
||
**cleanup-helpers.ts** - Multi-level cleanup strategies:
|
||
```typescript
|
||
import { N8nApiClient } from '../../../src/services/n8n-api-client';
|
||
import { getN8nCredentials, getTestN8nClient } from './credentials';
|
||
|
||
/**
|
||
* Clean up orphaned test workflows
|
||
* Run this periodically in CI to clean up failed test runs
|
||
*/
|
||
export async function cleanupOrphanedWorkflows(): Promise<void> {
|
||
const creds = getN8nCredentials();
|
||
const client = getTestN8nClient();
|
||
|
||
let allWorkflows: any[] = [];
|
||
let cursor: string | undefined;
|
||
|
||
// Fetch all workflows with pagination
|
||
do {
|
||
const response = await client.listWorkflows({ cursor, limit: 100 });
|
||
allWorkflows.push(...response.data);
|
||
cursor = response.nextCursor;
|
||
} while (cursor);
|
||
|
||
// Find test workflows
|
||
const testWorkflows = allWorkflows.filter(w =>
|
||
w.tags?.includes(creds.cleanup.tag) ||
|
||
w.name?.startsWith(creds.cleanup.namePrefix)
|
||
);
|
||
|
||
console.log(`Found ${testWorkflows.length} orphaned test workflows`);
|
||
|
||
// Delete them
|
||
for (const workflow of testWorkflows) {
|
||
try {
|
||
await client.deleteWorkflow(workflow.id);
|
||
console.log(`Deleted orphaned workflow: ${workflow.name} (${workflow.id})`);
|
||
} catch (error) {
|
||
console.warn(`Failed to delete workflow ${workflow.id}:`, error);
|
||
}
|
||
}
|
||
}
|
||
|
||
/**
|
||
* Clean up old executions (older than 24 hours)
|
||
*/
|
||
export async function cleanupOldExecutions(): Promise<void> {
|
||
const client = getTestN8nClient();
|
||
|
||
let allExecutions: any[] = [];
|
||
let cursor: string | undefined;
|
||
|
||
// Fetch all executions
|
||
do {
|
||
const response = await client.listExecutions({ cursor, limit: 100 });
|
||
allExecutions.push(...response.data);
|
||
cursor = response.nextCursor;
|
||
} while (cursor);
|
||
|
||
const oneDayAgo = Date.now() - 24 * 60 * 60 * 1000;
|
||
const oldExecutions = allExecutions.filter(e =>
|
||
new Date(e.startedAt).getTime() < oneDayAgo
|
||
);
|
||
|
||
console.log(`Found ${oldExecutions.length} old executions`);
|
||
|
||
for (const execution of oldExecutions) {
|
||
try {
|
||
await client.deleteExecution(execution.id);
|
||
} catch (error) {
|
||
console.warn(`Failed to delete execution ${execution.id}:`, error);
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
**fixtures.ts** - Reusable workflow templates:
|
||
```typescript
|
||
import { Workflow } from '../../../src/types/n8n-api';
|
||
|
||
export const SIMPLE_WEBHOOK_WORKFLOW: Partial<Workflow> = {
|
||
name: '[MCP-TEST] Simple Webhook',
|
||
nodes: [
|
||
{
|
||
id: 'webhook-1',
|
||
name: 'Webhook',
|
||
type: 'n8n-nodes-base.webhook',
|
||
typeVersion: 2,
|
||
position: [250, 300],
|
||
parameters: {
|
||
httpMethod: 'GET',
|
||
path: 'test-webhook'
|
||
}
|
||
}
|
||
],
|
||
connections: {}
|
||
};
|
||
|
||
export const SIMPLE_HTTP_WORKFLOW: Partial<Workflow> = {
|
||
name: '[MCP-TEST] Simple HTTP Request',
|
||
nodes: [
|
||
{
|
||
id: 'webhook-1',
|
||
name: 'Webhook',
|
||
type: 'n8n-nodes-base.webhook',
|
||
typeVersion: 2,
|
||
position: [250, 300],
|
||
parameters: {
|
||
httpMethod: 'GET',
|
||
path: 'trigger'
|
||
}
|
||
},
|
||
{
|
||
id: 'http-1',
|
||
name: 'HTTP Request',
|
||
type: 'n8n-nodes-base.httpRequest',
|
||
typeVersion: 4.2,
|
||
position: [450, 300],
|
||
parameters: {
|
||
url: 'https://httpbin.org/get',
|
||
method: 'GET'
|
||
}
|
||
}
|
||
],
|
||
connections: {
|
||
Webhook: {
|
||
main: [[{ node: 'HTTP Request', type: 'main', index: 0 }]]
|
||
}
|
||
}
|
||
};
|
||
|
||
// Add more fixtures for complex workflows
|
||
```
|
||
|
||
**webhook-workflows.ts** - Webhook workflow setup guide:
|
||
```typescript
|
||
/**
|
||
* Guide for setting up webhook workflows manually in n8n
|
||
*
|
||
* These workflows must be created manually and activated because
|
||
* n8n API doesn't support workflow activation.
|
||
*
|
||
* For each HTTP method, create a workflow with:
|
||
* 1. Single Webhook node
|
||
* 2. Configured for the specific HTTP method
|
||
* 3. Unique webhook path
|
||
* 4. Activated in n8n UI
|
||
* 5. Workflow ID added to .env
|
||
*/
|
||
|
||
export const WEBHOOK_WORKFLOW_CONFIGS = {
|
||
GET: {
|
||
name: '[MCP-TEST] Webhook GET',
|
||
description: 'Pre-activated webhook for GET method testing',
|
||
nodes: [
|
||
{
|
||
name: 'Webhook',
|
||
type: 'n8n-nodes-base.webhook',
|
||
typeVersion: 2,
|
||
parameters: {
|
||
httpMethod: 'GET',
|
||
path: 'mcp-test-get',
|
||
responseMode: 'lastNode'
|
||
}
|
||
}
|
||
]
|
||
},
|
||
POST: {
|
||
name: '[MCP-TEST] Webhook POST',
|
||
description: 'Pre-activated webhook for POST method testing',
|
||
nodes: [
|
||
{
|
||
name: 'Webhook',
|
||
type: 'n8n-nodes-base.webhook',
|
||
typeVersion: 2,
|
||
parameters: {
|
||
httpMethod: 'POST',
|
||
path: 'mcp-test-post',
|
||
responseMode: 'lastNode'
|
||
}
|
||
}
|
||
]
|
||
},
|
||
PUT: {
|
||
name: '[MCP-TEST] Webhook PUT',
|
||
description: 'Pre-activated webhook for PUT method testing',
|
||
nodes: [
|
||
{
|
||
name: 'Webhook',
|
||
type: 'n8n-nodes-base.webhook',
|
||
typeVersion: 2,
|
||
parameters: {
|
||
httpMethod: 'PUT',
|
||
path: 'mcp-test-put',
|
||
responseMode: 'lastNode'
|
||
}
|
||
}
|
||
]
|
||
},
|
||
DELETE: {
|
||
name: '[MCP-TEST] Webhook DELETE',
|
||
description: 'Pre-activated webhook for DELETE method testing',
|
||
nodes: [
|
||
{
|
||
name: 'Webhook',
|
||
type: 'n8n-nodes-base.webhook',
|
||
typeVersion: 2,
|
||
parameters: {
|
||
httpMethod: 'DELETE',
|
||
path: 'mcp-test-delete',
|
||
responseMode: 'lastNode'
|
||
}
|
||
}
|
||
]
|
||
}
|
||
};
|
||
|
||
export function printSetupInstructions(): void {
|
||
console.log(`
|
||
╔════════════════════════════════════════════════════════════════╗
|
||
║ WEBHOOK WORKFLOW SETUP REQUIRED ║
|
||
╠════════════════════════════════════════════════════════════════╣
|
||
║ ║
|
||
║ Integration tests require 4 pre-activated webhook workflows: ║
|
||
║ ║
|
||
║ 1. Create workflows manually in n8n UI ║
|
||
║ 2. Use the configurations shown below ║
|
||
║ 3. ACTIVATE each workflow in n8n UI ║
|
||
║ 4. Copy workflow IDs to .env file ║
|
||
║ ║
|
||
╚════════════════════════════════════════════════════════════════╝
|
||
|
||
Required workflows:
|
||
`);
|
||
|
||
Object.entries(WEBHOOK_WORKFLOW_CONFIGS).forEach(([method, config]) => {
|
||
console.log(`
|
||
${method} Method:
|
||
Name: ${config.name}
|
||
Path: ${config.nodes[0].parameters.path}
|
||
.env variable: N8N_TEST_WEBHOOK_${method}_ID
|
||
`);
|
||
});
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### Phase 2: Workflow Creation Tests (P0)
|
||
|
||
**Branch**: `feat/integration-tests-phase-2`
|
||
|
||
**File**: `tests/integration/n8n-api/workflows/create-workflow.test.ts`
|
||
|
||
**Test Approach**: Tests the `handleCreateWorkflow` MCP handler against real n8n instance
|
||
|
||
**MCP Handler Test Pattern:**
|
||
```typescript
|
||
import { handleCreateWorkflow } from '../../../../src/mcp/handlers-n8n-manager';
|
||
import { createMcpContext } from '../utils/mcp-context';
|
||
import { InstanceContext } from '../../../../src/types/instance-context';
|
||
|
||
describe('Integration: handleCreateWorkflow', () => {
|
||
let mcpContext: InstanceContext;
|
||
|
||
beforeEach(() => {
|
||
mcpContext = createMcpContext();
|
||
});
|
||
|
||
it('should create workflow using MCP handler', async () => {
|
||
const workflow = { name: 'Test', nodes: [...], connections: {} };
|
||
|
||
// Test MCP handler (the product layer)
|
||
const response = await handleCreateWorkflow({ ...workflow }, mcpContext);
|
||
|
||
// Verify MCP response structure
|
||
expect(response.success).toBe(true);
|
||
expect(response.data).toBeDefined();
|
||
|
||
// Extract actual workflow from MCP response
|
||
const result = response.data;
|
||
expect(result.id).toBeTruthy();
|
||
});
|
||
});
|
||
```
|
||
|
||
**15 Test Scenarios** (all testing MCP handlers):
|
||
1. Create workflow with base webhook node (verify P0 bug fix)
|
||
2. Create workflow with base HTTP request node
|
||
3. Create workflow with langchain agent node
|
||
4. Create complex multi-node workflow
|
||
5. Create workflow with complex connections
|
||
6. Create workflow with custom settings
|
||
7. Create workflow with n8n expressions
|
||
8. Create workflow with error handling
|
||
9. Error: Invalid node type (documents API behavior)
|
||
10. Error: Missing required parameters (documents API behavior)
|
||
11. Error: Duplicate node names (documents API behavior)
|
||
12. Error: Invalid connection references (documents API behavior)
|
||
13. Edge case: Minimal single node workflow
|
||
14. Edge case: Empty connections object
|
||
15. Edge case: Workflow without settings
|
||
|
||
---
|
||
|
||
### Phase 3: Workflow Retrieval Tests (P1)
|
||
|
||
**Branch**: `feat/integration-tests-phase-3`
|
||
|
||
**Test Approach**: Tests MCP handlers (`handleGetWorkflow`, `handleGetWorkflowDetails`, `handleGetWorkflowStructure`, `handleGetWorkflowMinimal`)
|
||
|
||
**MCP Handler Pattern:**
|
||
```typescript
|
||
import {
|
||
handleGetWorkflow,
|
||
handleGetWorkflowDetails,
|
||
handleGetWorkflowStructure,
|
||
handleGetWorkflowMinimal
|
||
} from '../../../../src/mcp/handlers-n8n-manager';
|
||
|
||
// Test MCP handler
|
||
const response = await handleGetWorkflow({ id: workflowId }, mcpContext);
|
||
expect(response.success).toBe(true);
|
||
const workflow = response.data;
|
||
|
||
// Note: handleGetWorkflowDetails returns nested structure
|
||
const detailsResponse = await handleGetWorkflowDetails({ id }, mcpContext);
|
||
const workflow = detailsResponse.data.workflow; // Extract from nested structure
|
||
const stats = detailsResponse.data.executionStats;
|
||
```
|
||
|
||
**Files**:
|
||
- `get-workflow.test.ts` (3 scenarios - tests handleGetWorkflow)
|
||
- `get-workflow-details.test.ts` (4 scenarios - tests handleGetWorkflowDetails)
|
||
- `get-workflow-structure.test.ts` (2 scenarios - tests handleGetWorkflowStructure)
|
||
- `get-workflow-minimal.test.ts` (2 scenarios - tests handleGetWorkflowMinimal)
|
||
|
||
---
|
||
|
||
### Phase 4: Workflow Update Tests (P1)
|
||
|
||
**Branch**: `feat/integration-tests-workflow-updates`
|
||
|
||
**Files**:
|
||
- `update-workflow.test.ts` (8+ scenarios)
|
||
- `update-partial-workflow.test.ts` (30+ scenarios covering all 15 operations)
|
||
|
||
---
|
||
|
||
### Phase 5: Workflow Management Tests (P2)
|
||
|
||
**Branch**: `feat/integration-tests-workflow-management`
|
||
|
||
**Files**:
|
||
- `delete-workflow.test.ts` (3 scenarios)
|
||
- `list-workflows.test.ts` (12+ scenarios with all filters and pagination)
|
||
|
||
---
|
||
|
||
### Phase 6A: Workflow Validation Tests (P2) ✅ COMPLETE
|
||
|
||
**Branch**: `feat/integration-tests-phase-6`
|
||
|
||
**Files**:
|
||
- ✅ `tests/integration/n8n-api/utils/node-repository.ts` - NodeRepository singleton for validation tests
|
||
- ✅ `validate-workflow.test.ts` (12 scenarios: 4 profiles + invalid detection + selective validation + error handling)
|
||
|
||
**Implementation Notes**:
|
||
- Created NodeRepository utility since handleValidateWorkflow requires repository parameter
|
||
- Tests cover all 4 validation profiles (strict, runtime, ai-friendly, minimal)
|
||
- Invalid workflow detection tests (bad node types, missing connections)
|
||
- Selective validation tests (nodes only, connections only, expressions only)
|
||
- Response structure correctly handles conditional errors/warnings fields
|
||
|
||
### Phase 6B: Workflow Autofix Tests (P2)
|
||
|
||
**Branch**: `feat/integration-tests-phase-6b` (or continue on `feat/integration-tests-phase-6`)
|
||
|
||
**Files**:
|
||
- `autofix-workflow.test.ts` (15-20 scenarios: 5 fix types × modes × confidence levels)
|
||
|
||
**Test Coverage Required**:
|
||
- 5 fix types: expression-format, typeversion-correction, error-output-config, node-type-correction, webhook-missing-path
|
||
- Preview mode (applyFixes: false) vs Apply mode (applyFixes: true)
|
||
- Confidence threshold filtering (high, medium, low)
|
||
- maxFixes parameter limiting
|
||
- Multiple fix types in single workflow
|
||
- No fixes available scenario
|
||
|
||
---
|
||
|
||
### Phase 7: Execution Management Tests (P2)
|
||
|
||
**Branch**: `feat/integration-tests-executions`
|
||
|
||
**Files**:
|
||
- `trigger-webhook.test.ts` (16+ scenarios: 4 HTTP methods × variations)
|
||
- `get-execution.test.ts` (20+ scenarios: 4 modes × filters)
|
||
- `list-executions.test.ts` (10+ scenarios)
|
||
- `delete-execution.test.ts` (3 scenarios)
|
||
|
||
**Special Considerations for Webhook Testing**:
|
||
- Use pre-activated workflows from `.env`
|
||
- Each HTTP method uses a different workflow ID
|
||
- Test both successful triggers and error cases
|
||
- Verify response data for synchronous executions
|
||
|
||
---
|
||
|
||
### Phase 8: System Tools Tests (P3)
|
||
|
||
**Branch**: `feat/integration-tests-system`
|
||
|
||
**Files**:
|
||
- `health-check.test.ts` (2 scenarios)
|
||
- `list-tools.test.ts` (1 scenario)
|
||
- `diagnostic.test.ts` (3 scenarios)
|
||
|
||
---
|
||
|
||
### Phase 9: CI/CD Integration
|
||
|
||
**Branch**: `feat/integration-tests-ci`
|
||
|
||
**GitHub Actions Workflow** (`.github/workflows/integration-tests.yml`):
|
||
|
||
```yaml
|
||
name: Integration Tests
|
||
|
||
on:
|
||
pull_request:
|
||
push:
|
||
branches: [main]
|
||
schedule:
|
||
- cron: '0 2 * * *' # Daily at 2 AM
|
||
workflow_dispatch:
|
||
|
||
jobs:
|
||
integration-tests:
|
||
runs-on: ubuntu-latest
|
||
|
||
steps:
|
||
- uses: actions/checkout@v4
|
||
|
||
- name: Setup Node.js
|
||
uses: actions/setup-node@v4
|
||
with:
|
||
node-version: '20'
|
||
cache: 'npm'
|
||
|
||
- name: Install dependencies
|
||
run: npm ci
|
||
|
||
- name: Build project
|
||
run: npm run build
|
||
|
||
- name: Run integration tests
|
||
env:
|
||
N8N_URL: ${{ secrets.N8N_URL }}
|
||
N8N_API_KEY: ${{ secrets.N8N_API_KEY }}
|
||
N8N_TEST_WEBHOOK_GET_URL: ${{ secrets.N8N_TEST_WEBHOOK_GET_URL }}
|
||
N8N_TEST_WEBHOOK_POST_URL: ${{ secrets.N8N_TEST_WEBHOOK_POST_URL }}
|
||
N8N_TEST_WEBHOOK_PUT_URL: ${{ secrets.N8N_TEST_WEBHOOK_PUT_URL }}
|
||
N8N_TEST_WEBHOOK_DELETE_URL: ${{ secrets.N8N_TEST_WEBHOOK_DELETE_URL }}
|
||
CI: true
|
||
run: npm run test:integration:n8n
|
||
|
||
- name: Cleanup orphaned workflows
|
||
if: always()
|
||
env:
|
||
N8N_URL: ${{ secrets.N8N_URL }}
|
||
N8N_API_KEY: ${{ secrets.N8N_API_KEY }}
|
||
run: npm run test:cleanup:orphans
|
||
```
|
||
|
||
**Add npm scripts to `package.json`**:
|
||
```json
|
||
{
|
||
"scripts": {
|
||
"test:integration:n8n": "vitest run tests/integration/n8n-api",
|
||
"test:cleanup:orphans": "tsx tests/integration/n8n-api/utils/cleanup-orphans.ts"
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Test Isolation Strategy
|
||
|
||
### Workflow Naming Convention
|
||
- Prefix: `[MCP-TEST]`
|
||
- Include test name: `[MCP-TEST] Create Workflow - Base Nodes`
|
||
- Include timestamp for uniqueness: `[MCP-TEST] Test Name ${Date.now()}`
|
||
|
||
### Workflow Tagging
|
||
- All test workflows tagged with: `mcp-integration-test`
|
||
- Enables bulk cleanup queries
|
||
|
||
### Cleanup Levels
|
||
1. **Test-level**: After each test via `afterEach` hook
|
||
2. **Suite-level**: After each test file via `afterAll` hook
|
||
3. **CI-level**: After CI job completes (always run)
|
||
4. **Orphan cleanup**: Periodic job to clean up failed test runs
|
||
|
||
---
|
||
|
||
## Pre-Test Setup Checklist
|
||
|
||
### Local Development
|
||
1. ✅ Install n8n locally or use Docker
|
||
2. ✅ Start n8n instance: `npx n8n start`
|
||
3. ✅ Create 4 webhook workflows (GET, POST, PUT, DELETE)
|
||
4. ✅ Activate all 4 webhook workflows in n8n UI
|
||
5. ✅ Get webhook URLs from the workflow's Webhook node
|
||
6. ✅ Copy `.env.example` to `.env`
|
||
7. ✅ Set `N8N_API_URL=<your-n8n-url>`
|
||
8. ✅ Generate API key in n8n Settings > API
|
||
9. ✅ Set `N8N_API_KEY=<your-key>`
|
||
10. ✅ Set all 4 `N8N_TEST_WEBHOOK_*_URL` variables with full webhook URLs
|
||
|
||
### CI/GitHub Actions (✅ COMPLETED)
|
||
1. ✅ Set up cloud n8n instance: `https://n8n-test.n8n-mcp.com`
|
||
2. ✅ Create 4 webhook workflows (GET, POST, PUT, DELETE)
|
||
3. ✅ Activate all 4 webhook workflows
|
||
4. ✅ Add GitHub secrets: `N8N_URL`, `N8N_API_KEY`
|
||
5. ✅ Add webhook URL secrets:
|
||
- `N8N_TEST_WEBHOOK_GET_URL=https://n8n-test.n8n-mcp.com/webhook/mcp-test-get`
|
||
- `N8N_TEST_WEBHOOK_POST_URL=https://n8n-test.n8n-mcp.com/webhook/mcp-test-post`
|
||
- `N8N_TEST_WEBHOOK_PUT_URL=https://n8n-test.n8n-mcp.com/webhook/mcp-test-put`
|
||
- `N8N_TEST_WEBHOOK_DELETE_URL=https://n8n-test.n8n-mcp.com/webhook/mcp-test-delete`
|
||
|
||
---
|
||
|
||
## Success Criteria
|
||
|
||
### Phase 1: Foundation ✅ COMPLETE
|
||
- ✅ Environment configuration (.env, GitHub secrets)
|
||
- ✅ All utility files created (8 files, ~1,520 lines of code)
|
||
- ✅ Pre-activated webhook workflows created and tested
|
||
- ✅ Cleanup helpers with pagination safety
|
||
- ✅ Resource tracking with TestContext
|
||
- ✅ Fixtures and factories for test data
|
||
- ✅ Documentation updated
|
||
- ✅ Environment loading fixed (loads .env before test defaults)
|
||
- ✅ Vitest integration config updated (removed MSW for n8n-api tests)
|
||
|
||
### Phase 2: Workflow Creation Tests ✅ COMPLETE
|
||
- ✅ 15 test scenarios implemented (all passing)
|
||
- ✅ Tests the `handleCreateWorkflow` MCP handler (product layer)
|
||
- ✅ All tests use MCP handler pattern with McpToolResponse validation
|
||
- ✅ P0 bug verification (FULL vs SHORT node type format)
|
||
- ✅ Base node tests (webhook, HTTP, langchain, multi-node)
|
||
- ✅ Advanced features (connections, settings, expressions, error handling)
|
||
- ✅ Error scenarios (4 tests documenting actual API behavior)
|
||
- ✅ Edge cases (3 tests for minimal/empty configurations)
|
||
- ✅ Test file: 563 lines covering all handleCreateWorkflow scenarios
|
||
- ✅ All tests passing against real n8n instance
|
||
|
||
### Overall Project (In Progress)
|
||
- ⏳ All 17 handlers have integration tests (11 of 17 complete)
|
||
- ⏳ All operations/parameters covered (99 of 150+ scenarios complete)
|
||
- ✅ Tests run successfully locally (Phases 1-6 verified)
|
||
- ⏳ Tests run successfully in CI (pending Phase 9)
|
||
- ✅ No manual cleanup required (automatic)
|
||
- ✅ Test coverage catches P0-level bugs (verified in Phase 2)
|
||
- ⏳ CI runs on every PR and daily (pending Phase 9)
|
||
- ✅ Clear error messages when tests fail
|
||
- ✅ Documentation for webhook workflow setup
|
||
- ✅ Code quality maintained (lint, typecheck, type safety)
|
||
|
||
---
|
||
|
||
## Timeline Estimate
|
||
|
||
- **Phase 1 (Foundation)**: ✅ COMPLETE (October 3, 2025)
|
||
- **Phase 2 (Workflow Creation)**: ✅ COMPLETE (October 3, 2025)
|
||
- **Phase 3 (Retrieval)**: ✅ COMPLETE (October 3, 2025)
|
||
- **Phase 4 (Updates)**: ✅ COMPLETE (October 4, 2025)
|
||
- **Phase 5 (Management)**: ✅ COMPLETE (October 4, 2025)
|
||
- **Phase 6A (Validation)**: ✅ COMPLETE (October 5, 2025)
|
||
- **Phase 6B (Autofix)**: ✅ COMPLETE (October 5, 2025)
|
||
- **Phase 7 (Executions)**: 2 days
|
||
- **Phase 8 (System)**: 1 day
|
||
- **Phase 9 (CI/CD)**: 1 day
|
||
|
||
**Total**: 6 days complete, ~4 days remaining
|
||
|
||
---
|
||
|
||
## Notes
|
||
|
||
- Each phase should be developed on a separate branch
|
||
- Phases can be parallelized where dependencies allow
|
||
- Run local tests frequently to catch issues early
|
||
- Document any n8n API quirks discovered during testing
|
||
|
||
## Key Learnings from Implementation
|
||
|
||
### Critical Testing Principle: Test the Product Layer
|
||
|
||
**The Mistake**: Initially, Phase 2 tests called `client.createWorkflow()` (raw API client) instead of `handleCreateWorkflow()` (MCP handler).
|
||
|
||
**Why This Was Wrong**:
|
||
- AI assistants interact with MCP handlers, not raw API client
|
||
- MCP handlers wrap responses in `McpToolResponse` format
|
||
- MCP handlers may transform/enrich API responses
|
||
- Bypassing MCP layer misses product-specific logic and bugs
|
||
|
||
**The Fix**: All tests updated to use MCP handlers:
|
||
```typescript
|
||
// ❌ BEFORE: Testing wrong layer
|
||
const result = await client.createWorkflow(workflow);
|
||
|
||
// ✅ AFTER: Testing the actual product
|
||
const response = await handleCreateWorkflow({ ...workflow }, mcpContext);
|
||
expect(response.success).toBe(true);
|
||
const result = response.data;
|
||
```
|
||
|
||
**Lesson Learned**: Always test the layer closest to the user/consumer. For n8n-mcp, that's the MCP handler layer.
|
||
|
||
### n8n API Behavior Discoveries
|
||
1. **Validation Timing**: n8n API accepts workflows with invalid node types and connection references at creation time. Validation only happens at execution time.
|
||
2. **Node Type Format**: FULL node type format (`n8n-nodes-base.*`) must be used in API requests. The P0 bug was confirmed fixed.
|
||
3. **Missing Parameters**: n8n accepts workflows with missing required parameters. They fail during execution, not creation.
|
||
4. **Duplicate Names**: n8n API handles duplicate node names gracefully (may auto-rename).
|
||
|
||
### Technical Implementation Insights
|
||
1. **MSW Interference**: Integration tests that need real network requests must NOT load MSW setup. Removed from vitest.config.integration.ts.
|
||
2. **Environment Loading**: Must load `.env` file BEFORE test defaults in global setup to preserve real credentials.
|
||
3. **Cleanup Safety**: TestContext pattern works well for tracking and cleaning up test resources.
|
||
4. **Test Isolation**: Each test creates unique workflows with timestamps to avoid conflicts.
|