Complete implementation of Phase 1 foundation for n8n API integration tests. Establishes core utilities, fixtures, and infrastructure for testing all 17 n8n API handlers against real n8n instance. Changes: - Add integration test environment configuration to .env.example - Create comprehensive test utilities infrastructure: * credentials.ts: Environment-aware credential management (local .env vs CI secrets) * n8n-client.ts: Singleton API client wrapper with health checks * test-context.ts: Resource tracking and automatic cleanup * cleanup-helpers.ts: Multi-level cleanup strategies (orphaned, age-based, tag-based) * fixtures.ts: 6 pre-built workflow templates (webhook, HTTP, multi-node, error handling, AI, expressions) * factories.ts: Dynamic node/workflow builders with 15+ factory functions * webhook-workflows.ts: Webhook workflow configs and setup instructions - Add npm scripts: * test:integration:n8n: Run n8n API integration tests * test:cleanup:orphans: Clean up orphaned test resources - Create cleanup script for CI/manual use Documentation: - Add comprehensive integration testing plan (550 lines) - Add Phase 1 completion summary with lessons learned Key Features: - Automatic credential detection (CI vs local) - Multi-level cleanup (test, suite, CI, orphan) - 6 workflow fixtures covering common scenarios - 15+ factory functions for dynamic test data - Support for 4 HTTP methods (GET, POST, PUT, DELETE) via pre-activated webhook workflows - TypeScript-first with full type safety - Comprehensive error handling with helpful messages Total: ~1,520 lines of production-ready code + 650 lines of documentation Ready for Phase 2: Workflow creation tests 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
43 KiB
P0 Priorities Implementation Plan
Critical Fixes for n8n-mcp Based on Production Telemetry Data
Date: October 2, 2025 Analysis Period: September 26 - October 2, 2025 Data Volume: 212,375 events | 5,751 workflows | 2,119 users Target: Reduce error rate from 5-10% to <2%
Executive Summary
This document provides a comprehensive implementation plan for the three P0 (Priority 0 - Critical) issues identified through deep analysis of production telemetry data. These fixes will eliminate 80% of all validation errors and significantly improve the AI agent experience.
Impact Summary
| Issue | Current Failure Rate | Post-Fix Target | Affected Users | Estimated Effort |
|---|---|---|---|---|
| P0-R1: Node Type Prefix Normalization | 80% of validation errors | <1% | Hundreds | 2 days |
| P0-R2: Null-Safety Audit | 10-18% TypeError rate | <1% | 30+ | 2 days |
| P0-R3: Pre-extract Template Configs + Remove get_node_for_task | 28% failure rate, 5.9% coverage | N/A (tool removed), 100% coverage | 197 (migrated) | 5 days |
Total Effort: 2 weeks (v2.15.0 release)
Table of Contents
- P0-R1: Auto-Normalize Node Type Prefixes
- P0-R2: Complete Null-Safety Audit
- P0-R3: Pre-extract Template Configurations + Remove get_node_for_task
- Implementation Order & Timeline
- Testing Strategy
- Rollback Plan
- Success Metrics
P0-R1: Auto-Normalize Node Type Prefixes
Problem Statement
Impact: 4,800+ validation errors (80% of all validation errors) from a single root cause
AI agents frequently produce nodes-base.X instead of n8n-nodes-base.X, causing validation failures. This is the single largest source of user frustration.
Example Error:
Error: Invalid node type: "nodes-base.set". Use "n8n-nodes-base.set" instead.
Root Cause Analysis
Current Implementation Issues:
-
Existing normalization is BACKWARD:
src/utils/node-type-utils.tsnormalizes TO short form (nodes-base.)- But validation expects full form (
n8n-nodes-base.) - This is the opposite of what we need
-
Location of the bug:
// src/utils/node-type-utils.ts:18-20 return type .replace(/^n8n-nodes-base\./, 'nodes-base.') // ❌ WRONG DIRECTION .replace(/^@n8n\/n8n-nodes-langchain\./, 'nodes-langchain.'); -
Why AI agents produce short form:
- Token efficiency (LLMs abbreviate to save tokens)
- Pattern learning from examples
- Natural language preference for concise names
Solution Architecture
Strategy: Normalize ALL node types to FULL form before validation
1. Create Universal Node Type Normalizer
File: src/utils/node-type-normalizer.ts (NEW)
/**
* Universal Node Type Normalizer
*
* Converts ANY node type variation to the canonical full form expected by n8n
*
* Handles:
* - Short form → Full form (nodes-base.X → n8n-nodes-base.X)
* - Already full form → Unchanged
* - LangChain nodes → Proper @n8n/ prefix
*/
export interface NodeTypeNormalizationResult {
original: string;
normalized: string;
wasNormalized: boolean;
package: 'base' | 'langchain' | 'community' | 'unknown';
}
export class NodeTypeNormalizer {
/**
* Normalize node type to canonical full form
*
* @example
* normalizeToFullForm('nodes-base.webhook')
* // → 'n8n-nodes-base.webhook'
*
* normalizeToFullForm('n8n-nodes-base.webhook')
* // → 'n8n-nodes-base.webhook' (unchanged)
*
* normalizeToFullForm('nodes-langchain.agent')
* // → '@n8n/n8n-nodes-langchain.agent'
*/
static normalizeToFullForm(type: string): string {
if (!type || typeof type !== 'string') {
return type;
}
// Already in full form - return unchanged
if (type.startsWith('n8n-nodes-base.')) {
return type;
}
if (type.startsWith('@n8n/n8n-nodes-langchain.')) {
return type;
}
// Normalize short forms to full form
if (type.startsWith('nodes-base.')) {
return type.replace(/^nodes-base\./, 'n8n-nodes-base.');
}
if (type.startsWith('nodes-langchain.')) {
return type.replace(/^nodes-langchain\./, '@n8n/n8n-nodes-langchain.');
}
if (type.startsWith('n8n-nodes-langchain.')) {
return type.replace(/^n8n-nodes-langchain\./, '@n8n/n8n-nodes-langchain.');
}
// No prefix - might be community node or error
return type;
}
/**
* Normalize with detailed result
*/
static normalizeWithDetails(type: string): NodeTypeNormalizationResult {
const original = type;
const normalized = this.normalizeToFullForm(type);
return {
original,
normalized,
wasNormalized: original !== normalized,
package: this.detectPackage(normalized)
};
}
/**
* Detect package type from node type
*/
private static detectPackage(type: string): 'base' | 'langchain' | 'community' | 'unknown' {
if (type.startsWith('n8n-nodes-base.')) return 'base';
if (type.startsWith('@n8n/n8n-nodes-langchain.')) return 'langchain';
if (type.includes('.')) return 'community';
return 'unknown';
}
/**
* Batch normalize multiple node types
*/
static normalizeBatch(types: string[]): Map<string, string> {
const result = new Map<string, string>();
for (const type of types) {
result.set(type, this.normalizeToFullForm(type));
}
return result;
}
/**
* Normalize all node types in a workflow
*/
static normalizeWorkflowNodeTypes(workflow: any): any {
if (!workflow?.nodes || !Array.isArray(workflow.nodes)) {
return workflow;
}
return {
...workflow,
nodes: workflow.nodes.map((node: any) => ({
...node,
type: this.normalizeToFullForm(node.type)
}))
};
}
}
2. Apply Normalization in All Entry Points
File: src/services/workflow-validator.ts
Change at line 250: (validateWorkflowStructure method)
// BEFORE (line 250-252):
const normalizedType = normalizeNodeType(singleNode.type);
const isWebhook = normalizedType === 'nodes-base.webhook' ||
normalizedType === 'nodes-base.webhookTrigger';
// AFTER:
import { NodeTypeNormalizer } from '../utils/node-type-normalizer';
const normalizedType = NodeTypeNormalizer.normalizeToFullForm(singleNode.type);
const isWebhook = normalizedType === 'n8n-nodes-base.webhook' ||
normalizedType === 'n8n-nodes-base.webhookTrigger';
Change at line 368-376: (validateAllNodes method)
// BEFORE:
// Get node definition - try multiple formats
let nodeInfo = this.nodeRepository.getNode(node.type);
// If not found, try with normalized type
if (!nodeInfo) {
const normalizedType = normalizeNodeType(node.type);
if (normalizedType !== node.type) {
nodeInfo = this.nodeRepository.getNode(normalizedType);
}
}
// AFTER:
// Normalize node type FIRST
const normalizedType = NodeTypeNormalizer.normalizeToFullForm(node.type);
const nodeInfo = this.nodeRepository.getNode(normalizedType);
// Update node type in place if normalized
if (normalizedType !== node.type) {
node.type = normalizedType;
}
File: src/mcp/handlers-n8n-manager.ts
Add normalization in handleCreateWorkflow (line 281-310):
// BEFORE validation:
const input = createWorkflowSchema.parse(args);
// AFTER: Add normalization
const input = createWorkflowSchema.parse(args);
// Normalize all node types before validation
const normalizedInput = NodeTypeNormalizer.normalizeWorkflowNodeTypes(input);
// Validate workflow structure
const errors = validateWorkflowStructure(normalizedInput);
Apply same pattern to:
handleUpdateWorkflow(line 520)validateWorkflowtool handler- Any other workflow creation/update entry points
3. Update Node Repository for Flexible Lookups
File: src/database/node-repository.ts
Enhance getNode method (line 54):
/**
* Get node with automatic type normalization
*/
getNode(nodeType: string): any {
// Try normalized type first
const normalizedType = NodeTypeNormalizer.normalizeToFullForm(nodeType);
const row = this.db.prepare(`
SELECT * FROM nodes WHERE node_type = ?
`).get(normalizedType) as any;
if (!row) {
// Fallback: try original type if normalization didn't help
if (normalizedType !== nodeType) {
const originalRow = this.db.prepare(`
SELECT * FROM nodes WHERE node_type = ?
`).get(nodeType) as any;
if (originalRow) return this.parseNodeRow(originalRow);
}
return null;
}
return this.parseNodeRow(row);
}
Testing Requirements
File: tests/unit/utils/node-type-normalizer.test.ts (NEW)
describe('NodeTypeNormalizer', () => {
describe('normalizeToFullForm', () => {
it('should normalize short base form to full form', () => {
expect(NodeTypeNormalizer.normalizeToFullForm('nodes-base.webhook'))
.toBe('n8n-nodes-base.webhook');
});
it('should normalize short langchain form to full form', () => {
expect(NodeTypeNormalizer.normalizeToFullForm('nodes-langchain.agent'))
.toBe('@n8n/n8n-nodes-langchain.agent');
});
it('should leave full forms unchanged', () => {
expect(NodeTypeNormalizer.normalizeToFullForm('n8n-nodes-base.webhook'))
.toBe('n8n-nodes-base.webhook');
});
it('should handle edge cases', () => {
expect(NodeTypeNormalizer.normalizeToFullForm('')).toBe('');
expect(NodeTypeNormalizer.normalizeToFullForm(null as any)).toBe(null);
});
});
describe('normalizeWorkflowNodeTypes', () => {
it('should normalize all nodes in workflow', () => {
const workflow = {
nodes: [
{ type: 'nodes-base.webhook', id: '1', name: 'Webhook' },
{ type: 'nodes-base.set', id: '2', name: 'Set' }
],
connections: {}
};
const result = NodeTypeNormalizer.normalizeWorkflowNodeTypes(workflow);
expect(result.nodes[0].type).toBe('n8n-nodes-base.webhook');
expect(result.nodes[1].type).toBe('n8n-nodes-base.set');
});
});
});
Success Criteria
- All workflow validation tests pass with both short and full node type forms
- 0 "Invalid node type" errors for variations of core nodes
- Telemetry shows <1% validation errors related to node type prefixes
- No breaking changes to existing workflows
Status: ✅ COMPLETED (October 2, 2025)
Commit: ed7de10
Estimated Effort
Total: 2-4 hours
- Implementation: 1-2 hours
- Testing: 1 hour
- Documentation: 30 minutes
- Code review: 30 minutes
P0-R2: Complete Null-Safety Audit
Problem Statement
Impact: 10-18% TypeError failures in node information tools affecting 1,000+ calls
TypeError: Cannot read property 'text' of undefined
Affected Tools:
get_node_essentials: 483 failures (10% of 4,909 calls)get_node_info: 352 failures (18% of 1,988 calls)get_node_documentation: 136 failures (7% of 1,919 calls)
Root Cause Analysis
From CHANGELOG 2.14.0:
"Fixed TypeErrors in get_node_info, get_node_essentials, and get_node_documentation tools" "Added null safety checks for undefined node properties"
The fix was incomplete. Residual issues remain in:
- Nested property access without guards
- Edge cases with unusual/legacy node structures
- Missing properties in database
- Assumptions about property structure
Current Implementation Analysis
File: src/database/node-repository.ts
Problem areas identified:
// Line 73-78: Good - has safeJsonParse
properties: this.safeJsonParse(row.properties_schema, []),
operations: this.safeJsonParse(row.operations, []),
credentials: this.safeJsonParse(row.credentials_required, []),
// But doesn't protect against:
// - properties being null after parse
// - Nested properties like properties[0].description.text
// - Missing fields in properties array
handlers for get_node_essentials/info need to be found and audited
Solution Architecture
1. Enhanced Safe Property Access Utilities
File: src/utils/safe-property-access.ts (NEW)
/**
* Safe Property Access Utilities
*
* Provides defensive property access with fallbacks
*/
export class SafePropertyAccess {
/**
* Safely get nested property with default
*/
static get<T>(obj: any, path: string, defaultValue: T): T {
if (!obj || typeof obj !== 'object') return defaultValue;
const keys = path.split('.');
let current = obj;
for (const key of keys) {
if (current === null || current === undefined) {
return defaultValue;
}
if (typeof current !== 'object') {
return defaultValue;
}
current = current[key];
}
return current !== undefined ? current : defaultValue;
}
/**
* Safely get array with default
*/
static getArray<T>(obj: any, path: string, defaultValue: T[] = []): T[] {
const value = this.get(obj, path, defaultValue);
return Array.isArray(value) ? value : defaultValue;
}
/**
* Safely get string with default
*/
static getString(obj: any, path: string, defaultValue: string = ''): string {
const value = this.get(obj, path, defaultValue);
return typeof value === 'string' ? value : defaultValue;
}
/**
* Safely get number with default
*/
static getNumber(obj: any, path: string, defaultValue: number = 0): number {
const value = this.get(obj, path, defaultValue);
return typeof value === 'number' && !isNaN(value) ? value : defaultValue;
}
/**
* Safely get boolean with default
*/
static getBoolean(obj: any, path: string, defaultValue: boolean = false): boolean {
const value = this.get(obj, path, defaultValue);
return typeof value === 'boolean' ? value : defaultValue;
}
/**
* Extract description from multiple possible locations
*/
static extractDescription(obj: any): string {
// Try common description locations
const locations = [
'description',
'properties.description',
'properties.description.text',
'subtitle',
'displayName'
];
for (const location of locations) {
const value = this.getString(obj, location);
if (value) return value;
}
return 'No description available';
}
/**
* Extract display name from multiple possible locations
*/
static extractDisplayName(obj: any, fallback: string = 'Unknown'): string {
const locations = [
'displayName',
'name',
'label',
'title'
];
for (const location of locations) {
const value = this.getString(obj, location);
if (value) return value;
}
return fallback;
}
}
2. Null-Safe Node Repository Methods
File: src/database/node-repository.ts
Refactor getNode method (line 54):
import { SafePropertyAccess } from '../utils/safe-property-access';
/**
* Get node with comprehensive null-safety
*/
getNode(nodeType: string): any | null {
try {
// Normalize type first
const normalizedType = NodeTypeNormalizer.normalizeToFullForm(nodeType);
const row = this.db.prepare(`
SELECT * FROM nodes WHERE node_type = ?
`).get(normalizedType) as any;
if (!row) return null;
// Use safe property access for all fields
return {
nodeType: SafePropertyAccess.getString(row, 'node_type', normalizedType),
displayName: SafePropertyAccess.extractDisplayName(row,
SafePropertyAccess.getString(row, 'display_name', 'Unknown Node')),
description: SafePropertyAccess.extractDescription(row),
category: SafePropertyAccess.getString(row, 'category', 'Uncategorized'),
developmentStyle: SafePropertyAccess.getString(row, 'development_style', 'declarative'),
package: SafePropertyAccess.getString(row, 'package_name', 'unknown'),
isAITool: SafePropertyAccess.getBoolean(row, 'is_ai_tool', false),
isTrigger: SafePropertyAccess.getBoolean(row, 'is_trigger', false),
isWebhook: SafePropertyAccess.getBoolean(row, 'is_webhook', false),
isVersioned: SafePropertyAccess.getBoolean(row, 'is_versioned', false),
version: SafePropertyAccess.getNumber(row, 'version', 1),
properties: this.safeParseProperties(row.properties_schema),
operations: this.safeParseArray(row.operations),
credentials: this.safeParseArray(row.credentials_required),
hasDocumentation: !!row.documentation,
outputs: row.outputs ? this.safeJsonParse(row.outputs, null) : null,
outputNames: row.output_names ? this.safeJsonParse(row.output_names, null) : null
};
} catch (error) {
console.error(`Error getting node ${nodeType}:`, error);
return null;
}
}
/**
* Safely parse properties with validation
*/
private safeParseProperties(json: string): any[] {
try {
const parsed = JSON.parse(json);
if (!Array.isArray(parsed)) return [];
// Validate each property has minimum required fields
return parsed.map(prop => ({
name: SafePropertyAccess.getString(prop, 'name', 'unknown'),
displayName: SafePropertyAccess.extractDisplayName(prop),
type: SafePropertyAccess.getString(prop, 'type', 'string'),
required: SafePropertyAccess.getBoolean(prop, 'required', false),
default: prop.default !== undefined ? prop.default : null,
description: SafePropertyAccess.extractDescription(prop),
options: SafePropertyAccess.getArray(prop, 'options', []),
displayOptions: prop.displayOptions || null
}));
} catch {
return [];
}
}
/**
* Safely parse array field
*/
private safeParseArray(json: string): any[] {
try {
const parsed = JSON.parse(json);
return Array.isArray(parsed) ? parsed : [];
} catch {
return [];
}
}
3. Find and Fix Handler Functions
Action Required: Search for handler functions that call getNode and add null checks
Pattern to search for:
grep -r "getNode\|getNodeEssentials\|getNodeInfo" src/mcp/ --include="*.ts"
Add null checks like:
const node = repository.getNode(nodeType);
if (!node) {
return {
success: false,
error: `Node type "${nodeType}" not found. Use search_nodes to find available nodes.`
};
}
Testing Requirements
File: tests/unit/database/node-repository-null-safety.test.ts (NEW)
describe('NodeRepository - Null Safety', () => {
it('should handle node with missing description', () => {
// Insert node with minimal data
const node = { type: 'test.node', name: 'Test' };
db.prepare('INSERT INTO nodes (node_type, display_name) VALUES (?, ?)').run(node.type, node.name);
const result = repository.getNode('test.node');
expect(result).not.toBeNull();
expect(result.description).toBe('No description available');
expect(result.properties).toEqual([]);
});
it('should handle node with malformed JSON', () => {
db.prepare('INSERT INTO nodes (node_type, properties_schema) VALUES (?, ?)').run('test.node', 'invalid json');
const result = repository.getNode('test.node');
expect(result).not.toBeNull();
expect(result.properties).toEqual([]);
});
it('should handle non-existent node gracefully', () => {
const result = repository.getNode('non.existent');
expect(result).toBeNull();
});
it('should handle null database row', () => {
// Simulate database returning null
const result = repository.getNode('null.node');
expect(result).toBeNull();
});
});
Success Criteria
- get_node_essentials failure rate: 10% → <1%
- get_node_info failure rate: 18% → <1%
- get_node_documentation failure rate: 7% → <1%
- 100% test coverage for null cases
- No TypeErrors in production logs
Estimated Effort
Total: 1 day (8 hours)
- Safe property access utility: 2 hours
- Repository refactoring: 3 hours
- Handler updates: 2 hours
- Testing: 1 hour
P0-R3: Pre-extract Template Configurations + Remove get_node_for_task
Problem Statement
Impact: 28% failure rate (worst-performing tool) + redundant with better alternatives
get_node_for_task failing 109 times out of 392 calls (27.8%)
Current State:
- Only 31 predefined tasks in
task-templates.ts(5.9% node coverage) - 22.5:1 usage ratio favoring
search_nodes(8,839 calls vs 392) - Hardcoded configurations require manual maintenance
- Tool provides no unique value over
search_nodes
Discovery: We have 2,646 real production workflow templates from n8n.io with:
- 3,820 httpRequest configurations
- 1,700 googleSheets configurations
- 466 webhook configurations
- 100% AI-generated metadata coverage
- Real-world best practices and patterns
Architectural Decision: Pre-extraction
Analysis: On-the-fly vs Pre-extraction (see /docs/local/TEMPLATE_MINING_ANALYSIS.md)
Decision: Pre-extract node configurations into separate table
Rationale:
- Performance: 1ms vs 30-60ms (30-60x faster)
- Storage: Only 513 KB for 2,625 configs (negligible)
- Simplicity: No cache management, TTL, or eviction logic
- Features: Enables filtering by complexity, auth (indexed queries)
- Scalability: Handles 10,000+ templates without degradation
- Predictability: Consistent sub-millisecond response times
Trade-offs (acceptable):
- +30-60 seconds rebuild time (rare operation)
- Incremental updates needed when templates change
Solution Architecture
Strategy:
- Pre-extract top 10 node configurations per node type into new table
- Enhance
get_node_essentialswith optional examples - Enhance
search_nodeswith optional examples - Remove
get_node_for_taskentirely (no redirect)
See /docs/local/TEMPLATE_MINING_ANALYSIS.md for complete analysis
1. Add Database Schema for Pre-extracted Configurations
File: src/database/schema.sql
Add new table after templates table:
-- Pre-extracted node configurations from templates
CREATE TABLE template_node_configs (
id INTEGER PRIMARY KEY,
node_type TEXT NOT NULL,
template_id INTEGER NOT NULL,
template_name TEXT NOT NULL,
template_views INTEGER DEFAULT 0,
-- Node configuration (extracted from workflow)
node_name TEXT, -- Node name in workflow (e.g., "HTTP Request")
parameters_json TEXT NOT NULL, -- JSON: node.parameters
credentials_json TEXT, -- JSON: node.credentials (if present)
-- Pre-calculated metadata for filtering
has_credentials INTEGER DEFAULT 0,
has_expressions INTEGER DEFAULT 0, -- Contains {{...}} or $json/$node
complexity TEXT CHECK(complexity IN ('simple', 'medium', 'complex')),
use_cases TEXT, -- JSON array from template.metadata.use_cases
-- Pre-calculated ranking (1 = best, 2 = second best, etc.)
rank INTEGER DEFAULT 0,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
FOREIGN KEY (template_id) REFERENCES templates(id) ON DELETE CASCADE
);
-- Indexes for fast queries
CREATE INDEX idx_config_node_type_rank
ON template_node_configs(node_type, rank);
CREATE INDEX idx_config_complexity
ON template_node_configs(node_type, complexity, rank);
CREATE INDEX idx_config_auth
ON template_node_configs(node_type, has_credentials, rank);
-- View for easy querying of top configs
CREATE VIEW ranked_node_configs AS
SELECT
node_type,
template_name,
template_views,
parameters_json,
credentials_json,
has_credentials,
has_expressions,
complexity,
use_cases,
rank
FROM template_node_configs
WHERE rank <= 5 -- Top 5 per node type
ORDER BY node_type, rank;
Migration Script: src/database/migrations/add-template-node-configs.sql
-- Migration for existing databases
-- Run during `npm run rebuild` or `npm run fetch:templates`
-- Check if table exists
CREATE TABLE IF NOT EXISTS template_node_configs (
-- ... schema as above
);
-- Populate from existing templates
-- (handled by extraction logic in fetch:templates script)
2. Add Extraction Logic to fetch:templates Script
File: src/scripts/fetch-templates.ts
Add extraction function:
import gzip from 'zlib';
/**
* Extract node configurations from a template workflow
*/
function extractNodeConfigs(
templateId: number,
templateName: string,
templateViews: number,
workflowCompressed: string,
metadata: any
): Array<{
node_type: string;
template_id: number;
template_name: string;
template_views: number;
node_name: string;
parameters_json: string;
credentials_json: string | null;
has_credentials: number;
has_expressions: number;
complexity: string;
use_cases: string;
}> {
try {
// Decompress workflow
const decompressed = gzip.gunzipSync(Buffer.from(workflowCompressed, 'base64'));
const workflow = JSON.parse(decompressed.toString('utf-8'));
const configs: any[] = [];
for (const node of workflow.nodes || []) {
// Skip UI-only nodes
if (node.type.includes('stickyNote') || !node.parameters) {
continue;
}
configs.push({
node_type: node.type,
template_id: templateId,
template_name: templateName,
template_views: templateViews,
node_name: node.name,
parameters_json: JSON.stringify(node.parameters),
credentials_json: node.credentials ? JSON.stringify(node.credentials) : null,
has_credentials: node.credentials ? 1 : 0,
has_expressions: detectExpressions(node.parameters) ? 1 : 0,
complexity: metadata?.complexity || 'medium',
use_cases: JSON.stringify(metadata?.use_cases || [])
});
}
return configs;
} catch (error) {
console.error(`Error extracting configs from template ${templateId}:`, error);
return [];
}
}
/**
* Detect n8n expressions in parameters
*/
function detectExpressions(params: any): boolean {
const json = JSON.stringify(params);
return json.includes('={{') || json.includes('$json') || json.includes('$node');
}
/**
* Insert extracted configs into database and rank them
*/
function insertAndRankConfigs(db: Database, configs: any[]) {
// Clear old configs for these templates
const templateIds = [...new Set(configs.map(c => c.template_id))];
db.prepare(`DELETE FROM template_node_configs WHERE template_id IN (${templateIds.join(',')})`).run();
// Insert new configs
const insertStmt = db.prepare(`
INSERT INTO template_node_configs (
node_type, template_id, template_name, template_views,
node_name, parameters_json, credentials_json,
has_credentials, has_expressions, complexity, use_cases
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
for (const config of configs) {
insertStmt.run(
config.node_type,
config.template_id,
config.template_name,
config.template_views,
config.node_name,
config.parameters_json,
config.credentials_json,
config.has_credentials,
config.has_expressions,
config.complexity,
config.use_cases
);
}
// Rank configs per node_type by template popularity
db.exec(`
UPDATE template_node_configs
SET rank = (
SELECT COUNT(*) + 1
FROM template_node_configs AS t2
WHERE t2.node_type = template_node_configs.node_type
AND t2.template_views > template_node_configs.template_views
)
`);
// Keep only top 10 per node_type
db.exec(`
DELETE FROM template_node_configs
WHERE id NOT IN (
SELECT id FROM template_node_configs
WHERE rank <= 10
)
`);
console.log(`Extracted and ranked ${configs.length} node configurations`);
}
3. Enhance get_node_essentials with Examples
File: src/mcp/handlers-*.ts or src/mcp/server.ts
Update get_node_essentials handler:
async function getNodeEssentials(
nodeType: string,
options?: { includeExamples?: boolean }
): Promise<any> {
const node = repository.getNode(nodeType);
if (!node) {
return {
success: false,
error: `Node type "${nodeType}" not found. Use search_nodes to find available nodes.`
};
}
const result = {
nodeType,
displayName: node.displayName,
description: node.description,
category: node.category,
// ... existing essentials fields ...
};
// NEW: Add real-world examples if requested
if (options?.includeExamples) {
const examples = db.prepare(`
SELECT
parameters_json,
template_name,
template_views,
complexity,
use_cases,
has_credentials,
has_expressions
FROM template_node_configs
WHERE node_type = ?
ORDER BY rank
LIMIT 3
`).all(nodeType);
result.examples = examples.map(ex => ({
config: JSON.parse(ex.parameters_json),
source: `${ex.template_name} (${(ex.template_views / 1000).toFixed(0)}k views)`,
complexity: ex.complexity,
useCases: JSON.parse(ex.use_cases).slice(0, 2),
hasAuth: ex.has_credentials === 1,
hasExpressions: ex.has_expressions === 1
}));
}
return result;
}
Tool definition update:
{
name: 'get_node_essentials',
description: 'Get essential information about a specific n8n node type...',
inputSchema: {
type: 'object',
properties: {
nodeType: {
type: 'string',
description: 'Full node type (e.g., "n8n-nodes-base.httpRequest")'
},
includeExamples: { // NEW
type: 'boolean',
description: 'Include 2-3 real configuration examples from popular templates',
default: false
}
},
required: ['nodeType']
}
}
4. Enhance search_nodes with Examples
File: src/mcp/handlers-*.ts or src/mcp/server.ts
Update search_nodes handler:
async function searchNodes(
query: string,
options?: {
limit?: number;
includeExamples?: boolean;
}
): Promise<any> {
const nodes = repository.searchNodes(query, 'OR', options?.limit || 20);
const results = nodes.map(node => {
const result = {
nodeType: node.nodeType,
displayName: node.displayName,
description: node.description,
category: node.category
};
// NEW: Add examples if requested
if (options?.includeExamples) {
const examples = db.prepare(`
SELECT parameters_json, template_name, complexity
FROM template_node_configs
WHERE node_type = ?
ORDER BY rank
LIMIT 2
`).all(node.nodeType);
result.examples = examples.map(ex => ({
config: JSON.parse(ex.parameters_json),
source: ex.template_name,
complexity: ex.complexity
}));
}
return result;
});
return results;
}
Tool definition update:
{
name: 'search_nodes',
description: 'Search for n8n nodes by keyword...',
inputSchema: {
type: 'object',
properties: {
query: { type: 'string', description: 'Search query' },
limit: { type: 'number', default: 20 },
includeExamples: { // NEW
type: 'boolean',
description: 'Include 2 real configuration examples per node',
default: false
}
},
required: ['query']
}
}
5. Remove get_node_for_task Tool Entirely
Files to modify:
src/mcp/server.ts- Remove handler functionsrc/mcp/tools.ts- Remove tool definitionsrc/mcp/tools-documentation.ts- Remove from documentationsrc/services/task-templates.ts- Can be deprecated (keep for now, remove in v2.16.0)README.md- Remove from available tools listCHANGELOG.md- Document removal
Steps:
# Search for all references
grep -r "get_node_for_task" src/
grep -r "getNodeForTask" src/
# Remove handler
# Remove tool definition
# Remove from documentation
# Update README
Migration note for users (add to CHANGELOG):
### BREAKING CHANGES in v2.15.0
- **Removed:** `get_node_for_task` tool
- **Replacement:** Use `search_nodes` with `includeExamples: true`
- **Migration:** `get_node_for_task({task: "webhook"})` → `search_nodes({query: "webhook", includeExamples: true})`
- **Benefit:** Access to 2,646 real templates vs 31 hardcoded tasks
Testing Requirements
File: tests/unit/services/template-config-extraction.test.ts (NEW)
describe('Template Config Extraction', () => {
it('should extract node configs from workflow', () => {
const workflow = {
nodes: [
{
type: 'n8n-nodes-base.httpRequest',
name: 'HTTP Request',
parameters: { url: 'https://api.example.com', method: 'GET' }
}
]
};
const configs = extractNodeConfigs(1, 'Test', 1000, compressWorkflow(workflow), {});
expect(configs).toHaveLength(1);
expect(configs[0].node_type).toBe('n8n-nodes-base.httpRequest');
});
it('should detect expressions in parameters', () => {
const params = { url: '={{$json.api_url}}' };
expect(detectExpressions(params)).toBe(true);
});
it('should rank configs by popularity', () => {
// Insert configs with different views
// Verify ranking order
});
});
File: tests/integration/enhanced-tools.test.ts (NEW)
describe('Enhanced Tools with Examples', () => {
it('get_node_essentials should return examples when requested', async () => {
const result = await getNodeEssentials('n8n-nodes-base.httpRequest', {
includeExamples: true
});
expect(result.examples).toBeDefined();
expect(result.examples.length).toBeGreaterThan(0);
expect(result.examples[0].config).toHaveProperty('url');
});
it('search_nodes should return examples when requested', async () => {
const result = await searchNodes('webhook', { includeExamples: true });
expect(result.length).toBeGreaterThan(0);
expect(result[0].examples).toBeDefined();
});
it('get_node_for_task should not exist', async () => {
expect(toolRegistry.has('get_node_for_task')).toBe(false);
});
});
Success Criteria
- Extract 2,000+ node configurations from templates
- Query performance: <1ms for pre-extracted configs
get_node_essentialswith examples: <5ms totalsearch_nodeswith examples: <10ms total- Database size increase: <1 MB
get_node_for_taskcompletely removed from codebase- All documentation updated
Estimated Effort
Total: 1 week (5 days)
-
Day 1: Database schema + migration (8 hours)
- Design schema
- Create migration script
- Test with existing database
-
Day 2: Extraction logic in fetch:templates (8 hours)
- Write extraction function
- Write ranking logic
- Test with 2,646 templates
-
Day 3: Enhance get_node_essentials + search_nodes (8 hours)
- Add includeExamples parameter
- Update tool definitions
- Integration testing
-
Day 4: Remove get_node_for_task + documentation (8 hours)
- Remove from all files
- Update README, CHANGELOG
- Update tools_documentation
- Migration guide
-
Day 5: Testing + optimization (8 hours)
- Unit tests
- Integration tests
- Performance testing
- Bug fixes
Implementation Order & Timeline
Version 2.15.0 - All P0 Fixes in One Release
Total Timeline: 2 weeks (10 working days)
Week 1: Foundation + P0-R1 + P0-R2
Monday (Day 1-2): P0-R1 - Node Type Normalization
- AM: Create NodeTypeNormalizer utility
- PM: Apply to workflow validator, handlers, and repository
- Testing and validation
- Deliverable: 80% of validation errors eliminated
Tuesday (Day 3): P0-R2 - Null-Safety Audit (Part 1)
- AM: Create SafePropertyAccess utility
- PM: Refactor node repository methods
- Deliverable: Safe property access framework
Wednesday (Day 4): P0-R2 - Null-Safety Audit (Part 2)
- AM: Find and fix all handlers
- PM: Comprehensive null-safety testing
- Deliverable: 10-18% TypeError rate → <1%
Thursday (Day 5): P0-R3 - Database Schema
- AM: Design and implement template_node_configs table
- PM: Create migration script and test with existing database
- Deliverable: Schema ready for extraction
Friday (Day 6): P0-R3 - Extraction Logic
- AM: Write extraction function in fetch:templates
- PM: Write ranking logic and test with 2,646 templates
- Deliverable: 2,000+ configs extracted and ranked
Week 2: P0-R3 Integration + Testing + Documentation
Monday (Day 7): Tool Enhancements
- AM: Enhance get_node_essentials with includeExamples
- PM: Enhance search_nodes with includeExamples
- Deliverable: Both tools return real examples
Tuesday (Day 8): Tool Removal + Documentation
- AM: Remove get_node_for_task from all files
- PM: Update README, CHANGELOG, tools_documentation
- Deliverable: Clean removal, migration guide complete
Wednesday (Day 9): Comprehensive Testing
- AM: Unit tests for extraction and enhanced tools
- PM: Integration tests for all P0 fixes
- Deliverable: 95%+ test coverage
Thursday (Day 10): Performance + Final Testing
- AM: Performance testing and optimization
- PM: E2E testing and bug fixes
- Deliverable: All success criteria met
Friday (Day 11): Release Preparation
- AM: Code review and documentation review
- PM: Prepare release notes, tag v2.15.0
- Deliverable: Ready for release
Parallel Activities
- Documentation updates: Days 1-11
- Code reviews: End of Days 2, 4, 6, 8, 10
- Telemetry preparation: Day 10-11 (prepare monitoring dashboard)
Testing Strategy
Unit Tests
Coverage Target: 95% for new code
- Node Type Normalizer: 20+ test cases
- Safe Property Access: 30+ test cases
- Task Discovery Service: 40+ test cases
Integration Tests
- Workflow validation with mixed node type forms
- Node repository with edge case data
- Task discovery with real node database
E2E Tests
- Create workflow with short-form node types → Should succeed
- Get node info for nodes with missing properties → Should return safe defaults
- Query task discovery with variations → Should find matches
Regression Tests
- All existing tests must pass
- No breaking changes to public APIs
Performance Tests
- Normalization overhead: <1ms per workflow
- Safe property access: <0.1ms per node
- Task discovery: <50ms average
Rollback Plan
If P0-R1 Causes Issues
- Symptom: Workflows fail validation after normalization
- Action: Revert node-type-normalizer changes
- Fallback: Use original normalizeNodeType
- Recovery time: 15 minutes
If P0-R2 Causes Performance Issues
- Symptom: Node lookup becomes slow
- Action: Cache safe property access results
- Fallback: Keep safe parsing but reduce validation
- Recovery time: 1 hour
If P0-R3 Template Extraction Causes Issues
- Symptom: Database bloat or slow queries
- Action: Reduce rank limit from 10 to 5 per node
- Fallback: Disable includeExamples parameter temporarily
- Recovery time: 15 minutes (just disable parameter)
If get_node_for_task Removal Causes User Issues
- Symptom: Users report missing tool
- Action: Add prominent migration guide to error messages
- Fallback: N/A (breaking change, users must migrate)
- Communication: Update docs, add migration examples
Success Metrics
Overall Goals
| Metric | Current | Target | How to Measure |
|---|---|---|---|
| Overall error rate | 5-10% | <2% | Telemetry events |
| Validation errors | 4,800/week | <100/week | Error logs |
| TypeError rate | 10-18% | <1% | Tool execution logs |
| Node configs extracted | 0 | 2,000+ | Database count |
| Config query performance | N/A | <1ms | Performance tests |
| get_node_for_task usage | 392 calls | 0 (removed) | Tool usage stats |
| search_nodes w/ examples | 0 | Monitored | New feature adoption |
Telemetry Monitoring
After deployment, monitor for 1 week:
- Error rate by tool (should decrease 80-90%)
- User success rate (should increase 5-10%)
- Average errors per user (should decrease from 2.5 to <0.5)
Dependencies
NPM Packages
No new NPM packages required - all functionality uses existing dependencies.
Internal Dependencies
- P0-R3 requires database schema update (template_node_configs table)
- P0-R3 requires migration script for existing databases
- All changes are backward compatible except removal of
get_node_for_task
Documentation Updates
Files to Update
- CHANGELOG.md - Add entries for each P0 fix + breaking changes
- README.md - Remove get_node_for_task, add includeExamples parameter
- src/mcp/tools-documentation.ts - Remove get_node_for_task documentation
- API.md - Document enhanced tool parameters
- MIGRATION.md - Add migration guide from get_node_for_task to search_nodes (NEW)
Example CHANGELOG Entry
## [2.15.0] - 2025-10-09
### BREAKING CHANGES
- **Removed:** `get_node_for_task` tool
- **Replacement:** Use `search_nodes` with `includeExamples: true`
- **Migration:** `get_node_for_task({task: "webhook"})` → `search_nodes({query: "webhook", includeExamples: true})`
- **Benefit:** Access to 2,646 real templates vs 31 hardcoded tasks
### Fixed
- **P0-R1:** Auto-normalize node type prefixes (eliminates 80% of validation errors)
- **P0-R2:** Complete null-safety audit for node information tools (reduces TypeError failures from 10-18% to <1%)
### Added
- `NodeTypeNormalizer` utility for universal node type normalization
- `SafePropertyAccess` utility for defensive property access
- `template_node_configs` table with 2,000+ pre-extracted configurations
- `includeExamples` parameter for `get_node_essentials` (returns 2-3 real configs)
- `includeExamples` parameter for `search_nodes` (returns 2 real configs per node)
- Real-world configuration examples from popular n8n templates
### Performance
- Node configuration queries: <1ms (30-60x faster than on-the-fly extraction)
- Sub-millisecond response time for configuration examples
Conclusion
These P0 fixes represent the highest-impact improvements we can make to n8n-mcp based on real production telemetry data. By implementing all three fixes in v2.15.0, we will:
- Eliminate 80% of validation errors (P0-R1: Node type normalization)
- Fix the majority of TypeError failures (P0-R2: Null-safety audit)
- Replace inferior tool with superior alternative (P0-R3: Template-based configs + remove get_node_for_task)
Expected Overall Impact:
- Error rate: 5-10% → <2%
- Configuration examples: 31 hardcoded → 2,000+ real templates
- Query performance: 30-60ms → <1ms (30-60x faster)
- User experience: Significant improvement across all tools
- Support burden: Reduced by 50%+
Key Innovation (P0-R3):
- Pre-extraction delivers 30-60x performance improvement
- 2,646 real templates provide richer context than hardcoded tasks
- Breaking change justified by superior replacement
- Database increase: Only +513 KB for 2,625 configurations
The implementation is well-architected, delivers exceptional value, and sets up future enhancements.
Next Steps:
- ✅ Review implementation plan with team (COMPLETED)
- ✅ Finalize architectural decisions (COMPLETED - pre-extraction chosen)
- ✅ Create feature branch:
feature/p0-priorities-fixes(COMPLETED) - ✅ P0-R1: Auto-Normalize Node Type Prefixes (COMPLETED - commit
ed7de10) - ⏳ P0-R2: Complete Null-Safety Audit (PENDING)
- ⏳ P0-R3: Pre-extract Template Configs + Remove get_node_for_task (PENDING)
- ⏳ Deploy v2.15.0 with monitoring and telemetry analysis
Target Release: v2.15.0 (estimated 1.5 weeks remaining)