Files
n8n-mcp/docs/local/P0_IMPLEMENTATION_PLAN.md
czlonkowski 2305aaab9e feat: implement integration testing foundation (Phase 1)
Complete implementation of Phase 1 foundation for n8n API integration tests.
Establishes core utilities, fixtures, and infrastructure for testing all 17 n8n API handlers against real n8n instance.

Changes:
- Add integration test environment configuration to .env.example
- Create comprehensive test utilities infrastructure:
  * credentials.ts: Environment-aware credential management (local .env vs CI secrets)
  * n8n-client.ts: Singleton API client wrapper with health checks
  * test-context.ts: Resource tracking and automatic cleanup
  * cleanup-helpers.ts: Multi-level cleanup strategies (orphaned, age-based, tag-based)
  * fixtures.ts: 6 pre-built workflow templates (webhook, HTTP, multi-node, error handling, AI, expressions)
  * factories.ts: Dynamic node/workflow builders with 15+ factory functions
  * webhook-workflows.ts: Webhook workflow configs and setup instructions

- Add npm scripts:
  * test:integration:n8n: Run n8n API integration tests
  * test:cleanup:orphans: Clean up orphaned test resources

- Create cleanup script for CI/manual use

Documentation:
- Add comprehensive integration testing plan (550 lines)
- Add Phase 1 completion summary with lessons learned

Key Features:
- Automatic credential detection (CI vs local)
- Multi-level cleanup (test, suite, CI, orphan)
- 6 workflow fixtures covering common scenarios
- 15+ factory functions for dynamic test data
- Support for 4 HTTP methods (GET, POST, PUT, DELETE) via pre-activated webhook workflows
- TypeScript-first with full type safety
- Comprehensive error handling with helpful messages

Total: ~1,520 lines of production-ready code + 650 lines of documentation

Ready for Phase 2: Workflow creation tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-03 13:12:42 +02:00

43 KiB

P0 Priorities Implementation Plan

Critical Fixes for n8n-mcp Based on Production Telemetry Data

Date: October 2, 2025 Analysis Period: September 26 - October 2, 2025 Data Volume: 212,375 events | 5,751 workflows | 2,119 users Target: Reduce error rate from 5-10% to <2%


Executive Summary

This document provides a comprehensive implementation plan for the three P0 (Priority 0 - Critical) issues identified through deep analysis of production telemetry data. These fixes will eliminate 80% of all validation errors and significantly improve the AI agent experience.

Impact Summary

Issue Current Failure Rate Post-Fix Target Affected Users Estimated Effort
P0-R1: Node Type Prefix Normalization 80% of validation errors <1% Hundreds 2 days
P0-R2: Null-Safety Audit 10-18% TypeError rate <1% 30+ 2 days
P0-R3: Pre-extract Template Configs + Remove get_node_for_task 28% failure rate, 5.9% coverage N/A (tool removed), 100% coverage 197 (migrated) 5 days

Total Effort: 2 weeks (v2.15.0 release)


Table of Contents

  1. P0-R1: Auto-Normalize Node Type Prefixes
  2. P0-R2: Complete Null-Safety Audit
  3. P0-R3: Pre-extract Template Configurations + Remove get_node_for_task
  4. Implementation Order & Timeline
  5. Testing Strategy
  6. Rollback Plan
  7. Success Metrics

P0-R1: Auto-Normalize Node Type Prefixes

Problem Statement

Impact: 4,800+ validation errors (80% of all validation errors) from a single root cause

AI agents frequently produce nodes-base.X instead of n8n-nodes-base.X, causing validation failures. This is the single largest source of user frustration.

Example Error:

Error: Invalid node type: "nodes-base.set". Use "n8n-nodes-base.set" instead.

Root Cause Analysis

Current Implementation Issues:

  1. Existing normalization is BACKWARD:

    • src/utils/node-type-utils.ts normalizes TO short form (nodes-base.)
    • But validation expects full form (n8n-nodes-base.)
    • This is the opposite of what we need
  2. Location of the bug:

    // src/utils/node-type-utils.ts:18-20
    return type
      .replace(/^n8n-nodes-base\./, 'nodes-base.')  // ❌ WRONG DIRECTION
      .replace(/^@n8n\/n8n-nodes-langchain\./, 'nodes-langchain.');
    
  3. Why AI agents produce short form:

    • Token efficiency (LLMs abbreviate to save tokens)
    • Pattern learning from examples
    • Natural language preference for concise names

Solution Architecture

Strategy: Normalize ALL node types to FULL form before validation

1. Create Universal Node Type Normalizer

File: src/utils/node-type-normalizer.ts (NEW)

/**
 * Universal Node Type Normalizer
 *
 * Converts ANY node type variation to the canonical full form expected by n8n
 *
 * Handles:
 * - Short form → Full form (nodes-base.X → n8n-nodes-base.X)
 * - Already full form → Unchanged
 * - LangChain nodes → Proper @n8n/ prefix
 */

export interface NodeTypeNormalizationResult {
  original: string;
  normalized: string;
  wasNormalized: boolean;
  package: 'base' | 'langchain' | 'community' | 'unknown';
}

export class NodeTypeNormalizer {

  /**
   * Normalize node type to canonical full form
   *
   * @example
   * normalizeToFullForm('nodes-base.webhook')
   * // → 'n8n-nodes-base.webhook'
   *
   * normalizeToFullForm('n8n-nodes-base.webhook')
   * // → 'n8n-nodes-base.webhook' (unchanged)
   *
   * normalizeToFullForm('nodes-langchain.agent')
   * // → '@n8n/n8n-nodes-langchain.agent'
   */
  static normalizeToFullForm(type: string): string {
    if (!type || typeof type !== 'string') {
      return type;
    }

    // Already in full form - return unchanged
    if (type.startsWith('n8n-nodes-base.')) {
      return type;
    }
    if (type.startsWith('@n8n/n8n-nodes-langchain.')) {
      return type;
    }

    // Normalize short forms to full form
    if (type.startsWith('nodes-base.')) {
      return type.replace(/^nodes-base\./, 'n8n-nodes-base.');
    }
    if (type.startsWith('nodes-langchain.')) {
      return type.replace(/^nodes-langchain\./, '@n8n/n8n-nodes-langchain.');
    }
    if (type.startsWith('n8n-nodes-langchain.')) {
      return type.replace(/^n8n-nodes-langchain\./, '@n8n/n8n-nodes-langchain.');
    }

    // No prefix - might be community node or error
    return type;
  }

  /**
   * Normalize with detailed result
   */
  static normalizeWithDetails(type: string): NodeTypeNormalizationResult {
    const original = type;
    const normalized = this.normalizeToFullForm(type);

    return {
      original,
      normalized,
      wasNormalized: original !== normalized,
      package: this.detectPackage(normalized)
    };
  }

  /**
   * Detect package type from node type
   */
  private static detectPackage(type: string): 'base' | 'langchain' | 'community' | 'unknown' {
    if (type.startsWith('n8n-nodes-base.')) return 'base';
    if (type.startsWith('@n8n/n8n-nodes-langchain.')) return 'langchain';
    if (type.includes('.')) return 'community';
    return 'unknown';
  }

  /**
   * Batch normalize multiple node types
   */
  static normalizeBatch(types: string[]): Map<string, string> {
    const result = new Map<string, string>();
    for (const type of types) {
      result.set(type, this.normalizeToFullForm(type));
    }
    return result;
  }

  /**
   * Normalize all node types in a workflow
   */
  static normalizeWorkflowNodeTypes(workflow: any): any {
    if (!workflow?.nodes || !Array.isArray(workflow.nodes)) {
      return workflow;
    }

    return {
      ...workflow,
      nodes: workflow.nodes.map((node: any) => ({
        ...node,
        type: this.normalizeToFullForm(node.type)
      }))
    };
  }
}

2. Apply Normalization in All Entry Points

File: src/services/workflow-validator.ts

Change at line 250: (validateWorkflowStructure method)

// BEFORE (line 250-252):
const normalizedType = normalizeNodeType(singleNode.type);
const isWebhook = normalizedType === 'nodes-base.webhook' ||
                 normalizedType === 'nodes-base.webhookTrigger';

// AFTER:
import { NodeTypeNormalizer } from '../utils/node-type-normalizer';

const normalizedType = NodeTypeNormalizer.normalizeToFullForm(singleNode.type);
const isWebhook = normalizedType === 'n8n-nodes-base.webhook' ||
                 normalizedType === 'n8n-nodes-base.webhookTrigger';

Change at line 368-376: (validateAllNodes method)

// BEFORE:
// Get node definition - try multiple formats
let nodeInfo = this.nodeRepository.getNode(node.type);

// If not found, try with normalized type
if (!nodeInfo) {
  const normalizedType = normalizeNodeType(node.type);
  if (normalizedType !== node.type) {
    nodeInfo = this.nodeRepository.getNode(normalizedType);
  }
}

// AFTER:
// Normalize node type FIRST
const normalizedType = NodeTypeNormalizer.normalizeToFullForm(node.type);
const nodeInfo = this.nodeRepository.getNode(normalizedType);

// Update node type in place if normalized
if (normalizedType !== node.type) {
  node.type = normalizedType;
}

File: src/mcp/handlers-n8n-manager.ts

Add normalization in handleCreateWorkflow (line 281-310):

// BEFORE validation:
const input = createWorkflowSchema.parse(args);

// AFTER: Add normalization
const input = createWorkflowSchema.parse(args);

// Normalize all node types before validation
const normalizedInput = NodeTypeNormalizer.normalizeWorkflowNodeTypes(input);

// Validate workflow structure
const errors = validateWorkflowStructure(normalizedInput);

Apply same pattern to:

  • handleUpdateWorkflow (line 520)
  • validateWorkflow tool handler
  • Any other workflow creation/update entry points

3. Update Node Repository for Flexible Lookups

File: src/database/node-repository.ts

Enhance getNode method (line 54):

/**
 * Get node with automatic type normalization
 */
getNode(nodeType: string): any {
  // Try normalized type first
  const normalizedType = NodeTypeNormalizer.normalizeToFullForm(nodeType);

  const row = this.db.prepare(`
    SELECT * FROM nodes WHERE node_type = ?
  `).get(normalizedType) as any;

  if (!row) {
    // Fallback: try original type if normalization didn't help
    if (normalizedType !== nodeType) {
      const originalRow = this.db.prepare(`
        SELECT * FROM nodes WHERE node_type = ?
      `).get(nodeType) as any;

      if (originalRow) return this.parseNodeRow(originalRow);
    }
    return null;
  }

  return this.parseNodeRow(row);
}

Testing Requirements

File: tests/unit/utils/node-type-normalizer.test.ts (NEW)

describe('NodeTypeNormalizer', () => {
  describe('normalizeToFullForm', () => {
    it('should normalize short base form to full form', () => {
      expect(NodeTypeNormalizer.normalizeToFullForm('nodes-base.webhook'))
        .toBe('n8n-nodes-base.webhook');
    });

    it('should normalize short langchain form to full form', () => {
      expect(NodeTypeNormalizer.normalizeToFullForm('nodes-langchain.agent'))
        .toBe('@n8n/n8n-nodes-langchain.agent');
    });

    it('should leave full forms unchanged', () => {
      expect(NodeTypeNormalizer.normalizeToFullForm('n8n-nodes-base.webhook'))
        .toBe('n8n-nodes-base.webhook');
    });

    it('should handle edge cases', () => {
      expect(NodeTypeNormalizer.normalizeToFullForm('')).toBe('');
      expect(NodeTypeNormalizer.normalizeToFullForm(null as any)).toBe(null);
    });
  });

  describe('normalizeWorkflowNodeTypes', () => {
    it('should normalize all nodes in workflow', () => {
      const workflow = {
        nodes: [
          { type: 'nodes-base.webhook', id: '1', name: 'Webhook' },
          { type: 'nodes-base.set', id: '2', name: 'Set' }
        ],
        connections: {}
      };

      const result = NodeTypeNormalizer.normalizeWorkflowNodeTypes(workflow);

      expect(result.nodes[0].type).toBe('n8n-nodes-base.webhook');
      expect(result.nodes[1].type).toBe('n8n-nodes-base.set');
    });
  });
});

Success Criteria

  • All workflow validation tests pass with both short and full node type forms
  • 0 "Invalid node type" errors for variations of core nodes
  • Telemetry shows <1% validation errors related to node type prefixes
  • No breaking changes to existing workflows

Status: COMPLETED (October 2, 2025) Commit: ed7de10

Estimated Effort

Total: 2-4 hours

  • Implementation: 1-2 hours
  • Testing: 1 hour
  • Documentation: 30 minutes
  • Code review: 30 minutes

P0-R2: Complete Null-Safety Audit

Problem Statement

Impact: 10-18% TypeError failures in node information tools affecting 1,000+ calls

TypeError: Cannot read property 'text' of undefined

Affected Tools:

  • get_node_essentials: 483 failures (10% of 4,909 calls)
  • get_node_info: 352 failures (18% of 1,988 calls)
  • get_node_documentation: 136 failures (7% of 1,919 calls)

Root Cause Analysis

From CHANGELOG 2.14.0:

"Fixed TypeErrors in get_node_info, get_node_essentials, and get_node_documentation tools" "Added null safety checks for undefined node properties"

The fix was incomplete. Residual issues remain in:

  1. Nested property access without guards
  2. Edge cases with unusual/legacy node structures
  3. Missing properties in database
  4. Assumptions about property structure

Current Implementation Analysis

File: src/database/node-repository.ts

Problem areas identified:

// Line 73-78: Good - has safeJsonParse
properties: this.safeJsonParse(row.properties_schema, []),
operations: this.safeJsonParse(row.operations, []),
credentials: this.safeJsonParse(row.credentials_required, []),

// But doesn't protect against:
// - properties being null after parse
// - Nested properties like properties[0].description.text
// - Missing fields in properties array

handlers for get_node_essentials/info need to be found and audited

Solution Architecture

1. Enhanced Safe Property Access Utilities

File: src/utils/safe-property-access.ts (NEW)

/**
 * Safe Property Access Utilities
 *
 * Provides defensive property access with fallbacks
 */

export class SafePropertyAccess {
  /**
   * Safely get nested property with default
   */
  static get<T>(obj: any, path: string, defaultValue: T): T {
    if (!obj || typeof obj !== 'object') return defaultValue;

    const keys = path.split('.');
    let current = obj;

    for (const key of keys) {
      if (current === null || current === undefined) {
        return defaultValue;
      }
      if (typeof current !== 'object') {
        return defaultValue;
      }
      current = current[key];
    }

    return current !== undefined ? current : defaultValue;
  }

  /**
   * Safely get array with default
   */
  static getArray<T>(obj: any, path: string, defaultValue: T[] = []): T[] {
    const value = this.get(obj, path, defaultValue);
    return Array.isArray(value) ? value : defaultValue;
  }

  /**
   * Safely get string with default
   */
  static getString(obj: any, path: string, defaultValue: string = ''): string {
    const value = this.get(obj, path, defaultValue);
    return typeof value === 'string' ? value : defaultValue;
  }

  /**
   * Safely get number with default
   */
  static getNumber(obj: any, path: string, defaultValue: number = 0): number {
    const value = this.get(obj, path, defaultValue);
    return typeof value === 'number' && !isNaN(value) ? value : defaultValue;
  }

  /**
   * Safely get boolean with default
   */
  static getBoolean(obj: any, path: string, defaultValue: boolean = false): boolean {
    const value = this.get(obj, path, defaultValue);
    return typeof value === 'boolean' ? value : defaultValue;
  }

  /**
   * Extract description from multiple possible locations
   */
  static extractDescription(obj: any): string {
    // Try common description locations
    const locations = [
      'description',
      'properties.description',
      'properties.description.text',
      'subtitle',
      'displayName'
    ];

    for (const location of locations) {
      const value = this.getString(obj, location);
      if (value) return value;
    }

    return 'No description available';
  }

  /**
   * Extract display name from multiple possible locations
   */
  static extractDisplayName(obj: any, fallback: string = 'Unknown'): string {
    const locations = [
      'displayName',
      'name',
      'label',
      'title'
    ];

    for (const location of locations) {
      const value = this.getString(obj, location);
      if (value) return value;
    }

    return fallback;
  }
}

2. Null-Safe Node Repository Methods

File: src/database/node-repository.ts

Refactor getNode method (line 54):

import { SafePropertyAccess } from '../utils/safe-property-access';

/**
 * Get node with comprehensive null-safety
 */
getNode(nodeType: string): any | null {
  try {
    // Normalize type first
    const normalizedType = NodeTypeNormalizer.normalizeToFullForm(nodeType);

    const row = this.db.prepare(`
      SELECT * FROM nodes WHERE node_type = ?
    `).get(normalizedType) as any;

    if (!row) return null;

    // Use safe property access for all fields
    return {
      nodeType: SafePropertyAccess.getString(row, 'node_type', normalizedType),
      displayName: SafePropertyAccess.extractDisplayName(row,
        SafePropertyAccess.getString(row, 'display_name', 'Unknown Node')),
      description: SafePropertyAccess.extractDescription(row),
      category: SafePropertyAccess.getString(row, 'category', 'Uncategorized'),
      developmentStyle: SafePropertyAccess.getString(row, 'development_style', 'declarative'),
      package: SafePropertyAccess.getString(row, 'package_name', 'unknown'),
      isAITool: SafePropertyAccess.getBoolean(row, 'is_ai_tool', false),
      isTrigger: SafePropertyAccess.getBoolean(row, 'is_trigger', false),
      isWebhook: SafePropertyAccess.getBoolean(row, 'is_webhook', false),
      isVersioned: SafePropertyAccess.getBoolean(row, 'is_versioned', false),
      version: SafePropertyAccess.getNumber(row, 'version', 1),
      properties: this.safeParseProperties(row.properties_schema),
      operations: this.safeParseArray(row.operations),
      credentials: this.safeParseArray(row.credentials_required),
      hasDocumentation: !!row.documentation,
      outputs: row.outputs ? this.safeJsonParse(row.outputs, null) : null,
      outputNames: row.output_names ? this.safeJsonParse(row.output_names, null) : null
    };
  } catch (error) {
    console.error(`Error getting node ${nodeType}:`, error);
    return null;
  }
}

/**
 * Safely parse properties with validation
 */
private safeParseProperties(json: string): any[] {
  try {
    const parsed = JSON.parse(json);
    if (!Array.isArray(parsed)) return [];

    // Validate each property has minimum required fields
    return parsed.map(prop => ({
      name: SafePropertyAccess.getString(prop, 'name', 'unknown'),
      displayName: SafePropertyAccess.extractDisplayName(prop),
      type: SafePropertyAccess.getString(prop, 'type', 'string'),
      required: SafePropertyAccess.getBoolean(prop, 'required', false),
      default: prop.default !== undefined ? prop.default : null,
      description: SafePropertyAccess.extractDescription(prop),
      options: SafePropertyAccess.getArray(prop, 'options', []),
      displayOptions: prop.displayOptions || null
    }));
  } catch {
    return [];
  }
}

/**
 * Safely parse array field
 */
private safeParseArray(json: string): any[] {
  try {
    const parsed = JSON.parse(json);
    return Array.isArray(parsed) ? parsed : [];
  } catch {
    return [];
  }
}

3. Find and Fix Handler Functions

Action Required: Search for handler functions that call getNode and add null checks

Pattern to search for:

grep -r "getNode\|getNodeEssentials\|getNodeInfo" src/mcp/ --include="*.ts"

Add null checks like:

const node = repository.getNode(nodeType);
if (!node) {
  return {
    success: false,
    error: `Node type "${nodeType}" not found. Use search_nodes to find available nodes.`
  };
}

Testing Requirements

File: tests/unit/database/node-repository-null-safety.test.ts (NEW)

describe('NodeRepository - Null Safety', () => {
  it('should handle node with missing description', () => {
    // Insert node with minimal data
    const node = { type: 'test.node', name: 'Test' };
    db.prepare('INSERT INTO nodes (node_type, display_name) VALUES (?, ?)').run(node.type, node.name);

    const result = repository.getNode('test.node');
    expect(result).not.toBeNull();
    expect(result.description).toBe('No description available');
    expect(result.properties).toEqual([]);
  });

  it('should handle node with malformed JSON', () => {
    db.prepare('INSERT INTO nodes (node_type, properties_schema) VALUES (?, ?)').run('test.node', 'invalid json');

    const result = repository.getNode('test.node');
    expect(result).not.toBeNull();
    expect(result.properties).toEqual([]);
  });

  it('should handle non-existent node gracefully', () => {
    const result = repository.getNode('non.existent');
    expect(result).toBeNull();
  });

  it('should handle null database row', () => {
    // Simulate database returning null
    const result = repository.getNode('null.node');
    expect(result).toBeNull();
  });
});

Success Criteria

  • get_node_essentials failure rate: 10% → <1%
  • get_node_info failure rate: 18% → <1%
  • get_node_documentation failure rate: 7% → <1%
  • 100% test coverage for null cases
  • No TypeErrors in production logs

Estimated Effort

Total: 1 day (8 hours)

  • Safe property access utility: 2 hours
  • Repository refactoring: 3 hours
  • Handler updates: 2 hours
  • Testing: 1 hour

P0-R3: Pre-extract Template Configurations + Remove get_node_for_task

Problem Statement

Impact: 28% failure rate (worst-performing tool) + redundant with better alternatives

get_node_for_task failing 109 times out of 392 calls (27.8%)

Current State:

  • Only 31 predefined tasks in task-templates.ts (5.9% node coverage)
  • 22.5:1 usage ratio favoring search_nodes (8,839 calls vs 392)
  • Hardcoded configurations require manual maintenance
  • Tool provides no unique value over search_nodes

Discovery: We have 2,646 real production workflow templates from n8n.io with:

  • 3,820 httpRequest configurations
  • 1,700 googleSheets configurations
  • 466 webhook configurations
  • 100% AI-generated metadata coverage
  • Real-world best practices and patterns

Architectural Decision: Pre-extraction

Analysis: On-the-fly vs Pre-extraction (see /docs/local/TEMPLATE_MINING_ANALYSIS.md)

Decision: Pre-extract node configurations into separate table

Rationale:

  • Performance: 1ms vs 30-60ms (30-60x faster)
  • Storage: Only 513 KB for 2,625 configs (negligible)
  • Simplicity: No cache management, TTL, or eviction logic
  • Features: Enables filtering by complexity, auth (indexed queries)
  • Scalability: Handles 10,000+ templates without degradation
  • Predictability: Consistent sub-millisecond response times

Trade-offs (acceptable):

  • +30-60 seconds rebuild time (rare operation)
  • Incremental updates needed when templates change

Solution Architecture

Strategy:

  1. Pre-extract top 10 node configurations per node type into new table
  2. Enhance get_node_essentials with optional examples
  3. Enhance search_nodes with optional examples
  4. Remove get_node_for_task entirely (no redirect)

See /docs/local/TEMPLATE_MINING_ANALYSIS.md for complete analysis

1. Add Database Schema for Pre-extracted Configurations

File: src/database/schema.sql

Add new table after templates table:

-- Pre-extracted node configurations from templates
CREATE TABLE template_node_configs (
  id INTEGER PRIMARY KEY,
  node_type TEXT NOT NULL,
  template_id INTEGER NOT NULL,
  template_name TEXT NOT NULL,
  template_views INTEGER DEFAULT 0,

  -- Node configuration (extracted from workflow)
  node_name TEXT,                  -- Node name in workflow (e.g., "HTTP Request")
  parameters_json TEXT NOT NULL,   -- JSON: node.parameters
  credentials_json TEXT,            -- JSON: node.credentials (if present)

  -- Pre-calculated metadata for filtering
  has_credentials INTEGER DEFAULT 0,
  has_expressions INTEGER DEFAULT 0,  -- Contains {{...}} or $json/$node
  complexity TEXT CHECK(complexity IN ('simple', 'medium', 'complex')),
  use_cases TEXT,                   -- JSON array from template.metadata.use_cases

  -- Pre-calculated ranking (1 = best, 2 = second best, etc.)
  rank INTEGER DEFAULT 0,

  created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
  FOREIGN KEY (template_id) REFERENCES templates(id) ON DELETE CASCADE
);

-- Indexes for fast queries
CREATE INDEX idx_config_node_type_rank
  ON template_node_configs(node_type, rank);

CREATE INDEX idx_config_complexity
  ON template_node_configs(node_type, complexity, rank);

CREATE INDEX idx_config_auth
  ON template_node_configs(node_type, has_credentials, rank);

-- View for easy querying of top configs
CREATE VIEW ranked_node_configs AS
SELECT
  node_type,
  template_name,
  template_views,
  parameters_json,
  credentials_json,
  has_credentials,
  has_expressions,
  complexity,
  use_cases,
  rank
FROM template_node_configs
WHERE rank <= 5  -- Top 5 per node type
ORDER BY node_type, rank;

Migration Script: src/database/migrations/add-template-node-configs.sql

-- Migration for existing databases
-- Run during `npm run rebuild` or `npm run fetch:templates`

-- Check if table exists
CREATE TABLE IF NOT EXISTS template_node_configs (
  -- ... schema as above
);

-- Populate from existing templates
-- (handled by extraction logic in fetch:templates script)

2. Add Extraction Logic to fetch:templates Script

File: src/scripts/fetch-templates.ts

Add extraction function:

import gzip from 'zlib';

/**
 * Extract node configurations from a template workflow
 */
function extractNodeConfigs(
  templateId: number,
  templateName: string,
  templateViews: number,
  workflowCompressed: string,
  metadata: any
): Array<{
  node_type: string;
  template_id: number;
  template_name: string;
  template_views: number;
  node_name: string;
  parameters_json: string;
  credentials_json: string | null;
  has_credentials: number;
  has_expressions: number;
  complexity: string;
  use_cases: string;
}> {
  try {
    // Decompress workflow
    const decompressed = gzip.gunzipSync(Buffer.from(workflowCompressed, 'base64'));
    const workflow = JSON.parse(decompressed.toString('utf-8'));

    const configs: any[] = [];

    for (const node of workflow.nodes || []) {
      // Skip UI-only nodes
      if (node.type.includes('stickyNote') || !node.parameters) {
        continue;
      }

      configs.push({
        node_type: node.type,
        template_id: templateId,
        template_name: templateName,
        template_views: templateViews,
        node_name: node.name,
        parameters_json: JSON.stringify(node.parameters),
        credentials_json: node.credentials ? JSON.stringify(node.credentials) : null,
        has_credentials: node.credentials ? 1 : 0,
        has_expressions: detectExpressions(node.parameters) ? 1 : 0,
        complexity: metadata?.complexity || 'medium',
        use_cases: JSON.stringify(metadata?.use_cases || [])
      });
    }

    return configs;
  } catch (error) {
    console.error(`Error extracting configs from template ${templateId}:`, error);
    return [];
  }
}

/**
 * Detect n8n expressions in parameters
 */
function detectExpressions(params: any): boolean {
  const json = JSON.stringify(params);
  return json.includes('={{') || json.includes('$json') || json.includes('$node');
}

/**
 * Insert extracted configs into database and rank them
 */
function insertAndRankConfigs(db: Database, configs: any[]) {
  // Clear old configs for these templates
  const templateIds = [...new Set(configs.map(c => c.template_id))];
  db.prepare(`DELETE FROM template_node_configs WHERE template_id IN (${templateIds.join(',')})`).run();

  // Insert new configs
  const insertStmt = db.prepare(`
    INSERT INTO template_node_configs (
      node_type, template_id, template_name, template_views,
      node_name, parameters_json, credentials_json,
      has_credentials, has_expressions, complexity, use_cases
    ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
  `);

  for (const config of configs) {
    insertStmt.run(
      config.node_type,
      config.template_id,
      config.template_name,
      config.template_views,
      config.node_name,
      config.parameters_json,
      config.credentials_json,
      config.has_credentials,
      config.has_expressions,
      config.complexity,
      config.use_cases
    );
  }

  // Rank configs per node_type by template popularity
  db.exec(`
    UPDATE template_node_configs
    SET rank = (
      SELECT COUNT(*) + 1
      FROM template_node_configs AS t2
      WHERE t2.node_type = template_node_configs.node_type
        AND t2.template_views > template_node_configs.template_views
    )
  `);

  // Keep only top 10 per node_type
  db.exec(`
    DELETE FROM template_node_configs
    WHERE id NOT IN (
      SELECT id FROM template_node_configs
      WHERE rank <= 10
    )
  `);

  console.log(`Extracted and ranked ${configs.length} node configurations`);
}

3. Enhance get_node_essentials with Examples

File: src/mcp/handlers-*.ts or src/mcp/server.ts

Update get_node_essentials handler:

async function getNodeEssentials(
  nodeType: string,
  options?: { includeExamples?: boolean }
): Promise<any> {
  const node = repository.getNode(nodeType);
  if (!node) {
    return {
      success: false,
      error: `Node type "${nodeType}" not found. Use search_nodes to find available nodes.`
    };
  }

  const result = {
    nodeType,
    displayName: node.displayName,
    description: node.description,
    category: node.category,
    // ... existing essentials fields ...
  };

  // NEW: Add real-world examples if requested
  if (options?.includeExamples) {
    const examples = db.prepare(`
      SELECT
        parameters_json,
        template_name,
        template_views,
        complexity,
        use_cases,
        has_credentials,
        has_expressions
      FROM template_node_configs
      WHERE node_type = ?
      ORDER BY rank
      LIMIT 3
    `).all(nodeType);

    result.examples = examples.map(ex => ({
      config: JSON.parse(ex.parameters_json),
      source: `${ex.template_name} (${(ex.template_views / 1000).toFixed(0)}k views)`,
      complexity: ex.complexity,
      useCases: JSON.parse(ex.use_cases).slice(0, 2),
      hasAuth: ex.has_credentials === 1,
      hasExpressions: ex.has_expressions === 1
    }));
  }

  return result;
}

Tool definition update:

{
  name: 'get_node_essentials',
  description: 'Get essential information about a specific n8n node type...',
  inputSchema: {
    type: 'object',
    properties: {
      nodeType: {
        type: 'string',
        description: 'Full node type (e.g., "n8n-nodes-base.httpRequest")'
      },
      includeExamples: {  // NEW
        type: 'boolean',
        description: 'Include 2-3 real configuration examples from popular templates',
        default: false
      }
    },
    required: ['nodeType']
  }
}

4. Enhance search_nodes with Examples

File: src/mcp/handlers-*.ts or src/mcp/server.ts

Update search_nodes handler:

async function searchNodes(
  query: string,
  options?: {
    limit?: number;
    includeExamples?: boolean;
  }
): Promise<any> {
  const nodes = repository.searchNodes(query, 'OR', options?.limit || 20);

  const results = nodes.map(node => {
    const result = {
      nodeType: node.nodeType,
      displayName: node.displayName,
      description: node.description,
      category: node.category
    };

    // NEW: Add examples if requested
    if (options?.includeExamples) {
      const examples = db.prepare(`
        SELECT parameters_json, template_name, complexity
        FROM template_node_configs
        WHERE node_type = ?
        ORDER BY rank
        LIMIT 2
      `).all(node.nodeType);

      result.examples = examples.map(ex => ({
        config: JSON.parse(ex.parameters_json),
        source: ex.template_name,
        complexity: ex.complexity
      }));
    }

    return result;
  });

  return results;
}

Tool definition update:

{
  name: 'search_nodes',
  description: 'Search for n8n nodes by keyword...',
  inputSchema: {
    type: 'object',
    properties: {
      query: { type: 'string', description: 'Search query' },
      limit: { type: 'number', default: 20 },
      includeExamples: {  // NEW
        type: 'boolean',
        description: 'Include 2 real configuration examples per node',
        default: false
      }
    },
    required: ['query']
  }
}

5. Remove get_node_for_task Tool Entirely

Files to modify:

  1. src/mcp/server.ts - Remove handler function
  2. src/mcp/tools.ts - Remove tool definition
  3. src/mcp/tools-documentation.ts - Remove from documentation
  4. src/services/task-templates.ts - Can be deprecated (keep for now, remove in v2.16.0)
  5. README.md - Remove from available tools list
  6. CHANGELOG.md - Document removal

Steps:

# Search for all references
grep -r "get_node_for_task" src/
grep -r "getNodeForTask" src/

# Remove handler
# Remove tool definition
# Remove from documentation
# Update README

Migration note for users (add to CHANGELOG):

### BREAKING CHANGES in v2.15.0

- **Removed:** `get_node_for_task` tool
  - **Replacement:** Use `search_nodes` with `includeExamples: true`
  - **Migration:** `get_node_for_task({task: "webhook"})``search_nodes({query: "webhook", includeExamples: true})`
  - **Benefit:** Access to 2,646 real templates vs 31 hardcoded tasks

Testing Requirements

File: tests/unit/services/template-config-extraction.test.ts (NEW)

describe('Template Config Extraction', () => {
  it('should extract node configs from workflow', () => {
    const workflow = {
      nodes: [
        {
          type: 'n8n-nodes-base.httpRequest',
          name: 'HTTP Request',
          parameters: { url: 'https://api.example.com', method: 'GET' }
        }
      ]
    };

    const configs = extractNodeConfigs(1, 'Test', 1000, compressWorkflow(workflow), {});
    expect(configs).toHaveLength(1);
    expect(configs[0].node_type).toBe('n8n-nodes-base.httpRequest');
  });

  it('should detect expressions in parameters', () => {
    const params = { url: '={{$json.api_url}}' };
    expect(detectExpressions(params)).toBe(true);
  });

  it('should rank configs by popularity', () => {
    // Insert configs with different views
    // Verify ranking order
  });
});

File: tests/integration/enhanced-tools.test.ts (NEW)

describe('Enhanced Tools with Examples', () => {
  it('get_node_essentials should return examples when requested', async () => {
    const result = await getNodeEssentials('n8n-nodes-base.httpRequest', {
      includeExamples: true
    });

    expect(result.examples).toBeDefined();
    expect(result.examples.length).toBeGreaterThan(0);
    expect(result.examples[0].config).toHaveProperty('url');
  });

  it('search_nodes should return examples when requested', async () => {
    const result = await searchNodes('webhook', { includeExamples: true });

    expect(result.length).toBeGreaterThan(0);
    expect(result[0].examples).toBeDefined();
  });

  it('get_node_for_task should not exist', async () => {
    expect(toolRegistry.has('get_node_for_task')).toBe(false);
  });
});

Success Criteria

  • Extract 2,000+ node configurations from templates
  • Query performance: <1ms for pre-extracted configs
  • get_node_essentials with examples: <5ms total
  • search_nodes with examples: <10ms total
  • Database size increase: <1 MB
  • get_node_for_task completely removed from codebase
  • All documentation updated

Estimated Effort

Total: 1 week (5 days)

  • Day 1: Database schema + migration (8 hours)

    • Design schema
    • Create migration script
    • Test with existing database
  • Day 2: Extraction logic in fetch:templates (8 hours)

    • Write extraction function
    • Write ranking logic
    • Test with 2,646 templates
  • Day 3: Enhance get_node_essentials + search_nodes (8 hours)

    • Add includeExamples parameter
    • Update tool definitions
    • Integration testing
  • Day 4: Remove get_node_for_task + documentation (8 hours)

    • Remove from all files
    • Update README, CHANGELOG
    • Update tools_documentation
    • Migration guide
  • Day 5: Testing + optimization (8 hours)

    • Unit tests
    • Integration tests
    • Performance testing
    • Bug fixes

Implementation Order & Timeline

Version 2.15.0 - All P0 Fixes in One Release

Total Timeline: 2 weeks (10 working days)

Week 1: Foundation + P0-R1 + P0-R2

Monday (Day 1-2): P0-R1 - Node Type Normalization

  • AM: Create NodeTypeNormalizer utility
  • PM: Apply to workflow validator, handlers, and repository
  • Testing and validation
  • Deliverable: 80% of validation errors eliminated

Tuesday (Day 3): P0-R2 - Null-Safety Audit (Part 1)

  • AM: Create SafePropertyAccess utility
  • PM: Refactor node repository methods
  • Deliverable: Safe property access framework

Wednesday (Day 4): P0-R2 - Null-Safety Audit (Part 2)

  • AM: Find and fix all handlers
  • PM: Comprehensive null-safety testing
  • Deliverable: 10-18% TypeError rate → <1%

Thursday (Day 5): P0-R3 - Database Schema

  • AM: Design and implement template_node_configs table
  • PM: Create migration script and test with existing database
  • Deliverable: Schema ready for extraction

Friday (Day 6): P0-R3 - Extraction Logic

  • AM: Write extraction function in fetch:templates
  • PM: Write ranking logic and test with 2,646 templates
  • Deliverable: 2,000+ configs extracted and ranked

Week 2: P0-R3 Integration + Testing + Documentation

Monday (Day 7): Tool Enhancements

  • AM: Enhance get_node_essentials with includeExamples
  • PM: Enhance search_nodes with includeExamples
  • Deliverable: Both tools return real examples

Tuesday (Day 8): Tool Removal + Documentation

  • AM: Remove get_node_for_task from all files
  • PM: Update README, CHANGELOG, tools_documentation
  • Deliverable: Clean removal, migration guide complete

Wednesday (Day 9): Comprehensive Testing

  • AM: Unit tests for extraction and enhanced tools
  • PM: Integration tests for all P0 fixes
  • Deliverable: 95%+ test coverage

Thursday (Day 10): Performance + Final Testing

  • AM: Performance testing and optimization
  • PM: E2E testing and bug fixes
  • Deliverable: All success criteria met

Friday (Day 11): Release Preparation

  • AM: Code review and documentation review
  • PM: Prepare release notes, tag v2.15.0
  • Deliverable: Ready for release

Parallel Activities

  • Documentation updates: Days 1-11
  • Code reviews: End of Days 2, 4, 6, 8, 10
  • Telemetry preparation: Day 10-11 (prepare monitoring dashboard)

Testing Strategy

Unit Tests

Coverage Target: 95% for new code

  • Node Type Normalizer: 20+ test cases
  • Safe Property Access: 30+ test cases
  • Task Discovery Service: 40+ test cases

Integration Tests

  • Workflow validation with mixed node type forms
  • Node repository with edge case data
  • Task discovery with real node database

E2E Tests

  • Create workflow with short-form node types → Should succeed
  • Get node info for nodes with missing properties → Should return safe defaults
  • Query task discovery with variations → Should find matches

Regression Tests

  • All existing tests must pass
  • No breaking changes to public APIs

Performance Tests

  • Normalization overhead: <1ms per workflow
  • Safe property access: <0.1ms per node
  • Task discovery: <50ms average

Rollback Plan

If P0-R1 Causes Issues

  1. Symptom: Workflows fail validation after normalization
  2. Action: Revert node-type-normalizer changes
  3. Fallback: Use original normalizeNodeType
  4. Recovery time: 15 minutes

If P0-R2 Causes Performance Issues

  1. Symptom: Node lookup becomes slow
  2. Action: Cache safe property access results
  3. Fallback: Keep safe parsing but reduce validation
  4. Recovery time: 1 hour

If P0-R3 Template Extraction Causes Issues

  1. Symptom: Database bloat or slow queries
  2. Action: Reduce rank limit from 10 to 5 per node
  3. Fallback: Disable includeExamples parameter temporarily
  4. Recovery time: 15 minutes (just disable parameter)

If get_node_for_task Removal Causes User Issues

  1. Symptom: Users report missing tool
  2. Action: Add prominent migration guide to error messages
  3. Fallback: N/A (breaking change, users must migrate)
  4. Communication: Update docs, add migration examples

Success Metrics

Overall Goals

Metric Current Target How to Measure
Overall error rate 5-10% <2% Telemetry events
Validation errors 4,800/week <100/week Error logs
TypeError rate 10-18% <1% Tool execution logs
Node configs extracted 0 2,000+ Database count
Config query performance N/A <1ms Performance tests
get_node_for_task usage 392 calls 0 (removed) Tool usage stats
search_nodes w/ examples 0 Monitored New feature adoption

Telemetry Monitoring

After deployment, monitor for 1 week:

  • Error rate by tool (should decrease 80-90%)
  • User success rate (should increase 5-10%)
  • Average errors per user (should decrease from 2.5 to <0.5)

Dependencies

NPM Packages

No new NPM packages required - all functionality uses existing dependencies.

Internal Dependencies

  • P0-R3 requires database schema update (template_node_configs table)
  • P0-R3 requires migration script for existing databases
  • All changes are backward compatible except removal of get_node_for_task

Documentation Updates

Files to Update

  1. CHANGELOG.md - Add entries for each P0 fix + breaking changes
  2. README.md - Remove get_node_for_task, add includeExamples parameter
  3. src/mcp/tools-documentation.ts - Remove get_node_for_task documentation
  4. API.md - Document enhanced tool parameters
  5. MIGRATION.md - Add migration guide from get_node_for_task to search_nodes (NEW)

Example CHANGELOG Entry

## [2.15.0] - 2025-10-09

### BREAKING CHANGES
- **Removed:** `get_node_for_task` tool
  - **Replacement:** Use `search_nodes` with `includeExamples: true`
  - **Migration:** `get_node_for_task({task: "webhook"})``search_nodes({query: "webhook", includeExamples: true})`
  - **Benefit:** Access to 2,646 real templates vs 31 hardcoded tasks

### Fixed
- **P0-R1:** Auto-normalize node type prefixes (eliminates 80% of validation errors)
- **P0-R2:** Complete null-safety audit for node information tools (reduces TypeError failures from 10-18% to <1%)

### Added
- `NodeTypeNormalizer` utility for universal node type normalization
- `SafePropertyAccess` utility for defensive property access
- `template_node_configs` table with 2,000+ pre-extracted configurations
- `includeExamples` parameter for `get_node_essentials` (returns 2-3 real configs)
- `includeExamples` parameter for `search_nodes` (returns 2 real configs per node)
- Real-world configuration examples from popular n8n templates

### Performance
- Node configuration queries: <1ms (30-60x faster than on-the-fly extraction)
- Sub-millisecond response time for configuration examples

Conclusion

These P0 fixes represent the highest-impact improvements we can make to n8n-mcp based on real production telemetry data. By implementing all three fixes in v2.15.0, we will:

  1. Eliminate 80% of validation errors (P0-R1: Node type normalization)
  2. Fix the majority of TypeError failures (P0-R2: Null-safety audit)
  3. Replace inferior tool with superior alternative (P0-R3: Template-based configs + remove get_node_for_task)

Expected Overall Impact:

  • Error rate: 5-10% → <2%
  • Configuration examples: 31 hardcoded → 2,000+ real templates
  • Query performance: 30-60ms → <1ms (30-60x faster)
  • User experience: Significant improvement across all tools
  • Support burden: Reduced by 50%+

Key Innovation (P0-R3):

  • Pre-extraction delivers 30-60x performance improvement
  • 2,646 real templates provide richer context than hardcoded tasks
  • Breaking change justified by superior replacement
  • Database increase: Only +513 KB for 2,625 configurations

The implementation is well-architected, delivers exceptional value, and sets up future enhancements.


Next Steps:

  1. Review implementation plan with team (COMPLETED)
  2. Finalize architectural decisions (COMPLETED - pre-extraction chosen)
  3. Create feature branch: feature/p0-priorities-fixes (COMPLETED)
  4. P0-R1: Auto-Normalize Node Type Prefixes (COMPLETED - commit ed7de10)
  5. P0-R2: Complete Null-Safety Audit (PENDING)
  6. P0-R3: Pre-extract Template Configs + Remove get_node_for_task (PENDING)
  7. Deploy v2.15.0 with monitoring and telemetry analysis

Target Release: v2.15.0 (estimated 1.5 weeks remaining)