Fixes production search failures where 69% of user searches returned zero
results for critical nodes (webhook, merge, split batch) despite nodes
existing in database.
Root Cause:
- schema.sql missing nodes_fts FTS5 virtual table
- No validation to detect empty database or missing FTS5
- rebuild.ts used schema without search index
- Result: 9 of 13 searches failed in production
Changes:
1. Schema Updates (src/database/schema.sql):
- Added nodes_fts FTS5 virtual table with full-text indexing
- Added INSERT/UPDATE/DELETE triggers for auto-sync
- Indexes: node_type, display_name, description, documentation, operations
2. Database Validation (src/scripts/rebuild.ts):
- Added empty database detection (fails if zero nodes)
- Added FTS5 existence and synchronization validation
- Added searchability tests for critical nodes
- Added minimum node count check (500+)
3. Runtime Health Checks (src/mcp/server.ts):
- Database health validation on first access
- Detects empty database with clear error
- Detects missing FTS5 with actionable warning
4. Test Suite (53 new tests):
- tests/integration/database/node-fts5-search.test.ts (14 tests)
- tests/integration/database/empty-database.test.ts (14 tests)
- tests/integration/ci/database-population.test.ts (25 tests)
5. Database Rebuild:
- data/nodes.db rebuilt with FTS5 index
- 535 nodes fully synchronized with FTS5
Impact:
- ✅ All critical searches now work (webhook, merge, split, code, http)
- ✅ FTS5 provides fast ranked search (< 100ms)
- ✅ Clear error messages if database empty
- ✅ CI validates committed database integrity
- ✅ Runtime health checks detect issues immediately
Performance:
- FTS5 search: < 100ms for typical queries
- LIKE fallback: < 500ms (unchanged, still functional)
Testing: LIKE search investigation revealed it was perfectly functional,
only failed because database was empty. No changes needed.
Related: Issue #296 Part 2 (Part 1: v2.18.4 fixed adapter bypass)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Implement OpenAI batch API integration for metadata generation
- Add metadata columns to database schema (metadata_json, metadata_generated_at)
- Create MetadataGenerator service with structured output schemas
- Create BatchProcessor for handling OpenAI batch jobs
- Add --generate-metadata flag to fetch-templates script
- Update template repository with metadata management methods
- Add OpenAI configuration to environment variables
- Include comprehensive tests for metadata generation
- Use gpt-4o-mini model with 50% cost savings via batch API
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add gzip compression for workflow JSONs (89% size reduction)
- Filter templates with ≤10 views to remove low-quality content
- Reduce template count from 4,505 to 2,596 high-quality templates
- Compress template data from ~75MB to 12.10MB
- Total database reduced from 117MB to 48MB
- Add on-the-fly decompression for template retrieval
- Update schema to support compressed workflow storage
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
## Problem
AI assistants were consistently connecting SplitInBatches node outputs backwards because:
- Output index 0 = "done" (runs after loop completes)
- Output index 1 = "loop" (processes items inside loop)
This counterintuitive ordering caused incorrect workflow connections.
## Solution
Enhanced the n8n-mcp system to expose and clarify output information:
### Database & Schema
- Added `outputs` and `output_names` columns to nodes table
- Updated NodeRepository to store/retrieve output information
### Node Parsing
- Enhanced NodeParser to extract outputs and outputNames from nodes
- Properly handles versioned nodes like SplitInBatchesV3
### MCP Server
- Modified getNodeInfo to return detailed output descriptions
- Added connection guidance for each output
- Special handling for loop nodes (SplitInBatches, IF, Switch)
### Documentation
- Enhanced DocsMapper to inject critical output guidance
- Added warnings about counterintuitive output ordering
- Provides correct connection patterns for loop nodes
### Workflow Validation
- Added validateSplitInBatchesConnection method
- Detects reversed connections and provides specific errors
- Added checkForLoopBack with depth limit to prevent stack overflow
- Smart heuristics to identify likely connection mistakes
## Testing
- Created comprehensive test suite (81 tests)
- Unit tests for all modified components
- Edge case handling for malformed data
- Performance testing with large workflows
## Impact
AI assistants will now:
- See explicit output indices and names (e.g., "Output 0: done")
- Receive clear connection guidance
- Get validation errors when connections are reversed
- Have enhanced documentation explaining the correct pattern
Fixes#97🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Added runtime FTS5 detection in database adapters
- Removed FTS5 from required schema to prevent "no such module" errors
- FTS5 tables/triggers created conditionally only if supported
- Template search automatically falls back to LIKE when FTS5 unavailable
- Works in ALL SQLite environments (Claude Desktop, restricted envs, etc.)
This ensures search_templates() works correctly regardless of SQLite build,
while still providing optimal performance when FTS5 is available.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- SQLite doesn't allow datetime('now') in CHECK constraints
- Drop tables before recreating to ensure clean schema
- 6-month filtering is already handled in application logic
- Successfully fetched and stored 199 templates
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
- Add 4 new MCP tools for workflow templates
- Integrate with n8n.io API to fetch community templates
- Filter templates to last 6 months only
- Store templates in SQLite with full workflow JSON
- Manual fetch system (not part of regular rebuild)
- Support search by nodes, keywords, and task categories
- Add fetch:templates and test:templates npm scripts
- Update to v2.4.1
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
