Commit Graph

7 Commits

Author SHA1 Message Date
czlonkowski
cfd2325ca4 fix: add FTS5 search index to prevent 69% search failure rate (v2.18.5)
Fixes production search failures where 69% of user searches returned zero
results for critical nodes (webhook, merge, split batch) despite nodes
existing in database.

Root Cause:
- schema.sql missing nodes_fts FTS5 virtual table
- No validation to detect empty database or missing FTS5
- rebuild.ts used schema without search index
- Result: 9 of 13 searches failed in production

Changes:
1. Schema Updates (src/database/schema.sql):
   - Added nodes_fts FTS5 virtual table with full-text indexing
   - Added INSERT/UPDATE/DELETE triggers for auto-sync
   - Indexes: node_type, display_name, description, documentation, operations

2. Database Validation (src/scripts/rebuild.ts):
   - Added empty database detection (fails if zero nodes)
   - Added FTS5 existence and synchronization validation
   - Added searchability tests for critical nodes
   - Added minimum node count check (500+)

3. Runtime Health Checks (src/mcp/server.ts):
   - Database health validation on first access
   - Detects empty database with clear error
   - Detects missing FTS5 with actionable warning

4. Test Suite (53 new tests):
   - tests/integration/database/node-fts5-search.test.ts (14 tests)
   - tests/integration/database/empty-database.test.ts (14 tests)
   - tests/integration/ci/database-population.test.ts (25 tests)

5. Database Rebuild:
   - data/nodes.db rebuilt with FTS5 index
   - 535 nodes fully synchronized with FTS5

Impact:
-  All critical searches now work (webhook, merge, split, code, http)
-  FTS5 provides fast ranked search (< 100ms)
-  Clear error messages if database empty
-  CI validates committed database integrity
-  Runtime health checks detect issues immediately

Performance:
- FTS5 search: < 100ms for typical queries
- LIKE fallback: < 500ms (unchanged, still functional)

Testing: LIKE search investigation revealed it was perfectly functional,
only failed because database was empty. No changes needed.

Related: Issue #296 Part 2 (Part 1: v2.18.4 fixed adapter bypass)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-10 09:16:20 +02:00
czlonkowski
b65a2f8f3d chore: update n8n dependencies to latest versions
- Updated n8n-nodes-base to 1.106.3
- Updated @n8n/n8n-nodes-langchain to 1.106.3
- Enhanced SQL.js compatibility in database adapter
- Fixed parameter binding and state management in SQLJSStatement
- Rebuilt node database with 535 nodes
- All tests passing with Node.js v22.17.0 LTS
2025-09-04 10:24:33 +02:00
czlonkowski
caf1e008d7 fix: respect NODE_DB_PATH environment variable in Docker (fixes #40)
- Fixed docker-entrypoint.sh to use NODE_DB_PATH instead of hardcoded paths
- Added log_message() helper to prevent stdio mode output corruption
- Fixed directory creation race condition with proper ownership
- Added path validation to ensure NODE_DB_PATH ends with .db
- Updated rebuild.ts to respect NODE_DB_PATH environment variable
- Added comprehensive documentation for custom database paths
- Bumped version to 2.7.16

The bug was caused by hardcoded /app/data/nodes.db paths in the Docker
entrypoint script that ignored the NODE_DB_PATH environment variable.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-16 09:49:41 +02:00
czlonkowski
e912b1c90d feat: integrate automatic template sanitization into rebuild process
- Update rebuild script to automatically sanitize templates after node processing
- Update documentation to reflect GitHub push protection changes
- Add notes about template sanitization in n8n update process
- Ensures databases are always clean before push

This prevents GitHub push protection from blocking database updates
due to API tokens in workflow templates.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-06 13:34:52 +02:00
czlonkowski
b476d36275 feat: implement universal Node.js compatibility with automatic database adapter fallback 2025-06-12 23:51:47 +02:00
czlonkowski
66f5d74e42 feat: implement node parser and property extractor with versioned node support 2025-06-12 18:45:20 +02:00
czlonkowski
8bf670c31e feat: Implement n8n-MCP Enhancement Plan v2.1 Final
- Implement simple node loader supporting n8n-nodes-base and langchain packages
- Create parser handling declarative, programmatic, and versioned nodes
- Build documentation mapper with 89% coverage (405/457 nodes)
- Setup SQLite database with minimal schema
- Create rebuild script for one-command database updates
- Implement validation script for critical nodes
- Update MCP server with documentation-focused tools
- Add npm scripts for streamlined workflow

Successfully loads 457/458 nodes with accurate documentation mapping.
Versioned node detection working (46 nodes detected).
3/4 critical nodes pass validation tests.

Known limitations:
- Slack operations extraction incomplete for some versioned nodes
- One langchain node fails due to missing dependency
- No AI tools detected (none have usableAsTool flag)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-12 14:18:19 +02:00