feat: Complete overhaul to enhanced documentation-only MCP server
- Removed all workflow execution capabilities per user requirements - Implemented enhanced documentation extraction with operations and API mappings - Fixed credential code extraction for all nodes - Fixed package info extraction (name and version) - Enhanced operations parser to handle n8n markdown format - Fixed documentation search to prioritize app nodes over trigger nodes - Comprehensive test coverage for Slack node extraction - All node information now includes: - Complete operations list (42 for Slack) - API method mappings with documentation URLs - Source code and credential definitions - Package metadata - Related resources and templates 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
czlonkowski
155
README-ENHANCED.md
Normal file
155
README-ENHANCED.md
Normal file
@@ -0,0 +1,155 @@
|
||||
# n8n-MCP Enhanced Documentation System
|
||||
|
||||
This is the enhanced n8n-MCP integration that provides comprehensive node documentation, including operations, API methods, examples, and rich metadata through the Model Context Protocol (MCP).
|
||||
|
||||
## Overview
|
||||
|
||||
The enhanced system provides:
|
||||
|
||||
- **Rich Node Documentation**: Complete documentation including markdown content, operations, API methods, and code examples
|
||||
- **Full-Text Search**: SQLite FTS5-powered search across node names, descriptions, and documentation
|
||||
- **Comprehensive Node Information**: Source code, credentials, examples, templates, and metadata in a single query
|
||||
- **Automatic Documentation Extraction**: Fetches and parses documentation from the official n8n-docs repository
|
||||
|
||||
## Available MCP Tools
|
||||
|
||||
### 1. `get_node_info`
|
||||
Get comprehensive information about a specific n8n node.
|
||||
|
||||
**Parameters:**
|
||||
- `nodeType` (string, required): The node type identifier (e.g., 'n8n-nodes-base.slack')
|
||||
|
||||
**Returns:**
|
||||
- Complete node information including:
|
||||
- Basic metadata (name, displayName, description, category)
|
||||
- Documentation (markdown, URL, title)
|
||||
- Operations and API methods
|
||||
- Code examples and templates
|
||||
- Related resources and required scopes
|
||||
- Source code (node and credential)
|
||||
- Example workflow and parameters
|
||||
|
||||
### 2. `search_nodes`
|
||||
Search n8n nodes with full-text search and advanced filtering.
|
||||
|
||||
**Parameters:**
|
||||
- `query` (string, optional): Search query for full-text search
|
||||
- `category` (string, optional): Filter by node category
|
||||
- `packageName` (string, optional): Filter by package name
|
||||
- `hasCredentials` (boolean, optional): Filter nodes that require credentials
|
||||
- `isTrigger` (boolean, optional): Filter trigger nodes only
|
||||
- `limit` (number, optional): Maximum results to return (default: 20)
|
||||
|
||||
**Returns:**
|
||||
- Array of matching nodes with summary information
|
||||
|
||||
### 3. `get_node_statistics`
|
||||
Get statistics about the node documentation database.
|
||||
|
||||
**Returns:**
|
||||
- Total nodes, packages, and storage statistics
|
||||
- Nodes with documentation, examples, and credentials
|
||||
- Package distribution
|
||||
|
||||
### 4. `rebuild_documentation_database`
|
||||
Rebuild the node documentation database with the latest information.
|
||||
|
||||
**Parameters:**
|
||||
- `packageFilter` (string, optional): Only rebuild nodes from specific package
|
||||
|
||||
**Returns:**
|
||||
- Rebuild statistics and status
|
||||
|
||||
## Database Schema
|
||||
|
||||
The system uses a SQLite database with the following main table:
|
||||
|
||||
```sql
|
||||
CREATE TABLE nodes (
|
||||
node_type TEXT UNIQUE NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
display_name TEXT,
|
||||
description TEXT,
|
||||
category TEXT,
|
||||
source_code TEXT NOT NULL,
|
||||
documentation_markdown TEXT,
|
||||
operations TEXT, -- JSON array of OperationInfo
|
||||
api_methods TEXT, -- JSON array of ApiMethodMapping
|
||||
documentation_examples TEXT, -- JSON array of CodeExample
|
||||
templates TEXT, -- JSON array of TemplateInfo
|
||||
related_resources TEXT, -- JSON array of RelatedResource
|
||||
-- ... additional fields
|
||||
);
|
||||
```
|
||||
|
||||
## Building the Documentation Database
|
||||
|
||||
To build or rebuild the documentation database:
|
||||
|
||||
```bash
|
||||
# Using npm script
|
||||
npm run docs:rebuild
|
||||
|
||||
# Or directly
|
||||
npx ts-node src/scripts/rebuild-database.ts
|
||||
```
|
||||
|
||||
This will:
|
||||
1. Clone/update the n8n-docs repository
|
||||
2. Extract source code for all available nodes
|
||||
3. Parse and extract enhanced documentation
|
||||
4. Generate example workflows
|
||||
5. Store everything in the SQLite database
|
||||
|
||||
## Usage Example
|
||||
|
||||
```typescript
|
||||
// Get comprehensive information about the Slack node
|
||||
const slackInfo = await mcpClient.callTool('get_node_info', {
|
||||
nodeType: 'n8n-nodes-base.slack'
|
||||
});
|
||||
|
||||
// Search for all trigger nodes with credentials
|
||||
const triggers = await mcpClient.callTool('search_nodes', {
|
||||
isTrigger: true,
|
||||
hasCredentials: true
|
||||
});
|
||||
|
||||
// Get database statistics
|
||||
const stats = await mcpClient.callTool('get_node_statistics', {});
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
The enhanced system consists of:
|
||||
|
||||
1. **NodeDocumentationService**: Main service that manages the SQLite database
|
||||
2. **EnhancedDocumentationFetcher**: Fetches and parses documentation from n8n-docs
|
||||
3. **ExampleGenerator**: Generates example workflows and parameters
|
||||
4. **MCP Server**: Exposes the tools through the Model Context Protocol
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
npm install
|
||||
|
||||
# Build TypeScript
|
||||
npm run build
|
||||
|
||||
# Run tests
|
||||
npm test
|
||||
|
||||
# Start MCP server
|
||||
npm start
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
- `NODE_DB_PATH`: Path to the SQLite database (default: `./data/nodes.db`)
|
||||
- `N8N_API_URL`: n8n instance URL
|
||||
- `N8N_API_KEY`: n8n API key for workflow operations
|
||||
|
||||
## License
|
||||
|
||||
Licensed under the Sustainable Use License v1.0
|
||||
285
README-v2.md
285
README-v2.md
@@ -1,285 +0,0 @@
|
||||
# n8n Node Documentation MCP Server
|
||||
|
||||
An MCP (Model Context Protocol) server that provides n8n node documentation, source code, and usage examples to AI assistants.
|
||||
|
||||
## Purpose
|
||||
|
||||
This MCP server serves as a knowledge base for AI assistants (like Claude) to understand and work with n8n nodes. It provides:
|
||||
|
||||
- **Complete node source code** - The actual implementation of each n8n node
|
||||
- **Official documentation** - Markdown documentation from the n8n-docs repository
|
||||
- **Usage examples** - Sample workflow JSON showing how to use each node
|
||||
- **Search capabilities** - Full-text search across node names, descriptions, and documentation
|
||||
|
||||
## Features
|
||||
|
||||
- 🔍 **Full-text search** - Search nodes by name, description, or documentation content
|
||||
- 📚 **Complete documentation** - Fetches and indexes official n8n documentation
|
||||
- 💻 **Source code access** - Provides full source code for each node
|
||||
- 🎯 **Usage examples** - Generates example workflows for each node type
|
||||
- 🔄 **Auto-rebuild** - Rebuilds the entire database on startup or on demand
|
||||
- ⚡ **Fast SQLite storage** - All data stored locally for instant access
|
||||
|
||||
## Installation
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Node.js 18+
|
||||
- n8n instance (for node extraction)
|
||||
- Git (for cloning n8n-docs)
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/yourusername/n8n-mcp.git
|
||||
cd n8n-mcp
|
||||
|
||||
# Install dependencies
|
||||
npm install
|
||||
|
||||
# Build the project
|
||||
npm run build
|
||||
|
||||
# Rebuild the database with all nodes
|
||||
npm run db:rebuild
|
||||
```
|
||||
|
||||
## Usage with Claude Desktop
|
||||
|
||||
### 1. Configure Claude Desktop
|
||||
|
||||
Add to your Claude Desktop configuration file:
|
||||
|
||||
**macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json`
|
||||
**Windows**: `%APPDATA%\Claude\claude_desktop_config.json`
|
||||
**Linux**: `~/.config/Claude/claude_desktop_config.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"n8n-nodes": {
|
||||
"command": "node",
|
||||
"args": ["/absolute/path/to/n8n-mcp/dist/index-v2.js"],
|
||||
"env": {
|
||||
"NODE_DB_PATH": "/absolute/path/to/n8n-mcp/data/nodes.db",
|
||||
"REBUILD_ON_START": "false"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Available MCP Tools
|
||||
|
||||
Once configured, you can ask Claude to:
|
||||
|
||||
- **List all n8n nodes**: "Show me all available n8n nodes"
|
||||
- **Get node information**: "Show me the IF node documentation and code"
|
||||
- **Search for nodes**: "Find all webhook-related nodes"
|
||||
- **Get examples**: "Show me an example of using the HTTP Request node"
|
||||
|
||||
### MCP Tools Reference
|
||||
|
||||
#### `list_nodes`
|
||||
Lists all available n8n nodes with basic information.
|
||||
|
||||
```
|
||||
Parameters:
|
||||
- category (optional): Filter by category
|
||||
- packageName (optional): Filter by package
|
||||
- isTrigger (optional): Show only trigger nodes
|
||||
```
|
||||
|
||||
#### `get_node_info`
|
||||
Gets complete information about a specific node including source code, documentation, and examples.
|
||||
|
||||
```
|
||||
Parameters:
|
||||
- nodeType (required): The node type (e.g., "n8n-nodes-base.if", "If", "webhook")
|
||||
```
|
||||
|
||||
#### `search_nodes`
|
||||
Searches for nodes by name, description, or documentation content.
|
||||
|
||||
```
|
||||
Parameters:
|
||||
- query (required): Search query
|
||||
- category (optional): Filter by category
|
||||
- hasDocumentation (optional): Only show nodes with docs
|
||||
- limit (optional): Max results (default: 20)
|
||||
```
|
||||
|
||||
#### `get_node_example`
|
||||
Gets example workflow JSON for a specific node.
|
||||
|
||||
```
|
||||
Parameters:
|
||||
- nodeType (required): The node type
|
||||
```
|
||||
|
||||
#### `get_node_source_code`
|
||||
Gets only the source code of a node.
|
||||
|
||||
```
|
||||
Parameters:
|
||||
- nodeType (required): The node type
|
||||
- includeCredentials (optional): Include credential definitions
|
||||
```
|
||||
|
||||
#### `get_node_documentation`
|
||||
Gets only the documentation for a node.
|
||||
|
||||
```
|
||||
Parameters:
|
||||
- nodeType (required): The node type
|
||||
- format (optional): "markdown" or "plain" (default: markdown)
|
||||
```
|
||||
|
||||
## Database Management
|
||||
|
||||
### Initial Setup
|
||||
|
||||
```bash
|
||||
# Build and populate the database
|
||||
npm run db:rebuild
|
||||
```
|
||||
|
||||
### Database Structure
|
||||
|
||||
The SQLite database stores:
|
||||
- Node source code
|
||||
- Official documentation from n8n-docs
|
||||
- Generated usage examples
|
||||
- Node metadata (category, triggers, webhooks, etc.)
|
||||
|
||||
### Rebuild Process
|
||||
|
||||
The rebuild process:
|
||||
1. Clears the existing database
|
||||
2. Fetches latest n8n-docs repository
|
||||
3. Extracts source code from all n8n nodes
|
||||
4. Fetches documentation for each node
|
||||
5. Generates usage examples
|
||||
6. Stores everything in SQLite with full-text search
|
||||
|
||||
## Example Responses
|
||||
|
||||
### IF Node Example
|
||||
|
||||
When asking for the IF node, the server returns:
|
||||
|
||||
```json
|
||||
{
|
||||
"nodeType": "n8n-nodes-base.if",
|
||||
"name": "If",
|
||||
"displayName": "If",
|
||||
"description": "Route items based on comparison operations",
|
||||
"sourceCode": "// Full TypeScript source code...",
|
||||
"documentation": "# If Node\n\nThe If node splits a workflow...",
|
||||
"exampleWorkflow": {
|
||||
"nodes": [{
|
||||
"parameters": {
|
||||
"conditions": {
|
||||
"options": {
|
||||
"caseSensitive": true,
|
||||
"leftValue": "",
|
||||
"typeValidation": "strict",
|
||||
"version": 2
|
||||
},
|
||||
"conditions": [{
|
||||
"id": "871274c8-dabf-465a-a6cf-655a1786aa55",
|
||||
"leftValue": "={{ $json }}",
|
||||
"rightValue": "",
|
||||
"operator": {
|
||||
"type": "object",
|
||||
"operation": "notEmpty",
|
||||
"singleValue": true
|
||||
}
|
||||
}],
|
||||
"combinator": "and"
|
||||
},
|
||||
"options": {}
|
||||
},
|
||||
"type": "n8n-nodes-base.if",
|
||||
"typeVersion": 2.2,
|
||||
"position": [220, 120],
|
||||
"id": "64b5d49f-ac2e-4456-bfa9-2d6eb9c7a624",
|
||||
"name": "If"
|
||||
}],
|
||||
"connections": {
|
||||
"If": {
|
||||
"main": [[], []]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Running in Development Mode
|
||||
|
||||
```bash
|
||||
# Start with auto-reload
|
||||
npm run dev
|
||||
|
||||
# Run tests
|
||||
npm test
|
||||
|
||||
# Type checking
|
||||
npm run typecheck
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```env
|
||||
# Database location
|
||||
NODE_DB_PATH=/path/to/nodes.db
|
||||
|
||||
# Rebuild database on server start
|
||||
REBUILD_ON_START=true
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL=debug
|
||||
|
||||
# Documentation repository location (optional)
|
||||
DOCS_REPO_PATH=/path/to/n8n-docs
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
n8n-mcp/
|
||||
├── src/
|
||||
│ ├── mcp/
|
||||
│ │ ├── server-v2.ts # MCP server implementation
|
||||
│ │ └── tools-v2.ts # MCP tool definitions
|
||||
│ ├── services/
|
||||
│ │ └── node-documentation-service.ts # Database service
|
||||
│ ├── utils/
|
||||
│ │ ├── documentation-fetcher.ts # n8n-docs fetcher
|
||||
│ │ ├── example-generator.ts # Example generator
|
||||
│ │ └── node-source-extractor.ts # Source extractor
|
||||
│ └── scripts/
|
||||
│ └── rebuild-database-v2.ts # Database rebuild
|
||||
└── data/
|
||||
└── nodes.db # SQLite database
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Database not found
|
||||
```bash
|
||||
npm run db:rebuild
|
||||
```
|
||||
|
||||
### No documentation for some nodes
|
||||
Some nodes may not have documentation in the n8n-docs repository. The server will still provide source code and generated examples.
|
||||
|
||||
### Rebuild takes too long
|
||||
The initial rebuild processes 500+ nodes and fetches documentation. Subsequent starts use the cached database unless `REBUILD_ON_START=true`.
|
||||
|
||||
## License
|
||||
|
||||
ISC
|
||||
BIN
data/demo-enhanced.db
Normal file
BIN
data/demo-enhanced.db
Normal file
Binary file not shown.
BIN
data/test-enhanced-docs.db
Normal file
BIN
data/test-enhanced-docs.db
Normal file
Binary file not shown.
BIN
data/test-enhanced.db
Normal file
BIN
data/test-enhanced.db
Normal file
Binary file not shown.
BIN
data/test-nodes-v2.db
Normal file
BIN
data/test-nodes-v2.db
Normal file
Binary file not shown.
BIN
data/test-slack-fix.db
Normal file
BIN
data/test-slack-fix.db
Normal file
Binary file not shown.
BIN
data/test-slack.db
Normal file
BIN
data/test-slack.db
Normal file
Binary file not shown.
133
docs/ENHANCED_DOCUMENTATION_PARSER.md
Normal file
133
docs/ENHANCED_DOCUMENTATION_PARSER.md
Normal file
@@ -0,0 +1,133 @@
|
||||
# Enhanced Documentation Parser for n8n-MCP
|
||||
|
||||
## Overview
|
||||
|
||||
We have successfully enhanced the markdown parser in DocumentationFetcher to extract rich, structured content from n8n documentation. This enhancement enables AI agents to have deeper understanding of n8n nodes, their operations, API mappings, and usage patterns.
|
||||
|
||||
## Key Features Implemented
|
||||
|
||||
### 1. Enhanced Documentation Structure
|
||||
|
||||
The `EnhancedDocumentationFetcher` class extracts and structures documentation into:
|
||||
|
||||
```typescript
|
||||
interface EnhancedNodeDocumentation {
|
||||
markdown: string; // Raw markdown content
|
||||
url: string; // Documentation URL
|
||||
title?: string; // Node title
|
||||
description?: string; // Node description
|
||||
operations?: OperationInfo[]; // Structured operations
|
||||
apiMethods?: ApiMethodMapping[]; // API endpoint mappings
|
||||
examples?: CodeExample[]; // Code examples
|
||||
templates?: TemplateInfo[]; // Template references
|
||||
relatedResources?: RelatedResource[]; // Related docs
|
||||
requiredScopes?: string[]; // OAuth scopes
|
||||
metadata?: DocumentationMetadata; // Frontmatter data
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Operations Extraction
|
||||
|
||||
The parser correctly identifies and extracts hierarchical operations:
|
||||
|
||||
- **Resource Level**: e.g., "Channel", "Message", "User"
|
||||
- **Operation Level**: e.g., "Archive", "Send", "Get"
|
||||
- **Descriptions**: Detailed operation descriptions
|
||||
|
||||
Example from Slack node:
|
||||
- Channel.Archive: "a channel"
|
||||
- Message.Send: "a message"
|
||||
- User.Get: "information about a user"
|
||||
|
||||
### 3. API Method Mapping
|
||||
|
||||
Extracts mappings between n8n operations and actual API endpoints from markdown tables:
|
||||
|
||||
```
|
||||
Channel.Archive → conversations.archive (https://api.slack.com/methods/conversations.archive)
|
||||
Message.Send → chat.postMessage (https://api.slack.com/methods/chat.postMessage)
|
||||
```
|
||||
|
||||
### 4. Enhanced Database Schema
|
||||
|
||||
Created a new schema to store the rich documentation:
|
||||
|
||||
- `nodes` table: Extended with documentation fields
|
||||
- `node_operations`: Stores all operations for each node
|
||||
- `node_api_methods`: Maps operations to API endpoints
|
||||
- `node_examples`: Stores code examples
|
||||
- `node_resources`: Related documentation links
|
||||
- `node_scopes`: Required OAuth scopes
|
||||
|
||||
### 5. Full-Text Search Enhancement
|
||||
|
||||
The FTS index now includes:
|
||||
- Documentation title and description
|
||||
- Operations and their descriptions
|
||||
- API method names
|
||||
- Full markdown content
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```javascript
|
||||
const fetcher = new EnhancedDocumentationFetcher();
|
||||
const doc = await fetcher.getEnhancedNodeDocumentation('n8n-nodes-base.slack');
|
||||
|
||||
// Access structured data
|
||||
console.log(`Operations: ${doc.operations.length}`);
|
||||
console.log(`API Methods: ${doc.apiMethods.length}`);
|
||||
```
|
||||
|
||||
### With Database Storage
|
||||
|
||||
```javascript
|
||||
const storage = new EnhancedSQLiteStorageService();
|
||||
const nodeInfo = await extractor.extractNodeSource('n8n-nodes-base.slack');
|
||||
const storedNode = await storage.storeNodeWithDocumentation(nodeInfo);
|
||||
|
||||
// Access counts
|
||||
console.log(`Stored ${storedNode.operationCount} operations`);
|
||||
console.log(`Stored ${storedNode.apiMethodCount} API methods`);
|
||||
```
|
||||
|
||||
## Benefits for AI Agents
|
||||
|
||||
1. **Comprehensive Understanding**: AI agents can now understand not just what a node does, but exactly which operations are available and how they map to API endpoints.
|
||||
|
||||
2. **Better Search**: Enhanced FTS allows searching across operations, descriptions, and documentation content.
|
||||
|
||||
3. **Structured Data**: Operations and API methods are stored as structured data, making it easier for AI to reason about node capabilities.
|
||||
|
||||
4. **Rich Context**: Related resources, examples, and metadata provide additional context for better AI responses.
|
||||
|
||||
## Implementation Files
|
||||
|
||||
- `/src/utils/enhanced-documentation-fetcher.ts`: Main parser implementation
|
||||
- `/src/services/enhanced-sqlite-storage-service.ts`: Database storage with rich schema
|
||||
- `/src/db/enhanced-schema.sql`: Enhanced database schema
|
||||
- `/tests/demo-enhanced-documentation.js`: Working demonstration
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
1. **Example Extraction**: Improve code example extraction from documentation
|
||||
2. **Parameter Parsing**: Extract operation parameters and their types
|
||||
3. **Credential Requirements**: Parse specific credential field requirements
|
||||
4. **Version Tracking**: Track documentation versions and changes
|
||||
5. **Caching**: Implement smart caching for documentation fetches
|
||||
|
||||
## Testing
|
||||
|
||||
Run the demo to see the enhanced parser in action:
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
node tests/demo-enhanced-documentation.js
|
||||
```
|
||||
|
||||
This will show:
|
||||
- Extraction of 40+ operations from Slack node
|
||||
- API method mappings with URLs
|
||||
- Resource grouping and organization
|
||||
- Related documentation links
|
||||
81
docs/SLACK_DOCUMENTATION_FIX.md
Normal file
81
docs/SLACK_DOCUMENTATION_FIX.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# Slack Documentation Fix Summary
|
||||
|
||||
## Issues Fixed
|
||||
|
||||
### 1. Documentation Fetcher Was Getting Wrong Files
|
||||
**Problem**: When searching for Slack node documentation, the fetcher was finding credential documentation instead of node documentation.
|
||||
|
||||
**Root Cause**:
|
||||
- Documentation files in n8n-docs repository are named with full node type (e.g., `n8n-nodes-base.slack.md`)
|
||||
- The fetcher was searching for just the node name (e.g., `slack.md`)
|
||||
- This caused it to find `slack.md` in the credentials folder first
|
||||
|
||||
**Fix Applied**:
|
||||
- Updated `getNodeDocumentation()` to search for full node type first
|
||||
- Added logic to skip credential documentation files by checking:
|
||||
- If file path includes `/credentials/`
|
||||
- If content has "credentials" in title without "node documentation"
|
||||
- Fixed search order to prioritize correct documentation
|
||||
|
||||
### 2. Node Source Extractor Case Sensitivity
|
||||
**Problem**: Slack node source code wasn't found because the directory is capitalized (`Slack/`) but search was case-sensitive.
|
||||
|
||||
**Root Cause**:
|
||||
- n8n node directories use capitalized names (e.g., `Slack/`, `If/`)
|
||||
- Extractor was searching with lowercase names from node type
|
||||
|
||||
**Fix Applied**:
|
||||
- Added case variants to try when searching:
|
||||
- Original case
|
||||
- Capitalized first letter
|
||||
- All lowercase
|
||||
- All uppercase
|
||||
- Now properly finds nodes regardless of directory naming convention
|
||||
|
||||
### 3. Missing Information in Database
|
||||
**Problem**: Node definitions weren't being properly parsed from compiled JavaScript.
|
||||
|
||||
**Fix Applied**:
|
||||
- Improved `parseNodeDefinition()` to extract individual fields using regex
|
||||
- Added extraction for:
|
||||
- displayName
|
||||
- description
|
||||
- icon
|
||||
- category/group
|
||||
- version
|
||||
- trigger/webhook detection
|
||||
|
||||
## Test Results
|
||||
|
||||
After applying fixes:
|
||||
- ✅ Slack node source code is correctly extracted
|
||||
- ✅ Slack node documentation (not credentials) is fetched
|
||||
- ✅ Documentation URL points to correct page
|
||||
- ✅ All information is properly stored in database
|
||||
|
||||
## Files Modified
|
||||
|
||||
1. `/src/utils/documentation-fetcher.ts`
|
||||
- Fixed path searching logic
|
||||
- Added credential documentation filtering
|
||||
- Improved search order
|
||||
|
||||
2. `/src/utils/node-source-extractor.ts`
|
||||
- Added case-insensitive directory searching
|
||||
- Improved path detection for different node structures
|
||||
|
||||
3. `/src/services/node-documentation-service.ts`
|
||||
- Enhanced node definition parsing
|
||||
- Better extraction of metadata from source code
|
||||
|
||||
## Verification
|
||||
|
||||
Run the test to verify the fix:
|
||||
```bash
|
||||
node tests/test-slack-fix.js
|
||||
```
|
||||
|
||||
This should show:
|
||||
- Source code found at correct location
|
||||
- Documentation is node documentation (not credentials)
|
||||
- All fields properly extracted and stored
|
||||
107
examples/enhanced-documentation-demo.js
Normal file
107
examples/enhanced-documentation-demo.js
Normal file
@@ -0,0 +1,107 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { DocumentationFetcher } = require('../dist/utils/documentation-fetcher');
|
||||
|
||||
async function demonstrateEnhancedDocumentation() {
|
||||
console.log('🎯 Enhanced Documentation Demo\n');
|
||||
|
||||
const fetcher = new DocumentationFetcher();
|
||||
const nodeType = 'n8n-nodes-base.slack';
|
||||
|
||||
console.log(`Fetching enhanced documentation for: ${nodeType}\n`);
|
||||
|
||||
try {
|
||||
const doc = await fetcher.getEnhancedNodeDocumentation(nodeType);
|
||||
|
||||
if (!doc) {
|
||||
console.log('No documentation found for this node.');
|
||||
return;
|
||||
}
|
||||
|
||||
// Display title and description
|
||||
console.log('📄 Basic Information:');
|
||||
console.log(`Title: ${doc.title || 'N/A'}`);
|
||||
console.log(`URL: ${doc.url}`);
|
||||
console.log(`Description: ${doc.description || 'See documentation for details'}\n`);
|
||||
|
||||
// Display operations
|
||||
if (doc.operations && doc.operations.length > 0) {
|
||||
console.log('⚙️ Available Operations:');
|
||||
// Group by resource
|
||||
const resourceMap = new Map();
|
||||
doc.operations.forEach(op => {
|
||||
if (!resourceMap.has(op.resource)) {
|
||||
resourceMap.set(op.resource, []);
|
||||
}
|
||||
resourceMap.get(op.resource).push(op);
|
||||
});
|
||||
|
||||
resourceMap.forEach((ops, resource) => {
|
||||
console.log(`\n ${resource}:`);
|
||||
ops.forEach(op => {
|
||||
console.log(` - ${op.operation}: ${op.description}`);
|
||||
});
|
||||
});
|
||||
console.log('');
|
||||
}
|
||||
|
||||
// Display API methods
|
||||
if (doc.apiMethods && doc.apiMethods.length > 0) {
|
||||
console.log('🔌 API Method Mappings (first 5):');
|
||||
doc.apiMethods.slice(0, 5).forEach(method => {
|
||||
console.log(` ${method.resource}.${method.operation} → ${method.apiMethod}`);
|
||||
if (method.apiUrl) {
|
||||
console.log(` Documentation: ${method.apiUrl}`);
|
||||
}
|
||||
});
|
||||
console.log(` ... and ${Math.max(0, doc.apiMethods.length - 5)} more\n`);
|
||||
}
|
||||
|
||||
// Display templates
|
||||
if (doc.templates && doc.templates.length > 0) {
|
||||
console.log('📋 Available Templates:');
|
||||
doc.templates.forEach(template => {
|
||||
console.log(` - ${template.name}`);
|
||||
if (template.description) {
|
||||
console.log(` ${template.description}`);
|
||||
}
|
||||
});
|
||||
console.log('');
|
||||
}
|
||||
|
||||
// Display related resources
|
||||
if (doc.relatedResources && doc.relatedResources.length > 0) {
|
||||
console.log('🔗 Related Resources:');
|
||||
doc.relatedResources.forEach(resource => {
|
||||
console.log(` - ${resource.title} (${resource.type})`);
|
||||
console.log(` ${resource.url}`);
|
||||
});
|
||||
console.log('');
|
||||
}
|
||||
|
||||
// Display required scopes
|
||||
if (doc.requiredScopes && doc.requiredScopes.length > 0) {
|
||||
console.log('🔐 Required Scopes:');
|
||||
doc.requiredScopes.forEach(scope => {
|
||||
console.log(` - ${scope}`);
|
||||
});
|
||||
console.log('');
|
||||
}
|
||||
|
||||
// Display summary
|
||||
console.log('📊 Summary:');
|
||||
console.log(` - Total operations: ${doc.operations?.length || 0}`);
|
||||
console.log(` - Total API methods: ${doc.apiMethods?.length || 0}`);
|
||||
console.log(` - Code examples: ${doc.examples?.length || 0}`);
|
||||
console.log(` - Templates: ${doc.templates?.length || 0}`);
|
||||
console.log(` - Related resources: ${doc.relatedResources?.length || 0}`);
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error:', error.message);
|
||||
} finally {
|
||||
await fetcher.cleanup();
|
||||
}
|
||||
}
|
||||
|
||||
// Run demo
|
||||
demonstrateEnhancedDocumentation().catch(console.error);
|
||||
@@ -6,18 +6,13 @@
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"dev": "nodemon --exec ts-node src/index.ts",
|
||||
"dev:v2": "nodemon --exec ts-node src/index-v2.ts",
|
||||
"dev:http": "nodemon --exec ts-node src/index-http.ts",
|
||||
"start": "node dist/index.js",
|
||||
"start:v2": "node dist/index-v2.js",
|
||||
"start:http": "node dist/index-http.js",
|
||||
"test": "jest",
|
||||
"lint": "tsc --noEmit",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"db:rebuild": "node dist/scripts/rebuild-database.js",
|
||||
"db:rebuild:v2": "node dist/scripts/rebuild-database-v2.js",
|
||||
"db:init": "node -e \"new (require('./dist/services/sqlite-storage-service').SQLiteStorageService)(); console.log('Database initialized')\"",
|
||||
"test:v2": "node tests/test-node-documentation-service.js"
|
||||
"docs:rebuild": "ts-node src/scripts/rebuild-database.ts"
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
|
||||
@@ -1,99 +0,0 @@
|
||||
-- Main nodes table with documentation and examples
|
||||
CREATE TABLE IF NOT EXISTS nodes (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
node_type TEXT UNIQUE NOT NULL, -- e.g., "n8n-nodes-base.if"
|
||||
name TEXT NOT NULL, -- e.g., "If"
|
||||
display_name TEXT, -- e.g., "If"
|
||||
description TEXT, -- Brief description from node definition
|
||||
category TEXT, -- e.g., "Core Nodes", "Flow"
|
||||
subcategory TEXT, -- More specific categorization
|
||||
icon TEXT, -- Icon identifier/path
|
||||
|
||||
-- Source code
|
||||
source_code TEXT NOT NULL, -- Full node source code
|
||||
credential_code TEXT, -- Credential type definitions
|
||||
code_hash TEXT NOT NULL, -- Hash for change detection
|
||||
code_length INTEGER NOT NULL, -- Source code size
|
||||
|
||||
-- Documentation
|
||||
documentation_markdown TEXT, -- Full markdown documentation from n8n-docs
|
||||
documentation_url TEXT, -- URL to documentation page
|
||||
|
||||
-- Example usage
|
||||
example_workflow TEXT, -- JSON example workflow using this node
|
||||
example_parameters TEXT, -- JSON example of node parameters
|
||||
properties_schema TEXT, -- JSON schema of node properties
|
||||
|
||||
-- Metadata
|
||||
package_name TEXT NOT NULL, -- e.g., "n8n-nodes-base"
|
||||
version TEXT, -- Node version
|
||||
codex_data TEXT, -- Additional codex/metadata JSON
|
||||
aliases TEXT, -- JSON array of alternative names
|
||||
|
||||
-- Flags
|
||||
has_credentials INTEGER DEFAULT 0,
|
||||
is_trigger INTEGER DEFAULT 0, -- Whether it's a trigger node
|
||||
is_webhook INTEGER DEFAULT 0, -- Whether it's a webhook node
|
||||
|
||||
-- Timestamps
|
||||
extracted_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Indexes for performance
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_package_name ON nodes(package_name);
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_category ON nodes(category);
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_code_hash ON nodes(code_hash);
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_name ON nodes(name);
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_is_trigger ON nodes(is_trigger);
|
||||
|
||||
-- Full Text Search virtual table for comprehensive search
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS nodes_fts USING fts5(
|
||||
node_type,
|
||||
name,
|
||||
display_name,
|
||||
description,
|
||||
category,
|
||||
documentation_markdown,
|
||||
aliases,
|
||||
content=nodes,
|
||||
content_rowid=id
|
||||
);
|
||||
|
||||
-- Triggers to keep FTS in sync
|
||||
CREATE TRIGGER IF NOT EXISTS nodes_ai AFTER INSERT ON nodes
|
||||
BEGIN
|
||||
INSERT INTO nodes_fts(rowid, node_type, name, display_name, description, category, documentation_markdown, aliases)
|
||||
VALUES (new.id, new.node_type, new.name, new.display_name, new.description, new.category, new.documentation_markdown, new.aliases);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS nodes_ad AFTER DELETE ON nodes
|
||||
BEGIN
|
||||
DELETE FROM nodes_fts WHERE rowid = old.id;
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS nodes_au AFTER UPDATE ON nodes
|
||||
BEGIN
|
||||
DELETE FROM nodes_fts WHERE rowid = old.id;
|
||||
INSERT INTO nodes_fts(rowid, node_type, name, display_name, description, category, documentation_markdown, aliases)
|
||||
VALUES (new.id, new.node_type, new.name, new.display_name, new.description, new.category, new.documentation_markdown, new.aliases);
|
||||
END;
|
||||
|
||||
-- Table for storing node documentation versions
|
||||
CREATE TABLE IF NOT EXISTS documentation_sources (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
source TEXT NOT NULL, -- 'n8n-docs-repo', 'inline', 'generated'
|
||||
commit_hash TEXT, -- Git commit hash if from repo
|
||||
fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Statistics table
|
||||
CREATE TABLE IF NOT EXISTS extraction_stats (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
total_nodes INTEGER NOT NULL,
|
||||
nodes_with_docs INTEGER NOT NULL,
|
||||
nodes_with_examples INTEGER NOT NULL,
|
||||
total_code_size INTEGER NOT NULL,
|
||||
total_docs_size INTEGER NOT NULL,
|
||||
extraction_date DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
@@ -1,34 +1,75 @@
|
||||
-- Main nodes table
|
||||
-- Enhanced n8n Node Documentation Database Schema
|
||||
-- This schema stores comprehensive node information including source code,
|
||||
-- documentation, operations, API methods, examples, and metadata
|
||||
|
||||
-- Main nodes table with rich documentation
|
||||
CREATE TABLE IF NOT EXISTS nodes (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
node_type TEXT UNIQUE NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
package_name TEXT NOT NULL,
|
||||
display_name TEXT,
|
||||
description TEXT,
|
||||
code_hash TEXT NOT NULL,
|
||||
code_length INTEGER NOT NULL,
|
||||
source_location TEXT NOT NULL,
|
||||
category TEXT,
|
||||
subcategory TEXT,
|
||||
icon TEXT,
|
||||
|
||||
-- Source code
|
||||
source_code TEXT NOT NULL,
|
||||
credential_code TEXT,
|
||||
package_info TEXT, -- JSON
|
||||
code_hash TEXT NOT NULL,
|
||||
code_length INTEGER NOT NULL,
|
||||
|
||||
-- Documentation
|
||||
documentation_markdown TEXT,
|
||||
documentation_url TEXT,
|
||||
documentation_title TEXT,
|
||||
|
||||
-- Enhanced documentation fields (stored as JSON)
|
||||
operations TEXT,
|
||||
api_methods TEXT,
|
||||
documentation_examples TEXT,
|
||||
templates TEXT,
|
||||
related_resources TEXT,
|
||||
required_scopes TEXT,
|
||||
|
||||
-- Example usage
|
||||
example_workflow TEXT,
|
||||
example_parameters TEXT,
|
||||
properties_schema TEXT,
|
||||
|
||||
-- Metadata
|
||||
package_name TEXT NOT NULL,
|
||||
version TEXT,
|
||||
codex_data TEXT,
|
||||
aliases TEXT,
|
||||
|
||||
-- Flags
|
||||
has_credentials INTEGER DEFAULT 0,
|
||||
is_trigger INTEGER DEFAULT 0,
|
||||
is_webhook INTEGER DEFAULT 0,
|
||||
|
||||
-- Timestamps
|
||||
extracted_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Indexes for performance
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_package_name ON nodes(package_name);
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_category ON nodes(category);
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_code_hash ON nodes(code_hash);
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_name ON nodes(name);
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_is_trigger ON nodes(is_trigger);
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_has_credentials ON nodes(has_credentials);
|
||||
|
||||
-- Full Text Search virtual table for node search
|
||||
-- Full Text Search table
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS nodes_fts USING fts5(
|
||||
node_type,
|
||||
name,
|
||||
display_name,
|
||||
description,
|
||||
package_name,
|
||||
category,
|
||||
documentation_markdown,
|
||||
aliases,
|
||||
content=nodes,
|
||||
content_rowid=id
|
||||
);
|
||||
@@ -36,8 +77,8 @@ CREATE VIRTUAL TABLE IF NOT EXISTS nodes_fts USING fts5(
|
||||
-- Triggers to keep FTS in sync
|
||||
CREATE TRIGGER IF NOT EXISTS nodes_ai AFTER INSERT ON nodes
|
||||
BEGIN
|
||||
INSERT INTO nodes_fts(rowid, node_type, name, display_name, description, package_name)
|
||||
VALUES (new.id, new.node_type, new.name, new.display_name, new.description, new.package_name);
|
||||
INSERT INTO nodes_fts(rowid, node_type, name, display_name, description, category, documentation_markdown, aliases)
|
||||
VALUES (new.id, new.node_type, new.name, new.display_name, new.description, new.category, new.documentation_markdown, new.aliases);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS nodes_ad AFTER DELETE ON nodes
|
||||
@@ -48,16 +89,55 @@ END;
|
||||
CREATE TRIGGER IF NOT EXISTS nodes_au AFTER UPDATE ON nodes
|
||||
BEGIN
|
||||
DELETE FROM nodes_fts WHERE rowid = old.id;
|
||||
INSERT INTO nodes_fts(rowid, node_type, name, display_name, description, package_name)
|
||||
VALUES (new.id, new.node_type, new.name, new.display_name, new.description, new.package_name);
|
||||
INSERT INTO nodes_fts(rowid, node_type, name, display_name, description, category, documentation_markdown, aliases)
|
||||
VALUES (new.id, new.node_type, new.name, new.display_name, new.description, new.category, new.documentation_markdown, new.aliases);
|
||||
END;
|
||||
|
||||
-- Statistics table for metadata
|
||||
-- Documentation sources tracking
|
||||
CREATE TABLE IF NOT EXISTS documentation_sources (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
source TEXT NOT NULL,
|
||||
commit_hash TEXT,
|
||||
fetched_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Statistics tracking
|
||||
CREATE TABLE IF NOT EXISTS extraction_stats (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
total_nodes INTEGER NOT NULL,
|
||||
total_packages INTEGER NOT NULL,
|
||||
nodes_with_docs INTEGER NOT NULL,
|
||||
nodes_with_examples INTEGER NOT NULL,
|
||||
total_code_size INTEGER NOT NULL,
|
||||
nodes_with_credentials INTEGER NOT NULL,
|
||||
total_docs_size INTEGER NOT NULL,
|
||||
extraction_date DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Views for common queries
|
||||
CREATE VIEW IF NOT EXISTS nodes_summary AS
|
||||
SELECT
|
||||
node_type,
|
||||
name,
|
||||
display_name,
|
||||
description,
|
||||
category,
|
||||
package_name,
|
||||
CASE WHEN documentation_markdown IS NOT NULL THEN 1 ELSE 0 END as has_documentation,
|
||||
CASE WHEN documentation_examples IS NOT NULL THEN 1 ELSE 0 END as has_examples,
|
||||
CASE WHEN operations IS NOT NULL THEN 1 ELSE 0 END as has_operations,
|
||||
has_credentials,
|
||||
is_trigger,
|
||||
is_webhook
|
||||
FROM nodes;
|
||||
|
||||
CREATE VIEW IF NOT EXISTS package_summary AS
|
||||
SELECT
|
||||
package_name,
|
||||
COUNT(*) as node_count,
|
||||
SUM(CASE WHEN documentation_markdown IS NOT NULL THEN 1 ELSE 0 END) as nodes_with_docs,
|
||||
SUM(CASE WHEN documentation_examples IS NOT NULL THEN 1 ELSE 0 END) as nodes_with_examples,
|
||||
SUM(has_credentials) as nodes_with_credentials,
|
||||
SUM(is_trigger) as trigger_nodes,
|
||||
SUM(is_webhook) as webhook_nodes
|
||||
FROM nodes
|
||||
GROUP BY package_name
|
||||
ORDER BY node_count DESC;
|
||||
@@ -1,101 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* n8n Documentation MCP Server
|
||||
* Copyright (c) 2025 n8n-mcp contributors
|
||||
*
|
||||
* This software is licensed under the Sustainable Use License.
|
||||
* See the LICENSE file in the root directory of this source tree.
|
||||
*/
|
||||
|
||||
import dotenv from 'dotenv';
|
||||
import { N8NDocumentationRemoteServer } from './mcp/remote-server';
|
||||
import { logger } from './utils/logger';
|
||||
import * as path from 'path';
|
||||
|
||||
// Load environment variables
|
||||
dotenv.config();
|
||||
|
||||
async function main() {
|
||||
try {
|
||||
// Get configuration from environment
|
||||
const config = {
|
||||
port: parseInt(process.env.MCP_PORT || '3000', 10),
|
||||
host: process.env.MCP_HOST || '0.0.0.0',
|
||||
domain: process.env.MCP_DOMAIN || 'localhost',
|
||||
authToken: process.env.MCP_AUTH_TOKEN,
|
||||
cors: process.env.MCP_CORS === 'true',
|
||||
tlsCert: process.env.MCP_TLS_CERT,
|
||||
tlsKey: process.env.MCP_TLS_KEY,
|
||||
};
|
||||
|
||||
// Validate required configuration
|
||||
if (!config.domain || config.domain === 'localhost') {
|
||||
logger.warn('MCP_DOMAIN not set or set to localhost. Using default: localhost');
|
||||
logger.warn('For production, set MCP_DOMAIN to your actual domain (e.g., n8ndocumentation.aiservices.pl)');
|
||||
}
|
||||
|
||||
if (!config.authToken) {
|
||||
logger.warn('MCP_AUTH_TOKEN not set. Server will run without authentication.');
|
||||
logger.warn('For production, set MCP_AUTH_TOKEN to a secure value.');
|
||||
}
|
||||
|
||||
// Set database path if not already set
|
||||
if (!process.env.NODE_DB_PATH) {
|
||||
process.env.NODE_DB_PATH = path.join(__dirname, '../data/nodes-v2.db');
|
||||
}
|
||||
|
||||
logger.info('Starting n8n Documentation MCP Remote Server');
|
||||
logger.info('Configuration:', {
|
||||
port: config.port,
|
||||
host: config.host,
|
||||
domain: config.domain,
|
||||
cors: config.cors,
|
||||
authEnabled: !!config.authToken,
|
||||
tlsEnabled: !!(config.tlsCert && config.tlsKey),
|
||||
databasePath: process.env.NODE_DB_PATH,
|
||||
});
|
||||
|
||||
const server = new N8NDocumentationRemoteServer(config);
|
||||
|
||||
// Start the server
|
||||
await server.start();
|
||||
|
||||
// Handle graceful shutdown
|
||||
const shutdown = async () => {
|
||||
logger.info('Received shutdown signal');
|
||||
await server.stop();
|
||||
process.exit(0);
|
||||
};
|
||||
|
||||
process.on('SIGINT', shutdown);
|
||||
process.on('SIGTERM', shutdown);
|
||||
|
||||
logger.info('Server is ready to accept connections');
|
||||
logger.info(`Claude Desktop configuration:`);
|
||||
logger.info(JSON.stringify({
|
||||
"mcpServers": {
|
||||
"n8n-nodes-remote": {
|
||||
"command": "curl",
|
||||
"args": [
|
||||
"-X", "POST",
|
||||
"-H", "Content-Type: application/json",
|
||||
"-H", `Authorization: Bearer ${config.authToken || 'YOUR_AUTH_TOKEN'}`,
|
||||
"-d", "@-",
|
||||
`https://${config.domain}/mcp`
|
||||
],
|
||||
"env": {}
|
||||
}
|
||||
}
|
||||
}, null, 2));
|
||||
|
||||
} catch (error) {
|
||||
logger.error('Failed to start server:', error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Run the server
|
||||
main().catch((error) => {
|
||||
logger.error('Unhandled error:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -1,67 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* n8n Documentation MCP Server
|
||||
* Copyright (c) 2025 n8n-mcp contributors
|
||||
*
|
||||
* This software is licensed under the Sustainable Use License.
|
||||
* See the LICENSE file in the root directory of this source tree.
|
||||
*/
|
||||
|
||||
import dotenv from 'dotenv';
|
||||
import { N8NDocumentationMCPServer } from './mcp/server-v2';
|
||||
import { MCPServerConfig } from './types';
|
||||
import { logger } from './utils/logger';
|
||||
import { NodeDocumentationService } from './services/node-documentation-service';
|
||||
|
||||
// Load environment variables
|
||||
dotenv.config();
|
||||
|
||||
async function main() {
|
||||
const config: MCPServerConfig = {
|
||||
port: parseInt(process.env.MCP_SERVER_PORT || '3000', 10),
|
||||
host: process.env.MCP_SERVER_HOST || 'localhost',
|
||||
authToken: process.env.MCP_AUTH_TOKEN,
|
||||
};
|
||||
|
||||
// Check if we should rebuild the database on startup
|
||||
const rebuildOnStart = process.env.REBUILD_ON_START === 'true';
|
||||
|
||||
if (rebuildOnStart) {
|
||||
logger.info('Rebuilding database on startup...');
|
||||
const service = new NodeDocumentationService();
|
||||
try {
|
||||
const stats = await service.rebuildDatabase();
|
||||
logger.info('Database rebuild complete:', stats);
|
||||
} catch (error) {
|
||||
logger.error('Failed to rebuild database:', error);
|
||||
process.exit(1);
|
||||
} finally {
|
||||
service.close();
|
||||
}
|
||||
}
|
||||
|
||||
const server = new N8NDocumentationMCPServer(config);
|
||||
|
||||
try {
|
||||
await server.start();
|
||||
} catch (error) {
|
||||
logger.error('Failed to start MCP server:', error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Handle graceful shutdown
|
||||
process.on('SIGINT', () => {
|
||||
logger.info('Received SIGINT, shutting down MCP server...');
|
||||
process.exit(0);
|
||||
});
|
||||
|
||||
process.on('SIGTERM', () => {
|
||||
logger.info('Received SIGTERM, shutting down MCP server...');
|
||||
process.exit(0);
|
||||
});
|
||||
|
||||
main().catch((error) => {
|
||||
logger.error('Unhandled error:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -1,592 +0,0 @@
|
||||
import express from 'express';
|
||||
import { createServer } from 'http';
|
||||
import { WebSocketServer, WebSocket } from 'ws';
|
||||
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
|
||||
// WebSocketServerTransport is not available in the SDK, we'll implement a custom solution
|
||||
import {
|
||||
CallToolRequestSchema,
|
||||
ErrorCode,
|
||||
ListResourcesRequestSchema,
|
||||
ListToolsRequestSchema,
|
||||
McpError,
|
||||
ReadResourceRequestSchema,
|
||||
} from '@modelcontextprotocol/sdk/types.js';
|
||||
import { NodeDocumentationService } from '../services/node-documentation-service';
|
||||
import { nodeDocumentationTools } from './tools-v2';
|
||||
import { logger } from '../utils/logger';
|
||||
import { authenticateRequest } from '../utils/auth-middleware';
|
||||
import * as crypto from 'crypto';
|
||||
|
||||
interface HttpServerConfig {
|
||||
port: number;
|
||||
host: string;
|
||||
domain: string;
|
||||
authToken?: string;
|
||||
cors?: boolean;
|
||||
tlsCert?: string;
|
||||
tlsKey?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* HTTP/WebSocket MCP Server for remote access
|
||||
*/
|
||||
export class N8NDocumentationHttpServer {
|
||||
private app: express.Application;
|
||||
private server: any;
|
||||
private wss!: WebSocketServer;
|
||||
private nodeService: NodeDocumentationService;
|
||||
private config: HttpServerConfig;
|
||||
private activeSessions: Map<string, any> = new Map();
|
||||
|
||||
constructor(config: HttpServerConfig) {
|
||||
this.config = config;
|
||||
this.app = express();
|
||||
this.nodeService = new NodeDocumentationService();
|
||||
|
||||
this.setupMiddleware();
|
||||
this.setupRoutes();
|
||||
this.setupWebSocket();
|
||||
}
|
||||
|
||||
private setupMiddleware(): void {
|
||||
// JSON parsing
|
||||
this.app.use(express.json());
|
||||
|
||||
// CORS if enabled
|
||||
if (this.config.cors) {
|
||||
this.app.use((req, res, next): void => {
|
||||
res.header('Access-Control-Allow-Origin', '*');
|
||||
res.header('Access-Control-Allow-Methods', 'GET, POST, OPTIONS');
|
||||
res.header('Access-Control-Allow-Headers', 'Content-Type, Authorization');
|
||||
if (req.method === 'OPTIONS') {
|
||||
res.sendStatus(200);
|
||||
return;
|
||||
}
|
||||
next();
|
||||
});
|
||||
}
|
||||
|
||||
// Request logging
|
||||
this.app.use((req, res, next): void => {
|
||||
logger.info(`${req.method} ${req.path}`, {
|
||||
ip: req.ip,
|
||||
userAgent: req.get('user-agent')
|
||||
});
|
||||
next();
|
||||
});
|
||||
}
|
||||
|
||||
private setupRoutes(): void {
|
||||
// Health check endpoint
|
||||
this.app.get('/health', (req, res) => {
|
||||
res.json({
|
||||
status: 'ok',
|
||||
service: 'n8n-documentation-mcp',
|
||||
version: '2.0.0',
|
||||
uptime: process.uptime()
|
||||
});
|
||||
});
|
||||
|
||||
// MCP info endpoint
|
||||
this.app.get('/mcp', (req, res) => {
|
||||
res.json({
|
||||
name: 'n8n-node-documentation',
|
||||
version: '2.0.0',
|
||||
description: 'MCP server providing n8n node documentation and source code',
|
||||
transport: 'websocket',
|
||||
endpoint: `wss://${this.config.domain}/mcp/websocket`,
|
||||
authentication: 'bearer-token',
|
||||
tools: nodeDocumentationTools.map(t => ({
|
||||
name: t.name,
|
||||
description: t.description
|
||||
}))
|
||||
});
|
||||
});
|
||||
|
||||
// Database stats endpoint (public)
|
||||
this.app.get('/stats', async (req, res) => {
|
||||
try {
|
||||
const stats = this.nodeService.getStatistics();
|
||||
res.json(stats);
|
||||
} catch (error) {
|
||||
logger.error('Failed to get statistics:', error);
|
||||
res.status(500).json({ error: 'Failed to retrieve statistics' });
|
||||
}
|
||||
});
|
||||
|
||||
// Rebuild endpoint (requires auth)
|
||||
this.app.post('/rebuild', authenticateRequest(this.config.authToken), async (req, res) => {
|
||||
try {
|
||||
logger.info('Database rebuild requested');
|
||||
const stats = await this.nodeService.rebuildDatabase();
|
||||
res.json({
|
||||
message: 'Database rebuild complete',
|
||||
stats
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Rebuild failed:', error);
|
||||
res.status(500).json({ error: 'Rebuild failed' });
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
private setupWebSocket(): void {
|
||||
// Create HTTP server
|
||||
this.server = createServer(this.app);
|
||||
|
||||
// Create WebSocket server
|
||||
this.wss = new WebSocketServer({
|
||||
server: this.server,
|
||||
path: '/mcp/websocket'
|
||||
});
|
||||
|
||||
this.wss.on('connection', async (ws: WebSocket, req: any) => {
|
||||
const sessionId = crypto.randomUUID();
|
||||
logger.info(`WebSocket connection established: ${sessionId}`);
|
||||
|
||||
// Authenticate WebSocket connection
|
||||
const authHeader = req.headers.authorization;
|
||||
if (this.config.authToken && authHeader !== `Bearer ${this.config.authToken}`) {
|
||||
logger.warn(`Unauthorized WebSocket connection attempt: ${sessionId}`);
|
||||
ws.close(1008, 'Unauthorized');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
// Create MCP server instance for this connection
|
||||
const mcpServer = new Server(
|
||||
{
|
||||
name: 'n8n-node-documentation',
|
||||
version: '2.0.0',
|
||||
},
|
||||
{
|
||||
capabilities: {
|
||||
tools: {},
|
||||
resources: {},
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
// Setup MCP handlers
|
||||
this.setupMcpHandlers(mcpServer);
|
||||
|
||||
// WebSocket transport not available in SDK - implement JSON-RPC over WebSocket
|
||||
// For now, we'll handle messages directly
|
||||
ws.on('message', async (data: Buffer) => {
|
||||
try {
|
||||
const request = JSON.parse(data.toString());
|
||||
// Process request through MCP server handlers
|
||||
// This would need custom implementation
|
||||
logger.warn('WebSocket MCP not fully implemented yet');
|
||||
ws.send(JSON.stringify({
|
||||
jsonrpc: '2.0',
|
||||
id: request.id,
|
||||
error: {
|
||||
code: -32601,
|
||||
message: 'WebSocket transport not implemented'
|
||||
}
|
||||
}));
|
||||
} catch (error) {
|
||||
logger.error('WebSocket message error:', error);
|
||||
}
|
||||
});
|
||||
|
||||
this.activeSessions.set(sessionId, { mcpServer, ws });
|
||||
logger.info(`MCP session established: ${sessionId}`);
|
||||
|
||||
// Handle disconnect
|
||||
ws.on('close', () => {
|
||||
logger.info(`WebSocket connection closed: ${sessionId}`);
|
||||
this.activeSessions.delete(sessionId);
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
logger.error(`Failed to establish MCP session: ${sessionId}`, error);
|
||||
ws.close(1011, 'Server error');
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
private setupMcpHandlers(server: Server): void {
|
||||
// List available tools
|
||||
server.setRequestHandler(ListToolsRequestSchema, async () => ({
|
||||
tools: nodeDocumentationTools,
|
||||
}));
|
||||
|
||||
// List available resources
|
||||
server.setRequestHandler(ListResourcesRequestSchema, async () => ({
|
||||
resources: [
|
||||
{
|
||||
uri: 'nodes://list',
|
||||
name: 'Available n8n Nodes',
|
||||
description: 'List of all available n8n nodes',
|
||||
mimeType: 'application/json',
|
||||
},
|
||||
{
|
||||
uri: 'nodes://statistics',
|
||||
name: 'Database Statistics',
|
||||
description: 'Statistics about the node documentation database',
|
||||
mimeType: 'application/json',
|
||||
},
|
||||
],
|
||||
}));
|
||||
|
||||
// Read resources
|
||||
server.setRequestHandler(ReadResourceRequestSchema, async (request) => {
|
||||
const { uri } = request.params;
|
||||
|
||||
try {
|
||||
if (uri === 'nodes://list') {
|
||||
const nodes = await this.nodeService.listNodes();
|
||||
return {
|
||||
contents: [
|
||||
{
|
||||
uri,
|
||||
mimeType: 'application/json',
|
||||
text: JSON.stringify(nodes.map(n => ({
|
||||
nodeType: n.nodeType,
|
||||
name: n.name,
|
||||
displayName: n.displayName,
|
||||
category: n.category,
|
||||
description: n.description,
|
||||
hasDocumentation: !!n.documentation,
|
||||
hasExample: !!n.exampleWorkflow,
|
||||
})), null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
if (uri === 'nodes://statistics') {
|
||||
const stats = this.nodeService.getStatistics();
|
||||
return {
|
||||
contents: [
|
||||
{
|
||||
uri,
|
||||
mimeType: 'application/json',
|
||||
text: JSON.stringify(stats, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Unknown resource: ${uri}`);
|
||||
} catch (error) {
|
||||
logger.error('Resource read error:', error);
|
||||
throw error instanceof McpError ? error : new McpError(
|
||||
ErrorCode.InternalError,
|
||||
`Failed to read resource: ${error instanceof Error ? error.message : String(error)}`
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
// Handle tool calls
|
||||
server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
||||
const { name, arguments: args } = request.params;
|
||||
|
||||
try {
|
||||
switch (name) {
|
||||
case 'list_nodes':
|
||||
return await this.handleListNodes(args);
|
||||
|
||||
case 'get_node_info':
|
||||
return await this.handleGetNodeInfo(args);
|
||||
|
||||
case 'search_nodes':
|
||||
return await this.handleSearchNodes(args);
|
||||
|
||||
case 'get_node_example':
|
||||
return await this.handleGetNodeExample(args);
|
||||
|
||||
case 'get_node_source_code':
|
||||
return await this.handleGetNodeSourceCode(args);
|
||||
|
||||
case 'get_node_documentation':
|
||||
return await this.handleGetNodeDocumentation(args);
|
||||
|
||||
case 'rebuild_database':
|
||||
return await this.handleRebuildDatabase(args);
|
||||
|
||||
case 'get_database_statistics':
|
||||
return await this.handleGetStatistics();
|
||||
|
||||
default:
|
||||
throw new McpError(ErrorCode.MethodNotFound, `Unknown tool: ${name}`);
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error(`Tool execution error (${name}):`, error);
|
||||
throw error instanceof McpError ? error : new McpError(
|
||||
ErrorCode.InternalError,
|
||||
`Tool execution failed: ${error instanceof Error ? error.message : String(error)}`
|
||||
);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Tool handlers (copied from server-v2.ts)
|
||||
private async handleListNodes(args: any): Promise<any> {
|
||||
const nodes = await this.nodeService.listNodes();
|
||||
|
||||
let filtered = nodes;
|
||||
|
||||
if (args.category) {
|
||||
filtered = filtered.filter(n => n.category === args.category);
|
||||
}
|
||||
|
||||
if (args.packageName) {
|
||||
filtered = filtered.filter(n => n.packageName === args.packageName);
|
||||
}
|
||||
|
||||
if (args.isTrigger !== undefined) {
|
||||
filtered = filtered.filter(n => n.isTrigger === args.isTrigger);
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(filtered.map(n => ({
|
||||
nodeType: n.nodeType,
|
||||
name: n.name,
|
||||
displayName: n.displayName,
|
||||
category: n.category,
|
||||
description: n.description,
|
||||
packageName: n.packageName,
|
||||
hasDocumentation: !!n.documentation,
|
||||
hasExample: !!n.exampleWorkflow,
|
||||
isTrigger: n.isTrigger,
|
||||
isWebhook: n.isWebhook,
|
||||
})), null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetNodeInfo(args: any): Promise<any> {
|
||||
if (!args.nodeType) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'nodeType is required');
|
||||
}
|
||||
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(args.nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${args.nodeType}`);
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify({
|
||||
nodeType: nodeInfo.nodeType,
|
||||
name: nodeInfo.name,
|
||||
displayName: nodeInfo.displayName,
|
||||
description: nodeInfo.description,
|
||||
category: nodeInfo.category,
|
||||
packageName: nodeInfo.packageName,
|
||||
sourceCode: nodeInfo.sourceCode,
|
||||
credentialCode: nodeInfo.credentialCode,
|
||||
documentation: nodeInfo.documentation,
|
||||
documentationUrl: nodeInfo.documentationUrl,
|
||||
exampleWorkflow: nodeInfo.exampleWorkflow,
|
||||
exampleParameters: nodeInfo.exampleParameters,
|
||||
propertiesSchema: nodeInfo.propertiesSchema,
|
||||
isTrigger: nodeInfo.isTrigger,
|
||||
isWebhook: nodeInfo.isWebhook,
|
||||
}, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleSearchNodes(args: any): Promise<any> {
|
||||
if (!args.query) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'query is required');
|
||||
}
|
||||
|
||||
const results = await this.nodeService.searchNodes({
|
||||
query: args.query,
|
||||
category: args.category,
|
||||
limit: args.limit || 20,
|
||||
});
|
||||
|
||||
let filtered = results;
|
||||
if (args.hasDocumentation) {
|
||||
filtered = filtered.filter(n => !!n.documentation);
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(filtered.map(n => ({
|
||||
nodeType: n.nodeType,
|
||||
name: n.name,
|
||||
displayName: n.displayName,
|
||||
category: n.category,
|
||||
description: n.description,
|
||||
hasDocumentation: !!n.documentation,
|
||||
hasExample: !!n.exampleWorkflow,
|
||||
})), null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetNodeExample(args: any): Promise<any> {
|
||||
if (!args.nodeType) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'nodeType is required');
|
||||
}
|
||||
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(args.nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${args.nodeType}`);
|
||||
}
|
||||
|
||||
if (!nodeInfo.exampleWorkflow) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: `No example available for node: ${args.nodeType}`,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(nodeInfo.exampleWorkflow, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetNodeSourceCode(args: any): Promise<any> {
|
||||
if (!args.nodeType) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'nodeType is required');
|
||||
}
|
||||
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(args.nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${args.nodeType}`);
|
||||
}
|
||||
|
||||
const response: any = {
|
||||
nodeType: nodeInfo.nodeType,
|
||||
sourceCode: nodeInfo.sourceCode,
|
||||
};
|
||||
|
||||
if (args.includeCredentials && nodeInfo.credentialCode) {
|
||||
response.credentialCode = nodeInfo.credentialCode;
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(response, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetNodeDocumentation(args: any): Promise<any> {
|
||||
if (!args.nodeType) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'nodeType is required');
|
||||
}
|
||||
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(args.nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${args.nodeType}`);
|
||||
}
|
||||
|
||||
if (!nodeInfo.documentation) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: `No documentation available for node: ${args.nodeType}`,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
const content = args.format === 'plain'
|
||||
? nodeInfo.documentation.replace(/[#*`]/g, '')
|
||||
: nodeInfo.documentation;
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: content,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleRebuildDatabase(args: any): Promise<any> {
|
||||
logger.info('Database rebuild requested via MCP');
|
||||
|
||||
const stats = await this.nodeService.rebuildDatabase();
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify({
|
||||
message: 'Database rebuild complete',
|
||||
stats,
|
||||
}, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetStatistics(): Promise<any> {
|
||||
const stats = this.nodeService.getStatistics();
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(stats, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
async start(): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
this.server.listen(this.config.port, this.config.host, () => {
|
||||
logger.info(`n8n Documentation MCP HTTP server started`);
|
||||
logger.info(`HTTP endpoint: http://${this.config.host}:${this.config.port}`);
|
||||
logger.info(`WebSocket endpoint: ws://${this.config.host}:${this.config.port}/mcp/websocket`);
|
||||
logger.info(`Domain: ${this.config.domain}`);
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async stop(): Promise<void> {
|
||||
logger.info('Stopping n8n Documentation MCP HTTP server...');
|
||||
|
||||
// Close all WebSocket connections
|
||||
this.wss.clients.forEach((ws: WebSocket) => ws.close());
|
||||
|
||||
// Close HTTP server
|
||||
return new Promise((resolve) => {
|
||||
this.server.close(() => {
|
||||
this.nodeService.close();
|
||||
logger.info('Server stopped');
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,608 +0,0 @@
|
||||
import express from 'express';
|
||||
import { createServer as createHttpServer } from 'http';
|
||||
import { createServer as createHttpsServer } from 'https';
|
||||
import {
|
||||
ErrorCode,
|
||||
McpError,
|
||||
} from '@modelcontextprotocol/sdk/types.js';
|
||||
import { NodeDocumentationService } from '../services/node-documentation-service';
|
||||
import { nodeDocumentationTools } from './tools-v2';
|
||||
import { logger } from '../utils/logger';
|
||||
import { authenticateRequest } from '../utils/auth-middleware';
|
||||
import * as fs from 'fs';
|
||||
|
||||
interface RemoteServerConfig {
|
||||
port: number;
|
||||
host: string;
|
||||
domain: string;
|
||||
authToken?: string;
|
||||
cors?: boolean;
|
||||
tlsCert?: string;
|
||||
tlsKey?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Remote MCP Server using Streamable HTTP transport
|
||||
* Based on MCP's modern approach for remote servers
|
||||
*/
|
||||
export class N8NDocumentationRemoteServer {
|
||||
private app: express.Application;
|
||||
private server: any;
|
||||
private nodeService: NodeDocumentationService;
|
||||
private config: RemoteServerConfig;
|
||||
|
||||
constructor(config: RemoteServerConfig) {
|
||||
this.config = config;
|
||||
this.app = express();
|
||||
this.nodeService = new NodeDocumentationService();
|
||||
|
||||
this.setupMiddleware();
|
||||
this.setupRoutes();
|
||||
}
|
||||
|
||||
private setupMiddleware(): void {
|
||||
// Parse JSON bodies with larger limit for MCP messages
|
||||
this.app.use(express.json({ limit: '10mb' }));
|
||||
|
||||
// CORS if enabled
|
||||
if (this.config.cors) {
|
||||
this.app.use((req, res, next): void => {
|
||||
res.header('Access-Control-Allow-Origin', '*');
|
||||
res.header('Access-Control-Allow-Methods', 'GET, POST, OPTIONS');
|
||||
res.header('Access-Control-Allow-Headers', 'Content-Type, Authorization, X-Request-ID');
|
||||
if (req.method === 'OPTIONS') {
|
||||
res.sendStatus(200);
|
||||
return;
|
||||
}
|
||||
next();
|
||||
});
|
||||
}
|
||||
|
||||
// Request logging
|
||||
this.app.use((req, res, next): void => {
|
||||
logger.info(`${req.method} ${req.path}`, {
|
||||
ip: req.ip,
|
||||
userAgent: req.get('user-agent'),
|
||||
requestId: req.get('X-Request-ID')
|
||||
});
|
||||
next();
|
||||
});
|
||||
}
|
||||
|
||||
private setupRoutes(): void {
|
||||
// Health check endpoint
|
||||
this.app.get('/health', (req, res) => {
|
||||
res.json({
|
||||
status: 'ok',
|
||||
service: 'n8n-documentation-mcp',
|
||||
version: '2.0.0',
|
||||
uptime: process.uptime(),
|
||||
domain: this.config.domain
|
||||
});
|
||||
});
|
||||
|
||||
// MCP info endpoint - provides server capabilities
|
||||
this.app.get('/', (req, res) => {
|
||||
res.json({
|
||||
name: 'n8n-node-documentation',
|
||||
version: '2.0.0',
|
||||
description: 'MCP server providing n8n node documentation and source code',
|
||||
transport: 'http',
|
||||
endpoint: `https://${this.config.domain}/mcp`,
|
||||
authentication: this.config.authToken ? 'bearer-token' : 'none',
|
||||
capabilities: {
|
||||
tools: nodeDocumentationTools.map(t => ({
|
||||
name: t.name,
|
||||
description: t.description
|
||||
})),
|
||||
resources: [
|
||||
{
|
||||
uri: 'nodes://list',
|
||||
name: 'Available n8n Nodes',
|
||||
description: 'List of all available n8n nodes',
|
||||
},
|
||||
{
|
||||
uri: 'nodes://statistics',
|
||||
name: 'Database Statistics',
|
||||
description: 'Statistics about the node documentation database',
|
||||
},
|
||||
]
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// Database stats endpoint (public)
|
||||
this.app.get('/stats', async (req, res) => {
|
||||
try {
|
||||
const stats = this.nodeService.getStatistics();
|
||||
res.json(stats);
|
||||
} catch (error) {
|
||||
logger.error('Failed to get statistics:', error);
|
||||
res.status(500).json({ error: 'Failed to retrieve statistics' });
|
||||
}
|
||||
});
|
||||
|
||||
// Rebuild endpoint (requires auth)
|
||||
this.app.post('/rebuild', authenticateRequest(this.config.authToken), async (req, res) => {
|
||||
try {
|
||||
logger.info('Database rebuild requested');
|
||||
const stats = await this.nodeService.rebuildDatabase();
|
||||
res.json({
|
||||
message: 'Database rebuild complete',
|
||||
stats
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Rebuild failed:', error);
|
||||
res.status(500).json({ error: 'Rebuild failed' });
|
||||
}
|
||||
});
|
||||
|
||||
// Main MCP endpoint - handles all MCP protocol messages
|
||||
this.app.post('/mcp', authenticateRequest(this.config.authToken), async (req, res) => {
|
||||
const requestId = req.get('X-Request-ID') || 'unknown';
|
||||
|
||||
try {
|
||||
// Process the JSON-RPC request directly
|
||||
const response = await this.handleJsonRpcRequest(req.body);
|
||||
res.json(response);
|
||||
} catch (error) {
|
||||
logger.error(`MCP request failed (${requestId}):`, error);
|
||||
|
||||
if (!res.headersSent) {
|
||||
res.status(500).json({
|
||||
jsonrpc: '2.0',
|
||||
id: req.body?.id || null,
|
||||
error: {
|
||||
code: -32603,
|
||||
message: 'Internal error',
|
||||
data: error instanceof Error ? error.message : String(error)
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
private async handleJsonRpcRequest(request: any): Promise<any> {
|
||||
const { jsonrpc, method, params, id } = request;
|
||||
|
||||
if (jsonrpc !== '2.0') {
|
||||
return {
|
||||
jsonrpc: '2.0',
|
||||
id: id || null,
|
||||
error: {
|
||||
code: -32600,
|
||||
message: 'Invalid Request',
|
||||
data: 'JSON-RPC version must be "2.0"'
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
let result;
|
||||
|
||||
switch (method) {
|
||||
case 'tools/list':
|
||||
result = await this.handleListTools();
|
||||
break;
|
||||
|
||||
case 'resources/list':
|
||||
result = await this.handleListResources();
|
||||
break;
|
||||
|
||||
case 'resources/read':
|
||||
result = await this.handleReadResource(params);
|
||||
break;
|
||||
|
||||
case 'tools/call':
|
||||
result = await this.handleToolCall(params);
|
||||
break;
|
||||
|
||||
default:
|
||||
return {
|
||||
jsonrpc: '2.0',
|
||||
id: id || null,
|
||||
error: {
|
||||
code: -32601,
|
||||
message: 'Method not found',
|
||||
data: `Unknown method: ${method}`
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
jsonrpc: '2.0',
|
||||
id: id || null,
|
||||
result
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error(`Error handling method ${method}:`, error);
|
||||
|
||||
const errorCode = error instanceof McpError ? error.code : -32603;
|
||||
const errorMessage = error instanceof Error ? error.message : 'Internal error';
|
||||
|
||||
return {
|
||||
jsonrpc: '2.0',
|
||||
id: id || null,
|
||||
error: {
|
||||
code: errorCode,
|
||||
message: errorMessage,
|
||||
data: error instanceof McpError ? error.data : undefined
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
private async handleListTools(): Promise<any> {
|
||||
return {
|
||||
tools: nodeDocumentationTools,
|
||||
};
|
||||
}
|
||||
|
||||
private async handleListResources(): Promise<any> {
|
||||
return {
|
||||
resources: [
|
||||
{
|
||||
uri: 'nodes://list',
|
||||
name: 'Available n8n Nodes',
|
||||
description: 'List of all available n8n nodes',
|
||||
mimeType: 'application/json',
|
||||
},
|
||||
{
|
||||
uri: 'nodes://statistics',
|
||||
name: 'Database Statistics',
|
||||
description: 'Statistics about the node documentation database',
|
||||
mimeType: 'application/json',
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleReadResource(params: any): Promise<any> {
|
||||
const { uri } = params;
|
||||
|
||||
if (uri === 'nodes://list') {
|
||||
const nodes = await this.nodeService.listNodes();
|
||||
return {
|
||||
contents: [
|
||||
{
|
||||
uri,
|
||||
mimeType: 'application/json',
|
||||
text: JSON.stringify(nodes.map(n => ({
|
||||
nodeType: n.nodeType,
|
||||
name: n.name,
|
||||
displayName: n.displayName,
|
||||
category: n.category,
|
||||
description: n.description,
|
||||
hasDocumentation: !!n.documentation,
|
||||
hasExample: !!n.exampleWorkflow,
|
||||
})), null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
if (uri === 'nodes://statistics') {
|
||||
const stats = this.nodeService.getStatistics();
|
||||
return {
|
||||
contents: [
|
||||
{
|
||||
uri,
|
||||
mimeType: 'application/json',
|
||||
text: JSON.stringify(stats, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Unknown resource: ${uri}`);
|
||||
}
|
||||
|
||||
private async handleToolCall(params: any): Promise<any> {
|
||||
const { name, arguments: args } = params;
|
||||
|
||||
switch (name) {
|
||||
case 'list_nodes':
|
||||
return await this.handleListNodes(args);
|
||||
|
||||
case 'get_node_info':
|
||||
return await this.handleGetNodeInfo(args);
|
||||
|
||||
case 'search_nodes':
|
||||
return await this.handleSearchNodes(args);
|
||||
|
||||
case 'get_node_example':
|
||||
return await this.handleGetNodeExample(args);
|
||||
|
||||
case 'get_node_source_code':
|
||||
return await this.handleGetNodeSourceCode(args);
|
||||
|
||||
case 'get_node_documentation':
|
||||
return await this.handleGetNodeDocumentation(args);
|
||||
|
||||
case 'rebuild_database':
|
||||
return await this.handleRebuildDatabase(args);
|
||||
|
||||
case 'get_database_statistics':
|
||||
return await this.handleGetStatistics();
|
||||
|
||||
default:
|
||||
throw new McpError(ErrorCode.MethodNotFound, `Unknown tool: ${name}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Tool handlers
|
||||
private async handleListNodes(args: any): Promise<any> {
|
||||
const nodes = await this.nodeService.listNodes();
|
||||
|
||||
let filtered = nodes;
|
||||
|
||||
if (args.category) {
|
||||
filtered = filtered.filter(n => n.category === args.category);
|
||||
}
|
||||
|
||||
if (args.packageName) {
|
||||
filtered = filtered.filter(n => n.packageName === args.packageName);
|
||||
}
|
||||
|
||||
if (args.isTrigger !== undefined) {
|
||||
filtered = filtered.filter(n => n.isTrigger === args.isTrigger);
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(filtered.map(n => ({
|
||||
nodeType: n.nodeType,
|
||||
name: n.name,
|
||||
displayName: n.displayName,
|
||||
category: n.category,
|
||||
description: n.description,
|
||||
packageName: n.packageName,
|
||||
hasDocumentation: !!n.documentation,
|
||||
hasExample: !!n.exampleWorkflow,
|
||||
isTrigger: n.isTrigger,
|
||||
isWebhook: n.isWebhook,
|
||||
})), null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetNodeInfo(args: any): Promise<any> {
|
||||
if (!args.nodeType) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'nodeType is required');
|
||||
}
|
||||
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(args.nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${args.nodeType}`);
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify({
|
||||
nodeType: nodeInfo.nodeType,
|
||||
name: nodeInfo.name,
|
||||
displayName: nodeInfo.displayName,
|
||||
description: nodeInfo.description,
|
||||
category: nodeInfo.category,
|
||||
packageName: nodeInfo.packageName,
|
||||
sourceCode: nodeInfo.sourceCode,
|
||||
credentialCode: nodeInfo.credentialCode,
|
||||
documentation: nodeInfo.documentation,
|
||||
documentationUrl: nodeInfo.documentationUrl,
|
||||
exampleWorkflow: nodeInfo.exampleWorkflow,
|
||||
exampleParameters: nodeInfo.exampleParameters,
|
||||
propertiesSchema: nodeInfo.propertiesSchema,
|
||||
isTrigger: nodeInfo.isTrigger,
|
||||
isWebhook: nodeInfo.isWebhook,
|
||||
}, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleSearchNodes(args: any): Promise<any> {
|
||||
if (!args.query) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'query is required');
|
||||
}
|
||||
|
||||
const results = await this.nodeService.searchNodes({
|
||||
query: args.query,
|
||||
category: args.category,
|
||||
limit: args.limit || 20,
|
||||
});
|
||||
|
||||
let filtered = results;
|
||||
if (args.hasDocumentation) {
|
||||
filtered = filtered.filter(n => !!n.documentation);
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(filtered.map(n => ({
|
||||
nodeType: n.nodeType,
|
||||
name: n.name,
|
||||
displayName: n.displayName,
|
||||
category: n.category,
|
||||
description: n.description,
|
||||
hasDocumentation: !!n.documentation,
|
||||
hasExample: !!n.exampleWorkflow,
|
||||
})), null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetNodeExample(args: any): Promise<any> {
|
||||
if (!args.nodeType) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'nodeType is required');
|
||||
}
|
||||
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(args.nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${args.nodeType}`);
|
||||
}
|
||||
|
||||
if (!nodeInfo.exampleWorkflow) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: `No example available for node: ${args.nodeType}`,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(nodeInfo.exampleWorkflow, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetNodeSourceCode(args: any): Promise<any> {
|
||||
if (!args.nodeType) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'nodeType is required');
|
||||
}
|
||||
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(args.nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${args.nodeType}`);
|
||||
}
|
||||
|
||||
const response: any = {
|
||||
nodeType: nodeInfo.nodeType,
|
||||
sourceCode: nodeInfo.sourceCode,
|
||||
};
|
||||
|
||||
if (args.includeCredentials && nodeInfo.credentialCode) {
|
||||
response.credentialCode = nodeInfo.credentialCode;
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(response, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetNodeDocumentation(args: any): Promise<any> {
|
||||
if (!args.nodeType) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'nodeType is required');
|
||||
}
|
||||
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(args.nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${args.nodeType}`);
|
||||
}
|
||||
|
||||
if (!nodeInfo.documentation) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: `No documentation available for node: ${args.nodeType}`,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
const content = args.format === 'plain'
|
||||
? nodeInfo.documentation.replace(/[#*`]/g, '')
|
||||
: nodeInfo.documentation;
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: content,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleRebuildDatabase(args: any): Promise<any> {
|
||||
logger.info('Database rebuild requested via MCP');
|
||||
|
||||
const stats = await this.nodeService.rebuildDatabase();
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify({
|
||||
message: 'Database rebuild complete',
|
||||
stats,
|
||||
}, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetStatistics(): Promise<any> {
|
||||
const stats = this.nodeService.getStatistics();
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(stats, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
async start(): Promise<void> {
|
||||
// Create server (HTTP or HTTPS)
|
||||
if (this.config.tlsCert && this.config.tlsKey) {
|
||||
const tlsOptions = {
|
||||
cert: fs.readFileSync(this.config.tlsCert),
|
||||
key: fs.readFileSync(this.config.tlsKey),
|
||||
};
|
||||
this.server = createHttpsServer(tlsOptions, this.app);
|
||||
} else {
|
||||
this.server = createHttpServer(this.app);
|
||||
}
|
||||
|
||||
return new Promise((resolve) => {
|
||||
this.server.listen(this.config.port, this.config.host, () => {
|
||||
const protocol = this.config.tlsCert ? 'https' : 'http';
|
||||
logger.info(`n8n Documentation MCP Remote server started`);
|
||||
logger.info(`Endpoint: ${protocol}://${this.config.host}:${this.config.port}`);
|
||||
logger.info(`Domain: ${this.config.domain}`);
|
||||
logger.info(`MCP endpoint: ${protocol}://${this.config.domain}/mcp`);
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async stop(): Promise<void> {
|
||||
logger.info('Stopping n8n Documentation MCP Remote server...');
|
||||
|
||||
return new Promise((resolve) => {
|
||||
this.server.close(() => {
|
||||
this.nodeService.close();
|
||||
logger.info('Server stopped');
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,435 +0,0 @@
|
||||
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
|
||||
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
|
||||
import {
|
||||
CallToolRequestSchema,
|
||||
ErrorCode,
|
||||
ListResourcesRequestSchema,
|
||||
ListToolsRequestSchema,
|
||||
McpError,
|
||||
ReadResourceRequestSchema,
|
||||
} from '@modelcontextprotocol/sdk/types.js';
|
||||
import { NodeDocumentationService } from '../services/node-documentation-service';
|
||||
import { nodeDocumentationTools } from './tools-v2';
|
||||
import { logger } from '../utils/logger';
|
||||
import { MCPServerConfig } from '../types';
|
||||
|
||||
/**
|
||||
* MCP Server focused on serving n8n node documentation and code
|
||||
*/
|
||||
export class N8NDocumentationMCPServer {
|
||||
private server: Server;
|
||||
private nodeService: NodeDocumentationService;
|
||||
|
||||
constructor(config: MCPServerConfig) {
|
||||
logger.info('Initializing n8n Documentation MCP server', { config });
|
||||
|
||||
this.server = new Server(
|
||||
{
|
||||
name: 'n8n-node-documentation',
|
||||
version: '2.0.0',
|
||||
},
|
||||
{
|
||||
capabilities: {
|
||||
tools: {},
|
||||
resources: {},
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
this.nodeService = new NodeDocumentationService();
|
||||
this.setupHandlers();
|
||||
}
|
||||
|
||||
private setupHandlers(): void {
|
||||
// List available tools
|
||||
this.server.setRequestHandler(ListToolsRequestSchema, async () => ({
|
||||
tools: nodeDocumentationTools,
|
||||
}));
|
||||
|
||||
// List available resources
|
||||
this.server.setRequestHandler(ListResourcesRequestSchema, async () => ({
|
||||
resources: [
|
||||
{
|
||||
uri: 'nodes://list',
|
||||
name: 'Available n8n Nodes',
|
||||
description: 'List of all available n8n nodes',
|
||||
mimeType: 'application/json',
|
||||
},
|
||||
{
|
||||
uri: 'nodes://statistics',
|
||||
name: 'Database Statistics',
|
||||
description: 'Statistics about the node documentation database',
|
||||
mimeType: 'application/json',
|
||||
},
|
||||
],
|
||||
}));
|
||||
|
||||
// Read resources
|
||||
this.server.setRequestHandler(ReadResourceRequestSchema, async (request) => {
|
||||
const { uri } = request.params;
|
||||
|
||||
try {
|
||||
if (uri === 'nodes://list') {
|
||||
const nodes = await this.nodeService.listNodes();
|
||||
return {
|
||||
contents: [
|
||||
{
|
||||
uri,
|
||||
mimeType: 'application/json',
|
||||
text: JSON.stringify(nodes.map(n => ({
|
||||
nodeType: n.nodeType,
|
||||
name: n.name,
|
||||
displayName: n.displayName,
|
||||
category: n.category,
|
||||
description: n.description,
|
||||
hasDocumentation: !!n.documentation,
|
||||
hasExample: !!n.exampleWorkflow,
|
||||
})), null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
if (uri === 'nodes://statistics') {
|
||||
const stats = this.nodeService.getStatistics();
|
||||
return {
|
||||
contents: [
|
||||
{
|
||||
uri,
|
||||
mimeType: 'application/json',
|
||||
text: JSON.stringify(stats, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
// Handle specific node URIs like nodes://info/n8n-nodes-base.if
|
||||
const nodeMatch = uri.match(/^nodes:\/\/info\/(.+)$/);
|
||||
if (nodeMatch) {
|
||||
const nodeType = nodeMatch[1];
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${nodeType}`);
|
||||
}
|
||||
|
||||
return {
|
||||
contents: [
|
||||
{
|
||||
uri,
|
||||
mimeType: 'application/json',
|
||||
text: JSON.stringify(nodeInfo, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Unknown resource: ${uri}`);
|
||||
} catch (error) {
|
||||
logger.error('Resource read error:', error);
|
||||
throw error instanceof McpError ? error : new McpError(
|
||||
ErrorCode.InternalError,
|
||||
`Failed to read resource: ${error instanceof Error ? error.message : String(error)}`
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
// Handle tool calls
|
||||
this.server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
||||
const { name, arguments: args } = request.params;
|
||||
|
||||
try {
|
||||
switch (name) {
|
||||
case 'list_nodes':
|
||||
return await this.handleListNodes(args);
|
||||
|
||||
case 'get_node_info':
|
||||
return await this.handleGetNodeInfo(args);
|
||||
|
||||
case 'search_nodes':
|
||||
return await this.handleSearchNodes(args);
|
||||
|
||||
case 'get_node_example':
|
||||
return await this.handleGetNodeExample(args);
|
||||
|
||||
case 'get_node_source_code':
|
||||
return await this.handleGetNodeSourceCode(args);
|
||||
|
||||
case 'get_node_documentation':
|
||||
return await this.handleGetNodeDocumentation(args);
|
||||
|
||||
case 'rebuild_database':
|
||||
return await this.handleRebuildDatabase(args);
|
||||
|
||||
case 'get_database_statistics':
|
||||
return await this.handleGetStatistics();
|
||||
|
||||
default:
|
||||
throw new McpError(ErrorCode.MethodNotFound, `Unknown tool: ${name}`);
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error(`Tool execution error (${name}):`, error);
|
||||
throw error instanceof McpError ? error : new McpError(
|
||||
ErrorCode.InternalError,
|
||||
`Tool execution failed: ${error instanceof Error ? error.message : String(error)}`
|
||||
);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
private async handleListNodes(args: any): Promise<any> {
|
||||
const nodes = await this.nodeService.listNodes();
|
||||
|
||||
// Apply filters
|
||||
let filtered = nodes;
|
||||
|
||||
if (args.category) {
|
||||
filtered = filtered.filter(n => n.category === args.category);
|
||||
}
|
||||
|
||||
if (args.packageName) {
|
||||
filtered = filtered.filter(n => n.packageName === args.packageName);
|
||||
}
|
||||
|
||||
if (args.isTrigger !== undefined) {
|
||||
filtered = filtered.filter(n => n.isTrigger === args.isTrigger);
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(filtered.map(n => ({
|
||||
nodeType: n.nodeType,
|
||||
name: n.name,
|
||||
displayName: n.displayName,
|
||||
category: n.category,
|
||||
description: n.description,
|
||||
packageName: n.packageName,
|
||||
hasDocumentation: !!n.documentation,
|
||||
hasExample: !!n.exampleWorkflow,
|
||||
isTrigger: n.isTrigger,
|
||||
isWebhook: n.isWebhook,
|
||||
})), null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetNodeInfo(args: any): Promise<any> {
|
||||
if (!args.nodeType) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'nodeType is required');
|
||||
}
|
||||
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(args.nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${args.nodeType}`);
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify({
|
||||
nodeType: nodeInfo.nodeType,
|
||||
name: nodeInfo.name,
|
||||
displayName: nodeInfo.displayName,
|
||||
description: nodeInfo.description,
|
||||
category: nodeInfo.category,
|
||||
packageName: nodeInfo.packageName,
|
||||
sourceCode: nodeInfo.sourceCode,
|
||||
credentialCode: nodeInfo.credentialCode,
|
||||
documentation: nodeInfo.documentation,
|
||||
documentationUrl: nodeInfo.documentationUrl,
|
||||
exampleWorkflow: nodeInfo.exampleWorkflow,
|
||||
exampleParameters: nodeInfo.exampleParameters,
|
||||
propertiesSchema: nodeInfo.propertiesSchema,
|
||||
isTrigger: nodeInfo.isTrigger,
|
||||
isWebhook: nodeInfo.isWebhook,
|
||||
}, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleSearchNodes(args: any): Promise<any> {
|
||||
if (!args.query) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'query is required');
|
||||
}
|
||||
|
||||
const results = await this.nodeService.searchNodes({
|
||||
query: args.query,
|
||||
category: args.category,
|
||||
limit: args.limit || 20,
|
||||
});
|
||||
|
||||
// Filter by documentation if requested
|
||||
let filtered = results;
|
||||
if (args.hasDocumentation) {
|
||||
filtered = filtered.filter(n => !!n.documentation);
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(filtered.map(n => ({
|
||||
nodeType: n.nodeType,
|
||||
name: n.name,
|
||||
displayName: n.displayName,
|
||||
category: n.category,
|
||||
description: n.description,
|
||||
hasDocumentation: !!n.documentation,
|
||||
hasExample: !!n.exampleWorkflow,
|
||||
})), null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetNodeExample(args: any): Promise<any> {
|
||||
if (!args.nodeType) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'nodeType is required');
|
||||
}
|
||||
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(args.nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${args.nodeType}`);
|
||||
}
|
||||
|
||||
if (!nodeInfo.exampleWorkflow) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: `No example available for node: ${args.nodeType}`,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(nodeInfo.exampleWorkflow, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetNodeSourceCode(args: any): Promise<any> {
|
||||
if (!args.nodeType) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'nodeType is required');
|
||||
}
|
||||
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(args.nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${args.nodeType}`);
|
||||
}
|
||||
|
||||
const response: any = {
|
||||
nodeType: nodeInfo.nodeType,
|
||||
sourceCode: nodeInfo.sourceCode,
|
||||
};
|
||||
|
||||
if (args.includeCredentials && nodeInfo.credentialCode) {
|
||||
response.credentialCode = nodeInfo.credentialCode;
|
||||
}
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(response, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetNodeDocumentation(args: any): Promise<any> {
|
||||
if (!args.nodeType) {
|
||||
throw new McpError(ErrorCode.InvalidParams, 'nodeType is required');
|
||||
}
|
||||
|
||||
const nodeInfo = await this.nodeService.getNodeInfo(args.nodeType);
|
||||
|
||||
if (!nodeInfo) {
|
||||
throw new McpError(ErrorCode.InvalidRequest, `Node not found: ${args.nodeType}`);
|
||||
}
|
||||
|
||||
if (!nodeInfo.documentation) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: `No documentation available for node: ${args.nodeType}`,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
const content = args.format === 'plain'
|
||||
? nodeInfo.documentation.replace(/[#*`]/g, '')
|
||||
: nodeInfo.documentation;
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: content,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleRebuildDatabase(args: any): Promise<any> {
|
||||
logger.info('Starting database rebuild...');
|
||||
|
||||
const stats = await this.nodeService.rebuildDatabase();
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify({
|
||||
message: 'Database rebuild complete',
|
||||
stats,
|
||||
}, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async handleGetStatistics(): Promise<any> {
|
||||
const stats = this.nodeService.getStatistics();
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: 'text',
|
||||
text: JSON.stringify(stats, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
async start(): Promise<void> {
|
||||
logger.info('Starting n8n Documentation MCP server...');
|
||||
|
||||
const transport = new StdioServerTransport();
|
||||
await this.server.connect(transport);
|
||||
|
||||
logger.info('n8n Documentation MCP server started successfully');
|
||||
}
|
||||
|
||||
async stop(): Promise<void> {
|
||||
logger.info('Stopping n8n Documentation MCP server...');
|
||||
await this.server.close();
|
||||
this.nodeService.close();
|
||||
logger.info('Server stopped');
|
||||
}
|
||||
}
|
||||
@@ -16,18 +16,18 @@ import { N8NApiClient } from '../utils/n8n-client';
|
||||
import { N8NMCPBridge } from '../utils/bridge';
|
||||
import { logger } from '../utils/logger';
|
||||
import { NodeSourceExtractor } from '../utils/node-source-extractor';
|
||||
import { SQLiteStorageService } from '../services/sqlite-storage-service';
|
||||
import { NodeDocumentationService } from '../services/node-documentation-service';
|
||||
|
||||
export class N8NMCPServer {
|
||||
private server: Server;
|
||||
private n8nClient: N8NApiClient;
|
||||
private nodeExtractor: NodeSourceExtractor;
|
||||
private nodeStorage: SQLiteStorageService;
|
||||
private nodeDocService: NodeDocumentationService;
|
||||
|
||||
constructor(config: MCPServerConfig, n8nConfig: N8NConfig) {
|
||||
this.n8nClient = new N8NApiClient(n8nConfig);
|
||||
this.nodeExtractor = new NodeSourceExtractor();
|
||||
this.nodeStorage = new SQLiteStorageService();
|
||||
this.nodeDocService = new NodeDocumentationService();
|
||||
logger.info('Initializing n8n MCP server', { config, n8nConfig });
|
||||
this.server = new Server(
|
||||
{
|
||||
@@ -164,12 +164,14 @@ export class N8NMCPServer {
|
||||
return this.getNodeSourceCode(args);
|
||||
case 'list_available_nodes':
|
||||
return this.listAvailableNodes(args);
|
||||
case 'extract_all_nodes':
|
||||
return this.extractAllNodes(args);
|
||||
case 'get_node_info':
|
||||
return this.getNodeInfo(args);
|
||||
case 'search_nodes':
|
||||
return this.searchNodes(args);
|
||||
case 'get_node_statistics':
|
||||
return this.getNodeStatistics(args);
|
||||
case 'rebuild_documentation_database':
|
||||
return this.rebuildDocumentationDatabase(args);
|
||||
default:
|
||||
throw new Error(`Unknown tool: ${name}`);
|
||||
}
|
||||
@@ -323,84 +325,87 @@ export class N8NMCPServer {
|
||||
}
|
||||
}
|
||||
|
||||
private async extractAllNodes(args: any): Promise<any> {
|
||||
|
||||
private async getNodeInfo(args: any): Promise<any> {
|
||||
try {
|
||||
logger.info(`Extracting all nodes`, args);
|
||||
logger.info('Getting comprehensive node information', args);
|
||||
const nodeInfo = await this.nodeDocService.getNodeInfo(args.nodeType);
|
||||
|
||||
// Get list of all nodes
|
||||
const allNodes = await this.nodeExtractor.listAvailableNodes();
|
||||
let nodesToExtract = allNodes;
|
||||
|
||||
// Apply filters
|
||||
if (args.packageFilter) {
|
||||
nodesToExtract = nodesToExtract.filter(node =>
|
||||
node.packageName === args.packageFilter ||
|
||||
node.location?.includes(args.packageFilter)
|
||||
);
|
||||
if (!nodeInfo) {
|
||||
throw new Error(`Node ${args.nodeType} not found`);
|
||||
}
|
||||
|
||||
if (args.limit) {
|
||||
nodesToExtract = nodesToExtract.slice(0, args.limit);
|
||||
}
|
||||
|
||||
logger.info(`Extracting ${nodesToExtract.length} nodes...`);
|
||||
|
||||
const extractedNodes = [];
|
||||
const errors = [];
|
||||
|
||||
for (const node of nodesToExtract) {
|
||||
try {
|
||||
const nodeType = node.packageName ? `${node.packageName}.${node.name}` : node.name;
|
||||
const nodeInfo = await this.nodeExtractor.extractNodeSource(nodeType);
|
||||
await this.nodeStorage.storeNode(nodeInfo);
|
||||
extractedNodes.push(nodeType);
|
||||
} catch (error) {
|
||||
errors.push({
|
||||
node: node.name,
|
||||
error: error instanceof Error ? error.message : 'Unknown error'
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const stats = await this.nodeStorage.getStatistics();
|
||||
|
||||
return {
|
||||
success: true,
|
||||
extracted: extractedNodes.length,
|
||||
failed: errors.length,
|
||||
totalStored: stats.totalNodes,
|
||||
errors: errors.slice(0, 10), // Limit error list
|
||||
statistics: stats
|
||||
nodeType: nodeInfo.nodeType,
|
||||
name: nodeInfo.name,
|
||||
displayName: nodeInfo.displayName,
|
||||
description: nodeInfo.description,
|
||||
category: nodeInfo.category,
|
||||
subcategory: nodeInfo.subcategory,
|
||||
icon: nodeInfo.icon,
|
||||
documentation: {
|
||||
markdown: nodeInfo.documentationMarkdown,
|
||||
url: nodeInfo.documentationUrl,
|
||||
title: nodeInfo.documentationTitle,
|
||||
},
|
||||
operations: nodeInfo.operations || [],
|
||||
apiMethods: nodeInfo.apiMethods || [],
|
||||
examples: nodeInfo.documentationExamples || [],
|
||||
templates: nodeInfo.templates || [],
|
||||
relatedResources: nodeInfo.relatedResources || [],
|
||||
requiredScopes: nodeInfo.requiredScopes || [],
|
||||
exampleWorkflow: nodeInfo.exampleWorkflow,
|
||||
exampleParameters: nodeInfo.exampleParameters,
|
||||
propertiesSchema: nodeInfo.propertiesSchema,
|
||||
metadata: {
|
||||
packageName: nodeInfo.packageName,
|
||||
version: nodeInfo.version,
|
||||
hasCredentials: nodeInfo.hasCredentials,
|
||||
isTrigger: nodeInfo.isTrigger,
|
||||
isWebhook: nodeInfo.isWebhook,
|
||||
aliases: nodeInfo.aliases,
|
||||
},
|
||||
sourceCode: {
|
||||
node: nodeInfo.sourceCode,
|
||||
credential: nodeInfo.credentialCode,
|
||||
},
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error(`Failed to extract all nodes`, error);
|
||||
throw new Error(`Failed to extract all nodes: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
logger.error(`Failed to get node info`, error);
|
||||
throw new Error(`Failed to get node info: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
}
|
||||
}
|
||||
|
||||
private async searchNodes(args: any): Promise<any> {
|
||||
try {
|
||||
logger.info(`Searching nodes`, args);
|
||||
|
||||
const results = await this.nodeStorage.searchNodes({
|
||||
logger.info('Searching nodes with enhanced filtering', args);
|
||||
const results = await this.nodeDocService.searchNodes({
|
||||
query: args.query,
|
||||
category: args.category,
|
||||
packageName: args.packageName,
|
||||
hasCredentials: args.hasCredentials,
|
||||
limit: args.limit || 20
|
||||
isTrigger: args.isTrigger,
|
||||
limit: args.limit || 20,
|
||||
});
|
||||
|
||||
return {
|
||||
nodes: results.map(node => ({
|
||||
nodeType: node.nodeType,
|
||||
name: node.name,
|
||||
packageName: node.packageName,
|
||||
displayName: node.displayName,
|
||||
description: node.description,
|
||||
codeLength: node.codeLength,
|
||||
category: node.category,
|
||||
packageName: node.packageName,
|
||||
hasDocumentation: !!node.documentationMarkdown,
|
||||
hasExamples: !!(node.documentationExamples && node.documentationExamples.length > 0),
|
||||
operationCount: node.operations?.length || 0,
|
||||
metadata: {
|
||||
hasCredentials: node.hasCredentials,
|
||||
location: node.sourceLocation
|
||||
isTrigger: node.isTrigger,
|
||||
isWebhook: node.isWebhook,
|
||||
},
|
||||
})),
|
||||
total: results.length
|
||||
total: results.length,
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error(`Failed to search nodes`, error);
|
||||
@@ -411,12 +416,11 @@ export class N8NMCPServer {
|
||||
private async getNodeStatistics(args: any): Promise<any> {
|
||||
try {
|
||||
logger.info(`Getting node statistics`);
|
||||
const stats = await this.nodeStorage.getStatistics();
|
||||
const stats = this.nodeDocService.getStatistics();
|
||||
|
||||
return {
|
||||
...stats,
|
||||
formattedTotalSize: `${(stats.totalCodeSize / 1024 / 1024).toFixed(2)} MB`,
|
||||
formattedAverageSize: `${(stats.averageNodeSize / 1024).toFixed(2)} KB`
|
||||
formattedTotalSize: stats.totalCodeSize ? `${(stats.totalCodeSize / 1024 / 1024).toFixed(2)} MB` : '0 MB',
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error(`Failed to get node statistics`, error);
|
||||
@@ -424,6 +428,23 @@ export class N8NMCPServer {
|
||||
}
|
||||
}
|
||||
|
||||
private async rebuildDocumentationDatabase(args: any): Promise<any> {
|
||||
try {
|
||||
logger.info('Rebuilding documentation database', args);
|
||||
const stats = await this.nodeDocService.rebuildDatabase();
|
||||
|
||||
return {
|
||||
success: true,
|
||||
message: 'Documentation database rebuilt successfully',
|
||||
statistics: stats,
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error(`Failed to rebuild documentation database`, error);
|
||||
throw new Error(`Failed to rebuild documentation database: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
async start(): Promise<void> {
|
||||
try {
|
||||
logger.info('Starting n8n MCP server...');
|
||||
|
||||
@@ -1,144 +0,0 @@
|
||||
import { ToolDefinition } from '../types';
|
||||
|
||||
/**
|
||||
* Simplified MCP tools focused on serving n8n node documentation and code
|
||||
*/
|
||||
export const nodeDocumentationTools: ToolDefinition[] = [
|
||||
{
|
||||
name: 'list_nodes',
|
||||
description: 'List all available n8n nodes with basic information',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
category: {
|
||||
type: 'string',
|
||||
description: 'Filter by category (e.g., "Core Nodes", "Flow", "Data Transformation")',
|
||||
},
|
||||
packageName: {
|
||||
type: 'string',
|
||||
description: 'Filter by package name (e.g., "n8n-nodes-base")',
|
||||
},
|
||||
isTrigger: {
|
||||
type: 'boolean',
|
||||
description: 'Filter to show only trigger nodes',
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'get_node_info',
|
||||
description: 'Get complete information about a specific n8n node including source code, documentation, and examples',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
nodeType: {
|
||||
type: 'string',
|
||||
description: 'The node type or name (e.g., "n8n-nodes-base.if", "If", "webhook")',
|
||||
},
|
||||
},
|
||||
required: ['nodeType'],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'search_nodes',
|
||||
description: 'Search for n8n nodes by name, description, or documentation content',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
query: {
|
||||
type: 'string',
|
||||
description: 'Search query (searches in node names, descriptions, and documentation)',
|
||||
},
|
||||
category: {
|
||||
type: 'string',
|
||||
description: 'Filter by category',
|
||||
},
|
||||
hasDocumentation: {
|
||||
type: 'boolean',
|
||||
description: 'Filter to show only nodes with documentation',
|
||||
},
|
||||
limit: {
|
||||
type: 'number',
|
||||
description: 'Maximum number of results to return',
|
||||
default: 20,
|
||||
},
|
||||
},
|
||||
required: ['query'],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'get_node_example',
|
||||
description: 'Get example workflow/usage for a specific n8n node',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
nodeType: {
|
||||
type: 'string',
|
||||
description: 'The node type or name',
|
||||
},
|
||||
},
|
||||
required: ['nodeType'],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'get_node_source_code',
|
||||
description: 'Get only the source code of a specific n8n node',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
nodeType: {
|
||||
type: 'string',
|
||||
description: 'The node type or name',
|
||||
},
|
||||
includeCredentials: {
|
||||
type: 'boolean',
|
||||
description: 'Include credential type definitions if available',
|
||||
default: false,
|
||||
},
|
||||
},
|
||||
required: ['nodeType'],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'get_node_documentation',
|
||||
description: 'Get only the documentation for a specific n8n node',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
nodeType: {
|
||||
type: 'string',
|
||||
description: 'The node type or name',
|
||||
},
|
||||
format: {
|
||||
type: 'string',
|
||||
enum: ['markdown', 'plain'],
|
||||
description: 'Documentation format',
|
||||
default: 'markdown',
|
||||
},
|
||||
},
|
||||
required: ['nodeType'],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'rebuild_database',
|
||||
description: 'Rebuild the entire node database with latest information from n8n and documentation',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
includeDocumentation: {
|
||||
type: 'boolean',
|
||||
description: 'Include documentation from n8n-docs repository',
|
||||
default: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'get_database_statistics',
|
||||
description: 'Get statistics about the node database',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {},
|
||||
},
|
||||
},
|
||||
];
|
||||
@@ -182,31 +182,40 @@ export const n8nTools: ToolDefinition[] = [
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'extract_all_nodes',
|
||||
description: 'Extract and store all available n8n nodes in the database',
|
||||
name: 'get_node_statistics',
|
||||
description: 'Get statistics about stored n8n nodes',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'get_node_info',
|
||||
description: 'Get comprehensive information about a specific n8n node including documentation, operations, API methods, and examples',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
packageFilter: {
|
||||
nodeType: {
|
||||
type: 'string',
|
||||
description: 'Optional package name to filter extraction',
|
||||
},
|
||||
limit: {
|
||||
type: 'number',
|
||||
description: 'Maximum number of nodes to extract',
|
||||
description: 'The node type identifier (e.g., n8n-nodes-base.slack)',
|
||||
},
|
||||
},
|
||||
required: ['nodeType'],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'search_nodes',
|
||||
description: 'Search for n8n nodes by name, package, or functionality',
|
||||
description: 'Search n8n nodes with full-text search and advanced filtering',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
query: {
|
||||
type: 'string',
|
||||
description: 'Search query',
|
||||
description: 'Search query for full-text search',
|
||||
},
|
||||
category: {
|
||||
type: 'string',
|
||||
description: 'Filter by node category',
|
||||
},
|
||||
packageName: {
|
||||
type: 'string',
|
||||
@@ -214,7 +223,11 @@ export const n8nTools: ToolDefinition[] = [
|
||||
},
|
||||
hasCredentials: {
|
||||
type: 'boolean',
|
||||
description: 'Filter nodes that have credentials',
|
||||
description: 'Filter nodes that require credentials',
|
||||
},
|
||||
isTrigger: {
|
||||
type: 'boolean',
|
||||
description: 'Filter trigger nodes only',
|
||||
},
|
||||
limit: {
|
||||
type: 'number',
|
||||
@@ -225,11 +238,16 @@ export const n8nTools: ToolDefinition[] = [
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'get_node_statistics',
|
||||
description: 'Get statistics about stored n8n nodes',
|
||||
name: 'rebuild_documentation_database',
|
||||
description: 'Rebuild the node documentation database with the latest information',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {},
|
||||
properties: {
|
||||
packageFilter: {
|
||||
type: 'string',
|
||||
description: 'Optional: Only rebuild nodes from specific package',
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
];
|
||||
@@ -1,67 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { NodeDocumentationService } from '../services/node-documentation-service';
|
||||
import { logger } from '../utils/logger';
|
||||
|
||||
async function rebuildDatabase() {
|
||||
console.log('🔄 Starting complete database rebuild...\n');
|
||||
|
||||
const service = new NodeDocumentationService();
|
||||
|
||||
try {
|
||||
const startTime = Date.now();
|
||||
|
||||
console.log('1️⃣ Initializing services...');
|
||||
console.log('2️⃣ Fetching n8n-docs repository...');
|
||||
console.log('3️⃣ Discovering available nodes...');
|
||||
console.log('4️⃣ Extracting node information...\n');
|
||||
|
||||
const stats = await service.rebuildDatabase();
|
||||
|
||||
const duration = ((Date.now() - startTime) / 1000).toFixed(2);
|
||||
|
||||
console.log('\n📊 Rebuild Results:');
|
||||
console.log(` Total nodes processed: ${stats.total}`);
|
||||
console.log(` Successfully stored: ${stats.successful}`);
|
||||
console.log(` Failed: ${stats.failed}`);
|
||||
console.log(` Duration: ${duration}s`);
|
||||
|
||||
if (stats.errors.length > 0) {
|
||||
console.log('\n⚠️ First 5 errors:');
|
||||
stats.errors.slice(0, 5).forEach(error => {
|
||||
console.log(` - ${error}`);
|
||||
});
|
||||
if (stats.errors.length > 5) {
|
||||
console.log(` ... and ${stats.errors.length - 5} more errors`);
|
||||
}
|
||||
}
|
||||
|
||||
// Get final statistics
|
||||
const dbStats = service.getStatistics();
|
||||
console.log('\n📈 Database Statistics:');
|
||||
console.log(` Total nodes: ${dbStats.totalNodes}`);
|
||||
console.log(` Nodes with documentation: ${dbStats.nodesWithDocs}`);
|
||||
console.log(` Nodes with examples: ${dbStats.nodesWithExamples}`);
|
||||
console.log(` Trigger nodes: ${dbStats.triggerNodes}`);
|
||||
console.log(` Webhook nodes: ${dbStats.webhookNodes}`);
|
||||
console.log(` Total packages: ${dbStats.totalPackages}`);
|
||||
|
||||
console.log('\n✨ Database rebuild complete!');
|
||||
|
||||
} catch (error) {
|
||||
console.error('\n❌ Database rebuild failed:', error);
|
||||
process.exit(1);
|
||||
} finally {
|
||||
service.close();
|
||||
}
|
||||
}
|
||||
|
||||
// Run if called directly
|
||||
if (require.main === module) {
|
||||
rebuildDatabase().catch(error => {
|
||||
console.error(error);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
|
||||
export { rebuildDatabase };
|
||||
@@ -1,129 +1,79 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
import { NodeSourceExtractor } from '../utils/node-source-extractor';
|
||||
import { SQLiteStorageService } from '../services/sqlite-storage-service';
|
||||
import { NodeDocumentationService } from '../services/node-documentation-service';
|
||||
import { logger } from '../utils/logger';
|
||||
import * as path from 'path';
|
||||
|
||||
/**
|
||||
* Rebuild the entire nodes database by extracting all available nodes
|
||||
* Rebuild the enhanced documentation database
|
||||
*/
|
||||
async function rebuildDatabase() {
|
||||
console.log('🔄 Starting database rebuild...\n');
|
||||
async function rebuildDocumentationDatabase() {
|
||||
console.log('🔄 Starting enhanced documentation database rebuild...\n');
|
||||
|
||||
const startTime = Date.now();
|
||||
const extractor = new NodeSourceExtractor();
|
||||
const storage = new SQLiteStorageService();
|
||||
const service = new NodeDocumentationService();
|
||||
|
||||
try {
|
||||
// Step 1: Clear existing database
|
||||
console.log('1️⃣ Clearing existing database...');
|
||||
await storage.rebuildDatabase();
|
||||
// Run the rebuild
|
||||
const results = await service.rebuildDatabase();
|
||||
|
||||
// Step 2: Get all available nodes
|
||||
console.log('2️⃣ Discovering available nodes...');
|
||||
const allNodes = await extractor.listAvailableNodes();
|
||||
console.log(` Found ${allNodes.length} nodes\n`);
|
||||
|
||||
// Step 3: Extract and store each node
|
||||
console.log('3️⃣ Extracting and storing nodes...');
|
||||
let processed = 0;
|
||||
let stored = 0;
|
||||
let failed = 0;
|
||||
const errors: Array<{ node: string; error: string }> = [];
|
||||
|
||||
// Process in batches for better performance
|
||||
const batchSize = 50;
|
||||
for (let i = 0; i < allNodes.length; i += batchSize) {
|
||||
const batch = allNodes.slice(i, Math.min(i + batchSize, allNodes.length));
|
||||
const nodeInfos = [];
|
||||
|
||||
for (const node of batch) {
|
||||
processed++;
|
||||
|
||||
try {
|
||||
const nodeType = node.packageName ? `${node.packageName}.${node.name}` : node.name;
|
||||
|
||||
// Show progress
|
||||
if (processed % 100 === 0) {
|
||||
const progress = ((processed / allNodes.length) * 100).toFixed(1);
|
||||
console.log(` Progress: ${processed}/${allNodes.length} (${progress}%)`);
|
||||
}
|
||||
|
||||
const nodeInfo = await extractor.extractNodeSource(nodeType);
|
||||
nodeInfos.push(nodeInfo);
|
||||
stored++;
|
||||
} catch (error) {
|
||||
failed++;
|
||||
const errorMsg = error instanceof Error ? error.message : 'Unknown error';
|
||||
errors.push({
|
||||
node: node.name,
|
||||
error: errorMsg
|
||||
});
|
||||
|
||||
// Log first few errors
|
||||
if (errors.length <= 5) {
|
||||
logger.debug(`Failed to extract ${node.name}: ${errorMsg}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Bulk store the batch
|
||||
if (nodeInfos.length > 0) {
|
||||
await storage.bulkStoreNodes(nodeInfos);
|
||||
}
|
||||
}
|
||||
|
||||
// Step 4: Save statistics
|
||||
console.log('\n4️⃣ Saving statistics...');
|
||||
const stats = await storage.getStatistics();
|
||||
await storage.saveExtractionStats(stats);
|
||||
|
||||
// Step 5: Display results
|
||||
const duration = ((Date.now() - startTime) / 1000).toFixed(2);
|
||||
|
||||
console.log('\n✅ Database rebuild completed!\n');
|
||||
console.log('\n✅ Enhanced documentation database rebuild completed!\n');
|
||||
console.log('📊 Results:');
|
||||
console.log(` Total nodes found: ${allNodes.length}`);
|
||||
console.log(` Successfully stored: ${stored}`);
|
||||
console.log(` Failed: ${failed}`);
|
||||
console.log(` Total nodes found: ${results.total}`);
|
||||
console.log(` Successfully processed: ${results.successful}`);
|
||||
console.log(` Failed: ${results.failed}`);
|
||||
console.log(` Duration: ${duration}s`);
|
||||
console.log(` Database size: ${(stats.totalCodeSize / 1024 / 1024).toFixed(2)} MB`);
|
||||
|
||||
if (results.errors.length > 0) {
|
||||
console.log(`\n⚠️ First ${Math.min(5, results.errors.length)} errors:`);
|
||||
results.errors.slice(0, 5).forEach(err => {
|
||||
console.log(` - ${err}`);
|
||||
});
|
||||
|
||||
if (results.errors.length > 5) {
|
||||
console.log(` ... and ${results.errors.length - 5} more errors`);
|
||||
}
|
||||
}
|
||||
|
||||
// Get and display statistics
|
||||
const stats = service.getStatistics();
|
||||
console.log('\n📈 Database Statistics:');
|
||||
console.log(` Total nodes: ${stats.totalNodes}`);
|
||||
console.log(` Nodes with documentation: ${stats.nodesWithDocs}`);
|
||||
console.log(` Nodes with examples: ${stats.nodesWithExamples}`);
|
||||
console.log(` Nodes with credentials: ${stats.nodesWithCredentials}`);
|
||||
console.log(` Trigger nodes: ${stats.triggerNodes}`);
|
||||
console.log(` Webhook nodes: ${stats.webhookNodes}`);
|
||||
|
||||
console.log('\n📦 Package distribution:');
|
||||
stats.packageDistribution.slice(0, 10).forEach(pkg => {
|
||||
stats.packageDistribution.slice(0, 10).forEach((pkg: any) => {
|
||||
console.log(` ${pkg.package}: ${pkg.count} nodes`);
|
||||
});
|
||||
|
||||
if (errors.length > 0) {
|
||||
console.log(`\n⚠️ First ${Math.min(5, errors.length)} errors:`);
|
||||
errors.slice(0, 5).forEach(err => {
|
||||
console.log(` - ${err.node}: ${err.error}`);
|
||||
});
|
||||
|
||||
if (errors.length > 5) {
|
||||
console.log(` ... and ${errors.length - 5} more errors`);
|
||||
}
|
||||
}
|
||||
|
||||
// Close database connection
|
||||
storage.close();
|
||||
service.close();
|
||||
|
||||
console.log('\n✨ Database is ready for use!');
|
||||
console.log('\n✨ Enhanced documentation database is ready!');
|
||||
console.log('💡 The database now includes:');
|
||||
console.log(' - Complete node source code');
|
||||
console.log(' - Enhanced documentation with operations and API methods');
|
||||
console.log(' - Code examples and templates');
|
||||
console.log(' - Related resources and required scopes');
|
||||
|
||||
} catch (error) {
|
||||
console.error('\n❌ Database rebuild failed:', error);
|
||||
storage.close();
|
||||
console.error('\n❌ Documentation database rebuild failed:', error);
|
||||
service.close();
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Run if called directly
|
||||
if (require.main === module) {
|
||||
rebuildDatabase().catch(error => {
|
||||
rebuildDocumentationDatabase().catch(error => {
|
||||
console.error('Fatal error:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
|
||||
export { rebuildDatabase };
|
||||
export { rebuildDocumentationDatabase };
|
||||
@@ -4,7 +4,15 @@ import path from 'path';
|
||||
import { promises as fs } from 'fs';
|
||||
import { logger } from '../utils/logger';
|
||||
import { NodeSourceExtractor } from '../utils/node-source-extractor';
|
||||
import { DocumentationFetcher } from '../utils/documentation-fetcher';
|
||||
import {
|
||||
EnhancedDocumentationFetcher,
|
||||
EnhancedNodeDocumentation,
|
||||
OperationInfo,
|
||||
ApiMethodMapping,
|
||||
CodeExample,
|
||||
TemplateInfo,
|
||||
RelatedResource
|
||||
} from '../utils/enhanced-documentation-fetcher';
|
||||
import { ExampleGenerator } from '../utils/example-generator';
|
||||
|
||||
interface NodeInfo {
|
||||
@@ -17,8 +25,15 @@ interface NodeInfo {
|
||||
icon?: string;
|
||||
sourceCode: string;
|
||||
credentialCode?: string;
|
||||
documentation?: string;
|
||||
documentationMarkdown?: string;
|
||||
documentationUrl?: string;
|
||||
documentationTitle?: string;
|
||||
operations?: OperationInfo[];
|
||||
apiMethods?: ApiMethodMapping[];
|
||||
documentationExamples?: CodeExample[];
|
||||
templates?: TemplateInfo[];
|
||||
relatedResources?: RelatedResource[];
|
||||
requiredScopes?: string[];
|
||||
exampleWorkflow?: any;
|
||||
exampleParameters?: any;
|
||||
propertiesSchema?: any;
|
||||
@@ -44,7 +59,7 @@ interface SearchOptions {
|
||||
export class NodeDocumentationService {
|
||||
private db: Database.Database;
|
||||
private extractor: NodeSourceExtractor;
|
||||
private docsFetcher: DocumentationFetcher;
|
||||
private docsFetcher: EnhancedDocumentationFetcher;
|
||||
|
||||
constructor(dbPath?: string) {
|
||||
const databasePath = dbPath || process.env.NODE_DB_PATH || path.join(process.cwd(), 'data', 'nodes-v2.db');
|
||||
@@ -57,7 +72,7 @@ export class NodeDocumentationService {
|
||||
|
||||
this.db = new Database(databasePath);
|
||||
this.extractor = new NodeSourceExtractor();
|
||||
this.docsFetcher = new DocumentationFetcher();
|
||||
this.docsFetcher = new EnhancedDocumentationFetcher();
|
||||
|
||||
// Initialize database with new schema
|
||||
this.initializeDatabase();
|
||||
@@ -88,6 +103,15 @@ CREATE TABLE IF NOT EXISTS nodes (
|
||||
-- Documentation
|
||||
documentation_markdown TEXT,
|
||||
documentation_url TEXT,
|
||||
documentation_title TEXT,
|
||||
|
||||
-- Enhanced documentation fields (stored as JSON)
|
||||
operations TEXT,
|
||||
api_methods TEXT,
|
||||
documentation_examples TEXT,
|
||||
templates TEXT,
|
||||
related_resources TEXT,
|
||||
required_scopes TEXT,
|
||||
|
||||
-- Example usage
|
||||
example_workflow TEXT,
|
||||
@@ -182,14 +206,16 @@ CREATE TABLE IF NOT EXISTS extraction_stats (
|
||||
INSERT OR REPLACE INTO nodes (
|
||||
node_type, name, display_name, description, category, subcategory, icon,
|
||||
source_code, credential_code, code_hash, code_length,
|
||||
documentation_markdown, documentation_url,
|
||||
documentation_markdown, documentation_url, documentation_title,
|
||||
operations, api_methods, documentation_examples, templates, related_resources, required_scopes,
|
||||
example_workflow, example_parameters, properties_schema,
|
||||
package_name, version, codex_data, aliases,
|
||||
has_credentials, is_trigger, is_webhook
|
||||
) VALUES (
|
||||
@nodeType, @name, @displayName, @description, @category, @subcategory, @icon,
|
||||
@sourceCode, @credentialCode, @hash, @codeLength,
|
||||
@documentation, @documentationUrl,
|
||||
@documentation, @documentationUrl, @documentationTitle,
|
||||
@operations, @apiMethods, @documentationExamples, @templates, @relatedResources, @requiredScopes,
|
||||
@exampleWorkflow, @exampleParameters, @propertiesSchema,
|
||||
@packageName, @version, @codexData, @aliases,
|
||||
@hasCredentials, @isTrigger, @isWebhook
|
||||
@@ -208,8 +234,15 @@ CREATE TABLE IF NOT EXISTS extraction_stats (
|
||||
credentialCode: nodeInfo.credentialCode || null,
|
||||
hash,
|
||||
codeLength: nodeInfo.sourceCode.length,
|
||||
documentation: nodeInfo.documentation || null,
|
||||
documentation: nodeInfo.documentationMarkdown || null,
|
||||
documentationUrl: nodeInfo.documentationUrl || null,
|
||||
documentationTitle: nodeInfo.documentationTitle || null,
|
||||
operations: nodeInfo.operations ? JSON.stringify(nodeInfo.operations) : null,
|
||||
apiMethods: nodeInfo.apiMethods ? JSON.stringify(nodeInfo.apiMethods) : null,
|
||||
documentationExamples: nodeInfo.documentationExamples ? JSON.stringify(nodeInfo.documentationExamples) : null,
|
||||
templates: nodeInfo.templates ? JSON.stringify(nodeInfo.templates) : null,
|
||||
relatedResources: nodeInfo.relatedResources ? JSON.stringify(nodeInfo.relatedResources) : null,
|
||||
requiredScopes: nodeInfo.requiredScopes ? JSON.stringify(nodeInfo.requiredScopes) : null,
|
||||
exampleWorkflow: nodeInfo.exampleWorkflow ? JSON.stringify(nodeInfo.exampleWorkflow) : null,
|
||||
exampleParameters: nodeInfo.exampleParameters ? JSON.stringify(nodeInfo.exampleParameters) : null,
|
||||
propertiesSchema: nodeInfo.propertiesSchema ? JSON.stringify(nodeInfo.propertiesSchema) : null,
|
||||
@@ -346,13 +379,13 @@ CREATE TABLE IF NOT EXISTS extraction_stats (
|
||||
// Parse node definition to get metadata
|
||||
const nodeDefinition = this.parseNodeDefinition(nodeData.sourceCode);
|
||||
|
||||
// Get documentation
|
||||
const docs = await this.docsFetcher.getNodeDocumentation(nodeType);
|
||||
// Get enhanced documentation
|
||||
const enhancedDocs = await this.docsFetcher.getEnhancedNodeDocumentation(nodeType);
|
||||
|
||||
// Generate example
|
||||
const example = ExampleGenerator.generateFromNodeDefinition(nodeDefinition);
|
||||
|
||||
// Prepare node info
|
||||
// Prepare node info with enhanced documentation
|
||||
const nodeInfo: NodeInfo = {
|
||||
nodeType: nodeType,
|
||||
name: node.name,
|
||||
@@ -363,8 +396,15 @@ CREATE TABLE IF NOT EXISTS extraction_stats (
|
||||
icon: nodeDefinition.icon,
|
||||
sourceCode: nodeData.sourceCode,
|
||||
credentialCode: nodeData.credentialCode,
|
||||
documentation: docs?.markdown,
|
||||
documentationUrl: docs?.url,
|
||||
documentationMarkdown: enhancedDocs?.markdown,
|
||||
documentationUrl: enhancedDocs?.url,
|
||||
documentationTitle: enhancedDocs?.title,
|
||||
operations: enhancedDocs?.operations,
|
||||
apiMethods: enhancedDocs?.apiMethods,
|
||||
documentationExamples: enhancedDocs?.examples,
|
||||
templates: enhancedDocs?.templates,
|
||||
relatedResources: enhancedDocs?.relatedResources,
|
||||
requiredScopes: enhancedDocs?.requiredScopes,
|
||||
exampleWorkflow: example,
|
||||
exampleParameters: example.nodes[0]?.parameters,
|
||||
propertiesSchema: nodeDefinition.properties,
|
||||
@@ -410,28 +450,88 @@ CREATE TABLE IF NOT EXISTS extraction_stats (
|
||||
* Parse node definition from source code
|
||||
*/
|
||||
private parseNodeDefinition(sourceCode: string): any {
|
||||
try {
|
||||
// Try to extract the description object from the source
|
||||
const descMatch = sourceCode.match(/description\s*[:=]\s*({[\s\S]*?})\s*[,;]/);
|
||||
if (descMatch) {
|
||||
// Clean up the match and try to parse it
|
||||
const descStr = descMatch[1]
|
||||
.replace(/(['"])?([a-zA-Z0-9_]+)(['"])?\s*:/g, '"$2":') // Quote property names
|
||||
.replace(/:\s*'([^']*)'/g, ': "$1"') // Convert single quotes to double
|
||||
.replace(/,\s*}/g, '}'); // Remove trailing commas
|
||||
|
||||
return JSON.parse(descStr);
|
||||
}
|
||||
} catch (error) {
|
||||
logger.debug('Failed to parse node definition:', error);
|
||||
}
|
||||
|
||||
// Return minimal definition if parsing fails
|
||||
return {
|
||||
const result: any = {
|
||||
displayName: '',
|
||||
description: '',
|
||||
properties: []
|
||||
properties: [],
|
||||
category: null,
|
||||
subcategory: null,
|
||||
icon: null,
|
||||
version: null,
|
||||
codex: null,
|
||||
alias: null
|
||||
};
|
||||
|
||||
try {
|
||||
// Extract individual properties using specific patterns
|
||||
|
||||
// Display name
|
||||
const displayNameMatch = sourceCode.match(/displayName\s*[:=]\s*['"`]([^'"`]+)['"`]/);
|
||||
if (displayNameMatch) {
|
||||
result.displayName = displayNameMatch[1];
|
||||
}
|
||||
|
||||
// Description
|
||||
const descriptionMatch = sourceCode.match(/description\s*[:=]\s*['"`]([^'"`]+)['"`]/);
|
||||
if (descriptionMatch) {
|
||||
result.description = descriptionMatch[1];
|
||||
}
|
||||
|
||||
// Icon
|
||||
const iconMatch = sourceCode.match(/icon\s*[:=]\s*['"`]([^'"`]+)['"`]/);
|
||||
if (iconMatch) {
|
||||
result.icon = iconMatch[1];
|
||||
}
|
||||
|
||||
// Category/group
|
||||
const groupMatch = sourceCode.match(/group\s*[:=]\s*\[['"`]([^'"`]+)['"`]\]/);
|
||||
if (groupMatch) {
|
||||
result.category = groupMatch[1];
|
||||
}
|
||||
|
||||
// Version
|
||||
const versionMatch = sourceCode.match(/version\s*[:=]\s*(\d+)/);
|
||||
if (versionMatch) {
|
||||
result.version = parseInt(versionMatch[1]);
|
||||
}
|
||||
|
||||
// Subtitle
|
||||
const subtitleMatch = sourceCode.match(/subtitle\s*[:=]\s*['"`]([^'"`]+)['"`]/);
|
||||
if (subtitleMatch) {
|
||||
result.subtitle = subtitleMatch[1];
|
||||
}
|
||||
|
||||
// Try to extract properties array
|
||||
const propsMatch = sourceCode.match(/properties\s*[:=]\s*(\[[\s\S]*?\])\s*[,}]/);
|
||||
if (propsMatch) {
|
||||
try {
|
||||
// This is complex to parse from minified code, so we'll skip for now
|
||||
result.properties = [];
|
||||
} catch (e) {
|
||||
// Ignore parsing errors
|
||||
}
|
||||
}
|
||||
|
||||
// Check if it's a trigger node
|
||||
if (sourceCode.includes('implements.*ITrigger') ||
|
||||
sourceCode.includes('polling:.*true') ||
|
||||
sourceCode.includes('webhook:.*true') ||
|
||||
result.displayName.toLowerCase().includes('trigger')) {
|
||||
result.isTrigger = true;
|
||||
}
|
||||
|
||||
// Check if it's a webhook node
|
||||
if (sourceCode.includes('webhooks:') ||
|
||||
sourceCode.includes('webhook:.*true') ||
|
||||
result.displayName.toLowerCase().includes('webhook')) {
|
||||
result.isWebhook = true;
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
logger.debug('Error parsing node definition:', error);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -448,8 +548,15 @@ CREATE TABLE IF NOT EXISTS extraction_stats (
|
||||
icon: row.icon,
|
||||
sourceCode: row.source_code,
|
||||
credentialCode: row.credential_code,
|
||||
documentation: row.documentation_markdown,
|
||||
documentationMarkdown: row.documentation_markdown,
|
||||
documentationUrl: row.documentation_url,
|
||||
documentationTitle: row.documentation_title,
|
||||
operations: row.operations ? JSON.parse(row.operations) : null,
|
||||
apiMethods: row.api_methods ? JSON.parse(row.api_methods) : null,
|
||||
documentationExamples: row.documentation_examples ? JSON.parse(row.documentation_examples) : null,
|
||||
templates: row.templates ? JSON.parse(row.templates) : null,
|
||||
relatedResources: row.related_resources ? JSON.parse(row.related_resources) : null,
|
||||
requiredScopes: row.required_scopes ? JSON.parse(row.required_scopes) : null,
|
||||
exampleWorkflow: row.example_workflow ? JSON.parse(row.example_workflow) : null,
|
||||
exampleParameters: row.example_parameters ? JSON.parse(row.example_parameters) : null,
|
||||
propertiesSchema: row.properties_schema ? JSON.parse(row.properties_schema) : null,
|
||||
|
||||
@@ -1,274 +0,0 @@
|
||||
import { NodeSourceInfo } from '../utils/node-source-extractor';
|
||||
import { logger } from '../utils/logger';
|
||||
import * as crypto from 'crypto';
|
||||
|
||||
export interface StoredNode {
|
||||
id: string;
|
||||
nodeType: string;
|
||||
name: string;
|
||||
packageName: string;
|
||||
displayName?: string;
|
||||
description?: string;
|
||||
codeHash: string;
|
||||
codeLength: number;
|
||||
sourceLocation: string;
|
||||
hasCredentials: boolean;
|
||||
extractedAt: Date;
|
||||
updatedAt: Date;
|
||||
sourceCode?: string;
|
||||
credentialCode?: string;
|
||||
packageInfo?: any;
|
||||
metadata?: Record<string, any>;
|
||||
}
|
||||
|
||||
export interface NodeSearchQuery {
|
||||
query?: string;
|
||||
packageName?: string;
|
||||
nodeType?: string;
|
||||
hasCredentials?: boolean;
|
||||
limit?: number;
|
||||
offset?: number;
|
||||
}
|
||||
|
||||
export class NodeStorageService {
|
||||
private nodes: Map<string, StoredNode> = new Map();
|
||||
private nodesByPackage: Map<string, Set<string>> = new Map();
|
||||
private searchIndex: Map<string, Set<string>> = new Map();
|
||||
|
||||
/**
|
||||
* Store a node in the database
|
||||
*/
|
||||
async storeNode(nodeInfo: NodeSourceInfo): Promise<StoredNode> {
|
||||
const codeHash = crypto.createHash('sha256').update(nodeInfo.sourceCode).digest('hex');
|
||||
|
||||
// Parse display name and description from source if possible
|
||||
const displayName = this.extractDisplayName(nodeInfo.sourceCode);
|
||||
const description = this.extractDescription(nodeInfo.sourceCode);
|
||||
|
||||
const storedNode: StoredNode = {
|
||||
id: crypto.randomUUID(),
|
||||
nodeType: nodeInfo.nodeType,
|
||||
name: nodeInfo.nodeType.split('.').pop() || nodeInfo.nodeType,
|
||||
packageName: nodeInfo.nodeType.split('.')[0] || 'unknown',
|
||||
displayName,
|
||||
description,
|
||||
codeHash,
|
||||
codeLength: nodeInfo.sourceCode.length,
|
||||
sourceLocation: nodeInfo.location,
|
||||
hasCredentials: !!nodeInfo.credentialCode,
|
||||
extractedAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
sourceCode: nodeInfo.sourceCode,
|
||||
credentialCode: nodeInfo.credentialCode,
|
||||
packageInfo: nodeInfo.packageInfo,
|
||||
};
|
||||
|
||||
// Store in memory (replace with real DB)
|
||||
this.nodes.set(nodeInfo.nodeType, storedNode);
|
||||
|
||||
// Update package index
|
||||
if (!this.nodesByPackage.has(storedNode.packageName)) {
|
||||
this.nodesByPackage.set(storedNode.packageName, new Set());
|
||||
}
|
||||
this.nodesByPackage.get(storedNode.packageName)!.add(nodeInfo.nodeType);
|
||||
|
||||
// Update search index
|
||||
this.updateSearchIndex(storedNode);
|
||||
|
||||
logger.info(`Stored node: ${nodeInfo.nodeType} (${codeHash.substring(0, 8)}...)`);
|
||||
return storedNode;
|
||||
}
|
||||
|
||||
/**
|
||||
* Search for nodes
|
||||
*/
|
||||
async searchNodes(query: NodeSearchQuery): Promise<StoredNode[]> {
|
||||
let results: StoredNode[] = [];
|
||||
|
||||
if (query.query) {
|
||||
// Text search
|
||||
const searchTerms = query.query.toLowerCase().split(' ');
|
||||
const matchingNodeTypes = new Set<string>();
|
||||
|
||||
for (const term of searchTerms) {
|
||||
const matches = this.searchIndex.get(term) || new Set();
|
||||
matches.forEach(nodeType => matchingNodeTypes.add(nodeType));
|
||||
}
|
||||
|
||||
results = Array.from(matchingNodeTypes)
|
||||
.map(nodeType => this.nodes.get(nodeType)!)
|
||||
.filter(Boolean);
|
||||
} else {
|
||||
// Get all nodes
|
||||
results = Array.from(this.nodes.values());
|
||||
}
|
||||
|
||||
// Apply filters
|
||||
if (query.packageName) {
|
||||
results = results.filter(node => node.packageName === query.packageName);
|
||||
}
|
||||
|
||||
if (query.nodeType) {
|
||||
results = results.filter(node => node.nodeType.includes(query.nodeType!));
|
||||
}
|
||||
|
||||
if (query.hasCredentials !== undefined) {
|
||||
results = results.filter(node => node.hasCredentials === query.hasCredentials);
|
||||
}
|
||||
|
||||
// Apply pagination
|
||||
const offset = query.offset || 0;
|
||||
const limit = query.limit || 50;
|
||||
|
||||
return results.slice(offset, offset + limit);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get node by type
|
||||
*/
|
||||
async getNode(nodeType: string): Promise<StoredNode | null> {
|
||||
return this.nodes.get(nodeType) || null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all packages
|
||||
*/
|
||||
async getPackages(): Promise<Array<{ name: string; nodeCount: number }>> {
|
||||
return Array.from(this.nodesByPackage.entries()).map(([name, nodes]) => ({
|
||||
name,
|
||||
nodeCount: nodes.size,
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Bulk store nodes
|
||||
*/
|
||||
async bulkStoreNodes(nodeInfos: NodeSourceInfo[]): Promise<{
|
||||
stored: number;
|
||||
failed: number;
|
||||
errors: Array<{ nodeType: string; error: string }>;
|
||||
}> {
|
||||
const results = {
|
||||
stored: 0,
|
||||
failed: 0,
|
||||
errors: [] as Array<{ nodeType: string; error: string }>,
|
||||
};
|
||||
|
||||
for (const nodeInfo of nodeInfos) {
|
||||
try {
|
||||
await this.storeNode(nodeInfo);
|
||||
results.stored++;
|
||||
} catch (error) {
|
||||
results.failed++;
|
||||
results.errors.push({
|
||||
nodeType: nodeInfo.nodeType,
|
||||
error: error instanceof Error ? error.message : 'Unknown error',
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate statistics
|
||||
*/
|
||||
async getStatistics(): Promise<{
|
||||
totalNodes: number;
|
||||
totalPackages: number;
|
||||
totalCodeSize: number;
|
||||
nodesWithCredentials: number;
|
||||
averageNodeSize: number;
|
||||
packageDistribution: Array<{ package: string; count: number }>;
|
||||
}> {
|
||||
const nodes = Array.from(this.nodes.values());
|
||||
const totalCodeSize = nodes.reduce((sum, node) => sum + node.codeLength, 0);
|
||||
const nodesWithCredentials = nodes.filter(node => node.hasCredentials).length;
|
||||
|
||||
const packageDistribution = Array.from(this.nodesByPackage.entries())
|
||||
.map(([pkg, nodeSet]) => ({ package: pkg, count: nodeSet.size }))
|
||||
.sort((a, b) => b.count - a.count);
|
||||
|
||||
return {
|
||||
totalNodes: nodes.length,
|
||||
totalPackages: this.nodesByPackage.size,
|
||||
totalCodeSize,
|
||||
nodesWithCredentials,
|
||||
averageNodeSize: nodes.length > 0 ? Math.round(totalCodeSize / nodes.length) : 0,
|
||||
packageDistribution,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract display name from source code
|
||||
*/
|
||||
private extractDisplayName(sourceCode: string): string | undefined {
|
||||
const match = sourceCode.match(/displayName:\s*["'`]([^"'`]+)["'`]/);
|
||||
return match ? match[1] : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract description from source code
|
||||
*/
|
||||
private extractDescription(sourceCode: string): string | undefined {
|
||||
const match = sourceCode.match(/description:\s*["'`]([^"'`]+)["'`]/);
|
||||
return match ? match[1] : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Update search index
|
||||
*/
|
||||
private updateSearchIndex(node: StoredNode): void {
|
||||
// Index by name parts
|
||||
const nameParts = node.name.toLowerCase().split(/(?=[A-Z])|[._-]/).filter(Boolean);
|
||||
for (const part of nameParts) {
|
||||
if (!this.searchIndex.has(part)) {
|
||||
this.searchIndex.set(part, new Set());
|
||||
}
|
||||
this.searchIndex.get(part)!.add(node.nodeType);
|
||||
}
|
||||
|
||||
// Index by display name
|
||||
if (node.displayName) {
|
||||
const displayParts = node.displayName.toLowerCase().split(/\s+/);
|
||||
for (const part of displayParts) {
|
||||
if (!this.searchIndex.has(part)) {
|
||||
this.searchIndex.set(part, new Set());
|
||||
}
|
||||
this.searchIndex.get(part)!.add(node.nodeType);
|
||||
}
|
||||
}
|
||||
|
||||
// Index by package name
|
||||
const pkgParts = node.packageName.toLowerCase().split(/[.-]/);
|
||||
for (const part of pkgParts) {
|
||||
if (!this.searchIndex.has(part)) {
|
||||
this.searchIndex.set(part, new Set());
|
||||
}
|
||||
this.searchIndex.get(part)!.add(node.nodeType);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Export all nodes for database import
|
||||
*/
|
||||
async exportForDatabase(): Promise<{
|
||||
nodes: StoredNode[];
|
||||
metadata: {
|
||||
exportedAt: Date;
|
||||
totalNodes: number;
|
||||
totalPackages: number;
|
||||
};
|
||||
}> {
|
||||
const nodes = Array.from(this.nodes.values());
|
||||
|
||||
return {
|
||||
nodes,
|
||||
metadata: {
|
||||
exportedAt: new Date(),
|
||||
totalNodes: nodes.length,
|
||||
totalPackages: this.nodesByPackage.size,
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -1,410 +0,0 @@
|
||||
import Database from 'better-sqlite3';
|
||||
import * as path from 'path';
|
||||
import * as fs from 'fs';
|
||||
import * as crypto from 'crypto';
|
||||
import { NodeSourceInfo } from '../utils/node-source-extractor';
|
||||
import { StoredNode, NodeSearchQuery } from './node-storage-service';
|
||||
import { logger } from '../utils/logger';
|
||||
|
||||
export class SQLiteStorageService {
|
||||
private db: Database.Database;
|
||||
private readonly dbPath: string;
|
||||
|
||||
constructor(dbPath?: string) {
|
||||
this.dbPath = dbPath || process.env.NODE_DB_PATH || path.join(process.cwd(), 'data', 'nodes.db');
|
||||
|
||||
// Ensure data directory exists
|
||||
const dataDir = path.dirname(this.dbPath);
|
||||
if (!fs.existsSync(dataDir)) {
|
||||
fs.mkdirSync(dataDir, { recursive: true });
|
||||
}
|
||||
|
||||
this.db = new Database(this.dbPath, {
|
||||
verbose: process.env.NODE_ENV === 'development' ? (msg: unknown) => logger.debug(String(msg)) : undefined
|
||||
});
|
||||
|
||||
// Enable WAL mode for better performance
|
||||
this.db.pragma('journal_mode = WAL');
|
||||
|
||||
this.initializeDatabase();
|
||||
}
|
||||
|
||||
/**
|
||||
* Initialize database with schema
|
||||
*/
|
||||
private initializeDatabase(): void {
|
||||
try {
|
||||
const schema = `
|
||||
-- Main nodes table
|
||||
CREATE TABLE IF NOT EXISTS nodes (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
node_type TEXT UNIQUE NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
package_name TEXT NOT NULL,
|
||||
display_name TEXT,
|
||||
description TEXT,
|
||||
code_hash TEXT NOT NULL,
|
||||
code_length INTEGER NOT NULL,
|
||||
source_location TEXT NOT NULL,
|
||||
source_code TEXT NOT NULL,
|
||||
credential_code TEXT,
|
||||
package_info TEXT, -- JSON
|
||||
has_credentials INTEGER DEFAULT 0,
|
||||
extracted_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Indexes for performance
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_package_name ON nodes(package_name);
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_code_hash ON nodes(code_hash);
|
||||
CREATE INDEX IF NOT EXISTS idx_nodes_name ON nodes(name);
|
||||
|
||||
-- Full Text Search virtual table for node search
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS nodes_fts USING fts5(
|
||||
node_type,
|
||||
name,
|
||||
display_name,
|
||||
description,
|
||||
package_name,
|
||||
content=nodes,
|
||||
content_rowid=id
|
||||
);
|
||||
|
||||
-- Triggers to keep FTS in sync
|
||||
CREATE TRIGGER IF NOT EXISTS nodes_ai AFTER INSERT ON nodes
|
||||
BEGIN
|
||||
INSERT INTO nodes_fts(rowid, node_type, name, display_name, description, package_name)
|
||||
VALUES (new.id, new.node_type, new.name, new.display_name, new.description, new.package_name);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS nodes_ad AFTER DELETE ON nodes
|
||||
BEGIN
|
||||
DELETE FROM nodes_fts WHERE rowid = old.id;
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS nodes_au AFTER UPDATE ON nodes
|
||||
BEGIN
|
||||
DELETE FROM nodes_fts WHERE rowid = old.id;
|
||||
INSERT INTO nodes_fts(rowid, node_type, name, display_name, description, package_name)
|
||||
VALUES (new.id, new.node_type, new.name, new.display_name, new.description, new.package_name);
|
||||
END;
|
||||
|
||||
-- Statistics table for metadata
|
||||
CREATE TABLE IF NOT EXISTS extraction_stats (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
total_nodes INTEGER NOT NULL,
|
||||
total_packages INTEGER NOT NULL,
|
||||
total_code_size INTEGER NOT NULL,
|
||||
nodes_with_credentials INTEGER NOT NULL,
|
||||
extraction_date DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
`;
|
||||
|
||||
this.db.exec(schema);
|
||||
logger.info('Database initialized successfully');
|
||||
} catch (error) {
|
||||
logger.error('Failed to initialize database:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Store a node in the database
|
||||
*/
|
||||
async storeNode(nodeInfo: NodeSourceInfo): Promise<StoredNode> {
|
||||
const codeHash = crypto.createHash('sha256').update(nodeInfo.sourceCode).digest('hex');
|
||||
|
||||
// Parse display name and description from source
|
||||
const displayName = this.extractDisplayName(nodeInfo.sourceCode);
|
||||
const description = this.extractDescription(nodeInfo.sourceCode);
|
||||
|
||||
const stmt = this.db.prepare(`
|
||||
INSERT OR REPLACE INTO nodes (
|
||||
node_type, name, package_name, display_name, description,
|
||||
code_hash, code_length, source_location, source_code,
|
||||
credential_code, package_info, has_credentials,
|
||||
updated_at
|
||||
) VALUES (
|
||||
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP
|
||||
)
|
||||
`);
|
||||
|
||||
const name = nodeInfo.nodeType.split('.').pop() || nodeInfo.nodeType;
|
||||
const packageName = nodeInfo.nodeType.split('.')[0] || 'unknown';
|
||||
|
||||
const result = stmt.run(
|
||||
nodeInfo.nodeType,
|
||||
name,
|
||||
packageName,
|
||||
displayName || null,
|
||||
description || null,
|
||||
codeHash,
|
||||
nodeInfo.sourceCode.length,
|
||||
nodeInfo.location,
|
||||
nodeInfo.sourceCode,
|
||||
nodeInfo.credentialCode || null,
|
||||
nodeInfo.packageInfo ? JSON.stringify(nodeInfo.packageInfo) : null,
|
||||
nodeInfo.credentialCode ? 1 : 0
|
||||
);
|
||||
|
||||
logger.info(`Stored node: ${nodeInfo.nodeType} (${codeHash.substring(0, 8)}...)`);
|
||||
|
||||
return {
|
||||
id: String(result.lastInsertRowid),
|
||||
nodeType: nodeInfo.nodeType,
|
||||
name,
|
||||
packageName,
|
||||
displayName,
|
||||
description,
|
||||
codeHash,
|
||||
codeLength: nodeInfo.sourceCode.length,
|
||||
sourceLocation: nodeInfo.location,
|
||||
hasCredentials: !!nodeInfo.credentialCode,
|
||||
extractedAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
sourceCode: nodeInfo.sourceCode,
|
||||
credentialCode: nodeInfo.credentialCode,
|
||||
packageInfo: nodeInfo.packageInfo
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Search for nodes using FTS
|
||||
*/
|
||||
async searchNodes(query: NodeSearchQuery): Promise<StoredNode[]> {
|
||||
let sql = `
|
||||
SELECT DISTINCT n.*
|
||||
FROM nodes n
|
||||
`;
|
||||
|
||||
const params: any[] = [];
|
||||
const conditions: string[] = [];
|
||||
|
||||
if (query.query) {
|
||||
// Use FTS for text search
|
||||
sql += ` JOIN nodes_fts fts ON n.id = fts.rowid`;
|
||||
conditions.push(`nodes_fts MATCH ?`);
|
||||
// Convert search query to FTS syntax (prefix search)
|
||||
const ftsQuery = query.query.split(' ')
|
||||
.map(term => `${term}*`)
|
||||
.join(' ');
|
||||
params.push(ftsQuery);
|
||||
}
|
||||
|
||||
if (query.packageName) {
|
||||
conditions.push(`n.package_name = ?`);
|
||||
params.push(query.packageName);
|
||||
}
|
||||
|
||||
if (query.nodeType) {
|
||||
conditions.push(`n.node_type LIKE ?`);
|
||||
params.push(`%${query.nodeType}%`);
|
||||
}
|
||||
|
||||
if (query.hasCredentials !== undefined) {
|
||||
conditions.push(`n.has_credentials = ?`);
|
||||
params.push(query.hasCredentials ? 1 : 0);
|
||||
}
|
||||
|
||||
if (conditions.length > 0) {
|
||||
sql += ` WHERE ${conditions.join(' AND ')}`;
|
||||
}
|
||||
|
||||
sql += ` ORDER BY n.name`;
|
||||
|
||||
if (query.limit) {
|
||||
sql += ` LIMIT ?`;
|
||||
params.push(query.limit);
|
||||
|
||||
if (query.offset) {
|
||||
sql += ` OFFSET ?`;
|
||||
params.push(query.offset);
|
||||
}
|
||||
}
|
||||
|
||||
const stmt = this.db.prepare(sql);
|
||||
const rows = stmt.all(...params);
|
||||
|
||||
return rows.map(row => this.rowToStoredNode(row));
|
||||
}
|
||||
|
||||
/**
|
||||
* Get node by type
|
||||
*/
|
||||
async getNode(nodeType: string): Promise<StoredNode | null> {
|
||||
const stmt = this.db.prepare(`
|
||||
SELECT * FROM nodes WHERE node_type = ?
|
||||
`);
|
||||
|
||||
const row = stmt.get(nodeType);
|
||||
return row ? this.rowToStoredNode(row) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all packages
|
||||
*/
|
||||
async getPackages(): Promise<Array<{ name: string; nodeCount: number }>> {
|
||||
const stmt = this.db.prepare(`
|
||||
SELECT package_name as name, COUNT(*) as nodeCount
|
||||
FROM nodes
|
||||
GROUP BY package_name
|
||||
ORDER BY nodeCount DESC
|
||||
`);
|
||||
|
||||
return stmt.all() as Array<{ name: string; nodeCount: number }>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Bulk store nodes (used for database rebuild)
|
||||
*/
|
||||
async bulkStoreNodes(nodeInfos: NodeSourceInfo[]): Promise<{
|
||||
stored: number;
|
||||
failed: number;
|
||||
errors: Array<{ nodeType: string; error: string }>;
|
||||
}> {
|
||||
const results = {
|
||||
stored: 0,
|
||||
failed: 0,
|
||||
errors: [] as Array<{ nodeType: string; error: string }>
|
||||
};
|
||||
|
||||
// Use transaction for bulk insert
|
||||
const insertMany = this.db.transaction((nodes: NodeSourceInfo[]) => {
|
||||
for (const nodeInfo of nodes) {
|
||||
try {
|
||||
this.storeNode(nodeInfo);
|
||||
results.stored++;
|
||||
} catch (error) {
|
||||
results.failed++;
|
||||
results.errors.push({
|
||||
nodeType: nodeInfo.nodeType,
|
||||
error: error instanceof Error ? error.message : 'Unknown error'
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
insertMany(nodeInfos);
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get statistics
|
||||
*/
|
||||
async getStatistics(): Promise<{
|
||||
totalNodes: number;
|
||||
totalPackages: number;
|
||||
totalCodeSize: number;
|
||||
nodesWithCredentials: number;
|
||||
averageNodeSize: number;
|
||||
packageDistribution: Array<{ package: string; count: number }>;
|
||||
}> {
|
||||
const stats = this.db.prepare(`
|
||||
SELECT
|
||||
COUNT(*) as totalNodes,
|
||||
COUNT(DISTINCT package_name) as totalPackages,
|
||||
SUM(code_length) as totalCodeSize,
|
||||
SUM(has_credentials) as nodesWithCredentials
|
||||
FROM nodes
|
||||
`).get() as any;
|
||||
|
||||
const packageDist = this.db.prepare(`
|
||||
SELECT package_name as package, COUNT(*) as count
|
||||
FROM nodes
|
||||
GROUP BY package_name
|
||||
ORDER BY count DESC
|
||||
`).all() as Array<{ package: string; count: number }>;
|
||||
|
||||
return {
|
||||
totalNodes: stats.totalNodes || 0,
|
||||
totalPackages: stats.totalPackages || 0,
|
||||
totalCodeSize: stats.totalCodeSize || 0,
|
||||
nodesWithCredentials: stats.nodesWithCredentials || 0,
|
||||
averageNodeSize: stats.totalNodes > 0 ? Math.round(stats.totalCodeSize / stats.totalNodes) : 0,
|
||||
packageDistribution: packageDist
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Rebuild entire database
|
||||
*/
|
||||
async rebuildDatabase(): Promise<void> {
|
||||
logger.info('Starting database rebuild...');
|
||||
|
||||
// Clear existing data
|
||||
this.db.exec('DELETE FROM nodes');
|
||||
this.db.exec('DELETE FROM extraction_stats');
|
||||
|
||||
logger.info('Database cleared for rebuild');
|
||||
}
|
||||
|
||||
/**
|
||||
* Save extraction statistics
|
||||
*/
|
||||
async saveExtractionStats(stats: {
|
||||
totalNodes: number;
|
||||
totalPackages: number;
|
||||
totalCodeSize: number;
|
||||
nodesWithCredentials: number;
|
||||
}): Promise<void> {
|
||||
const stmt = this.db.prepare(`
|
||||
INSERT INTO extraction_stats (
|
||||
total_nodes, total_packages, total_code_size, nodes_with_credentials
|
||||
) VALUES (?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
stmt.run(
|
||||
stats.totalNodes,
|
||||
stats.totalPackages,
|
||||
stats.totalCodeSize,
|
||||
stats.nodesWithCredentials
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Close database connection
|
||||
*/
|
||||
close(): void {
|
||||
this.db.close();
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert database row to StoredNode
|
||||
*/
|
||||
private rowToStoredNode(row: any): StoredNode {
|
||||
return {
|
||||
id: String(row.id),
|
||||
nodeType: row.node_type,
|
||||
name: row.name,
|
||||
packageName: row.package_name,
|
||||
displayName: row.display_name,
|
||||
description: row.description,
|
||||
codeHash: row.code_hash,
|
||||
codeLength: row.code_length,
|
||||
sourceLocation: row.source_location,
|
||||
hasCredentials: row.has_credentials === 1,
|
||||
extractedAt: new Date(row.extracted_at),
|
||||
updatedAt: new Date(row.updated_at),
|
||||
sourceCode: row.source_code,
|
||||
credentialCode: row.credential_code,
|
||||
packageInfo: row.package_info ? JSON.parse(row.package_info) : undefined
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract display name from source code
|
||||
*/
|
||||
private extractDisplayName(sourceCode: string): string | undefined {
|
||||
const match = sourceCode.match(/displayName:\s*["'`]([^"'`]+)["'`]/);
|
||||
return match ? match[1] : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract description from source code
|
||||
*/
|
||||
private extractDescription(sourceCode: string): string | undefined {
|
||||
const match = sourceCode.match(/description:\s*["'`]([^"'`]+)["'`]/);
|
||||
return match ? match[1] : undefined;
|
||||
}
|
||||
}
|
||||
@@ -1,49 +0,0 @@
|
||||
import { Request, Response, NextFunction } from 'express';
|
||||
import { logger } from './logger';
|
||||
|
||||
/**
|
||||
* Express middleware for authenticating requests with Bearer tokens
|
||||
*/
|
||||
export function authenticateRequest(authToken?: string) {
|
||||
return (req: Request, res: Response, next: NextFunction): void => {
|
||||
if (!authToken) {
|
||||
// No auth required
|
||||
return next();
|
||||
}
|
||||
|
||||
const authHeader = req.headers['authorization'];
|
||||
|
||||
if (!authHeader) {
|
||||
logger.warn('Missing authorization header', {
|
||||
ip: req.ip,
|
||||
path: req.path,
|
||||
});
|
||||
|
||||
res.status(401).json({
|
||||
error: 'Unauthorized',
|
||||
message: 'Missing authorization header',
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
// Support both "Bearer TOKEN" and just "TOKEN" formats
|
||||
const providedToken = authHeader.startsWith('Bearer ')
|
||||
? authHeader.substring(7)
|
||||
: authHeader;
|
||||
|
||||
if (providedToken !== authToken) {
|
||||
logger.warn('Invalid authentication token', {
|
||||
ip: req.ip,
|
||||
path: req.path,
|
||||
});
|
||||
|
||||
res.status(401).json({
|
||||
error: 'Unauthorized',
|
||||
message: 'Invalid authentication token',
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
next();
|
||||
};
|
||||
}
|
||||
@@ -1,241 +1,2 @@
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
import { logger } from './logger';
|
||||
import { execSync } from 'child_process';
|
||||
|
||||
interface NodeDocumentation {
|
||||
markdown: string;
|
||||
url: string;
|
||||
examples?: any[];
|
||||
}
|
||||
|
||||
export class DocumentationFetcher {
|
||||
private docsPath: string;
|
||||
private docsRepoUrl = 'https://github.com/n8n-io/n8n-docs.git';
|
||||
private cloned = false;
|
||||
|
||||
constructor(docsPath?: string) {
|
||||
this.docsPath = docsPath || path.join(process.cwd(), 'temp', 'n8n-docs');
|
||||
}
|
||||
|
||||
/**
|
||||
* Clone or update the n8n-docs repository
|
||||
*/
|
||||
async ensureDocsRepository(): Promise<void> {
|
||||
try {
|
||||
// Check if directory exists
|
||||
const exists = await fs.access(this.docsPath).then(() => true).catch(() => false);
|
||||
|
||||
if (!exists) {
|
||||
logger.info('Cloning n8n-docs repository...');
|
||||
await fs.mkdir(path.dirname(this.docsPath), { recursive: true });
|
||||
execSync(`git clone --depth 1 ${this.docsRepoUrl} ${this.docsPath}`, {
|
||||
stdio: 'pipe'
|
||||
});
|
||||
logger.info('n8n-docs repository cloned successfully');
|
||||
} else {
|
||||
logger.info('Updating n8n-docs repository...');
|
||||
execSync('git pull --ff-only', {
|
||||
cwd: this.docsPath,
|
||||
stdio: 'pipe'
|
||||
});
|
||||
logger.info('n8n-docs repository updated');
|
||||
}
|
||||
|
||||
this.cloned = true;
|
||||
} catch (error) {
|
||||
logger.error('Failed to clone/update n8n-docs repository:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get documentation for a specific node
|
||||
*/
|
||||
async getNodeDocumentation(nodeType: string): Promise<NodeDocumentation | null> {
|
||||
if (!this.cloned) {
|
||||
await this.ensureDocsRepository();
|
||||
}
|
||||
|
||||
try {
|
||||
// Convert node type to documentation path
|
||||
// e.g., "n8n-nodes-base.if" -> "if"
|
||||
const nodeName = this.extractNodeName(nodeType);
|
||||
|
||||
// Common documentation paths to check
|
||||
const possiblePaths = [
|
||||
path.join(this.docsPath, 'docs', 'integrations', 'builtin', 'core-nodes', `${nodeName}.md`),
|
||||
path.join(this.docsPath, 'docs', 'integrations', 'builtin', 'app-nodes', `${nodeName}.md`),
|
||||
path.join(this.docsPath, 'docs', 'integrations', 'builtin', 'trigger-nodes', `${nodeName}.md`),
|
||||
path.join(this.docsPath, 'docs', 'code-examples', 'expressions', `${nodeName}.md`),
|
||||
// Generic search in docs folder
|
||||
path.join(this.docsPath, 'docs', '**', `${nodeName}.md`)
|
||||
];
|
||||
|
||||
for (const docPath of possiblePaths) {
|
||||
try {
|
||||
const content = await fs.readFile(docPath, 'utf-8');
|
||||
const url = this.generateDocUrl(docPath);
|
||||
|
||||
return {
|
||||
markdown: content,
|
||||
url,
|
||||
examples: this.extractExamples(content)
|
||||
};
|
||||
} catch (error) {
|
||||
// Continue to next path
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
// If no exact match, try to find by searching
|
||||
const foundPath = await this.searchForNodeDoc(nodeName);
|
||||
if (foundPath) {
|
||||
const content = await fs.readFile(foundPath, 'utf-8');
|
||||
return {
|
||||
markdown: content,
|
||||
url: this.generateDocUrl(foundPath),
|
||||
examples: this.extractExamples(content)
|
||||
};
|
||||
}
|
||||
|
||||
logger.warn(`No documentation found for node: ${nodeType}`);
|
||||
return null;
|
||||
} catch (error) {
|
||||
logger.error(`Failed to get documentation for ${nodeType}:`, error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract node name from node type
|
||||
*/
|
||||
private extractNodeName(nodeType: string): string {
|
||||
// Handle different node type formats
|
||||
// "n8n-nodes-base.if" -> "if"
|
||||
// "@n8n/n8n-nodes-langchain.Agent" -> "agent"
|
||||
const parts = nodeType.split('.');
|
||||
const name = parts[parts.length - 1];
|
||||
return name.toLowerCase();
|
||||
}
|
||||
|
||||
/**
|
||||
* Search for node documentation file
|
||||
*/
|
||||
private async searchForNodeDoc(nodeName: string): Promise<string | null> {
|
||||
try {
|
||||
const result = execSync(
|
||||
`find ${this.docsPath}/docs -name "*.md" -type f | grep -i "${nodeName}" | head -1`,
|
||||
{ encoding: 'utf-8', stdio: 'pipe' }
|
||||
).trim();
|
||||
|
||||
return result || null;
|
||||
} catch (error) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate documentation URL from file path
|
||||
*/
|
||||
private generateDocUrl(filePath: string): string {
|
||||
const relativePath = path.relative(this.docsPath, filePath);
|
||||
const urlPath = relativePath
|
||||
.replace(/^docs\//, '')
|
||||
.replace(/\.md$/, '')
|
||||
.replace(/\\/g, '/');
|
||||
|
||||
return `https://docs.n8n.io/${urlPath}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract code examples from markdown content
|
||||
*/
|
||||
private extractExamples(markdown: string): any[] {
|
||||
const examples: any[] = [];
|
||||
|
||||
// Extract JSON code blocks
|
||||
const jsonCodeBlockRegex = /```json\n([\s\S]*?)```/g;
|
||||
let match;
|
||||
|
||||
while ((match = jsonCodeBlockRegex.exec(markdown)) !== null) {
|
||||
try {
|
||||
const json = JSON.parse(match[1]);
|
||||
examples.push(json);
|
||||
} catch (error) {
|
||||
// Not valid JSON, skip
|
||||
}
|
||||
}
|
||||
|
||||
// Extract workflow examples
|
||||
const workflowExampleRegex = /## Example.*?\n([\s\S]*?)(?=\n##|\n#|$)/gi;
|
||||
while ((match = workflowExampleRegex.exec(markdown)) !== null) {
|
||||
const exampleText = match[1];
|
||||
// Try to find JSON in the example section
|
||||
const jsonMatch = exampleText.match(/```json\n([\s\S]*?)```/);
|
||||
if (jsonMatch) {
|
||||
try {
|
||||
const json = JSON.parse(jsonMatch[1]);
|
||||
examples.push(json);
|
||||
} catch (error) {
|
||||
// Not valid JSON
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return examples;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all available documentation files
|
||||
*/
|
||||
async getAllDocumentationFiles(): Promise<Map<string, string>> {
|
||||
if (!this.cloned) {
|
||||
await this.ensureDocsRepository();
|
||||
}
|
||||
|
||||
const docMap = new Map<string, string>();
|
||||
|
||||
try {
|
||||
const findDocs = execSync(
|
||||
`find ${this.docsPath}/docs -name "*.md" -type f | grep -E "(core-nodes|app-nodes|trigger-nodes)/"`,
|
||||
{ encoding: 'utf-8', stdio: 'pipe' }
|
||||
).trim().split('\n');
|
||||
|
||||
for (const docPath of findDocs) {
|
||||
if (!docPath) continue;
|
||||
|
||||
const filename = path.basename(docPath, '.md');
|
||||
const content = await fs.readFile(docPath, 'utf-8');
|
||||
|
||||
// Try to extract the node type from the content
|
||||
const nodeTypeMatch = content.match(/node[_-]?type[:\s]+["']?([^"'\s]+)["']?/i);
|
||||
if (nodeTypeMatch) {
|
||||
docMap.set(nodeTypeMatch[1], docPath);
|
||||
} else {
|
||||
// Use filename as fallback
|
||||
docMap.set(filename, docPath);
|
||||
}
|
||||
}
|
||||
|
||||
logger.info(`Found ${docMap.size} documentation files`);
|
||||
return docMap;
|
||||
} catch (error) {
|
||||
logger.error('Failed to get documentation files:', error);
|
||||
return docMap;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean up cloned repository
|
||||
*/
|
||||
async cleanup(): Promise<void> {
|
||||
try {
|
||||
await fs.rm(this.docsPath, { recursive: true, force: true });
|
||||
this.cloned = false;
|
||||
logger.info('Cleaned up documentation repository');
|
||||
} catch (error) {
|
||||
logger.error('Failed to cleanup docs repository:', error);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Re-export everything from enhanced-documentation-fetcher
|
||||
export * from './enhanced-documentation-fetcher';
|
||||
621
src/utils/enhanced-documentation-fetcher.ts
Normal file
621
src/utils/enhanced-documentation-fetcher.ts
Normal file
@@ -0,0 +1,621 @@
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
import { logger } from './logger';
|
||||
import { execSync } from 'child_process';
|
||||
|
||||
// Enhanced documentation structure with rich content
|
||||
export interface EnhancedNodeDocumentation {
|
||||
markdown: string;
|
||||
url: string;
|
||||
title?: string;
|
||||
description?: string;
|
||||
operations?: OperationInfo[];
|
||||
apiMethods?: ApiMethodMapping[];
|
||||
examples?: CodeExample[];
|
||||
templates?: TemplateInfo[];
|
||||
relatedResources?: RelatedResource[];
|
||||
requiredScopes?: string[];
|
||||
metadata?: DocumentationMetadata;
|
||||
}
|
||||
|
||||
export interface OperationInfo {
|
||||
resource: string;
|
||||
operation: string;
|
||||
description: string;
|
||||
subOperations?: string[];
|
||||
}
|
||||
|
||||
export interface ApiMethodMapping {
|
||||
resource: string;
|
||||
operation: string;
|
||||
apiMethod: string;
|
||||
apiUrl: string;
|
||||
}
|
||||
|
||||
export interface CodeExample {
|
||||
title?: string;
|
||||
description?: string;
|
||||
type: 'json' | 'javascript' | 'yaml' | 'text';
|
||||
code: string;
|
||||
language?: string;
|
||||
}
|
||||
|
||||
export interface TemplateInfo {
|
||||
name: string;
|
||||
description?: string;
|
||||
url?: string;
|
||||
}
|
||||
|
||||
export interface RelatedResource {
|
||||
title: string;
|
||||
url: string;
|
||||
type: 'documentation' | 'api' | 'tutorial' | 'external';
|
||||
}
|
||||
|
||||
export interface DocumentationMetadata {
|
||||
contentType?: string[];
|
||||
priority?: string;
|
||||
tags?: string[];
|
||||
lastUpdated?: Date;
|
||||
}
|
||||
|
||||
export class EnhancedDocumentationFetcher {
|
||||
private docsPath: string;
|
||||
private docsRepoUrl = 'https://github.com/n8n-io/n8n-docs.git';
|
||||
private cloned = false;
|
||||
|
||||
constructor(docsPath?: string) {
|
||||
this.docsPath = docsPath || path.join(process.cwd(), 'temp', 'n8n-docs');
|
||||
}
|
||||
|
||||
/**
|
||||
* Clone or update the n8n-docs repository
|
||||
*/
|
||||
async ensureDocsRepository(): Promise<void> {
|
||||
try {
|
||||
const exists = await fs.access(this.docsPath).then(() => true).catch(() => false);
|
||||
|
||||
if (!exists) {
|
||||
logger.info('Cloning n8n-docs repository...');
|
||||
await fs.mkdir(path.dirname(this.docsPath), { recursive: true });
|
||||
execSync(`git clone --depth 1 ${this.docsRepoUrl} ${this.docsPath}`, {
|
||||
stdio: 'pipe'
|
||||
});
|
||||
logger.info('n8n-docs repository cloned successfully');
|
||||
} else {
|
||||
logger.info('Updating n8n-docs repository...');
|
||||
execSync('git pull --ff-only', {
|
||||
cwd: this.docsPath,
|
||||
stdio: 'pipe'
|
||||
});
|
||||
logger.info('n8n-docs repository updated');
|
||||
}
|
||||
|
||||
this.cloned = true;
|
||||
} catch (error) {
|
||||
logger.error('Failed to clone/update n8n-docs repository:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get enhanced documentation for a specific node
|
||||
*/
|
||||
async getEnhancedNodeDocumentation(nodeType: string): Promise<EnhancedNodeDocumentation | null> {
|
||||
if (!this.cloned) {
|
||||
await this.ensureDocsRepository();
|
||||
}
|
||||
|
||||
try {
|
||||
const nodeName = this.extractNodeName(nodeType);
|
||||
|
||||
// Common documentation paths to check
|
||||
const possiblePaths = [
|
||||
path.join(this.docsPath, 'docs', 'integrations', 'builtin', 'app-nodes', `${nodeType}.md`),
|
||||
path.join(this.docsPath, 'docs', 'integrations', 'builtin', 'core-nodes', `${nodeType}.md`),
|
||||
path.join(this.docsPath, 'docs', 'integrations', 'builtin', 'trigger-nodes', `${nodeType}.md`),
|
||||
path.join(this.docsPath, 'docs', 'integrations', 'builtin', 'core-nodes', `${nodeName}.md`),
|
||||
path.join(this.docsPath, 'docs', 'integrations', 'builtin', 'app-nodes', `${nodeName}.md`),
|
||||
path.join(this.docsPath, 'docs', 'integrations', 'builtin', 'trigger-nodes', `${nodeName}.md`),
|
||||
];
|
||||
|
||||
for (const docPath of possiblePaths) {
|
||||
try {
|
||||
const content = await fs.readFile(docPath, 'utf-8');
|
||||
logger.debug(`Checking doc path: ${docPath}`);
|
||||
|
||||
// Skip credential documentation files
|
||||
if (this.isCredentialDoc(docPath, content)) {
|
||||
logger.debug(`Skipping credential doc: ${docPath}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
logger.info(`Found documentation for ${nodeType} at: ${docPath}`);
|
||||
return this.parseEnhancedDocumentation(content, docPath);
|
||||
} catch (error) {
|
||||
// File doesn't exist, continue
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
// If no exact match, try to find by searching
|
||||
logger.debug(`No exact match found, searching for ${nodeType}...`);
|
||||
const foundPath = await this.searchForNodeDoc(nodeType);
|
||||
if (foundPath) {
|
||||
logger.info(`Found documentation via search at: ${foundPath}`);
|
||||
const content = await fs.readFile(foundPath, 'utf-8');
|
||||
|
||||
if (!this.isCredentialDoc(foundPath, content)) {
|
||||
return this.parseEnhancedDocumentation(content, foundPath);
|
||||
}
|
||||
}
|
||||
|
||||
logger.warn(`No documentation found for node: ${nodeType}`);
|
||||
return null;
|
||||
} catch (error) {
|
||||
logger.error(`Failed to get documentation for ${nodeType}:`, error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse markdown content into enhanced documentation structure
|
||||
*/
|
||||
private parseEnhancedDocumentation(markdown: string, filePath: string): EnhancedNodeDocumentation {
|
||||
const doc: EnhancedNodeDocumentation = {
|
||||
markdown,
|
||||
url: this.generateDocUrl(filePath),
|
||||
};
|
||||
|
||||
// Extract frontmatter metadata
|
||||
const metadata = this.extractFrontmatter(markdown);
|
||||
if (metadata) {
|
||||
doc.metadata = metadata;
|
||||
doc.title = metadata.title;
|
||||
doc.description = metadata.description;
|
||||
}
|
||||
|
||||
// Extract title and description from content if not in frontmatter
|
||||
if (!doc.title) {
|
||||
doc.title = this.extractTitle(markdown);
|
||||
}
|
||||
if (!doc.description) {
|
||||
doc.description = this.extractDescription(markdown);
|
||||
}
|
||||
|
||||
// Extract operations
|
||||
doc.operations = this.extractOperations(markdown);
|
||||
|
||||
// Extract API method mappings
|
||||
doc.apiMethods = this.extractApiMethods(markdown);
|
||||
|
||||
// Extract code examples
|
||||
doc.examples = this.extractCodeExamples(markdown);
|
||||
|
||||
// Extract templates
|
||||
doc.templates = this.extractTemplates(markdown);
|
||||
|
||||
// Extract related resources
|
||||
doc.relatedResources = this.extractRelatedResources(markdown);
|
||||
|
||||
// Extract required scopes
|
||||
doc.requiredScopes = this.extractRequiredScopes(markdown);
|
||||
|
||||
return doc;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract frontmatter metadata
|
||||
*/
|
||||
private extractFrontmatter(markdown: string): any {
|
||||
const frontmatterMatch = markdown.match(/^---\n([\s\S]*?)\n---/);
|
||||
if (!frontmatterMatch) return null;
|
||||
|
||||
const frontmatter: any = {};
|
||||
const lines = frontmatterMatch[1].split('\n');
|
||||
|
||||
for (const line of lines) {
|
||||
if (line.includes(':')) {
|
||||
const [key, ...valueParts] = line.split(':');
|
||||
const value = valueParts.join(':').trim();
|
||||
|
||||
// Parse arrays
|
||||
if (value.startsWith('[') && value.endsWith(']')) {
|
||||
frontmatter[key.trim()] = value
|
||||
.slice(1, -1)
|
||||
.split(',')
|
||||
.map(v => v.trim());
|
||||
} else {
|
||||
frontmatter[key.trim()] = value;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return frontmatter;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract title from markdown
|
||||
*/
|
||||
private extractTitle(markdown: string): string | undefined {
|
||||
const match = markdown.match(/^#\s+(.+)$/m);
|
||||
return match ? match[1].trim() : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract description from markdown
|
||||
*/
|
||||
private extractDescription(markdown: string): string | undefined {
|
||||
// Remove frontmatter
|
||||
const content = markdown.replace(/^---[\s\S]*?---\n/, '');
|
||||
|
||||
// Find first paragraph after title
|
||||
const lines = content.split('\n');
|
||||
let foundTitle = false;
|
||||
let description = '';
|
||||
|
||||
for (const line of lines) {
|
||||
if (line.startsWith('#')) {
|
||||
foundTitle = true;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (foundTitle && line.trim() && !line.startsWith('#') && !line.startsWith('*') && !line.startsWith('-')) {
|
||||
description = line.trim();
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return description || undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract operations from markdown
|
||||
*/
|
||||
private extractOperations(markdown: string): OperationInfo[] {
|
||||
const operations: OperationInfo[] = [];
|
||||
|
||||
// Find operations section
|
||||
const operationsMatch = markdown.match(/##\s+Operations\s*\n([\s\S]*?)(?=\n##|\n#|$)/i);
|
||||
if (!operationsMatch) return operations;
|
||||
|
||||
const operationsText = operationsMatch[1];
|
||||
|
||||
// Parse operation structure - handle nested bullet points
|
||||
let currentResource: string | null = null;
|
||||
const lines = operationsText.split('\n');
|
||||
|
||||
for (const line of lines) {
|
||||
const trimmedLine = line.trim();
|
||||
|
||||
// Skip empty lines
|
||||
if (!trimmedLine) continue;
|
||||
|
||||
// Resource level - non-indented bullet with bold text (e.g., "* **Channel**")
|
||||
if (line.match(/^\*\s+\*\*[^*]+\*\*\s*$/) && !line.match(/^\s+/)) {
|
||||
const match = trimmedLine.match(/^\*\s+\*\*([^*]+)\*\*/);
|
||||
if (match) {
|
||||
currentResource = match[1].trim();
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
// Skip if we don't have a current resource
|
||||
if (!currentResource) continue;
|
||||
|
||||
// Operation level - indented bullets (any whitespace + *)
|
||||
if (line.match(/^\s+\*\s+/) && currentResource) {
|
||||
// Extract operation name and description
|
||||
const operationMatch = trimmedLine.match(/^\*\s+\*\*([^*]+)\*\*(.*)$/);
|
||||
if (operationMatch) {
|
||||
const operation = operationMatch[1].trim();
|
||||
let description = operationMatch[2].trim();
|
||||
|
||||
// Clean up description
|
||||
description = description.replace(/^:\s*/, '').replace(/\.$/, '').trim();
|
||||
|
||||
operations.push({
|
||||
resource: currentResource,
|
||||
operation,
|
||||
description: description || operation,
|
||||
});
|
||||
} else {
|
||||
// Handle operations without bold formatting or with different format
|
||||
const simpleMatch = trimmedLine.match(/^\*\s+(.+)$/);
|
||||
if (simpleMatch) {
|
||||
const text = simpleMatch[1].trim();
|
||||
// Split by colon to separate operation from description
|
||||
const colonIndex = text.indexOf(':');
|
||||
if (colonIndex > 0) {
|
||||
operations.push({
|
||||
resource: currentResource,
|
||||
operation: text.substring(0, colonIndex).trim(),
|
||||
description: text.substring(colonIndex + 1).trim() || text,
|
||||
});
|
||||
} else {
|
||||
operations.push({
|
||||
resource: currentResource,
|
||||
operation: text,
|
||||
description: text,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return operations;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract API method mappings from markdown tables
|
||||
*/
|
||||
private extractApiMethods(markdown: string): ApiMethodMapping[] {
|
||||
const apiMethods: ApiMethodMapping[] = [];
|
||||
|
||||
// Find API method tables
|
||||
const tableRegex = /\|.*Resource.*\|.*Operation.*\|.*(?:Slack API method|API method|Method).*\|[\s\S]*?\n(?=\n[^|]|$)/gi;
|
||||
const tables = markdown.match(tableRegex);
|
||||
|
||||
if (!tables) return apiMethods;
|
||||
|
||||
for (const table of tables) {
|
||||
const rows = table.split('\n').filter(row => row.trim() && !row.includes('---'));
|
||||
|
||||
// Skip header row
|
||||
for (let i = 1; i < rows.length; i++) {
|
||||
const cells = rows[i].split('|').map(cell => cell.trim()).filter(Boolean);
|
||||
|
||||
if (cells.length >= 3) {
|
||||
const resource = cells[0];
|
||||
const operation = cells[1];
|
||||
const apiMethodCell = cells[2];
|
||||
|
||||
// Extract API method and URL from markdown link
|
||||
const linkMatch = apiMethodCell.match(/\[([^\]]+)\]\(([^)]+)\)/);
|
||||
|
||||
if (linkMatch) {
|
||||
apiMethods.push({
|
||||
resource,
|
||||
operation,
|
||||
apiMethod: linkMatch[1],
|
||||
apiUrl: linkMatch[2],
|
||||
});
|
||||
} else {
|
||||
apiMethods.push({
|
||||
resource,
|
||||
operation,
|
||||
apiMethod: apiMethodCell,
|
||||
apiUrl: '',
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return apiMethods;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract code examples from markdown
|
||||
*/
|
||||
private extractCodeExamples(markdown: string): CodeExample[] {
|
||||
const examples: CodeExample[] = [];
|
||||
|
||||
// Extract all code blocks with language
|
||||
const codeBlockRegex = /```(\w+)?\n([\s\S]*?)```/g;
|
||||
let match;
|
||||
|
||||
while ((match = codeBlockRegex.exec(markdown)) !== null) {
|
||||
const language = match[1] || 'text';
|
||||
const code = match[2].trim();
|
||||
|
||||
// Look for title or description before the code block
|
||||
const beforeCodeIndex = match.index;
|
||||
const beforeText = markdown.substring(Math.max(0, beforeCodeIndex - 200), beforeCodeIndex);
|
||||
const titleMatch = beforeText.match(/(?:###|####)\s+(.+)$/m);
|
||||
|
||||
const example: CodeExample = {
|
||||
type: this.mapLanguageToType(language),
|
||||
language,
|
||||
code,
|
||||
};
|
||||
|
||||
if (titleMatch) {
|
||||
example.title = titleMatch[1].trim();
|
||||
}
|
||||
|
||||
// Try to parse JSON examples
|
||||
if (language === 'json') {
|
||||
try {
|
||||
JSON.parse(code);
|
||||
examples.push(example);
|
||||
} catch (e) {
|
||||
// Skip invalid JSON
|
||||
}
|
||||
} else {
|
||||
examples.push(example);
|
||||
}
|
||||
}
|
||||
|
||||
return examples;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract template information
|
||||
*/
|
||||
private extractTemplates(markdown: string): TemplateInfo[] {
|
||||
const templates: TemplateInfo[] = [];
|
||||
|
||||
// Look for template widget
|
||||
const templateWidgetMatch = markdown.match(/\[\[\s*templatesWidget\s*\(\s*[^,]+,\s*'([^']+)'\s*\)\s*\]\]/);
|
||||
if (templateWidgetMatch) {
|
||||
templates.push({
|
||||
name: templateWidgetMatch[1],
|
||||
description: `Templates for ${templateWidgetMatch[1]}`,
|
||||
});
|
||||
}
|
||||
|
||||
return templates;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract related resources
|
||||
*/
|
||||
private extractRelatedResources(markdown: string): RelatedResource[] {
|
||||
const resources: RelatedResource[] = [];
|
||||
|
||||
// Find related resources section
|
||||
const relatedMatch = markdown.match(/##\s+(?:Related resources|Related|Resources)\s*\n([\s\S]*?)(?=\n##|\n#|$)/i);
|
||||
if (!relatedMatch) return resources;
|
||||
|
||||
const relatedText = relatedMatch[1];
|
||||
|
||||
// Extract links
|
||||
const linkRegex = /\[([^\]]+)\]\(([^)]+)\)/g;
|
||||
let match;
|
||||
|
||||
while ((match = linkRegex.exec(relatedText)) !== null) {
|
||||
const title = match[1];
|
||||
const url = match[2];
|
||||
|
||||
// Determine resource type
|
||||
let type: RelatedResource['type'] = 'external';
|
||||
if (url.includes('docs.n8n.io') || url.startsWith('/')) {
|
||||
type = 'documentation';
|
||||
} else if (url.includes('api.')) {
|
||||
type = 'api';
|
||||
}
|
||||
|
||||
resources.push({ title, url, type });
|
||||
}
|
||||
|
||||
return resources;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract required scopes
|
||||
*/
|
||||
private extractRequiredScopes(markdown: string): string[] {
|
||||
const scopes: string[] = [];
|
||||
|
||||
// Find required scopes section
|
||||
const scopesMatch = markdown.match(/##\s+(?:Required scopes|Scopes)\s*\n([\s\S]*?)(?=\n##|\n#|$)/i);
|
||||
if (!scopesMatch) return scopes;
|
||||
|
||||
const scopesText = scopesMatch[1];
|
||||
|
||||
// Extract scope patterns (common formats)
|
||||
const scopeRegex = /`([a-z:._-]+)`/gi;
|
||||
let match;
|
||||
|
||||
while ((match = scopeRegex.exec(scopesText)) !== null) {
|
||||
const scope = match[1];
|
||||
if (scope.includes(':') || scope.includes('.')) {
|
||||
scopes.push(scope);
|
||||
}
|
||||
}
|
||||
|
||||
return [...new Set(scopes)]; // Remove duplicates
|
||||
}
|
||||
|
||||
/**
|
||||
* Map language to code example type
|
||||
*/
|
||||
private mapLanguageToType(language: string): CodeExample['type'] {
|
||||
switch (language.toLowerCase()) {
|
||||
case 'json':
|
||||
return 'json';
|
||||
case 'js':
|
||||
case 'javascript':
|
||||
case 'typescript':
|
||||
case 'ts':
|
||||
return 'javascript';
|
||||
case 'yaml':
|
||||
case 'yml':
|
||||
return 'yaml';
|
||||
default:
|
||||
return 'text';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if this is a credential documentation
|
||||
*/
|
||||
private isCredentialDoc(filePath: string, content: string): boolean {
|
||||
return filePath.includes('/credentials/') ||
|
||||
(content.includes('title: ') &&
|
||||
content.includes(' credentials') &&
|
||||
!content.includes(' node documentation'));
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract node name from node type
|
||||
*/
|
||||
private extractNodeName(nodeType: string): string {
|
||||
const parts = nodeType.split('.');
|
||||
const name = parts[parts.length - 1];
|
||||
return name.toLowerCase();
|
||||
}
|
||||
|
||||
/**
|
||||
* Search for node documentation file
|
||||
*/
|
||||
private async searchForNodeDoc(nodeType: string): Promise<string | null> {
|
||||
try {
|
||||
// First try exact match with nodeType
|
||||
let result = execSync(
|
||||
`find ${this.docsPath}/docs/integrations/builtin -name "${nodeType}.md" -type f | grep -v credentials | head -1`,
|
||||
{ encoding: 'utf-8', stdio: 'pipe' }
|
||||
).trim();
|
||||
|
||||
if (result) return result;
|
||||
|
||||
// Try lowercase nodeType
|
||||
const lowerNodeType = nodeType.toLowerCase();
|
||||
result = execSync(
|
||||
`find ${this.docsPath}/docs/integrations/builtin -name "${lowerNodeType}.md" -type f | grep -v credentials | head -1`,
|
||||
{ encoding: 'utf-8', stdio: 'pipe' }
|
||||
).trim();
|
||||
|
||||
if (result) return result;
|
||||
|
||||
// Try node name pattern but exclude trigger nodes
|
||||
const nodeName = this.extractNodeName(nodeType);
|
||||
result = execSync(
|
||||
`find ${this.docsPath}/docs/integrations/builtin -name "*${nodeName}.md" -type f | grep -v credentials | grep -v trigger | head -1`,
|
||||
{ encoding: 'utf-8', stdio: 'pipe' }
|
||||
).trim();
|
||||
|
||||
return result || null;
|
||||
} catch (error) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate documentation URL from file path
|
||||
*/
|
||||
private generateDocUrl(filePath: string): string {
|
||||
const relativePath = path.relative(this.docsPath, filePath);
|
||||
const urlPath = relativePath
|
||||
.replace(/^docs\//, '')
|
||||
.replace(/\.md$/, '')
|
||||
.replace(/\\/g, '/');
|
||||
|
||||
return `https://docs.n8n.io/${urlPath}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean up cloned repository
|
||||
*/
|
||||
async cleanup(): Promise<void> {
|
||||
try {
|
||||
await fs.rm(this.docsPath, { recursive: true, force: true });
|
||||
this.cloned = false;
|
||||
logger.info('Cleaned up documentation repository');
|
||||
} catch (error) {
|
||||
logger.error('Failed to cleanup docs repository:', error);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,267 +1,140 @@
|
||||
import { logger } from './logger';
|
||||
|
||||
interface NodeExample {
|
||||
nodes: any[];
|
||||
connections: any;
|
||||
pinData?: any;
|
||||
meta?: any;
|
||||
}
|
||||
|
||||
interface NodeParameter {
|
||||
name: string;
|
||||
type: string;
|
||||
default?: any;
|
||||
options?: any[];
|
||||
displayOptions?: any;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generates example workflows and parameters for n8n nodes
|
||||
*/
|
||||
export class ExampleGenerator {
|
||||
/**
|
||||
* Generate example workflow for a node
|
||||
* Generate an example workflow from node definition
|
||||
*/
|
||||
static generateNodeExample(nodeType: string, nodeData: any): NodeExample {
|
||||
const nodeName = this.getNodeName(nodeType);
|
||||
const nodeId = this.generateNodeId();
|
||||
static generateFromNodeDefinition(nodeDefinition: any): any {
|
||||
const nodeName = nodeDefinition.displayName || 'Example Node';
|
||||
const nodeType = nodeDefinition.name || 'n8n-nodes-base.exampleNode';
|
||||
|
||||
// Base example structure
|
||||
const example: NodeExample = {
|
||||
nodes: [{
|
||||
parameters: this.generateExampleParameters(nodeType, nodeData),
|
||||
type: nodeType,
|
||||
typeVersion: nodeData.typeVersion || 1,
|
||||
position: [220, 120],
|
||||
id: nodeId,
|
||||
name: nodeName
|
||||
}],
|
||||
connections: {
|
||||
[nodeName]: {
|
||||
main: [[]]
|
||||
}
|
||||
},
|
||||
pinData: {},
|
||||
meta: {
|
||||
templateCredsSetupCompleted: true,
|
||||
instanceId: this.generateInstanceId()
|
||||
}
|
||||
};
|
||||
|
||||
// Add specific configurations based on node type
|
||||
this.addNodeSpecificConfig(nodeType, example, nodeData);
|
||||
|
||||
return example;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate example parameters based on node type
|
||||
*/
|
||||
private static generateExampleParameters(nodeType: string, nodeData: any): any {
|
||||
const params: any = {};
|
||||
|
||||
// Extract node name for specific handling
|
||||
const nodeName = nodeType.split('.').pop()?.toLowerCase() || '';
|
||||
|
||||
// Common node examples
|
||||
switch (nodeName) {
|
||||
case 'if':
|
||||
return {
|
||||
conditions: {
|
||||
options: {
|
||||
caseSensitive: true,
|
||||
leftValue: "",
|
||||
typeValidation: "strict",
|
||||
version: 2
|
||||
},
|
||||
conditions: [{
|
||||
name: `${nodeName} Example Workflow`,
|
||||
nodes: [
|
||||
{
|
||||
parameters: this.generateExampleParameters(nodeDefinition),
|
||||
id: this.generateNodeId(),
|
||||
leftValue: "={{ $json }}",
|
||||
rightValue: "",
|
||||
operator: {
|
||||
type: "object",
|
||||
operation: "notEmpty",
|
||||
singleValue: true
|
||||
}
|
||||
}],
|
||||
combinator: "and"
|
||||
name: nodeName,
|
||||
type: nodeType,
|
||||
typeVersion: nodeDefinition.version || 1,
|
||||
position: [250, 300],
|
||||
},
|
||||
options: {}
|
||||
],
|
||||
connections: {},
|
||||
active: false,
|
||||
settings: {},
|
||||
tags: ['example', 'generated'],
|
||||
};
|
||||
|
||||
case 'webhook':
|
||||
return {
|
||||
httpMethod: "POST",
|
||||
path: "webhook-path",
|
||||
responseMode: "onReceived",
|
||||
responseData: "allEntries",
|
||||
options: {}
|
||||
};
|
||||
|
||||
case 'httprequest':
|
||||
return {
|
||||
method: "GET",
|
||||
url: "https://api.example.com/data",
|
||||
authentication: "none",
|
||||
options: {},
|
||||
headerParametersUi: {
|
||||
parameter: []
|
||||
}
|
||||
};
|
||||
|
||||
case 'function':
|
||||
return {
|
||||
functionCode: "// Add your JavaScript code here\nreturn $input.all();"
|
||||
};
|
||||
|
||||
case 'set':
|
||||
return {
|
||||
mode: "manual",
|
||||
duplicateItem: false,
|
||||
values: {
|
||||
string: [{
|
||||
name: "myField",
|
||||
value: "myValue"
|
||||
}]
|
||||
}
|
||||
};
|
||||
|
||||
case 'split':
|
||||
return {
|
||||
batchSize: 10,
|
||||
options: {}
|
||||
};
|
||||
|
||||
default:
|
||||
// Generate generic parameters from node properties
|
||||
return this.generateGenericParameters(nodeData);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate generic parameters from node properties
|
||||
* Generate example parameters based on node properties
|
||||
*/
|
||||
private static generateGenericParameters(nodeData: any): any {
|
||||
static generateExampleParameters(nodeDefinition: any): any {
|
||||
const params: any = {};
|
||||
|
||||
if (nodeData.properties) {
|
||||
for (const prop of nodeData.properties) {
|
||||
if (prop.default !== undefined) {
|
||||
params[prop.name] = prop.default;
|
||||
} else if (prop.type === 'string') {
|
||||
params[prop.name] = '';
|
||||
} else if (prop.type === 'number') {
|
||||
params[prop.name] = 0;
|
||||
} else if (prop.type === 'boolean') {
|
||||
params[prop.name] = false;
|
||||
} else if (prop.type === 'options' && prop.options?.length > 0) {
|
||||
params[prop.name] = prop.options[0].value;
|
||||
// If properties are available, generate examples based on them
|
||||
if (Array.isArray(nodeDefinition.properties)) {
|
||||
for (const prop of nodeDefinition.properties) {
|
||||
if (prop.name && prop.type) {
|
||||
params[prop.name] = this.generateExampleValue(prop);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Add common parameters based on node type
|
||||
if (nodeDefinition.displayName?.toLowerCase().includes('trigger')) {
|
||||
params.pollTimes = {
|
||||
item: [
|
||||
{
|
||||
mode: 'everyMinute',
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
return params;
|
||||
}
|
||||
|
||||
/**
|
||||
* Add node-specific configurations
|
||||
* Generate example value based on property definition
|
||||
*/
|
||||
private static addNodeSpecificConfig(nodeType: string, example: NodeExample, nodeData: any): void {
|
||||
const nodeName = nodeType.split('.').pop()?.toLowerCase() || '';
|
||||
|
||||
// Add specific connection structures for different node types
|
||||
switch (nodeName) {
|
||||
case 'if':
|
||||
// IF node has true/false outputs
|
||||
example.connections[example.nodes[0].name] = {
|
||||
main: [[], []] // Two outputs: true, false
|
||||
};
|
||||
break;
|
||||
|
||||
case 'switch':
|
||||
// Switch node can have multiple outputs
|
||||
const outputs = nodeData.outputs || 3;
|
||||
example.connections[example.nodes[0].name] = {
|
||||
main: Array(outputs).fill([])
|
||||
};
|
||||
break;
|
||||
|
||||
case 'merge':
|
||||
// Merge node has multiple inputs
|
||||
example.nodes[0].position = [400, 120];
|
||||
// Add dummy input nodes
|
||||
example.nodes.push({
|
||||
parameters: {},
|
||||
type: "n8n-nodes-base.noOp",
|
||||
typeVersion: 1,
|
||||
position: [200, 60],
|
||||
id: this.generateNodeId(),
|
||||
name: "Input 1"
|
||||
});
|
||||
example.nodes.push({
|
||||
parameters: {},
|
||||
type: "n8n-nodes-base.noOp",
|
||||
typeVersion: 1,
|
||||
position: [200, 180],
|
||||
id: this.generateNodeId(),
|
||||
name: "Input 2"
|
||||
});
|
||||
example.connections = {
|
||||
"Input 1": { main: [[{ node: example.nodes[0].name, type: "main", index: 0 }]] },
|
||||
"Input 2": { main: [[{ node: example.nodes[0].name, type: "main", index: 1 }]] },
|
||||
[example.nodes[0].name]: { main: [[]] }
|
||||
};
|
||||
break;
|
||||
private static generateExampleValue(property: any): any {
|
||||
switch (property.type) {
|
||||
case 'string':
|
||||
if (property.name.toLowerCase().includes('url')) {
|
||||
return 'https://example.com';
|
||||
}
|
||||
|
||||
// Add credentials if needed
|
||||
if (nodeData.credentials?.length > 0) {
|
||||
example.nodes[0].credentials = {};
|
||||
for (const cred of nodeData.credentials) {
|
||||
example.nodes[0].credentials[cred.name] = {
|
||||
id: this.generateNodeId(),
|
||||
name: `${cred.name} account`
|
||||
};
|
||||
if (property.name.toLowerCase().includes('email')) {
|
||||
return 'user@example.com';
|
||||
}
|
||||
if (property.name.toLowerCase().includes('name')) {
|
||||
return 'Example Name';
|
||||
}
|
||||
return property.default || 'example-value';
|
||||
|
||||
case 'number':
|
||||
return property.default || 10;
|
||||
|
||||
case 'boolean':
|
||||
return property.default !== undefined ? property.default : true;
|
||||
|
||||
case 'options':
|
||||
if (property.options && property.options.length > 0) {
|
||||
return property.options[0].value;
|
||||
}
|
||||
return property.default || '';
|
||||
|
||||
case 'collection':
|
||||
case 'fixedCollection':
|
||||
return {};
|
||||
|
||||
default:
|
||||
return property.default || null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract display name from node type
|
||||
*/
|
||||
private static getNodeName(nodeType: string): string {
|
||||
const parts = nodeType.split('.');
|
||||
const name = parts[parts.length - 1];
|
||||
return name.charAt(0).toUpperCase() + name.slice(1);
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate a random node ID
|
||||
* Generate a unique node ID
|
||||
*/
|
||||
private static generateNodeId(): string {
|
||||
return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, (c) => {
|
||||
const r = Math.random() * 16 | 0;
|
||||
const v = c === 'x' ? r : (r & 0x3 | 0x8);
|
||||
return v.toString(16);
|
||||
return Math.random().toString(36).substring(2, 15) +
|
||||
Math.random().toString(36).substring(2, 15);
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate example based on node operations
|
||||
*/
|
||||
static generateFromOperations(operations: any[]): any {
|
||||
const examples: any[] = [];
|
||||
|
||||
if (!operations || operations.length === 0) {
|
||||
return examples;
|
||||
}
|
||||
|
||||
// Group operations by resource
|
||||
const resourceMap = new Map<string, any[]>();
|
||||
for (const op of operations) {
|
||||
if (!resourceMap.has(op.resource)) {
|
||||
resourceMap.set(op.resource, []);
|
||||
}
|
||||
resourceMap.get(op.resource)!.push(op);
|
||||
}
|
||||
|
||||
// Generate example for each resource
|
||||
for (const [resource, ops] of resourceMap) {
|
||||
examples.push({
|
||||
resource,
|
||||
operation: ops[0].operation,
|
||||
description: `Example: ${ops[0].description}`,
|
||||
parameters: {
|
||||
resource,
|
||||
operation: ops[0].operation,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate instance ID
|
||||
*/
|
||||
private static generateInstanceId(): string {
|
||||
return Array(64).fill(0).map(() => Math.floor(Math.random() * 16).toString(16)).join('');
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate example from node definition
|
||||
*/
|
||||
static generateFromNodeDefinition(nodeDefinition: any): NodeExample {
|
||||
const nodeType = nodeDefinition.description?.name || 'n8n-nodes-base.node';
|
||||
const nodeData = {
|
||||
typeVersion: nodeDefinition.description?.version || 1,
|
||||
properties: nodeDefinition.description?.properties || [],
|
||||
credentials: nodeDefinition.description?.credentials || [],
|
||||
outputs: nodeDefinition.description?.outputs || ['main']
|
||||
};
|
||||
|
||||
return this.generateNodeExample(nodeType, nodeData);
|
||||
return examples;
|
||||
}
|
||||
}
|
||||
@@ -21,6 +21,8 @@ export class NodeSourceExtractor {
|
||||
'/n8n-modules',
|
||||
// Common n8n installation paths
|
||||
process.env.N8N_CUSTOM_EXTENSIONS || '',
|
||||
// Additional local path for testing
|
||||
path.join(process.cwd(), 'node_modules'),
|
||||
].filter(Boolean);
|
||||
|
||||
/**
|
||||
@@ -75,22 +77,31 @@ export class NodeSourceExtractor {
|
||||
nodeName: string
|
||||
): Promise<NodeSourceInfo | null> {
|
||||
try {
|
||||
// First, try standard patterns
|
||||
// Try both the provided case and capitalized first letter
|
||||
const nodeNameVariants = [
|
||||
nodeName,
|
||||
nodeName.charAt(0).toUpperCase() + nodeName.slice(1), // Capitalize first letter
|
||||
nodeName.toLowerCase(), // All lowercase
|
||||
nodeName.toUpperCase(), // All uppercase
|
||||
];
|
||||
|
||||
// First, try standard patterns with all case variants
|
||||
for (const nameVariant of nodeNameVariants) {
|
||||
const standardPatterns = [
|
||||
`${packageName}/dist/nodes/${nodeName}/${nodeName}.node.js`,
|
||||
`${packageName}/dist/nodes/${nodeName}.node.js`,
|
||||
`${packageName}/nodes/${nodeName}/${nodeName}.node.js`,
|
||||
`${packageName}/nodes/${nodeName}.node.js`,
|
||||
`${nodeName}/${nodeName}.node.js`,
|
||||
`${nodeName}.node.js`,
|
||||
`${packageName}/dist/nodes/${nameVariant}/${nameVariant}.node.js`,
|
||||
`${packageName}/dist/nodes/${nameVariant}.node.js`,
|
||||
`${packageName}/nodes/${nameVariant}/${nameVariant}.node.js`,
|
||||
`${packageName}/nodes/${nameVariant}.node.js`,
|
||||
`${nameVariant}/${nameVariant}.node.js`,
|
||||
`${nameVariant}.node.js`,
|
||||
];
|
||||
|
||||
// Additional patterns for nested node structures (e.g., agents/Agent)
|
||||
const nestedPatterns = [
|
||||
`${packageName}/dist/nodes/*/${nodeName}/${nodeName}.node.js`,
|
||||
`${packageName}/dist/nodes/**/${nodeName}/${nodeName}.node.js`,
|
||||
`${packageName}/nodes/*/${nodeName}/${nodeName}.node.js`,
|
||||
`${packageName}/nodes/**/${nodeName}/${nodeName}.node.js`,
|
||||
`${packageName}/dist/nodes/*/${nameVariant}/${nameVariant}.node.js`,
|
||||
`${packageName}/dist/nodes/**/${nameVariant}/${nameVariant}.node.js`,
|
||||
`${packageName}/nodes/*/${nameVariant}/${nameVariant}.node.js`,
|
||||
`${packageName}/nodes/**/${nameVariant}/${nameVariant}.node.js`,
|
||||
];
|
||||
|
||||
// Try standard patterns first
|
||||
@@ -105,6 +116,7 @@ export class NodeSourceExtractor {
|
||||
const result = await this.searchWithGlobPattern(basePath, pattern, packageName, nodeName);
|
||||
if (result) return result;
|
||||
}
|
||||
}
|
||||
|
||||
// If basePath contains .pnpm, search in pnpm structure
|
||||
if (basePath.includes('node_modules')) {
|
||||
@@ -250,13 +262,49 @@ export class NodeSourceExtractor {
|
||||
try {
|
||||
const sourceCode = await fs.readFile(fullPath, 'utf-8');
|
||||
|
||||
// Try to find credential file
|
||||
const credentialPath = fullPath.replace('.node.js', '.credentials.js');
|
||||
// Try to find credential files
|
||||
let credentialCode: string | undefined;
|
||||
|
||||
// First, try alongside the node file
|
||||
const credentialPath = fullPath.replace('.node.js', '.credentials.js');
|
||||
try {
|
||||
credentialCode = await fs.readFile(credentialPath, 'utf-8');
|
||||
} catch {
|
||||
// Credential file is optional
|
||||
// Try in the credentials directory
|
||||
const possibleCredentialPaths = [
|
||||
// Standard n8n structure: dist/credentials/NodeNameApi.credentials.js
|
||||
path.join(packageBasePath, packageName, 'dist/credentials', `${nodeName}Api.credentials.js`),
|
||||
path.join(packageBasePath, packageName, 'dist/credentials', `${nodeName}OAuth2Api.credentials.js`),
|
||||
path.join(packageBasePath, packageName, 'credentials', `${nodeName}Api.credentials.js`),
|
||||
path.join(packageBasePath, packageName, 'credentials', `${nodeName}OAuth2Api.credentials.js`),
|
||||
// Without packageName in path
|
||||
path.join(packageBasePath, 'dist/credentials', `${nodeName}Api.credentials.js`),
|
||||
path.join(packageBasePath, 'dist/credentials', `${nodeName}OAuth2Api.credentials.js`),
|
||||
path.join(packageBasePath, 'credentials', `${nodeName}Api.credentials.js`),
|
||||
path.join(packageBasePath, 'credentials', `${nodeName}OAuth2Api.credentials.js`),
|
||||
// Try relative to node location
|
||||
path.join(path.dirname(path.dirname(fullPath)), 'credentials', `${nodeName}Api.credentials.js`),
|
||||
path.join(path.dirname(path.dirname(fullPath)), 'credentials', `${nodeName}OAuth2Api.credentials.js`),
|
||||
path.join(path.dirname(path.dirname(path.dirname(fullPath))), 'credentials', `${nodeName}Api.credentials.js`),
|
||||
path.join(path.dirname(path.dirname(path.dirname(fullPath))), 'credentials', `${nodeName}OAuth2Api.credentials.js`),
|
||||
];
|
||||
|
||||
// Try to find any credential file
|
||||
const allCredentials: string[] = [];
|
||||
for (const credPath of possibleCredentialPaths) {
|
||||
try {
|
||||
const content = await fs.readFile(credPath, 'utf-8');
|
||||
allCredentials.push(content);
|
||||
logger.debug(`Found credential file at: ${credPath}`);
|
||||
} catch {
|
||||
// Continue searching
|
||||
}
|
||||
}
|
||||
|
||||
// If we found credentials, combine them
|
||||
if (allCredentials.length > 0) {
|
||||
credentialCode = allCredentials.join('\n\n// --- Next Credential File ---\n\n');
|
||||
}
|
||||
}
|
||||
|
||||
// Try to get package.json info
|
||||
@@ -266,12 +314,16 @@ export class NodeSourceExtractor {
|
||||
path.join(packageBasePath, packageName, 'package.json'),
|
||||
path.join(path.dirname(path.dirname(fullPath)), 'package.json'),
|
||||
path.join(path.dirname(path.dirname(path.dirname(fullPath))), 'package.json'),
|
||||
// Try to go up from the node location to find package.json
|
||||
path.join(fullPath.split('/dist/')[0], 'package.json'),
|
||||
path.join(fullPath.split('/nodes/')[0], 'package.json'),
|
||||
];
|
||||
|
||||
for (const packageJsonPath of possiblePackageJsonPaths) {
|
||||
try {
|
||||
const packageJson = await fs.readFile(packageJsonPath, 'utf-8');
|
||||
packageInfo = JSON.parse(packageJson);
|
||||
logger.debug(`Found package.json at: ${packageJsonPath}`);
|
||||
break;
|
||||
} catch {
|
||||
// Try next path
|
||||
@@ -295,10 +347,26 @@ export class NodeSourceExtractor {
|
||||
*/
|
||||
async listAvailableNodes(category?: string, search?: string): Promise<any[]> {
|
||||
const nodes: any[] = [];
|
||||
const seenNodes = new Set<string>(); // Track unique nodes
|
||||
|
||||
for (const basePath of this.n8nBasePaths) {
|
||||
try {
|
||||
await this.scanDirectoryForNodes(basePath, nodes, category, search);
|
||||
// Check for n8n-nodes-base specifically
|
||||
const n8nNodesBasePath = path.join(basePath, 'n8n-nodes-base', 'dist', 'nodes');
|
||||
try {
|
||||
await fs.access(n8nNodesBasePath);
|
||||
await this.scanDirectoryForNodes(n8nNodesBasePath, nodes, category, search, seenNodes);
|
||||
} catch {
|
||||
// Try without dist
|
||||
const altPath = path.join(basePath, 'n8n-nodes-base', 'nodes');
|
||||
try {
|
||||
await fs.access(altPath);
|
||||
await this.scanDirectoryForNodes(altPath, nodes, category, search, seenNodes);
|
||||
} catch {
|
||||
// Try the base path directly
|
||||
await this.scanDirectoryForNodes(basePath, nodes, category, search, seenNodes);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
logger.debug(`Failed to scan ${basePath}: ${error}`);
|
||||
}
|
||||
@@ -314,7 +382,8 @@ export class NodeSourceExtractor {
|
||||
dirPath: string,
|
||||
nodes: any[],
|
||||
category?: string,
|
||||
search?: string
|
||||
search?: string,
|
||||
seenNodes?: Set<string>
|
||||
): Promise<void> {
|
||||
try {
|
||||
const entries = await fs.readdir(dirPath, { withFileTypes: true });
|
||||
@@ -330,8 +399,15 @@ export class NodeSourceExtractor {
|
||||
const descriptionMatch = content.match(/description:\s*['"`]([^'"`]+)['"`]/);
|
||||
|
||||
if (nameMatch) {
|
||||
const nodeName = entry.name.replace('.node.js', '');
|
||||
|
||||
// Skip if we've already seen this node
|
||||
if (seenNodes && seenNodes.has(nodeName)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
const nodeInfo = {
|
||||
name: entry.name.replace('.node.js', ''),
|
||||
name: nodeName,
|
||||
displayName: nameMatch[1],
|
||||
description: descriptionMatch ? descriptionMatch[1] : '',
|
||||
location: fullPath,
|
||||
@@ -347,6 +423,9 @@ export class NodeSourceExtractor {
|
||||
}
|
||||
|
||||
nodes.push(nodeInfo);
|
||||
if (seenNodes) {
|
||||
seenNodes.add(nodeName);
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Skip files we can't read
|
||||
@@ -354,10 +433,10 @@ export class NodeSourceExtractor {
|
||||
} else if (entry.isDirectory()) {
|
||||
// Special handling for .pnpm directories
|
||||
if (entry.name === '.pnpm') {
|
||||
await this.scanPnpmDirectory(path.join(dirPath, entry.name), nodes, category, search);
|
||||
await this.scanPnpmDirectory(path.join(dirPath, entry.name), nodes, category, search, seenNodes);
|
||||
} else if (entry.name !== 'node_modules') {
|
||||
// Recursively scan subdirectories
|
||||
await this.scanDirectoryForNodes(path.join(dirPath, entry.name), nodes, category, search);
|
||||
await this.scanDirectoryForNodes(path.join(dirPath, entry.name), nodes, category, search, seenNodes);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -373,7 +452,8 @@ export class NodeSourceExtractor {
|
||||
pnpmPath: string,
|
||||
nodes: any[],
|
||||
category?: string,
|
||||
search?: string
|
||||
search?: string,
|
||||
seenNodes?: Set<string>
|
||||
): Promise<void> {
|
||||
try {
|
||||
const entries = await fs.readdir(pnpmPath);
|
||||
@@ -382,7 +462,7 @@ export class NodeSourceExtractor {
|
||||
const entryPath = path.join(pnpmPath, entry, 'node_modules');
|
||||
try {
|
||||
await fs.access(entryPath);
|
||||
await this.scanDirectoryForNodes(entryPath, nodes, category, search);
|
||||
await this.scanDirectoryForNodes(entryPath, nodes, category, search, seenNodes);
|
||||
} catch {
|
||||
// Skip if node_modules doesn't exist
|
||||
}
|
||||
|
||||
51
tests/debug-slack-doc.js
Normal file
51
tests/debug-slack-doc.js
Normal file
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { execSync } = require('child_process');
|
||||
const path = require('path');
|
||||
|
||||
const tempDir = path.join(process.cwd(), 'temp', 'n8n-docs');
|
||||
|
||||
console.log('🔍 Debugging Slack documentation search...\n');
|
||||
|
||||
// Search for all Slack related files
|
||||
console.log('All Slack-related markdown files:');
|
||||
try {
|
||||
const allSlackFiles = execSync(
|
||||
`find ${tempDir}/docs/integrations/builtin -name "*slack*.md" -type f`,
|
||||
{ encoding: 'utf-8' }
|
||||
).trim().split('\n');
|
||||
|
||||
allSlackFiles.forEach(file => {
|
||||
console.log(` - ${file}`);
|
||||
});
|
||||
} catch (error) {
|
||||
console.log(' No files found');
|
||||
}
|
||||
|
||||
console.log('\n📄 Checking file paths:');
|
||||
const possiblePaths = [
|
||||
'docs/integrations/builtin/app-nodes/n8n-nodes-base.Slack.md',
|
||||
'docs/integrations/builtin/app-nodes/n8n-nodes-base.slack.md',
|
||||
'docs/integrations/builtin/core-nodes/n8n-nodes-base.Slack.md',
|
||||
'docs/integrations/builtin/core-nodes/n8n-nodes-base.slack.md',
|
||||
'docs/integrations/builtin/trigger-nodes/n8n-nodes-base.Slack.md',
|
||||
'docs/integrations/builtin/trigger-nodes/n8n-nodes-base.slack.md',
|
||||
'docs/integrations/builtin/credentials/slack.md',
|
||||
];
|
||||
|
||||
const fs = require('fs');
|
||||
possiblePaths.forEach(p => {
|
||||
const fullPath = path.join(tempDir, p);
|
||||
const exists = fs.existsSync(fullPath);
|
||||
console.log(` ${exists ? '✓' : '✗'} ${p}`);
|
||||
|
||||
if (exists) {
|
||||
// Read first few lines
|
||||
const content = fs.readFileSync(fullPath, 'utf-8');
|
||||
const lines = content.split('\n').slice(0, 10);
|
||||
const title = lines.find(l => l.includes('title:'));
|
||||
if (title) {
|
||||
console.log(` Title: ${title.trim()}`);
|
||||
}
|
||||
}
|
||||
});
|
||||
112
tests/demo-enhanced-documentation.js
Normal file
112
tests/demo-enhanced-documentation.js
Normal file
@@ -0,0 +1,112 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { EnhancedDocumentationFetcher } = require('../dist/utils/enhanced-documentation-fetcher');
|
||||
|
||||
async function demoEnhancedDocumentation() {
|
||||
console.log('=== Enhanced Documentation Parser Demo ===\n');
|
||||
console.log('This demo shows how the enhanced DocumentationFetcher extracts rich content from n8n documentation.\n');
|
||||
|
||||
const fetcher = new EnhancedDocumentationFetcher();
|
||||
|
||||
try {
|
||||
// Demo 1: Slack node (complex app node with many operations)
|
||||
console.log('1. SLACK NODE DOCUMENTATION');
|
||||
console.log('=' .repeat(50));
|
||||
const slackDoc = await fetcher.getEnhancedNodeDocumentation('n8n-nodes-base.slack');
|
||||
|
||||
if (slackDoc) {
|
||||
console.log('\n📄 Basic Information:');
|
||||
console.log(` • Title: ${slackDoc.title}`);
|
||||
console.log(` • Description: ${slackDoc.description}`);
|
||||
console.log(` • URL: ${slackDoc.url}`);
|
||||
|
||||
console.log('\n📊 Content Statistics:');
|
||||
console.log(` • Operations: ${slackDoc.operations?.length || 0} operations across multiple resources`);
|
||||
console.log(` • API Methods: ${slackDoc.apiMethods?.length || 0} mapped to Slack API endpoints`);
|
||||
console.log(` • Examples: ${slackDoc.examples?.length || 0} code examples`);
|
||||
console.log(` • Resources: ${slackDoc.relatedResources?.length || 0} related documentation links`);
|
||||
console.log(` • Scopes: ${slackDoc.requiredScopes?.length || 0} OAuth scopes`);
|
||||
|
||||
// Show operations breakdown
|
||||
if (slackDoc.operations && slackDoc.operations.length > 0) {
|
||||
console.log('\n🔧 Operations by Resource:');
|
||||
const resourceMap = new Map();
|
||||
slackDoc.operations.forEach(op => {
|
||||
if (!resourceMap.has(op.resource)) {
|
||||
resourceMap.set(op.resource, []);
|
||||
}
|
||||
resourceMap.get(op.resource).push(op);
|
||||
});
|
||||
|
||||
for (const [resource, ops] of resourceMap) {
|
||||
console.log(`\n ${resource} (${ops.length} operations):`);
|
||||
ops.slice(0, 5).forEach(op => {
|
||||
console.log(` • ${op.operation}: ${op.description}`);
|
||||
});
|
||||
if (ops.length > 5) {
|
||||
console.log(` ... and ${ops.length - 5} more`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Show API method mappings
|
||||
if (slackDoc.apiMethods && slackDoc.apiMethods.length > 0) {
|
||||
console.log('\n🔗 API Method Mappings (sample):');
|
||||
slackDoc.apiMethods.slice(0, 5).forEach(api => {
|
||||
console.log(` • ${api.resource}.${api.operation} → ${api.apiMethod}`);
|
||||
console.log(` URL: ${api.apiUrl}`);
|
||||
});
|
||||
if (slackDoc.apiMethods.length > 5) {
|
||||
console.log(` ... and ${slackDoc.apiMethods.length - 5} more mappings`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Demo 2: If node (core node with conditions)
|
||||
console.log('\n\n2. IF NODE DOCUMENTATION');
|
||||
console.log('=' .repeat(50));
|
||||
const ifDoc = await fetcher.getEnhancedNodeDocumentation('n8n-nodes-base.if');
|
||||
|
||||
if (ifDoc) {
|
||||
console.log('\n📄 Basic Information:');
|
||||
console.log(` • Title: ${ifDoc.title}`);
|
||||
console.log(` • Description: ${ifDoc.description}`);
|
||||
console.log(` • URL: ${ifDoc.url}`);
|
||||
|
||||
if (ifDoc.relatedResources && ifDoc.relatedResources.length > 0) {
|
||||
console.log('\n📚 Related Resources:');
|
||||
ifDoc.relatedResources.forEach(res => {
|
||||
console.log(` • ${res.title} (${res.type})`);
|
||||
console.log(` ${res.url}`);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Demo 3: Summary of enhanced parsing capabilities
|
||||
console.log('\n\n3. ENHANCED PARSING CAPABILITIES');
|
||||
console.log('=' .repeat(50));
|
||||
console.log('\nThe enhanced DocumentationFetcher can extract:');
|
||||
console.log(' ✓ Markdown frontmatter (metadata, tags, priority)');
|
||||
console.log(' ✓ Operations with resource grouping and descriptions');
|
||||
console.log(' ✓ API method mappings from markdown tables');
|
||||
console.log(' ✓ Code examples (JSON, JavaScript, YAML)');
|
||||
console.log(' ✓ Template references');
|
||||
console.log(' ✓ Related resources and documentation links');
|
||||
console.log(' ✓ Required OAuth scopes');
|
||||
console.log('\nThis rich content enables AI agents to:');
|
||||
console.log(' • Understand node capabilities in detail');
|
||||
console.log(' • Map operations to actual API endpoints');
|
||||
console.log(' • Provide accurate examples and usage patterns');
|
||||
console.log(' • Navigate related documentation');
|
||||
console.log(' • Understand authentication requirements');
|
||||
|
||||
} catch (error) {
|
||||
console.error('\nError:', error);
|
||||
} finally {
|
||||
await fetcher.cleanup();
|
||||
console.log('\n\n✓ Demo completed');
|
||||
}
|
||||
}
|
||||
|
||||
// Run the demo
|
||||
demoEnhancedDocumentation().catch(console.error);
|
||||
File diff suppressed because one or more lines are too long
94
tests/test-complete-fix.js
Executable file
94
tests/test-complete-fix.js
Executable file
@@ -0,0 +1,94 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { NodeDocumentationService } = require('../dist/services/node-documentation-service');
|
||||
|
||||
async function testCompleteFix() {
|
||||
console.log('=== Testing Complete Documentation Fix ===\n');
|
||||
|
||||
const service = new NodeDocumentationService('./data/test-nodes-v2.db');
|
||||
|
||||
try {
|
||||
// First check if we have any nodes
|
||||
const existingNodes = await service.listNodes();
|
||||
console.log(`📊 Current database has ${existingNodes.length} nodes`);
|
||||
|
||||
if (existingNodes.length === 0) {
|
||||
console.log('\n🔄 Rebuilding database with fixed documentation fetcher...');
|
||||
const stats = await service.rebuildDatabase();
|
||||
console.log(`\n✅ Rebuild complete:`);
|
||||
console.log(` - Total nodes found: ${stats.total}`);
|
||||
console.log(` - Successfully processed: ${stats.successful}`);
|
||||
console.log(` - Failed: ${stats.failed}`);
|
||||
|
||||
if (stats.errors.length > 0) {
|
||||
console.log('\n⚠️ Errors encountered:');
|
||||
stats.errors.slice(0, 5).forEach(err => console.log(` - ${err}`));
|
||||
}
|
||||
}
|
||||
|
||||
// Test specific nodes
|
||||
console.log('\n📋 Testing specific nodes:');
|
||||
|
||||
const testNodes = ['slack', 'if', 'httpRequest', 'webhook'];
|
||||
|
||||
for (const nodeName of testNodes) {
|
||||
const nodeInfo = await service.getNodeInfo(`n8n-nodes-base.${nodeName}`);
|
||||
|
||||
if (nodeInfo) {
|
||||
console.log(`\n✅ ${nodeInfo.displayName || nodeName}:`);
|
||||
console.log(` - Type: ${nodeInfo.nodeType}`);
|
||||
console.log(` - Description: ${nodeInfo.description?.substring(0, 80)}...`);
|
||||
console.log(` - Has source code: ${!!nodeInfo.sourceCode}`);
|
||||
console.log(` - Has documentation: ${!!nodeInfo.documentation}`);
|
||||
console.log(` - Documentation URL: ${nodeInfo.documentationUrl || 'N/A'}`);
|
||||
console.log(` - Has example: ${!!nodeInfo.exampleWorkflow}`);
|
||||
console.log(` - Category: ${nodeInfo.category || 'N/A'}`);
|
||||
|
||||
// Check if it's getting the right documentation
|
||||
if (nodeInfo.documentation) {
|
||||
const isCredentialDoc = nodeInfo.documentation.includes('credentials') &&
|
||||
!nodeInfo.documentation.includes('node documentation');
|
||||
console.log(` - Is credential doc: ${isCredentialDoc} ${isCredentialDoc ? '❌' : '✅'}`);
|
||||
}
|
||||
} else {
|
||||
console.log(`\n❌ ${nodeName}: Not found in database`);
|
||||
}
|
||||
}
|
||||
|
||||
// Test search functionality
|
||||
console.log('\n🔍 Testing search functionality:');
|
||||
|
||||
const searchTests = [
|
||||
{ query: 'webhook', label: 'Webhook nodes' },
|
||||
{ query: 'http', label: 'HTTP nodes' },
|
||||
{ query: 'slack', label: 'Slack nodes' }
|
||||
];
|
||||
|
||||
for (const test of searchTests) {
|
||||
const results = await service.searchNodes({ query: test.query });
|
||||
console.log(`\n ${test.label}: ${results.length} results`);
|
||||
results.slice(0, 3).forEach(node => {
|
||||
console.log(` - ${node.displayName} (${node.nodeType})`);
|
||||
});
|
||||
}
|
||||
|
||||
// Get final statistics
|
||||
console.log('\n📊 Final database statistics:');
|
||||
const stats = service.getStatistics();
|
||||
console.log(` - Total nodes: ${stats.totalNodes}`);
|
||||
console.log(` - Nodes with documentation: ${stats.nodesWithDocs}`);
|
||||
console.log(` - Nodes with examples: ${stats.nodesWithExamples}`);
|
||||
console.log(` - Trigger nodes: ${stats.triggerNodes}`);
|
||||
console.log(` - Webhook nodes: ${stats.webhookNodes}`);
|
||||
|
||||
console.log('\n✅ All tests completed!');
|
||||
|
||||
} catch (error) {
|
||||
console.error('\n❌ Test failed:', error);
|
||||
process.exit(1);
|
||||
} finally {
|
||||
service.close();
|
||||
}
|
||||
}
|
||||
|
||||
testCompleteFix().catch(console.error);
|
||||
38
tests/test-debug-enhanced.js
Normal file
38
tests/test-debug-enhanced.js
Normal file
@@ -0,0 +1,38 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { EnhancedDocumentationFetcher } = require('../dist/utils/enhanced-documentation-fetcher');
|
||||
|
||||
async function debugTest() {
|
||||
console.log('=== Debug Enhanced Documentation ===\n');
|
||||
|
||||
const fetcher = new EnhancedDocumentationFetcher();
|
||||
|
||||
try {
|
||||
await fetcher.ensureDocsRepository();
|
||||
|
||||
// Test Slack documentation parsing
|
||||
console.log('Testing Slack documentation...');
|
||||
const slackDoc = await fetcher.getEnhancedNodeDocumentation('n8n-nodes-base.slack');
|
||||
|
||||
if (slackDoc) {
|
||||
console.log('\nSlack Documentation:');
|
||||
console.log('- Operations found:', slackDoc.operations?.length || 0);
|
||||
|
||||
// Show raw markdown around operations section
|
||||
const operationsIndex = slackDoc.markdown.indexOf('## Operations');
|
||||
if (operationsIndex > -1) {
|
||||
console.log('\nRaw markdown around Operations section:');
|
||||
console.log('---');
|
||||
console.log(slackDoc.markdown.substring(operationsIndex, operationsIndex + 1000));
|
||||
console.log('---');
|
||||
}
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error:', error);
|
||||
} finally {
|
||||
await fetcher.cleanup();
|
||||
}
|
||||
}
|
||||
|
||||
debugTest().catch(console.error);
|
||||
57
tests/test-docs-fix.js
Executable file
57
tests/test-docs-fix.js
Executable file
@@ -0,0 +1,57 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { DocumentationFetcher } = require('../dist/utils/documentation-fetcher');
|
||||
const { NodeSourceExtractor } = require('../dist/utils/node-source-extractor');
|
||||
|
||||
async function testDocsFix() {
|
||||
console.log('=== Testing Documentation Fix ===\n');
|
||||
|
||||
const docsFetcher = new DocumentationFetcher();
|
||||
const extractor = new NodeSourceExtractor();
|
||||
|
||||
try {
|
||||
// Test nodes
|
||||
const testNodes = [
|
||||
'n8n-nodes-base.slack',
|
||||
'n8n-nodes-base.if',
|
||||
'n8n-nodes-base.httpRequest',
|
||||
'n8n-nodes-base.webhook'
|
||||
];
|
||||
|
||||
for (const nodeType of testNodes) {
|
||||
console.log(`\n📋 Testing ${nodeType}:`);
|
||||
|
||||
// Test documentation fetching
|
||||
const docs = await docsFetcher.getNodeDocumentation(nodeType);
|
||||
if (docs) {
|
||||
console.log(` ✅ Documentation found`);
|
||||
console.log(` 📄 URL: ${docs.url}`);
|
||||
const titleMatch = docs.markdown.match(/title:\s*(.+)/);
|
||||
if (titleMatch) {
|
||||
console.log(` 📝 Title: ${titleMatch[1]}`);
|
||||
}
|
||||
console.log(` 📏 Length: ${docs.markdown.length} characters`);
|
||||
console.log(` 🔧 Has examples: ${docs.examples && docs.examples.length > 0}`);
|
||||
} else {
|
||||
console.log(` ❌ No documentation found`);
|
||||
}
|
||||
|
||||
// Test source extraction
|
||||
try {
|
||||
const source = await extractor.extractNodeSource(nodeType);
|
||||
console.log(` ✅ Source code found at: ${source.location}`);
|
||||
} catch (error) {
|
||||
console.log(` ❌ Source extraction failed: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('\n✅ Test completed!');
|
||||
|
||||
} catch (error) {
|
||||
console.error('\n❌ Test failed:', error);
|
||||
} finally {
|
||||
await docsFetcher.cleanup();
|
||||
}
|
||||
}
|
||||
|
||||
testDocsFix().catch(console.error);
|
||||
141
tests/test-enhanced-documentation.js
Normal file
141
tests/test-enhanced-documentation.js
Normal file
@@ -0,0 +1,141 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { EnhancedDocumentationFetcher } = require('../dist/utils/enhanced-documentation-fetcher');
|
||||
const { EnhancedSQLiteStorageService } = require('../dist/services/enhanced-sqlite-storage-service');
|
||||
const { NodeSourceExtractor } = require('../dist/utils/node-source-extractor');
|
||||
|
||||
async function testEnhancedDocumentation() {
|
||||
console.log('=== Testing Enhanced Documentation Fetcher ===\n');
|
||||
|
||||
const fetcher = new EnhancedDocumentationFetcher();
|
||||
const storage = new EnhancedSQLiteStorageService('./data/test-enhanced.db');
|
||||
const extractor = new NodeSourceExtractor();
|
||||
|
||||
try {
|
||||
// Test 1: Fetch and parse Slack node documentation
|
||||
console.log('1. Testing Slack node documentation parsing...');
|
||||
const slackDoc = await fetcher.getEnhancedNodeDocumentation('n8n-nodes-base.slack');
|
||||
|
||||
if (slackDoc) {
|
||||
console.log('\n✓ Slack Documentation Found:');
|
||||
console.log(` - Title: ${slackDoc.title}`);
|
||||
console.log(` - Description: ${slackDoc.description}`);
|
||||
console.log(` - URL: ${slackDoc.url}`);
|
||||
console.log(` - Operations: ${slackDoc.operations?.length || 0} found`);
|
||||
console.log(` - API Methods: ${slackDoc.apiMethods?.length || 0} found`);
|
||||
console.log(` - Examples: ${slackDoc.examples?.length || 0} found`);
|
||||
console.log(` - Required Scopes: ${slackDoc.requiredScopes?.length || 0} found`);
|
||||
|
||||
// Show sample operations
|
||||
if (slackDoc.operations && slackDoc.operations.length > 0) {
|
||||
console.log('\n Sample Operations:');
|
||||
slackDoc.operations.slice(0, 5).forEach(op => {
|
||||
console.log(` - ${op.resource}.${op.operation}: ${op.description}`);
|
||||
});
|
||||
}
|
||||
|
||||
// Show sample API mappings
|
||||
if (slackDoc.apiMethods && slackDoc.apiMethods.length > 0) {
|
||||
console.log('\n Sample API Mappings:');
|
||||
slackDoc.apiMethods.slice(0, 5).forEach(api => {
|
||||
console.log(` - ${api.resource}.${api.operation} → ${api.apiMethod}`);
|
||||
});
|
||||
}
|
||||
} else {
|
||||
console.log('✗ Slack documentation not found');
|
||||
}
|
||||
|
||||
// Test 2: Test with If node (core node)
|
||||
console.log('\n\n2. Testing If node documentation parsing...');
|
||||
const ifDoc = await fetcher.getEnhancedNodeDocumentation('n8n-nodes-base.if');
|
||||
|
||||
if (ifDoc) {
|
||||
console.log('\n✓ If Documentation Found:');
|
||||
console.log(` - Title: ${ifDoc.title}`);
|
||||
console.log(` - Description: ${ifDoc.description}`);
|
||||
console.log(` - Examples: ${ifDoc.examples?.length || 0} found`);
|
||||
console.log(` - Related Resources: ${ifDoc.relatedResources?.length || 0} found`);
|
||||
}
|
||||
|
||||
// Test 3: Store node with documentation
|
||||
console.log('\n\n3. Testing node storage with documentation...');
|
||||
|
||||
// Extract a node
|
||||
const nodeInfo = await extractor.extractNodeSource('n8n-nodes-base.slack');
|
||||
if (nodeInfo) {
|
||||
const storedNode = await storage.storeNodeWithDocumentation(nodeInfo);
|
||||
|
||||
console.log('\n✓ Node stored successfully:');
|
||||
console.log(` - Node Type: ${storedNode.nodeType}`);
|
||||
console.log(` - Has Documentation: ${!!storedNode.documentationMarkdown}`);
|
||||
console.log(` - Operations: ${storedNode.operationCount}`);
|
||||
console.log(` - API Methods: ${storedNode.apiMethodCount}`);
|
||||
console.log(` - Examples: ${storedNode.exampleCount}`);
|
||||
console.log(` - Resources: ${storedNode.resourceCount}`);
|
||||
console.log(` - Scopes: ${storedNode.scopeCount}`);
|
||||
|
||||
// Get detailed operations
|
||||
const operations = await storage.getNodeOperations(storedNode.id);
|
||||
if (operations.length > 0) {
|
||||
console.log('\n Stored Operations (first 5):');
|
||||
operations.slice(0, 5).forEach(op => {
|
||||
console.log(` - ${op.resource}.${op.operation}: ${op.description}`);
|
||||
});
|
||||
}
|
||||
|
||||
// Get examples
|
||||
const examples = await storage.getNodeExamples(storedNode.id);
|
||||
if (examples.length > 0) {
|
||||
console.log('\n Stored Examples:');
|
||||
examples.forEach(ex => {
|
||||
console.log(` - ${ex.title || 'Untitled'} (${ex.type}): ${ex.code.length} chars`);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Test 4: Search with enhanced FTS
|
||||
console.log('\n\n4. Testing enhanced search...');
|
||||
|
||||
const searchResults = await storage.searchNodes({ query: 'slack message' });
|
||||
console.log(`\n✓ Search Results for "slack message": ${searchResults.length} nodes found`);
|
||||
|
||||
if (searchResults.length > 0) {
|
||||
console.log(' First result:');
|
||||
const result = searchResults[0];
|
||||
console.log(` - ${result.displayName || result.name} (${result.nodeType})`);
|
||||
console.log(` - Documentation: ${result.documentationTitle || 'No title'}`);
|
||||
}
|
||||
|
||||
// Test 5: Get statistics
|
||||
console.log('\n\n5. Getting enhanced statistics...');
|
||||
const stats = await storage.getEnhancedStatistics();
|
||||
|
||||
console.log('\n✓ Enhanced Statistics:');
|
||||
console.log(` - Total Nodes: ${stats.totalNodes}`);
|
||||
console.log(` - Nodes with Documentation: ${stats.nodesWithDocumentation}`);
|
||||
console.log(` - Documentation Coverage: ${stats.documentationCoverage}%`);
|
||||
console.log(` - Total Operations: ${stats.totalOperations}`);
|
||||
console.log(` - Total API Methods: ${stats.totalApiMethods}`);
|
||||
console.log(` - Total Examples: ${stats.totalExamples}`);
|
||||
console.log(` - Total Resources: ${stats.totalResources}`);
|
||||
console.log(` - Total Scopes: ${stats.totalScopes}`);
|
||||
|
||||
if (stats.topDocumentedNodes && stats.topDocumentedNodes.length > 0) {
|
||||
console.log('\n Top Documented Nodes:');
|
||||
stats.topDocumentedNodes.slice(0, 3).forEach(node => {
|
||||
console.log(` - ${node.display_name || node.name}: ${node.operation_count} operations, ${node.example_count} examples`);
|
||||
});
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error during testing:', error);
|
||||
} finally {
|
||||
// Cleanup
|
||||
storage.close();
|
||||
await fetcher.cleanup();
|
||||
console.log('\n\n✓ Test completed and cleaned up');
|
||||
}
|
||||
}
|
||||
|
||||
// Run the test
|
||||
testEnhancedDocumentation().catch(console.error);
|
||||
156
tests/test-enhanced-final.js
Normal file
156
tests/test-enhanced-final.js
Normal file
@@ -0,0 +1,156 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { EnhancedDocumentationFetcher } = require('../dist/utils/enhanced-documentation-fetcher');
|
||||
const { EnhancedSQLiteStorageService } = require('../dist/services/enhanced-sqlite-storage-service');
|
||||
const { NodeSourceExtractor } = require('../dist/utils/node-source-extractor');
|
||||
|
||||
async function testEnhancedDocumentation() {
|
||||
console.log('=== Enhanced Documentation Parser Test ===\n');
|
||||
|
||||
const fetcher = new EnhancedDocumentationFetcher();
|
||||
const extractor = new NodeSourceExtractor();
|
||||
|
||||
try {
|
||||
// Test 1: Parse Slack documentation
|
||||
console.log('1. Parsing Slack node documentation...');
|
||||
const slackDoc = await fetcher.getEnhancedNodeDocumentation('n8n-nodes-base.slack');
|
||||
|
||||
if (slackDoc) {
|
||||
console.log('\n✓ Slack Documentation Parsed:');
|
||||
console.log(` Title: ${slackDoc.title}`);
|
||||
console.log(` Description: ${slackDoc.description?.substring(0, 100)}...`);
|
||||
console.log(` URL: ${slackDoc.url}`);
|
||||
console.log(` Operations: ${slackDoc.operations?.length || 0} found`);
|
||||
console.log(` API Methods: ${slackDoc.apiMethods?.length || 0} found`);
|
||||
console.log(` Related Resources: ${slackDoc.relatedResources?.length || 0} found`);
|
||||
|
||||
// Show sample operations
|
||||
if (slackDoc.operations && slackDoc.operations.length > 0) {
|
||||
console.log('\n Sample Operations (first 10):');
|
||||
slackDoc.operations.slice(0, 10).forEach((op, i) => {
|
||||
console.log(` ${i + 1}. ${op.resource}.${op.operation}: ${op.description}`);
|
||||
});
|
||||
}
|
||||
|
||||
// Show sample API mappings
|
||||
if (slackDoc.apiMethods && slackDoc.apiMethods.length > 0) {
|
||||
console.log('\n Sample API Method Mappings (first 5):');
|
||||
slackDoc.apiMethods.slice(0, 5).forEach((api, i) => {
|
||||
console.log(` ${i + 1}. ${api.resource}.${api.operation} → ${api.apiMethod} (${api.apiUrl})`);
|
||||
});
|
||||
}
|
||||
|
||||
// Show related resources
|
||||
if (slackDoc.relatedResources && slackDoc.relatedResources.length > 0) {
|
||||
console.log('\n Related Resources:');
|
||||
slackDoc.relatedResources.forEach((res, i) => {
|
||||
console.log(` ${i + 1}. ${res.title} (${res.type}): ${res.url}`);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Test 2: Parse HTTP Request documentation (if available)
|
||||
console.log('\n\n2. Parsing HTTP Request node documentation...');
|
||||
const httpDoc = await fetcher.getEnhancedNodeDocumentation('n8n-nodes-base.httpRequest');
|
||||
|
||||
if (httpDoc) {
|
||||
console.log('\n✓ HTTP Request Documentation Parsed:');
|
||||
console.log(` Title: ${httpDoc.title}`);
|
||||
console.log(` Examples: ${httpDoc.examples?.length || 0} found`);
|
||||
|
||||
if (httpDoc.examples && httpDoc.examples.length > 0) {
|
||||
console.log('\n Code Examples:');
|
||||
httpDoc.examples.forEach((ex, i) => {
|
||||
console.log(` ${i + 1}. ${ex.title || 'Example'} (${ex.type}): ${ex.code.length} characters`);
|
||||
});
|
||||
}
|
||||
} else {
|
||||
console.log(' HTTP Request documentation not found');
|
||||
}
|
||||
|
||||
// Test 3: Database storage test with smaller database
|
||||
console.log('\n\n3. Testing enhanced database storage...');
|
||||
const storage = new EnhancedSQLiteStorageService('./data/demo-enhanced.db');
|
||||
|
||||
try {
|
||||
// Store Slack node with documentation
|
||||
const slackNodeInfo = await extractor.extractNodeSource('n8n-nodes-base.slack');
|
||||
if (slackNodeInfo) {
|
||||
const storedNode = await storage.storeNodeWithDocumentation(slackNodeInfo);
|
||||
|
||||
console.log('\n✓ Slack node stored with documentation:');
|
||||
console.log(` Node Type: ${storedNode.nodeType}`);
|
||||
console.log(` Documentation: ${storedNode.documentationTitle || 'No title'}`);
|
||||
console.log(` Operations stored: ${storedNode.operationCount}`);
|
||||
console.log(` API methods stored: ${storedNode.apiMethodCount}`);
|
||||
console.log(` Examples stored: ${storedNode.exampleCount}`);
|
||||
console.log(` Resources stored: ${storedNode.resourceCount}`);
|
||||
}
|
||||
|
||||
// Store a few more nodes
|
||||
const nodeTypes = ['n8n-nodes-base.if', 'n8n-nodes-base.webhook'];
|
||||
for (const nodeType of nodeTypes) {
|
||||
try {
|
||||
const nodeInfo = await extractor.extractNodeSource(nodeType);
|
||||
if (nodeInfo) {
|
||||
await storage.storeNodeWithDocumentation(nodeInfo);
|
||||
console.log(` ✓ Stored ${nodeType}`);
|
||||
}
|
||||
} catch (e) {
|
||||
console.log(` ✗ Failed to store ${nodeType}: ${e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Test search functionality
|
||||
console.log('\n\n4. Testing enhanced search...');
|
||||
|
||||
const searchTests = [
|
||||
{ query: 'slack', description: 'Search for "slack"' },
|
||||
{ query: 'message send', description: 'Search for "message send"' },
|
||||
{ query: 'webhook', description: 'Search for "webhook"' }
|
||||
];
|
||||
|
||||
for (const test of searchTests) {
|
||||
const results = await storage.searchNodes({ query: test.query });
|
||||
console.log(`\n ${test.description}: ${results.length} results`);
|
||||
if (results.length > 0) {
|
||||
const first = results[0];
|
||||
console.log(` Top result: ${first.displayName || first.name} (${first.nodeType})`);
|
||||
if (first.documentationTitle) {
|
||||
console.log(` Documentation: ${first.documentationTitle}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Get final statistics
|
||||
console.log('\n\n5. Database Statistics:');
|
||||
const stats = await storage.getEnhancedStatistics();
|
||||
|
||||
console.log(` Total Nodes: ${stats.totalNodes}`);
|
||||
console.log(` Nodes with Documentation: ${stats.nodesWithDocumentation} (${stats.documentationCoverage}% coverage)`);
|
||||
console.log(` Total Operations: ${stats.totalOperations}`);
|
||||
console.log(` Total API Methods: ${stats.totalApiMethods}`);
|
||||
console.log(` Total Examples: ${stats.totalExamples}`);
|
||||
console.log(` Total Resources: ${stats.totalResources}`);
|
||||
|
||||
if (stats.topDocumentedNodes && stats.topDocumentedNodes.length > 0) {
|
||||
console.log('\n Best Documented Nodes:');
|
||||
stats.topDocumentedNodes.forEach((node, i) => {
|
||||
console.log(` ${i + 1}. ${node.display_name || node.name}: ${node.operation_count} operations, ${node.example_count} examples`);
|
||||
});
|
||||
}
|
||||
|
||||
} finally {
|
||||
storage.close();
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('\nError:', error);
|
||||
} finally {
|
||||
await fetcher.cleanup();
|
||||
console.log('\n\n✓ Test completed and cleaned up');
|
||||
}
|
||||
}
|
||||
|
||||
// Run the test
|
||||
testEnhancedDocumentation().catch(console.error);
|
||||
163
tests/test-enhanced-integration.js
Normal file
163
tests/test-enhanced-integration.js
Normal file
@@ -0,0 +1,163 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { DocumentationFetcher } = require('../dist/utils/documentation-fetcher');
|
||||
const { NodeDocumentationService } = require('../dist/services/node-documentation-service');
|
||||
|
||||
async function testEnhancedIntegration() {
|
||||
console.log('🧪 Testing Enhanced Documentation Integration...\n');
|
||||
|
||||
// Test 1: DocumentationFetcher backward compatibility
|
||||
console.log('1️⃣ Testing DocumentationFetcher backward compatibility...');
|
||||
const docFetcher = new DocumentationFetcher();
|
||||
|
||||
try {
|
||||
// Test getNodeDocumentation (backward compatible method)
|
||||
const simpleDoc = await docFetcher.getNodeDocumentation('n8n-nodes-base.slack');
|
||||
if (simpleDoc) {
|
||||
console.log(' ✅ Simple documentation format works');
|
||||
console.log(` - Has markdown: ${!!simpleDoc.markdown}`);
|
||||
console.log(` - Has URL: ${!!simpleDoc.url}`);
|
||||
console.log(` - Has examples: ${simpleDoc.examples?.length || 0}`);
|
||||
}
|
||||
|
||||
// Test getEnhancedNodeDocumentation (new method)
|
||||
const enhancedDoc = await docFetcher.getEnhancedNodeDocumentation('n8n-nodes-base.slack');
|
||||
if (enhancedDoc) {
|
||||
console.log(' ✅ Enhanced documentation format works');
|
||||
console.log(` - Title: ${enhancedDoc.title || 'N/A'}`);
|
||||
console.log(` - Operations: ${enhancedDoc.operations?.length || 0}`);
|
||||
console.log(` - API Methods: ${enhancedDoc.apiMethods?.length || 0}`);
|
||||
console.log(` - Examples: ${enhancedDoc.examples?.length || 0}`);
|
||||
console.log(` - Templates: ${enhancedDoc.templates?.length || 0}`);
|
||||
console.log(` - Related Resources: ${enhancedDoc.relatedResources?.length || 0}`);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(' ❌ DocumentationFetcher test failed:', error.message);
|
||||
}
|
||||
|
||||
// Test 2: NodeDocumentationService with enhanced fields
|
||||
console.log('\n2️⃣ Testing NodeDocumentationService enhanced schema...');
|
||||
const docService = new NodeDocumentationService('data/test-enhanced-docs.db');
|
||||
|
||||
try {
|
||||
// Store a test node with enhanced documentation
|
||||
const testNode = {
|
||||
nodeType: 'test.enhanced-node',
|
||||
name: 'enhanced-node',
|
||||
displayName: 'Enhanced Test Node',
|
||||
description: 'A test node with enhanced documentation',
|
||||
sourceCode: 'const testCode = "example";',
|
||||
packageName: 'test-package',
|
||||
documentation: '# Test Documentation',
|
||||
documentationUrl: 'https://example.com/docs',
|
||||
documentationTitle: 'Enhanced Test Node Documentation',
|
||||
operations: [
|
||||
{
|
||||
resource: 'Message',
|
||||
operation: 'Send',
|
||||
description: 'Send a message'
|
||||
}
|
||||
],
|
||||
apiMethods: [
|
||||
{
|
||||
resource: 'Message',
|
||||
operation: 'Send',
|
||||
apiMethod: 'chat.postMessage',
|
||||
apiUrl: 'https://api.slack.com/methods/chat.postMessage'
|
||||
}
|
||||
],
|
||||
documentationExamples: [
|
||||
{
|
||||
title: 'Send Message Example',
|
||||
type: 'json',
|
||||
code: '{"text": "Hello World"}'
|
||||
}
|
||||
],
|
||||
templates: [
|
||||
{
|
||||
name: 'Basic Message Template',
|
||||
description: 'Simple message sending template'
|
||||
}
|
||||
],
|
||||
relatedResources: [
|
||||
{
|
||||
title: 'API Documentation',
|
||||
url: 'https://api.slack.com',
|
||||
type: 'api'
|
||||
}
|
||||
],
|
||||
requiredScopes: ['chat:write'],
|
||||
hasCredentials: true,
|
||||
isTrigger: false,
|
||||
isWebhook: false
|
||||
};
|
||||
|
||||
await docService.storeNode(testNode);
|
||||
console.log(' ✅ Stored node with enhanced documentation');
|
||||
|
||||
// Retrieve and verify
|
||||
const retrieved = await docService.getNodeInfo('test.enhanced-node');
|
||||
if (retrieved) {
|
||||
console.log(' ✅ Retrieved node with enhanced fields:');
|
||||
console.log(` - Has operations: ${!!retrieved.operations}`);
|
||||
console.log(` - Has API methods: ${!!retrieved.apiMethods}`);
|
||||
console.log(` - Has documentation examples: ${!!retrieved.documentationExamples}`);
|
||||
console.log(` - Has templates: ${!!retrieved.templates}`);
|
||||
console.log(` - Has related resources: ${!!retrieved.relatedResources}`);
|
||||
console.log(` - Has required scopes: ${!!retrieved.requiredScopes}`);
|
||||
}
|
||||
|
||||
// Test search
|
||||
const searchResults = await docService.searchNodes({ query: 'enhanced' });
|
||||
console.log(` ✅ Search found ${searchResults.length} results`);
|
||||
|
||||
} catch (error) {
|
||||
console.error(' ❌ NodeDocumentationService test failed:', error.message);
|
||||
} finally {
|
||||
docService.close();
|
||||
}
|
||||
|
||||
// Test 3: MCP Server integration
|
||||
console.log('\n3️⃣ Testing MCP Server integration...');
|
||||
try {
|
||||
const { N8NMCPServer } = require('../dist/mcp/server');
|
||||
console.log(' ✅ MCP Server loads with enhanced documentation support');
|
||||
|
||||
// Check if new tools are available
|
||||
const { n8nTools } = require('../dist/mcp/tools');
|
||||
const enhancedTools = [
|
||||
'get_node_documentation',
|
||||
'search_node_documentation',
|
||||
'get_node_operations',
|
||||
'get_node_examples'
|
||||
];
|
||||
|
||||
const hasAllTools = enhancedTools.every(toolName =>
|
||||
n8nTools.some(tool => tool.name === toolName)
|
||||
);
|
||||
|
||||
if (hasAllTools) {
|
||||
console.log(' ✅ All enhanced documentation tools are available');
|
||||
enhancedTools.forEach(toolName => {
|
||||
const tool = n8nTools.find(t => t.name === toolName);
|
||||
console.log(` - ${toolName}: ${tool.description}`);
|
||||
});
|
||||
} else {
|
||||
console.log(' ⚠️ Some enhanced tools are missing');
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error(' ❌ MCP Server integration test failed:', error.message);
|
||||
}
|
||||
|
||||
console.log('\n✨ Enhanced documentation integration tests completed!');
|
||||
|
||||
// Cleanup
|
||||
await docFetcher.cleanup();
|
||||
}
|
||||
|
||||
// Run tests
|
||||
testEnhancedIntegration().catch(error => {
|
||||
console.error('Fatal error:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
30
tests/test-package-info.js
Normal file
30
tests/test-package-info.js
Normal file
@@ -0,0 +1,30 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { NodeSourceExtractor } = require('../dist/utils/node-source-extractor');
|
||||
|
||||
async function testPackageInfo() {
|
||||
console.log('🧪 Testing Package Info Extraction\n');
|
||||
|
||||
const extractor = new NodeSourceExtractor();
|
||||
|
||||
const testNodes = [
|
||||
'n8n-nodes-base.Slack',
|
||||
'n8n-nodes-base.HttpRequest',
|
||||
'n8n-nodes-base.Function'
|
||||
];
|
||||
|
||||
for (const nodeType of testNodes) {
|
||||
console.log(`\n📦 Testing ${nodeType}:`);
|
||||
try {
|
||||
const result = await extractor.extractNodeSource(nodeType);
|
||||
console.log(` - Source Code: ${result.sourceCode ? '✅' : '❌'} (${result.sourceCode?.length || 0} bytes)`);
|
||||
console.log(` - Credential Code: ${result.credentialCode ? '✅' : '❌'} (${result.credentialCode?.length || 0} bytes)`);
|
||||
console.log(` - Package Name: ${result.packageInfo?.name || '❌ undefined'}`);
|
||||
console.log(` - Package Version: ${result.packageInfo?.version || '❌ undefined'}`);
|
||||
} catch (error) {
|
||||
console.log(` ❌ Error: ${error.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
testPackageInfo().catch(console.error);
|
||||
82
tests/test-parsing-operations.js
Normal file
82
tests/test-parsing-operations.js
Normal file
@@ -0,0 +1,82 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const markdown = `
|
||||
## Operations
|
||||
|
||||
* **Channel**
|
||||
* **Archive** a channel.
|
||||
* **Close** a direct message or multi-person direct message.
|
||||
* **Create** a public or private channel-based conversation.
|
||||
* **Get** information about a channel.
|
||||
* **Get Many**: Get a list of channels in Slack.
|
||||
* **File**
|
||||
* **Get** a file.
|
||||
* **Get Many**: Get and filter team files.
|
||||
* **Upload**: Create or upload an existing file.
|
||||
|
||||
## Templates and examples
|
||||
`;
|
||||
|
||||
function extractOperations(markdown) {
|
||||
const operations = [];
|
||||
|
||||
// Find operations section
|
||||
const operationsMatch = markdown.match(/##\s+Operations\s*\n([\s\S]*?)(?=\n##|\n#|$)/i);
|
||||
if (!operationsMatch) {
|
||||
console.log('No operations section found');
|
||||
return operations;
|
||||
}
|
||||
|
||||
const operationsText = operationsMatch[1];
|
||||
console.log('Operations text:', operationsText.substring(0, 200));
|
||||
|
||||
// Parse operation structure
|
||||
let currentResource = null;
|
||||
const lines = operationsText.split('\n');
|
||||
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const line = lines[i];
|
||||
const trimmedLine = line.trim();
|
||||
|
||||
// Resource level (e.g., "* **Channel**")
|
||||
if (trimmedLine.match(/^\*\s+\*\*([^*]+)\*\*/)) {
|
||||
currentResource = trimmedLine.match(/^\*\s+\*\*([^*]+)\*\*/)[1].trim();
|
||||
console.log(`Found resource: ${currentResource}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
// Skip if we don't have a current resource
|
||||
if (!currentResource) continue;
|
||||
|
||||
// Operation level - look for indented bullets (4 spaces + *)
|
||||
if (line.match(/^\s{4}\*\s+/)) {
|
||||
console.log(`Found operation line: "${line}"`);
|
||||
|
||||
// Extract operation name and description
|
||||
const operationMatch = trimmedLine.match(/^\*\s+\*\*([^*]+)\*\*(.*)$/);
|
||||
if (operationMatch) {
|
||||
const operation = operationMatch[1].trim();
|
||||
let description = operationMatch[2].trim();
|
||||
|
||||
// Clean up description
|
||||
description = description.replace(/^:\s*/, '').replace(/\.$/, '').trim();
|
||||
|
||||
operations.push({
|
||||
resource: currentResource,
|
||||
operation,
|
||||
description: description || operation,
|
||||
});
|
||||
console.log(` Parsed: ${operation} - ${description}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return operations;
|
||||
}
|
||||
|
||||
const operations = extractOperations(markdown);
|
||||
console.log('\nTotal operations found:', operations.length);
|
||||
console.log('\nOperations:');
|
||||
operations.forEach(op => {
|
||||
console.log(`- ${op.resource}.${op.operation}: ${op.description}`);
|
||||
});
|
||||
5378
tests/test-results/extracted-nodes.json
Normal file
5378
tests/test-results/extracted-nodes.json
Normal file
File diff suppressed because one or more lines are too long
760
tests/test-results/test-summary.json
Normal file
760
tests/test-results/test-summary.json
Normal file
@@ -0,0 +1,760 @@
|
||||
{
|
||||
"totalTests": 6,
|
||||
"passed": 6,
|
||||
"failed": 0,
|
||||
"startTime": "2025-06-08T10:57:55.233Z",
|
||||
"endTime": "2025-06-08T10:57:59.249Z",
|
||||
"tests": [
|
||||
{
|
||||
"name": "Basic Node Extraction",
|
||||
"status": "passed",
|
||||
"startTime": "2025-06-08T10:57:55.236Z",
|
||||
"endTime": "2025-06-08T10:57:55.342Z",
|
||||
"error": null,
|
||||
"details": {
|
||||
"results": [
|
||||
{
|
||||
"nodeType": "@n8n/n8n-nodes-langchain.Agent",
|
||||
"extracted": false,
|
||||
"error": "Node source code not found for: @n8n/n8n-nodes-langchain.Agent"
|
||||
},
|
||||
{
|
||||
"nodeType": "n8n-nodes-base.Function",
|
||||
"extracted": true,
|
||||
"codeLength": 7449,
|
||||
"hasCredentials": false,
|
||||
"hasPackageInfo": true,
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/Function/Function.node.js"
|
||||
},
|
||||
{
|
||||
"nodeType": "n8n-nodes-base.Webhook",
|
||||
"extracted": true,
|
||||
"codeLength": 10667,
|
||||
"hasCredentials": false,
|
||||
"hasPackageInfo": true,
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/Webhook/Webhook.node.js"
|
||||
}
|
||||
],
|
||||
"successCount": 2,
|
||||
"totalTested": 3
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "List Available Nodes",
|
||||
"status": "passed",
|
||||
"startTime": "2025-06-08T10:57:55.342Z",
|
||||
"endTime": "2025-06-08T10:57:55.689Z",
|
||||
"error": null,
|
||||
"details": {
|
||||
"totalNodes": 439,
|
||||
"packages": [
|
||||
"unknown"
|
||||
],
|
||||
"nodesByPackage": {
|
||||
"unknown": [
|
||||
"ActionNetwork",
|
||||
"ActiveCampaign",
|
||||
"ActiveCampaignTrigger",
|
||||
"AcuitySchedulingTrigger",
|
||||
"Adalo",
|
||||
"Affinity",
|
||||
"AffinityTrigger",
|
||||
"AgileCrm",
|
||||
"Airtable",
|
||||
"AirtableTrigger",
|
||||
"AirtableV1",
|
||||
"Amqp",
|
||||
"AmqpTrigger",
|
||||
"ApiTemplateIo",
|
||||
"Asana",
|
||||
"AsanaTrigger",
|
||||
"Automizy",
|
||||
"Autopilot",
|
||||
"AutopilotTrigger",
|
||||
"AwsLambda",
|
||||
"AwsSns",
|
||||
"AwsSnsTrigger",
|
||||
"AwsCertificateManager",
|
||||
"AwsComprehend",
|
||||
"AwsDynamoDB",
|
||||
"AwsElb",
|
||||
"AwsRekognition",
|
||||
"AwsS3",
|
||||
"AwsS3V1",
|
||||
"AwsS3V2",
|
||||
"AwsSes",
|
||||
"AwsSqs",
|
||||
"AwsTextract",
|
||||
"AwsTranscribe",
|
||||
"Bannerbear",
|
||||
"Baserow",
|
||||
"Beeminder",
|
||||
"BitbucketTrigger",
|
||||
"Bitly",
|
||||
"Bitwarden",
|
||||
"Box",
|
||||
"BoxTrigger",
|
||||
"Brandfetch",
|
||||
"Brevo",
|
||||
"BrevoTrigger",
|
||||
"Bubble",
|
||||
"CalTrigger",
|
||||
"CalendlyTrigger",
|
||||
"Chargebee",
|
||||
"ChargebeeTrigger",
|
||||
"CircleCi",
|
||||
"CiscoWebex",
|
||||
"CiscoWebexTrigger",
|
||||
"CitrixAdc",
|
||||
"Clearbit",
|
||||
"ClickUp",
|
||||
"ClickUpTrigger",
|
||||
"Clockify",
|
||||
"ClockifyTrigger",
|
||||
"Cloudflare",
|
||||
"Cockpit",
|
||||
"Coda",
|
||||
"Code",
|
||||
"CoinGecko",
|
||||
"CompareDatasets",
|
||||
"Compression",
|
||||
"Contentful",
|
||||
"ConvertKit",
|
||||
"ConvertKitTrigger",
|
||||
"Copper",
|
||||
"CopperTrigger",
|
||||
"Cortex",
|
||||
"CrateDb",
|
||||
"Cron",
|
||||
"CrowdDev",
|
||||
"CrowdDevTrigger",
|
||||
"Crypto",
|
||||
"CustomerIo",
|
||||
"CustomerIoTrigger",
|
||||
"DateTime",
|
||||
"DateTimeV1",
|
||||
"DateTimeV2",
|
||||
"DebugHelper",
|
||||
"DeepL",
|
||||
"Demio",
|
||||
"Dhl",
|
||||
"Discord",
|
||||
"Discourse",
|
||||
"Disqus",
|
||||
"Drift",
|
||||
"Dropbox",
|
||||
"Dropcontact",
|
||||
"E2eTest",
|
||||
"ERPNext",
|
||||
"EditImage",
|
||||
"Egoi",
|
||||
"ElasticSecurity",
|
||||
"Elasticsearch",
|
||||
"EmailReadImap",
|
||||
"EmailReadImapV1",
|
||||
"EmailReadImapV2",
|
||||
"EmailSend",
|
||||
"EmailSendV1",
|
||||
"EmailSendV2",
|
||||
"Emelia",
|
||||
"EmeliaTrigger",
|
||||
"ErrorTrigger",
|
||||
"EventbriteTrigger",
|
||||
"ExecuteCommand",
|
||||
"ExecuteWorkflow",
|
||||
"ExecuteWorkflowTrigger",
|
||||
"ExecutionData",
|
||||
"FacebookGraphApi",
|
||||
"FacebookTrigger",
|
||||
"FacebookLeadAdsTrigger",
|
||||
"FigmaTrigger",
|
||||
"FileMaker",
|
||||
"Filter",
|
||||
"Flow",
|
||||
"FlowTrigger",
|
||||
"FormTrigger",
|
||||
"FormIoTrigger",
|
||||
"FormstackTrigger",
|
||||
"Freshdesk",
|
||||
"Freshservice",
|
||||
"FreshworksCrm",
|
||||
"Ftp",
|
||||
"Function",
|
||||
"FunctionItem",
|
||||
"GetResponse",
|
||||
"GetResponseTrigger",
|
||||
"Ghost",
|
||||
"Git",
|
||||
"Github",
|
||||
"GithubTrigger",
|
||||
"Gitlab",
|
||||
"GitlabTrigger",
|
||||
"GoToWebinar",
|
||||
"GoogleAds",
|
||||
"GoogleAnalytics",
|
||||
"GoogleAnalyticsV1",
|
||||
"GoogleBigQuery",
|
||||
"GoogleBigQueryV1",
|
||||
"GoogleBooks",
|
||||
"GoogleCalendar",
|
||||
"GoogleCalendarTrigger",
|
||||
"GoogleChat",
|
||||
"GoogleCloudNaturalLanguage",
|
||||
"GoogleCloudStorage",
|
||||
"GoogleContacts",
|
||||
"GoogleDocs",
|
||||
"GoogleDrive",
|
||||
"GoogleDriveTrigger",
|
||||
"GoogleDriveV1",
|
||||
"GoogleFirebaseCloudFirestore",
|
||||
"GoogleFirebaseRealtimeDatabase",
|
||||
"GSuiteAdmin",
|
||||
"Gmail",
|
||||
"GmailTrigger",
|
||||
"GmailV1",
|
||||
"GmailV2",
|
||||
"GooglePerspective",
|
||||
"GoogleSheets",
|
||||
"GoogleSheetsTrigger",
|
||||
"GoogleSlides",
|
||||
"GoogleTasks",
|
||||
"GoogleTranslate",
|
||||
"YouTube",
|
||||
"Gotify",
|
||||
"Grafana",
|
||||
"GraphQL",
|
||||
"Grist",
|
||||
"GumroadTrigger",
|
||||
"HackerNews",
|
||||
"HaloPSA",
|
||||
"Harvest",
|
||||
"HelpScout",
|
||||
"HelpScoutTrigger",
|
||||
"HighLevel",
|
||||
"HomeAssistant",
|
||||
"Html",
|
||||
"HtmlExtract",
|
||||
"HttpRequest",
|
||||
"HttpRequestV1",
|
||||
"HttpRequestV2",
|
||||
"HttpRequestV3",
|
||||
"Hubspot",
|
||||
"HubspotTrigger",
|
||||
"HubspotV1",
|
||||
"HubspotV2",
|
||||
"HumanticAi",
|
||||
"Hunter",
|
||||
"ICalendar",
|
||||
"If",
|
||||
"Intercom",
|
||||
"Interval",
|
||||
"InvoiceNinja",
|
||||
"InvoiceNinjaTrigger",
|
||||
"ItemLists",
|
||||
"ItemListsV1",
|
||||
"ItemListsV2",
|
||||
"Iterable",
|
||||
"Jenkins",
|
||||
"Jira",
|
||||
"JiraTrigger",
|
||||
"JotFormTrigger",
|
||||
"Kafka",
|
||||
"KafkaTrigger",
|
||||
"Keap",
|
||||
"KeapTrigger",
|
||||
"Kitemaker",
|
||||
"KoBoToolbox",
|
||||
"KoBoToolboxTrigger",
|
||||
"Ldap",
|
||||
"Lemlist",
|
||||
"LemlistTrigger",
|
||||
"Line",
|
||||
"Linear",
|
||||
"LinearTrigger",
|
||||
"LingvaNex",
|
||||
"LinkedIn",
|
||||
"LocalFileTrigger",
|
||||
"LoneScale",
|
||||
"LoneScaleTrigger",
|
||||
"Mqtt",
|
||||
"MqttTrigger",
|
||||
"Magento2",
|
||||
"Mailcheck",
|
||||
"Mailchimp",
|
||||
"MailchimpTrigger",
|
||||
"MailerLite",
|
||||
"MailerLiteTrigger",
|
||||
"Mailgun",
|
||||
"Mailjet",
|
||||
"MailjetTrigger",
|
||||
"Mandrill",
|
||||
"ManualTrigger",
|
||||
"Markdown",
|
||||
"Marketstack",
|
||||
"Matrix",
|
||||
"Mattermost",
|
||||
"Mautic",
|
||||
"MauticTrigger",
|
||||
"Medium",
|
||||
"Merge",
|
||||
"MergeV1",
|
||||
"MergeV2",
|
||||
"MessageBird",
|
||||
"Metabase",
|
||||
"MicrosoftDynamicsCrm",
|
||||
"MicrosoftExcel",
|
||||
"MicrosoftExcelV1",
|
||||
"MicrosoftGraphSecurity",
|
||||
"MicrosoftOneDrive",
|
||||
"MicrosoftOutlook",
|
||||
"MicrosoftOutlookV1",
|
||||
"MicrosoftSql",
|
||||
"MicrosoftTeams",
|
||||
"MicrosoftToDo",
|
||||
"Mindee",
|
||||
"Misp",
|
||||
"Mocean",
|
||||
"MondayCom",
|
||||
"MongoDb",
|
||||
"MonicaCrm",
|
||||
"MoveBinaryData",
|
||||
"Msg91",
|
||||
"MySql",
|
||||
"MySqlV1",
|
||||
"N8n",
|
||||
"N8nTrainingCustomerDatastore",
|
||||
"N8nTrainingCustomerMessenger",
|
||||
"N8nTrigger",
|
||||
"Nasa",
|
||||
"Netlify",
|
||||
"NetlifyTrigger",
|
||||
"NextCloud",
|
||||
"NoOp",
|
||||
"NocoDB",
|
||||
"Notion",
|
||||
"NotionTrigger",
|
||||
"Npm",
|
||||
"Odoo",
|
||||
"OneSimpleApi",
|
||||
"Onfleet",
|
||||
"OnfleetTrigger",
|
||||
"OpenAi",
|
||||
"OpenThesaurus",
|
||||
"OpenWeatherMap",
|
||||
"Orbit",
|
||||
"Oura",
|
||||
"Paddle",
|
||||
"PagerDuty",
|
||||
"PayPal",
|
||||
"PayPalTrigger",
|
||||
"Peekalink",
|
||||
"Phantombuster",
|
||||
"PhilipsHue",
|
||||
"Pipedrive",
|
||||
"PipedriveTrigger",
|
||||
"Plivo",
|
||||
"PostBin",
|
||||
"PostHog",
|
||||
"Postgres",
|
||||
"PostgresTrigger",
|
||||
"PostgresV1",
|
||||
"PostmarkTrigger",
|
||||
"ProfitWell",
|
||||
"Pushbullet",
|
||||
"Pushcut",
|
||||
"PushcutTrigger",
|
||||
"Pushover",
|
||||
"QuestDb",
|
||||
"QuickBase",
|
||||
"QuickBooks",
|
||||
"QuickChart",
|
||||
"RabbitMQ",
|
||||
"RabbitMQTrigger",
|
||||
"Raindrop",
|
||||
"ReadBinaryFile",
|
||||
"ReadBinaryFiles",
|
||||
"ReadPDF",
|
||||
"Reddit",
|
||||
"Redis",
|
||||
"RedisTrigger",
|
||||
"RenameKeys",
|
||||
"RespondToWebhook",
|
||||
"Rocketchat",
|
||||
"RssFeedRead",
|
||||
"RssFeedReadTrigger",
|
||||
"Rundeck",
|
||||
"S3",
|
||||
"Salesforce",
|
||||
"Salesmate",
|
||||
"ScheduleTrigger",
|
||||
"SeaTable",
|
||||
"SeaTableTrigger",
|
||||
"SecurityScorecard",
|
||||
"Segment",
|
||||
"SendGrid",
|
||||
"Sendy",
|
||||
"SentryIo",
|
||||
"ServiceNow",
|
||||
"Set",
|
||||
"SetV1",
|
||||
"SetV2",
|
||||
"Shopify",
|
||||
"ShopifyTrigger",
|
||||
"Signl4",
|
||||
"Slack",
|
||||
"SlackV1",
|
||||
"SlackV2",
|
||||
"Sms77",
|
||||
"Snowflake",
|
||||
"SplitInBatches",
|
||||
"SplitInBatchesV1",
|
||||
"SplitInBatchesV2",
|
||||
"SplitInBatchesV3",
|
||||
"Splunk",
|
||||
"Spontit",
|
||||
"Spotify",
|
||||
"SpreadsheetFile",
|
||||
"SseTrigger",
|
||||
"Ssh",
|
||||
"Stackby",
|
||||
"Start",
|
||||
"StickyNote",
|
||||
"StopAndError",
|
||||
"Storyblok",
|
||||
"Strapi",
|
||||
"Strava",
|
||||
"StravaTrigger",
|
||||
"Stripe",
|
||||
"StripeTrigger",
|
||||
"Supabase",
|
||||
"SurveyMonkeyTrigger",
|
||||
"Switch",
|
||||
"SwitchV1",
|
||||
"SwitchV2",
|
||||
"SyncroMsp",
|
||||
"Taiga",
|
||||
"TaigaTrigger",
|
||||
"Tapfiliate",
|
||||
"Telegram",
|
||||
"TelegramTrigger",
|
||||
"TheHive",
|
||||
"TheHiveTrigger",
|
||||
"TheHiveProjectTrigger",
|
||||
"TimescaleDb",
|
||||
"Todoist",
|
||||
"TodoistV1",
|
||||
"TodoistV2",
|
||||
"TogglTrigger",
|
||||
"Totp",
|
||||
"TravisCi",
|
||||
"Trello",
|
||||
"TrelloTrigger",
|
||||
"Twake",
|
||||
"Twilio",
|
||||
"Twist",
|
||||
"Twitter",
|
||||
"TwitterV1",
|
||||
"TwitterV2",
|
||||
"TypeformTrigger",
|
||||
"UProc",
|
||||
"UnleashedSoftware",
|
||||
"Uplead",
|
||||
"UptimeRobot",
|
||||
"UrlScanIo",
|
||||
"VenafiTlsProtectDatacenter",
|
||||
"VenafiTlsProtectDatacenterTrigger",
|
||||
"VenafiTlsProtectCloud",
|
||||
"VenafiTlsProtectCloudTrigger",
|
||||
"Vero",
|
||||
"Vonage",
|
||||
"Wait",
|
||||
"Webflow",
|
||||
"WebflowTrigger",
|
||||
"Webhook",
|
||||
"Wekan",
|
||||
"WhatsApp",
|
||||
"Wise",
|
||||
"WiseTrigger",
|
||||
"WooCommerce",
|
||||
"WooCommerceTrigger",
|
||||
"Wordpress",
|
||||
"WorkableTrigger",
|
||||
"WorkflowTrigger",
|
||||
"WriteBinaryFile",
|
||||
"WufooTrigger",
|
||||
"Xero",
|
||||
"Xml",
|
||||
"Yourls",
|
||||
"Zammad",
|
||||
"Zendesk",
|
||||
"ZendeskTrigger",
|
||||
"ZohoCrm",
|
||||
"Zoom",
|
||||
"Zulip"
|
||||
]
|
||||
},
|
||||
"sampleNodes": [
|
||||
{
|
||||
"name": "ActionNetwork",
|
||||
"displayName": "Action Network",
|
||||
"description": "Consume the Action Network API",
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/ActionNetwork/ActionNetwork.node.js"
|
||||
},
|
||||
{
|
||||
"name": "ActiveCampaign",
|
||||
"displayName": "ActiveCampaign",
|
||||
"description": "Create and edit data in ActiveCampaign",
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/ActiveCampaign/ActiveCampaign.node.js"
|
||||
},
|
||||
{
|
||||
"name": "ActiveCampaignTrigger",
|
||||
"displayName": "ActiveCampaign Trigger",
|
||||
"description": "Handle ActiveCampaign events via webhooks",
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/ActiveCampaign/ActiveCampaignTrigger.node.js"
|
||||
},
|
||||
{
|
||||
"name": "AcuitySchedulingTrigger",
|
||||
"displayName": "Acuity Scheduling Trigger",
|
||||
"description": "Handle Acuity Scheduling events via webhooks",
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/AcuityScheduling/AcuitySchedulingTrigger.node.js"
|
||||
},
|
||||
{
|
||||
"name": "Adalo",
|
||||
"displayName": "Adalo",
|
||||
"description": "Consume Adalo API",
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/Adalo/Adalo.node.js"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "Bulk Node Extraction",
|
||||
"status": "passed",
|
||||
"startTime": "2025-06-08T10:57:55.689Z",
|
||||
"endTime": "2025-06-08T10:57:58.574Z",
|
||||
"error": null,
|
||||
"details": {
|
||||
"totalAttempted": 10,
|
||||
"successCount": 6,
|
||||
"failureCount": 4,
|
||||
"timeElapsed": 2581,
|
||||
"results": [
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"nodeType": "ActionNetwork",
|
||||
"name": "ActionNetwork",
|
||||
"codeLength": 15810,
|
||||
"codeHash": "c0a880f5754b6b532ff787bdb253dc49ffd7f470f28aeddda5be0c73f9f9935f",
|
||||
"hasCredentials": true,
|
||||
"hasPackageInfo": true,
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/ActionNetwork/ActionNetwork.node.js",
|
||||
"extractedAt": "2025-06-08T10:57:56.009Z"
|
||||
}
|
||||
},
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"nodeType": "ActiveCampaign",
|
||||
"name": "ActiveCampaign",
|
||||
"codeLength": 38399,
|
||||
"codeHash": "5ea90671718d20eecb6cddae2e21c91470fdb778e8be97106ee2539303422ad2",
|
||||
"hasCredentials": true,
|
||||
"hasPackageInfo": true,
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/ActiveCampaign/ActiveCampaign.node.js",
|
||||
"extractedAt": "2025-06-08T10:57:56.032Z"
|
||||
}
|
||||
},
|
||||
{
|
||||
"success": false,
|
||||
"nodeType": "ActiveCampaignTrigger",
|
||||
"error": "Node source code not found for: ActiveCampaignTrigger"
|
||||
},
|
||||
{
|
||||
"success": false,
|
||||
"nodeType": "AcuitySchedulingTrigger",
|
||||
"error": "Node source code not found for: AcuitySchedulingTrigger"
|
||||
},
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"nodeType": "Adalo",
|
||||
"name": "Adalo",
|
||||
"codeLength": 8234,
|
||||
"codeHash": "0fbcb0b60141307fdc3394154af1b2c3133fa6181aac336249c6c211fd24846f",
|
||||
"hasCredentials": true,
|
||||
"hasPackageInfo": true,
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/Adalo/Adalo.node.js",
|
||||
"extractedAt": "2025-06-08T10:57:57.330Z"
|
||||
}
|
||||
},
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"nodeType": "Affinity",
|
||||
"name": "Affinity",
|
||||
"codeLength": 16217,
|
||||
"codeHash": "e605ea187767403dfa55cd374690f7df563a0baa7ca6991d86d522dc101a2846",
|
||||
"hasCredentials": true,
|
||||
"hasPackageInfo": true,
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/Affinity/Affinity.node.js",
|
||||
"extractedAt": "2025-06-08T10:57:57.343Z"
|
||||
}
|
||||
},
|
||||
{
|
||||
"success": false,
|
||||
"nodeType": "AffinityTrigger",
|
||||
"error": "Node source code not found for: AffinityTrigger"
|
||||
},
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"nodeType": "AgileCrm",
|
||||
"name": "AgileCrm",
|
||||
"codeLength": 28115,
|
||||
"codeHash": "ce71c3b5dec23a48d24c5775e9bb79006ce395bed62b306c56340b5c772379c2",
|
||||
"hasCredentials": true,
|
||||
"hasPackageInfo": true,
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/AgileCrm/AgileCrm.node.js",
|
||||
"extractedAt": "2025-06-08T10:57:57.925Z"
|
||||
}
|
||||
},
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"nodeType": "Airtable",
|
||||
"name": "Airtable",
|
||||
"codeLength": 936,
|
||||
"codeHash": "2d67e72931697178946f5127b43e954649c4c5e7ad9e29764796404ae96e7db5",
|
||||
"hasCredentials": true,
|
||||
"hasPackageInfo": true,
|
||||
"location": "node_modules/n8n-nodes-base/dist/nodes/Airtable/Airtable.node.js",
|
||||
"extractedAt": "2025-06-08T10:57:57.941Z"
|
||||
}
|
||||
},
|
||||
{
|
||||
"success": false,
|
||||
"nodeType": "AirtableTrigger",
|
||||
"error": "Node source code not found for: AirtableTrigger"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "Database Schema Validation",
|
||||
"status": "passed",
|
||||
"startTime": "2025-06-08T10:57:58.574Z",
|
||||
"endTime": "2025-06-08T10:57:58.575Z",
|
||||
"error": null,
|
||||
"details": {
|
||||
"schemaValid": true,
|
||||
"tablesCount": 4,
|
||||
"estimatedStoragePerNode": 16834
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "Error Handling",
|
||||
"status": "passed",
|
||||
"startTime": "2025-06-08T10:57:58.575Z",
|
||||
"endTime": "2025-06-08T10:57:59.244Z",
|
||||
"error": null,
|
||||
"details": {
|
||||
"totalTests": 3,
|
||||
"passed": 2,
|
||||
"results": [
|
||||
{
|
||||
"name": "Non-existent node",
|
||||
"nodeType": "non-existent-package.FakeNode",
|
||||
"expectedError": "not found",
|
||||
"passed": true,
|
||||
"actualError": "Node source code not found for: non-existent-package.FakeNode"
|
||||
},
|
||||
{
|
||||
"name": "Invalid node type format",
|
||||
"nodeType": "",
|
||||
"expectedError": "invalid",
|
||||
"passed": false,
|
||||
"actualError": "Node source code not found for: "
|
||||
},
|
||||
{
|
||||
"name": "Malformed package name",
|
||||
"nodeType": "@invalid@package.Node",
|
||||
"expectedError": "not found",
|
||||
"passed": true,
|
||||
"actualError": "Node source code not found for: @invalid@package.Node"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "MCP Server Integration",
|
||||
"status": "passed",
|
||||
"startTime": "2025-06-08T10:57:59.244Z",
|
||||
"endTime": "2025-06-08T10:57:59.249Z",
|
||||
"error": null,
|
||||
"details": {
|
||||
"serverCreated": true,
|
||||
"config": {
|
||||
"port": 3000,
|
||||
"host": "0.0.0.0",
|
||||
"authToken": "test-token"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"extractedNodes": 6,
|
||||
"databaseSchema": {
|
||||
"tables": {
|
||||
"nodes": {
|
||||
"columns": {
|
||||
"id": "UUID PRIMARY KEY",
|
||||
"node_type": "VARCHAR(255) UNIQUE NOT NULL",
|
||||
"name": "VARCHAR(255) NOT NULL",
|
||||
"package_name": "VARCHAR(255)",
|
||||
"display_name": "VARCHAR(255)",
|
||||
"description": "TEXT",
|
||||
"version": "VARCHAR(50)",
|
||||
"code_hash": "VARCHAR(64) NOT NULL",
|
||||
"code_length": "INTEGER NOT NULL",
|
||||
"source_location": "TEXT",
|
||||
"extracted_at": "TIMESTAMP NOT NULL",
|
||||
"updated_at": "TIMESTAMP"
|
||||
},
|
||||
"indexes": [
|
||||
"node_type",
|
||||
"package_name",
|
||||
"code_hash"
|
||||
]
|
||||
},
|
||||
"node_source_code": {
|
||||
"columns": {
|
||||
"id": "UUID PRIMARY KEY",
|
||||
"node_id": "UUID REFERENCES nodes(id)",
|
||||
"source_code": "TEXT NOT NULL",
|
||||
"compiled_code": "TEXT",
|
||||
"source_map": "TEXT"
|
||||
}
|
||||
},
|
||||
"node_credentials": {
|
||||
"columns": {
|
||||
"id": "UUID PRIMARY KEY",
|
||||
"node_id": "UUID REFERENCES nodes(id)",
|
||||
"credential_type": "VARCHAR(255) NOT NULL",
|
||||
"credential_code": "TEXT NOT NULL",
|
||||
"required_fields": "JSONB"
|
||||
}
|
||||
},
|
||||
"node_metadata": {
|
||||
"columns": {
|
||||
"id": "UUID PRIMARY KEY",
|
||||
"node_id": "UUID REFERENCES nodes(id)",
|
||||
"package_info": "JSONB",
|
||||
"dependencies": "JSONB",
|
||||
"icon": "TEXT",
|
||||
"categories": "TEXT[]",
|
||||
"documentation_url": "TEXT"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
133
tests/test-slack-docs-issue.js
Executable file
133
tests/test-slack-docs-issue.js
Executable file
@@ -0,0 +1,133 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { DocumentationFetcher } = require('../dist/utils/documentation-fetcher');
|
||||
const { NodeSourceExtractor } = require('../dist/utils/node-source-extractor');
|
||||
const { execSync } = require('child_process');
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
|
||||
async function investigateSlackDocs() {
|
||||
console.log('=== Investigating Slack Node Documentation Issue ===\n');
|
||||
|
||||
const docsFetcher = new DocumentationFetcher();
|
||||
const extractor = new NodeSourceExtractor();
|
||||
|
||||
try {
|
||||
// 1. Ensure docs repo is available
|
||||
console.log('1️⃣ Ensuring documentation repository...');
|
||||
await docsFetcher.ensureDocsRepository();
|
||||
|
||||
// 2. Check what files exist for Slack
|
||||
console.log('\n2️⃣ Searching for Slack documentation files...');
|
||||
const docsPath = path.join(process.cwd(), 'temp', 'n8n-docs');
|
||||
|
||||
try {
|
||||
const slackFiles = execSync(
|
||||
`find ${docsPath} -name "*slack*" -type f | grep -v ".git"`,
|
||||
{ encoding: 'utf-8' }
|
||||
).trim().split('\n').filter(Boolean);
|
||||
|
||||
console.log(`Found ${slackFiles.length} files with "slack" in the name:`);
|
||||
slackFiles.forEach(file => {
|
||||
const relPath = path.relative(docsPath, file);
|
||||
console.log(` - ${relPath}`);
|
||||
});
|
||||
|
||||
// Check content of each file
|
||||
console.log('\n3️⃣ Checking content of Slack-related files...');
|
||||
for (const file of slackFiles.slice(0, 5)) { // Check first 5 files
|
||||
if (file.endsWith('.md')) {
|
||||
const content = fs.readFileSync(file, 'utf-8');
|
||||
const firstLine = content.split('\n')[0];
|
||||
const isCredential = content.includes('credential') || content.includes('authentication');
|
||||
console.log(`\n 📄 ${path.basename(file)}`);
|
||||
console.log(` First line: ${firstLine}`);
|
||||
console.log(` Is credential doc: ${isCredential}`);
|
||||
|
||||
// Check if it mentions being a node or credential
|
||||
if (content.includes('# Slack node')) {
|
||||
console.log(' ✅ This is the Slack NODE documentation!');
|
||||
console.log(` Path: ${file}`);
|
||||
} else if (content.includes('# Slack credentials')) {
|
||||
console.log(' ⚠️ This is the Slack CREDENTIALS documentation');
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.log('Error searching for Slack files:', error.message);
|
||||
}
|
||||
|
||||
// 4. Test the getNodeDocumentation method
|
||||
console.log('\n4️⃣ Testing getNodeDocumentation for Slack...');
|
||||
const slackDocs = await docsFetcher.getNodeDocumentation('n8n-nodes-base.slack');
|
||||
|
||||
if (slackDocs) {
|
||||
console.log(' ✅ Found documentation for Slack node');
|
||||
console.log(` URL: ${slackDocs.url}`);
|
||||
console.log(` Content preview: ${slackDocs.markdown.substring(0, 200)}...`);
|
||||
|
||||
// Check if it's credential or node docs
|
||||
const isCredentialDoc = slackDocs.markdown.includes('credential') ||
|
||||
slackDocs.markdown.includes('authentication') ||
|
||||
slackDocs.markdown.includes('# Slack credentials');
|
||||
const isNodeDoc = slackDocs.markdown.includes('# Slack node') ||
|
||||
slackDocs.markdown.includes('## Properties');
|
||||
|
||||
console.log(` Is credential doc: ${isCredentialDoc}`);
|
||||
console.log(` Is node doc: ${isNodeDoc}`);
|
||||
} else {
|
||||
console.log(' ❌ No documentation found for Slack node');
|
||||
}
|
||||
|
||||
// 5. Extract the Slack node source to understand its structure
|
||||
console.log('\n5️⃣ Extracting Slack node source code...');
|
||||
try {
|
||||
const slackNode = await extractor.extractNodeSource('n8n-nodes-base.slack');
|
||||
console.log(' ✅ Successfully extracted Slack node');
|
||||
console.log(` Location: ${slackNode.location}`);
|
||||
console.log(` Has credential code: ${!!slackNode.credentialCode}`);
|
||||
|
||||
// Parse the node definition
|
||||
const descMatch = slackNode.sourceCode.match(/description\s*[:=]\s*({[\s\S]*?})\s*[,;]/);
|
||||
if (descMatch) {
|
||||
console.log(' Found node description in source');
|
||||
}
|
||||
} catch (error) {
|
||||
console.log(' ❌ Failed to extract Slack node:', error.message);
|
||||
}
|
||||
|
||||
// 6. Check documentation structure
|
||||
console.log('\n6️⃣ Checking n8n-docs repository structure...');
|
||||
const docStructure = [
|
||||
'docs/integrations/builtin/app-nodes',
|
||||
'docs/integrations/builtin/core-nodes',
|
||||
'docs/integrations/builtin/trigger-nodes',
|
||||
'docs/integrations/builtin/credentials'
|
||||
];
|
||||
|
||||
for (const dir of docStructure) {
|
||||
const fullPath = path.join(docsPath, dir);
|
||||
try {
|
||||
const files = fs.readdirSync(fullPath);
|
||||
const slackFile = files.find(f => f.toLowerCase().includes('slack'));
|
||||
console.log(`\n 📁 ${dir}:`);
|
||||
if (slackFile) {
|
||||
console.log(` Found: ${slackFile}`);
|
||||
} else {
|
||||
console.log(` No Slack files found`);
|
||||
}
|
||||
} catch (error) {
|
||||
console.log(` Directory doesn't exist`);
|
||||
}
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('\n❌ Investigation failed:', error);
|
||||
} finally {
|
||||
// Cleanup
|
||||
await docsFetcher.cleanup();
|
||||
}
|
||||
}
|
||||
|
||||
// Run investigation
|
||||
investigateSlackDocs().catch(console.error);
|
||||
119
tests/test-slack-fix.js
Executable file
119
tests/test-slack-fix.js
Executable file
@@ -0,0 +1,119 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { NodeDocumentationService } = require('../dist/services/node-documentation-service');
|
||||
const { NodeSourceExtractor } = require('../dist/utils/node-source-extractor');
|
||||
const { DocumentationFetcher } = require('../dist/utils/documentation-fetcher');
|
||||
|
||||
async function testSlackFix() {
|
||||
console.log('=== Testing Slack Node Fix ===\n');
|
||||
|
||||
const extractor = new NodeSourceExtractor();
|
||||
const docsFetcher = new DocumentationFetcher();
|
||||
|
||||
try {
|
||||
// Test 1: Node source extraction
|
||||
console.log('1️⃣ Testing Slack node source extraction...');
|
||||
const slackSource = await extractor.extractNodeSource('n8n-nodes-base.slack');
|
||||
console.log(` ✅ Source code found at: ${slackSource.location}`);
|
||||
console.log(` 📏 Source length: ${slackSource.sourceCode.length} bytes`);
|
||||
|
||||
// Extract display name from source
|
||||
const displayNameMatch = slackSource.sourceCode.match(/displayName\s*[:=]\s*['"`]([^'"`]+)['"`]/);
|
||||
console.log(` 📛 Display name: ${displayNameMatch ? displayNameMatch[1] : 'Not found'}`);
|
||||
|
||||
// Test 2: Documentation fetching
|
||||
console.log('\n2️⃣ Testing Slack documentation fetching...');
|
||||
const slackDocs = await docsFetcher.getNodeDocumentation('n8n-nodes-base.slack');
|
||||
|
||||
if (slackDocs) {
|
||||
console.log(` ✅ Documentation found`);
|
||||
console.log(` 📄 URL: ${slackDocs.url}`);
|
||||
|
||||
// Extract title from markdown
|
||||
const titleMatch = slackDocs.markdown.match(/title:\s*(.+)/);
|
||||
console.log(` 📝 Title: ${titleMatch ? titleMatch[1] : 'Not found'}`);
|
||||
|
||||
// Check if it's the correct documentation
|
||||
const isNodeDoc = slackDocs.markdown.includes('Slack node') ||
|
||||
slackDocs.markdown.includes('node documentation');
|
||||
const isCredentialDoc = slackDocs.markdown.includes('Slack credentials') &&
|
||||
!slackDocs.markdown.includes('node documentation');
|
||||
|
||||
console.log(` ✅ Is node documentation: ${isNodeDoc}`);
|
||||
console.log(` ❌ Is credential documentation: ${isCredentialDoc}`);
|
||||
|
||||
if (isNodeDoc && !isCredentialDoc) {
|
||||
console.log('\n🎉 SUCCESS: Slack node documentation is correctly fetched!');
|
||||
} else {
|
||||
console.log('\n⚠️ WARNING: Documentation may not be correct');
|
||||
}
|
||||
|
||||
// Show first few lines of content
|
||||
console.log('\n📋 Documentation preview:');
|
||||
const lines = slackDocs.markdown.split('\n').slice(0, 15);
|
||||
lines.forEach(line => console.log(` ${line}`));
|
||||
|
||||
} else {
|
||||
console.log(' ❌ No documentation found');
|
||||
}
|
||||
|
||||
// Test 3: Complete node info using NodeDocumentationService
|
||||
console.log('\n3️⃣ Testing complete node info storage...');
|
||||
const service = new NodeDocumentationService('./data/test-slack-fix.db');
|
||||
|
||||
try {
|
||||
// Parse node definition
|
||||
const nodeDefinition = {
|
||||
displayName: displayNameMatch ? displayNameMatch[1] : 'Slack',
|
||||
description: 'Send messages to Slack channels, users and conversations',
|
||||
category: 'Communication',
|
||||
icon: 'file:slack.svg',
|
||||
version: 2
|
||||
};
|
||||
|
||||
// Store node info
|
||||
await service.storeNode({
|
||||
nodeType: 'n8n-nodes-base.slack',
|
||||
name: 'slack',
|
||||
displayName: nodeDefinition.displayName,
|
||||
description: nodeDefinition.description,
|
||||
category: nodeDefinition.category,
|
||||
icon: nodeDefinition.icon,
|
||||
sourceCode: slackSource.sourceCode,
|
||||
credentialCode: slackSource.credentialCode,
|
||||
documentation: slackDocs?.markdown,
|
||||
documentationUrl: slackDocs?.url,
|
||||
packageName: 'n8n-nodes-base',
|
||||
version: nodeDefinition.version,
|
||||
hasCredentials: !!slackSource.credentialCode,
|
||||
isTrigger: false,
|
||||
isWebhook: false
|
||||
});
|
||||
|
||||
console.log(' ✅ Node info stored successfully');
|
||||
|
||||
// Retrieve and verify
|
||||
const retrievedNode = await service.getNodeInfo('n8n-nodes-base.slack');
|
||||
if (retrievedNode) {
|
||||
console.log(' ✅ Node retrieved successfully');
|
||||
console.log(` 📛 Display name: ${retrievedNode.displayName}`);
|
||||
console.log(` 📝 Has documentation: ${!!retrievedNode.documentation}`);
|
||||
console.log(` 📄 Documentation URL: ${retrievedNode.documentationUrl || 'N/A'}`);
|
||||
}
|
||||
|
||||
service.close();
|
||||
} catch (error) {
|
||||
console.error(' ❌ Error with node service:', error.message);
|
||||
service.close();
|
||||
}
|
||||
|
||||
console.log('\n✅ All tests completed!');
|
||||
|
||||
} catch (error) {
|
||||
console.error('\n❌ Test failed:', error);
|
||||
} finally {
|
||||
await docsFetcher.cleanup();
|
||||
}
|
||||
}
|
||||
|
||||
testSlackFix().catch(console.error);
|
||||
137
tests/test-slack-node-complete.js
Normal file
137
tests/test-slack-node-complete.js
Normal file
@@ -0,0 +1,137 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const { NodeDocumentationService } = require('../dist/services/node-documentation-service');
|
||||
const { EnhancedDocumentationFetcher } = require('../dist/utils/documentation-fetcher');
|
||||
const { NodeSourceExtractor } = require('../dist/utils/node-source-extractor');
|
||||
const path = require('path');
|
||||
|
||||
async function testSlackNode() {
|
||||
console.log('🧪 Testing Slack Node Complete Information Extraction\n');
|
||||
|
||||
const dbPath = path.join(__dirname, '../data/test-slack.db');
|
||||
const service = new NodeDocumentationService(dbPath);
|
||||
const fetcher = new EnhancedDocumentationFetcher();
|
||||
const extractor = new NodeSourceExtractor();
|
||||
|
||||
try {
|
||||
console.log('📚 Fetching Slack node documentation...');
|
||||
const docs = await fetcher.getEnhancedNodeDocumentation('n8n-nodes-base.Slack');
|
||||
|
||||
console.log('\n✅ Documentation Structure:');
|
||||
console.log(`- Title: ${docs.title}`);
|
||||
console.log(`- Has markdown: ${docs.markdown?.length > 0 ? 'Yes' : 'No'} (${docs.markdown?.length || 0} chars)`);
|
||||
console.log(`- Operations: ${docs.operations?.length || 0}`);
|
||||
console.log(`- API Methods: ${docs.apiMethods?.length || 0}`);
|
||||
console.log(`- Examples: ${docs.examples?.length || 0}`);
|
||||
console.log(`- Templates: ${docs.templates?.length || 0}`);
|
||||
console.log(`- Related Resources: ${docs.relatedResources?.length || 0}`);
|
||||
console.log(`- Required Scopes: ${docs.requiredScopes?.length || 0}`);
|
||||
|
||||
console.log('\n📋 Operations by Resource:');
|
||||
const resourceMap = new Map();
|
||||
if (docs.operations) {
|
||||
docs.operations.forEach(op => {
|
||||
if (!resourceMap.has(op.resource)) {
|
||||
resourceMap.set(op.resource, []);
|
||||
}
|
||||
resourceMap.get(op.resource).push(op);
|
||||
});
|
||||
}
|
||||
|
||||
for (const [resource, ops] of resourceMap) {
|
||||
console.log(`\n ${resource}:`);
|
||||
ops.forEach(op => {
|
||||
console.log(` - ${op.operation}: ${op.description}`);
|
||||
});
|
||||
}
|
||||
|
||||
console.log('\n🔌 Sample API Methods:');
|
||||
if (docs.apiMethods) {
|
||||
docs.apiMethods.slice(0, 5).forEach(method => {
|
||||
console.log(` - ${method.operation} → ${method.apiMethod}`);
|
||||
});
|
||||
}
|
||||
|
||||
console.log('\n💻 Extracting Slack node source code...');
|
||||
const sourceInfo = await extractor.extractNodeSource('n8n-nodes-base.Slack');
|
||||
|
||||
console.log('\n✅ Source Code Extraction:');
|
||||
console.log(`- Has source code: ${sourceInfo.sourceCode ? 'Yes' : 'No'} (${sourceInfo.sourceCode?.length || 0} chars)`);
|
||||
console.log(`- Has credential code: ${sourceInfo.credentialCode ? 'Yes' : 'No'} (${sourceInfo.credentialCode?.length || 0} chars)`);
|
||||
console.log(`- Package name: ${sourceInfo.packageInfo?.name}`);
|
||||
console.log(`- Package version: ${sourceInfo.packageInfo?.version}`);
|
||||
|
||||
// Store in database
|
||||
console.log('\n💾 Storing in database...');
|
||||
await service.storeNode({
|
||||
nodeType: 'n8n-nodes-base.Slack',
|
||||
name: 'Slack',
|
||||
displayName: 'Slack',
|
||||
description: 'Send and receive messages, manage channels, and more',
|
||||
category: 'Communication',
|
||||
documentationUrl: docs?.url || 'https://docs.n8n.io/integrations/builtin/app-nodes/n8n-nodes-base.slack/',
|
||||
documentationMarkdown: docs?.markdown,
|
||||
documentationTitle: docs?.title,
|
||||
operations: docs?.operations,
|
||||
apiMethods: docs?.apiMethods,
|
||||
documentationExamples: docs?.examples,
|
||||
templates: docs?.templates,
|
||||
relatedResources: docs?.relatedResources,
|
||||
requiredScopes: docs?.requiredScopes,
|
||||
sourceCode: sourceInfo.sourceCode || '',
|
||||
credentialCode: sourceInfo.credentialCode,
|
||||
packageName: sourceInfo.packageInfo?.name || 'n8n-nodes-base',
|
||||
version: sourceInfo.packageInfo?.version,
|
||||
hasCredentials: true,
|
||||
isTrigger: false,
|
||||
isWebhook: false
|
||||
});
|
||||
|
||||
// Retrieve and verify
|
||||
console.log('\n🔍 Retrieving from database...');
|
||||
const storedNode = await service.getNodeInfo('n8n-nodes-base.Slack');
|
||||
|
||||
console.log('\n✅ Verification Results:');
|
||||
console.log(`- Node found: ${storedNode ? 'Yes' : 'No'}`);
|
||||
if (storedNode) {
|
||||
console.log(`- Has operations: ${storedNode.operations?.length > 0 ? 'Yes' : 'No'} (${storedNode.operations?.length || 0})`);
|
||||
console.log(`- Has API methods: ${storedNode.apiMethods?.length > 0 ? 'Yes' : 'No'} (${storedNode.apiMethods?.length || 0})`);
|
||||
console.log(`- Has examples: ${storedNode.documentationExamples?.length > 0 ? 'Yes' : 'No'} (${storedNode.documentationExamples?.length || 0})`);
|
||||
console.log(`- Has source code: ${storedNode.sourceCode ? 'Yes' : 'No'}`);
|
||||
console.log(`- Has credential code: ${storedNode.credentialCode ? 'Yes' : 'No'}`);
|
||||
}
|
||||
|
||||
// Test search
|
||||
console.log('\n🔍 Testing search...');
|
||||
const searchResults = await service.searchNodes('message send');
|
||||
const slackInResults = searchResults.some(r => r.nodeType === 'n8n-nodes-base.Slack');
|
||||
console.log(`- Slack found in search results: ${slackInResults ? 'Yes' : 'No'}`);
|
||||
|
||||
console.log('\n✅ Complete Information Test Summary:');
|
||||
const hasCompleteInfo =
|
||||
storedNode &&
|
||||
storedNode.operations?.length > 0 &&
|
||||
storedNode.apiMethods?.length > 0 &&
|
||||
storedNode.sourceCode &&
|
||||
storedNode.documentationMarkdown;
|
||||
|
||||
console.log(`- Has complete information: ${hasCompleteInfo ? '✅ YES' : '❌ NO'}`);
|
||||
|
||||
if (!hasCompleteInfo) {
|
||||
console.log('\n❌ Missing Information:');
|
||||
if (!storedNode) console.log(' - Node not stored properly');
|
||||
if (!storedNode?.operations?.length) console.log(' - No operations extracted');
|
||||
if (!storedNode?.apiMethods?.length) console.log(' - No API methods extracted');
|
||||
if (!storedNode?.sourceCode) console.log(' - No source code extracted');
|
||||
if (!storedNode?.documentationMarkdown) console.log(' - No documentation extracted');
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('❌ Test failed:', error);
|
||||
} finally {
|
||||
await service.close();
|
||||
}
|
||||
}
|
||||
|
||||
// Run the test
|
||||
testSlackNode().catch(console.error);
|
||||
Reference in New Issue
Block a user