mirror of
https://github.com/czlonkowski/n8n-mcp.git
synced 2026-01-30 22:42:04 +00:00
Compare commits
5 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
65f51ad8b5 | ||
|
|
af6efe9e88 | ||
|
|
3f427f9528 | ||
|
|
18b8747005 | ||
|
|
749f1c53eb |
43
CHANGELOG.md
43
CHANGELOG.md
@@ -7,6 +7,49 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [2.22.9] - 2025-11-04
|
||||
|
||||
### 🔄 Dependencies Update
|
||||
|
||||
**n8n Platform Update to 1.118.1**
|
||||
|
||||
Updated n8n and all related dependencies to the latest versions:
|
||||
|
||||
- **n8n**: 1.117.2 → 1.118.1
|
||||
- **n8n-core**: 1.116.0 → 1.117.0
|
||||
- **n8n-workflow**: 1.114.0 → 1.115.0
|
||||
- **@n8n/n8n-nodes-langchain**: 1.116.2 → 1.117.0
|
||||
|
||||
### 📊 Database Changes
|
||||
|
||||
- Rebuilt node database with **542 nodes**
|
||||
- 439 nodes from n8n-nodes-base
|
||||
- 103 nodes from @n8n/n8n-nodes-langchain
|
||||
- All node metadata synchronized with latest n8n release
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
**n8n 1.118.1+ Compatibility: Fixed versionCounter API Rejection**
|
||||
|
||||
Fixed integration test failures caused by n8n 1.118.1 API change where `versionCounter` property is returned in GET responses but rejected in PUT requests.
|
||||
|
||||
**Impact**:
|
||||
- Integration tests were failing with "request/body must NOT have additional properties" error
|
||||
- Workflow update operations via n8n API were failing
|
||||
|
||||
**Solution**:
|
||||
- Added `versionCounter` to property exclusion list in `cleanWorkflowForUpdate()` (src/services/n8n-validation.ts:136)
|
||||
- Added `versionCounter?: number` type definition to Workflow and WorkflowExport interfaces
|
||||
- Added test coverage to prevent regression
|
||||
|
||||
### ✅ Verification
|
||||
|
||||
- Database rebuild completed successfully
|
||||
- All node types validated
|
||||
- Documentation mappings updated
|
||||
|
||||
Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en
|
||||
|
||||
## [2.22.7] - 2025-10-26
|
||||
|
||||
### 📝 Documentation Fixes
|
||||
|
||||
@@ -1,5 +1,87 @@
|
||||
# n8n Update Process - Quick Reference
|
||||
|
||||
## ⚡ Recommended Fast Workflow (2025-11-04)
|
||||
|
||||
**CRITICAL FIRST STEP**: Check existing releases to avoid version conflicts!
|
||||
|
||||
```bash
|
||||
# 1. CHECK EXISTING RELEASES FIRST (prevents version conflicts!)
|
||||
gh release list | head -5
|
||||
# Look at the latest version - your new version must be higher!
|
||||
|
||||
# 2. Switch to main and pull
|
||||
git checkout main && git pull
|
||||
|
||||
# 3. Check for updates (dry run)
|
||||
npm run update:n8n:check
|
||||
|
||||
# 4. Run update and skip tests (we'll test in CI)
|
||||
yes y | npm run update:n8n
|
||||
|
||||
# 5. Create feature branch
|
||||
git checkout -b update/n8n-X.X.X
|
||||
|
||||
# 6. Update version in package.json (must be HIGHER than latest release!)
|
||||
# Edit: "version": "2.XX.X" (not the version from the release list!)
|
||||
|
||||
# 7. Update CHANGELOG.md
|
||||
# - Change version number to match package.json
|
||||
# - Update date to today
|
||||
# - Update dependency versions
|
||||
|
||||
# 8. Update README badge
|
||||
# Edit line 8: Change n8n version badge to new n8n version
|
||||
|
||||
# 9. Commit and push
|
||||
git add -A
|
||||
git commit -m "chore: update n8n to X.X.X and bump version to 2.XX.X
|
||||
|
||||
- Updated n8n from X.X.X to X.X.X
|
||||
- Updated n8n-core from X.X.X to X.X.X
|
||||
- Updated n8n-workflow from X.X.X to X.X.X
|
||||
- Updated @n8n/n8n-nodes-langchain from X.X.X to X.X.X
|
||||
- Rebuilt node database with XXX nodes (XXX from n8n-nodes-base, XXX from @n8n/n8n-nodes-langchain)
|
||||
- Updated README badge with new n8n version
|
||||
- Updated CHANGELOG with dependency changes
|
||||
|
||||
Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en
|
||||
|
||||
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||||
|
||||
Co-Authored-By: Claude <noreply@anthropic.com>"
|
||||
|
||||
git push -u origin update/n8n-X.X.X
|
||||
|
||||
# 10. Create PR
|
||||
gh pr create --title "chore: update n8n to X.X.X" --body "Updates n8n and all related dependencies to the latest versions..."
|
||||
|
||||
# 11. After PR is merged, verify release triggered
|
||||
gh release list | head -1
|
||||
# If the new version appears, you're done!
|
||||
# If not, the version might have already been released - bump version again and create new PR
|
||||
```
|
||||
|
||||
### Why This Workflow?
|
||||
|
||||
✅ **Fast**: Skip local tests (2-3 min saved) - CI runs them anyway
|
||||
✅ **Safe**: Unit tests in CI verify compatibility
|
||||
✅ **Clean**: All changes in one PR with proper tracking
|
||||
✅ **Automatic**: Release workflow triggers on merge if version is new
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Problem**: Release workflow doesn't trigger after merge
|
||||
**Cause**: Version number was already released (check `gh release list`)
|
||||
**Solution**: Create new PR bumping version by one patch number
|
||||
|
||||
**Problem**: Integration tests fail in CI with "unauthorized"
|
||||
**Cause**: n8n test instance credentials expired (infrastructure issue)
|
||||
**Solution**: Ignore if unit tests pass - this is not a code problem
|
||||
|
||||
**Problem**: CI takes 8+ minutes
|
||||
**Reason**: Integration tests need live n8n instance (slow)
|
||||
**Normal**: Unit tests (~2 min) + integration tests (~6 min) = ~8 min total
|
||||
|
||||
## Quick One-Command Update
|
||||
|
||||
For a complete update with tests and publish preparation:
|
||||
@@ -99,12 +181,14 @@ This command:
|
||||
|
||||
## Important Notes
|
||||
|
||||
1. **Always run on main branch** - Make sure you're on main and it's clean
|
||||
2. **The update script is smart** - It automatically syncs all n8n dependencies to compatible versions
|
||||
3. **Tests are required** - The publish script now runs tests automatically
|
||||
4. **Database rebuild is automatic** - The update script handles this for you
|
||||
5. **Template sanitization is automatic** - Any API tokens in workflow templates are replaced with placeholders
|
||||
6. **Docker image builds automatically** - Pushing to GitHub triggers the workflow
|
||||
1. **ALWAYS check existing releases first** - Use `gh release list` to see what versions are already released. Your new version must be higher!
|
||||
2. **Release workflow only triggers on version CHANGE** - If you merge a PR with an already-released version (e.g., 2.22.8), the workflow won't run. You'll need to bump to a new version (e.g., 2.22.9) and create another PR.
|
||||
3. **Integration test failures in CI are usually infrastructure issues** - If unit tests pass but integration tests fail with "unauthorized", this is typically because the test n8n instance credentials need updating. The code itself is fine.
|
||||
4. **Skip local tests - let CI handle them** - Running tests locally adds 2-3 minutes with no benefit since CI runs them anyway. The fast workflow skips local tests.
|
||||
5. **The update script is smart** - It automatically syncs all n8n dependencies to compatible versions
|
||||
6. **Database rebuild is automatic** - The update script handles this for you
|
||||
7. **Template sanitization is automatic** - Any API tokens in workflow templates are replaced with placeholders
|
||||
8. **Docker image builds automatically** - Pushing to GitHub triggers the workflow
|
||||
|
||||
## GitHub Push Protection
|
||||
|
||||
@@ -115,11 +199,27 @@ As of July 2025, GitHub's push protection may block database pushes if they cont
|
||||
3. If push is still blocked, use the GitHub web interface to review and allow the push
|
||||
|
||||
## Time Estimate
|
||||
|
||||
### Fast Workflow (Recommended)
|
||||
- Local work: ~2-3 minutes
|
||||
- npm install and database rebuild: ~2-3 minutes
|
||||
- File edits (CHANGELOG, README, package.json): ~30 seconds
|
||||
- Git operations (commit, push, create PR): ~30 seconds
|
||||
- CI testing after PR creation: ~8-10 minutes (runs automatically)
|
||||
- Unit tests: ~2 minutes
|
||||
- Integration tests: ~6 minutes (may fail with infrastructure issues - ignore if unit tests pass)
|
||||
- Other checks: ~1 minute
|
||||
|
||||
**Total hands-on time: ~3 minutes** (then wait for CI)
|
||||
|
||||
### Full Workflow with Local Tests
|
||||
- Total time: ~5-7 minutes
|
||||
- Test suite: ~2.5 minutes
|
||||
- npm install and database rebuild: ~2-3 minutes
|
||||
- The rest: seconds
|
||||
|
||||
**Note**: The fast workflow is recommended since CI runs the same tests anyway.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If tests fail:
|
||||
|
||||
34
README.md
34
README.md
@@ -5,23 +5,23 @@
|
||||
[](https://www.npmjs.com/package/n8n-mcp)
|
||||
[](https://codecov.io/gh/czlonkowski/n8n-mcp)
|
||||
[](https://github.com/czlonkowski/n8n-mcp/actions)
|
||||
[](https://github.com/n8n-io/n8n)
|
||||
[](https://github.com/n8n-io/n8n)
|
||||
[](https://github.com/czlonkowski/n8n-mcp/pkgs/container/n8n-mcp)
|
||||
[](https://railway.com/deploy/n8n-mcp?referralCode=n8n-mcp)
|
||||
|
||||
A Model Context Protocol (MCP) server that provides AI assistants with comprehensive access to n8n node documentation, properties, and operations. Deploy in minutes to give Claude and other AI assistants deep knowledge about n8n's 525+ workflow automation nodes.
|
||||
A Model Context Protocol (MCP) server that provides AI assistants with comprehensive access to n8n node documentation, properties, and operations. Deploy in minutes to give Claude and other AI assistants deep knowledge about n8n's 541 workflow automation nodes.
|
||||
|
||||
## Overview
|
||||
|
||||
n8n-MCP serves as a bridge between n8n's workflow automation platform and AI models, enabling them to understand and work with n8n nodes effectively. It provides structured access to:
|
||||
|
||||
- 📚 **536 n8n nodes** from both n8n-nodes-base and @n8n/n8n-nodes-langchain
|
||||
- 📚 **541 n8n nodes** from both n8n-nodes-base and @n8n/n8n-nodes-langchain
|
||||
- 🔧 **Node properties** - 99% coverage with detailed schemas
|
||||
- ⚡ **Node operations** - 63.6% coverage of available actions
|
||||
- 📄 **Documentation** - 90% coverage from official n8n docs (including AI nodes)
|
||||
- 🤖 **AI tools** - 263 AI-capable nodes detected with full documentation
|
||||
- 📄 **Documentation** - 87% coverage from official n8n docs (including AI nodes)
|
||||
- 🤖 **AI tools** - 271 AI-capable nodes detected with full documentation
|
||||
- 💡 **Real-world examples** - 2,646 pre-extracted configurations from popular templates
|
||||
- 🎯 **Template library** - 2,500+ workflow templates with smart filtering
|
||||
- 🎯 **Template library** - 2,709 workflow templates with 100% metadata coverage
|
||||
|
||||
|
||||
## ⚠️ Important Safety Warning
|
||||
@@ -51,6 +51,8 @@ npx n8n-mcp
|
||||
|
||||
Add to Claude Desktop config:
|
||||
|
||||
> ⚠️ **Important**: The `MCP_MODE: "stdio"` environment variable is **required** for Claude Desktop. Without it, you will see JSON parsing errors like `"Unexpected token..."` in the UI. This variable ensures that only JSON-RPC messages are sent to stdout, preventing debug logs from interfering with the protocol.
|
||||
|
||||
**Basic configuration (documentation tools only):**
|
||||
```json
|
||||
{
|
||||
@@ -531,7 +533,7 @@ When operations are independent, execute them in parallel for maximum performanc
|
||||
❌ BAD: Sequential tool calls (await each one before the next)
|
||||
|
||||
### 3. Templates First
|
||||
ALWAYS check templates before building from scratch (2,500+ available).
|
||||
ALWAYS check templates before building from scratch (2,709 available).
|
||||
|
||||
### 4. Multi-Level Validation
|
||||
Use validate_node_minimal → validate_node_operation → validate_workflow pattern.
|
||||
@@ -840,7 +842,7 @@ n8n_update_partial_workflow({
|
||||
### Core Behavior
|
||||
1. **Silent execution** - No commentary between tools
|
||||
2. **Parallel by default** - Execute independent operations simultaneously
|
||||
3. **Templates first** - Always check before building (2,500+ available)
|
||||
3. **Templates first** - Always check before building (2,709 available)
|
||||
4. **Multi-level validation** - Quick check → Full validation → Workflow validation
|
||||
5. **Never trust defaults** - Explicitly configure ALL parameters
|
||||
|
||||
@@ -943,7 +945,7 @@ Once connected, Claude can use these powerful tools:
|
||||
- **`get_node_as_tool_info`** - Get guidance on using any node as an AI tool
|
||||
|
||||
### Template Tools
|
||||
- **`list_templates`** - Browse all templates with descriptions and optional metadata (2,500+ templates)
|
||||
- **`list_templates`** - Browse all templates with descriptions and optional metadata (2,709 templates)
|
||||
- **`search_templates`** - Text search across template names and descriptions
|
||||
- **`search_templates_by_metadata`** - Advanced filtering by complexity, setup time, services, audience
|
||||
- **`list_node_templates`** - Find templates using specific nodes
|
||||
@@ -1098,17 +1100,17 @@ npm run dev:http # HTTP dev mode
|
||||
|
||||
## 📊 Metrics & Coverage
|
||||
|
||||
Current database coverage (n8n v1.113.3):
|
||||
Current database coverage (n8n v1.117.2):
|
||||
|
||||
- ✅ **536/536** nodes loaded (100%)
|
||||
- ✅ **528** nodes with properties (98.7%)
|
||||
- ✅ **470** nodes with documentation (88%)
|
||||
- ✅ **267** AI-capable tools detected
|
||||
- ✅ **541/541** nodes loaded (100%)
|
||||
- ✅ **541** nodes with properties (100%)
|
||||
- ✅ **470** nodes with documentation (87%)
|
||||
- ✅ **271** AI-capable tools detected
|
||||
- ✅ **2,646** pre-extracted template configurations
|
||||
- ✅ **2,500+** workflow templates available
|
||||
- ✅ **2,709** workflow templates available (100% metadata coverage)
|
||||
- ✅ **AI Agent & LangChain nodes** fully documented
|
||||
- ⚡ **Average response time**: ~12ms
|
||||
- 💾 **Database size**: ~15MB (optimized)
|
||||
- 💾 **Database size**: ~68MB (includes templates with metadata)
|
||||
|
||||
## 🔄 Recent Updates
|
||||
|
||||
|
||||
BIN
data/nodes.db
BIN
data/nodes.db
Binary file not shown.
@@ -4,7 +4,9 @@ Connect n8n-MCP to Claude Code CLI for enhanced n8n workflow development from th
|
||||
|
||||
## Quick Setup via CLI
|
||||
|
||||
### Basic configuration (documentation tools only):
|
||||
### Basic configuration (documentation tools only)
|
||||
|
||||
**For Linux, macOS, or Windows (WSL/Git Bash):**
|
||||
```bash
|
||||
claude mcp add n8n-mcp \
|
||||
-e MCP_MODE=stdio \
|
||||
@@ -13,9 +15,21 @@ claude mcp add n8n-mcp \
|
||||
-- npx n8n-mcp
|
||||
```
|
||||
|
||||
**For native Windows PowerShell:**
|
||||
```powershell
|
||||
# Note: The backtick ` is PowerShell's line continuation character.
|
||||
claude mcp add n8n-mcp `
|
||||
'-e MCP_MODE=stdio' `
|
||||
'-e LOG_LEVEL=error' `
|
||||
'-e DISABLE_CONSOLE_OUTPUT=true' `
|
||||
-- npx n8n-mcp
|
||||
```
|
||||
|
||||
|
||||
|
||||
### Full configuration (with n8n management tools):
|
||||
### Full configuration (with n8n management tools)
|
||||
|
||||
**For Linux, macOS, or Windows (WSL/Git Bash):**
|
||||
```bash
|
||||
claude mcp add n8n-mcp \
|
||||
-e MCP_MODE=stdio \
|
||||
@@ -26,6 +40,18 @@ claude mcp add n8n-mcp \
|
||||
-- npx n8n-mcp
|
||||
```
|
||||
|
||||
**For native Windows PowerShell:**
|
||||
```powershell
|
||||
# Note: The backtick ` is PowerShell's line continuation character.
|
||||
claude mcp add n8n-mcp `
|
||||
'-e MCP_MODE=stdio' `
|
||||
'-e LOG_LEVEL=error' `
|
||||
'-e DISABLE_CONSOLE_OUTPUT=true' `
|
||||
'-e N8N_API_URL=https://your-n8n-instance.com' `
|
||||
'-e N8N_API_KEY=your-api-key' `
|
||||
-- npx n8n-mcp
|
||||
```
|
||||
|
||||
Make sure to replace `https://your-n8n-instance.com` with your actual n8n URL and `your-api-key` with your n8n API key.
|
||||
|
||||
## Alternative Setup Methods
|
||||
@@ -133,9 +159,11 @@ For optimal results, create a `CLAUDE.md` file in your project root with the ins
|
||||
|
||||
## Tips
|
||||
|
||||
- If you're running n8n locally, use `http://localhost:5678` as the N8N_API_URL
|
||||
- The n8n API credentials are optional - without them, you'll have documentation and validation tools only
|
||||
- With API credentials, you'll get full workflow management capabilities
|
||||
- Use `--scope local` (default) to keep your API credentials private
|
||||
- Use `--scope project` to share configuration with your team (put credentials in environment variables)
|
||||
- Claude Code will automatically start the MCP server when you begin a conversation
|
||||
- If you're running n8n locally, use `http://localhost:5678` as the `N8N_API_URL`.
|
||||
- The n8n API credentials are optional. Without them, you'll only have access to documentation and validation tools. With credentials, you get full workflow management capabilities.
|
||||
- **Scope Management:**
|
||||
- By default, `claude mcp add` uses `--scope local` (also called "user scope"), which saves the configuration to your global user settings and keeps API keys private.
|
||||
- To share the configuration with your team, use `--scope project`. This saves the configuration to a `.mcp.json` file in your project's root directory.
|
||||
- **Switching Scope:** The cleanest method is to `remove` the server and then `add` it back with the desired scope flag (e.g., `claude mcp remove n8n-mcp` followed by `claude mcp add n8n-mcp --scope project`).
|
||||
- **Manual Switching (Advanced):** You can manually edit your `.claude.json` file (e.g., `C:\Users\YourName\.claude.json`). To switch, cut the `"n8n-mcp": { ... }` block from the top-level `"mcpServers"` object (user scope) and paste it into the nested `"mcpServers"` object under your project's path key (project scope), or vice versa. **Important:** You may need to restart Claude Code for manual changes to take effect.
|
||||
- Claude Code will automatically start the MCP server when you begin a conversation.
|
||||
|
||||
1923
package-lock.json
generated
1923
package-lock.json
generated
File diff suppressed because it is too large
Load Diff
10
package.json
10
package.json
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "n8n-mcp",
|
||||
"version": "2.22.7",
|
||||
"version": "2.22.9",
|
||||
"description": "Integration between n8n workflow automation and Model Context Protocol (MCP)",
|
||||
"main": "dist/index.js",
|
||||
"types": "dist/index.d.ts",
|
||||
@@ -140,15 +140,15 @@
|
||||
},
|
||||
"dependencies": {
|
||||
"@modelcontextprotocol/sdk": "^1.20.1",
|
||||
"@n8n/n8n-nodes-langchain": "^1.115.1",
|
||||
"@n8n/n8n-nodes-langchain": "^1.117.0",
|
||||
"@supabase/supabase-js": "^2.57.4",
|
||||
"dotenv": "^16.5.0",
|
||||
"express": "^5.1.0",
|
||||
"express-rate-limit": "^7.1.5",
|
||||
"lru-cache": "^11.2.1",
|
||||
"n8n": "^1.116.2",
|
||||
"n8n-core": "^1.115.1",
|
||||
"n8n-workflow": "^1.113.0",
|
||||
"n8n": "^1.118.1",
|
||||
"n8n-core": "^1.117.0",
|
||||
"n8n-workflow": "^1.115.0",
|
||||
"openai": "^4.77.0",
|
||||
"sql.js": "^1.13.0",
|
||||
"tslib": "^2.6.2",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "n8n-mcp-runtime",
|
||||
"version": "2.22.7",
|
||||
"version": "2.22.8",
|
||||
"description": "n8n MCP Server Runtime Dependencies Only",
|
||||
"private": true,
|
||||
"dependencies": {
|
||||
|
||||
99
scripts/process-batch-metadata.ts
Normal file
99
scripts/process-batch-metadata.ts
Normal file
@@ -0,0 +1,99 @@
|
||||
#!/usr/bin/env ts-node
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import { createDatabaseAdapter } from '../src/database/database-adapter';
|
||||
|
||||
interface BatchResponse {
|
||||
id: string;
|
||||
custom_id: string;
|
||||
response: {
|
||||
status_code: number;
|
||||
body: {
|
||||
choices: Array<{
|
||||
message: {
|
||||
content: string;
|
||||
};
|
||||
}>;
|
||||
};
|
||||
};
|
||||
error: any;
|
||||
}
|
||||
|
||||
async function processBatchMetadata(batchFile: string) {
|
||||
console.log(`📥 Processing batch file: ${batchFile}`);
|
||||
|
||||
// Read the JSONL file
|
||||
const content = fs.readFileSync(batchFile, 'utf-8');
|
||||
const lines = content.trim().split('\n');
|
||||
|
||||
console.log(`📊 Found ${lines.length} batch responses`);
|
||||
|
||||
// Initialize database
|
||||
const db = await createDatabaseAdapter('./data/nodes.db');
|
||||
|
||||
let updated = 0;
|
||||
let skipped = 0;
|
||||
let errors = 0;
|
||||
|
||||
for (const line of lines) {
|
||||
try {
|
||||
const response: BatchResponse = JSON.parse(line);
|
||||
|
||||
// Extract template ID from custom_id (format: "template-9100")
|
||||
const templateId = parseInt(response.custom_id.replace('template-', ''));
|
||||
|
||||
// Check for errors
|
||||
if (response.error || response.response.status_code !== 200) {
|
||||
console.warn(`⚠️ Template ${templateId}: API error`, response.error);
|
||||
errors++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Extract metadata from response
|
||||
const metadataJson = response.response.body.choices[0].message.content;
|
||||
|
||||
// Validate it's valid JSON
|
||||
JSON.parse(metadataJson); // Will throw if invalid
|
||||
|
||||
// Update database
|
||||
const stmt = db.prepare(`
|
||||
UPDATE templates
|
||||
SET metadata_json = ?
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
stmt.run(metadataJson, templateId);
|
||||
updated++;
|
||||
|
||||
console.log(`✅ Template ${templateId}: Updated metadata`);
|
||||
|
||||
} catch (error: any) {
|
||||
console.error(`❌ Error processing line:`, error.message);
|
||||
errors++;
|
||||
}
|
||||
}
|
||||
|
||||
// Close database
|
||||
if ('close' in db && typeof db.close === 'function') {
|
||||
db.close();
|
||||
}
|
||||
|
||||
console.log(`\n📈 Summary:`);
|
||||
console.log(` - Updated: ${updated}`);
|
||||
console.log(` - Skipped: ${skipped}`);
|
||||
console.log(` - Errors: ${errors}`);
|
||||
console.log(` - Total: ${lines.length}`);
|
||||
}
|
||||
|
||||
// Main
|
||||
const batchFile = process.argv[2] || '/Users/romualdczlonkowski/Pliki/n8n-mcp/n8n-mcp/docs/batch_68fff7242850819091cfed64f10fb6b4_output.jsonl';
|
||||
|
||||
processBatchMetadata(batchFile)
|
||||
.then(() => {
|
||||
console.log('\n✅ Batch processing complete!');
|
||||
process.exit(0);
|
||||
})
|
||||
.catch((error) => {
|
||||
console.error('\n❌ Batch processing failed:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -75,10 +75,15 @@ async function fetchTemplatesRobust() {
|
||||
|
||||
// Fetch detail
|
||||
const detail = await fetcher.fetchTemplateDetail(template.id);
|
||||
|
||||
// Save immediately
|
||||
repository.saveTemplate(template, detail);
|
||||
saved++;
|
||||
|
||||
if (detail !== null) {
|
||||
// Save immediately
|
||||
repository.saveTemplate(template, detail);
|
||||
saved++;
|
||||
} else {
|
||||
errors++;
|
||||
console.error(`\n❌ Failed to fetch template ${template.id} (${template.name}) after retries`);
|
||||
}
|
||||
|
||||
// Rate limiting
|
||||
await new Promise(resolve => setTimeout(resolve, 200));
|
||||
|
||||
@@ -133,6 +133,7 @@ export function cleanWorkflowForUpdate(workflow: Workflow): Partial<Workflow> {
|
||||
createdAt,
|
||||
updatedAt,
|
||||
versionId,
|
||||
versionCounter, // Added: n8n 1.118.1+ returns this but rejects it in updates
|
||||
meta,
|
||||
staticData,
|
||||
// Remove fields that cause API errors
|
||||
|
||||
@@ -40,7 +40,37 @@ export interface TemplateDetail {
|
||||
export class TemplateFetcher {
|
||||
private readonly baseUrl = 'https://api.n8n.io/api/templates';
|
||||
private readonly pageSize = 250; // Maximum allowed by API
|
||||
|
||||
private readonly maxRetries = 3;
|
||||
private readonly retryDelay = 1000; // 1 second base delay
|
||||
|
||||
/**
|
||||
* Retry helper for API calls
|
||||
*/
|
||||
private async retryWithBackoff<T>(
|
||||
fn: () => Promise<T>,
|
||||
context: string,
|
||||
maxRetries: number = this.maxRetries
|
||||
): Promise<T | null> {
|
||||
let lastError: any;
|
||||
|
||||
for (let attempt = 1; attempt <= maxRetries; attempt++) {
|
||||
try {
|
||||
return await fn();
|
||||
} catch (error: any) {
|
||||
lastError = error;
|
||||
|
||||
if (attempt < maxRetries) {
|
||||
const delay = this.retryDelay * attempt; // Exponential backoff
|
||||
logger.warn(`${context} - Attempt ${attempt}/${maxRetries} failed, retrying in ${delay}ms...`);
|
||||
await this.sleep(delay);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
logger.error(`${context} - All ${maxRetries} attempts failed, skipping`, lastError);
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch all templates and filter to last 12 months
|
||||
* This fetches ALL pages first, then applies date filter locally
|
||||
@@ -73,93 +103,105 @@ export class TemplateFetcher {
|
||||
let page = 1;
|
||||
let hasMore = true;
|
||||
let totalWorkflows = 0;
|
||||
|
||||
|
||||
logger.info('Starting complete template fetch from n8n.io API');
|
||||
|
||||
|
||||
while (hasMore) {
|
||||
try {
|
||||
const response = await axios.get(`${this.baseUrl}/search`, {
|
||||
params: {
|
||||
page,
|
||||
rows: this.pageSize
|
||||
// Note: sort_by parameter doesn't work, templates come in popularity order
|
||||
}
|
||||
});
|
||||
|
||||
const { workflows } = response.data;
|
||||
totalWorkflows = response.data.totalWorkflows || totalWorkflows;
|
||||
|
||||
allTemplates.push(...workflows);
|
||||
|
||||
// Calculate total pages for better progress reporting
|
||||
const totalPages = Math.ceil(totalWorkflows / this.pageSize);
|
||||
|
||||
if (progressCallback) {
|
||||
// Enhanced progress with page information
|
||||
progressCallback(allTemplates.length, totalWorkflows);
|
||||
}
|
||||
|
||||
logger.debug(`Fetched page ${page}/${totalPages}: ${workflows.length} templates (total so far: ${allTemplates.length}/${totalWorkflows})`);
|
||||
|
||||
// Check if there are more pages
|
||||
if (workflows.length < this.pageSize) {
|
||||
hasMore = false;
|
||||
}
|
||||
|
||||
const result = await this.retryWithBackoff(
|
||||
async () => {
|
||||
const response = await axios.get(`${this.baseUrl}/search`, {
|
||||
params: {
|
||||
page,
|
||||
rows: this.pageSize
|
||||
// Note: sort_by parameter doesn't work, templates come in popularity order
|
||||
}
|
||||
});
|
||||
return response.data;
|
||||
},
|
||||
`Fetching templates page ${page}`
|
||||
);
|
||||
|
||||
if (result === null) {
|
||||
// All retries failed for this page, skip it and continue
|
||||
logger.warn(`Skipping page ${page} after ${this.maxRetries} failed attempts`);
|
||||
page++;
|
||||
|
||||
// Rate limiting - be nice to the API (slightly faster with 250 rows/page)
|
||||
if (hasMore) {
|
||||
await this.sleep(300); // 300ms between requests (was 500ms with 100 rows)
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error(`Error fetching templates page ${page}:`, error);
|
||||
throw error;
|
||||
continue;
|
||||
}
|
||||
|
||||
const { workflows } = result;
|
||||
totalWorkflows = result.totalWorkflows || totalWorkflows;
|
||||
|
||||
allTemplates.push(...workflows);
|
||||
|
||||
// Calculate total pages for better progress reporting
|
||||
const totalPages = Math.ceil(totalWorkflows / this.pageSize);
|
||||
|
||||
if (progressCallback) {
|
||||
// Enhanced progress with page information
|
||||
progressCallback(allTemplates.length, totalWorkflows);
|
||||
}
|
||||
|
||||
logger.debug(`Fetched page ${page}/${totalPages}: ${workflows.length} templates (total so far: ${allTemplates.length}/${totalWorkflows})`);
|
||||
|
||||
// Check if there are more pages
|
||||
if (workflows.length < this.pageSize) {
|
||||
hasMore = false;
|
||||
}
|
||||
|
||||
page++;
|
||||
|
||||
// Rate limiting - be nice to the API (slightly faster with 250 rows/page)
|
||||
if (hasMore) {
|
||||
await this.sleep(300); // 300ms between requests (was 500ms with 100 rows)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
logger.info(`Fetched all ${allTemplates.length} templates from n8n.io`);
|
||||
return allTemplates;
|
||||
}
|
||||
|
||||
async fetchTemplateDetail(workflowId: number): Promise<TemplateDetail> {
|
||||
try {
|
||||
const response = await axios.get(`${this.baseUrl}/workflows/${workflowId}`);
|
||||
return response.data.workflow;
|
||||
} catch (error) {
|
||||
logger.error(`Error fetching template detail for ${workflowId}:`, error);
|
||||
throw error;
|
||||
}
|
||||
async fetchTemplateDetail(workflowId: number): Promise<TemplateDetail | null> {
|
||||
const result = await this.retryWithBackoff(
|
||||
async () => {
|
||||
const response = await axios.get(`${this.baseUrl}/workflows/${workflowId}`);
|
||||
return response.data.workflow;
|
||||
},
|
||||
`Fetching template detail for workflow ${workflowId}`
|
||||
);
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
async fetchAllTemplateDetails(
|
||||
workflows: TemplateWorkflow[],
|
||||
workflows: TemplateWorkflow[],
|
||||
progressCallback?: (current: number, total: number) => void
|
||||
): Promise<Map<number, TemplateDetail>> {
|
||||
const details = new Map<number, TemplateDetail>();
|
||||
|
||||
let skipped = 0;
|
||||
|
||||
logger.info(`Fetching details for ${workflows.length} templates`);
|
||||
|
||||
|
||||
for (let i = 0; i < workflows.length; i++) {
|
||||
const workflow = workflows[i];
|
||||
|
||||
try {
|
||||
const detail = await this.fetchTemplateDetail(workflow.id);
|
||||
|
||||
const detail = await this.fetchTemplateDetail(workflow.id);
|
||||
|
||||
if (detail !== null) {
|
||||
details.set(workflow.id, detail);
|
||||
|
||||
if (progressCallback) {
|
||||
progressCallback(i + 1, workflows.length);
|
||||
}
|
||||
|
||||
// Rate limiting (conservative to avoid API throttling)
|
||||
await this.sleep(150); // 150ms between requests
|
||||
} catch (error) {
|
||||
logger.error(`Failed to fetch details for workflow ${workflow.id}:`, error);
|
||||
// Continue with other templates
|
||||
} else {
|
||||
skipped++;
|
||||
logger.warn(`Skipped workflow ${workflow.id} after ${this.maxRetries} failed attempts`);
|
||||
}
|
||||
|
||||
if (progressCallback) {
|
||||
progressCallback(i + 1, workflows.length);
|
||||
}
|
||||
|
||||
// Rate limiting (conservative to avoid API throttling)
|
||||
await this.sleep(150); // 150ms between requests
|
||||
}
|
||||
|
||||
logger.info(`Successfully fetched ${details.size} template details`);
|
||||
|
||||
logger.info(`Successfully fetched ${details.size} template details (${skipped} skipped)`);
|
||||
return details;
|
||||
}
|
||||
|
||||
|
||||
@@ -496,10 +496,17 @@ export class TemplateRepository {
|
||||
// Count node usage
|
||||
const nodeCount: Record<string, number> = {};
|
||||
topNodes.forEach(t => {
|
||||
const nodes = JSON.parse(t.nodes_used);
|
||||
nodes.forEach((n: string) => {
|
||||
nodeCount[n] = (nodeCount[n] || 0) + 1;
|
||||
});
|
||||
if (!t.nodes_used) return;
|
||||
try {
|
||||
const nodes = JSON.parse(t.nodes_used);
|
||||
if (Array.isArray(nodes)) {
|
||||
nodes.forEach((n: string) => {
|
||||
nodeCount[n] = (nodeCount[n] || 0) + 1;
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
logger.warn(`Failed to parse nodes_used for template stats:`, error);
|
||||
}
|
||||
});
|
||||
|
||||
// Get top 10 most used nodes
|
||||
|
||||
@@ -66,6 +66,7 @@ export interface Workflow {
|
||||
updatedAt?: string;
|
||||
createdAt?: string;
|
||||
versionId?: string;
|
||||
versionCounter?: number; // Added: n8n 1.118.1+ returns this in GET responses
|
||||
meta?: {
|
||||
instanceId?: string;
|
||||
};
|
||||
@@ -152,6 +153,7 @@ export interface WorkflowExport {
|
||||
tags?: string[];
|
||||
pinData?: Record<string, unknown>;
|
||||
versionId?: string;
|
||||
versionCounter?: number; // Added: n8n 1.118.1+
|
||||
meta?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
|
||||
@@ -313,6 +313,7 @@ describe('n8n-validation', () => {
|
||||
createdAt: '2023-01-01',
|
||||
updatedAt: '2023-01-01',
|
||||
versionId: 'v123',
|
||||
versionCounter: 5, // n8n 1.118.1+ field
|
||||
meta: { test: 'data' },
|
||||
staticData: { some: 'data' },
|
||||
pinData: { pin: 'data' },
|
||||
@@ -333,6 +334,7 @@ describe('n8n-validation', () => {
|
||||
expect(cleaned).not.toHaveProperty('createdAt');
|
||||
expect(cleaned).not.toHaveProperty('updatedAt');
|
||||
expect(cleaned).not.toHaveProperty('versionId');
|
||||
expect(cleaned).not.toHaveProperty('versionCounter'); // n8n 1.118.1+ compatibility
|
||||
expect(cleaned).not.toHaveProperty('meta');
|
||||
expect(cleaned).not.toHaveProperty('staticData');
|
||||
expect(cleaned).not.toHaveProperty('pinData');
|
||||
@@ -349,6 +351,22 @@ describe('n8n-validation', () => {
|
||||
expect(cleaned.settings).toEqual({ executionOrder: 'v1' });
|
||||
});
|
||||
|
||||
it('should exclude versionCounter for n8n 1.118.1+ compatibility', () => {
|
||||
const workflow = {
|
||||
name: 'Test Workflow',
|
||||
nodes: [],
|
||||
connections: {},
|
||||
versionId: 'v123',
|
||||
versionCounter: 5, // n8n 1.118.1 returns this but rejects it in PUT
|
||||
} as any;
|
||||
|
||||
const cleaned = cleanWorkflowForUpdate(workflow);
|
||||
|
||||
expect(cleaned).not.toHaveProperty('versionCounter');
|
||||
expect(cleaned).not.toHaveProperty('versionId');
|
||||
expect(cleaned.name).toBe('Test Workflow');
|
||||
});
|
||||
|
||||
it('should add empty settings object for cloud API compatibility', () => {
|
||||
const workflow = {
|
||||
name: 'Test Workflow',
|
||||
|
||||
Reference in New Issue
Block a user