Compare commits

..

8 Commits

Author SHA1 Message Date
czlonkowski
a1291c59f3 fix: resolve TypeScript lint errors in telemetry tests
Fixed type issues in mutation-tracker and mutation-validator tests:
- Import and use MutationToolName enum instead of string literals
- Fix ValidationResult.errors to use proper object structure
- Add UpdateNodeOperation type assertion for operation with nodeName

All TypeScript errors resolved, lint now passes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 14:06:00 +01:00
czlonkowski
5cb1263468 chore: bump version to 2.22.16 with telemetry changelog
Updated package.json and package.runtime.json to version 2.22.16.
Added comprehensive CHANGELOG entry documenting workflow mutation
telemetry enhancements for better AI-powered workflow assistance.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 13:48:43 +01:00
czlonkowski
7ac748e73f feat: enhance workflow mutation telemetry for better AI responses
Improve workflow mutation tracking to capture comprehensive data that helps provide better responses when users update workflows. This enhancement collects workflow state, user intent, and operation details to enable more context-aware assistance.

Key improvements:
- Reduce auto-flush threshold from 5 to 2 for more reliable mutation tracking
- Add comprehensive workflow and credential sanitization to mutation tracker
- Document intent parameter in workflow update tools for better UX
- Fix mutation queue handling in telemetry manager (flush now handles 3 queues)
- Add extensive unit tests for mutation tracking and validation (35 new tests)

Technical changes:
- mutation-tracker.ts: Multi-layer sanitization (workflow, node, parameter levels)
- batch-processor.ts: Support mutation data flushing to Supabase
- telemetry-manager.ts: Auto-flush mutations at threshold 2, track mutations queue
- handlers-workflow-diff.ts: Track workflow mutations with sanitized data
- Tests: 13 tests for mutation-tracker, 22 tests for mutation-validator

The intent parameter messaging emphasizes user benefit ("helps to return better response") rather than technical implementation details.

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-13 13:42:15 +01:00
czlonkowski
6719628350 fix: improve mutation telemetry error logging and diagnostics
Changes:
- Upgrade error logging from debug to warn level for better visibility
- Add diagnostic logging to track mutation processing
- Log telemetry disabled state explicitly
- Add context info (sessionId, intent, operationCount) to error logs
- Remove 'await' from telemetry calls to make them truly non-blocking

This will help identify why mutations aren't being persisted to the
workflow_mutations table despite successful workflow operations.

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en
2025-11-12 17:52:52 +01:00
czlonkowski
00a2e77643 fix: reduce mutation auto-flush threshold from 5 to 2
Lower the auto-flush threshold for workflow mutations from 5 to 2 to ensure
more timely data persistence. Since mutations are less frequent than regular
telemetry events, a lower threshold provides:

- Faster data persistence (don't wait for 5 mutations)
- Better testing experience (easier to verify with fewer operations)
- Reduced risk of data loss if process exits before threshold
- More responsive telemetry for low-volume mutation scenarios

This complements the existing 5-second periodic flush and process exit
handlers, ensuring mutations are persisted promptly.

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en
2025-11-12 17:24:30 +01:00
czlonkowski
0ae8734148 fix: enable RLS policies for workflow_mutations table
Enable Row-Level Security and add policies:
- Allow anonymous (anon) inserts for telemetry data collection
- Allow authenticated reads for data analysis and querying

These policies are required for the telemetry system to function
correctly with Supabase, as the MCP server uses the anon key to
insert mutation data.

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en
2025-11-12 16:57:45 +01:00
czlonkowski
efe9437f20 fix: correct SQL syntax for expression index in workflow_mutations schema
The expression index for significant changes needs double parentheses
around the arithmetic expression to be valid PostgreSQL syntax.

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en
2025-11-12 16:39:06 +01:00
czlonkowski
61fdd6433a feat: add comprehensive telemetry for partial workflow updates
Implement telemetry infrastructure to track workflow mutations from
partial update operations. This enables data-driven improvements to
partial update tooling by capturing:

- Workflow state before and after mutations
- User intent and operation patterns
- Validation results and improvements
- Change metrics (nodes/connections modified)
- Success/failure rates and error patterns

New Components:
- Intent classifier: Categorizes mutation patterns
- Intent sanitizer: Removes PII from user instructions
- Mutation validator: Ensures data quality before tracking
- Mutation tracker: Coordinates validation and metric calculation

Extended Components:
- TelemetryManager: New trackWorkflowMutation() method
- EventTracker: Mutation queue management
- BatchProcessor: Mutation data flushing to Supabase

MCP Tool Enhancements:
- n8n_update_partial_workflow: Added optional 'intent' parameter
- n8n_update_full_workflow: Added optional 'intent' parameter
- Both tools now track mutations asynchronously

Database Schema:
- New workflow_mutations table with 20+ fields
- Comprehensive indexes for efficient querying
- Supports deduplication and data analysis

This telemetry system is:
- Privacy-focused (PII sanitization, anonymized users)
- Non-blocking (async tracking, silent failures)
- Production-ready (batching, retries, circuit breaker)
- Backward compatible (all parameters optional)

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en
2025-11-12 16:36:16 +01:00
11 changed files with 137 additions and 630 deletions

View File

@@ -7,77 +7,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [2.22.17] - 2025-11-13
### 🐛 Bug Fixes
**Critical Telemetry Improvements**
Fixed three critical issues in workflow mutation telemetry to improve data quality and security:
#### 1. Fixed Inconsistent Sanitization (Security Critical)
- **Problem**: 30% of workflows (178-188 records) were unsanitized, exposing potential credentials/tokens
- **Solution**: Replaced weak inline sanitization with robust `WorkflowSanitizer.sanitizeWorkflowRaw()`
- **Impact**: Now 100% sanitization coverage with 17 sensitive patterns detected and redacted
- **Files Modified**:
- `src/telemetry/workflow-sanitizer.ts`: Added `sanitizeWorkflowRaw()` method
- `src/telemetry/mutation-tracker.ts`: Removed redundant sanitization code, use centralized sanitizer
#### 2. Enabled Validation Data Capture (Data Quality Blocker)
- **Problem**: Zero validation metrics captured (validation_before/after all NULL)
- **Solution**: Added workflow validation before and after mutations using `WorkflowValidator`
- **Impact**: Can now measure mutation quality, track error resolution patterns
- **Implementation**:
- Validates workflows before mutation (captures baseline errors)
- Validates workflows after mutation (measures improvement)
- Non-blocking: validation errors don't prevent mutations
- Captures: errors, warnings, validation status
- **Files Modified**:
- `src/mcp/handlers-workflow-diff.ts`: Added pre/post mutation validation
#### 3. Improved Intent Capture (Data Quality)
- **Problem**: 92.62% of intents were generic "Partial workflow update"
- **Solution**: Enhanced tool documentation + automatic intent inference from operations
- **Impact**: Meaningful intents automatically generated when not explicitly provided
- **Implementation**:
- Enhanced documentation with specific intent examples and anti-patterns
- Added `inferIntentFromOperations()` function that generates meaningful intents:
- Single operations: "Add n8n-nodes-base.slack", "Connect webhook to HTTP Request"
- Multiple operations: "Workflow update: add 2 nodes, modify connections"
- Fallback inference when intent is missing, generic, or too short
- **Files Modified**:
- `src/mcp/tool-docs/workflow_management/n8n-update-partial-workflow.ts`: Enhanced guidance
- `src/mcp/handlers-workflow-diff.ts`: Added intent inference logic
### 📊 Expected Results
After deployment, telemetry data should show:
- **100% sanitization coverage** (up from 70%)
- **100% validation capture** (up from 0%)
- **50%+ meaningful intents** (up from 7.33%)
- **Complete telemetry dataset** for analysis
### 🎯 Technical Details
**Sanitization Coverage**: Now detects and redacts:
- Webhook URLs, API keys (OpenAI sk-*, GitHub ghp-*, etc.)
- Bearer tokens, OAuth credentials, passwords
- URLs with authentication, long tokens (20+ chars)
- Sensitive field names (apiKey, token, secret, password, etc.)
**Validation Metrics Captured**:
- Workflow validity status (true/false)
- Error/warning counts and details
- Node configuration errors
- Connection errors
- Expression syntax errors
- Validation improvement tracking (errors resolved/introduced)
**Intent Inference Examples**:
- `addNode` → "Add n8n-nodes-base.webhook"
- `rewireConnection` → "Rewire IF from ErrorHandler to SuccessHandler"
- Multiple operations → "Workflow update: add 2 nodes, modify connections, update metadata"
## [2.22.16] - 2025-11-13
### ✨ Enhanced Features

Binary file not shown.

View File

@@ -1,6 +1,6 @@
{
"name": "n8n-mcp",
"version": "2.22.17",
"version": "2.22.16",
"description": "Integration between n8n workflow automation and Model Context Protocol (MCP)",
"main": "dist/index.js",
"types": "dist/index.d.ts",

View File

@@ -1,6 +1,6 @@
{
"name": "n8n-mcp-runtime",
"version": "2.22.17",
"version": "2.22.16",
"description": "n8n MCP Server Runtime Dependencies Only",
"private": true,
"dependencies": {

View File

@@ -14,22 +14,6 @@ import { InstanceContext } from '../types/instance-context';
import { validateWorkflowStructure } from '../services/n8n-validation';
import { NodeRepository } from '../database/node-repository';
import { WorkflowVersioningService } from '../services/workflow-versioning-service';
import { WorkflowValidator } from '../services/workflow-validator';
import { EnhancedConfigValidator } from '../services/enhanced-config-validator';
// Cached validator instance to avoid recreating on every mutation
let cachedValidator: WorkflowValidator | null = null;
/**
* Get or create cached workflow validator instance
* Reuses the same validator to avoid redundant NodeSimilarityService initialization
*/
function getValidator(repository: NodeRepository): WorkflowValidator {
if (!cachedValidator) {
cachedValidator = new WorkflowValidator(repository, EnhancedConfigValidator);
}
return cachedValidator;
}
// Zod schema for the diff request
const workflowDiffSchema = z.object({
@@ -78,8 +62,6 @@ export async function handleUpdatePartialWorkflow(
const startTime = Date.now();
const sessionId = `mutation_${Date.now()}_${Math.random().toString(36).slice(2, 11)}`;
let workflowBefore: any = null;
let validationBefore: any = null;
let validationAfter: any = null;
try {
// Debug logging (only in debug mode)
@@ -110,24 +92,6 @@ export async function handleUpdatePartialWorkflow(
workflow = await client.getWorkflow(input.id);
// Store original workflow for telemetry
workflowBefore = JSON.parse(JSON.stringify(workflow));
// Validate workflow BEFORE mutation (for telemetry)
try {
const validator = getValidator(repository);
validationBefore = await validator.validateWorkflow(workflowBefore, {
validateNodes: true,
validateConnections: true,
validateExpressions: true,
profile: 'runtime'
});
} catch (validationError) {
logger.debug('Pre-mutation validation failed (non-blocking):', validationError);
// Don't block mutation on validation errors
validationBefore = {
valid: false,
errors: [{ type: 'validation_error', message: 'Validation failed' }]
};
}
} catch (error) {
if (error instanceof N8nApiError) {
return {
@@ -293,24 +257,6 @@ export async function handleUpdatePartialWorkflow(
let finalWorkflow = updatedWorkflow;
let activationMessage = '';
// Validate workflow AFTER mutation (for telemetry)
try {
const validator = getValidator(repository);
validationAfter = await validator.validateWorkflow(finalWorkflow, {
validateNodes: true,
validateConnections: true,
validateExpressions: true,
profile: 'runtime'
});
} catch (validationError) {
logger.debug('Post-mutation validation failed (non-blocking):', validationError);
// Don't block on validation errors
validationAfter = {
valid: false,
errors: [{ type: 'validation_error', message: 'Validation failed' }]
};
}
if (diffResult.shouldActivate) {
try {
finalWorkflow = await client.activateWorkflow(input.id);
@@ -352,8 +298,6 @@ export async function handleUpdatePartialWorkflow(
operations: input.operations,
workflowBefore,
workflowAfter: finalWorkflow,
validationBefore,
validationAfter,
mutationSuccess: true,
durationMs: Date.now() - startTime,
}).catch(err => {
@@ -386,8 +330,6 @@ export async function handleUpdatePartialWorkflow(
operations: input.operations,
workflowBefore,
workflowAfter: workflowBefore, // No change since it failed
validationBefore,
validationAfter: validationBefore, // Same as before since mutation failed
mutationSuccess: false,
mutationError: error instanceof Error ? error.message : 'Unknown error',
durationMs: Date.now() - startTime,
@@ -423,86 +365,11 @@ export async function handleUpdatePartialWorkflow(
}
}
/**
* Infer intent from operations when not explicitly provided
*/
function inferIntentFromOperations(operations: any[]): string {
if (!operations || operations.length === 0) {
return 'Partial workflow update';
}
const opTypes = operations.map((op) => op.type);
const opCount = operations.length;
// Single operation - be specific
if (opCount === 1) {
const op = operations[0];
switch (op.type) {
case 'addNode':
return `Add ${op.node?.type || 'node'}`;
case 'removeNode':
return `Remove node ${op.nodeName || op.nodeId || ''}`.trim();
case 'updateNode':
return `Update node ${op.nodeName || op.nodeId || ''}`.trim();
case 'addConnection':
return `Connect ${op.source || 'node'} to ${op.target || 'node'}`;
case 'removeConnection':
return `Disconnect ${op.source || 'node'} from ${op.target || 'node'}`;
case 'rewireConnection':
return `Rewire ${op.source || 'node'} from ${op.from || ''} to ${op.to || ''}`.trim();
case 'updateName':
return `Rename workflow to "${op.name || ''}"`;
case 'activateWorkflow':
return 'Activate workflow';
case 'deactivateWorkflow':
return 'Deactivate workflow';
default:
return `Workflow ${op.type}`;
}
}
// Multiple operations - summarize pattern
const typeSet = new Set(opTypes);
const summary: string[] = [];
if (typeSet.has('addNode')) {
const count = opTypes.filter((t) => t === 'addNode').length;
summary.push(`add ${count} node${count > 1 ? 's' : ''}`);
}
if (typeSet.has('removeNode')) {
const count = opTypes.filter((t) => t === 'removeNode').length;
summary.push(`remove ${count} node${count > 1 ? 's' : ''}`);
}
if (typeSet.has('updateNode')) {
const count = opTypes.filter((t) => t === 'updateNode').length;
summary.push(`update ${count} node${count > 1 ? 's' : ''}`);
}
if (typeSet.has('addConnection') || typeSet.has('rewireConnection')) {
summary.push('modify connections');
}
if (typeSet.has('updateName') || typeSet.has('updateSettings')) {
summary.push('update metadata');
}
return summary.length > 0
? `Workflow update: ${summary.join(', ')}`
: `Workflow update: ${opCount} operations`;
}
/**
* Track workflow mutation for telemetry
*/
async function trackWorkflowMutation(data: any): Promise<void> {
try {
// Enhance intent if it's missing or generic
if (
!data.userIntent ||
data.userIntent === 'Partial workflow update' ||
data.userIntent.length < 10
) {
data.userIntent = inferIntentFromOperations(data.operations);
}
const { telemetry } = await import('../telemetry/telemetry-manager.js');
await telemetry.trackWorkflowMutation(data);
} catch (error) {

View File

@@ -9,8 +9,7 @@ export const n8nUpdatePartialWorkflowDoc: ToolDocumentation = {
example: 'n8n_update_partial_workflow({id: "wf_123", operations: [{type: "rewireConnection", source: "IF", from: "Old", to: "New", branch: "true"}]})',
performance: 'Fast (50-200ms)',
tips: [
'ALWAYS provide intent parameter describing what you\'re doing (e.g., "Add error handling", "Fix webhook URL", "Connect Slack to error output")',
'DON\'T use generic intent like "update workflow" or "partial update" - be specific about your goal',
'Include intent parameter in every call - helps to return better responses',
'Use rewireConnection to change connection targets',
'Use branch="true"/"false" for IF nodes',
'Use case=N for Switch nodes',
@@ -368,7 +367,7 @@ n8n_update_partial_workflow({
],
performance: 'Very fast - typically 50-200ms. Much faster than full updates as only changes are processed.',
bestPractices: [
'Always include intent parameter with specific description (e.g., "Add error handling to HTTP Request node", "Fix authentication flow", "Connect Slack notification to errors"). Avoid generic phrases like "update workflow" or "partial update"',
'Always include intent parameter - it helps provide better responses',
'Use rewireConnection instead of remove+add for changing targets',
'Use branch="true"/"false" for IF nodes instead of sourceIndex',
'Use case=N for Switch nodes instead of sourceIndex',

View File

@@ -1,151 +0,0 @@
/**
* Test telemetry mutations with enhanced logging
* Verifies that mutations are properly tracked and persisted
*/
import { telemetry } from '../telemetry/telemetry-manager.js';
import { TelemetryConfigManager } from '../telemetry/config-manager.js';
import { logger } from '../utils/logger.js';
async function testMutations() {
console.log('Starting verbose telemetry mutation test...\n');
const configManager = TelemetryConfigManager.getInstance();
console.log('Telemetry config is enabled:', configManager.isEnabled());
console.log('Telemetry config file:', configManager['configPath']);
// Test data with valid workflow structure
const testMutation = {
sessionId: 'test_session_' + Date.now(),
toolName: 'n8n_update_partial_workflow',
userIntent: 'Add a Merge node for data consolidation',
operations: [
{
type: 'addNode',
nodeId: 'Merge1',
node: {
id: 'Merge1',
type: 'n8n-nodes-base.merge',
name: 'Merge',
position: [600, 200],
parameters: {}
}
},
{
type: 'addConnection',
source: 'previous_node',
target: 'Merge1'
}
],
workflowBefore: {
id: 'test-workflow',
name: 'Test Workflow',
active: true,
nodes: [
{
id: 'previous_node',
type: 'n8n-nodes-base.manualTrigger',
name: 'When called',
position: [300, 200],
parameters: {}
}
],
connections: {},
nodeIds: []
},
workflowAfter: {
id: 'test-workflow',
name: 'Test Workflow',
active: true,
nodes: [
{
id: 'previous_node',
type: 'n8n-nodes-base.manualTrigger',
name: 'When called',
position: [300, 200],
parameters: {}
},
{
id: 'Merge1',
type: 'n8n-nodes-base.merge',
name: 'Merge',
position: [600, 200],
parameters: {}
}
],
connections: {
'previous_node': [
{
node: 'Merge1',
type: 'main',
index: 0,
source: 0,
destination: 0
}
]
},
nodeIds: []
},
mutationSuccess: true,
durationMs: 125
};
console.log('\nTest Mutation Data:');
console.log('==================');
console.log(JSON.stringify({
intent: testMutation.userIntent,
tool: testMutation.toolName,
operationCount: testMutation.operations.length,
sessionId: testMutation.sessionId
}, null, 2));
console.log('\n');
// Call trackWorkflowMutation
console.log('Calling telemetry.trackWorkflowMutation...');
try {
await telemetry.trackWorkflowMutation(testMutation);
console.log('✓ trackWorkflowMutation completed successfully\n');
} catch (error) {
console.error('✗ trackWorkflowMutation failed:', error);
console.error('\n');
}
// Check queue size before flush
const metricsBeforeFlush = telemetry.getMetrics();
console.log('Metrics before flush:');
console.log('- mutationQueueSize:', metricsBeforeFlush.tracking.mutationQueueSize);
console.log('- eventsTracked:', metricsBeforeFlush.processing.eventsTracked);
console.log('- eventsFailed:', metricsBeforeFlush.processing.eventsFailed);
console.log('\n');
// Flush telemetry with 10-second wait for Supabase
console.log('Flushing telemetry (waiting 10 seconds for Supabase)...');
try {
await telemetry.flush();
console.log('✓ Telemetry flush completed\n');
} catch (error) {
console.error('✗ Flush failed:', error);
console.error('\n');
}
// Wait a bit for async operations
await new Promise(resolve => setTimeout(resolve, 2000));
// Get final metrics
const metricsAfterFlush = telemetry.getMetrics();
console.log('Metrics after flush:');
console.log('- mutationQueueSize:', metricsAfterFlush.tracking.mutationQueueSize);
console.log('- eventsTracked:', metricsAfterFlush.processing.eventsTracked);
console.log('- eventsFailed:', metricsAfterFlush.processing.eventsFailed);
console.log('- batchesSent:', metricsAfterFlush.processing.batchesSent);
console.log('- batchesFailed:', metricsAfterFlush.processing.batchesFailed);
console.log('- circuitBreakerState:', metricsAfterFlush.processing.circuitBreakerState);
console.log('\n');
console.log('Test completed. Check workflow_mutations table in Supabase.');
}
testMutations().catch(error => {
console.error('Test failed:', error);
process.exit(1);
});

View File

@@ -1,145 +0,0 @@
/**
* Test telemetry mutations
* Verifies that mutations are properly tracked and persisted
*/
import { telemetry } from '../telemetry/telemetry-manager.js';
import { TelemetryConfigManager } from '../telemetry/config-manager.js';
async function testMutations() {
console.log('Starting telemetry mutation test...\n');
const configManager = TelemetryConfigManager.getInstance();
console.log('Telemetry Status:');
console.log('================');
console.log(configManager.getStatus());
console.log('\n');
// Get initial metrics
const metricsAfterInit = telemetry.getMetrics();
console.log('Telemetry Metrics (After Init):');
console.log('================================');
console.log(JSON.stringify(metricsAfterInit, null, 2));
console.log('\n');
// Test data mimicking actual mutation with valid workflow structure
const testMutation = {
sessionId: 'test_session_' + Date.now(),
toolName: 'n8n_update_partial_workflow',
userIntent: 'Add a Merge node for data consolidation',
operations: [
{
type: 'addNode',
nodeId: 'Merge1',
node: {
id: 'Merge1',
type: 'n8n-nodes-base.merge',
name: 'Merge',
position: [600, 200],
parameters: {}
}
},
{
type: 'addConnection',
source: 'previous_node',
target: 'Merge1'
}
],
workflowBefore: {
id: 'test-workflow',
name: 'Test Workflow',
active: true,
nodes: [
{
id: 'previous_node',
type: 'n8n-nodes-base.manualTrigger',
name: 'When called',
position: [300, 200],
parameters: {}
}
],
connections: {},
nodeIds: []
},
workflowAfter: {
id: 'test-workflow',
name: 'Test Workflow',
active: true,
nodes: [
{
id: 'previous_node',
type: 'n8n-nodes-base.manualTrigger',
name: 'When called',
position: [300, 200],
parameters: {}
},
{
id: 'Merge1',
type: 'n8n-nodes-base.merge',
name: 'Merge',
position: [600, 200],
parameters: {}
}
],
connections: {
'previous_node': [
{
node: 'Merge1',
type: 'main',
index: 0,
source: 0,
destination: 0
}
]
},
nodeIds: []
},
mutationSuccess: true,
durationMs: 125
};
console.log('Test Mutation Data:');
console.log('==================');
console.log(JSON.stringify({
intent: testMutation.userIntent,
tool: testMutation.toolName,
operationCount: testMutation.operations.length,
sessionId: testMutation.sessionId
}, null, 2));
console.log('\n');
// Call trackWorkflowMutation
console.log('Calling telemetry.trackWorkflowMutation...');
try {
await telemetry.trackWorkflowMutation(testMutation);
console.log('✓ trackWorkflowMutation completed successfully\n');
} catch (error) {
console.error('✗ trackWorkflowMutation failed:', error);
console.error('\n');
}
// Flush telemetry
console.log('Flushing telemetry...');
try {
await telemetry.flush();
console.log('✓ Telemetry flushed successfully\n');
} catch (error) {
console.error('✗ Flush failed:', error);
console.error('\n');
}
// Get final metrics
const metricsAfterFlush = telemetry.getMetrics();
console.log('Telemetry Metrics (After Flush):');
console.log('==================================');
console.log(JSON.stringify(metricsAfterFlush, null, 2));
console.log('\n');
console.log('Test completed. Check workflow_mutations table in Supabase.');
}
testMutations().catch(error => {
console.error('Test failed:', error);
process.exit(1);
});

View File

@@ -41,8 +41,8 @@ export class MutationTracker {
}
// Sanitize workflows to remove credentials and sensitive data
const workflowBefore = WorkflowSanitizer.sanitizeWorkflowRaw(data.workflowBefore);
const workflowAfter = WorkflowSanitizer.sanitizeWorkflowRaw(data.workflowAfter);
const workflowBefore = this.sanitizeFullWorkflow(data.workflowBefore);
const workflowAfter = this.sanitizeFullWorkflow(data.workflowAfter);
// Sanitize user intent
const sanitizedIntent = intentSanitizer.sanitize(data.userIntent);
@@ -200,6 +200,98 @@ export class MutationTracker {
return metrics;
}
/**
* Sanitize a full workflow while preserving structure
* Removes credentials and sensitive data but keeps all nodes, connections, parameters
*/
private sanitizeFullWorkflow(workflow: any): any {
if (!workflow) return workflow;
// Deep clone to avoid modifying original
const sanitized = JSON.parse(JSON.stringify(workflow));
// Remove sensitive workflow-level fields
delete sanitized.credentials;
delete sanitized.sharedWorkflows;
delete sanitized.ownedBy;
delete sanitized.createdBy;
delete sanitized.updatedBy;
// Sanitize each node
if (sanitized.nodes && Array.isArray(sanitized.nodes)) {
sanitized.nodes = sanitized.nodes.map((node: any) => {
const sanitizedNode = { ...node };
// Remove credentials field
delete sanitizedNode.credentials;
// Sanitize parameters if present
if (sanitizedNode.parameters && typeof sanitizedNode.parameters === 'object') {
sanitizedNode.parameters = this.sanitizeParameters(sanitizedNode.parameters);
}
return sanitizedNode;
});
}
return sanitized;
}
/**
* Recursively sanitize parameters object
*/
private sanitizeParameters(params: any): any {
if (!params || typeof params !== 'object') return params;
const sensitiveKeys = [
'apiKey', 'api_key', 'token', 'secret', 'password', 'credential',
'auth', 'authorization', 'privateKey', 'accessToken', 'refreshToken'
];
const sanitized: any = Array.isArray(params) ? [] : {};
for (const [key, value] of Object.entries(params)) {
const lowerKey = key.toLowerCase();
// Check if key is sensitive
if (sensitiveKeys.some(sk => lowerKey.includes(sk.toLowerCase()))) {
sanitized[key] = '[REDACTED]';
} else if (typeof value === 'object' && value !== null) {
// Recursively sanitize nested objects
sanitized[key] = this.sanitizeParameters(value);
} else if (typeof value === 'string') {
// Sanitize string values that might contain sensitive data
sanitized[key] = this.sanitizeStringValue(value);
} else {
sanitized[key] = value;
}
}
return sanitized;
}
/**
* Sanitize string values that might contain sensitive data
*/
private sanitizeStringValue(value: string): string {
if (!value || typeof value !== 'string') return value;
let sanitized = value;
// Redact URLs with authentication
sanitized = sanitized.replace(/https?:\/\/[^:]+:[^@]+@[^\s/]+/g, '[REDACTED_URL_WITH_AUTH]');
// Redact long API keys/tokens (20+ alphanumeric chars)
sanitized = sanitized.replace(/\b[A-Za-z0-9_-]{32,}\b/g, '[REDACTED_TOKEN]');
// Redact OpenAI-style keys
sanitized = sanitized.replace(/\bsk-[A-Za-z0-9]{32,}\b/g, '[REDACTED_APIKEY]');
// Redact Bearer tokens
sanitized = sanitized.replace(/Bearer\s+[^\s]+/gi, 'Bearer [REDACTED]');
return sanitized;
}
/**
* Calculate validation improvement metrics

View File

@@ -27,32 +27,29 @@ interface SanitizedWorkflow {
workflowHash: string;
}
interface PatternDefinition {
pattern: RegExp;
placeholder: string;
preservePrefix?: boolean; // For patterns like "Bearer [REDACTED]"
}
export class WorkflowSanitizer {
private static readonly SENSITIVE_PATTERNS: PatternDefinition[] = [
private static readonly SENSITIVE_PATTERNS = [
// Webhook URLs (replace with placeholder but keep structure) - MUST BE FIRST
{ pattern: /https?:\/\/[^\s/]+\/webhook\/[^\s]+/g, placeholder: '[REDACTED_WEBHOOK]' },
{ pattern: /https?:\/\/[^\s/]+\/hook\/[^\s]+/g, placeholder: '[REDACTED_WEBHOOK]' },
/https?:\/\/[^\s/]+\/webhook\/[^\s]+/g,
/https?:\/\/[^\s/]+\/hook\/[^\s]+/g,
// URLs with authentication - MUST BE BEFORE BEARER TOKENS
{ pattern: /https?:\/\/[^:]+:[^@]+@[^\s/]+/g, placeholder: '[REDACTED_URL_WITH_AUTH]' },
{ pattern: /wss?:\/\/[^:]+:[^@]+@[^\s/]+/g, placeholder: '[REDACTED_URL_WITH_AUTH]' },
{ pattern: /(?:postgres|mysql|mongodb|redis):\/\/[^:]+:[^@]+@[^\s]+/g, placeholder: '[REDACTED_URL_WITH_AUTH]' }, // Database protocols - includes port and path
// API keys and tokens
/sk-[a-zA-Z0-9]{16,}/g, // OpenAI keys
/Bearer\s+[^\s]+/gi, // Bearer tokens
/[a-zA-Z0-9_-]{20,}/g, // Long alphanumeric strings (API keys) - reduced threshold
/token['":\s]+[^,}]+/gi, // Token fields
/apikey['":\s]+[^,}]+/gi, // API key fields
/api_key['":\s]+[^,}]+/gi,
/secret['":\s]+[^,}]+/gi,
/password['":\s]+[^,}]+/gi,
/credential['":\s]+[^,}]+/gi,
// API keys and tokens - ORDER MATTERS!
// More specific patterns first, then general patterns
{ pattern: /sk-[a-zA-Z0-9]{16,}/g, placeholder: '[REDACTED_APIKEY]' }, // OpenAI keys
{ pattern: /Bearer\s+[^\s]+/gi, placeholder: 'Bearer [REDACTED]', preservePrefix: true }, // Bearer tokens
{ pattern: /\b[a-zA-Z0-9_-]{32,}\b/g, placeholder: '[REDACTED_TOKEN]' }, // Long tokens (32+ chars)
{ pattern: /\b[a-zA-Z0-9_-]{20,31}\b/g, placeholder: '[REDACTED]' }, // Short tokens (20-31 chars)
// URLs with authentication
/https?:\/\/[^:]+:[^@]+@[^\s/]+/g, // URLs with auth
/wss?:\/\/[^:]+:[^@]+@[^\s/]+/g,
// Email addresses (optional - uncomment if needed)
// { pattern: /[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g, placeholder: '[REDACTED_EMAIL]' },
// /[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g,
];
private static readonly SENSITIVE_FIELDS = [
@@ -181,34 +178,19 @@ export class WorkflowSanitizer {
const sanitized: any = {};
for (const [key, value] of Object.entries(obj)) {
// Check if field name is sensitive
const isSensitive = this.isSensitiveField(key);
const isUrlField = key.toLowerCase().includes('url') ||
key.toLowerCase().includes('endpoint') ||
key.toLowerCase().includes('webhook');
// Check if key is sensitive
if (this.isSensitiveField(key)) {
sanitized[key] = '[REDACTED]';
continue;
}
// Recursively sanitize nested objects (unless it's a sensitive non-URL field)
// Recursively sanitize nested objects
if (typeof value === 'object' && value !== null) {
if (isSensitive && !isUrlField) {
// For sensitive object fields (like 'authentication'), redact completely
sanitized[key] = '[REDACTED]';
} else {
sanitized[key] = this.sanitizeObject(value);
}
sanitized[key] = this.sanitizeObject(value);
}
// Sanitize string values
else if (typeof value === 'string') {
// For sensitive fields (except URL fields), use generic redaction
if (isSensitive && !isUrlField) {
sanitized[key] = '[REDACTED]';
} else {
// For URL fields or non-sensitive fields, use pattern-specific sanitization
sanitized[key] = this.sanitizeString(value, key);
}
}
// For non-string sensitive fields, redact completely
else if (isSensitive) {
sanitized[key] = '[REDACTED]';
sanitized[key] = this.sanitizeString(value, key);
}
// Keep other types as-is
else {
@@ -230,42 +212,13 @@ export class WorkflowSanitizer {
let sanitized = value;
// Apply all sensitive patterns with their specific placeholders
for (const patternDef of this.SENSITIVE_PATTERNS) {
// Apply all sensitive patterns
for (const pattern of this.SENSITIVE_PATTERNS) {
// Skip webhook patterns - already handled above
if (patternDef.placeholder.includes('WEBHOOK')) {
if (pattern.toString().includes('webhook')) {
continue;
}
// Skip if already sanitized with a placeholder to prevent double-redaction
if (sanitized.includes('[REDACTED')) {
break;
}
// Special handling for URL with auth - preserve path after credentials
if (patternDef.placeholder === '[REDACTED_URL_WITH_AUTH]') {
const matches = value.match(patternDef.pattern);
if (matches) {
for (const match of matches) {
// Extract path after the authenticated URL
const fullUrlMatch = value.indexOf(match);
if (fullUrlMatch !== -1) {
const afterUrl = value.substring(fullUrlMatch + match.length);
// If there's a path after the URL, preserve it
if (afterUrl && afterUrl.startsWith('/')) {
const pathPart = afterUrl.split(/[\s?&#]/)[0]; // Get path until query/fragment
sanitized = sanitized.replace(match + pathPart, patternDef.placeholder + pathPart);
} else {
sanitized = sanitized.replace(match, patternDef.placeholder);
}
}
}
}
continue;
}
// Apply pattern with its specific placeholder
sanitized = sanitized.replace(patternDef.pattern, patternDef.placeholder);
sanitized = sanitized.replace(pattern, '[REDACTED]');
}
// Additional sanitization for specific field types
@@ -273,13 +226,9 @@ export class WorkflowSanitizer {
fieldName.toLowerCase().includes('endpoint')) {
// Keep URL structure but remove domain details
if (sanitized.startsWith('http://') || sanitized.startsWith('https://')) {
// If value has been redacted with URL_WITH_AUTH, preserve it
if (sanitized.includes('[REDACTED_URL_WITH_AUTH]')) {
return sanitized; // Already properly sanitized with path preserved
}
// If value has other redactions, leave it as is
// If value has been redacted, leave it as is
if (sanitized.includes('[REDACTED]')) {
return sanitized;
return '[REDACTED]';
}
const urlParts = sanitized.split('/');
if (urlParts.length > 2) {
@@ -347,37 +296,4 @@ export class WorkflowSanitizer {
const sanitized = this.sanitizeWorkflow(workflow);
return sanitized.workflowHash;
}
/**
* Sanitize workflow and return raw workflow object (without metrics)
* For use in telemetry where we need plain workflow structure
*/
static sanitizeWorkflowRaw(workflow: any): any {
// Create a deep copy to avoid modifying original
const sanitized = JSON.parse(JSON.stringify(workflow));
// Sanitize nodes
if (sanitized.nodes && Array.isArray(sanitized.nodes)) {
sanitized.nodes = sanitized.nodes.map((node: WorkflowNode) =>
this.sanitizeNode(node)
);
}
// Sanitize connections (keep structure only)
if (sanitized.connections) {
sanitized.connections = this.sanitizeConnections(sanitized.connections);
}
// Remove other potentially sensitive data
delete sanitized.settings?.errorWorkflow;
delete sanitized.staticData;
delete sanitized.pinData;
delete sanitized.credentials;
delete sanitized.sharedWorkflows;
delete sanitized.ownedBy;
delete sanitized.createdBy;
delete sanitized.updatedBy;
return sanitized;
}
}

View File

@@ -49,7 +49,7 @@ describe('WorkflowSanitizer', () => {
const sanitized = WorkflowSanitizer.sanitizeWorkflow(workflow);
expect(sanitized.nodes[0].parameters.webhookUrl).toBe('https://[webhook-url]');
expect(sanitized.nodes[0].parameters.webhookUrl).toBe('[REDACTED]');
expect(sanitized.nodes[0].parameters.method).toBe('POST'); // Method should remain
expect(sanitized.nodes[0].parameters.path).toBe('my-webhook'); // Path should remain
});
@@ -104,9 +104,9 @@ describe('WorkflowSanitizer', () => {
const sanitized = WorkflowSanitizer.sanitizeWorkflow(workflow);
expect(sanitized.nodes[0].parameters.url).toBe('https://[domain]/endpoint');
expect(sanitized.nodes[0].parameters.endpoint).toBe('https://[domain]/api');
expect(sanitized.nodes[0].parameters.baseUrl).toBe('https://[domain]');
expect(sanitized.nodes[0].parameters.url).toBe('[REDACTED]');
expect(sanitized.nodes[0].parameters.endpoint).toBe('[REDACTED]');
expect(sanitized.nodes[0].parameters.baseUrl).toBe('[REDACTED]');
});
it('should calculate workflow metrics correctly', () => {
@@ -480,8 +480,8 @@ describe('WorkflowSanitizer', () => {
expect(params.secret_token).toBe('[REDACTED]');
expect(params.authKey).toBe('[REDACTED]');
expect(params.clientSecret).toBe('[REDACTED]');
expect(params.webhookUrl).toBe('https://hooks.example.com/services/T00000000/B00000000/[REDACTED]');
expect(params.databaseUrl).toBe('[REDACTED_URL_WITH_AUTH]');
expect(params.webhookUrl).toBe('[REDACTED]');
expect(params.databaseUrl).toBe('[REDACTED]');
expect(params.connectionString).toBe('[REDACTED]');
// Safe values should remain
@@ -515,9 +515,9 @@ describe('WorkflowSanitizer', () => {
const sanitized = WorkflowSanitizer.sanitizeWorkflow(workflow);
const headers = sanitized.nodes[0].parameters.headers;
expect(headers[0].value).toBe('Bearer [REDACTED]'); // Authorization (Bearer prefix preserved)
expect(headers[0].value).toBe('[REDACTED]'); // Authorization
expect(headers[1].value).toBe('application/json'); // Content-Type (safe)
expect(headers[2].value).toBe('[REDACTED_TOKEN]'); // X-API-Key (32+ chars)
expect(headers[2].value).toBe('[REDACTED]'); // X-API-Key
expect(sanitized.nodes[0].parameters.methods).toEqual(['GET', 'POST']); // Array should remain
});