fix: add structural hash tracking for workflow mutations (#422)

* feat: add structural hashes and success tracking for workflow mutations

Enables cross-referencing workflow_mutations with telemetry_workflows by adding structural hashes (nodeTypes + connections) alongside existing full hashes.

**Database Changes:**
- Added workflow_structure_hash_before/after columns
- Added is_truly_successful computed column
- Created 3 analytics views: successful_mutations, mutation_training_data, mutations_with_workflow_quality
- Created 2 helper functions: get_mutation_success_rate_by_intent(), get_mutation_crossref_stats()

**Code Changes:**
- Updated mutation-tracker.ts to generate both hash types
- Updated mutation-types.ts with new fields
- Auto-converts to snake_case via existing toSnakeCase() function

**Testing:**
- Added 5 new unit tests for structural hash generation
- All 17 tests passing

**Tooling:**
- Created backfill script to populate hashes for existing 1,499 mutations
- Created comprehensive documentation (STRUCTURAL_HASHES.md)

**Impact:**
- Before: 0% cross-reference match rate
- After: Expected 60-70% match rate (post-backfill)
- Unlocks quality impact analysis, training data curation, and mutation pattern insights

Conceived by Romuald Członkowski - www.aiadvisors.pl/en

* fix: correct test operation types for structural hash tests

Fixed TypeScript errors in mutation-tracker tests by adding required
'updates' parameter to updateNode operations. Used 'as any' for test
operations to maintain backward compatibility while tests are updated.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en

* chore: remove documentation files from tracking

Removed internal documentation files from version control:
- Telemetry implementation docs
- Implementation roadmap
- Disabled tools analysis docs

These files are for internal reference only.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en

* chore: remove telemetry documentation files from tracking

Removed all telemetry analysis and documentation files from root directory.
These files are for internal reference only and should not be in version control.

Files removed:
- TELEMETRY_ANALYSIS*.md
- TELEMETRY_MUTATION_SPEC.md
- TELEMETRY_*_DATASET.md
- VALIDATION_ANALYSIS*.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en

* chore: bump version to 2.22.18 and update CHANGELOG

Version 2.22.18 adds structural hash tracking for workflow mutations,
enabling cross-referencing with workflow quality data and automated
success detection.

Key changes:
- Added workflowStructureHashBefore/After fields
- Added isTrulySuccessful computed field
- Enhanced mutation tracking with structural hashes
- All tests passing (17/17)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en

* chore: remove migration and documentation files from PR

Removed internal database migration files and documentation from
version control:
- docs/migrations/
- docs/telemetry/

Updated CHANGELOG to remove database migration references.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en
This commit is contained in:
Romuald Członkowski
2025-11-14 13:57:54 +01:00
committed by GitHub
parent 597bd290b6
commit 1bbfaabbc2
23 changed files with 484 additions and 7900 deletions

View File

@@ -7,6 +7,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [2.22.18] - 2025-11-14
### ✨ Features
**Structural Hash Tracking for Workflow Mutations**
Added structural hash tracking to enable cross-referencing between workflow mutations and workflow quality data:
#### Structural Hash Generation
- Added `workflowStructureHashBefore` and `workflowStructureHashAfter` fields to mutation records
- Hashes based on node types + connections (structural elements only)
- Compatible with `telemetry_workflows.workflow_hash` format for cross-referencing
- Implementation: Uses `WorkflowSanitizer.generateWorkflowHash()` for consistency
- Enables linking mutation impact to workflow quality scores and grades
#### Success Tracking Enhancement
- Added `isTrulySuccessful` computed field to mutation records
- Definition: Mutation executed successfully AND improved/maintained validation AND has known intent
- Enables filtering to high-quality mutation data
- Provides automated success detection without manual review
#### Testing & Verification
- All 17 mutation-tracker unit tests passing
- Verified with live mutations: structural changes detected (hash changes), config-only updates detected (hash stays same)
- Success tracking working accurately (64% truly successful rate in testing)
**Files Modified**:
- `src/telemetry/mutation-tracker.ts`: Generate structural hashes during mutation processing
- `src/telemetry/mutation-types.ts`: Add new fields to WorkflowMutationRecord interface
- `src/telemetry/workflow-sanitizer.ts`: Expose generateWorkflowHash() method
- `tests/unit/telemetry/mutation-tracker.test.ts`: Add 5 new test cases
**Impact**:
- Enables cross-referencing between mutation and workflow data
- Provides labeled dataset with quality indicators
- Maintains backward compatibility (new fields optional)
Conceived by Romuald Członkowski - https://www.aiadvisors.pl/en
## [2.22.17] - 2025-11-13
### 🐛 Bug Fixes