feat: add agent schema validation with comprehensive testing (#774)
Introduce automated validation for agent YAML files using Zod to ensure
schema compliance across all agent definitions. This feature validates
17 agent files across core and module directories, catching structural
errors and maintaining consistency.
Schema Validation (tools/schema/agent.js):
- Zod-based schema validating metadata, persona, menu, prompts, and critical actions
- Module-aware validation: module field required for src/modules/**/agents/,
optional for src/core/agents/
- Enforces kebab-case unique triggers and at least one command target per menu item
- Validates persona.principles as array (not string)
- Comprehensive refinements for data integrity
CLI Validator (tools/validate-agent-schema.js):
- Scans src/{core,modules/*}/agents/*.agent.yaml
- Parses with js-yaml and validates using Zod schema
- Reports detailed errors with file paths and field paths
- Exits 1 on failures, 0 on success
- Accepts optional project_root parameter for testing
Testing (679 lines across 3 test files):
- test/test-cli-integration.sh: CLI behavior and error handling tests
- test/unit-test-schema.js: Direct schema validation unit tests
- test/test-agent-schema.js: Comprehensive fixture-based tests
- 50 test fixtures covering valid and invalid scenarios
- ESLint configured to support CommonJS test files
- Prettier configured to ignore intentionally broken fixtures
CI Integration (.github/workflows/lint.yaml):
- Renamed from format-check.yaml to lint.yaml
- Added schema-validation job running npm run validate:schemas
- Runs in parallel with prettier and eslint jobs
- Validates on all pull requests
Data Cleanup:
- Fixed src/core/agents/bmad-master.agent.yaml: converted persona.principles
from string to array format
Documentation:
- Updated schema-classification.md with validation section
- Documents validator usage, enforcement rules, and CI integration
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
23
test/fixtures/agent-schema/valid/critical-actions/empty-critical-actions.agent.yaml
vendored
Normal file
23
test/fixtures/agent-schema/valid/critical-actions/empty-critical-actions.agent.yaml
vendored
Normal file
@@ -0,0 +1,23 @@
|
||||
# Test: Empty critical_actions array
|
||||
# Expected: PASS - empty array is valid for optional field
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: empty-critical-actions
|
||||
name: Empty Critical Actions
|
||||
title: Empty Critical Actions
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent with empty critical actions
|
||||
identity: I am a test agent with empty critical actions array.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test empty arrays
|
||||
|
||||
critical_actions: []
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
21
test/fixtures/agent-schema/valid/critical-actions/no-critical-actions.agent.yaml
vendored
Normal file
21
test/fixtures/agent-schema/valid/critical-actions/no-critical-actions.agent.yaml
vendored
Normal file
@@ -0,0 +1,21 @@
|
||||
# Test: No critical_actions field (optional)
|
||||
# Expected: PASS
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: no-critical-actions
|
||||
name: No Critical Actions
|
||||
title: No Critical Actions
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent without critical actions
|
||||
identity: I am a test agent without critical actions.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test optional fields
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
26
test/fixtures/agent-schema/valid/critical-actions/valid-critical-actions.agent.yaml
vendored
Normal file
26
test/fixtures/agent-schema/valid/critical-actions/valid-critical-actions.agent.yaml
vendored
Normal file
@@ -0,0 +1,26 @@
|
||||
# Test: critical_actions with valid strings
|
||||
# Expected: PASS
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: valid-critical-actions
|
||||
name: Valid Critical Actions
|
||||
title: Valid Critical Actions
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent with critical actions
|
||||
identity: I am a test agent with valid critical actions.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test valid arrays
|
||||
|
||||
critical_actions:
|
||||
- Load configuration from disk
|
||||
- Initialize user context
|
||||
- Set communication preferences
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
39
test/fixtures/agent-schema/valid/menu-commands/all-command-types.agent.yaml
vendored
Normal file
39
test/fixtures/agent-schema/valid/menu-commands/all-command-types.agent.yaml
vendored
Normal file
@@ -0,0 +1,39 @@
|
||||
# Test: Menu items with all valid command target types
|
||||
# Expected: PASS
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: all-commands
|
||||
name: All Command Types
|
||||
title: All Commands
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent with all command types
|
||||
identity: I test all available command target types.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test all command types
|
||||
|
||||
menu:
|
||||
- trigger: workflow-test
|
||||
description: Test workflow command
|
||||
workflow: path/to/workflow
|
||||
- trigger: validate-test
|
||||
description: Test validate-workflow command
|
||||
validate-workflow: path/to/validation
|
||||
- trigger: exec-test
|
||||
description: Test exec command
|
||||
exec: npm test
|
||||
- trigger: action-test
|
||||
description: Test action command
|
||||
action: perform_action
|
||||
- trigger: tmpl-test
|
||||
description: Test tmpl command
|
||||
tmpl: path/to/template
|
||||
- trigger: data-test
|
||||
description: Test data command
|
||||
data: path/to/data
|
||||
- trigger: run-workflow-test
|
||||
description: Test run-workflow command
|
||||
run-workflow: path/to/workflow
|
||||
23
test/fixtures/agent-schema/valid/menu-commands/multiple-commands.agent.yaml
vendored
Normal file
23
test/fixtures/agent-schema/valid/menu-commands/multiple-commands.agent.yaml
vendored
Normal file
@@ -0,0 +1,23 @@
|
||||
# Test: Menu item with multiple command targets
|
||||
# Expected: PASS - multiple targets are allowed
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: multiple-commands
|
||||
name: Multiple Commands
|
||||
title: Multiple Commands
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent with multiple command targets
|
||||
identity: I test multiple command targets per menu item.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test multiple targets
|
||||
|
||||
menu:
|
||||
- trigger: multi-command
|
||||
description: Menu item with multiple command targets
|
||||
workflow: path/to/workflow
|
||||
exec: npm test
|
||||
action: perform_action
|
||||
33
test/fixtures/agent-schema/valid/menu-triggers/kebab-case-triggers.agent.yaml
vendored
Normal file
33
test/fixtures/agent-schema/valid/menu-triggers/kebab-case-triggers.agent.yaml
vendored
Normal file
@@ -0,0 +1,33 @@
|
||||
# Test: Valid kebab-case triggers
|
||||
# Expected: PASS
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: kebab-triggers
|
||||
name: Kebab Case Triggers
|
||||
title: Kebab Triggers
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent with kebab-case triggers
|
||||
identity: I test kebab-case trigger validation.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test kebab-case format
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Single word trigger
|
||||
action: display_help
|
||||
- trigger: list-tasks
|
||||
description: Two word trigger
|
||||
action: list_tasks
|
||||
- trigger: workflow-init-process
|
||||
description: Three word trigger
|
||||
action: init_workflow
|
||||
- trigger: test123
|
||||
description: Trigger with numbers
|
||||
action: test
|
||||
- trigger: multi-word-kebab-case-trigger
|
||||
description: Long kebab-case trigger
|
||||
action: long_action
|
||||
30
test/fixtures/agent-schema/valid/menu/multiple-menu-items.agent.yaml
vendored
Normal file
30
test/fixtures/agent-schema/valid/menu/multiple-menu-items.agent.yaml
vendored
Normal file
@@ -0,0 +1,30 @@
|
||||
# Test: Menu with multiple valid items using different command types
|
||||
# Expected: PASS
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: multiple-menu
|
||||
name: Multiple Menu Items
|
||||
title: Multiple Menu
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent with multiple menu items
|
||||
identity: I am a test agent with diverse menu commands.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test multiple menu items
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
- trigger: start-workflow
|
||||
description: Start a workflow
|
||||
workflow: path/to/workflow
|
||||
- trigger: execute
|
||||
description: Execute command
|
||||
exec: npm test
|
||||
- trigger: use-template
|
||||
description: Use template
|
||||
tmpl: path/to/template
|
||||
21
test/fixtures/agent-schema/valid/menu/single-menu-item.agent.yaml
vendored
Normal file
21
test/fixtures/agent-schema/valid/menu/single-menu-item.agent.yaml
vendored
Normal file
@@ -0,0 +1,21 @@
|
||||
# Test: Menu with single valid item
|
||||
# Expected: PASS
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: single-menu
|
||||
name: Single Menu Item
|
||||
title: Single Menu
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent with single menu item
|
||||
identity: I am a test agent.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test minimal menu
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help information
|
||||
action: display_help
|
||||
23
test/fixtures/agent-schema/valid/metadata/empty-module-name-in-path.agent.yaml
vendored
Normal file
23
test/fixtures/agent-schema/valid/metadata/empty-module-name-in-path.agent.yaml
vendored
Normal file
@@ -0,0 +1,23 @@
|
||||
# Test: Empty module name in path (src/modules//agents/)
|
||||
# Expected: PASS - treated as core agent (empty module normalizes to null)
|
||||
# Path context: src/modules//agents/test.agent.yaml
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: empty-module-path
|
||||
name: Empty Module in Path
|
||||
title: Empty Module Path
|
||||
icon: 🧪
|
||||
# No module field - path has empty module name, treated as core
|
||||
|
||||
persona:
|
||||
role: Test agent for empty module name in path
|
||||
identity: I test the edge case where module name in path is empty.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test path parsing edge cases
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
23
test/fixtures/agent-schema/valid/metadata/malformed-path-treated-as-core.agent.yaml
vendored
Normal file
23
test/fixtures/agent-schema/valid/metadata/malformed-path-treated-as-core.agent.yaml
vendored
Normal file
@@ -0,0 +1,23 @@
|
||||
# Test: Malformed module path (no slash after module name) treated as core
|
||||
# Expected: PASS - malformed path returns null, treated as core agent
|
||||
# Path context: src/modules/bmm
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: malformed-path
|
||||
name: Malformed Path Test
|
||||
title: Malformed Path
|
||||
icon: 🧪
|
||||
# No module field - will be treated as core since path parsing returns null
|
||||
|
||||
persona:
|
||||
role: Test agent for malformed path edge case
|
||||
identity: I test edge cases in path parsing.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test edge case handling
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
23
test/fixtures/agent-schema/valid/metadata/module-agent-correct.agent.yaml
vendored
Normal file
23
test/fixtures/agent-schema/valid/metadata/module-agent-correct.agent.yaml
vendored
Normal file
@@ -0,0 +1,23 @@
|
||||
# Test: Valid module agent with correct module field
|
||||
# Expected: PASS
|
||||
# Path context: src/modules/bmm/agents/module-agent-correct.agent.yaml
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: bmm-test
|
||||
name: BMM Test Agent
|
||||
title: BMM Test
|
||||
icon: 🧪
|
||||
module: bmm
|
||||
|
||||
persona:
|
||||
role: Test module agent
|
||||
identity: I am a module-scoped test agent.
|
||||
communication_style: Professional
|
||||
principles:
|
||||
- Test module validation
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
23
test/fixtures/agent-schema/valid/persona/complete-persona.agent.yaml
vendored
Normal file
23
test/fixtures/agent-schema/valid/persona/complete-persona.agent.yaml
vendored
Normal file
@@ -0,0 +1,23 @@
|
||||
# Test: All persona fields properly filled
|
||||
# Expected: PASS
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: complete-persona
|
||||
name: Complete Persona Agent
|
||||
title: Complete Persona
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Comprehensive test agent with all persona fields
|
||||
identity: I am a test agent designed to validate complete persona structure with multiple characteristics and attributes.
|
||||
communication_style: Professional, clear, and thorough with attention to detail
|
||||
principles:
|
||||
- Validate all persona fields are present
|
||||
- Ensure array fields work correctly
|
||||
- Test comprehensive documentation
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
23
test/fixtures/agent-schema/valid/prompts/empty-prompts.agent.yaml
vendored
Normal file
23
test/fixtures/agent-schema/valid/prompts/empty-prompts.agent.yaml
vendored
Normal file
@@ -0,0 +1,23 @@
|
||||
# Test: Empty prompts array
|
||||
# Expected: PASS - empty array valid for optional field
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: empty-prompts
|
||||
name: Empty Prompts
|
||||
title: Empty Prompts
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent with empty prompts
|
||||
identity: I am a test agent with empty prompts array.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test empty arrays
|
||||
|
||||
prompts: []
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
21
test/fixtures/agent-schema/valid/prompts/no-prompts.agent.yaml
vendored
Normal file
21
test/fixtures/agent-schema/valid/prompts/no-prompts.agent.yaml
vendored
Normal file
@@ -0,0 +1,21 @@
|
||||
# Test: No prompts field (optional)
|
||||
# Expected: PASS
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: no-prompts
|
||||
name: No Prompts
|
||||
title: No Prompts
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent without prompts
|
||||
identity: I am a test agent without prompts field.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test optional fields
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
27
test/fixtures/agent-schema/valid/prompts/valid-prompts-minimal.agent.yaml
vendored
Normal file
27
test/fixtures/agent-schema/valid/prompts/valid-prompts-minimal.agent.yaml
vendored
Normal file
@@ -0,0 +1,27 @@
|
||||
# Test: Prompts with required id and content only
|
||||
# Expected: PASS
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: valid-prompts-minimal
|
||||
name: Valid Prompts Minimal
|
||||
title: Valid Prompts
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent with minimal prompts
|
||||
identity: I am a test agent with minimal prompt structure.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test minimal prompts
|
||||
|
||||
prompts:
|
||||
- id: prompt1
|
||||
content: This is a valid prompt content
|
||||
- id: prompt2
|
||||
content: Another valid prompt
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
29
test/fixtures/agent-schema/valid/prompts/valid-prompts-with-description.agent.yaml
vendored
Normal file
29
test/fixtures/agent-schema/valid/prompts/valid-prompts-with-description.agent.yaml
vendored
Normal file
@@ -0,0 +1,29 @@
|
||||
# Test: Prompts with optional description field
|
||||
# Expected: PASS
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: valid-prompts-description
|
||||
name: Valid Prompts With Description
|
||||
title: Valid Prompts Desc
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent with prompts including descriptions
|
||||
identity: I am a test agent with complete prompt structure.
|
||||
communication_style: Clear
|
||||
principles:
|
||||
- Test complete prompts
|
||||
|
||||
prompts:
|
||||
- id: prompt1
|
||||
content: This is a valid prompt content
|
||||
description: This prompt does something useful
|
||||
- id: prompt2
|
||||
content: Another valid prompt
|
||||
description: This prompt does something else
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
23
test/fixtures/agent-schema/valid/top-level/minimal-core-agent.agent.yaml
vendored
Normal file
23
test/fixtures/agent-schema/valid/top-level/minimal-core-agent.agent.yaml
vendored
Normal file
@@ -0,0 +1,23 @@
|
||||
# Test: Valid core agent with only required fields
|
||||
# Expected: PASS
|
||||
# Path context: src/core/agents/minimal-core-agent.agent.yaml
|
||||
|
||||
agent:
|
||||
metadata:
|
||||
id: minimal-test
|
||||
name: Minimal Test Agent
|
||||
title: Minimal Test
|
||||
icon: 🧪
|
||||
|
||||
persona:
|
||||
role: Test agent with minimal configuration
|
||||
identity: I am a minimal test agent used for schema validation testing.
|
||||
communication_style: Clear and concise
|
||||
principles:
|
||||
- Validate schema requirements
|
||||
- Demonstrate minimal valid structure
|
||||
|
||||
menu:
|
||||
- trigger: help
|
||||
description: Show help
|
||||
action: display_help
|
||||
Reference in New Issue
Block a user