diff --git a/bmad/bmb/workflows/audit-workflow/checklist.md b/bmad/bmb/workflows/audit-workflow/checklist.md new file mode 100644 index 00000000..8a7b6229 --- /dev/null +++ b/bmad/bmb/workflows/audit-workflow/checklist.md @@ -0,0 +1,136 @@ +# Audit Workflow - Validation Checklist + +## Structure + +- [ ] workflow.yaml file loads without YAML syntax errors +- [ ] instructions.md file exists and is properly formatted +- [ ] template.md file exists (if document workflow) with valid markdown +- [ ] All critical headers present in instructions (workflow engine reference, workflow.yaml reference) +- [ ] Workflow type correctly identified (document/action/interactive/autonomous/meta) +- [ ] All referenced files actually exist at specified paths +- [ ] No placeholder text remains (like {TITLE}, {WORKFLOW_CODE}, TODO, etc.) + +## Standard Config Block + +- [ ] workflow.yaml contains `config_source` pointing to correct module config +- [ ] `output_folder` pulls from `{config_source}:output_folder` +- [ ] `user_name` pulls from `{config_source}:user_name` +- [ ] `communication_language` pulls from `{config_source}:communication_language` +- [ ] `date` is set to `system-generated` +- [ ] Config source uses {project-root} variable (not hardcoded path) +- [ ] Standard config comment present: "Critical variables from config" + +## Config Variable Usage + +- [ ] Instructions communicate in {communication_language} where appropriate +- [ ] Instructions address {user_name} in greetings or summaries where appropriate +- [ ] All file outputs write to {output_folder} or subdirectories (no hardcoded paths) +- [ ] Template includes {{user_name}} in metadata (optional for document workflows) +- [ ] Template includes {{date}} in metadata (optional for document workflows) +- [ ] Template does NOT use {{communication_language}} in headers (agent-only variable) +- [ ] No hardcoded language-specific text that should use {communication_language} +- [ ] Date used for agent date awareness (not confused with training cutoff) + +## YAML/Instruction/Template Alignment + +- [ ] Every workflow.yaml variable (excluding standard config) is used in instructions OR template +- [ ] No unused yaml fields present (bloat removed) +- [ ] No duplicate fields between top-level and web_bundle section +- [ ] All template variables ({{variable}}) have corresponding yaml definitions OR tags +- [ ] All tags have corresponding template variables (if document workflow) +- [ ] Template variables use snake_case naming convention +- [ ] Variable names are descriptive (not abbreviated like {{puj}} instead of {{primary_user_journey}}) +- [ ] No hardcoded values in instructions that should be yaml variables + +## Web Bundle Validation (if applicable) + +- [ ] web_bundle section present if workflow needs deployment +- [ ] All paths in web_bundle use bmad/-relative format (NOT {project-root}) +- [ ] No {config_source} variables in web_bundle section +- [ ] instructions file listed in web_bundle_files array +- [ ] template file listed in web_bundle_files (if document workflow) +- [ ] validation/checklist file listed in web_bundle_files (if exists) +- [ ] All data files (CSV, JSON, YAML) listed in web_bundle_files +- [ ] All called workflows have their .yaml files in web_bundle_files +- [ ] All files referenced in instructions tags listed in web_bundle_files +- [ ] No files listed in web_bundle_files that don't exist +- [ ] Web bundle metadata (name, description, author) matches top-level metadata + +## Template Validation (if document workflow) + +- [ ] Template variables match tags in instructions exactly +- [ ] All required sections present in template structure +- [ ] Template uses {{variable}} syntax (double curly braces) +- [ ] Template variables use snake_case (not camelCase or PascalCase) +- [ ] Standard metadata header format correct (optional usage of {{date}}, {{user_name}}) +- [ ] No placeholders remain in template (like {SECTION_NAME}) +- [ ] Template structure matches document purpose + +## Instructions Quality + +- [ ] Each step has n="X" attribute with sequential numbering +- [ ] Each step has goal="clear goal statement" attribute +- [ ] Optional steps marked with optional="true" +- [ ] Repeating steps have appropriate repeat attribute (repeat="3", repeat="for-each-X", repeat="until-approved") +- [ ] Conditional steps have if="condition" attribute +- [ ] XML tags used correctly (, , , , , ) +- [ ] Steps are focused (single goal per step) +- [ ] Instructions are specific with limits ("Write 1-2 paragraphs" not "Write about") +- [ ] Examples provided where helpful +- [ ] tags save checkpoints for document workflows +- [ ] Flow control is logical and clear + +## Bloat Detection + +- [ ] Bloat percentage under 10% (unused yaml fields / total fields) +- [ ] No commented-out variables that should be removed +- [ ] No duplicate metadata between sections +- [ ] No variables defined but never referenced +- [ ] No redundant configuration that duplicates web_bundle + +## Final Validation + +### Critical Issues (Must fix immediately) + +_List any critical issues found:_ + +- Issue 1: +- Issue 2: +- Issue 3: + +### Important Issues (Should fix soon) + +_List any important issues found:_ + +- Issue 1: +- Issue 2: +- Issue 3: + +### Cleanup Recommendations (Nice to have) + +_List any cleanup recommendations:_ + +- Recommendation 1: +- Recommendation 2: +- Recommendation 3: + +--- + +## Audit Summary + +**Total Checks:** 70 +**Passed:** **\_** / 70 +**Failed:** **\_** / 70 +**Pass Rate:** **\_**% + +**Recommendation:** + +- Pass Rate ≥ 95%: Excellent - Ready for production +- Pass Rate 85-94%: Good - Minor fixes needed +- Pass Rate 70-84%: Fair - Important issues to address +- Pass Rate < 70%: Poor - Significant work required + +--- + +**Audit Completed:** {{date}} +**Auditor:** Audit Workflow (BMAD v6) diff --git a/bmad/bmb/workflows/audit-workflow/instructions.md b/bmad/bmb/workflows/audit-workflow/instructions.md new file mode 100644 index 00000000..b2a18b6b --- /dev/null +++ b/bmad/bmb/workflows/audit-workflow/instructions.md @@ -0,0 +1,370 @@ +# Audit Workflow - Workflow Quality Audit Instructions + +The workflow execution engine is governed by: {project-root}/bmad/core/tasks/workflow.xml +You MUST have already loaded and processed: {project-root}/bmad/bmb/workflows/audit-workflow/workflow.yaml + + + + +What is the path to the workflow you want to audit? (provide path to workflow.yaml or workflow folder) + +Load the workflow.yaml file from the provided path +Identify the workflow type (document, action, interactive, autonomous, meta) +List all associated files: + +- instructions.md (required for most workflows) +- template.md (if document workflow) +- checklist.md (if validation exists) +- Any data files referenced in yaml + +Load all discovered files + +Display summary: + +- Workflow name and description +- Type of workflow +- Files present +- Module assignment + + + +Check workflow.yaml for the standard config block: + +**Required variables:** + +- `config_source: "{project-root}/bmad/[module]/config.yaml"` +- `output_folder: "{config_source}:output_folder"` +- `user_name: "{config_source}:user_name"` +- `communication_language: "{config_source}:communication_language"` +- `date: system-generated` + +Validate each variable: + +**Config Source Check:** + +- [ ] `config_source` is defined +- [ ] Points to correct module config path +- [ ] Uses {project-root} variable + +**Standard Variables Check:** + +- [ ] `output_folder` pulls from config_source +- [ ] `user_name` pulls from config_source +- [ ] `communication_language` pulls from config_source +- [ ] `date` is set to system-generated + +Record any missing or incorrect config variables +config_issues + +If config issues found: +Add to issues list with severity: CRITICAL + + + +Extract all variables defined in workflow.yaml (excluding standard config block) +Scan instructions.md for variable usage: {variable_name} pattern +Scan template.md for variable usage: {{variable_name}} pattern (if exists) + +Cross-reference analysis: + +**For each yaml variable:** + +1. Is it used in instructions.md? (mark as INSTRUCTION_USED) +2. Is it used in template.md? (mark as TEMPLATE_USED) +3. Is it neither? (mark as UNUSED_BLOAT) + +**Special cases to ignore:** + +- Standard config variables (config_source, output_folder, user_name, communication_language, date) +- Workflow metadata (name, description, author) +- Path variables (installed_path, template, instructions, validation) +- Web bundle configuration (web_bundle block itself) + +Identify unused yaml fields (bloat) +Identify hardcoded values in instructions that should be variables +alignment_issues + +If unused variables found: +Add to issues list with severity: BLOAT + + + +Analyze instructions.md for proper config variable usage: + +**Communication Language Check:** + +- Search for phrases like "communicate in {communication_language}" +- Check if greetings/responses use language-aware patterns +- Verify NO usage of {{communication_language}} in template headers + +**User Name Check:** + +- Look for user addressing patterns using {user_name} +- Check if summaries or greetings personalize with {user_name} +- Verify optional usage in template metadata (not required) + +**Output Folder Check:** + +- Search for file write operations +- Verify all outputs go to {output_folder} or subdirectories +- Check for hardcoded paths like "/output/" or "/generated/" + +**Date Usage Check:** + +- Verify date is available for agent date awareness +- Check optional usage in template metadata +- Ensure no confusion between date and model training cutoff + +Record any improper config variable usage +config_usage_issues + +If config usage issues found: +Add to issues list with severity: IMPORTANT + + + +If workflow.yaml contains web_bundle section: + +Validate web_bundle structure: + +**Path Validation:** + +- [ ] All paths use bmad/-relative format (NOT {project-root}) +- [ ] No {config_source} variables in web_bundle section +- [ ] Paths match actual file locations + +**Completeness Check:** + +- [ ] instructions file listed in web_bundle_files +- [ ] template file listed (if document workflow) +- [ ] validation/checklist file listed (if exists) +- [ ] All data files referenced in yaml listed +- [ ] All files referenced in instructions listed + +**Workflow Dependency Scan:** +Scan instructions.md for tags +Extract workflow paths from invocations +Verify each called workflow.yaml is in web_bundle_files + +**File Reference Scan:** +Scan instructions.md for file references in tags +Check for CSV, JSON, YAML, MD files referenced +Verify all referenced files are in web_bundle_files + +Record any missing files or incorrect paths +web_bundle_issues + +If web_bundle issues found: +Add to issues list with severity: CRITICAL + +If no web_bundle section exists: +Note: "No web_bundle configured (may be intentional for local-only workflows)" + + + +Identify bloat patterns: + +**Unused YAML Fields:** + +- Variables defined but not used in instructions OR template +- Duplicate fields between top-level and web_bundle section +- Commented-out variables that should be removed + +**Hardcoded Values:** + +- File paths that should use {output_folder} +- Generic greetings that should use {user_name} +- Language-specific text that should use {communication_language} +- Static dates that should use {date} + +**Redundant Configuration:** + +- Variables that duplicate web_bundle fields +- Metadata repeated across sections + +Calculate bloat metrics: + +- Total yaml fields: {{total_yaml_fields}} +- Used fields: {{used_fields}} +- Unused fields: {{unused_fields}} +- Bloat percentage: {{bloat_percentage}}% + +Record all bloat items with recommendations +bloat_items + +If bloat detected: +Add to issues list with severity: CLEANUP + + + +Extract all template variables from template.md: {{variable_name}} pattern +Scan instructions.md for corresponding variable_name tags + +Cross-reference mapping: + +**For each template variable:** + +1. Is there a matching tag? (mark as MAPPED) +2. Is it a standard config variable? (mark as CONFIG_VAR - optional) +3. Is it unmapped? (mark as MISSING_OUTPUT) + +**For each tag:** + +1. Is there a matching template variable? (mark as USED) +2. Is it orphaned? (mark as UNUSED_OUTPUT) + +Verify variable naming conventions: + +- [ ] All template variables use snake_case +- [ ] Variable names are descriptive (not abbreviated) +- [ ] Standard config variables properly formatted + +Record any mapping issues +template_issues + +If template issues found: +Add to issues list with severity: IMPORTANT + + + +Compile all findings into a structured report + +Write audit report to {output_folder}/audit-report-{{workflow_name}}-{{date}}.md + +**Report Structure:** + +```markdown +# Workflow Audit Report + +**Workflow:** {{workflow_name}} +**Audit Date:** {{date}} +**Auditor:** Audit Workflow (BMAD v6) +**Workflow Type:** {{workflow_type}} + +--- + +## Executive Summary + +**Overall Status:** {{overall_status}} + +- Critical Issues: {{critical_count}} +- Important Issues: {{important_count}} +- Cleanup Recommendations: {{cleanup_count}} + +--- + +## 1. Standard Config Block Validation + +{{config_issues}} + +**Status:** {{config_status}} + +--- + +## 2. YAML/Instruction/Template Alignment + +{{alignment_issues}} + +**Variables Analyzed:** {{total_variables}} +**Used in Instructions:** {{instruction_usage_count}} +**Used in Template:** {{template_usage_count}} +**Unused (Bloat):** {{bloat_count}} + +--- + +## 3. Config Variable Usage + +{{config_usage_issues}} + +**Communication Language:** {{comm_lang_status}} +**User Name:** {{user_name_status}} +**Output Folder:** {{output_folder_status}} +**Date:** {{date_status}} + +--- + +## 4. Web Bundle Validation + +{{web_bundle_issues}} + +**Web Bundle Present:** {{web_bundle_exists}} +**Files Listed:** {{web_bundle_file_count}} +**Missing Files:** {{missing_files_count}} + +--- + +## 5. Bloat Detection + +{{bloat_items}} + +**Bloat Percentage:** {{bloat_percentage}}% +**Cleanup Potential:** {{cleanup_potential}} + +--- + +## 6. Template Variable Mapping + +{{template_issues}} + +**Template Variables:** {{template_var_count}} +**Mapped Correctly:** {{mapped_count}} +**Missing Mappings:** {{missing_mapping_count}} + +--- + +## Recommendations + +### Critical (Fix Immediately) + +{{critical_recommendations}} + +### Important (Address Soon) + +{{important_recommendations}} + +### Cleanup (Nice to Have) + +{{cleanup_recommendations}} + +--- + +## Validation Checklist + +Use this checklist to verify fixes: + +- [ ] All standard config variables present and correct +- [ ] No unused yaml fields (bloat removed) +- [ ] Config variables used appropriately in instructions +- [ ] Web bundle includes all dependencies +- [ ] Template variables properly mapped +- [ ] File structure follows v6 conventions + +--- + +## Next Steps + +1. Review critical issues and fix immediately +2. Address important issues in next iteration +3. Consider cleanup recommendations for optimization +4. Re-run audit after fixes to verify improvements + +--- + +**Audit Complete** - Generated by audit-workflow v1.0 +``` + +Display summary to user in {communication_language} +Provide path to full audit report + +Would you like to: + +- View the full audit report +- Fix issues automatically (invoke edit-workflow) +- Audit another workflow +- Exit + + +audit_report_path + + + diff --git a/bmad/bmb/workflows/audit-workflow/workflow.yaml b/bmad/bmb/workflows/audit-workflow/workflow.yaml new file mode 100644 index 00000000..db6b5624 --- /dev/null +++ b/bmad/bmb/workflows/audit-workflow/workflow.yaml @@ -0,0 +1,20 @@ +# Audit Workflow Configuration +name: "audit-workflow" +description: "Comprehensive workflow quality audit - validates structure, config standards, variable usage, bloat detection, and web_bundle completeness. Performs deep analysis of workflow.yaml, instructions.md, template.md, and web_bundle configuration against BMAD v6 standards." +author: "BMad" + +# Critical variables from config +config_source: "{project-root}/bmad/bmb/config.yaml" +output_folder: "{config_source}:output_folder" +user_name: "{config_source}:user_name" +communication_language: "{config_source}:communication_language" +date: system-generated + +# Module path and component files +installed_path: "{project-root}/bmad/bmb/workflows/audit-workflow" +template: false +instructions: "{installed_path}/instructions.md" +validation: "{installed_path}/checklist.md" + +# Output configuration +default_output_file: "{output_folder}/audit-report-{{workflow_name}}-{{date}}.md" diff --git a/src/modules/bmb/workflows/create-workflow/instructions.md b/src/modules/bmb/workflows/create-workflow/instructions.md index 1c118189..e7a556c8 100644 --- a/src/modules/bmb/workflows/create-workflow/instructions.md +++ b/src/modules/bmb/workflows/create-workflow/instructions.md @@ -201,16 +201,20 @@ Generate the template.md file following guide conventions: 1. Document structure with clear sections 2. Variable syntax: {{variable_name}} using snake_case 3. Variable names MUST match tags exactly from instructions -4. Include standard metadata header using config variables: +4. Include standard metadata header (optional - config variables available): ```markdown # Document Title **Date:** {{date}} **Author:** {{user_name}} - **Language:** {{communication_language}} ``` + Note: {{date}} and {{user_name}} are optional in headers. Primary purpose of these variables: + - {{date}} - Gives agent current date awareness (not confused with training cutoff) + - {{user_name}} - Optional author attribution + - {{communication_language}} - NOT for document output! Tells agent how to communicate during execution + 5. Follow naming conventions from guide: - Use descriptive names: {{primary_user_journey}} not {{puj}} - Snake_case for all variables @@ -225,11 +229,16 @@ Variable sources as per guide: Standard config variables in templates: -Templates MUST use standard config variables in headers: +Templates CAN optionally use these config variables: -- {{user_name}} - Document author -- {{date}} - Generation date -- {{communication_language}} - Content language (if multilingual) +- {{user_name}} - Document author (optional) +- {{date}} - Generation date (optional) + +IMPORTANT: {{communication_language}} is NOT for document headers! + +- Purpose: Tells agent how to communicate with user during workflow execution +- NOT for: Document output language or template headers +- Future: {{document_output_language}} will handle multilingual document generation These variables are automatically available from workflow.yaml config block. diff --git a/src/modules/bmb/workflows/edit-workflow/instructions.md b/src/modules/bmb/workflows/edit-workflow/instructions.md index 8625cfc2..afb92b47 100644 --- a/src/modules/bmb/workflows/edit-workflow/instructions.md +++ b/src/modules/bmb/workflows/edit-workflow/instructions.md @@ -204,7 +204,8 @@ If updating existing web bundle: - [ ] Instructions communicate in {communication_language} where appropriate - [ ] Instructions address {user_name} where appropriate - [ ] Instructions write to {output_folder} for file outputs -- [ ] Template includes {{user_name}}, {{date}} in metadata (if document workflow) +- [ ] Template optionally includes {{user_name}}, {{date}} in metadata (if document workflow) +- [ ] Template does NOT use {{communication_language}} in headers (agent-only variable) **YAML/File Alignment:** diff --git a/workflow-cleanup-progress.md b/workflow-cleanup-progress.md index fbd3cfe8..bcc67b9a 100644 --- a/workflow-cleanup-progress.md +++ b/workflow-cleanup-progress.md @@ -115,7 +115,27 @@ _None yet_ ## Issues/Notes Log -_Track any anomalies, decisions, or special cases here_ +### 2025-10-16: Created audit-workflow + +- **Location:** `/bmad/bmb/workflows/audit-workflow/` +- **Purpose:** Codifies all Phase 1 and Phase 2 standards into a reusable validation tool +- **Files Created:** + - `workflow.yaml` - Configuration with standard config block + - `instructions.md` - 8-step audit process + - `checklist.md` - 70-point validation checklist +- **Validation:** audit-workflow passes 100% of its own validation criteria +- **Next Use:** Will be used to perform Phase 3 systematic audits + +**Audit-Workflow Capabilities:** + +1. Standard config block validation +2. YAML/instruction/template alignment analysis +3. Config variable usage audit +4. Web bundle completeness validation +5. Bloat detection (unused yaml fields) +6. Template variable mapping verification +7. Comprehensive audit report generation +8. Integration with edit-workflow for fixes ---