Compare commits

...

1 Commits

Author SHA1 Message Date
Manfred Riem
9402ebd00a Feat/ai skills (#1632)
* implement ai-skills command line switch

* fix: address review comments, remove breaking change for existing projects, add tests

* fix: review comments

* fix: review comments

* fix: review comments

* fix: review comments

* fix: review comments, add test cases for all the agents

* fix: review comments

* fix: review comments

* chore: trigger CI

* chore: trigger CodeQL

* ci: add CodeQL workflow for code scanning

* ci: add actions language to CodeQL workflow, disable default setup

---------

Co-authored-by: dhilipkumars <s.dhilipkumar@gmail.com>
2026-02-19 13:24:41 -06:00
6 changed files with 920 additions and 1 deletions

32
.github/workflows/codeql.yml vendored Normal file
View File

@@ -0,0 +1,32 @@
name: "CodeQL"
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
security-events: write
contents: read
strategy:
fail-fast: false
matrix:
language: [ 'actions', 'python' ]
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{ matrix.language }}"

View File

@@ -2,11 +2,27 @@
<!-- markdownlint-disable MD024 --> <!-- markdownlint-disable MD024 -->
All notable changes to the Specify CLI and templates are documented here. All notable changes to the Specify CLI and templates are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.1.1] - 2026-02-13
### Added
- **Agent Skills Installation**: New `--ai-skills` CLI option to install Prompt.MD templates as agent skills following [agentskills.io specification](https://agentskills.io/specification)
- Skills are installed to agent-specific directories (e.g., `.claude/skills/`, `.gemini/skills/`, `.github/skills/`)
- Codex uses `.agents/skills/` following Codex agent directory conventions
- Default fallback directory is `.agents/skills/` for agents without a specific mapping
- Requires `--ai` flag to be specified
- Converts all 9 spec-kit command templates (specify, plan, tasks, implement, analyze, clarify, constitution, checklist, taskstoissues) to properly formatted SKILL.md files
- **New projects**: command files are not installed when `--ai-skills` is used (skills replace commands)
- **Existing repos** (`--here`): pre-existing command files are preserved — no breaking changes
- `pyyaml` dependency (already present) used for YAML frontmatter parsing
- **Unit tests** for `install_ai_skills`, `_get_skills_dir`, and `--ai-skills` CLI validation (51 test cases covering all 18 supported agents)
## [0.1.0] - 2026-01-28 ## [0.1.0] - 2026-01-28
### Added ### Added

View File

@@ -188,6 +188,7 @@ The `specify` command supports the following options:
| `--skip-tls` | Flag | Skip SSL/TLS verification (not recommended) | | `--skip-tls` | Flag | Skip SSL/TLS verification (not recommended) |
| `--debug` | Flag | Enable detailed debug output for troubleshooting | | `--debug` | Flag | Enable detailed debug output for troubleshooting |
| `--github-token` | Option | GitHub token for API requests (or set GH_TOKEN/GITHUB_TOKEN env variable) | | `--github-token` | Option | GitHub token for API requests (or set GH_TOKEN/GITHUB_TOKEN env variable) |
| `--ai-skills` | Flag | Install Prompt.MD templates as agent skills in agent-specific `skills/` directory (requires `--ai`) |
### Examples ### Examples
@@ -238,6 +239,12 @@ specify init my-project --ai claude --debug
# Use GitHub token for API requests (helpful for corporate environments) # Use GitHub token for API requests (helpful for corporate environments)
specify init my-project --ai claude --github-token ghp_your_token_here specify init my-project --ai claude --github-token ghp_your_token_here
# Install agent skills with the project
specify init my-project --ai claude --ai-skills
# Initialize in current directory with agent skills
specify init --here --ai gemini --ai-skills
# Check system requirements # Check system requirements
specify check specify check
``` ```

View File

@@ -1,6 +1,6 @@
[project] [project]
name = "specify-cli" name = "specify-cli"
version = "0.1.0" version = "0.1.1"
description = "Specify CLI, part of GitHub Spec Kit. A tool to bootstrap your projects for Spec-Driven Development (SDD)." description = "Specify CLI, part of GitHub Spec Kit. A tool to bootstrap your projects for Spec-Driven Development (SDD)."
requires-python = ">=3.11" requires-python = ">=3.11"
dependencies = [ dependencies = [

View File

@@ -32,6 +32,7 @@ import tempfile
import shutil import shutil
import shlex import shlex
import json import json
import yaml
from pathlib import Path from pathlib import Path
from typing import Optional, Tuple from typing import Optional, Tuple
@@ -983,6 +984,203 @@ def ensure_constitution_from_template(project_path: Path, tracker: StepTracker |
else: else:
console.print(f"[yellow]Warning: Could not initialize constitution: {e}[/yellow]") console.print(f"[yellow]Warning: Could not initialize constitution: {e}[/yellow]")
# Agent-specific skill directory overrides for agents whose skills directory
# doesn't follow the standard <agent_folder>/skills/ pattern
AGENT_SKILLS_DIR_OVERRIDES = {
"codex": ".agents/skills", # Codex agent layout override
}
# Default skills directory for agents not in AGENT_CONFIG
DEFAULT_SKILLS_DIR = ".agents/skills"
# Enhanced descriptions for each spec-kit command skill
SKILL_DESCRIPTIONS = {
"specify": "Create or update feature specifications from natural language descriptions. Use when starting new features or refining requirements. Generates spec.md with user stories, functional requirements, and acceptance criteria following spec-driven development methodology.",
"plan": "Generate technical implementation plans from feature specifications. Use after creating a spec to define architecture, tech stack, and implementation phases. Creates plan.md with detailed technical design.",
"tasks": "Break down implementation plans into actionable task lists. Use after planning to create a structured task breakdown. Generates tasks.md with ordered, dependency-aware tasks.",
"implement": "Execute all tasks from the task breakdown to build the feature. Use after task generation to systematically implement the planned solution following TDD approach where applicable.",
"analyze": "Perform cross-artifact consistency analysis across spec.md, plan.md, and tasks.md. Use after task generation to identify gaps, duplications, and inconsistencies before implementation.",
"clarify": "Structured clarification workflow for underspecified requirements. Use before planning to resolve ambiguities through coverage-based questioning. Records answers in spec clarifications section.",
"constitution": "Create or update project governing principles and development guidelines. Use at project start to establish code quality, testing standards, and architectural constraints that guide all development.",
"checklist": "Generate custom quality checklists for validating requirements completeness and clarity. Use to create unit tests for English that ensure spec quality before implementation.",
"taskstoissues": "Convert tasks from tasks.md into GitHub issues. Use after task breakdown to track work items in GitHub project management.",
}
def _get_skills_dir(project_path: Path, selected_ai: str) -> Path:
"""Resolve the agent-specific skills directory for the given AI assistant.
Uses ``AGENT_SKILLS_DIR_OVERRIDES`` first, then falls back to
``AGENT_CONFIG[agent]["folder"] + "skills"``, and finally to
``DEFAULT_SKILLS_DIR``.
"""
if selected_ai in AGENT_SKILLS_DIR_OVERRIDES:
return project_path / AGENT_SKILLS_DIR_OVERRIDES[selected_ai]
agent_config = AGENT_CONFIG.get(selected_ai, {})
agent_folder = agent_config.get("folder", "")
if agent_folder:
return project_path / agent_folder.rstrip("/") / "skills"
return project_path / DEFAULT_SKILLS_DIR
def install_ai_skills(project_path: Path, selected_ai: str, tracker: StepTracker | None = None) -> bool:
"""Install Prompt.MD files from templates/commands/ as agent skills.
Skills are written to the agent-specific skills directory following the
`agentskills.io <https://agentskills.io/specification>`_ specification.
Installation is additive — existing files are never removed and prompt
command files in the agent's commands directory are left untouched.
Args:
project_path: Target project directory.
selected_ai: AI assistant key from ``AGENT_CONFIG``.
tracker: Optional progress tracker.
Returns:
``True`` if at least one skill was installed or all skills were
already present (idempotent re-run), ``False`` otherwise.
"""
# Locate command templates in the agent's extracted commands directory.
# download_and_extract_template() already placed the .md files here.
agent_config = AGENT_CONFIG.get(selected_ai, {})
agent_folder = agent_config.get("folder", "")
if agent_folder:
templates_dir = project_path / agent_folder.rstrip("/") / "commands"
else:
templates_dir = project_path / "commands"
if not templates_dir.exists() or not any(templates_dir.glob("*.md")):
# Fallback: try the repo-relative path (for running from source checkout)
# This also covers agents whose extracted commands are in a different
# format (e.g. gemini uses .toml, not .md).
script_dir = Path(__file__).parent.parent.parent # up from src/specify_cli/
fallback_dir = script_dir / "templates" / "commands"
if fallback_dir.exists() and any(fallback_dir.glob("*.md")):
templates_dir = fallback_dir
if not templates_dir.exists() or not any(templates_dir.glob("*.md")):
if tracker:
tracker.error("ai-skills", "command templates not found")
else:
console.print("[yellow]Warning: command templates not found, skipping skills installation[/yellow]")
return False
command_files = sorted(templates_dir.glob("*.md"))
if not command_files:
if tracker:
tracker.skip("ai-skills", "no command templates found")
else:
console.print("[yellow]No command templates found to install[/yellow]")
return False
# Resolve the correct skills directory for this agent
skills_dir = _get_skills_dir(project_path, selected_ai)
skills_dir.mkdir(parents=True, exist_ok=True)
if tracker:
tracker.start("ai-skills")
installed_count = 0
skipped_count = 0
for command_file in command_files:
try:
content = command_file.read_text(encoding="utf-8")
# Parse YAML frontmatter
if content.startswith("---"):
parts = content.split("---", 2)
if len(parts) >= 3:
frontmatter = yaml.safe_load(parts[1])
if not isinstance(frontmatter, dict):
frontmatter = {}
body = parts[2].strip()
else:
# File starts with --- but has no closing ---
console.print(f"[yellow]Warning: {command_file.name} has malformed frontmatter (no closing ---), treating as plain content[/yellow]")
frontmatter = {}
body = content
else:
frontmatter = {}
body = content
command_name = command_file.stem
# Normalize: extracted commands may be named "speckit.<cmd>.md";
# strip the "speckit." prefix so skill names stay clean and
# SKILL_DESCRIPTIONS lookups work.
if command_name.startswith("speckit."):
command_name = command_name[len("speckit."):]
skill_name = f"speckit-{command_name}"
# Create skill directory (additive — never removes existing content)
skill_dir = skills_dir / skill_name
skill_dir.mkdir(parents=True, exist_ok=True)
# Select the best description available
original_desc = frontmatter.get("description", "")
enhanced_desc = SKILL_DESCRIPTIONS.get(command_name, original_desc or f"Spec-kit workflow command: {command_name}")
# Build SKILL.md following agentskills.io spec
# Use yaml.safe_dump to safely serialise the frontmatter and
# avoid YAML injection from descriptions containing colons,
# quotes, or newlines.
# Normalize source filename for metadata — strip speckit. prefix
# so it matches the canonical templates/commands/<cmd>.md path.
source_name = command_file.name
if source_name.startswith("speckit."):
source_name = source_name[len("speckit."):]
frontmatter_data = {
"name": skill_name,
"description": enhanced_desc,
"compatibility": "Requires spec-kit project structure with .specify/ directory",
"metadata": {
"author": "github-spec-kit",
"source": f"templates/commands/{source_name}",
},
}
frontmatter_text = yaml.safe_dump(frontmatter_data, sort_keys=False).strip()
skill_content = (
f"---\n"
f"{frontmatter_text}\n"
f"---\n\n"
f"# Speckit {command_name.title()} Skill\n\n"
f"{body}\n"
)
skill_file = skill_dir / "SKILL.md"
if skill_file.exists():
# Do not overwrite user-customized skills on re-runs
skipped_count += 1
continue
skill_file.write_text(skill_content, encoding="utf-8")
installed_count += 1
except Exception as e:
console.print(f"[yellow]Warning: Failed to install skill {command_file.stem}: {e}[/yellow]")
continue
if tracker:
if installed_count > 0 and skipped_count > 0:
tracker.complete("ai-skills", f"{installed_count} new + {skipped_count} existing skills in {skills_dir.relative_to(project_path)}")
elif installed_count > 0:
tracker.complete("ai-skills", f"{installed_count} skills → {skills_dir.relative_to(project_path)}")
elif skipped_count > 0:
tracker.complete("ai-skills", f"{skipped_count} skills already present")
else:
tracker.error("ai-skills", "no skills installed")
else:
if installed_count > 0:
console.print(f"[green]✓[/green] Installed {installed_count} agent skills to {skills_dir.relative_to(project_path)}/")
elif skipped_count > 0:
console.print(f"[green]✓[/green] {skipped_count} agent skills already present in {skills_dir.relative_to(project_path)}/")
else:
console.print("[yellow]No skills were installed[/yellow]")
return installed_count > 0 or skipped_count > 0
@app.command() @app.command()
def init( def init(
project_name: str = typer.Argument(None, help="Name for your new project directory (optional if using --here, or use '.' for current directory)"), project_name: str = typer.Argument(None, help="Name for your new project directory (optional if using --here, or use '.' for current directory)"),
@@ -995,6 +1193,7 @@ def init(
skip_tls: bool = typer.Option(False, "--skip-tls", help="Skip SSL/TLS verification (not recommended)"), skip_tls: bool = typer.Option(False, "--skip-tls", help="Skip SSL/TLS verification (not recommended)"),
debug: bool = typer.Option(False, "--debug", help="Show verbose diagnostic output for network and extraction failures"), debug: bool = typer.Option(False, "--debug", help="Show verbose diagnostic output for network and extraction failures"),
github_token: str = typer.Option(None, "--github-token", help="GitHub token to use for API requests (or set GH_TOKEN or GITHUB_TOKEN environment variable)"), github_token: str = typer.Option(None, "--github-token", help="GitHub token to use for API requests (or set GH_TOKEN or GITHUB_TOKEN environment variable)"),
ai_skills: bool = typer.Option(False, "--ai-skills", help="Install Prompt.MD templates as agent skills (requires --ai)"),
): ):
""" """
Initialize a new Specify project from the latest template. Initialize a new Specify project from the latest template.
@@ -1019,6 +1218,8 @@ def init(
specify init --here --ai codebuddy specify init --here --ai codebuddy
specify init --here specify init --here
specify init --here --force # Skip confirmation when current directory not empty specify init --here --force # Skip confirmation when current directory not empty
specify init my-project --ai claude --ai-skills # Install agent skills
specify init --here --ai gemini --ai-skills
""" """
show_banner() show_banner()
@@ -1035,6 +1236,11 @@ def init(
console.print("[red]Error:[/red] Must specify either a project name, use '.' for current directory, or use --here flag") console.print("[red]Error:[/red] Must specify either a project name, use '.' for current directory, or use --here flag")
raise typer.Exit(1) raise typer.Exit(1)
if ai_skills and not ai_assistant:
console.print("[red]Error:[/red] --ai-skills requires --ai to be specified")
console.print("[yellow]Usage:[/yellow] specify init <project> --ai <agent> --ai-skills")
raise typer.Exit(1)
if here: if here:
project_name = Path.cwd().name project_name = Path.cwd().name
project_path = Path.cwd() project_path = Path.cwd()
@@ -1150,6 +1356,11 @@ def init(
("extracted-summary", "Extraction summary"), ("extracted-summary", "Extraction summary"),
("chmod", "Ensure scripts executable"), ("chmod", "Ensure scripts executable"),
("constitution", "Constitution setup"), ("constitution", "Constitution setup"),
]:
tracker.add(key, label)
if ai_skills:
tracker.add("ai-skills", "Install agent skills")
for key, label in [
("cleanup", "Cleanup"), ("cleanup", "Cleanup"),
("git", "Initialize git repository"), ("git", "Initialize git repository"),
("final", "Finalize") ("final", "Finalize")
@@ -1172,6 +1383,29 @@ def init(
ensure_constitution_from_template(project_path, tracker=tracker) ensure_constitution_from_template(project_path, tracker=tracker)
if ai_skills:
skills_ok = install_ai_skills(project_path, selected_ai, tracker=tracker)
# When --ai-skills is used on a NEW project and skills were
# successfully installed, remove the command files that the
# template archive just created. Skills replace commands, so
# keeping both would be confusing. For --here on an existing
# repo we leave pre-existing commands untouched to avoid a
# breaking change. We only delete AFTER skills succeed so the
# project always has at least one of {commands, skills}.
if skills_ok and not here:
agent_cfg = AGENT_CONFIG.get(selected_ai, {})
agent_folder = agent_cfg.get("folder", "")
if agent_folder:
cmds_dir = project_path / agent_folder.rstrip("/") / "commands"
if cmds_dir.exists():
try:
shutil.rmtree(cmds_dir)
except OSError:
# Best-effort cleanup: skills are already installed,
# so leaving stale commands is non-fatal.
console.print("[yellow]Warning: could not remove extracted commands directory[/yellow]")
if not no_git: if not no_git:
tracker.start("git") tracker.start("git")
if is_git_repo(project_path): if is_git_repo(project_path):

630
tests/test_ai_skills.py Normal file
View File

@@ -0,0 +1,630 @@
"""
Unit tests for AI agent skills installation.
Tests cover:
- Skills directory resolution for different agents (_get_skills_dir)
- YAML frontmatter parsing and SKILL.md generation (install_ai_skills)
- Cleanup of duplicate command files when --ai-skills is used
- Missing templates directory handling
- Malformed template error handling
- CLI validation: --ai-skills requires --ai
"""
import pytest
import tempfile
import shutil
import yaml
from pathlib import Path
from unittest.mock import patch
import specify_cli
from specify_cli import (
_get_skills_dir,
install_ai_skills,
AGENT_SKILLS_DIR_OVERRIDES,
DEFAULT_SKILLS_DIR,
SKILL_DESCRIPTIONS,
AGENT_CONFIG,
app,
)
# ===== Fixtures =====
@pytest.fixture
def temp_dir():
"""Create a temporary directory for tests."""
tmpdir = tempfile.mkdtemp()
yield Path(tmpdir)
shutil.rmtree(tmpdir)
@pytest.fixture
def project_dir(temp_dir):
"""Create a mock project directory."""
proj_dir = temp_dir / "test-project"
proj_dir.mkdir()
return proj_dir
@pytest.fixture
def templates_dir(project_dir):
"""Create mock command templates in the project's agent commands directory.
This simulates what download_and_extract_template() does: it places
command .md files into project_path/<agent_folder>/commands/.
install_ai_skills() now reads from here instead of from the repo
source tree.
"""
tpl_root = project_dir / ".claude" / "commands"
tpl_root.mkdir(parents=True, exist_ok=True)
# Template with valid YAML frontmatter
(tpl_root / "specify.md").write_text(
"---\n"
"description: Create or update the feature specification.\n"
"handoffs:\n"
" - label: Build Plan\n"
" agent: speckit.plan\n"
"scripts:\n"
" sh: scripts/bash/create-new-feature.sh\n"
"---\n"
"\n"
"# Specify Command\n"
"\n"
"Run this to create a spec.\n",
encoding="utf-8",
)
# Template with minimal frontmatter
(tpl_root / "plan.md").write_text(
"---\n"
"description: Generate implementation plan.\n"
"---\n"
"\n"
"# Plan Command\n"
"\n"
"Plan body content.\n",
encoding="utf-8",
)
# Template with no frontmatter
(tpl_root / "tasks.md").write_text(
"# Tasks Command\n"
"\n"
"Body without frontmatter.\n",
encoding="utf-8",
)
# Template with empty YAML frontmatter (yaml.safe_load returns None)
(tpl_root / "empty_fm.md").write_text(
"---\n"
"---\n"
"\n"
"# Empty Frontmatter Command\n"
"\n"
"Body with empty frontmatter.\n",
encoding="utf-8",
)
return tpl_root
@pytest.fixture
def commands_dir_claude(project_dir):
"""Create a populated .claude/commands directory simulating template extraction."""
cmd_dir = project_dir / ".claude" / "commands"
cmd_dir.mkdir(parents=True, exist_ok=True)
for name in ["speckit.specify.md", "speckit.plan.md", "speckit.tasks.md"]:
(cmd_dir / name).write_text(f"# {name}\nContent here\n")
return cmd_dir
@pytest.fixture
def commands_dir_gemini(project_dir):
"""Create a populated .gemini/commands directory (TOML format)."""
cmd_dir = project_dir / ".gemini" / "commands"
cmd_dir.mkdir(parents=True)
for name in ["speckit.specify.toml", "speckit.plan.toml", "speckit.tasks.toml"]:
(cmd_dir / name).write_text(f'[command]\nname = "{name}"\n')
return cmd_dir
# ===== _get_skills_dir Tests =====
class TestGetSkillsDir:
"""Test the _get_skills_dir() helper function."""
def test_claude_skills_dir(self, project_dir):
"""Claude should use .claude/skills/."""
result = _get_skills_dir(project_dir, "claude")
assert result == project_dir / ".claude" / "skills"
def test_gemini_skills_dir(self, project_dir):
"""Gemini should use .gemini/skills/."""
result = _get_skills_dir(project_dir, "gemini")
assert result == project_dir / ".gemini" / "skills"
def test_copilot_skills_dir(self, project_dir):
"""Copilot should use .github/skills/."""
result = _get_skills_dir(project_dir, "copilot")
assert result == project_dir / ".github" / "skills"
def test_codex_uses_override(self, project_dir):
"""Codex should use the AGENT_SKILLS_DIR_OVERRIDES value."""
result = _get_skills_dir(project_dir, "codex")
assert result == project_dir / ".agents" / "skills"
def test_cursor_agent_skills_dir(self, project_dir):
"""Cursor should use .cursor/skills/."""
result = _get_skills_dir(project_dir, "cursor-agent")
assert result == project_dir / ".cursor" / "skills"
def test_unknown_agent_uses_default(self, project_dir):
"""Unknown agents should fall back to DEFAULT_SKILLS_DIR."""
result = _get_skills_dir(project_dir, "nonexistent-agent")
assert result == project_dir / DEFAULT_SKILLS_DIR
def test_all_configured_agents_resolve(self, project_dir):
"""Every agent in AGENT_CONFIG should resolve to a valid path."""
for agent_key in AGENT_CONFIG:
result = _get_skills_dir(project_dir, agent_key)
assert result is not None
assert str(result).startswith(str(project_dir))
# Should always end with "skills"
assert result.name == "skills"
def test_override_takes_precedence_over_config(self, project_dir):
"""AGENT_SKILLS_DIR_OVERRIDES should take precedence over AGENT_CONFIG."""
for agent_key in AGENT_SKILLS_DIR_OVERRIDES:
result = _get_skills_dir(project_dir, agent_key)
expected = project_dir / AGENT_SKILLS_DIR_OVERRIDES[agent_key]
assert result == expected
# ===== install_ai_skills Tests =====
class TestInstallAiSkills:
"""Test SKILL.md generation and installation logic."""
def test_skills_installed_with_correct_structure(self, project_dir, templates_dir):
"""Verify SKILL.md files have correct agentskills.io structure."""
result = install_ai_skills(project_dir, "claude")
assert result is True
skills_dir = project_dir / ".claude" / "skills"
assert skills_dir.exists()
# Check that skill directories were created
skill_dirs = sorted([d.name for d in skills_dir.iterdir() if d.is_dir()])
assert "speckit-plan" in skill_dirs
assert "speckit-specify" in skill_dirs
assert "speckit-tasks" in skill_dirs
assert "speckit-empty_fm" in skill_dirs
# Verify SKILL.md content for speckit-specify
skill_file = skills_dir / "speckit-specify" / "SKILL.md"
assert skill_file.exists()
content = skill_file.read_text()
# Check agentskills.io frontmatter
assert content.startswith("---\n")
assert "name: speckit-specify" in content
assert "description:" in content
assert "compatibility:" in content
assert "metadata:" in content
assert "author: github-spec-kit" in content
assert "source: templates/commands/specify.md" in content
# Check body content is included
assert "# Speckit Specify Skill" in content
assert "Run this to create a spec." in content
def test_generated_skill_has_parseable_yaml(self, project_dir, templates_dir):
"""Generated SKILL.md should contain valid, parseable YAML frontmatter."""
install_ai_skills(project_dir, "claude")
skill_file = project_dir / ".claude" / "skills" / "speckit-specify" / "SKILL.md"
content = skill_file.read_text()
# Extract and parse frontmatter
assert content.startswith("---\n")
parts = content.split("---", 2)
assert len(parts) >= 3
parsed = yaml.safe_load(parts[1])
assert isinstance(parsed, dict)
assert "name" in parsed
assert parsed["name"] == "speckit-specify"
assert "description" in parsed
def test_empty_yaml_frontmatter(self, project_dir, templates_dir):
"""Templates with empty YAML frontmatter (---\\n---) should not crash."""
result = install_ai_skills(project_dir, "claude")
assert result is True
skill_file = project_dir / ".claude" / "skills" / "speckit-empty_fm" / "SKILL.md"
assert skill_file.exists()
content = skill_file.read_text()
assert "name: speckit-empty_fm" in content
assert "Body with empty frontmatter." in content
def test_enhanced_descriptions_used_when_available(self, project_dir, templates_dir):
"""SKILL_DESCRIPTIONS take precedence over template frontmatter descriptions."""
install_ai_skills(project_dir, "claude")
skill_file = project_dir / ".claude" / "skills" / "speckit-specify" / "SKILL.md"
content = skill_file.read_text()
# Parse the generated YAML to compare the description value
# (yaml.safe_dump may wrap long strings across multiple lines)
parts = content.split("---", 2)
parsed = yaml.safe_load(parts[1])
if "specify" in SKILL_DESCRIPTIONS:
assert parsed["description"] == SKILL_DESCRIPTIONS["specify"]
def test_template_without_frontmatter(self, project_dir, templates_dir):
"""Templates without YAML frontmatter should still produce valid skills."""
install_ai_skills(project_dir, "claude")
skill_file = project_dir / ".claude" / "skills" / "speckit-tasks" / "SKILL.md"
assert skill_file.exists()
content = skill_file.read_text()
# Should still have valid SKILL.md structure
assert "name: speckit-tasks" in content
assert "Body without frontmatter." in content
def test_missing_templates_directory(self, project_dir):
"""Returns False when no command templates exist anywhere."""
# No .claude/commands/ exists, and __file__ fallback won't find anything
fake_init = project_dir / "nonexistent" / "src" / "specify_cli" / "__init__.py"
fake_init.parent.mkdir(parents=True, exist_ok=True)
fake_init.touch()
with patch.object(specify_cli, "__file__", str(fake_init)):
result = install_ai_skills(project_dir, "claude")
assert result is False
# Skills directory should not exist
skills_dir = project_dir / ".claude" / "skills"
assert not skills_dir.exists()
def test_empty_templates_directory(self, project_dir):
"""Returns False when commands directory has no .md files."""
# Create empty .claude/commands/
empty_cmds = project_dir / ".claude" / "commands"
empty_cmds.mkdir(parents=True)
# Block the __file__ fallback so it can't find real templates
fake_init = project_dir / "nowhere" / "src" / "specify_cli" / "__init__.py"
fake_init.parent.mkdir(parents=True, exist_ok=True)
fake_init.touch()
with patch.object(specify_cli, "__file__", str(fake_init)):
result = install_ai_skills(project_dir, "claude")
assert result is False
def test_malformed_yaml_frontmatter(self, project_dir):
"""Malformed YAML in a template should be handled gracefully, not crash."""
# Create .claude/commands/ with a broken template
cmds_dir = project_dir / ".claude" / "commands"
cmds_dir.mkdir(parents=True)
(cmds_dir / "broken.md").write_text(
"---\n"
"description: [unclosed bracket\n"
" invalid: yaml: content: here\n"
"---\n"
"\n"
"# Broken\n",
encoding="utf-8",
)
# Should not raise — errors are caught per-file
result = install_ai_skills(project_dir, "claude")
# The broken template should be skipped but not crash the process
assert result is False
def test_additive_does_not_overwrite_other_files(self, project_dir, templates_dir):
"""Installing skills should not remove non-speckit files in the skills dir."""
# Pre-create a custom skill
custom_dir = project_dir / ".claude" / "skills" / "my-custom-skill"
custom_dir.mkdir(parents=True)
custom_file = custom_dir / "SKILL.md"
custom_file.write_text("# My Custom Skill\n")
install_ai_skills(project_dir, "claude")
# Custom skill should still exist
assert custom_file.exists()
assert custom_file.read_text() == "# My Custom Skill\n"
def test_return_value(self, project_dir, templates_dir):
"""install_ai_skills returns True when skills installed, False otherwise."""
assert install_ai_skills(project_dir, "claude") is True
def test_return_false_when_no_templates(self, project_dir):
"""install_ai_skills returns False when no templates found."""
fake_init = project_dir / "missing" / "src" / "specify_cli" / "__init__.py"
fake_init.parent.mkdir(parents=True, exist_ok=True)
fake_init.touch()
with patch.object(specify_cli, "__file__", str(fake_init)):
assert install_ai_skills(project_dir, "claude") is False
def test_non_md_commands_dir_falls_back(self, project_dir):
"""When extracted commands are .toml (e.g. gemini), fall back to repo templates."""
# Simulate gemini template extraction: .gemini/commands/ with .toml files only
cmds_dir = project_dir / ".gemini" / "commands"
cmds_dir.mkdir(parents=True)
(cmds_dir / "speckit.specify.toml").write_text('[command]\nname = "specify"\n')
(cmds_dir / "speckit.plan.toml").write_text('[command]\nname = "plan"\n')
# The __file__ fallback should find the real repo templates/commands/*.md
result = install_ai_skills(project_dir, "gemini")
assert result is True
skills_dir = project_dir / ".gemini" / "skills"
assert skills_dir.exists()
# Should have installed skills from the fallback .md templates
skill_dirs = [d.name for d in skills_dir.iterdir() if d.is_dir()]
assert len(skill_dirs) >= 1
# .toml commands should be untouched
assert (cmds_dir / "speckit.specify.toml").exists()
@pytest.mark.parametrize("agent_key", list(AGENT_CONFIG.keys()))
def test_skills_install_for_all_agents(self, temp_dir, agent_key):
"""install_ai_skills should produce skills for every configured agent."""
proj = temp_dir / f"proj-{agent_key}"
proj.mkdir()
# Place .md templates in the agent's commands directory
agent_folder = AGENT_CONFIG[agent_key]["folder"]
cmds_dir = proj / agent_folder.rstrip("/") / "commands"
cmds_dir.mkdir(parents=True)
(cmds_dir / "specify.md").write_text(
"---\ndescription: Test command\n---\n\n# Test\n\nBody.\n"
)
result = install_ai_skills(proj, agent_key)
assert result is True
skills_dir = _get_skills_dir(proj, agent_key)
assert skills_dir.exists()
skill_dirs = [d.name for d in skills_dir.iterdir() if d.is_dir()]
assert "speckit-specify" in skill_dirs
assert (skills_dir / "speckit-specify" / "SKILL.md").exists()
class TestCommandCoexistence:
"""Verify install_ai_skills never touches command files.
Cleanup of freshly-extracted commands for NEW projects is handled
in init(), not in install_ai_skills(). These tests confirm that
install_ai_skills leaves existing commands intact.
"""
def test_existing_commands_preserved_claude(self, project_dir, templates_dir, commands_dir_claude):
"""install_ai_skills must NOT remove pre-existing .claude/commands files."""
# Verify commands exist before
assert len(list(commands_dir_claude.glob("speckit.*"))) == 3
install_ai_skills(project_dir, "claude")
# Commands must still be there — install_ai_skills never touches them
remaining = list(commands_dir_claude.glob("speckit.*"))
assert len(remaining) == 3
def test_existing_commands_preserved_gemini(self, project_dir, templates_dir, commands_dir_gemini):
"""install_ai_skills must NOT remove pre-existing .gemini/commands files."""
assert len(list(commands_dir_gemini.glob("speckit.*"))) == 3
install_ai_skills(project_dir, "gemini")
remaining = list(commands_dir_gemini.glob("speckit.*"))
assert len(remaining) == 3
def test_commands_dir_not_removed(self, project_dir, templates_dir, commands_dir_claude):
"""install_ai_skills must not remove the commands directory."""
install_ai_skills(project_dir, "claude")
assert commands_dir_claude.exists()
def test_no_commands_dir_no_error(self, project_dir, templates_dir):
"""No error when installing skills — commands dir has templates and is preserved."""
result = install_ai_skills(project_dir, "claude")
# Should succeed since templates are in .claude/commands/ via fixture
assert result is True
# ===== New-Project Command Skip Tests =====
class TestNewProjectCommandSkip:
"""Test that init() removes extracted commands for new projects only.
These tests run init() end-to-end via CliRunner with
download_and_extract_template patched to create local fixtures.
"""
def _fake_extract(self, agent, project_path, **_kwargs):
"""Simulate template extraction: create agent commands dir."""
agent_cfg = AGENT_CONFIG.get(agent, {})
agent_folder = agent_cfg.get("folder", "")
if agent_folder:
cmds_dir = project_path / agent_folder.rstrip("/") / "commands"
cmds_dir.mkdir(parents=True, exist_ok=True)
(cmds_dir / "speckit.specify.md").write_text("# spec")
def test_new_project_commands_removed_after_skills_succeed(self, tmp_path):
"""For new projects, commands should be removed when skills succeed."""
from typer.testing import CliRunner
runner = CliRunner()
target = tmp_path / "new-proj"
def fake_download(project_path, *args, **kwargs):
self._fake_extract("claude", project_path)
with patch("specify_cli.download_and_extract_template", side_effect=fake_download), \
patch("specify_cli.ensure_executable_scripts"), \
patch("specify_cli.ensure_constitution_from_template"), \
patch("specify_cli.install_ai_skills", return_value=True) as mock_skills, \
patch("specify_cli.is_git_repo", return_value=False), \
patch("specify_cli.shutil.which", return_value="/usr/bin/git"):
result = runner.invoke(app, ["init", str(target), "--ai", "claude", "--ai-skills", "--script", "sh", "--no-git"])
# Skills should have been called
mock_skills.assert_called_once()
# Commands dir should have been removed after skills succeeded
cmds_dir = target / ".claude" / "commands"
assert not cmds_dir.exists()
def test_commands_preserved_when_skills_fail(self, tmp_path):
"""If skills fail, commands should NOT be removed (safety net)."""
from typer.testing import CliRunner
runner = CliRunner()
target = tmp_path / "fail-proj"
def fake_download(project_path, *args, **kwargs):
self._fake_extract("claude", project_path)
with patch("specify_cli.download_and_extract_template", side_effect=fake_download), \
patch("specify_cli.ensure_executable_scripts"), \
patch("specify_cli.ensure_constitution_from_template"), \
patch("specify_cli.install_ai_skills", return_value=False), \
patch("specify_cli.is_git_repo", return_value=False), \
patch("specify_cli.shutil.which", return_value="/usr/bin/git"):
result = runner.invoke(app, ["init", str(target), "--ai", "claude", "--ai-skills", "--script", "sh", "--no-git"])
# Commands should still exist since skills failed
cmds_dir = target / ".claude" / "commands"
assert cmds_dir.exists()
assert (cmds_dir / "speckit.specify.md").exists()
def test_here_mode_commands_preserved(self, tmp_path, monkeypatch):
"""For --here on existing repos, commands must NOT be removed."""
from typer.testing import CliRunner
runner = CliRunner()
# Create a mock existing project with commands already present
target = tmp_path / "existing"
target.mkdir()
agent_folder = AGENT_CONFIG["claude"]["folder"]
cmds_dir = target / agent_folder.rstrip("/") / "commands"
cmds_dir.mkdir(parents=True)
(cmds_dir / "speckit.specify.md").write_text("# spec")
# --here uses CWD, so chdir into the target
monkeypatch.chdir(target)
def fake_download(project_path, *args, **kwargs):
pass # commands already exist, no need to re-create
with patch("specify_cli.download_and_extract_template", side_effect=fake_download), \
patch("specify_cli.ensure_executable_scripts"), \
patch("specify_cli.ensure_constitution_from_template"), \
patch("specify_cli.install_ai_skills", return_value=True), \
patch("specify_cli.is_git_repo", return_value=True), \
patch("specify_cli.shutil.which", return_value="/usr/bin/git"):
result = runner.invoke(app, ["init", "--here", "--ai", "claude", "--ai-skills", "--script", "sh", "--no-git"])
# Commands must remain for --here
assert cmds_dir.exists()
assert (cmds_dir / "speckit.specify.md").exists()
# ===== Skip-If-Exists Tests =====
class TestSkipIfExists:
"""Test that install_ai_skills does not overwrite existing SKILL.md files."""
def test_existing_skill_not_overwritten(self, project_dir, templates_dir):
"""Pre-existing SKILL.md should not be replaced on re-run."""
# Pre-create a custom SKILL.md for speckit-specify
skill_dir = project_dir / ".claude" / "skills" / "speckit-specify"
skill_dir.mkdir(parents=True)
custom_content = "# My Custom Specify Skill\nUser-modified content\n"
(skill_dir / "SKILL.md").write_text(custom_content)
result = install_ai_skills(project_dir, "claude")
# The custom SKILL.md should be untouched
assert (skill_dir / "SKILL.md").read_text() == custom_content
# But other skills should still be installed
assert result is True
assert (project_dir / ".claude" / "skills" / "speckit-plan" / "SKILL.md").exists()
assert (project_dir / ".claude" / "skills" / "speckit-tasks" / "SKILL.md").exists()
def test_fresh_install_writes_all_skills(self, project_dir, templates_dir):
"""On first install (no pre-existing skills), all should be written."""
result = install_ai_skills(project_dir, "claude")
assert result is True
skills_dir = project_dir / ".claude" / "skills"
skill_dirs = [d.name for d in skills_dir.iterdir() if d.is_dir()]
# All 4 templates should produce skills (specify, plan, tasks, empty_fm)
assert len(skill_dirs) == 4
# ===== SKILL_DESCRIPTIONS Coverage Tests =====
class TestSkillDescriptions:
"""Test SKILL_DESCRIPTIONS constants."""
def test_all_known_commands_have_descriptions(self):
"""All standard spec-kit commands should have enhanced descriptions."""
expected_commands = [
"specify", "plan", "tasks", "implement", "analyze",
"clarify", "constitution", "checklist", "taskstoissues",
]
for cmd in expected_commands:
assert cmd in SKILL_DESCRIPTIONS, f"Missing description for '{cmd}'"
assert len(SKILL_DESCRIPTIONS[cmd]) > 20, f"Description for '{cmd}' is too short"
# ===== CLI Validation Tests =====
class TestCliValidation:
"""Test --ai-skills CLI flag validation."""
def test_ai_skills_without_ai_fails(self):
"""--ai-skills without --ai should fail with exit code 1."""
from typer.testing import CliRunner
runner = CliRunner()
result = runner.invoke(app, ["init", "test-proj", "--ai-skills"])
assert result.exit_code == 1
assert "--ai-skills requires --ai" in result.output
def test_ai_skills_without_ai_shows_usage(self):
"""Error message should include usage hint."""
from typer.testing import CliRunner
runner = CliRunner()
result = runner.invoke(app, ["init", "test-proj", "--ai-skills"])
assert "Usage:" in result.output
assert "--ai" in result.output
def test_ai_skills_flag_appears_in_help(self):
"""--ai-skills should appear in init --help output."""
from typer.testing import CliRunner
runner = CliRunner()
result = runner.invoke(app, ["init", "--help"])
assert "--ai-skills" in result.output
assert "agent skills" in result.output.lower()