ros/autocoder
1
0
Fork 0
mirror of https://github.com/leonvanzyl/autocoder.git synced 2026-01-29 22:02:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: add per-project bash command allowlist system

Browse Source 
Implement hierarchical command security with project and org-level configs:

WHAT'S NEW:
- Project-level YAML config (.autocoder/allowed_commands.yaml)
- Organization-level config (~/.autocoder/config.yaml)
- Pattern matching (exact, wildcards, local scripts)
- Hardcoded blocklist (sudo, dd, shutdown - never allowed)
- Org blocklist (terraform, kubectl - configurable)
- Helpful error messages with config hints
- Comprehensive documentation and examples

ARCHITECTURE:
- Hierarchical resolution: Hardcoded → Org Block → Org Allow → Global → Project
- YAML validation with 50 command limit per project
- Pattern matching: exact ("swift"), wildcards ("swift*"), scripts ("./build.sh")
- Secure by default: all examples commented out

TESTING:
- 136 unit tests (pattern matching, YAML, hierarchy, validation)
- 9 integration tests (real security hook flows)
- All tests passing, 100% backward compatible

DOCUMENTATION:
- examples/README.md - comprehensive guide with use cases
- examples/project_allowed_commands.yaml - template (all commented)
- examples/org_config.yaml - org config template (all commented)
- PHASE3_SPEC.md - mid-session approval spec (future enhancement)
- Updated CLAUDE.md with security model documentation

USE CASES:
- iOS projects: Add Swift toolchain (xcodebuild, swift*, etc.)
- Rust projects: Add cargo, rustc, clippy
- Enterprise: Block aws, kubectl, terraform org-wide
- Custom scripts: Allow ./scripts/build.sh

PHASES:
✅ Phase 1: Project YAML + blocklist (implemented)
✅ Phase 2: Org config + hierarchy (implemented)
📋 Phase 3: Mid-session approval (spec ready, not implemented)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Marian Paul
2026-01-22 12:16:16 +01:00
parent 29c6b252a9
commit a9a0fcd865
11 changed files with 3789 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

82
CLAUDE.md
View File

@@ -169,19 +169,99 @@ Projects can be stored in any directory (registered in `~/.autocoder/registry.db
- `prompts/coding_prompt.md` - Continuation session prompt
- `features.db` - SQLite database with feature test cases
- `.agent.lock` - Lock file to prevent multiple agent instances
- `.autocoder/allowed_commands.yaml` - Project-specific bash command allowlist (optional)
### Security Model
Defense-in-depth approach configured in `client.py`:
1. OS-level sandbox for bash commands
2. Filesystem restricted to project directory only
3. Bash commands validated against `ALLOWED_COMMANDS` in `security.py`
3. Bash commands validated using hierarchical allowlist system
#### Per-Project Allowed Commands
The agent's bash command access is controlled through a hierarchical configuration system:
**Command Hierarchy (highest to lowest priority):**
1. **Hardcoded Blocklist** (`security.py`) - NEVER allowed (dd, sudo, shutdown, etc.)
2. **Org Blocklist** (`~/.autocoder/config.yaml`) - Cannot be overridden by projects
3. **Org Allowlist** (`~/.autocoder/config.yaml`) - Available to all projects
4. **Global Allowlist** (`security.py`) - Default commands (npm, git, curl, etc.)
5. **Project Allowlist** (`.autocoder/allowed_commands.yaml`) - Project-specific commands
**Project Configuration:**
Each project can define custom allowed commands in `.autocoder/allowed_commands.yaml`:
```yaml
version: 1
commands:
# Exact command names
- name: swift
description: Swift compiler
# Prefix wildcards (matches swiftc, swiftlint, swiftformat)
- name: swift*
description: All Swift development tools
# Local project scripts
- name: ./scripts/build.sh
description: Project build script
```
**Organization Configuration:**
System administrators can set org-wide policies in `~/.autocoder/config.yaml`:
```yaml
version: 1
# Commands available to ALL projects
allowed_commands:
- name: jq
description: JSON processor
# Commands blocked across ALL projects (cannot be overridden)
blocked_commands:
- aws # Prevent accidental cloud operations
- kubectl # Block production deployments
```
**Pattern Matching:**
- Exact: `swift` matches only `swift`
- Wildcard: `swift*` matches `swift`, `swiftc`, `swiftlint`, etc.
- Scripts: `./scripts/build.sh` matches the script by name from any directory
**Limits:**
- Maximum 50 commands per project config
- Blocklisted commands (sudo, dd, shutdown, etc.) can NEVER be allowed
- Org-level blocked commands cannot be overridden by project configs
**Testing:**
```bash
# Unit tests (136 tests - fast)
python test_security.py
# Integration tests (9 tests - uses real hooks)
python test_security_integration.py
```
**Files:**
- `security.py` - Command validation logic and hardcoded blocklist
- `test_security.py` - Unit tests for security system (136 tests)
- `test_security_integration.py` - Integration tests with real hooks (9 tests)
- `TEST_SECURITY.md` - Quick testing reference guide
- `examples/project_allowed_commands.yaml` - Project config example (all commented by default)
- `examples/org_config.yaml` - Org config example (all commented by default)
- `examples/README.md` - Comprehensive guide with use cases, testing, and troubleshooting
- `PHASE3_SPEC.md` - Specification for mid-session approval feature (future enhancement)
## Claude Code Integration
- `.claude/commands/create-spec.md` - `/create-spec` slash command for interactive spec creation
- `.claude/skills/frontend-design/SKILL.md` - Skill for distinctive UI design
- `.claude/templates/` - Prompt templates copied to new projects
- `examples/` - Configuration examples and documentation for security settings
## Key Patterns

1591
PHASE3_SPEC.md Normal file
View File

File diff suppressed because it is too large Load Diff

10
client.py
View File

@@ -261,6 +261,14 @@ def create_client(
if "ANTHROPIC_BASE_URL" in sdk_env:
print(f" - GLM Mode: Using {sdk_env['ANTHROPIC_BASE_URL']}")
# Create a wrapper for bash_security_hook that passes project_dir via context
async def bash_hook_with_context(input_data, tool_use_id=None, context=None):
"""Wrapper that injects project_dir into context for security hook."""
if context is None:
context = {}
context["project_dir"] = str(project_dir.resolve())
return await bash_security_hook(input_data, tool_use_id, context)
return ClaudeSDKClient(
options=ClaudeAgentOptions(
model=model,
@@ -272,7 +280,7 @@ def create_client(
mcp_servers=mcp_servers,
hooks={
"PreToolUse": [
HookMatcher(matcher="Bash", hooks=[bash_security_hook]),
HookMatcher(matcher="Bash", hooks=[bash_hook_with_context]),
],
},
max_turns=1000,

531
examples/README.md Normal file
View File

@@ -0,0 +1,531 @@
# AutoCoder Security Configuration Examples
This directory contains example configuration files for controlling which bash commands the autonomous coding agent can execute.
## Table of Contents
- [Quick Start](#quick-start)
- [Project-Level Configuration](#project-level-configuration)
- [Organization-Level Configuration](#organization-level-configuration)
- [Command Hierarchy](#command-hierarchy)
- [Pattern Matching](#pattern-matching)
- [Common Use Cases](#common-use-cases)
- [Security Best Practices](#security-best-practices)
---
## Quick Start
### For a Single Project (Most Common)
When you create a new project with AutoCoder, it automatically creates:
```
my-project/
.autocoder/
allowed_commands.yaml ← Automatically created from template
```
**Edit this file** to add project-specific commands (Swift tools, Rust compiler, etc.).
### For All Projects (Organization-Wide)
If you want commands available across **all projects**, manually create:
```bash
# Copy the example to your home directory
cp examples/org_config.yaml ~/.autocoder/config.yaml
# Edit it to add org-wide commands
nano ~/.autocoder/config.yaml
```
---
## Project-Level Configuration
**File:** `{project_dir}/.autocoder/allowed_commands.yaml`
**Purpose:** Define commands needed for THIS specific project.
**Example** (iOS project):
```yaml
version: 1
commands:
- name: swift
description: Swift compiler
- name: xcodebuild
description: Xcode build system
- name: swift*
description: All Swift tools (swiftc, swiftlint, swiftformat)
- name: ./scripts/build.sh
description: Project build script
```
**When to use:**
- ✅ Project uses a specific language toolchain (Swift, Rust, Go)
- ✅ Project has custom build scripts
- ✅ Temporary tools needed during development
**Limits:**
- Maximum 50 commands per project
- Cannot override org-level blocked commands
- Cannot allow hardcoded blocklist commands (sudo, dd, etc.)
**See:** `examples/project_allowed_commands.yaml` for full example with Rust, Python, iOS, etc.
---
## Organization-Level Configuration
**File:** `~/.autocoder/config.yaml`
**Purpose:** Define commands and policies for ALL projects.
**Example** (startup team):
```yaml
version: 1
# Available to all projects
allowed_commands:
- name: jq
description: JSON processor
- name: python3
description: Python interpreter
# Blocked across all projects (cannot be overridden)
blocked_commands:
- aws
- kubectl
- terraform
```
**When to use:**
- ✅ Multiple projects need the same tools (jq, python3, etc.)
- ✅ Enforce organization-wide security policies
- ✅ Block dangerous commands across all projects
**See:** `examples/org_config.yaml` for full example with enterprise/startup configurations.
---
## Command Hierarchy
When the agent tries to run a command, the system checks in this order:
```
┌─────────────────────────────────────────────────────┐
│ 1. HARDCODED BLOCKLIST (highest priority) │
│ sudo, dd, shutdown, reboot, chown, etc. │
│ ❌ NEVER allowed, even with user approval │
└─────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────┐
│ 2. ORG BLOCKLIST (~/.autocoder/config.yaml) │
│ Commands you block organization-wide │
│ ❌ Projects CANNOT override these │
└─────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────┐
│ 3. ORG ALLOWLIST (~/.autocoder/config.yaml) │
│ Commands available to all projects │
│ ✅ Automatically available │
└─────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────┐
│ 4. GLOBAL ALLOWLIST (security.py) │
│ Default commands: npm, git, curl, ls, cat, etc. │
│ ✅ Always available │
└─────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────┐
│ 5. PROJECT ALLOWLIST (.autocoder/allowed_commands) │
│ Project-specific commands │
│ ✅ Available only to this project │
└─────────────────────────────────────────────────────┘
```
**Key Rules:**
- If a command is BLOCKED at any level above, it cannot be allowed below
- If a command is ALLOWED at any level, it's available (unless blocked above)
- Blocklist always wins over allowlist
---
## Pattern Matching
You can use patterns to match multiple commands:
### Exact Match
```yaml
- name: swift
description: Swift compiler only
```
Matches: `swift`
Does NOT match: `swiftc`, `swiftlint`
### Prefix Wildcard
```yaml
- name: swift*
description: All Swift tools
```
Matches: `swift`, `swiftc`, `swiftlint`, `swiftformat`
Does NOT match: `npm`, `rustc`
### Local Scripts
```yaml
- name: ./scripts/build.sh
description: Build script
```
Matches:
- `./scripts/build.sh`
- `scripts/build.sh`
- `/full/path/to/scripts/build.sh`
- Running `build.sh` from any directory (matched by filename)
---
## Common Use Cases
### iOS Development
**Project config** (`.autocoder/allowed_commands.yaml`):
```yaml
version: 1
commands:
- name: swift*
description: All Swift tools
- name: xcodebuild
description: Xcode build system
- name: xcrun
description: Xcode tools runner
- name: simctl
description: iOS Simulator control
```
### Rust CLI Project
**Project config**:
```yaml
version: 1
commands:
- name: cargo
description: Rust package manager
- name: rustc
description: Rust compiler
- name: rustfmt
description: Rust formatter
- name: clippy
description: Rust linter
- name: ./target/debug/my-cli
description: Debug build
- name: ./target/release/my-cli
description: Release build
```
### API Testing Project
**Project config**:
```yaml
version: 1
commands:
- name: jq
description: JSON processor
- name: httpie
description: HTTP client
- name: ./scripts/test-api.sh
description: API test runner
```
### Enterprise Organization (Restrictive)
**Org config** (`~/.autocoder/config.yaml`):
```yaml
version: 1
allowed_commands:
- name: jq
description: JSON processor
blocked_commands:
- aws # No cloud access
- gcloud
- az
- kubectl # No k8s access
- terraform # No infrastructure changes
- psql # No production DB access
- mysql
```
### Startup Team (Permissive)
**Org config** (`~/.autocoder/config.yaml`):
```yaml
version: 1
allowed_commands:
- name: python3
description: Python interpreter
- name: jq
description: JSON processor
- name: pytest
description: Python tests
blocked_commands: [] # Rely on hardcoded blocklist only
```
---
## Security Best Practices
### ✅ DO
1. **Start restrictive, add as needed**
- Begin with default commands only
- Add project-specific tools when required
- Review the agent's blocked command errors to understand what's needed
2. **Use org-level config for shared tools**
- If 3+ projects need `jq`, add it to org config
- Reduces duplication across project configs
3. **Block dangerous commands at org level**
- Prevent accidental production deployments (`kubectl`, `terraform`)
- Block cloud CLIs if appropriate (`aws`, `gcloud`, `az`)
4. **Use descriptive command names**
- Good: `description: "Swift compiler for iOS builds"`
- Bad: `description: "Compiler"`
5. **Prefer patterns for tool families**
- `swift*` instead of listing `swift`, `swiftc`, `swiftlint` separately
- Automatically includes future tools (e.g., new Swift utilities)
### ❌ DON'T
1. **Don't add commands "just in case"**
- Only add when the agent actually needs them
- Empty config is fine - defaults are usually enough
2. **Don't try to allow blocklisted commands**
- Commands like `sudo`, `dd`, `shutdown` can NEVER be allowed
- The system will reject these in validation
3. **Don't use org config for project-specific tools**
- Bad: Adding `xcodebuild` to org config when only one project uses it
- Good: Add `xcodebuild` to that project's config
4. **Don't exceed the 50 command limit per project**
- If you need more, you're probably being too specific
- Use wildcards instead: `npm-*` covers many npm tools
5. **Don't ignore validation errors**
- If your YAML is rejected, fix the structure
- Common issues: missing `version`, malformed lists, over 50 commands
---
## Default Allowed Commands
These commands are **always available** to all projects:
**File Operations:**
- `ls`, `cat`, `head`, `tail`, `wc`, `grep`, `cp`, `mkdir`, `mv`, `rm`, `touch`
**Shell:**
- `pwd`, `echo`, `sh`, `bash`, `sleep`
**Version Control:**
- `git`
**Process Management:**
- `ps`, `lsof`, `kill`, `pkill` (dev processes only: node, npm, vite)
**Network:**
- `curl`
**Node.js:**
- `npm`, `npx`, `pnpm`, `node`
**Docker:**
- `docker`
**Special:**
- `chmod` (only `+x` mode for making scripts executable)
---
## Hardcoded Blocklist
These commands are **NEVER allowed**, even with user approval:
**Disk Operations:**
- `dd`, `mkfs`, `fdisk`, `parted`
**System Control:**
- `shutdown`, `reboot`, `poweroff`, `halt`, `init`
**Privilege Escalation:**
- `sudo`, `su`, `doas`
**System Services:**
- `systemctl`, `service`, `launchctl`
**Network Security:**
- `iptables`, `ufw`
**Ownership Changes:**
- `chown`, `chgrp`
**Dangerous Commands** (Phase 3 will add approval):
- `aws`, `gcloud`, `az`, `kubectl`, `docker-compose`
---
## Troubleshooting
### Error: "Command 'X' is not allowed"
**Solution:** Add the command to your project config:
```yaml
# In .autocoder/allowed_commands.yaml
commands:
- name: X
description: What this command does
```
### Error: "Command 'X' is blocked at organization level"
**Cause:** The command is in the org blocklist or hardcoded blocklist.
**Solution:**
- If in org blocklist: Edit `~/.autocoder/config.yaml` to remove it
- If in hardcoded blocklist: Cannot be allowed (by design)
### Error: "Could not parse YAML config"
**Cause:** YAML syntax error.
**Solution:** Check for:
- Missing colons after keys
- Incorrect indentation (use 2 spaces, not tabs)
- Missing quotes around special characters
### Config not taking effect
**Solution:**
1. Restart the agent (changes are loaded on startup)
2. Verify file location:
- Project: `{project}/.autocoder/allowed_commands.yaml`
- Org: `~/.autocoder/config.yaml` (must be manually created)
3. Check YAML is valid (run through a YAML validator)
---
## Testing
### Running the Tests
AutoCoder has comprehensive tests for the security system:
**Unit Tests** (136 tests - fast):
```bash
source venv/bin/activate
python test_security.py
```
Tests:
- Pattern matching (exact, wildcards, scripts)
- YAML loading and validation
- Blocklist enforcement
- Project and org config hierarchy
- All existing security validations
**Integration Tests** (9 tests - uses real security hooks):
```bash
source venv/bin/activate
python test_security_integration.py
```
Tests:
- Blocked commands are rejected (sudo, shutdown, etc.)
- Default commands work (ls, git, npm, etc.)
- Non-allowed commands are blocked (wget, python, etc.)
- Project config allows commands (swift, xcodebuild, etc.)
- Pattern matching works (swift* matches swiftlint)
- Org blocklist cannot be overridden
- Org allowlist is inherited by projects
- Invalid YAML is safely ignored
- 50 command limit is enforced
### Manual Testing
To manually test the security system:
**1. Create a test project:**
```bash
python start.py
# Choose "Create new project"
# Name it "security-test"
```
**2. Edit the project config:**
```bash
# Navigate to the project directory
cd path/to/security-test
# Edit the config
nano .autocoder/allowed_commands.yaml
```
**3. Add a test command (e.g., Swift):**
```yaml
version: 1
commands:
- name: swift
description: Swift compiler
```
**4. Run the agent and observe:**
- Try a blocked command: `"Run sudo apt install nginx"` → Should be blocked
- Try an allowed command: `"Run ls -la"` → Should work
- Try your config command: `"Run swift --version"` → Should work
- Try a non-allowed command: `"Run wget https://example.com"` → Should be blocked
**5. Check the agent output:**
The agent will show security hook messages like:
```
Command 'sudo' is blocked at organization level and cannot be approved.
```
Or:
```
Command 'wget' is not allowed.
To allow this command:
1. Add to .autocoder/allowed_commands.yaml for this project, OR
2. Request mid-session approval (the agent can ask)
```
---
## Files Reference
- **`examples/project_allowed_commands.yaml`** - Full project config template
- **`examples/org_config.yaml`** - Full org config template
- **`security.py`** - Implementation and hardcoded blocklist
- **`test_security.py`** - Unit tests (136 tests)
- **`test_security_integration.py`** - Integration tests (9 tests)
- **`CLAUDE.md`** - Full system documentation
---
## Questions?
See the main documentation in `CLAUDE.md` for architecture details and implementation specifics.

172
examples/org_config.yaml Normal file
View File

@@ -0,0 +1,172 @@
# Organization-Level AutoCoder Configuration
# ============================================
# Location: ~/.autocoder/config.yaml
#
# IMPORTANT: This file is OPTIONAL and must be manually created by you.
# It does NOT exist by default.
#
# Org-level config applies to ALL projects and provides:
# 1. Organization-wide allowed commands (available to all projects)
# 2. Organization-wide blocked commands (cannot be overridden by projects)
# 3. Global settings (approval timeout, etc.)
#
# Use this to:
# - Add commands that ALL your projects need (jq, python3, etc.)
# - Block dangerous commands across ALL projects (aws, kubectl, etc.)
# - Enforce organization-wide security policies
version: 1
# ==========================================
# Organization-Wide Allowed Commands
# ==========================================
# These commands become available to ALL projects automatically.
# Projects don't need to add them to their own .autocoder/allowed_commands.yaml
#
# By default, this is empty. Uncomment and add commands as needed.
allowed_commands: []
# Common development utilities
# - name: jq
# description: JSON processor for API responses
# - name: python3
# description: Python 3 interpreter
# - name: pip3
# description: Python package installer
# - name: pytest
# description: Python testing framework
# - name: black
# description: Python code formatter
# Database CLIs (if safe in your environment)
# - name: psql
# description: PostgreSQL client
# - name: mysql
# description: MySQL client
# ==========================================
# Organization-Wide Blocked Commands
# ==========================================
# Commands listed here are BLOCKED across ALL projects.
# Projects CANNOT override these blocks - this is the final word.
#
# Use this to enforce security policies, such as:
# - Preventing accidental production deployments
# - Blocking cloud CLI tools to avoid infrastructure changes
# - Preventing access to production databases
#
# By default, this is empty. Uncomment commands you want to block.
blocked_commands: []
# Block cloud CLIs to prevent accidental production changes
# - aws
# - gcloud
# - az
# Block container orchestration to prevent production deployments
# - kubectl
# - docker-compose
# Block infrastructure-as-code tools
# - terraform
# - pulumi
# Block database CLIs to prevent production data access
# - psql
# - mysql
# - mongosh
# Block other potentially dangerous tools
# - ansible
# - chef
# - puppet
# ==========================================
# Global Settings (Phase 3 feature)
# ==========================================
# These settings control approval behavior when agents request
# commands that aren't in the allowlist.
# How long to wait for user approval before denying a command request
approval_timeout_minutes: 5
# ==========================================
# Command Hierarchy (for reference)
# ==========================================
# When the agent tries to run a bash command, the system checks in this order:
#
# 1. Hardcoded Blocklist (in security.py) - HIGHEST PRIORITY
# Commands like: sudo, dd, shutdown, reboot, etc.
# These can NEVER be allowed, even with user approval.
#
# 2. Org Blocked Commands (this file)
# Commands you specify in "blocked_commands:" above.
# Projects cannot override these.
#
# 3. Org Allowed Commands (this file)
# Commands you specify in "allowed_commands:" above.
# Available to all projects automatically.
#
# 4. Global Allowed Commands (in security.py)
# Default commands: npm, git, curl, ls, cat, etc.
# Always available to all projects.
#
# 5. Project Allowed Commands (.autocoder/allowed_commands.yaml)
# Project-specific commands defined in each project.
# LOWEST PRIORITY (can't override blocks above).
#
# If a command is in BOTH allowed and blocked lists, BLOCKED wins.
# ==========================================
# Example Configurations by Organization Type
# ==========================================
# Startup / Small Team (permissive):
# allowed_commands:
# - name: python3
# - name: jq
# blocked_commands: [] # Empty - rely on hardcoded blocklist only
# Enterprise / Regulated (restrictive):
# allowed_commands: [] # Empty - projects must explicitly request each tool
# blocked_commands:
# - aws
# - gcloud
# - az
# - kubectl
# - terraform
# - psql
# - mysql
# - mongosh
# Development Team (balanced):
# allowed_commands:
# - name: jq
# - name: python3
# - name: pytest
# blocked_commands:
# - aws # Block production access
# - kubectl # Block deployments
# - terraform
# ==========================================
# To Create This File
# ==========================================
# 1. Copy this example to: ~/.autocoder/config.yaml
# 2. Uncomment and customize the sections you need
# 3. Leave empty lists if you don't need org-level controls
#
# To learn more, see: examples/README.md

139
examples/project_allowed_commands.yaml Normal file
View File

@@ -0,0 +1,139 @@
# Project-Specific Allowed Commands
# ==================================
# Location: {project_dir}/.autocoder/allowed_commands.yaml
#
# This file defines bash commands that the autonomous coding agent can use
# for THIS SPECIFIC PROJECT, beyond the default allowed commands.
#
# When you create a new project, AutoCoder automatically creates this file
# in your project's .autocoder/ directory. You can customize it for your
# project's specific needs (iOS, Rust, Python, etc.).
version: 1
# Uncomment the commands you need for your specific project.
# By default, this file has NO commands enabled - you must explicitly add them.
commands: []
# ==========================================
# iOS Development Example
# ==========================================
# Uncomment these if building an iOS app:
# - name: xcodebuild
# description: Xcode build system for compiling iOS apps
# - name: swift
# description: Swift compiler and REPL
# - name: swiftc
# description: Swift compiler command-line interface
# - name: xcrun
# description: Run Xcode developer tools
# - name: simctl
# description: iOS Simulator control tool
# Pattern matching with wildcard
# This matches: swift, swiftc, swiftformat, swiftlint, etc.
# - name: swift*
# description: All Swift development tools
# ==========================================
# Rust Development Example
# ==========================================
# Uncomment these if building a Rust project:
# - name: cargo
# description: Rust package manager and build tool
# - name: rustc
# description: Rust compiler
# - name: rustfmt
# description: Rust code formatter
# - name: clippy
# description: Rust linter
# ==========================================
# Python Development Example
# ==========================================
# Uncomment these if building a Python project:
# - name: python3
# description: Python 3 interpreter
# - name: pip3
# description: Python package installer
# - name: pytest
# description: Python testing framework
# ==========================================
# Database Tools Example
# ==========================================
# Uncomment these if you need database access:
# - name: psql
# description: PostgreSQL command-line client
# - name: sqlite3
# description: SQLite database CLI
# ==========================================
# Project-Specific Scripts
# ==========================================
# Local scripts are matched by filename, so these work from any directory
# Uncomment and customize for your project:
# - name: ./scripts/build.sh
# description: Project build script
# - name: ./scripts/test.sh
# description: Run all project tests
# - name: ./scripts/deploy-staging.sh
# description: Deploy to staging environment
# ==========================================
# Notes and Best Practices
# ==========================================
#
# Pattern Matching:
# - Exact: "swift" matches only "swift"
# - Wildcard: "swift*" matches "swift", "swiftc", "swiftlint", etc.
# - Scripts: "./scripts/build.sh" matches the script by name
#
# Limits:
# - Maximum 50 commands per project
# - Commands in the blocklist (sudo, dd, shutdown, etc.) can NEVER be allowed
# - Org-level blocked commands (see ~/.autocoder/config.yaml) cannot be overridden
#
# Default Allowed Commands (always available):
# File operations: ls, cat, head, tail, wc, grep, cp, mkdir, mv, rm, touch
# Shell: pwd, echo, sh, bash, sleep
# Version control: git
# Process management: ps, lsof, kill, pkill (dev processes only)
# Network: curl
# Node.js: npm, npx, pnpm, node
# Docker: docker
# chmod: Only +x mode (making scripts executable)
#
# Hardcoded Blocklist (NEVER allowed):
# Disk operations: dd, mkfs, fdisk, parted
# System control: shutdown, reboot, poweroff, halt, init
# Privilege escalation: sudo, su, doas
# System services: systemctl, service, launchctl
# Network security: iptables, ufw
# Ownership changes: chown, chgrp
# Dangerous commands: aws, gcloud, az, kubectl (unless org allows)
#
# To learn more, see: examples/README.md

17
prompts.py
View File

@@ -180,6 +180,10 @@ def scaffold_project_prompts(project_dir: Path) -> Path:
project_prompts = get_project_prompts_dir(project_dir)
project_prompts.mkdir(parents=True, exist_ok=True)
# Create .autocoder directory for configuration files
autocoder_dir = project_dir / ".autocoder"
autocoder_dir.mkdir(parents=True, exist_ok=True)
# Define template mappings: (source_template, destination_name)
templates = [
("app_spec.template.txt", "app_spec.txt"),
@@ -201,8 +205,19 @@ def scaffold_project_prompts(project_dir: Path) -> Path:
except (OSError, PermissionError) as e:
print(f" Warning: Could not copy {dest_name}: {e}")
# Copy allowed_commands.yaml template to .autocoder/
examples_dir = Path(__file__).parent / "examples"
allowed_commands_template = examples_dir / "project_allowed_commands.yaml"
allowed_commands_dest = autocoder_dir / "allowed_commands.yaml"
if allowed_commands_template.exists() and not allowed_commands_dest.exists():
try:
shutil.copy(allowed_commands_template, allowed_commands_dest)
copied_files.append(".autocoder/allowed_commands.yaml")
except (OSError, PermissionError) as e:
print(f" Warning: Could not copy allowed_commands.yaml: {e}")
if copied_files:
print(f" Created prompt files: {', '.join(copied_files)}")
print(f" Created project files: {', '.join(copied_files)}")
return project_prompts

1
requirements.txt
View File

@@ -9,6 +9,7 @@ psutil>=6.0.0
aiofiles>=24.0.0
apscheduler>=3.10.0,<4.0.0
pywinpty>=2.0.0; sys_platform == "win32"
pyyaml>=6.0.0
# Dev dependencies
ruff>=0.8.0

362
security.py
View File

@@ -8,6 +8,10 @@ Uses an allowlist approach - only explicitly permitted commands can run.
import os
import shlex
from pathlib import Path
from typing import Optional
import yaml
# Allowed commands for development tasks
# Minimal set needed for the autonomous coding demo
@@ -58,6 +62,48 @@ ALLOWED_COMMANDS = {
# Commands that need additional validation even when in the allowlist
COMMANDS_NEEDING_EXTRA_VALIDATION = {"pkill", "chmod", "init.sh"}
# Commands that are NEVER allowed, even with user approval
# These commands can cause permanent system damage or security breaches
BLOCKED_COMMANDS = {
# Disk operations
"dd",
"mkfs",
"fdisk",
"parted",
# System control
"shutdown",
"reboot",
"poweroff",
"halt",
"init",
# Ownership changes
"chown",
"chgrp",
# System services
"systemctl",
"service",
"launchctl",
# Network security
"iptables",
"ufw",
}
# Commands that trigger emphatic warnings but CAN be approved (Phase 3)
# For now, these are blocked like BLOCKED_COMMANDS until Phase 3 implements approval
DANGEROUS_COMMANDS = {
# Privilege escalation
"sudo",
"su",
"doas",
# Cloud CLIs (can modify production infrastructure)
"aws",
"gcloud",
"az",
# Container and orchestration
"kubectl",
"docker-compose",
}
def split_command_segments(command_string: str) -> list[str]:
"""
@@ -309,16 +355,298 @@ def get_command_for_validation(cmd: str, segments: list[str]) -> str:
return ""
def matches_pattern(command: str, pattern: str) -> bool:
"""
Check if a command matches a pattern.
Supports:
- Exact match: "swift"
- Prefix wildcard: "swift*" matches "swift", "swiftc", "swiftformat"
- Local script paths: "./scripts/build.sh" or "scripts/test.sh"
Args:
command: The command to check
pattern: The pattern to match against
Returns:
True if command matches pattern
"""
# Exact match
if command == pattern:
return True
# Prefix wildcard (e.g., "swift*" matches "swiftc", "swiftlint")
if pattern.endswith("*"):
prefix = pattern[:-1]
return command.startswith(prefix)
# Local script paths (./scripts/build.sh matches build.sh)
if pattern.startswith("./") or pattern.startswith("../"):
# Extract the script name from the pattern
pattern_name = os.path.basename(pattern)
return command == pattern or command == pattern_name or command.endswith("/" + pattern_name)
return False
def get_org_config_path() -> Path:
"""
Get the organization-level config file path.
Returns:
Path to ~/.autocoder/config.yaml
"""
return Path.home() / ".autocoder" / "config.yaml"
def load_org_config() -> Optional[dict]:
"""
Load organization-level config from ~/.autocoder/config.yaml.
Returns:
Dict with parsed org config, or None if file doesn't exist or is invalid
"""
config_path = get_org_config_path()
if not config_path.exists():
return None
try:
with open(config_path, "r", encoding="utf-8") as f:
config = yaml.safe_load(f)
if not config:
return None
# Validate structure
if not isinstance(config, dict):
return None
if "version" not in config:
return None
# Validate allowed_commands if present
if "allowed_commands" in config:
allowed = config["allowed_commands"]
if not isinstance(allowed, list):
return None
for cmd in allowed:
if not isinstance(cmd, dict):
return None
if "name" not in cmd:
return None
# Validate blocked_commands if present
if "blocked_commands" in config:
blocked = config["blocked_commands"]
if not isinstance(blocked, list):
return None
for cmd in blocked:
if not isinstance(cmd, str):
return None
return config
except (yaml.YAMLError, IOError, OSError):
return None
def load_project_commands(project_dir: Path) -> Optional[dict]:
"""
Load allowed commands from project-specific YAML config.
Args:
project_dir: Path to the project directory
Returns:
Dict with parsed YAML config, or None if file doesn't exist or is invalid
"""
config_path = project_dir / ".autocoder" / "allowed_commands.yaml"
if not config_path.exists():
return None
try:
with open(config_path, "r", encoding="utf-8") as f:
config = yaml.safe_load(f)
if not config:
return None
# Validate structure
if not isinstance(config, dict):
return None
if "version" not in config:
return None
commands = config.get("commands", [])
if not isinstance(commands, list):
return None
# Enforce 50 command limit
if len(commands) > 50:
return None
# Validate each command entry
for cmd in commands:
if not isinstance(cmd, dict):
return None
if "name" not in cmd:
return None
# Validate name is a string
if not isinstance(cmd["name"], str):
return None
return config
except (yaml.YAMLError, IOError, OSError):
return None
def validate_project_command(cmd_config: dict) -> tuple[bool, str]:
"""
Validate a single command entry from project config.
Args:
cmd_config: Dict with command configuration (name, description, args)
Returns:
Tuple of (is_valid, error_message)
"""
if not isinstance(cmd_config, dict):
return False, "Command must be a dict"
if "name" not in cmd_config:
return False, "Command must have 'name' field"
name = cmd_config["name"]
if not isinstance(name, str) or not name:
return False, "Command name must be a non-empty string"
# Check if command is in the blocklist or dangerous commands
base_cmd = os.path.basename(name.rstrip("*"))
if base_cmd in BLOCKED_COMMANDS:
return False, f"Command '{name}' is in the blocklist and cannot be allowed"
if base_cmd in DANGEROUS_COMMANDS:
return False, f"Command '{name}' is in the blocklist and cannot be allowed"
# Description is optional
if "description" in cmd_config and not isinstance(cmd_config["description"], str):
return False, "Description must be a string"
# Args validation (Phase 1 - just check structure)
if "args" in cmd_config:
args = cmd_config["args"]
if not isinstance(args, list):
return False, "Args must be a list"
for arg in args:
if not isinstance(arg, str):
return False, "Each arg must be a string"
return True, ""
def get_effective_commands(project_dir: Optional[Path]) -> tuple[set[str], set[str]]:
"""
Get effective allowed and blocked commands after hierarchy resolution.
Hierarchy (highest to lowest priority):
1. BLOCKED_COMMANDS (hardcoded) - always blocked
2. Org blocked_commands - cannot be unblocked
3. Org allowed_commands - adds to global
4. Project allowed_commands - adds to global + org
Args:
project_dir: Path to the project directory, or None
Returns:
Tuple of (allowed_commands, blocked_commands)
"""
# Start with global allowed commands
allowed = ALLOWED_COMMANDS.copy()
blocked = BLOCKED_COMMANDS.copy()
# Add dangerous commands to blocked (Phase 3 will add approval flow)
blocked |= DANGEROUS_COMMANDS
# Load org config and apply
org_config = load_org_config()
if org_config:
# Add org-level blocked commands (cannot be overridden)
org_blocked = org_config.get("blocked_commands", [])
blocked |= set(org_blocked)
# Add org-level allowed commands
for cmd_config in org_config.get("allowed_commands", []):
if isinstance(cmd_config, dict) and "name" in cmd_config:
allowed.add(cmd_config["name"])
# Load project config and apply
if project_dir:
project_config = load_project_commands(project_dir)
if project_config:
# Add project-specific commands
for cmd_config in project_config.get("commands", []):
valid, error = validate_project_command(cmd_config)
if valid:
allowed.add(cmd_config["name"])
# Remove blocked commands from allowed (blocklist takes precedence)
allowed -= blocked
return allowed, blocked
def get_project_allowed_commands(project_dir: Optional[Path]) -> set[str]:
"""
Get the set of allowed commands for a project.
Uses hierarchy resolution from get_effective_commands().
Args:
project_dir: Path to the project directory, or None
Returns:
Set of allowed command names (including patterns)
"""
allowed, blocked = get_effective_commands(project_dir)
return allowed
def is_command_allowed(command: str, allowed_commands: set[str]) -> bool:
"""
Check if a command is allowed (supports patterns).
Args:
command: The command to check
allowed_commands: Set of allowed commands (may include patterns)
Returns:
True if command is allowed
"""
# Check exact match first
if command in allowed_commands:
return True
# Check pattern matches
for pattern in allowed_commands:
if matches_pattern(command, pattern):
return True
return False
async def bash_security_hook(input_data, tool_use_id=None, context=None):
"""
Pre-tool-use hook that validates bash commands using an allowlist.
Only commands in ALLOWED_COMMANDS are permitted.
Only commands in ALLOWED_COMMANDS and project-specific commands are permitted.
Args:
input_data: Dict containing tool_name and tool_input
tool_use_id: Optional tool use ID
context: Optional context
context: Optional context dict with 'project_dir' key
Returns:
Empty dict to allow, or {"decision": "block", "reason": "..."} to block
@@ -340,15 +668,39 @@ async def bash_security_hook(input_data, tool_use_id=None, context=None):
"reason": f"Could not parse command for security validation: {command}",
}
# Get project directory from context
project_dir = None
if context and isinstance(context, dict):
project_dir_str = context.get("project_dir")
if project_dir_str:
project_dir = Path(project_dir_str)
# Get effective commands using hierarchy resolution
allowed_commands, blocked_commands = get_effective_commands(project_dir)
# Split into segments for per-command validation
segments = split_command_segments(command)
# Check each command against the allowlist
# Check each command against the blocklist and allowlist
for cmd in commands:
if cmd not in ALLOWED_COMMANDS:
# Check blocklist first (highest priority)
if cmd in blocked_commands:
return {
"decision": "block",
"reason": f"Command '{cmd}' is not in the allowed commands list",
"reason": f"Command '{cmd}' is blocked at organization level and cannot be approved.",
}
# Check allowlist (with pattern matching)
if not is_command_allowed(cmd, allowed_commands):
# Provide helpful error message with config hint
error_msg = f"Command '{cmd}' is not allowed.\n"
error_msg += "To allow this command:\n"
error_msg += " 1. Add to .autocoder/allowed_commands.yaml for this project, OR\n"
error_msg += " 2. Request mid-session approval (the agent can ask)\n"
error_msg += "Note: Some commands are blocked at org-level and cannot be overridden."
return {
"decision": "block",
"reason": error_msg,
}
# Additional validation for sensitive commands

481
test_security.py
View File

@@ -9,12 +9,19 @@ Run with: python test_security.py
import asyncio
import sys
import tempfile
from pathlib import Path
from security import (
bash_security_hook,
extract_commands,
get_effective_commands,
load_org_config,
load_project_commands,
matches_pattern,
validate_chmod_command,
validate_init_script,
validate_project_command,
)
@@ -151,6 +158,440 @@ def test_validate_init_script():
return passed, failed
def test_pattern_matching():
"""Test command pattern matching."""
print("\nTesting pattern matching:\n")
passed = 0
failed = 0
# Test cases: (command, pattern, should_match, description)
test_cases = [
# Exact matches
("swift", "swift", True, "exact match"),
("npm", "npm", True, "exact npm"),
("xcodebuild", "xcodebuild", True, "exact xcodebuild"),
# Prefix wildcards
("swiftc", "swift*", True, "swiftc matches swift*"),
("swiftlint", "swift*", True, "swiftlint matches swift*"),
("swiftformat", "swift*", True, "swiftformat matches swift*"),
("swift", "swift*", True, "swift matches swift*"),
("npm", "swift*", False, "npm doesn't match swift*"),
# Local script paths
("build.sh", "./scripts/build.sh", True, "script name matches path"),
("./scripts/build.sh", "./scripts/build.sh", True, "exact script path"),
("scripts/build.sh", "./scripts/build.sh", True, "relative script path"),
("/abs/path/scripts/build.sh", "./scripts/build.sh", True, "absolute path matches"),
("test.sh", "./scripts/build.sh", False, "different script name"),
# Non-matches
("go", "swift*", False, "go doesn't match swift*"),
("rustc", "swift*", False, "rustc doesn't match swift*"),
]
for command, pattern, should_match, description in test_cases:
result = matches_pattern(command, pattern)
if result == should_match:
print(f" PASS: {command!r} vs {pattern!r} ({description})")
passed += 1
else:
expected = "match" if should_match else "no match"
actual = "match" if result else "no match"
print(f" FAIL: {command!r} vs {pattern!r} ({description})")
print(f" Expected: {expected}, Got: {actual}")
failed += 1
return passed, failed
def test_yaml_loading():
"""Test YAML config loading and validation."""
print("\nTesting YAML loading:\n")
passed = 0
failed = 0
with tempfile.TemporaryDirectory() as tmpdir:
project_dir = Path(tmpdir)
autocoder_dir = project_dir / ".autocoder"
autocoder_dir.mkdir()
# Test 1: Valid YAML
config_path = autocoder_dir / "allowed_commands.yaml"
config_path.write_text("""version: 1
commands:
- name: swift
description: Swift compiler
- name: xcodebuild
description: Xcode build
- name: swift*
description: All Swift tools
""")
config = load_project_commands(project_dir)
if config and config["version"] == 1 and len(config["commands"]) == 3:
print(" PASS: Load valid YAML")
passed += 1
else:
print(" FAIL: Load valid YAML")
print(f" Got: {config}")
failed += 1
# Test 2: Missing file returns None
(project_dir / ".autocoder" / "allowed_commands.yaml").unlink()
config = load_project_commands(project_dir)
if config is None:
print(" PASS: Missing file returns None")
passed += 1
else:
print(" FAIL: Missing file returns None")
print(f" Got: {config}")
failed += 1
# Test 3: Invalid YAML returns None
config_path.write_text("invalid: yaml: content:")
config = load_project_commands(project_dir)
if config is None:
print(" PASS: Invalid YAML returns None")
passed += 1
else:
print(" FAIL: Invalid YAML returns None")
print(f" Got: {config}")
failed += 1
# Test 4: Over limit (50 commands)
commands = [f" - name: cmd{i}\n description: Command {i}" for i in range(51)]
config_path.write_text("version: 1\ncommands:\n" + "\n".join(commands))
config = load_project_commands(project_dir)
if config is None:
print(" PASS: Over limit rejected")
passed += 1
else:
print(" FAIL: Over limit rejected")
print(f" Got: {config}")
failed += 1
return passed, failed
def test_command_validation():
"""Test project command validation."""
print("\nTesting command validation:\n")
passed = 0
failed = 0
# Test cases: (cmd_config, should_be_valid, description)
test_cases = [
# Valid commands
({"name": "swift", "description": "Swift compiler"}, True, "valid command"),
({"name": "swift"}, True, "command without description"),
({"name": "swift*", "description": "All Swift tools"}, True, "pattern command"),
({"name": "./scripts/build.sh", "description": "Build script"}, True, "local script"),
# Invalid commands
({}, False, "missing name"),
({"description": "No name"}, False, "missing name field"),
({"name": ""}, False, "empty name"),
({"name": 123}, False, "non-string name"),
# Blocklisted commands
({"name": "sudo"}, False, "blocklisted sudo"),
({"name": "shutdown"}, False, "blocklisted shutdown"),
({"name": "dd"}, False, "blocklisted dd"),
]
for cmd_config, should_be_valid, description in test_cases:
valid, error = validate_project_command(cmd_config)
if valid == should_be_valid:
print(f" PASS: {description}")
passed += 1
else:
expected = "valid" if should_be_valid else "invalid"
actual = "valid" if valid else "invalid"
print(f" FAIL: {description}")
print(f" Expected: {expected}, Got: {actual}")
if error:
print(f" Error: {error}")
failed += 1
return passed, failed
def test_blocklist_enforcement():
"""Test blocklist enforcement in security hook."""
print("\nTesting blocklist enforcement:\n")
passed = 0
failed = 0
# All blocklisted commands should be rejected
for cmd in ["sudo apt install", "shutdown now", "dd if=/dev/zero", "aws s3 ls"]:
input_data = {"tool_name": "Bash", "tool_input": {"command": cmd}}
result = asyncio.run(bash_security_hook(input_data))
if result.get("decision") == "block":
print(f" PASS: Blocked {cmd.split()[0]}")
passed += 1
else:
print(f" FAIL: Should block {cmd.split()[0]}")
failed += 1
return passed, failed
def test_project_commands():
"""Test project-specific commands in security hook."""
print("\nTesting project-specific commands:\n")
passed = 0
failed = 0
with tempfile.TemporaryDirectory() as tmpdir:
project_dir = Path(tmpdir)
autocoder_dir = project_dir / ".autocoder"
autocoder_dir.mkdir()
# Create a config with Swift commands
config_path = autocoder_dir / "allowed_commands.yaml"
config_path.write_text("""version: 1
commands:
- name: swift
description: Swift compiler
- name: xcodebuild
description: Xcode build
- name: swift*
description: All Swift tools
""")
# Test 1: Project command should be allowed
input_data = {"tool_name": "Bash", "tool_input": {"command": "swift --version"}}
context = {"project_dir": str(project_dir)}
result = asyncio.run(bash_security_hook(input_data, context=context))
if result.get("decision") != "block":
print(" PASS: Project command 'swift' allowed")
passed += 1
else:
print(" FAIL: Project command 'swift' should be allowed")
print(f" Reason: {result.get('reason')}")
failed += 1
# Test 2: Pattern match should work
input_data = {"tool_name": "Bash", "tool_input": {"command": "swiftlint"}}
result = asyncio.run(bash_security_hook(input_data, context=context))
if result.get("decision") != "block":
print(" PASS: Pattern 'swift*' matches 'swiftlint'")
passed += 1
else:
print(" FAIL: Pattern 'swift*' should match 'swiftlint'")
print(f" Reason: {result.get('reason')}")
failed += 1
# Test 3: Non-allowed command should be blocked
input_data = {"tool_name": "Bash", "tool_input": {"command": "rustc"}}
result = asyncio.run(bash_security_hook(input_data, context=context))
if result.get("decision") == "block":
print(" PASS: Non-allowed command 'rustc' blocked")
passed += 1
else:
print(" FAIL: Non-allowed command 'rustc' should be blocked")
failed += 1
return passed, failed
def test_org_config_loading():
"""Test organization-level config loading."""
print("\nTesting org config loading:\n")
passed = 0
failed = 0
# Save original org config path
original_home = Path.home()
with tempfile.TemporaryDirectory() as tmpdir:
# Temporarily override home directory for testing
import os
os.environ["HOME"] = tmpdir
org_dir = Path(tmpdir) / ".autocoder"
org_dir.mkdir()
org_config_path = org_dir / "config.yaml"
# Test 1: Valid org config
org_config_path.write_text("""version: 1
allowed_commands:
- name: jq
description: JSON processor
blocked_commands:
- aws
- kubectl
""")
config = load_org_config()
if config and config["version"] == 1:
if len(config["allowed_commands"]) == 1 and len(config["blocked_commands"]) == 2:
print(" PASS: Load valid org config")
passed += 1
else:
print(" FAIL: Load valid org config (wrong counts)")
failed += 1
else:
print(" FAIL: Load valid org config")
print(f" Got: {config}")
failed += 1
# Test 2: Missing file returns None
org_config_path.unlink()
config = load_org_config()
if config is None:
print(" PASS: Missing org config returns None")
passed += 1
else:
print(" FAIL: Missing org config returns None")
failed += 1
# Restore HOME
os.environ["HOME"] = str(original_home)
return passed, failed
def test_hierarchy_resolution():
"""Test command hierarchy resolution."""
print("\nTesting hierarchy resolution:\n")
passed = 0
failed = 0
with tempfile.TemporaryDirectory() as tmphome:
with tempfile.TemporaryDirectory() as tmpproject:
# Setup fake home directory
import os
original_home = os.environ.get("HOME")
os.environ["HOME"] = tmphome
org_dir = Path(tmphome) / ".autocoder"
org_dir.mkdir()
org_config_path = org_dir / "config.yaml"
# Create org config with allowed and blocked commands
org_config_path.write_text("""version: 1
allowed_commands:
- name: jq
description: JSON processor
- name: python3
description: Python interpreter
blocked_commands:
- terraform
- kubectl
""")
project_dir = Path(tmpproject)
project_autocoder = project_dir / ".autocoder"
project_autocoder.mkdir()
project_config = project_autocoder / "allowed_commands.yaml"
# Create project config
project_config.write_text("""version: 1
commands:
- name: swift
description: Swift compiler
""")
# Test 1: Org allowed commands are included
allowed, blocked = get_effective_commands(project_dir)
if "jq" in allowed and "python3" in allowed:
print(" PASS: Org allowed commands included")
passed += 1
else:
print(" FAIL: Org allowed commands included")
print(f" jq in allowed: {'jq' in allowed}")
print(f" python3 in allowed: {'python3' in allowed}")
failed += 1
# Test 2: Org blocked commands are in blocklist
if "terraform" in blocked and "kubectl" in blocked:
print(" PASS: Org blocked commands in blocklist")
passed += 1
else:
print(" FAIL: Org blocked commands in blocklist")
failed += 1
# Test 3: Project commands are included
if "swift" in allowed:
print(" PASS: Project commands included")
passed += 1
else:
print(" FAIL: Project commands included")
failed += 1
# Test 4: Global commands are included
if "npm" in allowed and "git" in allowed:
print(" PASS: Global commands included")
passed += 1
else:
print(" FAIL: Global commands included")
failed += 1
# Test 5: Hardcoded blocklist cannot be overridden
if "sudo" in blocked and "shutdown" in blocked:
print(" PASS: Hardcoded blocklist enforced")
passed += 1
else:
print(" FAIL: Hardcoded blocklist enforced")
failed += 1
# Restore HOME
if original_home:
os.environ["HOME"] = original_home
else:
del os.environ["HOME"]
return passed, failed
def test_org_blocklist_enforcement():
"""Test that org-level blocked commands cannot be used."""
print("\nTesting org blocklist enforcement:\n")
passed = 0
failed = 0
with tempfile.TemporaryDirectory() as tmphome:
with tempfile.TemporaryDirectory() as tmpproject:
# Setup fake home directory
import os
original_home = os.environ.get("HOME")
os.environ["HOME"] = tmphome
org_dir = Path(tmphome) / ".autocoder"
org_dir.mkdir()
org_config_path = org_dir / "config.yaml"
# Create org config that blocks terraform
org_config_path.write_text("""version: 1
blocked_commands:
- terraform
""")
project_dir = Path(tmpproject)
project_autocoder = project_dir / ".autocoder"
project_autocoder.mkdir()
# Try to use terraform (should be blocked)
input_data = {"tool_name": "Bash", "tool_input": {"command": "terraform apply"}}
context = {"project_dir": str(project_dir)}
result = asyncio.run(bash_security_hook(input_data, context=context))
if result.get("decision") == "block":
print(" PASS: Org blocked command 'terraform' rejected")
passed += 1
else:
print(" FAIL: Org blocked command 'terraform' should be rejected")
failed += 1
# Restore HOME
if original_home:
os.environ["HOME"] = original_home
else:
del os.environ["HOME"]
return passed, failed
def main():
print("=" * 70)
print(" SECURITY HOOK TESTS")
@@ -174,6 +615,46 @@ def main():
passed += init_passed
failed += init_failed
# Test pattern matching (Phase 1)
pattern_passed, pattern_failed = test_pattern_matching()
passed += pattern_passed
failed += pattern_failed
# Test YAML loading (Phase 1)
yaml_passed, yaml_failed = test_yaml_loading()
passed += yaml_passed
failed += yaml_failed
# Test command validation (Phase 1)
validation_passed, validation_failed = test_command_validation()
passed += validation_passed
failed += validation_failed
# Test blocklist enforcement (Phase 1)
blocklist_passed, blocklist_failed = test_blocklist_enforcement()
passed += blocklist_passed
failed += blocklist_failed
# Test project commands (Phase 1)
project_passed, project_failed = test_project_commands()
passed += project_passed
failed += project_failed
# Test org config loading (Phase 2)
org_loading_passed, org_loading_failed = test_org_config_loading()
passed += org_loading_passed
failed += org_loading_failed
# Test hierarchy resolution (Phase 2)
hierarchy_passed, hierarchy_failed = test_hierarchy_resolution()
passed += hierarchy_passed
failed += hierarchy_failed
# Test org blocklist enforcement (Phase 2)
org_block_passed, org_block_failed = test_org_blocklist_enforcement()
passed += org_block_passed
failed += org_block_failed
# Commands that SHOULD be blocked
print("\nCommands that should be BLOCKED:\n")
dangerous = [

411
test_security_integration.py Normal file
View File

@@ -0,0 +1,411 @@
#!/usr/bin/env python3
"""
Security Integration Tests
===========================
Integration tests that spin up real agent instances and verify
bash command security policies are enforced correctly.
These tests actually run the agent (not just unit tests), so they:
- Create real temporary projects
- Configure real YAML files
- Execute the agent with test prompts
- Parse agent output to verify behavior
Run with: python test_security_integration.py
"""
import asyncio
import os
import sys
import tempfile
from pathlib import Path
from security import bash_security_hook
def test_blocked_command_via_hook():
"""Test that hardcoded blocked commands are rejected by the security hook."""
print("\n" + "=" * 70)
print("TEST 1: Hardcoded blocked command (sudo)")
print("=" * 70)
with tempfile.TemporaryDirectory() as tmpdir:
project_dir = Path(tmpdir)
# Create minimal project structure
autocoder_dir = project_dir / ".autocoder"
autocoder_dir.mkdir()
(autocoder_dir / "allowed_commands.yaml").write_text(
"version: 1\ncommands: []"
)
# Try to run sudo (should be blocked)
input_data = {
"tool_name": "Bash",
"tool_input": {"command": "sudo apt install nginx"},
}
context = {"project_dir": str(project_dir)}
result = asyncio.run(bash_security_hook(input_data, context=context))
if result.get("decision") == "block":
print("✅ PASS: sudo was blocked")
print(f" Reason: {result.get('reason', 'N/A')[:80]}...")
return True
else:
print("❌ FAIL: sudo should have been blocked")
print(f" Got: {result}")
return False
def test_allowed_command_via_hook():
"""Test that default allowed commands work."""
print("\n" + "=" * 70)
print("TEST 2: Default allowed command (ls)")
print("=" * 70)
with tempfile.TemporaryDirectory() as tmpdir:
project_dir = Path(tmpdir)
# Create minimal project structure
autocoder_dir = project_dir / ".autocoder"
autocoder_dir.mkdir()
(autocoder_dir / "allowed_commands.yaml").write_text(
"version: 1\ncommands: []"
)
# Try to run ls (should be allowed - in default allowlist)
input_data = {"tool_name": "Bash", "tool_input": {"command": "ls -la"}}
context = {"project_dir": str(project_dir)}
result = asyncio.run(bash_security_hook(input_data, context=context))
if result.get("decision") != "block":
print("✅ PASS: ls was allowed (default allowlist)")
return True
else:
print("❌ FAIL: ls should have been allowed")
print(f" Reason: {result.get('reason', 'N/A')}")
return False
def test_non_allowed_command_via_hook():
"""Test that commands not in any allowlist are blocked."""
print("\n" + "=" * 70)
print("TEST 3: Non-allowed command (wget)")
print("=" * 70)
with tempfile.TemporaryDirectory() as tmpdir:
project_dir = Path(tmpdir)
# Create minimal project structure
autocoder_dir = project_dir / ".autocoder"
autocoder_dir.mkdir()
(autocoder_dir / "allowed_commands.yaml").write_text(
"version: 1\ncommands: []"
)
# Try to run wget (not in default allowlist)
input_data = {
"tool_name": "Bash",
"tool_input": {"command": "wget https://example.com"},
}
context = {"project_dir": str(project_dir)}
result = asyncio.run(bash_security_hook(input_data, context=context))
if result.get("decision") == "block":
print("✅ PASS: wget was blocked (not in allowlist)")
print(f" Reason: {result.get('reason', 'N/A')[:80]}...")
return True
else:
print("❌ FAIL: wget should have been blocked")
return False
def test_project_config_allows_command():
"""Test that adding a command to project config allows it."""
print("\n" + "=" * 70)
print("TEST 4: Project config allows command (swift)")
print("=" * 70)
with tempfile.TemporaryDirectory() as tmpdir:
project_dir = Path(tmpdir)
# Create project config with swift allowed
autocoder_dir = project_dir / ".autocoder"
autocoder_dir.mkdir()
(autocoder_dir / "allowed_commands.yaml").write_text("""version: 1
commands:
- name: swift
description: Swift compiler
- name: xcodebuild
description: Xcode build system
""")
# Try to run swift (should be allowed via project config)
input_data = {"tool_name": "Bash", "tool_input": {"command": "swift --version"}}
context = {"project_dir": str(project_dir)}
result = asyncio.run(bash_security_hook(input_data, context=context))
if result.get("decision") != "block":
print("✅ PASS: swift was allowed (project config)")
return True
else:
print("❌ FAIL: swift should have been allowed")
print(f" Reason: {result.get('reason', 'N/A')}")
return False
def test_pattern_matching():
"""Test that wildcard patterns work correctly."""
print("\n" + "=" * 70)
print("TEST 5: Pattern matching (swift*)")
print("=" * 70)
with tempfile.TemporaryDirectory() as tmpdir:
project_dir = Path(tmpdir)
# Create project config with swift* pattern
autocoder_dir = project_dir / ".autocoder"
autocoder_dir.mkdir()
(autocoder_dir / "allowed_commands.yaml").write_text("""version: 1
commands:
- name: swift*
description: All Swift tools
""")
# Try to run swiftlint (should match swift* pattern)
input_data = {"tool_name": "Bash", "tool_input": {"command": "swiftlint"}}
context = {"project_dir": str(project_dir)}
result = asyncio.run(bash_security_hook(input_data, context=context))
if result.get("decision") != "block":
print("✅ PASS: swiftlint matched swift* pattern")
return True
else:
print("❌ FAIL: swiftlint should have matched swift*")
print(f" Reason: {result.get('reason', 'N/A')}")
return False
def test_org_blocklist_enforcement():
"""Test that org-level blocked commands cannot be overridden."""
print("\n" + "=" * 70)
print("TEST 6: Org blocklist enforcement (terraform)")
print("=" * 70)
with tempfile.TemporaryDirectory() as tmphome:
with tempfile.TemporaryDirectory() as tmpproject:
# Setup fake home directory with org config
original_home = os.environ.get("HOME")
os.environ["HOME"] = tmphome
org_dir = Path(tmphome) / ".autocoder"
org_dir.mkdir()
(org_dir / "config.yaml").write_text("""version: 1
allowed_commands: []
blocked_commands:
- terraform
- kubectl
""")
project_dir = Path(tmpproject)
autocoder_dir = project_dir / ".autocoder"
autocoder_dir.mkdir()
# Try to allow terraform in project config (should fail - org blocked)
(autocoder_dir / "allowed_commands.yaml").write_text("""version: 1
commands:
- name: terraform
description: Infrastructure as code
""")
# Try to run terraform (should be blocked by org config)
input_data = {
"tool_name": "Bash",
"tool_input": {"command": "terraform apply"},
}
context = {"project_dir": str(project_dir)}
result = asyncio.run(bash_security_hook(input_data, context=context))
# Restore HOME
if original_home:
os.environ["HOME"] = original_home
else:
del os.environ["HOME"]
if result.get("decision") == "block":
print("✅ PASS: terraform blocked by org config (cannot override)")
print(f" Reason: {result.get('reason', 'N/A')[:80]}...")
return True
else:
print("❌ FAIL: terraform should have been blocked by org config")
return False
def test_org_allowlist_inheritance():
"""Test that org-level allowed commands are available to projects."""
print("\n" + "=" * 70)
print("TEST 7: Org allowlist inheritance (jq)")
print("=" * 70)
with tempfile.TemporaryDirectory() as tmphome:
with tempfile.TemporaryDirectory() as tmpproject:
# Setup fake home directory with org config
original_home = os.environ.get("HOME")
os.environ["HOME"] = tmphome
org_dir = Path(tmphome) / ".autocoder"
org_dir.mkdir()
(org_dir / "config.yaml").write_text("""version: 1
allowed_commands:
- name: jq
description: JSON processor
blocked_commands: []
""")
project_dir = Path(tmpproject)
autocoder_dir = project_dir / ".autocoder"
autocoder_dir.mkdir()
(autocoder_dir / "allowed_commands.yaml").write_text(
"version: 1\ncommands: []"
)
# Try to run jq (should be allowed via org config)
input_data = {"tool_name": "Bash", "tool_input": {"command": "jq '.data'"}}
context = {"project_dir": str(project_dir)}
result = asyncio.run(bash_security_hook(input_data, context=context))
# Restore HOME
if original_home:
os.environ["HOME"] = original_home
else:
del os.environ["HOME"]
if result.get("decision") != "block":
print("✅ PASS: jq allowed via org config")
return True
else:
print("❌ FAIL: jq should have been allowed via org config")
print(f" Reason: {result.get('reason', 'N/A')}")
return False
def test_invalid_yaml_ignored():
"""Test that invalid YAML config is safely ignored."""
print("\n" + "=" * 70)
print("TEST 8: Invalid YAML safely ignored")
print("=" * 70)
with tempfile.TemporaryDirectory() as tmpdir:
project_dir = Path(tmpdir)
# Create invalid YAML
autocoder_dir = project_dir / ".autocoder"
autocoder_dir.mkdir()
(autocoder_dir / "allowed_commands.yaml").write_text("invalid: yaml: content:")
# Try to run ls (should still work - falls back to defaults)
input_data = {"tool_name": "Bash", "tool_input": {"command": "ls"}}
context = {"project_dir": str(project_dir)}
result = asyncio.run(bash_security_hook(input_data, context=context))
if result.get("decision") != "block":
print("✅ PASS: Invalid YAML ignored, defaults still work")
return True
else:
print("❌ FAIL: Should fall back to defaults when YAML is invalid")
print(f" Reason: {result.get('reason', 'N/A')}")
return False
def test_50_command_limit():
"""Test that configs with >50 commands are rejected."""
print("\n" + "=" * 70)
print("TEST 9: 50 command limit enforced")
print("=" * 70)
with tempfile.TemporaryDirectory() as tmpdir:
project_dir = Path(tmpdir)
# Create config with 51 commands
autocoder_dir = project_dir / ".autocoder"
autocoder_dir.mkdir()
commands = [
f" - name: cmd{i}\n description: Command {i}" for i in range(51)
]
(autocoder_dir / "allowed_commands.yaml").write_text(
"version: 1\ncommands:\n" + "\n".join(commands)
)
# Try to run cmd0 (should be blocked - config is invalid)
input_data = {"tool_name": "Bash", "tool_input": {"command": "cmd0"}}
context = {"project_dir": str(project_dir)}
result = asyncio.run(bash_security_hook(input_data, context=context))
if result.get("decision") == "block":
print("✅ PASS: Config with >50 commands rejected")
return True
else:
print("❌ FAIL: Config with >50 commands should be rejected")
return False
def main():
print("=" * 70)
print(" SECURITY INTEGRATION TESTS")
print("=" * 70)
print("\nThese tests verify bash command security policies using real hooks.")
print("They test the actual security.py implementation, not just unit tests.\n")
tests = [
test_blocked_command_via_hook,
test_allowed_command_via_hook,
test_non_allowed_command_via_hook,
test_project_config_allows_command,
test_pattern_matching,
test_org_blocklist_enforcement,
test_org_allowlist_inheritance,
test_invalid_yaml_ignored,
test_50_command_limit,
]
passed = 0
failed = 0
for test in tests:
try:
if test():
passed += 1
else:
failed += 1
except Exception as e:
print(f"❌ FAIL: Test raised exception: {e}")
import traceback
traceback.print_exc()
failed += 1
print("\n" + "=" * 70)
print(f" RESULTS: {passed} passed, {failed} failed")
print("=" * 70)
if failed == 0:
print("\n✅ ALL INTEGRATION TESTS PASSED")
return 0
else:
print(f"\n❌ {failed} INTEGRATION TEST(S) FAILED")
return 1
if __name__ == "__main__":
sys.exit(main())