fix: Prevent mock data implementations with infrastructure features

Problem:
The coding agent can implement in-memory storage (e.g., `dev-store.ts` with
`globalThis`) instead of a real database. These implementations pass all tests
because data persists during runtime, but data is lost on server restart.

This is a root cause for #68 - agent "passes" features that don't actually work.

Solution:
1. Add 5 mandatory Infrastructure Features (indices 0-4) that run first:
   - Feature 0: Database connection established
   - Feature 1: Database schema applied correctly
   - Feature 2: Data persists across server restart (CRITICAL)
   - Feature 3: No mock data patterns in codebase
   - Feature 4: Backend API queries real database

2. Add STEP 5.7: Server Restart Persistence Test to coding prompt:
   - Create test data, stop server, restart, verify data still exists

3. Extend grep patterns for mock detection in STEP 5.6:
   - globalThis., devStore, dev-store, mockData, fakeData
   - TODO.*real, STUB, MOCK, new Map() as data stores

Changes:
- .claude/templates/initializer_prompt.template.md - Infrastructure features
- .claude/templates/coding_prompt.template.md - STEP 5.6/5.7 enhancements
- .claude/commands/create-spec.md - Phase 3b database question

Backwards Compatible:
- Works with YOLO mode (uses bash/grep, not browser automation)
- Stateless apps can skip database features via create-spec question

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
cabana8471
2026-01-25 08:01:30 +01:00
parent 486979c3d9
commit 8e23fee094
3 changed files with 223 additions and 32 deletions

View File

@@ -156,6 +156,9 @@ Use browser automation tools:
- [ ] Deleted the test data - verified it's gone everywhere
- [ ] NO unexplained data appeared (would indicate mock data)
- [ ] Dashboard/counts reflect real numbers after my changes
- [ ] **Ran extended mock data grep (STEP 5.6) - no hits in src/ (excluding tests)**
- [ ] **Verified no globalThis, devStore, or dev-store patterns**
- [ ] **Server restart test passed (STEP 5.7) - data persists across restart**
#### Navigation Verification
@@ -174,10 +177,72 @@ Use browser automation tools:
### STEP 5.6: MOCK DATA DETECTION (Before marking passing)
1. **Search code:** `grep -r "mockData\|fakeData\|TODO\|STUB" --include="*.ts" --include="*.tsx"`
2. **Runtime test:** Create unique data (e.g., "TEST_12345") → verify in UI → delete → verify gone
3. **Check database:** All displayed data must come from real DB queries
4. If unexplained data appears, it's mock data - fix before marking passing.
**Run ALL these grep checks. Any hits in src/ (excluding test files) require investigation:**
```bash
# 1. In-memory storage patterns (CRITICAL - catches dev-store)
grep -r "globalThis\." --include="*.ts" --include="*.tsx" --include="*.js" src/
grep -r "dev-store\|devStore\|DevStore\|mock-db\|mockDb" --include="*.ts" --include="*.tsx" src/
# 2. Mock data variables
grep -r "mockData\|fakeData\|sampleData\|dummyData\|testData" --include="*.ts" --include="*.tsx" src/
# 3. TODO/incomplete markers
grep -r "TODO.*real\|TODO.*database\|TODO.*API\|STUB\|MOCK" --include="*.ts" --include="*.tsx" src/
# 4. Development-only conditionals
grep -r "isDevelopment\|isDev\|process\.env\.NODE_ENV.*development" --include="*.ts" --include="*.tsx" src/
# 5. In-memory collections as data stores (check lib/store/data directories)
grep -r "new Map()\|new Set()" --include="*.ts" --include="*.tsx" src/lib/ src/store/ src/data/ 2>/dev/null
```
**Rule:** If ANY grep returns results in production code → investigate → FIX before marking passing.
**Runtime verification:**
1. Create unique data (e.g., "TEST_12345") → verify in UI → delete → verify gone
2. Check database directly - all displayed data must come from real DB queries
3. If unexplained data appears, it's mock data - fix before marking passing.
### STEP 5.7: SERVER RESTART PERSISTENCE TEST (MANDATORY for data features)
**When required:** Any feature involving CRUD operations or data persistence.
**This test is NON-NEGOTIABLE. It catches in-memory storage implementations that pass all other tests.**
**Steps:**
1. Create unique test data via UI or API (e.g., item named "RESTART_TEST_12345")
2. Verify data appears in UI and API response
3. **STOP the server completely:**
```bash
pkill -f "node" || pkill -f "npm" || pkill -f "next"
sleep 5
# Verify server is stopped
pgrep -f "node" && echo "ERROR: Server still running!" && exit 1
```
4. **RESTART the server:**
```bash
./init.sh &
sleep 15 # Allow server to fully start
```
5. **Query for test data - it MUST still exist**
- Via UI: Navigate to data location, verify data appears
- Via API: `curl http://localhost:PORT/api/items` - verify data in response
6. **If data is GONE:** Implementation uses in-memory storage → CRITICAL FAIL
- Search for: `grep -r "globalThis\|devStore\|dev-store" src/`
- You MUST fix the mock data implementation before proceeding
- Replace in-memory storage with real database queries
7. **Clean up test data** after successful verification
**Why this test exists:** In-memory stores like `globalThis.devStore` pass all other tests because data persists during a single server run. Only a full server restart reveals this bug. Skipping this step WILL allow dev-store implementations to slip through.
**YOLO Mode Note:** Even in YOLO mode, this verification is MANDATORY for data features. Use curl instead of browser automation.
### STEP 6: UPDATE FEATURE STATUS (CAREFULLY!)