mirror of
https://github.com/bmad-code-org/BMAD-METHOD.git
synced 2026-01-30 04:32:02 +00:00
feat(bmgd): Add E2E testing methodology and scaffold workflow (#1322)
* feat(bmgd): Add E2E testing methodology and scaffold workflow - Add comprehensive e2e-testing.md knowledge fragment - Add e2e-scaffold workflow for infrastructure generation - Update qa-index.csv with e2e-testing fragment reference - Update game-qa.agent.yaml with ES trigger - Update test-design and automate instructions with E2E guidance - Update unity-testing.md with E2E section reference * fix(bmgd): improve E2E testing infrastructure robustness - Add WaitForValueApprox overloads for float/double comparisons - Fix assembly definition to use precompiledReferences for test runners - Fix CaptureOnFailure to yield before screenshot capture (main thread) - Add error handling to test file cleanup with try/catch - Fix ClickButton to use FindObjectsByType and check scene.isLoaded - Add engine-specific output paths (Unity/Unreal/Godot) to workflow - Fix knowledge_fragments paths to use correct relative paths * feat(bmgd): add E2E testing support for Godot and Unreal Godot: - Add C# testing with xUnit/NSubstitute alongside GDScript GUT - Add E2E infrastructure: GameE2ETestFixture, ScenarioBuilder, InputSimulator, AsyncAssert (all GDScript) - Add example E2E tests and quick checklist Unreal: - Add E2E infrastructure extending AFunctionalTest - Add GameE2ETestBase, ScenarioBuilder, InputSimulator classes - Add AsyncTestHelpers with latent commands and macros - Add example E2E tests for combat and turn cycle - Add CLI commands for running E2E tests --------- Co-authored-by: Scott Jennings <scott.jennings+CIGINT@cloudimperiumgames.com> Co-authored-by: Brian <bmadcode@gmail.com>
This commit is contained in:
@@ -22,6 +22,8 @@ agent:
|
||||
|
||||
critical_actions:
|
||||
- "Consult {project-root}/_bmad/bmgd/gametest/qa-index.csv to select knowledge fragments under knowledge/ and load only the files needed for the current task"
|
||||
- "For E2E testing requests, always load knowledge/e2e-testing.md first"
|
||||
- "When scaffolding tests, distinguish between unit, integration, and E2E test needs"
|
||||
- "Load the referenced fragment(s) from {project-root}/_bmad/bmgd/gametest/knowledge/ before giving recommendations"
|
||||
- "Cross-check recommendations with the current official Unity Test Framework, Unreal Automation, or Godot GUT documentation"
|
||||
- "Find if this exists, if it does, always treat it as the bible I plan and execute against: `**/project-context.md`"
|
||||
@@ -43,6 +45,10 @@ agent:
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/gametest/automate/workflow.yaml"
|
||||
description: "[TA] Generate automated game tests"
|
||||
|
||||
- trigger: ES or fuzzy match on e2e-scaffold
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/gametest/e2e-scaffold/workflow.yaml"
|
||||
description: "[ES] Scaffold E2E testing infrastructure"
|
||||
|
||||
- trigger: PP or fuzzy match on playtest-plan
|
||||
workflow: "{project-root}/_bmad/bmgd/workflows/gametest/playtest-plan/workflow.yaml"
|
||||
description: "[PP] Create structured playtesting plan"
|
||||
|
||||
1013
src/modules/bmgd/gametest/knowledge/e2e-testing.md
Normal file
1013
src/modules/bmgd/gametest/knowledge/e2e-testing.md
Normal file
File diff suppressed because it is too large
Load Diff
@@ -374,3 +374,502 @@ test:
|
||||
| Signal not detected | Signal not watched | Call `watch_signals()` before action |
|
||||
| Physics not working | Missing frames | Await `physics_frame` |
|
||||
| Flaky tests | Timing issues | Use proper await/signals |
|
||||
|
||||
## C# Testing in Godot
|
||||
|
||||
Godot 4 supports C# via .NET 6+. You can use standard .NET testing frameworks alongside GUT.
|
||||
|
||||
### Project Setup for C#
|
||||
|
||||
```
|
||||
project/
|
||||
├── addons/
|
||||
│ └── gut/
|
||||
├── src/
|
||||
│ ├── Player/
|
||||
│ │ └── PlayerController.cs
|
||||
│ └── Combat/
|
||||
│ └── DamageCalculator.cs
|
||||
├── tests/
|
||||
│ ├── gdscript/
|
||||
│ │ └── test_integration.gd
|
||||
│ └── csharp/
|
||||
│ ├── Tests.csproj
|
||||
│ └── DamageCalculatorTests.cs
|
||||
└── project.csproj
|
||||
```
|
||||
|
||||
### C# Test Project Setup
|
||||
|
||||
Create a separate test project that references your game assembly:
|
||||
|
||||
```xml
|
||||
<!-- tests/csharp/Tests.csproj -->
|
||||
<Project Sdk="Godot.NET.Sdk/4.2.0">
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net6.0</TargetFramework>
|
||||
<EnableDynamicLoading>true</EnableDynamicLoading>
|
||||
<IsPackable>false</IsPackable>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.8.0" />
|
||||
<PackageReference Include="xunit" Version="2.6.2" />
|
||||
<PackageReference Include="xunit.runner.visualstudio" Version="2.5.4" />
|
||||
<PackageReference Include="NSubstitute" Version="5.1.0" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="../../project.csproj" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
```
|
||||
|
||||
### Basic C# Unit Tests
|
||||
|
||||
```csharp
|
||||
// tests/csharp/DamageCalculatorTests.cs
|
||||
using Xunit;
|
||||
using YourGame.Combat;
|
||||
|
||||
public class DamageCalculatorTests
|
||||
{
|
||||
private readonly DamageCalculator _calculator;
|
||||
|
||||
public DamageCalculatorTests()
|
||||
{
|
||||
_calculator = new DamageCalculator();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Calculate_BaseDamage_ReturnsCorrectValue()
|
||||
{
|
||||
var result = _calculator.Calculate(100f, 1f);
|
||||
Assert.Equal(100f, result);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Calculate_CriticalHit_DoublesDamage()
|
||||
{
|
||||
var result = _calculator.Calculate(100f, 2f);
|
||||
Assert.Equal(200f, result);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(100f, 0.5f, 50f)]
|
||||
[InlineData(100f, 1.5f, 150f)]
|
||||
[InlineData(50f, 2f, 100f)]
|
||||
public void Calculate_Parameterized_ReturnsExpected(
|
||||
float baseDamage, float multiplier, float expected)
|
||||
{
|
||||
var result = _calculator.Calculate(baseDamage, multiplier);
|
||||
Assert.Equal(expected, result);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Testing Godot Nodes in C#
|
||||
|
||||
For tests requiring Godot runtime, use a hybrid approach:
|
||||
|
||||
```csharp
|
||||
// tests/csharp/PlayerControllerTests.cs
|
||||
using Godot;
|
||||
using Xunit;
|
||||
using YourGame.Player;
|
||||
|
||||
public class PlayerControllerTests : IDisposable
|
||||
{
|
||||
private readonly SceneTree _sceneTree;
|
||||
private PlayerController _player;
|
||||
|
||||
public PlayerControllerTests()
|
||||
{
|
||||
// These tests must run within Godot runtime
|
||||
// Use GodotXUnit or similar adapter
|
||||
}
|
||||
|
||||
[GodotFact] // Custom attribute for Godot runtime tests
|
||||
public async Task Player_Move_ChangesPosition()
|
||||
{
|
||||
var startPos = _player.GlobalPosition;
|
||||
|
||||
_player.SetInput(new Vector2(1, 0));
|
||||
|
||||
await ToSignal(GetTree().CreateTimer(0.5f), "timeout");
|
||||
|
||||
Assert.True(_player.GlobalPosition.X > startPos.X);
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_player?.QueueFree();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### C# Mocking with NSubstitute
|
||||
|
||||
```csharp
|
||||
using NSubstitute;
|
||||
using Xunit;
|
||||
|
||||
public class EnemyAITests
|
||||
{
|
||||
[Fact]
|
||||
public void Enemy_UsesPathfinding_WhenMoving()
|
||||
{
|
||||
var mockPathfinding = Substitute.For<IPathfinding>();
|
||||
mockPathfinding.FindPath(Arg.Any<Vector2>(), Arg.Any<Vector2>())
|
||||
.Returns(new[] { Vector2.Zero, new Vector2(10, 10) });
|
||||
|
||||
var enemy = new EnemyAI(mockPathfinding);
|
||||
|
||||
enemy.MoveTo(new Vector2(10, 10));
|
||||
|
||||
mockPathfinding.Received().FindPath(
|
||||
Arg.Any<Vector2>(),
|
||||
Arg.Is<Vector2>(v => v == new Vector2(10, 10)));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Running C# Tests
|
||||
|
||||
```bash
|
||||
# Run C# unit tests (no Godot runtime needed)
|
||||
dotnet test tests/csharp/Tests.csproj
|
||||
|
||||
# Run with coverage
|
||||
dotnet test tests/csharp/Tests.csproj --collect:"XPlat Code Coverage"
|
||||
|
||||
# Run specific test
|
||||
dotnet test tests/csharp/Tests.csproj --filter "FullyQualifiedName~DamageCalculator"
|
||||
```
|
||||
|
||||
### Hybrid Test Strategy
|
||||
|
||||
| Test Type | Framework | When to Use |
|
||||
| ------------- | ---------------- | ---------------------------------- |
|
||||
| Pure logic | xUnit/NUnit (C#) | Classes without Godot dependencies |
|
||||
| Node behavior | GUT (GDScript) | MonoBehaviour-like testing |
|
||||
| Integration | GUT (GDScript) | Scene and signal testing |
|
||||
| E2E | GUT (GDScript) | Full gameplay flows |
|
||||
|
||||
## End-to-End Testing
|
||||
|
||||
For comprehensive E2E testing patterns, infrastructure scaffolding, and
|
||||
scenario builders, see **knowledge/e2e-testing.md**.
|
||||
|
||||
### E2E Infrastructure for Godot
|
||||
|
||||
#### GameE2ETestFixture (GDScript)
|
||||
|
||||
```gdscript
|
||||
# tests/e2e/infrastructure/game_e2e_test_fixture.gd
|
||||
extends GutTest
|
||||
class_name GameE2ETestFixture
|
||||
|
||||
var game_state: GameStateManager
|
||||
var input_sim: InputSimulator
|
||||
var scenario: ScenarioBuilder
|
||||
var _scene_instance: Node
|
||||
|
||||
## Override to specify a different scene for specific test classes.
|
||||
func get_scene_path() -> String:
|
||||
return "res://scenes/game.tscn"
|
||||
|
||||
func before_each():
|
||||
# Load game scene
|
||||
var scene = load(get_scene_path())
|
||||
_scene_instance = scene.instantiate()
|
||||
add_child(_scene_instance)
|
||||
|
||||
# Get references
|
||||
game_state = _scene_instance.get_node("GameStateManager")
|
||||
assert_not_null(game_state, "GameStateManager not found in scene")
|
||||
|
||||
input_sim = InputSimulator.new()
|
||||
scenario = ScenarioBuilder.new(game_state)
|
||||
|
||||
# Wait for ready
|
||||
await wait_for_game_ready()
|
||||
|
||||
func after_each():
|
||||
if _scene_instance:
|
||||
_scene_instance.queue_free()
|
||||
_scene_instance = null
|
||||
input_sim = null
|
||||
scenario = null
|
||||
|
||||
func wait_for_game_ready(timeout: float = 10.0):
|
||||
var elapsed = 0.0
|
||||
while not game_state.is_ready and elapsed < timeout:
|
||||
await get_tree().process_frame
|
||||
elapsed += get_process_delta_time()
|
||||
assert_true(game_state.is_ready, "Game should be ready within timeout")
|
||||
```
|
||||
|
||||
#### ScenarioBuilder (GDScript)
|
||||
|
||||
```gdscript
|
||||
# tests/e2e/infrastructure/scenario_builder.gd
|
||||
extends RefCounted
|
||||
class_name ScenarioBuilder
|
||||
|
||||
var _game_state: GameStateManager
|
||||
var _setup_actions: Array[Callable] = []
|
||||
|
||||
func _init(game_state: GameStateManager):
|
||||
_game_state = game_state
|
||||
|
||||
## Load a pre-configured scenario from a save file.
|
||||
func from_save_file(file_name: String) -> ScenarioBuilder:
|
||||
_setup_actions.append(func(): await _load_save_file(file_name))
|
||||
return self
|
||||
|
||||
## Configure the current turn number.
|
||||
func on_turn(turn_number: int) -> ScenarioBuilder:
|
||||
_setup_actions.append(func(): _set_turn(turn_number))
|
||||
return self
|
||||
|
||||
## Spawn a unit at position.
|
||||
func with_unit(faction: int, position: Vector2, movement_points: int = 6) -> ScenarioBuilder:
|
||||
_setup_actions.append(func(): await _spawn_unit(faction, position, movement_points))
|
||||
return self
|
||||
|
||||
## Execute all configured setup actions.
|
||||
func build() -> void:
|
||||
for action in _setup_actions:
|
||||
await action.call()
|
||||
_setup_actions.clear()
|
||||
|
||||
## Clear pending actions without executing.
|
||||
func reset() -> void:
|
||||
_setup_actions.clear()
|
||||
|
||||
# Private implementation
|
||||
func _load_save_file(file_name: String) -> void:
|
||||
var path = "res://tests/e2e/test_data/%s" % file_name
|
||||
await _game_state.load_game(path)
|
||||
|
||||
func _set_turn(turn: int) -> void:
|
||||
_game_state.set_turn_number(turn)
|
||||
|
||||
func _spawn_unit(faction: int, pos: Vector2, mp: int) -> void:
|
||||
var unit = _game_state.spawn_unit(faction, pos)
|
||||
unit.movement_points = mp
|
||||
```
|
||||
|
||||
#### InputSimulator (GDScript)
|
||||
|
||||
```gdscript
|
||||
# tests/e2e/infrastructure/input_simulator.gd
|
||||
extends RefCounted
|
||||
class_name InputSimulator
|
||||
|
||||
## Click at a world position.
|
||||
func click_world_position(world_pos: Vector2) -> void:
|
||||
var viewport = Engine.get_main_loop().root.get_viewport()
|
||||
var camera = viewport.get_camera_2d()
|
||||
var screen_pos = camera.get_screen_center_position() + (world_pos - camera.global_position)
|
||||
await click_screen_position(screen_pos)
|
||||
|
||||
## Click at a screen position.
|
||||
func click_screen_position(screen_pos: Vector2) -> void:
|
||||
var press = InputEventMouseButton.new()
|
||||
press.button_index = MOUSE_BUTTON_LEFT
|
||||
press.pressed = true
|
||||
press.position = screen_pos
|
||||
|
||||
var release = InputEventMouseButton.new()
|
||||
release.button_index = MOUSE_BUTTON_LEFT
|
||||
release.pressed = false
|
||||
release.position = screen_pos
|
||||
|
||||
Input.parse_input_event(press)
|
||||
await Engine.get_main_loop().process_frame
|
||||
Input.parse_input_event(release)
|
||||
await Engine.get_main_loop().process_frame
|
||||
|
||||
## Click a UI button by name.
|
||||
func click_button(button_name: String) -> void:
|
||||
var root = Engine.get_main_loop().root
|
||||
var button = _find_button_recursive(root, button_name)
|
||||
assert(button != null, "Button '%s' not found in scene tree" % button_name)
|
||||
|
||||
if not button.visible:
|
||||
push_warning("[InputSimulator] Button '%s' is not visible" % button_name)
|
||||
if button.disabled:
|
||||
push_warning("[InputSimulator] Button '%s' is disabled" % button_name)
|
||||
|
||||
button.pressed.emit()
|
||||
await Engine.get_main_loop().process_frame
|
||||
|
||||
func _find_button_recursive(node: Node, button_name: String) -> Button:
|
||||
if node is Button and node.name == button_name:
|
||||
return node
|
||||
for child in node.get_children():
|
||||
var found = _find_button_recursive(child, button_name)
|
||||
if found:
|
||||
return found
|
||||
return null
|
||||
|
||||
## Press and release a key.
|
||||
func press_key(keycode: Key) -> void:
|
||||
var press = InputEventKey.new()
|
||||
press.keycode = keycode
|
||||
press.pressed = true
|
||||
|
||||
var release = InputEventKey.new()
|
||||
release.keycode = keycode
|
||||
release.pressed = false
|
||||
|
||||
Input.parse_input_event(press)
|
||||
await Engine.get_main_loop().process_frame
|
||||
Input.parse_input_event(release)
|
||||
await Engine.get_main_loop().process_frame
|
||||
|
||||
## Simulate an input action.
|
||||
func action_press(action_name: String) -> void:
|
||||
Input.action_press(action_name)
|
||||
await Engine.get_main_loop().process_frame
|
||||
|
||||
func action_release(action_name: String) -> void:
|
||||
Input.action_release(action_name)
|
||||
await Engine.get_main_loop().process_frame
|
||||
|
||||
## Reset all input state.
|
||||
func reset() -> void:
|
||||
Input.flush_buffered_events()
|
||||
```
|
||||
|
||||
#### AsyncAssert (GDScript)
|
||||
|
||||
```gdscript
|
||||
# tests/e2e/infrastructure/async_assert.gd
|
||||
extends RefCounted
|
||||
class_name AsyncAssert
|
||||
|
||||
## Wait until condition is true, or fail after timeout.
|
||||
static func wait_until(
|
||||
condition: Callable,
|
||||
description: String,
|
||||
timeout: float = 5.0
|
||||
) -> void:
|
||||
var elapsed := 0.0
|
||||
while not condition.call() and elapsed < timeout:
|
||||
await Engine.get_main_loop().process_frame
|
||||
elapsed += Engine.get_main_loop().root.get_process_delta_time()
|
||||
|
||||
assert(condition.call(),
|
||||
"Timeout after %.1fs waiting for: %s" % [timeout, description])
|
||||
|
||||
## Wait for a value to equal expected.
|
||||
static func wait_for_value(
|
||||
getter: Callable,
|
||||
expected: Variant,
|
||||
description: String,
|
||||
timeout: float = 5.0
|
||||
) -> void:
|
||||
await wait_until(
|
||||
func(): return getter.call() == expected,
|
||||
"%s to equal '%s' (current: '%s')" % [description, expected, getter.call()],
|
||||
timeout)
|
||||
|
||||
## Wait for a float value within tolerance.
|
||||
static func wait_for_value_approx(
|
||||
getter: Callable,
|
||||
expected: float,
|
||||
description: String,
|
||||
tolerance: float = 0.0001,
|
||||
timeout: float = 5.0
|
||||
) -> void:
|
||||
await wait_until(
|
||||
func(): return absf(expected - getter.call()) < tolerance,
|
||||
"%s to equal ~%s ±%s (current: %s)" % [description, expected, tolerance, getter.call()],
|
||||
timeout)
|
||||
|
||||
## Assert that condition does NOT become true within duration.
|
||||
static func assert_never_true(
|
||||
condition: Callable,
|
||||
description: String,
|
||||
duration: float = 1.0
|
||||
) -> void:
|
||||
var elapsed := 0.0
|
||||
while elapsed < duration:
|
||||
assert(not condition.call(),
|
||||
"Condition unexpectedly became true: %s" % description)
|
||||
await Engine.get_main_loop().process_frame
|
||||
elapsed += Engine.get_main_loop().root.get_process_delta_time()
|
||||
|
||||
## Wait for specified number of frames.
|
||||
static func wait_frames(count: int) -> void:
|
||||
for i in range(count):
|
||||
await Engine.get_main_loop().process_frame
|
||||
|
||||
## Wait for physics to settle.
|
||||
static func wait_for_physics(frames: int = 3) -> void:
|
||||
for i in range(frames):
|
||||
await Engine.get_main_loop().root.get_tree().physics_frame
|
||||
```
|
||||
|
||||
### Example E2E Test (GDScript)
|
||||
|
||||
```gdscript
|
||||
# tests/e2e/scenarios/test_combat_flow.gd
|
||||
extends GameE2ETestFixture
|
||||
|
||||
func test_player_can_attack_enemy():
|
||||
# GIVEN: Player and enemy in combat range
|
||||
await scenario \
|
||||
.with_unit(Faction.PLAYER, Vector2(100, 100)) \
|
||||
.with_unit(Faction.ENEMY, Vector2(150, 100)) \
|
||||
.build()
|
||||
|
||||
var enemy = game_state.get_units(Faction.ENEMY)[0]
|
||||
var initial_health = enemy.health
|
||||
|
||||
# WHEN: Player attacks
|
||||
await input_sim.click_world_position(Vector2(100, 100)) # Select player
|
||||
await AsyncAssert.wait_until(
|
||||
func(): return game_state.selected_unit != null,
|
||||
"Unit should be selected")
|
||||
|
||||
await input_sim.click_world_position(Vector2(150, 100)) # Attack enemy
|
||||
|
||||
# THEN: Enemy takes damage
|
||||
await AsyncAssert.wait_until(
|
||||
func(): return enemy.health < initial_health,
|
||||
"Enemy should take damage")
|
||||
|
||||
func test_turn_cycle_completes():
|
||||
# GIVEN: Game in progress
|
||||
await scenario.on_turn(1).build()
|
||||
var starting_turn = game_state.turn_number
|
||||
|
||||
# WHEN: Player ends turn
|
||||
await input_sim.click_button("EndTurnButton")
|
||||
await AsyncAssert.wait_until(
|
||||
func(): return game_state.current_faction == Faction.ENEMY,
|
||||
"Should switch to enemy turn")
|
||||
|
||||
# AND: Enemy turn completes
|
||||
await AsyncAssert.wait_until(
|
||||
func(): return game_state.current_faction == Faction.PLAYER,
|
||||
"Should return to player turn",
|
||||
30.0) # AI might take a while
|
||||
|
||||
# THEN: Turn number incremented
|
||||
assert_eq(game_state.turn_number, starting_turn + 1)
|
||||
```
|
||||
|
||||
### Quick E2E Checklist for Godot
|
||||
|
||||
- [ ] Create `GameE2ETestFixture` base class extending GutTest
|
||||
- [ ] Implement `ScenarioBuilder` for your game's domain
|
||||
- [ ] Create `InputSimulator` wrapping Godot Input
|
||||
- [ ] Add `AsyncAssert` utilities with proper await
|
||||
- [ ] Organize E2E tests under `tests/e2e/scenarios/`
|
||||
- [ ] Configure GUT to include E2E test directory
|
||||
- [ ] Set up CI with headless Godot execution
|
||||
|
||||
@@ -381,3 +381,17 @@ test:
|
||||
| NullReferenceException | Missing Setup | Ensure [SetUp] initializes all fields |
|
||||
| Tests hang | Infinite coroutine | Add timeout or max iterations |
|
||||
| Flaky physics tests | Timing dependent | Use WaitForFixedUpdate, increase tolerance |
|
||||
|
||||
## End-to-End Testing
|
||||
|
||||
For comprehensive E2E testing patterns, infrastructure scaffolding, and
|
||||
scenario builders, see **knowledge/e2e-testing.md**.
|
||||
|
||||
### Quick E2E Checklist for Unity
|
||||
|
||||
- [ ] Create `GameE2ETestFixture` base class
|
||||
- [ ] Implement `ScenarioBuilder` for your game's domain
|
||||
- [ ] Create `InputSimulator` wrapping Input System
|
||||
- [ ] Add `AsyncAssert` utilities
|
||||
- [ ] Organize E2E tests under `Tests/PlayMode/E2E/`
|
||||
- [ ] Configure separate CI job for E2E suite
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -14,4 +14,5 @@ input-testing,Input Testing,"Controller, keyboard, and touch input validation","
|
||||
localization-testing,Localization Testing,"Text, audio, and cultural validation for international releases","localization,i18n,text",knowledge/localization-testing.md
|
||||
certification-testing,Platform Certification,"Console TRC/XR requirements and certification testing","certification,console,trc,xr",knowledge/certification-testing.md
|
||||
smoke-testing,Smoke Testing,"Critical path validation for build verification","smoke-tests,bvt,ci",knowledge/smoke-testing.md
|
||||
test-priorities,Test Priorities Matrix,"P0-P3 criteria, coverage targets, execution ordering for games","prioritization,risk,coverage",knowledge/test-priorities.md
|
||||
test-priorities,Test Priorities Matrix,"P0-P3 criteria, coverage targets, execution ordering for games","prioritization,risk,coverage",knowledge/test-priorities.md
|
||||
e2e-testing,End-to-End Testing,"Complete player journey testing with infrastructure patterns and async utilities","e2e,integration,player-journeys,scenarios,infrastructure",knowledge/e2e-testing.md
|
||||
|
||||
|
@@ -209,6 +209,87 @@ func test_{feature}_integration():
|
||||
# Cleanup
|
||||
scene.queue_free()
|
||||
```
|
||||
### E2E Journey Tests
|
||||
|
||||
**Knowledge Base Reference**: `knowledge/e2e-testing.md`
|
||||
```csharp
|
||||
public class {Feature}E2ETests : GameE2ETestFixture
|
||||
{
|
||||
[UnityTest]
|
||||
public IEnumerator {JourneyName}_Succeeds()
|
||||
{
|
||||
// GIVEN
|
||||
yield return Scenario
|
||||
.{SetupMethod1}()
|
||||
.{SetupMethod2}()
|
||||
.Build();
|
||||
|
||||
// WHEN
|
||||
yield return Input.{Action1}();
|
||||
yield return AsyncAssert.WaitUntil(
|
||||
() => {Condition1}, "{Description1}");
|
||||
yield return Input.{Action2}();
|
||||
|
||||
// THEN
|
||||
yield return AsyncAssert.WaitUntil(
|
||||
() => {FinalCondition}, "{FinalDescription}");
|
||||
Assert.{Assertion}({expected}, {actual});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
## Step 3.5: Generate E2E Infrastructure
|
||||
|
||||
Before generating E2E tests, scaffold the required infrastructure.
|
||||
|
||||
### Infrastructure Checklist
|
||||
|
||||
1. **Test Fixture Base Class**
|
||||
- Scene loading/unloading
|
||||
- Game ready state waiting
|
||||
- Common service access
|
||||
- Cleanup guarantees
|
||||
|
||||
2. **Scenario Builder**
|
||||
- Fluent API for game state configuration
|
||||
- Domain-specific methods (e.g., `WithUnit`, `OnTurn`)
|
||||
- Yields for state propagation
|
||||
|
||||
3. **Input Simulator**
|
||||
- Click/drag abstractions
|
||||
- Button press simulation
|
||||
- Keyboard input queuing
|
||||
|
||||
4. **Async Assertions**
|
||||
- `WaitUntil` with timeout and message
|
||||
- `WaitForEvent` for event-driven flows
|
||||
- `WaitForState` for state machine transitions
|
||||
|
||||
### Generation Template
|
||||
```csharp
|
||||
// GameE2ETestFixture.cs
|
||||
public abstract class GameE2ETestFixture
|
||||
{
|
||||
protected {GameStateClass} GameState;
|
||||
protected {InputSimulatorClass} Input;
|
||||
protected {ScenarioBuilderClass} Scenario;
|
||||
|
||||
[UnitySetUp]
|
||||
public IEnumerator BaseSetUp()
|
||||
{
|
||||
yield return LoadScene("{main_scene}");
|
||||
GameState = Object.FindFirstObjectByType<{GameStateClass}>();
|
||||
Input = new {InputSimulatorClass}();
|
||||
Scenario = new {ScenarioBuilderClass}(GameState);
|
||||
yield return WaitForReady();
|
||||
}
|
||||
|
||||
// ... (fill from e2e-testing.md patterns)
|
||||
}
|
||||
```
|
||||
|
||||
**After scaffolding infrastructure, proceed to generate actual E2E tests.**
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
# E2E Infrastructure Scaffold Checklist
|
||||
|
||||
## Preflight Validation
|
||||
|
||||
- [ ] Test framework already initialized (`Tests/` directory exists with proper structure)
|
||||
- [ ] Game state manager class identified
|
||||
- [ ] Main gameplay scene identified and loads without errors
|
||||
- [ ] No existing E2E infrastructure conflicts
|
||||
|
||||
## Architecture Analysis
|
||||
|
||||
- [ ] Game engine correctly detected
|
||||
- [ ] Engine version identified
|
||||
- [ ] Input system type determined (New Input System, Legacy, Custom)
|
||||
- [ ] Game state manager class located
|
||||
- [ ] Ready/initialized state property identified
|
||||
- [ ] Key domain entities catalogued for ScenarioBuilder
|
||||
|
||||
## Generated Files
|
||||
|
||||
### Directory Structure
|
||||
- [ ] `Tests/PlayMode/E2E/` directory created
|
||||
- [ ] `Tests/PlayMode/E2E/Infrastructure/` directory created
|
||||
- [ ] `Tests/PlayMode/E2E/Scenarios/` directory created
|
||||
- [ ] `Tests/PlayMode/E2E/TestData/` directory created
|
||||
|
||||
### Infrastructure Files
|
||||
- [ ] `E2E.asmdef` created with correct assembly references
|
||||
- [ ] `GameE2ETestFixture.cs` created with correct class references
|
||||
- [ ] `ScenarioBuilder.cs` created with at least placeholder methods
|
||||
- [ ] `InputSimulator.cs` created matching detected input system
|
||||
- [ ] `AsyncAssert.cs` created with core assertion methods
|
||||
|
||||
### Example and Documentation
|
||||
- [ ] `ExampleE2ETest.cs` created with working infrastructure test
|
||||
- [ ] `README.md` created with usage documentation
|
||||
|
||||
## Code Quality
|
||||
|
||||
### GameE2ETestFixture
|
||||
- [ ] Correct namespace applied
|
||||
- [ ] Correct `GameStateClass` reference
|
||||
- [ ] Correct `SceneName` default
|
||||
- [ ] `WaitForGameReady` uses correct ready property
|
||||
- [ ] `UnitySetUp` and `UnityTearDown` properly structured
|
||||
- [ ] Virtual methods for derived class customization
|
||||
|
||||
### ScenarioBuilder
|
||||
- [ ] Fluent API pattern correctly implemented
|
||||
- [ ] `Build()` executes all queued actions
|
||||
- [ ] At least one domain-specific method added (or clear TODOs)
|
||||
- [ ] `FromSaveFile` method scaffolded
|
||||
|
||||
### InputSimulator
|
||||
- [ ] Matches detected input system (New vs Legacy)
|
||||
- [ ] Mouse click simulation works
|
||||
- [ ] Button click by name works
|
||||
- [ ] Keyboard input scaffolded
|
||||
- [ ] `Reset()` method cleans up state
|
||||
|
||||
### AsyncAssert
|
||||
- [ ] `WaitUntil` includes timeout and descriptive failure
|
||||
- [ ] `WaitForValue` provides current vs expected in failure
|
||||
- [ ] `AssertNeverTrue` for negative assertions
|
||||
- [ ] Frame/physics wait utilities included
|
||||
|
||||
## Assembly Definition
|
||||
|
||||
- [ ] References main game assembly
|
||||
- [ ] References Unity.InputSystem (if applicable)
|
||||
- [ ] `overrideReferences` set to true
|
||||
- [ ] `precompiledReferences` includes nunit.framework.dll
|
||||
- [ ] `precompiledReferences` includes UnityEngine.TestRunner.dll
|
||||
- [ ] `precompiledReferences` includes UnityEditor.TestRunner.dll
|
||||
- [ ] `UNITY_INCLUDE_TESTS` define constraint set
|
||||
|
||||
## Verification
|
||||
|
||||
- [ ] Project compiles without errors after scaffold
|
||||
- [ ] `ExampleE2ETests.Infrastructure_GameLoadsAndReachesReadyState` passes
|
||||
- [ ] Test appears in Test Runner under PlayMode → E2E category
|
||||
|
||||
## Documentation Quality
|
||||
|
||||
- [ ] README explains all infrastructure components
|
||||
- [ ] Quick start example is copy-pasteable
|
||||
- [ ] Extension instructions are clear
|
||||
- [ ] Troubleshooting table addresses common issues
|
||||
|
||||
## Handoff
|
||||
|
||||
- [ ] Summary output provided with all configuration values
|
||||
- [ ] Next steps clearly listed
|
||||
- [ ] Customization requirements highlighted
|
||||
- [ ] Knowledge fragments referenced
|
||||
1137
src/modules/bmgd/workflows/gametest/e2e-scaffold/instructions.md
Normal file
1137
src/modules/bmgd/workflows/gametest/e2e-scaffold/instructions.md
Normal file
File diff suppressed because it is too large
Load Diff
145
src/modules/bmgd/workflows/gametest/e2e-scaffold/workflow.yaml
Normal file
145
src/modules/bmgd/workflows/gametest/e2e-scaffold/workflow.yaml
Normal file
@@ -0,0 +1,145 @@
|
||||
# E2E Test Infrastructure Scaffold Workflow
|
||||
|
||||
workflow:
|
||||
id: e2e-scaffold
|
||||
name: E2E Test Infrastructure Scaffold
|
||||
version: 1.0
|
||||
module: bmgd
|
||||
agent: game-qa
|
||||
|
||||
description: |
|
||||
Scaffold complete E2E testing infrastructure for an existing game project.
|
||||
Creates test fixtures, scenario builders, input simulators, and async
|
||||
assertion utilities tailored to the project's architecture.
|
||||
|
||||
triggers:
|
||||
- "ES"
|
||||
- "e2e-scaffold"
|
||||
- "scaffold e2e"
|
||||
- "e2e infrastructure"
|
||||
- "setup e2e"
|
||||
|
||||
preflight:
|
||||
- "Test framework initialized (run `test-framework` workflow first)"
|
||||
- "Game has identifiable state manager"
|
||||
- "Main gameplay scene exists"
|
||||
|
||||
# Paths are relative to this workflow file's location
|
||||
knowledge_fragments:
|
||||
- "../../../gametest/knowledge/e2e-testing.md"
|
||||
- "../../../gametest/knowledge/unity-testing.md"
|
||||
- "../../../gametest/knowledge/unreal-testing.md"
|
||||
- "../../../gametest/knowledge/godot-testing.md"
|
||||
|
||||
inputs:
|
||||
game_state_class:
|
||||
description: "Primary game state manager class name"
|
||||
required: true
|
||||
example: "GameStateManager"
|
||||
|
||||
main_scene:
|
||||
description: "Scene name where core gameplay occurs"
|
||||
required: true
|
||||
example: "GameScene"
|
||||
|
||||
input_system:
|
||||
description: "Input system in use"
|
||||
required: false
|
||||
default: "auto-detect"
|
||||
options:
|
||||
- "unity-input-system"
|
||||
- "unity-legacy"
|
||||
- "unreal-enhanced"
|
||||
- "godot-input"
|
||||
- "custom"
|
||||
|
||||
# Output paths vary by engine. Generate files matching detected engine.
|
||||
outputs:
|
||||
unity:
|
||||
condition: "engine == 'unity'"
|
||||
infrastructure_files:
|
||||
description: "Generated E2E infrastructure classes"
|
||||
files:
|
||||
- "Tests/PlayMode/E2E/Infrastructure/GameE2ETestFixture.cs"
|
||||
- "Tests/PlayMode/E2E/Infrastructure/ScenarioBuilder.cs"
|
||||
- "Tests/PlayMode/E2E/Infrastructure/InputSimulator.cs"
|
||||
- "Tests/PlayMode/E2E/Infrastructure/AsyncAssert.cs"
|
||||
assembly_definition:
|
||||
description: "E2E test assembly configuration"
|
||||
files:
|
||||
- "Tests/PlayMode/E2E/E2E.asmdef"
|
||||
example_test:
|
||||
description: "Working example E2E test"
|
||||
files:
|
||||
- "Tests/PlayMode/E2E/ExampleE2ETest.cs"
|
||||
documentation:
|
||||
description: "E2E testing README"
|
||||
files:
|
||||
- "Tests/PlayMode/E2E/README.md"
|
||||
|
||||
unreal:
|
||||
condition: "engine == 'unreal'"
|
||||
infrastructure_files:
|
||||
description: "Generated E2E infrastructure classes"
|
||||
files:
|
||||
- "Source/{ProjectName}/Tests/E2E/GameE2ETestBase.h"
|
||||
- "Source/{ProjectName}/Tests/E2E/GameE2ETestBase.cpp"
|
||||
- "Source/{ProjectName}/Tests/E2E/ScenarioBuilder.h"
|
||||
- "Source/{ProjectName}/Tests/E2E/ScenarioBuilder.cpp"
|
||||
- "Source/{ProjectName}/Tests/E2E/InputSimulator.h"
|
||||
- "Source/{ProjectName}/Tests/E2E/InputSimulator.cpp"
|
||||
- "Source/{ProjectName}/Tests/E2E/AsyncAssert.h"
|
||||
build_configuration:
|
||||
description: "E2E test build configuration"
|
||||
files:
|
||||
- "Source/{ProjectName}/Tests/E2E/{ProjectName}E2ETests.Build.cs"
|
||||
example_test:
|
||||
description: "Working example E2E test"
|
||||
files:
|
||||
- "Source/{ProjectName}/Tests/E2E/ExampleE2ETest.cpp"
|
||||
documentation:
|
||||
description: "E2E testing README"
|
||||
files:
|
||||
- "Source/{ProjectName}/Tests/E2E/README.md"
|
||||
|
||||
godot:
|
||||
condition: "engine == 'godot'"
|
||||
infrastructure_files:
|
||||
description: "Generated E2E infrastructure classes"
|
||||
files:
|
||||
- "tests/e2e/infrastructure/game_e2e_test_fixture.gd"
|
||||
- "tests/e2e/infrastructure/scenario_builder.gd"
|
||||
- "tests/e2e/infrastructure/input_simulator.gd"
|
||||
- "tests/e2e/infrastructure/async_assert.gd"
|
||||
example_test:
|
||||
description: "Working example E2E test"
|
||||
files:
|
||||
- "tests/e2e/scenarios/example_e2e_test.gd"
|
||||
documentation:
|
||||
description: "E2E testing README"
|
||||
files:
|
||||
- "tests/e2e/README.md"
|
||||
|
||||
steps:
|
||||
- id: analyze
|
||||
name: "Analyze Game Architecture"
|
||||
instruction_file: "instructions.md#step-1-analyze-game-architecture"
|
||||
|
||||
- id: scaffold
|
||||
name: "Generate Infrastructure"
|
||||
instruction_file: "instructions.md#step-2-generate-infrastructure"
|
||||
|
||||
- id: example
|
||||
name: "Generate Example Test"
|
||||
instruction_file: "instructions.md#step-3-generate-example-test"
|
||||
|
||||
- id: document
|
||||
name: "Generate Documentation"
|
||||
instruction_file: "instructions.md#step-4-generate-documentation"
|
||||
|
||||
- id: complete
|
||||
name: "Output Summary"
|
||||
instruction_file: "instructions.md#step-5-output-summary"
|
||||
|
||||
validation:
|
||||
checklist: "checklist.md"
|
||||
@@ -91,6 +91,18 @@ Create comprehensive test scenarios for game projects, covering gameplay mechani
|
||||
| Performance | FPS, loading times | P1 |
|
||||
| Accessibility | Assist features | P1 |
|
||||
|
||||
### E2E Journey Testing
|
||||
|
||||
**Knowledge Base Reference**: `knowledge/e2e-testing.md`
|
||||
|
||||
| Category | Focus | Priority |
|
||||
|----------|-------|----------|
|
||||
| Core Loop | Complete gameplay cycle | P0 |
|
||||
| Turn Lifecycle | Full turn from start to end | P0 |
|
||||
| Save/Load Round-trip | Save → quit → load → resume | P0 |
|
||||
| Scene Transitions | Menu → Game → Back | P1 |
|
||||
| Win/Lose Paths | Victory and defeat conditions | P1 |
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Create Test Scenarios
|
||||
@@ -153,6 +165,39 @@ SCENARIO: Gameplay Under High Latency
|
||||
CATEGORY: multiplayer
|
||||
```
|
||||
|
||||
### E2E Scenario Format
|
||||
|
||||
For player journey tests, use this extended format:
|
||||
```
|
||||
E2E SCENARIO: [Player Journey Name]
|
||||
GIVEN [Initial game state - use ScenarioBuilder terms]
|
||||
WHEN [Sequence of player actions]
|
||||
THEN [Observable outcomes]
|
||||
TIMEOUT: [Expected max duration in seconds]
|
||||
PRIORITY: P0/P1
|
||||
CATEGORY: e2e
|
||||
INFRASTRUCTURE: [Required fixtures/builders]
|
||||
```
|
||||
|
||||
### Example E2E Scenario
|
||||
```
|
||||
E2E SCENARIO: Complete Combat Encounter
|
||||
GIVEN game loaded with player unit adjacent to enemy
|
||||
AND player unit has full health and actions
|
||||
WHEN player selects unit
|
||||
AND player clicks attack on enemy
|
||||
AND player confirms attack
|
||||
AND attack animation completes
|
||||
AND enemy responds (if alive)
|
||||
THEN enemy health is reduced OR enemy is defeated
|
||||
AND turn state advances appropriately
|
||||
AND UI reflects new state
|
||||
TIMEOUT: 15
|
||||
PRIORITY: P0
|
||||
CATEGORY: e2e
|
||||
INFRASTRUCTURE: ScenarioBuilder, InputSimulator, AsyncAssert
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Prioritize Test Coverage
|
||||
@@ -161,12 +206,12 @@ SCENARIO: Gameplay Under High Latency
|
||||
|
||||
**Knowledge Base Reference**: `knowledge/test-priorities.md`
|
||||
|
||||
| Priority | Criteria | Coverage Target |
|
||||
| -------- | ---------------------------- | --------------- |
|
||||
| P0 | Ship blockers, certification | 100% automated |
|
||||
| P1 | Major features, common paths | 80% automated |
|
||||
| P2 | Secondary features | 60% automated |
|
||||
| P3 | Edge cases, polish | Manual only |
|
||||
| Priority | Criteria | Unit | Integration | E2E | Manual |
|
||||
|----------|----------|------|-------------|-----|--------|
|
||||
| P0 | Ship blockers | 100% | 80% | Core flows | Smoke |
|
||||
| P1 | Major features | 90% | 70% | Happy paths | Full |
|
||||
| P2 | Secondary | 80% | 50% | - | Targeted |
|
||||
| P3 | Edge cases | 60% | - | - | As needed |
|
||||
|
||||
### Risk-Based Ordering
|
||||
|
||||
|
||||
Reference in New Issue
Block a user