builds/.agents/handoffs/forge-windows-parallel-replay.md

# Forge: Windows A/B of parallel draw-list replay + recording (engine a967d12b)

Why: Windows sits at 45–50 fps with the RTX 5080 idle. The render thread is the limiter (10–13 ms wall, `end_frame`
replay + submit 3.8–4.8 ms). The NVIDIA/WDDM driver path is paid serially on that thread for every `vkCmd*`.
Engine `06ecc5df` can cut large draw lists into chunks and record them into secondary command buffers on worker
threads (Vulkan only, **default off**). Design + Mac results: engine `.agents/wiki/parallel-draw-replay.md`.

Since then the engine also does **parallel recording** (Forward+ splits a pass's render list over worker threads, each
recording its own slice of the draw list; every driver; default off) — `gpucull record on|off|status`. It attacks the
other half of the render thread (Godot's per-draw bookkeeping before the driver). Same page, § Parallel recording.

Nothing is published or pinned for this yet. This is a test build on Forge only.

## 1. Build + oracle (Vulkan, Khronos layer)

1. `git checkout a967d12b` in the engine (replay + recording), build the Windows editor with the `windows-x64` manifest flags.
   GodotSharp regen is not needed for the test: the game reaches the new API through `Call`.
2. With the Vulkan SDK's validation layer available (`--gpu-validation` turns it on):
   `bash tests/gpu_oracle/parallel_draw_replay/run.sh <tinqs.windows.editor.x86_64.mono.exe> vulkan`
   Expect `ORACLE: 10 passed, 0 failed`, exit 0, **zero `VUID` / validation lines** in the output. Paste the
   output back.

## 2. In-game A/B (box otherwise idle, one engine, nothing else running)

Laptop setup first, and write the settings into the report (each one alone can move fps 8–15 %):
- Plugged in; Windows power mode **Best performance**; the maker's performance/turbo profile.
- **MUX switch on dGPU only** (Ultimate / Discrete mode in the maker's app; needs a reboot). In hybrid mode
  every frame is copied through the AMD integrated GPU to the panel. If the laptop has no MUX, say so.
- Record the CPU name (`Get-CimInstance Win32_Processor | Select-Object Name, NumberOfCores, NumberOfLogicalProcessors`).
  On a Ryzen AI 9 HX 370-class chip only 4 cores are full-speed; note in Process Explorer (Threads) which cores the
  engine's busiest threads sit on.

The game pins `4.7.2@e64890fd3`; skip the pin for this test build:

```bash
TINQS_ENGINE_SKIP_CHECK=1 GAME_ARGS=--gpu-profile GAME_RES=1600x900 bash tools/game.sh run
READY_TIMEOUT=400 bash tools/game.sh ready
curl -s -X POST 127.0.0.1:4328/command -d '{"command":"host_game"}'   # if it stops at the menu
# wait until in the world with fps settled (~50 s+), then:
curl -s -X POST 127.0.0.1:4328/command -d '{"command":"gpucull","value":"replay status"}'   # supported True
LOG="$APPDATA/Tinqs/app_userdata/Ariki/logs/godot.log"
OUT=.game-cli/replay-windows-$(date +%Y%m%d-%H%M)
ARIKI_API=http://127.0.0.1:4328 python tools/render_thread_profile.py --log "$LOG" \
  --poses tools/render_thread_poses.json --out "$OUT" --blocks 8 \
  --states "both off=gpucull:replay off;gpucull:record off" "replay=gpucull:replay on;gpucull:record off" \
           "record=gpucull:replay off;gpucull:record on" "both on=gpucull:replay on;gpucull:record on" \
  --restore "gpucull:replay off;gpucull:record off"
```

Also take three `curl -s 127.0.0.1:4328/mainthread` reads with replay off and with replay on (from `e3582841a`).
They show which thread paces the frame.

Also: `curl -s -X POST 127.0.0.1:4328/command -d '{"command":"occlusion","value":"off"}'` vs `on` at the spawn pose
(5 `/perf` reads each) — the 1.2–1.3 ms "Update Occlusion Buffer" label decides whether CPU occlusion culling earns its
cost now that vegetation is GPU-culled.

Also, after the run above (it can be its own run if the first one has already started), GPU depth-pyramid
occlusion for the gpu-culled vegetation (`rendering/occlusion_culling/gpu_hzb`, default off). It costs the render
thread nothing. On the Mac it cut kept plants by 16 % and GPU time by ~2 ms; it has never been measured on Windows:

```bash
OUT=.game-cli/hzb-windows-$(date +%Y%m%d-%H%M)
ARIKI_API=http://127.0.0.1:4328 python tools/render_thread_profile.py --log "$LOG" \
  --poses tools/render_thread_poses.json --out "$OUT" --blocks 8 \
  --states "hzb off=gpucull:hzb off" "hzb on=gpucull:hzb on" \
  --restore "gpucull:hzb off"
```

Eye check with `hzb on`: turn the camera fast past a hill and a house. Plants must not pop in a frame late or
vanish while visible.

Control, its own launch after the runs above: rendering on the main thread (`--render-thread safe`) instead of the
project's separate render thread. A frame should cost max(main, render) with the separate thread and main + render
without it; Windows reads ~22 ms against a ~11 ms `_draw` wall, which looks like main + render.

```bash
TINQS_ENGINE_SKIP_CHECK=1 GAME_ARGS="--gpu-profile --render-thread safe" GAME_RES=1600x900 bash tools/game.sh run
# same ready / host_game / settle as above, then the 4-state profile again with OUT=.game-cli/safe-windows-$(date +%Y%m%d-%H%M)
```

Plus three `/mainthread` reads with the separate thread (default launch) and three with `safe`, all with
replay + record off. Report `mainWorkMs`, `tailAndRenderSyncMs`, `verdict` p50/p95 for each. If `safe` costs about
the same frame time as `separate`, the separate thread is not overlapping anything on Windows.

Report per pose and state: frame ms, `_draw` wall, `end_frame`, render-CPU labels, fps p25, `/perf
gpucullCounters.parallelReplay` (lists / chunks), pixel diff vs the A/A floor, and the full `results.json`.

## 3. Soak + eye check

`gpucull replay on`, walk 10 minutes (shore, inland, village, a tree felling), with `--gpu-validation` on a second run:
no device lost, no `PARALLEL-REPLAY-BUG`, no validation errors. Eye check where tree/rock shadows fall on open sand.
A stepped shadow edge was seen in one Mac capture and is probably a static-tile boundary; compare replay off vs on
at the same spot.

## Decision rule

Default flips on (`rendering/rendering_device/parallel_draw_replay/enabled=true` in the game's `project.godot`,
then publish + pin) only if: oracle clean, soak clean, and frame time / `end_frame` drop at all 3 poses.
`gpu_hzb` flips on (game `project.godot`) if frame time and render-GPU ms do not rise at any pose and the eye check
is clean.

## 2026-10-19 Engine-side test results on Forge (ZCode) — build a967d12bf, NOT published (test-only per this handoff)

Tooling note: Vulkan SDK + Khronos validation layer and the D3D12 debug layer (Graphics Tools) are now
INSTALLED on Forge (admin install, C:\VulkanSDK) — the "--gpu-validation owed" notes in the manifest are
resolved from here on.

1. parallel_draw_replay oracle, vulkan, --gpu-validation (RTX 5080): **ORACLE: 10 passed, 0 failed, exit 0**
   (replay 64x8/8x32/120x2/16x3, record 96x8/8x32/200x2, record+replay 16x5, inline-again — all pixel-identical,
   real chunk splits up to 160 chunks).
   **VUID FINDING (not zero-VUID as this handoff expected):** 10x VUID-vkBeginCommandBuffer-commandBuffer-00050
   ("commandBuffer must be in the initial state … pool must have VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT").
   They fire in the FIRST parallel state (after the inline reference, before its PASS) and hit the duplicate limit.
   Control: 120 frames of the same project, validation on, feature OFF → 0 VUIDs. So the parallel path re-begins
   secondary command buffers from pools without the RESET bit — spec-UB that the current NVIDIA driver tolerates
   (and MoltenVK runs never flagged). Benign today, but a candidate fix is the pool flag or explicit
   vkResetCommandBuffer per chunk reuse. Pixel identity is unaffected.
2. draw_indirect_count oracle, d3d12, debug layer ON (run.sh): **22 passed, 0 failed, exit 0**, with the expected
   `ORACLE SKIP base-instance cases: D3D12 shaders see base instance 0` line and zero D3D12 ERROR lines — this now
   matches the wiki Windows checklist exactly (the earlier 22/2 was the no-debug-layer run where the cases ran
   instead of skipping). Old "owed" item closed.
3. multimesh_cull_pool oracle, d3d12, debug layer ON: **4524 passed, 0 failed, exit 0**, zero D3D12 ERROR lines —
   full parity with Mac Metal (4524/4524). The "base-instance gap may bite pooled draws" concern did NOT
   materialize; per the G2 handoff this un-gates making pool the default. Old "owed" item closed.

Still owed on Windows: the in-game A/Bs of this handoff (replay/record 4-state + occlusion on/off at the spawn
pose, render_thread_profile.py tables), the 10-minute replay soak + shadow-edge eye check, the tests=yes unit
suites (CrashReport/FreezeDetector/FreezeWatchdog/AgentLog/GpuCullArena/StaticShadow), and the 8-step crash
checklist. Publishing windows-x64 a967d12b + the default flip remain Ozan's call after the VUID finding.