How an AI Drove a PS2 RPG Through Its Own Frame Loop
Host simulation, an instrumented story runner, ordinary controller replay, and live decisions caught different classes of bugs.

I wanted the agent to exercise The Last Signal without requiring me to play through a story I’m deliberately saving for later. That set up a useful testing problem. A route can pass in the host application and still fall apart when motion runs through the PS2 frame loop. Replaying a route can find that problem while making the same dialogue choices every time. The project ended up with several ways to drive a build.
The shared code made an early test strategy straightforward. The campaign and RPG rules are C code used in both the native and desktop builds. A host test can create a state or load a retained checkpoint and inspect what happens when it advances. Once the claim includes the actual player experience, input sampling, scene transitions, audio, and the behavior of the fixed-step clock matter too.
Four forms of coverage
| Method | Input and observation | Evidence |
|---|---|---|
| Host checks | Call the shared C game code with authored fixtures and known expectations | Story, quest, dialogue, progression, and other regressions. The November 15 record reports 467 stock checks passing. |
| Instrumented PCSX2 journey | A test driver inside the PS2 build moves through the authored main route and collects frame timings and state | The November 15 voiced run reached the credits over 22,922 pictures. It logged two placements made when navigation failed to line up. Fights were exercised separately. |
| Controller replay | A route recorded as stick and button inputs runs through the ordinary game’s frame loop | The November 14 record reports a title-to-credits route with 94,566 pictures and about 650 walk-and-talk actions. |
| Live controller channel | Claude reads game-reported state and chooses the next action in four selected stretches | About 1,500 actions and 340 screens of text. It found continuity, UI, dialogue, and balance issues. |
Each row has different authority. The instrumented driver is useful for timing coverage and game-state checks. The replay exercises the input path and ordinary gameplay timing. The live channel exercises the consequences of decisions the agent makes as it goes.
The measurements were split alongside those layers. A separate walking census put every room under load with the camera turning. The fight tour ran fights with voices on and counted slow pictures through outcomes like a loss or a win. The story runner covered the main route and dialogue. There are 33 fights in the later tour, including two that needed the real choices and state surrounding them to start. An early version of the tour was timing the rooms without actually entering those two battles. Fixing the input to the test was part of making the performance claim defensible.
The controller interface
The replay tool feeds controller inputs from a recorded file. For a live session, the emulator exposes a host folder where the agent can send its next action. The game reports the room, conversation text and choices, or enough combat state to select an action. The loop for the agent is small.
- Read the state after the last input.
- Select a direction, button, or wait from what the game reports.
- Inject the next input and wait for a fresh state.
The logs can be replayed to recover representative screenshots. During live play the agent primarily read the exposed game state. The captures were separate visual evidence for selected moments.
In the implementation, LIVE.TXT on the test disc enables the live channel. The game opens host:PADLIVE.TXT and consumes new lines appended by the driver. An ordinary replay reads PAD.TXT and sequences the recorded actions in the same input loop. Live commands can express a wait, a walk to a point, or a button action. The game emits state after the actions so the driver can see how far it got. That keeps the critical feedback step observable.
There was a practical failure in the first live transport. Reopening the host file on every poll consumed emulator handles that weren’t reclaimed in that environment. Opening it once and reading forward made the approach usable long enough to test. A later fix retried if the host file was missing at startup. Both are examples of the tooling needing its own small lifecycle contract.
What to log at a controller boundary
The main loop normally samples the pad once per rendered frame and advances a 60 Hz fixed-step simulation. It queues button edges and tags them with the mode, scene, and dialogue node they came from. A confirmation sampled in one dialogue state should not silently confirm the next menu. The queue is bounded, and a controller disconnect clears queued input and stops play. An automation channel that drives this boundary can exercise those transitions as a player sees them.
For the live sessions, the durable artifact was the sequence of inputs and the results the game reported. With the same route logged, the project could reproduce a line that assumed a different branch, review four replies that dropped the player out of the conversation without a response, and capture the alternative case afterward. The evidence here is an input and an observed state. A developer can compare that pair with what the scene author meant.
The bug at 60 updates a second
The game normally simulates at 60 steps per second. Some earlier route checks took steps of a tenth of a second. A staged walk coordinated a guide and followers through a doorway, and at small steps a follower could get into a position it couldn’t route out of while the guide waited. Larger steps had hidden the stall.
There were two problems to address. A follower already in another actor’s occupied space needed permission to leave it. The next room’s route map also needed to be ready during the staged entry. The changes avoided promising the same impossible search again each frame, kept companions from stopping where the guide was about to be, built the required map during that entry, and bounded how long the guide would wait. The project added 1/60-second checks for the story and side stories.
There was a performance symptom as well. The sample stalled at about 22 FPS while the sequence waited. A game can look slow because it keeps asking for work that it will never be able to finish. To me, that’s the most useful result of putting the game on its own clock during testing.
The difficulty tests needed actual campaign state
The live sessions also made it clear that a solution built around a fully supplied party was smoothing over real problems. The team ran fight simulations from parties assembled along the story, with agents ranging from hands-off to deliberately well-equipped. Bringing fewer supplies into consecutive fights mattered. A hidden route problem could also change the balance when enemies overlapped.
The difficulty pass used those simulations to propose per-fight settings, followed by fresh tries and a smaller set of live fights. A separate tour measured frame time through encounters. Fight balance, fight rendering, and whether a player can select an action under pressure need separate evidence.
The November 14 difficulty record describes the modeled players and their starting condition. On Normal, the untrained hands-off model won 27–100% depending on the fight, with a typical result of 55%. The average model won 62–100%, typically 93%. On Hard, the tidy model with shop gear had a typical result of 73%, while the tidy player with nothing bought typically won 30%. These are sampled simulation outcomes. The shop loadout was granted for the test, so the model’s win rate doesn’t answer whether a player can afford the gear. For consecutive encounters, the report tested a few boards separately and showed how supplies changed the outcome.
The fixes themselves reached into gameplay and navigation. A dressing began restoring roughly a third of full health. Story and Normal let idle party members use a dressing when injured, and changing the difficulty restores the intended default for that rule. The simulation also exposed an overlapping-enemy case where nobody could route to either target. That was a movement defect, and tuning damage values around it would have preserved the wrong behavior.
A save failure belongs in a playtest
The November 14 emulator card checks deliberately damaged each slot in turn, filled a card, and ran without a card. When one save was damaged, the other could still be loaded. A first save onto a full card returned an error and let the player keep going. The test also checked an overwrite of an existing save and a subsequent save once space was available. In the story run, the saved payload was read back and compared with what the game meant to write.
Those are integration checks against the emulator’s memory card. The separate host migration tests cover files written by older builds. A real card, especially a card removed while a write is in progress, still needs physical testing to support the corresponding claim.
What I can take from the results
The recorded runs support specific claims about the November 14 and 15 builds in PCSX2. They show that the campaign can reach the credits under particular inputs, expose problems at real simulation steps, and demonstrate that an agent can choose inputs from reported state. The actual time a person spends reading, the feel of fights with a controller, and the first-play mystery are still things I want to judge in play.
The evidence comes from docs/CURRENT_BUILD.md, the input and state records under build/claude-harness/live-play-2026-11-14/, the PAD.TXT replay tooling, tools/sim/run_journey_fine_steps.sh, and the dated combat grid. They record the routes and assumptions behind each result. That makes an article about the test useful even after a different build takes its place.
Continue with the build and renderer history or the frame-capture reference study.