How Huawei’s Memento 3 Uses World Models to Beat 25 Games

Introduction

An AI agent playing an unfamiliar game must solve two problems at once: it has to discover how the environment works and then complete the task with as few actions as possible. Huawei Noah’s Ark Lab, together with UCL and other collaborators, designed Memento 3 around this challenge.

According to the supplied research report, Memento 3 cleared all 25 games and 183 levels in the public ARC-AGI-3 set, receiving a perfect RHAE score of 100. Human players used about 17,135 actions on the same collection, while the system used 7,518, a reduction of roughly 56%. The notable point is that the system did not create a game-specific model. It used external, executable knowledge to improve its behavior.

Code as a world model

Memento 3 uses Claude Opus 5 as its base model, without updating model weights or fine-tuning for individual games. It maintains two complementary artifacts:

• A rulebook: a living document describing observed mechanics, objectives, action effects, and unresolved assumptions.
• An executable world model: Python code that represents the current hypotheses and simulates how the game state should change.

The agent tests its hypotheses against the real environment. When the predicted screen differs from the observed screen, it revises the rulebook and the corresponding code, then checks the change by replaying earlier actions and comparing the resulting 64×64 frames. This makes the learning process inspectable rather than purely implicit.

One example involved a special floor tile. The agent initially believed that the symbol carried by the player was mirrored after entering the tile. Further interaction showed that the symbol actually rotated 90 degrees clockwise, independently of the entry direction. Memento 3 corrected both its written rule and its executable model. Across a symbol-decoding game, the rulebook went through 11 versions as later levels challenged earlier assumptions. Version history also made it possible to recover from a bad revision.

Ablation results reported in the material suggest that the rulebook matters. Across three games, keeping it reduced total actions from 677 to 617 and cut agent execution rounds by about 18%. In the wa30 game, two independently maintained world models reduced actions from 899 to 597. That second result has only been shown on one game, so it should not yet be treated as a general law.

From puzzles to Pong

The team also tested Memento 3 on Atari Pong, where the agent had to infer ball movement, collision behavior, and paddle control. It first formed an incorrect account of how different paddle regions affected the rebound, then revised the model after observing horizontal reflections and other trajectory errors.

Once the policy became reliable, code took over the control loop. The language model no longer had to decide every frame. In three different starting conditions, the resulting program won 21–0 each time, after learning from approximately 9,500 frames. This comparison highlights a useful division of labor: the model can explore, formulate hypotheses, and debug the model of the environment, while ordinary code executes a validated high-frequency policy.

Why it matters—and what it does not prove

Memento 3 presents a practical form of recursive self-improvement that does not require changing neural parameters. External memory, executable simulation, and versioned revisions can make a frozen general-purpose model more effective within a task family. It also offers a route toward reducing repeated reasoning calls in environments with stable rules and clear feedback.

The result should not be overgeneralized. These games are compact and relatively structured, while exploration, code generation, verification, and Claude Opus 5’s high-intensity reasoning still consume tokens and compute. Real-world tasks often involve incomplete observations, changing rules, delayed feedback, and costly mistakes. Whether this approach can remain reliable there is still an open question.

QbitAI