Velox Uses Spatial Memory to Speed Up Video World Models

Introduction

Video world models offer a natural interface for interactive simulation. Given actions and camera motion, they generate visual rollouts that can extend far beyond the original prompt or observation. Long-horizon interaction, however, creates a tension between coherence and responsiveness. A model must retain enough visual history to keep objects, layouts, and viewpoints consistent, but repeatedly attending to an ever-growing history makes later chunks increasingly expensive.

The paper Velox: Spatial Memory Makes World Models Efficient addresses this problem with a training-free attention router. Rather than changing the underlying model, Velox changes how the model selects historical context for each new region.

How Velox works

• A spatial representation of history: Historical blocks are represented as camera-frustum slabs, each associated with a depth interval. This turns the memory cache from a purely temporal sequence into a structure that can be related to the current view.
• Depth-aware correspondence: Velox estimates the depth of the current block using recency-priority multiwarp. The estimated geometry is then compared with historical frustum slabs to determine which regions are likely to correspond.
• Soft attention modulation: When a current region is well covered by historical slabs, their overlap softly adjusts the block importance produced by the model. Regions without reliable geometric coverage keep the model-derived routing, rather than being forced into a potentially incorrect spatial decision.
• Hardware-friendly granularity: Routing operates on fixed token blocks, matching tiled execution patterns used by FlashAttention-style kernels. Frustum construction can also run asynchronously with denoising on the same H200 GPU.

What the reported results show

According to the supplied abstract, Velox provides a 4.36× attention speedup and a 2.43× denoising speedup at chunk 48. On the full 158-case WBench navigation split, its API-free average score stays within 0.15 points of dense LingBot-Fast-Cam. Based on these results, selective access to history can reduce computation without materially changing the reported navigation outcome.

The important design choice is that Velox does not attempt to make the model store an entirely new form of memory. Instead, it uses geometry to decide which existing memory should be read. For camera-controlled generation, spatial correspondence can be more informative than recency alone: a frame from earlier in the rollout may still matter if it depicts the same region, while a more recent frame may be irrelevant if the camera has moved elsewhere.

Significance and open questions

Velox suggests a practical route toward more responsive world models because it is training-free and can be placed around an existing attention mechanism. Its fixed block structure also connects the algorithmic idea to efficient GPU execution rather than treating routing as an independent preprocessing stage.

There are nevertheless important boundaries to the current evidence. The approach depends on streaming geometry, depth estimation, and usable frustum correspondences. Occlusion, unstable geometry, or poorly covered regions could reduce the benefit, and the abstract only reports results for a navigation benchmark and a specific H200 setup. More evidence would be needed to determine how the method transfers to other video models, tasks, and hardware.

Velox reframes long-context efficiency as a memory-access problem: a world model may not need to inspect more history, but it does need a better way to find the history relevant to the current view.

Source: Hugging Face Daily Papers