Hy-Nomas Brings Multi-Turn Text and Camera Control to World Models

Introduction

An interactive world model must do more than produce a plausible video from a single prompt. A user may describe a scene, ask an object or character to act, then move the camera or issue a new instruction. The system must preserve the relevant state, account for earlier consequences, and make the next segment feel like a continuation rather than a restart. Hy-Nomas, introduced by a Tencent research team, is designed around this multi-turn setting.

Key points

• Free-form interaction: Hy-Nomas accepts natural-language instructions and continuous camera control. It generates the next video segment from current inputs together with the available interaction history.
• Variable-length segments: The model is not restricted to a single fixed-duration output. Segment length can vary with the interaction, while cached history connects successive generations.
• Separate persistent context from local actions: Its central design combines structured video captions with action-aware causal attention. Descriptive context remains accessible across segments to preserve the scene and its entities, while action instructions are restricted to their intended temporal windows. This is meant to reduce interference from outdated commands.
• A vision-language Prompt Enhancer: The Prompt Enhancer converts user requests and multimodal interaction histories into structured conditions for the next segment. It provides an intermediate representation of scene, action, and control intent before generation.
• A staged training curriculum: Training progressively introduces causal generation, text interaction, camera control, and model-generated histories, followed by diffusion distillation. The sequence is intended to bridge the gap between isolated video generation and continuing interaction.

Significance and limitations

Many video generators can create attractive results from a single prompt but struggle with identity drift, scene resets, or inconsistent actions over multiple turns. Hy-Nomas treats persistent world information and the execution of a new action as related but distinct problems. That design points toward world models that behave more like controllable simulations than collections of independent clips.

The supplied material reports that Hy-Nomas outperforms existing world models in human evaluation, but it does not provide numerical results. The claim should therefore be read as evidence of stronger overall performance in the evaluated interactive setting, not as proof that long-horizon consistency has been solved.

The release of model weights, code, benchmarks, and a lightweight adaptation workflow is also important. It gives researchers a reproducible system for studying history caching, causal attention, prompt structuring, and camera control together. Potential applications include game prototyping, virtual scene creation, and embodied interaction research, although compute cost, responsiveness, physical reliability, and long-term memory remain open challenges.

Source: Hugging Face Daily Papers