From Robot Policies to Agentic Robots: A Framework for Models, Data, and Environments

Introduction

Robot learning has often been framed as a mapping from perception to action: given an observation, predict the next movement. Foundation models are broadening that picture. A capable robot may need to interpret an ambiguous instruction, break a goal into subtasks, choose the right model or tool, act in a changing environment, and revise its plan when execution fails. The survey Towards Agentic Robots: Model, Data, Environment, and Harness uses the term “agentic robots” to describe this emerging direction and offers a unified view of its research landscape.

From action policies to system-level intelligence

Conventional robot policies are useful when tasks and environments are relatively constrained. Real-world operation is less predictable: objects move, instructions are incomplete, and actions may produce unexpected outcomes. An agentic robot is therefore defined not only by its control accuracy, but also by its ability to reason continuously about goals, observations, actions, and feedback.

The survey traces this transition through vision-language-action models, world models, and world-action models. Vision-language-action models connect visual observations and natural-language goals with physical behavior. World models focus on representing environments and anticipating possible outcomes, while world-action models connect such representations more directly to decisions. Together, these lines of work expand the question from “what action comes next?” to “what should the robot do next, and how should it respond to the consequences?”

Four functional domains

The paper organizes existing research into seventeen categories across four connected domains:

• Model: How agents support model learning, task reasoning, planning, and robot decisions, including policy and world-oriented modeling.
• Data: How robot data is collected, curated, filtered, and used. Data becomes part of an improvement loop rather than a static training resource.
• Environment: How worlds, tasks, and tests are constructed for interaction, learning, and evaluation. These settings provide ways to examine whether capabilities transfer beyond a narrow demonstration.
• Harness: How models, tools, and capabilities are orchestrated into a usable robot system that can plan, execute, receive feedback, and recover from errors.

Why the framework matters

This work is a survey, not a proposal for a single new robot model. Its value lies in addressing fragmentation. The word “agent” can refer to a task planner, a data-collection process, a training coordinator, or a system that directly operates a robot. Without a shared scope, it is difficult to compare studies or understand how an individual capability contributes to an end-to-end system.

The four-domain view also provides a practical checklist. Can the model understand the task? Does the data support adaptation? Does the environment test meaningful behavior? Can the harness coordinate planning, action, feedback, and recovery? These questions suggest that deployment challenges are not limited to the performance of one foundation model. They also concern the interfaces between models, data pipelines, simulated or physical environments, and execution systems.

Open challenges

The survey highlights the lack of a consistent recipe for combining agentic functions into reliable robots. Long-horizon interaction, autonomous error correction, evaluation, and safe operation all require more than isolated demonstrations. Future progress may depend on stronger data loops, more useful world representations, and system architectures that can revise their strategies under uncertainty. The central shift is therefore architectural: robotics is moving toward systems that do not merely perform actions, but organize the process of achieving goals.

Source: Hugging Face Daily Papers