RoboFollow Exposes the Instruction-Following Mirage in Embodied AI

Introduction

Task success is often treated as a convenient proxy for intelligence in robotics. Yet a robot can complete an action without actually following the instruction that was given. If a scene contains one obvious object, one valid goal, and one feasible motion, a policy may simply recognize the scene and execute its default behavior. Language can be present in the input without playing any causal role in the decision.

RoboFollow calls this gap the instruction-following mirage. Its explanation is a structural property of many benchmarks: low scene entropy. In a low-entropy scene, visual observations almost uniquely determine the task, so the model is not forced to resolve ambiguity through language.

Making language necessary

RoboFollow raises the entropy of the scene rather than merely adding longer instructions. Multiple tasks share the same scene configuration, but their action requirements differ in kinematically meaningful ways. Depending on the instruction, an agent may need to select another object, identify a spatial relation, obey a trajectory constraint, or choose a different logical branch. Vision alone is therefore insufficient; the policy has to connect linguistic content with the current scene.

The benchmark is built on RoboTwin and includes four scene families, expert demonstrations, evaluation protocols, and released code and data. Its design focuses on whether behavior changes when the instruction changes, not simply on whether some action eventually succeeds.

A layered diagnostic protocol

• High scene entropy creates several valid task branches in a shared visual setting.
• L0-L3 evaluation progressively perturbs visual layout and semantics. The levels probe spatial relations, attributes, trajectory constraints, and logic.
• Intent and Execution scores separate choosing the correct task from carrying it out physically.
• Controlled interaction simplifies objects and restricts actions to the trained repertoire, reducing unrelated motor difficulty.

This distinction is important. A failed episode may result from misunderstanding the instruction, selecting the wrong object, or executing the right plan poorly. A single success number cannot reliably separate those causes.

What the evaluation suggests

The authors evaluate nine VLA and WAM policies. Where policies achieve strong L0 results, that performance does not consistently transfer to L1-L3 under the reported fine-tuning setup. Several representative interventions—including stronger vision-language backbones, question-answering co-training, LangForce, and Classifier-Free Guidance—also fail to close the gap.

The result should not be read as proof that every existing robot policy ignores language. Rather, it shows that conventional benchmark success can leave this possibility unresolved. A system may be competent at visual recognition and motor imitation while remaining fragile when language must distinguish between otherwise plausible actions.

Why it matters

RoboFollow reframes instruction following as a measurable decision problem before it is a control problem. Future datasets may need counterfactual instructions, multiple tasks per scene, and explicit reporting of intent selection alongside execution. Model training may also benefit from stronger binding among language, scene state, and action planning.

Its broader lesson is simple: if an evaluation never makes language necessary, a high score cannot establish that the robot used language. RoboFollow offers a more demanding diagnostic lens for identifying that hidden limitation.

Source: Hugging Face Daily Papers