Giving 3D Diffusion Policies Foresight Without Explicit Trajectories

Introduction

Robot manipulation is rarely a sequence of independent reactions to single images. In tasks such as grasping, pushing, and placing, the action that is appropriate now often depends on how the object–gripper interaction is likely to evolve. 3D diffusion policies are effective at generating geometrically grounded actions from current observations, but their ability to anticipate future interaction is usually left to emerge implicitly from action learning.

A team from Dalian University of Technology proposes Movement Trend Guidance, a lightweight way to add this form of foresight without asking the policy to produce an explicit plan. The method builds on DP3 and retains its dense-action prediction and receding-horizon execution formulation.

How the method works

• A short history becomes a trend representation: Instead of relying only on the current frame, the policy processes a brief observation history and encodes the evolution of the interaction in a compact latent variable.
• Future supervision is used only during training: Sparse future gripper states supervise the latent representation, encouraging it to capture where the interaction is heading. At inference time, future states are not required; the policy uses the latent inferred from available history.
• Global conditioning plus bottleneck modulation: The latent provides global conditioning for action generation. A gated FiLM branch is also inserted at the UNet bottleneck so that trend information can influence a particularly important feature stage without broadly redesigning the network.
• No explicit trajectory planner is added: The policy continues to generate dense actions and execute them in a receding-horizon loop. The proposed component supplies directional context rather than a complete sequence of waypoints.

Reported results

In the 50-task mixed-training setting of RoboTwin2.0, the proposed approach reaches 62.8%, compared with 56.1% for DP3. On LIBERO-40, the reported results are 71.93% versus 37.08%. Across five real-robot tasks, the method achieves 72.0% compared with 49.0% for the baseline. The paper also reports that the addition increases the parameter count by only 3.52%, while improving over DP3 across the RoboTwin2.0, LIBERO-40, and DexArt evaluations described by the authors.

The central message is that a policy does not necessarily need an explicit future trajectory to become more anticipatory. A latent variable trained against sparse future states can provide useful directional information while remaining compact and implicit during deployment.

Why it matters

The work separates two capabilities that are often entangled in robot learning: estimating how an interaction is evolving and generating the next concrete action. This lets the diffusion policy retain its flexible action-generation behavior while reducing its dependence on purely local, frame-by-frame reactions. The small parameter overhead also makes the approach relatively straightforward to test as an extension to existing 3D diffusion policies.

The available material does not provide detailed ablations of history length, future-supervision density, or alternative trend representations. It therefore remains open how interpretable the latent is across tasks, how sensitive it is to observation noise, and whether the same benefit transfers to longer and more complex manipulation sequences.

Source: Hugging Face Daily Papers