GroundingPI: A Grounding Foundation for Physical Intelligence

Introduction

For embodied systems, recognizing an object is only the beginning. A robot must know which object a command refers to and where that object is before it can grasp, move around, or manipulate it. Autonomous vehicles face a similar requirement when locating cars, pedestrians, and other scene elements. GroundingPI is built around the idea that precise localization should be a first-class perceptual capability for physical intelligence, rather than an incidental feature inherited from a general-purpose vision-language model.

How it works

GroundingPI is a 4B-parameter grounding foundation model. It represents points and bounding boxes as quantized coordinates in a shared vocabulary, then produces those coordinates as generated outputs. This formulation exposes structured spatial information directly and offers a closer interface to downstream control than a purely textual description.

Its training pipeline combines several stages:

• Multimodal and spatial pretraining, linking visual semantics with spatial relationships.
• Supervised fine-tuning, using public datasets and dedicated data engines to broaden task coverage.
• GRPO-based reinforcement learning, which further optimizes grounding behavior and coordinate prediction.

The paper compares the model with 44 baselines across 34 benchmarks covering 11 perceptual capabilities. GroundingPI records an average score of 73.68%, above the reported 71.54% for the larger GPT-6 Astra. The result suggests that parameter count alone does not determine grounding quality; task-specific data and training objectives also matter substantially.

Downstream results

GroundingPI was also evaluated as a visual backbone for vision-language-action and world-action systems. On RoboTwin 2.0, it outperforms every mainstream backbone included in the study across four out-of-distribution settings, with a maximum relative gain of 24.8%. On RoboCasa-GR1, a model trained with 50% of the demonstrations surpasses comparison backbones trained with 75%, pointing to a possible data-efficiency advantage.

In autonomous driving experiments on nuScenes, GroundingPI achieves an average open-loop L2 error of 0.296 meters when used as the visual backbone. Scaling analyses show that larger-scale pretraining improves both manipulation and driving results. The authors also report that dense grounding benefits multiple perceptual capabilities, while OCR data may act as a useful catalyst for perceptual learning.

Why it matters

The broader contribution is a change in emphasis: embodied models may benefit from a dedicated perceptual foundation instead of relying entirely on generic vision-language or video-generation backbones. Fast, structured spatial outputs could serve as an interface between visual understanding and action generation.

The reported evidence still comes mainly from benchmarks, simulated environments, and open-loop evaluation. It does not by itself establish long-horizon reliability under real-world occlusion, sensor noise, or continuous interaction. Further hardware experiments will be needed. Still, the release of code, weights, and an inference stack makes GroundingPI a practical base for follow-up research.

Source: Hugging Face Daily Papers