GroundAnything Brings Parallel Diffusion Decoding to Visual Grounding

Introduction

Visual grounding requires more than identifying an object in an image. A model must connect a query to the correct entity, predict its location, and sometimes express spatial relations with other entities. Conventional autoregressive (AR) systems serialize these outputs token by token. That design introduces sequential latency and suggests a left-to-right causal order that may not exist in the grounding problem itself.

GroundAnything starts from a different interpretation: grounding is visual-evidence extraction. Objects, coordinates, and relations are jointly constrained by the image and the query, so their predictions can be proposed and refined together. This makes bidirectional diffusion a natural alternative to strictly sequential decoding.

How the model works

GroundAnything is a 4B-parameter grounding foundation model. Its training and inference design includes several components:

• Bidirectional diffusion decoding: spatial hypotheses are generated in parallel and progressively refined through iterative denoising.
• Blockwise denoising: predictions are organized into blocks that can be jointly optimized, preserving structure while reducing token-by-token dependence.
• AR-to-diffusion conversion: the pipeline begins with grounding pretraining on public datasets and data produced by dedicated engines, then converts an AR model to diffusion while retaining joint AR and diffusion objectives.
• Post-training: supervised fine-tuning is followed by GRPO-based reinforcement post-training.
• Flexible decoding: entropy-guided decoding and an optional self-speculative mode provide different trade-offs between quality and latency.

Results and speed trade-offs

On 30 grounding benchmarks, GroundAnything-VLM, the autoregressive variant, reports an average score of 72.42%. According to the supplied results, this is a state-of-the-art overall result among similarly sized models and is slightly above GPT-6 Astra at 71.35%.

The diffusion model paired with entropy-guided decoding averages 61.75%, compared with 53.32% for the similarly sized, fast MTP-based LocateAnything model. The work also reports that self-speculative decoding reaches a 4.51× speedup over the AR counterpart, with a 0.74 percentage-point drop in COCO F1m.

The project’s presentation summarizes the practical gain as roughly 4× faster inference. That figure should not be treated as universal: real speed depends on hardware, decoding settings, batch size, and the evaluation protocol.

Why it matters

The main contribution is conceptual as much as architectural. GroundAnything tests whether diffusion-style parallel generation can handle structured visual outputs without giving up precise localization. Lower serial latency could benefit interactive image understanding, embodied perception, and other applications that require frequent spatial queries.

There are also open questions. Diffusion inference still involves multiple denoising iterations, and the available material does not fully establish how throughput, memory use, and robustness vary across deployment environments. The model is therefore best understood as a practical compromise between AR precision and parallel decoding, rather than a universal replacement for autoregressive grounding.

The project releases code and weights, allowing further evaluation of its decoding strategies across tasks and hardware.

Source: Hugging Face Daily Papers