ProVisE Tests Spatial Reasoning by Letting Models Draw the Answer

Introduction

Spatial intelligence is central to building agents that can move beyond static image understanding and interact with the physical world. Yet many spatial questions are not naturally answered with a sentence, a multiple-choice option, or a pair of coordinates. In everyday reasoning, we often point to a region, trace a route, circle an object, or mark a boundary.

The paper “Show, Don’t Tell” focuses on this gap between task semantics and answer format. Most existing spatial reasoning benchmarks require textual or structured outputs, which works well for text-output vision-language models but creates friction for image-generation models. These models can express spatial judgments directly in pixels, but evaluation pipelines often force them into formats that may not match their strengths.

Key ideas

• ProVisE turns visual answers into measurable predictions
  The authors propose ProVisE, short for Protocolized Visual Evaluation. It is designed to be benchmark-agnostic: for a given task, ProVisE defines a constrained visual response protocol, elicits an image-based answer from a generation model, then parses that response into the structured prediction expected by the original benchmark. Scoring can therefore reuse the benchmark’s existing metrics.

• The framework tackles an answer-interface mismatch
  Many spatial tasks are grounded in continuous visual scenes. Locations, areas, paths, and relationships may be more naturally represented as marks on an image than as discrete text. If a benchmark only accepts coordinates or language, the measured score may reflect formatting limitations as much as spatial competence. ProVisE tries to compare models under the same task meaning while allowing different output interfaces.

• SpatialGen-Bench provides a focused diagnostic testbed
  The paper also introduces SpatialGen-Bench, a curated benchmark with 470 samples covering 14 spatial subtasks, four capability levels, and diverse answer forms. Rather than serving only as a leaderboard, it is positioned as a diagnostic tool for understanding where different model interfaces succeed or fail.

• Visual and textual answering are complementary
  The authors evaluate 31 text- and image-answering systems in a unified setup and validate the agentic protocol-construction process on six external spatial benchmarks. The reported results show a split: visual answering is competitive when the answer can be directly expressed in pixels, while text-output VLMs remain stronger on compositional reasoning tasks. One notable finding is that GPT Image 2 correctly solves 37% of the spatial cases missed by GPT-5.4.

Why it matters

The broader lesson is that evaluation is not only about datasets and metrics; it is also about the answer channel. If a spatial task is inherently visual, restricting every model to text may underestimate image-generation systems or obscure what kind of spatial reasoning is actually being tested.

This matters for embodied AI, visual agents, and robotics-oriented systems. In real-world interaction, spatial decisions often become actions, trajectories, or region selections rather than verbal explanations. A framework that allows models to “show” spatial understanding in pixels and then translates that output into standard metrics could be a useful bridge between generative visual ability and interactive intelligence.

At the same time, the paper does not claim that visual answers replace language-based reasoning. Its results point to complementarity: image-generation models can shine when spatial states are directly externalized, while text VLMs still have an edge in complex compositional reasoning. Future spatial intelligence evaluations may need to support both modes—asking models not only to explain, but also to draw what they know.

Source: Hugging Face Daily Papers