BlenderWorld Tests Whether Visual Agents Can Really Create in 3D

Introduction

Generating a plausible image is very different from delivering a usable 3D project. A capable 3D agent must infer shape, materials, scale, lighting, and spatial relationships from references, operate Blender, and revise its work after inspecting the result. BlenderWorld is designed to measure that broader workflow rather than rewarding a single attractive render.

A benchmark built around complete scenes

BlenderWorld contains 223 tasks spanning nine domains and 22 subcategories. The examples cover architecture, product design, natural environments, character animation, and scientific visualization. Inputs are deliberately varied: a task may be driven by text, an image, a video, or a structured specification file. This makes the benchmark closer to practical production settings, where creators combine several kinds of reference material.

The expected output is a complete, editable Blender project. The benchmark defines 2,651 task-specific binary criteria across 13 aspects, including geometry, spatial layout, materials, and functionality. Submitted projects are rendered from eight automatically selected key views. A vision-language model judges the resulting images, while an agent-based judge examines the underlying project structure. Overall quality is also scored across eight dimensions, including realism and physical plausibility.

This protocol addresses a weakness in many visual-generation evaluations. A scene can look convincing from one camera angle while containing missing surfaces, incorrect proportions, intersections, or an unusable internal structure. Multi-view rendering exposes inconsistencies, and project inspection helps distinguish a genuinely constructed scene from a visual approximation that only happens to look right in one frame.

Broad understanding is not enough

The study evaluates 13 frontier models. GPT-6-Astra records the highest fused score at 79.9, followed by Opus-5 at 73.0 and GPT-5.6-Sol at 67.7. The more revealing result is that even the best-performing model satisfies all fine-grained requirements on only 42.2% of tasks.

That gap points to a specific limitation in current visual agents. Models can often identify the subject and produce a reasonable high-level composition, but they have difficulty preserving exact positions, proportions, connections, and functional relationships. When the input is a reference image or video, they may reproduce the visible appearance without recovering the underlying structure. Errors introduced during an early modeling step can also spread through later edits, especially when the agent lacks a stable representation of scene state.

Why the benchmark matters

BlenderWorld is useful not simply because it produces a leaderboard, but because it turns 3D creation into a set of diagnosable capabilities. Fine-grained criteria can reveal whether a failure comes from geometry, materials, layout, or functionality. Multi-view outputs and inspection of the actual Blender file make the evaluation more relevant to professional workflows than image-only scoring.

For model developers, the results suggest that progress will require more than stronger visual recognition. Agents also need dependable tool use, persistent scene memory, geometric constraint reasoning, and iterative correction driven by rendered feedback. For artists and designers, current systems may be valuable for blockouts, initial scene construction, and localized edits, but human review remains important for scale, structure, and editability.

The project also provides public code and data, together with an interactive gallery for comparing model-generated scenes. These resources can support more systematic research into how agents plan, build, inspect, and repair 3D worlds.

Hugging Face Daily Papers