EmbodiedLab Tests Whether Agents Can Act to Know

Introduction

Many embodied-AI benchmarks ask a robot to reach a known target: pick up an object, move it, or arrange a scene into a specified state. Such tasks are useful for testing perception, planning, and control, but they leave out a central part of scientific work: what happens when the answer is not known beforehand and the agent must discover it through action?

EmbodiedLab is designed around that question. Its central idea is “act to know,” rather than merely acting to change the world. Given an objective and a laboratory apparatus, an agent must choose its own procedure, operate a physics-based simulated lab, read instruments through cameras, and explore until it can determine the result required by the task.

Key points

• Unknown outcomes instead of prescribed states. Parameters are resampled for every experiment instance, so an agent cannot simply recall the answer from training or from a fixed task template.
• The agent owns the experimental procedure. It must decide what to manipulate, what to measure, how to order the steps, and when the evidence is sufficient.
• Knowledge is evaluated quantitatively. The benchmark includes 30 experiments spanning five scientific disciplines and 14 apparatus platforms. Scoring checks what the agent learned, whether it achieved the required physical state, and whether its conclusions are supported by readings collected during the episode.
• The full interaction loop is tested. The study evaluates six frontier models in their developers’ coding agents, comparing native observation and control with additional perception, motion planning, and distilled manipulation skills.

Why it matters

Under native control, even the strongest evaluated agent succeeds less than half of the time. For every agent, most failures come from interacting with the apparatus rather than from abstract scientific reasoning alone. A model may be able to propose a plausible experiment, yet still fail to execute it reliably, read the instruments correctly, or maintain the physical configuration needed for a valid measurement.

The comparison also highlights the importance of basic manipulation. Adding manipulation skills helps weaker agents more than adding perception and motion planning. This does not make perception or planning irrelevant; instead, it suggests that experimental embodied intelligence may first be limited by the fundamentals of interaction: locating controls, applying the right action, preserving the apparatus state, and adjusting behavior when feedback changes.

EmbodiedLab brings scientific discovery, active exploration, and robot control into one evaluation loop. It asks more than whether a model can answer a science question or describe an experimental protocol. The agent must execute a procedure under unknown conditions, collect evidence, and produce a conclusion that can be checked against the recorded observations. That makes the benchmark useful for separating agents that can talk about experiments from systems that can actually conduct them.

Source: Hugging Face Daily Papers