VeriPhy Makes Physical Reliability in Generated Video Auditable

Generated video models are becoming better at producing scenes that look convincing at a glance. That visual fluency, however, does not guarantee physical reliability. An object may change position without a plausible transition, a count may drift across frames, or an action may occur in the wrong order. A single quality score cannot tell developers which obligation failed or where in the clip the failure occurred.

VeriPhy addresses this problem by treating video evaluation as a verification workflow. Before inspecting any frame, a text-only planner compiles the prompt into typed physical obligations. These obligations can concern referenced objects, quantities, spatial relations, motion, or the timing of an event. The planner also creates an execution plan that is checked statically, restricting which analyses may be called and how they relate to the declared claims.

At runtime, observations only gate calls that were already declared. The system uses frozen low-level experts for tasks such as segmentation and tracking, counting, depth estimation, OCR, and audio-event detection. It also applies eleven typed physical measurements to the resulting tracks. Each action emits an evidence record carrying provenance. When a result is usable, its payload is either a typed measurement or an explicitly tagged learned state, making the intermediate reasoning easier to inspect than an unconstrained model response.

Key points

• Prompts are compiled into explicit physical obligations before frame analysis.
• Tool calls are declared and scoped rather than freely invented during evaluation.
• Evidence records preserve the path from an observation to a final decision.
• A three-valued logic distinguishes supported, contradicted, and unknown cases.
• The system is built around human-localized failures in reference, space, and time.

The reported evaluation uses a 1,500-clip corpus with human-annotated flaw records. On a 149-clip core containing 304 records, VeriPhy accounted for 228. A published question-decomposition evaluator, given the same clips and claims, accounted for 164, while a monolithic prompting approach using the same backbone reached 222. The comparison is important but should be interpreted carefully: recall alone does not fully separate VeriPhy from the monolithic baseline. Its main distinction is that every decision retains an evidence record and provenance, allowing the verdict to be audited one case at a time.

This design matters for world-model development because an evaluator is useful only when it can explain what should be fixed. A diagnostic that identifies the object, relation, or moment of failure can potentially serve as an interface for improving future generations. VeriPhy’s unknown or abstain state is also significant: insufficient evidence is not automatically treated as either success or failure. The supplied material does not establish that the system covers all physical laws or remains equally reliable in every complex setting. Instead, it presents a practical pattern for building evaluators that are more explicit, inspectable, and potentially useful for refinement than opaque scalar scoring.

Source: Hugging Face Daily Papers