Vision & Video

Open-o3-Video-v2 extends video reasoning beyond fixed sparse frames by letting the model seek additional observations, ground relevant objects and timestamps, and expose the evidence behind its answers.

FramePack-U is a training-free framework for reducing arbitrarily long videos to a fixed-size visual representation. It combines prompt-aware frame ranking, diversity selection, and multi-resolution packing to keep memory usage constant.

QuadTok introduces a hierarchical quadtree tokenizer that allocates more visual capacity to complex regions and keeps homogeneous areas coarse. The same structure provides a spatially organized sequence for autoregressive image generation.

Long-horizon video generation needs more than a larger memory: it needs to decide which past frames will matter later. FrameMorrow predicts future information needs and uses them to select useful frames from the generation history.

OneStreamer is a 4B streaming video model designed to connect real-time perception, reusable memory, and proactive responses. It turns observed evidence into timestamped captions and event summaries so information can remain available after the original frames leave the visual window.

A new study tests visual self-supervised learning under a more realistic input regime: frames arrive in temporal order, without global shuffling or repeated epochs. Its proposed StreamMAE shows how adapting the data pipeline can recover much of the performance lost by standard methods.

FlashForward reuses the key–value caches already computed during denoising instead of rebuilding clean history with extra model passes. Sparse clean anchors then help control the visual and motion drift that can arise from using noisy, stage-matched history.

VideoPhysEdit moves video editing beyond appearance changes by reconstructing a rigid-body scene, simulating an intervention, and using the resulting trajectories to generate a counterfactual video.

A new interpretability study links implausible video motion to excessive spatial attention decay caused by RoPE during early denoising. The authors propose a step-dependent frequency adjustment to keep better motion candidates alive.

Vidu S2 combines a real-time interactive avatar model with a streaming video-editing model. Its broader goal is to let users continuously steer generated or transformed video as instructions, references, and scenes change.

ShallowStream uses shallow MLLM layers to encode incoming frames and maintain a lightweight retrieval index, reserving deeper computation for question answering. The paper reports up to 52.1× lower per-frame prefill latency and 11.9× lower end-to-end latency over 10 seconds.

A new arXiv paper proposes DM-Align, a single-stage framework that combines video generation distillation with human-preference alignment. It builds a preference-guided gradient from sample distribution gaps, aiming to reduce the cost and instability of conventional reinforcement-learning pipelines.

A controlled study separates frame selection, spatial compression, and budget reinvestment in long-video multimodal models. It finds that choosing the right frames matters most, while compression pays off only when its savings are used to inspect more moments.

Google DeepMind has introduced Agentic Video Understanding for Gemini, allowing the model to dynamically search and revisit relevant parts of a video across frames, audio and transcripts. Google reports up to 88% lower token usage, 66% lower cost and 7% higher accuracy on selected benchmarks.

Luce introduces a multimodal Gaussian representation that jointly models geometry and physically based materials for single-image-to-3D generation. Its decoder produces relightable PBR Gaussians and can optionally output a textured mesh with a tangent-space normal map.

Amap has released ABot-Recon, a streaming 3D reconstruction model that uses only the latest 12 frames as local context while reconstructing scenes spanning tens of thousands of frames. Its design targets drift, slowing inference, and growing memory usage in long video sequences.

Google DeepMind’s Gemini Omni 1.1 Flash adds scene extension, first-and-last-frame generation, low-resolution previews, video references, and upscaling to 4K.

SparsePR is a training-free sparse attention method for video generation and world models. It combines response-coupled partitioning with probe-fitted residual reconstruction to reduce executed attention pairs while preserving output quality.

InfinityEdit targets open-ended video streams rather than fixed clips, generating future segments that continue the scene while applying incoming edit instructions. Its lightweight adapter connects visual history, causal time flow, and edit prompts to a streaming video generator.

4DAnyone turns an uncalibrated monocular human video into reconstruction-ready, multiview-consistent videos and then lifts them into a 4D Gaussian Splatting representation. Its main contribution is a pair of context-management mechanisms designed to keep many generated views structurally and visually aligned.

Alibaba’s Wan 3.0 video generation model has entered public beta. Its most notable upgrade is support for document inputs such as doc, ppt, pdf, and md, pushing video generation closer to a productivity workflow.