Multimodal

Vision-language models can produce answers that sound convincing but are wrong. MOTIVE combines complementary verification views with a learned reliability score to decide whether an answer should be accepted or reconsidered.

Kaiming He’s team has proposed VISTA, a visual-native harness that lets existing multimodal models store and revisit raw observations during interactive tasks. The framework improves access to visual evidence without requiring additional training of the base model.

Cloudflare has announced three updates to its Clef decision-model lineup: the multimodal Clef-omni, a cheaper Clef-flash, and a faster Clef. The package targets the practical trade-offs between input flexibility, inference cost and latency.

FMEE is a parameter-free framework that helps deployed LLMs and VLMs accumulate experience through external expertise rather than weight updates. It combines procedural skills, evidence-backed memory, and a multimodal case base for medical reasoning.

Vision tokens are a major source of computational cost in vision-language models. V-CoLA introduces a training-free compression framework designed for linear-attention and hybrid architectures.

OneSearch-VL introduces a Visually Grounded Evidence Graph to connect visual grounding, external retrieval, and answer composition across image and video research. The framework combines purpose-built training data, reinforcement learning, and operation-level benchmarks.

OmniConfess is a training-free inference method that probes a multimodal model’s token-level dependence on text, image, audio, and video evidence before correcting unreliable commitments.

Kandinsky 6.0 Video is a family of models that generates five-second clips with synchronized 44 kHz audio and lip sync from text or images. Its dual-stream CrossDiT design jointly aligns visual and audio content instead of adding sound after video generation.

HUST Vision Lab introduces Multimodal Flow, a fully continuous approach for jointly modeling language and images. It replaces discrete visual tokens with continuous hyperchunks while retaining structure from both modalities.

ReImaGin gives multimodal language models access to image generation as a flexible visual reasoning mechanism. The paper reports gains over text-only reasoning and specialist vision tools across six tasks, with improvements of up to 25%.

A Berkeley-led study examines when image-to-image generation training transfers to image-to-text understanding. Its findings show that the gains are selective: the right generation tasks can strengthen specific perceptual and reasoning abilities.

SentZero is a sentence-centric vision-language pretraining framework for multi-task, zero-shot chest X-ray analysis. It addresses limited positive-pair diversity, false negatives in contrastive learning, and the mismatch between long radiology reports and simple prompts.

VQS replaces fragile answer voting with structured image records and fixed programs that generate and compute visual questions. The approach reaches 94% answer accuracy in human evaluation and improves Qwen3-VL across ten benchmarks.

A Tencent Hunyuan study compares multimodal language models with and without pretrained visual encoders. Its results suggest that encoder-free models may overcome their early disadvantage as model size and training compute increase.

AV-GRPO addresses entangled rewards, expensive joint optimization, and unstable synchronization evaluation in joint audio-video generation. Its companion 5DAV dataset introduces controllable difficulty through decoupled training dimensions.

PrismML has demonstrated a 1-bit Bonsai language model designed for AI glasses powered by Qualcomm’s Snapdragon AR1 Gen 1 platform. The roughly 2-billion-parameter model is intended to handle vision-language queries locally, although no product using it has been announced.

Google DeepMind has introduced Gemini 3.8 Live with Live Avatar, combining low-latency video generation with live speech conversations. The feature adds asynchronous tool use, multilingual synchronization across 97 languages, and customizable enterprise avatars.

Ovis-Embedding presents a native omni-modal embedding approach for encoding text, images, video, and audio with a shared multimodal backbone. Its contribution spans data organization, hard-example learning, similarity distillation, and flexible inference dimensions.

Realtime-Venus is a real-time interaction system designed to keep perceiving and responding while background reasoning or tool execution continues. It combines two separately trained 9B models for audio-visual and spoken interaction.

CARDEA is a vision-language model for coronary angiography that links its reasoning trace to image regions through Chain-of-Box. The study finds that verifiable-reward reinforcement learning improves zero-shot report generation as well as selected interpretation tasks.

SpatialBlock proposes a controlled alternative to expensive real-scene geometry annotation: train large vision-language models on synthetic block-manipulation problems first. Its SpatialBlock-15k dataset covers projection, viewpoint changes, structural composition, and color-based spatial anchors.