Multimodal

JoyAI-Echo-1.5 is a unified audio-visual generation system designed to preserve identities across long videos and support controllable, persistent world-model rollouts.

A new survey argues that smart glasses should be understood as first-person intelligence platforms, not merely cameras or displays. Their real test is whether they can sustain a reliable, correctable, and governed loop from perception to action.

LAION-BVD is an open video resource built from web-discovered URLs, containing about 80 million downloaded videos with a combined duration of 10 million hours. It supports video, audio, and image-text pre-training through scene extraction, synthetic captions, and key-frame mining.

DeepSeek has launched deepseek-v4-flash-vision-exp on its API platform, giving the V4 family an official image-understanding interface for the first time. The model also shows gains on several tool-use and coding-agent benchmarks.

DeepSeek has released DeepSeek-V4-Flash-Vision-Exp, an experimental vision-enabled version of V4-Flash. The model is available through the DeepSeek API while preserving the text capabilities highlighted for the original model.

One week into its public beta, DeepSeek Harness has released v0.1.0-rc.8 with native image requests and mixed text-image input. The update also expands sub-agent integration, adds persistent PowerShell sessions on Windows, and enables minimalist mode by default.

MOSS-VL is designed for real-time interaction rather than treating streaming vision as an afterthought. Its gated cross-attention, interaction-focused data, and staged training help the model keep processing incoming frames while generating responses.

SPARGen reframes several spatial intelligence tasks as instruction-conditioned generation within a native multimodal model. Its main contribution is a unified interface where geometric, correspondence, and reasoning supervision can shape shared representations.

UniSwap proposes a unified framework for replacing both the visual identity and vocal timbre in talking videos. Instead of stitching together separate face and voice models, it performs joint audio-visual transfer inside a single diffusion transformer.

Ex-Omni-2D tries to close a gap in today’s omni-modal assistants: they can talk, but they still feel visually absent. The paper combines text, personalized speech, and reference-conditioned video into one dialogue pipeline.

Evidence-RL targets a persistent weakness in vision-language models: correct answers may come from language priors or dataset shortcuts rather than image evidence. Its CED mechanism audits whether a response causally depends on the local visual region that should support it.

A new study argues that follow-up suggestions in image-creation chats should be visually grounded, not merely text-driven. Using real product data, the proposed three-stage framework improves suggestion relevance, user engagement, and visual consistency.

Kandinsky Lab has introduced KVAE, a family of tokenizers designed for latent diffusion models across audio, image and video. The report positions tokenization as a core part of generative modeling rather than a secondary preprocessing step.

This paper is less about a single model and more about the mechanisms behind unified multimodal training. It breaks the problem into knowledge flow, modality synergy, early unification, and practical recipes.

This paper argues that 3D should be treated as one unified multimodal problem, not separate tasks. Its main contribution is both an architecture and a large-scale data pipeline for training it.

VAD tackles a subtle problem in multimodal on-policy distillation: not every teacher correction is actually grounded in the image. The method reconstructs student targets from the part of the correction that can be attributed to visual evidence.

The paper introduces Evaluation-Verification Reward, a multi-dimensional reward mechanism designed to make multi-reference image editing more consistent and visually coherent through reinforcement learning.

Mage-VL proposes a codec-native approach to streaming multimodal perception, using signals already present in video compression to reduce visual token waste. Its key idea is to focus encoding on dynamic, high-entropy regions rather than treating every frame uniformly.

ClinFusion argues that medical MLLMs should be judged first by what they can see. The system targets unified understanding of heterogeneous 2D and native 3D medical images, along with evaluation methods closer to radiology practice.

VisCo reframes visual token compression as a self-encoding problem inside the pretrained vision-language model. Instead of adding a separate compressor, it uses a small set of memory tokens to carry compact visual information.

A Tencent Hunyuan paper studies how to scale native multimodal pre-training from scratch under a fixed compute budget. The work highlights that language and multimodal objectives follow different allocation behaviors.