Multimodal

Vivix has introduced A1, a real-time interactive multimodal model designed to make virtual characters listen, move and respond continuously. The release also includes W1, which aims to let users intervene in evolving narrative worlds.

Germany-based AI startup Black Forest Labs has introduced Flux3, a multimodal foundation model designed to unify image, video, audio and action representations. The model is described as using a Self-Flow architecture and supporting synchronized audio-video generation up to 20 seconds.

HPD-Parsing from the PaddleOCR team tackles a key bottleneck in VLM-based document parsers: full-page token-by-token generation. By separating global layout coordination from parallel block-level decoding, it reports much higher throughput while keeping competitive accuracy.

AV-Flamingo targets the hard problem of reasoning over long real-world audio-visual videos, not just short clips. It combines large-scale training data, staged curriculum learning, and timestamp-grounded reasoning to improve alignment and interpretability.

Hallo4D targets spatial and temporal hallucinations in 3D and 4D generation. Instead of retraining generators, it uses multimodal LLMs to detect inconsistencies and guide consensus-based corrections.

KnowAct-GUIClaw introduces a “Know Deeply, Act Perfectly” paradigm for personal GUI assistants, aiming to address OpenClaw’s limitations in cross-platform GUI interaction and self-evolution. The framework combines experience-based memory, a self-evolving skill library, and reflection to improve task execution over time.

Boogu-Image-0.1 is an open-source family of unified multimodal models for image generation, editing, fast inference, and bilingual text rendering. The paper argues that careful data, training, and inference-time design can narrow the gap with closed systems under constrained compute.

FM² tackles a practical barrier in medical AI: foundation models need data from many institutions, but privacy rules often prevent central aggregation. The paper focuses on modality heterogeneity across clients and proposes a unified federated training framework.

A new arXiv paper introduces CF-Net, a multimodal model for recognizing ambivalence and hesitancy in unconstrained video. Instead of looking for prototypical emotion cues, it focuses on subtle inconsistencies across vision, audio, and text.

A new arXiv paper introduces AgentHOI, a training-free agentic framework for human-object interaction detection in open-world scenes. Instead of learning fixed interaction classifiers, it coordinates multimodal reasoning and visual grounding modules.

A new arXiv paper proposes a slice-wise data synthesis pipeline to help 2D-pretrained medical multimodal large language models reason over 3D scans. The approach turns volumetric reading into structured, interpretable reasoning traces.

AspectCLIP addresses a subtle weakness in CLIP-style training: images contain many possible semantic aspects, while captions usually describe only one. The method uses text-guided attribute clusters to apply consistency where it is semantically appropriate.

Thinking Machines has released Inkling on Hugging Face, an open multimodal model designed to natively accept image, text, and audio inputs. With a 1M-token context window and a sparse MoE architecture, it targets advanced multimodal reasoning and downstream adaptation.