HOMIE Improves Human-Object Video Personalization

Introduction

In subject-driven video generation, realism is not only about making a person look consistent across frames. It is also about whether the model can produce believable interactions with objects. That becomes even harder when the object is abstract, such as a logo or a concept-like reference. HOMIE is proposed as a framework for human-object centric video personalization, a task the authors see as central to this space.

Key points

• Unified handling of two settings: the method is designed for both inter-subject personalization and intra-subject references.
• Better MLLM integration: HOMIE aims to extract reference-level relationship knowledge without weakening text-encoder controllability or requiring expensive re-alignment.
• Global multimodal guidance: it injects multimodal guidance into self-attention to align MLLM-derived semantic features with VAE tokens.
• Modality-reference embedding: it differentiates tokens from MLLM features and VAE tokens, while linking intra-subject reference image tokens.

Why it matters

This work points to a broader issue in video personalization: identity fidelity and interaction fidelity are not the same problem. A model can preserve appearance yet still fail at how a hand holds an object, how a person uses a prop, or how an abstract reference should be represented in motion. HOMIE’s contribution is mainly architectural, but its framing is important because it treats relationship understanding as a first-class requirement.

According to the abstract, the method achieves state-of-the-art results on multiple HOCVP benchmarks. If that holds up in practice, it suggests multimodal alignment inside the generation pipeline may be a more scalable path than repeatedly post-hoc correcting video outputs.

Source: Hugging Face Daily Papers