ID-V2V: Restyling Videos While Preserving Identity and Performance

Introduction

Generative video editing often faces a difficult trade-off: the more aggressively a scene is restyled, the more likely the model is to alter the person in the footage. Faces may drift, eye gaze can become unstable, expressions may lose nuance, and lip synchronization can degrade. For storytelling, that is not a minor artifact. Human performance is usually the emotional center of the shot.

Netflix’s research work ID-V2V: Identity-Preserving Video Restylization addresses this specific problem. Rather than generating a video from scratch, it focuses on video-to-video editing: take an existing performance, edit one or more keyframes to define a new visual direction, and propagate that change across the full video while keeping the actors recognizable and their performance intact.

Key ideas

• A production-oriented task definition: ID-V2V formalizes “identity-preserving video restylization,” where changes to scene, lighting, and style are guided by edited keyframes and applied to the source video over time.
• Performance is treated as a constraint, not a side effect: The method is designed to preserve facial likeness, subtle expressions, eye gaze, lip synchronization, body motion, and multi-person interactions.
• A response to missing paired data: Real-world paired examples of the same performance restyled in different ways while perfectly preserving identity are scarce. ID-V2V avoids relying on such rare datasets by constructing training pairs from a single video.
• Identity preservation becomes relighting: The central insight is that facial appearance and expressions should remain largely invariant, while illumination is the main acceptable change. The paper therefore frames identity preservation as a video relighting problem.
• Multiple controls work together: Relit facial regions and facial normal maps tightly constrain facial identity and performance. Edited keyframes and depth sequences guide flexible, temporally coherent synthesis for the rest of the scene.

Why it matters

The practical appeal is straightforward: capture the performance first, redesign the look later. For filmmaking, advertising, virtual production, and creator workflows, this could reduce the need for reshoots when the desired visual style changes after filming. A scene could be pushed toward a different lighting scheme, environment, or aesthetic treatment while retaining the original actor performance.

The work also highlights a broader evaluation issue for video generation. A generated or edited clip cannot be judged only by visual sharpness or style consistency. In human-centric footage, identity stability, expression continuity, gaze fidelity, and believable lip motion are central quality metrics.

ID-V2V should still be read as a research exploration rather than a fully proven production system. The material points to strong results against existing methods, including single- and multi-subject settings, but real-world robustness will depend on how it handles complex motion, occlusions, extreme stylization, and long sequences. Even so, its formulation is important: it shifts video restylization toward preserving the human performance as the non-negotiable anchor.

Source: Hugging Face Daily Papers