DiffGI Reframes Thin-Shell 3D Generation as Differentiable Geometry Images

Introduction

Many recent 3D generative systems are built around implicit volumetric representations. These approaches are powerful, but they often assume or encourage watertight topology. That assumption becomes a limitation for shapes such as garments, cloth panels, open surfaces, and other thin-shell or non-manifold structures. DiffGI, short for Differentiable Geometry Image, proposes a different route: encode surface geometry as a 2D geometry image, while keeping the reconstruction and optimization process differentiable from end to end.

Key ideas

• A surface-first representation: Instead of forcing thin surfaces into a volumetric format, DiffGI treats geometry images as a compact 2D carrier for 3D surface information. This is especially relevant for clothing-like assets, where the important structure is the surface itself rather than a filled volume.

• Continuous TSDF instead of binary occupancy: Earlier geometry-image pipelines often rely on binary occupancy maps. Their boundaries are tied to grid resolution, so downsampling can create stair-step artifacts and erase fine boundary information. DiffGI replaces those binary maps with a continuous 2D Truncated Signed Distance Function. This lets the representation encode boundary positions at subpixel precision within a fixed grid.

• Differentiable Marching Squares: Surface reconstruction is often treated as a post-processing step, disconnected from learning. DiffGI introduces a differentiable Marching Squares module based on analytical linear interpolation. As a result, losses measured on the reconstructed 3D surface can flow back into the 2D latent representation.

• Compact latent generation: The authors train a DiffGI-VAE with a geometry-aware normal rendering loss, compressing complex surfaces into a 32×32 latent space. On top of that space, they instantiate a Transformer-based latent diffusion model trained with a flow-matching objective for conditional 3D generation.

Why it matters

The main contribution is not just a new encoding format; it is the integration of representation, reconstruction, and optimization into one differentiable pipeline. For garment generation and other thin-shell assets, crisp boundaries and faithful surface structure matter more than watertight volume assumptions. A continuous TSDF representation helps reduce resolution-driven artifacts, while differentiable reconstruction gives the training process direct feedback from the final 3D surface.

The material reports stronger reconstruction fidelity and boundary precision than prior geometry-image and voxel-based baselines, while using fewer computational resources. The project page also highlights fast generation on consumer hardware, though broader validation will depend on the paper’s full experimental details, the forthcoming demo, and how well the method generalizes beyond the shown garment and object datasets.

DiffGI is therefore an interesting signal for the 3D generation field: better thin-shell generation may come not only from larger diffusion models, but also from representations that better match the geometry being generated.

Source: Hugging Face Daily Papers