A rendered view may show a neatly segmented object, but does that mean the object has been completely selected in the underlying 3D scene? Not necessarily. In 3D Gaussian Splatting (3DGS), many Gaussian primitives represent a scene, while common segmentation methods are trained and evaluated primarily through rendered images. A mask can therefore look right from a visible viewpoint even when some object-associated Gaussians remain unselected.
The mismatch stems partly from how rendering-based supervision reaches the primitives. Several Gaussians can contribute to the same pixel along a viewing ray, but alpha blending gives them different influence over the final pixel. Feature supervision based on the rendered result consequently concentrates learning on dominant contributors. Gaussians that are occluded or contribute less may receive weaker signals. A method can reproduce a visible mask without fully capturing the object in the 3D representation. This makes rendered-mask accuracy an incomplete measure of whether a segmentation is genuinely volumetric.
The paper proposes VODA, a plug-and-play volumetric supervision objective. It uses local depth structure around pixels to group the Gaussians contributing to them into depth clusters. Instead of relying only on contribution-weighted gradient delivery, VODA directly supervises all Gaussians in a selected cluster. The goal is to provide clearer learning signals to primitives that ordinary rendering supervision may under-train. The authors report that VODA recovers lower-contribution Gaussians. In their analysis, simple gradient reweighting and spatial propagation do not reproduce comparable gains.
The work also addresses how to evaluate completeness. Its synthetic Removal-3D benchmark tests object removal, and the proposed Removal Depth Error (RDE) compares depth changes produced by removing the predicted object with those produced by removing the ground-truth object. This shifts evaluation beyond whether a mask matches in a particular rendered view: if a segmentation misses part of an object, the result of removing that selection may differ from the result of removing the complete object, leaving a discrepancy in depth change.
According to the supplied paper summary, VODA consistently improves volumetric segmentation over four rendering-based baselines on Removal-3D. On LERF and ScanNet, it maintains or improves performance. The available summary does not provide numerical results, so it supports a qualitative account of the reported findings, not a judgment about the size of the gains.
The broader point is that a visually convincing 2D mask and a complete 3D object selection are different objectives. That distinction matters for tasks that select, remove, or otherwise operate on objects represented by Gaussians: omissions hidden in a rendered view may become relevant when the 3D object is manipulated. VODA targets the supervision gap, while Removal-3D and RDE offer a way to examine the consequences through removal. Together, they suggest a useful direction for 3DGS segmentation research: train and evaluate the scene representation, not only its rendered appearance. The supplied material does not establish how the method behaves across every scene type or data condition.
Source: Hugging Face Daily Papers
Comments
Checking sign-in status...
Loading comments...