GHOST: Geometry-Guided Hallucination of Opaque Surface Textures

Transparent objects are notoriously difficult for computer vision models because they violate the Lambertian assumption—the idea that surfaces reflect light uniformly. This leads to severe geometric distortions in depth estimation and 3D reconstruction. The authors ask: can we create a universal, plug-and-play preprocessing step that "repaints" transparent objects into opaque, structurally consistent versions, enabling existing state-of-the-art (SOTA) models to process them correctly without needing specialized training or expensive ground-truth data?

The authors propose GHOST (Geometry-Guided Hallucination of Opaque Surface Textures), a four-stage pipeline that processes an input image before it reaches a downstream model:

1. TransDINO: A segmentation model that uses DINOv3 features to identify transparent objects, even in complex scenes.
2. TransDecomp: A dual-head architecture that separates the image into an alpha matte (transparency) and a foreground grayscale map, effectively isolating the object from the background.
3. DAF-Net: A surface normal estimator that uses the extracted alpha and foreground cues to recover the object's geometric curvature.
4. GeoSemTransNet: A synthesis module that integrates these geometric and semantic cues to generate a texture-rich, opaque RGB image that preserves the object's 3D structure.

By feeding this "opaquified" image into standard models (like DepthAnything or Metric3Dv2), the authors demonstrate that they can bypass the need for task-specific, data-hungry models.

This work addresses the "perception silo" in 3D vision, where transparent objects are often excluded from general-purpose reconstruction pipelines. By treating transparency as a preprocessing challenge rather than a fundamental model limitation, GHOST allows researchers to leverage the rapid advancements in foundation models for depth and geometry without having to retrain them on scarce, costly transparent-object datasets. This significantly improves the robustness of robotic vision systems that must interact with glass, plastic, or other transparent materials in real-world environments.