EditVerse3D: High-Quality 3D Object Editing with Region-Aware Learning

Local 3D editing is historically difficult because it requires balancing precise modifications with global structural consistency. Existing methods often rely on complex multi-stage pipelines, such as rendering 2D views, editing them, and lifting them back to 3D, which frequently introduces cumulative errors and inconsistencies. EditVerse3D addresses this by proposing an end-to-end framework that accepts a coarse 3D bounding box and a reference 2D image to guide the editing process, eliminating the need for precise 3D masks or fully edited 2D views.

The authors utilize the TRELLIS 3D generative model as a backbone, adapting it to handle local edits through a novel region-aware adaptive loss. This loss function dynamically reweights the training objective to focus on hard-to-learn regions while ensuring that the target editing area and the preserved areas are balanced. To support this, the authors curated a large-scale dataset of 85k meshes and 500k editing pairs by simulating part-addition operations, which provides the necessary supervision for the model to learn robust editing behaviors.

The framework introduces several technical strategies to ensure high-fidelity results:

• Region-Aware Adaptive Loss: A loss reweighting mechanism that normalizes contributions from masked (target) and unmasked (preserved) regions, combined with a hard-example mining strategy to prioritize difficult-to-learn areas.
• Joint Normalization: A preprocessing step that computes a single normalization factor for both the 3D object and the mask, preventing spatial distortion that would otherwise occur if they were normalized independently.
• Data Augmentation: The model is trained with scaled 3D masks and random disturbances to the bounding box, which improves its robustness to the imprecise, coarse inputs typically provided by human users.

EditVerse3D simplifies the 3D editing workflow by moving away from computationally expensive and error-prone multi-stage pipelines. By enabling high-quality editing from coarse, user-friendly inputs, this approach makes 3D content creation more accessible and efficient. The release of a large-scale 3D editing dataset also provides a valuable resource for future research in 3D generative modeling and interactive 3D design.