A Unified Model for Cross-Domain Clone Detection via Model Merging

Research Question and Approach

Code clone detection is currently fragmented, with models specialized for specific languages or clone types (e.g., cross-language or AI-generated) failing when applied to out-of-distribution data. Training a single, unified model is often impractical due to data privacy, high computational costs, or the need for simultaneous access to all training corpora. This paper investigates whether post-hoc model merging—combining trained model checkpoints without retraining—can unify these specialists into a single, high-performing detector.

The authors evaluate five parameter-level merging techniques (Task Arithmetic, TIES, DARE-TIES, WUDI, and PCB) and architecture-level layer stitching. They test these methods across four pre-trained code models (UniXcoder, CodeBERT, and GraphCodeBERT) and three benchmarks, including BigCloneBench (same-language), CLCDSA (cross-language), and GPTCloneBench (AI-generated clones).

• Shared-Base Necessity: The study confirms that effective merging requires all specialist models to share the same pre-trained base. Cross-base merging, where models are initialized from different pre-training objectives, results in destructive interference that prevents the formation of a coherent unified model.
• TIES Merging Performance: Same-base TIES merging is the recommended approach. It achieves 93% of the performance of multi-task training without any training data at the merging step. While WUDI achieves slightly higher in-distribution F1 scores, TIES demonstrates superior generalization to unseen AI-generated clones.
• Robustness vs. In-Domain Performance: Merged models exhibit a distinct advantage over multi-task training: they generalize up to 4x better to unseen AI-generated clones. This suggests that while multi-task training may overfit to the combined training distribution, merging preserves the complementary knowledge of individual specialists.