Are we Merging the Right Models? Impact of Expert Training Duration on Model Merging for LLMs

Standard practice in model merging assumes that domain experts should be frozen at their individual validation optimum before being combined. This paper challenges this convention by investigating whether the optimal training duration for an expert depends on the specific merging method used to aggregate them. The authors hypothesize that some merging methods may benefit from the higher variance found in overfitted experts, similar to how random forests benefit from the inclusion of high-variance decision trees.

To test this, the researchers fine-tuned Qwen 3.5 models (0.8B, 2B, and 4B parameters) across five distinct domains: Math, Code, Instruction Following, Multilingual, and Safety. They saved checkpoints ranging from 25% to 500% of the optimal validation steps (T*). They then evaluated five common merging methods—Simple Averaging, Task Arithmetic, TIES-Merging, DARE+TIES, and Greedy Soup—to determine how each method responds to varying levels of expert training duration.

The study reveals a clear method-dependent pattern. Simple Averaging and Greedy Soup perform best with undertrained experts, as they lack mechanisms to handle the noise introduced by overfitting. In contrast, sparsification-based methods (TIES and DARE+TIES) show peak performance when experts are trained well past their validation optimum. The authors explain this through a bias-variance-covariance decomposition: overfitting reduces bias but increases variance, and sparsification acts as a variance-reduction mechanism that filters out the noise and sign-conflicting parameters introduced by overtraining. Consequently, practitioners using TIES or DARE+TIES should avoid early-stopping and instead train experts longer to maximize the quality of the final merged model.