Sharpness-aware Model Merging with Salience Recovery for LLM-based Cross-Domain Sequential Recommendation

Cross-Domain Sequential Recommendation (CDSR) aims to improve recommendation performance in a target domain by leveraging data from auxiliary domains. While Large Language Models (LLMs) have enhanced this process, current model merging techniques—which combine domain-specific adapters (like LoRA)—suffer from two primary issues: cross-domain knowledge conflicts caused by parameter misalignment and performance saturation due to statistical homogenization during linear fusion.

The authors propose SharpRec, a framework designed to overcome these bottlenecks through two key mechanisms:

1. Sharpness-aware Geometric Alignment (SGA): Instead of standard fine-tuning, SGA guides domain-specific models toward flat minima. By ensuring that models reside in geometrically connected low-loss basins, the framework reduces parameter interference when different domains are merged.

2. Preference Salience Activation (PSA): To combat the "mean filter" effect of linear merging, which smooths out important features into a generic Gaussian distribution, PSA uses a non-linear reparameterization strategy. This step reconstructs the merged parameters to restore heavy-tailed characteristics, effectively reactivating the salient signals necessary for high-quality recommendations.

Experimental results demonstrate that SharpRec consistently outperforms state-of-the-art baselines in both dual-domain and multi-domain scenarios. The authors show that their approach successfully mitigates negative transfer between divergent domains (e.g., Sport and Toy) and pushes past the performance plateaus typically encountered when scaling to multiple source domains. Visualizations confirm that SGA successfully aligns parameters geometrically, while PSA prevents the statistical homogenization that otherwise erodes model performance.