Diffusion Language Model for Recommendation

Traditional LLM-based recommender systems typically use an autoregressive paradigm, which generates items sequentially from left to right. This approach is often suboptimal for recommendation because it prioritizes sequential order over the structural, combinatorial dependencies between items that actually define user preferences. The authors investigate whether a discrete diffusion language model can better capture these global preference patterns by using bidirectional context and iterative refinement.

To address the limitations of autoregressive models, the authors propose DLMRec, which adapts discrete diffusion language modeling to the recommendation task through three primary innovations:

• Collaborative-Aware Stochastic Tokenizer (CAST): This module encodes multi-hop collaborative signals from user-item graphs into discrete tokens. It uses a topology-stochastic assignment mechanism to handle graph-based ambiguity, ensuring the tokens are expressive and compatible with diffusion processes.
• Curriculum-Driven Training: The training strategy aligns the denoising process with preference recovery by progressively learning at both the item and token levels. It incorporates negative sampling to help the model distinguish between actual user interests and plausible distractors.
• Stability-Aware Voting: Instead of relying on a single pass, this mechanism aggregates predictions across the iterative denoising trajectory. By voting on the most consistent outputs, the model improves generation robustness and reduces the impact of early-stage prediction errors.

Experimental results demonstrate that DLMRec outperforms autoregressive baselines on standard recommendation benchmarks. The bidirectional denoising process allows the model to capture global inter-item relationships more effectively than the unidirectional next-token prediction objective. Furthermore, the stability-aware voting mechanism significantly enhances the consistency of the generated recommendations, validating that discrete diffusion is a viable and effective alternative for generative recommendation tasks.