Reward Guided Decoding for Generative Recommendation

Generative recommendation models typically rely on maximum likelihood estimation, which optimizes for predicting historical user behavior. However, this likelihood-driven approach often conflicts with real-world business objectives, such as maximizing click-through rates, watch time, or commercial conversions. The authors investigate how to steer these generative models toward high-value outcomes during inference without the prohibitive cost of retraining the generator whenever business preferences shift.

The authors propose Reward Guided Decoding (RGD), a controllable framework that treats the base generative model as a reference policy. RGD introduces a lightweight, chain-structured reward model as a test-time controller. By formulating the task as a KL-regularized reward maximization problem, the authors derive a closed-form decoding distribution that combines generation likelihood with reward signals. This allows the system to inject reward feedback at each step of the beam search, effectively reshaping the search trajectory to favor high-value candidates before they are pruned.

RGD provides a principled method for combining heterogeneous signals—generation probability and business rewards—into a unified ranking objective. Because the reward model is applied at inference time, the system supports flexible, real-time switching between different business objectives (e.g., prioritizing clicks versus gifts) without modifying the underlying generator. Empirical results on both public datasets and the Kuaishou industrial platform demonstrate that RGD consistently improves business metrics compared to standard likelihood-based decoding and post-hoc reranking methods.

This work addresses a critical gap in industrial generative recommendation: the tension between historical behavior modeling and dynamic business utility. By moving from post-hoc reranking to integrated, reward-guided decoding, RGD enables more precise control over the recommendation pipeline. This flexibility is essential for modern platforms that must balance user personalization with evolving commercial goals.