VaLiDRec: Variable-Length LLM-Aligned Semantic IDs for Generative Recommendation

Generative recommendation (GR) models typically represent items using fixed-length semantic identifiers (SIDs) generated via clustering or quantization. These methods often suffer from over-compression, misalignment with LLM vocabularies, and high inference costs due to autoregressive decoding and beam search. This paper asks: can we construct variable-length, LLM-native semantic identifiers that are more expressive, efficient, and better aligned with pretrained language models?

VaLiDRec introduces a two-stage framework to address these limitations. In the first stage, it constructs SIDs directly from native LLM vocabulary tokens. It uses a combination of token importance scoring (based on contextual hidden-state norms and inverse document frequency) and a semantic-quality-aware greedy pruning strategy. This allows the length of an item's identifier to adapt naturally to its semantic complexity.

In the second stage, the model incorporates collaborative signals through graph-aware soft prompts, which are injected into the LLM. Instead of generating SIDs autoregressively, VaLiDRec reformulates the task as parallel token-set prediction. By aggregating token-level scores across an item's SID, the model produces item-level relevance scores directly, bypassing the need for expensive beam search.

Experiments across four real-world datasets demonstrate that VaLiDRec consistently outperforms state-of-the-art sequential and generative recommendation baselines. Key results include superior zero-shot performance in cold-start scenarios and an 87.49x speedup in inference time compared to the LC-Rec baseline. The authors show that using LLM-native tokens as identifiers provides a more expressive and efficient paradigm for generative recommendation than traditional artificial codebooks.