SISA-Rec: A Semantically Integrated Sequential Recommender with Contrastive Alignment

Most modern sequential recommendation systems, such as BERT4Rec, rely heavily on item identifier embeddings learned from interaction logs. While effective in dense settings, these models struggle in cold-start scenarios or with sparse data where interaction history is insufficient. This paper addresses the gap between behavioral modeling and semantic understanding, proposing a framework that embeds item semantics directly into the sequential modeling process.

SISA-Rec introduces a novel architecture designed to keep semantic information active throughout the entire encoding process. Key components include:

• Gated Fusion Module: Combines item ID embeddings with frozen BERT-based text embeddings at the input layer, allowing the model to learn the optimal balance between behavioral and semantic signals.
• Semantic-Aware Self-Attention: Injects a semantic similarity matrix directly into the attention score computation at each transformer layer, enabling semantically related items to influence the model even if they have never co-occurred in user histories.
• Attention-based Aggregation: Constructs a comprehensive user representation by learning a weighted sum over all positions in the sequence.
• Joint Training Objective: Combines Bayesian Personalized Ranking (BPR) loss with a contrastive alignment loss to ensure that behavioral and semantic user representations remain consistent.

SISA-Rec was tested on the Amazon Beauty and Amazon Toys & Games datasets, both characterized by 99.93% sparsity. The model consistently outperformed state-of-the-art baselines, including BERT4Rec. Specifically, SISA-Rec achieved significant improvements in Hit Ratio (HR@10) and Normalized Discounted Cumulative Gain (NDCG@10), with gains of up to 23.1% in HR@10 on the Toys & Games dataset. The results confirm that integrating semantic information directly into the attention mechanism provides a robust signal for users with limited interaction history.