Learning Sparse Representations of Multimodal Content for Enhanced Cold Item Recommendation

Modern recommender systems struggle with the 'cold-start' problem, where new items lack interaction history, and the storage of massive dense embedding tables is computationally expensive. This paper investigates whether sparse representations—where only a small subset of embedding dimensions are active—can effectively replace dense vectors in content-based cold-start recommendation, potentially offering better storage efficiency, faster retrieval, and improved interpretability.

The authors adapt existing content-based representation learning regimes (specifically the SEMCo framework) to produce sparse embeddings. They introduce a 'pre-sparsification activation' technique, inspired by linear attention mechanisms, to induce 'sharpness' and 'denoising' in item-item similarity scores. By applying top-K activation and alpha-entmax functions, the model forces the system to focus on the most relevant features while zeroing out noise, which is particularly beneficial for users with diverse, multi-interest profiles.

Sparse embeddings consistently outperform dense baselines across four multimodal datasets, with cold NDCG@20 improvements ranging from 16.6% to 75.5%. The authors demonstrate that the denoising effect of sparsity is especially effective for users with multiple interests, as it prevents irrelevant historical interactions from polluting preference predictions. Furthermore, the sparse representations maintain high accuracy even at significantly reduced storage budgets, and the resulting embeddings offer greater interpretability by highlighting the specific content features that drive a recommendation.

This work provides a practical pathway for deploying large-scale recommender systems that are both accurate and resource-efficient. By moving away from dense vectors toward sparse, interpretable representations, practitioners can reduce the infrastructure burden of storing massive item catalogs while simultaneously improving the quality of recommendations for new items, which is a critical requirement for dynamic digital platforms.