Breaking the Filter Bubble: A Semantic Pareto-DQN Framework for Multi-Objective Recommendation

Traditional recommender systems often prioritize immediate user engagement, which leads to "filter bubbles"—a phenomenon where users are trapped in a narrow, homogenized semantic space. This paper investigates whether a multi-objective reinforcement learning (MORL) approach can balance engagement with broader societal values like information diversity and provider fairness without significantly sacrificing platform performance.

The authors propose a "Semantic Pareto-DQN" framework that treats recommendation as a multi-objective Markov decision process. Key components of their approach include:

• Semantic State Representation: Using a Sentence Transformer (all-MiniLM-L6-v2) to map users and items into a shared, high-fidelity latent space.
• Pareto-DQN Agent: Instead of collapsing multiple objectives into a single scalar reward, the agent uses a vectorial reward function and estimates the Pareto frontier of achievable outcomes.
• Hypervolume-based Selection: The agent selects actions by maximizing the hypervolume of the predicted reward set, which mathematically ensures it explores the trade-off surface between conflicting objectives.
• Stratified Candidate Pooling: The system reserves a portion of the recommendation pool for long-tail items, providing a mechanism to inject diversity and fairness into the user experience.

Empirical evaluations on the MovieLens dataset demonstrate that the Pareto-DQN successfully disrupts the feedback loops responsible for semantic collapse. While standard single-objective DQN models drive user preferences toward a narrow, homogenized subspace, the Pareto-DQN maintains higher state-trajectory variance. This allows the system to achieve significant gains in diversity and fairness with only marginal impacts on engagement, proving that responsible recommendation is achievable without catastrophic losses in platform utility.