Multi-Objective Ranking for Live-Streaming: Balancing Fresh and Delayed Signals with Segment-Aware Targeting

Live-streaming platforms like Twitch operate as complex two-sided marketplaces where viewers engage in diverse, concurrent behaviors—such as watching, chatting, following, and spending—that occur with varying degrees of delay. Traditional recommendation systems, often designed for linear e-commerce funnels, struggle with these environments due to three primary issues: target sparsity (high-value actions like following occur rarely), delayed feedback (actions may happen days after exposure), and user segment bias (models are often dominated by highly engaged users, neglecting newer or less active ones).

The researchers introduced a multi-objective optimization framework to balance immediate engagement with long-term retention and monetization. Their approach consists of three main components:

1. Delayed Window Framework: Instead of relying on immediate signals, the system aggregates sparse actions (chat, follow, spend) over a 14-day window to create denser, more reliable training targets.
2. Multi-Model Architecture: The system separates 'fresh' signals (short-form watch time) from 'delayed' signals. Fresh Signal Models (FSM) handle immediate engagement, while Delayed Signal Models (DSM) process the aggregated historical data.
3. Viewer Segment Targeting (VST): To mitigate bias, the system applies segment-specific weights at inference time. This allows the model to prioritize different business objectives—such as retention for new users versus monetization for dedicated users—without requiring separate models for each segment.

Finally, the team integrated a Multi-gate Mixture-of-Experts (MMoE) architecture to jointly model correlated targets, which improved performance while reducing the total number of model parameters by 41.9%.

Online A/B testing on Twitch demonstrated significant improvements across key metrics. The multi-model architecture with the delayed window increased Daily Active Viewers (DAV) by 0.09% and boosted the Average Revenue Per User (ARPU) for highly engaged viewers by 0.56%. The VST module provided an additional 0.15% DAV lift for newer, less engaged viewers, and the MMoE integration further improved overall DAV and new follows. The system maintains low latency (under 110ms p99), making it highly scalable for production environments.