Personalizing Marketplace Policies with Competing Objectives and Constrained Experiments: Evidence from a Job Marketplace

Two-sided marketplaces, such as job platforms, face a fundamental tension: optimizing outcomes for one side (e.g., employer monetization) often degrades the experience for the other (e.g., job seeker engagement). Standard uplift modeling typically focuses on single-objective optimization and assumes sufficient treatment variation. However, in this job marketplace, the authors faced two major constraints: cross-side externalities that necessitate multi-objective optimization, and cluster-level randomization that limited experimental data to only two discrete treatment levels.

To address these, the authors developed an integrated framework consisting of three components: ensemble-based hybrid ranking models to manage competing objectives, a linear treatment effect extrapolation method to handle limited experimental variation, and a production deployment strategy that uses segment-level lookup tables for operational stability.

The core of the solution is a hybrid ranking algorithm that avoids the need for contested weight negotiations between stakeholders. Instead of combining target and guardrail metrics into a single weighted score, the framework ranks jobs by their target uplift for the top 50% of the distribution, and by their guardrail impact (from least to most negative) for the bottom 50%. This ensures that the platform maximizes monetization where possible while protecting job seeker engagement in segments where the risk is highest.

Because the experiment only provided two treatment levels, the authors used linear extrapolation to estimate effects at untested policy levels. They argue that this is the maximal model identifiable from the experimental design and that the policy remains robust because it relies on the ordinal ranking of segments rather than precise cardinal effect magnitudes.

The study demonstrates that principled personalization is achievable even under severe experimental constraints. By moving away from complex Pareto frontier navigation or arbitrary weighting, the authors provide a practical, interpretable, and stable method for multi-objective policy optimization. Post-launch validation confirmed that the framework achieved statistically significant target lift while successfully keeping engagement metrics within defined guardrails, providing a blueprint for other platforms facing similar structural tradeoffs.