Beyond Action Imitation: Learning a Decision-Aware User Simulator for Online Advertising

Traditional user simulators in online advertising often rely on single-domain interaction histories and focus exclusively on predicting observable actions like clicks. This approach frequently leads to model shortcuts, poor simulation fidelity, and a lack of diagnostic insight into why users behave the way they do. This paper asks: How can we build a user simulator that captures heterogeneous cross-domain preferences and models the underlying reasoning processes to improve both prediction accuracy and diagnostic utility?

The authors propose DASH (Decision-Aware Simulator with Heterogeneous context), which moves beyond simple action imitation by explicitly modeling user thinking traces. The framework operates in three stages:

1. Context Engineering: To handle massive, heterogeneous cross-domain logs, the authors implement a hierarchical compression strategy. This preserves high-information interactions (e.g., conversions) while compressing low-information ones (e.g., skips) and uses prompt optimization to ensure the LLM reasons effectively over this folded context.
2. Supervised Fine-Tuning (SFT): The team uses a strong teacher LLM to generate thinking trajectories. They curate both 'easy' samples (where the teacher's action matches ground truth) and 'hard' samples (where the teacher is forced to align with ground truth), filtering the results through a quality-control judge to create a high-quality training dataset.
3. Reinforcement Learning (RL): The model is further refined using a rubric-based reward system that evaluates thinking traces based on form, content, and logic, alongside traditional action-fidelity rewards. This ensures the generated thinking is not only plausible but also logically consistent with the user's profile and the ad context.

By generating explicit thinking traces, DASH provides researchers with a 'white-box' view of simulated user behavior. This allows for better diagnostic analysis of why recommendation systems fail, moving beyond simple click-through rate metrics. The use of heterogeneous data also ensures that the simulator captures a more holistic view of user preferences, making it a more robust tool for low-risk, offline experimentation in complex advertising ecosystems.