Ruixuan Sun, Matthew Zent, Minzhu Zhao, Thanmayee Boyapati, Xinyi Li, Joseph A. Konstan
9 min
Abstract
In this study, we applied the ``personalized diversity nudge framework'' with the goal of expanding user reading coverage in terms of news locality (i.e., domestic and world news). We designed a novel topic-locality dual calibration algorithmic nudge and a large language model-based news personalization presentation nudge, then launched a 5-week real-user study with 120 U.S. news readers on the news recommendation experiment platform POPROX. With user interaction logs and survey responses, we found that algorithmic nudges can successfully increase exposure and consumption diversity, while the impact of LLM-based presentation nudges varied. User-level topic interest is a strong predictor of user clicks, while highlighting the relevance of news articles to prior read articles outperforms generic topic-based and no personalization. We also demonstrate that longitudinal exposure to calibrated news may shift readers' reading habits to value a balanced news digest from both domestic and world articles. Our results provide direction for future work on nudging for diverse consumption in news recommendation systems.
Alex: Okay, so it's iteratively adding stories to fill out your usual topic pie chart, but now also making sure some world slices get in there. And the base liking score comes from their existing model?
Sam: Yes, that's NRMS—a standard tool that predicts clicks based on patterns in what you've read before, like a smart guesser trained on lots of user data. For the plain topic version, it only tunes the topic part; dual adds the place layer, penalizing picks that skew too domestic if your supply has world options. They tuned the weights by testing many combos to find what keeps things personalized yet pulls consumption closer to the full news supply. Then, for the extra nudge, they spot articles that made the dual list but wouldn't have in the plain topic one.
Alex: Right—like grabbing world stories on familiar topics that got bumped in. So for those, they rewrite the headlines to connect back to your history?
Sam: Exactly. They scan your past clicks for the most similar story—measuring similarity by how overlapping the key ideas are, like matching puzzle pieces. If a match fits, an AI like GPT rewrites the preview to link them, say tying a world politics piece to your prior U.S. one. No strong match? It weaves in your top topics from surveys. This happens just for those added articles, aiming to lower the skip rate on unfamiliar places.
Alex: Huh. So the rewrite bridges the gap, making global stuff feel like a natural next read. That could explain why exposure turned into more actual clicks.
Sam: The study suggests as much from the five-week logs on POPROX users. Domestic-heavy readers got nudged toward world views without big drops in opens or satisfaction, showing these mechanics can shift habits meaningfully over time.
Alex: So the logs show these nudges sticking over five weeks. But how did they pin down if exposure actually turned into more reading of world news?
Sam: They tracked two main measures: exposure diversity, which looks at the mix of topics and places in the top ten stories shown each day, and consumption diversity, based on what users actually clicked. To compare groups fairly, researchers built statistical models that account for differences between people and changes week to week—like adjusting for one user clicking more overall than another. These are called generalized linear mixed-effects models, or GLMMs for short. The models used the same math check we discussed, treating the full news supply as the goal mix and seeing how close exposure or clicks got to it. Both dual-calibration groups showed substantially lower divergence than the topic-only baseline.
Alex: Okay, so the models teased out group differences while controlling for individual habits and time. Meaning dual-calibration got feeds and clicks much closer to the available balance of domestic and global stories?
Sam: Yes. For exposure, the plain dual group cut divergence from supply substantially compared to baseline, and the preview-rewrite version did so as well—a clear step up that held over weeks. Consumption followed a similar pattern, aligning clicks tightly with supply variety. They also modeled clicks directly, factoring in story position, how well it matched your past topics or places, and group. Even with those controls, dual groups clicked world stories more without hurting overall engagement.
Alex: Huh. So position and past-match still mattered, but the nudges boosted clicks on less familiar places. What about user feelings—did surveys back this up?
Sam: Surveys were voluntary with just 28 people responding, so trends only. Baseline users felt less need for location mix by study's end, while nudge groups held steady on valuing domestic-global variety. No big shifts in satisfaction or control across groups. Overall, the click data from 120 users suggests these tweaks meaningfully bridge exposure to balanced reading without drawbacks.
Alex: You mentioned no big satisfaction drops, but what about clicks overall—did the nudges hurt engagement, like fewer total reads?
Sam: They checked that with a model predicting the chance of any click on a story. It factors in things like where the story sits in the list, the week number, and how much it matches your past interests—like weighing if it's your usual type to see what drives picks. No meaningful group differences emerged; user habits explained most variation.
Alex: So position and your history still ruled clicks, but dual groups leaned harder on topic fits without tanking totals. And for the rewrite group, any preview-type differences?
Sam: In that group, rewrites varied: some got event-linked ones tying to a specific past story, others topic-tied. Event ones showed a clear click lift, holding after controls. No overall engagement hit, and users there trended toward feeling more in control per surveys.
Alex: Huh, so specific story bridges worked best, even if rare. With small survey numbers, those trends point to valuing balance more in nudge groups?
Sam: Yes—trends only. Dual alone users agreed more than baseline on wanting future domestic-global mix. Rewrite group saw higher control sense. Logs back no long-term engagement fade.
Alex: Makes sense—the core list balance drives the shift, previews aid subtly. But with real-world rollout in mind, what limits should we note here?
Sam: The study highlights a few. Sample was 120 active U.S. users on POPROX, a newer platform prone to gradual drop-off—limiting how broadly results apply to bigger apps. They only tracked opens and reads, not finer details like time spent, due to ethical data rules. Surveys drew just 28 responses, so perceptual shifts are trends, not firm stats.
Alex: Fair points—small group and platform quirks mean caution on scaling, but the click patterns still suggest nudges can shift habits meaningfully.
Sam: Exactly. This points to algorithmic tweaks like dual calibration as a practical way to foster more balanced views in news feeds, turning available global stories into actual reading without hurting engagement. Overall, it's a clear step toward recommendation systems that build civic awareness through sustained diversity.
Alex: That's a grounded takeaway—nudges that personalize balance without the backlash. Thanks, Sam, for walking through the logic so clearly. And that's our look at balancing news perspectives. Thanks for listening to ResearchPod.