AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence

For decades, search engines functioned primarily as directories that directed user attention to third-party publishers via a list of links. The integration of generative artificial intelligence into search platforms, such as Google's AI Overviews and AI Mode, fundamentally alters this dynamic by synthesizing answers directly on the search results page. This raises critical questions about how AI features influence user behavior, shape information ecosystems, and impact web traffic referrals to publishers who supply the underlying content. This paper investigates the causal effects of these generative AI search features on user behavior, engagement, perceptions, and publisher traffic.

The researchers conducted a preregistered field experiment involving 1,100 participants recruited from Prolific and university work-study programs. Using a custom browser extension, participants were randomly assigned to one of three search conditions for a period of seven days: a No AI Search condition that hid AI Overviews and redirected AI Mode, a Current Search control group with unaltered Google Search functionality, and an AI Mode Search condition that routed all searches to a conversational AI experience. The researchers tracked daily browsing behavior, search sessions, click-through rates, and domain-specific clicks, complemented by pre- and post-experiment surveys measuring trust, satisfaction, and user agency.

Assigning users to the conversational AI Mode search condition caused a sharp decline in outbound clicks, reducing click-through rates to external sites by 18.8 percentage points. Specifically, user clicks to news publishers, Wikipedia, and Reddit dropped significantly, alongside a complete disappearance of ad clicks because AI Mode did not feature advertisements during the study. Contrary to claims that conversational search improves user experience, AI Mode reduced perceived usefulness, user satisfaction, and user agency while eroding overall trust in the information found on Google. Furthermore, forcing AI search caused users to increase search engine substitution by turning to competitor platforms like Bing or DuckDuckGo. Conversely, removing AI features through the No AI condition increased click-through rates to external publishers.