Auditing Source Exposure in Baidu and Google AI Search

As search engines transition from traditional ranked lists to AI-generated overviews, they are increasingly acting as curators of information. This study investigates how these overviews function across different platforms (Baidu and Google) and languages (Chinese and English). By auditing nearly 2,000 search queries, the researchers examine not only the content of the AI-generated answers but also the visibility of the sources cited within those answers.

The researchers used a set of 500 English queries from the MS MARCO dataset, translating them into Simplified Chinese to create aligned query pairs. These were submitted to Baidu and Google to observe four settings: Baidu Chinese, Baidu English, Google Chinese, and Google English. The team measured the frequency of overview appearance, the concentration of host domains (using Gini coefficients), the overlap of cited sources (using Jaccard similarity), and the semantic similarity of the generated answers (using cosine similarity of embeddings).

The audit reveals that while AI overviews are a standard feature on Google (appearing in over 97% of cases regardless of language), Baidu shows a strong language bias, triggering overviews for 82% of Chinese queries but only 4% of English queries.

Crucially, the study finds that high semantic similarity between answers does not imply similar source exposure. While the answers provided by different platforms for the same query intent are often semantically aligned, the sources cited are vastly different. Baidu Chinese, in particular, exhibits high source concentration, frequently citing its own subdomains (e.g., baike.baidu.com). In contrast, Google’s Chinese and English interfaces distribute visibility across a wider, more diverse set of external domains. The researchers conclude that source visibility and answer content are distinct dimensions of AI-mediated search that must be evaluated separately.