Using Gen AI for Early-Stage Market Research | James Brand et al. | ResearchPod

Transcript: Using Gen AI for Early-Stage Market Research

[[RP_SECTION:synthetic-market-research-limitations|Synthetic Market Research Limitations]]

Sam: [steady, matter-of-fact] Off-the-shelf language models can approximate average market signals for product features, but they tend to overrate novelty. That comes from James Brand, Ayelet Israeli, and Donald Ngwe, who studied synthetic customers for early-stage market research.

Alex: [curious, analytical] If the models overestimate interest in novel concepts, are they useless for testing genuinely disruptive, new-to-the-world products?

Sam: [grounded, teaching mode] Not necessarily, but it changes how you use them. Off-the-shelf models lack grounding, and they often read novelty as inherently valuable. The authors say category-specific fine-tuning is needed to correct for that. When you fine-tune on a company's proprietary historical survey data, the model's predictions get anchored in the behavioral patterns of that company's own customer base.

Alex: [processing] So the fine-tuning is an empirical anchor. But how does that work on a product the model has never seen? [[RP_SECTION:fine-tuning-for-predictive-accuracy|Fine-Tuning for Predictive Accuracy]]

Sam: [deliberate, clear] The base model has already absorbed implicit consumer preferences from its language training. If you fine-tune it on, say, several years of past toothpaste conjoint studies, you sharpen that signal. Then you prompt it with a new feature, like a cucumber-flavored toothpaste. It no longer leans on its general training, which defaults to excitement. It reflects the aggregate, more conservative choice patterns in your historical data.

Alex: [leaning in] That sounds like regression to past averages. Doesn't it make the model incapable of anticipating a market shift, or a new segment reacting differently? [[RP_SECTION:demographic-segmentation-challenges|Demographic Segmentation Challenges]]

Sam: [measured, acknowledging the point] That's the main limitation. The models handle broad, average trends reasonably well but fail at demographic segmentation. When the researchers simulated specific subgroups, the responses were inconsistent or exaggerated. In one case, the synthetic model predicted a price-sensitivity gap between two political groups nearly ten times larger than the gap in actual human surveys.

Alex: [analytical edge] That's a large discrepancy. Does fine-tuning help there, or does it compound the averaging?

Sam: [slower, for clarity] According to the paper, it makes the averaging more pronounced. The fine-tuned agents collapsed toward the mean and lost the ability to distinguish between personas. They would say both groups were willing to pay about the same. So you get a more accurate tool for the average customer, but it flattens heterogeneity, which is often the most valuable thing a market study reveals.

Using Gen AI for Early-Stage Market Research

James Brand, Ayelet Israeli, Donald Ngwe

Oct 1, 2026·4:07·user_upload

Synthetic CustomersFine-tuningConjoint AnalysisInnovation FunnelWillingness-to-Pay (WTP)

Listen

4 min

Briefing

The Role of Synthetic Customers in Innovation

Companies often struggle to efficiently filter early-stage product ideas, as traditional human-centric market research is frequently slow and costly. This paper explores the potential of using Large Language Models (LLMs) as synthetic customers to simulate market responses. By presenting structured product configurations to these models, researchers can estimate willingness-to-pay (WTP) and compare product alternatives before ever engaging a human respondent.

Methodology and Performance

The authors utilized a programmatic approach to prompt LLMs with choice-based questions, mimicking traditional conjoint analysis. Across various product categories like toothpaste and consumer electronics, the researchers found that LLMs could generate preference rankings that aligned with human-derived results. This capability allows firms to broaden the top of their innovation funnel, enabling the rapid testing of dozens or even hundreds of concepts at a fraction of the time and cost required for traditional studies.

The Importance of Fine-Tuning

While "off-the-shelf" LLMs provide a useful baseline, they are prone to overestimating interest in novel or unusual features. The authors demonstrate that fine-tuning models with a company’s proprietary historical survey data significantly improves accuracy. This process helps the model learn specific consumer trade-offs, preventing the exaggerated enthusiasm often seen in base models. However, this advantage is domain-specific; fine-tuning a model on one product category does not necessarily improve its performance in a different, even if related, category.

Strategic Implications

Synthetic customers should be viewed as a tool for augmentation rather than a replacement for human research. LLMs currently lack the emotional intelligence and demographic nuance required for final validation or deep segmentation. Instead, they are best deployed as a high-speed filter to prioritize promising directions. Firms that successfully integrate these synthetic insights with traditional human research can achieve a significant competitive advantage by innovating faster and reducing the risk of costly product failures.

Key Terms and Definitions

Synthetic Customers — AI-generated personas or model outputs used to simulate consumer preferences and responses to product concepts.
Conjoint Analysis — A statistical technique used to determine how people value different attributes (features, functions, benefits) that make up an individual product or service.
Fine-tuning — The process of adjusting an LLM’s parameters using a company’s specific historical survey data to improve its ability to predict preferences within a particular market context.
Willingness-to-Pay (WTP) — The maximum price at which a consumer is willing to purchase a specific product or feature, estimated here through simulated choice-based queries.

Using Gen AI for Early-Stage Market Research: Research Paper Podcast