Trustworthy synthetic data for campaign decision support: strategy simulation fidelity and the PolicySynth framework

Modern decision support systems (DSS) increasingly rely on synthetic customer populations to perform what-if analyses while adhering to privacy regulations. However, current evaluation methods focus on distributional similarity—ensuring the synthetic data looks like the real data statistically—rather than decision alignment. This creates a gap where a synthetic population might pass standard statistical tests but still lead managers to fund ineffective marketing campaigns. This paper addresses this by shifting the focus from statistical fidelity to decision fidelity.

To bridge the decision-alignment gap, the authors introduce three primary contributions:

1. Strategy Simulation Fidelity (SSF): A metric that quantifies the percentage of campaign strategies (go/no-go decisions) where the synthetic population yields the same result as the real population.
2. PolicySynth Architecture: A denoising diffusion-based generator that incorporates a decision-conditioning loop. By feeding the output of a production churn scorer back into the training process, the generator is forced to preserve the specific relationships that drive managerial decisions.
3. Three-Axis Reporting Standard: A deployment checklist requiring candidates to be evaluated on SSF, membership-inference resistance (privacy), and novel-record rate (to prevent verbatim copying).

Evaluated on telecommunications churn and banking acquisition datasets, PolicySynth demonstrated superior stability compared to standard architectures like CTGAN. While CTGAN showed significant variance between training cycles—leading to reversed campaign recommendations in one out of nine cases—PolicySynth’s recommendations remained stable, shifting by at most 1.2 percentage points. The authors demonstrate that while a bootstrap baseline can match SSF, it fails on privacy and novelty, proving that decision alignment alone is insufficient for a deployable system.