DialogPII: A multilingual dataset of synthetic dialog transcripts to detect personal information

As conversational data becomes increasingly vital for research in healthcare, social sciences, and customer support, the need for effective de-identification systems has grown. However, real-world conversational data is often restricted by ethical and legal privacy concerns, making it difficult to share. To address this, the authors developed DialogPII, a synthetic, multilingual dataset designed to support the development of automated systems for detecting personally identifiable information (PII) in spoken and written dialogs.

The researchers generated synthetic dialogs across eight scenarios—including emergency calls, therapy sessions, and medical interviews—using large language models. These dialogs were manually curated for plausibility and localized into 11 languages (Arabic, English, Finnish, French, German, Hindi, Italian, Polish, Portuguese, Spanish, and Turkish). To simulate real-world conditions, the team converted these texts into synthetic speech and then transcribed them using automatic speech recognition (ASR), creating a resource that accounts for transcription errors and speaker diarization challenges.

The authors provide a comprehensive technical validation of the dataset, including inter-annotator agreement and benchmark experiments using a transformer-based sequence labeling model (ModernBERT with CRF). The model achieved strong performance on the synthetic data, with an average lenient F1 score of 89.31% across all languages. When evaluated on speech-derived transcripts, performance decreased slightly (average lenient F1 of 85.54%), reflecting the impact of ASR-induced noise on entity recognition.

External validation on the real-world CallFriend corpus demonstrated that models trained on DialogPII can generalize to spontaneous, informal conversations, achieving type-agnostic F1 scores above 83%. This suggests that the synthetic data provides a viable foundation for training robust de-identification systems that can be adapted to diverse, naturally occurring conversational domains.

DialogPII lowers the barrier to entry for researchers working on privacy-preserving NLP. By providing a shareable, multilingual, and multi-modal resource, it enables the development of anonymization pipelines that can be applied to sensitive audio and text data. This is a critical step toward making genuine conversational datasets accessible for scientific research without compromising individual privacy.