Quantifying Training Membership Information in the Hyperspherical Embedding Geometry of Face Recognition Models

Research Approach

This study investigates whether face recognition models, which map faces to points on a unit hypersphere, retain geometric traces of their training data. The authors examine whether identities seen during training form tighter or more distinct clusters than unseen identities. To quantify this, they trained 180 distinct face recognition models using a factorial design that varied backbone size (IResNet-34/50/100), loss functions (ArcFace, CosFace, MagFace), training duration, and the number of training identities. They evaluated these models across nine benchmarks, including a same-domain held-out set designed to isolate membership signals from domain-specific noise.

Key Findings

The researchers computed four geometric statistics—pairwise cosine similarity, von Mises-Fisher concentration, penalised logit, and prototype softmax cross-entropy—to measure cluster tightness. They found that the number of training identities is the dominant factor in determining how well a model can distinguish between members and non-members. As the training set size increases, the membership signal in the embedding geometry weakens monotonically. Conversely, model architecture and the specific angular-margin loss head contribute very little to this separability.

The Role of Domain Shift

A critical finding is that standard benchmarks often conflate membership with domain shift. When evaluating models on datasets that differ from the training data in pose, age, or image quality, the apparent membership signal is artificially inflated. The authors demonstrate that by using a same-domain held-out reference, one can better isolate the true membership signal. Finally, they show that fusing multiple geometric statistics into a single learned classifier provides a more robust membership inference than relying on any individual statistic alone.

This work provides a systematic audit of data privacy in open-set face recognition. By demonstrating that membership information is encoded in the geometry of embedding clusters, the study highlights a potential privacy risk where an auditor could determine if a specific individual was part of a model's training set without needing access to the model's internal weights or training logs. It also cautions researchers that performance metrics on cross-domain benchmarks may be misleading if they do not account for the confounding effects of domain-specific geometric shifts.