Subliminal Learning is Non-Semantic Distillation

Subliminal Learning (SL) is a phenomenon where a student model adopts a bias or behavior from a teacher model by training on synthetic data that appears entirely unrelated to the bias. For example, a model prompted to 'love owls' might generate lists of random numbers that, when used to train a student model, cause the student to also develop a preference for owls. This study investigates the underlying mechanisms of this transfer, specifically whether the latent signals in the data are semantic (conceptually meaningful) or non-semantic (spurious noise).

The authors hypothesize that SL is mediated by non-semantic, spurious connections within the model's weights. To test this, they introduced Gaussian noise to the weights of both teacher and student models during the distillation process. They observed that adding this noise increased the magnitude of subliminal transfer by up to 1.9 times in Gemma and 1.3 times in Llama. This suggests that the 'subliminal distribution shift'—the mechanism by which the bias is encoded—is not a result of meaningful semantic associations, but rather a byproduct of the model's internal, near-random weight structures.

The researchers compared two methods of biasing the teacher: system prompting and steering vectors. They found that while both methods successfully induce bias in the student, the resulting students are mechanistically distinct. Students trained on data from steered teachers exhibit activation patterns that directly imitate the steering vectors used in the teacher. In contrast, students trained on prompted data do not show this imitation. This indicates that SL is a fine-grained process where the student learns not just the bias itself, but also the specific nature of the intervention applied to the teacher.