LLMs Silently Correct African American English: Auditing and Mitigating Dialect Bias via Activation Steering

This paper investigates the systematic tendency of large language models (LLMs) to "correct" African American English (AAE) into Standard American English (SAE), a phenomenon the authors term dialect preference bias. To address this, the researchers developed REAL-AAE, a large-scale, human-validated parallel corpus of 17,479 AAE/SAE/AAE-back triplets. They audited six instruction-tuned LLMs (ranging from 14B to 70B parameters) using both discriminative consistency metrics—specifically a new conditional Dialect Group Invariance (cDGI) metric—and generative preference metrics like perplexity and forced-choice continuation. Finally, they introduced a novel, training-free mitigation technique called activation steering, which uses causal tracing to identify and intervene in the specific model layers responsible for dialect bias.

The study reveals that all six tested LLMs exhibit a strong, systematic preference for SAE, even when the provided context is in AAE. This behavior is not merely a stylistic preference but a consistent bias that treats AAE as an error to be corrected. The feature-level localization analysis identified that syntactic constructions, particularly negative concord (e.g., "ain't nobody"), act as universal triggers for this bias across all models. The proposed activation steering method proved highly effective, reducing dialect bias 5 to 20 times more than standard prompting techniques while successfully preserving the fluency of the model's output.

As LLMs are increasingly deployed in high-stakes domains like hiring, healthcare, and content moderation, dialect bias poses a significant risk of linguistic discrimination. By demonstrating that this bias is not just a data artifact but a deeply embedded behavior in model activations, this work provides a scalable, inference-time solution that allows developers to mitigate bias without the high costs of retraining or the performance degradation often associated with architectural changes.