Language Shapes Instruction Hierarchy Compliance in Multilingual LLMs

As large language models (LLMs) are increasingly deployed in multilingual environments, it is critical to ensure they correctly follow instruction hierarchies (IH)—the principle that higher-priority instructions (e.g., system prompts) must override lower-priority ones (e.g., user messages). This paper investigates whether IH compliance remains stable across different languages or if the language of an instruction itself influences the model's ability to prioritize it correctly.

The authors introduce XIH-Bench, a large-scale benchmark designed to evaluate IH compliance across six typologically diverse languages (English, German, Spanish, French, Hindi, and Chinese). The benchmark holds the underlying task and conflict structure constant while systematically varying the languages assigned to higher- and lower-priority instructions. It covers four domains—Rule-Following, Task-Execution, Safety, and Persona Injection—and three hierarchy settings (System > User, System > Tool, and User > Tool), totaling nearly 79,000 evaluation instances across 13 different LLMs.

The study reveals that language is not a neutral carrier of instructions but actively shapes how models resolve conflicts. Two primary patterns emerge:

1. Language-Dependent Asymmetry: A language that strengthens a model's compliance when used in a high-priority position can become disruptive when used in a low-priority position. For example, dominant languages in a model's training data are often harder to suppress when they appear as lower-priority instructions, creating potential security and reliability risks.
2. The Language Boundary Effect: Models consistently demonstrate higher compliance in cross-language conflicts compared to same-language conflicts. This suggests that the linguistic contrast between instructions helps the model better distinguish between different levels of the hierarchy.

These results demonstrate that existing English-centric evaluations of instruction hierarchy are insufficient for global deployments, as they fail to account for the systematic variation introduced by multilingualism.