Diagnosing Task Insensitivity in Language Agents

Large language models (LLMs) are increasingly used as autonomous agents for long-horizon tasks. However, these agents often struggle with out-of-distribution (OOD) generalization. This paper investigates whether these failures stem from 'task insensitivity'—a tendency for models to ignore specific task instructions in favor of memorized action patterns learned during training.

The authors first diagnose this behavior by corrupting task descriptions in environments like ALFWorld, ScienceWorld, and WebShop. They observe that models often continue to execute actions associated with the original task even when the instruction is rendered nonsensical. They further analyze attention mechanisms during training, finding that models exhibit a consistent 'attention drift' where they prioritize local observations over the task instruction. To address this, the authors introduce Task-Perturbed NLL Optimization, a contrastive regularizer that penalizes the model if it fails to distinguish between the original task and a semantically similar but distinct replacement task.

The study demonstrates that agents frequently reconstruct and execute familiar tasks even when instructions are corrupted or replaced, indicating that they are not truly following instructions but rather defaulting to learned heuristics. This behavior is linked to an optimization bias where the model learns to rely on dynamic, step-specific state signals rather than the static task description. The proposed Task-Perturbed NLL Optimization effectively mitigates this by explicitly training the model to maintain a performance gap between original and perturbed task prompts, resulting in improved OOD robustness and more stable attention to task-relevant tokens.

As LLM agents are deployed in increasingly complex and varied environments, their ability to remain grounded in specific, novel instructions is critical. This work highlights a fundamental limitation in standard supervised and reinforcement learning pipelines for agents, providing a practical, lightweight intervention to ensure that agents remain sensitive to their goals rather than just their training history.