ResearchPod Summary
When an AI agent attempts a tool call that fails, standard practice is to append the failed action and the resulting error message to the transcript. The intuition is that this provides the model with corrective feedback, allowing it to learn from its mistake and try a different, successful approach. This study demonstrates that this intuition is fundamentally flawed for small language models. Instead of learning to avoid the error, these models are statistically more likely to repeat the exact same failed action after seeing it in the transcript.
The author defines the corrective gain of a failure record as the change in the log-probability of re-emitting the failed action. Across six different instruction-tuned models and two distinct environments—simulated tool calling and program repair—the corrective gain is consistently negative. This means the feedback record acts as a magnet for repetition rather than a deterrent. The effect is not merely an average; it holds for 90% to 100% of individual test items. When greedy decoding is used, the probability of the model repeating the failed call verbatim rises significantly after the failure is recorded.
By using counterfactual observations, the study separates the feedback record into two components: the surface form (the text of the failed action itself) and the semantic content (the error message indicating failure). The results show that the surface form accounts for roughly 83% of the damage. The model's tendency to copy text from its context window—a well-documented behavior in small models—overpowers its ability to interpret the error message. The problem is not that the model fails to understand the error, but that the harness provides a strong structural reason to repeat the action by placing the failed string directly into the context.
This finding dictates which interventions actually work. Simply instructing the model not to repeat itself is ineffective, as the model struggles to follow negative instructions that name the very string it is being told to avoid. Similarly, deleting the failed attempt to "clean" the context is counterproductive, as it restores the exact state that led to the original failure. The most effective remedies are structural: replacing the verbatim failed call with a runtime-generated description of the error removes the surface-form hazard while keeping the diagnostic information, and banning previously-failed strings at the decoder level prevents repetition without requiring model retraining.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.