MJ: Multi-turn LLM Jailbreaking via Decomposed Credit Assignment

Modern large language models (LLMs) are increasingly deployed in multi-turn conversational settings, where safety alignment techniques like RLHF have made them resistant to direct, single-turn harmful requests. However, these models remain vulnerable to multi-turn attacks that strategically shape context over time. The authors investigate how to effectively train automated attacker policies in this multi-turn environment, specifically addressing the challenge of credit assignment—determining which specific turns in a dialogue contribute to a successful jailbreak.

The authors propose Decomposed Credit GRPO (DC-GRPO), a reinforcement learning framework that extends Group Relative Policy Optimization (GRPO) to multi-turn dialogues. Standard GRPO typically assigns a single trajectory-level reward to all turns, which the authors argue is too coarse. DC-GRPO instead decomposes the credit into two components: immediate reward (the direct outcome of a turn) and future reward (the potential for future progress). By normalizing these components at the group level for each turn, the model can more accurately identify which actions lead to successful jailbreaks. The authors implement two weighting variants: static-weighted (using a fixed coefficient) and dynamic-weighted (using rollout-group statistics).

DC-GRPO demonstrates superior performance compared to existing state-of-the-art methods like SEMA and TROJail. Across four victim LLMs and three benchmarks, the dynamic-weighted and static-weighted variants achieved average Attack Success Rates (ASR) of 98.26% and 97.88%, respectively. The authors conclude that the primary benefit arises from the turn-level credit assignment structure itself, rather than the specific weighting rule used to balance immediate and future rewards. This approach allows for more precise policy updates, enabling lightweight models to learn effective, adaptive attack strategies.