Value-Aware Prediction for Robust Multi-Agent Coordination Under Communication Loss

In multi-agent reinforcement learning (MARL), agents often rely on shared observations to coordinate effectively. However, in real-world deployments like aerial swarms, communication is frequently interrupted. While existing methods use predictive models to estimate missing information during these gaps, they typically treat all environment transitions as equally important. This paper investigates whether prioritizing transitions that are critical to the agent's policy—rather than just reconstructing all data—can improve robustness during communication failure.

The authors propose Value-Aware MARO, an extension of the Multi-Agent Observation Sharing under Communication Dropout (MARO) framework. The core innovation is a value-aware loss function for the predictor. Instead of using a standard negative log-likelihood (NLL) objective, the predictor's loss is dynamically weighted by the advantage estimates derived from the underlying actor-critic architecture (specifically using PPO). By using the advantage as an importance weight, the model focuses its learning capacity on transitions that the policy identifies as high-return, effectively filtering out stochastic exploration noise and outdated dynamics.

The framework was evaluated in the Multi-Agent Particle Environment under varying levels of communication reliability. The results show that while the baseline performs adequately under high connectivity, it suffers from severe performance collapse as communication drops below 40%. In contrast, the Value-Aware MARO approach maintains stable performance in these high-attrition scenarios. Specifically, the method achieved an average improvement in mean returns of over 20% and reduced performance variance by approximately 64.7% compared to the unweighted baseline.