Gimitest: A Comprehensive Tool for Testing Reinforcement Learning Policies

Reinforcement learning (RL) policies are increasingly deployed in critical systems, yet they remain susceptible to unsafe behaviors and adversarial attacks. Testing these policies is notoriously difficult because existing methods are often fragmented, targeting specific environments or algorithms rather than providing a generalized approach. Researchers and developers currently lack a unified tool to systematically apply different testing strategies—such as Search-Based Software Testing (SBST), Metamorphic Testing (MT), and Adversarial Testing (AT)—across the diverse landscape of RL frameworks.

Gimitest addresses this gap by providing a comprehensive, open-source framework designed to standardize RL policy testing. The architecture centers on a decorator pattern that wraps standard environment simulators (e.g., Farama Gymnasium, PettingZoo). By utilizing a 'GTest' class, users can inject custom logic into the environment's step and reset functions. This allows for the systematic modification of environmental parameters, the injection of adversarial noise, or the configuration of specific initial states without needing to modify the underlying simulator code.

Beyond simple execution, Gimitest includes built-in logging capabilities that capture detailed execution traces, which are essential for diagnosing policy failures. The framework also provides a unified interface that facilitates integration with external tools, such as Large Language Models (e.g., GPT-4), to automate the generation of test cases. By abstracting the interaction between the policy and the environment, Gimitest enables researchers to leverage high-performance computing (HPC) resources to run large-scale simulations, significantly accelerating the identification of edge cases and policy vulnerabilities.