HEURISTIC LEARNING FOR ACTIVE FLOW CONTROL USING CODING AGENTS

This paper introduces a heuristic-learning framework that replaces traditional neural-network-based reinforcement learning (DRL) with an iterative program-synthesis approach. Instead of optimizing millions of weights via gradient descent, the authors use coding agents to search directly for explicit, executable feedback laws. These agents interact with fluid dynamics simulators through a fixed, public interface, proposing and refining controller code based on performance feedback. The goal is to achieve competitive control performance while maintaining full interpretability and inspectability of the resulting algorithms.

The framework was tested against state-of-the-art DRL baselines (such as PPO and SAC) across 13 diverse fluid-dynamics benchmarks, ranging from one-dimensional problems to complex three-dimensional simulations. The authors strictly enforced identical simulation budgets, observation spaces, and action constraints for both the heuristic-learning agents and the DRL baselines to ensure a fair comparison. The results show that the discovered heuristic controllers are not only competitive in performance but also provide physically meaningful insights into the underlying flow-control mechanisms.

Conventional DRL often produces "black-box" neural policies that are difficult to interpret, modify, or transfer across different physical configurations. By shifting the paradigm toward explicit, code-based controllers, this research offers a path to creating control systems that fluid dynamicists can directly analyze and trust. This approach bridges the gap between automated machine learning and traditional control theory, potentially accelerating the adoption of AI-driven methods in engineering fields where transparency and physical consistency are paramount.