DRLFluent: A distributed co-simulation framework coupling deep reinforcement learning with Ansys-Fluent on high-performance computing systems

For active flow control (AFC), several frameworks have been developed to enable dynamic interactions between deep reinforcement learning (DRL) agents and computational fluids dynamics (CFD) environments. However, the highly coupled modules in the existing frameworks require massive development efforts to be applied to different flow scenarios. Moreover, these frameworks do not support applications to run flexibly on multiple nodes in high-performance computing (HPC) systems. Herein, we propose a new framework named as DRLFluent coupling an open-source DRL package with a well-developed general CFD solver, Ansys-Fluent. The distributed HPC deployment of DRLFluent is achieved by a broker architecture which enables the DRL client and CFD server written in different languages to communicate across different nodes via a pair of standardized interfaces. We have evaluated DRLFluent along with two other representative frameworks using a benchmark case in which a laminar flow around a circular cylinder (Re=100) is controlled by two synthetic jets. The strategy discovered in DRLFluent is capable of neutralizing the small asymmetry of the shedding vortices, yielding a higher drag reduction than the other two frameworks. When the number of parallel environments exceeds 20, DRLFluent exhibits the largest speedup among the frameworks evaluated. Capable of distributing workload across multiple nodes, DRLFluent is suitable for implementing DRL for AFC of complex flow scenarios which requires hundreds of cores or more. Benefitting from a loose coupling between DRL and CFD, the standardized interfaces and the user-friendly CFD solver, DRLFluent is readily applicable to a wide range of flow scenarios without laborious low-level programming efforts.