强化学习方法在翼型拍动实验中的应用

张进; 周雷; 曹博超

doi:10.7638/kqdlxxb-2023.0071

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

强化学习方法在翼型拍动实验中的应用

研究论文 | 更新时间：2026-05-28

- 强化学习方法在翼型拍动实验中的应用
- Application of reinforcement learning method in flapping airfoil experiment
- 空气动力学学报 2023年41卷第9期页码：20-29
- 作者机构：
  
  复旦大学航天航空系,上海,200433
- 作者简介：
  
  张进（1999-），男，重庆彭水人，硕士研究生，研究方向：实验流体力学，强化学习. E-mail：zhangjin_132@163.com
  曹博超*，副教授，研究方向：空气动力学，实验流体力学，强化学习. E-mail：cbc@fudan.edu.cn
- 基金信息：
- DOI：10.7638/kqdlxxb-2023.0071
  中图分类号： O352;TP181
- 纸质出版：2023
- 稿件说明：
移动端阅览
张进, 周雷, 曹博超. 强化学习方法在翼型拍动实验中的应用[J]. 空气动力学学报, 2023, 41(9): 20−29.

ZHANG Jin, ZHOU Lei, CAO Bochao. Application of reinforcement learning method in flapping airfoil experiment[J]. 2023, 41(9): 20-29.
张进, 周雷, 曹博超. 强化学习方法在翼型拍动实验中的应用[J]. 空气动力学学报, 2023, 41(9): 20−29. DOI： 10.7638/kqdlxxb-2023.0071

ZHANG Jin, ZHOU Lei, CAO Bochao. Application of reinforcement learning method in flapping airfoil experiment[J]. 2023, 41(9): 20-29. DOI： 10.7638/kqdlxxb-2023.0071.

摘要

将深度强化学习方法应用于水洞实验，实现了实验室内的自动闭环优化框架，并用该框架优化了雷诺数

= 1.3×10

下纯俯仰运动的NACA0012翼型模型的推进效率。现有的相关研究往往将运动模式限制为某种周期性函数，具有局限性。借助于强化学习方法，实现了在更广的非周期动作空间中的动作搜索。在实验中，模型自动地与水洞环境进行交互，最终学习到了高效推进的非周期运动策略。另外，通过修改奖励函数，实现了在给定推力阈值以上的效率优化。研究结果显示，强化学习模型可以在实验过程中通过不断调整拍动动作的幅度和频率来实现推进效率的持续提升，并且最终通过强化学习方法获得的最优拍动动作均与正弦拍动动作接近，得到的最优推进效率基本位于同等幅度正弦动作效率的上边界。研究展示了强化学习方法用于复杂流动控制问题的可行性。

Abstract

This study presents the application of the deep reinforcement learning (DRL) method in water tunnel experiments

which establishes an automatic closed-loop optimization framework in the laboratory. The framework is used to optimize the propulsion efficiency of a NACA0012 airfoil model under pure pitching motion at a Reynolds number of

= 1.3×10

. Existing related studies often limit the motion patterns to periodic functions

thanks to the DRL method

the optimization process can take place in a broader non-periodic action space. In the experiment

the airfoil automatically interacts with the water tunnel environment

and ultimatel

y learns highly efficient non-periodic motions. By modifying the reward function

the efficiency optimization can be achieved above a given thrust threshold. The present findings demonstrate the DRL model can continuously enhance the propulsion efficiency by adjusting the amplitude and frequency of the flapping motion. Moreover

the optimal flapping motion obtained by DRL is close to the sinusoidal motion

and the achieved optimal propulsion efficiency lies on the upper bound of that of the sinusoidal motion with a similar amplitude. The present study demonstrates the feasibility of using the DRL method for complex flow control problems.

关键词

Keywords

references

BRUNTON S L, NOACK B R. Closed-loop turbulence control: progress and challenges[J]. Applied Mechanics Reviews, 2015, 67(5): 050801. DOI: 10.1115/1.4031175

SUTTON R S, BARTO A G. Reinforcement learning: an introduction[M]. Cambridge: MIT Press, 1998.

GARNIER P, VIQUERAT J, RABAULT J, et al. A review on deep reinforcement learning for fluid mechanics[J]. Computers Fluids, 2021, 225: 104973. DOI: 10.1016/j.compfluid.2021.104973

BRUNTON S L, NOACK B R, KOUMOUTSAKOS P. Machine learning for fluid mechanics[J]. Annual Review of Fluid Mechanics, 2020, 52: 477-508. DOI: 10.1146/annurev-fluid-010719-060214

REN F, GAO C Q, TANG H. Machine learning for flow control: Applications and development trends[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(4): 524686. (in Chinese)

REDDY G, CELANI A, SEJNOWSKI T J, et al. Learning to soar in turbulent environments[J]. Proceedings of the National Academy of Sciences of the United States of America, 2016, 113(33): E4877-E4884. DOI: 10.1073/pnas.1606075113

REDDY G, WONG-NG J, CELANI A, et al. Glider soaring via reinforcement learning in the field[J]. Nature, 2018, 562(7726): 236-239. DOI: 10.1038/s41586-018-0533-0

RABAULT J, KUCHTA M, JENSEN A, et al. Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control[J]. Journal of Fluid Mechanics, 2019, 865: 281-302. DOI: 10.1017/jfm.2019.62

FAN D X, YANG L, WANG Z C, et al. Reinforcement learning for bluff body active flow control in experiments and simulations[J]. Proceedings of the National Academy of Sciences of the United States of America, 2020, 117(42): 26091-26098. DOI: 10.1073/pnas.2004939117

LI J C, ZHANG M Q. Reinforcement-learning-based control of confined cylinder wakes with stability analyses[J]. Journal of Fluid Mechanics, 2022, 932: A44. DOI: 10.1017/jfm.2021.1045

YAO Z Y, SHI Z W, DONG Y Z. Deep reinforcement learning for the control of airfoil flow separation[J]. Journal of Experiments in Fluid Mechanics, 2022, 36(3): 55-64. (in Chinese)

GAZZOLA M, HEJAZIALHOSSEINI B, KOUMOUTSAKOS P. Reinforcement learning and wavelet adapted vortex methods for simulations of self-propelled swimmers[J]. SIAM Journal on Scientific Computing, 2014, 36(3): B622-B639. DOI: 10.1137/130943078

ZHU Y, TIAN F B, YOUNG J, et al. A numerical study of fish adaption behaviors in complex environments with a deep reinforcement learning and immersed boundary-lattice Boltzmann method[J]. Scientific Reports, 2021, 11: 1691. DOI: 10.1038/s41598-021-81124-8

LIAO J C. The role of the lateral line and vision on body kinematics and hydrodynamic preference of rainbow trout in turbulent flow[J]. Journal of Experimental Biology, 2006, 209(20): 4077-4090. DOI: 10.1242/jeb.02487

GAZZOLA M, TCHIEU A A, ALEXEEV D, et al. Learning to school in the presence of hydrodynamic interactions[J]. Journal of Fluid Mechanics, 2016, 789: 726-749. DOI: 10.1017/jfm.2015.686

NOVATI G, VERMA S, ALEXEEV D, et al. Synchronisation through learning for two self-propelled swimmers[J]. Bioinspiration Biomimetics, 2017, 12(3): 036001. DOI: 10.1088/1748-3190/aa6311

VERMA S, NOVATI G, KOUMOUTSAKOS P. Efficient collective swimming by harnessing vortices through deep reinforcement learning[J]. Proceedings of the National Academy of Sciences of the United States of America, 2018, 115(23): 5849-5854. DOI: 10.1073/pnas.1800923115

TRIANTAFYLLOU M S, TRIANTAFYLLOU G S, YUE D K P. Hydrodynamics of fishlike swimming[J]. Annual Review of Fluid Mechanics, 2000, 32: 33-53. DOI: 10.1146/annurev.fluid.32.1.33

WU T Y. Fish swimming and bird/insect flight[J]. Annual Review of Fluid Mechanics, 2011, 43: 25-58. DOI: 10.1146/annurev-fluid-122109-160648

SENTURK U, SMITS A J. Numerical simulations of the flow around a square pitching panel[J]. Journal of Fluids and Structures, 2018, 76: 454-468. DOI: 10.1016/j.jfluidstructs.2017.11.001

SENTURK U, SMITS A J. Reynolds number scaling of the propulsive performance of a pitching airfoil[J]. AIAA Journal, 2019, 57(7): 2663-2669. DOI: 10.2514/1.j058371

FLORYAN D, VAN BUREN T, ROWLEY C W, et al. Scaling the propulsive performance of heaving and pitching foils[J]. Journal of Fluid Mechanics, 2017, 822: 386-397. DOI: 10.1017/jfm.2017.302

LAGOPOULOS N S, WEYMOUTH G D, GANAPATHISUBRAMANI B. Universal scaling law for drag-to-thrust wake transition in flapping foils[J]. Journal of Fluid Mechanics, 2019, 872: R1. DOI: 10.1017/jfm.2019.361

LAUDER G V. Fish locomotion: recent advances and new directions[J]. Annual Review of Marine Science, 2015, 7: 521-545. DOI: 10.1146/annurev-marine-010814-015614

FLORYAN D, VAN BUREN T, SMITS A J. Forces and energetics of intermittent swimming[J]. Acta Mechanica Sinica, 2017, 33(4): 725-732. DOI: 10.1007/s10409-017-0694-3

AKOZ E, MOORED K W. Unsteady propulsion by an intermittent swimming gait[J]. Journal of Fluid Mechanics, 2018, 834: 149-172. DOI: 10.1017/jfm.2017.731

CHAO L M, PAN G, ZHANG D, et al. Numerical investigations on the force generation and wake structures of a nonsinusoidal pitching foil[J]. Journal of Fluids and Structures, 2019, 85: 27-39. DOI: 10.1016/j.jfluidstructs.2018.12.002

VAN BUREN T, FLORYAN D, QUINN D, et al. Nonsinusoidal gaits for unsteady propulsion[J]. Physical review fluids, 2017, 2(5): 053101. doi: 10.1103/PhysRevFluids.2.053101

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于条件扩散模型的火箭气动外形快速设计方法

基于机器学习的极端风场短时预测研究进展与思考

大型结构风效应流固耦合机器学习研究进展

基于深度强化学习的三维变形机翼反设计方法

基于机器学习的房颤病人左心耳血流动力学研究