朱振山, 陈豪, 陈炜龙, 黄缨惠. 基于深度强化学习的含储能船舶的海岛-海上渔排能源运输策略研究[J]. 中国电机工程学报, 2025, 45(7): 2486-2499. DOI: 10.13334/j.0258-8013.pcsee.242305
引用本文: 朱振山, 陈豪, 陈炜龙, 黄缨惠. 基于深度强化学习的含储能船舶的海岛-海上渔排能源运输策略研究[J]. 中国电机工程学报, 2025, 45(7): 2486-2499. DOI: 10.13334/j.0258-8013.pcsee.242305
ZHU Zhenshan, CHEN Hao, CHEN Weilong, HUANG Yinghui. Research on Energy Transportation Strategies Between Islands and Offshore Fish Farms for Ships With Energy Storage Based on Deep Reinforcement Learning[J]. Proceedings of the CSEE, 2025, 45(7): 2486-2499. DOI: 10.13334/j.0258-8013.pcsee.242305
Citation: ZHU Zhenshan, CHEN Hao, CHEN Weilong, HUANG Yinghui. Research on Energy Transportation Strategies Between Islands and Offshore Fish Farms for Ships With Energy Storage Based on Deep Reinforcement Learning[J]. Proceedings of the CSEE, 2025, 45(7): 2486-2499. DOI: 10.13334/j.0258-8013.pcsee.242305

基于深度强化学习的含储能船舶的海岛-海上渔排能源运输策略研究

Research on Energy Transportation Strategies Between Islands and Offshore Fish Farms for Ships With Energy Storage Based on Deep Reinforcement Learning

  • 摘要: 针对海上渔排与风光资源富余岛屿能源交互问题,该文提出含全电力船舶(all-electric ship,AES)的岛屿-海上渔排-海岸能源运输策略,利用能够很好处理海面风光不确定性问题以及适应较大规模能源转移模型的深度强化学习方法对上述能源运输模型进行求解。首先,将移动式储能电池组细化为满充电池、空载电池以及不完全充电电池;其次,将上述能源运输问题建模为含混合动作空间的马尔可夫决策过程;考虑到针对混合动作空间问题,提出一种适用于混合动作空间的基于多批次前向传播的参数化双深度Q网络,该方法通过多步前向传递策略对不相关离散与连续动作进行解耦,减少了智能体训练过程中的波动性并能够收敛于更优的解;最后,通过算例仿真可知,所提策略能够有效实现各地点间能量转移,所提算法相较于传统适用于离散动作空间的深度强化学习方法更加灵活,在目标场景下能够实现更优运行。此外,在模型逐渐扩大的情况下,将该文方法与传统方法求解效果进行对比,验证所提方法在解决大规模能源运输问题的优势。

     

    Abstract: This paper addresses the issue of energy interaction between offshore fish farms and islands with surplus wind and solar resources by developing an energy transportation strategy involving fully electric ships for the island-fish farm-coast system. The proposed strategy utilizes deep reinforcement learning, which is well-suited to managing the uncertainties of offshore wind and solar resources and can accommodate large-scale energy transfer models. First, the mobile energy storage battery group is detailed into fully charged, unloaded, and partially charged batteries. Then, the energy transportation problem is modeled as a Markov Decision Process with a hybrid action space. To solve the hybrid action space issue, a parameterized dual deep Q-network based on multi-batch forward propagation is proposed. This method decouples the unrelated discrete and continuous actions using a multi-step forward pass strategy, reducing volatility during the agent's training process and converging to a more optimal solution. Finally, simulation results verify that the proposed strategy effectively facilitates energy transfer between locations. Compared to traditional deep reinforcement learning methods suited for discrete action spaces, the proposed algorithm demonstrates greater flexibility and achieves superior performance in the target scenario. Additionally, comparative analysis in expanding model scales further validates the advantages of the proposed method in addressing large-scale energy transportation challenges.

     

/

返回文章
返回