基于深度强化学习的车联网频谱分配研究

林志聪; 王永华; 万频

doi:10.12052/gdutxb.240111

欢迎来到能源电力期刊网！

English Version 期刊登录

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的车联网频谱分配研究

信息与通信工程 | 更新时间：2026-05-27

- 基于深度强化学习的车联网频谱分配研究
- Deep Reinforcement Learning-based Spectrum Allocation in Vehicular Networks
- 广东工业大学学报 2025年42卷第3期页码：101-110
- 作者机构：
  
  1. 广东工业大学自动化学院,广东,广州,510006
  2. 广州理工学院智能制造与电气工程学院,广东,广州,510540
- 作者简介：
  
  林志聪(1997–)，男，硕士研究生，主要研究方向为认知无线电、强化学习，E-mail：2112104287@mail2.gdut.edu.cn
  王永华(1979–)，男，副教授，博士，主要研究方向为智能信息处理、机器学习，E-mail：wangyonghua@gdut.edu.cn
- 基金信息：
  
  国家自然科学基金资助项目(61971147);广东省基础与应用基础研究基金资助项目(2023A1515011888);广东省研究生教育创新计划项目(2024JGXM_049)
- DOI：10.12052/gdutxb.240111
  中图分类号： TN929.5
- 纸质出版：2025
- 稿件说明：
移动端阅览
林志聪, 王永华, 万频. 基于深度强化学习的车联网频谱分配研究[J]. 广东工业大学学报, 2025, 42(3): 101–110.

Lin Zhicong, Wang Yonghua, Wan Pin. Deep Reinforcement Learning-based Spectrum Allocation in Vehicular Networks[J]. 2025, 42(3): 101-110.
林志聪, 王永华, 万频. 基于深度强化学习的车联网频谱分配研究[J]. 广东工业大学学报, 2025, 42(3): 101–110. DOI： 10.12052/gdutxb.240111

Lin Zhicong, Wang Yonghua, Wan Pin. Deep Reinforcement Learning-based Spectrum Allocation in Vehicular Networks[J]. 2025, 42(3): 101-110. DOI： 10.12052/gdutxb.240111.

摘要

为了缓解车联网中车辆用户数持续增长与频谱资源有限之间的矛盾，提出一种基于单调值函数分解结合自注意力机制的多智能体深度强化学习算法，旨在通过学习优化车联网中车对车链路(Vehicle-to-Vehicle，V2V) 信道选择和功率控制的联合策略，实现车对基础设施链路(Vehicle-to-Infrastructure，V2I) 总信道容量最大化，同时保证V2V链路传输时延和可靠性要求。针对车联网环境动态变化导致V2V链路无法实时收集完整信道状态信息的问题，为每条V2V链路构建深度循环Q网络，能使每条V2V链路基于自身局部观测信息自主决策。为了确保每条V2V链路局部策略的优化方向与全局利益最大化保持一致，设计了具有单调性约束的全局混合网络指引训练优化。此外，通过构建基于自注意力机制的信息交互模块，进一步优化V2V链路间的协作。仿真结果表明，与基线算法相比，本文所提出的算法在V2I链路总信道容量上提高了1.44~8.24个百分点，在V2V链路传输时延上缩短了1.93~15.04个百分点，能有效指导V2V链路根据环境的变化及时调整信道选择和功率控制的决策，从而保证了链路的通信质量。

Abstract

A dynamic spectrum allocation scheme is proposed to address the growing number of vehicles and limited spectrum resources in vehicular networks. It integrates the self-attention mechanism and monotonic value function factorisation into deep multi-agent reinforcement learning to optimize channel selection and power levels for vehicle-to-vehicle (V2V) links. The global objective is to maximize the sum throughput of vehicle-to-infrastructure (V2I) links

while meeting the latency and reliability constraints of V2V links. To handle incomplete real-time channel state information due to dynamic environment

a deep recurrent Q-network is deployed for each V2V link

enabling autonomous decision-making based on local observations. To align the local strategy optimization of each V2V link with the global objective

a global mixing network with monotonicity constraints is designed to guide the algorithm training. Additionally

an information interaction model based on the self-attention mechanism further optimizes collaboration between V2V links. Compared with the baselines

the proposed algorithm increases the sum throughput of V2I links by 1.44~8.24 percentage point and reduces the transmission delay of V2V links by 1.93~15.04 percentage point . These results confirm its effectiveness in optimizing channel selection and power levels for improved communication quality.

关键词

Keywords

references

NOOR-A-RAHIM M, LIU Z, LEE H, et al . A survey on resource allocation in vehicular networks[J ] . IEEE Transactions on Intelligent Transportation Systems, 2022, 23(2): 701-721.

DENG Y K, ZHANG L, LI J. Overview of research on privacy protection of Internet of Vehicles[J]. Application Research of Computers, 2022, 39(10): 2891-2906.

SUN W, STRÖM E G, BRÄNNSTRÖM F, et al . Radio resource management for D2D-based V2V communication[J ] . IEEE Transactions on Vehicular Technology, 2016, 65(8): 6636-6650.

WEI Q, SUN W, BAI B, et al . Resource allocation for V2X communications: a local search based 3D matching approach[C ] //2017 IEEE International Conference on Communications. Paris: IEEE, 2017: 1-6.

MEI J, ZHENG K, ZHAO L, et al . A latency and reliability guaranteed resource allocation scheme for LTE V2V communication systems[J ] . IEEE Transactions on Wireless Communications, 2018, 17(6): 3850-3860.

LIANG L, LI G Y, XU W. Resource allocation for D2D-enabled vehicular communications[J]. IEEE Transactions on Communications, 2017, 65(7): 3186-3197.

LI T, ZHU K, LUONG N C, et al . Applications of multi-agent reinforcement learning in future Internet: a comprehensive survey[J ] . IEEE Communications Surveys Tutorials, 2022, 24(2): 1240-1279.

LIANG L, YE H, LI G Y. Spectrum sharing in vehicular networks based on multi-agent reinforcement learning[J]. IEEE Journal on Selected Areas in Communications, 2019, 37(10): 2282-2292.

HE Y S, WANG Y H, WAN P, et al . An improved double deep Q network for multi-user dynamic spectrum access[J ] . Journal of Guangdong University of Technology, 2023, 40(4): 85-93.

XIANG P, SHAN H, WANG M, et al . Multi-agent RL enables decentralized spectrum access in vehicular networks[J ] . IEEE Transactions on Vehicular Technology, 2021, 70(10): 10750-10762.

MAFUTA A D, MAHARAJ B T J, ALFA A S. Decentralized resource allocation-based multiagent deep learning in vehicular network[J]. IEEE Systems Journal, 2023, 17(1): 87-98.

WANG L, WANG Y H, HE Y S, et al . A channel allocation method for internet of vehicles based on GRU and LSTM hybrid model[J ] . Telecommunication Engineering, 2024, 64(2): 273-280.

WANG W N, SU J, CHEN Y, et al . Multi-agent reinforcement learning enabled spectrum sharing for vehicular networks[J ] . Acta Electronica Sinica, 2024, 52(5): 1690-1699.

FANG W W, WANG Y P, ZHANG H, et al . Optimized communication resource allocation in vehicular networks based on multi-agent deep reinforcement learning[J ] . Journal of Beijing Jiaotong University, 2022, 46(2): 64-72.

LI L, LI W, CHEN X, et al . Learning based multi-channel access in vehicular networks with illegal operators and users[J ] . IEEE Transactions on Vehicular Technology, 2023, 72(3): 4097-4102.

RASHID T, SAMVELYAN M, DE WITT C S, et al . Monotonic value function fact orisation for deep multi-agent reinforcement learning[J ] . Journal of Machine Learning Research, 2020, 21(178): 7234-7284.

LUO S, LI Y, LI J, et al . S2RL: do we really need to perceive all states in deep multi-agent reinforcement learning?[C ] //Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Washington: ACM, 2022: 1183-1191.

VASWANI A, SHAZEER N, PARMAR N, et al . Attention is all you need[C ] //Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: MIT Press, 2017: 6000-6010.

李小帅. V2X通信无线资源管理关键技术研究[D]. 黑龙江: 哈尔滨工业大学, 2022.

LIANG L, XIE S, LI G Y, et al . Graph-based resource sharing in vehicular communication[J ] . IEEE Transactions on Wireless Communications, 2018, 17(7): 4579-4592.

SUNEHAG P, LEVER G, GRUSLYS A, et al . Value-decomposition networks for cooperative multi-agent learning based on team reward[C ] //Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems. Stockholm: Springer, 2018: 2085-2087.

JI Y , WANG Y, ZHAO H, et al . Multi-agent reinforcement learning resources allocation method using dueling double deep Q-network in vehicular networks[J ] . IEEE Transactions on Vehicular Technology, 2023, 72(10): 13447-13460.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度强化学习的对话策略网络

基于强化学习的注塑工艺参数自动调优方法及应用

基于技能发现的元强化学习

基于自注意力和三维卷积的心脏多类分割方法

基于路况参数的混合动力机车能量管理策略研究