Deep Reinforcement Learning-based Spectrum Allocation in Vehicular Networks

Lin Zhicong; Wang Yonghua; Wan Pin

doi:10.12052/gdutxb.240111

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Deep Reinforcement Learning-based Spectrum Allocation in Vehicular Networks

更新时间：2026-05-27

- Deep Reinforcement Learning-based Spectrum Allocation in Vehicular Networks
- Vol. 42, Issue 3, Pages: 101-110(2025)
- 作者机构：
  
  1. 广东工业大学自动化学院,广东,广州,510006
  2. 广州理工学院智能制造与电气工程学院,广东,广州,510540
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.240111
  CLC： TN929.5
- Published：2025
- 稿件说明：
移动端阅览
林志聪, 王永华, 万频. 基于深度强化学习的车联网频谱分配研究[J]. 广东工业大学学报, 2025, 42(3): 101–110.

Lin Zhicong, Wang Yonghua, Wan Pin. Deep Reinforcement Learning-based Spectrum Allocation in Vehicular Networks[J]. 2025, 42(3): 101-110.
林志聪, 王永华, 万频. 基于深度强化学习的车联网频谱分配研究[J]. 广东工业大学学报, 2025, 42(3): 101–110. DOI： 10.12052/gdutxb.240111

Lin Zhicong, Wang Yonghua, Wan Pin. Deep Reinforcement Learning-based Spectrum Allocation in Vehicular Networks[J]. 2025, 42(3): 101-110. DOI： 10.12052/gdutxb.240111.

摘要

为了缓解车联网中车辆用户数持续增长与频谱资源有限之间的矛盾，提出一种基于单调值函数分解结合自注意力机制的多智能体深度强化学习算法，旨在通过学习优化车联网中车对车链路(Vehicle-to-Vehicle，V2V) 信道选择和功率控制的联合策略，实现车对基础设施链路(Vehicle-to-Infrastructure，V2I) 总信道容量最大化，同时保证V2V链路传输时延和可靠性要求。针对车联网环境动态变化导致V2V链路无法实时收集完整信道状态信息的问题，为每条V2V链路构建深度循环Q网络，能使每条V2V链路基于自身局部观测信息自主决策。为了确保每条V2V链路局部策略的优化方向与全局利益最大化保持一致，设计了具有单调性约束的全局混合网络指引训练优化。此外，通过构建基于自注意力机制的信息交互模块，进一步优化V2V链路间的协作。仿真结果表明，与基线算法相比，本文所提出的算法在V2I链路总信道容量上提高了1.44~8.24个百分点，在V2V链路传输时延上缩短了1.93~15.04个百分点，能有效指导V2V链路根据环境的变化及时调整信道选择和功率控制的决策，从而保证了链路的通信质量。

Abstract

A dynamic spectrum allocation scheme is proposed to address the growing number of vehicles and limited spectrum resources in vehicular networks. It integrates the self-attention mechanism and monotonic value function factorisation into deep multi-agent reinforcement learning to optimize channel selection and power levels for vehicle-to-vehicle (V2V) links. The global objective is to maximize the sum throughput of vehicle-to-infrastructure (V2I) links

while meeting the latency and reliability constraints of V2V links. To handle incomplete real-time channel state information due to dynamic environment

a deep recurrent Q-network is deployed for each V2V link

enabling autonomous decision-making based on local observations. To align the local strategy optimization of each V2V link with the global objective

a global mixing network with monotonicity constraints is designed to guide the algorithm training. Additionally

an information interaction model based on the self-attention mechanism further optimizes collaboration between V2V links. Compared with the baselines

the proposed algorithm increases the sum throughput of V2I links by 1.44~8.24 percentage point and reduces the transmission delay of V2V links by 1.93~15.04 percentage point . These results confirm its effectiveness in optimizing channel selection and power levels for improved communication quality.

关键词

Keywords

references

NOOR-A-RAHIM M, LIU Z, LEE H, et al . A survey on resource allocation in vehicular networks[J ] . IEEE Transactions on Intelligent Transportation Systems, 2022, 23(2): 701-721.

DENG Y K, ZHANG L, LI J. Overview of research on privacy protection of Internet of Vehicles[J]. Application Research of Computers, 2022, 39(10): 2891-2906.

SUN W, STRÖM E G, BRÄNNSTRÖM F, et al . Radio resource management for D2D-based V2V communication[J ] . IEEE Transactions on Vehicular Technology, 2016, 65(8): 6636-6650.

WEI Q, SUN W, BAI B, et al . Resource allocation for V2X communications: a local search based 3D matching approach[C ] //2017 IEEE International Conference on Communications. Paris: IEEE, 2017: 1-6.

MEI J, ZHENG K, ZHAO L, et al . A latency and reliability guaranteed resource allocation scheme for LTE V2V communication systems[J ] . IEEE Transactions on Wireless Communications, 2018, 17(6): 3850-3860.

LIANG L, LI G Y, XU W. Resource allocation for D2D-enabled vehicular communications[J]. IEEE Transactions on Communications, 2017, 65(7): 3186-3197.

LI T, ZHU K, LUONG N C, et al . Applications of multi-agent reinforcement learning in future Internet: a comprehensive survey[J ] . IEEE Communications Surveys Tutorials, 2022, 24(2): 1240-1279.

LIANG L, YE H, LI G Y. Spectrum sharing in vehicular networks based on multi-agent reinforcement learning[J]. IEEE Journal on Selected Areas in Communications, 2019, 37(10): 2282-2292.

HE Y S, WANG Y H, WAN P, et al . An improved double deep Q network for multi-user dynamic spectrum access[J ] . Journal of Guangdong University of Technology, 2023, 40(4): 85-93.

XIANG P, SHAN H, WANG M, et al . Multi-agent RL enables decentralized spectrum access in vehicular networks[J ] . IEEE Transactions on Vehicular Technology, 2021, 70(10): 10750-10762.

MAFUTA A D, MAHARAJ B T J, ALFA A S. Decentralized resource allocation-based multiagent deep learning in vehicular network[J]. IEEE Systems Journal, 2023, 17(1): 87-98.

WANG L, WANG Y H, HE Y S, et al . A channel allocation method for internet of vehicles based on GRU and LSTM hybrid model[J ] . Telecommunication Engineering, 2024, 64(2): 273-280.

WANG W N, SU J, CHEN Y, et al . Multi-agent reinforcement learning enabled spectrum sharing for vehicular networks[J ] . Acta Electronica Sinica, 2024, 52(5): 1690-1699.

FANG W W, WANG Y P, ZHANG H, et al . Optimized communication resource allocation in vehicular networks based on multi-agent deep reinforcement learning[J ] . Journal of Beijing Jiaotong University, 2022, 46(2): 64-72.

LI L, LI W, CHEN X, et al . Learning based multi-channel access in vehicular networks with illegal operators and users[J ] . IEEE Transactions on Vehicular Technology, 2023, 72(3): 4097-4102.

RASHID T, SAMVELYAN M, DE WITT C S, et al . Monotonic value function fact orisation for deep multi-agent reinforcement learning[J ] . Journal of Machine Learning Research, 2020, 21(178): 7234-7284.

LUO S, LI Y, LI J, et al . S2RL: do we really need to perceive all states in deep multi-agent reinforcement learning?[C ] //Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Washington: ACM, 2022: 1183-1191.

VASWANI A, SHAZEER N, PARMAR N, et al . Attention is all you need[C ] //Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: MIT Press, 2017: 6000-6010.

李小帅. V2X通信无线资源管理关键技术研究[D]. 黑龙江: 哈尔滨工业大学, 2022.

LIANG L, XIE S, LI G Y, et al . Graph-based resource sharing in vehicular communication[J ] . IEEE Transactions on Wireless Communications, 2018, 17(7): 4579-4592.

SUNEHAG P, LEVER G, GRUSLYS A, et al . Value-decomposition networks for cooperative multi-agent learning based on team reward[C ] //Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems. Stockholm: Springer, 2018: 2085-2087.

JI Y , WANG Y, ZHAO H, et al . Multi-agent reinforcement learning resources allocation method using dueling double deep Q-network in vehicular networks[J ] . IEEE Transactions on Vehicular Technology, 2023, 72(10): 13447-13460.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Deep Reinforcement Learning Based Dialogue Policy Network

Reinforcement Learning-based Automatic Tuning Method and Application of Injection Molding Process Parameters

Meta Reinforcement Learning Based on Skill Discovery

Cardiac Multiclass Segmentation Method Based on Self-attention and 3D Convolution

An Optimal Operation Strategy for Community Microgrid Alliances Considering Power Exchange under Multi-package Shared Energy Storage Model

Related Author

Xu Boyan

Hao Zhifeng

Zou Zhibin

Li Si

Liao Haolin

Ren Zhigang

Lin Jianghao

Wan Huilong

Related Institution

School of Computer Science and Technology, Guangdong University of Technology

College of Engineering, Shantou University

College of Science, Shantou University

School of Electronics and Information Technology, Guangdong Technical Normal University

Department of Cardiovascular Surgery, Guangdong Provincial People’s Hospital (Guangdong Academy of Medical Sciences)

AI问答

⁰