An Improved Double Deep Q Network for Multi-user Dynamic Spectrum Access

He Yi-shan; Wang Yong-hua; Wan Pin; Wang Lei; Wu Wen-tao

doi:10.12052/gdutxb.220159

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

An Improved Double Deep Q Network for Multi-user Dynamic Spectrum Access

更新时间：2026-05-27

- An Improved Double Deep Q Network for Multi-user Dynamic Spectrum Access
- Vol. 40, Issue 4, Pages: 85-93(2023)
- 作者机构：
  
  广东工业大学自动化学院,广东,广州,510006
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.220159
  CLC： TN929.5
- Online First：25 July 2023，
  
  Published：2023
- 稿件说明：
移动端阅览
何一汕, 王永华, 万频, 王磊, 伍文韬. 面向多用户动态频谱接入的改进双深度Q网络方法研究[J]. 广东工业大学学报, 2023, 40(4): 85-93.

He Yi-shan, Wang Yong-hua, Wan Pin, et al. An Improved Double Deep Q Network for Multi-user Dynamic Spectrum Access[J]. 2023, 40(4): 85-93.
何一汕, 王永华, 万频, 王磊, 伍文韬. 面向多用户动态频谱接入的改进双深度Q网络方法研究[J]. 广东工业大学学报, 2023, 40(4): 85-93. DOI： 10.12052/gdutxb.220159.

He Yi-shan, Wang Yong-hua, Wan Pin, et al. An Improved Double Deep Q Network for Multi-user Dynamic Spectrum Access[J]. 2023, 40(4): 85-93. DOI： 10.12052/gdutxb.220159.

摘要

随着移动通信技术的飞速发展，有限的频谱利用资源与大量频谱通信需求之间的矛盾也日益加剧，需要新的智能方法来提高频谱利用率。本文提出了一种基于分布式优先经验池结合双深度Q网络的多用户动态频谱接入方法。通过该方法，次用户可以在动态变化的认知无线网络环境下根据自己感知信息来不断地学习，选择空闲信道完成频谱接入任务来提高频谱利用率。该方法采用分布式强化学习框架，将每个次用户视为一个智能体，各个智能体采用标准单智能体强化学习方法进行学习以降低底层计算开销。另外，该方法在神经网络训练的基础上加入优先级采样，优化了神经网络的训练效率以帮助次用户选择出最优策略。仿真实验结果表明该方法能提高接入信道时的成功率、降低碰撞率和提升通信速率。

Abstract

With the rapid development of mobile communication technology

the contradiction between the limited spectrum utilization resources and the demand of a lot of spectrum communication is increasingly aggravated. New intelligent methods are needed to improve the utilization rate of spectrum. A multi-user dynamic spectrum access method based on distributed priority experience pool and double deep Q network is proposed. This method can help the secondary users to continuously learn according to their perceived environment information in the dynamic environment

and choose the idle channel to complete the spectrum access task for improving the spectrum utilization rate. In this method

a distributed reinforcement learning framework is adopted

and each secondary user is regarded as an agent. Each agent learns by using standard single-agent reinforcement learning method to reduce the underlying computing overhead. In addition

the method adds priority sampling on the basis of neural network training

and then optimizes the training efficiency of neural network to help sub-users choose the optimal strategy. The simulation results show that this method can improve the success rate

reduce the collision rate and improve the communication rate.

关键词

Keywords

references

SU H, ZHANG X. Cross-layer based opportunistic MAC protocols for QoS provisionings over Cognitive radio wireless networks [J]. IEEE Journal on Selected Areas in Communications, 2008, 26(1): 118-129.

WANG J, HUANG Y, JIANG H. Improved algorithm of spectrum allocation based on graph coloring model in cognitive radio[C]//2009 WRI International Conference on Communications and Mobile Computing. Kunming: IEEE, 2009: 353-357.

GAO L, DUAN L, HUANG J. Two-sided matching based cooperative spectrum sharing [J]. IEEE Transactions on Mobile Computing, 2017, 16(2): 538-551.

LIU X H, MA X R, WANG D W. Simulation modeling of cognitive radio spectrum allocation based on graph theory model [J]. Computer Telecommunication, 2021(3): 16-20.

LIU X, SUN C, ZHOU M, et al . Reinforcement learning based dynamic spectrum access in cognitive internet of vehicles [J ] . China Communications, 2021, 18(7): 58-68.

ZHENG S Y, CUI M, ZHANG G C. Reinforcement learning-based online trajectory optimization for secure UAV communications [J]. Journal of Guangdong University of Technology, 2021, 38(4): 59-64.

XU F, YANG F, BAO S, et al . DQN inspired joint computing and caching resource allocation approach for software defined information-centric Internet of Things network [J ] . IEEE Access, 2019, 7: 61987-61996.

CHEN Y, LI Y, XU D, et al . DQN-based power control for IoT transmission against jamming[C ] //2018 IEEE 87th Vehicular Technology Conference (VTC Spring) . Portugal: IEEE, 2018: 1-5.

SUN Y, PENG M, POOR H V. A distributed approach to improving spectral efficiency in uplink device-to-device-enabled cloud radio access networks [J]. IEEE Transactions on Communications, 2018, 66(12): 6511-6526.

KAI C H, MENG X W, MEI L S, et al. Deep reinforcement learning based user association and resource allocation for d2d-enabled wireless networks[C ] //2021 IEEE/CIC International Conference on Communications in China (ICCC) . Xiamen: IEEE, 2021: 1172-1177.

ASMUTH J, LI L, LITTMAN M L, et al . A bayesian sampling approach to exploration in reinforcement learning [J ] . Eprint Arxiv, 2009, 58(7): 1805-1810.

ZHANG R B, ZHONG Y, GU G C. A new accelerating algorithm for multi-agent reinforcement learning [J]. Journal of Harbin Institute of Technology, 2005, 12(1): 48-51.

VIRIYASITAVAT W, BOBAN M, TSAI H M, et al . Vehicular communications: survey and challenges of channel and propagation models [J ] . IEEE Vehicular Technology Magazine, 2015, 10(2): 55-66.

MEINIL J, KYSTI P, JMS T, et al . WINNER II channel models[M ] . New Jersey: John Wiley Sons, Ltd, 2009.

SUTTON R S, BARTO A G. Reinforcement learning: an introduction[M]. Cambridge: MIT Press, 2018.

傅波. 基于交替跟踪的分布式多智能体合作学习算法研究[D]. 长沙: 中南大学, 2014.

GUO H Q, CHEN F. A deep Q network method for dynamic arterial coordinated control [J]. Cyber Security and Data Governance, 2020, 39(6): 1-6.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Segmentation of Left Ventricular Endocardium Using Direction-constrained Reinforcement Learning

An AGV Path Planning Method for Discrete Manufacturing Smart Factory

A Research on a Training Model to Improve the Development Efficiency of Robot Reinforcement Learning

Trajectory Tracking and Dynamic Obstacle Avoidance of Mobile Robot Based on Deep Reinforcement Learning

Ground Deformation Monitoring and Influencing Factors Analysis in Fuzhou City Based on PSI Technology

Related Author

Zeng An

Pang Yao-xing

Pan Dan

Zhao Jing-liang

Guo Xin-de

Chris Hong-qiang Ding

Ye Wei-jie

Gao Jun-li

Related Institution

School of Computer Science and Technology, Guangdong University of Technology

School of Electronics and Information Engineering, Guangdong University of Technology and Education

Key Laboratory of Intelligent Information Processing and System Integration of IoT, Ministry of Education

The Chinese University of Hong Kong

School of Computers, Guangdong University of Technology

AI问答

⁰