Reinforcement Learning-Based Online Trajectory Optimization for Secure UAV Communications

Zheng Si-yuan; Cui Miao; Zhang Guang-chi

doi:10.12052/gdutxb.200113

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Reinforcement Learning-Based Online Trajectory Optimization for Secure UAV Communications

更新时间：2026-05-27

- Reinforcement Learning-Based Online Trajectory Optimization for Secure UAV Communications
- Vol. 38, Issue 4, Pages: 59-64(2021)
- 作者机构：
  
  广东工业大学信息工程学院,广东,广州,510006
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.200113
  CLC： TN929.5
- Online First：10 July 2021，
  
  Published：2021
- 稿件说明：
移动端阅览
郑思远, 崔苗, 张广驰. 基于强化学习的无人机安全通信轨迹在线优化策略[J]. 广东工业大学学报, 2021, 38(4): 59-64.

Zheng Si-yuan, Cui Miao, Zhang Guang-chi. Reinforcement Learning-Based Online Trajectory Optimization for Secure UAV Communications[J]. 2021, 38(4): 59-64.
郑思远, 崔苗, 张广驰. 基于强化学习的无人机安全通信轨迹在线优化策略[J]. 广东工业大学学报, 2021, 38(4): 59-64. DOI： 10.12052/gdutxb.200113.

Zheng Si-yuan, Cui Miao, Zhang Guang-chi. Reinforcement Learning-Based Online Trajectory Optimization for Secure UAV Communications[J]. 2021, 38(4): 59-64. DOI： 10.12052/gdutxb.200113.

摘要

物理层安全是无线通信实现安全通信的一种有效手段

无人机基站在存在地面窃听者的情况下向地面合法用户传输机密信息时

已有研究一般采用离线方法对无人机基站的飞行轨迹进行优化

得到的轨迹是固定的

保密传输缺乏应对通信环境变化的能力。针对该问题

本文研究无人机飞行轨迹的在线优化策略

使机密信息被安全传输的同时实现通信平均保密率最大化。文中采用了更符合实际的视距/非视距混合信道模型对空地通信链路进行建模

提出一种基于Q-learning算法的保障无人机通信系统安全的飞行轨迹在线优化方法。仿真结果表明

与两种基准算法相比

本文提出的算法能够有效提高无人机通信的平均保密率。

Abstract

Physical-layer security is an effective technique to achieve secure communication in wireless communication. Physical-layer security in an unmanned aerial vehicle (UAV) communication system is investigated

where a UAV base station transmits confidential information to a legitimate receiver in the presence of a potential eavesdropper. The existing works usually assume that the flight environment of the UAV is stable

and the corresponding trajectory optimization methods are offline

which are not able to adjust the trajectory according to changing environment. To solve this issue

online trajectory optimization for the UAV base station is studied to ensure the information transmission security and maximize the communication secrecy rate. Different from the existing works

a more accurate hybrid light-of-sight/non-light-of-sight channel model is adopted for the air-to-ground communication links and an online trajectory optimization method proposed based on the Q-learning algorithm to ensure the physical-layer security of UAV communication. Numerical results show that the proposed algorithm significantly improves the secrecy rate of the UAV communication system

as compared with two benchmark schemes.

关键词

Keywords

references

ZENG Y, ZHANG R, LIM T J. Wireless communications with unmanned aerial vehicles — opportunities and challenges [J]. IEEE Communications Magazine, 2016, 54(5): 36-42.

LYU J, ZENG Y, ZHANG R. Cyclical multiple access in UAV-aided communications — a throughput-delay tradeoff [J]. IEEE Wireless Communications Letters, 2016, 5(6): 600-603.

SOTHEARA S, ASO K, AOMI N, et al. Effective data gathering and energy efficient communication protocol in wireless sensor networks employing UAV[C]//Wireless Communications Networking Conference. Istanbul: IEEE, 2014: 2342-2347.

ZHENG G, KRIKIDIS I, LI J, et al . Improving physical layer secrecy using full-duplex jamming receivers [J ] . IEEE Transactions on Signal Processing, 2013, 61(20): 4962-4974.

WYNER A D. The wire-tap channel [J]. Bell System Technical Journal, 1975, 54(8): 1355-1387.

SHIU Y S, CHANG S Y, WU H C, et al . Physical layer security in wireless networks — a tutorial [J ] . IEEE Wireless Communications, 2011, 18(2): 66-74.

TONG H Z, ZHANG G C, ZHOU X L, et al . Joint energy and spectrum allocation in multiple adjacent cells with energy harvesting base stations [J ] . Journal of Guangdong University of Technology, 2018, 35(4): 72-78.

GOEL S, NEGI R. Guaranteeing secrecy using artificial noise [J]. IEEE Transactions on Wireless Communications, 2008, 7(6): 2180-2189.

GOPALA P K, LAI L, EL GAMAL H. On the secrecy capacity of fading channels [J]. IEEE Transactions on Information Theory, 2008, 54(10): 4687-4698.

WANG H, YIN Q, XIA X. Distributed beamforming for physical layer security of two-way relay networks [J]. IEEE Transactions on Signal Processing, 2012, 60(7): 3532-3545.

ZHANG G, LI X, CUI M, et al . Signal and artificial noise beamforming for secure simultaneous wireless information and power transfer multiple-input-multiple-output relaying systems [J ] . IET Communications, 2016, 10(7): 796-804.

ZHANG G, WU Q, CUI M, et al . Securing UAV communications via joint trajectory and power control [J ] . IEEE Transactions on Wireless Communications, 2019, 18(2): 1376-1389.

ZHANG J, ZENG Y, ZHANG R. Spectrum and energy efficiency maximization in UAV-enabled mobile relaying[C]//International Conference on Communications. Paris: IEEE, 2017: 1-6.

HUANG Y, CUI M, ZHANG G, et al . Bandwidth, power and trajectory optimization for UAV base station networks with back haul and user QoS constraints [J ] . IEEE Access, 2020(8): 67625-67634.

ZHOU Y, YEOH P L, CHEN H, et al. Secrecy outage probability and jamming coverage of UAV-enabled friendly jammer[C]//2017 11th International Conference on Signal Processing and Communication Systems. Gold Coast: IEEE, 2017: 1-6.

LIU J, LIU Z, ZENG Y, et al . Cooperative jammer placement for physical layer security enhancement [J ] . IEEE Network the Magazine of Global Internetworking, 2016, 30(6): 56-61.

FAN J, CUI M, ZHANG G, et al . Throughput improvement for multi-hop UAV relaying [J ] . IEEE Access, 2019(7): 147732-147742.

ZHANG G, YAN H, ZENG Y, et al . Trajectory optimization and power allocation for multi-hop UAV relaying communications [J ] . IEEE Access, 2018(6): 48566-48576.

SUTTON R, BARTO A. Reinforcement learning — an introduction[M]. 2nd ed. Cambridge: MIT Press, 2018.

FENG Q, TAMEH E, NIX A R, et al. Modelling the likelihood of line-of-sight for air-to-ground radio propagation in urban environments[C]// Globecom2006. San Francisco: IEEE, 2007: 1-5.

CUI M, ZHANG G, WU Q, et al . Robust trajectory and transmit power design for secure UAV communications [J ] . IEEE Transactions on Vehicular Technology, 2018, 67(9): 9042-9046.

AL HOURANI A, KANDEEPAN S, LARDNER S. Optimal lap altitude for maximum coverage [J]. Wireless Communications Letters IEEE, 2014, 3(6): 569-572.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Deep Reinforcement Learning Based Dialogue Policy Network

Deep Reinforcement Learning-based Spectrum Allocation in Vehicular Networks

Reinforcement Learning-based Automatic Tuning Method and Application of Injection Molding Process Parameters

Meta Reinforcement Learning Based on Skill Discovery

Reinforcement Learning Model for Automatic Inspection Route Based on Multi-agent Attention Mechanism

Related Author

Xu Boyan

Hao Zhifeng

Zou Zhibin

Wan Pin

Wang Yonghua

Lin Zhicong

Ren Zhigang

Lin Jianghao

Related Institution

School of Computer Science and Technology, Guangdong University of Technology

College of Engineering, Shantou University

School of Automation, Guangdong University of Technology

School of Intelligent Manufacturing and Electrical Engineering, Guangzhou Institute of Science and Technology

College of Science, Shantou University

AI问答

⁰