Path Planning of Opposite Q Learning Robot Based on Virtual Sub-Target in Unknown Environment

Wang Sheng-min; Lin Wei; Zeng Bi

doi:10.12052/gdutxb.180044

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Path Planning of Opposite Q Learning Robot Based on Virtual Sub-Target in Unknown Environment

更新时间：2026-05-27

- Path Planning of Opposite Q Learning Robot Based on Virtual Sub-Target in Unknown Environment
- Vol. 36, Issue 1, Pages: 51-56, 62(2019)
- 作者机构：
  
  广东工业大学　计算机学院,广东,广州,510006
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.180044
  CLC： TP242.6
- Online First：25 January 2019，
  
  Published：2019
- 稿件说明：
移动端阅览
汪盛民, 林伟, 曾碧. 未知环境下基于虚拟子目标的对立Q学习机器人路径规划[J]. 广东工业大学学报, 2019, 36(1): 51-56, 62.

Wang Sheng-min, Lin Wei, Zeng Bi. Path Planning of Opposite Q Learning Robot Based on Virtual Sub-Target in Unknown Environment[J]. 2019, 36(1): 51-56, 62.
汪盛民, 林伟, 曾碧. 未知环境下基于虚拟子目标的对立Q学习机器人路径规划[J]. 广东工业大学学报, 2019, 36(1): 51-56, 62. DOI： 10.12052/gdutxb.180044.

Wang Sheng-min, Lin Wei, Zeng Bi. Path Planning of Opposite Q Learning Robot Based on Virtual Sub-Target in Unknown Environment[J]. 2019, 36(1): 51-56, 62. DOI： 10.12052/gdutxb.180044.

摘要

针对Q学习算法在复杂的未知环境下

值更新速度慢

容易产生维数灾难等问题

提出了一种未知环境下基于虚拟子目标的对立Q学习机器人路径规划算法. 该算法根据移动机器人探索过的状态轨迹

建立了2个状态链分别记录状态−动作对和状态−反向动作对

并将每个单链当前状态的

值

依次反馈影响前一状态的

值

直到状态链的头端. 同时

在局部探测域内通过寻找最优虚拟子目标的方法解决了大规模环境下Q学习容易产生维数灾难的问题. 实验结果表明

在复杂的未知环境中

该算法可以有效地加快算法学习的收敛速度

提高学习效率

以较优的路径完成机器人导航任务.

Abstract

Aiming at the problem that in Q learning algorithm

value is slow in updating speed in complex unknown environment and the dimensionality disaster is easy to occur

a path planning algorithm based on virtual subtarget for Q learning robot in unknown environment is proposed. According to the state trajectory explored by the mobile robot

two state chains are established to record the state-action pair and the state-reverse action pair respectively. The

value of each single chain current state is fed back to the

value of the previous state in turn till it affects the head of a single chain. Meanwhile

the problem that

Q learning is prone to dimensionality disaster in large-scale environment is solved by finding the optimal virtual subtarget in the local detection domain. The experimental results show that the algorithm can effectively accelerate the convergence of the algorithm learning

improve the learning efficiency and complete the robot navigation task with a better path in the complex unknown environment.

关键词

Keywords

references

ZHAO X D, BAO F. Survey on cleaning robot path planning algorithm [J]. Journal of Mechanical Electrical Engineering, 2013, 30(11): 1440-1444.

LOPEZ A, PAREDES R, QUIROZ D, et al. R-obotman: A security robot for human-robot int-eraction[C]// International Conference on Adva-nced Robotics. Singapore Sands Expo: IEEE, 2017: 7-12.

ZHU D Q, YAN M Z. A survey of path planning techniques for mobile robots [J]. Control and Decision, 2010, 25(7): 961-967.

WANG Q Z, CHENG J Y, LI X L. Researchon robot path planning method in complex en-vironment [J]. Computer Simulation, 2017, 34(10): 296-300.

ZHANG H Y, LIN Z X, GUO T L. Robot obstacle avoidance path planning optimization control simulation [J]. Computer Simulation, 2017, 34(9): 325-330.

SALZMAN O, DAN H. Asymptotically near optimal RRT for fast, high-quality motion planning [J]. IEEE Transactions on Robotics, 2016, 32(3): 473-483.

LU Y P, ZHAO Y, MENG Y Q, et al . Path planning in narrow space by improved genetic algorithm [J ] . Application Research of Computers, 2015(2): 413-418.

HAN M, LIU J M, WU S M, et al . Path planning algorithm of mobile robot based on particle swarm optimization [J ] . Computer Application, 2017, 37(8): 2258-2263.

LAMINI C, FATHI Y, BENHLIMA S. H-MAS architecture and reinforcement learning method for autonomous robot path planning[C]//2017 Intelligent Systems and Computer Vision. Morocco, Fez: IEEE, 2017.

CETIN H, DURDU A. Path planning of mobile robots with Q-learning[C]// Signal Processing and Communications Applications Conference. India, Kerala: IEEE, 2014.

KONAR A, CHAKRABORTY I G, SINGH S J, et al . A deterministic Improved Q-learning for path planning of a mobile robot [J ] . IEEE Transactions on Systems Man Cybernetics Systems, 2013, 43(5): 1141-1153.

HWANG H J, VIET H H, CHUNG T C. Q ( λ ) based vector direction for path planning problem of autonomous mobile robots [J ] . Speech Communication, 2011, 17(3-4): 249-262.

PIETERS M, WIERING M A. Q-learning withexperience replay in a dynamic environment[C]//2016 IEEE Symposium Series on Computational Intelligence. Greece, Athens: IEEE, 2017.

LIANG J J, ZENG B, HE Y L. Research on path planning algorithm for cleaning robot based on improved potential field grid method [J]. Journal of Guangdong University of Technology, 2016, 33(4): 30-36.

LEI T, MING L. A robot exploration strategy based on Q-learning network[C]// IEEE Intern-ational Conference on Real-Time Computing a-nd Robotics. Okinawa: IEEE, 2016: 57-62.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Direct Sparse Visual Odometer Based on Enhanced Stereo-Camera Constraints

A Research on Docking Position Optimization Method of Mobile Robot for Grasping Task

Trajectory Tracking and Dynamic Obstacle Avoidance of Mobile Robot Based on Deep Reinforcement Learning

Energy Management Strategy of Hybrid Locomotive Based on Road Condition Parameters

SCA-YOLO: Real-time Target Detection Based on Deformable Convolution and Context-aware Attention

Related Author

Cai Xiaoming

Zhang Hui

Gu Longyi

Li Guanjian

Wang Qinruo

Ye Pei-chu

Li Dong

Zhang Yun

Related Institution

School of Automation, Guangdong University of Technology

AI问答

⁰