Convergence Condition of Value-iteration Based Adaptive Dynamic Programming

Liu Yi; Zhang Yun

doi:10.12052/gdutxb.170081

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

Convergence Condition of Value-iteration Based Adaptive Dynamic Programming

更新时间：2026-05-27

- Convergence Condition of Value-iteration Based Adaptive Dynamic Programming
- Vol. 34, Issue 5, Pages: 10-14(2017)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.170081
  CLC： TP273
- Online First：09 September 2017，
  
  Published：2017
- 稿件说明：
移动端阅览
刘毅, 章云. 基于值迭代的自适应动态规划的收敛条件[J]. 广东工业大学学报, 2017, 34(5): 10-14.

Liu Yi, Zhang Yun. Convergence Condition of Value-iteration Based Adaptive Dynamic Programming[J]. 2017, 34(5): 10-14.
刘毅, 章云. 基于值迭代的自适应动态规划的收敛条件[J]. 广东工业大学学报, 2017, 34(5): 10-14. DOI： 10.12052/gdutxb.170081.

Liu Yi, Zhang Yun. Convergence Condition of Value-iteration Based Adaptive Dynamic Programming[J]. 2017, 34(5): 10-14. DOI： 10.12052/gdutxb.170081.

摘要

研究了应用于离散时间非仿射非线性系统的基于值迭代的自适应动态规划的收敛条件，指出了迭代性能指标函数初始化为半正定函数可保证值迭代收敛到最优，并给出了证明.

Abstract

The convergence condition of value-iteration based adaptive dynamic programming which is applied to discrete time nonlinear non-affine system is studied. Convergence of value-iteration based adaptive dynamic programming is proven. The proof shows that value iteration will converge to the optimal when the initial iterative performance index function is a positive semi-definite function.

关键词

Keywords

references

张海舰, 成思源, 骆少明, 等. 基于动态规划法的B样条主动轮廓模型[J]. 广东工业大学学报, 2005, 22(4):26-30.ZHANG H J, CHENG S Y, LUO S M, et al. B-Spline active contour based on dynamic programming[J]. Journal of Guangdong University of Technology, 2005, 22(4):26-30.

BELLMAN R E. Dynamic Programming[M]. Princeton:Princeton University Press, 1957.

WERBOS P J. Advanced forecasting methods for global crisis warning and models of intelligence[J]. General Systems Yearbook, 1977, 22(6):25-38.

MILLER W T, SUTTON R S, WERBOS P J. A menu of designs for reinforcement learning over time, in neural networks for control[M]. Cambridge:MIT Press, 1991.

WERBOS P J. Approximate dynamic programming for real-time control and neural modeling, in handbook of intelligent control[M]. New York:Van Nostrand Reinhold, 1992.

MURRAY J J, COX C J, LENDARIS G G, et al. Adaptive dynamic programming[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 2002, 32(2):140-153.

PROKHOROV D V, WUNSCH D C. Adaptive critic designs[J]. IEEE Transactions on Neural Networks, 1997, 8(5):997-1007.

BERTSEKAS D P, TSITSIKLIS J N. Neuro-dynamic programming[M]. Belmont:Athena Scientific, 1996.

SUTTON R S, BARTO A G. Reinforcement learning:an introduction[M]. Cambridge:The MIT Press, 1998.

LEWIS F L, LIU D. Reinforcement learning and adaptive dynamic programming for feedback control[J]. IEEE Circuits & Systems Magazine, 2009, 9(3):32-50.

LEWIS F L, VRABIE D, VAMVOUDAKIS K G. Reinforcement learning and feedback control:using natural decision methods to design optimal adaptive controllers[J]. IEEE Control Systems, 2012, 32(6):76-105.

ABU-KHALAF M, LEWIS F L. Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach[J]. Automatica, 2005, 41(5):779-791.

LIU D, WEI Q. Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems[J]. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(3):621-634.

WANG F, JIN N, LIU D, WEI Q. Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound[J]. IEEE Transactions on Neural Networks, 2011, 22(1):24-36.

AL-TAMIMI A, LEWIS F L, ABU-KHALAF M. Discrete-time nonlinear HJB solution using approximate dynamic programming:convergence proof[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B:Cybernetics, 2008, 38(4):943-949.

ZHANG H, WEI Q, LUO Y. A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B:Cybernetics, 2008, 38(4):937-942.

WANG D, LIU D, WEI Q, et al. Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming[J]. Automatica, 2012, 48(8):1825-1832.

WEI Q, LIU D. A novel iterative θ-adaptive dynamic programming for discrete-time nonlinear systems[J]. IEEE Transactions on Automation Science and Engineering, 2014, 11(4):1176-1190.

WEI Q, LIU D, LIN H. Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems[J]. IEEE Transactions on Cybernetics, 2016, 46(3):840-853.

LIAO X, WANG L, YU P. Stability of dynamical systems[M]. Amsterdam:Elsevier Press, 2007.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A Cooperative Optimization Algorithm Based on Adaptive Dynamic Programming

A Data-Driven Prescribed Convergence Rate Design for Robust Tracking of Discrete-Time Systems

A Survey of Energy Management System Based on Adaptive Dynamic Programming

Multi-Scale Contrastive Embedding Enhanced Adaptive Intrusion Detection Model

A Resource Allocation Optimization Method for Multi-UAV Integrated Sensing, Computing, and Communication Towards Disaster Relief

Related Author

Liu Yi

Zhang Yun

Chen Ci

Xie Li-hua

Yuan Jun

Zhang Gui-dong

Li Zhong

Chen Zhe

Related Institution

School of Automation, Guangdong University of Technology

Guangdong Key Laboratory of IoT Information Technology

School of Electrical and Electronic Engineering, Nanyang Technological University

School of Mathematics and Computer Science, FernUniversität in Hagen, Hagen

Department of Energy Technology, Aalborg University, Aalborg

AI问答

⁰