Adaptive Constrained Optimal Control Design for Data-Based Nonlinear Discrete-Time Systems With Critic-Only Structure | |
Luo, Biao1; Liu, Derong2; Wu, Huai-Ning3 | |
刊名 | IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS |
2018-06-01 | |
卷号 | 29期号:6页码:2099-2111 |
关键词 | Adaptive Control Adaptive Dynamic Programming Constraints Critic-only Data-based Optimal Control Q-learning |
DOI | 10.1109/TNNLS.2017.2751018 |
文献子类 | Article |
英文摘要 | Reinforcement learning has proved to be a powerful tool to solve optimal control problems over the past few years. However, the data-based constrained optimal control problem of nonaffine nonlinear discrete-time systems has rarely been studied yet. To solve this problem, an adaptive optimal control approach is developed by using the value iteration-based Q-learning (VIQL) with the critic-only structure. Most of the existing constrained control methods require the use of a certain performance index and only suit for linear or affine nonlinear systems, which is unreasonable in practice. To overcome this problem, the system transformation is first introduced with the general performance index. Then, the constrained optimal control problem is converted to an unconstrained optimal control problem. By introducing the action-state value function, i.e., Q-function, the VIQL algorithm is proposed to learn the optimal Q-function of the data-based unconstrained optimal control problem. The convergence results of the VIQL algorithm are established with an easy-to-realize initial condition Q((0))(x, a) >= 0. To implement the VIQL algorithm, the critic-only structure is developed, where only one neural network is required to approximate the Q-function. The converged Q-function obtained from the critic-only VIQL method is employed to design the adaptive constrained optimal controller based on the gradient descent scheme. Finally, the effectiveness of the developed adaptive control method is tested on three examples with computer simulation. |
WOS关键词 | Optimal Tracking Control ; H-infinity Control ; Dynamic-programming Algorithm ; Linear-systems ; Unknown Dynamics ; Policy Iteration ; Neural-networks ; Control Scheme ; Equation ; Update |
WOS研究方向 | Computer Science ; Engineering |
语种 | 英语 |
WOS记录号 | WOS:000432398300005 |
资助机构 | National Natural Science Foundation of China(61503377 ; 61533017 ; 61625302 ; 61473011 ; U1501251) |
内容类型 | 期刊论文 |
源URL | [http://ir.ia.ac.cn/handle/173211/22045] |
专题 | 复杂系统管理与控制国家重点实验室_平行控制 |
作者单位 | 1.Chinese Acad Sci, State Key Lab Management & Control Complex Syst, Inst Automat, Beijing 100190, Peoples R China 2.Guangdong Univ Technol, Sch Automat, Guangzhou 510006, Guangdong, Peoples R China 3.Beihang Univ, Sci & Technol Aircraft Control Lab, Beijing 100191, Peoples R China |
推荐引用方式 GB/T 7714 | Luo, Biao,Liu, Derong,Wu, Huai-Ning. Adaptive Constrained Optimal Control Design for Data-Based Nonlinear Discrete-Time Systems With Critic-Only Structure[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2018,29(6):2099-2111. |
APA | Luo, Biao,Liu, Derong,&Wu, Huai-Ning.(2018).Adaptive Constrained Optimal Control Design for Data-Based Nonlinear Discrete-Time Systems With Critic-Only Structure.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,29(6),2099-2111. |
MLA | Luo, Biao,et al."Adaptive Constrained Optimal Control Design for Data-Based Nonlinear Discrete-Time Systems With Critic-Only Structure".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 29.6(2018):2099-2111. |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论