Adaptive Linear Quadratic Control Using Policy Iteration
Abstract¶
In this paper we present the stability and convergence results for dynamic programming-based reinforcement learning applied to linear quadratic regulation (LQR). The specific algorithm we analyze is based on Q-learning and it is proven to converge to an optimal controller provided that the underlying system is controllable and a particular signal vector is persistently excited. This is the first convergence result for DP-based reinforcement learning algorithms for a continuous problem.
Summary¶
Analyzes reinforcement-learning policy iteration for the continuous linear-quadratic regulator setting, proving convergence to the optimal controller under controllability and persistent excitation assumptions. The paper is an early bridge between adaptive control, dynamic programming, and model-free reinforcement learning for continuous-state control problems.