Low-Complexity Learning of Linear Quadratic Regulators from Noisy Data

Authors

Claudio De Persis, Pietro Tesi

Year

2020

Journal Paper

Algorithm

Low-Complexity Data-Driven LQR

Source

Automatica

Identifiers

arXiv 2005.01082 DOI 10.1016/j.automatica.2021.109548

Abstract¶

This paper considers the Linear Quadratic Regulator problem for linear systems with unknown dynamics, a central problem in data-driven control and reinforcement learning. We propose a method that uses data to directly return a controller without estimating a model of the system. Sufficient conditions are given under which this method returns a stabilizing controller with guaranteed relative error when the data used to design the controller are affected by noise. This method has low complexity as it only requires a finite number of samples of the system response to a sufficiently exciting input, and can be efficiently implemented as a semi-definite program. Further, the method does not require assumptions on the noise statistics, and the relative error nicely scales with the noise magnitude.

Summary¶

Designs LQR controllers directly from noisy trajectory data with finite-sample, relative-error guarantees and an SDP implementation. It is a concise data-driven LQR counterpart to the noiseless LMI formulas, emphasizing robustness without a probabilistic noise model.