Policy Iteration for Linear Quadratic Games with Stochastic Parameters
Abstract¶
Robustness is a key challenge in the integration of learning and control. In machine learning and robotics, two common approaches to promote robustness are adversarial training and domain randomization. Both of these approaches have analogs in control theory: adversarial training relates to H∞ control and dynamic game theory, while domain randomization relates to theory for systems with stochastic model parameters. We propose a stochastic dynamic game framework that integrates both of these complementary approaches to modeling uncertainty and promoting robustness. We describe policy iteration algorithms in both model-based and model-free settings to compute equilibrium strategies and value functions. We present numerical experiments that illustrate their effectiveness and the value of combining uncertainty representations in our integrated framework. We also provide an open-source implementation of the algorithms to facilitate their wider use.
Summary¶
Develops a polcy iteration algorithm for a state-feedback (fully observed) problem setting with linear dynamics, multiplicative noise (stochastic parameters), and adversarial disturbances. Nice compact notation for rather complicated Riccati equations, useful as a reference.