Skip to content

Policy Iteration Using Q-functions: Linear Dynamics with Multiplicative Noise

Authors
Peter Coppens, Panagiotis Patrinos
Year
2022
Preprint
Algorithm
Q-functions
Source
arXiv
Identifiers
arXiv 2212.01192

Abstract

This paper presents a novel model-free and fully data-driven policy iteration scheme for quadratic regulation of linear dynamics with state- and input-multiplicative noise. The implementation is similar to the least-squares temporal difference scheme for Markov decision processes, estimating Q-functions by solving a least-squares problem with instrumental variables. The scheme is compared with a model-based system identification scheme and natural policy gradient through numerical experiments.

Knowledge Base

Map Tree Search

External

Document arXiv Abstract arXiv PDF arXiv HTML Google Scholar OpenAlex

Tags

Policy gradients Policy iteration System identification Q-functions Multiplicative noise Instrumental variable Markov decision process

Similar Papers