Differential Dynamic Programming and Newton's Method for Discrete Optimal Control Problems

Abstract¶

The purpose of this paper is to draw a detailed comparison between Newton's method, as applied to discrete-time, unconstrained optimal control problems, and the second-order method known as differential dynamic programming (DDP). The main outcomes of the comparison are: (i) DDP does not coincide with Newton's method, but (ii) the methods are close enough that they have the same convergence rate, namely, quadratic. The comparison also reveals some other facts of theoretical and computational interest. For example, the methods differ only in that Newton's method operates on a linear approximation of the state at a certain point at which DDP operates on the exact value. This would suggest that DDP ought to be more accurate, an anticipation borne out in our computational example. Also, the positive definiteness of the Hessian of the objective function is easy to check within the framework of DDP. This enables one to propose a modification of DDP, so that a descent direction is produced at each iteration, regardless of the Hessian.

Summary¶

Murray and Yakowitz compare differential dynamic programming with Newton's method for unconstrained discrete optimal-control problems. The paper clarifies that DDP is not identical to Newton's method but shares quadratic convergence, and it explains why DDP can be computationally attractive and more accurate in certain trajectory-optimization settings.

Links¶

Tags¶

Similar Papers¶