The Method of Steepest Descent for Non-Linear Minimization Problems

Authors: Haskell B. Curry

Published: 1944 (Journal Paper)

Source: Quarterly of Applied Mathematics

Algorithm: Steepest Descent

DOI: 10.1090/qam/10667

Summary

Curry gives a practical numerical-computation exposition of Cauchy's method of steepest descent for minimizing nonlinear functions, motivated especially by nonlinear least-squares problems where more standard methods failed in Frankford Arsenal fire-control applications. The note describes the basic iteration of moving along the negative gradient and choosing a step by a one-dimensional search, explains the geometric right-angle behavior of successive exact line-search directions, and sketches conditions under which limit points of the resulting broken-line path are stationary points. Its comparison with Levenberg's contemporaneous method is useful historically: steepest descent avoids second derivatives and normal-equation solves, but the paper already recognizes its sensitivity to scaling and its uncertain practical advantage over more problem-specific least-squares methods.

Abstract

Links

Primary

Paper ams.org

Standard

DOI 10.1090/qam/10667

Alternate

jstor.org jstor.org

Tags

• Numerical optimization

• Steepest descent

• Gradient methods

• Nonlinear minimization

• Line search

• Nonlinear least squares

• First-order methods

• Optimization history