Skip to content

A Method for the Solution of Certain Non-Linear Problems in Least Squares

Authors: Kenneth Levenberg

Published: 1944 (Journal Paper)

Source: Quarterly of Applied Mathematics

Algorithm: Levenberg-Marquardt

DOI: 10.1090/qam/10666

Summary

Levenberg introduces the method of damped least squares for nonlinear least-squares problems, motivated by the failure of the usual linearized Gauss-Newton normal equations when the proposed parameter increments are large enough to invalidate the first-order approximation. The paper modifies the linearized least-squares objective by also penalizing the squared parameter increments, yielding damped normal equations that preserve the symmetry of the standard normal equations while adding positive damping terms to the principal diagonal. Levenberg shows that, away from a stationary point, a damping value can be chosen so that the true sum of squared residuals decreases, and suggests choosing or refining the damping parameter by trial values or graphical search. Historically, this is the foundational Levenberg half of the later Levenberg-Marquardt method: it frames damping as a way to prevent overshooting without second derivatives and gives a geometric interpretation as restricting distance in parameter space.

Abstract

Primary

Paper ams.org

Standard

DOI 10.1090/qam/10666

Alternate

jstor.org jstor.org

Tags

  • Numerical optimization

  • Levenberg-Marquardt

  • Damped least squares

  • Nonlinear least squares

  • Gauss-Newton method

  • Trust-region methods

  • Parameter estimation

  • Optimization history