On Automatic Differentiation¶
Authors: Andreas Griewank
Published: 1989 (Technical Report)
Source: Center for Research on Parallel Computation Technical Report
Algorithm: Automatic differentiation
Summary¶
Surveys automatic differentiation as a systematic way to compute exact derivatives of programs by propagating derivative information through elementary operations. The report helped frame AD as a practical computational tool for optimization, sensitivity analysis, and scientific computing.
Abstract¶
In comparison to symbolic differentiation and numerical differencing, the chain rule based technique of automatic differentiation is shown to evaluate partial derivatives accurately and cheaply. In particular it is demonstrated that the reverse mode of automatic differentiation yields any gradient vector at no more than five times the cost of evaluating the underlying scalar function. After developing the basic mathematics we describe several software implementations and briefly discuss the ramifications for optimization. Key words: gradient evaluation, automatic differentiation, symbolic differentiation, reverse accumulation, analytic derivatives.
Links¶
Primary
Alternate
Tags¶
-
Automatic differentiation
-
Algorithmic differentiation
-
Derivatives
-
Scientific computing
-
Optimization
-
Sensitivity analysis