AD Tutorial at the TU Berlin

Algorithmic Differentiation ↔ Taylor Polynomial Arithmetic
Basic Observation: Let f : R N → R, then
d
dt f (x + e nt)
∣ = (∇ x f (x)) T · e n = ∂f ,
t=0
∂x n
where e n is the n’th cartesian basis vector.
complete gradient can be obtained by taking e 1 , e 2 , . . . , e N
for notational brevity: introduction of the seed matrix
(∇ x f (x)) T · S = d dt f (x + St) ∣
∣∣∣t=0
e.g. S =
( 1
1)
S ∈ R N×P doesn’t have to be square and not necessarily be the identity
matrix
“using” a seed matrix also leads computationally more efficient
algorithms (vectorized AD, e.g. adolc.hov forward)
Sebastian F. Walter, HU Berlin () Not So Short Tutorial OnAlgorithmic Differentiation Wednesday, 04.06.2010 6 / 39