Algorithmic Differentiation in Python with Application Examples

Computational Model and the Evaluation Trace
All computer programs are a sequence of elementary functions
φ l ∈ {+, −, ∗, /, sin, exp, . . . }
Symbolic dependency is resolved at each elementary function:
pushforward of numerical values v j≺l
Example: Evaluate Function f (3, 7):
f : R 2 → R
x ↦→ y = f (x) = sin(x 1 +cos(x 2 )∗x 1 )
Computational Graph:
Computational Trace:
1 Id
independent v −1 = x 1 = 3
independent v 0 = x 2 = 7
v 1 = φ 1 (v 0 ) = cos(v 0 )
2 cos
v 2 = φ 2 (v 1 , v −1 ) = v 1 v −1
v 3 = φ 3 (v −1 , v 2 ) = v −1 + v 2
3 __mul__
v 4 = φ 4 (v 3 ) = sin(v 3 )
4 __add__
5 sin
0 Id
Sebastian F. Walter, Humboldt-Universität zu Berlin Algorithmic () Differentiation in Python with Application Examples Wednesday, 10.07.2010 10 / 27