Vectorizing the forward mode of ADOL-C on a GPU ... - Autodiff.org

. K. Kulshreshtha, A. Koniaeva 6 / 13 <strong>Vectorizing</strong> <strong>ADOL</strong>-C using CUDA Euro AD 10.06.2013Vector <strong>forward</strong> <strong>mode</strong>.Vector <strong>forward</strong> <strong>mode</strong>F : R n −→ R m , x ∈ R n , Ẋ ∈ R n×pẎ = F ′ (x)Ẋ ∈ R m×pḞ(x, Ẋ) := [F(x), F ′ (x)Ẋ]• Amortises <strong>the</strong> overhead by reusing intermediate values for variousdirections instead <strong>of</strong> recomputing• Serial: TIME(Ḟ)/TIME(F) ∈ [1 + p, 1 + 1.5p]• Problematic if p is large• Can easily be parallelised if derivatives required at a large number <strong>of</strong>evaluation points x• Can also be parallelised to propogate several direction simultaneously