What Color is the Non-Constant Part of Your Jacobian? - Stce.rwth ...

What Color is the Non-Constant Part of YourJacobian?Ebadollah VarnikUwe NaumannRWTH Aachen UniversitySIAM Workshop on CSC09Combinatorial Scientific Computing

ContentBackgroundNonzero Pattern PropagationNumerical Results

BackgroundConsider the vector function F : R n (=2) → R m (=2) with( ) [ ]y1 sin(x1 ) − x= F(x) =2y 2 x 1 + x2 ,2which is assumed to be d ≥ 1 times continuously differentiablein some neighborhood of x.The Single Assignment Code (SAC) of F is as follows.v 1 := x 1 ; v 2 = x 2 ;v 3 := sin(v 1 ); v 4 := v 3 − v 2 ; y 1 := v 4 ;v 5 := pow(v 2 ,2); v 6 := v 1 + v 5 ; y 2 := v 6 ;[ ]The Jacobian F ′ cos(x1 ) −1=, of F consist of variable1 2 · x 2(v) and constant nonzero (c) entries (sensitivities) respectivelycos(x 1 ), 2 · x 2 and −1, 1.

Jacobian Column Compression (SJ1)Vector Forward Mode (VFM) :B = F ′ · S ,with seed matrix S ∈ R n×p and compressed Jacobian B ∈ R m×p .Having the pattern of F ′ we getS1. S by coloring 1 G(F ′ ) using ColPack library;S2. B by applying VFM with the seed matrix S using ADOL-C;S3. F ′ by solving B = F ′ · S.Example :F ′ = B =[ ] cos(x1 ) −1·1 2 · x 2[ ] 1 00 1.1 A. H. Gebremedhin and A. Pothen What Color is Your Jacobian? Graph Coloring for Computing Derivatives.SIAM Rev. 47(4):629-705, 2005

Jacobian Column Compression (SJ2)Having F ′ c and the pattern of F ′ v we getS1. S by coloring the G(F ′ v );S2. B by applying VFM (B = F ′ · S);S3. F ′ v by solvingB = F ′ · S F′ =F v ′ +F′ c−→ B − F c ′ · S = F v ′ · S .Example :B =[ ] [ ] [ cos(x1 ) − 1 cos(x1 ) −1 1=·1 + 2 · x 2 1 2 · x 2 1].

Nonzero Pattern PropagationFor all SAC variables v k we define◮ The Index Domain of the d-TensorX d = {A := {i 1 ,...,i d } | i j ∈ X for j = 1,...,d}◮ Element of the d-Tensor at Position Ac d k,A = ∂d v k∂x A=◮ Nonzero Pattern of the d-Tensor∂ d v k∂x i1 ... ∂x idP k,d = {A ∈ X d | c d k,A ˆ=nz} = Pv k,d ∪ Pc k,d◮ Sub-Multiset PropertyA ∈ P k,d ∧ A ⊂ B ∈ P k,d+1 ⇒ A ∈ P v k,d◮ Constant Entries C k,d = {(A,c d k,A ) | A ∈ Pc k,d }.

Nonzero Pattern Propagation on Example (1)◮ For v 1 and v 2 and i ∈ {1,2}P i,1 = {{i}}; // ∂v i∂v i= 1→ P v i,1 = ∅;C i,1 = {({i},1.)};◮ v 3 = sin(v 1 )P 3,1 = {{1}}; // ∂v 3∂v 1= cos(v 1 )P 3,2 = {{1,1}}; // ∂2 v 3∂v 2 1= − sin(v 1 )P 3,3 = {{1,1,1}}; // ∂3 v 3∂v13 = − cos(v 1 )→ P v 3,1 = P 3,1; C 3,1 = ∅;→ P v 3,2 = P 3,2; C 3,2 = ∅;

Nonzero Pattern Propagation on Example (3)◮ v 5 = pow(v 2 ,2)◮ v 6 = v 1 + v 5P 5,1 = {{2}}; // ∂v 5∂v 2= 2 · v 2 ;P 5,2 = {{2,2}}; // ∂2 v 5∂v22 = 2→ P v 5,1 = {{2}}; C 5,1 = ∅;→ P v 5,2 = ∅; C 5,2 = {({2,2},2.)};P 6,1 = {{1}, {2}}; // ∂v 6∂v 1= 1; ∂v 6∂v 2= 2 · v 2 ;P 6,2 = {{2,2}}; // ∂2 v 6∂v22 = 2→ P v 4,1 = {{2}}; C 4,1 = {({1},1.)};→ P v 4,2 = ∅; C 4,2 = {({2,2},2.)};

Numerical Results (1)We consider a Simulated Moving Bed Process (SMB) 2 requiringlarge sparse Jacobians.SJ1SJ2Runtime (secs.)2.521.510.5 0.6 0.7 0.8 0.9 1N(#rows and columns) ·10 62 A. H. Gebremedhin and A. Pothen and A. Walther Exploiting Sparsity in Jacobian Computation via Coloringand Automatic Differentiation: A Case Study in a Simulated Moving Bed Process. C. H. Bischof et. al, eds.:Advances in Automatic Differentiation, Springer, 2008

Numerical Results (2)2S1 S2 S32S1 S2 S3Runtime (secs.)1.510.5Runtime (secs.)1.510.500.5 0.6 0.7 0.8 0.9 1N(#rows and columns) ·10 600.5 0.6 0.7 0.8 0.9 1N(#rows and columns) ·10 6(a)(b)Figure: Runtime of phases of SJ1 (a) and SJ2 (b) on SMB.

Conclusion and Outlook◮ Current state:◮ Implementation using operator overloading for Jacobiansand Hessians;◮ Work of progress:◮ Implementation using operator overloading of the aenericalgorithm as well as its◮ static formulation to make it a compile time activity◮ To get the current implementation send an email tovarnik@stce.rwth-aachen.de

End of the TalkTHANK YOU FOR YOURATTENTION !!ANY QUESTIONS ??

Hessian Compression (1)Second-Order Vector Adjoint Model (VFOR) :B = ȳ · F ′′ · S, with F : R n → R,F ′′ = F ′′ (x) ∈ R n×n , S ∈ R n×p , B ∈ R n×p and ȳ := 1 ∈ R.Example : The Hessian F ′′ ∈ R 2×2 ofy = cos(x 1 ) − 2 · x 1 · x 2 + exp(x 2 )[ ]−is F ′′ cos(x1 ) −2=.−2 exp(x 2 )Having F ′′c and pattern of F ′′v we getS1. S by coloring the G(F ′′v );S2. B by applying VFOR (B = ȳ · F ′′ · S) with ȳ = 1;S3. F ′′v by solving B = F ′′ · S.

Hessian Compression (2)Considering F ′′ = F v ′′ + F c ′′ with given F′′c and pattern of F′′v we getS1. S by coloring the G(F ′′v );S2. B by applying VFOR (B = ȳ · F ′′ · S) with ȳ = 1;S3. F ′′v by solvingB = F c ′′ · S F′ =F v ′′ +F c−→ ′′B − F c ′′ · S = F v ′′ · S .Example :B =[ ] [ ] [ cos(x1 ) − 2 cos(x1 ) −2 1=·−2 + exp(x 2 ) −2 exp(x 2 ) 1].