v2006.03.09 - Convex Optimization

D.1. DIRECTIONAL DERIVATIVE, TAYLOR SERIES 511D.1.7Correspondence of gradient to derivativeFrom the foregoing expressions for directional derivative, we derive arelationship between the gradient with respect to matrix X and the derivativewith respect to real variable t :D.1.7.1first-orderRemoving from (1385) the evaluation at t = 0 , D.3 we find an expression forthe directional derivative of g(X) in direction Y evaluated anywhere alonga line X+ t Y (parametrized by t) intersecting domg→Ydg(X+ t Y ) = d g(X+ t Y ) (1415)dtIn the general case g(X) : R K×L →R M×N , from (1378) and (1381) we findtr ( ∇ X g mn (X+ t Y ) T Y ) = d dt g mn(X+ t Y ) (1416)which is valid at t = 0, of course, when X ∈ domg . In the important caseof a real function g(X) : R K×L →R , from (1409) we have simplytr ( ∇ X g(X+ t Y ) T Y ) = d g(X+ t Y ) (1417)dtWhen, additionally, g(X) : R K →R has vector argument,∇ X g(X+ t Y ) T Y = d g(X+ t Y ) (1418)dtD.3 Justified by replacing X with X+ tY in (1378)-(1380); beginning,dg mn (X+ tY )| dX→Y= ∑ k,l∂g mn (X+ tY )Y kl∂X kl