download searchable PDF of Circuit Design book - IEEE Global ...

120 Gradient Optimization
The matrix algebra rules for differentiation applied to (5.6) produce
g(x)=b+Ax, (5.13)
where g(x) is often written as VF(x), called grad F. Whichever symbol is used,
g is a vector, like x; g is the gradient of F. From the central example in (5.7),
the vector b and matrix A can be identified so that (5.13) can be written as:
g(x) = [ ~~~]+[ -ig -m x . (5.14)
Some readers may be more comfortable differentiating (5.8) with respect to
both XI and x,:
gl=VIF=26xl-lOx,-60, (5.15)
g,=V,F=-lOxl+26x 2
-132. (5.16)
Appendix Program A5-2 evaluates the function value and the gradient elements
(derivatives) for this particular example. The reader is urged to use that
program in conjunction with Figure 5.5. Note that the gradient vectors are
always perpendicular to the level curves and point in the direction of steepest
ascent.
Finding the minimum of an n-variable quadratic function requires setting
each of the n gradient components equal to zero. This means setting (5.13)
equal to vector zero; this is equivalent to setting both (5.15) and (5.16) to zero
for that particular example. When (5.13) equals zero, then
Ax= -b. (5.17)
But the matrix inverse A-I is defined by the relationship
(5.18)
where the unit matrix U has all zero elements, except for l's on the main
diagonal. A property of the unit matrix is that when it multiplies a vector, the
result is just that vector. Multiplying both sides of (5.17) by A-I yields the x
values where g(x) = 0:
x=-A-Ib. (5.19)
Identifying b and A by comparing (5.13) with the example in (5.14), (5.19)
yields
XI] _ I [26 10] [ 60] _ [5 ]
[ x 2
- 576 10 26 132 - 7 '
(5.20)
where inverse matrix A-I was found by the three conceptual steps for finding
inverses: transpose, form the signed cofactors, and divide by the determinant
(see any book on matrix algebra, for instance, Noble, 1969). A glance at
Figure 5.5 shows that the center of the level curves is indeed at (5,7), the
vector from the origin to the center. The solution (5.19) is the translation of
the ellipses from the origin, as shown in Figure 5.7. The rotation of the ellipse
with respect to the major axes is discussed in the next section, as well as the
issue of whether (5.19) determines a true minimum function point.