download searchable PDF of Circuit Design book - IEEE Global ...

144 Gradient Optimization
5.4.1. Summary ofFletcher-Reeves Strutegy. The unconstrained, nonlinear
programming problem is:
minQ(x)=F(X I ,X 2 , .•• ,X N ),
,
(5.76)
where x is a vector composed of N variables. The process is easily visualized
by inspection of Figure 5.3. This objective function and its gradient VQ must
be added to the BASIC language computer code provided. The gradient is:
aF aF aF )T
VQ=g(x)= (-a'-a""'-a- . (5.77)
XI x 2 x N
The gradient may be described analytically, if available, or found numerically
by 0.01% finite differences. The user should consider an "awful warning"
concerning excessive numerical noise, such as might occur if a named variable
might inadvertently be declared an integer as opposed to a floating-point
number. The resulting discontinuous behavior of the objective function will
have a disastrous effect on partial derivatives obtained by finite differences.
Almost all gradient optimizer programs will appear unacceptably sluggish
under these circumstances.
Given an initial starting vector, xo, a sequence of linear (line) searches,
(5.78)
is performed in a calculated direction s in the variable a;. Each search
terminates when a minimum is approximated so that the directi.onal derivative
is nearly zero:
(5.79)
The comprehensive procedure to accomplish reasonably accurate line searches
on arbitrary functions of a; was discussed in Section 5.3.
The first linear search direction is the negative gradient (steepest descent),
i.e., with {31 =0 in the direction formula
i= 1,2, ...,N. (5.80)
This describes a sequence of directions calculated after estimating each linear
search minimum. The new search direction is simply the negative gradient
plus a fraction of the just-used search direction. The fraction is:
{31 =0;
where the squared-norm notation
Ilg i ll 2
{3;= IIgH II2 ;
i=2,3, ...,N,
(5.81 )
(5.82)
defines an inner product. It is seen from (5.80) that certain curvature information
is accumulated for influencing the choice of subsequent search directions.
This strategy was developed on the assumption of quadratic functions where