08.11.2012 Views

multivariate production systems optimization - Stanford University

multivariate production systems optimization - Stanford University

multivariate production systems optimization - Stanford University

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

educed by an order of magnitude. On the other hand, if a line search indicates that<br />

interpolation (a decrease in the step-length ρ) is necessary, the Marquardt parameter is<br />

increased by an order of magnitude.<br />

If the Marquardt parameter is large, the method imitates the method of steepest<br />

descent, which converges slowly but has excellent stability properties. If the Marquardt<br />

parameter is small, the method behaves like Newton’s Method, which has good<br />

convergence characteristics but is prone to difficulties. Thus, in ill-conditioned regions the<br />

Marquardt (1963) modification will imitate the method of steepest descent whereas in wellbehaved<br />

areas the method will imitate Newton’s Method. The modification seemingly<br />

offers the best of both worlds.<br />

The caveat is that the Marquardt (1963) modification can become “stuck” in illconditioned<br />

regions if the Marquardt parameter is allowed to increase unfettered.<br />

Expressing Equation 6.26 as<br />

1<br />

μ H + I p = - 1 μ g (6.27)<br />

it is apparent that as μ → ∞, the step-length becomes infinitesimal. Therefore, when<br />

implementing the Marquardt (1963) modification, the Marquardt parameter should always<br />

be given an upper bound of some kind and a line search procedure should always be used.<br />

6.2.3 The Greenstadt Modification<br />

Greenstadt (1967) proposed a procedure to ensure that the Hessian matrix is positive<br />

definite. The Greenstadt (1967) modification involves two steps:<br />

1) All of the negative eigenvalues are replaced with their absolute value;<br />

2) Any small eigenvalues are replaced with infinity.<br />

The first step requires spectral decomposition to ascertain the eigenvalues. Although this<br />

procedure is very expensive computationally, it avoids the main pitfall of the Marquardt<br />

(1963) modification--blindly changing the eigenvalues in lockstep. By taking the absolute<br />

value of the negative eigenvalues, the Greenstadt (1967) modification merely reverses the<br />

direction of the ascending eigenvectors and forces them to take the same step length in a<br />

descent direction. The relative proportions of all the eigenvalues are maintained, the only<br />

difference being that they are all moving downhill.<br />

70

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!