multivariate production systems optimization - Stanford University

IMSL (1987). For a broad overview of numerical optimization techniques, see Gill,
Murray, & Wright (1981). For a more extensive treatment of numerical optimization
applied to petroleum engineering problems, see Barua (1989).
6.1 Newton’s Method
Newton’s Method is one of the most common techniques used in nonlinear optimization. It
is the standard against which all other algorithms are measured. If Newton’s Method is
provided with a good initial estimate of the solution, quadratic convergence is achieved.
Newton’s Method achieves this rate of convergence by approximating the objective
function with a quadratic model. The quadratic model is chosen so that, at a given point,
its first and second derivatives are identical to the first and second derivatives of the
objective function. Therefore, at the given point, the objective function and the quadratic
model are identical in value, slope, and curvature. The quadratic model is solved for the
stationary point where its gradient goes to zero. If the quadratic model is a good
approximation of the objective function, then the stationary point of the quadratic model
should be near a stationary point of the objective function. The stationary point of the
quadratic model is taken as the new estimate of the objective function’s stationary point and
the process is repeated until some form of convergence criteria is satisfied.
Taylor’s theorem provides the mathematical basis for Newton’s Method. Taylor’s
theorem states that if a function and its derivatives are known at a single point, then
approximations to the function can be made at all points in the immediate neighborhood of
the point. The Taylor series expansion of a general function F about x gives a simple
approximation to the function F in the immediate neighborhood of x as
where x is the vector of variables,
F x+p = Fx + g T p + 1 2 p T H p + O ||p|| 3
x =
63
x1
x2
x3
xn
(6.4)
(6.5)