GAMS — The Solver Manuals - Available Software

524 SNOPT
2.2 Constraints and slack variables
Problem (32.1) contains n variables in x. Let m be the number of components of f(x) and A L x combined.
The upper and lower bounds on those terms define the general constraints of the problem. SNOPT converts
the general constraints to equalities by introducing a set of slack variables s = (s 1 , s 2 , ..., s m ) T . For example,
the linear constraint 5 ≤ 2x 1 + 3x 2 ≤ +∞ is replaced by 2x 1 + 3x 2 − s 1 = 0 together with the bounded slack
5 ≤ s 1 ≤ +∞. Problem (32.1) can be written in the equivalent form
minimize
x,s
subject to
f 0 (x)
( )
f(x)
− s = 0,
A L x
l ≤
(
)
x
≤ u.
s
where a maximization problem is cast into a minimization by multiplying the objective function by −1.
The linear and nonlinear general constraints become equalities of the form f(x) − s N = 0 and A L x − s L = 0,
where s L and s N are known as the linear and nonlinear slacks.
2.3 Major iterations
The basic structure of SNOPT’s solution algorithm involves major and minor iterations. The major iterations
generate a sequence of iterates (x k ) that satisfy the linear constraints and converge to a point that satisfies the
first-order conditions for optimality. At each iterate {x k } a QP subproblem is used to generate a search direction
towards the next iterate {x k+1 }. The constraints of the subproblem are formed from the linear constraints
A L x − s L = 0 and the nonlinear constraint linearization
f(x k ) + f ′ (x k )(x − x k ) − s N = 0,
where f ′ (x k ) denotes the Jacobian: a matrix whose rows are the first derivatives of f(x) evaluated at x k . The
QP constraints therefore comprise the m linear constraints
f ′ (x k )x −s N = −f(x k ) + f ′ (x k )x k ,
A L x −s L = 0,
where x and s are bounded by l and u as before. If the m × n matrix A and m-vector b are defined as
( )
(
)
f ′ (x k )
−f(x k ) + f ′ (x k )x k
A =
and b =
,
A L 0
then the QP subproblem can be written as
QP k
minimize q(x, x k ) = gk T (x − x k ) + 1
x,s
2 (x − x k) T H k (x − x k )
( )
x
subject to Ax − s = b, l ≤ ≤ u,
s
(32.2)
where q(x, x k ) is a quadratic approximation to a modified Lagrangian function [6]. The matrix H k is a quasi-
Newton approximation to the Hessian of the Lagrangian. A BFGS update is applied after each major iteration.
If some of the variables enter the Lagrangian linearly the Hessian will have some zero rows and columns. If the
nonlinear variables appear first, then only the leading n 1 rows and columns of the Hessian need be approximated,
where n 1 is the number of nonlinear variables.
2.4 Minor iterations
Solving the QP subproblem is itself an iterative procedure. Here, the iterations of the QP solver SQOPT[8] form
the minor iterations of the SQP method.