GAMS — The Solver Manuals - Available Software

CONOPT 129
Although GAMS has a SIGN function, it cannot be used in a model because of its discontinuous nature. Instead,
the pressure loss can be modeled with the equation dH =E= const * Q * ABS(Q)**(P-1), where the sign of the
Q-term takes care of the sign of dH, and the ABS function guaranties that the real power ** is applied to a
non-negative number. Although the expression involves the ABS function, the derivatives are smooth as long as
P is greater than 1. The derivative with respect to Q is const * (P-1) * ABS(Q)**(P-1) for Q > 0 and -const *
(P-1) * ABS(Q)**(P-1) for Q < 0. The limit for Q going to zero from both right and left is 0, so the derivative
is smooth in the critical point Q = 0 and the overall model is therefore smooth.
Another example of a smooth expression is the following terribly looking Sigmoid expression:
Sigmoid(x) = exp( min(x,0) ) / (1+exp(-abs(x)))
The standard definition of the sigmoid function is
Sigmoid(x) = exp(x) / ( 1+exp(x) )
This definition is well behaved for negative and small positive x, but it not well behaved for large positive x since
exp overflows. The alternative definition:
Sigmoid(x) = 1 / ( 1+exp(-x) )
is well behaved for positive and slightly negative x, but it overflows for very negative x. Ideally, we would like
to select the first expression when x is negative and the second when x is positive, i.e.
Sigmoid(x) = (exp(x)/(1+exp(x)))$(x lt 0) + (1/(1+exp(-x)))$(x gt 0)
but a $ -control that depends on an endogenous variable is illegal. The first expression above solves this problem.
When x is negative, the nominator becomes exp(x) and the denominator becomes 1+exp(x). And when x is
positive, the nominator becomes exp(0) = 1 and the denominator becomes 1+exp(-x). Since the two expressions
are mathematically identical, the combined expression is of course smooth, and the exp function is never evaluated
for a positive argument.
Unfortunately, GAMS cannot recognize this and similar special cases so you must always solve models with
endogenous ABS, MAX, or MIN as DNLP models, even in the cases where the model is smooth.
7.5 Are NLP Models Always Smooth
NLP models are defined as models in which all operators and functions are smooth. The derivatives of composite
functions, that can be derived using the chain rule, will therefore in general be smooth. However, it is not always
the case. The following simple composite function is not smooth: y = SQRT( SQR(x) ). The composite function
is equivalent to y = ABS(x), one of the non-smooth DNLP functions.
What went wrong The chain rule for computing derivatives of a composite function assumes that all intermediate
expressions are well defined. However, the derivative of SQRT grows without bound when the argument
approaches zero, violating the assumption.
There are not many cases that can lead to non-smooth composite functions, and they are all related to the case
above: The real power, x**y, for 0 < y < 1 and x approaching zero. The SQRT function is a special case since
it is equivalent to x**y for y = 0.5.
If you have expressions involving a real power with an exponent between 0 and 1 or a SQRT, you should in
most cases add bounds to your variables to ensure that the derivative or any intermediate terms used in their
calculation become undefined. In the example above, SQRT( SQR(x) ), a bound on x is not possible since x should
be allowed to be both positive and negative. Instead, changing the expression to SQRT( SQR(x) + SQR(delta))
may lead to an appropriate smooth formulation.
Again, GAMS cannot recognize the potential danger in an expression involving a real power, and the presence
of a real power operator is not considered enough to flag a model as a DNLP model. During the solution process,