Modeling and Multivariate Methods - SAS

274 Performing Nonlinear Regression Chapter 9
Statistical Details
Prediction Model:
b0 * First(T#1=1-(T#2=Exp(-b1*X)), T#3=-(-1*T#2*X))
The Derivative of Model with respect to the parameters is:
{T#1, T#3*b0}
The derivative facility works like this:
• In order to avoid calculating subexpressions repeatedly, the prediction model is threaded with
assignments to store the values of subexpressions that it needs for derivative calculations. The
assignments are made to names like T#1, T#2, and so on.
• When the prediction model needs additional subexpressions evaluated, it uses the First function,
which returns the value of the first argument expression, and also evaluates the other arguments. In this
case additional assignments are needed for derivatives.
• The derivative table itself is a list of expressions, one expression for each parameter to be fit. For
example, the derivative of the model with respect to b0 is T#1; its thread in the prediction model is
1–(Exp(-b1*X)). The derivative with respect to b1 is T#3*b0, which is –(–1*Exp(-b1*X)*X)*b0 if
you substitute in the assignments above. Although many optimizations are made, it does not always
combine the operations optimally. You can see this by the expression for T#3, which does not remove a
double negation.
If you ask for second derivatives, then you get a list of (m(m + 1))/2 second derivative expressions in a list,
where m is the number of parameters.
If you specify a loss function, then the formula editor takes derivatives with respect to parameters, if it has
any. And it takes first and second derivatives with respect to the model, if there is one.
If the derivative mechanism does not know how to take the analytic derivative of a function, then it takes
numerical derivatives, using the NumDeriv function. If this occurs, the platform shows the delta that it used
to evaluate the change in the function with respect to a delta change in the arguments. You might need to
experiment with different delta settings to obtain good numerical derivatives.
Tips
There are always many ways to represent a given model, and some ways behave much better than other
forms. Ratkowsky (1990) covers alternative forms in his text.
If you have repeated subexpressions that occur several places in a formula, then it is better to make an
assignment to a temporary variable. Then refer to it later in the formula. For example, one of the model
formulas above was this:
If(Y==0, Log(1/(1+Exp(model))), Log(1 - 1/(1 + Exp(model))));
This could be simplified by factoring out an expression and assigning it to a local variable:
temp=1/(1+Exp(model));
If(Y==0, Log(temp), Log(1-temp));
The derivative facility can track derivatives across assignments and conditionals.