Modeling and Multivariate Methods - SAS

Chapter 9 Performing Nonlinear Regression 273
Statistical Details
The nonlinear minimization formula works by taking the first two derivatives of
model, and forming the gradient and an approximate Hessian as follows:
n
L
=
ρ( f( β)
)
with respect to the
i = 1
n
∂L
-------
∂
--------------------
ρ( f( β)
) ∂f
=
∂β -------
j ∂f ∂β
i = 1
j
2 n
∂ L ∂ 2 2
---------------------------
ρ( f ( β)
)
∂β j
∂β k
( ∂f ) 2 -------
∂f
--------
∂f ∂
∂β j ∂β ---------------------- ρ( f ( β)
) ∂ f
=
+
k ∂f ∂ βk ∂ β j
i = 1
If f (•)
is linear in the parameters, the second term in the last equation is zero. If not, you can still hope that
its sum is small relative to the first term, and use
2 n
∂ L ∂ 2 -------------------------
ρ( f ( β)
) ∂f
-------
∂f
≅
∂β j
∂β
--------
k ( ∂f ) 2 ∂β j
∂β
i = 1
k
The second term is probably small if ρ is the squared residual because the sum of residuals is small. The
term is zero if there is an intercept term. For least squares, this is the term that distinguishes Gauss-Newton
from Newton-Raphson. In JMP, the second term is calculated only if the Second Derivative option is
checked.
Note: The standard errors, confidence intervals, and hypothesis tests are correct only if least squares
estimation is done, or if maximum likelihood estimation is used with a proper negative log likelihood.
ρ (•)
Notes Concerning Derivatives
The nonlinear platform takes symbolic derivatives for formulas with most common operations. This section
shows what type of derivative expressions result.
If you open the Negative Exponential.jmp nonlinear sample data example, the actual formula looks
something like this:
Parameter({b0=0.5, b1=0.5,}b0*(1-Exp(-b1*X)))
The Parameter block in the formula is hidden if you use the formula editor. That is how it is stored in the
column and how it appears in the Nonlinear Launch dialog. Two parameters named b0 and b1 are given
initial values and used in the formula to be fit.
The Nonlinear platform makes a separate copy of the formula, and edits it to extract the parameters from
the expression. Then it maps the references to them to the place where they are estimated. Nonlinear takes
the analytic derivatives of the prediction formula with respect to the parameters. If you use the Show
Derivatives command, you get the resulting formulas listed in the log, like this: