Modeling and Multivariate Methods - SAS

Chapter 10 Creating Neural Networks 285
Overview of Neural Networks
Fitting Options
Table 10.5 describes the model fitting options that you can specify.
Table 10.5 Fitting Options
Transform Covariates
Robust Fit
Penalty Method
Number of Tours
Transforms all continuous variables to near normality using either the
Johnson Su or Johnson Sb distribution. Transforming the continuous
variables helps to mitigate the negative effects of outliers or heavily skewed
distributions.
See the Save Transformed Covariates option in “Model Options” on
page 287.
Trains the model using least absolute deviations instead of least squares. This
option is useful if you want to minimize the impact of response outliers.
This option is available only for continuous responses.
Choose the penalty method. To mitigate the tendency neural networks have
to overfit data, the fitting process incorporates a penalty on the likelihood.
See “Penalty Method” on page 285.
Specify the number of times to restart the fitting process, with each iteration
using different random starting points for the parameter estimates. The
iteration with the best validation statistic is chosen as the final model.
Penalty Method
The penalty is λp( β i
), where λ is the penalty parameter, and p( ) is a function of the parameter estimates,
called the penalty function. Validation is used to find the optimal value of the penalty parameter.
Table 10.6 Descriptions of Penalty Methods
Method Penalty Function Description
Squared
Absolute
Weight Decay
β i
2
β i
β
2
i
---------------
1 + β
2
i
Use this method if you think that most of your X variables
are contributing to the predictive ability of the model.
Use either of these methods if you have a large number of X
variables, and you think that a few of them contribute more
than others to the predictive ability of the model.
NoPenalty none Does not use a penalty. You can use this option if you have a
large amount of data and you want the fitting process to go
quickly. However, this option can lead to models with lower
predictive performance than models that use a penalty.