Modeling and Multivariate Methods - SAS

282 Creating Neural Networks Chapter 10
Overview of Neural Networks
Table 10.2 Description of the Model Launch Dialog (Continued)
Go
Fits the neural network model and shows the model reports.
After you click Go to fit a model, you can reopen the Model Launch Dialog and change the settings to fit
another model.
Validation Method
Neural networks are very flexible models and have a tendency to overfit data. When that happens, the
model predicts the fitted data very well, but predicts future observations poorly. To mitigate overfitting, the
Neural platform does the following:
• applies a penalty on the model parameters
• uses an independent data set to assess the predictive power of the model
Validation is the process of using part of a data set to estimate model parameters, and using the other part to
assess the predictive ability of the model.
• The training set is the part that estimates model parameters.
• The validation set is the part that estimates the optimal value of the penalty, and assesses or validates the
predictive ability of the model.
• The test set is a final, independent assessment of the model’s predictive ability. The test set is available
only when using a validation column. See Table 10.3.
The training, validation, and test sets are created by subsetting the original data into parts. Table 10.3
describes several methods for subsetting a data set.
Table 10.3 Validation Methods
Excluded Rows
Holdback
KFold
Uses row states to subset the data. Rows that are unexcluded are used as
the training set, and excluded rows are used as the validation set.
For more information about using row states and how to exclude rows,
see Using JMP.
Randomly divides the original data into the training and validation sets.
You can specify the proportion of the original data to use as the
validation set (holdback).
Divides the original data into K subsets. In turn, each of the K sets is
used to validate the model fit on the rest of the data, fitting a total of K
models. The model giving the best validation statistic is chosen as the
final model.
This method is best for small data sets, because it makes efficient use of
limited amounts of data.