Final Technical Report: - Southwest Fisheries Science Center - NOAA

When cetacean experts were shown geographic contour plots of the predictions from the
competing simple and complex mgcv gam models for each species during the SWFSC Cetacean
Experts’ Workshop, the simple models were overwhelmingly preferred to the complex models.
The dominant criticisms of the complex models from the expert panel were twofold: the
predictions from the complex models either 1) exhibited relatively small-scale details in
population density that are unexplainable given existing knowledge of the dynamics of the
ecosystem, or 2) were nearly identical to those from the simple model and, therefore, the extra
model complexity was not necessary for capturing the spatial patterns.
Overall conclusions to be made from this investigation into the behavior of mgcv gam
(summarized in Tables 10 and 11) are as follows:
4.2.2 Encounter Rate Models
• The gam.method perf.magic produced the simple models with the greatest predictive
performance. The best complex models were developed using outer (6 models), perf.outer (4
models), and perf.magic (2 models).
• Cubic regression splines were preferred for building simple encounter rate models,
whereas the complex models were constructed using either cubic or thin plate regression splines.
• To our surprise, the preferred simple models were split almost equally between those
built using gamma = 1.0 (8 models) and 1.4 (6 models). The best complex models were
generally constructed using gamma = 1.0.
• The sum of absolute deviations of the observed-to-predicted ratios was smaller for the
complex models in most instances, although this is to be expected because the predictions were
based on the same data used to build the models for this exercise.
4.2.3 Group Size Models
• The gam.method magic produced the simple models with the greatest predictive
performance. The best complex models were divided among gam.methods mgcv and magic.
• The preferred simple models were constructed by thin plate regression splines, in general,
whereas cubic regression splines were found in more of the preferred complex models.
• The gamma parameter performed close to our expectations in the group size models, with
the majority of simple models constructed using gamma = 1.4 and the majority of complex
models using the default value of 1.0.
• The trend in the sum of absolute deviations of the observed-to-predicted ratios was
similar to that found for the encounter rate models, with simple models tending to have slightly
larger values.
49