Thematic Accuracy Assessment Procedures. Version 2 - USGS

vegetation data collection effort, (rather than simple determination of type by a qualified user in
the field using only the interpretive tools provided as products), and expertise that will not be
available to users once the project funding has ceased. Finally, “correction” of field observations
was occasionally guided to correspond with map class determinations. This practice violates a
central premise that the accuracy assessment results should be as independent of the evaluated
map data as possible.
Fuzzy sets theory (Gopal and Woodcock 1994) was often used to report higher (more acceptable)
accuracy rates, taking into account that some mapping errors are more understandable than
others. While this exercise may be helpful to map producers, a drawback of this approach for
project evaluators is that there is no nationally consistent fuzzy sets standard. Each project
developed its own criteria for the various levels of “correctness” of a field observation to map
class match. This practice made it difficult to assess the quality of an individual project against
comparable projects, a concern expressed by the 1994 guidelines (Environmental Systems
Research Institute et al. 1994). A further drawback to this approach for users that was also
expressed in the 1994 guidelines is that the fuzzy set criteria schemes reflected the perspective
and values of producers of maps and ecological classifications and might not reflect the needs of
potential users. Where the most thematically resolved map classes do not suffice for an
application, the map user may use error rates in a contingency table to derive his/her own “fuzzy
sets” or map class hierarchies in order to derive aggregated map classes that reflect user
application needs; the criteria applied by the production team may be irrelevant to many of these
needs. With the advent of more ecologically meaningful middle level units to the NVC hierarchy
(Federal Geographic Data Committee 2008), it should become more possible to develop a more
standardized scheme of fuzzy sets. Map class accuracies then might be reported at multiple
levels in the NVC hierarchy that are both ecologically meaningful and also reasonably
interpretable as a standard means of accuracy assessment at multiple levels of thematic
resolution.
Three basic components of a thematic accuracy assessment are the sampling design, the response
design, and estimation and analysis (Stehman and Czaplewski 1998). In evaluating these
components, as provided for by of the 1994 guidance (Environmental Systems Research Institute
et al. 1994) and as practiced within the NPS Vegetation Inventory in the subsequent 15 years,
two general conclusions may be made:
(1) The 1994 guidance addressed sampling design and estimation and analysis well, and relied
on well-documented and published methods. Published guidance in the remote sensing literature
is generally available for these two components, which generally are not specific to the scientific
discipline that is employed to create a mapping theme. Thus, this 2010 guidance follows mostly
the same methodological approach as that in the 1994 guidance, in the components of sampling
design and estimation and analysis. In addition, the 2010 guidance introduces some additional
recognized published methods not addressed in 1994, especially those for analysis of stratified
data and present more information on tactical methods (e.g. Geographic Information Systems)
for sampling design gained from practical experience.
(2) In contrast to methods that apply to the other components, methods for response design (field
assessment) are primarily tactical and specific to the methodologies of classification of the theme
being assessed (in this case, vegetation science) rather than to general published principles of
4