Extended Abstract

verification methods, though subjective evaluations can be varied and even misleading. More detailedinformation about the test cases and the subjective evaluations can be found in Ahijevych et al. (2009).3. Spatial verification methodsOne i mportant ou tcome of t he I CP ha s be en t he c ategorization of the n ew v erification m ethods intogroupings that characterize the way they treat the spatial features and attributes. The four main categoriesof m ethods are de picted in F igure 1. Panel a ) sh ows a test cas e f rom t he I CP t hat i s su bsequentlyevaluated in the other panels by a representative from each of the method categories. It was found that themethods generally fall into one of two main categories: filter and displacement.The f iltering a pproach includes ne ighborhood m ethods a nd s cale s eparation m ethods, bo th of w hichmeasure forecast quality as a function of scale. Neighborhood methods (Fig. 1b) relax the requirement foran exact match by allowing forecasts within spatial neighborhoods of the observation to be counted as (atleast partly) correct. Statistics are computed for a sequence of neighborhood sizes starting from a s inglegrid box (equivalent to traditional verification) and increasing to some much larger, but still meaningful,scale. Thus, this approach can provide forecast users with practical information on the scale at which anacceptable level of skill is attained. Ebert (2008) pr ovides an e xcellent overview of these t ypes ofapproaches, which include the Fractions Skill Score (FSS) described by Mittermaier and Roberts (2010).Scale separation methods (Fig. 1c ) diagnose and estimate the errors at distinct scales. For example,performance at small, m edium, an d l arge sca les may be ex amined sep arately using t hese ap proaches.Because different physical processes are associated with different spatial scales, this information can bevery useful for model and forecast developers. Examples of these methods include the wavelet approachby Briggs and Levine (1997) and the intensity-scale approach of Casati (2010).Forecast d isplacement approaches i nclude f eature-based methods (Fig. 1d) that identify and evaluateobjects of interest in the spatial fields, and field deformation methods (Fig. 1e) that alter the forecast tobetter m atch the obs ervations. F eature-based m ethods f ocus on t he a ttributes of w ell-defined sp atialobjects such as rain areas or wind maxima. Matching of forecast and observed objects allows evaluationof errors i n t he l ocation, size, sh ape, and ot her a ttributes of the forecast objects, w hich give u sefulinformation to both users and forecast developers. Even when matching is not possible, distributions offorecast an d o bserved f eature attributes can b e co mpared t o assess f orecast realism. Field d eformationmethods us e morphing t echniques to s tretch a nd squeeze t he f orecast t o resemble the o bservations. Incontrast to feature-based methods, field deformation methods yield an array of vector location errors forthe field. Once the forecast has been aligned with the observations, errors in amplitude can be measured.These methods are ideally suited for diagnosing phase errors.4. Comparison of types of methodsTable 1 briefly summarizes the methods by category in the context of several aspects of forecast quality.It s hould be noted that w hile t he table g ives y es o r no a nswers f or c ategories of methods, the a nswershown may not a pply t o a pa rticular m ethod i n t he c ategory. F or e xample, w hile t he di splacementmethods a re not generally designed to inform about scales at which a forecast has skill, it is certainlypossible to apply any of the methods to particular scales (e.g., an upscaled field). The method by whichMODE, a features-based displacement method (Davis et al. 2010), creates objects involves a convolutionof the field using a user-defined radius. The amount of convolution can be considered as a scale (similarto upscaling) so that skill at different scales can be obtained.All of the methods supply information about intensity errors, but they do s o in very different ways. Adeformation ap proach, f or ex ample, t ypically co mputes t raditional v erification sco res t hat p ertain t o-184-