H-SAF Product Validation Report (PVR) PR-OBS-3

Products Validation Report, 30 May 2010 - PVR-03 (Product PR-OBS-3) Page 40
4.2 Validation in Belgium (IRM)
The facilities available to IRM, and the methodology adopted for validation, are described in section
3.4.1. The ground truth is provided by meteorological radar.
Table 08 reports the results of the final validation campaign of the Development Phase for PR-OBS-3.
Table 08 - Summary results of PR-OBS-3 validation in Belgium by IMR
H03 v1.4 Belgium Land Validation period: 1 st January 2009 - 31 March 2010 - Threshold rain / no rain: 0.25 mm/h
Radar Class Jan Feb Mar Apr Mag Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar
Number of > 10 mm/h 593 224 195 3603 7660 11774 23062 7342 3632 4576 1850 372 198 458 3535
comparisons 1-10 mm/h 78350 89297 123601 214815 193171 190032 238585 99197 97072 113440 354361 202536 61870 143997 165356
(ground obs.) < 1 mm/h 136377 338700 413973 311765 235281 203011 227087 103687 105080 175380 525434 500719 268990 435240 289148
> 10 mm/h -12.4 -12.9 -13.3 -15.3 -20.6 -19.0 -20.1 -20.4 -14.4 -17.2 -12.9 -11.9 -12.5 -12.4 -13.8
ME (mm/h) 1-10 mm/h -2.33 -1.85 -1.82 -2.23 -2.41 -2.47 -2.85 -2.80 -2.48 -1.60 -2.07 -1.90 -2.23 -2.01 -2.29
< 1 mm/h -0.48 -0.47 -0.49 -0.44 -0.37 -0.24 -0.33 -0.38 -0.29 -0.03 -0.39 -0.46 -0.41 -0.48 -0.47
> 10 mm/h 2.88 3.93 3.60 7.54 18.8 12.0 14.9 15.3 6.48 10.4 4.17 2.01 3.81 2.82 4.56
SD (mm/h) 1-10 mm/h 1.52 1.07 0.95 1.51 1.75 1.90 2.08 1.97 1.92 1.74 1.36 1.16 1.41 1.26 1.62
< 1 mm/h 0.21 0.20 0.20 0.21 0.21 0.21 0.21 0.21 0.21 0.21 0.21 0.20 0.18 0.20 0.21
> 10 mm/h 12.8 13.5 13.7 17.0 27.9 22.5 25.0 25.5 15.8 20.1 13.5 12.1 13.0 12.7 14.5
RMSE (mm/h) 1-10 mm/h 2.79 2.14 2.05 2.71 3.03 3.29 3.62 3.47 3.21 4.25 2.53 2.26 2.64 2.37 2.81
< 1 mm/h 0.55 0.52 0.54 0.59 0.72 0.95 0.81 0.66 0.73 2.11 0.66 0.57 0.49 0.53 0.56
> 10 mm/h 100 98 100 100 99 98 99 99 97 99 99 99 100 100 99
RMSE (%) 1-10 mm/h 99 98 100 97 96 100 98 97 93 207 96 96 99 99 97
< 1 mm/h 103 100 99 112 154 208 173 133 155 457 132 108 103 99 104
> 10 mm/h 0.02 0.30 - 0.04 0.06 0.00 0.03 0.04 0.11 0.08 0.02 0.07 0.07 0.05 0.02
CC 1-10 mm/h 0.03 0.12 0.04 0.02 0.06 0.00 0.01 0.00 0.05 0.07 0.00 0.04 0.01 0.01 0.03
< 1 mm/h 0.01 0.03 0.02 0.00 0.01 0.02 0.02 0.04 0.04 0.06 0.03 0.04 0.08 0.04 0.03
POD ≥ 0.25 mm/h 0.04 0.03 0.01 0.11 0.15 0.20 0.15 0.14 0.23 0.19 0.14 0.07 0.06 0.02 0.08
FAR ≥ 0.25 mm/h 0.94 0.92 0.89 0.84 0.86 0.79 0.86 0.90 0.80 0.79 0.76 0.73 0.83 0.84 0.80
CSI ≥ 0.25 mm/h 0.02 0.02 0.01 0.07 0.08 0.11 0.08 0.06 0.12 0.11 0.10 0.06 0.05 0.02 0.06
The monthly values of the Mean Error (ME) and of the Root Mean Square Error (RMSE) are shown in
Fig. 30 and Fig. 31, respectively (all classes are merged). The ME shows an overall large
underestimation with a seasonal dependence. The ME ranges from –0.8 mm h -1 in winter to –2.5 mm h -1
in summer. It is worth noting that the ME of an earlier version of the product ranged from –0.1 mm h -1
in winter to +0.5 mm h -1 in summer (See REP-3.04). The RMSE ranges from 1 mm h -1 to 6 mm h -1 with
largest values during summer whereas the older version had a maximum monthly RMSE of 3 mm h -1 .
However, during summer, the Probability of Detection (POD) is better than during winter.