slides

Spatial covariance-robust minimaxprediction based on experimentaldesign ideasGunter Spoeckgunter.spoeck@uni-klu.ac.atDepartment of Statistics, University of Klagenfurt, Austria

OverviewBayesian linear kriging.Covariance-robust minimax krigingSpatial covariance-robust minimax prediction based on experimental design ideas – p.1

Bayesian linear kriging0.1 The modelGaussian model for the random field {Y (x)|x ∈ X}.Covariance function C(x 1 ,x 2 ) assumed to be known.Y (x) = f(x) T β + ǫ(x)E(Y (x)|β) = f(x) T βcov(Y (x 1 ),Y (x 2 )|β) = C(x 1 ,x 2 )Prior knowledge about the trend parameter vector β:A priori distribution for β is Gaussian with E(β) = µand cov(β) = Φ known.Spatial covariance-robust minimax prediction based on experimental design ideas – p.2

0.2 The predictorKnown from Gaussian distribution theory: predictivea posteriori meanŶ BK (x 0 ) = f(x 0 ) T ˆβBK + c T 0 K −1 (Y dat − Fˆβ BK ).The a posteriori mean of the trend parameter vectorβ:ˆβ BK = (F T K −1 F + Φ −1 ) −1(F T K −1 Y dat + Φ −1 µ)Spatial covariance-robust minimax prediction based on experimental design ideas – p.3

0.3 The quality of predictionThe Total Mean Squared Error of Prediction:TMSEP(ŶBK(x 0 )) = E((Y (x 0 ) − ŶBK(x 0 )) 2 ) =C(x 0 ,x 0 ) + f(x 0 ) T Φf(x 0 )−k T (K + FΦF T ) −1 k ,wherek = (c 0 + FΦf(x 0 )).TMSEP(ŶBK(x 0 )) ≤ TMSEP(Universal Kriging)Spatial covariance-robust minimax prediction based on experimental design ideas – p.4

0.4 Problems with krigingThe covariance functioncov(Y (x 1 ),Y (x 2 )|β) = C(x 1 ,x 2 ) is assumed to beknown exactly. But generally the covariance functionis just an estimate and therefore always is uncertain.Stein (1999): kriging predictor=empirical BLUP.Pilz et. al. (1997): kriging predictor=plug-in predictor.Consequence: Kriging plug-in predictor is non linear.Christensen (1991): Kriging TMSEP underestimatesthe true unknown TMSEP of the plug-in krigingpredictor.Spatial covariance-robust minimax prediction based on experimental design ideas – p.5

0.5 SolutionsMake use of the Bayesian paradigm and specify priordistributions over a space of covariance functions.How should one get prior knowledge about thecovariance function?What are noninformative priors for covariancefunctions?Make use of the minimax principle, specify a class ofplausible covariance functions and calculate apredictor in such a way that the maximum possibleTMSEP becomes a minimum (Spöck, 1997, 2005).Spatial covariance-robust minimax prediction based on experimental design ideas – p.6

1 Covariance FunctionsThe Matern class of covariance functions:K M θ,σ 2 (h = ||x 1 − x 2 ||) = σ 2 12 θ 2−1 Γ(θ 2 ) ( h θ 1) θ 2J θ2 ( h θ 1),where J θ2 is the modified Bessel function of order θ 2 .θ 1 > 0 controlling the range of correlation.θ 2 > 0 a smoothness parameter (differentiability).Exponential model (θ 2 = 0.5), K E θ,σ 2 (h) = σ 2 exp(− h θ 1).Gaussian model (θ 2 → ∞), K G θ,σ 2 (h) = σ 2 exp(− h2θ 2 1Spatial covariance-robust minimax prediction based on experimental design ideas – p.7).

Covariance-robust minimax kriging2 ModelY (x) = f(x) T β + ǫ(x), x ∈ X ⊂ R mE(Y (x)|β,ν) = f(x) T βcov(Y (x 1 ),Y (x 2 )|β,ν) = C ν (x 1 ,x 2 ) ∈ C,ν ∈ Θ , compactE(β|ν) = µcov(β|ν) = Φ.Spatial covariance-robust minimax prediction based on experimental design ideas – p.8

3 Characterization of the CRMKPRestriction to linear affine predictorsD a = {Ŷ = wT Y dat + w 0 ;w ∈ R n ,w 0 ∈ R}The TMSEPs for different predictors Ŷ ∈ D a:TMSEP(C ν ;Ŷ ) = E βE Y0 |β,ν((Ŷ − Y (x 0)) 2 ) == C ν (x 0 ,x 0 ) + f(x 0 ) T Φf(x 0 ) − 2w T (c 0,ν + FΦf(x 0 ))+ w T (K ν + FΦF T )w + {µ T (f(x 0 ) − F T w) − w 0 } 2 .Spatial covariance-robust minimax prediction based on experimental design ideas – p.9

In the squared marginal bias{µ T (f(x 0 ) − F T w) − w 0 } 2the covariance function does not appear.Consequence: Ŷ M (x 0 ) may be sought in the class ofall marginally unbiased predictorsD = {Ŷ = wT Y dat + µ T (f(x 0 ) − F T w)|w ∈ R n }.It is characterized by the relationshipsupC ν ∈CTMSEP(C ν ;Ŷ M (x 0 )) = infŶ ∈DsupC ν ∈CTMSEP(C ν ;Ŷ ).Spatial covariance-robust minimax prediction based on experimental design ideas – p.10

The definition of the minimax kriging predictornecessitates that the infima and suprema exist. Wetherefore suppose that the parametrization of thecovariance function is continuous on the compactparameter set Θ, i.e. lim ν→ν0 C ν (x 1 ,x 2 ) = C ν0 (x 1 ,x 2 )for all x 1 ,x 2 ∈ X.Under this assumption TMSEP(C ν ;Ŷ ), interpreted asa function of ν, is for every fixed Ŷ ∈ D a continuousfunction with compact domain Θ. Lemma 6.5 of Pilz(1991) then shows that∫sup TMSEP(C ν ;Ŷ ) = sup TMSEP(C ν ;Ŷ )ξ(dν),C ν ∈Cξ∈ΞΘSpatial covariance-robust minimax prediction based on experimental design ideas – p.11

where Ξ is the set of all probability measures ξ, thatare defined on the σ-algebra of the Borel sets of Θ.The equivalent new minimax problem now readssupC ν ∈CTMSEP(C ν ;Ŷ M (x 0 )) =infŶ ∈Dsupξ∈Ξ∫ΘTMSEP(C ν ;Ŷ )ξ(dν),This minimax problem may now be solved byinterchanging minimization and maximization.Spatial covariance-robust minimax prediction based on experimental design ideas – p.12

Reason: Minimax theorem of Sion (1958)The set of so-called average covariance matricesM(ξ) =∫Θ()C ν (x 0 ,x 0 ) c T 0,νξ(dν) =c 0,ν K ν()C ξ (x 0 ,x 0 ) c T 0,ξc 0,ξ K ξis convex and compact. The convexity andcompactness of this set follows from the continuityin ν of the covariance functions C ν (x 1 ,x 2 ) directlyfrom Lemma 5.1.8 in Bandemer et al. (1978) byinterpreting the average covariance matrices asinformation matrices from experimental design.Spatial covariance-robust minimax prediction based on experimental design ideas – p.13

The minimax theorem of Sion (1958) applies since∫TMSEP(C ν ;Ŷ )ξ(dν) =Θ= C ξ (x 0 ,x 0 ) + f(x 0 ) T Φf(x 0 ) − 2w T (c 0,ξ + FΦf(x 0 ))+ w T (K ξ + FΦF T )wis a continuous and concave function inM(ξ) ∈ M(Ξ) and a convex function in w ∈ R n .The average covariance matrix M(ξ) sought in theminimax problem and weight vector w ∈ R n aresaddle points of the above integrated TMSEP.Spatial covariance-robust minimax prediction based on experimental design ideas – p.14

Interchanging supremum and infimum we see thatthe integrated TMSEP becomes minimal if and only ifwe insert for Ŷ the Bayes kriging predictorŶM(ξ) BK (x 0) = f(x 0 ) T ˆβBK M(ξ) + cT 0,ξ K−1 ξ(Y dat − FˆβM(ξ) BK )ˆβM(ξ) BK = (FT K −1ξF + Φ −1 ) −1 (F T K −1ξY dat + Φ −1 µ).Thus, the minimax predictor is characterized as thatBayesian kriging predictor Ŷ M(ξ BK0 )that maximizesTMSEP(M(ξ);Ŷ BKM(ξ) ) = C ξ(x 0 ,x 0 ) + f(x 0 ) T Φf(x 0 ) −− (c 0,ξ + FΦf(x 0 )) T (K ξ + FΦF T ) −1 (c 0,ξ + FΦf(x 0 )).Spatial covariance-robust minimax prediction based on experimental design ideas – p.15

4 The Equivalence of the MinimaxProblem to an Experimental DesignProblemInstead of maximizing the Bayes risk in M(ξ) weconsider here the equivalent problem of minimizationof the reciprocalTMSEP(M(ξ);Ŷ BKM(ξ) )−1 → minξ∈Ξ .Spatial covariance-robust minimax prediction based on experimental design ideas – p.16

If we apply the block-inversion rule to the regular,blocked matrix, in the following called average totalcovariance matrix,M b (ξ) =()C ξ (x 0 ,x 0 ) + f(x 0 ) T Φf(x 0 ) (c 0,ξ + FΦf(x 0 )) Tc 0,ξ + FΦf(x 0 ) K ξ + FΦF T ,then the first block in the inverse M b (ξ) −1 is given byTMSEP(M(ξ);Ŷ BKM(ξ) )−1 ,the reciprocal of the integrated Bayes kriging TMSEP.Spatial covariance-robust minimax prediction based on experimental design ideas – p.17

Multiplication of the matrix M b (ξ) −1 with the unitvector c b = (1, 0, 0,... , 0) T ∈ R n+1 in the quadraticform c T b M b(ξ) −1 c b thus results exactly in thereciprokal TMSEP to be minimized.Defining M b (Ξ) = {M b (ξ),ξ ∈ Ξ}, U b = c b c T b andΨ(ξ) = Ψ(M b (ξ)) = tr(U b M b (ξ) −1 )with tr(.) the trace operator, the minimax predictormay thus be determined by minimizing the functionalΨ(.) in M b (ξ).Ψ(M b (ξ)) is in form equivalent to a design functional.Spatial covariance-robust minimax prediction based on experimental design ideas – p.18

Properties of the functionalΨ(M b (ξ)) = tr(U b M b (ξ) −1 )to be minimized are analogous to design functionals:From the compactness and convexity of the setM(Ξ) it follows that also the set M b (Ξ) is compactand convex.Analogiously to the proof of Theorem 12.2 in Pilz(1991) it may be shown that the above functionalis convex on the set M b (Ξ).Furthermore it may be shown that it is continuouswith respect to the usual Euclidean metric.Spatial covariance-robust minimax prediction based on experimental design ideas – p.19

Lemma 11.6 and 11.7 in Pilz (1991) show that forall pairs (ξ, ¯ξ) ∈ Ξ × Ξ the directional derivatives∆ Ψ (ξ, ¯ξ) = limα↓0Ψ((1 − α)ξ + α¯ξ) − Ψ(ξ)αexist, are given by∆ Ψ (ξ, ¯ξ) = Ψ(ξ) − tr(U b M b (ξ) −1 M b (¯ξ)M b (ξ) −1 ).and attain their minimum in direction of aprobability measure ¯ξ ν with one-point supportν ∈ Θ.Spatial covariance-robust minimax prediction based on experimental design ideas – p.20

5 Calculation of the CRMKPThe directional derivative plays an important role inthe construction of algorithms for the minimization ofthe convex functional Ψ(.).As a consequence of the analogy of this functional todesign functionals known from experimental designgradient descent algorithms may be used as they areknown from this theory.Inversion of the Matrix M b (ξ) can thereby becircumvented by means of update formulas given inPilz (1991) and based only on matrix multiplications.Spatial covariance-robust minimax prediction based on experimental design ideas – p.21

6 ExampleData set considered: 591 Caesium137measurements in the region of Gomel, Belarus, 10years after the Tchernobyl accident.The data set has been log-transformed in order tofollow a Gaussian distribution.By means of the empirical variogram estimator andweighted least squares a convex combination of aGaussian and an Exponential covariance functionmodel has been fit to the data.Spatial covariance-robust minimax prediction based on experimental design ideas – p.22

In order to get insight into the uncertainty of thecovariance function Gaussian random fields havebeen simulated at exactly the data locations and forevery simulation the covariance function has beenreestimated (parametric bootstrap). The covariancefunction used to get the simulations has been exactlythe one estimated from the data.After visual inspection of the simulated covariancefunctions a belt of plausible covariance functions hasbeen defined by means of restricting the parameterspace of the convex combination of the twocovariance types.Spatial covariance-robust minimax prediction based on experimental design ideas – p.23

This has been done by restricting a weightedEuclidean distance of the parametrized convexcombinations of the two used covariance functiontypes from the “central” estimated covariancefunction to be bounded.The covariance-robust minimax kriging predictor thenhas been calculated by means of an algorithm knownfrom experimental design searching for continuousdesigns ξ ∈ Ξ minimizing the reziprocal of theTMSEP, Ψ(M b (ξ)).Spatial covariance-robust minimax prediction based on experimental design ideas – p.24

54.543.532.521.510.500 20 40 60 80 100 120 140 160 180 200Figure 1: Semivariogram estimates simulated bymeans of a parametric bootstrapSpatial covariance-robust minimax prediction based on experimental design ideas – p.25

15032.51002501.5100.50−50−0.5−1−100−1.5−2−150−150 −100 −50 0 50 100 150Figure 2: Ordinary minimax kriging predictions forthe log-transformed Gomel dataSpatial covariance-robust minimax prediction based on experimental design ideas – p.26

1501.61001.5501.41.301.2−501.11−1000.9−150−150 −100 −50 0 50 100 1500.8Figure 3: Square root of the minimax risk for thelog-transformed Gomel dataSpatial covariance-robust minimax prediction based on experimental design ideas – p.27

1503100250100−50−1−100−2−150−150 −100 −50 0 50 100 150−3Figure 4: Ordinary kriging predictions for the logtransformedGomel dataSpatial covariance-robust minimax prediction based on experimental design ideas – p.28

1501.41001.250100.8−500.6−1000.4−150−150 −100 −50 0 50 100 1500.2Figure 5: Ordinary kriging standard deviations forthe log-transformed Gomel dataSpatial covariance-robust minimax prediction based on experimental design ideas – p.29

Figure 6: The semivariograms of the minimax predictors(red)Spatial covariance-robust minimax prediction based on experimental design ideas – p.30

7 ConclusionThe covariance-robust minimax kriging predictor is aworst case Bayesian kriging predictor.For the calculation of the covariance-robust minimaxkriging predictor algorithms based on directionalderivatives and borrowed from experimental designtheory may be used.Prior knowledge on the belt of plausible covariancefunctions may be gained by means of simulations(bootstrapping) from the sampling distribution of thecovariance estimator.Spatial covariance-robust minimax prediction based on experimental design ideas – p.31

It is visible from Fig. 6 that one seems to be minimaxin his decision if one uses kriging with a prioriminimum plausible range, maximum plausible nuggeteffect and maximum plausible sill. Minimum plausiblerange has been given preference to maximumplausible sill in the presented example.Spatial covariance-robust minimax prediction based on experimental design ideas – p.32

Acknowledgement. This work was partially funded by theEuropean Commission, under the Sixth FrameworkProgramme, by the Contract N. 033811 with DG INFSO,action Line IST-2005-2.5.12 ICT for Environmental RiskManagement. The views expressed herein are those ofthe authors and are not necessarily those of theEuropean Commission.Spatial covariance-robust minimax prediction based on experimental design ideas – p.33