Iterative Weighted Risk Estimation for Nonlinear Image Restoration ...

3. SELECTION OF REGULARIZATION PARAMETER β3.1 Generalized Cross Validation (GCV)GCV 2 is an attractive method for selecting β, especially in the context of linear algorithms. Fora generic linear reconstruction of the form, f β (y) =F β y (F β is matrix representing some type ofinverse filtering), GCV selects β by minimizingGCV(β) = N −1 ||Af β (y) − y|| 2(1 − N −1 tr{AF β }) 2 . (6)Calculation of the trace, tr{AF β } in the denominator of (6), can be performed either analyticallyin some special cases, e.g., when F β is circulant, or stochastically using Monte-Carlo methods 17 fora general F β . GCV(β) is simple to implement and is know to yield β that asymptotically providesan optimal reconstruction for linear algorithms. 2For nonlinear algorithms (denoted by f β ), Deshpande et al. 3 proposed the following NGCVmeasure 3, 4 (GCV for nonlinear algorithms) based on the principles of cross-validation:NGCV(β) = N −1 ‖y − Af β (y)‖ 2 2(1 − N −1 tr{AJ fβ (y)}) 2 , (7)where J fβ (y) is the Jacobian matrix consisting of partial derivatives of the components {f β,n (y)} N n=1of f β (y) with respect to the components {y n } N n=1 of y: thekl-th element of J fβ (y) isgivenby[J fβ (y)] kl = ∂f β,k(z)∂z l∣∣∣∣z=y. (8)NGCV(β) is a generalization of GCV(β) for nonlinear algorithms. 3, 4 It is more involved andcomputation intensive compared to GCV(β) (6) as it requires the evaluation of J fβ (y).3.2 Stein’s Principle for Estimating MSE-type MeasuresIn image reconstruction problems, mean squared error (MSE),MSE(β) △ = N −1 ‖x − f β (y)‖ 2 2, (9)is commonly used to determine quality of a reconstructed image and is an attractive alternative to(N)GCV for tuning β. However, MSE(β) cannot be directly used in practice due to its dependenceon the unknown x. For denoising applications, i.e., A = I N in (1), one can use Stein’s principle ∗to estimate MSE(β) when noise is modeled as Gaussian. This process leads to the so-called Stein’sUnbiased Risk Estimate (SURE) 5, 9 given by SURE(β) =N −1 ‖y−f β (y)‖ 2 2−σ 2 +2σ 2 N −1 tr{J fβ (y)}.SURE(β) is unbiased, i.e., E b {MSE(β)} = E b {SURE(β)} (where E b {·} denotes the expectationoperation with respect to b), but requires the knowledge of the noise variance σ 2 unlike (N)GCV.Evaluation of J fβ (y) inSURE(β) can be performed analytically for some special nonlinear denoisingalgorithms (e.g., wavelets-based denoising 9 and nonlocal means 11 ) or numerically using the Monte-Carlo method 10 for an arbitrary linear/nonlinear, iterative/noniterative algorithm f β .∗ Application of Stein’s principle requires the hypotheses that f β is (weakly) differentiable 6, 9 and decayssufficiently rapidly 6, 9 such that lim bn→∞ p(b)f β,n (y) =0∀ n, for the Gaussian probability density functionp(b) =(2πσ 2 ) −N/2 exp(−‖b‖ 2 2/2σ 2 ).SPIE-IS&T/ Vol. 8296 82960N-4Downloaded from SPIE Digital Library on 06 Apr 2012 to 141.213.236.110. Terms of Use: http://spiedl.org/terms

However, for inverse problems (1) involving A with a nontrivial null-space N(A), it is notpossible to estimate MSE since information concerning x that lies in N(A) cannotberecoveredfrom y. In such cases, Projected-MSE may be used as an alternative 6, 8 to MSE. Projected-MSEcomputes squared-error based on those components of x that lie in the orthogonal complement 6, 8of N(A) (equivalent to R(A ⊤ ), the range space of A ⊤ ) that are in turn accessible from y and thusallows for its estimation: 6, 8Projected-MSE(β) =N −1 ‖P(x − f β (y))‖ 2 2, (10)where P = A ⊤ (AA ⊤ ) † A is the matrix corresponding to a projection onto R(A ⊤ )and(·) † denotespseudo-inverse. Another squared-error measure that is amenable to estimation is Predicted-MSE(PMSE) 12 that computes the error in the measurement domain:Predicted-MSE(β) =N −1 ‖A(x − f β (y))‖ 2 2. (11)Both Projected-MSE and Predicted-MSE are special cases of the following weighted error measure(WMSE) that we consider for generality of exposition:WMSE(β) =N −1 ‖A(x − f β (y))‖ 2 W, (12)where W is a positive definite (W ≻ 0), § symmetric (W ⊤ = W) weighting matrix. It is easy tosee that W =(AA ⊤ ) † for Projected-MSE in (10) and W = I N for Predicted-MSE in (11). Similarto SURE(β), one can arrive at an estimate for WMSE(β) using Stein’s principle 6 as stated in thefollowing theorem.Theorem 3.1. Let f β in (2) be (weakly) differentiable and satisfy E b {|[WA f β (y)] n |} < ∞∀n.Then, the random variableWSURE(β) =N −1 ‖y − Af β (y)‖ 2 W − σ 2 N −1 tr{W} +2σ 2 N −1 tr{WAJ fβ (y)} (13)is an unbiased estimator of WMSE(β) (12), i.e., E b {WMSE(β)} = E b {WSURE(β)}. A proof of this result can be obtained by a straightforward extension of previous results, e.g.,of that of Eldar. 6 As with SURE(β), the key step in computing WSURE(β) is the calculationof tr{WAJ fβ (y)} that in turn requires J fβ (y). We propose to evaluate J fβ (y) analytically for aniterative reweighted least squares (IRLS)-type algorithm 13, 14 that can be applied to (2) for severalregularizers including TV (5), l 1 -regularization (3) and smooth edge-preserving criteria such as (4).4. EVALUATING THE JACOBIAN MATRIX J fβ (y)4.1 Standard IRLS AlgorithmThe iterative reweighted least squares (IRLS) algorithm performs the minimization in (2) by repetitivelysolving iteration-dependent linear systems 13 of the formH (i) u (i+1) = A ⊤ y. (14)§ A matrix W ∈ R N×N is said to be positive definite, i.e., W ≻ 0, ifz ⊤ Wz > 0 ∀ z ∈ R N .SPIE-IS&T/ Vol. 8296 82960N-5Downloaded from SPIE Digital Library on 06 Apr 2012 to 141.213.236.110. Terms of Use: http://spiedl.org/terms

At iteration i, H (i) = A ⊤ A+R ⊤ Ω −1(i) R and Ω (i) =diag{ω (i) } is a diagonal weighting matrix, wherethe n-th component of ω (i) is given by ω (i)n =t∣ n =0...PM − 1, and φt=|[Ru(i) ] n|, ′ is theβφ ′ (t)derivative of φ. Forthel 1 -regularization in (3), we have thatω (i)n = β −1 |[Ru (i) ] n |, (15)while for the Fair potential (4),ω (i)n =(βδ) −1 (δ + |[Ru (i) ] n |). (16)In case of TV, ω (i) = 1 P ⊗ ˘ω (i) ,where⊗ denotes Kronecker product, 1 Pand the m-th element of ˘ω (i) ∈ R M is given byis a P × 1 vector of 1s( P) 1/2∑˘ω (i)m = β −1 |[R p u (i) ] m | 2 . (17)p=1Standard iterative solvers, e.g., preconditioned conjugate gradient (PCG), for (14) may convergeslowly for nonsmooth regularizers such as l 1 -regularization (3) and TV (5) since Ru (i) tends tobecome sparse with increasing i and correspondingly Ω −1(i)becomes poorly conditioned for (15) and(17). To enhance numerical stability of IRLS (14), a small positive constant is often added to (15)and (17)—this is usually referred to as corner rounding 13 and is often administered to IRLS (14)when nonsmooth regularizers (such as l 1 -regularization and TV) are used.4.2 The IRLS-MIL Algorithm13, 18One can circumvent the use of corner rounding for IRLS by using a matrix splitting scheme.To see this, we rewrite (14) asB (i) u (i+1,j+1) = A ⊤ y +(C (i) − A ⊤ A)u (i+1,j) , (18)where B (i) = C (i) + R ⊤ Ω −1(i) R and C (i) is an invertible matrix such that C (i) − A ⊤ A ≻ 0. Solvingfor u (i+1,j+1) in (18), we obtain the following iterative schemeu (i+1,j+1) = B −1(i) (A⊤ y +(C (i) − A T A)u (i+1,j) ), (19)that is guaranteed 18 to converge on the j-iteration to a solution of (14). By applying matrix inversionlemma (MIL) 19 to B −1(i) ,wegetB−1 (i) = C−1 (i) − C−1 (i) R⊤ G −1i RC −1(i) ,whereG (i) = Ω (i) + RC −1(i) R⊤ .Thus, (19) with MIL leads tou (i+1,j+1) = z (i+1,j) − C −1(i) R⊤ v (i+1,j) , (20)solve {G (i) v (i+1,j) = Rz (i+1,j) } for v (i+1,j) , (21)△where z (i+1,j) = C−1(i) A⊤ y +(I N − C −1(i) A⊤ A)u (i+1,j) . We use the following iterative solver 18 for(21) in the inner loop of the algorithm:v (i+1,j,k+1) = Γ −1(i) (Rz (i+1,j) + Qv (i+1,j,k) ), (22)SPIE-IS&T/ Vol. 8296 82960N-6Downloaded from SPIE Digital Library on 06 Apr 2012 to 141.213.236.110. Terms of Use: http://spiedl.org/terms

△where Γ (i) = Ω(i) + λI PM is a diagonal matrix, Q = △ λ I PM − RC −1(i) R⊤ ,andλ is the maximumeigenvalue of RC −1(i) R⊤ . The update-rule (22) guarantees convergence 15, 18 of v (i+1,j,k) toasolutionof (21) and is implemented in favor of a CG-based solver since (22) is linear in both z (·) and v (·) andalso decouples the shift-variant component Ω (i) from the rest of the terms in G (i) : these featuresallow us to more easily evaluate the desired Jacobian matrix in (7) and (13) at each iteration aselucidated in the sequel. An important observation is that IRLS-MIL (20)-(22) depends on Γ (i)(that in turn relies on Ω (i) rather than Ω −1(i)) that is well-defined and thus does not require cornerrounding for handling (15) and (17).4.3 Implementation of IRLS-MILThe IRLS-MIL algorithm (20)-(21) generally exhibits faster convergence 13 over standard IRLS (14).The convergence speed of IRLS-MIL (20)-(22) depends primarily on the “proximity” of C (i) to A ⊤ Awhile ensuring C (i) − A ⊤ A ≻ 0. For image restoration with circulant A in (1), A ⊤ A ∈ R N×N is△also circulant. So we used C (i) = C ν = A ⊤ A + νI N ∀ i and implemented C −1ν using FFTs. Theparameter ν>0 was chosen to achieve a prescribed condition number of C ν , κ(C ν ), that can beeasily computed as a function of ν. In general, setting κ(C ν )toalargevaluecanleadtonumericalinstabilities in C −1ν and IRLS-MIL, while a small κ(C ν ) reduces convergence speed of IRLS-MIL. 13In our experiments, we found that ν leading to κ(C ν ) ∈ [20, 100] yielded good acceleration overstandard IRLS for a fixed number of outer (i.e., index by i) iterations, so we set ν such thatκ(C ν ) = 100.4.4 Jacobian Matrix Derivation for IRLS-MILWe propose to evaluate J fβ (y), or equivalently, J u(·)(y) (sinceu (·) is the reconstructed output atany stage of the IRLS-MIL algorithm in (20)-(22)) analytically as follows. Using linearity of (8),we have from (20) thatJ u(i+1,j+1) (y) =J z(i+1,j) (y) − C −1(i) R⊤ J v(i+1,j,K) (y), (23)where J z(i+1,j) (y) =C −1(i) A⊤ +(I N − C −1(i) A⊤ A)J u(i+1,j) (y). The term J v(i+1,j,K) (y) in (23) correspondsto v (i+1,j,K) that we obtain after performing K iterations of (22) and that we substitute inplace of v (i+1,j) in (20). We use product rule for Jacobian matrices 20 to obtain J v(i+1,j,K) (y) from(22) as follows:J v(i+1,j,k+1) (y) =Γ −1(i) (RJ z (i+1,j)(y)+QJ v(i+1,j,k) (y)) − Γ −2(i) D v (i+1,j,k)J ω(i) (y), (24)△where D v(i+1,j,k) =diag{Rz(i+1,j) + Qv (i+1,j,k) }. Since ω (i) is a function of Ru (i) ,weusechainrule 20 on J ω(i) (y) to get thatJ ω(i) (y) =J ω(i) (u (i) ) J u(i) (y). (25)We analytically evaluate J ω(i) (u (i) )forω (i) in (15)-(17), respectively, and obtain(l 1 -regularization) J ω(i) (u (i) ) = β −1 diag{τ (i) } R, (26)(Fair potential) J ω(i) (u (i) ) = (βδ) −1 diag{τ (i) } R, (27)( P)∑(TV) J ω(i) (u (i) ) = 1 P ⊗ β −2 diag{ϱ (i)p }R p , (28)p=1SPIE-IS&T/ Vol. 8296 82960N-7Downloaded from SPIE Digital Library on 06 Apr 2012 to 141.213.236.110. Terms of Use: http://spiedl.org/terms

1. Initialization: u △ (0,0) = A ⊤ y, J u(0,0) (y)n = △ A ⊤ n, i =02. Repeat Steps 3-17 until Stop Criterion is met3. If i =04. u (i+1,0) = u (i,0) , v (i+1,0,0) = Ru (i,0) , J u(i+1,0) (y)n = △ J u(i,0) (y)n, J v(i+1,0,0) (y)n = △ RJ u(i,0) (y)n5. Else6. u (i+1,0) = u (i,J) , v (i+1,0,0) = v (i,J−1,K) , J u(i+1,0) (y)n = △ J u(i,J) (y)n, J v(i+1,0,0) (y)n = △ J v(i,J−1,K) (y)n7. Compute Γ (i) ;setj =08. Run J iterations of Steps 9-149. Compute z (i+1,j) and J z(i+1,j) (y)n10. If j>0setv (i+1,j,0) = v (i+1,j−1,K) and J v(i+1,j,0) (y)n = △ J v(i+1,j−1,K) (y)n12. Run K iterations of (22) and (24) to get v (i+1,j,K) and J v(i+1,j,K) (y)n13. Compute u (i+1,j+1) (20) and J u(i+1,j+1) (y)n (23)14. Set j = j+1andreturntoStep916. Compute NGCV(β) and / or WSURE(β) at iteration i using (29)-(30), (7) and (13), respectively17. Set i = i +1andreturntoStep3Figure 1: Iterative computation of WSURE(β) andNGCV(β) for image restoration using IRLS-MILalgorithm (with J iterations of (20)-(21) and K iterations of (22)). We use a pregenerated binary randomvector n = n ±1 for Monte-Carlo computation (29)-(30) of the required traces in (7) and (13), respectively.Vectors of the form J·(·)n are stored and manipulated in place of actual matrices J·(·).where the n-th component of τ (i) ∈ R PM is τ (i)n =sign([Ru (i) ] n ), and the m-th component ofϱ (i)p ∈ R M , p =1...P,isϱ (i)pm =[R p u (i) ] m ˘ω −1(i)m . In (25)-(28), u (i) is to be interpreted asu (i) = u (i+1,0) , the initialization for the j-iterations (19) at (i + 1)-th outer iteration.To summarize our method, we run (20), (22) for restoration, and additionally execute theupdates in (23)-(24) using (25)-(28) to iteratively evaluate J u(·)(y) at any stage of IRLS-MIL.4.5 Monte-Carlo Trace EstimationThe Jacobian matrices J·(·) have enormous sizes for typical restoration settings and cannot be storedand manipulated directly to compute the desired traces, tr{AJ fβ (y)} in (7) and tr{WAJ fβ (y)} in(13), respectively. So we use a Monte-Carlo method 17 to estimate tr{AJ fβ (y)} and tr{WAJ fβ (y)}that is based on the following straightforward identity: for any deterministic matrix T ∈ R N×N ,E n {n ⊤ T n} =tr{T},where n ∈ R M is an i.i.d. zero-mean random vector with unit variance. To use this type of stochasticestimation for tr{AJ fβ (y)} and tr{WAJ fβ (y)}, we adopt the procedure proposed by Vonesch et al. 7where we take products with n in (23)-(25) and store and update vectors of the form J u(·)(·)n andJ v(·)(·)n in IRLS-MIL. At any point during the course of IRLS-MIL, the desired traces in (7) and(13) are stochastically approximated astr{AJ fβ (y)} ≈ ˆt AJu△= n ⊤ AJ u(·)(y)n, (29)tr{WAJ fβ (y)} ≈ ˆt WAJu△= n ⊤ WAJ u(·)(y)n, (30)respectively. To improve accuracy of (29)-(30), n can be designed to decrease the variance of ˆt AJuand ˆt WAJu : it has been shown 17 that the variance of a Monte-Carlo trace estimate (such as ˆt AJuor ˆt WAJu ) is lower for a binary random vector n ±1 whose elements are either +1 or −1 withSPIE-IS&T/ Vol. 8296 82960N-8Downloaded from SPIE Digital Library on 06 Apr 2012 to 141.213.236.110. Terms of Use: http://spiedl.org/terms

Table 1: Setup for image restoration (IR) experimentsExperiment Test image (256 × 256) Blur RegularizationIR-A Cameraman Uniform 9 × 9 R TV (5)IR-B Peppers (1 + x 1 + x 2 ) −1 , −7 ≤ x 1 ,x 2 ≤ 7 R l1 (3)IR-C House Gaussian with standard deviation 2 R FP (4)Table 2: ISNR (in dB) of Restored Images for Experiments IR-A, IR-B, and IR-C and varying BSNRExperiment BSNR σ 2 MSE Projected- Predicted- (oracle) SURE SURENGCV GCV20 3.08 × 10 1 3.85 3.73 3.84 3.84 2.45IR-A 30 3.08 5.85 5.84 5.85 5.85 2.4040 3.08 × 10 −1 8.50 8.50 8.49 8.49 2.4150 3.08 × 10 −2 11.02 10.97 11.00 11.01 2.3820 1.99 × 10 1 4.44 4.28 4.34 4.34 -0.26IR-B 30 1.99 8.44 8.43 8.44 8.44 -0.5140 1.99 × 10 −1 12.41 12.41 12.41 12.28 -0.5550 1.99 × 10 −2 15.54 15.53 15.44 15.53 -0.5520 1.76 × 10 1 3.98 3.57 3.62 3.61 -0.59IR-C 30 1.76 4.71 4.70 4.55 4.58 -1.0140 1.76 × 10 −1 6.19 6.14 6.12 6.12 -1.0650 1.76 × 10 −2 6.79 6.64 6.63 6.63 -1.07probability 0.5 than for a Gaussian random vector n ∼N(0, I M ). So in our experiments, we usedone realization of n ±1 in (29)-(30). Fig. 1 presents an outline for implementing IRLS-MIL withrecursions for J·(·)n ±1 to compute and monitor NGCV(β) andWSURE(β) as IRLS-MIL evolves.5. EXPERIMENTAL RESULTSWe performed simulations with standard test images and blur kernels 8 outlined in Table 1. Inall experiments, data y was simulated by synthetically adding Gaussian noise whose variance waschosen to meet a prescribed blurred signal to noise ratio, BSNR =10log △ 10 (Var(Ax)/σ 2 ). We ran100 iterations of IRLS-MIL (with J =1andK = 1 inner-iterations, respectively, see Figure 1)with the following regularizers: l 1 -wavelets (3) with two levels of undecimated Haar excluding theapproximation level for R, a smooth convex regularizer (4) with Fair Potential (δ =10 −4 )andfinite differences for R and TV (5). We measured quality of restored results using improvement insignal to noise ratio, ISNR(β) =10log △ 10 (‖y − x‖ 2 2/‖x − f β (y)‖ 2 2).In all experiments, we selected β so as to minimize Projected-SURE(β), Predicted-SURE(β) andNGCV(β) using golden-mean search method. We assumed that σ was available (in practice, σ canbe reliably estimated) for computing Projected-SURE(β), Predicted-SURE(β). We implementedW =(AA H ) † in Projected-SURE using FFTs, where we set the eigenvalues of AA H below athreshold of 10 −3 to zero for numerical stability of (AA H ) † . We also include results for β selectedby minimizing GCV(β) that applies only to linear algorithms, but has been suggested for use withnonlinear algorithms as well (with F β replaced by f β (y) in (6)) by Giryes et al. 8 We compare ourresults with those obtained by minimizing the true “unknown” (oracle) MSE(β).SPIE-IS&T/ Vol. 8296 82960N-9Downloaded from SPIE Digital Library on 06 Apr 2012 to 141.213.236.110. Terms of Use: http://spiedl.org/terms

(a)Predicted−MSE, Predicted−SURE (in dB)2.672.422.161.911.651.391.140.880.620.370.11−0.14−0.4−0.661.73Predicted−SURE−0.91Predicted−MSE 1.62Projected−SURE−1.171.52Projected−MSE−1.431.41−4.36 −3.76 −3.16 −2.56 −1.96 −1.36 −0.76 −0.16 0.44 1.04 1.64β (log 10scale)3.132.892.792.682.572.472.362.262.152.051.941.83Projected−MSE, Projected−SURE (in dB)ISNR (in dB)13.4712.2811.099.98.717.526.325.133.942.751.560.37−0.82−2.01−3.2−4.39ISNRMSE−optimalProjected−SURE−basedPredicted−SURE−basedNGCV−basedGCV−based−5.58−4.36 −3.76 −3.16 −2.56 −1.96 −1.36 −0.76β (log scale) 10−0.16 0.44 1.04 1.64Figure 2: Experiment corresponding to third row in IR-B in Table 2: (a) Plot of Projected-MSE,Projected-SURE(β), Predicted-MSE and Predicted-SURE(β), respectively, as functions of β. The estimatesclosely capture the trends of the respective MSE-curves and their minima (indicated by *) are closeto that of the (oracle) MSE indicated by the solid vertical line. (b) Plot of ISNR(β) as a function of β alongwith indication of ISNR values corresponding to MSE-, Predicted-SURE-, Projected-SURE-, NGCV-, andGCV-selections. ISNR values obtained using Projected-SURE, Predicted-SURE, and NGCV are agreeablyclose to the oracle (MSE) while GCV selection is noticeably worse.(b)In Table 2, we present ISNR of restoration results obtained by optimizing MSE(β), GCV(β),NGCV(β), Predicted-SURE(β), and Projected-SURE(β) for various BSNRs in each experiment.NGCV, Predicted-SURE, and Projected-SURE lead to ISNRs reasonably close to oracle-ISNRcorresponding to minimum-MSE in all experiments. In contrast, GCV (that is appropriate for linearalgorithms) yielded significantly lower ISNR values; this is likely because of the strong nonlinearityof our algorithm for the considered nonquadratic regularizers.We plot Projected-MSE(β), Predicted-MSE(β), Projected-SURE(β) and Predicted-SURE(β) inFigure 2a and ISNR(β) in Figure 2b, respectively, as functions of β for an instance of ExperimentIR-B. Predicted-SURE is almost an exact replica of Predicted-MSE, while Projected-SURE followsthe trend of Projected-MSE very closely (Figure 2a), deviating only slightly for some β-valuesaway from the minimum. NGCV-, Predicted-SURE-, and Projected-SURE-based selections lead toISNRs close to that of the MSE-based selection (Figure 2b), while the GCV-based one produces amuch lower ISNR value.Figure 3 shows a visual comparison of images restored with βs that minimized MSE(β), NGCV(β),Projected-SURE(β), Predicted-SURE(β), and GCV(β) for the same instance of Experiment IR-B.Figures 3d, 3e, and 3f corresponding to Projected-SURE, Predicted-SURE, and NGCV, respectively,are visually similar to the minimum-MSE result Figure 3(c). GCV-based restoration Figure3(g) is noticeably over-smoothed for reasons explained earlier. We obtained similar results in variousother experiments 14 (results not shown due to space limitations) indicating the consistency ofour approach.6. CONCLUSIONSelection of the regularization parameter β is an important step in regularized reconstruction methods.When data is corrupted by Gaussian noise, NGCV(β) (the nonlinear version of GCV) 3, 4 andSPIE-IS&T/ Vol. 8296 82960N-10Downloaded from SPIE Digital Library on 06 Apr 2012 to 141.213.236.110. Terms of Use: http://spiedl.org/terms

(a) (b) (c) (d)(e) (f) (g)Figure 3: Experiment corresponding to third row in IR-B in Table 2: Zoomed images of (a) Noise-freePeppers; (b) Blurred and noisy data; and TV-restored images with regularization parameter β selectedto minimize (c) (oracle) MSE(β) (12.41 dB); (d) Projected-SURE(β) (12.41 dB); (e) Predicted-SURE(β)(12.41 dB); (f) NGCV(β) (12.28 dB); (g) GCV(β) (-0.55 dB). Projected-SURE-, Predicted-SURE- andNGCV-based results, (d)-(f), respectively, visually resemble the oracle MSE-based result (c) very closely,while the GCV-based result is considerably over-smoothed.weighted MSE-estimates, 6 i.e., Predicted-SURE(β), Projected-SURE(β) can be used to tune β,but their computation necessitates the evaluation 6 of the trace of a linear transform of the Jacobianmatrix J fβ . In this paper, we introduced a method to recursively evaluate J fβ for theIRLS-MIL algorithm that can handle a variety of regularizers including TV, l 1 -regularization andsmooth edge-preserving criteria. We estimated the desired trace using a Monte-Carlo scheme. 17We demonstrated through simulations that β selected by minimizing NGCV, Predicted-SUREand Projected-SURE provide near-MSE-optimal results for restorations using TV, l 1 -wavelets andsmooth edge-preserving regularization with the Fair potential. Our results suggest that NGCV,Predicted-SURE and Projected-SURE can be useful for regularization parameter selection in nonlinearrestoration problems involving Gaussian noise. The proposed method can also be extendedto handle synthesis formulations, various other regularizers and restoration algorithms.ACKNOWLEDGMENTSThis work was supported in part by NIH grants R01 HL 098686 and P01 CA 87634.REFERENCES[1] Karl, W. C., “Regularization in image restoration and reconstruction,” in [Handbook of Image& Video Processing], Bovik, A., ed., 183–202, ELSEVIER, 2nd ed. (2005).[2] Craven, P. and Wahba, G., “Smoothing noisy data with spline functions,” Numer. Math. 31,377–403 (1979).SPIE-IS&T/ Vol. 8296 82960N-11Downloaded from SPIE Digital Library on 06 Apr 2012 to 141.213.236.110. Terms of Use: http://spiedl.org/terms

[3] Deshpande, L. N. and Girard, D. A., “Fast computation of cross-validated robust splines andother non-linear smoothing splines,” in [Curves and Surfaces], Laurent, L. M. and Schumaker,eds., 143–8, Academic, Boston (1991).[4] Girard, D. A., “The fast Monte-Carlo Cross-Validation and C L procedures: Comments, newresults and application to image recovery problems - Rejoinder,” Computation. Stat. 10, 251–258 (1995).[5] Stein, C., “Estimation of the mean of a multivariate normal distribution,” Ann. Stat. 9, 1135–51 (Nov. 1981).[6] Eldar, Y. C., “Generalized SURE for exponential families: applications to regularization,”IEEE Trans. Sig. Proc. 57, 471–81 (Feb. 2009).[7] Vonesch, C., Ramani, S., and Unser, M., “Recursive risk estimation for non-linear imagedeconvolution with a wavelet-domain sparsity constraint,” Proc. IEEE Intl. Conf. Img. Proc.(ICIP’08) , 665–8 (2008).[8] Giryes, R., Elad, M., and Eldar, Y. C., “The projected GSURE for automatic parameter tuningin iterative shrinkage methods,” Applied and Computational Harmonic Analysis 30, 407–22(May 2011).[9] Blu, T. and Luisier, F., “The SURE-LET approach to image denoising,” IEEE Trans. Im.Proc. 16, 2778–86 (Nov. 2007).[10] Ramani, S., Blu, T., and Unser, M., “Monte-Carlo SURE: A black-box optimization of regularizationparameters for general denoising algorithms,” IEEE Trans. Im. Proc. 17, 1540–54(Sept. 2008).[11] Ville, D. V. D. and Kocher, M., “Nonlocal means with dimensionality reduction and SUREbasedparameter selection,” IEEE Trans. Im. Proc. 20(9), 2683–90 (2011).[12] Thompson, A. M., Brown, J. C., Kay, J. W., and Titterington, D. M., “A study of methods ofchoosing the smoothing parameter in image restoration by regularization,” IEEE Trans. Patt.Anal. Mach. Intell. 13(4), 326–339 (1991).[13] Ramani, S. and Fessler, J. A., “An accelerated iterative reweighted least squares algorithm forcompressed sensing MRI,” in [Proc. IEEE Intl. Symp. Biomed. Imag.], 257–60 (2010).[14] Ramani, S., Rosen, J., Liu, Z., and Fessler, J. A., “Regularization parameter selection fornonlinear iterative image restoration and mri reconstruction using GCV and SURE-basedmethods,” IEEE Trans. Im. Proc. (submitted).[15] Selesnick, I. W. and Figueiredo, M. A. T., “Signal restoration with overcomplete wavelettransforms: comparison of analysis and synthesis priors,” in [SPIE-7446], 74460D (2009).Wavelets XIII.[16] Fair, R. C., “On the robust estimation of econometric models,” Ann. Econ. Social Measurement2, 667–77 (Oct. 1974).[17] Hutchinson, M. F., “A stochastic estimator for the trace of the influence matrix for Laplaciansmoothing splines,” Comm. in Statistics - Simulation and Computation 19(2), 433–50 (1990).[18] Hackbusch, W., [Iterative Solution of Large Sparse Systems of Equations], Springer-Verlag,New York (1994).[19] Golub, G. H. and Loan, C. F. V., [Matrix Computations], The Johns Hopkins University Press,Baltimore, MD, USA, third ed. (1996, p. 50).[20] Hjorungnes, A. and Gesbert, D., “Complex-valued matrix differentiation: techniques and keyresults,” IEEE Trans. Sig. Proc. 55(6), 2740–6 (2007).SPIE-IS&T/ Vol. 8296 82960N-12Downloaded from SPIE Digital Library on 06 Apr 2012 to 141.213.236.110. Terms of Use: http://spiedl.org/terms