Views
5 years ago

Non-local Sparse Models for Image Restoration - Département d ...

Non-local Sparse Models for Image Restoration - Département d ...

fixedorthogonaldictionaries). Experimentswithimages corruptedbysyntheticorrealnoiseshowthattheproposed methodoutperformsthestateoftheartinbothimagedenoisingandimagedemosaickingtasks,makingitpossible toeffectivelyrestorerawimagesfromdigitalcamerasata reasonablespeedandmemorycost.Furthermore,although itisdemonstratedonimagedenoisinganddemosaicking tasksinthispaper,ourmodelisgeneric,admitsstraightforwardextensionstovariousimageandvideorestoration taskssuchasinpainting,andcanadapttoalargeclassof data,e.g.,multispectralimagesorMRIdata. 2.RelatedWork Westartwithabriefdescriptionofwell-establishedapproachestoimagerestorationthatarerelevantandrelatedtotheapproachproposedinthenextsection.Sinceitisdifficulttodesignastandardmodelfordigitalcameranoise, thesemethodsassumewhiteGaussiannoise.Eventhough thisgenericsettingslightlydiffersfromthatofrealimage denoising,ithasallowedthedevelopmentofeffectivealgorithmsthatarenowwidelyusedindigitalcamerasand commercialsoftwarepackages. Wewillusethesameassumptionintherestofthispaper,butwilldemonstrateempiricallythatourapproachiseffectiveatrestoringrealimagescorruptedbynon-Gaussian,non-uniformnoise. 2.1.Non­LocalMeansFiltering EfrosandLeungshowedin[10]thattheself-similarities inherent to natural images could effectively be used in texturesynthesistasks. Followingtheirinsight,Buades, CollandMorelintroducedin[3]thenon-localmeansapproachtoimagedenoising,wheretheprominenceofselfsimilaritiesisusedasaprioronnaturalimages. 2 Concretely,letusconsideranoisyimagewrittenasacolumn vector yin R n ,anddenoteby y[i]the i-thpixelandby y i thepatchofsize mcenteredonthispixelforsomeappropriatesizem.Thisapproachexploitsthesimplebutveryeffectiveideathattwopixelsassociatedwithsimilarpatches y i and y jshouldhavesimilarvalues y[i]and y[j].Using y ias anexplanatoryvariablefor y[i]leadstothenon-localmeans formulation,wherethedenoisedpixel x[i]isobtainedbya weightedaverage(thecorrespondingNadaraya-Watsonestimator[3]): x[i] = n� j=1 Kh(yi − y � j) n l=1 Kh(y y[j], (1) i − yl) and KhisaGaussiankernelofbandwidth h. 2 Thisideahasinfactappearedintheliteratureinvariousguisesandunderdifferentequivalentinterpretations,e.g.,kerneldensityestimation[10],Nadaraya-Watsonestimators[3],mean-shiftiterations[1],diffusionprocessesongraphs[26],andlong-rangerandomfields[14]. 2.2.LearnedSparseCoding Analternativeistoassumethatthecleansignalcanbe approximatedbyasparselinearcombinationofelements fromabasissetcalleddictionary. Underthisassumption, denoisingapatch y iin R m withadictionary Din R m×k composedof kelements,amountstosolvingthesparsedecompositionproblem min αi∈Rk ||αi||p s.t. ||yi − Dα|| 2 2 ≤ ε, (2) where Dαisanestimateofthecleansignal,and ||α||pis asparsity-inducingregularizationterm.Thisregularizeris associatedwiththe ℓ1normwhen p = 1,leadingtothe well-knownLasso[27]andbasispursuit[5]problems,and withthe ℓ0pseudonormwhen p = 0. 3 Notethatthedictionarymaybeovercomplete—thatis,thenumberofcolumns of Dmaybegreaterthanthenumberofitsrows.Following [11,15], εcanbechosenaccordingtothe(supposedknown) standarddeviation σofthenoise. Oneindeedexpectsthe residual y i −DαitobehaveasaGaussianvector,andthus ||y i − Dαi|| 2 2/σ 2 tofollowachi-squareddistribution χ 2 m concentratedaround m. Thestrategyproposedin[15]is tothresholdthecumulativedistributionfunction Fmofthe χ2 mdistributionandchoose εas ε = σ2F −1 m (τ),where F −1 m istheinverseof Fm.Selectingthevalue τ = 0.9leadsin practicetoacceptablevaluesof ε[15]. Varioustypesofwavelets[17]havebeenusedasdictionariesfornaturalimages. Buildingonideasproposed in[19]tomodelneuronalresponsesintheV1areaofthe brain,EladandAharon[11]haveproposedinsteadtolearn adictionary Dadaptedtotheimageathand,anddemonstratedthatlearneddictionariesleadtobetterempiricalperformancethanoff-the-shelfones.Sinceimagesmaybeverylarge,efficiencyconcernsnaturallyleadtosparselydecomposingimagepatchesratherthanthefullimage. Foran imageofsize n,adictionaryin Rm×kadaptedtothe n overlappingpatchesofsize m(typically m = 8 × 8 ≪ n) associatedwiththeimagepixels,islearnedbyaddressing thefollowingoptimizationproblem min D∈C,A i=1 n� ||αi||p s.t. ||yi − Dαi|| 2 2 ≤ ε, (3) where Cisthesetofmatricesin R m×k withunit ℓ2-norm columns, A = [α1,...,αn]isamatrixin R k×n , y iisthe i-thpatchofthenoisyimage y, αiisthecorresponding code,and Dαiistheestimateofthedenoisedpatch.Note thatthisprocedureimplicitlyassumesthatthepatchesare independentfromeachother,whichisquestionablesince 3The ℓpnormofavector xin Rm △ isdefined,for p ≥ 1,by ||x||p = ( Pm i=1 |x[i]|p ) 1/p .Followingtradition,wedenoteby ||x||0thenumber ofnonzeroelementsofthevector x.This“ℓ0”sparsitymeasureisnota truenorm.

theyoverlap.However,thisapproximationmakesthecorrespondingoptimizationtractable.Indeed,althoughdictionarylearningistraditionallyconsideredasextremelycostly,onlineproceduressuchas[16]makeitpossibletoefficientlyprocessmillionsofpatches,allowingtheuseoflarge photographsand/orlargeimagedatabases. Oncethedictionary Dandcodes αihavebeenlearned, everypixeladmits mestimates(oneperpatchcontaining it),anditsvaluecanbecomputedbyaveragingthese: x = 1 m n� RiDαi, (4) i=1 where Riin R n×m isthebinarymatrixwhichplacespatch number iatitsproperpositionintheimage.Thisapproach learnsthedictionaryonthesetofoverlappingnoisypatches, therebyadaptingthedictionarytotheimageitself,whichis akeyelementinobtainingbetterresults. Howtochoosebetween p = 0or p = 1isnotaprioriclear. SolvingEq.(2)with p = 0isNPhard,leading toapproximatesolutionsobtainedwithagreedyalgorithm suchasforwardselection[30](alsoknownasorthogonal matchingpursuit[18]).When p = 1,theproblemisconvex andcanbesolvedefficientlywiththeLARSalgorithm[9]. FollowingEladandAharon[11],wehaveobservedexperimentallythat,givenafixeddictionary D,thereconstructed imageisingeneralofbetterqualitywhenusingthe ℓ0 pseudonormratherthanitsconvex ℓ1counterpart. However,wehavealsoobservedthatdictionarieslearnedwith the ℓ1normareusuallybetterfordenoising,evenwhenthe finalreconstructionisdonewiththe ℓ0pseudonorm. 2.3.BlockMatching3D(BM3D) Dabovetal.proposein[7]apatch-basedprocedurethat exploitsimageself-similaritiesandgivesstate-of-the-artresults. Asin[11],theyestimatethecodesofoverlapping patchesandaveragetheestimates.However,similartononlocalmeansfiltering[3],theyreconstructpatchesbyfinding similaronesintheimage(blockmatching),stackingthem togetherintoa3Dsignalblock,anddenoisingtheblock usinghardorsoftthresholding[8]witha3Dorthogonal dictionary(3Dfiltering).Inconjunctionwithafewheuristics, 4 thissimpleideahasproventobeveryefficientand givesbetterresultsthanregularnon-localmeans. Akey ideaofourpaperistoimplementasimilarjointdecompositionapproachinthecontextofsparsecodingwithlearned dictionaries,asexplainedinthenextsection. 3.ProposedFormulation Weshowinthissectionhowimageself-similaritiescan beusedtoimprovelearnedsparsemodelswithsimultane- 4 Namely,usingacombinationofweightedaveragesofoverlapping patches,Kaiserwindows,andWienerfilteringtofurtherimproveresults. Figure1.Sparsityvs.jointsparsity:Greysquaresrepresentsnonzerosvaluesinvectors(left)ormatrix(right). oussparsecoding,whichencouragessimilarpatchestoadmitsimilarsparsedecompositions. 3.1.SimultaneousSparseCoding A joint sparsity pattern—that is, a common set of nonzerocoefficients—canbeimposedtoasetofvectors α1,...,αlthroughagrouped-sparsityregularizeronthe matrix A = [α1,...,αl]in Rk×l (Figure1).Thisamounts torestrictingthenumberofnonzerorowsof A,orreplacing the ℓpvector(pseudo)norminEq.(3)bythe ℓp,q(pseudo) matrixnorm ||A||p,q △ = k� ||α i || p q, (5) i=1 where α i denotesthe i-throwof A.Inpractice,oneusually choosesforthepair (p,q)thevalues (1,2)or (0, ∞),the formerleadingtoaconvexnorm,whilethelatteractually countsthenumberofnonzerorowsandisonlyapseudo norm[28]. 3.2.PrincipleoftheFormulation Non-localmeansfilteringhasprovenveryeffectivein general,butitfailsinsomecases. Intheextreme,when apatchdoesnotlooklikeanyotheroneintheimage,it isimpossibletoexploitself-similaritiestodenoisethecorrespondingpixelvalue. Sparseimagemodelscanhandle suchsituationsbyexploitingtheredundancybetweenoverlappingpatches,buttheysufferfromanotherdrawback: Similarpatchessometimesadmitverydifferentestimates duetothepotentialinstabilityofsparsedecompositions(the ℓ0pseudonormis,afterall,piecewiseconstant,andits ℓ1 counterpartisonlypiecewisedifferentiable),whichcanresultinpracticeinnoticeablereconstructionartefacts.Inthis paper,weaddressthisproblembyforcingsimilarpatchesto admitsimilardecompositions.Concretely,letusdefinefor eachpatch y itheset Siofsimilarpatchesas Si △ = {j = 1,...,n s.t. ||y i − y j|| 2 2 ≤ ξ}, (6) where ξissomethreshold.Letusalsoconsiderforthemomentafixeddictionary Din Rm×k .Decomposingthepatch yiwithagrouped-sparsityregularizerontheset Siamounts tosolving min ||Ai||p,q s.t. Ai � ||yj − Dαij|| j∈Si 2 2 ≤ εi, (7)

Local and Non-Metric Similarities Between Images ... - Pixel shaker
Desingularization of non local dynamic models by means of ...
Free True Confessions of Nude Photography: A Step-By-Step Guide to Recruiting Beautiful Models, Lighting, Photographing Nudes, Post-Processing Images, and Maybe Even Getting Paid to Do It (3rd Edition) | Download file
Centralized Sparse Representation for Image Restoration
Carved Visual Hulls for Image-Based Modeling - Département d ...
Online Dictionary Learning for Sparse Coding - Département d ...
Effective Separation of Sparse and Non-Sparse Image Features for ...
Image restoration by sparse 3D transform-domain ... - CiteSeerX
Non Local Image Restoration Using Orientation Optimization By ...
IMAGE RESTORATION USING A SPARSE QUADTREE ...
Sparse Coding Models of Natural Images: Algorithms for Efficient ...
VARIATIONAL RESTORATION OF NON-FLAT IMAGE FEATURES ...
Image Interpolation, Restoration, and Vector Set Models
Sparse kernel regression modeling using combined locally ...
Predictive spatio-temporal models for spatially sparse ... - IMAGe
Image and Video Restoration via Non-Local Kernel Regression
Sparse PDF Maps for Non-Linear Multi-Resolution Image Operations
Sparse Models for Adversarial Learning
Restoration of images corrupted by additive non ... - IEEE Xplore
Shape Reconstruction and Image Restoration for Non-Flat Surfaces ...
Local Image Reconstruction and Sub-Pixel Restoration Algorithms
Training sparse natural image models with a fast Gibbs sampler of ...
Restoring Warped Document Images through 3D Shape Modeling
Adaptive image restoration using a generalized Gaussian model for ...
Image Deblurring and Super-resolution by Adaptive Sparse Domain ...
Image Deblurring and Super-Resolution by Adaptive Sparse ...
Non-negative Local Coordinate Factorization for Image ... - People
Adaptive LASSO for sparse high-dimensional regression models
Sparse Bayesian Modeling With Adaptive Kernel Learning