Views
5 years ago

Non-local Sparse Models for Image Restoration - Département d ...

Non-local Sparse Models for Image Restoration - Département d ...

σ 5 10 15 20 25 50 100

σ 5 10 15 20 25 50 100 house 39.93 36.96 35.35 34.16 33.15 30.04 25.83 peppers 38.18 34.80 32.82 31.37 30.21 26.62 23.00 camera. 38.32 34.21 32.01 30.57 29.51 26.42 23.08 lena 38.69 35.83 34.15 32.90 31.87 28.87 25.82 barbara 38.48 34.97 33.00 31.57 30.47 27.06 23.59 boat 37.35 34.02 32.20 30.89 29.87 26.74 23.84 hill 37.17 33.67 31.89 30.71 29.80 27.05 24.44 couple 37.45 33.98 32.06 30.69 29.61 26.30 23.28 man 37.89 34.06 32.01 30.64 29.63 26.69 24.00 fingerp. 36.70 32.57 30.31 28.78 27.62 24.25 21.26 bridge 35.78 31.22 28.92 27.46 26.42 23.68 21.46 flintst. 36.13 32.46 30.78 29.63 28.71 25.16 21.10 Av. 37.67 34.06 32.12 30.78 29.74 26.57 23.39 Table1.Quantitativedenoisingexperimentson12standardimages. ThePSNRvaluesareaveragedover 5experimentswith 5 differentnoiserealizationsandvaluesof σbetween5and100. Thevarianceisnegligibleandnotreportedduetospacelimitations. σ [22] [24] [11] [7] SC LSC LSSC 5 37.05 37.03 37.42 37.62 37.46 37.66 37.67 10 33.34 33.11 33.62 34.00 33.76 33.98 34.06 15 31.31 30.99 31.58 32.05 31.72 31.99 32.12 20 29.91 29.62 30.18 30.73 30.29 30.60 30.78 25 28.84 28.36 29.10 29.72 29.18 29.52 29.74 50 25.66 24.36 25.61 26.38 25.83 26.18 26.57 100 22.80 21.36 22.10 23.25 22.46 22.62 23.39 Table2.Quantitativecomparativeevaluation.WecompareouralgorithmtoGSM[22],FoE[24],K-SVD[11]andBM3D[7],thatwerethetopperformerssofaronthisbenchmark,andwhoseimplementationsareavailableonline.ThePSNRischosenasbefore asperformancemeasure.Bestresultsareinbold. Figure2.Qualitativeevaluationofourdenoisingmethodwith standardimages. Left: noisyimages. Right: restoredimages. Notethatwereproducetheoriginalbricktextureinthehouseimage(σ = 15)andthehairtextureforthemanimage(σ = 50), bothhardlyvisibleinthenoisyimages.(Thedetailsarebetterseen byzoomingonacomputerscreen.) Figure3.Left: DemosaickingwithLSCsometimescausesartefactssuchastheyellowandbluepixelsinthemiddleofthefence. Right:ThereconstructionobtainedwiththeLSSCalgorithmdoes notexhibitsuchartefacts.(Thisfigureshouldbeviewedincolor.) havebeenusedforall24photos. WeevaluatetheperformanceofthethreevariantsSC, LSC,LSSCofourframeworkdefinedintheprevioussubsection,andcomparethemwiththestateoftheartusing theexperimentalprotocolofPaliyetal.[20]whoseLPA methodis,tothebestofourknowledge,thetopperformer sofarintermsofPSNR(orequivalentlymean-squarederror)ontheKodakPhotoCDbenchmark.Following[20],we haveexcludeda15-pixelborderinfairnesstomethodsthat aresusceptibletoboundaryeffects.Table3addsourresults tothosereportedin[20]foreachoneofthe24photos.The proposedLSSCmethodoutperformsthestate-of-the-artalgorithmsAP[13],DL[32]andLPA[20]byasignificant marginof 0.87dBeventhoughourformulationisgeneric andnottunedtothetaskofdemosaicking,demonstrating thepromiseofourimagemodel. Whenincludingtheimagebordersoastobeabletocompareourresultswiththoseof[15],itisinterestingtonote that,intheSCsetting,weachieveameanPSNRof 40.72dB onthe 24images,comparedtothe 39.56dBof[15].Clearly, itisthuspreferableinthiscasetolearnthedictionaryfrom alargedatasetofnaturalimages.WithLSC,weachievea meanPSNRof 40.98dB,comparedtothe 40.32dBof[15], reachingameanPSNRof 41.24dBwithLSSC.Although thisquantitativeimprovementmayseemsmall,itisqualitativelyquitesignificant.EventhoughSCandLSCperformverywellintermsofPSNR,theysufferfromclassicaldemosaickingartefacts,asshownbytheexampleofFigure3. Ontheotherhand,ournewLSSCmodel,whichexploits self-similaritiesaswellaslearnedsparsecoding,isusually freeofmostoftheseartefacts. 4.3.Denoising–RealNoise Toevaluatequalitativelyourdenoisingmethodonreal images, wehavetakenthreeRAWphotographsusinga CanonPowershotG9digitalcameraat1600ISOwitha shorttimeexposure. Atsuchasetting, theimagesare quitenoisy. Wehaveextractedthemosaickeddatafrom theRAWimageusingtheopen-sourcedcrawsoftware.We havethenscaledmanuallytheR,G,Bchannelssothatthey

Im. AP DL LPA SC LSC LSSC 1 37.84 38.46 40.47 40.84 40.92 41.36 2 39.64 40.89 41.36 41.76 42.03 42.24 3 41.40 42.66 43.47 43.15 43.92 44.24 4 39.92 40.49 40.84 41.99 42.14 42.45 5 37.28 38.07 37.51 38.72 39.15 39.45 6 38.69 40.19 40.92 41.29 41.36 41.71 7 41.75 42.35 43.06 43.30 43.59 44.06 8 35.58 36.02 37.13 37.42 37.38 37.57 9 41.84 43.05 43.50 43.17 43.74 43.83 10 41.93 42.54 42.77 43.01 43.17 43.33 11 39.25 40.01 40.51 41.19 41.29 41.51 12 42.62 43.45 44.01 44.29 44.49 44.90 13 34.28 34.75 36.08 36.16 36.29 36.35 14 35.66 36.91 36.86 37.64 38.48 38.77 15 39.17 39.82 40.09 41.04 41.24 41.74 16 42.10 43.75 44.02 44.36 44.42 44.91 17 41.23 41.68 41.75 41.75 41.86 41.98 18 37.31 37.64 37.59 38.05 38.27 38.38 19 39.99 41.01 41.55 41.58 41.71 42.31 20 40.63 41.24 41.48 41.95 42.25 42.27 21 38.72 39.10 39.61 40.55 40.59 40.65 22 37.63 38.37 38.44 38.73 38.97 39.24 23 41.93 43.22 43.92 43.47 43.93 44.34 24 34.74 35.55 35.44 35.59 35.85 35.89 Av. 39.21 40.05 40.52 40.88 41.13 41.39 Table3.Comparisonofdemosaickingperformanceintermsof PSNRbetweenAP[13],DL[32],LPA[20]andtheSC,LSCand LSSCvariantsofourmethod.Bestresultsareinbold. visuallyappeartocontainsimilaramountsofnoise.Atthis point,thenoiseis,toafirstapproximation,roughlyuniform,andweapplyourdenoisingalgorithmtothescaled mosaickedimage,beforeperformingdemosaicking,white balance,sRGBspaceconversion,gammacorrection,and contrastenhancementtoreconstructthefinalimage. This approachhasprovenexperimentallytoleadtobetterresultsthandenoisingeachR,G,Bchannelindependently.Of course,assumingthatthenoiseisuniformisonlyarough approximation. Non-spatiallyuniformnoisemodelsare availableforspecificcameras,andexploitedbycommercialsoftwarepackagessuchatthosediscussedlaterinthis section.Incorporatingthesemodelsintoourframeworkis feasible(following[15]),butbeyondthescopeofthispaper. Instead,wedemonstratethat,evenwithauniformassumption,ouralgorithmisqualitativelycompetitivewithtop-ofthe-linecommercialdenoisingsoftware. Theparameterswehaveusedareapatchsizeof m = 8 × 8pixels,and k = 256dictionaryelements,whichis typicalforsparsecodingmethods[11,15].Thenoiselevel σisestimatedbytheuserandassumedtobeuniformacross theimage,and ξischosenaccordingtotheempiricalrule presentedinSection4.1.Demosaickingisperformedusing thesameparametersasinSection4.2.Figure4compares closeupsoftheimagesreconstructedfromtheRAWfile bythecameraitself(jpegoutput),theimageobtainedwith AdobeCameraRaw5.0(nodenoising),twostate-of-the-art denoisingsoftwaresNoiseWare4.2andtheDxOOpticsPro 5.3package,andourmethod. Thecommercialprograms havebeenrunwiththeirdefaultparameters,andthesecould certainlybefurthertunedtoimproveimagequalityabit. 6 However,notethat,unlikeours,theseprogramsdotakeadvantageofadetailed,non-uniformnoisemodelspecificto thecamera,yetdonotappeartogivequalitativelybetter results. Althoughaquantitativecomparisonisnotpossible,webelieve(subjectively)thatourmethoddoesbeston thefirstandthirdimages,whileDxOOpticsProisslightly betterforthesecondone.Asinourpreviousexperiments, LSSCsuffersfromfewerartefactsthanLSCingeneral.The noise’snon-uniformitydoesnotseemtoaffectourresults much,exceptperhapsforthebackgroundofthethirdimage,wherepartofthenoiseisreconstructed. 5.Conclusion Wehaveproposedinthispaperanewimagemodel thatcombinesthenon-localmeansandsparsecodingapproaches to image restoration into a unified framework wheresimilarpatchesaredecomposedusingsimilarsparsitypatterns.Quantitativeandqualitativeexperimentswith imagescorruptedwithsyntheticorrealnoisehaveshown thattheproposedalgorithmoutperformsthestateofthe artinimagedemosaickinganddenoisingtasks. Nexton ouragendaistoincludenon-uniformnoisemodelsinthe reconstructionprocess,thenadaptourapproachtoother challengingimagemanipulationproblemsincomputational photography,includingdeblurring,inpainting,andtexture synthesisinstillimagesandvideosequences. Acknowledgments ThispaperwassupportedinpartbyANRundergrant MGA.TheworkofGuillermoSapiroispartiallysupported byNSF,NGA,ONR,ARO,andDARPA.Wewouldlike tothankFrédéricGuichardandFrédéricCaofromDxOfor interestingdiscussions. References [1] S. Awate and R. Whitaker. Unsupervised, informationtheoretic, adaptive image filtering for image restoration. IEEET.PAMI,364–376,2006. [2] M.Bertalmio,G.Sapiro,V.Caselles,andC.Ballester.Image inpainting.InProc.Comp.Graph.andInteract.Tech.,2000. [3] A.Buades,B.Coll,andJ.Morel.Anon-localalgorithmfor imagedenoising.InProc.IEEECVPR,2005. [4] A.Buades,B.Coll,J.Morel,andC.Sbert.Non-localdemosaicing.Technicalreport,2007.PreprintCMLA2007-15. [5] S.Chen,D.Donoho,andM.Saunders.Atomicdecompositionbybasispursuit.SIAMJ.Sc.Comp.,20:33–61,1999. 6 NotethatNoiseWaredoesnotprocessdirectlytheRAWfiles,butrequiresatfirsttouseademosaickingsoftware.Wehavechosentocombine NoiseWareandAdobeCameraRawinourexperiments.

Desingularization of non local dynamic models by means of ...
Local and Non-Metric Similarities Between Images ... - Pixel shaker
Carved Visual Hulls for Image-Based Modeling - Département d ...
Centralized Sparse Representation for Image Restoration
Online Dictionary Learning for Sparse Coding - Département d ...
Effective Separation of Sparse and Non-Sparse Image Features for ...
IMAGE RESTORATION USING A SPARSE QUADTREE ...
Non Local Image Restoration Using Orientation Optimization By ...
Image restoration by sparse 3D transform-domain ... - CiteSeerX
Sparse Coding Models of Natural Images: Algorithms for Efficient ...
VARIATIONAL RESTORATION OF NON-FLAT IMAGE FEATURES ...
Image Interpolation, Restoration, and Vector Set Models
Sparse kernel regression modeling using combined locally ...
Predictive spatio-temporal models for spatially sparse ... - IMAGe
Image and Video Restoration via Non-Local Kernel Regression
Sparse PDF Maps for Non-Linear Multi-Resolution Image Operations
Sparse Models for Adversarial Learning
Training sparse natural image models with a fast Gibbs sampler of ...
Restoring Warped Document Images through 3D Shape Modeling
Shape Reconstruction and Image Restoration for Non-Flat Surfaces ...
Restoration of images corrupted by additive non ... - IEEE Xplore
Local Image Reconstruction and Sub-Pixel Restoration Algorithms
Image Deblurring and Super-Resolution by Adaptive Sparse ...
Adaptive image restoration using a generalized Gaussian model for ...
Image Deblurring and Super-resolution by Adaptive Sparse Domain ...
Image Restoration and Reconstruction & Color Imaging and ...
Non-negative Local Coordinate Factorization for Image ... - People
Adaptive LASSO for sparse high-dimensional regression models
Sparse Bayesian Modeling With Adaptive Kernel Learning