Views
5 years ago

Non-local Sparse Models for Image Restoration - Département d ...

Non-local Sparse Models for Image Restoration - Département d ...

σ 5 10 15 20 25 50 100

σ 5 10 15 20 25 50 100 house 39.93 36.96 35.35 34.16 33.15 30.04 25.83 peppers 38.18 34.80 32.82 31.37 30.21 26.62 23.00 camera. 38.32 34.21 32.01 30.57 29.51 26.42 23.08 lena 38.69 35.83 34.15 32.90 31.87 28.87 25.82 barbara 38.48 34.97 33.00 31.57 30.47 27.06 23.59 boat 37.35 34.02 32.20 30.89 29.87 26.74 23.84 hill 37.17 33.67 31.89 30.71 29.80 27.05 24.44 couple 37.45 33.98 32.06 30.69 29.61 26.30 23.28 man 37.89 34.06 32.01 30.64 29.63 26.69 24.00 fingerp. 36.70 32.57 30.31 28.78 27.62 24.25 21.26 bridge 35.78 31.22 28.92 27.46 26.42 23.68 21.46 flintst. 36.13 32.46 30.78 29.63 28.71 25.16 21.10 Av. 37.67 34.06 32.12 30.78 29.74 26.57 23.39 Table1.Quantitativedenoisingexperimentson12standardimages. ThePSNRvaluesareaveragedover 5experimentswith 5 differentnoiserealizationsandvaluesof σbetween5and100. Thevarianceisnegligibleandnotreportedduetospacelimitations. σ [22] [24] [11] [7] SC LSC LSSC 5 37.05 37.03 37.42 37.62 37.46 37.66 37.67 10 33.34 33.11 33.62 34.00 33.76 33.98 34.06 15 31.31 30.99 31.58 32.05 31.72 31.99 32.12 20 29.91 29.62 30.18 30.73 30.29 30.60 30.78 25 28.84 28.36 29.10 29.72 29.18 29.52 29.74 50 25.66 24.36 25.61 26.38 25.83 26.18 26.57 100 22.80 21.36 22.10 23.25 22.46 22.62 23.39 Table2.Quantitativecomparativeevaluation.WecompareouralgorithmtoGSM[22],FoE[24],K-SVD[11]andBM3D[7],thatwerethetopperformerssofaronthisbenchmark,andwhoseimplementationsareavailableonline.ThePSNRischosenasbefore asperformancemeasure.Bestresultsareinbold. Figure2.Qualitativeevaluationofourdenoisingmethodwith standardimages. Left: noisyimages. Right: restoredimages. Notethatwereproducetheoriginalbricktextureinthehouseimage(σ = 15)andthehairtextureforthemanimage(σ = 50), bothhardlyvisibleinthenoisyimages.(Thedetailsarebetterseen byzoomingonacomputerscreen.) Figure3.Left: DemosaickingwithLSCsometimescausesartefactssuchastheyellowandbluepixelsinthemiddleofthefence. Right:ThereconstructionobtainedwiththeLSSCalgorithmdoes notexhibitsuchartefacts.(Thisfigureshouldbeviewedincolor.) havebeenusedforall24photos. WeevaluatetheperformanceofthethreevariantsSC, LSC,LSSCofourframeworkdefinedintheprevioussubsection,andcomparethemwiththestateoftheartusing theexperimentalprotocolofPaliyetal.[20]whoseLPA methodis,tothebestofourknowledge,thetopperformer sofarintermsofPSNR(orequivalentlymean-squarederror)ontheKodakPhotoCDbenchmark.Following[20],we haveexcludeda15-pixelborderinfairnesstomethodsthat aresusceptibletoboundaryeffects.Table3addsourresults tothosereportedin[20]foreachoneofthe24photos.The proposedLSSCmethodoutperformsthestate-of-the-artalgorithmsAP[13],DL[32]andLPA[20]byasignificant marginof 0.87dBeventhoughourformulationisgeneric andnottunedtothetaskofdemosaicking,demonstrating thepromiseofourimagemodel. Whenincludingtheimagebordersoastobeabletocompareourresultswiththoseof[15],itisinterestingtonote that,intheSCsetting,weachieveameanPSNRof 40.72dB onthe 24images,comparedtothe 39.56dBof[15].Clearly, itisthuspreferableinthiscasetolearnthedictionaryfrom alargedatasetofnaturalimages.WithLSC,weachievea meanPSNRof 40.98dB,comparedtothe 40.32dBof[15], reachingameanPSNRof 41.24dBwithLSSC.Although thisquantitativeimprovementmayseemsmall,itisqualitativelyquitesignificant.EventhoughSCandLSCperformverywellintermsofPSNR,theysufferfromclassicaldemosaickingartefacts,asshownbytheexampleofFigure3. Ontheotherhand,ournewLSSCmodel,whichexploits self-similaritiesaswellaslearnedsparsecoding,isusually freeofmostoftheseartefacts. 4.3.Denoising–RealNoise Toevaluatequalitativelyourdenoisingmethodonreal images, wehavetakenthreeRAWphotographsusinga CanonPowershotG9digitalcameraat1600ISOwitha shorttimeexposure. Atsuchasetting, theimagesare quitenoisy. Wehaveextractedthemosaickeddatafrom theRAWimageusingtheopen-sourcedcrawsoftware.We havethenscaledmanuallytheR,G,Bchannelssothatthey

Im. AP DL LPA SC LSC LSSC 1 37.84 38.46 40.47 40.84 40.92 41.36 2 39.64 40.89 41.36 41.76 42.03 42.24 3 41.40 42.66 43.47 43.15 43.92 44.24 4 39.92 40.49 40.84 41.99 42.14 42.45 5 37.28 38.07 37.51 38.72 39.15 39.45 6 38.69 40.19 40.92 41.29 41.36 41.71 7 41.75 42.35 43.06 43.30 43.59 44.06 8 35.58 36.02 37.13 37.42 37.38 37.57 9 41.84 43.05 43.50 43.17 43.74 43.83 10 41.93 42.54 42.77 43.01 43.17 43.33 11 39.25 40.01 40.51 41.19 41.29 41.51 12 42.62 43.45 44.01 44.29 44.49 44.90 13 34.28 34.75 36.08 36.16 36.29 36.35 14 35.66 36.91 36.86 37.64 38.48 38.77 15 39.17 39.82 40.09 41.04 41.24 41.74 16 42.10 43.75 44.02 44.36 44.42 44.91 17 41.23 41.68 41.75 41.75 41.86 41.98 18 37.31 37.64 37.59 38.05 38.27 38.38 19 39.99 41.01 41.55 41.58 41.71 42.31 20 40.63 41.24 41.48 41.95 42.25 42.27 21 38.72 39.10 39.61 40.55 40.59 40.65 22 37.63 38.37 38.44 38.73 38.97 39.24 23 41.93 43.22 43.92 43.47 43.93 44.34 24 34.74 35.55 35.44 35.59 35.85 35.89 Av. 39.21 40.05 40.52 40.88 41.13 41.39 Table3.Comparisonofdemosaickingperformanceintermsof PSNRbetweenAP[13],DL[32],LPA[20]andtheSC,LSCand LSSCvariantsofourmethod.Bestresultsareinbold. visuallyappeartocontainsimilaramountsofnoise.Atthis point,thenoiseis,toafirstapproximation,roughlyuniform,andweapplyourdenoisingalgorithmtothescaled mosaickedimage,beforeperformingdemosaicking,white balance,sRGBspaceconversion,gammacorrection,and contrastenhancementtoreconstructthefinalimage. This approachhasprovenexperimentallytoleadtobetterresultsthandenoisingeachR,G,Bchannelindependently.Of course,assumingthatthenoiseisuniformisonlyarough approximation. Non-spatiallyuniformnoisemodelsare availableforspecificcameras,andexploitedbycommercialsoftwarepackagessuchatthosediscussedlaterinthis section.Incorporatingthesemodelsintoourframeworkis feasible(following[15]),butbeyondthescopeofthispaper. Instead,wedemonstratethat,evenwithauniformassumption,ouralgorithmisqualitativelycompetitivewithtop-ofthe-linecommercialdenoisingsoftware. Theparameterswehaveusedareapatchsizeof m = 8 × 8pixels,and k = 256dictionaryelements,whichis typicalforsparsecodingmethods[11,15].Thenoiselevel σisestimatedbytheuserandassumedtobeuniformacross theimage,and ξischosenaccordingtotheempiricalrule presentedinSection4.1.Demosaickingisperformedusing thesameparametersasinSection4.2.Figure4compares closeupsoftheimagesreconstructedfromtheRAWfile bythecameraitself(jpegoutput),theimageobtainedwith AdobeCameraRaw5.0(nodenoising),twostate-of-the-art denoisingsoftwaresNoiseWare4.2andtheDxOOpticsPro 5.3package,andourmethod. Thecommercialprograms havebeenrunwiththeirdefaultparameters,andthesecould certainlybefurthertunedtoimproveimagequalityabit. 6 However,notethat,unlikeours,theseprogramsdotakeadvantageofadetailed,non-uniformnoisemodelspecificto thecamera,yetdonotappeartogivequalitativelybetter results. Althoughaquantitativecomparisonisnotpossible,webelieve(subjectively)thatourmethoddoesbeston thefirstandthirdimages,whileDxOOpticsProisslightly betterforthesecondone.Asinourpreviousexperiments, LSSCsuffersfromfewerartefactsthanLSCingeneral.The noise’snon-uniformitydoesnotseemtoaffectourresults much,exceptperhapsforthebackgroundofthethirdimage,wherepartofthenoiseisreconstructed. 5.Conclusion Wehaveproposedinthispaperanewimagemodel thatcombinesthenon-localmeansandsparsecodingapproaches to image restoration into a unified framework wheresimilarpatchesaredecomposedusingsimilarsparsitypatterns.Quantitativeandqualitativeexperimentswith imagescorruptedwithsyntheticorrealnoisehaveshown thattheproposedalgorithmoutperformsthestateofthe artinimagedemosaickinganddenoisingtasks. Nexton ouragendaistoincludenon-uniformnoisemodelsinthe reconstructionprocess,thenadaptourapproachtoother challengingimagemanipulationproblemsincomputational photography,includingdeblurring,inpainting,andtexture synthesisinstillimagesandvideosequences. Acknowledgments ThispaperwassupportedinpartbyANRundergrant MGA.TheworkofGuillermoSapiroispartiallysupported byNSF,NGA,ONR,ARO,andDARPA.Wewouldlike tothankFrédéricGuichardandFrédéricCaofromDxOfor interestingdiscussions. References [1] S. Awate and R. Whitaker. Unsupervised, informationtheoretic, adaptive image filtering for image restoration. IEEET.PAMI,364–376,2006. [2] M.Bertalmio,G.Sapiro,V.Caselles,andC.Ballester.Image inpainting.InProc.Comp.Graph.andInteract.Tech.,2000. [3] A.Buades,B.Coll,andJ.Morel.Anon-localalgorithmfor imagedenoising.InProc.IEEECVPR,2005. [4] A.Buades,B.Coll,J.Morel,andC.Sbert.Non-localdemosaicing.Technicalreport,2007.PreprintCMLA2007-15. [5] S.Chen,D.Donoho,andM.Saunders.Atomicdecompositionbybasispursuit.SIAMJ.Sc.Comp.,20:33–61,1999. 6 NotethatNoiseWaredoesnotprocessdirectlytheRAWfiles,butrequiresatfirsttouseademosaickingsoftware.Wehavechosentocombine NoiseWareandAdobeCameraRawinourexperiments.

Desingularization of non local dynamic models by means of ...
Local and Non-Metric Similarities Between Images ... - Pixel shaker
Centralized Sparse Representation for Image Restoration
Carved Visual Hulls for Image-Based Modeling - Département d ...
Online Dictionary Learning for Sparse Coding - Département d ...
Effective Separation of Sparse and Non-Sparse Image Features for ...
Image restoration by sparse 3D transform-domain ... - CiteSeerX
Non Local Image Restoration Using Orientation Optimization By ...
IMAGE RESTORATION USING A SPARSE QUADTREE ...
Sparse Coding Models of Natural Images: Algorithms for Efficient ...
Image Interpolation, Restoration, and Vector Set Models
VARIATIONAL RESTORATION OF NON-FLAT IMAGE FEATURES ...
Image and Video Restoration via Non-Local Kernel Regression
Predictive spatio-temporal models for spatially sparse ... - IMAGe
Sparse kernel regression modeling using combined locally ...
Sparse PDF Maps for Non-Linear Multi-Resolution Image Operations
Sparse Models for Adversarial Learning
Restoration of images corrupted by additive non ... - IEEE Xplore
Shape Reconstruction and Image Restoration for Non-Flat Surfaces ...
Local Image Reconstruction and Sub-Pixel Restoration Algorithms
Image Deblurring and Super-resolution by Adaptive Sparse Domain ...
Adaptive image restoration using a generalized Gaussian model for ...
Restoring Warped Document Images through 3D Shape Modeling
Training sparse natural image models with a fast Gibbs sampler of ...
Non-negative Local Coordinate Factorization for Image ... - People
Adaptive LASSO for sparse high-dimensional regression models
Sparse Bayesian Modeling With Adaptive Kernel Learning
adaptive lasso for sparse high-dimensional regression models
adaptive image restoration using a local neural ... - PASCAL EPrints