Exploiting Redundancy for Aerial Image Fusion using Convex ...

More documents

Recommendations

Info

2 Stefan Kluckner, Thomas Pock and Horst Bischof Fig. 1. An observed scene with overlapping camera positions. The scene is taken from different viewpoints. We exploit computed range images to transform the point cloud into a common orthographic aerial view. Our joint inpainting and denoising approach takes redundant observations as input data and generates an improved fused image. Note that some observations include many undefined areas (black pixels) caused by occlusions and non-stationary objects. and an orthographic view, which is related to novel view synthesis [2–4, 8]. Due to missing data in the individual height fields (e.g. caused by non-stationary objects or occlusions) the initial alignment causes undefined areas, artifacts or outliers in the novel view. Figure 1 depicts an urban scene taken from different camera positions and a set of redundant images, geometrically transformed to a common view. Some image tiles show large areas of missing information and erroneous pixel values. Our task can also be interpreted as an image fusion from multiple input observations of the same scene by joint inpainting and denoising. While inpainting fills undefined areas, the denoising removes strong outliers and noise by exploiting the high redundancy in the input data. This paper has several contributions: First, we present a novel variational framework for gray and color image fusion, which provides a smooth solution over the image domain by exploiting redundant input images. In order to compute natural appearing images we further introduce a wavelet transform [9], providing an improved texture prior for regularization, in our convex optimization framework (Section 3). In the experimental section we show that our framework can be successfully applied to image recovery and orthographic image generation in high-resolution aerial imagery (Section 4). In addition, we present results for exemplar-based inpainting, which enables an integrated removal of undesired, non-stationary objects like cars. Finally, Section 5 concludes our work and gives an outlook on future work. 2 Related Work The challenging task of reconstructing an original image from given (noisy) observations is known to be ill-posed. Although fast mean or median computation
Exploiting Redundancy for Aerial Image Fusion using Convex Optimization 3 over multiple pixel observations will suppress noisy or undefined areas, each pixel in the result is treated independently. A variety of proposed algorithms for a fusion of redundant information is based on image priors [2, 8], image transforms [10], Markov random field optimization procedures [3] and generative models [4]. Variational formulations are well-suited for finding smooth and consistent solutions of the inverse problem by exploiting different types of regularizations [7, 11–14]. The quadratic model [11] uses the L 2 norm for regularization, which causes smoothed edges. Introducing a total variation (TV) norm instead leads to the edge preserving denoising model proposed by Rudin, Osher and Fatemi (ROF) [12]. The authors in [13] proposed to also use a L 1 norm in the data term to estimate the deviation between sought solution and input observation. Thus the resulting TV-L 1 model is more effective in removing impulse noise containing strong outliers than the ROF model. Zach et al. [7] applied the TV-L 1 to robust range image integration from multiple views. Although TVbased methods are well suited for tasks like range data integration, in texture inpainting the regularization produces results that look unnatural near recovered edges (too much contrast). To overcome the problem of synthetic appearance, natural image priors based on multi-level transforms like wavelets [9, 15, 16] or curvelets [17] can be used within the inpainting and fusion model [14, 18]. These transforms provide a compact yet sparse image representation obtained with low computational costs. Similar to [14], we exploit a wavelet transform for natural regularization within our proposed variational fusion framework capable to handle multiple input observations. 3 Convex Fusion Model In this section we describe our generic fusion model which takes into account multiple observations of the same scene. For clarity, we derive our model for grayvalued images, however the formulation can be easily extended to vector-valued data like color images. 3.1 The Proposed Model We consider a discrete image domain Ω as a regular grid of size W × H pixels with Ω = {(i, j) : 1 ≤ i ≤ W, 1 ≤ j ≤ H}, where the tupel (i, j) denotes a pixel position in the domain Ω. Our fusion model, which takes into account multiple observations and a waveletbased regularization, can be seen as an extension of the TV-L 1 denoising model proposed by Nikolova [13]. In the discrete setting the minimization problem of the common TV-L 1 model for an image domain Ω is formulated as ⎧ ⎫ ⎨ min u∈X ⎩ ‖∇u‖ 1 + λ ∑ ⎬ |u i,j − f i,j | ⎭ , (1) i,j∈Ω where X = R W H is a finite-dimensional vector space provided with a scalar product 〈u, v〉 X = ∑ i,j u i,jv i,j , u, v ∈ X. The first term denotes the TV of the
Page 1: Exploiting Redundancy for Aerial Im
Page 5 and 6: Exploiting Redundancy for Aerial Im

Exploiting Redundancy for Aerial Image Fusion using Convex ...

Create successful ePaper yourself

Delete template?

Save as template?