A FAST AND ROBUST FRAMEWORK FOR IMAGE FUSION AND ...

More documents

Recommendations

Info

3.16 Multi-frame color super-resolution implemented on a real data sequence. (a) shows one of the input low-resolution images (of size [141 × 147 × 3]) demosaiced by [3] and (b) is one of the input low-resolution images demosaiced by the more sophisticated [2]. (c) is the result of applying the proposed colorsuper-resolution method on 31 low-resolution images each demosaiced by [3] method (high-resolution image of size [423 × 441 × 3]). (d) is the result of applying the proposed color-super-resolution method on 31 low-resolution images each demosaiced by [2] method. ......................... 80 3.17 Multi-frame color super-resolution implemented on a real data sequence. The result of applying our method on the original mosaiced raw low-resolution images (without using the inter color dependence term) is shown in (a) (highresolution image of size [423 × 441 × 3]). (b) is the result of applying our method on the original mosaiced raw low-resolution images. .......... 81 3.18 Multi-frame color super-resolution implemented on a real data sequence. (a) shows one of the input low-resolution images (of size [81×111×3]) demosaiced by [3] and (b) is one of the input low-resolution images demosaiced by the more sophisticated [2]. (c) is the result of applying the proposed color-superresolution method on 30 low-resolution images each demosaiced by [3] method (high-resolution image of size [243 × 333 × 3]). (d) is the result of applying the proposed color-super-resolution method on 30 low-resolution images each demosaiced by [2] method. The result of applying our method on the original mosaiced raw low-resolution images (without using the inter color dependence term) is shown in (e). (f) is the result of applying our method on the original mosaiced raw low-resolution images. . . ..................... 82 4.1 The diagonal matrix GB on the right is the result of applying the up-sampling operation (D T GSD) on an arbitrary diagonal matrix GS on the left. The matrix GS can be retrieved by applying the down-sampling operation (DGBD T ). The up-sampling/down-sampling factor for this example is two. . .......... 90 4.2 Block diagram representation of (4.10), where ˆ Z(t), the new input high-resolution output frame is the weighted average of Y (t), the current input low-resolution frame and ˆ Z f (t), the previous estimate of the high-resolution image after motion compensation. ................................ 92 4.3 Block diagram representation of (4.16), where ˆ Zs(t), the new Rauch-Tung- Striebel smoothed high-resolution output frame is the weighted average of ˆ Z(t), (t), the previous the forward Kalman high-resolution estimate at time t, and ˆ Zb s smoothed estimate of the high-resolution image ( ˆ Zb s (t) =F T (t+1) ˆ Zs(t+1)), after motion compensation. ........................... 96 4.4 Block diagram representation of the overall dynamic SR process for color filtered images. The feedback loops are omitted to simplify the diagram. Note ˆZ i∈{R,G,B}(t) represents the forward dynamic Shift-and-Add estimate studied in Section 4.2.2. . ................................ 100 x
4.5 A sequence of 250 low-resolution color filtered images where recursively fused (Section 4.2), increasing their resolution by the factor of 4 in each direction. They were further deblurred and demosaiced (Section 4.4), resulting in images with much higher-quality than the input low-resolution frames. In (a) & (e) we see the ground-truth for frames #50 and #100 of size [100 × 128 × 3], and (b) & (f) are the corresponding synthesized low-resolution frames of size [25×32×3]. In (c) & (g) we see the recursively fused high-resolution frames and (d) & (h) of size [100 × 128 × 3] show the deblurred-demosaiced frames. ........ 103 4.6 A sequence of 250 low-resolution color filtered images where recursively fused (Section 4.2), increasing their resolution by the factor of 4 in each direction. They were further deblurred and demosaiced (Section 4.4), resulting in images with much higher-quality than the input low-resolution frames. In (a) & (e) we see the ground-truth for frames #150 and #200 of size [100×128×3], and (b) & (f) are the corresponding synthesized low-resolution frames of size [25×32×3]. In (c) & (g) we see the recursively fused high-resolution frames and (d) & (h) of size [100 × 128 × 3] show the deblurred-demosaiced frames. ........ 104 4.7 A sequence of 250 low-resolution color filtered images where recursively fused (Section 4.2), increasing their resolution by the factor of 4 in each direction. They were further deblurred and demosaiced (Section 4.4), resulting in images with much higher-quality than the input low-resolution frames. In (a) we see the ground-truth for frame #250 of size [100 × 132 × 3], and (b) is the corresponding synthesized low-resolution frame of size [25 × 32 × 3]. In (c) we see the recursively fused high-resolution frame and (d) of size [100×128×3] show the deblurred-demosaiced frame. . . . ..................... 105 4.8 PSNR values in dB for the synthesized 250 frames sequence of the experiment in Figure 4.5. . . . ................................ 106 4.9 A sequence of 60 real-world low-resolution compressed color frames (a & d of size [141 × 71 × 3]) are recursively fused (Section 4.2), increasing their resolution by the factor of four in each direction (b & e of size [564 × 284 × 3]). They were further deblurred (Section 4.4), resulting in images with much higher-quality than the input low-resolution frames (c & f). . .......... 107 4.10 A sequence of 74 real-world low-resolution uncompressed color filtered frames of size [76×65×3] (a & f show frames #1 and #69, respectively) are recursively fused (Forward data fusion method of Section 4.2.2), increasing their resolution by the factor of three in each direction (b & g of size [228 × 195 × 3]). They were further deblurred (Section 4.4), resulting in images with much higherquality than the input low-resolution frames (c & h). The smoothed data fusion method of Section 4.2.3 further improves the quality of reconstruction. The smoothed Shift-and-Add result for frame #1 is shown in (d). This image was further deblurred-demosaiced (Section 4.4) and the result is shown in (e). . . . 108 5.1 Common strategies used for registering frames of a video sequence. (a) Fixed reference (“anchored”) estimation. (b) Pairwise (“progressive”) estimation. . . 111 5.2 The consistent flow properties: (a) Jacobi Identity and (b) Skew Anti-Symmetry. 114 xi
Page 1 and 2: UNIVERSITY OF CALIFORNIA SANTA CRUZ
Page 3 and 4: Contents List of Figures vi List of
Page 5 and 6: Appendix F Appendix: Affine Motion
Page 7 and 8: 2.3 Effect of upsampling D T matrix
Page 9: 3.11 Multi-frame color super-resolu
Page 13 and 14: List of Tables 2.1 The true motion
Page 15 and 16: show that using such constraints wi
Page 17 and 18: I thank all the handsome gentlemen
Page 19 and 20: It’s the robustness, stupid! xix
Page 21 and 22: quality. We develop the theory and
Page 23 and 24: That is, ⎧ ⎪⎨ ⎪⎩ y1 = h1.
Page 25 and 26: problem of super-resolution is line
Page 27 and 28: accomplished by way of Lagrangian t
Page 29 and 30: which addresses the artifacts resul
Page 31 and 32: noise and Y is the resulting discre
Page 33 and 34: ods, where the regularization-like
Page 35 and 36: 2.2 Robust Super-Resolution 2.2.1 R
Page 37 and 38: and horizontal directions and subsa
Page 39 and 40: Note that if p =2then (2.8) will be
Page 41 and 42: Figure 2.3: Effect of upsampling D
Page 43 and 44: where λ, the regularization parame
Page 45 and 46: Although a relatively large regular
Page 47 and 48: image(X). Figure 2.5(b) is the corr
Page 49 and 50: of high-resolution frame � Xn. Bl
Page 51 and 52: as conjugate gradient is not a stra
Page 53 and 54: of minimization problem (2.23) can
Page 55 and 56: norm is not robust to motion error,
Page 57 and 58: e one. Figure 2.12(f) is the result
Page 59 and 60: 50 100 150 200 250 300 350 50 100 1
Page 61 and 62:
a: One of 8 LR Frames b: Cubic Spli
Page 63 and 64:
a: Frame 1 of 55 LR Frames b: Frame
Page 65 and 66:
Chapter 3 Multi-Frame Demosaicing a
Page 67 and 68:
chrominance layers are separated fr
Page 69 and 70:
Other examples of popular demosaici
Page 71 and 72:
a: Original b: Down-sampled c: Blur
Page 73 and 74:
3.3 Mathematical Model and Solution
Page 75 and 76:
epresents the down-sampling operato
Page 77 and 78:
image (Spatial Luminance Penalty Te
Page 79 and 80:
Acquire a Set of LR Color Filtered
Page 81 and 82:
espect to the other color channels.
Page 83 and 84:
in different color bands. Our propo
Page 85 and 86:
the robust super-resolution method
Page 87 and 88:
We used the method described in [48
Page 89 and 90:
the color artifacts of a set of low
Page 91 and 92:
a: Reconst. with lumin. regul. b: R
Page 93 and 94:
a: Reconst. with all terms. Figure
Page 95 and 96:
a: LR b: Shift-and-Add c: SR [4] on
Page 97 and 98:
a b c d e f Figure 3.14: Multi-fram
Page 99 and 100:
a b c d Figure 3.16: Multi-frame co
Page 101 and 102:
a b c d e f Figure 3.18: Multi-fram
Page 103 and 104:
ates from them in several important
Page 105 and 106:
order. The current image X(t) is of
Page 107 and 108:
With this alternative definition of
Page 109 and 110:
GS a 0 0 0 b 0 0 0 c D T GSD =⇒
Page 111 and 112:
Figure 4.2: Block diagram represent
Page 113 and 114:
ˆZ(t), necessitating a further joi
Page 115 and 116:
Figure 4.3: Block diagram represent
Page 117 and 118:
4.3 Simultaneous Deblurring and Int
Page 119 and 120:
Y (t) Input CFA Data Y R (t) Y G (t
Page 121 and 122:
4.5(b) & 4.5(f) show frames #50 and
Page 123 and 124:
a b c d e f g h Figure 4.6: A seque
Page 125 and 126:
dB 24 22 20 18 16 14 12 10 8 0 50 1
Page 127 and 128:
a b c d e f g h Figure 4.10: A sequ
Page 129 and 130:
Chapter 5 Constrained, Globally Opt
Page 131 and 132:
In this chapter, we study such prio
Page 133 and 134:
(a) (b) Figure 5.2: The consistent
Page 135 and 136:
vector field δi,j is spatially con
Page 137 and 138:
5.3.3 Robust Multi-Frame Registrati
Page 139 and 140:
MSE 10 −1 10 −2 10 −3 10 0 10
Page 141 and 142:
(a) (b) (c) (d) Figure 5.6: Experim
Page 143 and 144:
we advocated the use of the L1 norm
Page 145 and 146:
y the user. - The user is able to s
Page 147 and 148:
There is need for more research on
Page 149 and 150:
than the corresponding single-frame
Page 151 and 152:
In [111], Elad proved that such fil
Page 153 and 154:
all iterations, which means estimat
Page 155 and 156:
Appendix C Noise Modeling Based on
Page 157 and 158:
Appendix D Error Modeling Experimen
Page 159 and 160:
Appendix E Derivation of the Inter-
Page 161 and 162:
Appendix F Appendix: Affine Motion
Page 163 and 164:
Bibliography [1] A. Zomet, A. Rav-A
Page 165 and 166:
[26] H. Ur and D. Gross, “Improve
Page 167 and 168:
[55] K. Hirakawa and T. Parks, “A
Page 169 and 170:
[84] S. Farsiu, D. Robinson, M. Ela
Page 171:
[112] S. M. Kay, Fundamentals of st
show all

A FAST AND ROBUST FRAMEWORK FOR IMAGE FUSION AND ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?