SR Non-Uniform Interpolation
SR Non-Uniform Interpolation
SR Non-Uniform Interpolation
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
www.ceva-dsp.com<br />
www.ceva-dsp.com<br />
www.ceva-dsp.com CEVA Confidential<br />
Multi-Image Super<br />
Resolution Using<br />
<strong>Non</strong>-<strong>Uniform</strong><br />
<strong>Interpolation</strong><br />
Agenda<br />
Adar Paz<br />
Danny Gal<br />
DSP trends<br />
The <strong>SR</strong> challenge<br />
<strong>SR</strong> non-uniform interpolation<br />
CEVA’s <strong>SR</strong> Solution<br />
Summary<br />
February 2013<br />
slide 2
www.ceva-dsp.com<br />
Image Processing Trends<br />
www.ceva-dsp.com<br />
DSP Trends<br />
High<br />
res<br />
High<br />
res<br />
High<br />
frame<br />
rate<br />
Computational demands \<br />
BW Demands<br />
Clock speed<br />
High<br />
frame<br />
rate<br />
Wide<br />
access<br />
slide 3<br />
slide 4<br />
2D<br />
access
www.ceva-dsp.com<br />
DSP Application Requirements<br />
> The industry is interested in algorithms that produce<br />
high quality Super Resolution within real-time<br />
processing and power limitations:<br />
> DSP<br />
Parallel processing<br />
Predictable data access<br />
> Real-Time<br />
Low and bounded cycle count<br />
Low bandwidth<br />
Robust<br />
www.ceva-dsp.com<br />
<strong>SR</strong> Challenge<br />
> Create high resolution image using several low<br />
resolution images<br />
Low end sensor High end sensor<br />
slide 5<br />
slide 6
www.ceva-dsp.com<br />
<strong>SR</strong> Algorithms<br />
How can we increase image resolution ?<br />
www.ceva-dsp.com<br />
> Iterative Back-Projection Based<br />
> Frequency Domain<br />
> Normalized Convolution<br />
> Statistical<br />
> <strong>Non</strong>-<strong>Uniform</strong> <strong>Interpolation</strong><br />
<strong>SR</strong> <strong>Non</strong>–<strong>Uniform</strong> <strong>Interpolation</strong> (Cont.)<br />
slide 7<br />
slide 8
www.ceva-dsp.com<br />
<strong>SR</strong> <strong>Non</strong>-<strong>Uniform</strong> <strong>Interpolation</strong> -<br />
Advantages for the DSP Application<br />
Performance<br />
Easy to parallelize calculations (no dependencies)<br />
Simple filter (using MAC) - Efficient in DSP cores<br />
Bandwidth<br />
Serial data access - Data can be accessed in<br />
continuous “raster like” order<br />
Performance + Bandwidth<br />
www.ceva-dsp.com<br />
<strong>Non</strong> iterative - Bounded and predicted calculation time<br />
<strong>SR</strong> <strong>Non</strong>–<strong>Uniform</strong> <strong>Interpolation</strong><br />
(Training)<br />
A. Gilman, and D. G. Bailey, “Near optimal non-uniform interpolation for<br />
image super-resolution from multiple images,” 2006<br />
slide 9<br />
slide 10
www.ceva-dsp.com<br />
<strong>SR</strong> <strong>Non</strong>–<strong>Uniform</strong> <strong>Interpolation</strong><br />
(Training – Cont.)<br />
www.ceva-dsp.com<br />
A. Gilman, and D. G. Bailey, “Near optimal non-uniform interpolation for<br />
image super-resolution from multiple images,” 2006<br />
Original HR<br />
Step 1<br />
Coeff exract<br />
Step 2<br />
<strong>Interpolation</strong><br />
<strong>SR</strong> <strong>Non</strong>–<strong>Uniform</strong> <strong>Interpolation</strong><br />
(Training – Cont.)<br />
A. Gilman, and D. G. Bailey, “Near optimal non-uniform interpolation for<br />
image super-resolution from multiple images,” 2006<br />
Original HR<br />
HR<br />
HR<br />
slide 11<br />
slide 12
www.ceva-dsp.com<br />
<strong>SR</strong> <strong>Non</strong>–<strong>Uniform</strong> <strong>Interpolation</strong><br />
(Training – Cont.)<br />
A. Gilman, and D. G. Bailey, “Near optimal non-uniform interpolation for<br />
image super-resolution from multiple images,” 2006<br />
Method MSE (10-5 Method MSE )<br />
Optimal 100 % (1.25)<br />
Coeffs from ‘Sleep’ 106 % (1.32)<br />
Coeffs from ‘Disk’ 116 % (1.45)<br />
www.ceva-dsp.com<br />
Sleep Disk<br />
<strong>SR</strong> <strong>Non</strong>–<strong>Uniform</strong> <strong>Interpolation</strong><br />
Disadvantage<br />
> Requires just-in-time training<br />
Cycles<br />
Bandwidth<br />
1<br />
HR<br />
LRs<br />
2<br />
slide 13<br />
Interpolated<br />
HR<br />
slide 14
www.ceva-dsp.com<br />
Alternative: Assume Analytic Prior<br />
(Michaeli & Eldar SSP’09)<br />
> Derive optimal filter based on priors<br />
> Assume prior knowledge on<br />
Image spectrum<br />
Sampling kernel (PSF)<br />
Noise statistics<br />
LR image displacement<br />
www.ceva-dsp.com<br />
Michaeli & Eldar SSP’09 (cont.)<br />
Continuous Signal Image<br />
Estimated HR Image<br />
Blurred with PSF kernel<br />
Displacement and Sampling<br />
(LR Images)<br />
slide 15<br />
slide 16
www.ceva-dsp.com<br />
Michaeli & Eldar SSP’09 (cont.)<br />
> Downside: Requires unlimited filter support<br />
www.ceva-dsp.com<br />
CEVAs Solution<br />
Empirical solution<br />
Gilman & Bailey<br />
Training<br />
Finite support<br />
Combined solution<br />
CEVA<br />
Analytic and Robust<br />
Finite support<br />
Analytical Solution<br />
Michaeli & Eldar<br />
Analytic and Robust<br />
Infinite support<br />
slide 17<br />
slide 18
www.ceva-dsp.com<br />
Gilman’s <strong>SR</strong><br />
www.ceva-dsp.com<br />
CEVA's <strong>SR</strong><br />
System overview<br />
System overview<br />
slide 19<br />
slide 20
www.ceva-dsp.com<br />
Quality Comparisons<br />
The test suite and ISO 12233 test pattern are from<br />
LCAV - Audiovisual Communications Laboratory.<br />
• Iterated Back-Projection. M. Irani and S. Peleg, Graphical Models and Image<br />
Processing, 1991.<br />
• Robust Super-Resolution. A. Zomet, A. Rav-Acha, and S. Peleg, CVPR, 2001.<br />
• Normalized Convolution. Tuan Q. Pham, Lucas J. van Vliet and Klamer Schutte,<br />
EURASIP Journal on Applied Signal Processing, 2006.<br />
www.ceva-dsp.com<br />
Quality Comparisons (Cont.)<br />
Photograph taken with a Canon 550D camera<br />
LCAV - Audiovisual Communications Laboratory.<br />
slide 21<br />
slide 22
www.ceva-dsp.com<br />
Performance Comparison<br />
Method<br />
Complexity<br />
[op / pixel]<br />
Real-time disadvantage<br />
Iterated Back-Projection More than 10,000 Iterative, high BW<br />
Robust Super Resolution More than 12,000 Iterative, high BW<br />
Structure Adaptive<br />
Normalized Convolution<br />
Very complex:<br />
Photo Acute 100-400<br />
SVD for every pixel<br />
CEVA <strong>SR</strong> Less than 100<br />
CEVA <strong>SR</strong> on MM3101 8 cycles/pixel<br />
www.ceva-dsp.com<br />
CEVA’s MM3101<br />
<strong>Non</strong>-Linear complexity<br />
> In order to achieve Real-Time performance we<br />
implemented solution on CEVA’s MM3101 core<br />
Parallelism<br />
> SIMD VLIW architecture<br />
> 32 multiply operation in one cycle<br />
Dedicated instructions<br />
> SAD x64 in one cycle<br />
> 4 tap filter x16 in one cycle<br />
Advance memory access<br />
> Wide 2D memory access<br />
> Complex data manipulations/permutations<br />
slide 23<br />
slide 24
www.ceva-dsp.com<br />
Summary and Conclusions<br />
> <strong>Interpolation</strong> approach for Super Resolution shows<br />
very good results comparing to the leading and<br />
commercial solutions, while having major<br />
advantages for Real-Time DSP application<br />
> Together with CEVA-MM3101 platform we propose a<br />
high quality and fast super resolution solution that<br />
satisfies Real-Time processing demands<br />
www.ceva-dsp.com<br />
Thank You<br />
slide 25
Change language