An Integrated and Scalable Approach to Video Enhancement in ...

8
…
1 2 3 4 5 6 7 8 9 ... W-1 W
Fig. 10: Fast calculation of 1-D local minimum value.
in the YUV space to match the input format of most practical
video applications:
Y out (x) =
U out (x) =
V out (x) =
Y in (x) − ωA min( min ( I(y)
c y∈Ω(x)
A ))
1 − ω min( min ( I(y)
c y∈Ω(x)
A )) , (11)
U in (x) − 128
1 − ω min( min
c
( I(y)
y∈Ω(x)
A
V in (x) − 128
1 − ω min( min
c
( I(y)
y∈Ω(x)
A
))
+ 128, (12)
))
+ 128. (13)
Finally, to further speed up the implementation, we exploited
the inherent redundancies in the pixel-wise calculations of the
minimization in equations (10) - (13), which corresponds to a
complexity of k 2 × W × H comparisons for an input frame of
resolution W ×H, and a search window (for the minimization)
of size k × k pixels. To expedite the process, we first find
and store the smaller of every two horizontally neighboring
pixels in the frame using a sliding horizontal window of size
2, requiring W × H comparisons. Then, by again using a
horizontal sliding window of 2 over the values stored in the
previous step, we can find the minimum of every 4 horizontally
neighboring pixels in the original input frame. This process is
repeated in both the horizontal and vertical directions, until we
have found the minimum of all k × k neighborhoods of the
input. It is easy to find that such a strategy has a complexity
of roughly 2 log 2 k × W × H comparisons, as opposed to
k 2 × W × H for the simplistic implementation. This process
is illustrated for one row of W pixels in Fig. 10, where the
red and black lines refer to the comparisons made, each with
a sliding window of 2 values.
Fig. 11: Example of low lighting video enhancement algorithm:
Original input (Left), and the enhancement result
(Right).
Fig. 12: Example of high dynamic range video enhancement
algorithm: Original input (Left), and the enhancement result
(Right).
Examples of the enhancement outputs for low lighting, high
dynamic range and hazy videos are shown in Fig. 11, Fig. 12
and Fig. 15 respectively. As we can see from these figures,
the improvements in visibility are obvious. In Fig. 11, the
yellow light from the windows and signs such as “Hobby
Town” and other Chinese characters were recovered in correct
color. In Fig. 12, the headlight of the car in the original input
made letters on the license plate very difficult to read. After
V. EXPERIMENTAL RESULTS
To evaluate the proposed approach, a series of experiments
were conducted with a Windows PC (Intel Core 2 Duo
processor running at 2.0 GHz with 3G of RAM) and an iPhone
4. On the iPhone, our software could process the images and
videos from the camera directly or from the photo album.
After the processing is complete, the output would be shown
on the screen and saved in the photo album automatically. The
resolution of test videos in our experiments was 640 × 480 on
PC and 192 × 144 on iPhone 4. The enhancement effects and
processing speed are listed below. Due to time constraints,
we only implemented the frame-by-frame base line system on
the iPhone and did not employ many further possible ways
of optimization on the PC platform (e.g. by using assembly
coding).
Fig. 15: Example of haze removal algorithm: Original input
(Left), and the enhancement result (Right).
Fig. 16: Example of rainy video enhancement using haze
removal algorithm: Original input (Left), and the enhancement
result (Right).