Real-time feature extraction from video stream data for stream ...

2.2. Shot Boundary Detection

have a huge difference compared to the same pixel (x 1 ,y 1 ) in frame F i . There**for**e ”a large

number of pixels will be judged as changed even if the pan entails a shift of only a few

pixels” [Zhang et al., 1993]. Similar problems occure, when an object in the **for**eground

is moving in front of a fixed background. Hence the number of detected shot boundaries

is usually too high, as camera and/or object motions are mistakenly classified as shot

boundaries (false positives). To handle this problem, it has proven to be useful to apply

a smoothing filter to the image be**for**e calculating the pair-wise pixel difference. For

example, Zhang et al. tried to smooth the images applying a 3 × 3 unweighted image

smoothing filter be**for**e computing the pair-wise pixel comparison [Zhang et al., 1993].

For further in**for**mation on image filtering, please see chapter 6.2.1.

Statistical difference

Instead of comparing all pixels pair-wise, there are also approaches that compute statistical

measures over adjacent frames and compare these statistical measures. Simple

statistical measures are i.g. the average value of all pixels after trans**for**ming the image

to grayscale. Other approaches propose more complex statistics. For example, Kasturi

and Jain [Kasturi and Jain, 1991] proposed an approach, computing the mean value (m)

and the standard deviation (S) of the gray levels of two successive frames and have a

measure called likelihood ratio based on these two values. If the likelihood ratio exceeds

a given threshold T , a shot boundary is declared.

[ S i+S i+1

2

+ ( m i−m i+1

2

) 2 ] 2

S i ∗ S i+1

This approach is quite resistant against noise, but un**for**tunately the computation of the

statistical measures is slow due to the complexity of the statistical **for**mulas.

> T

Histogram difference

Other approaches do not rely on statistical measures but on color histograms. Color

histograms are a discretized representation of the color distribution of an image. For each

discretized color value (=color bin) it is counted, how many pixels fall into that bin. The

evaluation of color histograms is described in chapter 6.2.1. Based on the histograms of

two adjacent frames, a shot boundary is declared, when the bin-wise difference of the

two histograms exceeds a given threshold T . The histograms can be either calculated

on grayscale images [Tonomura and Abe, 1989] or the color image itself. Nagasaka and

Tanaka [Nagasaka and Tanaka, 1992], **for** example, use color histograms with 64 bins (2

bits **for** each RGB color component [Lefevre et al., 2003]).

G∑

|H i (j) − H i+1 (j)| > T

j=1

In this **for**mula H i (j) denotes the percentage of pixels in bin j, with 1 ≤ j ≤ G, **for** the

ith frame. G is the total number of bins.

13