Real-time feature extraction from video stream data for stream ...

Real-time feature extraction from video stream data for stream ...

2.2. Shot Boundary Detection

have a huge difference compared to the same pixel (x 1 ,y 1 ) in frame F i . Therefore ”a large

number of pixels will be judged as changed even if the pan entails a shift of only a few

pixels” [Zhang et al., 1993]. Similar problems occure, when an object in the foreground

is moving in front of a fixed background. Hence the number of detected shot boundaries

is usually too high, as camera and/or object motions are mistakenly classified as shot

boundaries (false positives). To handle this problem, it has proven to be useful to apply

a smoothing filter to the image before calculating the pair-wise pixel difference. For

example, Zhang et al. tried to smooth the images applying a 3 × 3 unweighted image

smoothing filter before computing the pair-wise pixel comparison [Zhang et al., 1993].

For further information on image filtering, please see chapter 6.2.1.

Statistical difference

Instead of comparing all pixels pair-wise, there are also approaches that compute statistical

measures over adjacent frames and compare these statistical measures. Simple

statistical measures are i.g. the average value of all pixels after transforming the image

to grayscale. Other approaches propose more complex statistics. For example, Kasturi

and Jain [Kasturi and Jain, 1991] proposed an approach, computing the mean value (m)

and the standard deviation (S) of the gray levels of two successive frames and have a

measure called likelihood ratio based on these two values. If the likelihood ratio exceeds

a given threshold T , a shot boundary is declared.

[ S i+S i+1


+ ( m i−m i+1


) 2 ] 2

S i ∗ S i+1

This approach is quite resistant against noise, but unfortunately the computation of the

statistical measures is slow due to the complexity of the statistical formulas.

> T

Histogram difference

Other approaches do not rely on statistical measures but on color histograms. Color

histograms are a discretized representation of the color distribution of an image. For each

discretized color value (=color bin) it is counted, how many pixels fall into that bin. The

evaluation of color histograms is described in chapter 6.2.1. Based on the histograms of

two adjacent frames, a shot boundary is declared, when the bin-wise difference of the

two histograms exceeds a given threshold T . The histograms can be either calculated

on grayscale images [Tonomura and Abe, 1989] or the color image itself. Nagasaka and

Tanaka [Nagasaka and Tanaka, 1992], for example, use color histograms with 64 bins (2

bits for each RGB color component [Lefevre et al., 2003]).


|H i (j) − H i+1 (j)| > T


In this formula H i (j) denotes the percentage of pixels in bin j, with 1 ≤ j ≤ G, for the

ith frame. G is the total number of bins.


More magazines by this user
Similar magazines