an innovative algorithm for key frame extraction in video ...

... ... S tF(t) F(t+1)F(t+n) F(t+γ NF -1)Diff(F(t+n),F KF (t+n))~ ~~ ~F(t) F(t+1)F(t+n) F(t+γ NF -1)... ...FIA(F KF (t+2),F KF (t+γ NF -1),n,2,γ NF -1)F KF (t) F KF (t+2) F(t+γ NF -1)KF tFig. 3. The computation of the SRD measure. The video sequence is reconstructed using the keyframes extracted and a frame interpolation algorithm (FIA). The interpolated frames are comparedwith the corresponding frames in the original sequence, and the computed frame differences areused to compute the final SRD measure.3.3 Compression ratioA video summary should not contain too many key frames since the aim of thesummarization process is to allow users to quickly grasp the content of a videosequence. For this reason, we have also evaluated the compactness of thesummary (compression ratio). The compression ratio is computed by dividing thenumber of key frames in the summary by the length of video sequence. For agiven video sequence S t , the compression rate is thus defined as:CRatioγ( S ) = 1 −NKFt(9)γNFwhere γNKFis the number of key frames in the summary, and γNFis the totalnumber of frames in the video sequence. Ideally, a good summary produced by akey frame extraction algorithm will present both high quality measure (in terms ofFidelity or SRD) and a high compression ratio (i.e. small number of key frames).4. New key frame extraction approachThe proposed key frame extraction algorithm functions as shown in Fig. 4. Thealgorithm employs the information obtained by a video segmentation algorithm toprocess each shot. We have developed a prototypical segmentation algorithm thatfor the moment detects only abrupt changes and fades, since these are the mostcommon editing effects. For abrupt changes we have implemented a thresholdbasedalgorithm coupled with a frame difference measure computed fromhistograms and texture descriptors. The same frame difference measure is used forthe key frame extraction algorithm. To detect fades we have implemented amodified version of the algorithm proposed by Fernando et al. [28]. The resultsobtained by these algorithms are submitted for evaluation to a decision modulewhich gives the final response. This allows us to cope with conflicting results, orwith groups of frames that are not meaningful, such as those between the end of afade out and the start of a fade-in, increasing the robustness of the detection phase.A gradual transition detection algorithm is currently being developed, and will beintegrated in a similar manner.7