an innovative algorithm for key frame extraction in video ...

More documents

Recommendations

Info

where MaxDiff is the largest possible value that the Diff( ) frame differencedistance can assume. High Fidelity values indicate that the key frames extractedfrom the video sequence provide a good global description of the visual content ofthe sequence. A visual representation of the Fidelity mesaure computation isshown in Fig. 2.... ... S tF(t) F(t+n) F(t+γ NF -1)F KF (t) F KF (t+2) Diff(F(t+γ NF -1),F KF (t+2))F(t+γ NF -1)KF tFig. 2. Computation of the Fidelity measure. Each frame in the video sequence is compared witheach key frame in the summary, and the minimal distance of each is retained. The greatest of thesedistances provides the Fidelity measure.3.2 Shot Reconstruction DegreeThe idea underlying the Shot Reconstruction Degree (SRD) measure is that, usinga suitable frame interpolation algorithm, we should be able to reconstruct thewhole sequence, from the set of key frames. The better the reconstructionapproximates the original video sequence, the better the key frames summarize itscontent. Let the video sequence St and the set of key frames KFt be defined as inthe previous paragraph. Let FIA( ) be a frame interpolation algorithm thatcomputes a frame from a pair of key frames in KFt:~( t + n)= FIA F (t + n ),F (t + n ),n,n ,n n ≤ n n (6)(KF j KF j+ 1 j j+1)j
... ... S tF(t) F(t+1)F(t+n) F(t+γ NF -1)Diff(F(t+n),F KF (t+n))~ ~~ ~F(t) F(t+1)F(t+n) F(t+γ NF -1)... ...FIA(F KF (t+2),F KF (t+γ NF -1),n,2,γ NF -1)F KF (t) F KF (t+2) F(t+γ NF -1)KF tFig. 3. The computation of the SRD measure. The video sequence is reconstructed using the keyframes extracted and a frame interpolation algorithm (FIA). The interpolated frames are comparedwith the corresponding frames in the original sequence, and the computed frame differences areused to compute the final SRD measure.3.3 Compression ratioA video summary should not contain too many key frames since the aim of thesummarization process is to allow users to quickly grasp the content of a videosequence. For this reason, we have also evaluated the compactness of thesummary (compression ratio). The compression ratio is computed by dividing thenumber of key frames in the summary by the length of video sequence. For agiven video sequence S t , the compression rate is thus defined as:CRatioγ( S ) = 1 −NKFt(9)γNFwhere γNKFis the number of key frames in the summary, and γNFis the totalnumber of frames in the video sequence. Ideally, a good summary produced by akey frame extraction algorithm will present both high quality measure (in terms ofFidelity or SRD) and a high compression ratio (i.e. small number of key frames).4. New key frame extraction approachThe proposed key frame extraction algorithm functions as shown in Fig. 4. Thealgorithm employs the information obtained by a video segmentation algorithm toprocess each shot. We have developed a prototypical segmentation algorithm thatfor the moment detects only abrupt changes and fades, since these are the mostcommon editing effects. For abrupt changes we have implemented a thresholdbasedalgorithm coupled with a frame difference measure computed fromhistograms and texture descriptors. The same frame difference measure is used forthe key frame extraction algorithm. To detect fades we have implemented amodified version of the algorithm proposed by Fernando et al. [28]. The resultsobtained by these algorithms are submitted for evaluation to a decision modulewhich gives the final response. This allows us to cope with conflicting results, orwith groups of frames that are not meaningful, such as those between the end of afade out and the start of a fade-in, increasing the robustness of the detection phase.A gradual transition detection algorithm is currently being developed, and will beintegrated in a similar manner.7
Page 1 and 2: AN INNOVATIVE ALGORITHM FOR KEYFRAM
Page 3 and 4: Section 2 of this paper presents se
Page 5: into the summarization algorithm to
Page 9 and 10: histogram. The threshold has been h
Page 11 and 12: the first pass the algorithm detect
Page 13 and 14: Fig. 7. An example of key frame sel
Page 15 and 16: energy feature, the magnitudes of t
Page 17 and 18: was available we also selected the
Page 19 and 20: 6.2 Computational TimeTable 4 shows
Page 21 and 22: performance of the SRDI algorithm e
Page 23 and 24: Table 6. Fidelity measure results c
Page 25 and 26: eeopenFidelityHSTFidelityHWDSRDHSTS
Page 27 and 28: AcknowledgementsThe video indexing

an innovative algorithm for key frame extraction in video ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?