Real-time feature extraction from video stream data for stream ...

Real-time feature extraction from video stream data for stream ...


Feature Extraction

Videos are basically a sequence of images (frames) and short audio-samples. Therefore,

most methods developed for learning on images can also be applied to video data. The

following section gives an overview of the features I have extracted from the video data

for this diploma thesis. Since the extraction of features from images is a common task

in computer vision and countless approaches have been developed over the years, I can

only focus on a small subset of all possibly extractable features. Of course, the following

collection is surely not exhaustive and there might be useful features I have not identified

yet, but it will turn out, that the features extracted so far are sufficient to solve the

specified learning tasks. Nevertheless the extraction of further features might turn out

to be necessary during the next phases of the ViSTA-TV Project.

In order to evaluate which image features are useful for solving the defined learning tasks

(see chapter 4), I have first of all extracted a huge amount of features using RapidMiner.

The RapidMiner Image Mining Extension takes JPEG images, gained from the video

data by using xuggler, as its’ input and enables us to extract and evaluate features

quickly. The features extraction is described in section 6.2. As RapidMiner does not

provide the user with any stream functionality, I have afterwards developed processors

for the streams framework, enabling me to extract those features, which turned out to be

useful, on live video data in real-time as well. An overview of the implemented methods

can be found in section 6.3).

6.1. Naming convention for extracted features

In order to keep track of the huge amount of features that might potentially be created

using all sorts of available data (i.g video, audio, EPG and user-data), a consistent

naming convention is necessary. Therefore we have developed a hierarchical naming

scheme. We have decided to separate each level of the hierarchy by using a ”:”. On the

first level of the hierarchy we should tell, from which kind of data we gained the feature.

Possible values are: ”video:”, ”audio:”, ”epg:”, ”user:” or ”dbpedia:”, just to mention a


More magazines by this user
Similar magazines