Project Proposal (PDF) - Oxford Brookes University

FP7-ICT-2011-9 STREP proposal
18/01/12 v1 [Dynact]
Work package description
Work package number 3 Start date or starting event: Month 1
Work package title Dynamical discriminative modeling
Activity type 14 RTD
Participant number 1
Participant short name OBU
Person-months per participant 39
Objectives
The goal of this WP is to develop a novel framework of discriminative models able to explicitly capture the
spatio-temporal structure of human motions. This involves:
– identifying the most discriminative parts of an action or activity via Multiple Instance Learning (MIL)
from a weakly labeled training set;
– learning and classifying pictorial structure models representing constellations of elementary, most
discriminative parts.
Description of work
The work can be articulated into the following tasks:
Task 3.1 – Multiple instance learning of discriminative action parts. Let an action-part be defined as a
Bag of Features model bounded in a space-time cube. The task here is to learn models for a set of most
discriminative action parts, given a training set of video sequences in which an action class is assigned to
each video clip, assuming that one action occurs in each clip. This is a weakly labeled scenario, where it is
known that a positive example of the action exists within the clip, but the exact location of the action is
unknown. The learning task can then be cast in a Multiple Instance Learning (MIL) framework, in which
the training set consists of a set of “bags” (the training sequences), containing a number of BoF models (in
our case SVM classifiers learnt for each sub-volume of the spatio-temporal sequence), and the
corresponding ground truth class labels.
Task 3.2 – Learning and classifying structured discriminative models of actions. Once the most
discriminative action parts are learnt by applying MIL to the “bags” corresponding to the given training
spatio-temporal sequences, we can construct tree-like ensembles of action parts which can be later used for
both localizing and efficiently classifying complex activities. A cost function can be defined as a function of
both the appearance models of the individual parts and of the relative positions between pairs of action
parts, whose maximization yields the best action configuration.
The problem can be efficiently solved by dynamic programming.
Deliverables
D3.1 A scientific report on the Multiple Instance Learning framework for the detection of the most
discriminative action parts (month 16).
D3.2 The efficient algorithmic implementation of the MIL framework as a software prototype (month 18).
D3.3 A scientific report detailing the approach based on the optimization of constellations of elementary
actions for the localization and classification of complex activities (month 24).
D3.4 The efficient implementation of our novel framework for “structural” discriminative action modelling,
localization and classification as a software prototype (month 24).
14 Please indicate one activity per work package:
RTD = Research and technological development; DEM = Demonstration; MGT = Management of the
consortium.
Proposal Part B: page [30] of [67]