Learning manifolds of dynamical models for activity recognition

More documents

Recommendations

Info

in this context performance is influenced by factors as diverse as viewpoint, emotional state, illumination, presence of clothes/occlusions, etcetera, that can be modeled through tensor algebra. He has recently published a book chapter [14] with IGI on this topic. In the wider field of articulated motion analysis he has published several papers on spectral motion capture techniques [40], focusing in particular on the crucial issue of how to select and map eigenspaces generated by two different shapes in order to track 3D points on their surfaces or consistently segment bodyparts along sequences of voxelsets [17], as a preprocessing step to action recognition. In direct relation to the topic of this proposal, he is now exploring manifold learning techniques for dynamical models representing (human) motions, in order to learn the optimal metric of the space they live in and maximize classification performance. Another book chapter [16] collecting his preliminary results on the topic has been recently accepted by Springer. Proposer’s Other Scientific Contributions. Dr Cuzzolin is also recognized as one of the most prominent experts in the field of non-additive probabilities and belief functions. He has been recently elected member of the Board of Directors of the newly founded “Belief Functions and Applications Society”, and is member of the “Society for Imprecise Probabilities and Their Applications”. His most important contribution in the field of uncertainty theory and imprecise probabilities is a general geometric approach to uncertainty measures, in which probabilities, possibilities and belief functions can all be represented as points of a Cartesian space and there analyzed [12]. Evidence aggregation operators (the analogues of Bayes’ rule in the Bayesian formalism) can also be seen as geometric operators. The issues of how to approximate a belief function with an additive probability or a possibility measure, or what probability transformation is appropriate for decision making can be all solved by geometric means. In his recent award-winning paper [15], Dr Cuzzolin has also investigated alternative combinatorial foundations for the theory of belief functions, and their algebraic properties. He is currently working on the generalization of the total probability theorem to finite random sets, as a key contribution to the field of non-additive probabilities. He is in the process of finalizing a book entitled “The geometry of uncertainty” which will collect all his contributions to the mathematics of uncertainty. Past Collaborations and Professional Links. Dr Cuzzolin acquired considerable international experience by working in the past for some of the most prominent research laboratories in both the US and Europe. He gave seminar and invited talks at several world-leading institutions such as MIT, EPFL, GeorgiaTech, Microsoft Research Europe, INRIA. His network of collaborations with groups of researchers in both Europe and the United States is quite large and expanding. Dr Cuzzolin is currently arranging meetings with several groups of researchers all around Europe to set up active collaborations to support his goal of establishing a fairly large group of five-ten people in a few years time in perspective of reaching a professorial position in the medium term. He is in talks with M. Zaffalon (IDSIA, Switzerland) for a STREP on imprecise Markov chains for gesture recognition. He is setting up with INRIA’s Radu Horaud, Alejan- dro Frangi (Pompeu Fabra) and Technion (R. Kimmel, M. Bronstein) an interdisciplinary Future Emerging Technology (FET) EU proposal on large scale manifold learning, with applications to scene understanding. He is discussing a collaborative project on uncertainty theory at UK level with J. Lawry (Head of Department of Bristol’s Engineering Mathematics) and F. Coolen (Durham’s Dept of Statistics), and exploring the opportunity of a European Network of Excellence in the same field. He plans to apply for the European Research Council Starting Grant in October 2010. Dr Cuzzolin enjoys personal links with several world class companies (many of them with research divisions in the UK) such as Microsoft Research, Honeywell Labs (I. Cohen), Boston’s MERL (M. Brand, S. Ramalingam), GE (G. Doretto), Google (A. Bissacco), Riya. The host organization: Oxford Brookes University, School of Technology, (OBU). In the department there are around 30 academic staff, these include, in computer graphics, Prof. David Duce (co-chair of Eurographics 2003, 2006 conference), Bob Hopgood OBE, Prof. M.K. Pidcock, world leader in Electrical Impedance Tomography, and in AI and image processing, Prof. William Clocksin. The Computer Science department had the following break down in the recent RAE: 4* 15%, 3* 35%, 2* 35% and 1* 15%, which means that 85% of output was deemed internationally leading and that no research output was considered unclassified. The School of Technology has recently established a new doctoral training programme with the theme of intelligent transport systems ( http://tech.brookes.ac.uk/research/), which includes many computer vision problems, with a set of courses and associated infrastructure which will be directly beneficial to this project. Dr Fabio Cuzzolin belongs to the Oxford Brookes vision group founded by Professor Philip Torr ( cms.brookes.ac.uk/research/visiongroup/), which comprises some seventeen staff, students and post-docs who will add value to this project. Professor Torr was awarded the Marr Prize, the most prestigious prize in computer vision, in 1998. Members of the group have recently received awards in 4 other conferences, including best paper at CVPR 08 and honourary mention at NIPS, the top machine learning conference. The group was mentioned four times in the UKCRC Submission to the EPSRC International Review Sep. 20064. It enjoys ongoing collaborations with companies such as 2d3, Vicon Life, Yotta, Microsoft Research Europe, Sharp Laboratories Europe, Sony Entertainments Europe. The group’s work with the Oxford Metrics Group in a Knowledge Transfer Partnership 2005-9 won the National Best Knowledge Transfer Partnership of the year at the 2009 awards, sponsored by the Technology Strategy Board, selected out of several hundred projects. Oxford Brookes also has close links with Oxford University, with both the Active Vision Group and Prof. Zissermans Visual Geometry group, including a joint EP- SRC grant and EU collaboration as well as co-supervision. Members of the all groups regularly attend each others reading groups and seminars. Dr Cuzzolin holds a Early Career Fellow position with minimal undergraduate teaching duties and hence has sufficient time to conduct the research listed in this proposal.
2 Proposed research and its context 2.1 Background Topic of research. Recognizing human activities from video is a natural application of computer vision. Since the classic experiment of Johansson [] showing that moving light displays were enough for people to recognize motions or even identities, the matter has been subject of increasing interest from the vision community. The problem consists on telling, given one or more image sequences capturing one or more people performing an activity, what category (among those previously learned) this activity belongs to. A useful even though quite fuzzy distinction is that between “actions”, meant as simple (usually stationary) motion patterns, and “activities” as more complex sequences of actions, sometimes overlapping in time as they are performed by different parts of the body or different agents in the scene. State of the art. The activity recognition problem involves, as we have seen, numerous challenges at different levels of representation. Accordingly, activity recognition frameworks can be effectively described in terms of three layers: 1. feature extraction from single images o videos; 2. action description or modeling; 3. high-level semantic activity modeling. Critical issues of activity recognition. Though the formulation of the problem is simple and intuitive, activity recognition is a much harder problem than it may look [?]. Motions inherently possess an extremely high degree of variability. Movements quite different from each can in fact carry the same meaning, or represent the same gesture. Even in perfectly controlled situations (constant illumination, static background) this inherent variability makes recognition hard (e.g., the trajectory followed by a walking person is irrelevant to the classification of the action they perform). In addition, motions are subject to a large number of nuisance or “covariate” factors [37], such as: illumination, background, viewpoint. The list goes on and on. The combination of inherent variability and nuisance factors explains why experiments have been so far conducted in small, controlled environments to reduce their complexity. Attempts have been recently done to go beyond such controlled environments. Liu, Luo and Shah [37] has taken on the challenge of recognizing actions “in the wild” and coping with the tremendous nuisance variations of unconstrained videos. Motion statistics are used to prune motion and static features. Adaboost is chosen to integrate all those single-feature classifiers. The algorithm is tested on the KTH dataset and on YouTube videos. Hu et al [59] have also investigated action detection in complex, cluttered scenes, representing candidates regions are bags of instances. In their SMILE-SVM framework human action detectors are learnt based on imprecise action locations. They test their approach on both the CMU dataset and videos shot in a shopping mall. Yao and Zhu [58] learn deformable action templates from cluttered real-world videos. Such action templates are sequences of image templates consisting of a set of shape and motion primitives. Templates are deformed to match KTH and CMU videos by space-time warping. If we remove the assumption that a single motion of interest is present in our video, locality emerges too as a critical factor. For instance, a person can walk and wave at the same time. Different actions performed by different agents can go on in different regions of the image sequence, without being necessarily synchronized or coordinated. Gilbert et al [27], for instance, perform multi-action recognition using very dense spatio-temporal corner features in a hierarchical classification framework, testing it on the multi-KTH dataset and the Real-World Movie dataset. Reddy et al [45] propose to cope with incremental recognition using feature trees that can handle multiple actions without intensive training. Local features are recognized by NN, while actions are later labeled using a simple voting strategy. They use the KTH and IXMAS datasets. If we also remove the assumption of having a single body moving in the field of interest, problems like occlusion assume a critical importance []. On the other hand, the presence of one of more (static) objects in the vicinity of the motion can effectively help to disambiguate the activity class (recognition “in context”). Marszalek, Laptev and Schmid [39] apply a joint SVM classifier of action and scene to bags of features, using movie scripts as means of automatic supervision for training (as proposed in [21]) on the newly proposed “Hollywood movies” database. Han et al [29], for instance, introduce bag-of-detectors scene descriptors to encode such contextual presence, and use Gaussian processes to select and weight multiple features for classification on the KTH and Hollywood dataset. Sun et al [53] use the Hollywood and LSCOM databases to model and test spatio-temporal context in a hierarchical way, using SIFT for point-level context and Markov processes to model trajectory proximity and transition descriptors, on top of which they apply multi-channel non-linear SVM. In opposition, sometimes very few instances of each action category are available. Seo and Milanfar [49] detect actions from single examples by computing sense space-time local regression kernels, which are used as descriptors and passed through PCA to extract salient features. A matrix generalization of cosine similarity is then used to compare videos, on the Irani database. Bag-of-features on spatio-temporal volumes methods. Recently, recognition methods which neglect action dynamics (typically extracting spatio-temporal [6] features from the 3D volume associated with a video [32]) have proven very effective (http://www.wisdom.weizmann.ac.il/ vision/SpaceTimeActions.html). Kim and Cipolla [32] do spatio-temporal pattern matching, representing volumes as tensors. They propose an extension of canonical correlation analysis to tensor to detect actions on a 3D window search with exemplars on the KTH dataset. Bregonzio et al [7] also use a global spatio-temporal distribution of interest points. They extract and select “holistic” features from clouds of interest points over multiple temporal scales. Rapantzikos et al [44] also dense spatio-temporal features detected using saliency measures, which incorporate color, motion and intensity, in a multi-scale volumetric representation. Yuan at el [60] describe actions as collections of spatio-temporal invariant features, and propose a naive Bayes mutual information maximization method for multi-class. A search algorithm is also proposed to locate the optimal 3D subvolume in which the action can be detected. Experiments
Page 1: Learning manifolds of dynamical mod
Page 5 and 6: tion such as HMMs or variable lengt
Page 7 and 8: Figure 3: A bird’s eye view of ou
Page 9 and 10: A Justification of resources B Diag
Page 11: E Impact plan Please detail how the

Learning manifolds of dynamical models for activity recognition

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?