D2.1 Requirements and Specification - CORBYS

D2.1 Requirements and Specification
include anomaly detection, assisted living, PDA with pen-gestures, speech and graphics input capability etc.
11.2 Situation Assessment
Endsley (2000) defines situation assessment as “the perception of elements in the environment within a
volume of time and space, the comprehension of their meaning and the projection of their status in the near
future”. According to Lambert (1999, 2001, 2003, 2006), situation assessment involves assessment of
situations where situations are “fragments of the world represented by a set of assertions”. This differs from
object recognition in the sense that it requires a shift in the procedure from numeric to symbolic
representations, a problem coined as the semantic challenge for information fusion (Lambert, 2003). Lambert
(2006) proposes an interesting approach to semantic fusion using a formal theory by following a development
path that involves sequential construction of the problem in terms of philosophy, mathematics and
computation. This approach is illustrated using a formal theory for existence in Lambert’s work (Lambert,
2006).
Level 1:
Perception of
elements in
current
situation
Situation Awareness
Level 2:
Comprehension
of current
situation
Goals and
objectives
preconceptions
Figure 26: Lambert's approach to semantic Fusion
There exist two main stages for integration and fusion of multimodal input according to Corradini et al (2005)
namely:
• Integration of signal at feature level,
Feedback
• Integration of information at semantic level.
Level 3:
Projection of
future status
Environment
Decision
The feature level signal fusion, also referred to lower level fusion is related to “closely coupled and
synchronised” modalities such as speech and lip movements. This does not scale with ease, requires extensive
training data sets and incurs high computational costs (Corradini et al. 2005). Higher level symbolic or
semantic fusion on the other hand, is related to modalities that “differ in time scale characteristics of their
features”. This entails time stamping of all modalities to aid the fusion process. Semantic fusion involves a
number of benefits such as off-the-shelf usage, reusability, simplicity etc. (Corradini et al. 2005). Semantic
fusion is a process that unifies input at a “meaning level” from various modalities that are part of a multimodal
system (Gibbon et al. 2000). It is said to occur in two steps: a) input events for a user’s command are taken
from various modalities and fused at a low level to form a single multimodal input event that signifies the
113
Action