User Interface Service Software Developerís Guide - Hitech Projects

More documents

Recommendations

Info

January 2008 Public 3.3.3 Multimodal Dialogue Manager 3.3.3.1 Limitations and scope of services The UIS Dialogue Manager (DM) is a blackboard based system i.e. shared memory (database) area where applications and modalities, i.e. DM clients, can share their inputs and outputs. DM manages the communication between modality services (speech, GUI gestures…), Multi Modal Fusion (MMF) and applications. The architecture is flexible making it possible to add new modalities, applications and data abstraction modules (such as MMF and application specific dialogue managers) as the system grow. Figure 1 presents the conceptual architecture of the UIS Dialogue Manager. Applications, modality services and MMF first register themselves as UI event producers and consumers. When new applications are registered to DM they have to provide their user interface XML description (See GUI service ServiceDescription parameter format), which describes the functions and capabilities of the application. DM will pass the description to XUI service which generates the optimised and personalised UI representation that can be used in automatic generation of GUI and speech UI representations. Since UI generation is based on same user and environment models the presented output (GUI or voice) will have uniform look and feel. Application specific task and dialogue management is applications responsibility. DM provides flexible mechanism for UI event sharing and synchronisation, but it cannot provide application specific task management. GUI Applications Speech Service XUI Service 2D Gesture Service UIMessage 3D Gesture Service Direct modality access for certain devices Publish events Dialogue Manager UIMessage Subsribe Publish Fused events MMF Data Abstraction Module Subsribe to events BB cleaner Appl. Light Switching Appl. Owen Appl. ... Light1 Proxy Light2 Proxy Owen Proxy Figure 1. Conceptual architecture of the UIS. There are different reasons, why a user interacts with the Amigo System. Amigo UIS should support the following interaction modes: Amigo IST-2004-004182 83/114
January 2008 Public • Operation mode: The interaction has been initiated by the user. He wants to change something. So the goal of the interaction has to be told to the system or has to be discovered by the system. Using a GUI or speech the user has to select the operation out of the offered functionalities of the whole services. He has also to specify values needed for the responsible service. Because there could be more than one hundred functionalities offered by the services of one home, these functions have to be sorted into groups and an arbitrary number of subgroups to give the user a structured overview. The functions and its groups could then be represented for example by a menu hierarchy either by GUI or speech. The operation mode includes also the possibility to check the status of different services and the values of parameters of this service. To be used by the operation mode a service has to describe itself, to define possible user interaction functionalities and to give a model of its internal structure to the UIS. The UIS will use data models, dependency-graphs and user dependent menu logic descriptions to build a personalised menu structure. • Event mode: The event mode offers the system the possibility to initiate an interaction with the user. The dialog flow and the requested input parameters are fully controlled by the application interacting with the user. The event mode can be used for warnings, messages, questions etc. 3.3.3.2 Integration with the Context Management Service User interactions form relevant context information, especially when considering implicit user interactions. For instance, an application can react to specific contextual information, including user location, but also user gestures and user speech. Two options may thus exist: the first one is that every user interaction service implements the Context Source interface. We have not opted for this solution, as this may impact the performances of the overall UIS. Furthermore, it dramatically increases the variety of context sources related to user interfaces, and context consumers may have difficulties to choose the most relevant one. The second option consists in providing only one context source, namely the dialogue manager. Then, every user interface provider and consumer interacts with the dialogue manager, which centralizes all the contextual information related to user interaction. A typical usage scenario can take the form (Figure 2): Figure 2. Typical DM user scenario. Possible issues: The time delay induced by the CMS broker and the web service eventing mechanism may not be acceptable from the user point of view Amigo IST-2004-004182 84/114
Page 1 and 2:
IST Amigo Project Deliverable D4.7
Page 3 and 4:
January 2008 Public Table of Conten
Page 5 and 6:
January 2008 Public 3.2.3.4 Configu
Page 7 and 8:
January 2008 Public 1 Speech based
Page 9 and 10:
January 2008 Public Bugs - Patches
Page 11 and 12:
January 2008 Public 1.2 Deployment
Page 13 and 14:
January 2008 Public SWIsvcMonPort=
Page 15 and 16:
January 2008 Public # RealSpeak por
Page 17 and 18:
January 2008 Public 6666 20 10
Page 19 and 20:
January 2008 Public true C:/Temp/
Page 21 and 22:
January 2008 Public [Recording_TCP]
Page 23 and 24:
January 2008 Public Example string
Page 25 and 26:
January 2008 Public 1.3.2 Explicit
Page 27 and 28:
January 2008 Public • the user sp
Page 29 and 30:
January 2008 Public parameters. The
Page 31 and 32:
January 2008 Public In this example
Page 33 and 34: January 2008 Public Assuming that y
Page 35 and 36: January 2008 Public Step 3 Rebuild
Page 37 and 38: January 2008 Public 1.4.1.3 Run Tra
Page 39 and 40: January 2008 Public On a successful
Page 41: January 2008 Public At the end the
Page 44 and 45: January 2008 Public At the end the
Page 46 and 47: January 2008 Public Step 5 In the S
Page 48 and 49: January 2008 Public Step 8 In Serve
Page 50 and 51: January 2008 Public 1.4.2.5 Run Voi
Page 52 and 53: January 2008 Public 1.4.2.6 Run Voi
Page 54 and 55: January 2008 Public As soon as the
Page 56 and 57: January 2008 Public • SoapBox bas
Page 58 and 59: January 2008 Public 2.2.1.1.4 Confi
Page 60 and 61: January 2008 Public The gesture ser
Page 62 and 63: January 2008 Public • Folder Volu
Page 64 and 65: January 2008 Public After installat
Page 66 and 67: January 2008 Public Up 65 In
Page 68 and 69: January 2008 Public Figure below pr
Page 70 and 71: January 2008 Public Development sta
Page 72 and 73: January 2008 Public Tools Oscar Fil
Page 74 and 75: January 2008 Public Bugs None Patch
Page 76 and 77: January 2008 Public 3.2 Deployment
Page 78 and 79: January 2008 Public Scene descripti
Page 80 and 81: January 2008 Public 3.2.2.5 Comp
Page 82 and 83: January 2008 Public 3.3 Component A
Page 86 and 87: January 2008 Public Fallback soluti
Page 88 and 89: January 2008 Public 3.3.3.4 Compone
Page 90 and 91: January 2008 Public 3.4 Tutorial 3.
Page 92 and 93: January 2008 Public scene, the imag
Page 94 and 95: January 2008 Public objectDesc VCR
Page 96 and 97: January 2008 Public 3.5 Assessment
Page 98 and 99: January 2008 Public 3.6 Appendix 3.
Page 100 and 101: January 2008 Public Intended audien
Page 102 and 103: January 2008 Public 3.9 Component A
Page 104 and 105: January 2008 Public interface input
Page 106 and 107: January 2008 Public Figure 7 Classe
Page 108 and 109: January 2008 Public different sub-m
Page 110 and 111: January 2008 Public Figure 3: Menu
Page 112 and 113: January 2008 Public 30% 10% 7% 0% 5
Page 114 and 115: January 2008 Public Figure 10: Conc
show all

User Interface Service Software Developerís Guide - Hitech Projects

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?