User Interface Service Software Developerís Guide - Hitech Projects

More documents

Recommendations

Info

January 2008 Public 3.3.3.4 Component interfaces IContextSource Containing the following methods: public String query(String contextQueryExpression); public String subscribe(String contextSubscriptionCharacterisation, String contextSubscriptionReference); public boolean unsubscribe(String contextSubscriptionID); IDialogueManager Containing the following method: IDialogueManager::publish(Data) Creates a new entry on the blackboard. This entry contains a user input or a system output. Note that no knowledge source can actually delete an existing entry on the blackboard. At most, they can mark any entry as “having been processed”, but the actual deletion can only be realized by the blackboard controller. Every item that is published onto the blackboard shall conform to the following structure, which is undefined yet, but which shall minimally contain the following fields: • ID: unique identifier (given by the blackboard controller) • Time: time of publication • Type: speech input, gesture input, graphical output, … • Semantic content: describes the semantic content of the input or output. • Ambiguous [Boolean]: when the input is ambiguous (e.g. the user command “delete this”), it shall be noted here. The Multi-modal Fusion Module can then try to resolve such ambiguities by combining several. • ProcessedBy: the list of clients id that have processed this item so far. • Priority: Several knowledge sources may change the priority of treatment of this item: for instance, when concurrent outputs shall be delivered to the same user. It may be possible to further provide a “subscription helper” that simplifies the subscription mechanism for external applications: the idea would be to implement a simple subscription() function (with attribute/value matching instead of a full SPARQL query) that activates the complete subscription() method and returns another helper, which can be used by the application to simplify accordingly the SPARQL answer. 3.3.4 Multimodal Fusion 3.3.4.1 Component interface 3.3.4.1.1 Multimodal Fusion Interface The javadoc for the MFM interface is: Method Summary void cancelUserGesture(java.lang.String timeStamp) Signal that a gesture movement has stopped without any information void cancelUserSelection(java.lang.String timeStamp) Amigo IST-2004-004182 87/114
January 2008 Public Signal that a selection movement has stopped without any information void cancelUserSentence(java.lang.String timeStamp) Signal that a speech event has stopped without any information void setAppliConcepts(java.lang.String owlOntology) Set the ontology to be used to test semantic compatibility of objects void setAppliReferenceableCommands(java.lang.String mmilCommands) Set the commands managed by the application. void setAppliReferenceableObjects(java.lang.String mmilObjects) Set the objects managed by the application. void setUserGesture(java.lang.String mmilInputGesture, java.lang.String startStamp, java.lang.String endStamp) Set a gesture's meaning (the mmil component encapsulated in the XML String must have at least one participant) void setUserSelection(java.lang.String mmilInputSelection, java.lang.String startStamp, java.lang.String endStamp) Set a referential selection event (the mmil component encapsulated in the XML String must have at least one event) - the origin of the selection itself is not relevant but can be expressed (GUI event, Gesture event). void setUserSentence(java.lang.String mmilInputSentence, java.lang.String startStamp, java.lang.String endStamp) Set a referential expression (the mmil component encapsulated in the XML String must have a participant) void signalUserGesture(java.lang.String timeStamp) Signal that a gesture movement has started, but is not yet completed void signalUserSelection(java.lang.String timeStamp) Signal that a selection movement has started, but is not yet completed void signalUserSentence(java.lang.String timeStamp) Signal that a speech event has started, but is not yet completed The format of timestamp parameters is like this: 2006-08-24T14:22:20.358+02:00 (standard XML Gregorian Calendar format, for java, use javax.xml.datatype.XMLGregorianCalendar). The MMIL messages examples are available in the appendix. Amigo IST-2004-004182 88/114
Page 1 and 2:
IST Amigo Project Deliverable D4.7
Page 3 and 4:
January 2008 Public Table of Conten
Page 5 and 6:
January 2008 Public 3.2.3.4 Configu
Page 7 and 8:
January 2008 Public 1 Speech based
Page 9 and 10:
January 2008 Public Bugs - Patches
Page 11 and 12:
January 2008 Public 1.2 Deployment
Page 13 and 14:
January 2008 Public SWIsvcMonPort=
Page 15 and 16:
January 2008 Public # RealSpeak por
Page 17 and 18:
January 2008 Public 6666 20 10
Page 19 and 20:
January 2008 Public true C:/Temp/
Page 21 and 22:
January 2008 Public [Recording_TCP]
Page 23 and 24:
January 2008 Public Example string
Page 25 and 26:
January 2008 Public 1.3.2 Explicit
Page 27 and 28:
January 2008 Public • the user sp
Page 29 and 30:
January 2008 Public parameters. The
Page 31 and 32:
January 2008 Public In this example
Page 33 and 34:
January 2008 Public Assuming that y
Page 35 and 36:
January 2008 Public Step 3 Rebuild
Page 37 and 38: January 2008 Public 1.4.1.3 Run Tra
Page 39 and 40: January 2008 Public On a successful
Page 41: January 2008 Public At the end the
Page 44 and 45: January 2008 Public At the end the
Page 46 and 47: January 2008 Public Step 5 In the S
Page 48 and 49: January 2008 Public Step 8 In Serve
Page 50 and 51: January 2008 Public 1.4.2.5 Run Voi
Page 52 and 53: January 2008 Public 1.4.2.6 Run Voi
Page 54 and 55: January 2008 Public As soon as the
Page 56 and 57: January 2008 Public • SoapBox bas
Page 58 and 59: January 2008 Public 2.2.1.1.4 Confi
Page 60 and 61: January 2008 Public The gesture ser
Page 62 and 63: January 2008 Public • Folder Volu
Page 64 and 65: January 2008 Public After installat
Page 66 and 67: January 2008 Public Up 65 In
Page 68 and 69: January 2008 Public Figure below pr
Page 70 and 71: January 2008 Public Development sta
Page 72 and 73: January 2008 Public Tools Oscar Fil
Page 74 and 75: January 2008 Public Bugs None Patch
Page 76 and 77: January 2008 Public 3.2 Deployment
Page 78 and 79: January 2008 Public Scene descripti
Page 80 and 81: January 2008 Public 3.2.2.5 Comp
Page 82 and 83: January 2008 Public 3.3 Component A
Page 84 and 85: January 2008 Public 3.3.3 Multimoda
Page 86 and 87: January 2008 Public Fallback soluti
Page 90 and 91: January 2008 Public 3.4 Tutorial 3.
Page 92 and 93: January 2008 Public scene, the imag
Page 94 and 95: January 2008 Public objectDesc VCR
Page 96 and 97: January 2008 Public 3.5 Assessment
Page 98 and 99: January 2008 Public 3.6 Appendix 3.
Page 100 and 101: January 2008 Public Intended audien
Page 102 and 103: January 2008 Public 3.9 Component A
Page 104 and 105: January 2008 Public interface input
Page 106 and 107: January 2008 Public Figure 7 Classe
Page 108 and 109: January 2008 Public different sub-m
Page 110 and 111: January 2008 Public Figure 3: Menu
Page 112 and 113: January 2008 Public 30% 10% 7% 0% 5
Page 114 and 115: January 2008 Public Figure 10: Conc
show all

User Interface Service Software Developerís Guide - Hitech Projects

Create successful ePaper yourself

Delete template?

Save as template?