MobileBIT: - Instituto de Telecomunicações

MobileBIT:A Framework for Mobile Interaction Recording and DisplayMichel Cânovas 1 , Hugo Silva 1 , André Lourenço 1,2 and Ana Fred 11 Instituto de Telecomunicações, Instituto Superior TécnicoAvenida Rovisco Pais, 1, 1049-001 Lisboa, Portugal2 Instituto Superior de Engenharia de LisboaRua Conselheiro Emídio Navarro, 1, 1959-007 Lisboa, Portugal{michel.canovas, hsilva,afred}@lx.it.pt, alourenco@deetc.isel.ipl.ptKeywords:Abstract:Android, Smartphone, ECG, Rapid Application DevelopmentThe proliferation of mobile devices and smartphones, together with their built-in sensors and the ability toeasily connect to other peripherals using Bluetooth, is enabling new and promising signal processing applications.However, the development of mobile applications that use these features is still in the early days,requiring advanced development skills. In our work, we propose a generic framework for rapid prototypingin a mobile environment. The proposed framework enables real-time data acquisition, processing, recording,communication and visualization. As proof of concept we present an application used for electrocardiographic(ECG) signals monitoring.1 INTRODUCTIONNowadays, smartphones and tablets devices play afundamental role, enabling ubiquitous access to realtimeinformation in an always-on/always-connectedapproach (Wan et al., 2009). However, applicationdevelopment can be a thorough and time-consumingprocess for researchers, which often falls out of thescope of their core interests.Considering this, the main motivation behind ourwork is to create a tool that addresses real-timebiomedical signal acquisition, processing, recording,communication and display, using mobile or webplatforms. Our ultimate purpose is to give biomedicalresearchers, physicians, psychologists or otherhealth-related professionals, an intuitive tool to easilydevelop mobile signal processing and communicationapplications tailored to their needs. These can be usedin telemedicine and mobile monitoring of health, improvingquality of care and independent living (Wanet al., 2009; Silva et al., 2011a; Silva et al., 2011b).2 SYSTEM ARCHITECTUREThe architecture is arranged in layers decoupling thepresentation from the processing counterpart. It usesthe Model-View-Controller (MVC) architecture, asproposed by Silva et al. (Silva et al., 2012). Moreover,the definition of the application is made by assemblinga network of functional blocks that runs ontop of a Workflow Manager (WFM). This network definesthe behavior of the application, in such a waythat each block performs a specialized task, includingdata acquisition from a source, signal processing, anddisplay of results.The WFM is the core of the framework. It dealswith the instantiation and control of a previously designedsystem, described according to a structure definedin JSON. This structure uses a specification,called Data Processing Language, that is automaticallyloaded and instantiated at runtime by the framework.Figure 1 gives an overview of the architecture.2.1 Functional BlocksThese are the operating units of the system; it is theirconfiguration and combination that defines the ultimatebehavior of the end-user applications designedwith the proposed framework. All blocks must extendan abstract class, which includes methods to implementthe observer pattern (Cooper, 2000) and otherconvenient methods which give handle to the WFM.The communication between blocks is made throughan event-based Message Passing Protocol using theAndroid OS Message.class.

It has a simple organization as follows:{"":{"config": {"":"",...},"in": {"":""|"/",...},"out": {"":"",...}},"":...}This structure can be seen as a collection ofkey/value pairs through which the blocks, that theend-user application will use, are identified, andtheir basic configuration information is provided. Aslong as there is a correspondence between the representedblocks (CLASS in "") and Javaclasses (CLASS.class, in Java), the WFM will beable to instantiate and build the defined system in runtime.This structure is not only easily read but can alsobe written or edited very simply. Future work will befocused on having this structure generated through avisual programming tool and fed to the applicationautomatically.Figure 1: Architecture of our demo application, wherebiosignals are acquired from an external hardware device,displayed in a mobile device through a rich user interface,and sent to a remote server for efficient processing; the resultsof the processing stage is returned back to the phonefor presentation to the user.Conceptually, the framework uses three types ofblocks: a) the normal blocks, which take input data,perform a specific task, and output results; b) theSources, which are a special type of blocks that onlyhave outputs; c) Sinks, that are a special type of blockwhich only have inputs. Our framework has the underlyingassumption that the functional blocks aretriggered by the output data produced by the Sourceblocks, and thus, a normal system operational flowstarts on the sources and generally ends on Sinks (aspart 1 of Figure 1).All the information about the type, input/outputconnections and configuration of the blocks, which,overall, characterize a realization of a system, is representedusing a structure defined according to theData Processing Language, which will be further detailedin the next section.2.2 Data Processing LanguageThe description of a system to be instantiated by theWFM, is described using a JSON-based specificationto which we called Data Processing Language (DPL).2.3 Workflow ManagerThe WFM is the core of the mobile framework, andit manages the execution and interaction between thedifferent blocks. It contains a handle to every importantinstance of the system, and it is through the WFMinstance that most of the others communicate.The most demanding task performed by the WFMis the initial network setup from the DPL structure.As soon as the end-user application starts, a JSONstructure, following the specification described in theprevious section, is fed to the WFM. Then, using athe high performance JSON processing library, allblocks are instantiated sequentially, configured, andconnected among themselves through their input andoutput channels.Once the whole system is assembled, all that remainsto be done is to trigger it, namely by startingthe sources. Then, the arrival of the data itself to theinput of the next block triggers its operation, and soon. Finally, the processed data will arrive to a sinkwhich gets the flow to an end. Sinks end the flow eitherby saving, streaming or displaying their input.Usually, this action of starting the sources, istriggered by an UI event. Since the UI is shownthrough a Webview, which, in turn displays HTMLcontent, the native part of the application must beable to exchange events with the Javascript side.Hence, the Webview class provides a method calledaddJavascriptInterface, that binds a Java objectto Javascript. This object turns out to be the interfacebetween the Java (the native Android) and Javascript

(in the HTML page) worlds (further explanation continueson the next section).The WFM communicates with this interface(hereafter called JSI) through messages containingcommands to be executed. For this reason, the WFMhas also the very important task, which is to parse andexecute these command messages. Likewise, sincethe only way to the JSI is across the WFM, all datathat needs to be displayed to the user has to arrive tothe WFM to be correctly dispatched.2.4 User InterfaceThe UI is, as in most applications, one of the mostimportant parts of the platform. It needs to be able notonly to accurately display the data to the user, but alsoto get his inputs, all this in intuitive and aestheticalways (Tidwell, 2011).Another concern taken into account in the designof our framework was the unification of the layoutof the platform, between the mobile or other clients.This led to the use of the Android OS Webview component,that enabled the interaction and display ofHTML content, which also allows the use of exactlythe same UI both on the mobile phone, and on a standardPC interface.The presentation layer is decoupled from the processingbackbone. In the Android case, the Javascriptinterface is configured to receive commands from JSI;in a client running in a browser, it is configured to receivecommands from a Websocket.The mechanism behind this versatility is hidden inthe script contained in the HTML file. This script isresponsible for the seamless behavior of the UI eitheron the browser or in the Android: all the commands tobe executed by the JavaScript world are handled by anobject bound to the JSI instance in Android. So, if itdoes not exist, it is created as a JS websocket, linkingthe page in the browser directly to the server.In order not to computationally overload the Webkitengine, responsible for rendering the HTML pagewith the layout (as it does in the built in browser ofthe OS) all the processing is done at the Java backend,before reaching the Webview, since, in addition,we believe that, in Android, native methods are fasterand more efficient than web scripts.3 MOBILEBIT PROOF OFCONCEPT: CARDIOWATCHTo demonstrate the potential of our framework, ademo application was built, entitled ”CardioWatch”.The goal of this application is to show the subjectheartbeat, measured using a bioPLUX, biosignal acquisitiondevice connected to an 1 lead ECG sensor.The acquisition unit is interfaced with the Android environmentvia Bluetooth. On the other end, there isa remote server that performs the signal processing,which is accessible over the web.Inside the Android-powered device this is materializedthrough a system composed by a sequence offour blocks: a BioPlux, a Websocket, a FlotBlockand an HTMLfield, which are, respectively, the virtualrepresentation of: the acquisition device; the implementedsocket connection to the server; the ECGline chart; and the heart rate (HR) numeric display.The Data Processing Language structure, correspondingto our application example, is the following:{"BioPlux:Bio":{"config":{"MACaddress":"12:34:56:78:9A:BC"},"in":{},"out":{"0":"ECG"}},"Websocket:Wskt":{"config":{"Port":"9999","Host_URL":"123.456.7.890"},"in":{"0":"ECG"},"out":{"0":"SV","1":"HR"}},"FlotBlock:Flot":{"config":{"FlotDiv_id":"graphHolder"},"in":{"2":"SV"},"out":{}},"HTMLfield:HR":{"config":{"FieldDiv_id":"hr"},"in":{"0":"Wskt/1"},"out":{}}}Here we can see how the connections between theinput (in) and output (out) channels of the blocks areperformed and also how the configuration parameters(fields under config) vary amongst the blocks. Eachblock extends the abstract Block class, although theyperform very distinct tasks.For instance, the BioPlux block implements methodsfor controlling a bioPLUX (Gamboa and Silva,2007) device via Bluetooth. This device is connectedto a pair of electrodes that gathers the ECG signalfrom the hands, at a configurable sampling frequencythat can go up to 1000Hz, and sends them to the Androiddevice, namely to the BioPlux block (whichends up being a virtual image of the actual device).The Websocket is a block that implements a fullduplexsingle socket connection (Websocket, 2012)over which messages can be sent/received to/from aremote server. With this, the gathered signal can beremotely filtered, analyzed and returned along withsome of its features. In this demo case, at the server,the signal is filtered, downsampled, and the instantheart rate is computed and retrieved. This functionalityprovides a rapid prototyping framework of com-

plex algorithms in a mobile environment, that otherwisewould require more time and effort to implementusing more traditional programming approaches.At the mobile device, the Websocket block receivesthe JSON message from the server, parses it,and notifies the FlotBlock and HTMLfield blocks, topresent the data to the user interface, namely the realtime,filtered ECG signal and the heart rate. FlotBlockowes its name to the Javascript library used for plottingin jQuery, that produces graphical plots of arbitrarydatasets on-the-fly on the client-side(Flot, 2012).The application connects automatically to theserver on start-up, and as soon as the start button ispressed, the Android device connects to the bioPLUXunit and starts to receive real time data. The bottompanel displays the processed data, namely the filteredECG signal trace, and the heart rate information.4 CONCLUSION AND FUTUREWORKIn this paper we presented a rapid application developmentframework for mobile devices. It can be usedboth for standalone or client/server applications.The presented framework is based on a MVCparadigm for rapid application development, thatcano be used by researchers and clinicians to facilitatethe prototyping of mobile applications.Alongside, we have devised a Data ProcessingLanguage, that can be used to easily define the structureof the end-user application.Furthermore, the UI of our mobile framework issupported on web technologies, allowing easy configurationof the layout, look-and-feel, among otheraspects of the graphical interface with the user, whichis fundamental given the latest trends in mobile applicationdevelopment.The proposed architecture has undoubtable benefits,namely the ones arising from its versatility, dueto the fact that the user can define his application behaviorby combining the different blocks, and reusethe layout of a web UI.Due to its sophistication, we assessed the performanceof our framework in terms of computationtimes and message exchange robustness. The profilingdone to the demo version shows that the systemcomponents running in the native part of the Androidhave sub-millisecond performance. Since this is areal-time demo applications, and one that communicateswith a remote server through the web, most ofthe time consumed in this communication.In the overall, tests have shown that our frameworkperforms adequately for rapid prototyping ofmobile applications. The example application presentedserves not only as a proof of concept but alsoas way to demonstrate the potential of the platform.The integration of the Websocket block to communicatewith a remote server, provides a testbed for functionalitiesthat can be supported by algorithms alreadyfeatured on the server toolbox.ACKNOWLEDGEMENTSThis work was partially funded by the IT - Instituto deTelecomunicações under the grant ”Android BiometricsPlatform”, by the Fundação para a Ciência e Tecnologia(FCT) under grants SFRH/BD/65248/2009and SFRH /PROTEC/49512/2009, whose support theauthors gratefully acknowledge. The authors alsothank the Institute for Systems and Technologiesof Information, Control and Communication (IN-STICC), the graphic designer André Lista, Prof. PedroOliveira, and the Instituto Superior de Educação eCiências (ISEC), for their support to this work.REFERENCESCooper, J. (2000). Java Design Patterns: A Tutorial.Addison-Wesley.Flot (2012). Flot. http://code.google.com/p/flot/ (last accessedon 30/08/2012).Gamboa, H. and Silva, H. (2007). bioPLUXwireless biosignal acquisition system.http://www.bioplux.com/bioplux.Silva, H., Lourenço, A., and Paz, N. (2011a). Real-timebiosignal acquisition and telemedicine platform forAAL based on Android OS. Proc INSTICC InternationalLiving Usability Lab Workshop on AAL LatestSolutions, Trends and Applications - AAL, Rome,Italy.Silva, H., Palma, S., and Gamboa, H. (2011b). AAL+: Continuousinstitutional and home care through wirelessbiosignal monitoring systems. Handbook of DigitalHomecare, pages 115–142.Silva, M., Guerreiro, T., Gonçalves, D., and Silva, H.(2012). A web-based application to address individualinterests of children with autism spectrum disorders.DSAI 2012: 4th International Conference on SoftwareDevelopment for Enhancing Accessibility and FightingInfo-exclusion, Douro, Portugal.Tidwell, J. (2011). Designing Interfaces. O’Reilly Media,second edition edition.Wan, D., Mentzer, K., and Nash, R. (2009). Always on,always connected how technology will transform thefuture of chronic care. Technical report, Accenture.Websocket (2012). Websocket. http://www.websocket.org/(last accessed on 30/08/2012).