A Scalable Approach to Human-Robot Interaction - USC Robotics ...

odometry for laser beacon-based localization andnavigation. The sonar is used for obstacle avoidance.The first system is a multi-robot, multi-target trackingsystem [9], configured to detect three different targetsidentified by colored beacons. Hence, three services aregenerated and provide target location information. For theexperiments, the system consists of three robotsconfigured to be either stationary or dynamic trackers.Using the infrastructure, the system client can manage upto 10 requests simultaneously.The second system offers one service and consists of adelivery robot that navigates to goal locations provided bythe requesting user. It provides task status details ofproximity to the goal location and the robot’s currentpose. A new goal location can be entered at any time. Thedelivery robot has a ‘home’ location in an office where itreturns to whenever it is not tasked. For long durationexperiments, this location could contain a docking stationfor recharging.The two systems share the same environment map andcoordinate reference frame so location information iscommon between them. This map was used in the earliersimulation experiments. No calibration had to be carriedout for real robot localization, which is a testament to theaccuracy of the map used in Stage and the effectiveness ofusing Stage for real robot system development.Experiment 1 – Small Scalability AnalysisThe first experiment involves several users requestingtasks from the systems. The three tracking robots areconfigured to act as stationary sensors. The area of thetracking system’s coverage is shown by the gray areas inFigure 3. The target to be tracked is the delivery robotwhich is tasked through the infrastructure to travel tolocations in the corridors within the tracking system’scoverage.The infrastructure operates as follows. The systemsconnect to the infrastructure, register their services andwait for requests. The users connect to the infrastructureand receive the global list of services, consisting of thethree tracking services for the differently colored targetsand one delivery service. The users select whicheverservice they desire and are connected to the applicablesystem. They then have a one-to-one interaction for eachsystem they are connected to. Additionally, they retainaccess to the global service list for many-to-manyinteraction.The experiment ran for approximately 50 minuteswith 10 users connecting to the infrastructure andperiodically requesting and terminating services. Figure 4shows the server’s performance during the experiment.The data shown represents the time for the server toprocess service request, list request, and servicetermination messages. The instantaneous number ofonline users is also shown. As can be seen, the number ofusers has little effect on the response times. The mainreason for this is that the infrastructure’s operations areevent-based on the user’s side. Therefore, connected usersrequire no maintenance when there is no activity. On thesystem side, a ping signal is sent from the server every 20seconds to ensure all systems connected to it are stillserviceable. A system has 10 seconds to respond before itand its services are removed from the server.50454035302520151050RequestListTerminate#Users1 306 610 913 1217 1522 1828 2132 2436 2739 3044Time (s)Figure 4. Server response times for Experiment 1. The numberof users connected at any point in time is represented by thesolid line and the right y axis. The server’s response times to thevarious messages are shown by the symbols and correlate to theleft y axis.While the server’s response times to requests provideevidence of its efficiency, the response times from theuser’s perspective provides a better indication to theefficiency of the infrastructure. In addition to the timetaken by the server to process the requests, thecommunication delays between infrastructure layers,system client processing times and UDP packet queueprocessing can also contribute to these response times.Table 1 lists the statistics pertaining to requests forservices and service lists. These constitute the two mosttime-intensive messages used in the infrastructure. Thetimes are measured from when the user made the requestto when a response was received which is either from theserver if the request is refused or from the system tocommence interaction.Table 1. Statistics from the user’s perspectiveService ListRequestsServiceRequestsNumber of Requests 30 19Maximum Response Time 170 ms 64 sMedian Response Time 43 ms 691 msMinimum Response Time 5 ms 38 ms1210864201668

Requests for the global service list from the server aregenerally carried out quickly since the transactioninvolves only the user clients and the server. Requests forservices take longer since they are processed by the serverand the system clients. Apart from the transition times, itmay take some time for the system client to respond to theuser’s request depending on its ongoing activities. For thetracking system, one robot was responsible for managingall users’ requests in addition to its tracking duties. As aconsequence of these duties and the limited processingcapability and memory on the robot, the response timessometimes took several seconds.The statistics for the service requests are listed inTable 2. The users selected 19 tasks, 14 of which weregranted. Three tracking service requests were refused dueto the maximum quota of 10 simultaneous requests beingexceeded 1 . The system client notified the users in thesecases. Two request failures occurred due to the robotbeing overburdened with too many requests to managesimultaneously. Although the maximum number ofrequests that the system client manages was set to 10, themaximum number the robot’s internal system couldreasonably sustain while carrying out its part as a trackerwas eight. During peak activity, eight users were beingmanaged by the tracking system for more than 20minutes, providing updates at 1 Hz.Table 2. Service statistics showing the number of tasks selectedand service running timesDeliveryServiceTrackingServiceNumber of Granted Tasks 3/3 11/16Maximum Time 1754 s 2057 sMedian Time 341 s 1478 sMinimum Time 185 s 2 sExperiment 2 – Service LinkingThe second experiment demonstrates the ability of theinfrastructure to link heterogeneous services to provide ahigher-level function to the user. It is similar to ourprevious service-linking experiment [17] except it iscarried out in a real building environment rather thansimulated open space.In this experiment, both tracking and delivery systemsare used. The tracking system provides users with thelocations of particular colored targets whereas thedelivery robot requires a location to deliver to. By linkingthese semantically similar services, a dynamic locationdelivery function is created. In this scenario, the deliverytarget is a person wandering around the environment.1 Even though there were only 10 users, a user could request morethan one task from a system if the system allows it.The robots are configured the same as Experiment 1,except one of the tracking robots is now mobile (T3 inFigure 3). Although this robot can traverse the wholeenvironment while searching and tracking, it is confinedto the corridor where the stationary trackers are located.Once the systems and the infrastructure are connected,a user logs on and selects the target tracking service todetermine whether the target is currently being tracked.Once it is, the user selects the linking option from theservice menu, which prompts for the data-providingservice and the command service. These are selected asthe applicable target tracking and delivery servicerespectively.The operation of the infrastructure at this point is thesame as if two separate services were selected. Bothconnect to the user client, which checks if the data serviceis part of a link. Since it is in this case, the information issent to the command service’s system client as well asbeing displayed to the user. The command service’ssystem client parses the received information for the goallocation. This effectively automates the user’s manualgoal location entry. Consequently, the delivery robot isconstantly provided with the updated target location. Inthe cases when the target is lost or there is no detectedlocation information in the data sent to the delivery robot,it proceeds to the last provided location. This helps dealwith temporary communication failures and target loss.Once the delivery robot is within a meter of the goallocation, it halts and sends a message to the user that it isat its commanded location.In this experiment, the tracking information issometimes provided by the mobile tracking robot, whichresulted in a two-robot convoy following the target. Thisapplication of a linked service could easily be extended togenerate a leader-follower behavior for multiple robots ifthey need to be relocated en masse.5 ConclusionsIn the context of developing a scalable human-robotinteraction mechanism, we have assumed that the systemsinvolved have some degree of autonomy. This assumptionallows a general and simple mechanism to be developed.Our interaction infrastructure follows this paradigm andallows users to interact with systems at two levels: manyto-manyand one-to-one. The many-to-many level isrequired to expose the tasks or robots’ services to thehumans, and upon selection the users have more personalone-to-one interaction. These levels are expected to berequired in large scale robot applications such asassembling solar panels in space or robot-assisted searchfor casualties after an earthquake. The experimentspresented in this paper demonstrate the infrastructure’sapplication towards these types of applications.1669

A future step for enhancing the infrastructure is todistribute the server to provide greater redundancy. Thiswill allow more robust operation in the presence of eithercommunication or server failures. Currently, if the serverfails, the infrastructure is unusable for new interactions,however any existing interaction between systems andusers remains unaffected.Establishing the potential numbers of systems andusers the infrastructure can support is important butlogistically difficult with real humans and systems. It canhowever, be analytically determined by developing aqueuing model from the infrastructure’s performance.Results obtained from the model can then be correlated insimulation.AcknowledgementsThis work is supported in part by the DARPA MobileAutonomous Robot Software (MARS) Program undergrants DABT63-99-1-0015 and 5-39509-A (via UPenn),the DARPA Software for Distributed Robotics (SDR)Program under contract 4400057784 (via SAIC), and theONR DURIP program under grant 00014-00-1-0638.Bibliography[1] A. Agah, "Human Interactions with IntelligentSystems: Research Taxonomy", Computers andElectrical Engineering, 2001, no. 27, pp. 71-107.[2] K. Ali, and R. Arkin, "Integration of Reactive andTelerobotic Control in Multi-agent RoboticsSystems", Proceedings, From Animals to Animats 3:Simulation of Adaptive Behavior 1994, pp. 473-478,MIT Press.[3] R. Arkin, T. Collins, and Y. Endo, "Tactical MobileRobot Mission Specification and Execution",Proceedings, SPIE-99, Mobile Robots XIV, Boston,MA, September, 1999, pp. 150-163.[4] I. Elhajj, J. Tan, N. Xi, W. Fung, Y. Liu, T. Kaga, Y.Hasegawa, and T. Fukuda, "Multi-site Internet-basedCooperative Control of Robotic Operations",Proceedings of the 2000 IEEE/RSJ InternationalConference on Intelligent Robots and Systems,Takamatsu, Japan, 2000, pp.826-831.[5] A. Fox, B. Johanson, P. Hanrahan, and T. Winograd,"Integrating Information Appliances into anInteractive Workspace", IEEE Computer Graphicsand Applications, May/June, 2000, pp. 54-65.[6] B. Gerkey, R. Vaughan, K. Støy, A. Howard, G.Sukhatme, and M. Matari , "Most Valuable Player: ARobot Device Server for Distributed Control",Proceedings of the IEEE/RSJ InternationalConference on Intelligent Robots and Systems (IROS2001), Wailea, Hawaii, Oct. 29-Nov. 3, 2001, pp.1226-1231.[7] S. Goldsmith, S. Spires, and L. Phillips, "ObjectFrameworks for Agent System Development", SandiaNational Laboratories, Tech. Rep. SAND98-0742A,1998.[8] H. Jones, and M. Snyder, M. "Supervisory Control ofMultiple Robots Based on a Real-time Strategy GameInteraction Paradigm", Proceedings of the 2001 IEEEInternational Conference on Systems, Man, andCybernetics, Tucson, AZ, Oct. 2001, pp. 383-388.[9] B. Jung, and G. Sukhatme, "A Region-basedApproach for Cooperative Multi-Target Tracking in aStructured Environment," IEEE/RSJ InternationalConference on Intelligent Robots and Systems, 2002,Sep. 30-Oct. 4, pp. 2764-2769.[10] J. Lee, and H. Hashimoto, "Intelligent Space - Itsconcept and contents", Proceedings of the 2000IEEE/RSJ International Conference on IntelligentRobots and Systems (IROS), Takamatsu, Japan, 2000,vol. 2. pp. 1358-1363.[11] R. Murphy, and E. Rogers, "Cooperative Assistancefor Remote Robot Supervision", Presence, SpecialIssue, vol. 5, no. 2, pp. 224-240, 1996.[12] L. Phillips, S. Goldsmith, and S. Spires, "CHI: AGeneral Agent Communication Framework", SandiaNational Laboratories, Tech. Rep. SAND98-2825C,1998.[13] S. Ponnekanti, B. Lee, A. Fox, P. Hanrahan, and T.Winograd, "ICrafter: A Service Framework forUbiquitous Computing Environments", Proc.Ubiquitous Computing Conference. Lecture Notes inComputer Science", vol. 2201, 2001, pp. 56-75.[14] P. Rybski, I. Burt, T. Dahlin, M. Gini, D. Hougen, D.Krantz, F. Nageotte, N. Papanikolopoulos, and S.Stoeter, "System Architecture for VersatileAutonomous and Teleoperated Control of MultipleMiniature Robots", Proceedings of the 2001 IEEEInternational Conference on Robotics andAutomation, Seoul, Korea, 2001.[15] H. Surmann, and M. Theißinger, "ROBODIS: ADispatching System for Multiple AutonomousService Robots", Proceedings of the FSR'99 RoboticsApplications for the Next Millenium, Pittsburgh, PA,1999, pp. 168-173.[16] A. Tews, G. Sukhatme, and M. Matari , “AnInfrastructure for Large-Scale Human-RobotInteraction”, Technical Report CRES-02-001, Centerfor Robotic and Embedded Systems, University ofSouthern California, August, 2002.[17] A. Tews, M. Matari , G. Sukhatme, and B. Gerkey,“G’day Mate. Let me Introduce you to Everyone: AnInfrastructure for Scalable Human-SystemInteraction”, Technical Report CRES-02-004, Centerfor Robotic and Embedded Systems, University ofSouthern California, August, 2002.1670