D.-s. Eom et al.: A Multi-player Arcade Video Game Platformwith a Wireless Tangible User Interface1819A Multi-player Arcade Video Game Platformwith a Wireless Tangible User InterfaceDoo-seop Eom, Taeyoung Kim, HyunHo Jee, Hyoil Lee and JungHyun HanAbstract — A recent game development trend is naturaluser interfaces. This paper presents a motion-trackingplatform, which provides natural interfaces for multi-playerarcade video games. The platform is composed ofacceleration sensors and transmitters/receivers of radiofrequency and ultrasonic signals. The game players operatephysical devices in the real world, and the positions andorientations of the devices are computed and represented inthe virtual devices of the game world. The prototype provesthat various off-the-shelf wireless sensor nodes can beintegrated, resulting in a new type of natural interface forarcade video games 1 .Index Terms — arcade video game, user interface, motiontrackingContributed PaperManuscript received August 15, 2008I. INTRODUCTIONA recent game development trend is a natural userinterface which seamlessly integrates physical space andvirtual space. For example, recognition of a game player’sgesture often replaces traditional pointing devices such as amouse, such that the player can interact with virtual space in asmoother fashion. Similarly, a tangible interface enables thegame player to interact with virtual space via real-worldphysical objects, such that the state of the tangible objectsinstantiates key aspects of the digital state of virtual space.Such a natural or tangible interface is essential, especiallyfor arcade video games which provide interactive virtual 3Denvironments for game players. One of the key aspects ofconstructing virtual environments is tracking a player’smotion, and representing it in virtual environments.We designed and developed a motion tracking system usingvarious off-the-shelf wireless sensor nodes, and built aprototype of an arcade video game which was executed on thesystem. The motion tracking system comprised accelerationsensors and transmitters/receivers of radio frequency (RF) and1 This research was supported by the Ministry of Knowledge Economy,Korea, under the ITRC (Information Technology Research Center) supportprogram supervised by the Institute of Information Technology Advancement(IITA-2008-(C1090-0801-0046)) .Doo-seop Eom is with the Department of Electronics and ComputerEngineering, Korea University, Seoul, Korea (e-mail: email@example.com).Taeyoung Kim is with the Department of Electronics and ComputerEngineering, Korea University, Seoul, Korea (e-mail: firstname.lastname@example.org).Hyunho Jee is with the Department of Computer and RadioCommunications Engineering, Korea University, Seoul, Korea (e-mail:email@example.com).Hyoil Lee is with the Department of Computer and Radio CommunicationsEngineering, Korea University, Seoul, Korea (e-mail: firstname.lastname@example.org).JungHyun Han is the corresponding author, and is with the Game ResearchCenter and the Department of Computer and Radio CommunicationsEngineering, Korea University, Seoul, Korea (e-mail: email@example.com).0098 3063/08/$20.00 © 2008 IEEEultrasonic signals. In the prototype game, multiple playersoperated physical devices in the real world. Then, thepositions and orientations of the devices were computed andrepresented in the virtual devices of the game world, displayedon a wall-mounted screen. Furthermore, the angular velocitiesof the devices were computed, to create special effects in thegame. This paper presents the hardware architecture of themotion tracking system, and the software modules running onit, which detect the identities, poses and velocities of multiplephysical devices.The organization of the paper is as follows. Section 2reviews the related work. Section 3 presents the hardwarearchitecture and the software modules of the motion trackingsystem. Section 4 reports the experimental results andperformance analysis. Section 5 presents the implementationof a prototype game. Finally, Section 6 concludes the paper.II. RELATED WORKThe best-known location-sensing system is the GlobalPositioning System (GPS) . However, it does not workindoors, and therefore is not universally applicable. Manyindoor location-sensing systems have been developed toovercome this drawback. Active Badge  involves fixedinfrared receivers and mobile infrared transmitters, where aparticular transmitter is associated with the fixed location ofthe receiver that hears it. Using periodic infrared light pulsessent between transmitters and receivers, the position of thetransmitters is tracked at room-scale granularity. The RADARsystem  adopts RF communications. The RF signal strengthis used as an indicator of the distance between a transmitterand a receiver, and the location is computed from the distanceinformation through triangulation.For supplying positioning information with centimeterlevelaccuracy, a few fine-grained indoor location systemshave been proposed, using both RF and ultrasonic signals.Active Bat  adopts the architecture of fixed receivers andmobile transmitters. The system works in a synchronized orcentralized mode, controlled by an RF base station, and thereceivers estimate the distance to the transmitter using thearrival times of the ultrasonic signal. In contrast, Cricket, can be considered the opposite of the Active Bat:ceiling- or wall-mounted transmitters send RF and ultrasonicsignals to mobile receivers. Each transmitter broadcasts itslocation using an RF band. Simultaneously, the transmitteremits an ultrasound pulse. A receiver deduces its distancefrom the transmitter using the difference in arrival times of thefast RF signal and slow ultrasound signal. Note that Cricketworks in an asynchronous or decentralized mode. In suchfine-grained location-sensing systems, Hazas and Hopper investigated the use of broadband ultrasound for indoorAuthorized licensed use limited to: National Cheng Kung University. Downloaded on April 20, 2009 at 04:49 from IEEE Xplore. Restrictions apply.
1820positioning systems, in order to overcome the limitations ofnarrowband transducers.Obvious applications of location-sensing systems includegames that provide a virtual reality (VR) environment. In fact,a prototype game was built upon a Cricket-like VR gameplatform proposed by Eom et al . In the prototype,however, only a single player’s movement was detected. Alsonote that, in developing a VR game with a tangible interface,information about the orientations of a tangible object, as wellas its locations, is often quite useful. However, Active Bat andCricket compute the orientations in a brute-force way, byadding multiple ultrasound transmitters and receivers to themobile device, respectively. It is not only cumbersome butalso error-prone.IEEE Transactions on Consumer Electronics, Vol. 54, No. 4, NOVEMBER 2008of the game sticks. For this purpose, the sink node, beaconsand game sticks communicate using RF signals. An RFtransceiver (transmitter and receiver) is installed in each ofthem.The game sticks send ultrasonic signals to the beacons, formeasuring the distances between them. For this purpose, boththe beacon and the game stick have an ultrasonic sensor. Inaddition to the RF transceiver and ultrasonic sensor, the gamestick has a 3D acceleration sensor that measures theacceleration along three axes.III. ARCADE VIDEO GAME PLATFORMA. Hardware configuration(a) Sink node(b) Beacon(c) Game stickFig. 2. Wireless sensor nodesFig. 1. Game spaceThe physical space of the arcade video game is illustratedin Fig. 1. The hardware platform comprises a game-service PC,a sink node, a wide screen, beacons spatially distributed in thegame space (typically located at the corners of the screen), andthe game sticks operated by the game players. The gameplatform heavily relies on wireless communications, where thesink node, beacon, and game stick constitute the main wirelesssensor nodes. (In the current experiments, the sink nodecommunicates with the game-service PC via serialcommunication; however, it can be seamlessly replaced by RFcommunication.)Table IComponents of the wireless sensor nodesRFUltrasonicsensorAcceleration sensortransceiverSink node XBeacon X XGame stick X X XTable I shows the components of the wireless sensor nodes.The sink node synchronizes and coordinates the beacons andgame sticks, and then collects the distance/acceleration dataFig. 2 shows the three wireless sensor nodes of the gameplatform. The sizes of the sink node and the beacon are8.9cm×12cm×5.1cm and 8.2cm×7.9cm×6.6cm, respectively.The game stick is 50-cm long, and only its handle is shown inFig. 2-(c).B. Computation of distances and accelerationThe sink node plays a key role in the proposed gameplatform: It collects the distance/acceleration data of the gamesticks. Fig. 3 shows the flowchart of the sink node’s algorithm.Firstly, the sink node broadcasts RF signals to all game sticks.Each game stick responds to the broadcast with an RFmessage, which contains its ID. Thus, the sink node identifiesevery active game stick. On the other hand, the beacons arefixed, and their IDs are known to the sink node in advance.The sink node synchronizes the beacons and the active gamesticks by sending an RF message START to them.All nodes in the game platform work in a synchronizedmode. The game sticks sequentially send ultrasonic signals toall of the beacons. When a beacon receives an ultrasonicsignal, it uses the standard time-of-arrival technique toestimate the distance of the game stick. Suppose that we havefour beacons. The distances between the beacons and a gamestick, d 1 , d 2 , d 3 and d 4 , are computed simultaneously. Then, thefour beacons sequentially deliver the estimated distances tothe sink node in RF messages.Authorized licensed use limited to: National Cheng Kung University. Downloaded on April 20, 2009 at 04:49 from IEEE Xplore. Restrictions apply.
D.-s. Eom et al.: A Multi-player Arcade Video Game Platformwith a Wireless Tangible User Interface 1821C. Pose computationGiven the distance and acceleration data of a game stick,the game-service PC computes its position and orientation.For computing the position of a game stick, the well-knowntrilateration method is used. Three distance values aresufficient for trilateration, and therefore the largest valueamong the four distance values is discarded. Note that, incomputing the distance using the time-of-arrival technique, alarger distance value usually has a larger error. Let us denotethe remaining three distance values by (d 1 , d 2 , d 3 ).Fig. 3. Flow chart of the sink node’s algorithmThe acceleration data (a x , a y , a z ) for the three axes of agame stick are computed by the acceleration sensor of thegame stick. Immediately after sending the ultrasonic signal tothe beacons, each game stick sends an RF message containingthe acceleration data to the sink node. (The acceleration dataarrive at the sink node, while the distances are computed bythe beacons. This is why, in Fig. 3, ‘reception of accelerationdata’ appears ahead of ‘reception of distance data.’)0 1 2 3 4 5 6 7 8 9START Stick ID d 1 d 2 d 3 d 410 11 12 13 14 15 16 17a x a y a z Button on/off ENDFig. 4. Packet of each game stickFig. 4 shows the data for a game stick, collected by thesink node. The 18-byte packet includes the distance data (d 1 ,d 2 , d 3 , d 4 ) from the four beacons, the acceleration data (a x , a y ,a z ) for three axes, and the on/off status of the button attachedto the game stick, which is useful for creating the game playscenario. (The button is white, and can be found in Fig. 2-(c).)The packet for a game stick is sent to the game-service PC,which computes the position and orientation, i.e. pose, of thegame stick, as is discussed in the next subsection.For every active game stick, the sink node constructs an18-byte packet. When all the active game sticks have beenprocessed, the game-service PC renders the scene in virtualspace by updating the poses of the game sticks.Immediately after sending the packet of the last game stickto the PC, the sink node synchronizes the beacons and gamesticks, and then performs another iteration to render the nextscene.Fig. 5. TrilaterationFig. 5 illustrates the trilateration method. The i-th beaconis denoted by b i , and the distance between b i and the gamestick located at (x, y, z) is d i . Then, we obtain the followingequation for each of the three beacons:222 2( x bix) + ( y − biy) + ( z − biz) = di− (1)The system of equations is solved using Newton-Raphsonmethod. In the current implementation, the initial position (x 0 ,y 0 , z 0 ) of the game stick is set to the center of the game space.The acceleration data (a x , a y , a z ) for three axes arenormalized, to determine the orientation of the game stick.Further, the angular velocities of the game stick areapproximated using the differences of orientation databetween consecutive frames, and are used for generatingspecial effects in the game, as discussed in Section 5.IV. PERFORMANCE ANALYSIS IN POSITION SENSINGRecall that the distance between the game stick and thebeacon is measured using the arrival time of the ultrasonicsignal. For the purpose of performance analysis, a game stickwas fixed at a particular position, and the distance from abeacon was measured 1,500 times. Fig. 6 shows the statisticalresults, where the distance between the game stick and thebeacon is 100cm. The statistical results prove that theproposed game platform can provide precise location data,and can be successfully integrated with location-sensingarcade video games.Authorized licensed use limited to: National Cheng Kung University. Downloaded on April 20, 2009 at 04:49 from IEEE Xplore. Restrictions apply.
1822IEEE Transactions on Consumer Electronics, Vol. 54, No. 4, NOVEMBER 2008Distance(cm)Frequency(1,500 times in total)99.9655 1199.9765 18999.9875 31899.9985 587100.0095 274100.0205 111(a) Measured distancesDistance(cm)Frequency(1,500 times in total)100.8515 23100.8635 233100.8745 225100.8855 311100.8965 321100.9075 258100.9195 116100.9305 12100.9415 1(a) Measured distances(b) Distances in graphMean X = 99.99722 cmVariance S 2 = 0.000160388 cmStandard deviation S = 0.01266 cmStandard error SE = 0.000326994 cm(c) Statistical analysisFig. 6. Test result #1 (at 25°C)Note that the speed of sound varies with the atmosphericconditions, and the most important factor is the temperature.The speed of sound usually increases with increasingtemperature. We used the following equation to calculate thesound speed:S air= 331 .3 + 0. 606 × t (2)where 331.3 (m/s) represents the speed at 0°C and t is thetemperature. Equation (2) is derived from the Taylorexpansion of (3) :tS air= 331 .3 1 +(3)273 .15The test described in Fig. 6 was performed in a room at25 ° C, and t in (2) was set to 25 ° C. We lowered the roomtemperature to 20 ° C, but t was held constant. Fig. 7 shows thestatistical results where the distance between the game stickand beacon was held constant at 100cm. Note that thedistances were overestimated, because the speed of sound wasdecreased.Fig. 8 shows the test result when the room temperaturewas raised to 30 ° C, but t was held constant at 25 ° C. Thedistances were underestimated. The tests show that thetemperature of the game room must be used as input of thegame program, in order to maintain the accuracy of thelocation-sensing system.(b) Distances in graphMean X = 100.88912 cmVariance S 2 = 0.000319619 cmStandard deviation S = 0.01788 cmStandard error SE = 0.000461605 cm(c) Statistical analysisFig. 7. Test result #2 (at 20°C)V. IMPLEMENTATION AND PROTOTYPE GAMEThe wireless sensor nodes in the proposed game platformare fabricated from off-the-shelf components. They are verysmall, and sufficiently light-weight to ensure that they areeasy to place in an indoor environment, such as rooms andcorridors. Little physical infrastructure is required. Thewireless sensor nodes can also be easily embedded in aphysical device, such as the game stick, as shown in Fig. 2-(c).We developed a prototype game, the physical space ofwhich is shown in Fig. 9, where the beacons are located at thefour corners of the screen. It is a two-player game, and themotion of each game stick is tracked, to determine thecorresponding virtual stick’s motion in the game world. Notethat it is rare to have more than two players involved in anarcade game.In the current implementation, the RF transceiver uses2.4GHz RF, a 16-bit timer, and an 8MHz 8-bit MCU. Themotion data of the two game sticks are sampled at 50Hz, i.e.the ID and pose of each game stick are sampled at 25Hz,which is sufficient to control the virtual stick in the gameworld.Authorized licensed use limited to: National Cheng Kung University. Downloaded on April 20, 2009 at 04:49 from IEEE Xplore. Restrictions apply.
D.-s. Eom et al.: A Multi-player Arcade Video Game Platformwith a Wireless Tangible User Interface 1823Distance(cm)Frequency(1,500 times in total)99.1015 799.1125 12399.1235 19899.1345 41599.1455 31999.1565 26999.1675 15599.1785 14(a) Measured distances(a) Special effect for swing(b) Special effect for twistFig. 10. Game screenshots(b) Distances in graphMean X = 99.14077 cmVariance S 2 = 0.000259055 cmStandard deviation S = 0.0161 cmStandard error SE = 0.000415576 cm(c) Statistical analysisFig. 10 shows screenshots of game play. The gameprogram is implemented in C++ and executed on a 1.4 GHzPentium M CPU with a 512 MB RAM. A graphics API isused for rendering, and a game physics engine is used forphysical simulation, i.e. collision detection and resolutionamong the characters and game objects, such as boxes.Fig. 8. Test result #3 (at 30°C)Fig. 11. Acceleration sensor’s coordinatesFig. 9. A prototype gameLet us describe how to detect the orientation changes ofthe game stick, for approximating the angular velocities,which are used for creating special effects in the game. Fig. 11shows the local coordinates associated with the accelerationsensor, where the x- and y-axes are parallel to the sensor chip,and the z-axis is orthogonal to the chip. In general, rotationsabout the x-, y-, and z-axes are called rolling, panning, andtilting, respectively. The prototype game was designed torecognize rolling and tilting. For example, if the player swingsthe game stick downward, the motion is detected as tilting,and then beam lights emanate from the floor, as shown in Fig.10-(a). Similarly, twisting the game stick’s handle is detectedas rolling, which results in boxes and balls falling from theceiling, as shown in Fig. 10-(b).Authorized licensed use limited to: National Cheng Kung University. Downloaded on April 20, 2009 at 04:49 from IEEE Xplore. Restrictions apply.
1824Immediately after determining the game stick’s orientation,the game program computes the orientation change betweenthe current frame and the previous one. If the changes alongthe x- and y-axes are significant, i.e. greater than apredetermined threshold, the player’s motion is considered asswinging or tilting. If the changes along the y- and z-axes aresignificant, the motion is considered as twisting or rolling.VI. CONCLUSIONThis paper presented an arcade video game platform thatcan detect the motions of multiple game players. Theimplementation results proved that various off-the-shelfwireless sensor nodes can be used for VR games that requirethe tracking of the motions of users or physical objects.Ubiquitous or context-aware computing is featured as thecapability to recognize and react to real-world contexts. A keyfactor in these contexts is the user’s location and motion. Inparticular, due to the evolution of mobile computing andwireless networking, location-aware and motion-trackingtechnologies have attracted considerable attention. Manyapplications could benefit from location and motion awarenessof a user or an object. An arcade video game is a goodexample of such applications.The concept of ubiquitous games has recently emerged,and the integration of ubiquitous computing and computergames has been investigated. A recent report on ubiquitousgames can be found in . However, previous studies havelargely focused on toy examples. Unlike previous studies, thispaper presents the first serious attempt to import the coretechnology of ubiquitous computing into the field of arcadevideo games.In the current implementation, the motion data are sampledat 50Hz. The sampling rate is 25Hz per player, for a twoplayergame. This is a reasonable sampling rate, because it israre to have more than two players involved in an arcadegame. The authors believe that the proposed platform isoptimized for arcade video games. For special arcade gameswith three or more players, however, a 50Hz sampling rate isinsufficient for real-time interaction. Some applicationsbesides arcade games may also benefit from tracking themotions of a large number of objects. For this purpose, thereshould be more efforts to improve the communicationperformance among wireless sensor nodes, and the scalabilityof the system. The current system is being extended in thesedirections.REFERENCES I. Getting, “The global positioning system,” IEEE Spectrum, vol. 30, no.12, pp. 36-47, Dec. 1993. R. Want, A. Hopper, V. Falcao, and J. Gibbons, “The Active Badgelocation system,” ACM Transactions on Information Systems, vol. 10, no.1, pp. 91-102, Jan. 1992. P. Bahl and V. Padmanabhan, “RADAR: an in-building RF-based userlocation and tracking system,” Proc. IEEE INFOCOM, Tel-Aviv, Israel,Mar. 2000. A. Harter, A. Hopper, P. Steggles, A. Ward, and P. Webster, “Theanatomy of a context-aware application,” Proc. 5th ACM MOBICOMConf., Seattle, WA, Aug. 1999.IEEE Transactions on Consumer Electronics, Vol. 54, No. 4, NOVEMBER 2008 N. Priyantha, A. Chakraborty, and H. Balakrishnan, “The cricketlocation-support system,” Proc. 6th FACM MOBICOM Conf., Boston,MA, Aug. 2000. S. Teller, J. Chen, H. Balakrishnan, “Pervasive pose-aware applicationsand infrastructure,” IEEE Computer Graphics and Applications, vol. 23,no. 4, pp. 14-18, Jul/Aug, 2003. M. Hazas and A. Hopper, “Broadband ultrasonic location systems forimproved indoor positioning,” IEEE Transactions on Mobile Computing,vol. 5, no. 5, pp. 536-547, May 2006. D. Eom, J. Jang, T. Kim, and J. Han, “A VR game platform built uponwireless sensor network,” Proc. International Symposium on VisualComputing, Lake Tahoe, USA, pp. 1062–1071, Nov. 2006. F. Everest, The Master Handbook of Acoustics, New York, McGraw-Hill, 2001. S. Björk, J. Holopainen, P. Ljungstrand, and K. Akesson, “Designingubiquitous computing games - a report from a workshop exploringubiquitous computing entertainment,” Personal and UbiquitousComputing, vol. 6, no. 6, pp. 443-458, 2002.Doo-seop Eom is an associate professor in theDepartment of Electronics and Computer Engineering atKorea University. He received B.S. and M.S. degrees inElectronics Engineering from Korea University, and aD.E. degree in Information and Computer Sciences fromOsaka University, Japan. He was awarded a JapaneseGovernment Scholarship during 1996-1999, the period ofhis academic career at Osaka University. Prior to joining Korea University, Dr.Eom worked for the Communication Systems Division, Electronics andTelecommunications Research Institute (ETRI), which is the largest researchinstitute in Korea. His research interests include wireless sensor networks,RFID, and Internet QoS.Taeyoung Kim is a Ph.D. student in the Department ofElectronics and Computer Engineering at KoreaUniversity. He received a B.S. degree in ElectronicsEngineering from Konkuk University, and an M.S.degree in Electronics and Computer Engineering fromKorea University. His research interests include wirelesssensor networks and RFID.HyunHo Jee is an M.S. student in the Department ofComputer and Radio Communications Engineering atKorea University. He received a B.S. degree in ComputerEngineering from Korea University. His research interestsinclude 3D graphics and character animation.Hyoil Lee is an M.S. student in the Department ofComputer and Radio Communications Engineering atKorea University. He received a B.S. degree in ComputerScience from Sejong University. His research interestsinclude real-time rendering and user interfaces in 3Dgames.JungHyun Han is an associate professor in theDepartment of Computer and Radio CommunicationsEngineering at Korea University, where he directs theInteractive 3D Media Laboratory and the Game ResearchCenter, supported by the Korea Ministry of Culture,Sports and Tourism. Prior to joining Korea University, heworked at the School of Information andCommunications Engineering of Sungkyunkwan University, in Korea, and theManufacturing Systems Integration Division of the US Department ofCommerce National Institute of Standards and Technology (NIST). Dr. Hanreceived a B.S. degree in Computer Engineering at Seoul National University,an M.S. degree in Computer Science at the University of Cincinnati, and aPh.D. degree in Computer Science at USC. His research interests include HCIand game design.Authorized licensed use limited to: National Cheng Kung University. Downloaded on April 20, 2009 at 04:49 from IEEE Xplore. Restrictions apply.