12.07.2015 Views

Usability of Legacy p2p Multicast in Multi-hop Ad hoc Networks: an ...

Usability of Legacy p2p Multicast in Multi-hop Ad hoc Networks: an ...

Usability of Legacy p2p Multicast in Multi-hop Ad hoc Networks: an ...

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

1<strong>Usability</strong> <strong>of</strong> <strong>Legacy</strong> <strong>p2p</strong> <strong><strong>Multi</strong>cast</strong><strong>in</strong> <strong>Multi</strong>-<strong>hop</strong> <strong>Ad</strong> <strong>hoc</strong> <strong>Networks</strong>: <strong>an</strong> Experimental StudyAndrea Passarella <strong>an</strong>d Fr<strong>an</strong>ca DelmastroPervasive Comput<strong>in</strong>g & Network<strong>in</strong>g Laboratory (PerLab)IIT-CNR, Via G. Moruzzi 1, 56124 Pisa, Italy{firstname.lastname}@iit.cnr.itAbstractThere has recently been <strong>an</strong> <strong>in</strong>creas<strong>in</strong>g <strong>in</strong>terest <strong>in</strong> convergence <strong>of</strong> <strong>p2p</strong> <strong>an</strong>d ad <strong>hoc</strong> network research.Actually, <strong>p2p</strong> systems <strong>an</strong>d multi-<strong>hop</strong> ad <strong>hoc</strong> networks share similar features, such a self org<strong>an</strong>isation, decentralisation,self heal<strong>in</strong>g, etc. It is thus <strong>in</strong>terest<strong>in</strong>g to underst<strong>an</strong>d if <strong>p2p</strong> systems designed for the wired Internetbe suitable also for ad <strong>hoc</strong> networks <strong>an</strong>d, if they are not, <strong>in</strong> which direction they should be improved. Inthis paper we report our experience <strong>in</strong> runn<strong>in</strong>g <strong>p2p</strong> applications <strong>in</strong> real multi-<strong>hop</strong> ad <strong>hoc</strong> network testbeds.Specifically, we used Group-Communication Applications, that require <strong>p2p</strong> systems made up <strong>of</strong> <strong>an</strong> overlaynetwork <strong>an</strong>d a <strong>p2p</strong> multicast protocol. In this paper we present experimental results specifically related tothe perform<strong>an</strong>ce <strong>of</strong> a well-known <strong>p2p</strong> shared-tree multicast protocol (Scribe). Our results show that such asolution is far from be<strong>in</strong>g efficient on ad <strong>hoc</strong> networks. We highlight that the structured multicast approachis one <strong>of</strong> the ma<strong>in</strong> causes <strong>of</strong> <strong>in</strong>efficiency, <strong>an</strong>d suggest that stateless solutions could be preferable.I. INTRODUCTIONThe <strong>in</strong>tegration <strong>of</strong> <strong>p2p</strong> systems <strong>in</strong>to multi-<strong>hop</strong> ad <strong>hoc</strong> networks is a very <strong>in</strong>terest<strong>in</strong>g <strong>an</strong>d challeng<strong>in</strong>g topicthat is attract<strong>in</strong>g <strong>in</strong>creas<strong>in</strong>g attention [1], [2]. Actually, <strong>p2p</strong> systems <strong>an</strong>d ad <strong>hoc</strong> networks share a number<strong>of</strong> similar features. P2p systems provide a decentralised, self-org<strong>an</strong>is<strong>in</strong>g <strong>an</strong>d self-heal<strong>in</strong>g environment,along with features like subject-based rout<strong>in</strong>g, distributed data storage/retrieval, <strong>an</strong>d load bal<strong>an</strong>c<strong>in</strong>g. Suchfeatures, orig<strong>in</strong>ally devised for <strong>p2p</strong> overlay networks <strong>in</strong> the legacy Internet, are also well suited for adecentralised <strong>an</strong>d dynamic environment like a multi-<strong>hop</strong> ad <strong>hoc</strong> network. It is thus extremely import<strong>an</strong>t tounderst<strong>an</strong>d how <strong>p2p</strong> systems designed for the Internet could be <strong>in</strong>tegrated <strong>in</strong>to ad <strong>hoc</strong> networks.Our approach to this study is tw<strong>of</strong>old. Firstly, we run experiments on a real testbed, so as to take <strong>in</strong>toaccount all the wireless l<strong>in</strong>ks complexities that real systems have to deal with. Secondly, we deploy <strong>in</strong> ourtestbed a realistic prototype that <strong>in</strong>cludes all layers <strong>of</strong> the stack, from the physical up to the application one.Specifically, we have developed a distributed whiteboard (WB) at the application layer. WB allows users toshare content with<strong>in</strong> members <strong>of</strong> a community. It is <strong>an</strong> <strong>in</strong>st<strong>an</strong>ce <strong>of</strong> Group-Communication (GC) Applicationsthat are <strong>an</strong> <strong>in</strong>terest<strong>in</strong>g example <strong>of</strong> <strong>p2p</strong> applications oriented to ad <strong>hoc</strong> networks (see Section III-A).


2We believe that both <strong>of</strong> these features are import<strong>an</strong>t to achieve clear underst<strong>an</strong>d<strong>in</strong>g <strong>of</strong> the <strong>p2p</strong>-systembehavior <strong>in</strong> ad <strong>hoc</strong> networks. Actually, most <strong>of</strong> the research on multi-<strong>hop</strong> ad <strong>hoc</strong> networks has privilegeda simulation <strong>an</strong>d/or theoretical-driven approach. Simulation <strong>an</strong>d theoretical <strong>an</strong>alysis are very helpful toolss<strong>in</strong>ce they allow to m<strong>an</strong>age large scales, control parameters’ variation, etc. However, propagation overwireless medium is so complicated to precisely model, that experiment<strong>in</strong>g on real test-beds is a must. Justrely<strong>in</strong>g on simulation <strong>an</strong>d theoretical models might lead to conclusions that do not match real measures [3],[4]. Furthermore, little effort has been devoted to <strong>an</strong>alyse the behavior <strong>of</strong> multi-<strong>hop</strong> ad <strong>hoc</strong> networks withrespect to realistic applications. Much effort has been spent on evaluat<strong>in</strong>g the behavior <strong>of</strong> s<strong>in</strong>gle protocols(mostly at the rout<strong>in</strong>g layer) us<strong>in</strong>g CBR- or FTP-like applications. Just a few works (e.g., [5]) try to envisionapplication scenarios <strong>in</strong> which ad <strong>hoc</strong> networks could be <strong>an</strong> enabl<strong>in</strong>g factor for applications, rather th<strong>an</strong> ahostile network<strong>in</strong>g environment to cope with.The Whiteboard application exploits services provided by a <strong>p2p</strong> multicast system. Therefore, we haveused a <strong>p2p</strong> system consist<strong>in</strong>g <strong>of</strong> <strong>an</strong> overlay network <strong>an</strong>d a <strong>p2p</strong> multicast protocol. We have used Pastry toimplement the overlay network, <strong>an</strong>d Scribe to build multicast trees on top <strong>of</strong> the overlay. Pastry <strong>an</strong>d Scribeare among the most efficient solutions developed for the legacy Internet [6]. F<strong>in</strong>ally, we have tested thesystem both on proactive <strong>an</strong>d on reactive rout<strong>in</strong>g protocols (i.e., OLSR <strong>an</strong>d AODV, respectively).While <strong>in</strong> previous papers we have focused on the perform<strong>an</strong>ce <strong>of</strong> Pastry on ad <strong>hoc</strong> networks [7], [8],[9], <strong>in</strong> this work we ma<strong>in</strong>ly concentrate on the perform<strong>an</strong>ce <strong>of</strong> Scribe. Experimental results are discussed<strong>in</strong> Sections IV <strong>an</strong>d V. Specifically, we focus on the QoS provided to WB users <strong>in</strong> terms <strong>of</strong> average delay<strong>an</strong>d packet loss. Purposely, we report only results gathered from static ad <strong>hoc</strong> networks. Tak<strong>in</strong>g also <strong>in</strong>toaccount users mobility would have added further dimensions to the parameters space, mak<strong>in</strong>g results quitedifficult to underst<strong>an</strong>d. Our experiments highlight that proactive rout<strong>in</strong>g protocols are much more efficient<strong>in</strong> our scenario (Section IV). Actually, AODV is practically not able to support <strong>an</strong>y configuration <strong>of</strong> ourtestbed. However, even when OLSR is used, Scribe shows severe limitations over ad <strong>hoc</strong> networks. Eventhough ref<strong>in</strong>ed s<strong>of</strong>tware releases might improve the user QoS (Section V), there are <strong>in</strong>tr<strong>in</strong>sic Scribe featuresthat h<strong>in</strong>der from us<strong>in</strong>g it <strong>in</strong> this network<strong>in</strong>g environment.We argue that they ma<strong>in</strong>ly stem from the design choice <strong>of</strong> concentrat<strong>in</strong>g all the application-level trafficon one s<strong>in</strong>gle node (the Scribe Root Node) before deliver<strong>in</strong>g it to the f<strong>in</strong>al dest<strong>in</strong>ations. In fact, from theexperimental results, we c<strong>an</strong> conclude that the structured multicast approach used by Scribe is one <strong>of</strong> thema<strong>in</strong> reasons <strong>of</strong> its <strong>in</strong>efficiency. Specifically, structured multicast tends to concentrate the application loadon few nodes <strong>an</strong>d l<strong>in</strong>ks that may become easily overloaded. As a topic <strong>of</strong> future research, we highlight thatstructure-less (or stateless) multicast approaches c<strong>an</strong> avoid such concentration, represent<strong>in</strong>g a simple <strong>an</strong>d<strong>in</strong>terest<strong>in</strong>g possibility to implement efficient <strong>p2p</strong> multicast systems <strong>in</strong> multi-<strong>hop</strong> ad <strong>hoc</strong> networks.


3II. RELATED WORKExperiment-based research on wireless networks, <strong>an</strong>d specifically on multi-<strong>hop</strong> ad <strong>hoc</strong> networks, isga<strong>in</strong><strong>in</strong>g momentum <strong>in</strong> the last few years [10], [11]. Hav<strong>in</strong>g controllable, reproducible <strong>an</strong>d reasonable-sizewireless testbeds is not trivial. Thus, several research efforts are focus<strong>in</strong>g on how to design <strong>an</strong>d implementtestbeds that the whole community c<strong>an</strong> exploit [12], [13], [14], [15]. One <strong>of</strong> the ma<strong>in</strong> issues <strong>of</strong> simulation<strong>an</strong>d theoretical <strong>an</strong>alysis is the accuracy <strong>of</strong> wireless ch<strong>an</strong>nel models. Therefore, several papers <strong>an</strong>alysewireless ch<strong>an</strong>nel features aim<strong>in</strong>g at provid<strong>in</strong>g realistic models (see, for example, [3] <strong>an</strong>d references here<strong>in</strong>).Other research efforts target the experimental evaluation <strong>of</strong> rout<strong>in</strong>g ([9], [16]) or tr<strong>an</strong>sport ([10], [11])protocols on multi-<strong>hop</strong> ad <strong>hoc</strong> networks. Signific<strong>an</strong>t effort has also been devoted to build experimentaltestbeds for mesh networks ([17] <strong>an</strong>d references here<strong>in</strong>).The root <strong>of</strong> this paper is our previous work on experimental <strong>an</strong>alysis <strong>of</strong> rout<strong>in</strong>g <strong>an</strong>d middleware platformsfor multi-<strong>hop</strong> ad <strong>hoc</strong> networks. The work <strong>in</strong> [7] <strong>an</strong>d [8] focuses on issues related to structured overlaynetworks runn<strong>in</strong>g on proactive <strong>an</strong>d reactive rout<strong>in</strong>g protocols. Specifically, these papers <strong>an</strong>alyse Pastryperform<strong>an</strong>ce runn<strong>in</strong>g on top <strong>of</strong> OLSR [18] <strong>an</strong>d AODV [19]. Work <strong>in</strong> [7], [8], [9] showed that OLSRperforms better th<strong>an</strong> AODV <strong>in</strong> terms <strong>of</strong> packet loss <strong>an</strong>d delays, when runn<strong>in</strong>g either a light applicationsuch as the p<strong>in</strong>g utility, or a structured <strong>p2p</strong> system. Furthermore, <strong>in</strong> [20], [21] we started <strong>an</strong>alys<strong>in</strong>gthe perform<strong>an</strong>ce <strong>of</strong> a full <strong>p2p</strong> stack <strong>in</strong>clud<strong>in</strong>g a complete <strong>p2p</strong> multicast system based on Scribe <strong>an</strong>d Pastry,<strong>an</strong>d a realistic GC application. In this paper we provide a systematic <strong>an</strong>d comprehensive view <strong>of</strong> our majorf<strong>in</strong>d<strong>in</strong>gs <strong>in</strong> us<strong>in</strong>g legacy <strong>p2p</strong> systems to support GC applications <strong>in</strong> multi-<strong>hop</strong> ad <strong>hoc</strong> networks.From the study reported <strong>in</strong> [7], [8], [9], the design <strong>of</strong> <strong>an</strong> optimised <strong>p2p</strong> overlay network for multi-<strong>hop</strong>ad <strong>hoc</strong> networks has arisen. We called it CrossROAD [22], s<strong>in</strong>ce it exploits a cross-layer <strong>in</strong>teraction with aproactive rout<strong>in</strong>g protocol. The comparison between Pastry <strong>an</strong>d CrossROAD shows that such <strong>in</strong>teractionsc<strong>an</strong> be <strong>an</strong> enabler to implement structured overlay networks over ad <strong>hoc</strong> networks <strong>in</strong> a very efficient way.In this paper we do not focus exclusively on the overlay network, but take <strong>in</strong>to account also the multicast<strong>an</strong>d application layers. To have good h<strong>in</strong>ts on how to design <strong>an</strong> efficient <strong>p2p</strong> multicast system for multi<strong>hop</strong>ad <strong>hoc</strong> networks, we do not consider at this stage <strong>an</strong>y possible cross-layer optimisation. Instead, we<strong>an</strong>alyse how a legacy protocol (i.e., Scribe) works <strong>in</strong> the environment it has been designed for (i.e., Pastry)when used to support GC applications. The results we provide <strong>in</strong> this paper tell that <strong>in</strong> our experimentalenvironment the limits <strong>of</strong> such a legacy <strong>p2p</strong> system c<strong>an</strong> be ma<strong>in</strong>ly accounted to design choices <strong>of</strong> Scribe(see Sections VI <strong>an</strong>d VII). Therefore, simply replac<strong>in</strong>g Pastry with CrossROAD may help because the DHTwill become more efficient, but will not solve the problem.<strong><strong>Multi</strong>cast</strong> support implemented at the <strong>p2p</strong> layer is just one <strong>of</strong> the available options proposed <strong>in</strong> theliterature. Usually, multicast protocols are classified as operat<strong>in</strong>g at the network (L3) layer, or at theapplication layer, where application denotes all possible layers above the tr<strong>an</strong>sport.In the legacy wired networks, application-level multicast has been <strong>in</strong>troduced to address the practi-


4cal problems <strong>of</strong> implement<strong>in</strong>g L3 multicast <strong>in</strong> the Internet core. L3 multicast requires modification atthe routers, s<strong>in</strong>ce they have to ma<strong>in</strong>ta<strong>in</strong> the multicast groups state, which contrasts with the orig<strong>in</strong>alstatelessness <strong>of</strong> the IP protocol. Instead, application-level multicast runs at edge nodes, <strong>an</strong>d just requiresst<strong>an</strong>dard unicast support from the core 1 . Several application-level multicast protocols have been proposedfor the wired Internet (e.g., [23] [24] [25] [26] [27] [28]). The most recent proposals among them([24] [25] [26] [28]) run the multicast protocol on top <strong>of</strong> <strong>an</strong> overlay network based on a DHT. Thisapproach is very <strong>in</strong>terest<strong>in</strong>g for a number <strong>of</strong> reasons. Firstly, the task <strong>of</strong> def<strong>in</strong><strong>in</strong>g a network structure thatjust encompasses the edge nodes is assigned to the DHT, <strong>an</strong>d has not to be implemented by the multicastprotocol itself (as, for example, <strong>in</strong> [23]). Secondly, the multicast protocol leverages the self-org<strong>an</strong>is<strong>in</strong>g <strong>an</strong>dself-recovery features <strong>of</strong> the DHT. F<strong>in</strong>ally, the same DHT c<strong>an</strong> be shared by several higher-level servicesrunn<strong>in</strong>g besides the multicast protocol. To the best <strong>of</strong> our knowledge, the feasibility <strong>of</strong> such systems onmulti-<strong>hop</strong> ad <strong>hoc</strong> networks has not been <strong>in</strong>vestigated yet. In this work we choose Scribe ([24]), because itis one <strong>of</strong> the most recent <strong>an</strong>d popular <strong>p2p</strong> multicast protocols, <strong>an</strong>d has shown to outperform other similarapproaches shar<strong>in</strong>g the same concepts [6].Other papers propose multicast solutions explicitly designed for ad <strong>hoc</strong> networks (e.g., [29] [30] [31] [32],<strong>an</strong>d see the survey <strong>in</strong> [33]). Also <strong>in</strong> this case, it is possible to identify L3 approaches (e.g., [31] [32]) <strong>an</strong>dapplication-layer approaches (e.g., [29] [30]). The argument used to justify application-level multicast <strong>in</strong>wired networks does not hold <strong>in</strong> ad <strong>hoc</strong> networks <strong>an</strong>ymore, as <strong>in</strong> this case there is no dist<strong>in</strong>ction betweencore <strong>an</strong>d edge hosts. The ma<strong>in</strong> reason people highlight to implement multicast at the application level also <strong>in</strong>ad <strong>hoc</strong> networks is the fact that <strong>in</strong> this way the burden <strong>of</strong> m<strong>an</strong>ag<strong>in</strong>g the multicast structure falls exclusivelyon nodes that are actually <strong>in</strong>terested <strong>in</strong> be<strong>in</strong>g part <strong>of</strong> the multicast group. However, it should be po<strong>in</strong>tedout that application-level multicast potentially generates path stretch because just a subset <strong>of</strong> nodes c<strong>an</strong> beused to deliver the data. Moreover, nodes that do not participate <strong>in</strong> multicast groups have to forward dat<strong>an</strong>evertheless. Therefore, it is not yet clear whether multicast for ad <strong>hoc</strong> networks should be implemented atthe rout<strong>in</strong>g or at the application level. To the best <strong>of</strong> our knowledge, application level multicast approachesdesigned for ad <strong>hoc</strong> networks do not leverage underly<strong>in</strong>g <strong>p2p</strong> systems as some solutions for wired networksdo. We do believe that exploit<strong>in</strong>g DHTs to build multicast systems c<strong>an</strong> be a valid option <strong>in</strong> ad <strong>hoc</strong> networkstoo, due to the adv<strong>an</strong>tages highlighted before. This is why we have chosen to evaluate Scribe on ad <strong>hoc</strong>networks <strong>in</strong> this paper.F<strong>in</strong>ally, it is worth mention<strong>in</strong>g the orig<strong>in</strong>al br<strong>an</strong>ch represented by gossip<strong>in</strong>g protocols applied to multicast(e.g. [34]). The ma<strong>in</strong> idea <strong>of</strong> this approach is that senders <strong>of</strong> a multicast group select a r<strong>an</strong>dom subset <strong>of</strong>nodes <strong>in</strong> the group to send data to. The same process is repeated at receiv<strong>in</strong>g nodes for a given number <strong>of</strong>turns, this number <strong>an</strong>d the size <strong>of</strong> the r<strong>an</strong>dom subsets be<strong>in</strong>g protocol parameters. It has been shown thatsuch protocols are actually able to deliver messages with high probability to all <strong>in</strong>tended receivers [34]. A1 Runn<strong>in</strong>g exclusively on edge hosts, or end systems, application-level multicast is sometimes referred to as end-system multicast


5side effect <strong>of</strong> gossip<strong>in</strong>g is that the message replication rate is quite difficult to control. Evaluat<strong>in</strong>g such <strong>an</strong>approach <strong>in</strong> comparison with <strong>p2p</strong> multicast<strong>in</strong>g is <strong>an</strong> <strong>in</strong>terest<strong>in</strong>g topic which is however out <strong>of</strong> the scope <strong>of</strong>this work.Similarly, <strong>in</strong> this work we do not specifically consider reliability issues that may arise also <strong>in</strong> <strong>p2p</strong>multicast<strong>in</strong>g scenarios (we elaborate on this po<strong>in</strong>t <strong>in</strong> Section III-C). As a future work, it will be <strong>in</strong>terest<strong>in</strong>gto <strong>in</strong>vestigate how to efficiently <strong>in</strong>tegrate multicast reliability techniques available <strong>in</strong> literature.III. EXPERIMENTAL SCENARIO AND SETUPA. Application <strong>an</strong>d Protocol StackOne <strong>of</strong> our targets is to envision realistic applications oriented to multi-<strong>hop</strong> ad <strong>hoc</strong> networks <strong>an</strong>dunderst<strong>an</strong>d how they could be developed <strong>in</strong> practice. From this st<strong>an</strong>dpo<strong>in</strong>t, GC applications are quite<strong>in</strong>terest<strong>in</strong>g. They fit well the overall features <strong>of</strong> multi-<strong>hop</strong> ad <strong>hoc</strong> networks s<strong>in</strong>ce they are distributed, selforg<strong>an</strong>is<strong>in</strong>g,<strong>an</strong>d decentralised <strong>in</strong> nature. As a simple - yet signific<strong>an</strong>t - example, we developed a Whiteboardapplication (WB), which implements a distributed whiteboard among the network users. WB usage isvery <strong>in</strong>tuitive. Users run a WB <strong>in</strong>st<strong>an</strong>ce on their own devices, select a topic they w<strong>an</strong>t to jo<strong>in</strong>, <strong>an</strong>d startdraw<strong>in</strong>g on the c<strong>an</strong>vas. Draw<strong>in</strong>gs are distributed to all the devices subscribed to that topic, <strong>an</strong>d renderedon each c<strong>an</strong>vas. WB is just <strong>an</strong> example <strong>of</strong> a broader r<strong>an</strong>ge <strong>of</strong> applications, <strong>in</strong>clud<strong>in</strong>g distributed messag<strong>in</strong>g,distributed gam<strong>in</strong>g, etc. We believe that this k<strong>in</strong>d <strong>of</strong> applications c<strong>an</strong> be valuable to users, <strong>an</strong>d they c<strong>an</strong>thus contribute to br<strong>in</strong>g multi-<strong>hop</strong> ad <strong>hoc</strong> networks technologies <strong>in</strong>to everyday life.WB has been developed on top <strong>of</strong> the network protocol stack shown <strong>in</strong> Figure 1(a). Specifically, WBruns on top <strong>of</strong> a <strong>p2p</strong> middleware layer made up <strong>of</strong> a structured overlay network (Pastry [35]), <strong>an</strong>d <strong>an</strong>application-level multicast protocol (Scribe [24]). WB maps each <strong>in</strong>terest group (i.e., each topic) to amulticast tree, <strong>an</strong>d exploits the multicast protocol services to deliver <strong>in</strong>formation to group members. Pastryuses both TCP <strong>an</strong>d UDP at the tr<strong>an</strong>sport layer, <strong>an</strong>d thus we have <strong>in</strong>cluded both these protocols <strong>in</strong> ourarchitecture. Note that the traffic generated by WB <strong>an</strong>d Scribe is sent over TCP connections. In l<strong>in</strong>e withour previous experiments, at the rout<strong>in</strong>g layer we used OLSR [18] as a proactive rout<strong>in</strong>g protocol, <strong>an</strong>dAODV [19] as a reactive one. The referred implementations <strong>of</strong> OLSR <strong>an</strong>d AODV provide a st<strong>an</strong>dard L3forward<strong>in</strong>g platform. Therefore, we could run v<strong>an</strong>illa TCP <strong>an</strong>d UDP shipped with the L<strong>in</strong>ux kernel on top<strong>of</strong> them.Before proceed<strong>in</strong>g, it is useful to give some more details about Pastry <strong>an</strong>d Scribe features. This helps tounderst<strong>an</strong>d the experimental results presented afterwards.Pastry implements <strong>an</strong> overlay network based on a Distributed Hash Table (DHT), on which nodes <strong>an</strong>ddata are logically mapped. Specifically, each node gets a logical address with<strong>in</strong> a circular address space asthe hashed value <strong>of</strong> its IP address, while a key is associated to each data to be stored/retrieved <strong>in</strong>/fromthe overlay. The onus <strong>of</strong> def<strong>in</strong><strong>in</strong>g a key for each message lies with the layer runn<strong>in</strong>g on top <strong>of</strong> Pastry(Scribe <strong>in</strong> this case, or <strong>an</strong>y other distributed application <strong>in</strong> general). Pastry guar<strong>an</strong>tees that the message


6is delivered to the node <strong>in</strong> the overlay with the closest logical address to the hashed key. Thus, the ma<strong>in</strong>feature <strong>of</strong> Pastry is implement<strong>in</strong>g subject-based rout<strong>in</strong>g, allow<strong>in</strong>g upper layers to be completely unaware<strong>of</strong> the eventual-dest<strong>in</strong>ation address. In more detail, a message generated at a node is sent to the nodewith the closest address (<strong>in</strong> the circular space) to the hashed key, to the best <strong>of</strong> the local-node knowledge.For scalability reasons, <strong>in</strong> general, each node knows only a subset <strong>of</strong> the other particip<strong>an</strong>ts <strong>in</strong> the sameoverlay. Thus, the message may reach the f<strong>in</strong>al dest<strong>in</strong>ation follow<strong>in</strong>g a multi-<strong>hop</strong> path on the overlay,possibly result<strong>in</strong>g <strong>in</strong> physical-path stretch. Actually, Pastry trades path stretch for scalability <strong>an</strong>d ability <strong>of</strong>implement<strong>in</strong>g subject-based rout<strong>in</strong>g.In addition, <strong>in</strong> order to jo<strong>in</strong> the overlay, each node has to contact <strong>an</strong>other node already <strong>in</strong> the overlay, <strong>an</strong>dcollect <strong>in</strong>formation needed to <strong>in</strong>itialise its <strong>in</strong>ternal data structures from the other nodes <strong>in</strong> the overlay (see[35] for details). In case this procedure fails, the jo<strong>in</strong><strong>in</strong>g node creates its own <strong>in</strong>dependent overlay, <strong>an</strong>d hasno possibility to rejo<strong>in</strong> the orig<strong>in</strong>al network without execut<strong>in</strong>g the bootstrap procedure <strong>an</strong>ew. Failures <strong>of</strong>the bootstrap procedure becomes quite likely <strong>in</strong> ad <strong>hoc</strong> networks, due to the <strong>in</strong>tr<strong>in</strong>sic <strong>in</strong>stability <strong>of</strong> wirelessl<strong>in</strong>ks. As shown <strong>in</strong> the follow<strong>in</strong>g, we have actually experienced such failures several times.AR2R1ECBDF(a) protocol stack(b) network topologyFig. 1.Protocol stack <strong>an</strong>d network topology <strong>of</strong> our testbed.Scribe has been developed on top <strong>of</strong> Pastry because the presence <strong>of</strong> a DHT facilitates the creation <strong>an</strong>dma<strong>in</strong>ten<strong>an</strong>ce <strong>of</strong> the shared trees among groups <strong>of</strong> nodes. Each tree is identified by a topic. Scribe def<strong>in</strong>es aRoot Node, as the node <strong>in</strong> the overlay whose address is the closest one to the hashed topic. In Figures 2a)<strong>an</strong>d b) the root is node C. The tree is built as the union <strong>of</strong> the reverse paths from the members to the root.Each node will<strong>in</strong>g to jo<strong>in</strong> the tree sends a subscribe message specify<strong>in</strong>g the topic as the key. If the localnode does not directly know the root <strong>of</strong> the topic, the message is forwarded us<strong>in</strong>g a multi-<strong>hop</strong> route atthe overlay level. An <strong>in</strong>termediate node receiv<strong>in</strong>g such message either subscribes itself to the same tree bysend<strong>in</strong>g its own subscribe message towards the root (e.g., node B after step 1 <strong>in</strong> Figure 2a), or discards themessage if it is already a member <strong>of</strong> the tree (e.g., node B <strong>in</strong> step 3 <strong>in</strong> Figure 2a). In both cases, it enrollsthe node from which it received the message as a child. Messages to be delivered over the tree are firstsent towards the root <strong>of</strong> the topic (step 1 <strong>in</strong> Figure 2b), <strong>an</strong>d subsequently delivered by each parent to its


8In all the experiments nodes A to E r<strong>an</strong> the WB application while the others just worked as routers.Specifically, the “WB nodes” tried to jo<strong>in</strong> a s<strong>in</strong>gle overlay <strong>an</strong>d consequently a s<strong>in</strong>gle tree related to a specifictopic <strong>of</strong> <strong>in</strong>terest. In our configuration every node always assumed the same logical identifier obta<strong>in</strong>ed byhash<strong>in</strong>g its IP address, <strong>an</strong>d the topic used by the WB users was always the same.Under the hypothesis that Pastry generated a s<strong>in</strong>gle overlay encompass<strong>in</strong>g all WB nodes, the Root <strong>of</strong> theScribe tree (i.e., the node whose id is closest to the WB topic id) was the same through all the experiments,<strong>an</strong>d was node C <strong>in</strong> Figure 1b). Users <strong>in</strong>teractions with the WB were simulated by a s<strong>of</strong>tware agent thatalternated between active <strong>an</strong>d idle phases. Specifically, <strong>in</strong> each active phase the agent generated trafficon the network correspond<strong>in</strong>g to strokes drawn on the WB. Both the number <strong>of</strong> strokes drawn dur<strong>in</strong>g <strong>an</strong>active phase, <strong>an</strong>d the duration <strong>of</strong> <strong>an</strong> idle phase were exponentially distributed. Such a traffic pr<strong>of</strong>ile isbursty, represent<strong>in</strong>g the typical behavior <strong>of</strong> a user that sends content to be shared with the group, <strong>an</strong>d then“idles”, look<strong>in</strong>g at data generated by other users.We r<strong>an</strong> experiments by vary<strong>in</strong>g both the average idle-phase duration, <strong>an</strong>d the average number <strong>of</strong> strokesper active phase. Each trial was composed by 100 active/idle cycles, <strong>an</strong>d we took care that each noderunn<strong>in</strong>g WB generated at least 100 messages 2 . To make trials start at the same time at different nodes, wesynchronised the nodes before each trial, <strong>an</strong>d scheduled the trial to start at the same time on each node.Then, the Pastry bootstrap sequence occurred as follows: node C started first, <strong>an</strong>d generated the overlay.Nodes E <strong>an</strong>d D started 5 seconds after C, <strong>an</strong>d bootstrapped from C. Node B started 5 seconds after D <strong>an</strong>dbootstrapped from D. Node A started 5 seconds after B <strong>an</strong>d bootstrapped from B. F<strong>in</strong>ally, node F started5 seconds after E <strong>an</strong>d bootstrapped from E. After all nodes jo<strong>in</strong>ed the overlay the Scribe tree was created<strong>an</strong>d, f<strong>in</strong>ally, WB <strong>in</strong>st<strong>an</strong>ces started send<strong>in</strong>g application messages.The time <strong>in</strong>tervals used for the bootstrap sequence were def<strong>in</strong>ed to reduce the probability <strong>of</strong> failuresdur<strong>in</strong>g the bootstrap procedure.Furthermore, the bootstrap sequence was def<strong>in</strong>ed to make each nodebootstrap from a physical neighbour. This is a quite realistic assumption, <strong>an</strong>d also <strong>in</strong>creases the probability<strong>of</strong> nodes to correctly complete the bootstrap procedure.However, <strong>of</strong>ten this was not sufficient. In fact,the <strong>in</strong>stability <strong>of</strong> the wireless l<strong>in</strong>ks caused high packet loss result<strong>in</strong>g <strong>in</strong> TCP connection failures <strong>an</strong>d theconsequent generation <strong>of</strong> <strong>an</strong> isolated overlay.In the perform<strong>an</strong>ce <strong>an</strong>alysis, each trial is identified by the rout<strong>in</strong>g protocol used <strong>an</strong>d by <strong>an</strong> applicationload<strong>in</strong>dex, measured as the number <strong>of</strong> Packets Per Second (pps) generated by each user. This <strong>in</strong>dex isdef<strong>in</strong>ed as the ratio between the average number <strong>of</strong> strokes generated <strong>in</strong> the active/idle cycle, <strong>an</strong>d theaverage duration <strong>of</strong> the cycle. We have found that this simple <strong>in</strong>dex is sufficient to correctly identify usagecases <strong>of</strong> a GC application <strong>in</strong> our scenario.The ma<strong>in</strong> perform<strong>an</strong>ce <strong>in</strong>dices presented <strong>in</strong> the follow<strong>in</strong>g are the packet loss <strong>an</strong>d the average delayexperienced by each node dur<strong>in</strong>g the experiment. The packet loss is def<strong>in</strong>ed as 1− ∑ RiN , where R i isj=1 Sj2 A dist<strong>in</strong>ct message was sent for each stroke. The size <strong>of</strong> each message was 1448 bytes.


9Packet Loss (%)100806040xPastry 1.3 Packet LossAODV(0.1pps): r<strong>in</strong>gs (A),(BCDEF)AODV(0.2pps): r<strong>in</strong>gs (A),(BCDEF)OLSR(0.2pps): r<strong>in</strong>gs (ABCDEF)OLSR(0.5pps): r<strong>in</strong>gs (A),(CDEF)OLSR(0.8pps): r<strong>in</strong>gs (ABCDEF)Avgerage Delay (s)450400350300250200150Pastry 1.3 Average DelayAODV(0.1pps): r<strong>in</strong>gs (A),(BCDEF)AODV(0.2pps): r<strong>in</strong>gs (A),(BCDEF)OLSR(0.2pps): r<strong>in</strong>gs (ABCDEF)OLSR(0.5pps): r<strong>in</strong>gs (A),(CDEF)OLSR(0.8pps): r<strong>in</strong>gs (ABCDEF)x20100500ABC(R)NodeDEF0ABC(R)NodeDEF(a) Packet Loss(b) Average DelayFig. 3.Pastry perform<strong>an</strong>ce on AODV <strong>an</strong>d OLSRthe number <strong>of</strong> messages received by node i, <strong>an</strong>d S j is the number <strong>of</strong> messages tr<strong>an</strong>smitted by the j-thnode. S<strong>in</strong>ce Pastry uses TCP at the tr<strong>an</strong>sport layer, one could expect not to see <strong>an</strong>y packet loss. Actually,Pastry uses <strong>an</strong> <strong>in</strong>ternal queue to store messages go<strong>in</strong>g to be sent. Packet loss actually occurs when thisqueue fills up, <strong>an</strong>d is thus a side effect <strong>of</strong> network congestion 3 . The average delay experienced by node iis def<strong>in</strong>ed as ∑ R ij=1 d ij/R i where d ij is the delay experienced by node i <strong>in</strong> receiv<strong>in</strong>g packet j.We have also def<strong>in</strong>ed a usability threshold for the application, <strong>in</strong>dicat<strong>in</strong>g reference values for both delays<strong>an</strong>d packet loss. Beyond these thresholds the application perform<strong>an</strong>ce is deemed not compli<strong>an</strong>t with usersexpectations. To have reasonable values, <strong>in</strong> our case we assume 10s delay <strong>an</strong>d 15% packet loss as thresholds.Of course they closely depend on the specific application requirements. We replicated each configurationseveral times, obta<strong>in</strong><strong>in</strong>g quite variable results. They are ma<strong>in</strong>ly due to the variability <strong>of</strong> the wireless medium.In this paper we show the best results measured <strong>in</strong> each configuration.IV. IMPACT OF ROUTING PROTOCOLS ON SYSTEM PERFORMANCEResults presented <strong>in</strong> this section aim at evaluat<strong>in</strong>g the impact <strong>of</strong> the underly<strong>in</strong>g rout<strong>in</strong>g protocol on themulticast tree creation <strong>an</strong>d ma<strong>in</strong>ten<strong>an</strong>ce. Figures 3 a) <strong>an</strong>d b) show the packet loss <strong>an</strong>d the delay <strong>in</strong>dicesexperienced by the WB nodes at different traffic loads. Specifically, we consider AODV experiments withnodes generat<strong>in</strong>g 0.1pps <strong>an</strong>d 0.2pps, <strong>an</strong>d OLSR experiments with nodes generat<strong>in</strong>g 0.2, 0.5, <strong>an</strong>d 0.8 pps,respectively. System perform<strong>an</strong>ce runn<strong>in</strong>g AODV are quite bad even with such a light traffic load, thus therewas no reason to further <strong>in</strong>crease it. An “x” label for a particular node <strong>an</strong>d a particular experiment denotesthat for that experiment we are not able to derive the <strong>in</strong>dex related to the node (for example, becausesome component <strong>of</strong> the stack crashed dur<strong>in</strong>g the experiment).3 Properly dimension<strong>in</strong>g the queue to f<strong>in</strong>d the right bal<strong>an</strong>ce between delay <strong>an</strong>d packet loss depends on the particular applicationdem<strong>an</strong>ds (actually, <strong>in</strong> other set <strong>of</strong> experiments we have completely removed packet losses by allow<strong>in</strong>g the queue to grow unlimited).


10First <strong>of</strong> all, the impact <strong>of</strong> the Pastry bootstrap procedure should be highlighted. In both cases weexperienced several failures dur<strong>in</strong>g the bootstrap procedure <strong>of</strong> Pastry that highly <strong>in</strong>fluence the systemperform<strong>an</strong>ce. The bootstrap procedure is actually a critical po<strong>in</strong>t for Pastry on wireless networks. In thisphase, the bootstrapp<strong>in</strong>g node has to <strong>in</strong>itialise its <strong>in</strong>ternal data structures by open<strong>in</strong>g TCP connectionswith several other nodes, <strong>an</strong>d gather<strong>in</strong>g portions <strong>of</strong> their <strong>in</strong>ternal data structures. The <strong>in</strong>tr<strong>in</strong>sic <strong>in</strong>stability<strong>of</strong> wireless l<strong>in</strong>ks results <strong>in</strong> possible failures <strong>of</strong> some <strong>of</strong> these connections, which prevents the bootstrapprocedure to successfully complete, <strong>an</strong>d the node to jo<strong>in</strong> the overlay. Such events were quite frequent <strong>in</strong>our testbed. When a node fails to jo<strong>in</strong> the overlay, it also creates <strong>an</strong> <strong>in</strong>dependent tree <strong>an</strong>d it is not able toreceive WB messages from the other particip<strong>an</strong>ts <strong>in</strong> the ma<strong>in</strong> overlay. This generates a high packet loss onthe isolated node, <strong>an</strong>d <strong>in</strong>creases the packet loss also on the other nodes that c<strong>an</strong>not receive the messages<strong>of</strong> the isolated node. For this reason we report <strong>in</strong> the legend <strong>of</strong> the plots the overlays configuration builtdur<strong>in</strong>g each trial. Each experiment related to a specific traffic load has been repeated several times observ<strong>in</strong>gdifferent outcomes <strong>of</strong> the bootstrap procedure. We report <strong>in</strong> this paper the results obta<strong>in</strong>ed select<strong>in</strong>g theexperiments <strong>in</strong> which the number <strong>of</strong> overlay partitions is m<strong>in</strong>imised. This allows us to focus the perform<strong>an</strong>ceevaluation ma<strong>in</strong>ly on the multicast protocol <strong>an</strong>d the application usability, limit<strong>in</strong>g the effects <strong>of</strong> problemsdur<strong>in</strong>g the Pastry bootstrap phase. From the experiments shown <strong>in</strong> Figure 3 we c<strong>an</strong> note that the probability<strong>of</strong> bootstrap failure is higher runn<strong>in</strong>g AODV th<strong>an</strong> OLSR, even at lighter traffic loads. This is due to highdelays required by AODV to establish a new route towards a dest<strong>in</strong>ation, especially with unstable l<strong>in</strong>ks,caus<strong>in</strong>g TCP-connection failures [38], [9].Failures <strong>of</strong> the bootstrap procedure have a signific<strong>an</strong>t effect on packet loss (Figure 3a). In fact, <strong>in</strong> bothAODV experiments, node A is isolated <strong>an</strong>d creates its own overlay. This results <strong>in</strong> packet loss higherth<strong>an</strong> 80% at node A (i.e., it just receives its own WB messages, which counts for about one sixth <strong>of</strong> theoverall WB traffic). The packet loss is about 16% at node C (the root node), <strong>an</strong>d about 70% at the othernodes. Qualitatively, similar remarks apply to the “OLSR 0.5pps” experiment ma<strong>in</strong>ly because some s<strong>of</strong>twarecomponent crashed on node B dur<strong>in</strong>g the experiment, <strong>an</strong>d node A was forced to fail its jo<strong>in</strong> operation try<strong>in</strong>gto connect to node B. Otherwise, <strong>in</strong> the other experiments runn<strong>in</strong>g OLSR, all nodes correctly jo<strong>in</strong>ed theoverlay, giv<strong>in</strong>g a complete view <strong>of</strong> WB perform<strong>an</strong>ce. Specifically, <strong>in</strong> the case “OLSR 0.2pps” the packet lossis 0 at nodes A, B, C, <strong>an</strong>d D, while it is about 3% at node E, <strong>an</strong>d about 13% at node F, s<strong>in</strong>ce its connectionwith the root node is quite less stable th<strong>an</strong> the other nodes’ connection with the root node. In the case“OLSR 0.8pps” the packet loss is higher at all nodes. Specifically, node C measures a packet loss <strong>of</strong> about6%, node A, B, D, <strong>an</strong>d E measure about 20% packet loss, <strong>an</strong>d node F about 45% packet loss. The <strong>in</strong>creasedpacket loss stems from <strong>an</strong> architectural design choice <strong>of</strong> Scribe. Due to the Scribe algorithm, each WBmessage to be distributed on the tree is firstly sent to root, <strong>an</strong>d then forwarded over the tree. Often, thisis <strong>an</strong> excessive load for the root node, which, as the application load <strong>in</strong>creases, becomes unable to deliverall the received messages, <strong>an</strong>d drops them at the send<strong>in</strong>g queue. This event not only depends on the trafficload generated by the application, but also on the rout<strong>in</strong>g protocol used. The fact that the root node drops


11messages at the send<strong>in</strong>g queue is also the reason why the root node always experiences lower packet losswith respect to the other nodes.Even though we have replicated the experiments <strong>in</strong> each configuration several times, we have not beenable to make all the nodes execute the bootstrap procedure correctly <strong>in</strong> the case “OLSR 0.5pps”, while wehave been able <strong>in</strong> the case “OLSR 0.8pps”. This actually does not me<strong>an</strong> that the system works for a higherload (0.8 pps), <strong>an</strong>d does not for a lower load (0.5 pps). In fact, the bootstrap procedure is not <strong>in</strong>fluencedby the application load, because it is executed before the application starts. Therefore, the probability thatall nodes correctly bootstrap exclusively depends on the l<strong>in</strong>ks’ stability. From <strong>an</strong> usability st<strong>an</strong>dpo<strong>in</strong>t, resultsshown for the case “OLSR 0.5pps” are useful even if the network configuration is different from the otherOLSR cases. The packet loss measured on the ma<strong>in</strong> overlay is essentially due to the isolation <strong>of</strong> node A,while the other nodes are able to receive almost all messages from each other. On the other h<strong>an</strong>d, thepacket loss becomes clearly too high at 0.8 pps. We c<strong>an</strong> thus conclude that the system is surely not usablebeyond 0.5pps <strong>in</strong> case <strong>of</strong> OLSR, while the threshold clearly drops to 0.1pps <strong>in</strong> case <strong>of</strong> AODV. In this case,besides never be<strong>in</strong>g able to correctly bootstrap, the system drops m<strong>an</strong>y messages also <strong>in</strong> the ma<strong>in</strong> overlaynetwork. Thus, as far as the packet loss, we c<strong>an</strong> conclude that OLSR outperforms AODV, because it makesthe overlay more stable. It should be po<strong>in</strong>ted out that the better perform<strong>an</strong>ce <strong>of</strong> OLSR with respect toAODV confirms our previous f<strong>in</strong>d<strong>in</strong>gs, e.g. <strong>in</strong> [7], even <strong>in</strong> mobile configurations. This is ma<strong>in</strong>ly becauseAODV builds routes try<strong>in</strong>g to use also unidirectional <strong>an</strong>d asymmetric l<strong>in</strong>ks. Therefore, <strong>in</strong> our experimentsthe network turned out to be always far less stable with AODV th<strong>an</strong> with OLSR.Similar observations c<strong>an</strong> be drawn by focus<strong>in</strong>g on the delay <strong>in</strong>dex (Figure 3b). First <strong>of</strong> all, it should bepo<strong>in</strong>ted out that the delay related to nodes that are the sole member <strong>of</strong> their own overlay (e.g., node A <strong>in</strong>the “AODV 0.1 <strong>an</strong>d 0.2 pps” case) is obviously negligible. Furthermore, it is confirmed that OLSR performsbetter th<strong>an</strong> AODV. In fact, while AODV leads to average delays <strong>of</strong> the order <strong>of</strong> 100 seconds, OLSR producesdelays <strong>of</strong> at most 20 seconds, at 0.8pps. Aga<strong>in</strong>, the root node (C) always experiences a lower delay withrespect to the other nodes <strong>in</strong> the same overlay. From OLSR experiments we c<strong>an</strong> note that at 0.5pps themaximum delay is about 2 seconds, while at 0.8pps node C measures <strong>an</strong> average delay <strong>of</strong> about 5 seconds,nodes A, B, D, <strong>an</strong>d E measure about 10 seconds, <strong>an</strong>d node F about 25 seconds. This confirms the packetloss <strong>an</strong>alysis, also with respect to the usability thresholds.All these experiments have been run us<strong>in</strong>g the FreePastry 1.3 release <strong>of</strong> Pastry code. Even thoughthe experiments have been run <strong>in</strong> a particular setup, <strong>an</strong>d the results refer to particular experiments,the outcome <strong>of</strong> our measurements c<strong>an</strong> be generalised fairly well. Actually, we c<strong>an</strong> conclude that thePastry/Scribe platform was practically unable to support even light application loads, even though thesystem perform<strong>an</strong>ce runn<strong>in</strong>g OLSR demonstrates that a proactive approach at the rout<strong>in</strong>g layer c<strong>an</strong> <strong>in</strong>creasethe usability threshold. However, <strong>in</strong> the last few months new versions <strong>of</strong> FreePastry have been released,<strong>an</strong>d we decided to replicate this <strong>an</strong>alysis us<strong>in</strong>g one <strong>of</strong> the latest versions (1.4.1) to highlight possibleimprovements <strong>in</strong>troduced by s<strong>of</strong>tware ref<strong>in</strong>ements <strong>of</strong> Pastry. In the next section, we present a comparison


12<strong>of</strong> the two releases. As OLSR drastically outperformed AODV <strong>in</strong> all cases, we used only OLSR <strong>in</strong> this newset <strong>of</strong> experiments.V. IMPACT OF PASTRY SOFTWARE REFINEMENTSIn order to compare system perform<strong>an</strong>ce depend<strong>in</strong>g on the s<strong>of</strong>tware release, we do not <strong>an</strong>alyse packetloss <strong>an</strong>d delays per node, but we consider the same <strong>in</strong>dices for the root node, <strong>an</strong>d <strong>an</strong> average value forall the other nodes. The root node represents the best case <strong>of</strong> the system perform<strong>an</strong>ce s<strong>in</strong>ce each messagehas to be firstly sent to it <strong>an</strong>d then forwarded to the others.Figures 4 a) <strong>an</strong>d b) show the perform<strong>an</strong>ce we measured <strong>in</strong> terms <strong>of</strong> average delay <strong>an</strong>d packet loss,respectively, for both s<strong>of</strong>tware releases. Focus<strong>in</strong>g on “Pastry 1.3” curves, we c<strong>an</strong> summarise the <strong>an</strong>alysispresented <strong>in</strong> the previous section. The root node experiences a reasonable QoS for all application loads (i.e.,0.2 pps, 0.5 pps, <strong>an</strong>d 0.8 pps). Specifically, the root-node average delay is always below 5s, <strong>an</strong>d the packetloss below 15%. But other nodes obta<strong>in</strong> a largely unsatisfactory service. We c<strong>an</strong> identify a critical po<strong>in</strong>tfor <strong>an</strong> application load between 0.5 <strong>an</strong>d 0.8 pps. At 0.5 pps, the system perform<strong>an</strong>ce is highly <strong>in</strong>fluencedby the bootstrap failure <strong>of</strong> node A. As far as the packet loss, isolation <strong>of</strong> node A raises the average packetloss <strong>of</strong> non-root nodes to 32.86%. However, the average delay experienced by node A is clearly negligible.Thus, the delay averaged over non-root nodes, which is about 2.3s, is a lower bound <strong>of</strong> the real delaythat non-root nodes would have experienced under this traffic load if the overlay network had been builtcorrectly. When the application load <strong>in</strong>creases to 0.8 pps, all nodes belong to the same overlay. The packetloss slightly decreases to 26.50% at the expenses <strong>of</strong> a sharp <strong>in</strong>crease <strong>of</strong> the average delay, which raises to13.17s. Hence, we c<strong>an</strong> confirm that, under Pastry 1.3, the system is reasonably usable only for very lightapplication loads, <strong>an</strong>d deploy<strong>in</strong>g applications like WB on this platform becomes quite questionable.In order to further reduce the impact <strong>of</strong> Pastry <strong>in</strong>efficiencies on the system perform<strong>an</strong>ce, <strong>in</strong> the secondset <strong>of</strong> experiments runn<strong>in</strong>g the new s<strong>of</strong>tware release we discarded the trials affected by bootstrap failures,only consider<strong>in</strong>g the formation <strong>of</strong> a s<strong>in</strong>gle overlay, even though this represents <strong>an</strong> optimistic assumption <strong>in</strong>real experiments. This might sound unfair to the FreePastry 1.3 version. However, it should be po<strong>in</strong>ted outthat, even consider<strong>in</strong>g only experiments <strong>in</strong> which the bootstrap procedure correctly completed, FreePastry1.3 is clearly unusable at 0.8 pps, while FreePastry 1.4 rema<strong>in</strong>s usable for much greater application loads,as shown by Figures 4a) <strong>an</strong>d b).Analys<strong>in</strong>g “Pastry 1.4” results we noticed that the major modifications to the overlay build<strong>in</strong>g <strong>an</strong>dma<strong>in</strong>ten<strong>an</strong>ce procedures have drastically reduced the overhead <strong>an</strong>d improved the overlay stability [36].Thus, it has been <strong>in</strong>terest<strong>in</strong>g to explore whether this new release improves also the application perform<strong>an</strong>ce<strong>in</strong> our scenario.By look<strong>in</strong>g at Figures 4 a) <strong>an</strong>d b) the perform<strong>an</strong>ce improvement is evident. The critical po<strong>in</strong>t moves byabout one order <strong>of</strong> magnitude, ly<strong>in</strong>g now between 5 <strong>an</strong>d 10 pps. Indeed, at 5 pps also non-root nodesexperience reasonable QoS, s<strong>in</strong>ce the average delay is about 3.5s, <strong>an</strong>d the packet loss is 5%. On the other


13Packet Loss (%)70605040302010Packet Loss - Root Node <strong>an</strong>d Avg non-Root Nodes00.10.2Pastry 1.3 - rootPastry 1.4 - rootPastry 1.3 - avgPastry 1.4 - avg0.512510Application traffic load (pps)2050Delay (s)Average delay - Root Node <strong>an</strong>d Avg-non-Root Nodes22201816141210864200.10.2Pastry 1.3 - rootPastry 1.4 - rootPastry 1.3 - avgPastry 1.4 - avg0.512510Application traffic load (pps)2050(a) Packet loss(b) Average delayFig. 4.Pastry releases comparisonh<strong>an</strong>d, at 10 pps <strong>an</strong>d beyond the application becomes hardy usable at <strong>an</strong>y node. Note that, even though theaverage delay at root node would be almost acceptable also at 10 <strong>an</strong>d 20pps (i.e., it is below the usabilitythreshold), the packet loss <strong>in</strong>creases to 35% <strong>an</strong>d 42%, respectively.Note also that between 10pps <strong>an</strong>d 20pps the delay curve relative to the root node flattens. This is actuallya side effect <strong>of</strong> the higher packet loss experienced at 20pps. In detail, <strong>in</strong> the topology <strong>of</strong> our experiments,non-root nodes are either 1 or 2 <strong>hop</strong>s away from root. In the next section we show that even at 20pps1-<strong>hop</strong>-away nodes are able to send to root almost all the traffic locally generated. The additional packetloss experienced by root at 20pps is thus due to less messages received from 2-<strong>hop</strong>-away nodes. In otherwords, out <strong>of</strong> the whole bunch <strong>of</strong> messages received by root, the fraction <strong>of</strong> messages received by 1-<strong>hop</strong>-away nodes <strong>in</strong>creases when the traffic load shifts from 10 to 20pps. S<strong>in</strong>ce messages from 1-<strong>hop</strong>-awaynodes experience signific<strong>an</strong>t lower delay th<strong>an</strong> messages from 2-<strong>hop</strong>-away nodes, the fraction <strong>of</strong> messagesexperienc<strong>in</strong>g low delays <strong>in</strong>creases too. This reduces the average delay measured at root, result<strong>in</strong>g <strong>in</strong> the flatshape <strong>of</strong> the curve. The same phenomenon does not apply to non-root nodes. Recall that messages haveto reach the root before be<strong>in</strong>g delivered to the other nodes. The delay experienced by non-root nodes <strong>in</strong>receiv<strong>in</strong>g messages from root <strong>in</strong>creases between 10 <strong>an</strong>d 20 pps. Thus, even though <strong>in</strong> both cases messagesexperience – on average – the same delay between the orig<strong>in</strong>at<strong>in</strong>g node <strong>an</strong>d root, <strong>in</strong> the 20 pps case theyundergo higher delay along the path between root <strong>an</strong>d the f<strong>in</strong>al dest<strong>in</strong>ation.To summarise, the above <strong>an</strong>alysis shows a steep improvement <strong>in</strong> the user-perceived QoS when mov<strong>in</strong>gfrom a FreePastry release to a more adv<strong>an</strong>ced one. However, a critical po<strong>in</strong>t still exists, beyond which itpractically makes no sense to use GC applications like WB on this platform. At this po<strong>in</strong>t it is really import<strong>an</strong>tto underst<strong>an</strong>d whether this critical po<strong>in</strong>t is go<strong>in</strong>g to eventually disappear th<strong>an</strong>ks to future s<strong>of</strong>tware releases,or it is <strong>in</strong>tr<strong>in</strong>sic <strong>in</strong> the Pastry/Scribe design. In the follow<strong>in</strong>g sections we <strong>an</strong>alyse the results achieved withFreePastry 1.4.1 more <strong>in</strong> depth, <strong>an</strong>d show that, <strong>in</strong>dependently <strong>of</strong> s<strong>of</strong>tware ref<strong>in</strong>ements, the design <strong>of</strong> Scribe


14Packet Loss (%)10080604020Packet loss to the Scribe Root node1 pps5 pps10 pps20 pps1 <strong>hop</strong>2 <strong>hop</strong>sPacket Loss (%)10080604020Packet loss from the Scribe Root node1 pps5 pps10 pps20 pps1 <strong>hop</strong>2 <strong>hop</strong>s0EDFNodeAB0EDFNodeAB(a) Packet loss towards root(b) Packet loss from rootFig. 5.Packet loss <strong>an</strong>alysis<strong>in</strong>cludes features not suitable for the multi-<strong>hop</strong> ad <strong>hoc</strong> networks environment.VI. ROOT AS THE CENTRAL NODE: A RATHER OPTIMISTIC SETUPIn the set <strong>of</strong> experiments presented so far (related to Pastry 1.4.1) we have placed the root node at thecenter <strong>of</strong> the topology to m<strong>in</strong>imise the average <strong>hop</strong> dist<strong>an</strong>ce to <strong>an</strong>y other node. S<strong>in</strong>ce it is well-known thatTCP perform<strong>an</strong>ce drastically worsens as the <strong>hop</strong> dist<strong>an</strong>ce <strong>in</strong>creases ([4], [38]), this represents <strong>an</strong> optimisticsetup. To have a clearer picture <strong>of</strong> the system behavior, we now focus on the average delay <strong>an</strong>d packet lossexperienced by each s<strong>in</strong>gle node. Specifically, curves <strong>in</strong> Figure 3a) <strong>an</strong>d 3b) show the average perform<strong>an</strong>ceexperienced by non-root nodes, <strong>an</strong>d thus provide <strong>in</strong>dications about the average QoS a user may expect. Inthis section we <strong>an</strong>alyse the perform<strong>an</strong>ce <strong>of</strong> nodes at 1 <strong>an</strong>d 2 <strong>hop</strong>s from root separately. Together with the“root-curves” <strong>in</strong> Figures 3a) <strong>an</strong>d 3b) this provides a precise view <strong>of</strong> the expected QoS with respect to theposition <strong>of</strong> a node <strong>in</strong> our topology.Figures 5a) <strong>an</strong>d b) show the packet loss experienced by each s<strong>in</strong>gle node towards <strong>an</strong>d from root,respectively. The packet loss <strong>of</strong> the i-th node towards root is def<strong>in</strong>ed as the ratio between the number<strong>of</strong> messages generated at the i-th node <strong>an</strong>d not received by root, <strong>an</strong>d the number <strong>of</strong> messages generated atthe i-th node, i.e., 1− R(i) r, where R (i)G (i) r is the number <strong>of</strong> messages generated by the i-th node <strong>an</strong>d receivedby root, <strong>an</strong>d G (i) is the number <strong>of</strong> messages generated by the i-th node. The packet loss <strong>of</strong> the i-th nodefrom root is def<strong>in</strong>ed as the ratio between the number <strong>of</strong> messages not received by the i-th node out <strong>of</strong>the total number <strong>of</strong> messages sent by root, <strong>an</strong>d the total number <strong>of</strong> messages sent by root, i.e., 1− R(r) i,S (r)where R (r)i is the number <strong>of</strong> messages sent by root <strong>an</strong>d received by the i-th node, <strong>an</strong>d S (r) is the number<strong>of</strong> messages sent by root.Note that, due to the Scribe architecture, root not only sends messages locallygenerated, but also messages that it receives from all the other nodes.Due to packet loss towards root,the number <strong>of</strong> messages sent by root is less th<strong>an</strong> the sum <strong>of</strong> messages generated by the sender nodes, i.e.


15S (r) < ∑ Ni=1 G(i) . In addition, the i-th node experiences packet loss 0 only if i) the packet loss <strong>of</strong> allnodes towards root is 0, <strong>an</strong>d ii) the packet loss <strong>of</strong> the i-th node from root is 0.On the one h<strong>an</strong>d, Figures 5a) <strong>an</strong>d b) confirm that, as far as the packet loss, the critical po<strong>in</strong>t for WBusability lies between 5 <strong>an</strong>d 10pps. Indeed, below 5pps the packet loss towards root is 0 at all nodes. Thisme<strong>an</strong>s that all nodes are able to send messages locally generated to root. Apart from node F, also thepacket loss from root is 0 at all nodes. Thus, the overall packet loss experienced by all nodes except F is 0at 5pps <strong>an</strong>d below, mak<strong>in</strong>g WB usable.On the other h<strong>an</strong>d, Figures 5a) <strong>an</strong>d b) show a drastic difference between nodes at 1 <strong>hop</strong> <strong>an</strong>d 2 <strong>hop</strong>sdist<strong>an</strong>ce from root. Even though the magnitude <strong>of</strong> this difference is quite surpris<strong>in</strong>g, a steep perform<strong>an</strong>cedecrease <strong>in</strong> the case <strong>of</strong> multi-<strong>hop</strong> connections was expected (see, for example, [4], [38]). Specifically,Figure 5a) shows that, even at 10 pps <strong>an</strong>d 20 pps, 1-<strong>hop</strong>-away nodes are able to send almost all theirmessages to root. Instead, 2-<strong>hop</strong>-away nodes see their outgo<strong>in</strong>g traffic cut by 50% to 89%. Figure 5b)shows a similar trend, <strong>in</strong> the sense that at 10 <strong>an</strong>d 20 pps 2-<strong>hop</strong>-away nodes experience a far higher packetloss th<strong>an</strong> 1-<strong>hop</strong>-away nodes. However, there is a difference worth to be noted. While for 2-<strong>hop</strong>-away nodesthe packet loss is higher <strong>in</strong> the direction towards root, for 1-<strong>hop</strong>-away nodes the packet loss is higher <strong>in</strong>the direction from root. At a higher level, one could note that as the traffic load <strong>in</strong>creases, Scribe cuts itbecause the root node becomes overloaded. In our configuration, while 1-<strong>hop</strong>-away nodes suffer only <strong>in</strong>the direction from root, 2-<strong>hop</strong>-away nodes ma<strong>in</strong>ly suffer <strong>in</strong> the direction towards root. Actually, we havefound configurations <strong>in</strong> which for both cases (i.e., 1 <strong>an</strong>d 2 <strong>hop</strong>s) the ma<strong>in</strong> traffic cut occurs <strong>in</strong> the directionfrom root. Underst<strong>an</strong>d<strong>in</strong>g the reason <strong>of</strong> this behavior is not trivial, thus we are currently <strong>an</strong>alys<strong>in</strong>g thesystem even more deeply. However, it should be noted that, as far as the application-level QoS, the precisedirection along which the ma<strong>in</strong> traffic cut occurs is not that import<strong>an</strong>t.Figures 5a) <strong>an</strong>d 5b) f<strong>in</strong>ally show that the presence <strong>of</strong> 2-<strong>hop</strong>-away nodes makes the application unusablealso for 1-<strong>hop</strong>-away nodes. For example, let us focus on the 10 pps case. Nodes 1-<strong>hop</strong> away from rootmeasure 0 packet loss on both directions. However, they are unable to receive most <strong>of</strong> the messagesgenerated at nodes 2-<strong>hop</strong>s away from root, because those nodes suffer very high packet loss towards root.This highlights that, s<strong>in</strong>ce all messages have to be firstly sent to the root, a poor connection between aparticular node <strong>an</strong>d root makes all other nodes unable to receive the messages generated by this node.Analys<strong>in</strong>g the delay figures allows us to highlight a further feature <strong>of</strong> the system. Specifically, Table Ishows the average delay <strong>an</strong>d the ma<strong>in</strong> percentiles depend<strong>in</strong>g on the application-traffic load, <strong>an</strong>d on the<strong>hop</strong>-dist<strong>an</strong>ce from root. By look<strong>in</strong>g at the average delays only, one could conclude that the application isusable even at 10pps by nodes at most 1-<strong>hop</strong> away from root. However, if the usability threshold is def<strong>in</strong>edwith respect to the 90th percentile <strong>in</strong>stead <strong>of</strong> the average value, the critical po<strong>in</strong>t shifts below 5pps (forall nodes).Table I also shows a drastic difference between 1pps <strong>an</strong>d the other traffic loads. Indeed, at 1pps WBperform<strong>an</strong>ce are completely satisfactory, as the 99th percentile for 2-<strong>hop</strong>-away nodes is about 600ms. At


16avg/percentiles root 1 <strong>hop</strong> 2 <strong>hop</strong>s1pps avg 0.032 0.055 0.10090 0.070 0.115 0.22195 0.126 0.159 0.31699 0.282 0.321 0.6045pps avg 2.710 2.900 3.92490 11.00 11.29 13.1595 13.74 13.95 16.6299 23.19 23.28 25.7110pps avg 10.22 10.57 14.5290 34.06 34.79 43.1195 65.06 65.11 71.3699 101.4 101.4 101.820pps avg 9.11 13.12 21.1590 23.09 31.11 88.0395 86.30 89.30 115.199 145.4 145.7 146.4TABLE IDELAYS DEPENDING ON THE HOP-DISTANCE FROM ROOT.higher traffic loads there is a signific<strong>an</strong>t difference between the average delays <strong>an</strong>d the 90th percentiles.This suggests that the delay distributions have a long tail. This is confirmed by look<strong>in</strong>g at Figure 6a), whichplots the CCDF <strong>of</strong> delays measured at root node. Clearly, for each traffic load, the CCDF at root is a lowerbound <strong>of</strong> CCDFs at <strong>an</strong>y other node. Figure 6 shows that CCDFs for application loads <strong>of</strong> 5pps <strong>an</strong>d beyond c<strong>an</strong>be lower bounded by a Pareto distribution with parameter 0.25. Specifically, they show a long-tail pattern<strong>in</strong> the r<strong>an</strong>ge from 5ms to 10s. Figures 6b) <strong>an</strong>d c) show the same feature for 1-<strong>hop</strong>-away <strong>an</strong>d 2-<strong>hop</strong>-awaynodes, as well. Be<strong>in</strong>g the coefficients <strong>of</strong> the approximat<strong>in</strong>g Pareto distribution far below 1, the tail is veryheavy also <strong>in</strong> these cases. At the application level, this me<strong>an</strong>s that, even though the average delay c<strong>an</strong> laybelow the usability threshold, delay values are highly variable, <strong>an</strong>d thus no strict QoS guar<strong>an</strong>tees c<strong>an</strong> begr<strong>an</strong>ted.From results presented <strong>in</strong> this section we c<strong>an</strong> conclude that the centralised approach <strong>of</strong> Scribe generallyh<strong>in</strong>ders the use <strong>of</strong> GC applications over multi-<strong>hop</strong> ad <strong>hoc</strong> networks. Specifically, we have highlighted twoma<strong>in</strong> <strong>in</strong>efficiencies:• even few nodes poorly connected to root prevent all the nodes from us<strong>in</strong>g GC application properly.This is because those nodes will experience very high packet losses towards the root, <strong>an</strong>d all the othernodes will be unable to receive their messages. Even nodes that could be physically very close to the“poorly connected” nodes, <strong>an</strong>d thus might be potentially able to correctly communicate with them,will just get a small fraction <strong>of</strong> their messages.


171Delays CCDF - root node1Delays CCDF - 1-<strong>hop</strong> nodesP(Delay>d)0.10.0120pps10pps5pps1ppspareto(0.25)0.0010.001 0.01 0.1 1 10 100 1000d (s)(a) root nodeP(Delay>d)0.10.0120pps10pps5pps1ppspareto(0.5)0.0010.001 0.01 0.1 1 10 100 1000d (s)(b) 1-<strong>hop</strong>-away nodes1Delays CCDF - 2-<strong>hop</strong> nodesP(Delay>d)0.10.0120pps10pps5pps1ppspareto(0.5)0.0010.001 0.01 0.1 1 10 100 1000d (s)(c) 2-<strong>hop</strong>-away nodesFig. 6.Delay CCDFs• The root node very likely experiences a long-tailed delay distribution. In these cases <strong>an</strong>y other nodeexperiences the same pattern <strong>in</strong> the delay distribution.Both these drawbacks are <strong>in</strong>tr<strong>in</strong>sic to the Scribe design, <strong>an</strong>d c<strong>an</strong>not be completely addressed either byref<strong>in</strong>ed s<strong>of</strong>tware releases, or by simply improv<strong>in</strong>g the underly<strong>in</strong>g DHT. Note also that our experimentswere run with quite powerful laptops. Perform<strong>an</strong>ce could thus be even worse <strong>in</strong> case <strong>of</strong> less powerfuldevices, such as PDAs, that are natural c<strong>an</strong>didates to be used <strong>in</strong> multi-<strong>hop</strong> ad <strong>hoc</strong> networks environments.VII. SYSTEM PERFORMANCE VARYING THE SCRIBE ROOT NODE LOCATIONIn the experiments presented so far, the Scribe Root Node was always at the center <strong>of</strong> the networktopology <strong>an</strong>d all the other nodes were 2 <strong>hop</strong>s away at most. However, <strong>in</strong> the general case no assumptionsabout the root location c<strong>an</strong> be done. Therefore, we r<strong>an</strong> a further set <strong>of</strong> experiments by plac<strong>in</strong>g the rootnode at one edge <strong>of</strong> the network. This might sound like a pessimistic configuration, but it should be notedthat hav<strong>in</strong>g the root node at one edge <strong>of</strong> the network is more likely th<strong>an</strong> hav<strong>in</strong>g it at the center <strong>of</strong> the


18Packet loss at each s<strong>in</strong>gle nodeAverage delay at each s<strong>in</strong>gle nodePacket Loss (%)10080604020root1 pps5 pps10 pps20 pps1 h2 <strong>hop</strong>s 3 h 4 hDelay (s)605040302010root1 pps5 pps10 pps20 pps1 h2 <strong>hop</strong>s3 h4 h0C(R)BADEF0C(R)BADEFNodeNode(a) Packet Loss(b) Average delayFig. 7.S<strong>in</strong>gle node <strong>an</strong>alysistopology. In detail, with respect to the topology <strong>in</strong> Figure 1(b), we swapped the positions <strong>of</strong> nodes C (rootnode) <strong>an</strong>d A. This setup also allows us to better <strong>an</strong>alyse the impact <strong>of</strong> longer paths on Scribe, s<strong>in</strong>ce wehave TCP connections sp<strong>an</strong>n<strong>in</strong>g 1 to 4 <strong>hop</strong>s.Figures 7a) <strong>an</strong>d b) show the perform<strong>an</strong>ce measured at each s<strong>in</strong>gle node <strong>in</strong> terms <strong>of</strong> average delay <strong>an</strong>dpacket loss respectively. They confirm that the system now is usable just at lighter application loads. Evenat 1 pps, while the packet loss is 0 at all nodes, the average delay r<strong>an</strong>ges between 5 <strong>an</strong>d 10s. Thus, even at1 pps, the system is usable just for applications with loose delay constra<strong>in</strong>ts. At a load <strong>of</strong> 5 pps the system iscompletely unusable for nodes 3-<strong>hop</strong> away from root <strong>an</strong>d beyond. At higher loads, the perform<strong>an</strong>ce mightbe too low even for the root node.VIII. DISCUSSION AND FUTURE WORKScribe was designed hav<strong>in</strong>g <strong>in</strong> m<strong>in</strong>d a resource-rich environment, such as the legacy wired Internet,<strong>an</strong>d it has shown to perform very well <strong>in</strong> it [6]. However, when used over multi-<strong>hop</strong> ad <strong>hoc</strong> networks,the centralised approach, <strong>in</strong> which a root node is <strong>in</strong> charge <strong>of</strong> deliver<strong>in</strong>g all application messages, leadsto very low perform<strong>an</strong>ce <strong>in</strong> terms <strong>of</strong> packet loss <strong>an</strong>d delay.One might th<strong>in</strong>k <strong>of</strong> us<strong>in</strong>g solutions likeSplitStream [28], which splits a s<strong>in</strong>gle multicast group <strong>in</strong> several trees, thus reduc<strong>in</strong>g the concentration atthe root. However, SplitStream requires signific<strong>an</strong>t a-priori knowledge about the application traffic load,<strong>an</strong>d a quite signific<strong>an</strong>t pl<strong>an</strong>n<strong>in</strong>g effort. Actually, it is aga<strong>in</strong> a system designed for large-scale wired networks,which <strong>in</strong>troduces even further network<strong>in</strong>g structures to the shared tree used by Scribe.In general, we believe that one <strong>of</strong> the ma<strong>in</strong> reasons <strong>of</strong> the low perform<strong>an</strong>ce is <strong>in</strong>deed the use <strong>of</strong> astructured multicast solution. Besides requir<strong>in</strong>g signific<strong>an</strong>t overhead <strong>in</strong> terms <strong>of</strong> m<strong>an</strong>agement traffic, suchsolutions tend to concentrate the costs <strong>of</strong> the application (<strong>in</strong> terms <strong>of</strong> network/computation resources)on a few nodes <strong>an</strong>d l<strong>in</strong>ks. If these nodes/l<strong>in</strong>ks are underprovisioned, or happen to be placed <strong>in</strong> adverse


19locations, the whole system may implode, mak<strong>in</strong>g it unable to support the application. Structured solutionsare a good choice <strong>in</strong> large-scale systems designed for the legacy Internet (like Scribe is). Indeed, <strong>in</strong> thiscase the network <strong>an</strong>d computational resources are not a big issue, <strong>an</strong>d structured solutions allow thesystem to scale up to thous<strong>an</strong>ds <strong>of</strong> nodes. However, grow<strong>in</strong>g up to such a scale is not reasonable for multi<strong>hop</strong>ad <strong>hoc</strong> networks. Theoretical results <strong>an</strong>d practical experiences (e.g., [37], [4]) show that the mostreasonable scale for flat ad <strong>hoc</strong> networks based on 802.11 technologies is up to 10-20 nodes, <strong>an</strong>d 2-3 <strong>hop</strong>s.Furthermore, concentrat<strong>in</strong>g the load on a few resources <strong>an</strong>d m<strong>an</strong>ag<strong>in</strong>g the multicast structure may becomeserious problems <strong>in</strong> these network<strong>in</strong>g environments.Based on these remarks, we believe that structure-less (or stateless) multicast be a more reasonable choice<strong>in</strong> this environment. For example, approaches like Differential Dest<strong>in</strong>ation <strong><strong>Multi</strong>cast</strong> (DDM, [39]) <strong>an</strong>d RouteDriven Gossip (RDG, [34]) seem to be very <strong>in</strong>terest<strong>in</strong>g solutions. Typically, accord<strong>in</strong>g to this approach, eachmember <strong>of</strong> a multicast group knows the other members <strong>of</strong> the same group (actually, RDG also works withpartial knowledge). When a message is locally generated, it is sent to the group members by us<strong>in</strong>g theunderly<strong>in</strong>g rout<strong>in</strong>g protocol, without requir<strong>in</strong>g <strong>an</strong>y multicast structure. Mech<strong>an</strong>isms are <strong>in</strong>cluded to sendjust once a message addressed to dest<strong>in</strong>ations shar<strong>in</strong>g <strong>an</strong> <strong>in</strong>itial portion <strong>of</strong> the path from a send<strong>in</strong>g node.These approaches spread the load more evenly over the nodes <strong>an</strong>d l<strong>in</strong>ks <strong>of</strong> the network, avoid concentrationon a few nodes/l<strong>in</strong>ks, <strong>an</strong>d typically work remarkably well <strong>in</strong> small <strong>an</strong>d medium size networks. However,they require costly mech<strong>an</strong>isms to collect <strong>an</strong>d ma<strong>in</strong>ta<strong>in</strong> the (partial) list <strong>of</strong> group members at each node.Typically, nodes have to periodically flood the network to <strong>an</strong>nounce their presence, or to check for othernodes’ liveness. Therefore, we are currently design<strong>in</strong>g <strong>an</strong> improved version <strong>of</strong> Scribe, that reta<strong>in</strong>s thestructure-less features <strong>of</strong> DDM <strong>an</strong>d RDG, but avoids such costly mech<strong>an</strong>isms by exploit<strong>in</strong>g a cross-layerapproach. In this view, further adv<strong>an</strong>tages would be brought <strong>in</strong> by replac<strong>in</strong>g Pastry with CrossROAD. Inall previous experiments, CrossROAD has shown to outperform Pastry <strong>in</strong> multi-<strong>hop</strong> ad <strong>hoc</strong> networks byfix<strong>in</strong>g all its <strong>in</strong>efficiencies (e.g., path stretches, bootstrap failures, nodes’ isolation, etc.). S<strong>in</strong>ce the resultspresented <strong>in</strong> this paper clearly show that even one <strong>of</strong> the best legacy <strong>p2p</strong> multicast system needs drasticimprovements to work <strong>in</strong> ad <strong>hoc</strong> networks, us<strong>in</strong>g CrossROAD to support <strong>an</strong> enh<strong>an</strong>ced <strong>p2p</strong> multicast systemis a natural choice.REFERENCES[1] M. Gerla, C. L<strong>in</strong>dem<strong>an</strong>n, <strong>an</strong>d A. Rowstron, “P2P MANET’s - New Research Issues,” <strong>in</strong> Perspectives Works<strong>hop</strong>: Peer-to-PeerMobile <strong>Ad</strong> Hoc <strong>Networks</strong> - New Research Issues, M. Gerla, C. L<strong>in</strong>dem<strong>an</strong>n, <strong>an</strong>d A. Rowstron, Eds., 2005. [Onl<strong>in</strong>e]. Available:http://drops.dagstuhl.de/opus/volltexte/2005/213[2] (2006, September) ACM Mobicom MobiShare Works<strong>hop</strong>. [Onl<strong>in</strong>e]. Available: http://www.mobishare.org/[3] G. Anastasi, E. Borgia, M. Conti, E. Gregori, <strong>an</strong>d A. Passarella, “Underst<strong>an</strong>d<strong>in</strong>g the real behavior <strong>of</strong> mote <strong>an</strong>d 802.11 ad <strong>hoc</strong>networks: <strong>an</strong> experimental approach,” Pervasive <strong>an</strong>d Mobile Comput<strong>in</strong>g, vol. 1, no. 2, pp. 237–256, July 2005.[4] P. Gunn<strong>in</strong>gberg, H. Lundgren, E. Nordstroem, <strong>an</strong>d C. Tschud<strong>in</strong>, “Lessons from experimental m<strong>an</strong>et research,” <strong>Ad</strong> Hoc <strong>Networks</strong>Journal, vol. 3, no. 2, pp. 221–233, Mar. 2005.


20[5] E. Hu<strong>an</strong>g, W. Hu, J. Crowcr<strong>of</strong>t, <strong>an</strong>d I. Wassell, “Towards commercial mobile ad <strong>hoc</strong> network applications: a radio dispatchsystem,” <strong>in</strong> Proc. <strong>of</strong> ACM MobiHoc, Urb<strong>an</strong>a, IL, May 2005.[6] M. Castro, M. B. Jones, A.-M. Kermarrec, A. Rowstron, M. Theimer, H. W<strong>an</strong>g, <strong>an</strong>d A. Wolm<strong>an</strong>, “An evaluation <strong>of</strong> scalableapplication-level multicast built us<strong>in</strong>g peer-to-peer overlays,” <strong>in</strong> Proc. <strong>of</strong> INFOCOM, S<strong>an</strong> Fr<strong>an</strong>cisco, CA, Apr. 2003.[7] E. Borgia, M. Conti, F. Delmastro, <strong>an</strong>d E. Gregori, “Experimental comparison <strong>of</strong> Rout<strong>in</strong>g <strong>an</strong>d Middleware solutions for Mobile<strong>Ad</strong> Hoc <strong>Networks</strong>: <strong>Legacy</strong> vs Cross-Layer approach,” <strong>in</strong> Proc. <strong>of</strong> the ACM SIGCOMM E-WIND Works<strong>hop</strong>, Philadelphia, Aug. 2005.[8] E. Borgia, M.Conti, F. Delmastro, <strong>an</strong>d A. Passarella, “MANET perspective: current <strong>an</strong>d forthcom<strong>in</strong>g perspective,” <strong>in</strong> Proc. <strong>of</strong> ISTMobile <strong>an</strong>d Wireless Communication Summit 2006, Myconos, June 2006.[9] E. Borgia, “Experimental Evaluation <strong>of</strong> <strong>Ad</strong> Hoc Rout<strong>in</strong>g Protocols,” <strong>in</strong> Proc. <strong>of</strong> IEEE PerCom PWN Works<strong>hop</strong>, Kauai Isl<strong>an</strong>d, Hawaii,Mar. 2005.[10] (2005, July) IEEE ICPS REALMAN Works<strong>hop</strong>. S<strong>an</strong>tor<strong>in</strong>i, Greece. [Onl<strong>in</strong>e]. Available: http://www.cl.cam.ac.uk/realm<strong>an</strong>/05/[11] (2005, Aug.) ACM SIGCOMM E-WIND Works<strong>hop</strong>. Philadelphia, PA. [Onl<strong>in</strong>e]. Available: http://www-ece.rice.edu/E-WIND/[12] Department <strong>of</strong> Computer Systems at Uppsala (Sweden). APE: <strong>Ad</strong> <strong>hoc</strong> Protocol Evaluation testbed. [Onl<strong>in</strong>e]. Available:http://apetestbed.sourceforge.net/[13] N. Vaidya, J. Bernhard, V. Veeravalli, P. Kumar, <strong>an</strong>d R. Iyer, “Ill<strong>in</strong>ois wireless w<strong>in</strong>d tunnel: A testbed for experimental evaluation<strong>of</strong> wireless networks,” <strong>in</strong> Proc. <strong>of</strong> the ACM SIGCOMM E-WIND Works<strong>hop</strong>, Philadelphia, PA, Aug. 2005.[14] K. Ramach<strong>an</strong>dr<strong>an</strong>, S. Kaul, S. Mathur, M. Gruteser, <strong>an</strong>d I. Seskar, “Towards large-scale mobile network emulation through spatialswitch<strong>in</strong>g on a wireless grid,” <strong>in</strong> Proc. <strong>of</strong> the ACM SIGCOMM E-WIND Works<strong>hop</strong>, Philadelphia, PA, Aug. 2005.[15] P. De, A. R<strong>an</strong>iwala, S. Sharma, <strong>an</strong>d T. Chiueh, “MiNT: A M<strong>in</strong>iaturized Network Testbed for Mobile Wireless Research,” <strong>in</strong> Proc.<strong>of</strong> INFOCOM 2005, Miami, FL, Mar. 2005.[16] R. S. Gray, D. Kotz, C. Newport, N. Dubrovsky, A. Fiske, J. Liu, C. Masone, S. McGrath, <strong>an</strong>d Y. Yu<strong>an</strong>, “Outdoor ExperimentalComparison <strong>of</strong> Four <strong>Ad</strong> Hoc Rout<strong>in</strong>g Algorithms,” <strong>in</strong> Proc. <strong>of</strong> MSWiM 2004, Venice, Italy, October 2004.[17] R. Bruno, M. Conti, <strong>an</strong>d E. Gregori, “Mesh <strong>Networks</strong>: Commodity <strong>Multi</strong><strong>hop</strong> <strong>Ad</strong> Hoc <strong>Networks</strong>,” IEEE Communications Magaz<strong>in</strong>e,March 2005.[18] A. Tonnesen. OLSR: Optimized l<strong>in</strong>k state rout<strong>in</strong>g protocol. Institute for <strong>in</strong>formatics at the University <strong>of</strong> Oslo (Norway).[Onl<strong>in</strong>e]. Available: http://www.olsr.org[19] AODV: <strong>Ad</strong> <strong>hoc</strong> on dem<strong>an</strong>d distnce vector rout<strong>in</strong>g. Dept. <strong>of</strong> Information technology at Uppsala University (Sweden). [Onl<strong>in</strong>e].Available: http://user.it.uu.se/ henrikl/aodv/[20] F. Delmastro <strong>an</strong>d A. Passarella, “An experimental study <strong>of</strong> <strong>p2p</strong> group-communication applications <strong>in</strong> real-world m<strong>an</strong>ets,” <strong>in</strong> Proc.<strong>of</strong> IEEE ICPS REALMAN 2005 Works<strong>hop</strong>, S<strong>an</strong>tor<strong>in</strong>i, Greece, July 2005.[21] F. Delmastro, A. Passarella, <strong>an</strong>d M. Conti, “Experimental <strong>an</strong>alysis <strong>of</strong> <strong>p2p</strong> shared-tree multicast on m<strong>an</strong>ets: the case <strong>of</strong> scribe,”<strong>in</strong> The IFIP Fifth Annual Mediterr<strong>an</strong>e<strong>an</strong> <strong>Ad</strong> Hoc Network<strong>in</strong>g Works<strong>hop</strong> (Med-Hoc-Net), June 2006.[22] F. Delmastro, “From Pastry to CrossROAD: Cross-layer R<strong>in</strong>g Overlay for AD <strong>hoc</strong> networks,” <strong>in</strong> Proc. <strong>of</strong> IEEE PerCom MP2PWorks<strong>hop</strong>, Kauai Isl<strong>an</strong>d, Hawaii, Mar. 2005.[23] Y. hua Chu, S. G. Cao, S. Sesh<strong>an</strong>, <strong>an</strong>d H. Zh<strong>an</strong>g, “A case for end system multicast,” IEEE Journal on Selected Areas <strong>in</strong> Communication(JSAC), vol. 20, no. 8, pp. 1456–1471, October 2002.[24] M. Castro, P. Druschel, A.-M. Kermarrec, <strong>an</strong>d A. Rowstron, “Scribe: A large-scale <strong>an</strong>d decentralised application-level multicast<strong>in</strong>frastructure,” IEEE JSAC, vol. 20, no. 8, Oct. 2002.[25] S. Ratnasamy, M. H<strong>an</strong>dley, R. Karp, <strong>an</strong>d S. Shenker, “Application-level multicast us<strong>in</strong>g content-addressable networks,” <strong>in</strong> 3rdInternational Works<strong>hop</strong> on Networked Group Communication, November 2001.[26] S. Q. Zhu<strong>an</strong>g, B. Y. Zh<strong>an</strong>g, A. D. Joseph, R. H. Katz, <strong>an</strong>d J. D. Kubiatowicz, “Bayeux: An architecture for scalable <strong>an</strong>d faulttoler<strong>an</strong>twide-area data dissem<strong>in</strong>ation,” <strong>in</strong> Eleventh International Works<strong>hop</strong> on Network <strong>an</strong>d Operat<strong>in</strong>g Systems Support for DigitalAudio <strong>an</strong>d Video (NOSSDAV), 2001.[27] S. B<strong>an</strong>erjee, B. Bhattacharjee, <strong>an</strong>d C. Kommareddy, “Scalable application layer multicast,” <strong>in</strong> ACM SIGCOMM, August 2002.[28] M. Castro, P. Druschel, A.-M. Kermarrec, A. N<strong>an</strong>di, A. Rowstron, <strong>an</strong>d A. S<strong>in</strong>gh, “SplitStream: High b<strong>an</strong>dwidth multicast <strong>in</strong>cooperative environments,” <strong>in</strong> ACM SOSP, October 2003.


21[29] M. Ge, S. V. Krishnamurthy, <strong>an</strong>d M. Faloutsos, “Application versus Network Layer <strong><strong>Multi</strong>cast</strong><strong>in</strong>g <strong>in</strong> <strong>Ad</strong> Hoc <strong>Networks</strong>: The ALMARout<strong>in</strong>g Protocol,” Elsevier <strong>Ad</strong> Hoc <strong>Networks</strong> Journal, to appear.[30] C. Gui <strong>an</strong>d P. Mohapatra, “Overlay <strong><strong>Multi</strong>cast</strong> for MANETs Us<strong>in</strong>g Dynamic Virtual Mesh,” ACM/Spr<strong>in</strong>ger Wireless <strong>Networks</strong>(WINET), to appear.[31] E. Royer <strong>an</strong>d C. Perk<strong>in</strong>s, “<strong><strong>Multi</strong>cast</strong> ad <strong>hoc</strong> on-dem<strong>an</strong>d dist<strong>an</strong>ce vector (maodv) rout<strong>in</strong>g,” 2000. [Onl<strong>in</strong>e]. Available:http://www3.ietf.org/proceed<strong>in</strong>gs/00jul/I-D/m<strong>an</strong>et-maodv-00.txt[32] S.-J. Lee, W. Su, <strong>an</strong>d M. Gerla, “On-Dem<strong>an</strong>d <strong><strong>Multi</strong>cast</strong> Rout<strong>in</strong>g Protocol (ODMRP) for <strong>Ad</strong> Hoc <strong>Networks</strong>,” 2000. [Onl<strong>in</strong>e].Available: http://www.cs.ucla.edu/NRL/wireless/PAPER/draft-ietf-m<strong>an</strong>et-odmrp-02.txt[33] S. Y<strong>an</strong>g <strong>an</strong>d J. Wu, New Technologies <strong>of</strong> <strong><strong>Multi</strong>cast</strong><strong>in</strong>g <strong>in</strong> MANETS, Y. Xiao <strong>an</strong>d Y. P<strong>an</strong>, Eds. Nova Publishers, 2005.[34] J. Luo, P. Eugster, <strong>an</strong>d J.-P. Hubaux, “Probabilistic reliable multicast <strong>in</strong> ad <strong>hoc</strong> networks,” <strong>Ad</strong> Hoc <strong>Networks</strong>, vol. 2, pp. 369–386,2004.[35] A. Rowstron <strong>an</strong>d P. Druschel, “Pastry: Scalable, decentralized object localtion <strong>an</strong>d rout<strong>in</strong>g for large scale peer-to-peer systems,”LNCS, vol. 2218, pp. 329–350, 2001.[36] FreePastry, Rice University. [Onl<strong>in</strong>e]. Available: http://freepastry.rice.edu[37] P. Gupta <strong>an</strong>d P. Kumar, “The capacity <strong>of</strong> wireless networks,” IEEE Tr<strong>an</strong>sactions on Information Theory, vol. 46, no. 2, pp. 388–404,Mar. 2000.[38] E. Borgia, M. Conti, F. Delmastro, <strong>an</strong>d L. Pelusi, “Lessons from <strong>an</strong> <strong>Ad</strong>-Hoc Network Test-Bed: Middleware <strong>an</strong>d Rout<strong>in</strong>g Issues,”<strong>in</strong> <strong>Ad</strong> Hoc & Sensor Wireless <strong>Networks</strong>, An International Journal, Vol.1, Numbres 1-2, 2005.[39] L. Ji <strong>an</strong>d M. Corson, “Explicit multicast<strong>in</strong>g for mobile ad <strong>hoc</strong> networks,” Mobile <strong>Networks</strong> <strong>an</strong>d Applications, vol. 8, pp. 525–549,2003.

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!