Cross-layer Optimization of Peer-to-Peer Video Streaming in ...

Cross-layer Optimization of
Peer-to-Peer Video Streaming in
OpenFlow-based ISP Networks
Diplomarbeit
Jeremias Blendin
PS-D-0003
Fachbereich Elektrotechnik
und Informationstechnik
Institut für Datentechnik
Fachgebiet P2P Systems Engineering
Prof. Dr. David Hausheer

Cross-layer Optimization of Peer-to-Peer Video Streaming in OpenFlow-based ISP Networks
Diplomarbeit
PS-D-0003
Eingereicht von Jeremias Blendin
Tag der Einreichung: 30. April 2013
Gutachter: Prof. Dr. David Hausheer
Zweitgutachter: Prof. Dr.-Ing. Ralf Steinmetz
Betreuer: M.Sc. Julius Rückert
Technische Universität Darmstadt
Fachbereich Elektrotechnik und Informationstechnik
Institut für Datentechnik
Fachgebiet P2P Systems Engineering
Prof. Dr. David Hausheer

Ehrenwörtliche Erklärung
Hiermit versichere ich, die vorliegende Diplomarbeit ohne Hilfe Dritter und nur mit den angegebenen
Quellen und Hilfsmitteln angefertigt zu haben. Alle Stellen, die aus den Quellen entnommen wurden,
sind als solche kenntlich gemacht worden. Diese Arbeit hat in dieser oder ähnlicher Form noch keiner
Prüfungsbehörde vorgelegen. Die schriftliche Fassung stimmt mit der elektronischen Fassung überein.
Darmstadt, den 30. April 2013
Jeremias Blendin
i

Contents
1. Introduction 1
1.1. Goal and Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2. Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Background 3
2.1. Live Video Streaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1.1. Multicast on the Internet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1.2. Peer-to-Peer Live Video Streaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2. Internet Service Providers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.1. Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.2. Network Topology and Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.3. ISPs and Peer-to-Peer Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3. Openflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3.1. The OpenFlow Standard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.2. OpenFlow Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3. Related Work 17
3.1. ISP and Peer-to-Peer Cooperation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.1.1. ISP Network Information Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.1.2. ISP-Owned Peers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2. Peer-to-Peer Live Video Streaming and Super Peers . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2.1. mTreebone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2.2. SALSA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3. Overlay Networks and Multicast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.4. Improvements to IP Multicast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.4.1. Recursive Unicast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.4.2. Explicit Multicast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.4.3. LIPSIN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.4.4. Labelcast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.5. OpenFlow and IP Multicast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.5.1. IP multicast with Fast Tree Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.5.2. CastFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4. Assumptions and Requirements 25
4.1. Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2. Requirements of ISPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2.1. Service Resource Consumption Requirements . . . . . . . . . . . . . . . . . . . . . . . 26
4.2.2. Service Integration and Management Requirements . . . . . . . . . . . . . . . . . . . 26
4.3. Requirements of Peer-to-Peer Live Video Streaming . . . . . . . . . . . . . . . . . . . . . . . . 27
4.3.1. Technical Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
iii

4.3.2. Requirements of Users, Operators and Application Providers . . . . . . . . . . . . . . 27
5. System Design 29
5.1. Description and Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
5.1.1. Functional Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
5.1.2. Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.2. Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.2.1. OpenFlow based Network Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
5.2.2. System Structure and Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.3. Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.3.1. Rent-a-Super Peer Controller Component . . . . . . . . . . . . . . . . . . . . . . . . . . 36
5.3.2. Virtual Peer Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.3.3. Software Defined Multicast Component . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.3.4. Explicit Multicast Subcomponent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.4. Peer-to-Peer System Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.4.1. General Considerations and Peer Behavior Adaptation . . . . . . . . . . . . . . . . . . 47
5.4.2. Super Peer Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
5.4.3. Rent-a-Super Peer Instance Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6. Prototype 49
6.1. Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
6.2. Network Emulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
6.2.1. Mininet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
6.2.2. OpenFlow Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
6.2.3. Topology Database and Network Experiments . . . . . . . . . . . . . . . . . . . . . . . 54
6.2.4. OpenFlow Routing and Forwarding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
6.3. The Ryu OpenFlow Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.3.1. Architecture and Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.3.2. Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
6.3.3. Ryu Application Blueprint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
6.4. Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
6.4.1. Rent-a-Super Peer Controller and Public API . . . . . . . . . . . . . . . . . . . . . . . . 60
6.4.2. Topology Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
6.4.3. Virtual Peer Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
6.4.4. Software-Defined Multicast Application . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
7. Evaluation 69
7.1. Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
7.1.1. ISP Core Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
7.1.2. ISP Access and Aggregation Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
7.1.3. Applied Network Topology Variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
7.1.4. Peer-to-Peer Live Video Streaming Model and Emulation . . . . . . . . . . . . . . . . 76
7.1.5. Overlay Topology Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
7.2. Transmission Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
7.2.1. Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
7.2.2. Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
iv
Contents

7.2.3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
7.3. System Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
7.3.1. Number of Required OpenFlow Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
7.3.2. OpenFlow Messaging Overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
7.3.3. API Management Overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
7.4. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
8. Conclusion and Future Work 95
8.1. Future Work on the RASP Implementation and Integration into P2P LVS Systems . . . . . . 96
8.2. Future Work on the Large Scale Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
8.3. Future Work on Using the RASP Service with other Applications . . . . . . . . . . . . . . . . 97
A. Evaluation Statistics 99
A.1. t-test Statistics of the Intra-ISP Traffic Volume Ratio Differences . . . . . . . . . . . . . . . . 99
A.2. t-test Statistics of the Cross-Border Traffic Volume Differences . . . . . . . . . . . . . . . . . . 100
Acronyms 103
Bibliography 104
Contents
v

List of Figures
2.1. A P2P LVS layer model (based on [67], p. 73). . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2. Tree and mesh topologies for video stream distribution in P2P LVS. Blue boxes denote
existing, white boxes missing parts of the video. . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3. Hierarchical topology of the Internet [58, p. 78]. . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4. ISP network topology by network types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.5. A traditional networking scheme and the OpenFlow networking scheme [see 72, p. 2]. . . 11
2.6. Operating schema of the OpenFlow version 1.0 flow table. . . . . . . . . . . . . . . . . . . . . 13
2.7. Operating schema of the OpenFlow version 1.1 packet-processing pipeline. . . . . . . . . . 14
3.1. The forwarding scheme of REUNITE (adapted from [104, p. 1645]). . . . . . . . . . . . . . 21
5.1. A simplified illustration of the operation of a P2P LVS system. . . . . . . . . . . . . . . . . . . 30
5.2. A simplified illustration of the operation of a P2P LVS system using the RASP service. . . . 31
5.3. OpenFlow network architecture (adapted from [82, p. 7]). . . . . . . . . . . . . . . . . . . . 34
5.4. Architecture of the RASP service. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.5. The names of entities involved with the RASP system. . . . . . . . . . . . . . . . . . . . . . . . 37
5.6. OpenFlow rules for implementing a virtual peer for a connection initiated by peer P to
super peer SP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.7. Comparison of an IP multicast and a SDM domain. . . . . . . . . . . . . . . . . . . . . . . . . 41
5.8. Schema of the SDM concept. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5.9. Schematic overview of an Explicit SDM domain and its subdomains. . . . . . . . . . . . . . . 46
6.1. The components of the RASP service that are part of the prototype. . . . . . . . . . . . . . . 49
6.2. Overview of the prototype implementation and the accompanying emulation system of
RASP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
6.3. Throughput of the three OpenFlow software switches as measured by iperf. . . . . . . . . . 53
6.4. The process schema of running an emulated network experiment. . . . . . . . . . . . . . . . 54
6.5. An example network based on the topology database information used for creating emulated
networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
6.6. Simplified diagram of an OpenFlow LAN switch in the emulated network. . . . . . . . . . . 56
6.7. The architecture of Ryu [77]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
6.8. UML class diagram of a typical Ryu application skeleton. . . . . . . . . . . . . . . . . . . . . . 59
6.9. Overview over the parts of the prototype whose implementations are described in Section
6.4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
6.10.Example of a super peer using the RASP API to set up a video stream and adding a member
to it. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
6.11.The structure of the RASP controller implementation. . . . . . . . . . . . . . . . . . . . . . . . 62
6.12.The structure of the topology application implementation. . . . . . . . . . . . . . . . . . . . . 63
6.13.The structure of the Virtual Peer application implementation. . . . . . . . . . . . . . . . . . . 63
6.14.The structure of the SDM application implementation. . . . . . . . . . . . . . . . . . . . . . . 64
vii

6.15.Multicast tree construction problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
6.16.Schematic cutout of a SDM multicast tree with three different forwarding states. . . . . . . 67
7.1. Deutsche Telekom’s core network [26, p. 4]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
7.2. The network topology model used as basis for network topology with annotations on the
scaling possibilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
7.3. Access and aggregation networks: Ethernet application and PPP encapsulation. . . . . . . . 72
7.4. Network model instance DT small. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
7.5. Network model instance scaled by increasing the number of connected host per access
switch (DT Hosts). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
7.6. Network model instance scaled by adding core nodes (DT Core). . . . . . . . . . . . . . . . . 76
7.7. Network model instance scaled by increasing the trees depths (DT Depths). . . . . . . . . . 77
7.8. Overlay network algorithm examples based on the DT Small topology. . . . . . . . . . . . . 79
7.9. The cumulative distribution functions of two characteristics of the three P2P video overlay
tree algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
7.10.The intra ISP data volume per peer for each topology and overlay network algorithm. . . . 84
7.11.The intra ISP data volume per peer for each topology and overlay network algorithm
normalized by the mean of the Random overlay algorithm for each topology. . . . . . . . . 85
7.12.The cross-border data volume for each topology and overlay network algorithm. . . . . . . 85
7.13.The link stretch for each topology and overlay. . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
7.14.The RASP system architecture with annotations on system cost related parts. . . . . . . . . 87
7.15.Schema of the SDM distribution tree and the relevant parts for assessing the number of
used OpenFlow rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
7.16.The number of OpenFlow flow modification messages that install new rules during the
network experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
7.17.The number of OpenFlow message required for adding peers to a RASP instance. . . . . . . 92
viii
List of Figures

List of Tables
2.1. The packet header match fields available in OpenFlow 1.0. . . . . . . . . . . . . . . . . . . . 13
5.1. Comparison of multicast concepts (based on [46, p. 158]). . . . . . . . . . . . . . . . . . . . 33
5.2. OpenFlow rule schema for marking packets addresses to a SDM multicast group on the
group’s marking switches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
5.3. OpenFlow rule schema for writing a member’s socket information to packets of a SDM
multicast group on the group’s member switches. . . . . . . . . . . . . . . . . . . . . . . . . . 44
5.4. OpenFlow rule schema for switch S4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5.5. OpenFlow rule schema for switch S9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
6.1. Throughput of the three OpenFlow software switches as measured by iperf. . . . . . . . . . 53
6.2. OpenFlow 1.0 flowtables for a LAN switch in the emulated network. . . . . . . . . . . . . . . 56
6.3. RASP controller service API overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
6.4. Topology application API overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
6.5. Virtual Peer application API overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6.6. SDM application API overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
6.7. OpenFlow matcher G match for packets of SDM group G. . . . . . . . . . . . . . . . . . . . . . . 67
6.8. OpenFlow rules during the three steps of the SDM tree forwarding example. . . . . . . . . 68
7.1. Overview of the methods and tools used for evaluation. . . . . . . . . . . . . . . . . . . . . . 82
7.2. Average number of OpenFlow rules per device class for the scenario used for evaluation. . 89
ix

1 Introduction
Internet video streaming causes the second largest transfer volume and is the second fastest growing
application class in Internet traffic analysis [58, p. 82]. In the American broadband access market,
streaming video is the source of 50% of the traffic volume and still expected to increase its volume
share [101]. Many large video streaming services use CDNs or closed multicast systems. An approach
that works without fixed infrastructure is Peer-to-Peer (P2P) video streaming. Using P2P systems enables
Internet wide use of video streaming, but it is a burden for Internet Service Providers (ISPs) due to its
hard to control, unpredictable, and unfavorable traffic patterns.
Software Defined Networking (SDN) and especially its OpenFlow variant is a network paradigm that
gains traction in the network market [1] and which is already used in production environments [40].
OpenFlow enables network domains and, more importantly, the forwarding hardware network switches
and routers to be controlled remotely by software. At the same time, it allows a logically centralized view
on the network state. These features are of immediate relevance for efficient and centralized network
management. Furthermore, new networking concepts can be applied that have been unfeasible before.
This work investigates how OpenFlow can be used to improve Peer-to-Peer Live Video Streaming (P2P
LVS) for users, operators and ISPs. The approach is based on cross-layer optimization, as OpenFlow
is a network layer technology and P2P LVS an application layer technology. It focuses on the network
domains of residential broadband access ISPs, whose customers are the main users of video streaming
applications.
The concept presented in this work, Rent-a-Super Peer (RASP), is a service that enables users of P2P
LVS systems to increase their network resources within a ISP network, thereby providing them super peer
like capabilities. The service is offered by residential broadband ISPs for users outside their networks to
distribute their video streams within the ISP’s network in an efficient way. This concept allows using
the system on-demand. The generic cross-layer system enables P2P LVS to increase its number of super
peers, which leads to improved system performance. At the same time, it improves the ISPs’ role in P2P
LVS by increasing their control on P2P LVS traffic and making its transmission more efficient and, thus,
reducing the generated traffic volume.
The approach of the service and its design is presented and discussed in this work. A prototypical
implementation is described and its conceptual characteristics evaluated. OpenFlow is a new technology
and its usage as well as its application to specific application domains is not extensively documented yet.
Hence, this work should give an insight on how an OpenFlow application can be designed and evaluated.
For designing the service, it is assumed that, in the future, OpenFlow will be adopted in most networking
domains, including ISPs.
1.1 Goal and Contribution
The goals of the RASP service are threefold: improving P2P LVS systems as well as increasing the transmission
efficiency and the control of P2P LVS traffic in ISP networks.
The service aims to improve P2P LVS systems by creating an on-demand service in ISP networks that
provides super peer like network performance to normal peers. The services works for peers outside the
ISP network by providing network resources for other peers inside the network. In addition to that, the
1

service provides them with a network layer proxy presence within the ISP network. The transmission
efficiency is increased by using a network layer multicast mechanism for distributing the video stream. It
uses unicast IP addresses and its usage is transparent for its users outside the ISP’s OpenFlow network.
This approach in combination with the network layer proxy enables the ISP to have full control of the
service and its network traffic.
The contribution of this work is the design of network layer service for enabling generic, on-demand
ISP-owned Peers [89]. It is based OpenFlow and is aimed at exploiting the performance and efficiency of
OpenFlow hardware. The RASP service provides the network resources, while the P2P LVS users provide
the peer’s computing resources and operation. The respective applications providers add the integration
of the service into P2P LVS systems. Due to its generic approach, the service can be integrated into a wide
range of P2P LVS systems. This is possible because it does not require peers to support special network
protocols such as IP multicast.
1.2 Outline
An overview of P2P LVS systems, ISPs, their relationship, and OpenFlow is given in Chapter 2. The state
of the art on improvements of P2P LVS systems, their cooperation with ISPs, and approaches to integrate
network layer multicast in P2P LVS systems is described in Chapter 3. The requirements for OpenFlowbased
services offered by ISPs to P2P LVS systems are discussed in Chapter 4. The system design of
RASP is described in Chapter 5, its implementation for evaluation in Chapter 6. The system is evaluated
in Chapter 7. Its results are summed up in Chapter 8, finally and an outlook on the next steps in the
research is given.
2 1. Introduction

2 Background
In this section, the relevant background on the research areas of P2P LVS, ISPs and OpenFlow is given.
The historical development of live video streaming on the Internet is described in Section 2.1, beginning
from IP multicast to today’s P2P LVS systems. An introduction into the structure of the Internet and the
role of ISPs is given in Section 2.2. Furthermore, their relationship with P2P LVS systems is described.
Lastly, the SDN variant OpenFlow is introduced and relevant features described in Section 2.3.
2.1 Live Video Streaming
Live video streaming is only one of three forms of Internet video application types, the others are video
file sharing, and video on demand. Video file sharing works similar to sharing of other files; the videos
are downloaded and watched some time later. When using video on demand, the videos are played
back while still being downloaded. In addition to that, live video streams are watched synchronously by
all clients. The synchronicity of the playback of live video streaming is defined by application specific
deadlines. If the deadlines are met the playback is considered synchronous [119, pp. 3548,3589]. The
two major applications of live video streaming are video conferencing and Internet Protocol Television
(IPTV). Typical differences in the usage characteristics between video conferencing and IPTV are the
delay deadline and the number of users. In video conferencing, the number of participating users is
small as is the delay deadline. For IPTV the delay deadline is higher, and the number of users is usually
much larger [65, p. 14].
In live video streaming, a source sends a video stream to multiple of consumers. The natural way
of handling this communication is multicast. The concept and application of multicast is described in
Section 2.1.1, its application for streaming live video is described in Section 2.1.2
2.1.1 Multicast on the Internet
The different approaches on implementing multicast services are presented in this section. Multicast
communication is a service for sending one message to multiple recipients. Without such a service, the
sender has to send the message to each recipient separately. The sender and its recipients in multicast
services constitute a multicast group; the recipients are also called members in this context. Computer
systems connected to the network (hosts) can join and leave groups. Members of a group receive its
messages and–depending on the system–can send messages to the group. However, in this work only
multicast services with one sender per group are discussed. Components required by all types of multicast
services are functionality for creating, joining and leaving groups. Further required functionalities are
routing and signaling of the multicast group information as well as an addressing scheme for identifying
multicast packets and their destinations.
A multicast service can be implemented on different levels of the ISO/OSI network layer reference
model. In the next paragraphs, a short overview over the historical development of multicast services is
given.
3

IP Multicast
The implementation of multicast services at the network layer is very efficient. This concept, IP multicast,
has been a research topic since 1990 [22]. The goal of IP multicast is to create facilities for Internet
wide multicast communications. The fundamental mechanism of all IP multicast services is the distribution
tree. The members of a multicast group are connected by a tree spanning, which consists of IP
multicast enabled routers. Multicast Packets are forwarded along the tree and duplicated at joints of the
tree.
IP multicast consists of a number of components that are implemented using different network protocols.
Multicast groups are identified and addressed by IP addresses from a special range [21]. Group management,
joining and leaving is enabled by a network layer group management protocol (e.g. IGMPv3
[15]) that is used between hosts and routers. To join a group, a host sends an IGMP message which is
forwarded through the network towards the multicast management router. The management router adds
the hosts to the member list, and adjusts the multicast tree accordingly. The setup and maintenance of
the routes that constitute the multicast tree is provided by IP multicast routing protocols, such as PIM-SM
[28]. Forwarding of IP multicast packets is done by the normal method of packet forwarding but with
special IP addresses. The protocols described up to this point are for usage within a network domain.
Interconnecting multicast domains is enabled by inter-domain multicast routing protocols (e.g. MSDP
[27]). A wide range of IP multicast protocols exists, the ones mentioned here are prominent examples.
Today, IP multicast is not deployed on a global basis. Only isolated IP multicast domains, called multicast
islands, exist. For example, some ISPs use IP multicast to deliver services like IPTV within their
networks as for example described in [66]. The reasons for the failure of IP multicast to be deployed
globally on the Internet are discussed in [23]. Although the paper is more than 10 years old, many
arguments still hold. From a technical perspective, the deployment, management, and operations of IP
multicast are complex and thus expensive. One problem that hinders the global deployment of IP multicast
is the necessity to store the state of multicast groups in the network. IP multicast enabled routers
have to store information on the multicast distribution tree on each group they are part of. This characteristic
is the reason for the unfavorable scaling characteristics of IP multicast, which is a cause for the
failure of Internet-wide IP multicast. Today the scalability aspect of IP multicast is still an active research
topic [92]. Admission control and security features are not part of most IP multicast protocols. Hence,
ISPs have no control over IP multicast communications, its users and usage, while its implementation
is expensive. Hence, there are no incentives for the major network providers to deploy Internet wide IP
multicast services for public consumption.
Overlay Multicast and Application Level Multicast
Application layer multicast and overlay multicast were developed in response to the slow adaption
of IP Multicast. Both approaches construct a virtual application layer network. “The main differences
among [...] IP multicast, application layer multicast, and overlay multicast, depend on whether the group
management and data delivery control are implemented in network routers, and hosts, or intermediate
proxy nodes.” [113, 526] In application layer multicast, hosts replicate the packets and manage the group
features, which is approach similar to P2P. The approach of overlay multicast is to connect IP multicast
domains using specialized hosts. They act as multicast proxies, connecting the multicast islands in a
P2P fashion. IP multicast and overlay multicast rely on the IP multicast protocol, while application layer
multicast uses IP unicast only and therefore does not rely specialized routers.
4 2. Background

2.1.2 Peer-to-Peer Live Video Streaming
While IP multicast is the most efficient method to implement one to many communications, it is not
widely deployed. Hence, alternatives such as application layer multicast are researched and used. One
form of application layer multicast are P2P LVS systems. The concept of such systems is described in this
paragraph. In the following paragraph, an insight into current research on P2P LVS is given.
Applications are called P2P systems because they do net rely on servers to communicate. Instead,
the application is run by the participating clients, the peers. Their communication paths form a virtual
network on top of the underlying one, called overlay network. The underlying network is called underlay
network in this context.
P2P systems are distributed systems in which participating entities (peers) share resources with the
goal of running applications mutually. Each peer acts as client and server, the application is consumed
and provided by peers at the same time. Different P2P systems have been developed to offer a range
of applications such as file sharing, content distribution, and application-level multicast [67, p. 72].
The topology of the overlay network can be arbitrary and is not bound to the topology of the underlay
network. However, the overlay networks performance is bounded by the resources and qualities of the
underlay network.
P2P LVS System
Live video streaming
Service: Video distribution
Overlay network
Network communication
Underlay network
Figure 2.1.: A P2P LVS layer model (based on [67], p. 73).
P2P LVS is an application of P2P systems for distributing live video streams. A layer model for such
applications is depicted in Figure 2.1. The system is built on the underlay network, which is used for creating
overlay network connections to other peers. The system uses them for management, maintenance,
and message transport. A video distribution service is built on top of the overlay network. It defines
the topology and the behavior of peers during video distribution. Finally, the live video streaming layer
creates a video stream for the consumption of the user. The general approach to live video streaming
in P2P systems is to split into small pieces, junks, and distribute them using the overlay network. The
methods for distributing the video is either tree or mesh-based. In tree-based systems, the video distribution
topology forms a tree as shown in Figure 2.2a. Tree-based systems are application layer multicast
systems. For each stream an arborescence is created, in which each peer immediately forwards the video
data it receives along the arcs. The root of the arborescence is the source of the video, denoted by node A
in the figure. The video parts 1,2, and 3 are available, as depicted by the blue boxes. These parts are
consecutively sent to the downstream nodes B and C. Node C itself has two downstream nodes, to which
it sends each part immediately after it is received. Nodes B, D, and E are leaf nodes, they receive the
video stream but do not take part in its distribution.
Mesh-based systems work similar to common file sharing systems. As shown in Figure 2.2b, each
peer maintains a list of currently relevant parts of the video. Neighbors exchange information on the
2.1. Live Video Streaming 5

parts they already received and on those they need. Based on this information the missing parts are
exchanged, initiated by either the receiver or sender of a part. In contrast to the tree topology there is no
predefined video dissemination path and no predefined sender and receiver roles. Neighbors are treated
individually, while in tree systems all downstream neighbors of a node get the next part as soon as it is
available [119, p. 3552,3553].
1 2
3
A
1
2 3 4
2
2
A
2
1 2
3
B
C
1 2
3
1
2 3 4
B
3
C
1
2 3 4
1
1
3
2
1 2
3
D
E
1 2
3
1
2 3 4
D
1
E
1
2 3 4
(a) Tree topology
(b) Mesh topology
Figure 2.2.: Tree and mesh topologies for video stream distribution in P2P LVS. Blue boxes denote existing, white
boxes missing parts of the video.
The topology of P2P LVS systems influences the performance characteristics of the system. Tree-based
systems have the advantage of achieving much smaller playback delays than mesh-based systems [119,
p. 3559]. Their transmission efficiency is higher, because the distribution tree is fixed. For mesh-based
systems the distribution of the stream is negotiated between neighbors for every junk, hence the management
overhead is higher. The disadvantage of tree-based systems is their susceptibility to peers leaving
the system unexpectedly. If a node fails, its complete tree of downstream node loses its video stream
source. If a node in a level near the video source fails, such an event affects a large number of nodes.
Multiple trees are used in some systems to mitigate this characteristic of tree topologies. The video is
split in several sub streams at the video source and each sub stream is distributed through a different
tree. The sub streams are recombined to a complete video stream before playback. Mesh-based systems
have the advantage of being more resilient to peers leaving and entering the system. Due to this tradeoff,
hybrid systems are developed which include either both topology approaches for different parts of the
system or allow the dynamic switching for adaptation to different situations [4, p. 183,184].
P2P LVS issues
A number of issues existing for P2P LVS systems. An overview over current research topics is given
in [4]. The issues can be categorized in three groups: underlay issues, overlay issues and application
layer issues. Underlay issues comprise quality issues of network connections such as packet loss, high
latency or failing links. Also, insufficient network resources influence the systems’ performance. One
example are peers that are connected to the Internet by residential broadband Internet connections.
These connections often exhibit asymmetric transmission capacities for upload and download, which
restrict the number of downstream nodes a peer can forward the video stream to. In addition, the costs
of the data volume might be an issue, for examples for broadband users with data volume limits in their
contract.
6 2. Background

The upload capacity of peers plays an important role in both topologies. In trees it constraints out
degree of the participating nodes, which in turn influences the trees depths and the playback delay [4,
p. 181]. In mesh systems, uplink capacities of a peers neighbor influences the number of them needed
to receive all parts in time. In sum, the upload capacity determines how many neighbors a peer can
distribute a video stream to. Also, studies show that the ratio of peers with low uplink capacities to peers
with high uplink capacities influences the performance of the whole P2P LVS system [57],[64, p. 349].
Overlay issues concern the P2P network itself. Problems include bad overlay connections, unresponsive
peers or improper overlay topologies. The unpredictable joining and leaving of peers in the system is
called churn. It is of relevance in P2P systems, as the peers are part of the overlay networks infrastructure.
Peer churn can cause instability in the system or be the reason for packet loss in the video stream in case
of P2P LVS systems. Another problem is a high rate of new peers, which can overwhelm the system’s
ability to integrate new peers; this phenomenon is also called flash crowd. Similar problems arise during
high rates of peers leaving the system.
Lastly application layer problems could arise, especially if the ISP of the user tries to mitigate the
negative influence of P2P systems on the network. This could lead to throttling or blocking of P2P
overlay connections, obstructing the user to use the overlay network.
Approaches for mitigating these issues are an active research topic. Besides optimizing the P2P LVS systems,
mechanisms for the cooperation of different layers, called cross-layer optimization, are researched.
One is underlay awareness, which is the concept of considering the characteristics of the underlay network
and adapting the overlay network accordingly.
2.2 Internet Service Providers
ISPs play an important role in the Internet architecture. This work focuses on residential broadband
Internet access. Other means of connecting to the Internet like cellular networks are not discussed, as
P2P systems are discussed in the context of fixed Internet connections only.
In Section 2.2.1 an overview is given over the entities involved in the Internet architecture and the
role and classification of ISPs. The network characteristics of broadband access ISPs are described in
Section 2.2.2. The issue of P2P systems and their relation to ISPs is discussed in Section 2.2.3.
2.2.1 Classification
The Internet is made up of autonomous systems, administrative domains that are operated by entities
that take the role of network operators. This section gives an overview of different types of network
operators with a special focus on ISPs. ISPs are network operators whose purpose is to provide network
connectivity to other entities. The networks differ in geographical extent, network degree, data volume,
and place in the topology of the Internet. As shown in Figure 2.3, the networks can be classified into
a hierarchy. On the bottom of the hierarchy are networks that are completely managed by a single
entity and do net sell their connectivity (customer networks). They are geographically small, placed
at the edge of the Internet topology, exchange small data volumes and are mostly connected to only on
other network, their ISP. Examples for this class are home, company, and educational networks as well as
small hosting companies. The next class are regional or tier2 ISPs, that offer transit connectivity for other
networks, and large hosting and content companies. They might span a larger area, and are connected to
a larger number of other networks, and exchange larger data volumes. Examples for this class are small
ISPs and midsized hosting companies. The top class of the hierarchy is called the global Internet core.
2.2. Internet Service Providers 7

It consists of worldwide operating transit ISPs (tier1), large content and hosting companies, and large
consumer access providers. They have a high network degree and high transit data volumes. Examples
for this class are tier1 ISPs like Verizon and Deutsche Telekom, large hosting and content companies
like Google and content distribution companies like Akamai [58, pp. 78-80]. Internet eXchange Points
(IXPs) are located between the top class and the middle class in the hierarchy. They are small networks
(consisting only of a few hundred nodes), span a small area but their transit traffic volume and network
degree [7] are huge.
Global Internet
Core
Global Transit &
National
Backbone ISPs
"Hyper Giants"
Large Content Provider Networks,
Consumer Access Provider ,
Content Distribution Networks
IXP
IXP
IXP
Regional/Tier 2
ISPs
ISP 1 ISP 2
Customer IP
Networks
Figure 2.3.: Hierarchical topology of the Internet [58, p. 78].
2.2.2 Network Topology and Characteristics
Most entities from the global Internet core have no direct business contacts with end customers. Large
content and transit providers sell their services to other ISPs. The same is true for IXPs, content providers
like Google and content distribution providers like Akamai. Often, IXPs operate at layer 2 only [7, p.
165] and are not involved with the Internet protocol or services provided by upper layers of the ISO/OSI
reference model because of that. The only exceptions are large consumer access providers, for example
Comcast. The approach described in this work is applicable for ISP with end costumers, because they are
the ones interested in P2P LVS services. Only a few consumer access providers are “Hyper Giants”, hence
can be seen as a special case.
Providing a service for P2P LVS systems and users implies a business relationship with residential
broadband access users. P2P LVS are alternatives to Content Delivery Networks (CDNs), hence the
content providers are not the target group. This leaves tier2 ISPs as the main audience for P2P LVS
optimization services. Consumer ISPs share certain network characteristics. A typical structure of a residential
customer ISP is shown in Figure 2.4.
It consists of three types of networks: access networks, aggregation networks and one core network.
Access networks consist of devices needed for the network links to the customers. Aggregation networks
aggregate the customer connections and connect them to the backbone. Both access and aggregation
networks are either local or regional networks of which a number exists. There is one core network,
which enables connectivity between the aggregation networks, the Internet and other parts of the ISP
[33, p. 1][36, p. 1284]. The system is hierarchical; a tree is formed with the core network as root node,
the access networks as leaves and aggregation networks as inner nodes.
8 2. Background

Internet
ISP B
ISP
ISP C
Access
Core
Aggregation
Access
Access
Aggregation
Access
Access
Access
Broadband Customer Networks
Figure 2.4.: ISP network topology by network types.
Global routing on the Internet is provided by network connections to neighboring networks. The connectivity
is redundant for resilience and performance. Each network operator applies its own policies for
routing, with which the forwarding of its own and transit traffic can be influenced. This is not true for
incoming traffic however; it can enter the ISP network on any link that is connected to another network.
Predictions on where traffic from other networks enters the ISP network are hence not possible. Applying
special treatment for traffic from certain networks is only possible when it is enabled on all incoming
links.
In the core network, IP and often MPLS are used for forwarding. It is optimized for high throughput
and efficient forwarding, not for service delivery or its implementation. In the aggregation and access
networks, the network protocol used for forwarding depends on the access technology. When offering Internet
connectivity to consumers, the network access technology used has a big influence on the network.
Network access technology groups are Digital Subscriber Line (DSL), cable and Fiber-to-the-x (FTTX).
DSL uses existing telephone wires as medium, cable works on existing TV cables while FTTX relies on
fiber cables to or near the access location. The currently prevalent residential network access technology
is DSL both worldwide and in Western Europe. The worldwide and western European market share of
DSL is about 60%. The market shares of cable are similar for both areas at about 20%. The market share
of FTTX shows a difference, it is globally at about 20% and in western Europe below 10%. The remaining
market share represents other connection technologies that are not listed separately [105, 93].
2.2. Internet Service Providers 9

2.2.3 ISPs and Peer-to-Peer Systems
P2P traffic makes up a considerable part of the traffic volumes [69, p. 90][58, p. 82]. Hence, it affects the
ISPs and their networks, and here especially residential broadband access providers. This is because the
users of P2P systems are predominantly residential broadband users. Most of the data volume is caused
by P2P file sharing.
The effects of P2P systems on ISPs are an active research topic. While many issues arise from the usage
of P2P file sharing system, many aspects are valid for P2P LVS systems too, even when they are used at a
smaller scale. A good overview of the research is given in [25] and [20]. P2P systems cause considerable
network traffic, while bringing no additional profit for the ISPs. Many overlay systems are unaware of
the underlying network topology leading to inefficient network usage. This could result in increased
usage of expensive links. The amount of data volume leaving the network could also increase. This is an
issue, because of the billing structure between network operators on the Internet. While transit and tier1
providers and large networks often have different approaches for settling their peering, smaller tier2
providers have to pay for the data volume transferred. In addition, controlling P2P traffic is difficult, as
it is often encrypted. Unpredictable usage pattern and traffic flows in the overlay network make it more
difficult for ISPs to manage their networks and to ensure service quality for all users.
A large body of approaches on how to improve this situation have been developed, good overviews
are given in [20] or [25]. Unilateral remedies deployed today by ISP are throttling and blocking of P2P
network traffic. However, due to the market competition, and legal issues such as net neutrality, research
suggests other solutions. One approach that can be used by P2P systems is implementing underlay awareness
as described in Section 2.1.2. Even greater improvements are expected from cooperation between
ISPs and P2P systems. Examples are topology information services provided by ISPs that are used by P2P
systems to improve their underlay awareness.
2.3 Openflow
OpenFlow is a variant of the SDN approach. Unlike earlier variants, it is supported by hardware vendors
and dedicated high performance OpenFlow hardware is available. The technology is already deployed in
the industry [40], and is popular in the research community.
In traditional networks, the forwarding elements, such as switches or routers, consist of specialized
hardware for packet forwarding, the datapath, and a general purpose CPU that runs management software.
The management software configures the datapath; both, the software and the hardware are
vendor specific and not standardized. The forwarding elements exchange relevant information using
feature specific, standards based network protocols as shown in Figure 2.5a. The management software
of such a network represents a distributed system. This has the advantage of high robustness against
failures of single forwarding elements. However, as no central entity exists, it is managed by changing
settings in the configuration of the management software of single forwarding elements. Due to the distributed
network state, configuration or network changes take time to disseminate through the network
[90, p. 8]. It is generally not possible for researchers or users to change the behavior of forwarding
elements as the vendors do not disclose details of their implementation. Hence, often routers based on
PCs are used, for which free open source routing software exists. However, low performance makes software
routers unsuitable for large experiments or production deployments. To change the behavior of the
network, the modification of the network protocols used is necessary. This is difficult, since for changed
network protocols the forwarding element’s software needs to be changed. Hence, standards need to be
10 2. Background

changed, which might in turn be implemented by the vendors. This situation slows down the research of
networks as well as the introduction of new networking features [71, p. 12].
Feature Specific,
Standards Based
Network Protocols
OpenFlow
Normal Secure
Software Channel
Normal Flow
Datapath Table
OpenFlow
Normal Secure
Software Channel
Normal Flow
Datapath Table
OpenFlow
Normal Secure
Software Channel
Normal Flow
Datapath Table
(a) Traditional networking scheme.
OpenFlow
Controller
OpenFlow
Switch
Secure
Channel
Flow
Table
OpenFlow
Protocol
OpenFlow
Switch
Secure
Channel
Flow
Table
OpenFlow
Switch
Secure
Channel
Flow
Table
(b) OpenFlow networking scheme.
Figure 2.5.: A traditional networking scheme and the OpenFlow networking scheme [see 72, p. 2].
OpenFlow [72] introduces a concept for managing computer networks that aims to change this situation.
OpenFlow is a standard that defines the configuration of datapaths and a protocol for the remote
configuration of them: the OpenFlow protocol. This allows moving the management of the forwarding
device to a central software system, the OpenFlow controller, as shown in Figure 2.5b. An OpenFlow
network is still a distributed system, but centralized management is possible. OpenFlow forwarding elements
are called OpenFlow switches, the datapath of an OpenFlow switch is called flow table. OpenFlow
datapaths can be implemented in hardware, which allows the configuration of high performance forwarding
hardware with an open standard protocol. The scopes of OpenFlow are Ethernet and TCP/IP
based networks, specifically the OSI/ISO network layers 2-4. The OpenFlow controller is software that
runs on a standard PC. It is connected to the OpenFlow switches, handles incoming information from
them and sends configuration messages.
The general packet-forwarding scheme of OpenFlow is based on rules stored in the flow table. Each
rule consists of a matcher and actions. The matcher consists of values for packet header fields of Layers
2-4 of the TCP/IP network stack, that are matched against the values of arriving packets. If all values
match, the list of associated actions is applied. Actions include the changing of header field values, send-
2.3. Openflow 11

ing packets out a network interface, dropping packets and sending incoming packets to the OpenFlow
controller.
In Section 2.3.1 an overview is given on OpenFlow, its implementations and the versions of the Open-
Flow standard. The latter are described in more detail in Section 2.3.1.
2.3.1 The OpenFlow Standard
The OpenFlow standards, named “OpenFlow Switch Specification”, define a network protocol and associated
roles for communication between an OpenFlow controller and an OpenFlow switch. The purpose
of the OpenFlow protocol is to enable the remote operation of the OpenFlow switch from the OpenFlow
controller. The main responsibilities are the configuration of the flow table on OpenFlow switches and
handling associated events. These include packets that are forwarded from the OpenFlow switch to the
controller for inspection and flow table related events like rules that are remove due to timeouts. The
protocol also enables the controller to configure and gather statistical data on the flow table. Finally,
OpenFlow switches send events on their status to the controller; examples are a change of a network
interface status. The configuration of the switch hardware is not part of the standard, however an accompanying
standard, the “OpenFlow Management and configuration protocol” (OF-Config) [83]. Its
responsibility includes the configuration of the network ports and accompanying queues as well as configuring
the switches flow tables and their controller connections. The OF-Config protocol is beyond the
scope of this work.
The OpenFlow standard describes a network protocol and by defining the semantics of the flow table
configuration messages, the behavior of an OpenFlow switch. An overview over the versions of the
OpenFlow standard and their port prominent features is given, then relevant versions are described in
detail in the remainder of this section. The first OpenFlow standard for public use, version 1.0, was
released in 2009 [78]. It contains a simple flow table model, which also illustrates basic OpenFlow
concepts. A number of updates of the OpenFlow were released. Version 1.1 was published in 2011 [80]
and includes, among other changes, a fundamental change of the flow table behavior. All OpenFlow
versions release after 1.1 introduces incremental changes, but no major change as from version 1.0 to 1.1.
OpenFlow version 1.2 [79] – published in 2011 – introduces a new approach for packet matching. Two
versions of the standard were released in 2012, version 1.3 [81] and 1.3.1 [84]. OpenFlow version 1.3
adds a number of supported network protocols, improves the connection to the controller and introduces
interoperability to the OF-Config protocol. The latest version of the standard, version 1.3.1, contains
predominantly corrections and small improvements. The last two standards will not be discussed in
details, as currently no complete implementations are available, neither for the controller nor for the
switch.
OpenFlow 1.0
The process of packet handling in OpenFlow version 1.0 [78] starts with an incoming packet. It is
matched against the rules in the flow table; the action list associated with the first matching rule is
applied to the packet. The rules in the flow table are ordered by their priority value. Figure 2.6 shows
the operating schema of the flow table in OpenFlow version 1.0.
For matching, a set of packet header fields and the ingress port of the packets can be used as shown
in Table 2.1. Matching rules consist of values for one or more of the fields, which are matched against
the values of incoming packets. Fields that should not be matched can be marked as wildcard value, in
this case all values match. For all fields except the IP addresses, matching means that all bits of the given
value and the packet data are equal. In case of IP addresses, a subnet mask may be set, in which case
12 2. Background

Incoming packet
Matcher
Matcher
Matcher
Flow Table
.
.
.
.
.
.
Action List
Find matching rule
Action List
Action List
Apply actions
Figure 2.6.: Operating schema of the OpenFlow version 1.0 flow table.
only a part of the IP addresses bits are used for matching. The matching of IP address subnet masks is an
optional feature, not every OpenFlow is required to support it. The same is true for a number of actions.
Due to the conceptual and explorative nature of this work, from here on, no distinction will be made
between required and optional features. Instead, it is assumed that all features they are described in the
standards are available for use.
Ingress Port
Packet Match Field
Ethernet source address
Ethernet destination address
Ether type
VLAN id
VLAN priority
IP source address
IP destination address
IP transport protocol ID
IP ToS bits
TCP/UDP source port or ICMP type
TCP/UDP destination port or ICMP code
Table 2.1.: The packet header match fields available in OpenFlow 1.0.
If a rule matches, the associated actions are applied. Available actions include the changing of the
values of the packet header fields that can be used for matching, except for ICMP code and type. In
addition to that, a packet can be forwarded to a single port or all ports, to the controller or it can be
dropped. The actions given in the list are applied in the order specified. An action list may include the
repeated writing of header fields and multiple forward actions. If no rules match the packet, it is send
to the controller for further inspection. For all ports, rules and tables, counters are kept to keep statistics
on the packets processed.
An important aspect of the OpenFlow standard is the interaction of OpenFlow switches with the controller.
As described, incoming packets are sent to the controller in case no rules match or the matching
rules include a corresponding forwarding action. There is no required behavior for the controller soft-
2.3. Openflow 13

ware in this case defined in the standard. For certain applications however, the controller can use the
packet information as basis for installing new rules in the flow table. In addition to that, the controller
may also send packets to an OpenFlow switch. They are associated with an action list that is applied on
the switch. This list may include all actions described above and it is possible to handle the packet by the
flow table. This facility enables a controller to send packets through connected OpenFlow switches.
The main responsibility of the OpenFlow controller is the management of the flow table rules. Methods
for adding, modifying and deleting rules are available. For management, flow table rules are identified
either by their exact matcher or by the notion of matching equivalency. In the first case, a rule management
command applies to the rule that exactly matches the management commands matcher. In the
latter case, all rules that are matched by the commands matcher are affected. Hence, a delete command
whose matcher contains wildcards for all fields would remove all rules from the flow table.
Rules installed can be given timeout values, a soft and a hard time. A rule with a hard timeout value is
removed after the given time. A rule with a soft timeout value is removed if there is no matching packet
for the given time. In both cases, a message is send to the controller containing information about which
rule was removed due to what reason.
OpenFlow 1.1
OpenFlow version 1.1 introduces a new packet-processing scheme. Instead of the single matcher that
causes the application of an action list, a pipelined more flexible model is used. The packet processing
consists of multiple tables, each with a different set of rules. Packets traversing the table pipeline are
associated with a metadata field and an action set, which is executed at the end of the processing and
which can be changed by each rule. The action set is kept for each packet during the processing pipeline.
Each available action can be included in the set once; the execution order of the set is predefined by the
action types. The packet-processing model of OpenFlow 1.1 is shown in Figure 2.7.
Incoming packets are processed in flow table 0, similar to the method used in OpenFlow 1.0. However,
each rule has an associated instruction set instead of an action list. The instruction set may contain
one rule of each instruction type. Instructions exist for modification of the action set. In addition, an
instruction allows the immediate application of an action list, similar to the OpenFlow 1.0 flow table
operation. As shown in Figure 2.7, at the end of the processing pipeline each packets action set is applied.
P P action set
P
action set
Incoming packet
Flow
Table 0
Flow
Table 1
...
Flow
Table n
P
action set
Apply instruction
set
Apply instruction
set
Apply action
set
Figure 2.7.: Operating schema of the OpenFlow version 1.1 packet-processing pipeline.
The pipelined packet processing allows the reuse and combination of rules. In OpenFlow 1.0 each
packet is matched by exactly one matcher, which is associated with exactly one predefined action list.
Pipelined processing changes this situation, as each packet can be matched by a number of rules. For example,
a firewall implementation in OpenFlow is considered. First, each packet is checked if the security
policy allows it to be forwarded. If this is the case, the output port is set and the packet is forwarded.
14 2. Background

With OpenFlow 1.0, for each combination of m security policy entries and n next hops a rule has to be
created. With OpenFlow 1.1, the security policies and forwarding rules can be implemented using two
different tables. Using OpenFlow 1.0 n ∗ m rules need to be created; OpenFlow 1.1 only requires n + m
rules.
The table layout can be defined by the controller. For each rule, the next table id to continue processing
can be set using a corresponding instruction. Furthermore, for each table, the behavior in case no rules
matches can be defined. This includes the possibility to continue the processing at another table. One
restriction is that the next table id for processing has to be larger than the old one.
In addition to pipelined processing, OpenFlow version 1.1 introduces groups. Groups are associated
with a type and a list of actions buckets, which in turn contain action sets. The group type all replicates
the packet for each of the action buckets. After application, pipelined processing is continued for all
packets. Groups are referenced by their id and can be utilized by applying a corresponding action to
packets. An application example for the group type all is multicast. For each multicast group a group
entry is created, for each next hop of a group an action bucket is added. Each action bucket then contains
an output action for the corresponding interface where the next hop is connected.
OpenFlow 1.2
The main change in OpenFlow 1.2 is extensible match support. Instead of using a predefined set of
matchers, extensible matchers introduce a data structure for describing match fields. For selecting which
fields should match, the notion of wildcards – as used in OpenFlow 1.0 – is abandoned. Instead, for
each field its extensible match description and comparison value is included. The description includes
the field that should be matched and a bitmask that is applied for matching. This approach enables the
application of arbitrary bitmasks for matching of most fields. This approach enables e.g. the matching of
a single bit of a packets IP address.
In addition, OpenFlow version 1.2 introduces support for IPv6 and MPLS and the VLAN support is
improved.
2.3.2 OpenFlow Switches
OpenFlow has attracted a large number of companies and organizations [1]. A wide variety of OpenFlow
controller software, OpenFlow switch software and hardware is available, both by commercial vendors
and by open source projects. While OpenFlow software switches are convenient for testing and some
usage scenarios, the main advantage of OpenFlow are hardware switches. The most prominent feature
of OpenFlow hardware is the matching hardware, of which the most efficient implementation is based on
Content-addressable Memory (CAM). CAM allows the lookup of bit field matches in one clock cycle [88,
p. 712]. Two types of CAM exists, one that implements exact bit matches and one that allows wildcard
bits. CAM is already used in today’s network hardware, the OpenFlow protocol enables its usage based
on an open standard. However, CAM has a high power consumption, its size is restricted [88, p. 1] and
is expensive [68, p. 81].
As each OpenFlow matcher requires a certain amount of CAM, the amount of CAM becomes a restriction.
Therefore, one restriction of OpenFlow hardware switches is the number of rules the devices
can host. The maximum number of supported flow entries for hardware switches could not be found
in literature, however a vendor claims for a contemporary OpenFlow switch to support up to 160,000
flows [74]. Information on older hardware shows much smaller numbers, for example between 512
and 1024 entries in [75, p. 30]. Like most controllers and switches available at the time of writing, it
2.3. Openflow 15

supports OpenFlow 1.0 only. Information on the numbers of supported OpenFlow 1.1 rules is not available
yet. The OpenFlow features that are implemented in hardware vary between devices. Furthermore,
the number of supported rules is different for each feature [86]. Hence, the hardware restrictions of
contemporary OpenFlow switch hardware are not obvious; they require an analysis for each device.
The sizes and prices of available CAM are analyzed in [68]. Commercial available devices include
CAM chips with 512,000 40-bit entries for matching. Given that each OpenFlow 1.1 match field can be
implemented in one entry and consumes at maximum 128bit for an IPv6 address, devices with 100,000
flow entries seem feasible. This number is supported by the vendor reference mentioned earlier in this
section on devices with a 160,000 OpenFlow rule capacity.
16 2. Background

3 Related Work
No relevant approaches exist that aim to provide a similar service that is local to an ISP. The goal of
most works is to improve and analyze P2P LVS system at a global level. However, due to its cross-layer
approach, which aims to be useful for ISPs and P2P LVS system operators as well as users, the RASP
service is related to a wide range of research topics.
Research on ISP and P2P cooperation is presented in Section 3.1. Improvements for P2P systems by
integrating IP multicast islands and using super peers are described in Section 3.3 and Section 3.2. Approaches
on improving IP multicast are introduced in Section 3.4, OpenFlow-based multicast approaches
in Section 3.5.
3.1 ISP and Peer-to-Peer Cooperation
The approach in this work is using cross-layer optimization for P2P systems and ISPs. Specifically, both
stakeholders should cooperate with the goal of improving their situations. An overview of the topic is
given in Section 2.2.3. Here, works that a relevant for this thesis are presented and discussed.
3.1.1 ISP Network Information Services
A number of approaches exist that propose that ISPs offer information on their network topology to their
customers. Peers of P2P systems can use this information to optimize the overlay network topology. The
ISPs benefits through P2P locality, the P2P users benefit from a higher Quality of Service (QoS) due to
increased traffic efficiency. In addition to that, ISPs can use the systems to influence the behavior of P2P
users. This can be achieved by adapting the offered information to their cost structure or their traffic
engineering approach.
Older approaches are the network Oracle [8] and P4P [115]. For using the Oracle, peers send network
addresses on prospective neighbors to the Oracle system. The system orders the address according to
its preferences and sends them back to the peer. A more recent approach is Application-Layer Traffic
Optimization (ALTO) [102], which is developed based on the P4P concept. An Internet Engineering Task
Force (IETF) standard for the ALTO protocol exists, and it is already used in real networks [61]. The
ALTO service specifies different request methods. Request similar to the ones use by the Oracle approach
are supported, but the system also supports the distribution of network cost maps, which can be used by
clients without sending a request every time.
The IETF ALTO standard service [11] aims to enable a mechanism for applications to gather information
on the network the application is running in. ALTO servers are discovered by clients that can send
queries. Offered information are peer-to-peer cost paths and network cost maps for specific peers.
The approaches described in this section aim at improving the influence of ISPs on the traffic of P2P
LVS, which is one of the goals of the RASP service. While it uses a different approach, the approaches
described are complementary to the RASP service, hence could be used alongside for further improvements.
17

3.1.2 ISP-Owned Peers
The concept of using ISP-owned peers to enable the ISP to influence P2P networks is presented in [89].
An ISP-owned peer participates in a P2P system while being controlled or influenced by the ISP it is
operated in. It could either be operated by the ISP or by a different entity, that agrees to implement
the goals of the ISP in exchange e.g. for network resources. The approach is used in P2P file sharing
networks to reduce the traffic volume crossing the network boundaries. The evaluation shows that the
approach works and increases the control of the ISP on the P2P traffic.
However, ISP-owned peers contain P2P application specific parts. They have to be updated and managed
along the P2P software [20, p. 224]. There are a number of different P2P applications used today.
Hence, it is not clear how the approach can be efficiently implemented and operated on a larger scale.
Also, participating in an P2P overlay network can have juridical consequences, for example if illegal
content is distributed and stored on the ISP-owned peer.
ISP-owned peers join a P2P system with the goal of influencing it in the interest of the network operator.
This idea is the basis for approach used with RASP. However, the RASP service improves the concept
by introducing a generic on-demand network layer service. This approach shifts the burden of operating
and integrating the service into different P2P LVS systems from the ISP to P2P LVS users and applications
providers.
3.2 Peer-to-Peer Live Video Streaming and Super Peers
The super peer concept for P2P systems is widely used and researched. Super peers are peers that fulfill
special tasks that could not be achieved by normal peers. This might be due to their stability, their trust
relationship or their available resources for computing or bandwidth. While this hierarchy for peers is
somewhat contradictory to the concept that all peers are equal, a number of works have shown that using
them has an advantage. These concepts are relevant to this thesis, as applying cross-layer optimizations
in technological islands leads to natural clusters and to the question if the empowerment of single peers
can improve the whole P2P LVS system performance. The idea of empowering single peers by providing
them network resources is the RASP systems benefit for P2P LVS systems. Hence, it is interesting to see
how this empowerment can be used to not only improve P2P LVS, but how its effects can be further
increase by specifically exploiting them. Super Peers are specific to their P2P LVS system and set up by
the P2P LVS system operator or its users.
3.2.1 mTreebone
mTreebone [110] is a P2P LVS approach, which introduces a hybrid tree and mesh topology for improved
resilience. To combine the advantages of tree and mesh topologies, a tree topology for video distribution
is created for video stream delivery. A mesh topology overlay exists to handle packet loss in the tree
topology. To increase the resilience of the tree topology, the system relies on a backbone, called treebone,
which consists of super peers. These super peers are elected because of their stability. They make up the
upper levels of the distribution tree. Other, normal peers are connected to the super peers. The mesh
overlay network is created over all nodes and is only used in case the tree delivery fails.
The authors show that their approach works and increases the resilience and quality of the system.
This result stresses the relevance and usefulness of super peers. Hence, mTreebone would be well suited
to use the RASP service.
18 3. Related Work

3.2.2 SALSA
In [51], a Super-Peer Assisted Live Streaming Architecture (SALSA) is presented. The basic idea is to
auction video streaming quality in exchange for video distribution. The proposed P2P LVS system consists
of three kinds of nodes, the single video source, super peers, and leaf peers. Super peers get the video
stream from the source and distribute it to leaf peers, which do not distribute the video stream further.
The system uses a multi tree approach, where the video stream is split into sub streams, which can be
recombined arbitrarily. Each additional sub stream increases the quality. The number of sub streams the
super peers get varies, while leaf peers always get the lowest quality. The goal of the video source is to
service as much leaf nodes as possible. The video source auctions full quality videos streams to super
peers. The payment in the auction is video streaming time to leaf peers. A super peer offers streaming
time to a number of leaf peers; the highest bidder gets the full quality video stream. This incentive
motivates super peers to use as much of their upstream bandwidth as possible for distributing the video
stream. Due to this mechanism, the tree construction is influenced by the auctions. The authors show
by simulation that their approach works and that is has the desired property of motivating upstream
bandwidth contribution.
SALSA is based on the contributions of super peers, which makes it interesting for the RASP service.
The work shows that to account for super peers during P2P LVS system design is beneficial. Furthermore,
an approach is described on how to motivate users to increase their upstream bandwidth. Hence, it could
be adopted for motivating users to use the RASP service.
3.3 Overlay Networks and Multicast
The approaches on combining IP multicast and P2P systems can be distinguished in two groups: one
aims to support and improve IP multicast with P2P systems and one that aims to improve P2P systems
with IP Multicast. In this thesis, we are interested in the latter approach. A P2P system should be optimized
by integrating network multicast services where available. However, much of the literature tries
to connect multicast islands to enable global multicast. While these approaches have not the same goal,
it is interesting to investigate the approaches used and how they are related to RASP.
There are a number of approaches on this topic. One is the Scalable Adaptive Multicast (SAM) research
group of the Internet Research Task Force that investigates multicast protocols including hybrid
techniques that combine application layer multicast with IP multicast [49]. The group proposes to an
existing generic P2P overlay for connecting multicast islands.
Another approach is called island multicast. Its goal is to improve application layer multicast by integrating
IP multicast islands into the overlay network where available. A number of island multicast
approaches exists [45, 113, 62].
In [46] an overview is given on the research on island multicast until the year 2009, hence, [113, 62]
are not discussed there. The fundamental concept is similar with all approaches. Most systems exhibit a
global tree-based overlay network, and an IP multicast control group local to the corresponding island.
Peers first detect if they are located within an IP multicast island by contacting the local control overlay.
If there is no IP multicast support, the peer forms its own island and uses the application layer multicast
overlay network for operations. If the peer is located within an island, it connects to the other peers of
the island. Some of them are connected to the global control overlay to peers of other islands. Thereby
all islands with its peers are connected into an Internet wide multicast network.
3.3. Overlay Networks and Multicast 19

The focus of the research is the efficient and resilient interconnection of the IP multicast islands.
Island multicast systems can be classified by the implementation of their functional elements: island
management, overlay joining, ingress and egress peer election, and connecting multicast islands [46, p.
159]. Some approaches designate a local peer (leader) for island management. If there is no leader, all
local peers join the global overlay network.
Island multicast approaches combine the global connectivity and robustness of application layer multicast
with the efficiency of IP multicast. Efficient and reliable ways of interconnecting the multicast island
exist. Open research questions are error correction within multicast islands and the integration of these
approaches with mesh-based overlay networks. However, the application scenario of these approaches is
not clearly defined. They aim to overcome the limitations of IP multicast islands without considering the
reasons for the failure of global multicast as described e.g. in [23] (see Section 2.1.1). Most of the island
multicast systems require globally unique multicast IP addresses for operations, which is an unsolved
problem. IP multicast authentication, authorization, encryption and data integrity are still open issues,
which might cause ISPs to refraining from opening their IP multicast platform to third parties. The business
case is not discussed; still it could be assumed that an operator of an application layer multicast
system is required to contract for IP island operations. Most approaches assume in their evaluations that
a significant number of IP multicast islands are available. For efficient operations, large deployments are
needed which improve the situation of operators using the service. In addition, the system is based on IP
multicast, all routers and the end device have to support IP multicast for operations.
The island multicast approaches share a goal with the RASP service: to exploit locally available network
layer multicast mechanisms for global communication systems. However, the approaches are very
different. Many island multicast approaches aim at connecting IP multicast island for their global use.
This approach has its disadvantages, as described in the last paragraph. Hence, for RASP a local approach
is chosen, that allows single multicast islands to be integrated into existing P2P LVS systems. This
approach has the advantage that it is useful even with a small number of deployments. Furthermore, its
deployment does not require new application layer multicast systems as the island multicast approaches
do.
3.4 Improvements to IP Multicast
While IP multicast is deployed, it is not globally connected. Nevertheless, improvements on multicast
are researched. Some of these approaches are interesting to consider when implementing multicast with
OpenFlow.
IP multicast is a research topic for already more than 20 years. A large number of concepts has been
proposed and analyzed. In this section, a number of approaches are introduced, that propose concepts
that are be helpful for integrating P2P with OpenFlow based multicast.
The flexibility and performance of OpenFlow allows the implementation of concepts that have hitherto
been impractical to implement. This concerns especially addressing schemes and concepts for addressing
the multicast scaling problem.
3.4.1 Recursive Unicast
In [104] the network layer multicast scheme REUNITE is presented, where unicast IP addresses are
used for forwarding and delivery to implement a multicast-forwarding scheme. The main goal of the
approach is to increase the scalability of network layer multicast by reducing the amount of multicast
20 3. Related Work

elated information stored on routers in the network. IP unicast IP addresses are used for forwarding,
a small number of multicast-enabled routers are responsible for replicating the packets on joints of the
multicast tree.
The following description of the forwarding scheme of REUNITE references corresponding elements
in Figure 3.1. A multicast group is identified by the IP address of the multicast tree root node (S1) and
the source port number of the packet stream. For multicast routing, signaling, and group membership
management two separate protocols are introduced, which are not discussed here. A multicast packet
P1 is send by the tree root node, using the multicast groups IP address and source port. The destination
address is a unicast IP address of a host H1 that is a member of the multicast group. The packet is
forwarded using the unicast-forwarding scheme of the IP (R2). On routers which constitute a joint of the
multicast tree, an additional multicast routing table is present (R1, R3, R4). There, packets are identified
by their group id. For each group id, a number of destination IP addresses exist, representing a different
subtree of the multicast tree. The incoming multicast packet uses already one of those IP addresses as
destination address and is forwarded according to the unicast routing table (R1:P1, R3:P1, R4:P2). For
each other destination address in the table entry, the packet is replicated and the address written as
destination (R3:P2, R4:P2). Each replicated packet is send according to the unicast routing table. Using
this scheme on each tree joint, each member of the multicast group gets the multicast packet.
S1
R1
Multicast Router
Multicast Table:
S1 ➔ H1
Unicast Router
R2
R3
Multicast Router
Multicast Table:
S1 ➔ H1, H2
R4
Multicast Router
Multicast Table:
S1 ➔ H2, H3
P1:
S1 ➔ H1
P2:
S1 ➔ H2
P3:
S1 ➔ H3
H1 H2 H3
Figure 3.1.: The forwarding scheme of REUNITE (adapted from [104, p. 1645]).
REHASH [10] improves REUNITE by introducing more efficient multicast forwarding tables. In [118]
a OSPF protocol extension is presented, that implements an improved version of REUNITE. The multicast
approach of REUNITE has the advantage of forgoing class D IP addresses for forwarding and not requiring
IP multicast features on all routers in the network. However, it introduces new network protocols
and is not widely deployed as of today.
3.4. Improvements to IP Multicast 21

REUNITE uses unicast IP addresses for multicast forwarding and describes one way how this can be
achieved without SDN. It also shows that using unicast addresses for multicast is not possible without
special hardware or SDN. The RASP approach shows that it is possible with OpenFlow.
3.4.2 Explicit Multicast
In explicit multicast, each multicast packet contains all destination information without requiring multicast
network state information. One approach, called Xcast (Explicit Multi-unicast) is described [43].
There, an additional header field contains all unicast IP addresses of the multicast group. The destination
address of Xcast packets is a special IP multicast group, which all Xcast capable routers belong to. A host
sends an Xcast packet to a capable router, the router analyses the Xcast header and forwards the packet
according to the contained unicast IP addresses. If it contains different subsets of Xcast destination addresses
with different next hop addresses, the packet is replicated and each packets Xcast header is set
to the respective subset of destination addresses. If a subset emerges that only contains one address, the
packet is transformed into a normal unicast packet. The mode of operation is similar to recursive unicast
schemes, with the difference that the replication information is contained in the packet header, not in
the router.
The advantages and disadvantages are discussed in [43]. The main advantages are that no network
state is required, group management is done ad-hoc by senders and not all routers need to include Xcast
features. However, due to the restricted space in the packet, only small groups can rely on Xcast. In
addition, no incentives are given to the ISPs for rolling out Xcast, except of data volume savings. Still,
specialized routers are needed which rely on multicast IP addresses. Furthermore, the Xcast mechanisms
could be exploited for unwanted packet duplication in combination with source address spoofing.
Xcast is a multicast system based on the idea of embedding all destination addresses of a multicast
group in each packet header. The work shows that its approach works and can reduce the multicast state
in the network. However, is only applicable for small groups. RASP takes up the idea and applies it to
OpenFlow, where it can be implemented for larger groups and without special purpose hardware.
3.4.3 LIPSIN
LIPSIN [47] is an approach for network multicast forwarding in publish/subscribe systems. The goal
of the system is to reduce the state information that is stored in the network and thereby increases
the scalability of publish/subscribe systems. The main contribution is multicast forwarding where each
packets contains its distribution path encoded in a Bloom [106] filter, similar to source routing. Each
path contains the list of network links to travel, which is evaluated by each switch in the network and
forwarded along the given paths.
The authors evaluate their system with a simulator and with a prototype of a LIPSIN compatible router.
The result shows that their approach works and can improve publish/subscribe systems. The concept of
multicast state reduction in networks using path information in the packet header is promising. However,
a new network protocol is assumed, that enables the storage of the Bloom filter in the packet.
Furthermore, specialized hardware is required for forwarding. While the concept is interesting, large
deployments are unlikely to happen due to the hardware requirements.
The goal of LIPSIN is similar to the one of Xcast. However, the approach is different, not the destination
addresses are encoded in each packet, but their paths. The approach is interesting and investigated for
its use in RASP, but ultimately not used due to its storage space required in each packet.
22 3. Related Work

3.4.4 Labelcast
In [112] a network protocol for IPTV data delivery is introduced. The goals of the approach are twofold:
first, decouple the multicast tree management from the user management, second enable the application
of traffic engineering and load balancing to multicast traffic. In the PIM-SM protocol, the path a group
membership request takes to the group management node is used for multicast delivery to that user. Labelcast
decouples the two functions by introducing a network protocol, which combines RTP/RTCP with
a multicast-forwarding scheme, based on the traffic engineering idea of MPLS. The Labelcast header
includes content metadata information on such as priority and flow bandwidth. Routers are assumed
to add a special Labelcast handling facility, which is different from the IP routing facility. The Labelcast
protocol handler includes means for packet duplication. The path management is done by a label
distribution protocol similar to MPLS. It includes facilities for monitoring of the video stream.
While the goals of Labelcast are interesting, its implementation using a new network protocol is not
ideal. Routers and clients are required to support it, which is a big obstacle for deployments.
The goals of Labelcast – to apply traffic engineering to multicast traffic and making multicast routing
more flexible – are taken up by the RASP approach. Again, the work shows that implementing these
features without SDN is difficult. The decoupling of tree management from the user management is
implemented in RASP. Furthermore, the idea of adding metadata on the multicast groups is interesting.
While the RASP service does not include this idea, it could be evaluated in future works for better quality
of service.
3.5 OpenFlow and IP Multicast
The concept of SDN and OpenFlow enable new possibilities for IP multicast. This section presents some
of the approaches on how to implement multicast with OpenFlow in particular.
3.5.1 IP multicast with Fast Tree Switching
In [56] the authors propose an IP multicast-based forwarding system optimized for fast recovery in case
of path failures. A network using IP multicast is set up using OpenFlow switches. For each multicast
group, the OpenFlow controller calculates two different multicast trees spanning all switches of the
network. If a switch fails, the controller disables the currently used tree and enables the complementary
tree, which might be unaffected by the failure. The authors implement a tree algorithm that creates two
redundant multicast trees. They evaluate their approach using a NEC 8800 hardware switch used as
nine virtual switches. Low packet loss and fast switching times are achieved. An in-switch method, that
could ease the implementation of the proposed mechanism, is available from OpenFlow version 1.2 on
by using the group feature.
The use of the group feature for multicast forwarding shows the development of OpenFlow and is
beneficial for use in the implementation of the RASP service. The application of redundant trees are
interesting for improved resilience and could be investigated in future research on RASP.
3.5.2 CastFlow
CastFlow, presented in [70] proposes the adaptation of IP multicast to the capabilities of OpenFlow. It
is assumed that the network used consists of OpenFlow switches. For group management, IGMP could
3.5. OpenFlow and IP Multicast 23

e used with the difference to normal IP multicast that the IGMP messages are forwarded to an Open-
Flow controller when they are received by the first switch. The OpenFlow controller hosts the multicast
functionality in one place in contrast to the distributed nature of IP multicast. The distribution tree is
calculated by the controller software and the network is configured accordingly using OpenFlow. To mitigate
issues of the OpenFlow controller as a single point of failure, they propose the use of a distributed
OpenFlow management system. For quick response times in case of system events, a wide range of multicast
trees is calculated in advance. Furthermore, the tree management system is designed to require as
little changes as possible for group member changes.
The system’s implementation based on the NOX [35] controller is described and evaluated with Mininet
[59]. They use BRITE [73] to generate network topologies and a custom program to emulate topology
events. The authors asses the performance of their system when reacting to events generated in the
network. In addition to that, they analyze the efficiency of their tree construction on varying network
sizes.
While the adaptation of IP multicast to an OpenFlow setting is convincing the analysis is not. It is not
clear how the response time of their system can be transferred to other settings. Especially considering
their use of Mininet for network emulation, whose performance similarities to real networks is unclear.
Furthermore, the approach is very limited. It is not clear how it can be integrated with other networks
or OpenFlow domains.
The RASP service does not implement IP multicast. However, the tree construction and the management
concept for multicast from CastFlow are adopted by the RASP service prototype, which is described
in Chapter 6.
24 3. Related Work

4 Assumptions and Requirements
The aim of the design of the RASP service is to improve P2P LVS systems and the situation of ISPs. Hence,
the requirements of both stakeholders of the system are described in this chapter. A general requirement
is that the systems must be realizable with available technology. Specifically, only available versions of
the OpenFlow standard and other deployed network technologies must be used. The system should be
easy to implement and require no hard- and software except OpenFlow and related technologies.
The basic assumptions for the service design are described in Section 4.1. The requirements of ISPs
are discussed in Section 4.2, the requirements of P2P LVS systems in Section 4.3
4.1 Assumptions
For this work, a future scenario is assumed where OpenFlow is a dominant technology. For the technological
environment of the RASP service, today’s ISP network topologies and P2P LVS systems are assumed.
ISP core networks are assumed to be completely based on OpenFlow switches. While no specific assumptions
are required for other technologies used, the ones deployed today are used for comparison. For
aggregation and access networks, different scenarios are considered based on available information.
For approximation of possible future rule numbers, the contemporary available CAM sizes, as described
in Section 2.3.2, are used as a starting point. Up to 160,000 rules can be used on devices that are sold
at the time of writing. For the future scenario rule capacities between 50,000 for cheap low-end devices,
such as access switches, between 100,000 and 250,000 for mid-range devices such as Broadband Remote
Access Servers (BRASs), and 1,000,000 for high-end devices such as core switches are assumed.
A P2P LVS system is considered. No further technical details are specified for the RASP service design,
as it should be as generic as possible. For the deployment, it is assumed that the P2P LVS system video
sources, management, and many of its peers are located outside the ISP network. Due to the number
of ISPs and P2P LVS systems, this assumption is statistically true for most systems in most networks.
The special case when a video source node or existing peer is located within the ISP network is not
considered. In such a setting, other approaches, such as increasing the peer’s bandwidth or allowing the
use of a hosting center, are available today.
The selection of suitable peers for using the RASP service is left to the P2P LVS system operators and
users. The service provides network resources; hence, the stability of peers should be a requirement. The
identification and selection of stable peers is an active research topic (see e.g. [111]). It is assumed, that
sufficiently stable nodes are selected for the use of the RASP service.
4.2 Requirements of ISPs
This work describes an approach for optimizing the use of P2P LVS with OpenFlow in ISP networks with
residential broadband access customers. The application of OpenFlow in ISP networks requires some
consideration. The requirements of ISPs regarding the resource consumption of services are described
in Section 4.2.1. The requirements for the integration with other services and their management are
presented in Section 4.2.2.
25

4.2.1 Service Resource Consumption Requirements
ISPs can have millions customers, hence one import requirement for OpenFlow-based services is their
appropriate scalability. For deploying services in ISP networks, three characteristics should be considered:
the number of customers is high, the traffic volume is high, and the potential number of applications is
high. The scalability of OpenFlow applications has two aspects: first, the traffic volume, second, the
number of OpenFlow rules and rule changes. The first aspect is an issue for all network applications. The
issue of restricting the number of OpenFlow rules used and their dynamics is not unique to OpenFlow
either, IP multicast has similar issues, but it is more important for OpenFlow due to its capabilities and
broad applicability.
The special structure of ISP and their large number of customers necessitate different requirements for
different network parts. Core networks transport traffic from all customers, including, but not limited to,
residential broadband customers. Hence, applications should restrict their bandwidth usage. However,
what is more important is the consumption of OpenFlow resources: the number of rules of OpenFlow
services should be kept as low as possible. The number of rules per service must not scale linearly with
their amount of users, but at much lower factor. Considering that one router could forward the traffic of
millions of customers, using one rule per user is clearly not feasible. The failure of IP multicast shows that
the number of worldwide multicast groups is too high to require single devices to store all of them. In
addition, as potentially many applications will be deployed in ISP networks, the number of rules should
be kept low at all costs.
The situation is different in access and aggregation networks. While access networks such as a Digital
Subscriber Line Access Multiplexer (DSLAM) typically serves up to a few hundred customers, aggregation
networks and especially broadband customer access devices such as BRASs connect tens of
thousands customers. The number of available OpenFlow rules per device is assumed about 50.000
in access switches and 200.000 in broadband customer access devices, as described in Section 4.1. Considering
these number, it is feasible to store one rule per customer on access devices. However, it is less
so for broadband aggregation devices.
The same considerations should be applied for OpenFlow rule dynamics. No real world measurements
on the performance of large-scale OpenFlow network deployments are available, hence their capabilities
in terms of OpenFlow message processing is difficult to estimate. Still, its number should be kept low to
allow other services to work in parallel.
4.2.2 Service Integration and Management Requirements
For deployment of the RASP service, compatibility with other services running in the same OpenFlow
domain is required. It should be able to work with different means of transport encapsulation, such as
MPLS, and should not rely on specific network paths or forwarding mechanisms. The interactions with
other services should be predictable and comprehensively specified. The RASP service should be able to
adapt to the IPv6 protocol. Furthermore, the service should not be restricted to single use cases; it should
be a generic as possible.
The whole service should be defined comprehensively and clearly. This includes predictable resource
usage and the possibility to apply traffic engineering. Reducing the network traffic inside the network
and to the outside is also beneficial.
26 4. Assumptions and Requirements

4.3 Requirements of Peer-to-Peer Live Video Streaming
The RASP service’s goal is to improve the performance and quality of P2P LVS systems. It achieves this
goal by increasing the network upload capacity of single peers. The general requirements of P2P LVS
systems are meeting the deadline for synchronicity, providing a good stream quality, and system stability.
However, the RASP does not alter the P2P LVS system directly, but offers a network layer service.
Hence, the service can only influence the associated parameters of the transmission. The resulting technical
requirements are described in Section 4.3.1. The requirements of users, operators and application
providers [38] are described in Section 4.3.2.
4.3.1 Technical Requirements
The system should support a wide range of P2P LVS systems. It should not restrict its applicability to
a single type of topology, such as tree or mesh. While tree topologies are more natural for network
layer multicast, many commercially operational systems use a mesh topology [66, p. 236] for its better
resilience. In general, the integration of the RASP service into P2P LVS systems should require as little
effort as possible.
The system is required to improve the effective network upstream capacity of the peer using it. Other
parameters of the P2P LVS systems should not be impaired by the service. Specifically, it should not
increase the bandwidth requirements for the peers, increase packet loss or increase the latency of the
transmission.
For real deployments, P2P LVS systems are required to support Network Address Translation (NAT)
due to its prevalent used in residential broadband networks. However, the goal of this work is to provide
the concept of the service and analyze its characteristics. Hence, the support of NAT is beyond the scope
of this work. The same is true for the support of heterogeneous peers and the behavior under of peer
churn and flash crowds [65, p. 20].
4.3.2 Requirements of Users, Operators and Application Providers
The usage of the service must not require technical changes except those on the P2P LVS software itself.
Specifically, IP multicast protocols, which are often not supported by consumer grade equipment used in
home networks, should not be required. The API should be as simple as possible. Still, it should be able
to provide the peer using it with all relevant information on the system.
4.3. Requirements of Peer-to-Peer Live Video Streaming 27

5 System Design
The concept of the RASP service is presented in this chapter. Its design is based on the requirements defined
in Chapter 4. The goal is to provide a flexible, on-demand service for live video stream distribution
for peers of P2P LVS systems. The concept is a cross-layer application offered by ISPs to peers of the P2P
system. The service design is component based, it combines the functionality of two functional OpenFlow
service components: a network layer proxy application and a network layer multicast application. The
first provides the peer a virtual presence in the ISP’s network; the latter provides efficient and reliable
multicast data distribution. A third component, the RASP controller integrates the other components and
provides a public API.
The remainder of this chapter is organized as follows: in Section 5.1.1 an overview is given on the
RASP service’s concept and its rationale is discussed. In Section 5.2 the system architecture is explained
and the systems components described. The integration of the RASP service into P2P LVS systems is
explained in Section 5.4.
5.1 Description and Rationale
Before discussing the rationale of the RASP service and its benefits for the involved parties in Section
5.1.2, an overview of its and functionality is given in the following Section 5.1.1.
5.1.1 Functional Description
The focus of RASP is the network of the ISP deploying the system. For an exemplary description of the
operations of RASP, a schematic setting of an ISP network and a P2P LVS system is used as shown in
Figure 5.1. The ISP network is shown as a box on the left, consisting of the aggregation and access
network, and the core network. The core network is connected to the internet, the access network to
the broadband access customers of the ISP. Peers P0 through P5 participate in a P2P LVS system. The
overlay network connections are displayed in blue, the video stream tree, starting in P0, is depicted by
orange arrows. Peers P3 through P5 are located in customer networks of the ISP, peers P0 through P2
are located in arbitrary networks connected by the Internet.
The ISP network after introducing the RASP service is shown in Figure 5.2. All routers within the ISP
network are OpenFlow based. A hosting network area is added in the ISP network, which is used for
placement of the public service API. It is accessible by any Internet-based host and enables the use of
the RASP service. Users of the system enter a contractual relationship with the ISP before gaining access
to its features. In the example shown in the figure, peer P1 accesses the API to use the RASP service,
as indicated by the green arrow. After registration, P1 requests the creation of a RASP instance, which
causes the set up a virtual peer 1 . A virtual peer is a network layer proxy that establishes a presence for
P4 in the ISP network. Peers (e.g. P3) in the customer networks of the ISP are made aware of the new
peer in the network and choose it for joining the P2P LVS system. Therefore, they send overlay network
connection requests to the virtual peer. There, the source and destination IP address and ports of each
1
The use of the term “virtual peer ” is unrelated to its earlier use in [6]
29

ISP
Internet
Core
P0
P1
P2
Aggregation/
Access
P3 P4 P5
Figure 5.1.: A simplified illustration of the operation of a P2P LVS system.
packet are rewritten. As source IP address the virtual peer’s address is used, as destination IP address the
one of P1. As destination port, the number registered by P1 is used, as source port a hitherto unused by
the virtual peer is selected. This approach allows the identification of associated packets traveling in the
opposite direction, as used with network and port address translation [103]. Therefore, the packets from
peers send to the virtual peer are relayed by the virtual peer to peer P1. Packets traversing the opposite
direction from P1 to P3 are addressed to the virtual peer. There, the packets source and destination
addresses and ports are rewritten, redirecting the packet to P3.
By using the virtual peer, P3 and P1 have established an overlay network connection. P4 and P5 do the
same and remove the direct overlay connection to P1, leading to the overlay network topology as shown
in Figure 5.2. Note that each peer has established its own, dedicated network connection. Thus both
connection oriented and connectionless network protocols can be used for joining the overlay network.
For the distribution of the video stream, P1 offers video stream transmission to every peer connected
through the virtual peer, regardless of its own network upload capacity. The video delivery is done by
means of an OpenFlow based network layer multicast system, hence only connectionless protocols such
as User Datagram Protocol (UDP) can be used as transport protocol. Also, using the existing overlay
network connections for in band video stream delivery cannot be used. When a peer requests the video
stream, it supplies the local port number where the stream should be addressed to. P1 adds the IP address
of the peer and its port number to the multicast recipient list of the RASP system by using the service
API. P1 sends the video stream packets addressed to the virtual peer and the previously defined streams
port. The multicast system calculates the distribution tree for the video stream. It uses the virtual peer’s
IP address location as starting point and adds all registered peers to the destination list. The multicast
distribution tree is configured on the OpenFlow switches as required. Video stream packets arriving
from P1 are redirected by the OpenFlow rules. On their way towards the peers, they are duplicated as
needed by corresponding OpenFlow rules. In the access network, before leaving the ISPs network, the
destination address and port are rewritten to the ones required by the peer.
30 5. System Design

Internet
P0
ISP
Server/Hosting
Core
P1
Service
API
OF
OF
Virtual
Peer
OF
OF
P2
Aggregation/
Access
OF
OF
P3 P4 P5
Figure 5.2.: A simplified illustration of the operation of a P2P LVS system using the RASP service.
The virtual presence of P1 and its improved video stream delivery capacity inside the ISP network
provide it super peer like network performance. This is enabled by the RASP service, hence its name. In
this context, P1 is called super peer, which is enabled by a RASP instance.
The RASP service can be considered as combination of two distinct functional components: a network
layer proxy and a multicast service. The first is called Virtual Peer and the latter Software Defined
Multicast (SDM).
5.1.2 Rationale
The RASP service concept is introduced in the preceding section. However, for better comprehensibility
the introduction lacked a rational for the system and its approach. Before describing the systems architecture
and its components in Section 5.2.1, the design approach of the complete system is discussed.
The RASP service is an on-demand network service reduces the burden of P2P LVS for ISPs. For P2P
LVS, it offers a generic stream distribution service for users and operators. The design approach is to exploit
the hardware features of OpenFlow switches for high performance and low resource consumption.
RASP from a P2P LVS Perspective
As described in Section 2.1.2, the upload capacity of peers in P2P LVS systems is crucial for their
performance. The RASP system provides network capacity, enabling normal peers of a P2P LVS system
to act like super peers. Given that stable peers exist with low upload capacities, they can use the RASP
service to increase their network capacity. Stable peers are important for P2P LVS systems, as described
in Section 4.1, in combination with a high upload capacity they can expand their influence in their P2P
LVS system. Hence, the RASP service should increase the system performance by increasing the resources
of single peers.
5.1. Description and Rationale 31

RASP from an ISPs’ Perspective
For ISPs the RASP service has two advantages: increased control of P2P LVS network traffic and improved
transmission efficiency. When the RASP service is used by P2P LVS users, the amount of P2P LVS
traffic passing through the ISP network is reduced and at the same time, the identified network traffic to
which traffic engineering can be applied is increased. The RASP service usage is fully controllable, the
bandwidth for video streams can be monitored, and the number of users is known and can be restricted
if required. Before a new user is added, the system can verify if the required resources to serve it are
available. This is an improvement over IP multicast systems, which often lack admission control. The
RASP service is used on demand. Resources are freed after the usage period, which increases the flexibility
of the available resources for the ISP compared to IP multicast whose resource allocation is not
subject to time restrictions.
Reasons for Using a Network Layer Proxy
The RASP service could be used without a proxy service; in this case, it would be a pure multicast
system. However, this would reduce the control of the ISP over the P2P traffic. By using the virtual peer
functionality, the ISP can verify that each peer the video stream is delivered to has an open network
connection to the video source. In case of unwanted behavior or failure of the peer using the RASP service,
all connections between the peers and the RASP service can be removed by deleting the associated
OpenFlow rules. Furthermore, with the RASP instance IP address location inside the ISP network, it can
be announced as a preferred peer by the ISP as described in Section 5.4.3 without taking the risk of
promoting malicious or spoofed IP addresses.
An alternative for achieving the same amount of influence on P2P LVS systems are ISP owned peers
[89]. However, such systems are P2P LVS system dependent, which requires the ISP to support a vast
range of P2P LVS systems. Also, a complete peer has to be operated by the ISP, which requires more computational
resources. With the RASP service, only the network transmission has to be operated, which
can be implemented using OpenFlow hardware features. The system allows users to access available
network resources, while the P2P LVS system operators or users operate the peer and take care of integration
into the respective P2P LVS system. The only dedicated computational resources required for
operating the RASP service a public API service and two OpenFlow applications.
Reasons for SDM
The use of network layer multicast is the most efficient approach on distributing data in one to many
communications. Recent studies comparing multicast to P2P LVS systems conclude that this is the case
even when underlay awareness is used [17]. It also has the advantage of having only one packet stream
on the network, compared to many different, which increases the ability to control the traffic and apply
traffic engineering.
Technical features from established IP multicast approaches are not used. Instead, its methods for
creating and maintaining multicast trees are combined with a unicast address scheme. This method has
the advantage that it is completely transparent for systems outside the OpenFlow domain. Peers using
the system are not required to use IP multicast or be specially adapted. If IP multicast were required, this
would require all network devices in the residential network of the P2P LVS user to support IP multicast
as well. For this reason, ISPs offering IP multicast IPTV services often demand from their customers to
use specific devices that support the required features. Because the RASP service is not such a prominent
feature for customers, such requirements could hinder the acceptance of the service. Therefore, the
alternative for using SDM, IP multicast, is not used.
32 5. System Design

The forwarding approach used with RASP does not use the destination IP address of multicast packets.
It allows multicast-forwarding schemes the reuse the unused fields for different purposes. This is illustrated
by the Explicit Multicast subcomponent as described in Section 5.3.4. It uses the IP address field
for reducing the network state requirements during forwarding.
One reason traditional IP multicast systems failed is their lack of control by the network operator. For
that reason, they are used within network domains only. The same is true for approaches such as island
multicast, which aims to connect those domains, but fail to improve controllability of IP multicast. A
comparison between IP multicast, overlay multicast, and SDM is shown in Table 5.1. Except from not
using specialized network protocols, IP multicast and SDM are very similar. Their main difference is the
control the ISP has over the system. Overlay multicast is more flexible, but too, is hard to control and is
less efficient for data delivery.
Multicast tree
Deployment requirement
Transport layer
connection
Scalability
Congestion control/loss
recovery
IP multicast Overlay multicast Software Defined Multicast
Interior tree nodes are
routers, and leaves are
hosts
Require multicast-capable
routers
UDP
Low — multicast-capable
routers are not scalable
No
Both interior nodes and
leaves are hosts
Can be directly deployed
on the current Internet
Can freely choose TCP or
UDP
High — fully distributed
and scalable protocols are
available
May use existing solutions
for unicast
In the SDM network
nodes are routers, outside,
nodes and leaves are
routers or hosts
Requires OpenFlowcapable
routers within
the SDM domain, no requirements
outside
UDP
Depends on the Open-
Flow hardware and
management architecture
used
Delivery efficiency High Low High in the SDM domain,
low outside of it
ISP control Low None High
Table 5.1.: Comparison of multicast concepts (based on [46, p. 158]).
No
Choosing a Generic Approach
The RASP service offers a network layer service that is not specific to certain P2P LVS systems. This
property should allow the service to be used with a wide range of P2P LVS systems, while on the other
hand require the P2P application providers to adapt their application to the service. Furthermore, it
means that the system can be used in different ways by P2P LVS systems. Some P2P LVS systems with tree
topology split the video stream in multiple substreams, which are distributed over different distribution
trees, for example SplitStream [16]. This approach increases the systems resilience, because if one tree
fails, the other substreams may still work. This allows to continue to show the video, albeit in degraded
quality, instead of the stopping it. Such systems can use the RASP service to stream multiple sub streams
over the same instance or by using different instances.
5.1. Description and Rationale 33

Due to its generic approach, the system can also be used for other purposes. This is possible, because
the service offered is a generic network service, consisting of a network layer proxy and transparent
network layer multicast. The service API is more specific, as monitoring and contractual details must be
included for real deployments. However, the technical part of the API could be standardized. The generic
approach could also lead to other uses of the service.
5.2 Architecture
The architecture of the RASP system is described in this chapter. The design goals for the architecture are
to yield a stable and clear system as well as enable the reuse of functional components. The OpenFlow
network architecture for the system is discussed Section 5.2.1. An overview over the components, their
interaction and dependencies is given in Figure 5.2.2
5.2.1 OpenFlow based Network Architectures
The reference architecture for OpenFlow-based networks is shown in Figure 5.3, as presented in [82,
p. 7]. The foundation is the infrastructure layer, which represents the OpenFlow devices and the links
between them. Its API is the OpenFlow protocol, which is used for communication with OpenFlow controllers.
The latter are part of the control layer, which also includes network applications that are built
on top of a management abstraction. The network applications can be accessed through their API, which
is also called northbound API. No standards exist for these APIs and their implementations yet. The
network applications are used by applications from the application layer to implement cross-layer functionality.
The control layer applications implement technical features, which can be used for different
business purposes. Control layer applications are not meant for public consumption, they should be used
by trusted entities. Applications from the application layer serve a business purpose. Their usage can be
offered to outside entities as their purpose and their restrictions are clearly stated.
Application
Layer
Business
Application
...
Business
Application
Control
Layer
OpenFlow
Application
API Access
...
OpenFlow
Application
OpenFlow Interface
OpenFlow Protocol
Infrastructure
Layer
OpenFlow switches
Figure 5.3.: OpenFlow network architecture (adapted from [82, p. 7]).
34 5. System Design

The architecture of OpenFlow applications is an active research topic. As described in Section 2.3.1,
the infrastructure layer and the OpenFlow protocol changed fast over the last years. In the control layer,
a number of open issues exist. The abstractions the OpenFlow controller should offer and how it should
be implemented is not yet determined. Considering the size of most networks, especially ISP networks
that are discussed in this thesis, a single controller is not sufficient. Hence, the concept of the network
operating system is introduced in [35]. A network OS “provides a uniform and centralized programmatic
interface to the entire network” [35, p. 105]. It should offer a single API for accessing the functions of an
OpenFlow-based network, which is a large distributed system made up from switches, links, controllers,
and miscellaneous software. Various aspects of this approach are open research issues; an overview of
them is given in [55]. The topic includes the question for the appropriate level of abstraction, [108],
[109], consistency [98] and approaches to composition of applications [97]. Details on the design and
implementation of OpenFlow-based network operating system is beyond the scope of this document.
For the remainder of this document, it is assumed that a logically central configuration entity exists for
the OpenFlow network. In a real network, this would be a distributed system of OpenFlow controllers
and other components; the task however, is still the configuration of the elements of the infrastructure
layer. Hence, for the sake of simplicity the term OpenFlow controller is used for the logically central
configuration entity.
5.2.2 System Structure and Interactions
The RASP service can be divided into three distinct parts: the RASP controller component with the public
service API and two network components as shown in Figure 5.4.
Application
Layer
RASP Controller
with Service API
Control
Layer
Explicit
Multicast
SDM
Application
Virtual Peer
Application
OpenFlow Controller/
Network OS
Topology
Database
Infrastructure
Layer
OpenFlow switches
Figure 5.4.: Architecture of the RASP service.
The technical functionality of RASP can be split in two parts: a Virtual Peer component and a SDM component.
The first implements the network layer proxy functionality, the latter the network layer multicast
5.2. Architecture 35

functionality. The SDM and Virtual Peer application should only implement functions that are strictly and
uniquely needed to implement their respective task. This separation of features ensures that the Virtual
Peer and SDM components are reusable for other purposes. The RASP controller component’s task is to
translate incoming API calls to the control layer components and to coordinate their behavior. It is also
responsible for the management and selection of network resources for the control layer components.
The controller component is application dependent, hence cannot be reused for other purposes and is
kept as small as possible for this reason. The SDM component relies on multicast forwarding subcomponents
for parts of functionality. Besides a basic multicast concept, a subcomponent for Explicit Multicast
forwarding is presented. It reduces the amount of state information stored in the network compared the
basic multicast forwarding propose as default for the SDM component.
Both control layer applications are based on an OpenFlow controller or network OS, which provides
a topology database that contains information on the infrastructure layer. This database is commonly
required by control layer applications. The interactions between the components are kept as low as possible
in order to increase reuse and flexibility. The RASP service in the application layer interacts with
both control layer applications. The interactions include the management of Virtual Peer and SDM instances.
Interactions are started from the RASP service; exceptions are error or resource status change
notifications, which are initiated by the control layer. There are no interactions between the two control
layer applications. However, both access the underlying network OS and its topology database. Interactions
patterns are bidirectional as management commands are passed to the network OS and network
resource status information are passed from the network OS to the applications.
5.3 Components
The components that constitute the RASP service are the RASP controller component, the Virtual Peer
component and the SDM component. The names of the relevant entities of the RASP service are depicted
in Figure 5.5, . The entity using the RASP system is called super peer. The entities using an RASP instance
to participate in the P2P LVS system are called peers in this context.
The functionality and interfaces of the components are described in the sections 5.3.1, 5.3.2 and 5.3.3.
Furthermore an extension to the SDM application for improved multicast group scalability is presented
in Section 5.3.4.
5.3.1 Rent-a-Super Peer Controller Component
The RASP service is managed and controlled by the service controller. This application offers the service
by means of a public API and relies on the OpenFlow application for its delivery. It implements the nonnetworks
part of the application such as authenticating super peers, logging the service usage, enforcing
non technical-constraints in service usage and allocating the networks resources. The design goal is to
provide the service as generic and functional as possible. For modeling of a P2P LVS system generic
abstractions are used: such a system consists of a control channel, and one or more data channels for
video streams; each video channel has a number of receivers. Except for expecting the service user to be
involved in P2P LVS no assumptions on the specific use of the service are made.
The API should offer fundamental services such as establishing the connection to an interested super
peer and keeping the connection. During all operations, the service controller manages accounting information
on the service usage. Once a connection exists, the following functions are needed to provide
the RASP service:
36 5. System Design

Internet
Super-Peer
ISP
RASP
Service
API
Server/Hosting
OF
Core
OF
Virtual
Peer
OF
OF
Aggregation/
Access
OF
OF
Software Defined
Multicast
Peers
Figure 5.5.: The names of entities involved with the RASP system.
1. Create/delete/monitor RASP instances
2. Confirm peers
3. Add/remove video streams to RASP instances
4. Add/remove receivers of a video stream
5. Asynchronous message exchange for exchanging system events
The functions are described by assuming successful requests. Errors and their messages are not described
in detail; still, they are expected to be sent in case of errors. An error might occur for example if
a given resource is already in use.
Create/delete/monitor RASP instances
This function accepts information on the RASP instance to be created: The transport protocol and port
to be used for forwarding P2P overlay connections and the network port of the super peer where the
overlay connections should be forwarded to. The IP address of the super peer could also be provided or
the IP address of the corresponding communications endpoint could be used. The latter should reduce
the susceptibility to misuse as the IP address’ existence and location is validated by the communications
between the API server and the super peer.
Upon receiving the request, the service controller allocates an IP address for the RASP instance and
sends it as reply to the super peer. Also, it uses the instance address to create a Virtual Peer instance using
the application’s OpenFlow API. The instance address is unique and hence is used for identification.
During the whole time of operating the RASP instance, the service controller should maintain the
identity and network information of the super peer. In case the identity or network information check of
the super peer fails during operation, the RASP instance should be deleted as described below. Alterna-
5.3. Components 37

tively, the service controller could offer means for re-establishing the identity and network information
by appropriate measures.
For deletion of a RASP instance, the super peer can send a deletion request. The service controller
replies with a confirmation. For deletion, the service controller uses the SDM and Virtual Peer application’s
APIs to remove the associated multicast groups and the virtual peer.
Monitoring information includes data on the service performance as well as contractual and cost information.
However, these aspects of the service are not the scope of this work.
Confirm peers
Peers connect to the super peer through the network proxy of the RASP instance. The setup of the
proxy rules happens without the involvement of the super peer. To reduce the effects of malicious connection
attempts, the super peer has to confirm each overlay connection of each peer that uses the Virtual
Peer component. Once the overlay connection between the super peer and the peer is active, the super
peer sends a confirmation message to the API, making the proxy-forwarding configuration for this peer
in the Virtual Peer component permanent. If it fails to do so within a given time, the forwarding configuration
is removed. Details on this mechanism are given in description of the Virtual Peer component in
Section 5.3.2.
Add/remove a video stream
For adding a video stream to a RASP instance, the super peer supplies the port number to be used as
source port for the video stream (virtual stream port) and the port number (stream source port) used by
the super peer for sending the video stream. The service controller sends a confirmation message.
The service controller uses the SDM OpenFlow API to create a new multicast group. For removing a
video stream, the super peer sends a remove request and supplies the instance address and virtual stream
port. The service controller uses the SDM API to remove the stream and sends a confirmation.
Add/remove a receiver for a video stream
For adding a receiver to a video stream, the super peer requests the addition of a receiver. It supplies
the video stream port and the new receivers IP address and receive port. The service controller adds the
receiver to the selected multicast group by using the SDM API, providing the RASP instance address,
stream port, and the receivers address and port number of the new group member. For removing a
receiver, the super peer requests its deletion and supplies the receivers IP address and port number. The
service controller passes this information to the OpenFlow API and sends a confirmation message to the
super peer.
Asynchronous message exchange
Unexpected events can occur during operation of a RASP instance. Network devices can fail, clients
might disconnect unexpectedly or the service contract might expire. For such cases, the service controller
should be able to send information to the service user in an asynchronous way. The implementation of
this communication channel is up to the service operator and depends on the API technology used. It
might be implemented by pushing messages to the user or offering functionality where the user can poll
for new messages. Details on error handling and recovery are beyond the scope of this work.
38 5. System Design

5.3.2 Virtual Peer Component
The Virtual Peer application provides the creation and management of virtual peers, which enabling
network layer communication between two hosts through a proxy. A virtual peer (VP) enables the proxy
presence of the super peer (SP) in the network. The proxies operate at the network layer because its
implementation at the application layer could not be implemented using OpenFlow features only. Such
an approach would be less generic and would not be able to exploit the performance of OpenFlow
hardware.
A virtual peer is created by selecting an IP address, which should be used as proxy IP address
(VP address ). For integration with other network applications and services, the addresses should be taken
from an IP address range reserved for this purpose. The OpenFlow rules that implement the functionality
of a virtual peer are located on an OpenFlow switch or a number of switches. All packets addressed to
VP address must pass this switch (VP switch ) with the associated OpenFlow rules. The routing scheme used
in the underlying network must be set up accordingly.
Before creating a virtual peer, the super peer supplies a transport layer protocol (SP protocol ) and port
number (VP port ). This protocol and port number is used for accepting connection request addressed to
the VP. The general idea is to accept connections addressed to VP address using VP protocol and destination
port VP port and forward them to the super peer by rewriting source and destination addresses and ports.
The same is done for reciprocal packets coming from the super peer, which constitutes the network proxy
functionality.
This approach is required for redirecting network connections transparently from the virtual peer IP
address to a different address. For an explanation, consider the alternative where NAT would only be
applied in one direction, for example from H1 to H2 through proxy PR. Packets from H1 are address to
PR, their destination address is rewritten in P to H2, they arrive there with source address H1. Replies to
that packet would be forwarded to H1, without necessarily passing PR. Finally, they arrive at H1 which
expects the packets’ source address to be PR. Because the source address is H2 the packet is discarded;
no bidirectional communication is possible.
An OpenFlow rule is installed on VP switch which forwards packets that are addressed to VP address using
VP protocol and destination port VP port to the OpenFlow controller. There, this information is delivered to
the Virtual Peer application. The application checks the packet header for consistency and creates two
rules if it contains a hitherto unknown source address and port combination which identifies a new peer.
The host from which the packets are coming from is called peer P with address P address using P port as
source port. This information is extracted from the received packet.
One of the created rule matches on packets coming from P address as shown in Table 5.6a, the reciprocal
rule matches on packets coming from SP address as shown in Table 5.6b. The pair of rules implements
source and destination NAT [103]. The first rule rewrites the destination address of a packet to the one
of the super peer, SP address and the source address is rewritten to VP address . The source port is rewritten to
a port selected by the controller, which uniquely identifies the connection (peer NAT port, P NAT
port
). This port
is used for matching packets of the same network connection in the reciprocal rule. These rules match
packets coming from VP address with the SP port as source port and destination port P NAT
port
. The destination
address and port of the packet are rewritten to the associated peer’s incoming values, the source address
is set to VP address .
Each connection attempt from a peer causes two rules to be installed on the switch. To reduce the
number of rules and the impact of malicious connection attempts, both rules are set with a hard timeout
of T h seconds. The super peer sends a confirmation message to the Virtual Peer application for each
5.3. Components 39

Field Match value Change value New value
Transport protocol SP protocol no –
Network source address P address yes VP address
TCP/UDP source port P port yes P NAT
port
Network destination address VP address yes SP address
TCP/UDP destination port VP port yes SP port
(a) OpenFlow rule schema for handling packets from peer P to super peer SP.
Field Match value Change value New value
Protocol SP protocol no –
Network source address SP address yes VP address
TCP/UDP source port SP port yes VP port
Network destination address VP address yes P address
TCP/UDP destination port P NAT
port
yes P port
(b) OpenFlow rule schema for handling packets from super peer SP to peer P.
Figure 5.6.: OpenFlow rules for implementing a virtual peer for a connection initiated by peer P to super peer SP.
incoming connection from a peer it chooses to accept. If the message arrives within T h seconds from the
installation time of the rule, the hard timeout is replaced by a soft timeout of T s seconds. If it fails to do
so, the rules are removed automatically by the OpenFlow switch. The rules are deleted if the super peer
notifies the Virtual Peer application to do so, a soft timeout occurs, or when the virtual peer is removed.
The value of T s should be set to values that are appropriate for the transport protocol used.
The usage of one IP address per virtual peer restricts the maximum number of connections to 64511
as the port number for both Transmission Control Procol (TCP) and UDP is a 16-bit value and the port
numbers up to 1024 should not be used. Maintaining 64511 network connections, even if they are
used for low traffic volumes only, requires the super peer to dispose sufficient processing and network
resources. Hence, connecting to 64511 other hosts might be sufficient for many use cases. Additional IP
addresses could be used as source addresses, increasing the maximum number of supported clients by
the service. This approach is beyond the scope of this work.
Interactions with other components.
The Virtual Peer application does interact with the OpenFlow controller it is based upon in two ways.
First, the commands for the OpenFlow switch configuration are passed to the controller. Second, status
information on the OpenFlow switch and its configuration are passed from the controller to the application.
This includes information concerning the operational status of the VP switch and timeout events
related to the OpenFlow rules managed by the application. Both types of events are passed to the users
of the API, depending on the technology used either by push or pull interaction pattern.
5.3.3 Software Defined Multicast Component
The task of the SDM application is to distribute video streams efficiently in the ISP’s network. However,
SDM does not use rely on IP multicast features or protocols to achieve this goal. Instead, the capabilities
40 5. System Design

of OpenFlow are used for implementing network layer multicast with IP unicast addresses and cross
layer group management.
Multicast systems require three functions: group management, multicast tree management and signaling,
and addressing and forwarding. Figure 5.7 shows an IP multicast domain and an SDM domain.
IP multicast uses IGMP for group management. The multicast tree is managed by a specialized router.
The forwarding elements in IP multicast are routers that use protocols like PIM-SM for signaling. For
forwarding, addresses from the IP multicast range are used. Each network device on the path from the
source to the destinations is required to support IP multicast.
Multicast
Packet Stream
IP Multicast
Domain
Distribution Tree
Management Router
Intradomain
Multicast Routing
Protocol: PIM-SM
Group Management
Protocol: IGMP
(a) IP Multicast Domain.
Group Management
SDM API Access
Unicast packet stream
OpenFlow
API Server
OpenFlow
signaling
Multicast
Packet Stream
Software Defined
Multicast Domain
Group
Management
Server
Group Management
Client Access
Multicast packet delivery
with unicast addresses
(b) Software Defined Multicast Domain.
Figure 5.7.: Comparison of an IP multicast and a SDM domain.
With SDM, arbitrary application layer protocols can be used for group management as depicted by
the green arrows in Figure 5.7b. For security reasons, the group membership API should not be used
by every group member individually. Hence, a group member is elected to act as proxy for the group
management features. In case of the RASP system, the super peer is the group manager. The OpenFlow
SDM application manages the distribution tree inside the SDM domain. The OpenFlow controller uses
the OpenFlow protocol for signaling of the multicast tree. The whole network consists of OpenFlow
switches, no specialized devices or protocols are needed. For forwarding, normal unicast IP addresses are
5.3. Components 41

used outside the SDM domain, inside the packets are marked as explained below. The data source sends
the packet stream to a special network addressed as shown by the topmost red arrow in Figure 5.7b. The
packets are labeled for multicast transport and forwarded as shown by the blue lines. Before exiting the
system, the destination address and port are rewritten. The packets exit the system as unicast packets,
which is shown in Figure 5.7b as the red arrows near the bottom.
The approach of SDM exploits the features of OpenFlow: the separation of routing and forwarding,
the efficient rewriting of packet header fields, and the easy integration with other software. The latter
enables the use of a application layer API for the service instead of using network layer protocols as with
IP multicast. This is beneficial for integration with the rest of the RASP service as no separate mechanism
is required for the use of multicast.
For multicast forwarding, SDM has to perform three functions: identification and marking packets
addressed at the border of the SDM domain, distribute the packets to the OpenFlow switches at the
other border of the network and finally rewrite the packets fields to match the values expected by the
each group member. This schema is depicted in Figure 5.8.
Group source
Unicast addressing
Ingress switches
SDM Domain
Forwarding by
SDM mark / SDM
group socket
Egress switches
Unicast addressing
Group members
Figure 5.8.: Schema of the SDM concept.
The identification of packets addresses at a SDM group is implemented on the SDM domain’s ingress
switches. The set of ingress switches includes all switches where packets address to SDM groups could
enter the SDM domain. The packets are marked as SDM packets after identification for simple recognition
inside the SDM domain. Finally, before leaving the SDM domain, for each member of a multicast group
the corresponding destination address and port is written to the multicast packets on the SDM egress
switches. For optimal efficiency, the SDM domain should span the whole OpenFlow domain. This means,
that egress and ingress switches should be as close a possible to the border of the OpenFlow domain.
The alternative is to locate the ingress and egress switches further inside the OpenFlow domain. This
approach reduces the part of the OpenFlow domain where the SDM packets are efficiently forwarded
using the multicast tree and where they are an easy subject to traffic management. Once the destination
address of the respective peer is written to each packet, matching multicast traffic requires one rule for
each group member.
This approach of marking incoming packets is used, as SDM multicast packets can only be identified by
their specific source and destination address and port, and the used transport protocol. The alternative to
42 5. System Design

marking the packets is to use OpenFlow rules with 5 match fields on all switches for multicast forwarding
in the SDM domain. The used match fields make it impossible to match more than one SDM group at
once in one OpenFlow matcher. On OpenFlow hardware devices that allocate the CAM per match field,
this approach also increases the resource usage. To mitigate this, the packets are marked in a generic
fashion, identifying them as multicast packets. In addition to that, their multicast group membership
is revealed by the packets source address. Marking packets on entering a network and using the mark
for forwarding inside the network is an established concept in networking; one example is MPLS [100].
For forwarding inside the SDM domain, the mark is used for identification of multicast packets. This
allows matching multicast packets by using one OpenFlow match field only and the group by using two
additional match fields. The destination address and port fields in the multicast packet headers are not
used; they can be used for other purposes if necessary.
The detailed operation of the SDM concept and the required OpenFlow rule schemata are described
in the following paragraphs. An IP address, a transport protocol, and a port number identifying a communications
endpoint is called socket. Only connectionless transport protocols are suitable for network
layer multicast. Hence, UDP is the transport protocol assumed for all sockets described in this section.
SDM groups are identified by their group socket G socket = (G address , G port ). It is used as source address for
packet delivery and as destination address for the packet stream coming from the group’s source socket
(S socket ). Alternatively, other unused fields could be used for group identification. For example, the data
link layer addresses are likely be not used for forwarding in an ISP core network. A unique group ID
can be assigned to each group and the source and destination port used for different purposes. However,
two more fields of the multicast packets have to be rewritten when the packets leaves the SDM domain,
which might increase the resource requirements on the egress switches. Furthermore, the data link layer
addresses can also be used for different purposes as described in the Explicit Multicast extension.
Creating a new SDM group through the API requires the group manager to supply G socket and S socket .
Based on this information, OpenFlow rules for matching and marking multicast packets belonging to
that group are created on the relevant ingress switches, which are denoted by SWC G . This could be
mark
either a single OpenFlow switch or a set of switches. Packets addressed to G address need to be forwarded
to the SWC G , which should be ensured by the routing application used within the OpenFlow domain.
mark
The requirement is automatically fulfilled if all ingress switches where packets addressed at the SDM
group G may enter the SDM domain are contained in SWC G . The OpenFlow rules needed for marking
mark
the packets are shown in Table 5.2. The rule matches on the source and destination address and port.
It writes the SDM mark to the packet and sets its source address and port to the ones of the multicast
group G.
Field Match value Change value New value
Protocol UDP no –
Network source address S address yes G address
TCP/UDP source port S port yes G port
Network destination address G address no –
TCP/UDP destination port G port no –
SDM mark field – yes SDM mark
Table 5.2.: OpenFlow rule schema for marking packets addresses to a SDM multicast group on the group’s marking
switches.
5.3. Components 43

The packet header fields and its values used for marking are left to the discretion of the network operator.
This allows easy integration with other services and the existing technical environment. Depending
on the OpenFlow version used, a VLAN or MPLS tag, or any other header field that is unused during
forwarding, like the destination IP address field, can be used as mark field.
After creating a SDM group, members can be added by the group manager. A member M G of group
G is identified by its socket M G socket = (MG address , MG port
), which is supplied by the group manager. The
last OpenFlow switch in the network on the path from G address to M G is identified. This switch is
address
called the member switch M G switch , the set of member switches in a SDM group is called SWCG member .
Potential member switches are the egress switches of the SDM domain. When a member is added, a rule
is installed on M G that rewrites the multicast packets destination IP address and port to the values
switch
from the M G . The schema for this rule is shown in Table 5.3.
socket
Field Match value Change value New value
Protocol UDP no –
SDM mark field SDM mark no –
Network destination address – yes M G address
TCP/UDP destination port – yes M G port
Table 5.3.: OpenFlow rule schema for writing a member’s socket information to packets of a SDM multicast group
on the group’s member switches.
The multicast packets are identified by the SDM mark to separate them from other traffic. The group
a packet belongs to can be identified by the source address and port of the packet. The identification
method is similar to Source-Specific Multicast (SSM) [41]. This approach enables the network operator
to apply traffic engineering to all SDM multicast data streams or individual multicast groups.
The approach on the multicast tree management can be chosen arbitrarily by the network operator.
The use of marked packets for multicast forwarding in the network and the unused destination address
fields allow the application of different multicast-forwarding schemes. One application of this is use of
multicast group state reduction mechanisms in the network as done by the Explicit Multicast concept,
which is presented in Section 5.3.4.
For illustration, a simple group-based distribution tree is described. The distribution tree is determined
by two sets of switches: SWC G mark and SWCG . The minimum tree spanning both sets of
member
switches (G tree ) is calculated for each SDM group. On the OpenFlow switches that are part of G tree a
rule is installed for forwarding packets coming from SWC G mark to SWCG . Packets are duplicated by
member
the OpenFlow rules installed on internal nodes of G tree . One more efficient example is introduced in
Section 5.3.4.
Interactions with other components.
The SDM application relies on the OpenFlow controller and the topology database for its operation.
The topology database is used for gathering information on ingress and egress nodes, finding member
switches M G for group members, and for determining the multicast tree. The application processes
switch
information on topology changes adapting the multicast distribution tree if needed. If a multicast group
or the forwarding to a member fails, the respective users are notified.
44 5. System Design

5.3.4 Explicit Multicast Subcomponent
The SDM concept introduced in this chapter separates the delivery of a multicast packet to a group
member from the multicast forwarding within the network. The SDM system’s core functionality is the
group management, the identification of incoming packets and the preparation of outgoing packets. The
multicast routing inside the network, whose task it is to deliver packets from an ingress switch to a number
of egress switches, can be implemented by a separate subsystem. To illustrate the possibilities of this
approach, a stateless hierarchical multicast-forwarding scheme based on Explicit Multicast is presented
in this section. It is enabled by SDM because the destination IP address is not used for forwarding inside
the multicast domain.
The packet-forwarding scheme for SDM as discussed in Section 5.3.3 requires one rule per multicast
group on each OpenFlow switch that is part of the distribution tree. For large numbers of groups this is
disadvantageous, especially for use in the core network of an ISP. In OpenFlow networks, the number
of rules an application requires determines its applicability. The issue of scaling multicast group state
for IP multicast is an open research topic [60]. Also, the topic is researched in the context of SDN.
In [19] forwarding without group state is achieved by including packet-processing instructions for the
switches in each packets header. However, this approach cannot be used with the available versions of
the OpenFlow standard. Other alternatives rely on Bloom filters for reducing network state, e.g. [76]
in which each packet contains the whole path it should travel. However, as explained at the end of this
section, Bloom filters are not suitable in this context.
With ability to rewrite packets and having a logically centralized OpenFlow controller hitherto unfeasible
approaches can be reevaluated. In the approach outlined in this section, hierarchical multicast [21,
p. 107] and explicit multicast [43] are combined. The basic concept is to use explicit multicast, where
all destinations are contained in the header of each multicast packet. To keep the size of destination
addresses small, the multicast domain is structured hierarchically. In higher layers of the hierarchy, one
destination address denotes a complete subnetwork. When entering a subnetwork, each packets header
is rewritten to contain destination addresses of the specific subnetwork only. In the context of SDM the
term multicast domain is used, therefore the hierarchical structure consists of SDM subdomains.
The SDM domain is divided hierarchically into subdomains, each consisting of less than 128 destinations.
When using core Point of Presences (PoPs) as destinations, this assumption covers more than
90% of commercial, research, and educational networks [54, p. 1772]. Within a subdomain, each packet
header includes a bitmap of all locations it should be forwarded to. This requires a 128-bit field in the
packets header. Because the destination IP address field is not used for forwarding in SDM it can be used
for other purposes. Using bit masks as addressing scheme for multicast is described in [92]. For IPv4 the
network destination address with 32 bit and both data link layer addresses with 48 bits each are used.
For IPv6 the destination address could be used for a 128 bit field, and both data link layer addresses
for an even larger bitmap. The idea of changing the semantics of address fields with SDN is presented
in [32, p. 5]. Each subnetwork contains n ≤ 128 border locations. Each switch in all locations contains
routes to locations, yielding n routing entries per device.
For illustration, an SDM domain with 3 subdomains and a 3 bit address field is shown in Figure 5.9.
Instead of locations, individual switches are used as destinations in the example. The SDM domain has
the ingress switches S1 and S2 and the egress switch S7, S8, S10, S11. Subdomain 1 has the border
switches S1, S2, S4. Subdomain 2 has the border switches S4, S7, S8, Subdomain 3 has the border
switches S4, S10, S11. The SDM multicast group depicted by the example has 4 members, which are
connected to the egress switches S7, S8, and S11, the group’s mark switch is S1. Packets forwarded by
5.3. Components 45

Unicast addressing
Ingress switches
S1 S2
100 010
SDM Subdomain 1
S6
Subdomain border
001
100
S4
100
SDM Subdomain 2
S9
S12
010 001
Egress switches
Unicast addressing
S7 S8
010 001
S10
S11
Figure 5.9.: Schematic overview of an Explicit SDM domain and its subdomains.
S1 are addressed at 001, with S4 as the destination switch. Switch S4 is a border switch, its task is
to rewrite the packet destination addresses to the respective subdomain destination addresses for each
multicast group. The OpenFlow rule schema of S4 is shown in Table 5.4.
Match
Group G
Actions
Write L3 dst = 011, forward to S9; Write L3 dst = 001, Forward to S12
Table 5.4.: OpenFlow rule schema for switch S4.
The rule schema on switch S9 illustrates the forwarding behavior inside a subdomain. Only the destination
addresses are matched and the packets forwarded based on this information. No group related
matching is required. This approach allows multicast forwarding inside subdomains with a very small
number of rules. This is important for switches in the core of an ISP network, which are part of the forwarding
path of most applications used in the network. Hence, the number of rules used for individual
applications should be as small as possible.
L3 destination address field
Match bitmask
Comparison value
Action
010 010 Forward to S7
001 001 Forward to S8
Table 5.5.: OpenFlow rule schema for switch S9.
Matching individual bits of the destination address fields is required for the approach to work. This
feature is included in OpenFlow version 1.2 or later, it is not possible to implement this system with
OpenFlow version 1.0 or 1.1.
46 5. System Design

Bloom filters are an often used alternative approach for storing addressing information, an overview
of their use is given in [106, p. 146-148]. An OpenFlow based approach using Bloom filters is described
in [76]. However, the Bloom filter is used to encode the whole path of the packet not its destination
addresses. The goal of the approach described here is to implement group scale independent forwarding
with OpenFlow and without introducing new header fields. Hence only existing and unused header fields
can be used, which offers less than 200-bit storage space. Furthermore, Bloom filters are probabilistic;
their false positive rates influence the ratio of filter size and number of elements that can be stored. A
Bloom filter with a size of 128 bit and a false positive rate of 0.01 can store − 128(ln(2))2 = 13.4 elements
ln(0.01)
[106, p. 133]. 128 elements can be stored when each node is encoded as a number and a bitmap is
used. However, Bloom filters are used in multicast approaches without relying on OpenFlow, but using
forwarding methods similar to source routing [47, 96]. In these approaches, each packet contains the
whole forwarding path in its header, encoded in a Bloom filter, similar to the OpenFlow base approach
described before.
5.4 Peer-to-Peer System Integration
The goal of RASP is to offer upload capacity offload to P2P LVS peers. However, the usage of RASP
requires the P2P LVS system to support its technical implementation. Specifically, RASP requires the
super peer to support its service API and the feeding of the video stream to the service. Other peers
are required to receive each video stream pushed by the super peer on a dedicated unidirectional UDP
network connection.
5.4.1 General Considerations and Peer Behavior Adaptation
The adaptation to these requirements is up to the P2P LVS system operators and application providers.
Still, certain elements are expected to be part of most adaptation processes. The overlay connection does
not require any adjustments. Hence, peers can connect to a super peer using a RASP instance normally.
The super peer should inform the peers upon connection that the connection used is a special one, which
employs certain characteristics. If the peer is able to receive the RASP service and agrees to do so, it
continues with the connection. If not, it should terminate the connection. Other P2P communication
should proceed normally, like announcing the list of available channels and other information. If a peer
requests a video stream, the super peer should notify the peer on the conditions of the delivery. This
information includes the port number of the video stream source. The peer provides his IP address and
port number where it wished to receive the video stream. The super peer then proceeds as described in
the functional description in Section 5.1.1.
Many P2P LVS systems use the overlay network for control message and video stream delivery. RASP
requires the video stream to be delivered using a dedicated network connection. Hence, the systems
need to implement this delivery method in order to use the RASP service. The video stream delivery
method of the service has the same characteristics as tree topology. The video is pushed from the super
peer to the peer using a unidirectional connection. If the P2P LVS system is not tree but mesh-based,
adaptation system can be considered. One example for such systems is Transit [99], which allows to
adapt the topology and scheduling of P2P LVS systems dynamically. Peers connecting to a RASP super
peer could dynamically switch to tree-based topology.
5.4. Peer-to-Peer System Integration 47

5.4.2 Super Peer Adaptation
The super peer is required to implement the API access to the RASP service for its usage. This includes
the setup of a RASP instance and the feeding of the relevant video streams into the system. If connected
peers request the delivery of a video stream, the super peer uses the RASP API for adding them to the
stream’s member list. When a peer leaves, another call to the RASP API is required.
Because the super peer has a contractual relationship with the ISP, it should prohibit the misuse of
its RASP instance. The overlay connections should not be used for data delivery as the virtual peer is
not designed for high data rates and data volume. For maximal efficiency and P2P LVS performance, the
super peer should accept all peers that request to join the system through the RASP instance.
Special measures should be taken to make sure the system is able to take over the duties of a super peer
in the system. This can be done on discretion of the super peer operator or the P2P LVS system operator.
Approaches on how to select stable peers could are described in the literature, e.g. [111]. Furthermore,
approaches on motivating peers to increase their upstream bandwidth are researched, e.g. in [51]. Once
the RASP instance is running, its contact information should be passed to neighbors.
5.4.3 Rent-a-Super Peer Instance Discovery
RASP instances should be given precedence for peers within ISP networks. One way to achieve this is to
propagate the information in the P2P LVS system. Peers should propagate the RASP instance information
in their neighborhood. Another way of achieving RASP instance precedence is using information supplied
by ISPs for users of P2P systems. The ALTO service [11] is a standard protocol for exchange of such
information. It enables P2P users to query their ISPs for information about potential peers. The ALTO
endpoint cost service [11, p. 12] is used for prioritization of peer connections. A peer sends a list of
potential neighbors to the service, which replies with network costs for these clients. When the ALTO
service is requested to give the costs of an IP address of a RASP instance, the connection costs should
be minimal to encourage peering. This could be implemented by reserving certain IP networks for RASP
instances. All addresses in these networks are advertised with minimal connections costs. As each RASP
instance is used for only one P2P system, only peers using this network will attempt to connect to it,
which eschews the problem for the ALTO service of identifying the used P2P LVS system before giving
information.
48 5. System Design

6 Prototype
This chapter describes the prototypical implementation of RASP for evaluation. The basic components
described in the RASP design architecture, which is shown in Figure 6.1 are part of the prototype.
The prototype allows the evaluation of the basic RASP concept and its characteristics. They include the
transmission efficiency and the overhead generated by the RASP service. Therefore, the implementation
focuses on the key components of RASP: the RASP controller with the service’s API, the Virtual Peer, and
the SDM component. The explicit multicast subcomponent is not required for evaluating the concept. It
is relevant for reducing the network state requirements of the SDM component. Furthermore, it requires
OpenFlow version 1.2 for its implementation, for which no evaluation environment is available; the
reason for that is described in Section 6.2.2. Of these components, only functional parts required for
evaluating the RASP service are implemented. Features concerning security such as authentication and
authorization and accounting are not part of the prototype.
Application
Layer
RASP Controller
with Service API
Control
Layer
Explicit
Multicast
SDM
Application
Virtual Peer
Application
OpenFlow Controller/
Network OS
Topology
Database
Infrastructure
Layer
OpenFlow switches
Figure 6.1.: The components of the RASP service that are part of the prototype.
Furthermore, their actual implementations using OpenFlow and their integration with other OpenFlow
applications running in the same network are relevant. As the service should be offered by ISPs, the
characteristics of ISP networks and their influence on the RASP concept are evaluated as well. Hence,
the prototype needs to support these settings, too.
No sufficiently large testbed is available to the author; hence, an emulated network is used. The
emulation is based on a Mininet [59], which makes the automated creation and operation of emulated
network based on OpenFlow switches possible. Either hardware switches or software switches can be
used with the system. Normal Linux programs can be used on the emulated hosts of Mininet.
The use of OpenFlow version 1.1 or higher is advantageous for the RASP applications, because it
enables more efficient combination of OpenFlow rules, as described in Section 2.3.1, which simplifies
the integration of different OpenFlow applications. Furthermore, the Explicit Multicast subcomponent
49

elies on OpenFlow 1.2, which is still a new standard, so that it is not supported by all relevant software
yet. In the following options that allow the support of OpenFlow 1.1 or 1.2 will be included if possible.
However, as shown in Section 6.2.2 the available software does not allow its evaluation on a reasonable
scale.
This chapter is organized as follows: first, an overview of the prototypes architecture is given in Section
6.1 and on the network emulation in Section 6.2. Being the foundation of the RASP applications
components, the OpenFlow controller software is described in Section 6.3. The description of the implementation
of the components of the RASP service follows in Section 6.4.
6.1 Architecture
An overview of the prototype, its components, and the corresponding setup for evaluation is given in
Figure 6.2. In the figure, elements, which are colored in green, correspond to components that constitute
the RASP system, blue elements to supporting applications written for this work, and grey colored ones
to already existing software.
Network
Experiment
RASP Controller
with Service API
REST API REST API REST API
Topology
App
SDM
App
Virtual
Peer App
Network
Topology
Ryu OpenFlow Controller
Mininet
OpenFlow Network
Figure 6.2.: Overview of the prototype implementation and the accompanying emulation system of RASP.
The prototype of RASP consists of three components: the public service interface and two OpenFlow
controller applications. The controller applications are the SDM and Virtual Peer applications. The public
service API allows the usage of the RASP service, whose functionality is implemented by the two
OpenFlow controller applications.
50 6. Prototype

As foundation of these applications, OpenFlow controller software and a network topology database
is required: for the controller, an existing application is used, for the topology database, an OpenFlow
controller application is created. The topology application uses the network topology data, which has
been already used for creating the emulated network, for implementing the forwarding behavior of the
network. In addition to that, it offers topology information on the network to other OpenFlow controller
applications.
Mininet is used for emulating networks. To create network topologies within Mininet, a network topology
configuration is used, which defines the switches, hosts as well as their connections. As the switches
are OpenFlow devices, their behavior is defined by the OpenFlow controller. The hosts’ settings are part
of the topology configuration as well as the networks’ forwarding behavior. Finally, the network experiment
controls the networks behavior during tests. Various scenarios can be created by starting data
streams using the RASP service and running emulated P2P networks.
In this work, during the test the network topology is assumed static. This means that network connections,
peers, and switches are not expected to change their status during tests, what greatly simplifies
the implementation of OpenFlow controller applications. Still, this assumption allows evaluation of the
relevant properties of the system.
6.2 Network Emulation
Before discussing the actual RASP application and its implementation, the environment of the application
needs to be described. The RASP application is meant to run in an ISP network environment; hence, the
goal of the emulated network is to create adequate network settings. The specific assumptions made
should be minimal, while still implement the required characteristics.
For creating and running the emulated network, Mininet, as described in Section 6.2.1, is used. The
simulated network and its topology are described in Section 6.2.3
6.2.1 Mininet
The network emulation tool Mininet [59] version 2.0 is used. Mininet runs on Linux and uses Linux
features for its operation. A Mininet network consists of OpenFlow switches and hosts. For switches,
Mininet creates instances of OpenFlow software switches as required; the available software is discussed
in Section 6.2.2. Hosts are running as normal Linux programs using their own virtualized network stack
instance, called network namespace. Mininet uses virtual Ethernet pairs to connect these network namespaces
with the OpenFlow software switch instances. This approach allows the creation and control of
emulated networks and the programs running on the emulated hosts.
Mininet is written in the Python programming language. It includes programs for running automatically
generated networks and interacting with them. An API is available for creating custom programs,
experiments and network topologies. Therefore, it can be used as library to run a network and its hosts
automatically without the command line interface.
6.2.2 OpenFlow Switch
The Mininet network emulator includes two OpenFlow switch software implementations: Open vSwitch
[91] 1 , and the OpenFlow reference software switch implementation 2 . Both switches support OpenFlow
1
http://openvswitch.org/ [Accessed 21.04.2013]
2
http://yuba.stanford.edu/git/gitweb.cgip=openflow.git;a=summary [Accessed 21.04.2013]
6.2. Network Emulation 51

version 1.0 only. Their main difference is that Open vSwitch implements its forwarding functions in
the Operating System (OS) kernel space while the OpenFlow reference implementation runs in the OS
user space, which influences their performance. Another software switch is the OpenFlow 1.2 software
switch by CPqD 3 which also runs in the user space. It is the only available software switch that supports
OpenFlow 1.2 and integrates with Mininet.
The integration of OpenFlow hardware switches with Mininet networks is also possible. The switch
available to the author is a HP 3500yl with 24 network ports, which supports OpenFlow 1.0. However,
a number of OpenFlow features required for this work are not implemented with hardware by the
switch, but with software. These include the matching of layer 2 addresses, the modification of IP addresses
and the forwarding of packets to multiple switch ports [39, pp. 40, 41]. If a part of an OpenFlow
rule is processed in software, this applies to the whole rule [39, p. 42], so that all OpenFlow features
would run on the switch’s OpenFlow software implementation. Therefore, using the HP switches software
implementation would have no advantages over using the available software implementations, yet
it would require a much more complex setup. Also, the HP switch’s OpenFlow software performance was
reported to be very low [31], yielding 1 Mbit/s throughput for a simple OpenFlow application. The available
switch hardware is not used; the available OpenFlow software switches’ performance is analyzed in
the following paragraph.
Comparison of software switch performance
From the available OpenFlow switches, the performance of Open vSwitch, the OpenFlow 1.0 reference
implementation and the CPqD OpenFlow 1.2 implementation have been compared on the available
computer system. The system is a XEN 4 based virtual machine (VM). Two Intel XEON E5-1410 cores
with 2,8Ghz and 8GB Memory are allocated to the VM.
For testing setups with tens to hundreds of nodes, a used software switch should exhibit sufficient performance
of at least several Mbit/s per node. The data rate of video streams is expected to be between
0.1 Mbit/s and 1 Mbit/s. The lower data rate can be used if necessary with the disadvantage of representing
smaller videos only. Since each stream traverses multiple switches in the simulated network, several
Mbit/s per host are required to enable a sustained video stream delivery. Using the iperf 5 software that
comes with Mininet, the three soft switches are tested for throughput.
Three network sizes are analyzed, with 1, 5, and 10 OpenFlow switches. The switches are connected in
a line; the hosts for running the performance test are connected at the opposing ends of the topology. An
OpenFlow learning switch application is used for forwarding. iperf is run 20 times using the TCP protocol
for measurement. The average of the runs for each protocol is documented in Figure 6.3. Because of the
large differences in the results, the y axis is logarithmically scaled for improved visibility. The averages
for the means of the three different network sizes are connected by colored lines; the 95% confidence
intervals are indicated by black lines. The exact results of the measurement are given in Table 6.1.
The performance differences are huge, Open vSwitch achieves more than 4 Gbit/s with one switch and
1,3 Gbit/s with 10 switches. The OpenFlow version 1.0 reference switch achieves 440 Mbit/s with one
switch and 60 Mbit/s with 10 switches. The CPqD OpenFlow version 1.2 switch performs worst. For one
switch, the throughput is 15 Mbit/s and for 10 switches less than 3 Mbit/s of throughput is achieved. The
scaling behavior also differs greatly. While the throughput of Open vSwitch for 1 switch is about 3 times
higher than the throughput for 10 switches, the ratio is higher for the other two switches. The OpenFlow
3
https://github.com/CPqD/of12softswitch [Accessed 21.04.2013]
4
http://www.xen.org/ [Accessed 21.04.2013]
5
http://iperf.sourceforge.net/ [Accessed 21.04.2013]
52 6. Prototype

version 1.0 reference throughput for 1 switch is about 7 times higher than for 10 switches, for the CPqD
OpenFlow version 1.2 switch it is about 5 times higher.
The numbers for the OpenFlow version 1.0 reference switch that comes with Mininet is similar to the
observation made in [59, p. 4]. The Open vSwitch throughput measured for 1 switch is twice as high
as in the paper, for 10 switches it is still 30% higher. This is due to the better system hardware used in
this analysis over the hardware used in the paper. Furthermore, it shows that the performance of Open
vSwitch improves significantly with better hardware, while this does not seem the case for the OpenFlow
version 1.0 reference switch.
5000
1000
Throughput [Mbit/s]
500
100
50
OpenFlow switch software:
Open vSwitch
Ver. 1.0 reference switch
CPqD Ver. 1.2 switch
10
5
1 5 10
Number of OpenFlow switches
Figure 6.3.: Throughput of the three OpenFlow software switches as measured by iperf.
Switch software Switch count Throughput
[Mbit/s]
Open vSwitch
Ver. 1.0
reference
switch
CPqD Ver.
1.2 switch
95% confidence
interval [Mbit/s]
Standard
deviation
1 4189 ±32 15
5 2116 ±21 10
10 1290 ±8 4
1 440 ±0.8 0.4
5 133 ±0.3 0.1
10 60 ±0.5 0.3
1 15.0 ±0 0
5 5.75 ±0.01 0.005
10 2.85 ±0.003 0.001
Table 6.1.: Throughput of the three OpenFlow software switches as measured by iperf.
For emulation of networks with the largest possible networks on the given hardware, Open vSwitch
offers the best throughput. The OpenFlow version 1.0 reference switch should be usable for smaller
6.2. Network Emulation 53

networks; however, it has no advantage against Open vSwitch. The throughput of the CPqD OpenFlow
version 1.2 switch is too low for networks larger than a few hosts. As it is the only available OpenFlow
1.2 software switch that integrates with Mininet, OpenFlow 1.2 is not used for the evaluation. OpenFlow
1.0 and the Open vSwitch implementation are used for measurements.
6.2.3 Topology Database and Network Experiments
The API of Mininet is used to start and control emulated networks. A program called runtopo.py implements
the approach described in this section. For an experiment, the program requires two parameters:
a topology database and a network experiment. The process of running an experiment is depicted in Figure
6.4. A topology database is loaded and instantiated by Mininet. The resulting object is transmitted to
the topology application that runs on the Ryu OpenFlow controller. Then, the emulated hosts are started
and configured using the information from the topology database. The emulated network is started as
well as the topology application. This enables the forwarding in the emulated network. After that, the
network experiment is loaded, instantiated, supplied with the topology database and gets started. The
experiment is supplied by the user and takes control of the emulated network. When it is finished, the
topology as well as the emulated network is stopped.
Mininet
Ryu
TopologyApp
Create topology database
PUT /topology/mininet
serialized RASPTopo object
200
Setup Hosts
Start Mininet network
PUT /topology/control
start
200
Setup forwarding according to
the RASPTopo object
Run network experiments
PUT /topology/control
stop
200
Figure 6.4.: The process schema of running an emulated network experiment.
54 6. Prototype

The topology database Python class inherits from the Mininet Topo class. In addition to OpenFlow
switches, hosts and connections between them, it contains information on the ISP network. For running
an emulated network for P2P LVS, not only nodes inside the ISP network are required, but also nodes
which are located on the Internet. Hence, for each host and switch the location information is added.
Each node is either located in the ISP or Internet part of the network. For enabling routing and differentiation
between different uses of switches, they are tagged with a type attribute. A switch is called access
switch if there are hosts directly connected. A switch is called hosting switch, if it is used as dedicated
switch for the OpenFlow applications of the RASP application. A switch is called core switch, if there are
no hosts connected and if it is not a hosting switch.
An example network is shown in Figure 6.5. Core switches are depicted in orange, access switches in
blue, and hosting switches in green if they are located in the ISP network. Nodes, which are located in
the Internet part of the network, are depicted in gray. Hosts are not shown individually but as groups
depicted by a rectangle and labeled with the number of hosts contained in the group.
5 H
Access switches
Hosting switch
Core switches
ISP
Number of connected hosts
5 H 5 H
Internet
5 H 5 H
5 H
Figure 6.5.: An example network based on the topology database information used for creating emulated networks.
6.2.4 OpenFlow Routing and Forwarding
For switches where hosts are directly connected, including access switches, a local area network (LAN)
is created. In reality their behavior is different, as in most cases, hosts are not directly attached, but
connected by an additional router. However, this difference should have no influence on the outcome of
experiments but reduces the amount of switches needed for tests.
To each switch, a dedicated IP network is assigned, which connected hosts are a part of. Rules are
installed that imitate the behavior of a switching hub. Packets from one host to another are identified by
the destination IP and MAC address and transmitted to the appropriate physical port. For IP addresses
outside the local network, a default router address is configured on all hosts in addition to a specific
6.2. Network Emulation 55

MAC address. The rules on the switch create a virtual default router by matching each packet’s address
to the default router’s MAC address and sending it out the uplink port. Complementary rules exist for
incoming packets. They are identified by their source port and their network layer destination address.
A rule matches these and writes the MAC address of the default gateway as source address and the MAC
destination address corresponding to the destination IP address to the packet header. The packet is then
sent out the host’s switch port. A simplified LAN switch is shown in Figure 6.6.
Host A
IP A
MAC A
P1
OF Switch
P3
Uplink port
Rest of the network
Host B
IP B
MAC B
P2
Figure 6.6.: Simplified diagram of an OpenFlow LAN switch in the emulated network.
Host A and B are connected to switch ports 1 and 2 respectively. The uplink port connecting the
switch with the network is port 3. The flowtable for OpenFlow version 1.0 for a LAN switch is shown
in Table 6.2. Layer 2 and 3 addresses are denoted by their host’s name, for the default router they are
denoted by DR.
Priority Match Action
Default + 1 In: P3; To: IP A Set: L2 dst=MAC A; Out: P1
Default + 1 In: P3; To: IP B Set: L2 dst=MAC B; Out: P2
Default To: MAC A Out: P1
Default To: MAC B Out: P2
Default To: MAC DR Out: P3
Table 6.2.: OpenFlow 1.0 flowtables for a LAN switch in the emulated network.
OpenFlow 1.0 relies on a simple first match scheme as described in Section 2.3. For integration of
different network features, there are different rule priorities. Each rule contains all actions which should
be applied to the matching packets. This requires each rule to match all packets, to which a certain action
list is applied. Hence the forwarding rules in the network match relevant packets, their matchers must
not overlap. Still, matchers for rules for forwarding and other network features might overlap. This is
true for rules matching incoming packets on a LAN switch, which have a higher priority for that reason.
The reason for this is the OpenFlow specification, if two rules containing a wild card match – as they do
in the example shown in Table 6.2 – the switch implementation can choose a method to determine which
rule is applied. As the system can be used with different OpenFlow software switches, such situations
would effectively show an unknown behavior.
The rule priority approach illustrated in this example is used in the whole emulated network. Rules
for outputting traffic to directly connected hosts matching layer 2 addresses have the lowest priority.
Rules involving packets from directly connected hosts and forwarding them to other switches have a
slightly higher priority. Rules for forwarding traffic that is not addressed to directly connected hosts
have a higher priority. Finally, rules for the RASP component applications have the highest priority. This
approach makes sure that packets are checked for matching application rules first.
56 6. Prototype

The example discussed above is based on LAN switches. Another kind of switches are hosting switches.
They are used for hosting RASP specific functionality. Hosting switches use a single port for incoming
and outgoing traffic. This approach allows using hosting switches with standardized rules while being
able to place them arbitrarily on the network.
The task of the other switches in the emulated network is to forward packets based on their network
layer addresses. The specific behavior can be configured by the topology configuration used. Different
forwarding mechanisms and their use are discussed in the evaluation in Chapter 7. The forwarding does
not, other than by fulfilling this function, directly influence the RASP component applications. This is the
reason why the evaluation results are also valid for networks which rely on other forwarding mechanism.
6.3 The Ryu OpenFlow Controller
Ryu 6 version 1.7 is used as OpenFlow controller. It supports the OpenFlow versions 1.0, 1.2 and 1.3,
which allows using the prototype with different OpenFlow switches supporting different OpenFlow versions.
Ryu is implemented in Python, which eases the integration with Mininet, the network emulation
system used for evaluation, which is also implemented in Python. Ryu is the only available open source
controller software that supports OpenFlow versions 1.0 and 1.2 [85] at the same time. As discussed in
the introduction, the use of OpenFlow 1.1 or a higher version is advantageous for the RASP application.
Hence, the Ryu OpenFlow controller is used for its flexibility regarding the use of different OpenFlow
versions.
In the following Section 6.3.1 an overview is given and the architecture of Ryu is explained. Then
the limitations of Ryu are discussed in Section 6.3.2 and finally, in Section 6.3.3, a blueprint for Ryu
applications is presented.
6.3.1 Architecture and Overview
Ryu adapts the event driven concept of OpenFlow for its internal architecture, which is shown in Figure
6.7. The core of Ryu is the event system, which creates events and distributes them. Applications
interact with the system by registering for the handling of events. The AppManager is responsible for
loading, starting and managing applications while the OpenFlow library maintains connections to Open-
Flow devices and generates related events.
The event processing starts with an event source, which adds an event to its event queue. The event
queue eventually dequeues the event and delivers it to all registered methods. Registering for events
works by annotating 7 a method or function with the types of events it should receive.
Ryu’s event system is based on the gevent 8 library. As the source of actions in Ryu, it provides means
for cooperative multitasking. When loading the gevent library, other libraries used by the same program
are patched to put the calling thread to sleep, given another thread the chance to take over, once an
I/O operation is called. This approach makes multitasking possible, given that most control threads are
I/O bound. Running long computations or other CPU bound operations impede this mechanism and can
freeze the application. Therefore, CPU bound operations should not be implemented in a Ryu application
[13].
The event system is used for inter-application communications. Applications can declare events and
register for receiving events of other applications. However, the main task of the event system is the
6
http://osrg.github.io/ryu/ [Accessed 21.04.2013]
7
This process is called decorating in the Python programming language
8
http://www.gevent.org/ [Accessed 21.04.2013]
6.3. The Ryu OpenFlow Controller 57

App
1
App
2
...
App
n
Event System
OpenFlow Protocol
Library
App-
Manager
Figure 6.7.: The architecture of Ryu [77].
distribution of OpenFlow related events, which are generated by the Ryu OpenFlow library. It is used
for managing connections to OpenFlow switches, parsing OpenFlow messages and generating events
based on them. Due to the event handling, the applications can choose which OpenFlow events they
wish to receive. To enable precise event selection, OpenFlow events include the status of the switch
which they are caused by. Upon the connection of an OpenFlow switch, the event dispatcher is set to
the status HANDSHAKE_DISPATCHER. The library automatically processes the handshake and the switch
configuration messages, finally setting the event dispatch status to MAIN_DISPATCHER. In this status an
OpenFlow switch is ready for operations. Once the connection to an OpenFlow switch is lost the status
of the event queue changes to DEAD_DISPATCHER.
6.3.2 Limitations
Ryu is in development at the time of writing, the API is still changing. Because of that, the documentation
is not complete; nevertheless, the architecture and the basic working principles are described. The application
is small so that it can be understood by reading the source code. The OpenFlow implementation
is closely modeled after the OpenFlow standards [78, 79, 81], which can serve as documentation.
Besides, the message passing mechanism used has some limitations: it is unidirectional by nature,
making it difficult to use bidirectional communication patterns. For example, when OpenFlow messages
are sent, corresponding error messages are not automatically processed. An application needs to register
for error message events and implement a method for correlating error messages to outgoing messages.
As this prototype is assumed to be used in a static network, no errors should occur during the operations.
Therefore the limitations of Ryu are not relevant for this work.
6.3.3 Ryu Application Blueprint
Applications running on the Ryu OpenFlow controller have a specific structure that is introduced in this
section. Ryu applications consist of an application class and, for enabling a RESTful API, a controller
class. Representational State Transfer (REST) [30] is a Hypertext Transfer Protocol (HTTP) based Remote
Procedure Call (RPC) method for resource management. It is used because it is well structured,
easy to implement and to debug. An UML class diagram of a typical application skeleton is shown in
Figure 6.8. The application class inherits from ryu.base.app_manager.RyuApp. This marks the class for
instantiation by Ryu’s application manager. In order to access Ryu’s builtin applications, the names of
58 6. Prototype

the applications are added to a list that is maintained by the class for this purpose. During instantiation,
the application manager reads this list and supplies the needed objects as parameters to the application
classes constructor. In Figure 6.8, the DPset application is used by Application. The DPset application
maintains a list of connected OpenFlow switches 9 and their status. It is also used for mapping switch
IDs to switch objects that are used to send OpenFlow messages. For this reason all of the applications
described in this section use the DPset application.
wsgi.ControllerBase
app_manager.RyuApp
ryu.controller.dpset.DPset
ApplicationController
REST API
Application
Ryu Service Applications
Event handler methods
Application API methods
Figure 6.8.: UML class diagram of a typical Ryu application skeleton.
For the implementation of Application Programming Interfaces (APIs) RESTful web interfaces [30] using
JavaScript Object Notation (JSON) as data format are selected for ease of use. REST (representational
state transfer) is a remote resource management principle based on the HTTP protocol [29]. Resources
are identified by URIs (unified resource identifiers), the mode of interaction is set by the HTTP verb used.
The verb GET is used to retrieve information, POST to create a resource, PUT for changing it and DELETE
for deleting it. The POST verb is special, as it is the only one that is not expected to be idempotent. The
requirement for all other verbs is that the repeated usage with a specific resource and parameters does
not change the system after the first application.
Ryu’s implementation of a RESTful API application, WSGIApplication, can be accessed through the application
manager as described above. However, while WSGIApplications are managed by the application
manager, the class does not inherit from RyuApp. It enables the mapping of URIs and HTTP verbs to methods
of a controller class. The implementation is based on the implementation of the Web Server Gateway
Interface (WSGI) 10 standard by the gevent library. This controller class inherits from wsgi.ControllerBase
and is supplied by each application; in the example figure, it is called ApplicationController. The controller
class implements a method for each API function, handles incoming data from the API requests
and is responsible for sending response messages. Incoming data is encoded with JSON, the same is true
for response data, for which in addition a HTTP response code should be supplied.
A typical API call sequence starts with an incoming request for the WSGIApplication. The request’s URI
and HTTP verb are used to select which method of ApplicationController is called. The method evaluates
the incoming parameters, verifies their values and converts them from strings and JSON data to Python
data structures. Then, methods of the Application are called for further processing of the request. If the
called method returns result data, it is converted to JSON and sent out with an appropriate response
code.
The handling of events is done by methods of the application class, which are annotated with the event
type that they should receive, as described in Section 6.3.1. Incoming events are filtered by their class
9
OpenFlow switches are called datapaths in the context of Ryu
10
http://wsgi.org [Accessed 21.04.2013]
6.3. The Ryu OpenFlow Controller 59

and optionally by the status of the event source; further inspection of the event is implemented by each
method individually. For events that were generated from incoming OpenFlow messages, this typically
includes the check if the source switch is one managed by the application. If the event matches the filter,
further information can be gathered, and, finally, another method is called that implements the event
processing logic.
6.4 Implementation
In this section, the implementation of the RASP service is described. As explained in Section 6.1, it consists
of three components. For each component, the structure is illustrated using an UML class diagram
and an overview is given on its API. On this background, the process of the usage of an exemplary API
function is described. The parts of the prototype whose implementations are described in this section
are the RASP service application, the SDM application, the Virtual Peer application, and the topology
application as depicted in Figure 6.9
Network
Experiment
RASP Controller
with Service API
RASP Controller
with Service API
REST API REST API REST API
Topology
App
SDM
App
Virtual
Peer App
Network
Topology
Ryu OpenFlow Controller
Mininet
OpenFlow Network
REST API REST API REST API
Topology
App
SDM
App
Virtual
Peer App
Figure 6.9.: Overview over the parts of the prototype whose implementations are described in Section 6.4.
First, the RASP controller and its API is described in Section 6.4.1. Then, in Section 6.4.2, the topology
application running on the OpenFlow controller is presented. The two OpenFlow controller applications
that are used for implementing the RASP service, the Virtual Peer application and the SDM application
are explained in sections 6.4.3 and 6.4.4.
6.4.1 Rent-a-Super Peer Controller and Public API
The implementation of the RASP controller follows the API requirements described in Section 5.3.1. The
implementation covers the required functionality for evaluating the system. Other functionality such as
authentication and billing are not implemented, as they are not needed for this purpose. The API is
implemented with RESTful HTTP, as used with Ryu applications, because of its simplicity. Alternative
mechanisms such as web services are more complex yet have no advantages compared to the RESTful
approach for this specific application.
An application is identified by a base URI, in this case http://:/rasp. In the
remainder of this description, the /rasp part will be used synonymously for the whole URI including
the server address part. In Table 6.3 an overview is given on the controller’s API. A RASP instance is
identified by its IP address, a new one can be created by using the HTTP verb POST on the URI /rasp.
Information on the super peer are passed as HTTP parameters. These are the IP address, the transport
layer protocol, and the transport layer port number that is used for P2P overlay network connections.
The same port number is used by the RASP instance for accepting incoming overlay connections. The
60 6. Prototype

server replies with the IP address of the RASP instance, in this example the IP address is 192.168.0.1. An
example of the usage of the API is shown in Figure 6.10.
Super Peer
RASP Controller
VirtualPeerApp
SDMApp
POST /rasp
Parameters:
Super peer: 172.16.0.1:5000, TCP
Virtual peer port: 4000
200: Virtual peer address: 192.168.0.1
PUT /vp/192.168.0.1
Parameters:
Super peer: 172.16.0.1:5000
Service: TCP, 4000
200
PUT /rasp/192.168.0.1/streams/8000
Parameters:
Super peer port: 7000
200
PUT /sdm/192.168.0.1:8000
Parameters:
Data source: 172.16.0.1:7000
200
PUT /rasp/192.168.0.1/streams/8000/members/10.0.0.1:2000
200
PUT /sdm/192.168.0.1:8000/members/10.0.0.1:2000
200
Figure 6.10.: Example of a super peer using the RASP API to set up a video stream and adding a member to it.
The text above the line describes the HTTP request that is send, listed are the HTTP verb, the URI
and the respective parameters. Below the line is the HTTP return code followed by data contained in
the reply. Information on the RASP instance (ri) can be retrieved by using /rasp/. Since a
RASP instance can be the source of multiple streams, the list of streams is identified by the URI /rasp//streams. Using the GET verb with this URI returns a list of active streams, identified by their
source port.
The combination of an IP address with an associated transport layer port separated by a colon is called
a socket (e.g. "10.0.1.1:4000"). For each stream, appending /members to the URI and using the GET
verb returns the list of peers that receive this stream, identified by their socket. Using the PUT verb with
the URI /rasp//streams//members/ can be used for adding a
member to a stream.
Verbs URI Description
POST /rasp Creates a new RASP instance
GET /rasp/ Get the configuration information
on the instance
GET /rasp//streams Get the list of streams
GET,PUT,DELETE /rasp//streams/ Add/get/delete a stream
GET
GET,PUT,DELETE
/rasp//streams//
members
/rasp//streams//
members/
Table 6.3.: RASP controller service API overview.
Get the list of members
Add/get/remove a member
The RASP controller implementation consists of the service API implementation and the controller, a
part of the application that processes incoming requests and translates them into API calls to the Virtual
Peer and the SDM application. An example for this is given in Figure 6.10. In contrast to the other
components described in the remainder of this chapter, the RASP controller application does not run
in the context of the Ryu application manager. Hence, it runs its own HTTP server class, RASPServer,
which is implemented based on the pywsgi.WSGIServer class. Incoming requests are handled by the
class RASPController. This approach is similar to the one described in the Ryu application blueprint in
6.4. Implementation 61

Section 6.3.3. The application logic such as the management of the available IP addresses is implemented
in the class RASPManager. Most requests are processes and translated into requests to the Virtual Peer
and SDM application. The API calls to them are implements by the classes VPeerClient and SDMClient.
Many of the functions the RASP controller that would be implemented for a real deployment are not
needed for this prototype, like authentication and authorization. For that, the RASP controller implementation
is kept as simple as possible.
pywsgi.WSGIServer
wsgi.ControllerBase
RASPServer
RASPController
RASPController
REST API
RASPManager
VPeerClient: vpeer_client
SDMClient: sdm_client
API function
implementations
VPeerClient
Client for Virtual-Peer App
API
SDMClient
Client for SDM App API
Figure 6.11.: The structure of the RASP controller implementation.
6.4.2 Topology Application
Information on the topology of the OpenFlow network used is important for OpenFlow applications.
Some patches for Ryu exist, that could be used as basis for a topology discovery application using the
LLDP protocol. However, the result would be a graph of OpenFlow devices, lacking additional information
on the network such as the location or the type of the switches. As an ISP network is assumed,
information on the different network parts and switch types are required. This kind of information is not
offered by the existing topology discovery approaches, so it is not used. Instead, as topology database for
the OpenFlow controller application serves the data structure used for setting up the emulated network.
It provides a single coherent view of the network, which constitutes the basis for decisions the applications
make. The concept is similar to the one described in [35, p. 106]. For this, the API, as described
in Table 6.4, can be used to set the topology applications network information database. For transfer, an
instance of the lib.topo.RASPTopo class is used. It is serialized to JSON using the jsonpickle 11 library. The
RASPTopo class is a subtype of Mininet’s Mininet.topo.Topo class containing additional information on the
network and its devices.
Verbs URI Description
PUT /topology/mininet Set Mininet topology object
PUT /topology/control Starts/stops the topology application
Table 6.4.: Topology application API overview.
An overview of the implementation of TopologyApp is given in Figure 6.12. It consists of two parts, the
TopologyApp class, and the ForwardingManager class. The first stores the topology database object and implements
the REST API’s functionality. The latter implements the functions to enable the network’s basic
behavior which is defined by the topology database object. This object therefore controls the network’s
forwarding behavior.
11
https://github.com/jsonpickle/jsonpickle [Accessed 21.04.2013]
62 6. Prototype

wsgi.ControllerBase
app_manager.RyuApp
ryu.controller.dpset.DPset
mininet.topo.Topo
TopologyController
REST API
TopologyApp
RASPTopo: topology
set_topology(...)
start(...)
stop(...)
ForwardingManager
RASPTopo: topology
setup_lan(...)
setup_lan_forwarding(...)
setup_forwarding(...)
lib.topo.RASPTopo
Topology Information
EventTopologySet
RASPTopo: topology
openflow.topology10
Figure 6.12.: The structure of the topology application implementation.
If the topology database object is set, the TopologyApp emits an EventTopologySet event. At the time
of writing, Ryu offers a mechanism neither for accessing other applications nor for bidirectional event
passing. Hence, a simple method for disseminating the topology information to other applications is
needed. In order to achieve it, the EventTopologySet event object contains a reference to the topology
database object. Applications receiving the event are able to access at the whole topology database as it
is contained in the event object.
6.4.3 Virtual Peer Application
The Virtual Peer application provides a REST API for managing virtual peers and their clients (peers). The
basic application layout follows the description in Section 6.3.3. The relevant part of its class diagram is
shown in Figure 6.13. Methods for creating virtual peers are provided by the class VirtualPeerApp. It uses
a dictionary that maps IP address strings to VirtualPeer objects. These objects store the data for each peer
and provide methods for adding and removing peers. Peers can be lookup up either by their mapped port
or by a socket string.
wsgi.ControllerBase
app_manager.RyuApp
ryu.controller.dpset.DPset
VirtualPeerController
REST API
VirtualPeerApp
int: datapath_id
VirtualPeer: virtual_peers
add_virtual_peer(...)
del_virtual_peer(...)
packet_in_handler(...)
VirtualPeer
IPAddress: address
IPAddress: super_peer_address
int: service_protocol
int: service_port
add_peer(...)
del_peer(...)
1 0..* 1 0..*
Peer
Socket: socket
int: map_port
bool: confirmed
openflow.virtualpeer10
Figure 6.13.: The structure of the Virtual Peer application implementation.
An overview of the API of the application is given in Table 6.5 Before creating a virtual peer, the
OpenFlow switch to be used by the application needs to be configured. Then virtual peers can be added
by using their desired IP address as name and supplying the address of the super peer, the transport
6.4. Implementation 63

protocol and the port number to be used by both. When a virtual peer is added, an OpenFlow rule is
installed on the switch that matches packets addressed at the associated virtual peer. For identification
of the packets, the destination address, the service port, and protocol are matched. The rule’s action is
to forward packets to the OpenFlow controller.
Events matching an OpenFlow PACKET_IN message are processed by the VirtualPeerApp objects
packet_in_handler method. This method checks if the incoming packet matches the data of the virtual
peer. If it does, the packets source address and port are used for adding a new peer to the virtual peer.
The add method of the VirtualPeer object installs rules for implementing the packet header rewrite actions.
The OpenFlow implementation of this functionality is a straightforward implementation of the rule
schema described in Table 5.6 in Section 5.3.2. No special adaption is necessary.
Verbs URI Description
PUT,GET /vp/datapath_id Set/get datapath id
GET /vp/vps Get the list of virtual peers
PUT,GET,DELETE /vp/ Create/get/delete virtual peer
GET /vp//peers Get the list of peers
PUT,GET,DELETE /vp//peers/ Add/confirm/get/remove a peer
Table 6.5.: Virtual Peer application API overview.
6.4.4 Software-Defined Multicast Application
The SDM application, SDMApp, implements the multicast features for the RASP service. Its UML class
diagram is shown in Figure 6.14. The application class is responsible for handling EventTopologySet
events, which are emitted by the topology application. When such an event is received, the contained
wsgi.ControllerBase
app_manager.RyuApp
ryu.controller.dpset.DPset
mininet.topo.Topo
lib.topo.RASPTopo
Topology Information
SDMController
REST API
SDMApp
RASPTopo: topology
set_topology(...)
MulticastTree
NetworkX.Graph: tree
str: root_switch
get_active_tree(...)
add_delivery_destination(...)
del_delivery_destiation(...)
str: root_switch
add_group(...)
del_group(...)
SDMManager 1 0..*
MulticastGroup
Socket: data_source
1 0..*
Socket: tree_root
add_member(...)
del_member(...)
MulticastMember
Socket: socket
str: switch
int: switchport
openflow.topology10
Figure 6.14.: The structure of the SDM application implementation.
RASPTopo object is passed to a newly created SDMManager object. This object is responsible for managing
multicast groups and so for implementing most functions of the applications API.
The SDMManager needs to know the name of the OpenFlow switch that constitutes the root node of
the multicast tree. Once it is set, the multicast tree is calculated as described in the next paragraph. It
64 6. Prototype

is stored as arborescence which starts at the root node and connects all access switches in the network.
Using a pre computed multicast tree for improved performance is an approach widely used in literature,
e.g. in [70]. When a new group is added by creating a MulticastGroup object, this tree is copied and used
as basis for the management of the active multicast tree. The tree management is implemented in the
MulticastTree class, of which each MulticastGroup object contains an instance. After a new group is set
up, an OpenFlow rule is created that matches the incoming packet stream and transforms it according
to the multicast-forwarding scheme.
Verbs URI Description
PUT,GET /sdm/datapath_id Set/get tree root datapath id
GET /sdm/groups Get the list of multicast groups
PUT, GET,
DELETE
/sdm/
Create/get/delete a multicast
group
GET /sdm//members Get the list of members
PUT, GET,
DELETE
/sdm//members/
Table 6.6.: SDM application API overview.
Add/get/remove a member
When a new member is added to a multicast group, it is identified by its receive socket. The receive
socket is the combination of IP address and UDP port where the member wishes to receive the multicast
packets. First, the switch and port is determined where this new member is connected to by using the
topology database. Using this information, a MulticastMember object is created and stored in the member
list. At the switch where the member is connected, the member switch, a rule is added that creates a copy
of each incoming multicast packet and writes the destination address and port to the correct values.
The forwarding of the multicast data is managed by the MulticastTree object. If the member switch of
the new member is not part of the multicast tree yet, it is added by calling the add_delivery_destination
method. Starting with this switch, the method traverses the multicast tree towards the root node. On the
way, switches that are not part of the active tree yet are identified. Then, for each of these switches, the
rules on the respective predecessor are modified to include them in the packet distribution tree. In [70,
p. 000098] the same approach is used.
An alternative to the approach described in the preceding paragraph is to calculate a new multicast
tree every time a member joins or leaves the multicast group. This approach has some disadvantages.
First, the new multicast tree calculated after changes to the group membership could be node disjoint
for a large number of the trees inner nodes. Therefore, the forwarding rules are removed from the inner
nodes of the old tree and rules added to the inner nodes of the new tree. This could lead to a large
number of rule changes required for a single group member joining or leaving. In contrast, the approach
used for the SDM application limits the number of changes to the number of nodes on the path between
the new member and the tree root. The calculation of a new tree is also computationally more complex.
Hence, it could lead to computing times that influence the whole Ryu controller, because it relies on
cooperative multitasking.
Multicast tree construction
The networks topology contained in the RASPTopo object is available as a Graph object of the NetworkX
12 library. It is used as database for computation of the multicast tree. The construction of mul-
12
http://networkx.github.com/ [Accessed 21.04.2013]
6.4. Implementation 65

ticast trees still is a research topic [60]. The approach for this implementation is to create a simple yet
robust method that solves the multicast tree problem.
The construction of the multicast tree is determined by the topology of the network. Multicast packets
exit the network through access switches. They traveled the core switches to get there. For selection of
the source of the multicast tree, several approaches exist. One approach is to use a dedicated part of the
network as multicast tree source and calculate a tree from there to all access switches. Another approach
is to set the source to the border or access router, where the video stream enters the network. The point
of entry might be stable when the video source is located inside the ISPs network. If it is located on the
Internet, the point can change any time due to routing policy changes of the neighbor networks. Because
the point of entry of the video stream is likely to change, this approach is more complex. It requires
dynamic changes to the routes of the network, still; it does not fundamentally change the system. Hence,
the approach with one static source node is used.
Internet
Tree Source
Hosting
Core
Core
Core Network
Core
Core
Core
Core
Access Access Access Access
Figure 6.15.: Multicast tree construction problem.
The multicast tree construction from a graph theoretic perspective is shown in Figure 6.15. The nodes
to be connected by the multicast tree are colored blue. The goal is to find a minimum spanning tree
for a subset of the nodes of the network graph. This problem is also called the Steiner tree problem,
which is known to be NP-hard. However, a number of approximations exist, one being the Ravi-Klein
algorithm [53]. It is an approximation for the node weighted Steiner problem, which – besides edge
weights – includes node weights in its calculation. This feature is useful for influencing the multicast tree
construction, for example by taking a switch’s load or the existing number of OpenFlow rules on it into
consideration. The algorithms complexity is logarithmic; no exact bounds are given on the accuracy of
the solution. The Ravi-Klein algorithm is used in the prototype because it is available as a NetworkX compatible
Python module as part of GenRev 13 . The other NetworkX compatible Python implementation of
a Steiner tree approximation available is Kuo’s algorithm [18], which is part of the GOGrapher project 14 .
However, this implementation does not work as expected; it fails on some small example graphs and is
not used because of that.
13
http://bioinfo.mc.vanderbilt.edu/GenRev.html [Accessed 29.04.2013]
14
https://projects.dbbe.musc.edu/trac/GOGrapher [Accessed 29.04.2013]
66 6. Prototype

In [70] the authors implement IP multicast OpenFlow forwarding with NOX 15 . They propose the
usage of the PRIM algorithm for calculating the multicast tree for its computational efficiency. However,
the PRIM algorithm calculates the minimum spanning tree of the network. Depending on the network
topology, the minimum spanning tree might differ significantly from the Steiner tree that is required
for multicast. This in turn might influence the efficiency of the SDM application. Hence, the simplified
approach is not used.
OpenFlow implementation
For identification of packets of a SDM group within the ISP, the group address and port are used as
source and destination socket. Hence, the OpenFlow rules used for forwarding on core switches and
the ones used on hosts access switches rely on the same OpenFlow matcher. The SDM group address is
denoted by G addr , the port by G port . The OpenFlow 1.0 match fields and their respective values are listed
in Table 6.7. The port number is set to the associated tree upstream port. Its usage is required, as the
packet-marking scheme does not allow the identification of the packet’s direction of travel, hence it has
to be identified by the incoming port.
In port L3 proto L3 src L3 dst L4 proto L4 src port L4 dst port
IP G addr G addr UDP G port G port
Table 6.7.: OpenFlow matcher G match for packets of SDM group G.
The rule for writing the packet header information to a multicast packet on an access switch are a
combination of the SDM matcher and the actions as described in Table 5.3 in Section 5.3.2.
For a description of the OpenFlow rules used for SDM multicast tree forwarding, consider step 1 of
the schematic cutout of a multicast tree in Figure 6.16. Switch Core 1 receives the packet stream of
SDM multicast group G though port 1. It forwards it though port 2 to switch Access 1, where peers are
connected that receive group Gs packet stream. Switch Access 2, which is connected through port 3 at
this time does not receive the packet stream.
SDM group packets
1
1
1
Core 1
Core 1
Core 1
2 3
2 3
2 3
Access
1
Access
2
Access
1
Access
2
Access
1
Access
2
(a) Step 1
(b) Step 2
(c) Step 3
Figure 6.16.: Schematic cutout of a SDM multicast tree with three different forwarding states.
For step 1, the OpenFlow rule of switch Core 1 is shown in Table 6.8. The matcher of group G is
denoted by G match . The rule contains one action, which forwards incoming packets to port 2. In step 2,
a host connected to switch Access 2 joins the multicast group. The request is processed and switch Access
2 is added to the multicast tree. The existing OpenFlow rule is overwritten; the rule now includes two
15
https://github.com/caioviel/CastFlow [Accessed 21.04.2013]
6.4. Implementation 67

actions, forward to port 2 and forward to port 3. Then, all clients connected to the switches Access 1
and 2 leave the group. The change is processed and the two access switches as well as switch Core 1 are
removed from the tree. The SDM multicast rule is removed from switch Core 1.
Step Matcher Actions
1 G match Forward: Port 2
2 G match Forward: Port 2, Forward: Port 3
3 No rule
Table 6.8.: OpenFlow rules during the three steps of the SDM tree forwarding example.
68 6. Prototype

7 Evaluation
The goals of RASP are twofold: support P2P LVS systems and help ISPs to cope with P2P traffic. The focus
of this evaluation is on the P2P LVS service prototype’s behavior from the perspective of the ISP offering
it. The transmission efficiency and the influence of different network topologies on it are investigated.
The resource requirements of operating the P2P LVS service are evaluated by analyzing the required
number of OpenFlow rules in the network and the system’s management overhead.
The improved control of the P2P LVS traffic is given by marking and separating the P2P traffic that
runs through the RASP service. Subsequently, the ISP can apply traffic engineering. Further evaluation of
this topic depends on the ISP network architecture, which is beyond the scope of this thesis. For P2P LVS
systems, on-demand super peer functionality increases the stability and scalability. The beneficial effect
of super peers and especially high upload bandwidth is well documented, as described in Section 2.1.2
and 3.2. Evaluating the effects of the increased upload bandwidth on complete P2P LVS systems would
require such a system to be simulated or emulated. Such an analysis requires a different evaluation
environment than the one used for analyzing the transmission efficiency. Therefore, the investigation of
the impact of the system on P2P LVS systems is beyond the scope of this thesis.
The remainder of this chapter is organized as follows: the scenario used throughout this chapter is
introduced in Section 7.1. The evaluation of the transmission efficiency and its results are presented in
Section 7.2. The network resources required for operating the RASP service are described in Section 7.3.
The results of this chapter are summarized in the conclusion in Section 7.4.
7.1 Scenario
The goal of the evaluation is twofold: first, comparing the transmission efficiency of P2P LVS systems
with and without using the RASP service. The second goal is analyzing the influence of the ISP’s network
topology on the transmission efficiency. Therefore, ISP network topologies and P2P LVS overlay
network topologies, which are representative of the field of application, are selected for comparison. The
characteristics of both ISP networks topologies and P2P LVS overlay topologies are active research topics
and comprise a large number of elements in their real world application. The used emulation system is
capable of handling ISP and P2P LVS overlay networks of sizes of up to a few hundred nodes. However,
real systems are often much larger, especially overlay networks might consist of thousands of nodes.
Hence, for evaluation the sizes of the systems have to be adapted to the available resources. Models are
introduced that include certain aspects of real networks at reduced sizes.
In Section 7.1.1, ISP network topologies and common simulation approaches are discussed. The influence
of technologies used in access and aggregation networks is presented in Section 7.1.2. Both
considerations are summarized in Section 7.1.3, where the network models used for evaluation are described.
The approach used to build overlay networks mimicking real P2P LVS system in this evaluation
is discussed in Section 7.1.4.
69

7.1.1 ISP Core Networks
The characterization and analysis of real world ISP networks is an active research area. For many ISP networks,
some details on their topology exist as described e.g. in [54]. However, the available information
differs in its details and level of abstraction.
Today’s networks often use IP addresses or IP capable devices in the core network only. Most research
focuses on this area because it can be remotely analyzed without depending on the network owner. For
this evaluation, only the plain topology is relevant. Additional information such as link capacities and
lengths are not relevant and not included in the model that is sought after for this evaluation.
Networks are analyzed on different levels of detail: on the router level or on the Point of Presence (PoP)
level. PoP level topologies contain the relevant structural information such as long distance network
links needed for the topology model. Router level information would also include details on the internal
structure of PoPs. For analyzing the systems performance and transport efficiency these details are not
required. Hence, PoP level topologies are discussed from here on, the PoPs are also referred to as nodes.
The research shows a wide range of ISP network topologies and characteristics. It is difficult to identify
phenomena that are characteristic for such networks [54, p. 1773]. Often example networks [24, p.
1529],[107, p. 1166] or network generators such as TIERS 1 , GT-ITM 2 , or BRITE [73] are used for experiments.
However, the goal of the generators is mainly to generate large networks and they require a large
set of parameters to be selected for customization of the generated network topologies. Hence, parameters
settings are required that reflect the characteristics of real networks. Some information on parameter
settings is available in [37]. They are specifically matched to topologies of the German research network
(Deutsches Forschungsnetz, DFN) and AT&T. However, DFN is not an ISP with residential broadband
customers, AT&T has, but its dominant role is the one of an Internet backbone provider. Hence, both do
not represent the target audience of the RASP service. Other approaches from literature include an old
long distance network of the USA [116]. However, it is not clear what kind of network it represents and
if the modeled network is still representative. Therefore, a simplified topology of a real network is used
for this evaluation, closely following publicly available information on characteristics of ISP networks.
Information on the German core network topology of Deutsche Telekom (DT) is available [26, p. 4].
Deutsche Telekom has a large number of residential customers and hence is part of the target audience of
RASP. It is hierarchical organized as depicted in Figure 7.1. Beginning from the highest hierarchy level,
the inner core features three and the outer core nine PoPs. To these 63 regional nodes are attached,
which constitute the lower hierarchy level. Small groups of regional nodes are each connected to a node
of the outer core. Each node has a direct link to the outer core node and one or more links to its next
neighbors. Those links add an extra hop for paths with destinations on other core nodes. Hence, their
primary task is assumed to provide redundancy. The structure of the regional node groups is virtually
tree shaped. The basic shape of the inner core is a triangle, the outer core adds PoPs by adding links to
two different inner core locations. The whole network shape resembles a so-called snow flake structure.
A similar shape is used for evaluation of MPLS traffic engineering approaches in [117, p. 9-11]. The core
is a triangle; the rest of the network is tree-shaped. The authors argue that this reflects the fact that
only the regional PoPs are generating revenue and that any additional links introduce costs, which are
avoided by ISPs if possible.
Based on these two network models, the ISP network model described in the remainder of this section
is used for evaluation. The model should be scalable while keeping certain characteristics of the reference
1
http://www.isi.edu/nsnam/ns/ns-topogen.html#tiers [Accessed 28.04.2013]
2
http://www.cc.gatech.edu/projects/gtitm/ [Accessed 28.04.2013]
70 7. Evaluation

5 regional
nodes
7 regional
nodes
6 regional
nodes
10 regional
nodes
Outer Core
5 regional
nodes
10 regional
nodes
Inner Core
5 regional
nodes
8 regional
nodes
3 regional
nodes
5 regional
nodes
Figure 7.1.: Deutsche Telekom’s core network [26, p. 4].
topologies. The network topology model is shown in Figure 7.2. It consists of two parts, a part made up
of triangular shapes and a part made up of tree shapes. The minimal network consists of a single triangle
with one tree node connected to each of the triangle’s nodes. The size of the network can be increase
either by adding nodes in the triangular part or the tree part. In the triangular part, nodes are added
by attaching them to two nodes of the starting triangle, resembling the topology of Deutsche Telekom.
Nodes can be added to the tree part by adding them to any of the existing trees. For easy generation and
handling, all size variations of the model are done symmetrically.
Scale the triangular network part by linking
a node to two of the starting nodes
Scale the tree part by increasing
its depths
Scale the tree part by increasing its degree
Figure 7.2.: The network topology model used as basis for network topology with annotations on the scaling
possibilities.
7.1. Scenario 71

7.1.2 ISP Access and Aggregation Networks
Access and aggregation networks are fundamentally different from core networks. Core networks are
researched because they are part of the public reachable Internet and run the IP protocol. Often this is
not the case in access and aggregation networks. This has an impact on the applicability of OpenFlow
in these network areas, because network protocols used are often not support by OpenFlow. Hence,
access and aggregation networks need to be analyzed in more detail to assess the impact on the network
topology model.
Access networks are mostly implemented using technologies, which are specifically developed for this
application domain. Today, the prevalent technology used in access networks is DSL, as described in
Section 2.2.2. Ethernet based OpenFlow technologies are rarely used, which rules out the application
of the Ethernet-based OpenFlow. Aggregation networks are implemented with Ethernet today, however
the technology used for packet transport is influenced by the technology used in the connected access
networks. The applicability of Ethernet is shown in Figure 7.3 on the left.
Core PoP
BRAS
Use of Ethernet
possible and likely
Aggregation switches
PPP encapsulation
Use of Ethernet
unlikely
...
...
Access devices, eg. DSLAM
...
...
Figure 7.3.: Access and aggregation networks: Ethernet application and PPP encapsulation.
Customer network traffic in DSL networks is often managed by Broadband Remote Access Servers
(BRAS). Such a device is often placed in the core PoP, aggregating residential subscriber traffic and
managing the offered network services. Their tasks include authentication and IP address assignment [9,
p. 8]. In order to differentiate the traffic from different subscribers, it is encapsulated on its way to the
BRAS. Today, the encapsulation is enabled by creating a PPP session between the subscribers residential
gateway and the BRAS as shown in Figure 7.3 on the right.
The SDM component of the RASP service, proposed in this thesis, relies on manipulation of the IP
header of packets. For manipulating the protocol header of a network packet with OpenFlow the protocol
to be manipulated and all underlying protocols must be supported by the OpenFlow standard
specification. Besides the standard protocols of the TCP/IP network stack like IP, TCP, and UDP, Open-
Flow version 1.0 supports the manipulation of VLAN tags. In addition to that, OpenFlow version 1.2
specifies the manipulation of MPLS tags. PPP is not a supported protocol, which prevents the manipulation
of other protocol headers which are encapsulated within PPP such as the IP header. Hence, the SDM
component of the RASP service can only be used in access networks if Ethernet is used as data link layer
protocol and the subscriber traffic is encapsulated in a supported protocol header or not encapsulated at
all.
Starting from today’s DSL-based standard configuration, possible OpenFlow compatible settings are
explored. The placement of the BRAS in a core PoP is the standard configuration today (e.g. [52, p.
5],[50, p. 413],[34, p. 355]). An alternative approach, called distributed BRAS, is also available, in
which some of the BRAS functionality is moved from the core PoP to locations nearer to the DSLAMs
72 7. Evaluation

[42, p. 10,11], thereby enlarging the IP-based network. However, industry reports from 2005 [9, p. 6]
and virtually none more recent literature on this topic suggest that their distribution is limited.
Another approach for enabling OpenFlow in access networks is to forgo subscriber traffic encapsulation
and use Ethernet as data link layer protocol. For the latter, a number of approaches exist. A required
functionality in the network between the DSLAM and the BRAS is subscriber isolation. Today, this is often
implemented by using PPP. Alternative approaches exist: the Broadband Forum suggests using Ethernet
with VLAN tagging [2]. In [63], an approach called seamless MPLS is introduced. The authors propose a
MPLS architecture that also spans access networks. In both scenarios PPP over Ethernet (PPPoE) can be
used as encapsulation. An alternative encapsulation exists: IP over Ethernet (IPoE), which forgoes PPP. It
is standardized in [2], and further detailed in WT-146, which is a work in progress and hence not freely
available. When using IPoE, DHCP is used for IP address management, while a number of other features
PPPoE offers are not available [12]. Literature suggests it is deployed in production [48, p. 4].
Approaches for implementing ISP networks with OpenFlow are researched, with SPARC [52] being
one research project in this area. Different scenarios for OpenFlow usage are discussed, including implementing
PPP-based BRAS functionality and an approach on using IPoE/DHCP with OpenFlow [3].
The overview of current research literature on technologies used, propose that PPP-based aggregation
networks are used in the future, even in OpenFlow-based aggregation networks. Also, IPoE/DHCP approaches
are used and it is assumed that they will see an increasing adoption in the future. Furthermore,
approaches on moving the BRAS from the access network towards the access network exist and are likely
to be used more broadly in the future. Aggregation networks are mostly logically tree shaped. Hence,
the impact of changing the tree depths at the borders of the ISP network is analyzed in this evaluation.
This gives first hints on how the RASP system would behave if the BRAS location is changed or if it is not
used at all.
7.1.3 Applied Network Topology Variations
In Section 7.1.2 and Section 7.1.1 topologies for core and access networks are discussed. A model for
provider networks is presented, based on the network topology of Deutsche Telekom (DT). The nodes or
PoPs in the model are represented by OpenFlow switches. For evaluation, a number of instances of this
topology are chosen and features for running the RASP service are added. These include hosts on which
P2P LVS peers can run, a hosting node for implementing the RASP service and a network part mimicking
the Internet. The instances of the network model and their selection are described in this section.
For illustration, a small example instance of the network topology model exhibiting all relevant features,
called DT small, is shown in Figure 7.4. The hosting switch is connected to a switch of the core
triangle. Hosts are not directly connected to core switches, but to access switches. To emulate the effects
of the interaction between hosts inside the ISP network and on the Internet, Internet switches and hosts
are added. Each switch of the ISP network part’s triangular core is connected to one Internet switch.
The three Internet core switches are interconnected and one Internet access switch is connected per core
switch. One half of the hosts in the network is located in the ISP part, the other half is located on the Internet
part. The topology contains a total of 13 switches. In Figure 7.4 the ISP core switches are depicted
in orange, the access switches in blue; Internet switches and hosts are depicted in gray.
Three network topologies with the same amount of hosts are used for assessing the influence of their
characteristics on the efficiency of the RASP service. Each network includes 180 hosts, 90 of which
are located in the ISP network and 90 on the Internet. Larger networks were tested on the available
hardware, but could not be used due to excessive packet loss when running an emulated P2P LVS overlay
network. The total of 180 hosts is chosen for its adequate performance requirements and because it
7.1. Scenario 73

5 H
Access switches
Hosting switch
Core switches
ISP
Number of connected hosts
5 H 5 H
Internet
5 H 5 H
5 H
Figure 7.4.: Network model instance DT small.
allows the three topologies to contain the same number of hosts. 150 hosts in the ISP network and the
same number of hosts in the Internet part, running a P2P LVS emulation resulted in high packet loss due
to an overload situation computer system running the test.
The largest topology that is used in this thesis contains 34 switches. The video stream used for
evaluation has an average bit rate of 150Kbit/s, which is discussed in Section 7.1.4. For an upper
bound of the throughput per second, it is assumed that each host uses the full video bandwidth
on every switch in the network. This yields an upper bound for the total throughput requirement of
180 ∗ 0.150Mbit/s ∗ 34 = 918Mbit/s. These are upper bounds; the real throughput requirement is expected
to be much lower. The maximum network diameter of the network topologies described later in
this section is 7. Assuming that the video stream crosses 7 switches to reach each host, this lower bound
yields 180 ∗ 0.150Mbit/s ∗ 7 = 189Mbit/s.
The resulting numbers of this consideration are not consistent with the anticipated performance based
on the analysis of the switch performance in Section 6.2.2. This is true even if the high processing
resource usage of the P2P LVS emulation is considered. The performance test showed an aggregated
throughput of 1.3Gbit/s for 10 switches, tests for larger networks where not conducted. Some results
can be deducted from [59, p. 4]. In the evaluation described there, the aggregated throughput of the
Open vSwitch software drops by 40% when the number of switches is increased from 10 to 20. When
the number of switches is increased from 10 to 30, the aggregated throughput drops by 70%. Assuming
the same behavior of the hardware used in this thesis, the results suggest a maximum throughput of
1.3Gbit/s ∗ 0.3 = 0.39Gbit/s. This value is twice as high as the upper bound derived for the throughput
performance required for this evaluation. Hence, the network throughput is unlikely to be the limiting
performance metric in this setting. Further analysis of these performance considerations is beyond of the
scope of this work.
Each network topology instance starts from the DT Small instances and scales its size by taking a
different approach. All topologies scale the number of access switches per core node. One approach is
74 7. Evaluation

to add more hosts per access switch, which is taken for DT Hosts, shown in Figure 7.5. 10 hosts are
connected to each access node and five access nodes are connected to each core node, for a total of 19
switches. The ISP network part consists of 13 switches and has a network diameter of five. This topology
represents networks with high user density in this comparison.
10 H
10 H 10 H
10 H
10 H
10 H
10 H
10 H
10 H
Internet
30 H 30 H
30 H
Figure 7.5.: Network model instance scaled by increasing the number of connected host per access switch (DT
Hosts).
Another approach is to add more core switches, similar to the outer core nodes in the DTAG network.
Three core nodes are connected with two access switches each for DT Core, shown in Figure 7.6. At each
access switch, six hosts are connected. The topology consists of 28 switches; the ISP part consists of 22
switches and has a network diameter of six. This topology represents networks with a high link density
in the core.
Finally, scaling can be achieved by increasing the depths of the tree-shaped part of the network. For DT
Depth, at each core node two switches are added as shown in Figure 7.7. 5 connected hosts per access
switch yield the same number of hosts as DT Hosts, but with an additional tree level and a total of 34
switches. The ISP network consists of 28 switches and has a network diameter of seven. This topology
represents networks with a high ratio of tree-shaped network parts. ISP networks where OpenFlow can
be used in the complete aggregation network parts, as described in Section 7.1.2, are represented by this
topology.
A simple routing model is used for the network topologies. Each access switch is associated with an
IP network. A forwarding path is created for each pair of switches and their associated networks. The
selection of the paths is done by calculating the shortest path between the two access switches. Each
link in the network topology is assigned a length of 1. Exceptions are the links in the Internet part of
the network. The links between the Internet core switches are given a length of 3 to enforce a balanced
usage of the links between the ISP and the Internet network part. To these links, a length of 5 is assigned
7.1. Scenario 75

6 H
6 H 6 H
6 H
6 H
6 H
6 H
6 H
6 H
6 H
6 H
6 H
6 H
6 H 6 H
Internet
30 H 30 H
30 H
Figure 7.6.: Network model instance scaled by adding core nodes (DT Core).
to ensure that routes between two hosts in the Internet part do not use links from the ISP network part.
The link length is also used for the creation of the overlay network, which is described in Section 7.1.5.
The generated networks used in this evaluation are assumed static, they do not change during the
time the network emulation is running. In addition, no bandwidth restrictions are applied. For a first
basic evaluation of the RASP concept, this setting is assumed sufficient. Evaluations going beyond these
topologies are considered future work.
7.1.4 Peer-to-Peer Live Video Streaming Model and Emulation
The P2P LVS emulation defines the workload of the evaluation. It runs on the network topology described
in the last section. No software is available for running P2P LVS systems in the chosen network emulator
setting. Therefore, the P2P LVS system behavior is emulated with custom software. The emulation consists
of three parts: peers, an overlay topology and a video stream. Peers run on the emulated network,
they are configured by the evaluation system. Their main task is to establish network connections to their
neighbors and forward the video stream. The overlay topology is created by the evaluation system before
configuring the peers, its design is discussed in Section 7.1.5. Then, a video is streamed to the root node
of the overlay topology, which forwards it through the overlay network.
Besides the overlay network approach, the hosts, which are included in the P2P LVS system, can be
used to influence the systems workload. In a real system, the users of such systems are expected to be
randomly distributed in the network. This behavior is simulated by choosing 80% of the hosts for the
operation of peers of the P2P LVS system. The peer selection for each topology is done randomly during
initialization of an overlay network emulation.
76 7. Evaluation

5 H
5 H
5 H
5 H
5 H
5 H
5 H
5 H
5 H
5 H
5 H
5 H
5 H
5 H
5 H
5 H
5 H
Internet
5 H
30 H 30 H
30 H
Figure 7.7.: Network model instance scaled by increasing the trees depths (DT Depths).
The overlay topology is calculated in advance, the peers are configured with their neighbors and their
associated roles. Each peer creates a TCP network connection to each of its neighbors. Small messages
are send over these connections every few seconds, no functional information is exchanged. Their main
use is to verify if the RASP system is working correctly during the experiments.
The video player and streaming application VLC 3 is used for streaming a video in the P2P LVS overlay
network using UDP as transport protocol. A video 4 , which is encoded for an average bit rate of 150kbit/s,
is used for the stream. It is small enough to allow its use with a larger number of peers in the available
emulated network environment. Still, it is large enough to be a viewable video and to create a sufficient
amount of network traffic for measurement. Obtained qualitative results are assumed representative also
for streams with higher bit rates and scenarios with more peers.
7.1.5 Overlay Topology Generation
As introduced in Section 7.1.4, a tree-based P2P LVS overlay is assumed. A method for the construction
of the emulated topologies is presented in this section. RASP should be most suitable for systems with
centralized control. Hence, we assume an operator who employs RASP for the streaming, e.g. of a live
event. The video stream source is placed somewhere on the Internet as part of the respective topology.
The ISP represents – compared to the whole Internet – a small number of users. Hence, a significant
3
http://www.videolan.org/ [Accessed 29.04.2013]
4
http://www.bigbuckbunny.org/ [Accessed 21.04.2013]
7.1. Scenario 77

number of peers connect to the system before the first peers inside the ISP join to account for this
characteristic situation.
For simplicity of the emulation, a centralized overlay network management is assumed. The scheme
used is based on the one used by CoopNet [87], whose operation is described in this paragraph. A peer
willing to join the video distribution tree requests information on potential upstream peers from a central
authority, which aims to create a balanced distribution tree. Therefore, the lowest level with available
capacity in the tree is chosen for connecting the new peer. From this level, peers with available capacity
are determined and one is randomly selected and proposed to the new peer. The new peer joins the
network, receives the video stream and informs the overlay manager on its upload capacity.
A method adapted from the approach of CoopNet is used as a model for simple P2P LVS systems that
are underlay network oblivious. This overlay topology algorithm is called Random. Many P2P LVS systems
consider information on the underlay network during the construction of their overlay network. In
these systems, the upstream peer is not selected randomly but by estimated virtual or physical distance
or location. Such a representative approach is used in this evaluation; the algorithm is called Underlayaware.
The basic approach is similar to the one of CoopNet. It differs when a new peer is inside the ISP
network. In this case, upstream peers from the same network are preferred 90% of the time. From the
available peers the one with the shortest distance is chosen. In the other 10% of the requests, from the
three closest peers located on the Internet part of the network, one is chosen randomly. This concept
is inspired by P2P LVS systems that rely on network information services such as ALTO [11]. Such systems
can provide distance information on nodes inside the ISP network to support ISP-friendly overlay
topology constructions. For distance measurement on the Internet, different approaches exist like determining
the hop count or latency measurements between two nodes. For this emulation, the path length
information available from the network topologies is used in both cases.
A method based on the Underlay-aware overlay algorithm is used as model for P2P LVS systems that
integrate the RASP service. The overlay algorithm is called RASP, it assumes an existing RASP instance
for its operation, which is always a directly connected peer of the video source. The RASP algorithm
differs from the Underlay-aware algorithm in the upstream peer selection. In 90% of the cases, the RASP
instance is chosen as upstream peer if a new peer is located in the ISP network. The preference of
the RASP instance is achieved by setting the distance in the network information service between the
RASP instance and all other hosts in the ISP network to zero. In the other 10% of the cases, from the
three closest peers at the right level, one is chosen randomly. The random selection approach reflects
a worst-case scenario for tree-based systems from the perspective of transmission efficiency, while the
Underlay-aware approach exhibits features that it shares with underlay-aware P2P systems.
The simulated P2P networks are small compared to real systems with thousands or millions of users.
Many P2P systems prefer a shallow tree to reduce the transmission latency. The out degree of peers is in
many cases restricted by their network upstream capacity. Given that DSL is the prevalent access network
technology and its asynchronous transmission capacity, each peer is assumed to have a small number of
neighbors to which they transmit the video stream to. The alternative approach of using large numbers
of neighbors would lead to trees with only a few levels of depth because of the small number of peers
available. This setting would not be representative and could increase the variability of the results, as the
few randomly selected peers near the tree root could have a big influence on the systems performance.
The advantage of a RASP instance is a high out degree. However, with a small number of peers between
tens and hundreds and an out degree, which covers a significant part of all peers, the advantage of a
RASP instance would not appear as it is expected in real deployments. To mitigate this, an out degree of
four is used for peers in the generated overlay network topologies. It ensures a tree depth of at least two
levels for less than 50 peers.
78 7. Evaluation

H
H
H
H
H
H
H
H
H
H
H
H
H
Access
H
H
H
Access
H
H
H
Access
H
H
H
Access
H
H
H
Hosting
Core
Core
Access
H
H
Hosting
Core
Core
Access
H
Core
H
Core
H
H
INTERNET
H
INTERNET
H
INTERNET
INTERNET
INTERNET
H
H
INTERNET
INTERNET
INTERNET
H
H
INTERNET
H
H
INTERNET
H
H
H
INTERNET
H
H
H
INTERNET
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
(a) Random
(b) Underlay-aware
H
H
H
H
H
H
H
Access
H
H
H
Access
H
H
H
Hosting
Core
Core
Access
H
Core
H
H
INTERNET
H
INTERNET
INTERNET
INTERNET
H
H
INTERNET
H
H
H
INTERNET
H
H
H
H
H
H
H
H
(c) RASP
Figure 7.8.: Overlay network algorithm examples based on the DT Small topology.
The analysis of the three overlay network algorithms, Random, Underlay-aware, and RASP, is described
in the remainder of this section. For comparison of the random algorithm for overlay network construction
and the Underlay-aware supported approach the DT Small network topology is used with 80% P2P
usage and 24 peers in total. The lengths of network links are as described in Section 7.1.3. Inside the ISP
network they are 1, the network links between the ISP and the Internet have a length of 5. The lengths
of links between Internet switches are 3. The three overlay network algorithms are run 100 times. The
average path lengths of overlay links inside the ISP network and the percentage of overlay connections
crossing the network boundaries are used for comparison.
7.1. Scenario 79

For illustration, example overlay networks that were generated by the three algorithms on the small
DTAG network topology are shown in Figure 7.8. Blue arrows denote overlay network links; the direction
of the arrows represents the direction of the video stream. Hosts are depicted by boxes with an “H”,
switches by ovals. Other edges are network links; the Internet part of the network is depicted in gray to
emphasize the ISP network part. The different properties of the overlay networks algorithms are clearly
visible in the depicted networks. The number of edges crossing from the ISP to the Internet part is much
smaller for the Underlay-aware and the RASP approach. Furthermore, the overlay links are more uniform,
long edges occur less often. For the RASP algorithm, the virtual peer is represented by the green hosting
switch.
1.00
1.00
Cummulative probability
0.75
0.50
0.25
0.00
0 1 2 3 4 5
Average overlay link lengths inside
the ISP network
0.75
0.50
0.25
0.00
0.0 0.2 0.4 0.6
Average ratio of network border
crossing overlay links
Overlay algorithm: Random RASP Underlay−aware
Figure 7.9.: The cumulative distribution functions of two characteristics of the three P2P video overlay tree algorithms.
The average overlay link path lengths and the average ratio of overlay network links crossing the ISP
network’s border are depicted in Figure 7.9. The Underlay-aware overlay algorithm improves the overlay
network with respect to the overlay link length compared to the Random algorithm. The average edge
length inside the ISP network is reduced by a small amount and the spread in reduced. This behavior
can be explained by the nature of the network topology. If no peer is available on the same access switch,
other peers inside the ISP network are at least 5 hops away. The small reduction of the average means
that peers connected to the same access switch are chosen more often than in the Random algorithm.
The average path length of the RASP approach is zero for a third of the generated network topologies.
This is due to the RASP instance being chosen as upstream neighbor by many peers in the ISP network;
its distance to all other hosts in the network is defined as zero.
The average ratio of overlay links that cross the network boundary is nearly 50% for the Random approach,
about 30% percent for the Underlay-aware and about 20% for the RASP approach. The spread
is similar for the Underlay-aware and the Random algorithm, the spread of the RASP algorithm his significantly
higher. Still, the Underlay-aware and RASP approaches yield a significant lower percentage of
network crossing links than the Random approach.
The number of cross-border overlay links is significantly reduced, as it is expected for underlay aware
P2P LVS approaches. For the Underlay-aware approach, the average overlay link length is slightly re-
80 7. Evaluation

duced. However, due to the characteristics of the topology used for comparison, this is to be expected.
Lastly, in the RASP approach, many peers in the ISP network use the RASP instance as upstream neighbor,
as it should be.
7.2 Transmission Efficiency
Different emulated P2P LVS overlay networks, one with and two without RASP integration, are used
to analyze the effect of the RASP service on the transmission efficiency inside an ISP network. Furthermore,
three network topologies are used to analyze the influence of their different characteristics
on the transmission efficiency. Finally, the impact of the RASP service on the overlay path lengths is
assessed. The prototype and the emulated network described in Chapter 6 are used for emulating the
system. The metrics used for this evaluation are discussed in Section 7.2.1 followed by the description
of the implementation measurements in Section 7.2.2. The results of the measurements are presented in
Section 7.2.3.
7.2.1 Metrics
The RASP service should improve the efficiency of the P2P video stream transport in an ISP network.
From the point of view an ISP, the transport efficiency of a system can be measured by the data volume
that is send through its network when the system is used. A distinction is made between the data volume
that is transported within the ISP network and the volume that is exchanged with neighboring networks.
In addition to resource usage, the latter causes costs as described in Section 2.2. The data volume is
influenced by the number of peers that reside inside the ISP network. For that reason, the transmitted
volume inside the ISP network is divided by the peers residing there. This method shows the resource
consumption for a P2P system for an ISP network with and without the RASP service. Furthermore, it
allows to analyze the improvement per peer which is used for comparison with the costs per peer of
the system. A direct comparison of the data volume of different network sizes is not possible with this
approach, because the size of the network influences the number of links which the data is send over
and hence the whole data volume.
The set of all devices in the network is denoted by D. Nodes n ∈ N ⊆ D and peers p ∈ P ⊆ N are
indexed by their location, ISP or Internet. Links l ∈ L ⊆ D × D are considered bidirectional and can be
denoted by the location of their adjacent nodes. Hence links inside the ISP are denoted by L ISP , links
crossing the network boundary by L ISP,Internet . The data volume send through the network is calculated
by summing the data volume transmitted in both directions on each link v l ∈ V of the network: v ISP =
∑
l∈L ISP
v l . The data volume that crosses the network boundary is the sum of the data volume transmitted
in both directions of links that cross it: v ISP,Internet = ∑ l∈L ISP,Internet
v l . The equation for calculating the
intra-domain data volume per peer (ivp) is shown in Equation 7.1, the one for calculating the network
boundary crossing data volume per peer (bvp) in Equation 7.2.
Intra domain data volume per peer: ivp(L, P, V ) =
|P ISP |
∑
Network boundary crossing data volume per peer: bvp(L, P, V ) =
∑
l∈L ISP
v l
l∈L ISP,Internet
v l
|P ISP |
(7.1)
(7.2)
7.2. Transmission Efficiency 81

P2P networks are often evaluated by metrics like link stress [44, p. 6]. Link stress is defined as the
number of copies of the same packet that are transmitted over a link. Based on this metric other metrics
such as resource usage, the sum of the link stress of active links multiplied with its delay, are defined.
However, these metrics aim at measuring the properties of P2P systems. The metrics are less relevant
from the point of view of an ISP.
To assess the impact of the RASP service on the latency of the video delivery, the differences in the
overlay link lengths with and without using the service are investigated. The ratio of overlay connection
path lengths and direct unicast path lengths is called link stretch [14]; often the paths latency is used as
measure for the link lengths. Here, the lengths of P2P LVS connection paths in the respective network
topology are compared with the corresponding direct unicast connection path. The forwarding inside the
ISP network is done by the same OpenFlow switches for all traffic. Therefore, except for other network
traffic, the length is the only factor that influences the latency of a network path. The relevant paths
are the ones from the video source to the peers that consume the video. A link l a,b ∈ L from node
a ∈ D to node b ∈ D has length len(l a,b ). A network path, or route, from device a to b is a set of links
r a,b ⊂ R ⊆ L. Overlay connections o ps ,p ∈ O ⊆ 2 R with p s , p ∈ P are a set of overlay links, represented
by routes r a,b ∈ o ps ,p, that connect a peer to the source peer p s of the video stream o ps ,p ⊆ R. The link
stretch for the video stream delivery is defined in Equation 7.3.
∑ ∑
r∈o p s,p l∈r len(l)
Link stretch : ls(L, P, O) = ∑l∈r len(l) (7.3)
p s,p
7.2.2 Implementation
An overview of the different parts of the emulated system as well as the methods and tools for implementing
them is given in Table 7.1. The system relies on the network emulation and prototype described
in Chapter 6. For other parts, such as the emulated P2P LVS system, custom Python programs are used
that run on the emulated Mininet hosts.
System/Component Method Tools
P2P LVS system Emulation Python program/Mininet control/VLC video streaming
Forwarding/Routing Prototype/Emulation RASP prototype, Mininet/Ryu program
Network Emulation Mininet
Table 7.1.: Overview of the methods and tools used for evaluation.
For each of the three network topologies (DT Hosts, DT Core, DT Tree), three overlay algorithms
(Random, Underlay-aware, RASP) are used to generate 30 different overlay topologies. The combination
of a network topology and one overlay network generated by an overlay algorithm constitutes an
iteration of the experiment. For each iteration, the values of the relevant measures are saved for analysis.
80% of the available hosts are selected randomly for use as peers to mimic a realistic distribution
of peers in the network. In each iteration, first the hosts for use as peers are selected. Then the overlay
algorithm is applied to generate a topology. The Mininet network emulator is started and basic settings
are applied to the OpenFlow switches as well as the hosts to enable normal network communications.
If the RASP overlay algorithm is used, the RASP service is configured. A RASP instance is added and all
82 7. Evaluation

peers as defined by the overlay network topology. Then, on each selected host, a program emulating a
P2P LVS peer is started and configured according to the overlay network topology. After waiting 15s, the
byte counter is initialized for measurement of the traffic volume. The waiting period is added because
the emulation startup causes a high CPU usage for some seconds, in which the forwarding does not work
reliably. Then, the traffic volume is measured for 60 seconds. After that, the network is torn down and
the next iteration begins.
The measurement of the data volume is implemented by reading the byte counters of the used interfaces
before and after the experiment. The interface names maps the node and network port names in
the network topology. By combining the information, the traffic volume transferred on each link in the
network during the 60 seconds of operation is calculated.
The computer system used for evaluation is a XEN 5 based virtual machine (VM). Two Intel XEON E5-
1410 cores with 2,8Ghz and 8 Gbyte Memory are allocated to the VM. For processing and analysis of the
measured data, a custom Python program is used. The graphs are created with ggplot2 [114], based on
preprocessing in R [94].
7.2.3 Results
The results of the measurements are described in this section. 30 iterations of the experiments are conducted
for each combination of the network topologies and the overlay network algorithms. The traffic
volume per peer in the ISP network for each network topology is shown in Figure 7.10a. Box plots are
used for depicting the mean, the confidence interval and the standard deviation. The mean is denoted
by the thick horizontal bar in the middle, the box denotes the 95% confidence interval, and the standard
deviation is depicted by the whiskers. The colored dots denote the measurements, one per iteration of
the experiment. They give an improved impression on the spread of the values as well as the minima
and maxima. For summarizing the metrics, the mean value is used because the RASP system’s use case is
a deployment over time period. For the evaluation of the efficiency of the system, the total sum of traffic
volume is relevant for the ISP. Hence, the appropriate measure is the mean traffic volume [95, p. 185].
In Figure 7.10a the order of the volume per peer is the same for all network topologies. Unsurprisingly
the Random overlay algorithms generates the highest volume, nearly the same amount is generated by
the Underlay-aware algorithm while the RASP algorithm generates significant less traffic volume. The
small differences between the two overlay algorithms that do not use the RASP system indicate that
the differences in the total sum of overlay path lengths in small. The amount of traffic volume per peer
differs slightly between the DT Hosts and the DT Core topologies. The volume is about 20% higher for all
overlay algorithms for the DT Depths topology.
The average traffic volume generated by transferring the video stream over a network link is 60s ∗
150KBit/s = 60s ∗ 18.75KByte/s = 1125KByte = 1.125MByte. The RASP algorithm uses per peer on
average about three times the traffic volume a single video stream generates over one link during the
same time for delivering the video throughout the ISP network. Running on the DT Hosts topology
the RASP algorithm uses a little less traffic volume, on the DT Depths it uses a little more. Hence, for
delivering the video stream, it uses the traffic volume equivalent of an average of 3 links per peer. The
other two overlay algorithms use at least 4 times the traffic volume of the video stream generated per
link.
The cumulative distribution functions of the traffic volumes per peer are depicted in Figure 7.10b. The
variation is similar for all overlay algorithms. It increases with the size of the topologies. The smallest
5
http://www.xen.org/ [Accessed 21.04.2013]
7.2. Transmission Efficiency 83

DT Hosts DT Core DT Depths
1.00
Intra−ISP network traffic volume per peer [Mbyte]
7.5
5
2.5
0
Cummulative probability
0.75
0.50
0.25
0.00
0 2.5 5 7.5
Intra−ISP network traffic volume per peer [Mbyte]
Overlay algorithm:
Random
Underlay−aware
RASP
Overlay algorithm:
●
●
Random
Underlay−aware
●
RASP
Topology: DT Hosts DT Core DT Depths
(a) Boxplot with mean, confidence interval, and standard deviation.
(b) Distributions of all topologies and overlay algorithms.
Figure 7.10.: The intra ISP data volume per peer for each topology and overlay network algorithm.
variation exhibits DT Hosts with 13 switches in the ISP network. It is slightly bigger for DT Core, whose
ISP part consists of 22 switches. DT Depths features the largest variation and its ISP network consists of
28 switches.
For assessing the influence of the network topology on the efficiency of the RASP service, a different
representation is helpful. In Figure 7.11 the intra ISP traffic volume is normalized by the mean value of
the overlay algorithm that uses the highest average traffic volume. This representation shows the ratio
between the algorithm with highest traffic volume, Random, and the other approaches. The mean traffic
volume per peer of the Underlay-aware approach is about 10% smaller in the DT Hosts topology, and
about 5% smaller in the other two topologies than the Random algorithm. The mean traffic volume ratio
of the RASP overlay algorithm is about 40% smaller than the Random algorithm in all cases. In the DT
Hosts and the DT Depths topologies the mean is less and the ratio larger. However, there is no statistical
significant difference between the mean traffic volume ratio of the RASP algorithm in the DT Hosts and
DT Depths topologies. The same is true for DT Depths and DT Core, only the difference between DT Core
and DT Hosts is statistically significant. The t-test statistics are listed in Section A.1. It can be concluded
that the different topologies used for this evaluation do not influence the efficiency of the RASP overlay
algorithm statistically significant compared to the alternatives.
The traffic volume crossing the ISP network border is relevant due to its costs. While it is not the goal
of the RASP service to reduce this type of network traffic, it should not increase it either. In Figure 7.12
the traffic volume per peer crossing the network border is depicted. The rasp overlay algorithm’s average
traffic volume is not generally smaller than that of the Underlay-aware algorithm. The difference between
both overlay algorithms in the topologies DT Core and DT Depths is not statistically significant. However,
the difference is significant in the DT Hosts topology. For the t-test statistics refer to Section A.2
84 7. Evaluation

DT Hosts DT Core DT Depths
Normalized intra−ISP data volume per peer
1.0
0.8
0.6
0.4
0.2
0.0
Overlay algorithm: ● Random ● Underlay−aware ● RASP
Figure 7.11.: The intra ISP data volume per peer for each topology and overlay network algorithm normalized by
the mean of the Random overlay algorithm for each topology.
DT Hosts DT Core DT Depths
Cross−border network traffic volume
per peer [Mbyte]
1.5
1
0.5
0
Overlay algorithm: ● Random ● Underlay−aware ● RASP
Figure 7.12.: The cross-border data volume for each topology and overlay network algorithm.
The overlay algorithm that uses the RASP service requires 40% less traffic volume inside the ISP
network compared to an underlay oblivious overlay network. About 30% less traffic volume inside the
ISP network is required compared to an underlay aware overlay network. The topologies used in this
7.2. Transmission Efficiency 85

evaluation do not influence this ratio significantly. The cross-border traffic does not decrease significantly
when using the RASP service compared to an underlay oblivious overlay network. Both, the Underlayaware
and the RASP overlay algorithms use about 30% less cross-border traffic volume than the Random
overlay algorithm.
The link stretch for the three topologies and overlay algorithms is depicted in Figure 7.13. The mean
link stretch of the Random overlay algorithm is the highest, between 2.3 − 2.4 in the three topologies.
The Underlay-aware algorithm has a slightly smaller mean, between 2.2 − 2.3. The mean of the RASP
algorithm is noticeably smaller at 1.9. The confidence intervals in Figure 7.13a are too small to be
distinguish them from the mean bar. For all overlay algorithms, the values are very similar for the three
network topologies, which is observable in Figure 7.13b.
The link stretch metric used in this thesis is based on the link lengths, not on measured latencies. Nevertheless,
the results of the link stretch analysis support the assumption that the transmission latencies
are not increase by using the RASP service; in the contrary they are slightly smaller.
DT Hosts DT Core DT Depths
1.00
Link stretch
4
3
2
Cummulative probability
0.75
0.50
0.25
1
0.00
0 1 2 3 4 5
Link stretch
0
Overlay algorithm:
Random
Underlay−aware
RASP
Overlay algorithm:
●
●
Random
Underlay−aware
●
RASP
Topology: DT Hosts DT Core DT Depths
(a) Boxplot with mean, confidence interval, and standard deviation.
(b) Distributions of all topologies and overlay algorithms.
Figure 7.13.: The link stretch for each topology and overlay.
7.3 System Costs
The evaluation showed that using the RASP system improves the efficiency of video stream distribution
in an ISP network. However, the systems usage also consumes resources during operation, which constitute
operating costs. Two aspects of costs caused by the system are identified: OpenFlow rules and
management overhead. OpenFlow rules represent the state information that is required to be kept in
the network during operation. As described in Section 2.3.2, the number of available OpenFlow rules
is restricted. Therefore, the number of rules is relevant when considering the systems advantages and
86 7. Evaluation

disadvantages. The messages exchanged for management of the RASP system by the super peer is called
management overhead. The RASP system architecture is shown in Figure 7.14. Messages are exchanged
between the super peer and the public service interface, the public service interface and the OpenFlow
applications and between the OpenFlow applications and the OpenFlow switches. This is also the location
where the OpenFlow rules are stored during operations. For this analysis, the prototype is used
for reference. However, as it lacks certain features, at some points the RASP concept is considered for
completing the analysis. For each part of the system, the resources required for adding an additional
peer are described. This is an indicator on the scaling properties of the system. The messages exchanged
between the application layer and the control layer are a direct consequence of the behavior of either
layers. Therefore, their behavior is not analyzed further.
Super Peer
Public Service API messages
Application
Layer
RASP Service
with public API
OpenFlow Application API message
Control
Layer
SDM
Application
Virtual Peer
Application
OpenFlow messages
Infrastructure
Layer
OpenFlow switches
Number of OpenFlow rules in the network
Figure 7.14.: The RASP system architecture with annotations on system cost related parts.
The use of OpenFlow by the RASP service has two aspects. One is the number of OpenFlow rules
that are used on the devices in the network. The other is the number of OpenFlow messages that are
exchanges during operation. Both aspects are closely related, as OpenFlow messages are used to install
and remove rules. Due to the nature of the experiments conducted for this evaluation, measured data is
only available on the setup of RASP instances. Hence, only this aspect of the RASP services operation is
measured and described. Other aspects are deducted from the available data.
The number of OpenFlow rules required is described in Section 7.3.1, the number of OpenFlow management
messages in Section 7.3.2. Finally, the management overhead of the public API is described in
Section 7.3.3.
7.3. System Costs 87

7.3.1 Number of Required OpenFlow Rules
The OpenFlow rules are installed by the two OpenFlow applications used in the RASP system: the Virtual
Peer application and the SDM application. Both are analyzed separately before summarizing their results
for an overview of the complete system. OpenFlow rules are a peer devices resource; each device has
a predefined rule capacity. Hence, both the rule requirements per device and for the whole OpenFlow
domain are investigated. Measurement data on the number of OpenFlow messages used to install new
OpenFlow rules during the experiments is used to validate the results.
The task of the virtual peer application is to manage the network layer proxy functionality of the RASP
service. It uses a dedicated switch for this task, which is used by the SDM application too. Each RASP
instance i ∈ I uses a switch for both applications, it is called instance Switch (R i SWC ).
For each RASP instance, the Virtual Peer application uses one OpenFlow rule on the instance switch
to forward connection requests from peers addressed at a RASP instance to the OpenFlow controller.
For each peer p ∈ R i P
that uses the instance, two OpenFlow rules for NAT on the dedicated switch are
required.
Super Peer
SDM distribution tree root
Potential
distribution tree
Distribution tree in
use
SDM egress switches
Per member
rules
ISP Customers
Stream
members
Figure 7.15.: Schema of the SDM distribution tree and the relevant parts for assessing the number of used Open-
Flow rules.
The usage pattern of the OpenFlow network of the SDM application is more complex. Different parts
of the network are used differently, as shown in Figure 7.15. For each video stream v ∈ R i V
that is
added to a RASP instance, a SDM stream with an associated distribution tree is created. The distribution
tree root is always the instance switch, from which an arborescence of potential forwarding paths spans
all SDM egress switches. For establishing a SDM tree, a rule is installed on the instance switch which
captures incoming video stream packets, denoted by the purple arrow to the tree root, which are sent
by the super peer for SDM forwarding. For each member of a stream one OpenFlow rule is installed on
its associated SDM egress switch, which writes the members destination information to the packet. For
each switch in the distribution tree that is on the path between a used egress switch and the tree root,
one rule per stream is required. Each core switch of the network topology model is a potential member
of a SDM distribution tree. Each access switch in the model is an SDM egress switch because it is the
88 7. Evaluation

last OpenFlow switch on the path to the connected hosts, that has the ability to modify the packets as
required by SDM.
Switches are denoted by s ∈ S ⊆ D, the switch type used as index specifies the type. For the analysis,
it assumed that ∀ i∈I |R i p | ≫ |S access| ≫ |S core |, which is true for the scenario used in this evaluation, as
described in Section 7.1. The scenario uses only one RASP instance and one video stream. Furthermore,
the SDM multicast tree spans all core switches, and at maximum one peer is used per host. The number
of OpenFlow rules is analyzed for the general case and for the described scenario. For the scenario, the
average number of rules is shown in Table 7.2. The rule numbers of the RASP instance switch are exact;
they do not change with different usage scenarios. The rules number for the core switches are influenced
by the distribution of the peers in the network; hence, it is a statistical value. However, in the network
topologies used for this evaluation, it is unlikely that a core switch will not be part of the active SDM
distribution tree. The same is true for access switches, given the random process used to select the hosts
to run peers on, it should reflect the relationship described in the table.
RASP instance property
Number of required OpenFlow rules per device
RASP Instance Switch Core switch Access switch
Number of instances |I| |I| 0 0
Number of peers ∑ i∈I |Ri P | 2 ∑ ∑
i∈I |Ri P | 0 i∈I |Ri P |
|S access |
∑
Number of streams ∑ i∈I |Ri V |
Sum for all instances, members, and peers
∑
i∈I
i∈I |Ri V | ∑i∈I |Ri V | 0
∑
i∈I |Ri V |
2|R
i
P | + |Ri V | + |I|
∑
i∈I |Ri P |
|S access |
Table 7.2.: Average number of OpenFlow rules per device class for the scenario used for evaluation.
The rule number given for the RASP instance switch in Table 7.2 is an exact value; the value for core
switches is an upper bound. To get a complete picture of the upper bound, the rule numbers for the
access switch are derived. The theoretical maximum number of rules for one access switch are installed,
if all peers of the RASP service are connected to the same switch. Hence, the upper bound is ∑ i∈I |Ri P |.
However, this value is assumed unrealistic in sufficiently large deployments.
Based on the analysis for each device class, upper bounds for the number of OpenFlow rules for the
whole OpenFlow domain is given in Equation 7.4. The values for the RASP instance switch are taken
from Table 7.2. The number of rules for core switches already is an upper bound; hence, it is multiplied
with the number of core switches. The total number of rules that are installed on access switches is the
number of peers in the system.
Max. number of rules: ofr max (I, R I V , Ri P , S) =|I| + ∑
i∈I
3|R
i
P | + (|S core| + 1)|R I V | (7.4)
Data from the evaluated scenario supports the results of Equation 7.4. The network experiment used
for evaluation creates rules, but does not remove them. The prototype implementation sends out one
OpenFlow flow modification message per rule that is installed on the RASP instance switch or access
switches. For core switches, the first message to a specific core switch installs a new OpenFlow rule;
additional messages modify the existing one. Hence, the number OpenFlow flow modification messages
without duplicate messages to the same core switches equals the number of OpenFlow rules in the
network. The numbers of these OpenFlow message that are send during the experiments are depicted in
Figure 7.16.
7.3. System Costs 89

Number of outgoing Openflow
flow modification messages
that install new rules
200
150
100
50
0
● ●
●
● ● ●
● ●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
30 40 50 60
Number of RASP peers
Topology: ● DT Hosts ● DT Core ● DT Depths
Figure 7.16.: The number of OpenFlow flow modification messages that install new rules during the network
experiments.
The black line denotes a linear regression model fitted to the data. Its confidence interval is shown
as a gray shaded band, but is too small to be clearly visible. For comparison, the dotted line denotes
Equation 7.4, the analytically derived upper bound for the number of OpenFlow rules in the network.
For the number of core switches in Equation 7.4, the maximum value of the three topologies is used,
which is |S core | = 9 for DT Depths. The differences between the topologies are consistent but small. DT
Host requires the lowest number rules, DT Depths the most, and the rule number of DT Core is in between
the two others. This order correlates with order of the number of switches in the respective ISP network
parts, which is 13, 22, and 28 for DT Host, DT Core, and DT Depths. The slope of the linear model is 3.18
and its intercept is 0.11. The upper bound holds for the segment shown in Figure 7.16. However, the
slope of the upper bound is 3, which means that it is not an upper bound for the linear model.
The reason for that could be that the model derived from the experiments conducted in this evaluation
is not precise enough due to its size. Still for the segment that is analyzed it shows that the upper bound
holds. In addition, it shows that the scenario used for evaluation is near the upper bound for OpenFlow
rule usage.
The bounds for the number of rules required by the RASP derived in this section are relevant for the
decision if an OpenFlow domain can host the service. Furthermore, it could be used for admission control,
to check whether enough resources are available for adding an additional instance, video stream, or peer.
The most resources are required on the RASP instance switch. However, a dedicated switch could be used,
which eases the burden of integrating the resource requirements with other services and the monitoring
of the resource usage. The requirement of an average of one rule per video stream on each core switch
is more demanding. Considering that the traffic millions of users could pass such a device, and many
OpenFlow applications run on it, the resource requirements might be an issue for integration in a large
network. However this is expected, the Explicit Multicast method for SDM, described in Section 5.3.4
is designed as remedy for this, but could not be evaluated due to incompatible OpenFlow versions.
On access switches, one rule is required for each user on any one of the access switches. The resource
restrictions on access switches are expected to be lower, hence the requirement might be easier to justify
than the one of the core switches.
An upper bound can be given for adding an additional peer to a RASP service instance and one video
stream. The additional required OpenFlow rules are fixed for the RASP instance switch and the access
90 7. Evaluation

switch. For the core switches, the maximum number of additional rules is one rule per switch on the path
from egress switch of the peer to the hosting switch. Hence, the longest path from the RASP instance
switch to any egress switch max(dist(R SWC i, s)|s ∈ S egress ) is an upper bound for the number of additional
core switch rules required. The bound is defined in Equation 7.5.
Max. number additional rules per peer: ofr max
peer (Ri SWC , S) =3 + max(dist(Ri SWC , s)|s ∈ S egress)) (7.5)
7.3.2 OpenFlow Messaging Overhead
The number of messages send during the operation of the RASP service describes the load the system imposes
on the OpenFlow network and its management facilities. The numbers of OpenFlow rules that are
installed during operation are a lower bound for the number of messages send. The reason for that is that
each rule installation requires one OpenFlow flow modification message. Hence, the findings concerning
the number of OpenFlow rules described in Section 7.3.1 are valid for this section as well. However,
more messages are exchanged, some of which do not depend on the OpenFlow applications used. These
messages, such as periodic echo messages between the OpenFlow controller and the OpenFlow switches,
are not relevant and hence not counted. In the next paragraphs, aspects of the OpenFlow applications
are described where they send and receive OpenFlow messages other than for rule installation. The latter
is described in Section 7.3.1.
The operation of the Virtual Peer application relies on reactive installation of OpenFlow rules. When
a hitherto unknown peer contacts a virtual peer instance, its switch forwards the packet wrapped in an
OpenFlow message to the Virtual Peer application. The corresponding OpenFlow message type is called
a PacketIn. There, two OpenFlow flow modification messages are send for installing the NAT rules. After
that, the received packet is send back to the instance switch, for putting it back on its path to the super
peer. The corresponding OpenFlow message type is called a PacketOut. Depending on the performance
of the OpenFlow controller, the peer can send several packets to the virtual peer before the NAT rules
are installed. This causes the repeated exchange of PacketIn and PacketOut messages.
The operation of the SDM application relies on proactive OpenFlow rule installation. Hence, no packets
are send from the instance switch. However, during operation multiple flow modification messages are
send to the same core switch. Only one rule installed on the core switch. The flow modification messages
are required to modify the existing rule for enlarging the distribution tree.
These are the two sources of OpenFlow messages besides the installation of rules. Their impact on the
total number of sent messages is depicted in Figure 7.17. In both figures, the black line denotes a linear
regression model of the surrounding data points. In the left diagram, the number of flow modification
messages from one experiment are depicted as a green dot, the total numbers of OpenFlow messages
from one experiment are depicted as a red dot. The number of flow modification messages send is stable
and varies only slightly. While it shows all sent OpenFlow flow modification messages, Figure 7.16 only
shows the ones that install a new rule, but not the ones that modify existing rules. The PacketIn and
PacketOut messages show a much larger variability. The slope of the linear regression of the number of
all messages is 6.99. The slope of the regression model of the flow modification messages is 3.46. Hence,
as shown in the right figure, the slope of the number of PacketIn and PacketOut messages is similar with
3.53, albeit its confidence interval is larger. For these messages, the network topology does not seem to
have a clear influence on the number of messages send.
7.3. System Costs 91

500
500
Number of OpenFlow messages
400
300
200
100
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
● ● ● ●
●
● ●
● ● ● ●
●
●
●
●
●
●
● ●
● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ● ● ●
●
●
● ● ●
● ●
●
●
● ●
●
●
●
● ● ●
● ●
●
●
● ●
●
●
●
●
●
● ●
●
●
● ● ● ●
●
● ●
●
● ●
● ●
Number of PacketIn and PacketOut
OpenFlow messages
400
300
200
100
●
●
● ●
●
●
● ● ● ●
● ●
● ● ●
●
●
●
●
● ● ●
●
● ●
●
●
●
● ● ●
●
●
●
● ● ●
●
●
● ● ● ● ● ●
●
●
● ● ●
● ● ● ● ●
●
● ●
●
●
●
●
●
●
● ●
0
30 40 50 60
Number of RASP peers
0
30 40 50 60
Number of RASP peers
Message types: ● all ●
flow mod
Topology: ● DT Hosts ● DT Core ● DT Depths
Figure 7.17.: The number of OpenFlow message required for adding peers to a RASP instance.
The upper bound of OpenFlow flow modification messages required to add a peer to the RASP instance
and one video stream, is identical to the upper bound for OpenFlow rules state in the last section in
Equation 7.5. However, an upper bound for the number of PacketIn and PacketOut messages cannot be
stated. For the emulated network and setup in this evaluation, on average an additional 3.5 messages
are sent. However, the setup used for this evaluation is very specific and likely not representative for
larger systems with respect to its OpenFlow controller architecture and the OpenFlow switches used.
7.3.3 API Management Overhead
In this section, the management overhead for a super peer using the RASP API is described. The API
management overhead adds to the data volume transferred by the super peer and increases the load of
both sides of the communications channel.
For creating an RASP instance, the super peer sets up a connection to the public service API of the RASP
system. Using the prototype, this requires two messages, one request and one reply. In a real system,
additional steps for authentication an authorization would be required. As this happens only once from
the point of view of a super peer, its influence is small and thus is ignored in this analysis. Once a RASP
instance is created, video streams and peers can be added. Adding a video stream requires that the super
peer sends one message to the API, which replies one message. When a peer joins the overlay network
through the RASP instance, no messaging is required when using the prototype implementation. For
a complete implementation of the system, one message for confirming the peer’s connection through
the RASP instance has to be sent to the API. The same is true when a peer leaves or a video stream is
removed. In both cases, one message is send to, and one message replied by the API.
The exact number of messages exchanged with the API required for adding one peer to an instance
and a video stream in the prototype is two messages. For a system following the RASP concept, two
additional API messages are transmitted.
92 7. Evaluation

7.4 Conclusion
The evaluation showed that using a RASP service lowers the transmitted data volume by 40% compared
to using an underlay oblivious P2P LVS system, and by 30% compared to an underlay-aware system. The
three different network topologies analyzed do not influence the transmitted data volume significantly.
The data volume crossing the ISP network’s border is not increased compared to underlay aware systems.
The same is true for the link stretch when using a RASP instance: in the presented evaluation, it did not
increase, neither compared to underlay-oblivious nor -aware P2P LVS systems.
With respect to the required number of OpenFlow rules, the system showed linear scaling. At maximum
3 + max(dist(R i SWC , s)|s ∈ S egress)) of additional OpenFlow rules are required for adding one peer to an
existing RASP instance and video stream. This equation is supported by measurements made during the
evaluation. The process causes at maximum the same number of OpenFlow flow modification messages
and a number of OpenFlow PacketIn and PacketOut messages. The latter depends on the system setup
and performance. In addition to that, two API messages are sent for adding a peer or a stream. The
messaging overhead for the super peer is very low, two to four transferred message calls cause negligible
network traffic. The load imposed on the OpenFlow network and its management system is higher. Since
there are no such systems known for reference, the numbers cannot be validated for their applicability.
For deployment of the system, is it important to know its limitations and potential bottlenecks. The
maximum number of peers supported by one RASP instance is 64, 511, as described in Section 5.3.2. The
maximum number of peers supported by an OpenFlow domain is restricted by the number of available
RASP instance switches and the rule capacities of the access switches. For supporting 64, 511 peers, one
RASP instance switch has to use 129, 022 rules, and all the access switches in the network have to support
64, 511 rules together. The RASP instance switches are likely to become the bottleneck of the system,
because of their vital role and the resource requirements of the RASP service. However, these switches
are expected to be dedicated to their specific role, hence their resource usage can be monitored and their
rule load can be adapted accordingly. The resource requirements for the core and access switches are
clearly stated. Hence, the requirements for integration into other services are met.
7.4. Conclusion 93

8 Conclusion and Future Work
Internet video streaming is responsible for a growing part of the internet traffic. P2P systems are one
delivery method for live video streaming. Their use is beneficial for its users and operators, because no
infrastructure is required for operation. However, Peer-to-Peer Live Video Streaming (P2P LVS) puts a
burden on ISPs because of its hard to control, unpredictable, and unfavorable traffic patterns. In this
work, the Rent-a-Super Peer (RASP) service is introduced to improve the situation of ISPs by giving
them the opportunity to offer a generic network layer service for efficient distribution of P2P video
streams. The service is designed to be deployed by residential broadband ISPs, whose customers are
the main users of P2P LVS. The RASP service provides single peers of P2P LVS systems on-demand
additional upstream capacity within the ISP’s network. Its generic cross-layer approach should allow
easy integration into different P2P LVS systems. In exchange for providing the service, the ISPs get full
control of the P2P traffic and benefit from a reduction of the traffic volume because of the improved
transmission efficiency. The RASP service is enabled by the recently introduced SDN variant OpenFlow.
Its adoption in ISP networks is well possible in the near future owing to its growing market acceptance
and available hardware.
In Chapter 4 the requirements for this kind of service are described. The design of the RASP service
is discussed in Chapter 5. Its component based approach defines a service controller component as well
as two functional OpenFlow components, a network layer proxy called Virtual Peer and a network layer
multicast component called SDM. The first provides the user of a P2P LVS system with a virtual presence
within the ISP network; the latter provides efficient multicast transmission using IP unicast addresses. A
multicast distribution mechanism based on the Explicit Multicast concept [43] for the SDM component
is described. It exhibits improved scaling properties compared to the default multicast mechanism used
by SDM.
A prototypical implementation of selected features of the RASP systems is described in Chapter 6.
Based on the requirements for an evaluation of fundamental system properties, the two functional components,
and the service controller component are implemented. The Ryu OpenFlow controller software
is used as base for the implementation of the components. Reasons for its use are its easy integration
with Mininet and its flexibility regarding the use of different OpenFlow versions. The architectural approach
on designing the RASP OpenFlow applications is described. A custom emulation environment
for testing emulated P2P LVS systems in different topologies is designed and implemented based on the
network emulator Mininet. The Open vSwitch OpenFlow version 1.0 software switch is selected for use
with the network emulation for its performance. An assessment of the performance of available Open-
Flow software switches reveals that it is the only viable alternative. The Explicit Multicast concept is not
implemented due to its requirement for OpenFlow version 1.2, which is not met by Open vSwitch.
The objectives for the evaluation of the RASP service, introduced in Chapter 7, are to compare the
transmission efficiency of P2P LVS system with and without using the service in different network
topologies as well as analyzing its resource requirements and management overhead. The relevant characteristics
of ISP networks and their topologies are described taking the current research into account
as well as the literature on deployed technologies. Against this background, three different network
topologies based on the core network topology model of Deutsche Telekom are selected to assess the
influence of the network topology on the efficiency of RASP service. Three different P2P LVS overlay
95

topology generators are created for evaluation. The first approach represents underlay-oblivious overlay,
the second represents underlay-aware overlay networks. The third overlay network generator is also
underlay-aware, but integrates the RASP service. Their conformance to the expected characteristics of
their respective represented overlay network topologies is assessed by evaluating the properties of their
generated topologies. The emulated networks consist of 19, 28, and 34 OpenFlow switches. On average,
the emulated P2P LVS overlay networks consist of 144 peers.
The evaluation results show a significant improvement in the average transmission efficiency of P2P
LVS systems using overlay networks that integrate the RASP service over overlay networks that do not
integrate the service. Compared to random overlay network topologies, underlay aware overlay topologies
with RASP service integration reduce the traffic volume by 40%. A reduction of 30% is achieved
when comparing underlay-aware overlay topologies with and without the use of the RASP service. The
investigated network topologies do not significantly influence the transmission efficiency of the service.
Other characteristics of the overlay networks do not deteriorate when using the RASP service. The traffic
volume crossing the network border as well as the link stretch does not increase with the use of the RASP
service. It is even slightly lower than with the comparable underlay-aware overlay topologies.
The resource requirements and management overhead of the RASP service are analyzed by using measured
data and giving analytically derived upper bounds. The service imposes a very small management
overhead for peers using it. The resource requirements on the ISP’ management system are higher but
well defined. The system relies on dedicated OpenFlow switches for the core of its operation. The resource
demand for the rest ISP network is lower. It is well defined and upper bounds are given, which
is expected to allow easy integration into OpenFlow-based ISP networks. The RASP service achieves its
objectives without relying on IP multicast or other specialized network protocols. Integration into a wide
range of different network management regimes is possible because of the flexibility of multicast forwarding
mechanism. One reason for this is that it is compatible with different network protocols such as
MPLS that are used for forwarding or encapsulation.
8.1 Future Work on the RASP Implementation and Integration into P2P LVS Systems
The prototype in this work is implemented with OpenFlow 1.0 because no adequate OpenFlow 1.2
software or hardware switch is available to the author. This restriction is the reason why the Explicit
Multicast subcomponent of SDM could not be evaluated. An implementation and evaluation is required
to verify if this approach scales better than the available prototype.
This evaluation in this work provides details on the resource consumption of the RASP service. For
evaluating the impact of these resource requirements on real OpenFlow hardware and software architectures,
appropriate testing is required. Relevant are the real world throughput of hardware OpenFlow
switches given rules that heavily rely on rewriting packet headers as they are used on the SDM egress
switches and the RASP instance switches. Furthermore, the RASP system should be evaluated using
adequate OpenFlow controller architectures for ISPs.
The RASP service provides a generic network layer service. The integration into P2P LVS systems is
crucial for its adoption. Some P2P LVS systems are especially suitable for this purpose. mTreebone [110]
is such a system, as it is based on super peers. SALSA [51] is a system based on super peers as well but it
also proposes an incentive system for users to offer additional upstream bandwidth. Besides the effects of
the RASP service, the applicability of SALSA’s incentive system should give an insight on how to motivate
the users to adopt the RASP service. Mesh-based systems necessitate special adaptation for integrating
the RASP service because SDM requires a tree like push model for connected peers. Hence, adaptation
mechanisms such as the Transit [99] approach are required for the integration of the RASP service. Their
96 8. Conclusion and Future Work

applicability for integration should be investigated, and a general approach on how to integrate with
mesh-based systems should be derived.
8.2 Future Work on the Large Scale Evaluation
The prototype of the RASP service used for evaluation in this thesis is adequate for evaluating basic
properties. Furthermore, it demonstrates the feasibility of the concept. However, the prototypical implementation,
the associated evaluation system, and the available emulated network environment limits the
scale of the network topologies and the P2P LVS systems used. Therefore, 180 hosts, 19 to 34 switches
and on average 144 peers are used for evaluation. Both, ISP and P2P LVS networks are often much
larger in reality. P2P LVS systems might have tens of thousands of users; some ISP networks consist of
hundreds of locations with hundreds of routers each. Hence, large-scale tests are required to get a better
understanding on the performance characteristics of the RASP service at such scales.
Given that many questions concerning the use of OpenFlow in ISP networks are not explored yet,
simulations could provide a better understanding of the RASP service. Furthermore, simulations could
be used for investigation its integration into P2P LVS systems and the impact of the increased upstream
bandwidth. The same is true for experiments with large ISP networks, given their size; simulations are
the only viable way to study the impact of a large number of RASP instances and users. The large-scale
dynamics of the RASP should also be investigated. Peer churn and flash crowds are issues current P2P LVS
systems face. The RASP system should improve the P2P LVS system’s behavior in these circumstances.
8.3 Future Work on Using the RASP Service with other Applications
The generic nature of the RASP service and especially its functional components allow its use with other
applications than P2P LVS. Specifically, the applicability of the SDM approach to different application
areas, such as online gaming, should be investigated. Another application area that could be investigated
is multicast communication in wireless networks such as WiFi, WiMAX, or cellular technologies. In this
area, due to technological restrictions, combinations of unicast and multicast are beneficial. However, the
use of IP multicast is disadvantageous in many areas [5]. The applicability of SDM should be investigates.
Its use could be beneficial in this area due to its efficient combination of network layer multicast and IP
unicast addressing.
8.2. Future Work on the Large Scale Evaluation 97

A Evaluation Statistics
The code of the R programming language shown in the following sections is based on traffic data processed
with the Python module rasp.evaluation.traffic-processing.py and prepared with the traffic-plot.R
script. For reference of the data frame names, their definition is listed below; the base data frame has
the name traffic in the code.
> ic_random ic_underlay ic_rasp oc_random oc_underlay oc_rasp tree_random tree_underlay tree_rasp t.test(ic_rasp$isp_volume_peer_norm, tree_rasp$isp_volume_peer_norm)
Welch Two Sample t-test
data: ic_rasp$isp_volume_peer_norm and tree_rasp$isp_volume_peer_norm
t = -1.5846, df = 51.584, p-value = 0.1192
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.051917052 0.006106836
sample estimates:
mean of x mean of y
0.5058354 0.5287405
> t.test(ic_rasp$isp_volume_peer_norm, oc_rasp$isp_volume_peer_norm)
Welch Two Sample t-test
data: ic_rasp$isp_volume_peer_norm and oc_rasp$isp_volume_peer_norm
t = -3.8517, df = 55.302, p-value = 0.0003071
alternative hypothesis: true difference in means is not equal to 0
99

95 percent confidence interval:
-0.07715070 -0.02434784
sample estimates:
mean of x mean of y
0.5058354 0.5565846
> t.test(tree_rasp$isp_volume_peer_norm, oc_rasp$isp_volume_peer_norm)
Welch Two Sample t-test
data: tree_rasp$isp_volume_peer_norm and oc_rasp$isp_volume_peer_norm
t = -1.7706, df = 56.838, p-value = 0.08198
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.059335984 0.003647667
sample estimates:
mean of x mean of y
0.5287405 0.5565846
>
A.2 t-test Statistics of the Cross-Border Traffic Volume Differences
> t.test(ic_underlay$cross_volume_peer, ic_rasp$cross_volume_peer)
Welch Two Sample t-test
data: ic_underlay$cross_volume_peer and ic_rasp$cross_volume_peer
t = 4.2714, df = 57.637, p-value = 7.355e-05
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
97046.99 268268.29
sample estimates:
mean of x mean of y
862875.9 680218.3
> t.test(oc_underlay$cross_volume_peer, oc_rasp$cross_volume_peer)
Welch Two Sample t-test
data: oc_underlay$cross_volume_peer and oc_rasp$cross_volume_peer
t = 0.441, df = 57.465, p-value = 0.6609
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-58013.40 90788.76
100 A. Evaluation Statistics

sample estimates:
mean of x mean of y
803199.8 786812.1
> t.test(tree_underlay$cross_volume_peer, tree_rasp$cross_volume_peer)
Welch Two Sample t-test
data: tree_underlay$cross_volume_peer and tree_rasp$cross_volume_peer
t = 1.7822, df = 50.983, p-value = 0.08067
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-10397.38 174806.56
sample estimates:
mean of x mean of y
828671.8 746467.2
A.2. t-test Statistics of the Cross-Border Traffic Volume Differences 101

Acronyms
ALTO Application-Layer Traffic Optimization.
API Application Programming Interface.
BRAS Broadband Remote Access Server.
CAM Content-addressable Memory.
CDN Content Delivery Network.
DHCP Dynamic Host Configuration Protocol.
DSL Digital Subscriber Line.
DSLAM Digital Subscriber Line Access Multiplexer.
FTTX Fiber-to-the-x.
HTTP Hypertext Transfer Protocol.
IETF Internet Engineering Task Force.
IGMP Internet Group Management Protocol.
IPoE IP over Ethernet.
IPTV Internet Protocol Television.
ISP Internet Service Provider.
IXP Internet eXchange Point.
JSON JavaScript Object Notation.
LLDP Link Layer Discovery Protocol.
MPLS Multiprotocol Label Switching.
NAT Network Address Translation.
OS Operating System.
OSPF Open Shortest Path First.
P2P Peer-to-Peer.
P2P LVS Peer-to-Peer Live Video Streaming.
PIM-SM Protocol Independent Multicast, Sparse Mode.
PoP Point of Presence.
PPP Point-to-Point Protocol.
PPPoE PPP over Ethernet.
QoS Quality of Service.
RASP Rent-a-Super Peer.
REST Representational State Transfer.
103

RPC Remote Procedure Call.
SDM Software Defined Multicast.
SDN Software Defined Networking.
SSM Source-Specific Multicast.
TCP Transmission Control Procol.
UDP User Datagram Protocol.
VLAN Virtual Local Area Network.
104 Acronyms

Bibliography
[1] Open Networking Foundation Members. URL https://www.opennetworking.org/membership/
member-listing [Accessed 29.04.2013].
[2] Migration to Ethernet-Based Broadband Aggregation. Technical Report TR-101, Broadband Forum,
July 2011.
[3] Split Architecture for Large Scale Wide Area Networks. Technical Report ICT-258457, SPARC,
2012.
[4] O. Abboud, K. Pussep, A. Kovacevic, K. Mohr, S. Kaune, and R. Steinmetz. Enabling resilient P2P
video streaming: survey and analysis. Multimedia Systems, 17(3):177–197, March 2011.
[5] A. Abdollahpouri, B. E. Wolfinger, and J. Lai. Unicast versus Multicast for Live TV Delivery in
Networks with Tree Topology. In Proceedings of the International Conference on Wired/Wireless
Internet Communications (WWIC), 2010.
[6] K. Abe, T. Ueda, M. Shikano, H. Ishibashi, and T. Matsuura. Toward Fault-Tolerant P2P Systems:
Constructing a Stable Virtual Peer from Multiple Unstable Peers. In Proceedings of the
International Conference on Advances in P2P Systems (AP2PS), 2009.
[7] B. Ager, N. Chatzis, A. Feldmann, N. Sarrar, S. Uhlig, and W. Willinger. Anatomy of a large european
IXP. ACM SIGCOMM Computer Communication Review, 42(4), September 2012.
[8] V. Aggarwal, A. Feldmann, and C. Scheideler. Can ISPS and P2P users cooperate for improved
performance ACM SIGCOMM Computer Communication Review, 37(3), July 2007.
[9] Agilent Technologies. Understanding DSLAM and BRAS Access Devices, 2006. URL http://cp.
literature.agilent.com/litweb/pdf/5989-4766EN.pdf [Accessed 29.04.2013].
[10] B. M. Ali, S. A. Al-Talib, S. Khatun, and S. Subramaniam. REHASH: integrating recursive unicast
with Hash algorithm to provide multicast service to receivers. In Proceedings of the IEEE
International Conference on Networks (ICON), 2005.
[11] R. Alimi, R. Penno, and Y. Yang. ALTO Protocol. Internet Engineering Task Force, February
2013. URL http://tools.ietf.org/pdf/draft-ietf-alto-protocol-14.pdf [Accessed
29.04.2013].
[12] M. Bernstein. Understanding PPPoE and DHCP. Juniper Networks, May 2006. URL http:
//www.broadbandreports.com/r0/download/1468367~39a604b7c30141db72e55620ebf95879/
Understanding%20PPPoE%20and%20DHCP_200187.pdf [Accessed 29.04.2013].
[13] D. Bilenko. gevent Introduction. URL http://www.gevent.org/intro.html [Accessed
29.04.2013].
[14] J. Buford and M. Kolberg. Hybrid Overlay Multicast Simulation and Evaluation. In Proceedings
of the IEEE Consumer Communications and Networking Conference (CCNC), 2009.
105

[15] B. Cain, S. E. Deering, I. Kouvelas, B. Fenner, and A. Thyagarajan. Internet Group Management
Protocol, Version 3. RFC 3376, Internet Engineering Task Force, October 2002.
[16] M. Castro, P. Druschel, A.-M. Kermarrec, A. Nandi, A. Rowstron, and A. Singh. SplitStream: highbandwidth
multicast in cooperative environments. In Proceedings of the ACM Symposium on
Operating Systems Principles (SOSP), 2003.
[17] M. Cha, P. Rodriguez, S. Moon, and J. Crowcroft. On next-generation telco-managed P2P TV
architectures. In Proceedings of the USENIX International Conference on Peer-to-Peer Systems
(IPTPS), 2008.
[18] S.-K. Chang. The Generation of Minimal Trees with a Steiner Topology. Journal of the ACM, 19
(4):699–711, October 1972.
[19] Y. Chiba, Y. Shinohara, and H. Shimonishi. Source flow: handling millions of flows on flow-based
nodes. ACM SIGCOMM Computer Communication Review, 40(4):465–466, 2010.
[20] G. Dán, T. Hoßfeld, S. Oechsner, P. Cholda, R. Stankiewicz, I. Papafili, and G. Stamoulis. Interaction
patterns between P2P content distribution systems and ISPs. IEEE Communications
Magazine, 49(5):222–230, May 2011.
[21] S. E. Deering. Host Extensions for IP Multicasting. RFC 1112, Internet Engineering Task Force,
August 1989.
[22] S. E. Deering and D. R. Cheriton. Multicast routing in datagram internetworks and extended
LANs. ACM Transactions on Computer Systems, 8(2):85–110, 1990.
[23] C. Diot, B. N. Levine, B. Lyles, H. Kassem, and D. Balensiefen. Deployment issues for the IP
multicast service and architecture. IEEE Network, 14(1):78–88, 2000.
[24] R. D. Doverspike, G. Li, K. N. Oikonomou, K. K. Ramakrishnan, and D. Wang. IP Backbone Design
for Multimedia Distribution: Architecture and Performance. In Proceedings of the IEEE International
Conference on Computer Communications (INFOCOM), 2007.
[25] R. Dunaytsev, D. Moltchanov, Y. Koucheryavy, O. Strandberg, and H. Flinck. A Survey of P2P Traffic
Management Approaches: Best Practices and Future Directions. Journal of Internet Engineering,
5(1), July 2012.
[26] M. Düser and A. Gladisch. Evaluation of Next Generation Network Architectures and Further Steps
for a Clean Slate Networking Approach. In ITG EuroView, Presentation, 2006. URL http://
www3.informatik.uni-wuerzburg.de/euroview/2006/presentations/talk_Dueser.pdf [Accessed
29.04.2013].
[27] B. Fenner and D. Meyer. Multicast Source Discovery Protocol. RFC 3618, Internet Engineering
Task Force, October 2003.
[28] B. Fenner, M. Handley, H. Holbrook, and I. Kouvelas. Protocol Independent Multicast - Sparse
Mode (PIM-SM): Protocol Specification (Revised). RFC 4601, Internet Engineering Task Force,
August 2006.
[29] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee. Hypertext
Transfer Protocol – HTTP/1.1. RFC 2616, Internet Engineering Task Force, June 1999.
106 Bibliography

[30] R. T. Fielding. Architectural styles and the design of network-based software architectures. PhD
thesis, University of California, Irvine, 2000.
[31] P. Georgopoulos. Openflow HP switch (3500yl) - Processes flows on software, January 2013.
URL http://thread.gmane.org/gmane.network.routing.openflow.general/2275 [Accessed
29.04.2013].
[32] A. Gladisch. Programmable BitPipe. In European Workshop on Software Defined Networks
(EWSDN), Presentation, 2012. URL http://www.ewsdn.eu/presentations/Gladisch.pdf [Accessed
29.04.2013].
[33] A. Gladisch and F.-J. Westphal. Directions of next generation transport network development. In
Optical Fiber Communication Conference and Exposition and the National Fiber Optic Engineers
Conference (OFC/NFOEC), 2012.
[34] K. Grobe, J.-P. Elbers, and S. Neidlinger. Broadband Access Networks. In A. Shami, M. Maier, and
C. Assi, editors, Next-Generation Broadband Access Networks and Technologies, pages 353–372.
Springer, 2009.
[35] N. Gude, T. Koponen, J. Pettit, B. Pfaff, M. Casado, N. McKeown, and S. Shenker. NOX: towards
an operating system for networks. ACM SIGCOMM Computer Communication Review, 38(3):
105–110, 2008.
[36] G. Hasslinger. Improving Peer-to-Peer Transport Paths for Content Distribution. In X. Shen,
H. Yu, J. Buford, and M. Akon, editors, Handbook of Peer-to-Peer Networking, pages 1277–1291.
Springer US, 2010.
[37] O. Heckmann, M. Piringer, J. Schmitt, and R. Steinmetz. Generating realistic ISP-level network
topologies. IEEE Communications Letters, 7(7):335–336, 2003.
[38] M. V. J. Heikkinen, T. Casey, and F. Hecht. Value analysis of centralized and distributed communications
and video streaming. The Journal of Policy, Regulation and Strategy for Telecommunications,
Information and Media (INFO), 12(5):42–58, 2010.
[39] Hewlett-Packard. HP OpenFlow Switches, 1 edition, September 2012. URL http://bizsupport1.
austin.hp.com/bc/docs/support/SupportManual/c03512348/c03512348.pdf [Accessed
29.04.2013].
[40] U. Hoelzle. Openflow@google. In Open Networking Summit, Presentation, 2012. URL
http://www.opennetsummit.org/archives/apr12/hoelzle-tue-openflow.pdf [Accessed
29.04.2013].
[41] H. Holbrook and B. Cain. Source-Specific Multicast for IP. RFC 4607, Internet Engineering Task
Force, August 2006.
[42] M. Howard. Routers on the IP Edge. Infonetics Research, October 2012. URL
http://www.infonetics.com/whitepapers/2012-Infonetcs-Whitepaper-Routers-onthe-IP-Edge.pdf
[Accessed 29.04.2013].
[43] Y. Imai, N. Feldman, R. Boivie, W. Livens, and D. Ooms. Explicit Multicast (Xcast) Concepts and
Options. Internet Engineering Task Force, November 2007. URL http://tools.ietf.org/html/
rfc5058 [Accessed 29.04.2013].
Bibliography 107

[44] X. Jin, K.-L. Cheng, and S. H. G. Chan. Scalable Island Multicast for Peer-to-Peer Streaming.
Advances in Multimedia, 2007:1–9, 2007.
[45] X. Jin, K.-L. Cheng, and S. H. G. Chan. Island Multicast: Combining IP Multicast With Overlay
Data Distribution. IEEE Transactions on Multimedia, 11(5):1024–1036, July 2009.
[46] X. Jin, W. Tu, and S. H. G. Chan. Challenges and advances in using IP multicast for overlay data
delivery. IEEE Communications Magazine, 47(6):157–163, 2009.
[47] P. Jokela, A. Zahemszky, C. E. Rothenberg, S. Arianfar, and P. Nikander. LIPSIN: line speed publish/subscribe
inter-networking. In Proceedings of the ACM Special Interest Group on Data Communication
Conference (SIGCOMM), 2009.
[48] Juniper Networks. Do You Need a Broadband Remote Access Server, 2009. URL http://www.
juniper.net/kr/kr/local/pdf/whitepapers/2000259-en.pdf [Accessed 29.04.2013].
[49] S. Kadadi and J. Buford. SAM Problem Statement. Internet Engineering Task Force, March
2008. URL http://tools.ietf.org/pdf/draft-irtf-sam-problem-statement-02 [Accessed
29.04.2013].
[50] A. Keller. Breitbandkabel und Zugangsnetze. Springer, 2011.
[51] J. Kim, Y. Lee, and S. Bahk. SALSA: Super-Peer Assisted Live Streaming Architecture. In Proceedings
of the IEEE International Conference on Communications (ICC), 2009.
[52] M. Kind, F.-J. Westphal, A. Gladisch, and S. Topp. SplitArchitecture: Applying the Software Defined
Networking Concept to Carrier Networks. In World Telecommunications Congress (WTC), 2012.
[53] P. Klein and R. Ravi. A Nearly Best-Possible Approximation Algorithm for Node-Weighted Steiner
Trees. Journal of Algorithms, 19(1):104–115, July 1995.
[54] S. Knight, H. X. Nguyen, N. Falkner, R. Bowden, and M. Roughan. The Internet Topology Zoo.
IEEE Journal on Selected Areas in Communications, 29(9):1765–1775, 2011.
[55] T. Koponen, M. Casado, N. Gude, J. Stribling, L. Poutievski, M. Zhu, R. Ramanathan, Y. Iwata,
H. Inoue, T. Hama, and S. Shenker. Onix: a distributed control platform for large-scale production
networks. In Proceedings of the USENIX Symposium on Operating Systems Design and
Implementation (OSDI), 2010.
[56] D. Kotani, K. Suzuki, and H. Shimonishi. A Design and Implementation of OpenFlow Controller
Handling IP Multicast with Fast Tree Switching. In Proceedings of the IEEE/IPSJ International
Symposium on Applications and the Internet (SAINT), 2012.
[57] R. Kumar, Y. Liu, and K. Ross. Stochastic Fluid Theory for P2P Streaming Systems. In Proceedings
of the IEEE International Conference on Computer Communications (INFOCOM), 2007.
[58] C. Labovitz, S. Iekel-Johnson, D. McPherson, J. Oberheide, and F. Jahanian. Internet inter-domain
traffic. In Proceedings of the ACM Special Interest Group on Data Communication Conference
(SIGCOMM), 2010.
[59] B. Lantz, B. Heller, and N. McKeown. A network in a laptop. In Proceedings of the ACM Workshop
on Hot Topics in Networks (HotNets), 2010.
108 Bibliography

[60] L. Lao, J.-H. Cui, and M. Gerla. Tackling group-to-tree matching in large scale group communications.
Computer Networks, 51(11):3069–3089, 2007.
[61] K. Lee and G. Jian. ALTO and DECADE service trial within China Telecom. Internet Engineering
Task Force, March 2012. URL http://tools.ietf.org/id/draft-lee-alto-chinatelecomtrial-04.txt
[Accessed 29.04.2013].
[62] S. Lee and S. Sahu. Optimized Hybrid Overlay Design for Real Time Broadcasting. In Proceedings
of the IEEE International Conference on Computer Communications and Networks (ICCCN),
2010.
[63] N. Leyman, B. Decraene, C. Filsfils, M. Konstantynowicz, and D. Steinberger. Seamless MPLS
Architecture. Internet Engineering Task Force, October 2012. URL http://tools.ietf.org/
pdf/draft-ietf-mpls-seamless-mpls-02.pdf [Accessed 29.04.2013].
[64] C. Liang, Y. Guo, and Y. Liu. Investigating the Scheduling Sensitivity of P2P Video Streaming: An
Experimental Study. IEEE Transactions on Multimedia, 11(3):348–360, 2009.
[65] J. Liu, S. G. Rao, B. Li, and H. Zhang. Opportunities and Challenges of Peer-to-Peer Internet Video
Broadcast. Proceedings of the IEEE, 96(1):11–24, 2008.
[66] Y. Liu, Z. Liu, X. Wu, J. Wang, and C. C.-Y. Yang. IPTV System Design: An ISP’s Perspective.
In International Conference on Cyber-enabled Distributed Computing and Knowledge Discovery
(CyberC), 2011.
[67] E. K. Lua, J. Crowcroft, M. Pias, R. Sharma, and S. Lim. A survey and comparison of peer-to-peer
overlay network schemes. IEEE Communications Surveys & Tutorials, 7(2):72–93, 2005.
[68] L. Luo, G. Xie, S. Uhlig, L. Mathy, K. Salamatian, and Y. Xie. Towards TCAM-based scalable
virtual routers. In Proceedings of the ACM International Conference on Emerging Networking
Experiments and Technologies (CoNEXT), 2012.
[69] G. Maier, A. Feldmann, V. Paxson, and M. Allman. On dominant characteristics of residential
broadband internet traffic. In Proceedings of the ACM SIGCOMM Conference on Internet Measurement
(IMC), 2009.
[70] C. A. C. Marcondes, T. P. C. Santos, A. P. Godoy, C. C. Viel, and C. A. C. Teixeira. CastFlow:
Clean-slate multicast approach using in-advance path processing in programmable networks. In
Proceedings of the IEEE Symposium on Computers and Communications (ISCC), 2012.
[71] N. McKeown. Software-defined networking. In International Conference on Computer Communications
(INFOCOM) Keynote Talk, Presentation, 2009. URL http://klamath.stanford.edu/
~nickm/talks/infocom_brazil_2009_v1-1.pdf [Accessed 29.04.2013].
[72] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson, J. Rexford, S. Shenker, and
J. Turner. OpenFlow. ACM SIGCOMM Computer Communication Review, 38(2):69–74, March
2008.
[73] A. Medina, A. Lakhina, I. Matta, and J. Byers. BRITE: an approach to universal topology generation.
In Proceedings of the International Symposium on Modeling, Analysis and Simulation of
Computer and Telecommunication Systems (MASCOT), 2001.
Bibliography 109

[74] NEC. NEC ProgrammableFlow Univerge PF5240 Product Sheet. URL http://www.necam.com/
docs/id=5ce9b8d9-e3f3-41de-a5c2-6bd7c9b37246 [Accessed 29.04.2013].
[75] NEC. IP8800/S3640 OpenFlow Feature Guide (Version 11.1 Compatible), 1 edition, June
2010. URL http://support.necam.com/kbtools/sDocs.cfmid=fcbdcb3e-45fa-4ec4-9311-
215bd9ab9f81 [Accessed 29.04.2013].
[76] F. Németh, Á. Stipkovits, B. Sonkoly, and A. Gulyás. Towards SmartFlow: case studies on enhanced
programmable forwarding in OpenFlow switches. ACM SIGCOMM Computer Communication
Review, 42(4):85–86, September 2012.
[77] NTT laboratories OSRG Group. Ryu 1.7 documentation, March 2013.
[78] Open Networking Foundation. OpenFlow Switch Specification 1.0, December 2009.
[79] Open Networking Foundation. OpenFlow Switch Specification 1.2, December 2011.
[80] Open Networking Foundation. OpenFlow Switch Specification 1.1, February 2011.
[81] Open Networking Foundation. OpenFlow Switch Specification 1.3, July 2012.
[82] Open Networking Foundation. Software-Defined Networking: The New Norm for Networks, April
2012. URL https://www.opennetworking.org/images/stories/downloads/sdn-resources/
white-papers/wp-sdn-newnorm.pdf [Accessed 29.04.2013].
[83] Open Networking Foundation. OpenFlow Management and Configuration Protocol 1.1, June
2012.
[84] Open Networking Foundation. OpenFlow Switch Specification 1.3.1, September 2012.
[85] B. Owens. What’s Next for OpenFlow and Open Source, March 2013. URL http://
packetpushers.net/whats-next-for-openflow-and-open-source/ [Accessed 29.04.2013].
[86] B. Owens. OpenFlow Switching Performance: Not All TCAM Is Created Equal, February
2013. URL http://packetpushers.net/openflow-switching-performance-not-all-tcamis-created-equal/
[Accessed 29.04.2013].
[87] V. N. Padmanabhan, H. J. Wang, P. A. Chou, and K. Sripanidkulchai. Distributing streaming media
content using cooperative networking. In Proceedings of the ACM International Workshop on
Network and Operating Systems Support for Digital Audio and Video (NOSSDAV), 2002.
[88] K. Pagiamtzis and A. Sheikholeslami. Content-addressable memory (CAM) circuits and architectures:
a tutorial and survey. IEEE Journal of Solid-State Circuits, 41(3):712–727, 2006.
[89] I. Papafili, S. Soursos, and G. D. Stamoulis. Improvement of BitTorrent Performance and Interdomain
Traffic by Inserting ISP-Owned Peers. In Proceedings of the International Workshop on
Internet Charging and Qos Technologies (ICQT), 2009.
[90] I. Pepelnjak. OpenFlow and SDN: hype, useful tools or panacea In RIPE 65, Presentation,
2012. URL https://ripe65.ripe.net/presentations/19-OpenFlow_and_SDN_(RIPE)
.pdf [Accessed 29.04.2013].
110 Bibliography

[91] J. Pettit, J. Gross, B. Pfaff, M. Casado, and S. Crosby. Virtual switching in an era of advanced
edges. In Workshop on Data Center – Converged and Virtual Ethernet Switching (DC-CAVES),
2010.
[92] K. T. Phan, J. Moulierac, C. N. Tran, and N. Thoai. Xcast6 treemap islands: revisiting multicast
model. In Proceedings of the ACM International Conference on Emerging Networking Experiments
and Technologies (CoNEXT) Student Workshop, 2012.
[93] Point Topic. World Broadband Statistics Q2 2012. October 2012. URL http://point-
topic.com/wp-content/uploads/2013/02/Sample-Report-Global-Broadband-Statistics-
Q2-2012.pdf [Accessed 29.04.2013].
[94] R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for
Statistical Computing, 2013. URL http://www.R-project.org [Accessed 29.04.2013].
[95] J. Raj. The Art of Computer Systems Performance Analysis. Wiley-Interscience, 1991.
[96] S. Ratnasamy, A. Ermolinskiy, and S. Shenker. Revisiting IP multicast. In Proceedings of the ACM
Special Interest Group on Data Communication Conference (SIGCOMM), 2006.
[97] J. Reich, C. Monsanto, N. Foster, J. Rexford, and D. Walker. Composing Software Defined Networks.
In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation
(NSDI), 2013.
[98] M. Reitblatt, N. Foster, J. Rexford, and D. Walker. Consistent updates for software-defined networks.
In Proceedings of the ACM Workshop on Hot Topics in Networks (HotNets), 2011.
[99] B. Richerzhagen. Supporting Transitions in Peer-to-Peer Video Streaming. Master’s thesis, Technische
Universität Darmstadt, November 2012.
[100] E. Rosen, A. Viswanathan, and R. Callon. Multiprotocol Label Switching Architecture. RFC 3031,
Internet Engineering Task Force, January 2001.
[101] Sandvine Inc. Global Internet Phenomena Report: 2H 2012, 2012. URL http://www.sandvine.
com/downloads/documents/Phenomena_2H_2012/Sandvine_Global_Internet_Phenomena_
Report_2H_2012.pdf [Accessed 29.04.2013].
[102] J. Seedorf, S. Kiesel, and M. Stiemerling. Traffic localization for P2P-applications: The ALTO
approach. In Proceedings of the IEEE International Conference on Peer-to-Peer Computing (P2P),
2009.
[103] P. Srisuresh and K. Egevang. Traditional IP Network Address Translator (Traditional NAT). Technical
Report RFC 3022, Internet Engineering Task Force, January 2001.
[104] I. Stoica, T. S. E. Ng, and H. Zhang. REUNITE: a recursive unicast approach to multicast. In
Proceedings of the Joint Conference of the IEEE Computer and Communications Societies (INFO-
COM), 2000.
[105] K. Stordahl. Long-term penetration and traffic forecasts for the Western European fixed broadband
market. In Proceedings of the European Regional Conference of the International Telecommunications
Society (ITS), 2011.
Bibliography 111

[106] S. Tarkoma, C. E. Rothenberg, and E. Lagerspetz. Theory and Practice of Bloom Filters for Distributed
Systems. IEEE Communications Surveys & Tutorials, 14(1):131–155, February 2012.
[107] X. Tian, Y. Cheng, and B. Liu. Design of a Scalable Multicast Scheme With an Application-Network
Cross-Layer Approach. IEEE Transactions on Multimedia, 11(6):1160–1169, 2009.
[108] A. Voellmy, A. Agarwal, and P. Hudak. Nettle: Functional Reactive Programming for OpenFlow
Networks. Technical Report YALEU/DCS/RR-1431, Yale University, July 2010.
[109] A. Voellmy, H. Kim, and N. Feamster. Procera: a language for high-level reactive network control.
In Proceedings of the ACM SIGCOMM workshop on Hot topics in software defined networks
(HotSDN), 2012.
[110] F. Wang, Y. Xiong, and J. Liu. mTreebone: A Hybrid Tree/Mesh Overlay for Application-Layer Live
Video Multicast. In Proceedings of the IEEE International Conference on Distributed Computing
Systems (ICDCS), 2007.
[111] F. Wang, J. Liu, and Y. Xiong. On Node Stability and Organization in Peer-to-Peer Video Streaming
Systems. IEEE Systems Journal, 5(4):440–450, 2011.
[112] H. Wang, Z. Sun, B. Wang, H. Zhang, and L. Liu. A Point-to-Multipoint Distribution Mechanism for
IPTV Video Network. In IEEE International Conference on Networking, Architecture, and Storage
(NAS), 2011.
[113] C.-C. Wen, C.-J. Chiu, C.-S. Wu, H.-K. Su, and Y.-S. Chu. An Integrated Two-Tier Multicast-Agent
Architecture for All-IP Multicast Transport Services. In International Conference on Broadband
and Wireless Computing, Communication and Applications (BWCCA), 2011.
[114] H. Wickham. ggplot2: Elegant Graphics for Data Analysis. Springer, 2009.
[115] H. Xie, A. Krishnamurthy, A. Silberschatz, and Y. R. Yang. P4P: Explicit communications for cooperative
control between P2P and network providers. Technical report, P4P Working Group, 2007.
URL http://www.dcia.info/documents/P4P_Overview.pdf [Accessed 29.04.2013].
[116] Y. Xiong and L. G. Mason. Restoration strategies and spare capacity requirements in self-healing
ATM networks. In Proceedings of the Joint Conference of the IEEE Computer and Communications
Societies (INFOCOM), 1997.
[117] S. Yasukawa, A. Farrel, and O. Komolafe. An Analysis of Scaling Issues in MPLS-TE Core Networks.
Internet Engineering Task Force, February 2009. URL http://tools.ietf.org/pdf/rfc5439.
pdf [Accessed 29.04.2013].
[118] B. Zhang and H. T. Mouftah. Providing multicast through recursive unicast. IEEE Communications
Magazine, 43(1):115–121, January 2005.
[119] X. Zhang and H. Hassanein. A survey of peer-to-peer live video streaming schemes — An algorithmic
perspective. Computer Networks, 56(15):3548–3579, October 2012.
112 Bibliography