Programming Model and Protocols for Reconfigurable Distributed ...

More documents

Recommendations

Info

TRITA-ICT/ECS AVH 13:07 ISSN 1653-6363 ISRN KTH/ICT/ECS/AVH-13/07-SE ISBN 978-91-7501-694-8 KTH – Royal Institute of Technology School of Information and Communication Technology Electrum 229, SE-164 40 Kista, Sweden SICS Dissertation Series 62 ISSN 1101-1335 ISRN SICS-D-62-SE Swedish Institute of Computer Science Computer Systems Laboratory Box 1263, SE-164 29 Kista, Sweden © Cosmin Ionel Arad, April 2013 Printed and bound by Universitetsservice US-AB
Abstract Distributed systems are everywhere. From large datacenters to mobile devices, an ever richer assortment of applications and services relies on distributed systems, infrastructure, and protocols. Despite their ubiquity, testing and debugging distributed systems remains notoriously hard. Moreover, aside from inherent design challenges posed by partial failure, concurrency, or asynchrony, there remain significant challenges in the implementation of distributed systems. These programming challenges stem from the increasing complexity of the concurrent activities and reactive behaviors in a distributed system on the one hand, and the need to effectively leverage the parallelism offered by modern multi-core hardware, on the other hand. This thesis contributes Kompics, a programming model designed to alleviate some of these challenges. Kompics is a component model and programming framework for building distributed systems by composing message-passing concurrent components. Systems built with Kompics leverage multi-core machines out of the box, and they can be dynamically reconfigured to support hot software upgrades. A simulation framework enables deterministic execution replay for debugging, testing, and reproducible behavior evaluation for largescale Kompics distributed systems. The same system code is used for both simulation and production deployment, greatly simplifying the system development, testing, and debugging cycle. We highlight the architectural patterns and abstractions facilitated by Kompics through a case study of a non-trivial distributed key-value storage system. CATS is a scalable, fault-tolerant, elastic, and self-managing key-value store which trades off service availability for guarantees of atomic data consistency and tolerance to network partitions. We present the composition architecture for the numerous protocols employed by the CATS system, as well as our methodology for testing the correctness of key CATS algorithms using the Kompics simulation framework. Results from a comprehensive performance evaluation attest that CATS achieves its claimed properties and delivers a level of performance competitive with similar systems which provide only weaker consistency guarantees. More importantly, this testifies that Kompics admits efficient system implementations. Its use as a teaching framework as well as its use for rapid prototyping, development, and evaluation of a myriad of scalable distributed systems, both within and outside our research group, confirm the practicality of Kompics. v
Page 1: Programming Model and Protocols for
Page 5: Programming Model and Protocols for
Page 11 and 12: Acknowledgements I am truly honoure
Page 13 and 14: Contents List of Figures List of So
Page 15 and 16: CONTENTS xiii 4.4 Testing and Debug
Page 17 and 18: List of Figures 2.1 An example of p
Page 19 and 20: List of Source Code Listings 2.1 A
Page 21 and 22: Chapter 1 Introduction A large and
Page 23 and 24: 1.1. MOTIVATION 3 1.1 Motivation Mo
Page 25 and 26: 1.3. THESIS CONTRIBUTIONS 5 This fa
Page 27 and 28: 1.3. THESIS CONTRIBUTIONS 7 systems
Page 29 and 30: 1.5. ORGANIZATION 9 • Cosmin Arad
Page 31: 1.5. ORGANIZATION 11 admits efficie
Page 35 and 36: Chapter 2 Component Model Kompics i
Page 37 and 38: 2.1. CONCEPTS IN KOMPICS 17 specifi
Page 39 and 40: 2.1. CONCEPTS IN KOMPICS 19 + + F
Page 41 and 42: 2.1. CONCEPTS IN KOMPICS 21 MyCompo
Page 43 and 44: 2.2. KOMPICS OPERATIONS 23 Code 2.7
Page 45 and 46: 2.3. PUBLISH-SUBSCRIBE MESSAGE PASS
Page 47 and 48: 2.4. CHANNEL EVENT FILTERING 27 Fig
Page 49 and 50: 2.5. REQUEST-RESPONSE INTERACTION 2
Page 51 and 52: 2.6. COMPONENT INITIALIZATION AND L
Page 53 and 54: 2.8. NON-BLOCKING RECEIVE 33 + som
Page 55: 2.9. DYNAMIC RECONFIGURATION 35 2.9
Page 58 and 59:
38 CHAPTER 3. PROGRAMMING PATTERNS
Page 60 and 61:
Page 62 and 63:
Page 64 and 65:
Page 66 and 67:
Page 68 and 69:
Page 70 and 71:
Page 72 and 73:
Page 74 and 75:
54 CHAPTER 4. IMPLEMENTATION ASPECT
Page 76 and 77:
Page 78 and 79:
Page 80 and 81:
Page 82 and 83:
Page 84 and 85:
Page 86 and 87:
Page 88 and 89:
Page 90 and 91:
Page 92 and 93:
Page 94 and 95:
Page 96 and 97:
Page 98 and 99:
Page 100 and 101:
Page 102 and 103:
Page 104 and 105:
Page 106 and 107:
86 CHAPTER 5. KOMPICS DISCUSSION AN
Page 108 and 109:
Page 110 and 111:
Page 112 and 113:
Page 115 and 116:
Chapter 6 Background, Motivation, a
Page 117 and 118:
CHAPTER 6. BACKGROUND, MOTIVATION,
Page 119 and 120:
6.2. CONSISTENCY, AVAILABILITY, AND
Page 121 and 122:
6.4. QUORUM-BASED REPLICATION SYSTE
Page 123:
6.5. PROBLEM STATEMENT 103 5 10 15
Page 126 and 127:
106 CHAPTER 7. CONSISTENT QUORUMS g
Page 128 and 129:
108 CHAPTER 7. CONSISTENT QUORUMS A
Page 130 and 131:
110 CHAPTER 7. CONSISTENT QUORUMS v
Page 132 and 133:
112 CHAPTER 7. CONSISTENT QUORUMS b
Page 134 and 135:
114 CHAPTER 7. CONSISTENT QUORUMS A
Page 136 and 137:
116 CHAPTER 7. CONSISTENT QUORUMS v
Page 138 and 139:
118 CHAPTER 7. CONSISTENT QUORUMS t
Page 140 and 141:
120 CHAPTER 7. CONSISTENT QUORUMS C
Page 142 and 143:
122 CHAPTER 7. CONSISTENT QUORUMS F
Page 144 and 145:
124 CHAPTER 8. CATS SYSTEM ARCHITEC
Page 146 and 147:
Page 148 and 149:
Page 150 and 151:
Page 152 and 153:
Page 154 and 155:
Page 156 and 157:
Page 159 and 160:
Chapter 9 Scalability, Elasticity,
Page 161 and 162:
9.2. THROUGHPUT AND LATENCY 141 Thr
Page 163 and 164:
9.3. SCALABILITY 143 Throughput [10
Page 165 and 166:
9.5. OVERHEAD OF CONSISTENT QUORUMS
Page 167 and 168:
9.5. OVERHEAD OF CONSISTENT QUORUMS
Page 169 and 170:
9.6. COMPARISON WITH CASSANDRA 149
Page 171 and 172:
Chapter 10 CATS Discussion and Comp
Page 173 and 174:
10.2. SEQUENTIAL CONSISTENCY AT SCA
Page 175 and 176:
10.6. SCALABLE AND CONSISTENT KEY-V
Page 177 and 178:
Chapter 11 Conclusions We are witne
Page 179 and 180:
CHAPTER 11. CONCLUSIONS 159 replica
Page 181 and 182:
11.1. KOMPICS LIMITATIONS AND LESSO
Page 183 and 184:
11.2. CATS LIMITATIONS AND LESSONS
Page 185 and 186:
11.3. FUTURE WORK 165 distribution
Page 187 and 188:
Appendix A Kompics Abstract Syntax
Page 189:
APPENDIX A. KOMPICS ABSTRACT SYNTAX
Page 192 and 193:
172 APPENDIX B. KOMPICS OPERATIONAL
Page 194 and 195:
Page 196 and 197:
Page 199 and 200:
Bibliography [1] Daniel Abadi. Cons
Page 201 and 202:
BIBLIOGRAPHY 181 [16] Cosmin Arad,
Page 203 and 204:
BIBLIOGRAPHY 183 [38] Kenneth P. Bi
Page 205 and 206:
BIBLIOGRAPHY 185 [59] Brian F. Coop
Page 207 and 208:
BIBLIOGRAPHY 187 [77] Patrick Th. E
Page 209 and 210:
BIBLIOGRAPHY 189 [98] Vassos Hadzil
Page 211 and 212:
BIBLIOGRAPHY 191 [120] David Karger
Page 213 and 214:
BIBLIOGRAPHY 193 [142] Wyatt Lloyd,
Page 215 and 216:
BIBLIOGRAPHY 195 [164] Salman Niazi
Page 217 and 218:
BIBLIOGRAPHY 197 [186] Rackspace Cl
Page 219 and 220:
BIBLIOGRAPHY 199 [205] Yair Sovran,
Page 221:
BIBLIOGRAPHY 201 [226] András Varg
Page 224 and 225:
204 ACRONYMS DES discrete-event sim
Page 226 and 227:
206 ACRONYMS SMART Service Migratio
Page 228 and 229:
208 SICS DISSERTATION SERIES 9. Lar
Page 230 and 231:
210 SICS DISSERTATION SERIES 40. Ma
Page 232:
Colophon This dissertation was type
show all

Programming Model and Protocols for Reconfigurable Distributed ...

Create successful ePaper yourself

Delete template?

Save as template?