Graph Processing & Bulk Synchronous Parallel Model

More documents

Recommendations

Info

<strong>Graph</strong> ParFFoning • VerFces are assigned to machines based on hash(vertex.id) mod N • Can define other parFFons: co-‐locate all web pages from the same site • Sparsest Cut Problem: minimize the edges across parFFons Lecture 14 : 590.02 Spring 13 22
<strong>Processing</strong> • Master coordinates a set of workers. – Determines the number of parFFons – Determines assignment of parFFons to workers • Worker processes one or more parFFons – Workers know the enFre set of parFFon to worker assignments and the parFFon funcFon – All verFces in Worker’s parFFon are iniFalized to acFve – Worker loops through vertex list and sends any messages asynchronously – Worker noFfies master of # acFve verFces at the end of a superstep Lecture 14 : 590.02 Spring 13 23
Page 1 and 2: Graph Processing & Bulk Synchro
Page 3 and 4: This Class • IteraFon Aware M
Page 5 and 6: IteraFve ComputaFons PageRank: d
Page 7 and 8: HaLoop Map Reduce Map Reduce Loop
Page 9 and 10: PREGEL Lecture 14 : 590.02 Spr
Page 11 and 12: Pregel Overview • Processing o
Page 13 and 14: Example 3 6 2 1 Superstep 0 6 6 2
Page 15 and 16: Vertex API • MessageIterator c
Page 17 and 18: Combiners • If messages are a
Page 19 and 20: AggregaFon • Global communicaFo
Page 21: PREGEL ARCHITECTURE Lecture 14
Page 25: Summary • Map-‐reduce has

Graph Processing & Bulk Synchronous Parallel Model

Create successful ePaper yourself

Delete template?

Save as template?