Introduction and MapReduce - SNAP - Stanford University

More documents

Recommendations

Info

Hadoop Wiki Introduction http://wiki.apache.org/lucene-hadoop/ Getting Started http://wiki.apache.org/lucenehadoop/GettingStartedWithHadoop Map/Reduce Overview http://wiki.apache.org/lucene-hadoop/HadoopMapReduce http://wiki.apache.org/lucenehadoop/HadoopMapRedClasses Eclipse Environment http://wiki.apache.org/lucene-hadoop/EclipseEnvironment Javadoc http://lucene.apache.org/hadoop/docs/api/ 1/8/2012 Jure Leskovec, Stanford CS246: Mining Massive Datasets, http://cs246.stanford.edu 65
Releases from Apache download mirrors http://www.apache.org/dyn/closer.cgi/lucene/had oop/ Nightly builds of source http://people.apache.org/dist/lucene/hadoop/nig htly/ Source code from subversion http://lucene.apache.org/hadoop/version_control .html 1/8/2012 Jure Leskovec, Stanford CS246: Mining Massive Datasets, http://cs246.stanford.edu 66
Page 1 and 2:
CS246: Mining Massive Datasets Jure
Page 3 and 4:
Course website: http://cs246.stanfo
Page 5 and 6:
Short weekly quizzes: 20% Short e-
Page 7 and 8:
Recitation sessions: Review of pro
Page 9 and 10:
CS345a: Data mining got split into
Page 11 and 12:
Much of the course will be devoted
Page 13 and 14: High-dimensional data: Locality Se
Page 16 and 17: Discovery of patterns and models th
Page 18 and 19: Scalability Dimensionality Comple
Page 20 and 21: A big data-mining risk is that you
Page 22: He told these people they had ESP a
Page 25 and 26: 20+ billion web pages x 20KB = 400+
Page 27 and 28: Large-scale computing for data mini
Page 29 and 30: Problem If nodes fail, how to stor
Page 31 and 32: Reliable distributed file system D
Page 33 and 34: Case 1: File too large for memory,
Page 35 and 36: Input key-value pairs k k k … v v
Page 37 and 38: Input: a set of key/value pairs Pr
Page 39 and 40: map(key, value): // key: document n
Page 41 and 42: MAP: reads input and produces a set
Page 43 and 44: Input and final output are stored o
Page 45 and 46: Map worker failure Map tasks compl
Page 47 and 48: Fine granularity tasks: map tasks >
Page 49 and 50: Often a map task will produce many
Page 51 and 52: 1/8/2012 Jure Leskovec, Stanford CS
Page 53 and 54: Statistical machine translation: N
Page 55 and 56: Use a hash function h from B-values
Page 57 and 58: For a map-reduce algorithm: Commun
Page 59 and 60: Total communication cost = O(|R|+|S
Page 61 and 62: Google Not available outside Googl
Page 63: Jeffrey Dean and Sanjay Ghemawat: M
show all

Introduction and MapReduce - SNAP - Stanford University

Create successful ePaper yourself

Delete template?

Save as template?