Part 4: Index Construction

More documents

Recommendations

Info

SPIMI: Compression Sec. 4.3 Compression makes SPIMI even more efficient. • Compression of terms • Compression of postings See next – (optional)
Distributed indexing Sec. 4.4 For web-scale indexing (don’t try this at home!): must use a distributed computing cluster Individual machines are fault-prone • Can unpredictably slow down or fail How do we exploit such a pool of machines?
Page 1 and 2: Part 4: Index Construction Francesc
Page 3 and 4: Sec. 4.1 Hardware basics Many desi
Page 5 and 6: Sec. 4.1 Hardware basics Servers u
Page 7 and 8: Hardware assumptions Sec. 4.1 symb
Page 9 and 10: Reuters RCV1 statistics Sec. 4.2 sy
Page 11 and 12: Key step After all documents have
Page 13 and 14: Sort-based index construction Sec.
Page 15 and 16: Bottleneck Sec. 4.2 Parse and buil
Page 17 and 18: Blocks consider different documents
Page 19 and 20: Block sorted-based indexing Sec. 4.
Page 21 and 22: Remaining problem with sort-based a
Page 23: SPIMI-Invert Sec. 4.3 When the memo
Page 27 and 28: Google data centers If in a non-fa
Page 29 and 30: Parallel tasks Sec. 4.4 We will us
Page 31 and 32: Inverters Sec. 4.4 An inverter col
Page 33 and 34: MapReduce Sec. 4.4 The index const
Page 35 and 36: Schema for index construction in Ma
Page 37 and 38: Simplest approach Sec. 4.5 Maintai
Page 39 and 40: Sec. 4.5 Logarithmic merge Maintai
Page 41 and 42: Logarithmic merge Sec. 4.5 Auxilia
Page 43 and 44: Dynamic indexing at search engines
Page 45 and 46: Other sorts of indexes Sec. 4.5 P

Part 4: Index Construction

Create successful ePaper yourself

Delete template?

Save as template?