Part 4: Index Construction

More documents

Recommendations

Info

A Reuters RCV1 document Sec. 4.2
Reuters RCV1 statistics Sec. 4.2 symbol statistic value N documents 800,000 L avg. # tokens per doc 200 M terms (= word types) 400,000 avg. # bytes per token 6 (incl. spaces/punct.) avg. # bytes per token 4.5 (without spaces/punct.) avg. # bytes per term 7.5 T non-positional postings 100,000,000 •4.5 bytes per word token vs. 7.5 bytes per word type: why? •Why T \= N*L?
Page 1 and 2: Part 4: Index Construction Francesc
Page 3 and 4: Sec. 4.1 Hardware basics Many desi
Page 5 and 6: Sec. 4.1 Hardware basics Servers u
Page 7: Hardware assumptions Sec. 4.1 symb
Page 11 and 12: Key step After all documents have
Page 13 and 14: Sort-based index construction Sec.
Page 15 and 16: Bottleneck Sec. 4.2 Parse and buil
Page 17 and 18: Blocks consider different documents
Page 19 and 20: Block sorted-based indexing Sec. 4.
Page 21 and 22: Remaining problem with sort-based a
Page 23 and 24: SPIMI-Invert Sec. 4.3 When the memo
Page 25 and 26: Distributed indexing Sec. 4.4 For
Page 27 and 28: Google data centers If in a non-fa
Page 29 and 30: Parallel tasks Sec. 4.4 We will us
Page 31 and 32: Inverters Sec. 4.4 An inverter col
Page 33 and 34: MapReduce Sec. 4.4 The index const
Page 35 and 36: Schema for index construction in Ma
Page 37 and 38: Simplest approach Sec. 4.5 Maintai
Page 39 and 40: Sec. 4.5 Logarithmic merge Maintai
Page 41 and 42: Logarithmic merge Sec. 4.5 Auxilia
Page 43 and 44: Dynamic indexing at search engines
Page 45 and 46: Other sorts of indexes Sec. 4.5 P

Part 4: Index Construction

Create successful ePaper yourself

Delete template?

Save as template?