Part 4: Index Construction

More documents

Recommendations

Info

Dynamic indexing Sec. 4.5 Up to now, we have assumed that collections are static They rarely are: • Documents come in over time and need to be inserted • Documents are deleted and modified This means that the dictionary and postings lists have to be modified: • Postings updates for terms already in dictionary • New terms added to dictionary.
Simplest approach Sec. 4.5 Maintain big main index New docs go into small auxiliary index Search across both, merge results Deletions • Invalidation bit-vector for deleted docs • Filter docs output on a search result by this invalidation bit-vector Periodically, re-index into one main index.
Page 1 and 2: Part 4: Index Construction Francesc
Page 3 and 4: Sec. 4.1 Hardware basics Many desi
Page 5 and 6: Sec. 4.1 Hardware basics Servers u
Page 7 and 8: Hardware assumptions Sec. 4.1 symb
Page 9 and 10: Reuters RCV1 statistics Sec. 4.2 sy
Page 11 and 12: Key step After all documents have
Page 13 and 14: Sort-based index construction Sec.
Page 15 and 16: Bottleneck Sec. 4.2 Parse and buil
Page 17 and 18: Blocks consider different documents
Page 19 and 20: Block sorted-based indexing Sec. 4.
Page 21 and 22: Remaining problem with sort-based a
Page 23 and 24: SPIMI-Invert Sec. 4.3 When the memo
Page 25 and 26: Distributed indexing Sec. 4.4 For
Page 27 and 28: Google data centers If in a non-fa
Page 29 and 30: Parallel tasks Sec. 4.4 We will us
Page 31 and 32: Inverters Sec. 4.4 An inverter col
Page 33 and 34: MapReduce Sec. 4.4 The index const
Page 35: Schema for index construction in Ma
Page 39 and 40: Sec. 4.5 Logarithmic merge Maintai
Page 41 and 42: Logarithmic merge Sec. 4.5 Auxilia
Page 43 and 44: Dynamic indexing at search engines
Page 45 and 46: Other sorts of indexes Sec. 4.5 P

Part 4: Index Construction

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?