Hadoop: The Definitive Guide - Cdn.oreilly.com

15. Sqoop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527
Getting Sqoop 527
Sqoop Connectors 529
A Sample Import 529
Text and Binary File Formats 532
Generated Code 532
Additional Serialization Systems 533
Imports: A Deeper Look 533
Controlling the Import 535
Imports and Consistency 536
Direct-mode Imports 536
Working with Imported Data 536
Imported Data and Hive 537
Importing Large Objects 540
Performing an Export 542
Exports: A Deeper Look 543
Exports and Transactionality 545
Exports and SequenceFiles 545
16. Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547
Hadoop Usage at Last.fm 547
Last.fm: The Social Music Revolution 547
Hadoop at Last.fm 547
Generating Charts with Hadoop 548
The Track Statistics Program 549
Summary 556
Hadoop and Hive at Facebook 556
Hadoop at Facebook 556
Hypothetical Use Case Studies 559
Hive 562
Problems and Future Work 566
Nutch Search Engine 567
Data Structures 568
Selected Examples of Hadoop Data Processing in Nutch 571
Summary 580
Log Processing at Rackspace 581
Requirements/The Problem 581
Brief History 582
Choosing Hadoop 582
Collection and Storage 582
MapReduce for Logs 583
Cascading 589
Fields, Tuples, and Pipes 590
Table of Contents | xiii