Hadoop Development - CSC

More documents

Recommendations

Info

Volume (cont’d) • If you have a few 100 terabytes of data then products like Teradata, Netezza, Oracle Database Machine, etc., can help you - $$$$$$$ – Note these usually also require “structured” data • If you have many (even tens) of petabytes of data that need to be stored and analyzed – Products like those are cost prohibitive for most of us (assuming that the product can scale that far) • Complexity of analytics is also now becoming a problem – Difficult to express within the constraints of the tools (e.g. SQL) – Time taken to get results is unacceptable TBSC 2009 11/10/2011 12:53 PM 0725-23_TBSC 2009 6
Variety • Continuing increase in the types of data needing to be stored: – Video, voice/sound, images, RFID tags, SMS messages, “chat”, etc. • Many of these are not easy to store/process in relational databases for analysis • Many sources of such data are “unstructured” and/or not easy to structure – Often need to know at design time what kind of “questions” will need to be asked TBSC 2009 11/10/2011 12:53 PM 0725-23_TBSC 2009 7
Page 1 and 2: ©2011 CSC INNOVATE AND DELIVER AN
Page 3 and 4: The Data Challenge •3 dimensions:
Page 5: Volume (cont’d) • IDC now predi
Page 9 and 10: What Is Hadoop? • Google famously
Page 11 and 12: What Is Hadoop? (cont’d) • Hado
Page 13 and 14: Hadoop Architectural Overview (cont
Page 15 and 16: Hadoop Architectural Overview (cont
Page 17 and 18: What is Map-Reduce? • Map-Reduce
Page 19 and 20: What is Map-Reduce? (cont’d) Map
Page 21 and 22: What is Map-Reduce? (cont’d) •
Page 23 and 24: Real world example - a Cyber proble
Page 31 and 32: How to get started... • Download
Page 33 and 34: Questions? TBSC 2009 ? 11/10/2011 1
Page 35: TBSC 2009 11/10/2011 12:53 PM 0725-

Hadoop Development - CSC

Create successful ePaper yourself

Delete template?

Save as template?