Hadoop, and what Hadoop can do for us?

It's a brief intro for Hadoop. Although it's spark's world now.

Transcript

1. S Hadoop, and what Hadoop can do for us?
2. How to get the info you want from totally 10TB log files? S  After 1 hour S  After 2 hours S  After 3 hours S  … S  After 2 days
3. Hadoop S  is a Java-based framework S  contains S  Hadoop Common S  Hadoop HDFS S  Hadoop MapReduce S  and has lots of children and friends
4. How it works? DEBUG - 2011-09-22 13:56:15 --> Session Class Initialized : OBSession 2.0.0 DEBUG - 2011-09-22 13:56:15 --> Database Driver Class Initialized DEBUG - 2011-09-22 13:56:15 --> A session cookie was not found. DEBUG - 2011-09-22 13:56:16 --> Sending session cookie DEBUG - 2011-09-22 13:56:16 --> New session started. DEBUG - 2011-09-22 13:56:16 --> Controller Class Initialized DEBUG - 2011-09-22 13:56:16 --> [MYLanguage] Language file loaded: language/EN-US/portal_lang.xml ERROR - 2011-09-22 13:56:16 --> [MYConfig] MY Config class in framework loaded DEBUG - 2011-09-22 13:56:16 --> Config Class Initialized ERROR - 2011-09-22 13:56:16 --> Hooks Class Initialized Reducer DEBUG 8 ERROR 2
5. What are the difficult parts? S  Mapper & Reducer S  Mapper & Reducer S  Mapper & Reducer S  Mapper & Reducer S  Mapper & Reducer
6. and next? S  Search & index
7. CloudGo!