Introduction to Cascalog Stefan Hübner, Nokia Berlin EuroClojure 2012 

Nokia Maps • Address & POI Search • Category Search • Nearby Recommendations 

Hadoop • Batch Processing • (Very) Large Scale • Distributed Filesystem • Parallel Computation • Fault-Tolerant 

Hadoop • Batch Processing • (Very) Large Scale • Distributed Filesystem • Parallel Computation • Fault-Tolerant 

Hadoop • Batch Processing • (Very) Large Scale • Distributed Filesystem • Parallel Computation • Fault-Tolerant 

Hadoop MapReduce API • Tedious and verbose • Hard to test • Hard to refactor 

• Tedious and verbose • Hard to test • Hard to refactor Hadoop MapReduce API 

Pig and Hive • Define their own query language • Custom operations in Java, Python, ... • Non-intuitive integration 

(()) 

Star Trek I - "The Motion Picture", Paramount Pictures V'Gr's question 

Star Trek I - "The Motion Picture", Paramount Pictures "Ist das wirklich alles? Ist da sonst gar nichts mehr?" 

Cascalog Cascalog Cascading Hadoop Abstraction Variables and logic Tuples, data workflows Key/value pairs, simple aggregation slide (c) Nathan Marz, reproduced with permission 

Queries (<- ; defines a query [?person] ; output variables (age ?person ?age) ; generator with two variables (< ?age 30)) ; filter 

Queries (<- [?person] (age ?person ?age) ; generator with two variables (< ?age 30)) ; filter Predicates 

Predicates • Functions • Filters • Aggregators • Generators 

No content

Thank you! Stefan Hübner, @sthuebner http://knowyourmeme.com/memes/cereal-guy