Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cascalog

Avatar for αλεx π αλεx π
May 22, 2013
Technology
4
150

 Cascalog

Short demo talk on Cascalog on Hadoop UG in Munich

Avatar for αλεx π

αλεx π

May 22, 2013
Tweet

More Decks by αλεx π

See All by αλεx π
Scalable Time Series With Cassandra
Avatar for αλεx π ifesdjeen
1
390
Bayesian Inference is known to make machines biased
Avatar for αλεx π ifesdjeen
2
380
Cassandra for Data Analytics Backends
Avatar for αλεx π ifesdjeen
7
450
Stream Processing and Functional Programming
Avatar for αλεx π ifesdjeen
1
760
PolyConf 2015 - Rocking the Time Series boat with C, Haskell and ClojureScript
Avatar for αλεx π ifesdjeen
0
490
Clojure - A Sweetspot for Analytics
Avatar for αλεx π ifesdjeen
8
2.1k
Going Off Heap
Avatar for αλεx π ifesdjeen
3
1.9k
Always be learning
Avatar for αλεx π ifesdjeen
1
150
Learn Yourself Emacs For Great Good workshop slides
Avatar for αλεx π ifesdjeen
3
340

Other Decks in Technology

See All in Technology
usermode linux without MMU - fosdem2026 kernel devroom
Avatar for Hajime Tazaki thehajime
0
230
All About Sansan – for New Global Engineers
Avatar for Sansan, Inc. sansan33
PRO
1
1.3k
GSIが複数キー対応したことで、俺達はいったい何が嬉しいのか？
Avatar for Makky12 smt7174
3
150
こんなところでも（地味に）活躍するImage Modeさんを知ってるかい？- Image Mode for OpenShift -
Avatar for Masataka Tsukamoto tsukaman
0
120
Codex 5.3 と Opus 4.6 にコーポレートサイトを作らせてみた / Codex 5.3 vs Opus 4.6
Avatar for ama-ch ama_ch
0
120
Introduction to Sansan, inc / Sansan Global Development Center, Inc.
Avatar for Sansan, Inc. sansan33
PRO
0
3k
Bedrock PolicyでAmazon Bedrock Guardrails利用を強制してみた
Avatar for Yudai Jinno yuu551
0
190
Claude_CodeでSEOを最適化する_AI_Ops_Community_Vol.2__マーケティングx_AIはここまで進化した.pdf
Avatar for 小笠原理久 riku_423
2
530
20260208_第66回 コンピュータビジョン勉強会
Avatar for KeiichiIto1978 keiichiito1978
0
100
顧客の言葉を、そのまま信じない勇気
Avatar for yamatai12 yamatai1212
1
350
Kiro IDEのドキュメントを全部読んだので地味だけどちょっと嬉しい機能を紹介する
Avatar for khmoryz khmoryz
0
180
ブロックテーマ、WordPress でウェブサイトをつくるということ / 2026.02.07 Gifu WordPress Meetup
Avatar for Toro_Unit (Hiroshi Urabe) torounit
0
170

Featured

See All Featured
Money Talks: Using Revenue to Get Sh*t Done
Avatar for Nikki Halliwell nikkihalliwell
0
150
Mozcon NYC 2025: Stop Losing SEO Traffic
Avatar for Sam Torres samtorres
0
140
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
Avatar for Eileen M. Uchitelle eileencodes
141
34k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
Avatar for panda_program panda_program
122
21k
Agile Leadership in an Agile Organization
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
79
Music & Morning Musume
Avatar for Bryan Veloso bryan
47
7.1k
Bash Introduction
Avatar for André Augusto Costa Santos 62gerente
615
210k
Optimising Largest Contentful Paint
Avatar for Harry Roberts csswizardry
37
3.6k
Into the Great Unknown - MozCon
Avatar for Noah Learner thekraken
40
2.3k
Building Experiences: Design Systems, User Experience, and Full Site Editing
Avatar for Michelle Schulp Hunt marktimemedia
0
410
Practical Orchestrator
Avatar for Shlomi Noach shlominoach
191
11k
Are puppies a ranking factor?
Avatar for Jono Alderson jonoalderson
1
2.7k

Transcript

  1. Cascalog Hassle-free MapReduce that matches your scale Thursday, May 23,

    13

  2. Thursday, May 23, 13

  3. Setting expecations •This is not a guide •And not a

    tutorial •Doesn’t claim to be complete •Mostly to give you an idea •And encourage you to explore further Thursday, May 23, 13

  4. How much time do you spend on writing logic that

    framework should take care of? Thursday, May 23, 13

  5. How easy is it to debug your map/reduce aggragation? Thursday,

    May 23, 13

  6. Hadoop + Java composable, but too vebrose Pig, Hive too

    concrete, lack of abstraction and composition Thursday, May 23, 13

  7. Thursday, May 23, 13

  8. • Clear, declarative syntax • Inner and outer joins •

    Aggregators • Functions • Subqueries, composition • Sorting • Performant Thursday, May 23, 13

  9. Casca-WHAT? • Built on top of Hadoop (MapReduce) • Cascading

    (tuples, workflows, job execution) • Written in Clojure • Datalog (logic programming) Thursday, May 23, 13

  10. Abstract evrthn! Thursday, May 23, 13

  11. Source where data pours from Thursday, May 23, 13

  12. Pipe that data flows through Thursday, May 23, 13

  13. Filter that makes sure that only good stuff goes through

    Thursday, May 23, 13

  14. Tuple they actually flow Thursday, May 23, 13

  15. Thursday, May 23, 13

  16. Query anatomy Thursday, May 23, 13

  17. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Thursday, May 23, 13

  18. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Output Thursday, May 23, 13

  19. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    output vars Thursday, May 23, 13

  20. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Input input vars Thursday, May 23, 13

  21. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Logic/aggregations Thursday, May 23, 13

  22. Sources and sinks • HDFS (go figure) • Cassandra •

    MongoDB • SQL data sources • File system • Memory sources Thursday, May 23, 13

  23. (?<- (stdout) [?person] (age ?person 25)) Exact match of second

    element in a tuple Thursday, May 23, 13

  24. (defn younger-than? [limit age] (< age limit)) (?<- (stdout) [?person

    ?age] (age ?person ?age) (younger-than? 32 ?age)) Predicate match, fn call Predicate Thursday, May 23, 13

  25. (?<- (stdout) [?person ?count] (follows ?person _) (c/count ?count)) Aggregation

    Thursday, May 23, 13

  26. SHOWTIME! Thursday, May 23, 13

  27. Benefits •Query language is same as application language •Subqueries, reusability

    •Ad-hoc querying •Cascading underneath, so taps for all DBs work •Reuse application logic •Text editor integration Thursday, May 23, 13

  28. @ifesdjeen (twitter/github) Thursday, May 23, 13