Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cascalog

Avatar for αλεx π αλεx π
May 22, 2013
Technology
4
140

 Cascalog

Short demo talk on Cascalog on Hadoop UG in Munich

Avatar for αλεx π

αλεx π

May 22, 2013
Tweet

More Decks by αλεx π

See All by αλεx π
Scalable Time Series With Cassandra
Avatar for αλεx π ifesdjeen
1
370
Bayesian Inference is known to make machines biased
Avatar for αλεx π ifesdjeen
2
370
Cassandra for Data Analytics Backends
Avatar for αλεx π ifesdjeen
7
430
Stream Processing and Functional Programming
Avatar for αλεx π ifesdjeen
1
750
PolyConf 2015 - Rocking the Time Series boat with C, Haskell and ClojureScript
Avatar for αλεx π ifesdjeen
0
470
Clojure - A Sweetspot for Analytics
Avatar for αλεx π ifesdjeen
8
2.1k
Going Off Heap
Avatar for αλεx π ifesdjeen
3
1.9k
Always be learning
Avatar for αλεx π ifesdjeen
1
140
Learn Yourself Emacs For Great Good workshop slides
Avatar for αλεx π ifesdjeen
3
330

Other Decks in Technology

See All in Technology
ゼロコード計装導入後のカスタム計装でさらに可観測性を高めよう
Avatar for SansanTech sansantech
PRO
1
600
新米エンジニアをTech Leadに任命する ー 成長を支える挑戦的な人と組織のマネジメント
Avatar for Naoto Sato naopr
1
330
AI機能プロジェクト炎上の 3つのしくじりと学び
Avatar for nakawai nakawai
0
180
abema-trace-sampling-observability-cost-optimization
Avatar for tetsuya28 tetsuya28
0
400
ざっくり学ぶ 『エンジニアリングリーダー 技術組織を育てるリーダーシップと セルフマネジメント』 / 50 minute Engineering Leader
Avatar for iwashi iwashi86
8
3.9k
マルチエージェントのチームビルディング_2025-10-25
Avatar for shino shinoyamada
0
230
アウトプットから始めるOSSコントリビューション 〜eslint-plugin-vueの場合〜 #vuefes
Avatar for 弁護士ドットコム bengo4com
3
1.9k
SOTA競争から人間を超える画像認識へ
Avatar for Yosuke Shinya shinya7y
0
650
初海外がre:Inventだった人間の感じたこと
Avatar for tommy tommy0124
1
140
SRE × マネジメントレイヤーが挑戦した組織・会社のオブザーバビリティ改革 ― ビジネス価値と信頼性を両立するリアルな挑戦
Avatar for coconala_engineer coconala_engineer
0
340
AWSが好きすぎて、41歳でエンジニアになり、AAIを経由してAWSパートナー企業に入った話
Avatar for Yuuki Yamashita yama3133
2
220
[re:Inent2025事前勉強会(有志で開催)] re:Inventで見つけた人生をちょっと変えるコツ
Avatar for sh_fk2 sh_fk2
1
1.1k

Featured

See All Featured
Mobile First: as difficult as doing things right
Avatar for Swwweet swwweet
225
10k
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
38
2.9k
[RailsConf 2023] Rails as a piece of cake
Avatar for Vladimir Dementyev palkan
57
6k
Visualization
Avatar for Eitan Lees eitanlees
150
16k
Balancing Empowerment & Direction
Avatar for Lara Hogan lara
5
710
Automating Front-end Workflow
Avatar for Addy Osmani addyosmani
1371
200k
GitHub's CSS Performance
Avatar for Jon Rohan jonrohan
1032
470k
Into the Great Unknown - MozCon
Avatar for Noah Learner thekraken
40
2.1k
The Cult of Friendly URLs
Avatar for Andy Hume andyhume
79
6.6k
Keith and Marios Guide to Fast Websites
Avatar for Keith Pitt keithpitt
412
23k
The Art of Delivering Value - GDevCon NA Keynote
Avatar for David Neal reverentgeek
16
1.7k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
37
2.6k

Transcript

  1. Cascalog Hassle-free MapReduce that matches your scale Thursday, May 23,

    13

  2. Thursday, May 23, 13

  3. Setting expecations •This is not a guide •And not a

    tutorial •Doesn’t claim to be complete •Mostly to give you an idea •And encourage you to explore further Thursday, May 23, 13

  4. How much time do you spend on writing logic that

    framework should take care of? Thursday, May 23, 13

  5. How easy is it to debug your map/reduce aggragation? Thursday,

    May 23, 13

  6. Hadoop + Java composable, but too vebrose Pig, Hive too

    concrete, lack of abstraction and composition Thursday, May 23, 13

  7. Thursday, May 23, 13

  8. • Clear, declarative syntax • Inner and outer joins •

    Aggregators • Functions • Subqueries, composition • Sorting • Performant Thursday, May 23, 13

  9. Casca-WHAT? • Built on top of Hadoop (MapReduce) • Cascading

    (tuples, workflows, job execution) • Written in Clojure • Datalog (logic programming) Thursday, May 23, 13

  10. Abstract evrthn! Thursday, May 23, 13

  11. Source where data pours from Thursday, May 23, 13

  12. Pipe that data flows through Thursday, May 23, 13

  13. Filter that makes sure that only good stuff goes through

    Thursday, May 23, 13

  14. Tuple they actually flow Thursday, May 23, 13

  15. Thursday, May 23, 13

  16. Query anatomy Thursday, May 23, 13

  17. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Thursday, May 23, 13

  18. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Output Thursday, May 23, 13

  19. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    output vars Thursday, May 23, 13

  20. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Input input vars Thursday, May 23, 13

  21. (?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))

    Logic/aggregations Thursday, May 23, 13

  22. Sources and sinks • HDFS (go figure) • Cassandra •

    MongoDB • SQL data sources • File system • Memory sources Thursday, May 23, 13

  23. (?<- (stdout) [?person] (age ?person 25)) Exact match of second

    element in a tuple Thursday, May 23, 13

  24. (defn younger-than? [limit age] (< age limit)) (?<- (stdout) [?person

    ?age] (age ?person ?age) (younger-than? 32 ?age)) Predicate match, fn call Predicate Thursday, May 23, 13

  25. (?<- (stdout) [?person ?count] (follows ?person _) (c/count ?count)) Aggregation

    Thursday, May 23, 13

  26. SHOWTIME! Thursday, May 23, 13

  27. Benefits •Query language is same as application language •Subqueries, reusability

    •Ad-hoc querying •Cascading underneath, so taps for all DBs work •Reuse application logic •Text editor integration Thursday, May 23, 13

  28. @ifesdjeen (twitter/github) Thursday, May 23, 13