Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Cascalog
Search
αλεx π
May 22, 2013
Technology
4
140
Cascalog
Short demo talk on Cascalog on Hadoop UG in Munich
αλεx π
May 22, 2013
Tweet
Share
More Decks by αλεx π
See All by αλεx π
Scalable Time Series With Cassandra
ifesdjeen
1
370
Bayesian Inference is known to make machines biased
ifesdjeen
2
370
Cassandra for Data Analytics Backends
ifesdjeen
7
430
Stream Processing and Functional Programming
ifesdjeen
1
750
PolyConf 2015 - Rocking the Time Series boat with C, Haskell and ClojureScript
ifesdjeen
0
470
Clojure - A Sweetspot for Analytics
ifesdjeen
8
2.1k
Going Off Heap
ifesdjeen
3
1.9k
Always be learning
ifesdjeen
1
140
Learn Yourself Emacs For Great Good workshop slides
ifesdjeen
3
330
Other Decks in Technology
See All in Technology
非エンジニアのあなたもできる&もうやってる!コンテキストエンジニアリング
findy_eventslides
3
870
KMP の Swift export
kokihirokawa
0
300
業務自動化プラットフォーム Google Agentspace に入門してみる #devio2025
maroon1st
0
180
Modern_Data_Stack最新動向クイズ_買収_AI_激動の2025年_.pdf
sagara
0
170
北海道の人に知ってもらいたいGISスポット / gis-spot-in-hokkaido-2025
sakaik
0
200
それでも私はContextに値を詰めたい | Go Conference 2025 / go conference 2025 fill context
budougumi0617
4
1.1k
Geospatialの世界最前線を探る [2025年版]
dayjournal
3
450
タスクって今どうなってるの?3.14の新機能 asyncio ps と pstree でasyncioのデバッグを (PyCon JP 2025)
jrfk
1
200
【新卒研修資料】LLM・生成AI研修 / Large Language Model・Generative AI
brainpadpr
23
16k
バイブコーディングと継続的デプロイメント
nwiizo
2
380
AIが書いたコードをAIが検証する!自律的なモバイルアプリ開発の実現
henteko
1
240
“2件同時配達”の開発舞台裏 〜出前館PMが挑んだダブルピック実現に向けた体験設計〜
demaecan
0
180
Featured
See All Featured
Intergalactic Javascript Robots from Outer Space
tanoku
273
27k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
34
6.1k
Faster Mobile Websites
deanohume
310
31k
How to Think Like a Performance Engineer
csswizardry
27
2k
Balancing Empowerment & Direction
lara
4
670
Bash Introduction
62gerente
615
210k
Designing for humans not robots
tammielis
254
25k
RailsConf 2023
tenderlove
30
1.2k
Thoughts on Productivity
jonyablonski
70
4.8k
jQuery: Nuts, Bolts and Bling
dougneiner
64
7.9k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
32
1.6k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
162
15k
Transcript
Cascalog Hassle-free MapReduce that matches your scale Thursday, May 23,
13
Thursday, May 23, 13
Setting expecations •This is not a guide •And not a
tutorial •Doesn’t claim to be complete •Mostly to give you an idea •And encourage you to explore further Thursday, May 23, 13
How much time do you spend on writing logic that
framework should take care of? Thursday, May 23, 13
How easy is it to debug your map/reduce aggragation? Thursday,
May 23, 13
Hadoop + Java composable, but too vebrose Pig, Hive too
concrete, lack of abstraction and composition Thursday, May 23, 13
Thursday, May 23, 13
• Clear, declarative syntax • Inner and outer joins •
Aggregators • Functions • Subqueries, composition • Sorting • Performant Thursday, May 23, 13
Casca-WHAT? • Built on top of Hadoop (MapReduce) • Cascading
(tuples, workflows, job execution) • Written in Clojure • Datalog (logic programming) Thursday, May 23, 13
Abstract evrthn! Thursday, May 23, 13
Source where data pours from Thursday, May 23, 13
Pipe that data flows through Thursday, May 23, 13
Filter that makes sure that only good stuff goes through
Thursday, May 23, 13
Tuple they actually flow Thursday, May 23, 13
Thursday, May 23, 13
Query anatomy Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Output Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
output vars Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Input input vars Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Logic/aggregations Thursday, May 23, 13
Sources and sinks • HDFS (go figure) • Cassandra •
MongoDB • SQL data sources • File system • Memory sources Thursday, May 23, 13
(?<- (stdout) [?person] (age ?person 25)) Exact match of second
element in a tuple Thursday, May 23, 13
(defn younger-than? [limit age] (< age limit)) (?<- (stdout) [?person
?age] (age ?person ?age) (younger-than? 32 ?age)) Predicate match, fn call Predicate Thursday, May 23, 13
(?<- (stdout) [?person ?count] (follows ?person _) (c/count ?count)) Aggregation
Thursday, May 23, 13
SHOWTIME! Thursday, May 23, 13
Benefits •Query language is same as application language •Subqueries, reusability
•Ad-hoc querying •Cascading underneath, so taps for all DBs work •Reuse application logic •Text editor integration Thursday, May 23, 13
@ifesdjeen (twitter/github) Thursday, May 23, 13