Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Cascalog
Search
αλεx π
May 22, 2013
Technology
4
130
Cascalog
Short demo talk on Cascalog on Hadoop UG in Munich
αλεx π
May 22, 2013
Tweet
Share
More Decks by αλεx π
See All by αλεx π
Scalable Time Series With Cassandra
ifesdjeen
1
340
Bayesian Inference is known to make machines biased
ifesdjeen
2
340
Cassandra for Data Analytics Backends
ifesdjeen
7
400
Stream Processing and Functional Programming
ifesdjeen
1
720
PolyConf 2015 - Rocking the Time Series boat with C, Haskell and ClojureScript
ifesdjeen
0
430
Clojure - A Sweetspot for Analytics
ifesdjeen
8
2k
Going Off Heap
ifesdjeen
3
1.9k
Always be learning
ifesdjeen
1
130
Learn Yourself Emacs For Great Good workshop slides
ifesdjeen
3
310
Other Decks in Technology
See All in Technology
.NET Profiler in 2024.
kkamegawa
2
2.5k
さらばあのボタンとは言わせない SORACOM LTE-M Button powerd by AWSをまだ使えるようにした(前編?)
miura55
0
110
kcp: Kubernetes APIs Are All You Need #techfeed_live / TechFeed Experts Night 28th
ytaka23
1
160
プロンプトエンジニアリングでがんばらない-Agentic Workflow へ-近藤憲児
kenjikondobai
6
1.3k
【SORACOM UG 東海】あらゆるモノがつながる社会へ、IoT と SORACOM
soracom
PRO
1
360
AWSに詳しくない人でも始められるコスト最適化ガイド
yuhta28
2
420
DX企業CTOとして考える技術への向き合い方
shoheitai
0
120
Cypress or Playwright?
rainerhahnekamp
0
180
同じ様なUIをiOS/Android間で合わせるヒントNo.2
fumiyasac0921
1
110
今年のRubyKaigiはProfiler Year🤘
osyoyu
0
520
成長をサポートするピープルマネジメントのやり方
sioncojp
9
1.4k
IaCからAWSに入門した初心者が CloudFormationを通して考えた「AWS操作」の使い分け
maimyyym
3
620
Featured
See All Featured
Designing Dashboards & Data Visualisations in Web Apps
destraynor
226
51k
KATA
mclloyd
16
12k
Build The Right Thing And Hit Your Dates
maggiecrowley
25
2k
Typedesign – Prime Four
hannesfritz
36
2.1k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
123
39k
A Philosophy of Restraint
colly
197
16k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
15
1.6k
For a Future-Friendly Web
brad_frost
172
9k
Designing for humans not robots
tammielis
247
25k
Side Projects
sachag
451
41k
Fireside Chat
paigeccino
22
2.6k
The Mythical Team-Month
searls
217
42k
Transcript
Cascalog Hassle-free MapReduce that matches your scale Thursday, May 23,
13
Thursday, May 23, 13
Setting expecations •This is not a guide •And not a
tutorial •Doesn’t claim to be complete •Mostly to give you an idea •And encourage you to explore further Thursday, May 23, 13
How much time do you spend on writing logic that
framework should take care of? Thursday, May 23, 13
How easy is it to debug your map/reduce aggragation? Thursday,
May 23, 13
Hadoop + Java composable, but too vebrose Pig, Hive too
concrete, lack of abstraction and composition Thursday, May 23, 13
Thursday, May 23, 13
• Clear, declarative syntax • Inner and outer joins •
Aggregators • Functions • Subqueries, composition • Sorting • Performant Thursday, May 23, 13
Casca-WHAT? • Built on top of Hadoop (MapReduce) • Cascading
(tuples, workflows, job execution) • Written in Clojure • Datalog (logic programming) Thursday, May 23, 13
Abstract evrthn! Thursday, May 23, 13
Source where data pours from Thursday, May 23, 13
Pipe that data flows through Thursday, May 23, 13
Filter that makes sure that only good stuff goes through
Thursday, May 23, 13
Tuple they actually flow Thursday, May 23, 13
Thursday, May 23, 13
Query anatomy Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Output Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
output vars Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Input input vars Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Logic/aggregations Thursday, May 23, 13
Sources and sinks • HDFS (go figure) • Cassandra •
MongoDB • SQL data sources • File system • Memory sources Thursday, May 23, 13
(?<- (stdout) [?person] (age ?person 25)) Exact match of second
element in a tuple Thursday, May 23, 13
(defn younger-than? [limit age] (< age limit)) (?<- (stdout) [?person
?age] (age ?person ?age) (younger-than? 32 ?age)) Predicate match, fn call Predicate Thursday, May 23, 13
(?<- (stdout) [?person ?count] (follows ?person _) (c/count ?count)) Aggregation
Thursday, May 23, 13
SHOWTIME! Thursday, May 23, 13
Benefits •Query language is same as application language •Subqueries, reusability
•Ad-hoc querying •Cascading underneath, so taps for all DBs work •Reuse application logic •Text editor integration Thursday, May 23, 13
@ifesdjeen (twitter/github) Thursday, May 23, 13