Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Cascalog
Search
αλεx π
May 22, 2013
Technology
4
140
Cascalog
Short demo talk on Cascalog on Hadoop UG in Munich
αλεx π
May 22, 2013
Tweet
Share
More Decks by αλεx π
See All by αλεx π
Scalable Time Series With Cassandra
ifesdjeen
1
360
Bayesian Inference is known to make machines biased
ifesdjeen
2
370
Cassandra for Data Analytics Backends
ifesdjeen
7
430
Stream Processing and Functional Programming
ifesdjeen
1
740
PolyConf 2015 - Rocking the Time Series boat with C, Haskell and ClojureScript
ifesdjeen
0
460
Clojure - A Sweetspot for Analytics
ifesdjeen
8
2.1k
Going Off Heap
ifesdjeen
3
1.9k
Always be learning
ifesdjeen
1
140
Learn Yourself Emacs For Great Good workshop slides
ifesdjeen
3
330
Other Decks in Technology
See All in Technology
Wasmで社内ツールを作って配布しよう
askua
0
130
大規模組織にAIエージェントを迅速に導入するためのセキュリティの勘所 / AI agents for large-scale organizations
i35_267
6
240
Railsの限界を超えろ!「家族アルバム みてね」の画像・動画の大規模アップロードを支えるアーキテクチャの変遷
ojima_h
3
450
会社もクラウドも違うけど 通じたコスト削減テクニック/Cost optimization strategies effective regardless of company or cloud provider
aeonpeople
2
170
「現場で活躍するAIエージェント」を実現するチームと開発プロセス
tkikuchi1002
6
1k
そもそも AWS FIS について。なぜ今 FIS のハンズオンなのか?などなど
kazzpapa3
2
120
20250719_JAWS_kobe
takuyay0ne
1
160
少人数でも回る! DevinとPlaybookで支える運用改善
ishikawa_pro
3
670
RapidPen: AIエージェントによる高度なペネトレーションテスト自動化の研究開発
laysakura
1
390
メモ整理が苦手な者による頑張らないObsidian活用術
optim
0
130
ゼロから始めるSREの事業貢献 - 生成AI時代のSRE成長戦略と実践 / Starting SRE from Day One
shinyorke
PRO
0
240
Power Automate のパフォーマンス改善レシピ / Power Automate Performance Improvement Recipes
karamem0
0
200
Featured
See All Featured
Music & Morning Musume
bryan
46
6.7k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Fireside Chat
paigeccino
37
3.5k
A better future with KSS
kneath
238
17k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
29
9.6k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
181
54k
Site-Speed That Sticks
csswizardry
10
720
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Java REST API Framework Comparison - PWX 2021
mraible
31
8.7k
Transcript
Cascalog Hassle-free MapReduce that matches your scale Thursday, May 23,
13
Thursday, May 23, 13
Setting expecations •This is not a guide •And not a
tutorial •Doesn’t claim to be complete •Mostly to give you an idea •And encourage you to explore further Thursday, May 23, 13
How much time do you spend on writing logic that
framework should take care of? Thursday, May 23, 13
How easy is it to debug your map/reduce aggragation? Thursday,
May 23, 13
Hadoop + Java composable, but too vebrose Pig, Hive too
concrete, lack of abstraction and composition Thursday, May 23, 13
Thursday, May 23, 13
• Clear, declarative syntax • Inner and outer joins •
Aggregators • Functions • Subqueries, composition • Sorting • Performant Thursday, May 23, 13
Casca-WHAT? • Built on top of Hadoop (MapReduce) • Cascading
(tuples, workflows, job execution) • Written in Clojure • Datalog (logic programming) Thursday, May 23, 13
Abstract evrthn! Thursday, May 23, 13
Source where data pours from Thursday, May 23, 13
Pipe that data flows through Thursday, May 23, 13
Filter that makes sure that only good stuff goes through
Thursday, May 23, 13
Tuple they actually flow Thursday, May 23, 13
Thursday, May 23, 13
Query anatomy Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Output Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
output vars Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Input input vars Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Logic/aggregations Thursday, May 23, 13
Sources and sinks • HDFS (go figure) • Cassandra •
MongoDB • SQL data sources • File system • Memory sources Thursday, May 23, 13
(?<- (stdout) [?person] (age ?person 25)) Exact match of second
element in a tuple Thursday, May 23, 13
(defn younger-than? [limit age] (< age limit)) (?<- (stdout) [?person
?age] (age ?person ?age) (younger-than? 32 ?age)) Predicate match, fn call Predicate Thursday, May 23, 13
(?<- (stdout) [?person ?count] (follows ?person _) (c/count ?count)) Aggregation
Thursday, May 23, 13
SHOWTIME! Thursday, May 23, 13
Benefits •Query language is same as application language •Subqueries, reusability
•Ad-hoc querying •Cascading underneath, so taps for all DBs work •Reuse application logic •Text editor integration Thursday, May 23, 13
@ifesdjeen (twitter/github) Thursday, May 23, 13