Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Cascalog
Search
αλεx π
May 22, 2013
Technology
4
130
Cascalog
Short demo talk on Cascalog on Hadoop UG in Munich
αλεx π
May 22, 2013
Tweet
Share
More Decks by αλεx π
See All by αλεx π
Scalable Time Series With Cassandra
ifesdjeen
1
350
Bayesian Inference is known to make machines biased
ifesdjeen
2
360
Cassandra for Data Analytics Backends
ifesdjeen
7
410
Stream Processing and Functional Programming
ifesdjeen
1
720
PolyConf 2015 - Rocking the Time Series boat with C, Haskell and ClojureScript
ifesdjeen
0
440
Clojure - A Sweetspot for Analytics
ifesdjeen
8
2k
Going Off Heap
ifesdjeen
3
1.9k
Always be learning
ifesdjeen
1
140
Learn Yourself Emacs For Great Good workshop slides
ifesdjeen
3
320
Other Decks in Technology
See All in Technology
生成AIが変えるデータ分析の全体像
ishikawa_satoru
0
150
The Rise of LLMOps
asei
7
1.6k
インフラとバックエンドとフロントエンドをくまなく調べて遅いアプリを早くした件
tubone24
1
430
CysharpのOSS群から見るModern C#の現在地
neuecc
2
3.4k
OS 標準のデザインシステムを超えて - より柔軟な Flutter テーマ管理 | FlutterKaigi 2024
ronnnnn
0
110
OTelCol_TailSampling_and_SpanMetrics
gumamon
1
170
B2B SaaSから見た最近のC#/.NETの進化
sansantech
PRO
0
840
データプロダクトの定義からはじめる、データコントラクト駆動なデータ基盤
chanyou0311
2
320
iOS/Androidで同じUI体験をネ イティブで作成する際に気をつ けたい落とし穴
fumiyasac0921
1
110
誰も全体を知らない ~ ロールの垣根を超えて引き上げる開発生産性 / Boosting Development Productivity Across Roles
kakehashi
1
230
開発生産性を上げながらビジネスも30倍成長させてきたチームの姿
kamina_zzz
2
1.7k
Incident Response Practices: Waroom's Features and Future Challenges
rrreeeyyy
0
160
Featured
See All Featured
Music & Morning Musume
bryan
46
6.2k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
28
8.2k
How GitHub (no longer) Works
holman
310
140k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9.1k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
0
96
Into the Great Unknown - MozCon
thekraken
32
1.5k
Git: the NoSQL Database
bkeepers
PRO
427
64k
How to train your dragon (web standard)
notwaldorf
88
5.7k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
159
15k
Fireside Chat
paigeccino
34
3k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.3k
Why Our Code Smells
bkeepers
PRO
334
57k
Transcript
Cascalog Hassle-free MapReduce that matches your scale Thursday, May 23,
13
Thursday, May 23, 13
Setting expecations •This is not a guide •And not a
tutorial •Doesn’t claim to be complete •Mostly to give you an idea •And encourage you to explore further Thursday, May 23, 13
How much time do you spend on writing logic that
framework should take care of? Thursday, May 23, 13
How easy is it to debug your map/reduce aggragation? Thursday,
May 23, 13
Hadoop + Java composable, but too vebrose Pig, Hive too
concrete, lack of abstraction and composition Thursday, May 23, 13
Thursday, May 23, 13
• Clear, declarative syntax • Inner and outer joins •
Aggregators • Functions • Subqueries, composition • Sorting • Performant Thursday, May 23, 13
Casca-WHAT? • Built on top of Hadoop (MapReduce) • Cascading
(tuples, workflows, job execution) • Written in Clojure • Datalog (logic programming) Thursday, May 23, 13
Abstract evrthn! Thursday, May 23, 13
Source where data pours from Thursday, May 23, 13
Pipe that data flows through Thursday, May 23, 13
Filter that makes sure that only good stuff goes through
Thursday, May 23, 13
Tuple they actually flow Thursday, May 23, 13
Thursday, May 23, 13
Query anatomy Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Output Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
output vars Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Input input vars Thursday, May 23, 13
(?<- (stdout) [?person ?person-age] (age ?person ?person-age) (< ?person-age 30))
Logic/aggregations Thursday, May 23, 13
Sources and sinks • HDFS (go figure) • Cassandra •
MongoDB • SQL data sources • File system • Memory sources Thursday, May 23, 13
(?<- (stdout) [?person] (age ?person 25)) Exact match of second
element in a tuple Thursday, May 23, 13
(defn younger-than? [limit age] (< age limit)) (?<- (stdout) [?person
?age] (age ?person ?age) (younger-than? 32 ?age)) Predicate match, fn call Predicate Thursday, May 23, 13
(?<- (stdout) [?person ?count] (follows ?person _) (c/count ?count)) Aggregation
Thursday, May 23, 13
SHOWTIME! Thursday, May 23, 13
Benefits •Query language is same as application language •Subqueries, reusability
•Ad-hoc querying •Cascading underneath, so taps for all DBs work •Reuse application logic •Text editor integration Thursday, May 23, 13
@ifesdjeen (twitter/github) Thursday, May 23, 13