Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Berlin Buzzwords 2012: Introduction to Cascalog

Stefan Hübner
June 18, 2012
Technology
1
570

Berlin Buzzwords 2012: Introduction to Cascalog

These are the slides for my talk "Introduction to Cascalog: Functional Data Processing for Hadoop". http://berlinbuzzwords.de/sessions/introducing-cascalog-functional-data-processing-hadoop

The talk features a live demonstration of Cascalog. Check out the video on Vimeo: https://vimeo.com/album/1978224/video/43804713

Stefan Hübner

June 18, 2012
Tweet

More Decks by Stefan Hübner

See All by Stefan Hübner
EuroClojure 2012: Introduction to Cascalog
sthuebner
2
640
Introduction to Cascalog
sthuebner
1
170

Other Decks in Technology

See All in Technology
プラットフォームってつくることより計測することが重要なんじゃないかという話 / Platform Engineering Meetup #8
taishin
1
340
KubeConにproposalを送りたい人へのアドバイス
sat
PRO
3
230
VS CodeでAWSを操作しよう
smt7174
7
1.6k
日本におけるデータエンジニアリングのこれまでとこれから
foursue
16
4.2k
AOAI をきっかけに​ 社内の Azure 管理を見直した話​
recruitengineers
PRO
1
250
Postman v10リリース後を振り返る
nagix
0
170
よく聞くけど使ったことないソフトウェアNo.1 KafkaとSnowflake
foursue
4
340
On Your Data を超えていく！
hirotomotaguchi
2
650
Terraformあれやこれ/terraform-this-and-that
emiki
8
1.3k
EMとして2023年度に頑張ったこと / What we did well in FY2023 as a EM
pauli
1
160
推しは推せるときに推せ！ プロダクトにフィードバックしていこう
nakasho
0
280
MLOpsの「壁」を乗り越える、LINEヤフーの Data Quality as Code
lycorptech_jp
PRO
5
390

Featured

See All Featured
How GitHub Uses GitHub to Build GitHub
holman
468
290k
Designing the Hi-DPI Web
ddemaree
276
33k
The Invisible Customer
myddelton
114
12k
Bash Introduction
62gerente
604
210k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
25
2.3k
No one is an island. Learnings from fostering a developers community.
thoeni
16
2.1k
Imperfection Machines: The Place of Print at Facebook
scottboms
260
12k
Typedesign – Prime Four
hannesfritz
36
2.1k
Six Lessons from altMBA
skipperchong
21
3k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
221
21k
jQuery: Nuts, Bolts and Bling
dougneiner
59
7.1k
What’s in a name? Adding method to the madness
productmarketing
PRO
16
2.6k

Transcript

  1. Introduction to Cascalog Stefan Hübner, Nokia Berlin Berlin Buzzwords 2012

  2. Nokia Maps • Address & POI Search • Category Search

    • Nearby Recommendations

  3. Hadoop • Batch Processing • (Very) Large Scale • Distributed

    Filesystem • Parallel Computation • Fault-Tolerant

  4. Hadoop • Batch Processing • (Very) Large Scale • Distributed

    Filesystem • Parallel Computation • Fault-Tolerant

  5. Hadoop MapReduce API • Tedious and verbose • Hard to

    test • Hard to refactor

  6. • Tedious and verbose • Hard to test • Hard

    to refactor Hadoop MapReduce API

  7. Pig and Hive • Define their own query language •

    Custom operations in Java, Python, ... • Non-intuitive integration

  8. (())

  9. Cascalog Cascalog Cascading Hadoop Abstraction Variables and logic Tuples, data

    workflows Key/value pairs, simple aggregation slide (c) Nathan Marz, reproduced with permission

  10. Thank you! Stefan Hübner, @sthuebner http://knowyourmeme.com/memes/cereal-guy