Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Szymon Sobczak - Hadoop + Storm
Search
Base Lab
May 07, 2015
Technology
0
100
Szymon Sobczak - Hadoop + Storm
Combo for realtime big data systems
Base Lab
May 07, 2015
Tweet
Share
More Decks by Base Lab
See All by Base Lab
Slawek Skowron - Monitoring @ Scale
baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
baselab
0
110
Tomasz Nowak - Web Application Testing made easy
baselab
0
290
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
baselab
0
140
Jerzy Chałupski - Offline mode in Android apps
baselab
3
480
Jerzy Chałupski - Data model on Android
baselab
4
230
Other Decks in Technology
See All in Technology
Dylib Hijacking on macOS: Dead or Alive?
patrickwardle
0
440
Implementing and Evaluating a High-Level Language with WasmGC and the Wasm Component Model: Scala’s Case
tanishiking
0
170
AI AgentをLangflowでサクッと作って、1日働かせてみた!
yano13
1
120
Bill One 開発エンジニア 紹介資料
sansan33
PRO
4
14k
旅で応援する✈️ NEWTが目指すコミュニティ支援とあたらしい旅行 / New Travel: Supporting by NEWT on Your Journey
mii3king
0
140
Sansan Engineering Unit 紹介資料
sansan33
PRO
1
3k
React19.2のuseEffectEventを追う
maguroalternative
2
600
「タコピーの原罪」から学ぶ間違った”支援” / the bad support of Takopii
piyonakajima
0
120
初めてのDatabricks Apps開発
taka_aki
1
240
serverless team topology
_kensh
2
130
会社を支える Pythonという言語戦略 ~なぜPythonを主要言語にしているのか?~
curekoshimizu
2
520
NLPコロキウム20251022_超効率化への挑戦: LLM 1bit量子化のロードマップ
yumaichikawa
1
190
Featured
See All Featured
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
23
1.5k
Scaling GitHub
holman
463
140k
Context Engineering - Making Every Token Count
addyosmani
7
290
GitHub's CSS Performance
jonrohan
1032
470k
Git: the NoSQL Database
bkeepers
PRO
431
66k
Faster Mobile Websites
deanohume
310
31k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
Building a Scalable Design System with Sketch
lauravandoore
463
33k
Being A Developer After 40
akosma
91
590k
Fireside Chat
paigeccino
40
3.7k
Optimizing for Happiness
mojombo
379
70k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
130k
Transcript
Szymon Sobczak
Hadoop + Storm Combo for realtime big data systems
Plan • Hadoop & Storm • Our setup • What
projects are we running • Decisions we had to make
Hadoop
None
Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,
“ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”
Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"
“ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”
Common traits
None
Base infrastructure services
Understand how the entire Base system works services
Big Data S3 uploader
Four example projects • Debugging • Reporting • Email intelligence
• Forecasting
Debugging S3 uploader
Reporting
Email analysis S3 uploader
Forecasting S3 uploader
Decisions we made ☑ Collect *all* data ☑ Put them
in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm
Summary S3 uploader
Questions?
Thank you
[email protected]
bigdata.getbase.com