Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Szymon Sobczak - Hadoop + Storm
Search
Base Lab
May 07, 2015
Technology
0
100
Szymon Sobczak - Hadoop + Storm
Combo for realtime big data systems
Base Lab
May 07, 2015
Tweet
Share
More Decks by Base Lab
See All by Base Lab
Slawek Skowron - Monitoring @ Scale
baselab
0
120
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
baselab
0
110
Tomasz Nowak - Web Application Testing made easy
baselab
0
290
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
baselab
0
240
Mateusz Herych - LIKE '%smth%' is not the way
baselab
0
140
Jerzy Chałupski - Offline mode in Android apps
baselab
3
470
Jerzy Chałupski - Data model on Android
baselab
4
220
Other Decks in Technology
See All in Technology
関数型プログラミングで 「脳がバグる」を乗り越える
manabeai
1
190
IPA&AWSダブル全冠が明かす、人生を変えた勉強法のすべて
iwamot
PRO
2
140
OPENLOGI Company Profile
hr01
0
67k
American airlines ®️ USA Contact Numbers: Complete 2025 Support Guide
airhelpsupport
0
380
CDKTFについてざっくり理解する!!~CloudFormationからCDKTFへ変換するツールも作ってみた~
masakiokuda
1
140
20250705 Headlamp: 專注可擴展性的 Kubernetes 用戶界面
pichuang
0
270
MUITにおける開発プロセスモダナイズの取り組みと開発生産性可視化の取り組みについて / Modernize the Development Process and Visualize Development Productivity at MUIT
muit
1
16k
Delta airlines®️ USA Contact Numbers: Complete 2025 Support Guide
airtravelguide
0
340
無意味な開発生産性の議論から抜け出すための予兆検知とお金とAI
i35_267
4
13k
改めてAWS WAFを振り返る~業務で使うためのポイント~
masakiokuda
2
250
fukabori.fm 出張版: 売上高617億円と高稼働率を陰で支えた社内ツール開発のあれこれ話 / 20250704 Yoshimasa Iwase & Tomoo Morikawa
shift_evolve
PRO
2
7.7k
OPENLOGI Company Profile for engineer
hr01
1
34k
Featured
See All Featured
Making the Leap to Tech Lead
cromwellryan
134
9.4k
Making Projects Easy
brettharned
116
6.3k
A Modern Web Designer's Workflow
chriscoyier
695
190k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.9k
The Power of CSS Pseudo Elements
geoffreycrofte
77
5.9k
YesSQL, Process and Tooling at Scale
rocio
173
14k
Writing Fast Ruby
sferik
628
62k
We Have a Design System, Now What?
morganepeng
53
7.7k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
Scaling GitHub
holman
460
140k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
Bash Introduction
62gerente
613
210k
Transcript
Szymon Sobczak
Hadoop + Storm Combo for realtime big data systems
Plan • Hadoop & Storm • Our setup • What
projects are we running • Decisions we had to make
Hadoop
None
Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,
“ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”
Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"
“ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”
Common traits
None
Base infrastructure services
Understand how the entire Base system works services
Big Data S3 uploader
Four example projects • Debugging • Reporting • Email intelligence
• Forecasting
Debugging S3 uploader
Reporting
Email analysis S3 uploader
Forecasting S3 uploader
Decisions we made ☑ Collect *all* data ☑ Put them
in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm
Summary S3 uploader
Questions?
Thank you
[email protected]
bigdata.getbase.com