Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Szymon Sobczak - Hadoop + Storm
Search
Base Lab
May 07, 2015
Technology
0
100
Szymon Sobczak - Hadoop + Storm
Combo for realtime big data systems
Base Lab
May 07, 2015
Tweet
Share
More Decks by Base Lab
See All by Base Lab
Slawek Skowron - Monitoring @ Scale
baselab
0
120
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
baselab
0
110
Tomasz Nowak - Web Application Testing made easy
baselab
0
290
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
baselab
0
240
Mateusz Herych - LIKE '%smth%' is not the way
baselab
0
140
Jerzy Chałupski - Offline mode in Android apps
baselab
3
470
Jerzy Chałupski - Data model on Android
baselab
4
220
Other Decks in Technology
See All in Technology
KubeCon + CloudNativeCon Japan 2025 Recap
ren510dev
1
380
What’s new in Android development tools
yanzm
0
310
LLM時代の検索
shibuiwilliam
1
150
スタートアップに選択肢を 〜生成AIを活用したセカンダリー事業への挑戦〜
nstock
0
190
マーケットプレイス版Oracle WebCenter Content For OCI
oracle4engineer
PRO
3
960
20250705 Headlamp: 專注可擴展性的 Kubernetes 用戶界面
pichuang
0
270
MobileActOsaka_250704.pdf
akaitadaaki
0
120
AI時代の開発生産性を加速させるアーキテクチャ設計
plaidtech
PRO
3
160
fukabori.fm 出張版: 売上高617億円と高稼働率を陰で支えた社内ツール開発のあれこれ話 / 20250704 Yoshimasa Iwase & Tomoo Morikawa
shift_evolve
PRO
2
7.7k
生成AI開発案件におけるClineの業務活用事例とTips
shinya337
0
260
IPA&AWSダブル全冠が明かす、人生を変えた勉強法のすべて
iwamot
PRO
2
140
関数型プログラミングで 「脳がバグる」を乗り越える
manabeai
1
190
Featured
See All Featured
[RailsConf 2023] Rails as a piece of cake
palkan
55
5.7k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
20
1.3k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
35
2.4k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
A better future with KSS
kneath
238
17k
The Pragmatic Product Professional
lauravandoore
35
6.7k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
281
13k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Building Adaptive Systems
keathley
43
2.7k
Raft: Consensus for Rubyists
vanstee
140
7k
Transcript
Szymon Sobczak
Hadoop + Storm Combo for realtime big data systems
Plan • Hadoop & Storm • Our setup • What
projects are we running • Decisions we had to make
Hadoop
None
Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,
“ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”
Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"
“ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”
Common traits
None
Base infrastructure services
Understand how the entire Base system works services
Big Data S3 uploader
Four example projects • Debugging • Reporting • Email intelligence
• Forecasting
Debugging S3 uploader
Reporting
Email analysis S3 uploader
Forecasting S3 uploader
Decisions we made ☑ Collect *all* data ☑ Put them
in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm
Summary S3 uploader
Questions?
Thank you
[email protected]
bigdata.getbase.com