Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
110

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
140
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
120
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
310
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
240

Other Decks in Technology

See All in Technology
Oracle Cloud Infrastructure：2026年3月度サービス・アップデート
Avatar for oracle4engineer oracle4engineer
PRO
0
200
会社紹介資料 / Sansan Company Profile
Avatar for Sansan, Inc. sansan33
PRO
16
410k
FastMCP OAuth Proxy with Cognito
Avatar for iganin hironobuiga
3
220
タスク管理も1on1も、もう「管理」じゃない - KiroとBedrock AgentCoreで変わった“判断の仕事”
Avatar for Yusuke Shimizu yusukeshimizu
0
140
LLMに何を任せ、何を任せないか
Avatar for an cap120
10
6.2k
AgentCoreとLINEを使った飲食店おすすめアプリを作ってみた
Avatar for やくも yakumo
2
260
GitHub Copilot CLI で Azure Portal to Bicep
Avatar for Yuta Matsumura tsubakimoto_s
0
290
15年メンテしてきたdotfilesから開発トレンドを振り返る 2011 - 2026
Avatar for giginet giginet
PRO
1
210
「AIエージェントで変わる開発プロセス―レビューボトルネックからの脱却」
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
0
190
FlutterでPiP再生を実装した話
Avatar for 白井翔大 s9a17
0
230
ハーネスエンジニアリング×AI適応開発
Avatar for Ryohei Kamiya aictokamiya
1
780
AIエージェント時代に必要な オペレーションマネージャーのロールとは
Avatar for KentaroFujii kentarofujii
0
220

Featured

See All Featured
Rails Girls Zürich Keynote
Avatar for Gregor Martynus gr2m
96
14k
A better future with KSS
Avatar for Kyle Neath kneath
240
18k
Imperfection Machines: The Place of Print at Facebook
Avatar for Scott Boms scottboms
269
14k
How People are Using Generative and Agentic AI to Supercharge Their Products, Projects, Services and Value Streams Today
Avatar for Helen Beal helenjbeal
1
140
Mozcon NYC 2025: Stop Losing SEO Traffic
Avatar for Sam Torres samtorres
0
190
The Anti-SEO Checklist Checklist. Pubcon Cyber Week
Avatar for Ryan Jones ryanjones
0
110
The Psychology of Web Performance [Beyond Tellerrand 2023]
Avatar for Tammy Everts tammyeverts
49
3.3k
Building an army of robots
Avatar for Kyle Neath kneath
306
46k
Odyssey Design
Avatar for Ryan Kendrick rkendrick25
PRO
2
560
Darren the Foodie - Storyboard
Avatar for Kevinho.art khoart
PRO
3
3.1k
Exploring the relationship between traditional SERPs and Gen AI search
Avatar for Ray Grieselhuber raygrieselhuber
PRO
2
3.8k
Designing Experiences People Love
Avatar for Jonathan Moore moore
143
24k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com