Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
140
0

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
170
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
150
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
330
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
270
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
170
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
500
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
270

Other Decks in Technology

See All in Technology
生き様としてのSRE
Avatar for Yutaro Sugai hokkai7go
1
150
Flutter Widget Previewerで始めるUIプレビューとVRTの自動化
Avatar for Aokiti sakusaku3939
0
190
渡米前の準備を制す！re:Invent 2026徹底準備術 / The Ultimate Preparation Guide for re:Invent 2026
Avatar for Yuji Oshima yuj1osm
1
500
SLOを阻む不安
Avatar for sraku sraku2159
0
210
[re:Invent2026事前勉強会(有志開催)]続きは、ラスベガスの廊下で / continue-in-the-vegas-hallway
Avatar for sh_fk2 sh_fk2
0
490
アクセラレーターで実現するServiceNow活用の高速化 / ServiceNow World Forum 2026
Avatar for Visional Engineering ＆ Design visional_engineering_and_design
0
120
Technical Project Manager の役割とやりがい、少しだけ AI
Avatar for Ryo Yoshii yoshiiryo1
0
560
攻撃 3 時間 vs 復旧 10 時間。アクセスキー流出の非対称性を体感するために CTF を作った話
Avatar for kazzpapa3 kazzpapa3
1
130
Copilot Coworkで「みのるん式」やってみた
Avatar for みのるん minorun365
PRO
1
1k
AIがGISを代替する日：〜すべてをAIが創る時代に、私たちがFOSS4Gに関わる意味〜
Avatar for Yuji Kobayashi lehupa
0
190
Content Mapper APIの応用例～CSS Modules、言語ファイル～
Avatar for uhyo uhyo
2
510
バイブコーディングの特徴と「コケて直す」ループ
Avatar for コロッケそば happysamurai294
0
140

Featured

See All Featured
Reality Check: Gamification 10 Years Later
Avatar for Sebastian Deterding codingconduct
0
2.4k
技術選定の審美眼（2025年版） / Understanding the Spiral of Technologies 2025 edition
Avatar for Takuto WADA twada
PRO
120
120k
From Legacy to Launchpad: Building Startup-Ready Communities
Avatar for Dug Song dugsong
0
370
SEOcharity - Dark patterns in SEO and UX: How to avoid them and build a more ethical web
Avatar for Sara Fernández Carmona sarafernandez
0
320
Sam Torres - BigQuery for SEOs
Avatar for Tech SEO Connect techseoconnect
PRO
0
600
[SF Ruby Conf 2025] Rails X
Avatar for Vladimir Dementyev palkan
3
1.5k
B2B Lead Gen: Tactics, Traps & Triumph
Avatar for Sophie Logan marketingsoph
1
300
AI: The stuff that nobody shows you
Avatar for John Nunemaker jnunemaker
PRO
10
1.2k
AI Search: Implications for SEO and How to Move Forward - #ShenzhenSEOConference
Avatar for Aleyda Solis aleyda
1
1.4k
How Software Deployment tools have changed in the past 20 years
Avatar for Geshan Manandhar geshan
2
35k
Evolving SEO for Evolving Search Engines
Avatar for Ryan Jones ryanjones
0
350
Music & Morning Musume
Avatar for Bryan Veloso bryan
48
7.5k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com