Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Base Lab
May 07, 2015
Technology
0
94

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
baselab
0
110
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
baselab
0
110
Tomasz Nowak - Web Application Testing made easy
baselab
0
290
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
baselab
0
240
Mateusz Herych - LIKE '%smth%' is not the way
baselab
0
140
Jerzy Chałupski - Offline mode in Android apps
baselab
3
470
Jerzy Chałupski - Data model on Android
baselab
4
220

Other Decks in Technology

See All in Technology
Developer Summit 2025 [14-D-1] Yuki Hattori
yuhattor
19
6.2k
現場で役立つAPIデザイン
nagix
33
12k
SA Night #2 FinatextのSA思想/SA Night #2 Finatext session
satoshiimai
1
140
クラウドサービス事業者におけるOSS
tagomoris
2
850
Developers Summit 2025 浅野卓也(13-B-7 LegalOn Technologies)
legalontechnologies
PRO
0
730
リーダブルテストコード 〜メンテナンスしやすい テストコードを作成する方法を考える〜 #DevSumi #DevSumiB / Readable test code
nihonbuson
11
7.3k
OpenID BizDay#17 KYC WG活動報告（法人） / 20250219-BizDay17-KYC-legalidentity
oidfj
0
250
デスクトップだけじゃないUbuntu
mtyshibata
0
120
インフラをつくるとはどういうことなのか、 あるいはPlatform Engineeringについて
nwiizo
5
2.6k
全文検索+セマンティックランカー+LLMの自然文検索サ−ビスで得られた知見
segavvy
2
110
室長と気ままに学ぶマイクロソフトのビジネスアプリケーションとビジネスプロセス
ryoheig0405
0
370
Nekko Cloud、 これまでとこれから ～学生サークルが作る、 小さなクラウド
logica0419
2
970

Featured

See All Featured
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
A Tale of Four Properties
chriscoyier
158
23k
BBQ
matthewcrist
87
9.5k
Become a Pro
speakerdeck
PRO
26
5.1k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
366
25k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
47
5.2k
Site-Speed That Sticks
csswizardry
4
380
How STYLIGHT went responsive
nonsquared
98
5.4k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
27
1.6k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
6
550
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
29
1k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com