Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
110
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
480
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
230

Other Decks in Technology

See All in Technology
"おまじない"はもう卒業！ デバッガで探るSpring Bootの裏側と「学び方」の学び方
Avatar for 竹内 雅和 takeuchi_132917
0
180
ABEMAのCM配信を支えるスケーラブルな分散カウンタの実装
Avatar for Honoka Toda hono0130
3
820
【M3】攻めのセキュリティの実践！プロアクティブなセキュリティ対策の実践事例
Avatar for mizu axelmizu
0
170
2ヶ月で新規事業のシステムを0から立ち上げるスタートアップの舞台裏
Avatar for Shoma Okamoto shmokmt
0
190
JavaScript パーサーに using 対応をする過程で与えたエコシステムへの影響
Avatar for Yuichiro Yamashita baseballyama
1
100
なぜインフラコードのモジュール化は難しいのか - アプリケーションコードとの本質的な違いから考える
Avatar for Gosuke Miyashita mizzy
55
18k
[mercari GEARS 2025] Keynote
Avatar for mercari mercari
PRO
1
310
Flutterにしてよかった？出前館アプリを2年運用して気づいたことを全部話します
Avatar for 株式会社出前館 demaecan
0
220
Devoxx Morocco 2025 - Like Spring but faster: The new Java Jedi
Avatar for Eric Deandrea edeandrea
PRO
0
100
生成AIではじめるテスト駆動開発
Avatar for puku0x puku0x
0
120
仕様は“書く”より“語る” - 分断を超えたチーム開発の実践 / 20251115 Naoki Takahashi
Avatar for SHIFT EVOLVE shift_evolve
PRO
1
990
「もっと正確に、もっと効率的に」ANDPADの写真書き込み機能における、 現場の声を形にしたエンハンス
Avatar for ANDPAD inc andpad
0
110

Featured

See All Featured
10 Git Anti Patterns You Should be Aware of
Avatar for Lemi Orhan Ergin lemiorhan
PRO
658
61k
The Success of Rails: Ensuring Growth for the Next 100 Years
Avatar for Eileen M. Uchitelle eileencodes
46
7.8k
Fashionably flexible responsive web design (full day workshop)
Avatar for Andy Clarke malarkey
407
66k
StorybookのUI Testing Handbookを読んだ
Avatar for Shingo Yamazaki zakiyama
31
6.3k
Product Roadmaps are Hard
Avatar for C. Todd Lombardo iamctodd
PRO
55
12k
Six Lessons from altMBA
Avatar for Skipper Chong Warson skipperchong
29
4.1k
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
65
8k
Keith and Marios Guide to Fast Websites
Avatar for Keith Pitt keithpitt
413
23k
Raft: Consensus for Rubyists
Avatar for Patrick Van Stee vanstee
140
7.2k
Understanding Cognitive Biases in Performance Measurement
Avatar for Philip Tellis bluesmoon
31
2.7k
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
56
14k
How to Think Like a Performance Engineer
Avatar for Harry Roberts csswizardry
28
2.3k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com