Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
120
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
240

Other Decks in Technology

See All in Technology
CDK対応したAWS DevOps Agentを試そう_20260201
Avatar for モブエンジニア（Masaki Okuda） masakiokuda
1
370
フルカイテン株式会社 エンジニア向け採用資料
Avatar for FULL KAITEN fullkaiten
0
10k
OWASP Top 10:2025 リリースと 少しの日本語化にまつわる裏話
Avatar for Riotaro OKADA okdt
PRO
3
830
コンテナセキュリティの最新事情 ~ 2026年版 ~
Avatar for Kyohei Mizumoto kyohmizu
2
370
Bill One 開発エンジニア 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
5
17k
Embedded SREの終わりを設計する　「なんとなく」から計画的な自立支援へ
Avatar for SansanTech sansantech
PRO
3
2.6k
SchooでVue.js/Nuxtを技術選定している理由
Avatar for Okuto Oyama yamanoku
3
130
こんなところでも（地味に）活躍するImage Modeさんを知ってるかい？- Image Mode for OpenShift -
Avatar for Masataka Tsukamoto tsukaman
1
160
StrandsとNeptuneを使ってナレッジグラフを構築する
Avatar for やくも yakumo
1
120
超初心者からでも大丈夫！オープンソース半導体の楽しみ方〜今こそ！オレオレチップをつくろう〜
Avatar for Yamada3 keropiyo
0
120
プロダクト成長を支える開発基盤とスケールに伴う課題
Avatar for yuu26 yuu26
4
1.4k
AI駆動開発を事業のコアに置く
Avatar for 鬼澤輔 tasukuonizawa
1
300

Featured

See All Featured
The Spectacular Lies of Maps
Avatar for Per Axbom axbom
PRO
1
520
Getting science done with accelerated Python computing platforms
Avatar for Jacob Tomlinson jacobtomlinson
2
120
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
38
2.7k
Site-Speed That Sticks
Avatar for Harry Roberts csswizardry
13
1.1k
Six Lessons from altMBA
Avatar for Skipper Chong Warson skipperchong
29
4.2k
Odyssey Design
Avatar for Ryan Kendrick rkendrick25
PRO
1
500
The AI Search Optimization Roadmap by Aleyda Solis
Avatar for Aleyda Solis aleyda
1
5.2k
Measuring & Analyzing Core Web Vitals
Avatar for Philip Tellis bluesmoon
9
760
Easily Structure & Communicate Ideas using Wireframe
Avatar for Afnizar Nur Ghifari afnizarnur
194
17k
Claude Code どこまでも/ Claude Code Everywhere
Avatar for nwiizo nwiizo
61
52k
How to audit for AI Accessibility on your Front & Back End
Avatar for davetheseo davetheseo
0
180
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
Avatar for Tammy Everts tammyeverts
11
830

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com