Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
130
0

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
160
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
140
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
320
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
260
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
160
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
500
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
260

Other Decks in Technology

See All in Technology
ラジオの科学
Avatar for Frieve-A frievea
0
280
Sansan Engineering Unit 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
1
4.9k
[しろおび夏祭り2026] チャットするAIから、作業するAIへ - 使われ方の変化と、その裏側で起きていること
Avatar for kk0n kk0n
0
1.7k
Goでデータパイプラインを作ろう
Avatar for SansanTech sansantech
PRO
1
360
FORENSIA: ローカルLLMフォレンジックハーネス
Avatar for S.Nakano sumeshi
2
350
【Google Cloud Next Tokyo'26】Gemini Enterprise と Oracle AI Database で実現する、 業務データ活用を実現する AI エージェント実装
Avatar for KoheiOgawa shisyu_gaku
0
230
社内の7割が使うデータ基盤を、 データチーム2人で回すためにやったこと
Avatar for KOH koh_yoshi
4
1.2k
20260608_Codexの可能性_ノンプログラマー向け_大城追記
Avatar for NobuakiOshiro doradora09
PRO
0
780
【５分でわかる】セーフィー エンジニア向け会社紹介
Avatar for Safie safie_recruit
0
53k
MIRU 2026 チュートリアル
Avatar for Keisuke Fujii keisuke198619
0
870
え？フロントエンドエンジニアの ワイがインフラも！？
Avatar for puku0x puku0x
1
440
toio・myCobotでフィジカルAIっぽいことを行うための検討（とりあえず調査） / フィジカルAI LT（IoTLTによる開催）
Avatar for you（@youtoy） you
PRO
0
300

Featured

See All Featured
Deep Space Network (abreviated)
Avatar for Tony Rice tonyrice
0
250
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
38
2.9k
Easily Structure & Communicate Ideas using Wireframe
Avatar for Afnizar Nur Ghifari afnizarnur
194
17k
Fashionably flexible responsive web design (full day workshop)
Avatar for Andy Clarke malarkey
408
67k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
230
23k
SERP Conf. Vienna - Web Accessibility: Optimizing for Inclusivity and SEO
Avatar for Sara Fernández Carmona sarafernandez
2
1.5k
The Power of CSS Pseudo Elements
Avatar for Geoffrey Crofte geoffreycrofte
82
6.5k
B2B Lead Gen: Tactics, Traps & Triumph
Avatar for Sophie Logan marketingsoph
0
200
Into the Great Unknown - MozCon
Avatar for Noah Learner thekraken
41
2.7k
Agile that works and the tools we love
Avatar for Rasmus Luckow-Nielsen rasmusluckow
331
22k
Digital Projects Gone Horribly Wrong (And the UX Pros Who Still Save the Day) - Dean Schuster
Avatar for UX Y'all uxyall
1
2.3k
Applied NLP in the Age of Generative AI
Avatar for Ines Montani inesmontani
PRO
4
2.4k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com