Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →
Avatar for Base Lab Base Lab
May 07, 2015
Technology
120
0

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
150
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
130
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
310
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
260
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
160
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
250

Other Decks in Technology

See All in Technology
Oracle AI Database@Google Cloud：サービス概要のご紹介
Avatar for oracle4engineer oracle4engineer
PRO
6
1.5k
権限管理設計を完全に理解した
Avatar for r-sugi rsugi
2
230
イベントストーミングとKiroの仕様駆動開発で実現する要件の認識合わせプロセス
Avatar for syobochim syobochim
7
940
Cloud Run のアップデート 触ってみる＆紹介
Avatar for Gre212 gre212
0
230
JEP 522 Deep Dive - G1 GC同期コスト削減によるスループット向上を徹底検証&解説
Avatar for Daishi Tabata tabatad
1
330
脅威をエンジニアリングの糧にして：恐怖を乗り越えた先にあったもの / Turn threats into fuel for engineering: what lay beyond overcoming fear
Avatar for nrs nrslib
1
340
コードレビューを制するチームがソフトウェアデリバリーのフローを制す / Beyond Code Review: Distributing Its Responsibilities Across the SDLC
Avatar for mtx2s mtx2s
1
360
Agentic Design Patterns
Avatar for Guillaume Laforge glaforge
0
280
AI Adaptable なテストを整える工夫 / Ways to Make Your Tests AI-Adaptable
Avatar for 株式会社ビットキー / Bitkey Inc. bitkey
PRO
2
160
自称宇宙最速で不合格となったAIP-C01にリベンジを果たすべくAIで問題集アプリを作ってみた。
Avatar for Yuuki Yamashita yama3133
0
240
AI-DLCを活用した高品質・安全なAI駆動開発実践 / AI Driven Development
Avatar for 吉田真吾 yoshidashingo
1
230
Claude Codeですべての日常業務を爆速化しよう！
Avatar for みのるん minorun365
PRO
16
15k

Featured

See All Featured
Leo the Paperboy
Avatar for Maya Tellez mayatellez
7
1.8k
Impact Scores and Hybrid Strategies: The future of link building
Avatar for Tamara Novitovic tamaranovitovic
0
290
AI: The stuff that nobody shows you
Avatar for John Nunemaker jnunemaker
PRO
7
660
CSS Pre-Processors: Stylus, Less & Sass
Avatar for Bermon Painter bermonpainter
360
30k
職位にかかわらず全員がリーダーシップを発揮するチーム作り / Building a team where everyone can demonstrate leadership regardless of position
Avatar for MADOX madoxten
62
54k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
Avatar for Michelle Schulp Hunt marktimemedia
31
2.8k
Amusing Abliteration
Avatar for ianozsvald ianozsvald
1
190
A designer walks into a library…
Avatar for Paul Jervis Heath pauljervisheath
211
24k
Fantastic passwords and where to find them - at NoRuKo
Avatar for Phil Nash philnash
52
3.7k
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
66
8.5k
Organizational Design Perspectives: An Ontology of Organizational Design Elements
Avatar for Dr. Kim W Petersen kimpetersen
PRO
1
710
Leadership Guide Workshop - DevTernity 2021
Avatar for David Neal reverentgeek
1
290

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com