Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Base Lab Base Lab
May 07, 2015
Technology
120
0

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
160
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
140
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
310
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
260
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
160
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
500
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
260

Other Decks in Technology

See All in Technology
2026 TECHFRESH 畢業分享會 - 開發日常大解密！從領域驅動到企業級上線
Avatar for LINE Developers Taiwan line_developers_tw
PRO
0
1.1k
人材育成分科会.pdf
Avatar for _awache _awache
4
270
小さく始める AI 活用推進 ― 日経電子版 Web チームの事例/nikkei-tech-talk47
Avatar for 日本経済新聞社 エンジニア採用事務局 nikkei_engineer_recruiting
0
270
MUSUBI 田中裕一『AIと共に行う「しごとのリデザイン」- スモールバックオフィス編』AI Ops Lab #4
Avatar for MUSUBI musubi
0
200
作って終わりにしない タイミーのセマンティックレイヤー育成の現在地
Avatar for chanyou0311 chanyou0311
4
2.4k
SONiC Scale-Up Working Group から探る Scale-UpやUltraEthernet機能の実装方法
Avatar for ebiken ebiken
PRO
2
350
2026TECHFRESH畢業分享會 - Lightning Talk - 資料也要 CI/CD? 用 Airbyte 自動化資料同步
Avatar for LINE Developers Taiwan line_developers_tw
PRO
0
1.1k
中期計画、2回作ってみた　～業務委託と正社員、両方の視点から～
Avatar for 株式会社出前館 demaecan
1
910
Kiroで書いた 設計書 が AI レビューの 採点基準 になる
Avatar for shinji ezaki ezaki
0
120
Oracle AI Database@Google Cloud：サービス概要のご紹介
Avatar for oracle4engineer oracle4engineer
PRO
6
1.5k
日本 Fintech 未来予測レポート 2027〜2028年（オリジナル版）
Avatar for 8maki 8maki
0
2.2k
Claude Code の Sandbox 機能を Anthropic Sandbox Runtime(srt) で試そう！/lets-play-anthropic-sandbox-runtime
Avatar for tomoki10 tomoki10
1
620

Featured

See All Featured
RailsConf 2023
Avatar for Aaron Patterson tenderlove
30
1.5k
A Guide to Academic Writing Using Generative AI - A Workshop
Avatar for Kenji Saito ks91
PRO
1
330
The Mindset for Success: Future Career Progression
Avatar for Greg Gifford greggifford
PRO
0
360
Hiding What from Whom? A Critical Review of the History of Programming languages for Music
Avatar for Tomoya Matsuura tomoyanonymous
2
860
GraphQLとの向き合い方2022年版
Avatar for Yosuke Kurami quramy
50
15k
HU Berlin: Industrial-Strength Natural Language Processing with spaCy and Prodigy
Avatar for Ines Montani inesmontani
PRO
0
410
How to Ace a Technical Interview
Avatar for Jacob Kaplan-Moss jacobian
281
24k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
Avatar for Tammy Everts tammyeverts
11
940
Build your cross-platform service in a week with App Engine
Avatar for Jose L jlugia
234
18k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
Avatar for Irina Nazarova irinanazarova
9
1.4k
Dominate Local Search Results - an insider guide to GBP, reviews, and Local SEO
Avatar for Greg Gifford greggifford
PRO
0
190
Pawsitive SEO: Lessons from My Dog (and Many Mistakes) on Thriving as a Consultant in the Age of AI
Avatar for David Carrasco davidcarrasco
0
160

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com