Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
120
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
240

Other Decks in Technology

See All in Technology
外部キー制約の知っておいて欲しいこと - RDBMSを正しく使うために必要なこと / FOREIGN KEY Night
Avatar for soudai sone soudai
PRO
12
5.6k
Ruby版 JSXのRuxが気になる
Avatar for SansanTech sansantech
PRO
0
160
プロポーザルに込める段取り八分
Avatar for ShoheiMitani shoheimitani
1
540
日本の85%が使う公共SaaSは、どう育ったのか
Avatar for たけだ taketakekaho
1
230
Frontier Agents (Kiro autonomous agent / AWS Security Agent / AWS DevOps Agent) の紹介
Avatar for msysh msysh
3
180
プロダクト成長を支える開発基盤とスケールに伴う課題
Avatar for yuu26 yuu26
4
1.4k
小さく始めるBCP ― 多プロダクト環境で始める最初の一歩
Avatar for kekke-n kekke_n
1
460
2026年、サーバーレスの現在地 -「制約と戦う技術」から「当たり前の実行基盤」へ- /serverless2026
Avatar for Serverless Operations slsops
2
260
Oracle AI Database 移行・アップグレード勉強会 - RAT活用編
Avatar for oracle4engineer oracle4engineer
PRO
0
100
ブロックテーマでサイトをリニューアルした話 / 2026-01-31 Kansai WordPress Meetup
Avatar for Toro_Unit (Hiroshi Urabe) torounit
0
470
Tebiki Engineering Team Deck
Avatar for Tebiki, Inc. tebiki
0
24k
セキュリティについて学ぶ会 / 2026 01 25 Takamatsu WordPress Meetup
Avatar for Rocket Martue rocketmartue
1
310

Featured

See All Featured
Java REST API Framework Comparison - PWX 2021
Avatar for Matt Raible mraible
34
9.1k
Gemini Prompt Engineering: Practical Techniques for Tangible AI Outcomes
Avatar for Mfonobong Umondia mfonobong
2
280
We Analyzed 250 Million AI Search Results: Here's What I Found
Avatar for Josh Blyskal joshbly
1
740
KATA
Avatar for Megan Lloyd mclloyd
PRO
34
15k
How to optimise 3,500 product descriptions for ecommerce in one day using ChatGPT
Avatar for Katarina Dahlin katarinadahlin
PRO
0
3.4k
Context Engineering - Making Every Token Count
Avatar for Addy Osmani addyosmani
9
660
Done Done
Avatar for chrislema chrislema
186
16k
Why Our Code Smells
Avatar for Brandon Keepers bkeepers
PRO
340
58k
The Limits of Empathy - UXLibs8
Avatar for Cassini Nazir cassininazir
1
220
The Art of Delivering Value - GDevCon NA Keynote
Avatar for David Neal reverentgeek
16
1.8k
The Web Performance Landscape in 2024 [PerfNow 2024]
Avatar for Tammy Everts tammyeverts
12
1k
Building AI with AI
Avatar for Ines Montani inesmontani
PRO
1
700

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com