Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
110
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
480
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
230

Other Decks in Technology

See All in Technology
ブロックテーマとこれからの WordPress サイト制作 / Toyama WordPress Meetup Vol.81
Avatar for Toro_Unit (Hiroshi Urabe) torounit
0
370
Symfony AI in Action
Avatar for Christopher Hertel el_stoffel
2
390
Design System Documentation Tooling 2025
Avatar for takanorip takanorip
2
990
Karate+Database RiderによるAPI自動テスト導入工数をCline＋GitLab MCPを使って2割削減を目指す！ / 20251206 Kazuki Takahashi
Avatar for SHIFT EVOLVE shift_evolve
PRO
1
330
安いGPUレンタルサービスについて
Avatar for Aratako aratako
2
2.6k
Claude Code はじめてガイド -1時間で学べるAI駆動開発の基本と実践-
Avatar for Oikon oikon48
45
27k
会社紹介資料 / Sansan Company Profile
Avatar for Sansan, Inc. sansan33
PRO
11
390k
EM歴1年10ヶ月のぼくがぶち当たった苦悩とこれからへ向けて
Avatar for maaaato maaaato
0
160
技術以外の世界に『越境』しエンジニアとして進化を遂げる 〜Kotlinへの愛とDevHRとしての挑戦を添えて〜
Avatar for subroh_0508 subroh0508
1
310
AIと二人三脚で育てた、個人開発アプリグロース術
Avatar for ZOZO Developers zozotech
PRO
0
560
バグハンター視点によるサプライチェーンの脆弱性
Avatar for morioka12 scgajge12
2
630
MS Ignite 2025で発表されたFoundry IQをRecap
Avatar for Akira Sato satodayo
3
250

Featured

See All Featured
ReactJS: Keep Simple. Everything can be a component!
Avatar for Pedro Nauck pedronauck
666
130k
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
128
54k
Art, The Web, and Tiny UX
Avatar for Lynn Fisher lynnandtonic
303
21k
Building Flexible Design Systems
Avatar for Yesenia Perez-Cruz yeseniaperezcruz
329
39k
Visualization
Avatar for Eitan Lees eitanlees
150
16k
Speed Design
Avatar for Sergey Chernyshev sergeychernyshev
33
1.4k
Building an army of robots
Avatar for Kyle Neath kneath
306
46k
Building Applications with DynamoDB
Avatar for Matt Wood mza
96
6.8k
Why You Should Never Use an ORM
Avatar for John Nunemaker jnunemaker
PRO
60
9.6k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
Avatar for Irina Nazarova irinanazarova
9
1.1k
Embracing the Ebb and Flow
Avatar for Simon Collison colly
88
4.9k
Understanding Cognitive Biases in Performance Measurement
Avatar for Philip Tellis bluesmoon
31
2.7k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com