Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
120
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
240

Other Decks in Technology

See All in Technology
歴史から学ぶ、Goのメモリ管理基礎
Avatar for Takuto Nagami logica0419
14
2.6k
業務の煩悩を祓うAI活用術108選 / AI 108 Usages
Avatar for 株式会社スマートバンク smartbank
9
21k
AWS re:Invent2025最新動向まとめ（NRIグループre:Cap 2025）
Avatar for KoheiGamo gamogamo
0
170
AI時代のアジャイルチームを目指して ー スクラムというコンフォートゾーンからの脱却 ー / Toward Agile Teams in the Age of AI
Avatar for TAKAKING22 takaking22
11
6.1k
SES向け、生成AI時代におけるエンジニアリングとセキュリティ
Avatar for LongbowXXX longbowxxx
0
310
製造業から学んだ「本質を守り現場に合わせるアジャイル実践」
Avatar for yaskamito kamitokusari
0
560
投資戦略を量産せよ 2 - マケデコセミナー(2025/12/26)
Avatar for tomo gamella
1
640
スクラムマスターが
スクラムチームに入って取り組む5つのこと
- スクラムガイドには書いてないけど入った当初から取り組んでおきたい大切なこと -
Avatar for scrummasudar scrummasudar
2
1.9k
コミュニティが持つ「学びと成長の場」としての作用 / RSGT2026
Avatar for ama-ch ama_ch
0
190
Sansan Engineering Unit 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
1
3.6k
202512_AIoT.pdf
Avatar for AIxIoTビジネス共創ラボ iotcomjpadmin
0
190
純粋なイミュータブルモデルを設計してからイベントソーシングと組み合わせるDeciderの実践方法の紹介 /Introducing Decider Pattern with Event Sourcing
Avatar for Tomohisa Takaoka tomohisa
1
920

Featured

See All Featured
Agile Actions for Facilitating Distributed Teams - ADO2019
Avatar for Mark Kilby mkilby
0
100
Digital Ethics as a Driver of Design Innovation
Avatar for Per Axbom axbom
PRO
0
140
Agile Leadership in an Agile Organization
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
67
We Have a Design System, Now What?
Avatar for Morgane Peng morganepeng
54
8k
Product Roadmaps are Hard
Avatar for C. Todd Lombardo iamctodd
PRO
55
12k
How To Stay Up To Date on Web Technology
Avatar for Chris Coyier chriscoyier
791
250k
Odyssey Design
Avatar for Ryan Kendrick rkendrick25
PRO
0
460
Building a Modern Day 
E-commerce SEO Strategy
Avatar for Aleyda Solis aleyda
45
8.5k
Building Experiences: Design Systems, User Experience, and Full Site Editing
Avatar for Michelle Schulp Hunt marktimemedia
0
370
Winning Ecommerce Organic Search in an AI Era - #searchnstuff2025
Avatar for Aleyda Solis aleyda
0
1.8k
Bootstrapping a Software Product
Avatar for Garrett Dimon garrettdimon
PRO
307
120k
The Straight Up "How To Draw Better" Workshop
Avatar for Dennis Kardys denniskardys
239
140k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com