Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
110
0

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
140
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
120
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
310
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
160
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
250

Other Decks in Technology

See All in Technology
Revisiting [CLS] and Patch Token Interaction in Vision Transformers
Avatar for yu4u yu4u
0
400
GKE Agent SandboxでAIが生成したコードを 安全に実行してみた
Avatar for Lamaglama39 lamaglama39
0
110
需要創出(Chatwork)×供給(BPaaS) フライホイールとMoat 実行能力の最適配置とAI戦略
Avatar for kubell kubell_hr
0
990
AIはハッカーを減らすのか、増やすのか？──現役ホワイトハッカーから見るAI時代のリアル【MEGU-Meet】
Avatar for サイバーセキュリティクラウド cscengineer
PRO
0
220
20260428_Product Management Summit_tadokoroyoshiro
Avatar for tadokoro_yoshiro tadokoro_yoshiro
14
15k
AIが盛んな時代に 技術記事を書き始めて起きた私の中での小さな変化
Avatar for 松尾淳平 peintangos
0
260
20260428_Product Management Summit_Loglass_JoeHirose
Avatar for 広瀬丈 loglassjoe
3
4k
Standards et agents IA : un tour d’horizon de MCP, A2A, ADK et plus encore
Avatar for Guillaume Laforge glaforge
0
210
【技術書典20】OpenFOAM（自宅で深める流体解析）流れと熱移動（２）
Avatar for kamakiri1225 kamakiri1225
0
280
Building a Standalone Programming Environment
Avatar for Harukasan harukasan
PRO
1
100
プラットフォームエンジニアリングの実践 - AWS コンテナサービスで構築する社内プラットフォーム / AWS Containers Platform Meetup #1
Avatar for Masatoshi Hayashi literalice
1
210
Keeping Ruby Running on Cygwin
Avatar for fd0 fd0
0
180

Featured

See All Featured
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
Avatar for Tammy Everts tammyeverts
55
3.3k
Lessons Learnt from Crawling 1000+ Websites
Avatar for Charles Meaden charlesmeaden
PRO
1
1.2k
Product Roadmaps are Hard
Avatar for C. Todd Lombardo iamctodd
PRO
55
12k
SEOcharity - Dark patterns in SEO and UX: How to avoid them and build a more ethical web
Avatar for Sara Fernández Carmona sarafernandez
0
170
SEO Brein meetup: CTRL+C is not how to scale international SEO
Avatar for Linda Hogenes lindahogenes
1
2.6k
How Software Deployment tools have changed in the past 20 years
Avatar for Geshan Manandhar geshan
0
33k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
231
23k
AI Search:
Where Are We & What Can We Do About It?
Avatar for Aleyda Solis aleyda
0
7.4k
The innovator’s Mindset - 
Leading Through an Era of Exponential Change - McGill University 2025
Avatar for Jean-Marc De Jonghe jdejongh
PRO
1
160
4 Signs Your Business is Dying
Avatar for Josh Pigford shpigford
187
22k
Stewardship and Sustainability of Urban and Community Forests
Avatar for Eric Wiseman pwiseman
0
190
We Have a Design System, Now What?
Avatar for Morgane Peng morganepeng
55
8.1k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com