Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Base Lab
May 07, 2015
Technology
0
94

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
baselab
0
110
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
baselab
0
100
Tomasz Nowak - Web Application Testing made easy
baselab
0
290
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
baselab
0
240
Mateusz Herych - LIKE '%smth%' is not the way
baselab
0
140
Jerzy Chałupski - Offline mode in Android apps
baselab
3
470
Jerzy Chałupski - Data model on Android
baselab
4
220

Other Decks in Technology

See All in Technology
社外コミュニティで学び社内に活かす共に学ぶプロジェクトの実践/backlogworld2024
nishiuma
0
260
Qiita埋め込み用スライド
naoki_0531
0
4.8k
Jetpack Composeで始めるServer Cache State
ogaclejapan
2
170
UI State設計とテスト方針
rmakiyama
2
580
日本版とグローバル版のモバイルアプリ統合の開発の裏側と今後の展望
miichan
1
130
なぜCodeceptJSを選んだか
goataka
0
160
マルチプロダクト開発の現場でAWS Security Hubを1年以上運用して得た教訓
muziyoshiz
3
2.3k
スタートアップで取り組んでいるAzureとMicrosoft 365のセキュリティ対策/How to Improve Azure and Microsoft 365 Security at Startup
yuj1osm
0
210
成果を出しながら成長する、アウトプット駆動のキャッチアップ術 / Output-driven catch-up techniques to grow while producing results
aiandrox
0
310
Snowflake女子会#3 Snowpipeの良さを5分で語るよ
lana2548
0
230
どちらを使う?GitHub or Azure DevOps Ver. 24H2
kkamegawa
0
760
LINE Developersプロダクト(LIFF/LINE Login)におけるフロントエンド開発
lycorptech_jp
PRO
0
120

Featured

See All Featured
Typedesign – Prime Four
hannesfritz
40
2.4k
Put a Button on it: Removing Barriers to Going Fast.
kastner
59
3.6k
Visualization
eitanlees
146
15k
Being A Developer After 40
akosma
87
590k
Optimising Largest Contentful Paint
csswizardry
33
3k
Code Reviewing Like a Champion
maltzj
520
39k
Facilitating Awesome Meetings
lara
50
6.1k
Reflections from 52 weeks, 52 projects
jeffersonlam
347
20k
Gamification - CAS2011
davidbonilla
80
5.1k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
44
9.3k
RailsConf 2023
tenderlove
29
940
Intergalactic Javascript Robots from Outer Space
tanoku
270
27k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com