Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Base Lab
May 07, 2015
Technology
0
90

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
baselab
0
110
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
baselab
0
96
Tomasz Nowak - Web Application Testing made easy
baselab
0
280
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
baselab
0
240
Mateusz Herych - LIKE '%smth%' is not the way
baselab
0
130
Jerzy Chałupski - Offline mode in Android apps
baselab
3
470
Jerzy Chałupski - Data model on Android
baselab
4
210

Other Decks in Technology

See All in Technology
疲弊しない！AWSセキュリティ統制の考え方 #devio_osakaday1
masahirokawahara
6
5.9k
Data and AI Governance: Existing Challenges and Emerging Trends
scotthsieh825
0
160
AIQ株式会社　エンジニア向け会社紹介資料
aiqlab
0
380
株式会社EventHub・エンジニア採用資料
eventhub
0
1.9k
アプリがつくるNOT A HOTELブランド
hokuts
1
450
GraphQL 成熟度モデルの紹介と、プロダクトに当てはめた事例 / GraphQL maturity model
mh4gf
4
190
転移学習とドメイン適応の基礎
kmatsui
2
570
カオナビの利用実績をアウトカムへつなげる旅 / example-of-data-management-startup-in-kaonavi
kaonavi
0
120
プロトタイピングによる不確実性の低減 / Reducing Uncertainty through Prototyping
ohbarye
4
280
シン・Kafka / shin-kafka
oracle4engineer
PRO
7
2.7k
検証を通して見えてきたTiDBの性能特性
lycorptech_jp
PRO
6
3.4k
Tableau事例紹介 / Tableau Case Study of Eureka
kazuya_araki_tokyo
1
170

Featured

See All Featured
The Power of CSS Pseudo Elements
geoffreycrofte
59
5k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
124
32k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
240
1.2M
Web Components: a chance to create the future
zenorocha
305
41k
BBQ
matthewcrist
80
8.7k
Designing with Data
zakiwarfel
95
4.8k
The Mythical Team-Month
searls
215
42k
The Cost Of JavaScript in 2023
addyosmani
14
3.8k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
6
990
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
273
13k
Making Projects Easy
brettharned
108
5.5k
Pencils Down: Stop Designing & Start Developing
hursman
117
11k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com