Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
120
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
240

Other Decks in Technology

See All in Technology
Introduction to Sansan for Engineers / エンジニア向け会社紹介
Avatar for Sansan, Inc. sansan33
PRO
6
68k
ブロックテーマでサイトをリニューアルした話 / 2026-01-31 Kansai WordPress Meetup
Avatar for Toro_Unit (Hiroshi Urabe) torounit
0
470
今日から始める Amazon Bedrock AgentCore
Avatar for Har1101 har1101
4
410
小さく始めるBCP ― 多プロダクト環境で始める最初の一歩
Avatar for kekke-n kekke_n
1
450
2026年、サーバーレスの現在地 -「制約と戦う技術」から「当たり前の実行基盤」へ- /serverless2026
Avatar for Serverless Operations slsops
2
260
フルカイテン株式会社 エンジニア向け採用資料
Avatar for FULL KAITEN fullkaiten
0
10k
AIエージェントを開発しよう！-AgentCore活用の勘所-
Avatar for YukiOgawa yukiogawa
0
170
Bedrock PolicyでAmazon Bedrock Guardrails利用を強制してみた
Avatar for Yudai Jinno yuu551
0
250
外部キー制約の知っておいて欲しいこと - RDBMSを正しく使うために必要なこと / FOREIGN KEY Night
Avatar for soudai sone soudai
PRO
12
5.6k
Oracle Base Database Service 技術詳細
Avatar for oracle4engineer oracle4engineer
PRO
15
93k
FinTech SREのAWSサービス活用/Leveraging AWS Services in FinTech SRE
Avatar for maaaato maaaato
0
130
こんなところでも（地味に）活躍するImage Modeさんを知ってるかい？- Image Mode for OpenShift -
Avatar for Masataka Tsukamoto tsukaman
1
160

Featured

See All Featured
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
39
3k
How to Get Subject Matter Experts Bought In and Actively Contributing to SEO & PR Initiatives.
Avatar for Liv Day livdayseo
0
67
VelocityConf: Rendering Performance Case Studies
Avatar for Addy Osmani addyosmani
333
24k
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
32
2.1k
Being A Developer After 40
Avatar for Adrian Kosmaczewski akosma
91
590k
Ten Tips & Tricks for a 🌱 transition
Avatar for Manu Carrasco Molina stuffmc
0
71
Leveraging Curiosity to Care for An Aging Population
Avatar for Cassini Nazir cassininazir
1
160
The Power of CSS Pseudo Elements
Avatar for Geoffrey Crofte geoffreycrofte
80
6.2k
Sam Torres - BigQuery for SEOs
Avatar for Tech SEO Connect techseoconnect
PRO
0
190
RailsConf 2023
Avatar for Aaron Patterson tenderlove
30
1.3k
Keith and Marios Guide to Fast Websites
Avatar for Keith Pitt keithpitt
413
23k
A brief & incomplete history of 
UX Design for the World Wide Web: 1989–2019
Avatar for Jason CranfordTeague jct
1
300

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com