Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
120
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
240

Other Decks in Technology

See All in Technology
AI駆動開発を事業のコアに置く
Avatar for 鬼澤輔 tasukuonizawa
1
280
15 years with Rails and DDD (AI Edition)
Avatar for Andrzej Krzywda andrzejkrzywda
0
200
Frontier Agents (Kiro autonomous agent / AWS Security Agent / AWS DevOps Agent) の紹介
Avatar for msysh msysh
3
180
Amazon S3 Vectorsを使って資格勉強用AIエージェントを構築してみた
Avatar for usanchuu usanchuu
3
450
Context Engineeringが企業で不可欠になる理由
Avatar for Hirosato Gamo hirosatogamo
PRO
3
620
OCI Database Management サービス詳細
Avatar for oracle4engineer oracle4engineer
PRO
1
7.4k
ブロックテーマでサイトをリニューアルした話 / 2026-01-31 Kansai WordPress Meetup
Avatar for Toro_Unit (Hiroshi Urabe) torounit
0
470
セキュリティについて学ぶ会 / 2026 01 25 Takamatsu WordPress Meetup
Avatar for Rocket Martue rocketmartue
1
310
Contract One Engineering Unit 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
0
13k
量子クラウドサービスの裏側　〜Deep Dive into OQTOPUS〜
Avatar for OQTOPUS oqtopus
0
130
AzureでのIaC - Bicep? Terraform? それ早く言ってよ会議
Avatar for Toru Makabe torumakabe
1
580
生成AIを活用した音声文字起こしシステムの2つの構築パターンについて
Avatar for Kazuki Miura miu_crescent
PRO
3
210

Featured

See All Featured
Introduction to Domain-Driven Design and Collaborative software design
Avatar for Kenny Baas-Schwegler baasie
1
590
Visualization
Avatar for Eitan Lees eitanlees
150
17k
Unsuck your backbone
Avatar for Amy Palamountain ammeep
671
58k
Easily Structure & Communicate Ideas using Wireframe
Avatar for Afnizar Nur Ghifari afnizarnur
194
17k
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
240
Improving Core Web Vitals using Speculation Rules API
Avatar for Sergey Chernyshev sergeychernyshev
21
1.4k
Bash Introduction
Avatar for André Augusto Costa Santos 62gerente
615
210k
The Curious Case for Waylosing
Avatar for Cassini Nazir cassininazir
0
240
Designing Powerful Visuals for Engaging Learning
Avatar for Mike Taylor tmiket
0
240
Designing Dashboards & Data Visualisations in Web Apps
Avatar for Des Traynor destraynor
231
54k
Speed Design
Avatar for Sergey Chernyshev sergeychernyshev
33
1.5k
How to build a perfect <img>
Avatar for Jono Alderson jonoalderson
1
4.9k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com