Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
130
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
110
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
230

Other Decks in Technology

See All in Technology
半年で、AIゼロ知識から AI中心開発組織の変革担当に至るまで
Avatar for ryu rfdnxbro
0
120
20251218_AIを活用した開発生産性向上の全社的な取り組みの進め方について / How to proceed with company-wide initiatives to improve development productivity using AI
Avatar for yayoi_dd yayoi_dd
0
590
Microsoft Agent 365 についてゆっくりじっくり理解する！
Avatar for skmkzyk skmkzyk
0
440
AWSインフルエンサーへの道 / load of AWS Influencer
Avatar for WHIsaiyo whisaiyo
0
190
「もしもデータ基盤開発で『強くてニューゲーム』ができたなら今の僕はどんなデータ基盤を作っただろう」
Avatar for AEON aeonpeople
0
170
AlmaLinux + KVM + Cockpit で始めるお手軽仮想化基盤 ～ 開発環境などでの利用を想定して ～
Avatar for koedoyoshida koedoyoshida
0
150
AIの長期記憶と短期記憶の違いについてAgentCoreを例に深掘ってみた
Avatar for やくも yakumo
4
470
AIBuildersDay_track_A_iidaxs
Avatar for iidaxs iidaxs
4
950
AWS Security Agentの紹介/introducing-aws-security-agent
Avatar for tomoki10 tomoki10
0
380
AWS re:Invent 2025~初参加の成果と学び~
Avatar for Kubo kubomasataka
0
170
会社紹介資料 / Sansan Company Profile
Avatar for Sansan, Inc. sansan33
PRO
11
390k
さくらのクラウド開発ふりかえり2025
Avatar for kazeburo kazeburo
2
190

Featured

See All Featured
The Curious Case for Waylosing
Avatar for Cassini Nazir cassininazir
0
190
VelocityConf: Rendering Performance Case Studies
Avatar for Addy Osmani addyosmani
333
24k
New Earth Scene 8
Avatar for Sydney Stone popppiees
0
1.2k
A Soul's Torment
Avatar for Nat Ma seathinner
1
1.9k
Deep Space Network (abreviated)
Avatar for Tony Rice tonyrice
0
20
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
56
14k
Bootstrapping a Software Product
Avatar for Garrett Dimon garrettdimon
PRO
307
120k
More Than Pixels: Becoming A User Experience Designer
Avatar for Michelle Schulp Hunt marktimemedia
2
250
Heart Work Chapter 1 - Part 1
Avatar for Lake Fama lfama
PRO
3
35k
What's in a price? How to price your products and services
Avatar for Michael Herold michaelherold
246
13k
The Impact of AI in SEO - AI Overviews June 2024 Edition
Avatar for Aleyda Solis aleyda
5
680
[RailsConf 2023] Rails as a piece of cake
Avatar for Vladimir Dementyev palkan
58
6.2k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com