Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
0
100

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015
Tweet

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
140
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
120
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
300
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
250
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
150
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
490
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
240

Other Decks in Technology

See All in Technology
JAWS DAYS 2026 CDP道場 事前説明会 / JAWS DAYS 2026 CDP Dojo briefing document
Avatar for Naomi Yamasaki naospon
0
170
Lookerの最新バージョンv26.2がやばい話
Avatar for Yuta Ozaki waiwai2111
1
160
「ヒットする」＋「近い」を同時にかなえるスマートサジェストの作り方.pdf
Avatar for なかしょ nakasho
0
110
「データとの対話」の現在地と未来
Avatar for kobakou kobakou
0
1.3k
マルチロールEMが実践する「組織のレジリエンス」を高めるための組織構造と人材配置戦略
Avatar for coconala_engineer coconala_engineer
2
420
越境する組織づくり ─ 多様性を前提にしたチームビルディングとリードの実践知
Avatar for hrpnx kido_engineer
1
110
LINE Messengerの次世代ストレージ選定
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
19
7.4k
kintone開発のプラットフォームエンジニアの紹介
Avatar for Cybozu cybozuinsideout
PRO
0
820
【SLO】"多様な期待値" と向き合ってみた
Avatar for kaita z63d
2
310
Master Dataグループ紹介資料
Avatar for Sansan, Inc. sansan33
PRO
1
4.4k
Windows ネットワークを再確認する
Avatar for Murachi Akira murachiakira
PRO
0
260
Eight Engineering Unit 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
1
6.9k

Featured

See All Featured
Why Your Marketing Sucks and What You Can Do About It - Sophie Logan
Avatar for Sophie Logan marketingsoph
0
99
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
231
22k
How STYLIGHT went responsive
Avatar for nonsquared nonsquared
100
6k
Marketing to machines
Avatar for Jono Alderson jonoalderson
1
5k
Gemini Prompt Engineering: Practical Techniques for Tangible AI Outcomes
Avatar for Mfonobong Umondia mfonobong
2
300
How to Build an AI Search Optimization Roadmap - Criteria and Steps to Take #SEOIRL
Avatar for Aleyda Solis aleyda
1
1.9k
Skip the Path - Find Your Career Trail
Avatar for Mark Kilby mkilby
1
72
RailsConf 2023
Avatar for Aaron Patterson tenderlove
30
1.4k
Automating Front-end Workflow
Avatar for Addy Osmani addyosmani
1370
200k
Building AI with AI
Avatar for Ines Montani inesmontani
PRO
1
760
Designing for humans not robots
Avatar for Tammie Lister tammielis
254
26k
Save Time (by Creating Custom Rails Generators)
Avatar for Garrett Dimon garrettdimon
PRO
32
2.3k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com