Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Szymon Sobczak - Hadoop + Storm

Avatar for Base Lab Base Lab
May 07, 2015
Technology
130
0

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
160
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
140
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
320
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
260
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
160
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
500
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
260

Other Decks in Technology

See All in Technology
Introduction to Sansan, inc / Sansan Global Development Center, Inc.
Avatar for Sansan, Inc. sansan33
PRO
0
3.2k
AI駆動開発をチームに根付かせる - 「1行も書かない」チームがHarnessを育てた1年 -
Avatar for Ken'ichirou Kimura kenichirokimura
6
3.3k
「ミスを許さない手順書」を作ってみた 〜 個人的にはこれ以上できることはあまりなさそう/20260827-ssmjp-operation-procedure-update
Avatar for opelab opelab
13
9.1k
atproto spaces概要
Avatar for yamarten yamarten
0
110
サイボウズ 開発本部採用ピッチ / Cybozu Engineer Recruit
Avatar for Cybozu cybozuinsideout
PRO
12
86k
ブラウザで変わるID連携（OAuth/OIDC Numa (Immersion) Workshop 2026）
Avatar for OpenID Foundation Japan oidfj
PRO
0
350
Digital Credentials API × OpenID4VP ブラウザ完結型本人確認の実装知見（OAuth/OIDC Numa (Immersion) Workshop 2026）
Avatar for OpenID Foundation Japan oidfj
PRO
0
340
Bill One 開発エンジニア 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
7
20k
現場の暗黙知を継承するAIエージェント — 対話から生まれる長期記憶と Skills
Avatar for Atsuki Shirasawa atsukish
0
130
顧客の要望は2次情報である 〜アンテナを張るFDEの構造論〜
Avatar for Noriaki Oji noriakioji
5
1k
みてねにおけるAI-DLC導入活動とAIドリブン開発の現在地/JAWS-UG AI-DLC #2
Avatar for Isao Shimizu isaoshimizu
2
280
Distributed Transactions Under Fire: Building a Zero-Oversell Flash Sale Platform with Amazon Aurora DSQL
Avatar for Yuuki Yamashita yama3133
1
110

Featured

See All Featured
The untapped power of vector embeddings
Avatar for Frank van Dijk frankvandijk
2
1.8k
The SEO identity crisis: Don't let AI make you average
Avatar for Varn varn
0
540
エンジニアに許された特別な時間の終わり
Avatar for watany watany
108
250k
Redefining SEO in the New Era of Traffic Generation
Avatar for Szymon Słowik szymonslowik
1
400
Imperfection Machines: The Place of Print at Facebook
Avatar for Scott Boms scottboms
270
14k
10 Git Anti Patterns You Should be Aware of
Avatar for Lemi Orhan Ergin lemiorhan
PRO
659
62k
Evolving SEO for Evolving Search Engines
Avatar for Ryan Jones ryanjones
0
270
Art, The Web, and Tiny UX
Avatar for Lynn Fisher lynnandtonic
304
22k
Why Our Code Smells
Avatar for Brandon Keepers bkeepers
PRO
340
58k
Jamie Indigo - Trashchat’s Guide to Black Boxes: Technical SEO Tactics for LLMs
Avatar for Tech SEO Connect techseoconnect
PRO
0
630
VelocityConf: Rendering Performance Case Studies
Avatar for Addy Osmani addyosmani
331
25k
Applied NLP in the Age of Generative AI
Avatar for Ines Montani inesmontani
PRO
4
2.4k

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com