Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Szymon Sobczak - Hadoop + Storm

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for Base Lab Base Lab
May 07, 2015
Technology
130
0

Szymon Sobczak - Hadoop + Storm

Combo for realtime big data systems

Avatar for Base Lab

Base Lab

May 07, 2015

More Decks by Base Lab

See All by Base Lab
Slawek Skowron - Monitoring @ Scale
Avatar for Base Lab baselab
0
170
Karol Nowak - Monitoring clock drift in Amazon EC2 environment
Avatar for Base Lab baselab
0
140
Tomasz Nowak - Web Application Testing made easy
Avatar for Base Lab baselab
0
320
Szymon Pawlik - UX i Automatyzacja czyli jak testerzy mogą poprawić produkt.
Avatar for Base Lab baselab
0
260
Mateusz Herych - LIKE '%smth%' is not the way
Avatar for Base Lab baselab
0
160
Jerzy Chałupski - Offline mode in Android apps
Avatar for Base Lab baselab
3
500
Jerzy Chałupski - Data model on Android
Avatar for Base Lab baselab
4
270

Other Decks in Technology

See All in Technology
AI時代の「技術的負債」の変質ー概念の終焉と再解釈、エージェントと共に向かう先
Avatar for nwiizo nwiizo
1
2.9k
AI時代、データエンジニアが一番おもろい
Avatar for ゲンシュン genshun9
0
670
CLIライブラリ開発を支える技術
Avatar for Hiroshi TANABE htnabe
0
130
2026-09-11 【Snowflake World Tour Tokyo 2026】Snowflakeを起点に、AI Agentが自律稼働し続ける未来へ / Driving AI Agents with Snowflake
Avatar for civitaspo civitaspo
0
700
Reactの設計論
Avatar for uhyo uhyo
24
14k
AIで社員の自主発信に広報目線を組み込む
Avatar for もっさん _mossann_t
0
130
JSONataとAWS Step Functionsで目指すRuntimelessな世界
Avatar for mu7889yoon / Yuta Nakamura mu7889yoon
0
180
技術的負債から考える、AI時代のエンジニアリング投資 — ビズリーチの技術的負債と向き合った経験から、変更し続けられるソフトウェアを考える/ technical-debt-con2026
Avatar for Visional Engineering ＆ Design visional_engineering_and_design
4
3.2k
SREは、MCPとAutopilotをこう使え！
Avatar for kazumax55 kazumax55
3
890
新機種発売前に見直そう！端末移行で再ログインが要るアプリ・要らないアプリは何が違うのか 〜シームレスに再開できる設計と実装〜
Avatar for ZOZO Developers zozotech
PRO
0
210
ソフトウェアDNAとクラウドエージェントのススメ
Avatar for Cloud Ace cloudace
0
110
「守り」で活用するオンデバイスLLM 〜写ってはいけないを総力戦で防ぐ〜 / iOSDC Japan 2026
Avatar for nakamuuu nakamuuu
0
170

Featured

See All Featured
Navigating the moral maze — ethical principles for Al-driven product design
Avatar for Skipper Chong Warson skipperchong
2
550
Stewardship and Sustainability of Urban and Community Forests
Avatar for Eric Wiseman pwiseman
0
530
How To Speak Unicorn (iThemes Webinar)
Avatar for Michelle Schulp Hunt marktimemedia
1
580
Designing Experiences People Love
Avatar for Jonathan Moore moore
143
24k
Max Prin - Stacking Signals: How International SEO Comes Together (And Falls Apart)
Avatar for Tech SEO Connect techseoconnect
PRO
0
470
Bridging the Design Gap: How Collaborative Modelling removes blockers to flow between stakeholders and teams @FastFlow conf
Avatar for Kenny Baas-Schwegler baasie
0
700
Reality Check: Gamification 10 Years Later
Avatar for Sebastian Deterding codingconduct
0
2.3k
The Hidden Cost of Media on the Web [PixelPalooza 2025]
Avatar for Tammy Everts tammyeverts
2
500
The Cost Of JavaScript in 2023
Avatar for Addy Osmani addyosmani
55
10k
The innovator’s Mindset - 
Leading Through an Era of Exponential Change - McGill University 2025
Avatar for Jean-Marc De Jonghe jdejongh
PRO
1
340
AI: The stuff that nobody shows you
Avatar for John Nunemaker jnunemaker
PRO
10
1k
Leadership Guide Workshop - DevTernity 2021
Avatar for David Neal reverentgeek
1
370

Transcript

  1. Szymon Sobczak

  2. Hadoop + Storm Combo for realtime big data systems

  3. Plan • Hadoop & Storm • Our setup • What

    projects are we running • Decisions we had to make

  4. Hadoop

  5. None

  6. Storm “Ala ma kota Artura" “ala”, “ma”, “kota”, “artura" “ala”,

    “ma”, “kota”, “artura" a: 2 k: 1 m: 1 “ala”

  7. Storm “Ala ma kota Artura" “ma” “ala”, “ma”, “kota”, “artura"

    “ala” a: 2 m: 1 k: 1 “ala”, “artura" “kota”

  8. Common traits

  9. None

  10. Base infrastructure services

  11. Understand how the entire Base system works services

  12. Big Data S3 uploader

  13. Four example projects • Debugging • Reporting • Email intelligence

    • Forecasting

  14. Debugging S3 uploader

  15. Reporting

  16. Email analysis S3 uploader

  17. Forecasting S3 uploader

  18. Decisions we made ☑ Collect *all* data ☑ Put them

    in one place ☐ Build platform for engineers ☐ Same code on Hadoop and Storm

  19. Summary S3 uploader

  20. Questions?

  21. Thank you [email protected] bigdata.getbase.com