Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
0
69

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017
Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
130
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
98
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
230
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
160
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
59
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
400
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
340

Other Decks in Technology

See All in Technology
【CEDEC2025】現場を理解して実現！ゲーム開発を効率化するWebサービスの開発と、利用促進のための継続的な改善
Avatar for Cygames cygames
PRO
0
370
東京海上日動におけるセキュアな開発プロセスの取り組み
Avatar for miyabit miyabit
0
200
Turn Your Community into a Fundraising Catalyst for Black Philanthropy Month
Avatar for Auctria auctria
PRO
0
200
Ktor + Google Cloud Tasks/PubSub におけるOTel Messaging計装の実践
Avatar for SansanTech sansantech
PRO
1
330
私とAWSとの関わりの歩み～意志あるところに道は開けるかも？～
Avatar for nagisa_53 nagisa53
1
130
The Madness of Multiple Gemini CLIs Developing Simultaneously with Jujutsu
Avatar for Gunther Brunner gunta
1
2.8k
AIに全任せしないコーディングとマネジメント思考
Avatar for kichion kikuchikakeru
0
280
ユーザー理解の爆速化とPdMの価値
Avatar for KAKEHASHI kakehashi
PRO
1
110
増え続ける脆弱性に立ち向かう: 事前対策と優先度づけによる 持続可能な脆弱性管理 / Confronting the Rise of Vulnerabilities: Sustainable Management Through Proactive Measures and Prioritization
Avatar for NTT docomo Business nttcom
1
210
Step Functions First - サーバーレスアーキテクチャの新しいパラダイム
Avatar for Taiki Sugawara taikis
1
280
株式会社島津製作所_研究開発（集団協業と知的生産）の現場を支える、OSS知識基盤システムの導入
Avatar for Kuniharu, AKAHANE (赤羽根/州晴) akahane92
1
1.3k
AI時代の知識創造 ─GeminiとSECIモデルで読み解く “暗黙知”と創造の境界線
Avatar for Yukinaga OISHI nyagasan
0
160

Featured

See All Featured
Documentation Writing (for coders)
Avatar for Carmen Chung carmenintech
72
4.9k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
Avatar for Eileen M. Uchitelle eileencodes
138
34k
The MySQL Ecosystem @ GitHub 2015
Avatar for Sam Lambert samlambert
251
13k
Art, The Web, and Tiny UX
Avatar for Lynn Fisher lynnandtonic
301
21k
ピンチをチャンスに：未来をつくるプロダクトロードマップ #pmconf2020
Avatar for Aki / @LoveIdahoBurger aki_iinuma
126
53k
Building a Scalable Design System with Sketch
Avatar for Laura Van Doore lauravandoore
462
33k
Large-scale JavaScript Application Architecture
Avatar for Addy Osmani addyosmani
512
110k
Six Lessons from altMBA
Avatar for Skipper Chong Warson skipperchong
28
3.9k
It's Worth the Effort
Avatar for 3n 3n
185
28k
Building Applications with DynamoDB
Avatar for Matt Wood mza
95
6.5k
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
37
2.8k
Visualization
Avatar for Eitan Lees eitanlees
146
16k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio