Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
0
69

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017
Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
130
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
100
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
240
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
160
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
59
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
400
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
350

Other Decks in Technology

See All in Technology
Automating Web Accessibility Testing with AI Agents
Avatar for South maminami373
0
1.3k
Django's GeneratedField by example - DjangoCon US 2025
Avatar for Paolo Melchiorre pauloxnet
0
150
Agile PBL at New Grads Trainings
Avatar for Yasunobu Kawaguchi kawaguti
PRO
1
440
「何となくテストする」を卒業するためにプロダクトが動く仕組みを理解しよう
Avatar for kawabeaver kawabeaver
0
420
フルカイテン株式会社 エンジニア向け採用資料
Avatar for FULL KAITEN fullkaiten
0
8.8k
これでもう迷わない！Jetpack Composeの書き方実践ガイド
Avatar for ZOZO Developers zozotech
PRO
0
900
Language Update: Java
Avatar for Yuichi.Sakuraba skrb
2
300
Rustから学ぶ 非同期処理の仕組み
Avatar for skanehira skanehira
1
140
TS-S205_昨年対比2倍以上の機能追加を実現するデータ基盤プロジェクトでのAI活用について
Avatar for K.Mitsuhashi kaz3284
1
170
【初心者向け】ローカルLLMの色々な動かし方まとめ
Avatar for Aratako aratako
7
3.5k
5年目から始める Vue3 サイト改善 #frontendo
Avatar for Kihara, Takuya tacck
PRO
3
220
まずはマネコンでちゃちゃっと作ってから、それをCDKにしてみよか。
Avatar for ヤマダ（北野） yamada_r
2
110

Featured

See All Featured
The Invisible Side of Design
Avatar for Vitaly Friedman smashingmag
301
51k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
Avatar for Stéphanie Walter stephaniewalter
285
13k
BBQ
Avatar for Matthew Crist matthewcrist
89
9.8k
Dealing with People You Can't Stand - Big Design 2015
Avatar for Cassini Nazir cassininazir
367
27k
The Power of CSS Pseudo Elements
Avatar for Geoffrey Crofte geoffreycrofte
77
6k
How GitHub (no longer) Works
Avatar for Zach Holman holman
315
140k
It's Worth the Effort
Avatar for 3n 3n
187
28k
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
56
13k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
Avatar for Morgane Peng morganepeng
131
19k
How to Ace a Technical Interview
Avatar for Jacob Kaplan-Moss jacobian
279
23k
Why You Should Never Use an ORM
Avatar for John Nunemaker jnunemaker
PRO
59
9.5k
Typedesign – Prime Four
Avatar for Hannes Fritz hannesfritz
42
2.8k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio