Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
0
69

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017
Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
130
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
100
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
240
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
160
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
59
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
400
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
350

Other Decks in Technology

See All in Technology
アノテーション作業書作成のGood Practice
Avatar for Cierpa & Company cierpa0905
PRO
1
410
Data Engineering Guide 2025 #data_summit_findy by @Kazaneya_PR / 20251106
Avatar for 風音屋 (Kazaneya) kazaneya
PRO
8
1.5k
技術の総合格闘技！？AIインフラの現在と未来。
Avatar for ebiken ebiken
PRO
0
160
SREのキャリアから経営に近づく - Enterprise Risk Managementを基に -
Avatar for shonansurvivors shonansurvivors
1
740
メタプログラミングRuby読書会の活用
Avatar for Shinichi Maeshima willnet
0
120
datadog-incident-management-intro
Avatar for tetsuya28 tetsuya28
0
120
今から間に合う re:Invent 準備グッズと現地の地図、その他ラスベガスを周る際の Tips/reinvent-preparation-guide
Avatar for emi emiki
1
290
Boxを“使われる場”にする統制と自動化の仕組み
Avatar for 株式会社出前館 demaecan
0
200
最近読んで良かった本 / Yokohama North Meetup #10
Avatar for Takuya Mukohira mktakuya
0
920
AIがコードを書いてくれるなら、新米エンジニアは何をする？ / komekaigi2025
Avatar for Yukiya Nakagawa nkzn
25
17k
어떤 개발자가 되고 싶은가?
Avatar for Arawn Park arawn
1
450
Design and implementation of "Markdown to Google Slides" / phpconfuk 2025
Avatar for Ken’ichiro Oyama k1low
1
200

Featured

See All Featured
Why You Should Never Use an ORM
Avatar for John Nunemaker jnunemaker
PRO
60
9.6k
Designing for Performance
Avatar for Lara Hogan lara
610
69k
Bash Introduction
Avatar for André Augusto Costa Santos 62gerente
615
210k
The Success of Rails: Ensuring Growth for the Next 100 Years
Avatar for Eileen M. Uchitelle eileencodes
46
7.8k
Writing Fast Ruby
Avatar for Erik Berlin sferik
630
62k
GraphQLとの向き合い方2022年版
Avatar for Yosuke Kurami quramy
49
14k
Scaling GitHub
Avatar for Zach Holman holman
463
140k
Producing Creativity
Avatar for Steve Smith orderedlist
PRO
348
40k
The Straight Up "How To Draw Better" Workshop
Avatar for Dennis Kardys denniskardys
239
140k
BBQ
Avatar for Matthew Crist matthewcrist
89
9.9k
Stop Working from a Prison Cell
Avatar for Chris Michel hatefulcrawdad
272
21k
Being A Developer After 40
Avatar for Adrian Kosmaczewski akosma
91
590k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio