Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
0
67

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017
Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
120
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
93
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
230
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
150
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
57
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
380
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
330

Other Decks in Technology

See All in Technology
Perl歴約10年のエンジニアがフルスタックTypeScriptに出会ってみた
Avatar for papix papix
1
250
30代からでも遅くない！ 内製開発の世界に飛び込み、最前線で戦うLLMアプリ開発エンジニアになろう
Avatar for みのるん minorun365
PRO
16
5k
Mastraに入門してみた ~AWS CDKを添えて~
Avatar for つくぼし tsukuboshi
0
370
AIと共に乗り越える、 入社後２ヶ月の苦労と学習の軌跡
Avatar for Sai Kaneko sai_kaneko
0
180
SnowflakeとDatabricks両方でRAGを構築してみた
Avatar for camay kameitomohiro
1
550
MySQL Indexes and Histograms – How they really speed up your queries
Avatar for lefred lefred
0
140
Porting PicoRuby to Another Microcontroller: ESP32
Avatar for Y_uuu yuuu
4
520
グループ ポリシー再確認 (2)
Avatar for Murachi Akira murachiakira
0
200
Microsoft Fabric vs Databricks vs (Snowflake) -若手エンジニアがそれぞれの強みと違いを比較してみた- "A Young Engineer's Comparison of Their Strengths and Differences"
Avatar for 大竹礼二（REIJI OTAKE） reireireijinjin6
1
120
コードや知識を組み込む / Incorporating Codes and Knowledge
Avatar for Kenji Saito ks91
PRO
0
150
新卒エンジニアがCICDをモダナイズしてみた話
Avatar for Akashi_SN akashi_sn
2
280
バクラクの認証基盤の成長と現在地 / bakuraku-authn-platform
Avatar for convto convto
4
870

Featured

See All Featured
Stop Working from a Prison Cell
Avatar for Chris Michel hatefulcrawdad
268
20k
Fashionably flexible responsive web design (full day workshop)
Avatar for Andy Clarke malarkey
407
66k
Thoughts on Productivity
Avatar for Jon Yablonski jonyablonski
69
4.6k
The Pragmatic Product Professional
Avatar for Laura Van Doore lauravandoore
33
6.6k
Practical Tips for Bootstrapping Information Extraction Pipelines
Avatar for Matthew Honnibal honnibal
PRO
19
1.2k
Typedesign – Prime Four
Avatar for Hannes Fritz hannesfritz
41
2.6k
YesSQL, Process and Tooling at Scale
Avatar for Rocio Delgado rocio
172
14k
Building a Scalable Design System with Sketch
Avatar for Laura Van Doore lauravandoore
462
33k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
227
22k
Code Review Best Practice
Avatar for Trisha Gee trishagee
67
18k
Put a Button on it: Removing Barriers to Going Fast.
Avatar for kastner kastner
60
3.8k
For a Future-Friendly Web
Avatar for Brad Frost brad_frost
177
9.7k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | vmaggio@fbk.eu Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio