Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
76
0

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
140
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
120
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
280
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
180
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
65
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
420
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
390

Other Decks in Technology

See All in Technology
AI Engineering Summit Tokyo 2026 AIの前に、やることがある 〜医療データ企業の4フェーズ〜
Avatar for Daisuke Taniwaki dtaniwaki
0
2.5k
新しいVibe Codingと”自走”について
Avatar for watany watany
5
290
EventBridge Connection
Avatar for kensh _kensh
5
690
失敗を経て、Harness Engineering で 大切にしたいことを考える / Learning from Failure: What Matters in Harness Engineering
Avatar for 株式会社ビットキー / Bitkey Inc. bitkey
PRO
1
290
How Timee Delivers Day 1 Production Ready LLM Features
Avatar for tomoyks tomoyks
0
110
AmazonRoute 53ではじめてのドメイン取得！HTTPS化までの道のりを整理してみた
Avatar for usanchuu usanchuu
3
130
日本 Fintech 未来予測レポート 2027〜2028年（手動編集版）
Avatar for 8maki 8maki
0
1.5k
Djangoユーザが知っ得なPostgreSQL機能 - 設計の選択肢を増やす / Djang-use-PostgreSQL
Avatar for soudai sone soudai
PRO
1
230
On-behalf-of Token exchange with AgentCore Identity
Avatar for iganin hironobuiga
2
140
地球に⽣きるAI —GeoAIと「中間領域」— / AI Living on Earth — GeoAI and the “Intermediate Layer” —
Avatar for Kiyota Yoji, Ph.D. ykiyota
0
270
データサイエンスを価値につなげるプロジェクト設計 〜 DS一年目が現場で得た気づき 〜
Avatar for Yusei Doi ysd113
1
150
「速く作る」から「正しく作る」へ ─ 生成AI時代の開発フロー改革の ロードマップと実行 ─
Avatar for starfish719 starfish719
0
9.8k

Featured

See All Featured
HU Berlin: Industrial-Strength Natural Language Processing with spaCy and Prodigy
Avatar for Ines Montani inesmontani
PRO
0
410
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
66
8.5k
Ten Tips & Tricks for a 🌱 transition
Avatar for Manu Carrasco Molina stuffmc
0
130
The AI Search Optimization Roadmap by Aleyda Solis
Avatar for Aleyda Solis aleyda
1
5.9k
Docker and Python
Avatar for Tania Allard trallard
47
3.9k
What does AI have to do with Human Rights?
Avatar for Per Axbom axbom
PRO
1
2.2k
How to build a perfect <img>
Avatar for Jono Alderson jonoalderson
1
5.6k
It's Worth the Effort
Avatar for 3n 3n
188
29k
SEO Brein meetup: CTRL+C is not how to scale international SEO
Avatar for Linda Hogenes lindahogenes
1
2.7k
Deep Space Network (abreviated)
Avatar for Tony Rice tonyrice
0
170
A brief & incomplete history of 
UX Design for the World Wide Web: 1989–2019
Avatar for Jason CranfordTeague jct
2
390
The Art of Delivering Value - GDevCon NA Keynote
Avatar for David Neal reverentgeek
16
2k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio