Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
0
69

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017
Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
130
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
100
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
240
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
160
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
59
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
400
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
360

Other Decks in Technology

See All in Technology
AWS IAM Identity Centerによる権限設定をグラフ構造で可視化＋グラフRAGへの挑戦
Avatar for Yuta Kimi ykimi
2
540
InsightX 会社説明資料/ Company deck
Avatar for InsightX insightx
0
220
re:Inventに行きたい　いつか行きたい　行けるようにできることは？
Avatar for Yuuki Yamashita yama3133
0
110
今から間に合う re:Invent 準備グッズと現地の地図、その他ラスベガスを周る際の Tips/reinvent-preparation-guide
Avatar for emi emiki
1
340
Master Dataグループ紹介資料
Avatar for Sansan, Inc. sansan33
PRO
1
3.9k
AI-ready"のための"データ基盤 〜 LLMOpsで事業貢献するための基盤づくり
Avatar for Izumu KUSUNOKI ismk
0
130
今のコンピュータ、AI にも Web にも 向いていないので 作り直そう!!
Avatar for piacerex piacerex
0
720
CloudComposerによる大規模ETL 「制御と実行の分離」の実践
Avatar for Tech Leverages leveragestech
0
200
20251102 WordCamp Kansai 2025
Avatar for Chiaki Okamoto chiilog
1
700
Spec Driven Development入門/spec_driven_development_for_learners
Avatar for Ryo Tomidokoro hanhan1978
1
980
設計に疎いエンジニアでも始めやすいアーキテクチャドキュメント
Avatar for Tomonori Hayashi / ぴーはや phaya72
30
20k
技術の総合格闘技！？AIインフラの現在と未来。
Avatar for ebiken ebiken
PRO
0
190

Featured

See All Featured
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
56
14k
ReactJS: Keep Simple. Everything can be a component!
Avatar for Pedro Nauck pedronauck
666
130k
[RailsConf 2023 Opening Keynote] The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
31
9.7k
Optimising Largest Contentful Paint
Avatar for Harry Roberts csswizardry
37
3.5k
The Power of CSS Pseudo Elements
Avatar for Geoffrey Crofte geoffreycrofte
80
6k
Being A Developer After 40
Avatar for Adrian Kosmaczewski akosma
91
590k
Navigating Team Friction
Avatar for Lara Hogan lara
190
15k
Principles of Awesome APIs and How to Build Them.
Avatar for Keavy McMinn keavy
127
17k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
Avatar for Tammy Everts tammyeverts
55
3.1k
The Straight Up "How To Draw Better" Workshop
Avatar for Dennis Kardys denniskardys
239
140k
[RailsConf 2023] Rails as a piece of cake
Avatar for Vladimir Dementyev palkan
57
6k
Making the Leap to Tech Lead
Avatar for Ryan Cromwell cromwellryan
135
9.6k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio