Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
0
68

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017
Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
120
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
95
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
230
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
160
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
58
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
390
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
340

Other Decks in Technology

See All in Technology
カンファレンスのつくりかた / The Conference Code: What Makes It All Work
Avatar for HASEGAWA Tomoki tomzoh
8
1k
MCP Clientを活用するための設計と実装上の工夫
Avatar for Yudai Hayashi yudai00
1
900
Amazon DevOps Guru のベースラインを整備して1ヶ月ほど運用してみた #jawsug_asa / Amazon DevOps Guru trial
Avatar for MasahiroKawahara masahirokawahara
3
200
Generational ZGCのメモリ運用改善 - その物理メモリ使用量、本当に正しい？
Avatar for Daishi Tabata tabatad
0
260
DevOpsDays Taipei 2025 - Opening Remarks
Avatar for Cheng-Wei Chen cheng_wei_chen
0
120
おれのAI活用の現状とこれから
Avatar for TsukasaSekiguchi tsukasagr
0
110
AWS Lambdaでサーバレス設計を学ぼう_ベンダーロックインの懸念を超えて-サーバレスの真価を探る
Avatar for ふくちーぬ（Takaya Fukuchi） fukuchiiinu
4
430
Zero Data Loss Autonomous Recovery Service サービス概要
Avatar for oracle4engineer oracle4engineer
PRO
2
7.3k
Java 30周年記念! Javaの30年をふりかえる
Avatar for Yuichi.Sakuraba skrb
4
2.6k
LT:組込み屋さんのオシロが壊れた！
Avatar for windy windy_pon
0
580
組織とセキュリティ文化と、自分の一歩
Avatar for mai miya maimyyym
3
1.4k
Data Observability：企業資料管理技術的未來顯學
Avatar for Cheng-Wei Chen cheng_wei_chen
0
290

Featured

See All Featured
Making Projects Easy
Avatar for Brett Harned brettharned
116
6.2k
The MySQL Ecosystem @ GitHub 2015
Avatar for Sam Lambert samlambert
251
13k
Fontdeck: Realign not Redesign
Avatar for Paul Robert Lloyd paulrobertlloyd
84
5.5k
Rebuilding a faster, lazier Slack
Avatar for samanthasiow samanthasiow
81
9k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
Avatar for Tammy Everts tammyeverts
52
2.8k
The Straight Up "How To Draw Better" Workshop
Avatar for Dennis Kardys denniskardys
233
140k
KATA
Avatar for Megan Lloyd mclloyd
29
14k
GraphQLの誤解/rethinking-graphql
Avatar for sonatard sonatard
71
11k
Faster Mobile Websites
Avatar for Dean Hume deanohume
307
31k
[RailsConf 2023 Opening Keynote] The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
29
9.5k
Put a Button on it: Removing Barriers to Going Fast.
Avatar for kastner kastner
60
3.9k
Building Flexible Design Systems
Avatar for Yesenia Perez-Cruz yeseniaperezcruz
329
39k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | vmaggio@fbk.eu Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio