Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
0
69

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017
Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
130
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
100
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
240
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
160
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
59
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
400
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
350

Other Decks in Technology

See All in Technology
Android Audio: Beyond Winning On It
Avatar for Atsushi Eno atsushieno
0
880
初めてAWSを使うときのセキュリティ覚書〜初心者支部編〜
Avatar for cm-usuda-keisuke cmusudakeisuke
1
260
AWSを利用する上で知っておきたい名前解決のはなし（10分版）
Avatar for nagisa_53 nagisa53
10
3.2k
株式会社ログラス - 会社説明資料【エンジニア】/ Loglass Engineer
Avatar for 株式会社ログラス loglass2019
4
64k
複数サービスを支える マルチテナント型 Batch MLプラットフォーム
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
1
440
まずはマネコンでちゃちゃっと作ってから、それをCDKにしてみよか。
Avatar for ヤマダ（北野） yamada_r
2
110
AI時代を生き抜くエンジニアキャリアの築き方 (AI-Native 時代、エンジニアという道は 「最大の挑戦の場」となる) / Building an Engineering Career to Thrive in the Age of AI (In the AI-Native Era, the Path of Engineering Becomes the Ultimate Arena of Challenge)
Avatar for JaeSoon Jeong jeongjaesoon
0
180
[ JAWS-UG 東京 CommunityBuilders Night #2 ]SlackとAmazon Q Developerで 運用効率化を模索する
Avatar for sh_fk2 sh_fk2
3
440
実践！カスタムインストラクション＆スラッシュコマンド
Avatar for puku0x puku0x
0
440
これでもう迷わない！Jetpack Composeの書き方実践ガイド
Avatar for ZOZO Developers zozotech
PRO
0
900
Agile PBL at New Grads Trainings
Avatar for Yasunobu Kawaguchi kawaguti
PRO
1
440
新アイテムをどう使っていくか？みんなであーだこーだ言ってみよう / 20250911-rpi-jam-tokyo
Avatar for Akira Ouchi akkiesoft
0
290

Featured

See All Featured
GraphQLとの向き合い方2022年版
Avatar for Yosuke Kurami quramy
49
14k
RailsConf 2023
Avatar for Aaron Patterson tenderlove
30
1.2k
The Pragmatic Product Professional
Avatar for Laura Van Doore lauravandoore
36
6.9k
Build The Right Thing And Hit Your Dates
Avatar for Maggie C. maggiecrowley
37
2.9k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
Avatar for Dan Newman danielanewman
229
22k
Git: the NoSQL Database
Avatar for Brandon Keepers bkeepers
PRO
431
66k
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
56
13k
[RailsConf 2023 Opening Keynote] The Magic of Rails
Avatar for Eileen M. Uchitelle eileencodes
30
9.7k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
Avatar for soudai sone soudai
PRO
188
55k
It's Worth the Effort
Avatar for 3n 3n
187
28k
Writing Fast Ruby
Avatar for Erik Berlin sferik
628
62k
Product Roadmaps are Hard
Avatar for C. Todd Lombardo iamctodd
PRO
54
11k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio