Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
0
69

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017
Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
130
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
100
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
240
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
160
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
59
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
400
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
350

Other Decks in Technology

See All in Technology
AWSでAgentic AIを開発するための前提知識の整理
Avatar for Kiminori Yokoi nasuvitz
2
170
AWS Control Tower に学ぶ！ IAM Identity Center 権限設計の第一歩 / IAM Identity Center with Control Tower
Avatar for Yuta Otani y___u
0
170
プロダクトのコードから見るGoによるデザインパターンの実践 #go_night_talk
Avatar for 弁護士ドットコム bengo4com
1
2.6k
ガバメントクラウド（AWS）へのデータ移行戦略の立て方【虎の巻】 / 20251011 Mitsutosi Matsuo
Avatar for SHIFT EVOLVE shift_evolve
PRO
2
200
カンファレンスに託児サポートがあるということ / Having Childcare Support at Conferences
Avatar for nobu09 nobu09
1
590
アイテムレビュー機能導入からの学びと改善
Avatar for ZOZO Developers zozotech
PRO
0
170
WEBサービスを成り立たせるAWSサービス
Avatar for Takano takano0131
1
170
物体検出モデルでシイタケの収穫時期を自動判定してみた。 #devio2025
Avatar for Lamaglama39 lamaglama39
0
160
業務効率化をさらに加速させる、ノーコードツールとStep Functionsのハイブリッド化
Avatar for Makky12 smt7174
2
140
プレーリーカードを活用しよう❗❗デジタル名刺交換からはじまるイベント会場交流のススメ
Avatar for Masataka Tsukamoto tsukaman
0
170
リセラー企業のテクサポ担当が考える、生成 AI 時代のトラブルシュート 2025
Avatar for kazzpapa3 kazzpapa3
1
350
ニッポンの人に知ってもらいたいGISスポット
Avatar for sakaik sakaik
0
150

Featured

See All Featured
Being A Developer After 40
Avatar for Adrian Kosmaczewski akosma
91
590k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
Avatar for Joe Casabona jcasabona
33
2.5k
Optimising Largest Contentful Paint
Avatar for Harry Roberts csswizardry
37
3.5k
GitHub's CSS Performance
Avatar for Jon Rohan jonrohan
1032
470k
GraphQLとの向き合い方2022年版
Avatar for Yosuke Kurami quramy
49
14k
A better future with KSS
Avatar for Kyle Neath kneath
239
18k
The Invisible Side of Design
Avatar for Vitaly Friedman smashingmag
302
51k
Testing 201, or: Great Expectations
Avatar for Joseph Mastey jmmastey
45
7.7k
Practical Orchestrator
Avatar for Shlomi Noach shlominoach
190
11k
Imperfection Machines: The Place of Print at Facebook
Avatar for Scott Boms scottboms
269
13k
KATA
Avatar for Megan Lloyd mclloyd
32
15k
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
49
51k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio