Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

ernestoarbitrio
April 09, 2017
Technology
0
65

Fast In-memory Analytics for Retail Data with Columnar Databases

ernestoarbitrio

April 09, 2017
Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
pamaron
0
110
PyConZA 2022 Best practices for good(ish) and clean(ish) code
pamaron
0
84
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
pamaron
0
210
Bokeh: Using python for interactive data visualization
pamaron
1
150
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
pamaron
0
47
Indexing and search tons of data with ElasticSearch and Django
pamaron
0
340
Interactive plot with django and highchart JS (without JS)
pamaron
0
310

Other Decks in Technology

See All in Technology
AI時代のデータセンターネットワーク
lycorptech_jp
PRO
1
280
終了の危機にあった15年続くWebサービスを全力で存続させる - phpcon2024
yositosi
1
760
TSKaigi 2024 の登壇から広がったコミュニティ活動について
tsukuha
0
160
Amazon Kendra GenAI Index 登場でどう変わる？ 評価から学ぶ最適なRAG構成
naoki_0531
0
100
AIのコンプラは何故しんどい？
shujisado
1
190
権威ドキュメントで振り返る2024 #年忘れセキュリティ2024
hirotomotaguchi
2
740
PHP ユーザのための OpenTelemetry 入門 / phpcon2024-opentelemetry
shin1x1
1
170
大幅アップデートされたRagas v0.2をキャッチアップ
os1ma
2
520
統計データで2024年の クラウド・インフラ動向を眺める
ysknsid25
2
840
Oracle Cloud Infrastructure：2024年12月度サービス・アップデート
oracle4engineer
PRO
0
170
LINEヤフーのフロントエンド組織・体制の紹介【24年12月】
lycorp_recruit_jp
0
530
WACATE2024冬セッション資料（ユーザビリティ）
scarletplover
0
190

Featured

See All Featured
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
44
9.3k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
38
1.9k
Fashionably flexible responsive web design (full day workshop)
malarkey
405
66k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
29k
StorybookのUI Testing Handbookを読んだ
zakiyama
27
5.3k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
48k
The World Runs on Bad Software
bkeepers
PRO
65
11k
Unsuck your backbone
ammeep
669
57k
Designing for Performance
lara
604
68k
The MySQL Ecosystem @ GitHub 2015
samlambert
250
12k
Code Reviewing Like a Champion
maltzj
520
39k
Optimizing for Happiness
mojombo
376
70k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio