Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
0
69

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017
Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
130
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
96
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
230
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
160
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
58
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
400
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
340

Other Decks in Technology

See All in Technology
生成AI時代 文字コードを学ぶ意義を見出せるか？
Avatar for hiroshi ueda hrsued
1
730
Tech-Verse 2025 Keynote
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
0
1.3k
CI/CD/IaC 久々に0から環境を作ったらこうなりました
Avatar for Kaz Watanabe kaz29
1
200
整頓のジレンマとの戦い〜Tidy First?で振り返る事業とキャリアの歩み〜/Fighting the tidiness dilemma〜Business and Career Milestones Reflected on in Tidy First?〜
Avatar for 株式会社ビットキー / Bitkey Inc. bitkey
0
220
さくらのIaaS基盤のモニタリングとOpenTelemetry/OSC Hokkaido 2025
Avatar for FUJIWARA Shunichiro fujiwara3
2
230
AI専用のリンターを作る #yumemi_patch
Avatar for 弁護士ドットコム bengo4com
3
1.8k
高速なプロダクト開発を実現、創業期から掲げるエンタープライズアーキテクチャ
Avatar for かわうそ kawauso
1
160
作曲家がボカロを使うようにPdMはAIを使え
Avatar for Takuya Ito itotaxi
0
380
OPENLOGI Company Profile for engineer
Avatar for OPENLOGI hr01
1
33k
Tech-Verse 2025 Global CTO Session
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
0
1.1k
【５分でわかる】セーフィー エンジニア向け会社紹介
Avatar for Safie safie_recruit
0
26k
KubeCon + CloudNativeCon Japan 2025 Recap by CA
Avatar for YuyaKoda ponkio_o
PRO
0
240

Featured

See All Featured
Making Projects Easy
Avatar for Brett Harned brettharned
116
6.3k
Helping Users Find Their Own Way: Creating Modern Search Experiences
Avatar for Dan Newman danielanewman
29
2.7k
Java REST API Framework Comparison - PWX 2021
Avatar for Matt Raible mraible
31
8.7k
Intergalactic Javascript Robots from Outer Space
Avatar for Vicent Martí tanoku
271
27k
The MySQL Ecosystem @ GitHub 2015
Avatar for Sam Lambert samlambert
251
13k
Product Roadmaps are Hard
Avatar for C. Todd Lombardo iamctodd
PRO
54
11k
Optimising Largest Contentful Paint
Avatar for Harry Roberts csswizardry
37
3.3k
Designing for humans not robots
Avatar for Tammie Lister tammielis
253
25k
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
48
50k
Designing Dashboards & Data Visualisations in Web Apps
Avatar for Des Traynor destraynor
231
53k
Producing Creativity
Avatar for Steve Smith orderedlist
PRO
346
40k
GraphQLとの向き合い方2022年版
Avatar for Yosuke Kurami quramy
49
14k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio