Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
77
0

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
140
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
120
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
280
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
180
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
66
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
420
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
400

Other Decks in Technology

See All in Technology
ゴールデンパスは敷いただけでは道にならない ─ 企画部門のエンジニアが技術標準を事業価値に変えるまで
Avatar for みずほリサーチ&テクノロジーズ株式会社 先端技術研究部 mhrtech
1
170
凡エンジニアがこの先生きのこるためには。〜TypeScript完全に理解したい〜
Avatar for Ryotaro Harada alchemy1115
2
260
Amplify Gen2でbackend.tsにCDKを定義する/しない事によるCDKの挙動の違いとユースケース
Avatar for Makky12 smt7174
1
270
インフラと開発の垣根を超えていき！〜元AWSインフラエンジニアがAWS開発で奮闘している話〜
Avatar for da-hatakeyama hatahata021
3
250
地域 SRE コミュニティ最前線 / SRE NEXT 2026 Discussion Night Track C
Avatar for Masahiro Yoshizawa muziyoshiz
0
220
AI Agent SaaS を支える自社仮想化基盤への挑戦と実運用 / ai-agent-saas-virtualization
Avatar for GMO Flatt Security flatt_security
2
4k
Data + AI Summit 2026 イベントレポート： 「AIがビジネスで意思決定するデータ基盤」へ
Avatar for KJ(Kenji Matsuda) nek0128
0
170
世界、断片、モデル。そして理解
Avatar for SAKO Hiroshi ardbeg1958
1
120
Network Firewallやっていき！
Avatar for 荒木 ARK news_it_enj
0
120
インフラ寄りSREでも 開発に踏み出せる〜境界を越えてユーザー体験に向き合いたい〜
Avatar for SansanTech sansantech
PRO
2
3.9k
生成AI×AWS CDK×AWS FISで"振り返れる"ミニGameDayをつくろう
Avatar for yoshimi0227 yoshimi0227
1
160
知らん間に、回ってる
Avatar for ming ming_ayami
0
620

Featured

See All Featured
Writing Fast Ruby
Avatar for Erik Berlin sferik
630
63k
Pawsitive SEO: Lessons from My Dog (and Many Mistakes) on Thriving as a Consultant in the Age of AI
Avatar for David Carrasco davidcarrasco
0
190
The SEO identity crisis: Don't let AI make you average
Avatar for Varn varn
0
510
The browser strikes back
Avatar for Jono Alderson jonoalderson
0
1.4k
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
66
8.5k
エンジニアに許された特別な時間の終わり
Avatar for watany watany
107
250k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
Avatar for Morgane Peng morganepeng
133
19k
DBのスキルで生き残る技術 - AI時代におけるテーブル設計の勘所
Avatar for soudai sone soudai
PRO
67
56k
Introduction to Domain-Driven Design and Collaborative software design
Avatar for Kenny Baas-Schwegler baasie
1
900
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
49
10k
A Soul's Torment
Avatar for Nat Ma seathinner
6
3.1k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
Avatar for Irina Nazarova irinanazarova
9
1.4k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio