Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
78
0

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
150
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
130
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
290
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
180
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
68
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
420
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
400

Other Decks in Technology

See All in Technology
1000⼈規模のClaude Enterprise運⽤を「Oktaのグループ」と「Slack」に集約する
Avatar for SansanTech sansantech
PRO
1
520
Digital Credentials API × OpenID4VP ブラウザ完結型本人確認の実装知見（OAuth/OIDC Numa (Immersion) Workshop 2026）
Avatar for OpenID Foundation Japan oidfj
PRO
0
360
AI時代のデータ基盤を考える問い
Avatar for Jun Ernesto Okumura pacocat
0
750
会計事務所と顧問先の契約関係をOIDC・OAuthで表現する
Avatar for てらら terara
0
520
「ミスを許さない手順書」を作ってみた 〜 個人的にはこれ以上できることはあまりなさそう/20260827-ssmjp-operation-procedure-update
Avatar for opelab opelab
15
16k
Cloudflare製品を活用した AIガバナンス実践入門 / AI governance with Cloudflare Service
Avatar for 株式会社DELTA delta_tech
1
130
Kiro Crew で始める マルチエージェント開発
Avatar for Kazuki Miura miu_crescent
PRO
0
210
Introduction to Sansan for Engineers / エンジニア向け会社紹介
Avatar for Sansan, Inc. sansan33
PRO
6
77k
分割40%キーボードにスムーズに入門するには
Avatar for Yuki Ishikawa hoto17296
1
200
1.5時間を無駄にして学んだwsl2におけるaptとsnapの選択と仕組み
Avatar for y-osaka0123 yosaka0123
0
470
EUDIWの枠組みを出発点に民間エコシステムの在り方を考える（OAuth/OIDC Numa (Immersion) Workshop 2026）
Avatar for OpenID Foundation Japan oidfj
PRO
0
370
わたしが知り合いゼロの勉強会に 行けるようになるまで
Avatar for R-524 r5ni4
2
840

Featured

See All Featured
エンジニアに許された特別な時間の終わり
Avatar for watany watany
108
250k
How to audit for AI Accessibility on your Front & Back End
Avatar for davetheseo davetheseo
0
520
Technical Leadership for Architectural Decision Making
Avatar for Kenny Baas-Schwegler baasie
3
540
The B2B funnel & how to create a winning content strategy
Avatar for Katarina Dahlin katarinadahlin
PRO
1
490
Ten Tips & Tricks for a 🌱 transition
Avatar for Manu Carrasco Molina stuffmc
0
180
How to Ace a Technical Interview
Avatar for Jacob Kaplan-Moss jacobian
281
24k
Joys of Absence: A Defence of Solitary Play
Avatar for Sebastian Deterding codingconduct
1
450
Put a Button on it: Removing Barriers to Going Fast.
Avatar for kastner kastner
60
4.5k
Beyond borders and beyond the search box: How to win the global "messy middle" with AI-driven SEO
Avatar for David Carrasco davidcarrasco
3
220
Testing 201, or: Great Expectations
Avatar for Joseph Mastey jmmastey
46
8.3k
Avoiding the “Bad Training, Faster” Trap in the Age of AI
Avatar for Mike Taylor tmiket
0
220
Claude Code どこまでも/ Claude Code Everywhere
Avatar for nwiizo nwiizo
67
57k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio