Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

Fast In-memory Analytics for Retail Data with C...

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
78
0

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
150
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
130
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
300
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
180
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
69
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
430
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
400

Other Decks in Technology

See All in Technology
Gitは怖い？共有ワークスペースから始めるSnowflakeチーム開発
Avatar for あべ coco_se
0
180
2026-09-09 【sigma_ucj#1】Sigma を IaC 管理したい！ / IaC for Sigma
Avatar for civitaspo civitaspo
0
120
新機種発売前に見直そう！端末移行で再ログインが要るアプリ・要らないアプリは何が違うのか 〜シームレスに再開できる設計と実装〜
Avatar for ZOZO Developers zozotech
PRO
0
120
データ界隈LT祭 第1回LT登壇
Avatar for タロウ taromatsui_cccmkhd
0
230
synctest時代のhttptest Go 1.27で変わるHTTPサーバテストの裏側 / go conference2026 synctest and httptest
Avatar for Yoichiro Shimizu budougumi0617
1
2.7k
Sigmaユーザーのための有用リソース一挙公開 & Sigmaで使えるMCP #sigma_ucj /useful-resources-for-sigma-computing-users-and-mcps-with-sigma
Avatar for shinyaa31 shinyaa31
0
200
2026_devsumi_ozono.pdf
Avatar for O3(ozono) o3
3
420
Claude in Chrome 入門 / Introduction to Claude in Chrome
Avatar for yuta cielo1985
0
560
ペアプロの価値はコードを書くことだけじゃない
Avatar for コドモン開発チーム codmoninc
PRO
0
200
AI時代のAPI品質を支えるガードレール / API Guardrails for API quality in the AI ​​era
Avatar for Yoichi Kawasaki yokawasa
1
220
Microsoft 365 Copilot chat -tekoälypalvelun tietosuojaongelmat
Avatar for Harto Pönkä hponka
0
740
AI時代、データエンジニアが一番おもろい
Avatar for ゲンシュン genshun9
0
250

Featured

See All Featured
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
49
10k
jQuery: Nuts, Bolts and Bling
Avatar for Doug Neiner dougneiner
66
8.6k
The untapped power of vector embeddings
Avatar for Frank van Dijk frankvandijk
2
1.9k
The Invisible Side of Design
Avatar for Vitaly Friedman smashingmag
301
52k
Impact Scores and Hybrid Strategies: The future of link building
Avatar for Tamara Novitovic tamaranovitovic
0
440
How to Talk to Developers About Accessibility
Avatar for Jason CranfordTeague jct
2
540
It's Worth the Effort
Avatar for 3n 3n
188
29k
Context Engineering - Making Every Token Count
Avatar for Addy Osmani addyosmani
9
1.1k
GitHub's CSS Performance
Avatar for Jon Rohan jonrohan
1033
470k
Odyssey Design
Avatar for Ryan Kendrick rkendrick25
PRO
2
800
Unlocking the hidden potential of vector embeddings in international SEO
Avatar for Frank van Dijk frankvandijk
0
930
How STYLIGHT went responsive
Avatar for nonsquared nonsquared
100
6.3k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio