Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
73
0

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
140
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
120
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
270
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
180
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
65
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
420
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
380

Other Decks in Technology

See All in Technology
2026-05-14 要件定義からソース管理まで！IBM Bob基礎ハンズオン
Avatar for YutaNonaka yutanonaka
0
160
なぜ、私がCommunity Builderに?〜活動期間1か月半でも選出されたワケ〜
Avatar for Yuuki Yamashita yama3133
0
130
Terragrunt x Snowflake + dbt で作るマルチテナントなデータ基盤構築プラットフォーム
Avatar for Gaku TASHIRO gak_t12
0
170
Vision Banana: Image Generators are Generalist Vision Learners
Avatar for Kazuyuki Miyazawa kzykmyzw
0
390
Claude Code / Codex / Kiro に AWS 権限を 渡すとき、何を設計すべきか
Avatar for Kazuki Adachi k_adachi_01
5
1.5k
Claude Code で使える DuckDB Skills を試してみた / DuckDB Skills and Claude Code
Avatar for MasahiroKawahara masahirokawahara
1
360
マンション備え付けのネットワークとLTE回線を組み合わせた ネットワークの安定化の考案
Avatar for harutiro harutiro
1
130
"スキルファースト"で作る、AIの自走環境
Avatar for subroh_0508 subroh0508
0
510
サンプリングは「作る」のか「使う」のか？ 分散トレースのコストと運用を両立する実践的戦略 / Why you need the tail sampling and why you don't want it
Avatar for ymotongpoo ymotongpoo
4
180
20260516_SecJAWS_Days
Avatar for Takuya Yonezawa takuyay0ne
2
430
Oracle Base Database Service 技術詳細
Avatar for oracle4engineer oracle4engineer
PRO
15
100k
ServiceによるKubernetes通信制御ーClusterIPを例に
Avatar for 周学勤（Zhou Xueqin） miku01
1
170

Featured

See All Featured
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
49
9.9k
Public Speaking Without Barfing On Your Shoes - THAT 2023
Avatar for David Neal reverentgeek
1
390
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
Avatar for Rebecca Miller-Webster rmw
35
3.5k
Designing Experiences People Love
Avatar for Jonathan Moore moore
143
24k
Groundhog Day: Seeking Process in Gaming for Health
Avatar for Sebastian Deterding codingconduct
0
180
Design of three-dimensional binary manipulators for pick-and-place task avoiding obstacles (IECON2024)
Avatar for konakalab konakalab
0
420
Building Adaptive Systems
Avatar for Chris Keathley keathley
44
3k
Building Experiences: Design Systems, User Experience, and Full Site Editing
Avatar for Michelle Schulp Hunt marktimemedia
0
500
Ten Tips & Tricks for a 🌱 transition
Avatar for Manu Carrasco Molina stuffmc
0
110
Ethics towards AI in product and experience design
Avatar for Skipper Chong Warson skipperchong
2
270
Data-driven link building: lessons from a $708K investment (BrightonSEO talk)
Avatar for Szymon Słowik szymonslowik
1
1.1k
What’s in a name? Adding method to the madness
Avatar for The Alliance productmarketing
PRO
24
4k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio