Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Sponsored · Ship Features Fearlessly Turn features on and off without deploys. Used by thousands of Ruby developers. →
Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
73
0

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
140
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
110
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
270
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
180
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
65
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
420
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
380

Other Decks in Technology

See All in Technology
Rapid Start: Faster Internet Connections, with Ruby's Help
Avatar for kazuho kazuho
2
740
Good Enough Types: Heuristic Type Inference for Ruby
Avatar for Shia riseshia
1
270
自分のハンドルは自分で握れ！ ― 自分のケイパビリティを増やし、メンバーのケイパビリティ獲得を支援する ― / Take the wheel yourself
Avatar for TAKAKING22 takaking22
1
950
260420_スマートホーム採用説明 - wakinchan
Avatar for Yusuke Kuroiwa wakinchan
0
120
AgentCore Managed Harness を使ってみよう
Avatar for やくも yakumo
2
140
AI: Making Admin and Users, Lives Better
Avatar for Keith Brooks kbmsg
0
110
Hacobu Tech Deck
Avatar for Hacobu hacobu
PRO
0
120
データを"持てない"環境でのアノテーション基盤設計
Avatar for SansanTech sansantech
PRO
1
140
サイボウズ 開発本部採用ピッチ / Cybozu Engineer Recruit
Avatar for Cybozu cybozuinsideout
PRO
10
78k
巨大プラットフォームを進化させる「第3のROI」
Avatar for Recruit recruitengineers
PRO
2
380
Microsoft 365 / Microsoft 365 Copilot : 自分の状態を確認する「ラベル」について
Avatar for Taichi Nakamura taichinakamura
0
330
AI時代のガードレールとしてのAPIガバナンス
Avatar for 草薙昭彦 nagix
0
300

Featured

See All Featured
Context Engineering - Making Every Token Count
Avatar for Addy Osmani addyosmani
9
840
Writing Fast Ruby
Avatar for Erik Berlin sferik
630
63k
Jamie Indigo - Trashchat’s Guide to Black Boxes: Technical SEO Tactics for LLMs
Avatar for Tech SEO Connect techseoconnect
PRO
0
110
Mobile First: as difficult as doing things right
Avatar for Swwweet swwweet
225
10k
The State of eCommerce SEO: How to Win in Today's Products SERPs - #SEOweek
Avatar for Aleyda Solis aleyda
2
10k
Become a Pro
Avatar for Speaker Deck speakerdeck
PRO
31
5.9k
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
Avatar for Sam Stephenson sstephenson
162
16k
The MySQL Ecosystem @ GitHub 2015
Avatar for Sam Lambert samlambert
251
13k
30 Presentation Tips
Avatar for Ian Lurie portentint
PRO
1
280
RailsConf 2023
Avatar for Aaron Patterson tenderlove
30
1.4k
Game over? The fight for quality and originality in the time of robots
Avatar for Wayne Barker wayneb77
1
160
Fashionably flexible responsive web design (full day workshop)
Avatar for Andy Clarke malarkey
408
66k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio