Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

Avatar for ernestoarbitrio ernestoarbitrio
April 09, 2017
Technology
0
69

Fast In-memory Analytics for Retail Data with Columnar Databases

Avatar for ernestoarbitrio

ernestoarbitrio

April 09, 2017
Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
Avatar for ernestoarbitrio pamaron
0
130
PyConZA 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
100
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
Avatar for ernestoarbitrio pamaron
0
240
Bokeh: Using python for interactive data visualization
Avatar for ernestoarbitrio pamaron
1
160
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
Avatar for ernestoarbitrio pamaron
0
59
Indexing and search tons of data with ElasticSearch and Django
Avatar for ernestoarbitrio pamaron
0
400
Interactive plot with django and highchart JS (without JS)
Avatar for ernestoarbitrio pamaron
0
350

Other Decks in Technology

See All in Technology
フルカイテン株式会社 エンジニア向け採用資料
Avatar for FULL KAITEN fullkaiten
0
9.1k
o11yで育てる、強い内製開発組織
Avatar for _awache _awache
3
130
そのWAFのブロック、どう活かす？ サービスを守るための実践的多層防御と思考法 / WAF blocks defense decision
Avatar for 株式会社カミナシ kaminashi
0
110
Geospatialの世界最前線を探る [2025年版]
Avatar for Yasunori Kirimoto dayjournal
0
150
『OCI で学ぶクラウドネイティブ 実践 × 理論ガイド』 書籍概要
Avatar for oracle4engineer oracle4engineer
PRO
2
140
Optuna DashboardにおけるPLaMo2連携機能の紹介 / PFN LLM セミナー
Avatar for Preferred Networks pfn
PRO
2
920
ガバメントクラウドの概要と自治体事例（名古屋市）
Avatar for 高橋広和 techniczna
2
200
Modern_Data_Stack最新動向クイズ_買収_AI_激動の2025年_.pdf
Avatar for Sagara sagara
0
230
AI時代こそ求められる設計力- AWSクラウドデザインパターン3選で信頼性と拡張性を高める-
Avatar for Ken'ichirou Kimura kenichirokimura
3
160
自動テストのコストと向き合ってみた
Avatar for SmartHR 品質保証部 qa
0
200
Trust as Infrastructure
Avatar for Bryan Cantrill bcantrill
1
360
データエンジニアがこの先生きのこるには...?
Avatar for 10xinc 10xinc
0
460

Featured

See All Featured
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
Avatar for Marc Duiker marcduiker
32
2.3k
Learning to Love Humans: Emotional Interface Design
Avatar for Aarron Walter aarron
274
41k
CSS Pre-Processors: Stylus, Less & Sass
Avatar for Bermon Painter bermonpainter
358
30k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
Avatar for Kevin Liebholz kevinliebholz
34
6.1k
A Modern Web Designer's Workflow
Avatar for Chris Coyier chriscoyier
697
190k
How STYLIGHT went responsive
Avatar for nonsquared nonsquared
100
5.8k
Put a Button on it: Removing Barriers to Going Fast.
Avatar for kastner kastner
60
4k
A better future with KSS
Avatar for Kyle Neath kneath
239
18k
Automating Front-end Workflow
Avatar for Addy Osmani addyosmani
1371
200k
10 Git Anti Patterns You Should be Aware of
Avatar for Lemi Orhan Ergin lemiorhan
PRO
657
61k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
Avatar for Irina Nazarova irinanazarova
9
970
Being A Developer After 40
Avatar for Adrian Kosmaczewski akosma
91
590k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio