Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fast In-memory Analytics for Retail Data with C...

ernestoarbitrio
April 09, 2017
Technology
0
65

Fast In-memory Analytics for Retail Data with Columnar Databases

ernestoarbitrio

April 09, 2017
Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio
Enable effective Observability with Python
pamaron
0
110
PyConZA 2022 Best practices for good(ish) and clean(ish) code
pamaron
0
90
PyCon Italia 2022 Best practices for good(ish) and clean(ish) code
pamaron
0
220
Bokeh: Using python for interactive data visualization
pamaron
1
150
Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure
pamaron
0
52
Indexing and search tons of data with ElasticSearch and Django
pamaron
0
360
Interactive plot with django and highchart JS (without JS)
pamaron
0
320

Other Decks in Technology

See All in Technology
The Future of SEO: The Impact of AI on Search
badams
0
200
なぜ私は自分が使わないサービスを作るのか？ / Why would I create a service that I would not use?
aiandrox
0
750
現場の種を事業の芽にする - エンジニア主導のイノベーションを事業戦略に装着する方法 -
kzkmaeda
2
2.1k
PHPカンファレンス名古屋-テックリードの経験から学んだ設計の教訓
hayatokudou
2
330
君も受託系GISエンジニアにならないか
sudataka
2
440
Building Products in the LLM Era
ymatsuwitter
10
5.5k
Classmethod AI Talks(CATs) #17 司会進行スライド(2025.02.19) / classmethod-ai-talks-aka-cats_moderator-slides_vol17_2025-02-19
shinyaa31
0
120
速くて安いWebサイトを作る
nishiharatsubasa
10
13k
OpenID BizDay#17 KYC WG活動報告（法人） / 20250219-BizDay17-KYC-legalidentity
oidfj
0
250
データ資産をシームレスに伝達するためのイベント駆動型アーキテクチャ
kakehashi
PRO
2
550
明日からできる！技術的負債の返済を加速するための実践ガイド～『ホットペッパービューティー』の事例をもとに～
recruitengineers
PRO
3
410
Amazon S3 Tablesと外部分析基盤連携について / Amazon S3 Tables and External Data Analytics Platform
nttcom
0
140

Featured

See All Featured
Building Your Own Lightsaber
phodgson
104
6.2k
The Invisible Side of Design
smashingmag
299
50k
Building Applications with DynamoDB
mza
93
6.2k
[RailsConf 2023] Rails as a piece of cake
palkan
53
5.2k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
33
2.8k
Practical Orchestrator
shlominoach
186
10k
GitHub's CSS Performance
jonrohan
1030
460k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
226
22k
YesSQL, Process and Tooling at Scale
rocio
172
14k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
21
2.5k
Java REST API Framework Comparison - PWX 2021
mraible
28
8.4k
Making the Leap to Tech Lead
cromwellryan
133
9.1k

Transcript

  1. Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto

    Arbitrio - Valerio Maggio
 arbitrio | [email protected] Florence April 6, 2017

  2. Retail Data • Overview of data we have • granularity

    • refresh/update rate • Quantity and storage required (space) • services developed around these data

  3. “Materialized Views” • Description of what they are (non-technical) •

    Some examples of Analytics we do on this data

  4. The Problem! ~1 TByte Data We need OLAP Performance:
 75M

    rows -> 5hours

  5. The Solution! Use a Column-oriented Database
 (i.e. Just swap Rows

    with Columns) Chuck Norris Test 
 Passed!
  6. None

  7. Query

  8. None

  9. Thank you get in touch @__pamaron__ @leriomaggio