Fast In-memory Analytics for Retail Data with Columnar Databases

ernestoarbitrio

April 09, 2017

58

Fast In-memory Analytics for Retail Data with Columnar Databases

ernestoarbitrio

April 09, 2017

Tweet

More Decks by ernestoarbitrio

See All by ernestoarbitrio

Enable effective Observability with Python

0

69

PyConZA 2022 Best practices for good(ish) and clean(ish) code

0

83

PyCon Italia 2022 Best practices for good(ish) and clean(ish) code

0

180

Bokeh: Using python for interactive data visualization

1

140

Keystroke Behavioural Analysis For Fraud Detection: Deep Learning as-a-service Infrastructure

0

42

Indexing and search tons of data with ElasticSearch and Django

0

280

Interactive plot with django and highchart JS (without JS)

0

290

Other Decks in Technology

See All in Technology

Java EE/Jakarta EEの現状と将来―クラウドネイティブ時代にJava EEは対応できるのか？―

1

150

サーバー間 GraphQL と webmock-graphql の話 / server-to-server graphql and webmock-graphql

2

180

Databricks における『MLOps』

databricksjapan

2

170

自己改善からチームを動かす！「セルフエンジニアリングマネージャー」のすゝめ

6

420

Kernel MemoryでAzure OpenAI Serviceとお手軽データソース連携

1

240

レガシーをぶっ壊せ。AEONで始めるDevRelの話 / Qiita Night 2024-2-22

3

1.3k

Gitlab本から学んだこと - そーだいなるプレイバック / gitlab-book

3

130

長期間TiDBを使ってきた話 @ 私たちはなぜNewSQLを使うのかTiDB選定5社が語る選定理由と活用LT / Experiences with TiDB Over Time

2

890

エンジニア候補者向け資料2024.04.24.pdf

0

3.3k

長期運用プロジェクトでのMySQLからTiDB移行の検証

2

850

Hands-on Gemini, the Google DeepMind LLM

1

110

開発生産性向上サービスを作るFindyが自分たちで開発生産性を爆上げした組織づくりの歩み / Findy's path to boosting its own development productivity 2024-04-17

3

650

Featured

See All Featured

Testing 201, or: Great Expectations

28

6.3k

Large-scale JavaScript Application Architecture

504

110k

YesSQL, Process and Tooling at Scale

164

13k

Web Components: a chance to create the future

305

41k

Pencils Down: Stop Designing & Start Developing

117

11k

Bash Introduction

604

210k

Designing the Hi-DPI Web

276

33k

Helping Users Find Their Own Way: Creating Modern Search Experiences

20

1.9k

The MySQL Ecosystem @ GitHub 2015

243

12k

Infographics Made Easy

238

18k

For a Future-Friendly Web

172

9k

Learning to Love Humans: Emotional Interface Design

267

39k

Transcript

Fast In-memory Analytics for Retail Data with Columnar Databases Ernesto
Arbitrio - Valerio Maggio  arbitrio | [email protected] Florence April 6, 2017
Retail Data • Overview of data we have • granularity
• refresh/update rate • Quantity and storage required (space) • services developed around these data
“Materialized Views” • Description of what they are (non-technical) •
Some examples of Analytics we do on this data
The Problem! ~1 TByte Data We need OLAP Performance:  75M
rows -> 5hours
The Solution! Use a Column-oriented Database  (i.e. Just swap Rows
with Columns) Chuck Norris Test   Passed!
None
Query
None
Thank you get in touch @__pamaron__ @leriomaggio