Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Агрегация и обработка статистики
Search
Yehor Nazarkin
August 12, 2013
Programming
0
5.7k
Агрегация и обработка статистики
Доклад Валерия Змиевского, kharkivpy #8
Yehor Nazarkin
August 12, 2013
Tweet
Share
More Decks by Yehor Nazarkin
See All by Yehor Nazarkin
Flask, гордость и предубеждения
nimnull
0
300
Python non-mainstream web
nimnull
0
130
Write once, run everywhere
nimnull
0
6.2k
GPU Computing in Python
nimnull
1
5.7k
To Flask or not to Flask
nimnull
5
620
MongoDB Без ORM
nimnull
0
280
REST in Flask (And small client-side intro)
nimnull
10
1.7k
Other Decks in Programming
See All in Programming
Amazon Bedrock Agentsを用いてアプリ開発してみた!
har1101
0
290
破壊せよ!データ破壊駆動で考えるドメインモデリング / data-destroy-driven
minodriven
17
4.3k
PLoP 2024: The evolution of the microservice architecture pattern language
cer
PRO
0
2.3k
OpenTelemetryでRailsのパフォーマンス分析を始めてみよう(KoR2024)
ymtdzzz
5
2k
Compose 1.7のTextFieldはPOBox Plusで日本語変換できない
tomoya0x00
0
160
現場で役立つモデリング 超入門
masuda220
PRO
15
3.1k
Nurturing OpenJDK distribution: Eclipse Temurin Success History and plan
ivargrimstad
0
400
色々なIaCツールを実際に触って比較してみる
iriikeita
0
310
Pinia Colada が実現するスマートな非同期処理
naokihaba
4
200
Why Jakarta EE Matters to Spring - and Vice Versa
ivargrimstad
0
570
GCCのプラグインを作る / I Made a GCC Plugin
shouth
1
160
C++でシェーダを書く
fadis
6
3.9k
Featured
See All Featured
ReactJS: Keep Simple. Everything can be a component!
pedronauck
665
120k
How GitHub (no longer) Works
holman
310
140k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
10
700
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
29k
No one is an island. Learnings from fostering a developers community.
thoeni
19
3k
Become a Pro
speakerdeck
PRO
25
5k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.3k
Ruby is Unlike a Banana
tanoku
96
11k
Mobile First: as difficult as doing things right
swwweet
222
8.9k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
169
50k
5 minutes of I Can Smell Your CMS
philhawksworth
202
19k
Making Projects Easy
brettharned
115
5.9k
Transcript
Агрегация и обработка статистики реального времени Валерий Змиевской
Что нам нужно? • База данных • Отказаться от запросов:
group by keyfield … order by sum(datafield) desc • Upsert MongoDB, MySQL - есть нативный синтаксис
Как хранить данные Хранить данные для каждого ключа группировки в
отдельной таблице: user : [last_login, orders, coupons_used, …] category, date : [orders_amount, items_sold] region, date : [orders_amount, items_sold] ….
OLAP-куб на одной таблице Ключи : данные [region, category, year,
month] : [ord, isold] [<all>, 12, 2013, 01] : [100, 665] [<all>, <all>, 2013, 01] : [1600, 5200] ….
Срезы [region, category, year, month] [0, 0, 1, 0], [0,
0, 1, 1], [1, 0, 1, 0], [1, 0, 1, 1] [0, 1, 1, 0], [0, 1, 1, 1], [1, 1, 1, 0], [1, 1, 1, 1] [0, 1, 0, 0], [1, 1, 0, 0] Не нужны срезы: [*, *, 0, 1] и [*, 0, 0, 0]
Показываем: category, графики по месяцам Сортируем: ord, isold Индексы: •
region, category, year, month - unique|Charts • region, year, month, ord - Orders/Category • region, year, month, isold - It. sold/Category Индексы
Запросы: Категории: region=<all>, year=<all>, month=<all> region=EU, year=<all>, month=<all> region=EU, year=2013,
month=<all> График по категории Goods за 2013 год region=EU, year=2013, category=’Goods’