Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Probablistic Data Structures
Search
Sergey Arkhipov
November 11, 2017
Programming
300
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Probablistic Data Structures
My talk on rannts #18 (11.11.2017)
Sergey Arkhipov
November 11, 2017
Transcript
42 slides
Вероятностные структуры данных Сергей Архипов, 2017
None
None
curl http://site.com
curl -x myproxy.ru:3128 http://site.com
curl -x proxy.crawlera.com:8010 http://site.com
None
evt evt evt evt evt
{ "user": "sarkhipov", "hostname": "rannts.ru", "status": "ok", "status_description": "" }
{ "user": "sarkhipov", "hostname": "rannts.ru", "status": "ok", "status_description": "" }
None
collector collector collector
{ "user": "sarkhipov", "hostname": "rannts.ru", "status": "ok", "status_description": "" }
{ "user": "sarkhipov", "hostname": "rannts.ru", "status": "ok", "status_description": "" } { "user": "sarkhipov", "hostname": "rannts.ru", "status": "ok", "status_description": "" } { "user": "sarkhipov", "hostname": "rannts.ru", "ok": 234, "banned": 12, "errors": 3, }
Consumer 1 { "user": "sarkhipov", "hostname": "rannts.ru", "ok": 234, "banned":
12, "errors": 3, } Consumer 2 { "user": "sarkhipov", "hostname": "rannts.ru", "ok": 250, "banned": 3, "errors": 0, } Consumer 3 { "user": "sarkhipov", "hostname": "rannts.ru", "ok": 0, "banned": 124, "errors": 84, }
INSERT INTO stats ( date, user, hostname, ok, ban, error
) VALUES ( :date, :user, :hostname, :ok, :ban, :error ) ON DUPLICATE KEY UPDATE ok = ok + VALUES(ok), ban = ban + VALUES(ban), error = error + VALUES(error);
{ "user": "sarkhipov", "hostname": "rannts.ru", "status": "ok", "status_description": "", "response_time":
2861, }
(20 + 10) + 11 = (20 + 11) +
10
F(x)=P{σ<x} { P(x⩽x α )⩾α P(x⩾x α )⩾1−α
Ω(N 1 p )
collector collector collector pworker pworker pworker
None
None
var memCount = 75604275; var memPerSec = 1.38176367782; function updateCount()
{ next = -(1000 / memPerSec) * Math.log(Math.random()); memCountString = ''+memCount; len = memCountString.length; memCountString = memCountString.substr(0, len - 6) + ’ < span style = ”font - size: 8 px” > < /span>’+memCountString.substr(len-6,3)+‘ < span style = ”font - size: 8 px” > < /span>’+memCountString.substr(len-3,3); ge(‘memCount’).innerHTML = memCountString; memCount = memCount + 1; setTimeout(updateCount, next); } addEvent(window, ‘load’, updateCount);
3500 3671 3400 3502 3463 3371 3607 6012 6168 6211
6017
3500 3507 3671 3667 3400 3410 3502 3502 3463 3466
3371 3330 3607 3599 6012 6009 6168 6152 6211 6215 6017 6016
Count-Min Sketch 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Count-Min Sketch 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Count-Min Sketch 0 0 1 0 0 0 0 0
1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 1
Count-Min Sketch 32 11 1 18 200 126 184 78
1 0 91 59 30 24 8 82 76 34 48 72 11 200 129 136 14
Count-Min Sketch 32 11 1 18 200 126 184 78
1 0 91 59 30 24 8 82 76 34 48 72 11 200 129 136 14
MinHash J (A , B)= |A∩B| |A∪B| k=[ 1 ε2
]
HyperLogLog 010010000110010101101100011011000110111100100001 b 26 = 64 1001 b = 9
100001 b = 33 σ= 1.04 √2k E= α(k)4k ∑ j 2−M j
t-digest
t-digest
t-digest
t-digest X=x 1 , x 2 ,…, x n X={s
1 ,s 2 ,…,s m } s i ={x l e f t(i) ,…, x r i ght(i) }
t-digest k(q,δ)≝δ (sin−1 (2q−1) π + 1 2 ) K(i)≝k(
r i ght(i) n ,δ)−k( le f t(i)−1 n ,δ) K (i)⩽1 K(i)+K (i+1)>1
t-digest
t-digest
collector collector collector pworker pworker pworker
None
Q/A
More Decks by Sergey Arkhipov
See All by Sergey Arkhipov
Fingerprinting
9seconds
0
210
Concurrency Models
9seconds
0
320
Own Mustache
9seconds
0
380
Daemonize
9seconds
0
380
Stuff That Works
9seconds
0
410
Evidence
9seconds
0
140
Redneck Monads
9seconds
1
150
Latency
9seconds
0
170
Oh Blindfold Russia!
9seconds
0
340
Other Decks in Programming
See All in Programming
Growing PicoRuby's Ecosystem on ESP32
yuuu
2
150
コード生成時代のAPI定義を支えるRequest Spec移行
pvcresin
0
300
SONY CISC-NEWS NWS-1750 + NWB-225 ハードウェアメンテとフレームバッファカラー対応 / KOF2026
tsutsui
0
130
エージェントの管理もエージェントに – herdr-core と自作バックログで作る司令塔 –
nearme_tech
PRO
1
110
React Native経験者が Flutterを30⽇間触って驚いたこと
3saito
0
210
はじめてのKubernetesコントリビューション
nasa9084
0
320
flutter_test のその先へ — Flutter アプリの LLM 機能に「評価」という品質保証を
kosukesaigusa
0
210
AI時代のシニアエンジニアとは
yosuke_furukawa
PRO
15
12k
Debian JP Projectのサーバー構成
uwabami
1
180
AI駆動TDD ― 2026年10月版
hrmstrsmgs
0
110
Getting Started with Vibe Coding
miccheng
0
250
将来の標準フォーマッター?ktfmtを深堀る
t45k
0
250
Featured
See All Featured
Git: the NoSQL Database
bkeepers
PRO
433
67k
The SEO Collaboration Effect
kristinabergwall1
1
630
We Are The Robots
honzajavorek
0
460
Building a Scalable Design System with Sketch
lauravandoore
464
34k
What Being in a Rock Band Can Teach Us About Real World SEO
427marketing
0
1.1k
エンジニアに許された特別な時間の終わり
watany
109
250k
Prompt Engineering for Job Search
mfonobong
0
610
Build The Right Thing And Hit Your Dates
maggiecrowley
39
3.5k
AI: The stuff that nobody shows you
jnunemaker
PRO
12
1.3k
A designer walks into a library…
pauljervisheath
211
25k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
659
62k
Test your architecture with Archunit
thirion
2
2.5k