Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Probablistic Data Structures
Search
Sergey Arkhipov
November 11, 2017
Programming
0
200
Probablistic Data Structures
My talk on rannts #18 (11.11.2017)
Sergey Arkhipov
November 11, 2017
Tweet
Share
More Decks by Sergey Arkhipov
See All by Sergey Arkhipov
Fingerprinting
9seconds
0
140
Concurrency Models
9seconds
0
140
Own Mustache
9seconds
0
260
Daemonize
9seconds
0
240
Stuff That Works
9seconds
0
230
Evidence
9seconds
0
71
Redneck Monads
9seconds
1
78
Latency
9seconds
0
85
Oh Blindfold Russia!
9seconds
0
220
Other Decks in Programming
See All in Programming
Findy - エンジニア向け会社紹介 / Findy Letter for Engineers
findyinc
2
81k
Javaの現状2024夏 / Java current status 2024 summer
kishida
4
1.4k
CSC307 Lecture 11
javiergs
PRO
0
240
Clean Architecture by TypeScript & NestJS
ryounasso
0
150
3 Effective Rules for Success with Signals in Angular
manfredsteyer
PRO
0
120
なぜ宣言的 UI は壊れにくいのか / Why declarative UI is less fragile
uenitty
29
13k
Harnessing Large Language Models for Training-free Video Anomaly Detection
tereka114
1
1.3k
CSC307 Lecture 14
javiergs
PRO
0
220
最近追加した型の紹介とその振り返り
aki19035vc
0
170
わかりやすい正解を捨てて、コトに向き合う - スクラムフェス金沢2024 スポンサーセッション
yusukekokubo
0
170
Google's Recipe for Scaling (Web) Security – LocoMocoSec 2024
lweichselbaum
0
170
さきがけから振り返るアーキテクチャ刷新 / Reflecting on the Architectural Renewal from the Vanguard
nrslib
2
770
Featured
See All Featured
Learning to Love Humans: Emotional Interface Design
aarron
269
39k
Bootstrapping a Software Product
garrettdimon
PRO
304
110k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
228
16k
How to train your dragon (web standard)
notwaldorf
79
5.5k
Git: the NoSQL Database
bkeepers
PRO
423
64k
Raft: Consensus for Rubyists
vanstee
134
6.5k
Speed Design
sergeychernyshev
9
270
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
78
15k
The World Runs on Bad Software
bkeepers
PRO
63
11k
YesSQL, Process and Tooling at Scale
rocio
166
14k
StorybookのUI Testing Handbookを読んだ
zakiyama
15
4.9k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
24
1.8k
Transcript
Вероятностные структуры данных Сергей Архипов, 2017
None
None
curl http://site.com
curl -x myproxy.ru:3128 http://site.com
curl -x proxy.crawlera.com:8010 http://site.com
None
evt evt evt evt evt
{ "user": "sarkhipov", "hostname": "rannts.ru", "status": "ok", "status_description": "" }
{ "user": "sarkhipov", "hostname": "rannts.ru", "status": "ok", "status_description": "" }
None
collector collector collector
{ "user": "sarkhipov", "hostname": "rannts.ru", "status": "ok", "status_description": "" }
{ "user": "sarkhipov", "hostname": "rannts.ru", "status": "ok", "status_description": "" } { "user": "sarkhipov", "hostname": "rannts.ru", "status": "ok", "status_description": "" } { "user": "sarkhipov", "hostname": "rannts.ru", "ok": 234, "banned": 12, "errors": 3, }
Consumer 1 { "user": "sarkhipov", "hostname": "rannts.ru", "ok": 234, "banned":
12, "errors": 3, } Consumer 2 { "user": "sarkhipov", "hostname": "rannts.ru", "ok": 250, "banned": 3, "errors": 0, } Consumer 3 { "user": "sarkhipov", "hostname": "rannts.ru", "ok": 0, "banned": 124, "errors": 84, }
INSERT INTO stats ( date, user, hostname, ok, ban, error
) VALUES ( :date, :user, :hostname, :ok, :ban, :error ) ON DUPLICATE KEY UPDATE ok = ok + VALUES(ok), ban = ban + VALUES(ban), error = error + VALUES(error);
{ "user": "sarkhipov", "hostname": "rannts.ru", "status": "ok", "status_description": "", "response_time":
2861, }
(20 + 10) + 11 = (20 + 11) +
10
F(x)=P{σ<x} { P(x⩽x α )⩾α P(x⩾x α )⩾1−α
Ω(N 1 p )
collector collector collector pworker pworker pworker
None
None
var memCount = 75604275; var memPerSec = 1.38176367782; function updateCount()
{ next = -(1000 / memPerSec) * Math.log(Math.random()); memCountString = ''+memCount; len = memCountString.length; memCountString = memCountString.substr(0, len - 6) + ’ < span style = ”font - size: 8 px” > < /span>’+memCountString.substr(len-6,3)+‘ < span style = ”font - size: 8 px” > < /span>’+memCountString.substr(len-3,3); ge(‘memCount’).innerHTML = memCountString; memCount = memCount + 1; setTimeout(updateCount, next); } addEvent(window, ‘load’, updateCount);
3500 3671 3400 3502 3463 3371 3607 6012 6168 6211
6017
3500 3507 3671 3667 3400 3410 3502 3502 3463 3466
3371 3330 3607 3599 6012 6009 6168 6152 6211 6215 6017 6016
Count-Min Sketch 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Count-Min Sketch 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Count-Min Sketch 0 0 1 0 0 0 0 0
1 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 1
Count-Min Sketch 32 11 1 18 200 126 184 78
1 0 91 59 30 24 8 82 76 34 48 72 11 200 129 136 14
Count-Min Sketch 32 11 1 18 200 126 184 78
1 0 91 59 30 24 8 82 76 34 48 72 11 200 129 136 14
MinHash J (A , B)= |A∩B| |A∪B| k=[ 1 ε2
]
HyperLogLog 010010000110010101101100011011000110111100100001 b 26 = 64 1001 b = 9
100001 b = 33 σ= 1.04 √2k E= α(k)4k ∑ j 2−M j
t-digest
t-digest
t-digest
t-digest X=x 1 , x 2 ,…, x n X={s
1 ,s 2 ,…,s m } s i ={x l e f t(i) ,…, x r i ght(i) }
t-digest k(q,δ)≝δ (sin−1 (2q−1) π + 1 2 ) K(i)≝k(
r i ght(i) n ,δ)−k( le f t(i)−1 n ,δ) K (i)⩽1 K(i)+K (i+1)>1
t-digest
t-digest
collector collector collector pworker pworker pworker
None
Q/A