Velocity London 2017 - A tour of sketching data structures

As the scale of data our systems produce continues to increase, the techniques our systems use to process it must evolve. Kiran Bhattaram explains why sketches are a good option for leveraging more sophisticated data structures.

Sketching data structures are probabilistic structures that store a summary of the full dataset. They’re specialized to answer specific questions (e.g., how many unique values a large dataset contains or what the p95 of the dataset is). By leveraging some neat mathematical properties, sketching data structures trades accuracy for a significant increase in both computational and storage efficiency.

Kiran covers real-world use cases of a few basic sketching data structures and explores the statistical underpinnings that make them work.

Transcript

1. 1 S K E T C H I N G D ATA S T R U C T U R E S Velocity London
2. 2 S K E T C H I N G D ATA S T R U C T U R E S Velocity London
3. 3 Kiran Bhattaram @kiranb
4. 4 Timeline W h e r e t o g o f r o m h e r e S o l v i n g p r o b l e m s M o t i v a t i o n S y s t e m s i n P r o d u c t i o n !
5. 4 Timeline W h e r e t o g o f r o m h e r e S o l v i n g p r o b l e m s M o t i v a t i o n S y s t e m s i n P r o d u c t i o n !
6. 4 Timeline W h e r e t o g o f r o m h e r e S o l v i n g p r o b l e m s M o t i v a t i o n S y s t e m s i n P r o d u c t i o n !
7. 4 Timeline W h e r e t o g o f r o m h e r e S o l v i n g p r o b l e m s M o t i v a t i o n S y s t e m s i n P r o d u c t i o n !
8. 4 Timeline W h e r e t o g o f r o m h e r e S o l v i n g p r o b l e m s M o t i v a t i o n S y s t e m s i n P r o d u c t i o n !
9. 5 Background 1 motivation, history, system models
10. 6 Algorithm Efficiency Axes time space
11. 6 Algorithm Efficiency Axes time space error probability
12. 6 Algorithm Efficiency Axes time space implementation complexity error probability
13. 7 Classic algorithms time space error probability = 0
14. 8 Probabilistic Algorithms time space error probability
15. 9 If you can tolerate error… 4 x 109 => 0.5 GiB to store IPv4 addresses how many IP addresses have we seen?
16. 9 If you can tolerate error… 4 x 109 => 0.5 GiB to store IPv4 addresses vs. 1.5kB with a 2% error how many IP addresses have we seen?
17. 9 If you can tolerate error… 4 x 109 => 0.5 GiB to store IPv4 addresses vs. 1.5kB with a 2% error how many IP addresses have we seen? x 358,000
18. 10 What are sketches?
19. 10 What are sketches? probabilistic algorithms
20. 10 What are sketches? probabilistic algorithms summarize stream of data
21. 10 What are sketches? probabilistic algorithms summarize stream of data streaming data/online queries
22. 11 how they work
23. 11 how they work [ ] stream of data . . .
24. 11 how they work P(n) hash! uniform distribution [ ] stream of data . . .
25. 11 how they work P(n) hash! uniform distribution [ ] stream of data . . . data structure
26. 11 how they work P(n) hash! uniform distribution estimator [ ] stream of data . . . data structure
27. 11 how they work P(n) hash! uniform distribution estimator guess +/- ε [ ] stream of data . . . data structure
28. 12 Estimators & Observables ✦ Order statistics: [10, 11, 10, 01] ex: smallest value seen so far ✦ Bit-pattern: ex: longest run of contiguous 0s 10001010 ✦ Presence: ex: is the bit set?
29. 13 But also! Horizontal Scalability! :( :) :) :) :) :) :) :)
30. 14 A Case Study: Story Reader
31. 15 Editor: Features
32. 15 Editor: Features 1. Feed of short stories without duplicates
33. 15 Editor: Features 1. Feed of short stories without duplicates 2. Working vocabulary size (# of unique words)
34. 15 Editor: Features 1. Feed of short stories without duplicates 2. Working vocabulary size (# of unique words) 3. Word length statistics
35. 16 Editor: Analytics Requirements Fast: want real-time statistics Okay to be good ~enough Cheap to run: no data analytics team!
36. 17 Bloom Filters 2 set membership
37. 18 The Problem is this element in this set? [ ]
38. 18 The Problem Google Chrome: ”is this URL known to be malicious?" is this element in this set? [ ]
39. 18 The Problem Google Chrome: ”is this URL known to be malicious?" is this element in this set? [ ] Databases/LSM trees: “is this data on disk?”
40. 18 The Problem Google Chrome: ”is this URL known to be malicious?" is this element in this set? [ ] Databases/LSM trees: “is this data on disk?” Story Feed: “have I read this short story?”
41. 19 Hash Set hash to a bitmap; test for presence [ ]
42. 20 Hash Functions
43. 20 Hash Functions 34248a9bfcbd589d 9b5fccb6a0ac6963 2fc01ec765ec0cb3 dcc559126de20b30 1. Deterministic
44. 20 Hash Functions 34248a9bfcbd589d 9b5fccb6a0ac6963 2fc01ec765ec0cb3 dcc559126de20b30 1. Deterministic 2. Uniform P(n)
45. 21 Hash Set — Insertion hash to a bitmap; test for presence [ ] array of size m
46. 21 Hash Set — Insertion hash to a bitmap; test for presence [ ] array of size m hash ( ) mod m
47. 21 Hash Set — Insertion hash to a bitmap; test for presence [ ] array of size m hash ( ) mod m
48. 21 Hash Set — Insertion hash to a bitmap; test for presence [ ] array of size m hash ( ) mod m
49. 21 Hash Set — Insertion hash to a bitmap; test for presence [ ] array of size m hash ( ) mod m
50. 21 Hash Set — Insertion hash to a bitmap; test for presence [ ] array of size m hash ( ) mod m
51. 21 Hash Set — Insertion hash to a bitmap; test for presence [ ] array of size m hash ( ) mod m
52. 22 Hash Set — Testing hash to a bitmap; test for presence
53. 22 Hash Set — Testing hash to a bitmap; test for presence
54. 22 Hash Set — Testing hash to a bitmap; test for presence
55. 22 Hash Set — Testing hash to a bitmap; test for presence
56. 23 Hash Set — Collisions
57. 24 The system now
58. 24 The system now
59. 24 The system now
60. 24 The system now
61. 24 The system now
62. 25 Scaling the system x100
63. 25 Scaling the system x100
64. 25 Scaling the system x100
65. 26 Intuition 1: don’t store the entire object! false positives! m bits in the array P(bit = 0)
66. 26 Intuition 1: don’t store the entire object! false positives! ( )n m bits in the array P(bit = 0)
67. 26 Intuition 1: don’t store the entire object! false positives! ( )n number of elements inserted m bits in the array P(bit = 0)
68. 26 Intuition 1: don’t store the entire object! false positives! ( )n 1 - number of elements inserted m bits in the array P(bit = 0)
69. 27 Intuition 2 — Multiply Hashing! run through k independent hash functions Bloom, Burton H. (1970), "Space/Time Trade-offs in Hash Coding with Allowable Errors"
70. 27 Intuition 2 — Multiply Hashing! run through k independent hash functions Bloom, Burton H. (1970), "Space/Time Trade-offs in Hash Coding with Allowable Errors"
71. 27 Intuition 2 — Multiply Hashing! run through k independent hash functions h1(x) h2(x) h3(x) Bloom, Burton H. (1970), "Space/Time Trade-offs in Hash Coding with Allowable Errors"
72. 27 Intuition 2 — Multiply Hashing! run through k independent hash functions Bloom, Burton H. (1970), "Space/Time Trade-offs in Hash Coding with Allowable Errors"
73. 27 Intuition 2 — Multiply Hashing! run through k independent hash functions Bloom, Burton H. (1970), "Space/Time Trade-offs in Hash Coding with Allowable Errors"
74. 27 run through k independent hash functions Bloom, Burton H. (1970), "Space/Time Trade-offs in Hash Coding with Allowable Errors" Bloom Filter!
75. 28 Bloom Filter — Testing! hash to a bitmap; test for presence
76. 29 Bloom Filter — Testing! false positives!
77. 30 Bloom Filter — Error Rates! false positives! oooh!
78. 30 Bloom Filter — Error Rates! false positives! oooh!
79. 30 Bloom Filter — Error Rates! false positives! oooh!
80. 30 Bloom Filter — Error Rates! false positives! oooh!
81. 31 Bloom Filter — Error Rates! false positives! number of hash functions (k) false positive possibility optimal k!
82. 32 Bloom Filters: a summary No false negatives Smaller memory footprint
 (store 4-8 bits vs. entire obj) Small (and tunable!) false positive rate Can’t retrieve or delete items
83. 33 how they work: Bloom Filters [ ] stream of data . . .
84. 33 how they work: Bloom Filters [ ] stream of data . . . P(n) hash! uniform distribution
85. 33 how they work: Bloom Filters [ ] stream of data . . . P(n) hash! uniform distribution data structure: bitmap
86. 33 how they work: Bloom Filters [ ] stream of data . . . P(n) hash! uniform distribution data structure: bitmap estimator: presence
87. 33 how they work: Bloom Filters [ ] stream of data . . . P(n) hash! uniform distribution data structure: bitmap estimator: presence guess +/- ε
88. 34 Story Feed: feed architecture
89. 34 Story Feed: feed architecture
90. 34 Story Feed: feed architecture
91. 34 Story Feed: feed architecture
92. 35 Merging Bloom Filters Bitwise OR =
93. 36 An extension: Counting Bloom Filters allows for deletions Fan, Li et al. (2000), "Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol" 0 0 0 0 0 0 0 0 0
94. 36 An extension: Counting Bloom Filters allows for deletions 1 1 1 Fan, Li et al. (2000), "Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol" 0 0 0 0 0 0
95. 36 An extension: Counting Bloom Filters allows for deletions 1 1 1 1 1 1 Fan, Li et al. (2000), "Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol" 0 0 0
96. 36 An extension: Counting Bloom Filters allows for deletions 2 2 1 1 1 1 1 Fan, Li et al. (2000), "Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol" 0 0
97. 36 An extension: Counting Bloom Filters allows for deletions 2 1 1 1 Fan, Li et al. (2000), "Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol" 1 0 0 0 0
98. 2 1 2 1 37 An extension: Count-Min Sketch keep a count of the frequency of items seen min() estimator 2 3 1 2 4 h1 h2 h3 Cormode, Graham (2009). "Count-min sketch"
99. 2 1 2 1 37 An extension: Count-Min Sketch keep a count of the frequency of items seen min() estimator 2 3 1 2 4 h1 h2 h3 Cormode, Graham (2009). "Count-min sketch"
100. 38 Bloom Filters: A Summary
101. 38 Bloom Filters: A Summary • Hash Sets —> Bloom Filters
102. 38 Bloom Filters: A Summary • Hash Sets —> Bloom Filters • bits & multiple hashing!
103. 38 Bloom Filters: A Summary • Hash Sets —> Bloom Filters • bits & multiple hashing! • Extensions: Counting
104. 38 Bloom Filters: A Summary • Hash Sets —> Bloom Filters • bits & multiple hashing! • Extensions: Counting • Extensions: Count-min sketch
105. 39 Hyper Log Log 3 counting uniques
106. 40 Editor: Text Analytics denote read stories (Bloom filters!) count unique words used/unique users
107. 41 The Problem: Cardinality number of unique values in a collection [ ]
108. 41 The Problem: Cardinality number of unique values in a collection [ ] advertising: number of “uniques”
109. 41 The Problem: Cardinality traffic modeling: # of unique IP addresses number of unique values in a collection [ ] advertising: number of “uniques”
110. 41 The Problem: Cardinality traffic modeling: # of unique IP addresses number of unique values in a collection [ ] advertising: number of “uniques” natural language processing: number of unique words
111. 42 Measuring Cardinality size = N (number of unique values)
112. 42 Measuring Cardinality size = N (number of unique values) (ex) IPv4: 232 bits
113. 42 Measuring Cardinality size = N (number of unique values) (ex) IPv4: 232 bits 0.5 GiB
114. 43 The system now
115. 43 The system now
116. 43 The system now
117. 43 The system now
118. 43 The system now
119. 43 The system now
120. 43 The system now
121. 43 The system now
122. 44 The Paper Flajolet, Fusy, et al. 2007
123. 45 Flipping coins!
124. 46 Flipping coins! seeing a rare combination => I’ve seen a lot of trials!
125. 47 Bit patterns! 0101 1010 0010 0001 1100 1011 0101 1011 1010 run of 3 0s => likely seen 8 numbers!
126. 48 Making a uniform distribution Hashing! (ex: murmurhash) 01 10 11
127. 49 But the cardinality estimate could be so wrong! Techniques for increasing accuracy ~8 friends ~4 friends ~4 friends = ~5.33 friends x 3 trials
128. 50 The Algorithm 000 001 010 011 100 101 110 111 a register of m=8 bytes
129. 50 The Algorithm 000 001 010 011 100 101 110 111 a register of m=8 bytes 010 00010
130. 50 The Algorithm Bucket the first log2 8 bits 000 001 010 011 100 101 110 111 a register of m=8 bytes 010 00010
131. 50 The Algorithm Bucket the first log2 8 bits 000 001 010 011 100 101 110 111 count leading 0s a register of m=8 bytes 010 00010
132. 50 The Algorithm Bucket the first log2 8 bits 000 001 010 011 100 101 110 111 count leading 0s a register of m=8 bytes 010 00010
133. 50 The Algorithm 3 Bucket the first log2 8 bits 000 001 010 011 100 101 110 111 count leading 0s a register of m=8 bytes 010 00010
134. 51 The Algorithm 110 01111 111 00100 111 00111 110 01010 011 00000 100 00100 101 00011 101 01010 010 00011 000 01001 001 00111 001 01111 1 2 3 5 2 3 1 2 000 001 010 011 100 101 110 111 … …
135. 52 The Algorithm 1 2 3 5 2 3 1 2 000 001 010 011 100 101 110 111 take the harmonic mean of all of these!
136. 52 The Algorithm 1 2 3 5 2 3 1 2 000 001 010 011 100 101 110 111 take the harmonic mean of all of these! = 8 * 3.93 = 31.5
137. 52 The Algorithm 1 2 3 5 2 3 1 2 000 001 010 011 100 101 110 111 take the harmonic mean of all of these! = 8 * 3.93 = 31.5 (I used 28 values)
138. 52 The Algorithm 1 2 3 5 2 3 1 2 000 001 010 011 100 101 110 111 take the harmonic mean of all of these! = 8 * 3.93 = 31.5 (I used 28 values) Plus corrections for small and large values!
139. 53 Merging Hyper Log Logs 1 2 3 5 2 1 4 8 max() for each register 2 2 4 8 =
140. 54 Hyper Log Log — Error Rates! over and under estimating Cardinality Space required (m) Error 109 1.5kB ~2% (vs. 0.5GiB!)
141. 55 how they work: Hyper Log Logs [ ] stream of data . . .
142. 55 how they work: Hyper Log Logs [ ] stream of data . . . P(n) hash! uniform distribution
143. 55 how they work: Hyper Log Logs [ ] stream of data . . . P(n) hash! uniform distribution data structure 1 2 3 5 2 3 1 2 000 001 010 011 100 101 110 111
144. 55 how they work: Hyper Log Logs [ ] stream of data . . . P(n) hash! uniform distribution estimator (run of 0s) data structure 1 2 3 5 2 3 1 2 000 001 010 011 100 101 110 111
145. 55 how they work: Hyper Log Logs [ ] stream of data . . . P(n) hash! uniform distribution estimator (run of 0s) guess +/- ε data structure 1 2 3 5 2 3 1 2 000 001 010 011 100 101 110 111
146. 56 Estimating your working vocabulary
147. 56 Estimating your working vocabulary
148. 56 Estimating your working vocabulary
149. 56 Estimating your working vocabulary
150. 56 Estimating your working vocabulary
151. 57 HyperLogLog: A Summary
152. 57 HyperLogLog: A Summary • uniform distributions!
153. 57 HyperLogLog: A Summary • uniform distributions! • log! log! space!
154. 57 HyperLogLog: A Summary • uniform distributions! • log! log! space! • commutativity!
155. 58 t-digests 4 estimating quantiles
156. 59 Editor: Text Analytics denote read stories (Bloom filters!) count unique words used estimate percentiles for word length
157. 60 The Algorithm - CDFs! Value Frequency
158. 60 The Algorithm - CDFs! Value Frequency Value Probability
159. 60 The Algorithm - CDFs! Value Frequency Value Probability 0.5
160. 60 The Algorithm - CDFs! Value Frequency Value Probability 0.5 0.95
161. 60 The Algorithm - CDFs! Value Frequency Value Probability 0.5 0.95 0.99
162. 61 The Algorithm - compression q-digests
163. 61 The Algorithm - compression q-digests Value Probability
164. 61 The Algorithm - compression q-digests Value Probability 0.5
165. 61 The Algorithm - compression q-digests Value Probability 0.5 0.95
166. 61 The Algorithm - compression q-digests Value Probability 0.5 0.95 0.99
167. 62 t digest — Error Rates! non-constant error rates
168. 63 how they work - t digests
169. 63 how they work - t digests [ ] stream of data . . .
170. 63 how they work - t digests [ ] stream of data . . . data structure
171. 63 how they work - t digests [ ] stream of data . . . estimator data structure
172. 63 how they work - t digests [ ] stream of data . . . estimator guess +/- ε data structure
173. 64 Estimating your word length quantiles
174. 64 Estimating your word length quantiles
175. 64 Estimating your word length quantiles
176. 65 in production 5 Veneur, Summingbird, Sawzall
177. 66 Veneur
178. 67 Analytics Overall API Latency (p90, p95, p99)
179. 68 previously, on Stripe monitoring => no global statistics!
180. 69 previously, on Stripe monitoring => UDP packet drops :( :(
181. 70 previously, on Stripe monitoring aggregation! :)
182. 71 Sketches! HyperLogLogs (sets) t-digests (quantiles/histograms)
183. 72 Algebird
184. 73 Sketches HyperLogLogs (sets) Bloom Filters Count-min sketch MinHasher and more!
185. 74 Google Sawzall
186. 75 Sketches! HyperLogLogs (sets) Munro & Peterson quantile estimation (quantiles)
187. 76 Where to go from here 6 evaluating sketches, further resources
188. 77 Evaluating Sketches • performance • error rate • distortion • tuning
189. 78 A brief list of other sketches • Skip Lists • frequency: count-min sketch, heavy hitters, etc • membership: Bloom filters, Cuckoo hashing • cardinality: hyperloglog • geometric data: coresets, locality-sensitive hashing
190. 79 tl;dr — error is a tradeoff in algorithms approximations are often Good Enough and a hell of a lot cheaper
191. 80 Thanks! @ k i r a n b kiranbot.com
192. 81 Appendix!
193. 82 Database Semi-Joins city author 1 Kiran 2 or peer-to-peer networks!
194. 82 Database Semi-Joins city author 1 Kiran 2 or peer-to-peer networks!
195. 82 Database Semi-Joins city author 1 Kiran 2 or peer-to-peer networks!
196. 83 Small Value Corrections 1 3 2
197. 83 Small Value Corrections 1 3 2 Estimate = m*log(m/# of un-init registers) = ~ 3.75 values
198. 84 Large Value Corrections
199. 84 Large Value Corrections as the number of unique values approaches 2^(2^m), you start seeing hash collisions!
200. 84 Large Value Corrections as the number of unique values approaches 2^(2^m), you start seeing hash collisions! => use a 64 bit hash & more bits in the registers!
201. 85 History query cost estimation
202. 86 Sphere: dealing with fraudsters :(
203. 87 Sphere: building a fraud shield :)