How IBLT Works

Transcript

1. How IBLT works @cipepser

2. Structure of IBLT 1 2 3 ... ... ... m

   m cells each cell have 3 fields: count keySum valueSum

3. Supported method ・Insert(key, value) ・Get(key) ・Delete(key, value) ・ListEntries()

4. Insert for each hash function h i [key], i =

   1, …, k (like standard BloomFilter) T: IBLT T[h i [key]].count++ T[h i [key]].sumKey += key T[h i [key]].sumValue += value

5. Insert ex) m = 7 cells, k = 3 hash

   functions, h i (x) = (10 * i + x) mod m (simply) Insert(key=5, value=10) h 1 (key=5) = 1 0 c: 1 ks: 5 vs: 10 1 c: 1 ks: 5 vs: 10 2 3 4 c: 1 ks: 5 vs: 10 5 6 count: c keySum: ks valueSum: vs h 2 (key=5) = 4 h 3 (key=5) = 0

6. Insert ex) m = 7 cells, k = 3 hash

   functions, h i (x) = (10 * i + x) mod m (simply) Insert(key=2, value=30) h 1 (key=2) = 5 0 c: 1 ks: 5 vs: 10 1 c: 2 ks: 7 vs: 40 2 3 4 c: 2 ks: 7 vs: 40 5 c: 1 ks: 2 vs: 30 6 count: c keySum: ks valueSum: vs h 2 (key=2) = 1 h 3 (key=2) = 4

7. Get T[h i [key]].count == 0? YES NO return false

   T[h i [key]].count == 1? return sumValue for i = 1, …, k end for T[h i [key]].sumKey == key? YES YES return false NO NO

8. Get ex) m = 7 cells, k = 3 hash

   functions, h i (x) = (10 * i + x) mod m (simply) (key, value) = (5, 10), (2, 30) have been inserted Get(key=2) →return 30 0 c:1 ks: 5 vs: 10 1 c:2 ks: 7 vs: 40 2 3 4 c:2 ks: 7 vs: 40 5 c:1 ks: 2 vs: 30 6 h 1 (key=2) = 5 T[5].count = 1 T[5].keySum = 2 T[5].valueSum = 30

9. Get ex) m = 7 cells, k = 3 hash

   functions, h i (x) = (10 * i + x) mod m (simply) (key, value) = (5, 10), (2, 30) have been inserted Get(key=3) →return false 0 c:1 ks: 5 vs: 10 1 c:2 ks: 7 vs: 40 2 3 4 c:2 ks: 7 vs: 40 5 c:1 ks: 2 vs: 30 6 c:0 ks: 0 vs: 0 h 1 (key=3) = 6 T[6].count = 0 != 1

10. Get ex) m = 7 cells, k = 3 hash

    functions, h i (x) = (10 * i + x) mod m (simply) (key, value) = (5, 10), (2, 30) have been inserted Get(key=9) →return false 0 c:1 ks: 5 vs: 10 1 c:2 ks: 7 vs: 40 2 3 4 c:2 ks: 7 vs: 40 5 c:1 ks: 2 vs: 30 6 h 1 (key=9) = 5 T[5].keySum = 2 h 2 (key=9) = 1 T[1].keySum = 7 h 3 (key=9) = 4 T[4].keySum = 7

11. Get ex) m = 7 cells, k = 3 hash

    functions, h i (x) = (10 * i + x) mod m (simply) (key, value) = (5, 10), (2, 30), (3, 20) have been inserted Get(key=2) →return false (false-negative) 0 c:1 ks: 5 vs: 10 1 c:2 ks: 7 vs: 40 2 c:1 ks: 3 vs: 20 3 4 c:2 ks: 7 vs: 40 5 c:2 ks: 5 vs: 50 6 c:1 ks: 3 vs: 20 h 1 (key=2) = 5 T[5].keySum = 5 != 2 h 2 (key=2) = 1 T[1].keySum = 7 != 2 h 3 (key=2) = 4 T[4].keySum = 7 != 2

12. Delete for each hash function h i [key], i =

    1, …, k T: IBLT T[h i [key]].count-- T[h i [key]].sumKey -= key T[h i [key]].sumValue -= value

13. Delete ex) m = 7 cells, k = 3 hash

    functions, h i (x) = (10 * i + x) mod m (simply) (key, value) = (5, 10), (2, 30) have been inserted 0 c:1 ks: 5 vs: 10 1 c:2 ks: 7 vs: 40 2 3 4 c:2 ks: 7 vs: 40 5 c:1 ks: 2 vs: 30 6 Delete(key=2, value=30) h 1 (key=2) = 5 0 c: 1 ks: 5 vs: 10 1 c: 1 ks: 5 vs: 10 2 3 4 c: 1 ks: 5 vs: 10 5 c: 0 ks: 0 vs: 0 6 h 2 (key=2) = 1 h 3 (key=2) = 4

14. ListEntries T[i].count == 1? YES NO for i = 1,

    …, m end for push( key = T[i].sumKey, value = T[i].sumValue ) Delete(key, value) START

15. ListEntries ex) m = 7 cells, k = 3 hash

    functions, h i (x) = (10 * i + x) mod m (simply) (key, value) = (5, 10), (2, 30) have been inserted 0 c:1 ks: 5 vs: 10 1 c:2 ks: 7 vs: 40 2 3 4 c:2 ks: 7 vs: 40 5 c:1 ks: 2 vs: 30 6 T[0].count == 1 push(key=5,value=10) Delete(key=5,value=10) 0 c:0 ks: 0 vs: 0 1 c:1 ks: 2 vs: 30 2 3 4 c:1 ks: 2 vs: 30 5 c:1 ks: 2 vs: 30 6 T[1].count == 1 push(key=2,value=30) Delete(key=2,value=30) 0 c:0 ks: 0 vs: 0 1 c:0 ks: 0 vs: 0 2 3 4 c:0 ks: 0 vs: 0 5 c:0 ks: 0 vs: 0 6 pushed key-value pairs: {(5, 10), (2, 30)}