How IBLT Works

Transcript

1. How IBLT works
   @cipepser
   

   View Slide

2. Structure of IBLT
   1 2 3 ... ... ... m
   m cells
   each cell have 3 fields:
   count
   keySum
   valueSum
   

   View Slide

3. Supported method
   ・Insert(key, value)
   ・Get(key)
   ・Delete(key, value)
   ・ListEntries()
   

   View Slide

4. Insert
   for each hash function h
   i
   [key], i = 1, …, k (like standard BloomFilter)
   T: IBLT
   T[h
   i
   [key]].count++
   T[h
   i
   [key]].sumKey += key
   T[h
   i
   [key]].sumValue += value
   

   View Slide

5. Insert
   ex) m = 7 cells, k = 3 hash functions,
   h
   i
   (x) = (10 * i + x) mod m (simply)
   Insert(key=5, value=10)
   h
   1
   (key=5) = 1
   0
   c: 1
   ks: 5
   vs: 10
   1
   c: 1
   ks: 5
   vs: 10
   2 3 4
   c: 1
   ks: 5
   vs: 10
   5 6 count: c
   keySum: ks
   valueSum: vs
   h
   2
   (key=5) = 4 h
   3
   (key=5) = 0
   

   View Slide

6. Insert
   ex) m = 7 cells, k = 3 hash functions,
   h
   i
   (x) = (10 * i + x) mod m (simply)
   Insert(key=2, value=30)
   h
   1
   (key=2) = 5
   0
   c: 1
   ks: 5
   vs: 10
   1
   c: 2
   ks: 7
   vs: 40
   2 3 4
   c: 2
   ks: 7
   vs: 40
   5
   c: 1
   ks: 2
   vs: 30
   6 count: c
   keySum: ks
   valueSum: vs
   h
   2
   (key=2) = 1 h
   3
   (key=2) = 4
   

   View Slide

7. Get
   T[h
   i
   [key]].count == 0?
   YES
   NO
   return
   false
   T[h
   i
   [key]].count == 1?
   return
   sumValue
   for i = 1, …, k
   end for
   T[h
   i
   [key]].sumKey == key?
   YES
   YES
   return
   false
   NO
   NO
   

   View Slide

8. Get
   ex) m = 7 cells, k = 3 hash functions,
   h
   i
   (x) = (10 * i + x) mod m (simply)
   (key, value) = (5, 10), (2, 30) have been inserted
   Get(key=2)
   →return 30
   0
   c:1
   ks: 5
   vs: 10
   1
   c:2
   ks: 7
   vs: 40
   2 3 4
   c:2
   ks: 7
   vs: 40
   5
   c:1
   ks: 2
   vs: 30
   6
   h
   1
   (key=2) = 5
   T[5].count = 1
   T[5].keySum = 2
   T[5].valueSum = 30
   

   View Slide

9. Get
   ex) m = 7 cells, k = 3 hash functions,
   h
   i
   (x) = (10 * i + x) mod m (simply)
   (key, value) = (5, 10), (2, 30) have been inserted
   Get(key=3)
   →return false
   0
   c:1
   ks: 5
   vs: 10
   1
   c:2
   ks: 7
   vs: 40
   2 3 4
   c:2
   ks: 7
   vs: 40
   5
   c:1
   ks: 2
   vs: 30
   6
   c:0
   ks: 0
   vs: 0
   h
   1
   (key=3) = 6
   T[6].count = 0 != 1
   

   View Slide

10. Get
    ex) m = 7 cells, k = 3 hash functions,
    h
    i
    (x) = (10 * i + x) mod m (simply)
    (key, value) = (5, 10), (2, 30) have been inserted
    Get(key=9)
    →return false
    0
    c:1
    ks: 5
    vs: 10
    1
    c:2
    ks: 7
    vs: 40
    2 3 4
    c:2
    ks: 7
    vs: 40
    5
    c:1
    ks: 2
    vs: 30
    6
    h
    1
    (key=9) = 5
    T[5].keySum = 2
    h
    2
    (key=9) = 1
    T[1].keySum = 7
    h
    3
    (key=9) = 4
    T[4].keySum = 7
    

    View Slide

11. Get
    ex) m = 7 cells, k = 3 hash functions,
    h
    i
    (x) = (10 * i + x) mod m (simply)
    (key, value) = (5, 10), (2, 30), (3, 20) have been inserted
    Get(key=2)
    →return false (false-negative)
    0
    c:1
    ks: 5
    vs: 10
    1
    c:2
    ks: 7
    vs: 40
    2
    c:1
    ks: 3
    vs: 20
    3 4
    c:2
    ks: 7
    vs: 40
    5
    c:2
    ks: 5
    vs: 50
    6
    c:1
    ks: 3
    vs: 20
    h
    1
    (key=2) = 5
    T[5].keySum = 5 != 2
    h
    2
    (key=2) = 1
    T[1].keySum = 7 != 2
    h
    3
    (key=2) = 4
    T[4].keySum = 7 != 2
    

    View Slide

12. Delete
    for each hash function h
    i
    [key], i = 1, …, k
    T: IBLT
    T[h
    i
    [key]].count--
    T[h
    i
    [key]].sumKey -= key
    T[h
    i
    [key]].sumValue -= value
    

    View Slide

13. Delete
    ex) m = 7 cells, k = 3 hash functions,
    h
    i
    (x) = (10 * i + x) mod m (simply)
    (key, value) = (5, 10), (2, 30) have been inserted
    0
    c:1
    ks: 5
    vs: 10
    1
    c:2
    ks: 7
    vs: 40
    2 3 4
    c:2
    ks: 7
    vs: 40
    5
    c:1
    ks: 2
    vs: 30
    6
    Delete(key=2, value=30)
    h
    1
    (key=2) = 5
    0
    c: 1
    ks: 5
    vs: 10
    1
    c: 1
    ks: 5
    vs: 10
    2 3 4
    c: 1
    ks: 5
    vs: 10
    5
    c: 0
    ks: 0
    vs: 0
    6
    h
    2
    (key=2) = 1 h
    3
    (key=2) = 4
    

    View Slide

14. ListEntries
    T[i].count == 1?
    YES
    NO
    for i = 1, …, m
    end for
    push(
    key = T[i].sumKey,
    value = T[i].sumValue
    )
    Delete(key, value)
    START
    

    View Slide

15. ListEntries
    ex) m = 7 cells, k = 3 hash functions,
    h
    i
    (x) = (10 * i + x) mod m (simply)
    (key, value) = (5, 10), (2, 30) have been inserted
    0
    c:1
    ks: 5
    vs: 10
    1
    c:2
    ks: 7
    vs: 40
    2 3 4
    c:2
    ks: 7
    vs: 40
    5
    c:1
    ks: 2
    vs: 30
    6
    T[0].count == 1
    push(key=5,value=10)
    Delete(key=5,value=10)
    0
    c:0
    ks: 0
    vs: 0
    1
    c:1
    ks: 2
    vs: 30
    2 3 4
    c:1
    ks: 2
    vs: 30
    5
    c:1
    ks: 2
    vs: 30
    6
    T[1].count == 1
    push(key=2,value=30)
    Delete(key=2,value=30)
    0
    c:0
    ks: 0
    vs: 0
    1
    c:0
    ks: 0
    vs: 0
    2 3 4
    c:0
    ks: 0
    vs: 0
    5
    c:0
    ks: 0
    vs: 0
    6
    pushed key-value pairs:
    {(5, 10), (2, 30)}
    

    View Slide