Class 24: Privacy

Class 24: Privacy
https://uvammm.github.io/class24

Markets, Mechanisms, and Machines
University of Virginia
cs4501/econ4559 Spring 2019
David Evans and Denis Nekipelov
https://uvammm.github.io/

Transcript

1. MARKETS, MECHANISMS, MACHINES University of Virginia, Spring 2019
   Class 24:
   Privacy
   11 April 2019
   cs4501/econ4559 Spring 2019
   David Evans and Denis Nekipelov
   https://uvammm.github.io
   

   View Slide

2. View Slide

3. https://www.youtube.com/watch?v=A_6uV9A12ok
   2
   

   View Slide

4. Plan
   Last Tuesday: Economics of Information
   Value of Information ⟹ Value of Privacy
   Last Thursday: Joe Calandrino, FTC
   privacy abuses and regulations
   Today: Mechanisms for Privacy
   Next Tuesday: Privacy-Aware Mechanism Design
   3
   

   View Slide

5. Obtaining Sensitive Statistics
   4
   https://projects.fivethirtyeight.com/2019-march-madness-predictions/
   

   View Slide

6. Randomized Response
   5
   If you have a RED card: answer
   If you have a BLACK card: answer “Did you expect UVA to lose?”
   

   View Slide

7. How much privacy?
   6
   ! "#
   $%&'()#
   = “,(&”) ! "#
   $%&'()#
   = “/0”)
   

   View Slide

8. Alternative Randomized Response Methods
   Secretly roll a 6-sided die:
   1: Answer !
   2-6: Answer not !
   7
   

   View Slide

9. Flipped Randomized Response Methods
   Secretly flip a coin:
   heads: Answer !
   tails: secretly flip coin again, answer (coin = heads)
   8
   

   View Slide

10. Formalizing Privacy
    9
    

    View Slide

11. Differential Privacy
    10
    TCC 2006
    

    View Slide

12. Definition
    11
    A randomized mechanism ! satisfies (#)-Differential
    Privacy if for any two neighboring datasets % and %’:
    Pr[!(%) ∈ +]
    Pr[!(%-) ∈ +]
    ≤ /0
    “Neighboring” datasets differ in at most one entry.
    

    View Slide

13. Definition
    12
    A randomized mechanism ! satisfies (#)-Differential
    Privacy if for any two neighboring datasets % and %&:
    Pr[*(+)∈-]
    Pr[*(+/)∈-]
    ≤ 12
    Pr[*(+/)∈-]
    Pr[*(+)∈-]
    ≤ 12
    “Neighboring” datasets differ in at most one entry: definition is symmetrical
    132 ≤
    Pr[*(+)∈-]
    Pr[*(+/)∈-]
    ≤ 12
    

    View Slide

14. Definition
    13
    A randomized mechanism ! satisfies (#, %)-Differential
    Privacy if for two neighboring datasets ' and '’:
    Pr[!(') ∈ -]
    Pr[!('/) ∈ -]
    ≤ 12 + %
    

    View Slide

15. 14
    Image taken from “Differential Privacy and Pan-Private Algorithms” slides by Cynthia Dwork
    Pr[$(&) ∈ )] Pr[$(&′) ∈ )]
    Pr[$(&) ∈ )]
    Pr[$(&,) ∈ )]
    ≤ ./ + 1
    

    View Slide

16. 15
    Differential privacy describes a
    promise, made by a data
    holder, or curator, to a data
    subject: “You will not be
    affected, adversely or
    otherwise, by allowing your
    data to be used in any study or
    analysis, no matter what other
    studies, data sets, or
    information sources, are
    available.”
    

    View Slide

17. Randomized Response: Local DP
    16
    Pr[$(&) ∈ )]
    Pr[$(&+) ∈ )]
    ≤ -. + 0
    Randomized Response Mechanism:
    $ 1 :
    3 ← 0, 1 , 8 ← 0, 1
    if 3: output &
    else: output 1
    

    View Slide

18. Randomized Response: Local DP
    17
    Pr[$(&) ∈ )]
    Pr[$(&+) ∈ )]
    ≤ -. + 0
    Randomized Response Mechanism:
    $ 1 :
    3 ← 0, 1 , 8 ← 0, 1
    if 3: output &
    else: output 1
    Pr[$(1) ∈ {1}]
    Pr[$(0) ∈ {1}]
    ≤ -. + 0
    3
    4
    1
    4
    ≤ -. + 0
    -. ≥ 3 0 = 0
    H = ln 3 ≈ 1.1
    

    View Slide

19. Composition
    What if I can query ! " multiple times?
    18
    

    View Slide

20. Composition
    What if I can query ! " multiple times?
    19
    Pr[!(1) ∈ 1 ∧ !′(1) ∈ {1}]
    Pr[!(0) ∈ 1 ∧ !′(0) ∈ 1 ]
    ≤ 12 + 4
    

    View Slide

21. DP Composition
    20
    Composition Theorem:
    ! executions of an ", $ -DP mechanism satisfies !", !$ -DP.
    

    View Slide

22. 21
    https://chromium.googlesource.com/chromium/src/+/master/tools/metrics/rappor/rappor.xml
    What if you want to learn answers to lots of questions?
    

    View Slide

23. RAPPOR
    22
    ACM CCS 2014
    

    View Slide

24. Bloom Filter
    23
    1970
    (Original) Design Goals:
    small (<< |"|) data structure, to record # ⊆ " items
    lookup(+):
    + ∈ #: always returns 789:
    + ∉ #: likely to return =>?@: (but ocassionaly 789:)
    [note: no privacy goal, and does not guarantee any
    useful privacy properties!]
    

    View Slide

25. Bloom Filter Design
    24
    0 1 2 3 4 5 6 7 8 9 10 11 12 13
    Set of ! independent hash functions:
    "#
    : % → '
    

    View Slide

26. Bloom Filter Design
    25
    0 1 2 3 4 5 6 7 8 9 10 11 12 13
    Set of ! independent hash functions:
    "#
    : % → {0, … , + − 1}
    initialize: for i in 0, … , + − 1 : 4[6] = 0
    insert(9):
    for i in {0, … , ! − 1}: 4["#
    9 ] = 1
    

    View Slide

27. Bloom Filter Design
    26
    0 1 2 3 4 5 6 7 8 9 10 11 12 13
    Set of ! independent hash functions:
    "#
    : % → '
    initialize: for i in 0, … , B − 1 : 3[5] = 0
    insert(8):
    for i in {0, … , ! − 1}: 3["#
    8 ] = 1
    lookup(8):
    ⋀#<=
    >?@ 3["#
    8 ]
    Does this provide differential privacy?
    

    View Slide

28. False Positive Rate?
    After inserting ! items in "-bit filter,
    what is the probability a bit is still 0?
    27
    0 1 2 3 4 5 6 7 8 9 10 11 12 13
    

    View Slide

29. False Positive Rate?
    After inserting ! items in "-bit filter,
    what is the probability a bit is still 0?
    28
    0 1 2 3 4 5 6 7 8 9 10 11 12 13
    1 −
    1
    "
    %&
    For lookup of item not present,
    what is probability all bits are 1?
    

    View Slide

30. False Positive Rate?
    After inserting ! items in "-bit filter,
    what is the probability a bit is still 0?
    29
    0 1 2 3 4 5 6 7 8 9 10 11 12 13
    1 −
    1
    "
    %&
    For lookup of item not present,
    what is probability all bits are 1?
    1 − 1 −
    1
    "
    %& %
    ≈ 1 − (
    )%&
    *
    %
    

    View Slide

31. Bloom Filter with Noise
    30
    

    View Slide

32. Bloom Filter with Noise
    31
    ℎ = 4, % = 0.5, ) = 0.75, + = 0.5.
    

    View Slide

33. Permanent Randomized Response Privacy
    32
    

    View Slide

34. View Slide

35. View Slide

36. View Slide

37. View Slide

38. Data Analysis Pipeline
    37
    Data Subjects
    Data
    Collection
    Data Owner
    Data
    Collection
    Model Training
    Trained
    Model
    Deployed
    Model
    Hyperparameters
    User
    Machine Learning Service
    API
    User
    

    View Slide

39. Privacy Mechanisms
    38
    Data Subjects
    Data
    Collection
    Data Owner
    Data
    Collection
    Model Training
    Trained
    Model
    Deployed
    Model
    Hyperparameters
    User
    Machine Learning Service
    API
    User
    Randomized Response,
    Local Differential Privacy
    Output
    Perturbation
    Objective Perturbation
    Gradient Perturbation
    

    View Slide

40. 39
    Image: https://en.wikipedia.org/wiki/Laplace_distribution
    Laplace Distribution
    !"#$,&
    ' =
    1
    2+
    ,-
    &-.
    $
    

    View Slide

41. Charge
    Project Proposals due Tonight, 8:59pm
    40
    

    View Slide