Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Class 24: Privacy

David Evans
April 11, 2019

Class 24: Privacy

Class 24: Privacy
https://uvammm.github.io/class24

Markets, Mechanisms, and Machines
University of Virginia
cs4501/econ4559 Spring 2019
David Evans and Denis Nekipelov
https://uvammm.github.io/

David Evans

April 11, 2019
Tweet

More Decks by David Evans

Other Decks in Science

Transcript

  1. MARKETS, MECHANISMS, MACHINES University of Virginia, Spring 2019
    Class 24:
    Privacy
    11 April 2019
    cs4501/econ4559 Spring 2019
    David Evans and Denis Nekipelov
    https://uvammm.github.io

    View full-size slide

  2. https://www.youtube.com/watch?v=A_6uV9A12ok
    2

    View full-size slide

  3. Plan
    Last Tuesday: Economics of Information
    Value of Information ⟹ Value of Privacy
    Last Thursday: Joe Calandrino, FTC
    privacy abuses and regulations
    Today: Mechanisms for Privacy
    Next Tuesday: Privacy-Aware Mechanism Design
    3

    View full-size slide

  4. Obtaining Sensitive Statistics
    4
    https://projects.fivethirtyeight.com/2019-march-madness-predictions/

    View full-size slide

  5. Randomized Response
    5
    If you have a RED card: answer
    If you have a BLACK card: answer “Did you expect UVA to lose?”

    View full-size slide

  6. How much privacy?
    6
    ! "#
    $%&'()#
    = “,(&”) ! "#
    $%&'()#
    = “/0”)

    View full-size slide

  7. Alternative Randomized Response Methods
    Secretly roll a 6-sided die:
    1: Answer !
    2-6: Answer not !
    7

    View full-size slide

  8. Flipped Randomized Response Methods
    Secretly flip a coin:
    heads: Answer !
    tails: secretly flip coin again, answer (coin = heads)
    8

    View full-size slide

  9. Formalizing Privacy
    9

    View full-size slide

  10. Differential Privacy
    10
    TCC 2006

    View full-size slide

  11. Definition
    11
    A randomized mechanism ! satisfies (#)-Differential
    Privacy if for any two neighboring datasets % and %’:
    Pr[!(%) ∈ +]
    Pr[!(%-) ∈ +]
    ≤ /0
    “Neighboring” datasets differ in at most one entry.

    View full-size slide

  12. Definition
    12
    A randomized mechanism ! satisfies (#)-Differential
    Privacy if for any two neighboring datasets % and %&:
    Pr[*(+)∈-]
    Pr[*(+/)∈-]
    ≤ 12
    Pr[*(+/)∈-]
    Pr[*(+)∈-]
    ≤ 12
    “Neighboring” datasets differ in at most one entry: definition is symmetrical
    132 ≤
    Pr[*(+)∈-]
    Pr[*(+/)∈-]
    ≤ 12

    View full-size slide

  13. Definition
    13
    A randomized mechanism ! satisfies (#, %)-Differential
    Privacy if for two neighboring datasets ' and '’:
    Pr[!(') ∈ -]
    Pr[!('/) ∈ -]
    ≤ 12 + %

    View full-size slide

  14. 14
    Image taken from “Differential Privacy and Pan-Private Algorithms” slides by Cynthia Dwork
    Pr[$(&) ∈ )] Pr[$(&′) ∈ )]
    Pr[$(&) ∈ )]
    Pr[$(&,) ∈ )]
    ≤ ./ + 1

    View full-size slide

  15. 15
    Differential privacy describes a
    promise, made by a data
    holder, or curator, to a data
    subject: “You will not be
    affected, adversely or
    otherwise, by allowing your
    data to be used in any study or
    analysis, no matter what other
    studies, data sets, or
    information sources, are
    available.”

    View full-size slide

  16. Randomized Response: Local DP
    16
    Pr[$(&) ∈ )]
    Pr[$(&+) ∈ )]
    ≤ -. + 0
    Randomized Response Mechanism:
    $ 1 :
    3 ← 0, 1 , 8 ← 0, 1
    if 3: output &
    else: output 1

    View full-size slide

  17. Randomized Response: Local DP
    17
    Pr[$(&) ∈ )]
    Pr[$(&+) ∈ )]
    ≤ -. + 0
    Randomized Response Mechanism:
    $ 1 :
    3 ← 0, 1 , 8 ← 0, 1
    if 3: output &
    else: output 1
    Pr[$(1) ∈ {1}]
    Pr[$(0) ∈ {1}]
    ≤ -. + 0
    3
    4
    1
    4
    ≤ -. + 0
    -. ≥ 3 0 = 0
    H = ln 3 ≈ 1.1

    View full-size slide

  18. Composition
    What if I can query ! " multiple times?
    18

    View full-size slide

  19. Composition
    What if I can query ! " multiple times?
    19
    Pr[!(1) ∈ 1 ∧ !′(1) ∈ {1}]
    Pr[!(0) ∈ 1 ∧ !′(0) ∈ 1 ]
    ≤ 12 + 4

    View full-size slide

  20. DP Composition
    20
    Composition Theorem:
    ! executions of an ", $ -DP mechanism satisfies !", !$ -DP.

    View full-size slide

  21. 21
    https://chromium.googlesource.com/chromium/src/+/master/tools/metrics/rappor/rappor.xml
    What if you want to learn answers to lots of questions?

    View full-size slide

  22. RAPPOR
    22
    ACM CCS 2014

    View full-size slide

  23. Bloom Filter
    23
    1970
    (Original) Design Goals:
    small (<< |"|) data structure, to record # ⊆ " items
    lookup(+):
    + ∈ #: always returns 789:
    + ∉ #: likely to return =>?@: (but ocassionaly 789:)
    [note: no privacy goal, and does not guarantee any
    useful privacy properties!]

    View full-size slide

  24. Bloom Filter Design
    24
    0 1 2 3 4 5 6 7 8 9 10 11 12 13
    Set of ! independent hash functions:
    "#
    : % → '

    View full-size slide

  25. Bloom Filter Design
    25
    0 1 2 3 4 5 6 7 8 9 10 11 12 13
    Set of ! independent hash functions:
    "#
    : % → {0, … , + − 1}
    initialize: for i in 0, … , + − 1 : 4[6] = 0
    insert(9):
    for i in {0, … , ! − 1}: 4["#
    9 ] = 1

    View full-size slide

  26. Bloom Filter Design
    26
    0 1 2 3 4 5 6 7 8 9 10 11 12 13
    Set of ! independent hash functions:
    "#
    : % → '
    initialize: for i in 0, … , B − 1 : 3[5] = 0
    insert(8):
    for i in {0, … , ! − 1}: 3["#
    8 ] = 1
    lookup(8):
    ⋀#<=
    >?@ 3["#
    8 ]
    Does this provide differential privacy?

    View full-size slide

  27. False Positive Rate?
    After inserting ! items in "-bit filter,
    what is the probability a bit is still 0?
    27
    0 1 2 3 4 5 6 7 8 9 10 11 12 13

    View full-size slide

  28. False Positive Rate?
    After inserting ! items in "-bit filter,
    what is the probability a bit is still 0?
    28
    0 1 2 3 4 5 6 7 8 9 10 11 12 13
    1 −
    1
    "
    %&
    For lookup of item not present,
    what is probability all bits are 1?

    View full-size slide

  29. False Positive Rate?
    After inserting ! items in "-bit filter,
    what is the probability a bit is still 0?
    29
    0 1 2 3 4 5 6 7 8 9 10 11 12 13
    1 −
    1
    "
    %&
    For lookup of item not present,
    what is probability all bits are 1?
    1 − 1 −
    1
    "
    %& %
    ≈ 1 − (
    )%&
    *
    %

    View full-size slide

  30. Bloom Filter with Noise
    30

    View full-size slide

  31. Bloom Filter with Noise
    31
    ℎ = 4, % = 0.5, ) = 0.75, + = 0.5.

    View full-size slide

  32. Permanent Randomized Response Privacy
    32

    View full-size slide

  33. Data Analysis Pipeline
    37
    Data Subjects
    Data
    Collection
    Data Owner
    Data
    Collection
    Model Training
    Trained
    Model
    Deployed
    Model
    Hyperparameters
    User
    Machine Learning Service
    API
    User

    View full-size slide

  34. Privacy Mechanisms
    38
    Data Subjects
    Data
    Collection
    Data Owner
    Data
    Collection
    Model Training
    Trained
    Model
    Deployed
    Model
    Hyperparameters
    User
    Machine Learning Service
    API
    User
    Randomized Response,
    Local Differential Privacy
    Output
    Perturbation
    Objective Perturbation
    Gradient Perturbation

    View full-size slide

  35. 39
    Image: https://en.wikipedia.org/wiki/Laplace_distribution
    Laplace Distribution
    !"#$,&
    ' =
    1
    2+
    ,-
    &-.
    $

    View full-size slide

  36. Charge
    Project Proposals due Tonight, 8:59pm
    40

    View full-size slide