Class 24: Privacy - Speaker Deck

Slide 1

Slide 1 text

MARKETS, MECHANISMS, MACHINES University of Virginia, Spring 2019 Class 24: Privacy 11 April 2019 cs4501/econ4559 Spring 2019 David Evans and Denis Nekipelov https://uvammm.github.io

Slide 2

Slide 2 text

Slide 3

Slide 3 text

https://www.youtube.com/watch?v=A_6uV9A12ok 2

Slide 4

Slide 4 text

Plan Last Tuesday: Economics of Information Value of Information ⟹ Value of Privacy Last Thursday: Joe Calandrino, FTC privacy abuses and regulations Today: Mechanisms for Privacy Next Tuesday: Privacy-Aware Mechanism Design 3

Slide 5

Slide 5 text

Obtaining Sensitive Statistics 4 https://projects.fivethirtyeight.com/2019-march-madness-predictions/

Slide 6

Slide 6 text

Randomized Response 5 If you have a RED card: answer If you have a BLACK card: answer “Did you expect UVA to lose?”

Slide 7

Slide 7 text

How much privacy? 6 ! "# $%&'()# = “,(&”) ! "# $%&'()# = “/0”)

Slide 8

Slide 8 text

Alternative Randomized Response Methods Secretly roll a 6-sided die: 1: Answer ! 2-6: Answer not ! 7

Slide 9

Slide 9 text

Flipped Randomized Response Methods Secretly flip a coin: heads: Answer ! tails: secretly flip coin again, answer (coin = heads) 8

Slide 10

Slide 10 text

Formalizing Privacy 9

Slide 11

Slide 11 text

Differential Privacy 10 TCC 2006

Slide 12

Slide 12 text

Definition 11 A randomized mechanism ! satisfies (#)-Differential Privacy if for any two neighboring datasets % and %’: Pr[!(%) ∈ +] Pr[!(%-) ∈ +] ≤ /0 “Neighboring” datasets differ in at most one entry.

Slide 13

Slide 13 text

Definition 12 A randomized mechanism ! satisfies (#)-Differential Privacy if for any two neighboring datasets % and %&: Pr[*(+)∈-] Pr[*(+/)∈-] ≤ 12 Pr[*(+/)∈-] Pr[*(+)∈-] ≤ 12 “Neighboring” datasets differ in at most one entry: definition is symmetrical 132 ≤ Pr[*(+)∈-] Pr[*(+/)∈-] ≤ 12

Slide 14

Slide 14 text

Definition 13 A randomized mechanism ! satisfies (#, %)-Differential Privacy if for two neighboring datasets ' and '’: Pr[!(') ∈ -] Pr[!('/) ∈ -] ≤ 12 + %

Slide 15

Slide 15 text

14 Image taken from “Differential Privacy and Pan-Private Algorithms” slides by Cynthia Dwork Pr[$(&) ∈ )] Pr[$(&′) ∈ )] Pr[$(&) ∈ )] Pr[$(&,) ∈ )] ≤ ./ + 1

Slide 16

Slide 16 text

15 Differential privacy describes a promise, made by a data holder, or curator, to a data subject: “You will not be affected, adversely or otherwise, by allowing your data to be used in any study or analysis, no matter what other studies, data sets, or information sources, are available.”

Slide 17

Slide 17 text

Randomized Response: Local DP 16 Pr[$(&) ∈ )] Pr[$(&+) ∈ )] ≤ -. + 0 Randomized Response Mechanism: $ 1 : 3 ← 0, 1 , 8 ← 0, 1 if 3: output & else: output 1

Slide 18

Slide 18 text

Randomized Response: Local DP 17 Pr[$(&) ∈ )] Pr[$(&+) ∈ )] ≤ -. + 0 Randomized Response Mechanism: $ 1 : 3 ← 0, 1 , 8 ← 0, 1 if 3: output & else: output 1 Pr[$(1) ∈ {1}] Pr[$(0) ∈ {1}] ≤ -. + 0 3 4 1 4 ≤ -. + 0 -. ≥ 3 0 = 0 H = ln 3 ≈ 1.1

Slide 19

Slide 19 text

Composition What if I can query ! " multiple times? 18

Slide 20

Slide 20 text

Composition What if I can query ! " multiple times? 19 Pr[!(1) ∈ 1 ∧ !′(1) ∈ {1}] Pr[!(0) ∈ 1 ∧ !′(0) ∈ 1 ] ≤ 12 + 4

Slide 21

Slide 21 text

DP Composition 20 Composition Theorem: ! executions of an ", $ -DP mechanism satisfies !", !$ -DP.

Slide 22

Slide 22 text

21 https://chromium.googlesource.com/chromium/src/+/master/tools/metrics/rappor/rappor.xml What if you want to learn answers to lots of questions?

Slide 23

Slide 23 text

RAPPOR 22 ACM CCS 2014

Slide 24

Slide 24 text

Bloom Filter 23 1970 (Original) Design Goals: small (<< |"|) data structure, to record # ⊆ " items lookup(+): + ∈ #: always returns 789: + ∉ #: likely to return =>?@: (but ocassionaly 789:) [note: no privacy goal, and does not guarantee any useful privacy properties!]

Slide 25

Slide 25 text

Bloom Filter Design 24 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Set of ! independent hash functions: "# : % → '

Slide 26

Slide 26 text

Bloom Filter Design 25 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Set of ! independent hash functions: "# : % → {0, … , + − 1} initialize: for i in 0, … , + − 1 : 4[6] = 0 insert(9): for i in {0, … , ! − 1}: 4["# 9 ] = 1

Slide 27

Slide 27 text

Bloom Filter Design 26 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Set of ! independent hash functions: "# : % → ' initialize: for i in 0, … , B − 1 : 3[5] = 0 insert(8): for i in {0, … , ! − 1}: 3["# 8 ] = 1 lookup(8): ⋀#<= >?@ 3["# 8 ] Does this provide differential privacy?

Slide 28

Slide 28 text

False Positive Rate? After inserting ! items in "-bit filter, what is the probability a bit is still 0? 27 0 1 2 3 4 5 6 7 8 9 10 11 12 13

Slide 29

Slide 29 text

False Positive Rate? After inserting ! items in "-bit filter, what is the probability a bit is still 0? 28 0 1 2 3 4 5 6 7 8 9 10 11 12 13 1 − 1 " %& For lookup of item not present, what is probability all bits are 1?

Slide 30

Slide 30 text

False Positive Rate? After inserting ! items in "-bit filter, what is the probability a bit is still 0? 29 0 1 2 3 4 5 6 7 8 9 10 11 12 13 1 − 1 " %& For lookup of item not present, what is probability all bits are 1? 1 − 1 − 1 " %& % ≈ 1 − ( )%& * %

Slide 31

Slide 31 text

Bloom Filter with Noise 30

Slide 32

Slide 32 text

Bloom Filter with Noise 31 ℎ = 4, % = 0.5, ) = 0.75, + = 0.5.

Slide 33

Slide 33 text

Permanent Randomized Response Privacy 32

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Data Analysis Pipeline 37 Data Subjects Data Collection Data Owner Data Collection Model Training Trained Model Deployed Model Hyperparameters User Machine Learning Service API User

Slide 39

Slide 39 text

Privacy Mechanisms 38 Data Subjects Data Collection Data Owner Data Collection Model Training Trained Model Deployed Model Hyperparameters User Machine Learning Service API User Randomized Response, Local Differential Privacy Output Perturbation Objective Perturbation Gradient Perturbation

Slide 40

Slide 40 text

39 Image: https://en.wikipedia.org/wiki/Laplace_distribution Laplace Distribution !"#$,& ' = 1 2+ ,- &-. $

Slide 41

Slide 41 text

Charge Project Proposals due Tonight, 8:59pm 40