Class 24: Privacy

Class 24: Privacy
https://uvammm.github.io/class24

Markets, Mechanisms, and Machines
University of Virginia
cs4501/econ4559 Spring 2019
David Evans and Denis Nekipelov
https://uvammm.github.io/

Transcript

1. MARKETS, MECHANISMS, MACHINES University of Virginia, Spring 2019 Class 24: Privacy 11 April 2019 cs4501/econ4559 Spring 2019 David Evans and Denis Nekipelov https://uvammm.github.io
3. https://www.youtube.com/watch?v=A_6uV9A12ok 2
4. Plan Last Tuesday: Economics of Information Value of Information ⟹ Value of Privacy Last Thursday: Joe Calandrino, FTC privacy abuses and regulations Today: Mechanisms for Privacy Next Tuesday: Privacy-Aware Mechanism Design 3
5. Obtaining Sensitive Statistics 4 https://projects.fivethirtyeight.com/2019-march-madness-predictions/
6. Randomized Response 5 If you have a RED card: answer If you have a BLACK card: answer “Did you expect UVA to lose?”
7. How much privacy? 6 ! "# $%&'()# = “,(&”) ! "# $%&'()# = “/0”)
8. Alternative Randomized Response Methods Secretly roll a 6-sided die: 1: Answer ! 2-6: Answer not ! 7
9. Flipped Randomized Response Methods Secretly flip a coin: heads: Answer ! tails: secretly flip coin again, answer (coin = heads) 8
10. Formalizing Privacy 9
11. Differential Privacy 10 TCC 2006
12. Definition 11 A randomized mechanism ! satisfies (#)-Differential Privacy if for any two neighboring datasets % and %’: Pr[!(%) ∈ +] Pr[!(%-) ∈ +] ≤ /0 “Neighboring” datasets differ in at most one entry.
13. Definition 12 A randomized mechanism ! satisfies (#)-Differential Privacy if for any two neighboring datasets % and %&: Pr[*(+)∈-] Pr[*(+/)∈-] ≤ 12 Pr[*(+/)∈-] Pr[*(+)∈-] ≤ 12 “Neighboring” datasets differ in at most one entry: definition is symmetrical 132 ≤ Pr[*(+)∈-] Pr[*(+/)∈-] ≤ 12
14. Definition 13 A randomized mechanism ! satisfies (#, %)-Differential Privacy if for two neighboring datasets ' and '’: Pr[!(') ∈ -] Pr[!('/) ∈ -] ≤ 12 + %
15. 14 Image taken from “Differential Privacy and Pan-Private Algorithms” slides by Cynthia Dwork Pr[$(&) ∈ )] Pr[$(&′) ∈ )] Pr[$(&) ∈ )] Pr[$(&,) ∈ )] ≤ ./ + 1
16. 15 Differential privacy describes a promise, made by a data holder, or curator, to a data subject: “You will not be affected, adversely or otherwise, by allowing your data to be used in any study or analysis, no matter what other studies, data sets, or information sources, are available.”
17. Randomized Response: Local DP 16 Pr[$(&) ∈ )] Pr[$(&+) ∈ )] ≤ -. + 0 Randomized Response Mechanism: $ 1 : 3 ← 0, 1 , 8 ← 0, 1 if 3: output & else: output 1
18. Randomized Response: Local DP 17 Pr[$(&) ∈ )] Pr[$(&+) ∈ )] ≤ -. + 0 Randomized Response Mechanism: $ 1 : 3 ← 0, 1 , 8 ← 0, 1 if 3: output & else: output 1 Pr[$(1) ∈ {1}] Pr[$(0) ∈ {1}] ≤ -. + 0 3 4 1 4 ≤ -. + 0 -. ≥ 3 0 = 0 H = ln 3 ≈ 1.1
19. Composition What if I can query ! " multiple times? 18
20. Composition What if I can query ! " multiple times? 19 Pr[!(1) ∈ 1 ∧ !′(1) ∈ {1}] Pr[!(0) ∈ 1 ∧ !′(0) ∈ 1 ] ≤ 12 + 4
21. DP Composition 20 Composition Theorem: ! executions of an ", $ -DP mechanism satisfies !", !$ -DP.
22. 21 https://chromium.googlesource.com/chromium/src/+/master/tools/metrics/rappor/rappor.xml What if you want to learn answers to lots of questions?
23. RAPPOR 22 ACM CCS 2014
24. Bloom Filter 23 1970 (Original) Design Goals: small (<< |"|) data structure, to record # ⊆ " items lookup(+): + ∈ #: always returns 789: + ∉ #: likely to return =>?@: (but ocassionaly 789:) [note: no privacy goal, and does not guarantee any useful privacy properties!]
25. Bloom Filter Design 24 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Set of ! independent hash functions: "# : % → '
26. Bloom Filter Design 25 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Set of ! independent hash functions: "# : % → {0, … , + − 1} initialize: for i in 0, … , + − 1 : 4[6] = 0 insert(9): for i in {0, … , ! − 1}: 4["# 9 ] = 1
27. Bloom Filter Design 26 0 1 2 3 4 5 6 7 8 9 10 11 12 13 Set of ! independent hash functions: "# : % → ' initialize: for i in 0, … , B − 1 : 3[5] = 0 insert(8): for i in {0, … , ! − 1}: 3["# 8 ] = 1 lookup(8): ⋀#<= >?@ 3["# 8 ] Does this provide differential privacy?
28. False Positive Rate? After inserting ! items in "-bit filter, what is the probability a bit is still 0? 27 0 1 2 3 4 5 6 7 8 9 10 11 12 13
29. False Positive Rate? After inserting ! items in "-bit filter, what is the probability a bit is still 0? 28 0 1 2 3 4 5 6 7 8 9 10 11 12 13 1 − 1 " %& For lookup of item not present, what is probability all bits are 1?
30. False Positive Rate? After inserting ! items in "-bit filter, what is the probability a bit is still 0? 29 0 1 2 3 4 5 6 7 8 9 10 11 12 13 1 − 1 " %& For lookup of item not present, what is probability all bits are 1? 1 − 1 − 1 " %& % ≈ 1 − ( )%& * %
31. Bloom Filter with Noise 30
32. Bloom Filter with Noise 31 ℎ = 4, % = 0.5, ) = 0.75, + = 0.5.
33. Permanent Randomized Response Privacy 32
38. Data Analysis Pipeline 37 Data Subjects Data Collection Data Owner Data Collection Model Training Trained Model Deployed Model Hyperparameters User Machine Learning Service API User
39. Privacy Mechanisms 38 Data Subjects Data Collection Data Owner Data Collection Model Training Trained Model Deployed Model Hyperparameters User Machine Learning Service API User Randomized Response, Local Differential Privacy Output Perturbation Objective Perturbation Gradient Perturbation
40. 39 Image: https://en.wikipedia.org/wiki/Laplace_distribution Laplace Distribution !"#$,& ' = 1 2+ ,- &-. $
41. Charge Project Proposals due Tonight, 8:59pm 40