David Evans
April 11, 2019
580

Class 24: Privacy

Class 24: Privacy
https://uvammm.github.io/class24

Markets, Mechanisms, and Machines
University of Virginia
cs4501/econ4559 Spring 2019
David Evans and Denis Nekipelov
https://uvammm.github.io/

April 11, 2019

Transcript

1. MARKETS, MECHANISMS, MACHINES University of Virginia, Spring 2019
Class 24:
Privacy
11 April 2019
cs4501/econ4559 Spring 2019
David Evans and Denis Nekipelov
https://uvammm.github.io

2. 1

2

4. Plan
Last Tuesday: Economics of Information
Value of Information ⟹ Value of Privacy
Last Thursday: Joe Calandrino, FTC
privacy abuses and regulations
Today: Mechanisms for Privacy
Next Tuesday: Privacy-Aware Mechanism Design
3

5. Obtaining Sensitive Statistics
4

6. Randomized Response
5
If you have a RED card: answer
If you have a BLACK card: answer “Did you expect UVA to lose?”

7. How much privacy?
6
! "#
\$%&'()#
= “,(&”) ! "#
\$%&'()#
= “/0”)

8. Alternative Randomized Response Methods
Secretly roll a 6-sided die:
7

9. Flipped Randomized Response Methods
Secretly flip a coin:
8

10. Formalizing Privacy
9

11. Differential Privacy
10
TCC 2006

12. Definition
11
A randomized mechanism ! satisfies (#)-Differential
Privacy if for any two neighboring datasets % and %’:
Pr[!(%) ∈ +]
Pr[!(%-) ∈ +]
≤ /0
“Neighboring” datasets differ in at most one entry.

13. Definition
12
A randomized mechanism ! satisfies (#)-Differential
Privacy if for any two neighboring datasets % and %&:
Pr[*(+)∈-]
Pr[*(+/)∈-]
≤ 12
Pr[*(+/)∈-]
Pr[*(+)∈-]
≤ 12
“Neighboring” datasets differ in at most one entry: definition is symmetrical
132 ≤
Pr[*(+)∈-]
Pr[*(+/)∈-]
≤ 12

14. Definition
13
A randomized mechanism ! satisfies (#, %)-Differential
Privacy if for two neighboring datasets ' and '’:
Pr[!(') ∈ -]
Pr[!('/) ∈ -]
≤ 12 + %

15. 14
Image taken from “Differential Privacy and Pan-Private Algorithms” slides by Cynthia Dwork
Pr[\$(&) ∈ )] Pr[\$(&′) ∈ )]
Pr[\$(&) ∈ )]
Pr[\$(&,) ∈ )]
≤ ./ + 1

16. 15
Differential privacy describes a
holder, or curator, to a data
subject: “You will not be
otherwise, by allowing your
data to be used in any study or
analysis, no matter what other
studies, data sets, or
information sources, are
available.”

17. Randomized Response: Local DP
16
Pr[\$(&) ∈ )]
Pr[\$(&+) ∈ )]
≤ -. + 0
Randomized Response Mechanism:
\$ 1 :
3 ← 0, 1 , 8 ← 0, 1
if 3: output &
else: output 1

18. Randomized Response: Local DP
17
Pr[\$(&) ∈ )]
Pr[\$(&+) ∈ )]
≤ -. + 0
Randomized Response Mechanism:
\$ 1 :
3 ← 0, 1 , 8 ← 0, 1
if 3: output &
else: output 1
Pr[\$(1) ∈ {1}]
Pr[\$(0) ∈ {1}]
≤ -. + 0
3
4
1
4
≤ -. + 0
-. ≥ 3 0 = 0
H = ln 3 ≈ 1.1

19. Composition
What if I can query ! " multiple times?
18

20. Composition
What if I can query ! " multiple times?
19
Pr[!(1) ∈ 1 ∧ !′(1) ∈ {1}]
Pr[!(0) ∈ 1 ∧ !′(0) ∈ 1 ]
≤ 12 + 4

21. DP Composition
20
Composition Theorem:
! executions of an ", \$ -DP mechanism satisfies !", !\$ -DP.

22. 21
What if you want to learn answers to lots of questions?

23. RAPPOR
22
ACM CCS 2014

24. Bloom Filter
23
1970
(Original) Design Goals:
small (<< |"|) data structure, to record # ⊆ " items
lookup(+):
+ ∈ #: always returns 789:
+ ∉ #: likely to return =>?@: (but ocassionaly 789:)
[note: no privacy goal, and does not guarantee any
useful privacy properties!]

25. Bloom Filter Design
24
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Set of ! independent hash functions:
"#
: % → '

26. Bloom Filter Design
25
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Set of ! independent hash functions:
"#
: % → {0, … , + − 1}
initialize: for i in 0, … , + − 1 : 4[6] = 0
insert(9):
for i in {0, … , ! − 1}: 4["#
9 ] = 1

27. Bloom Filter Design
26
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Set of ! independent hash functions:
"#
: % → '
initialize: for i in 0, … , B − 1 : 3[5] = 0
insert(8):
for i in {0, … , ! − 1}: 3["#
8 ] = 1
lookup(8):
⋀#<=
>?@ 3["#
8 ]
Does this provide differential privacy?

28. False Positive Rate?
After inserting ! items in "-bit filter,
what is the probability a bit is still 0?
27
0 1 2 3 4 5 6 7 8 9 10 11 12 13

29. False Positive Rate?
After inserting ! items in "-bit filter,
what is the probability a bit is still 0?
28
0 1 2 3 4 5 6 7 8 9 10 11 12 13
1 −
1
"
%&
For lookup of item not present,
what is probability all bits are 1?

30. False Positive Rate?
After inserting ! items in "-bit filter,
what is the probability a bit is still 0?
29
0 1 2 3 4 5 6 7 8 9 10 11 12 13
1 −
1
"
%&
For lookup of item not present,
what is probability all bits are 1?
1 − 1 −
1
"
%& %
≈ 1 − (
)%&
*
%

31. Bloom Filter with Noise
30

32. Bloom Filter with Noise
31
ℎ = 4, % = 0.5, ) = 0.75, + = 0.5.

33. Permanent Randomized Response Privacy
32

34. 33

35. 34

36. 35

37. 36

38. Data Analysis Pipeline
37
Data Subjects
Data
Collection
Data Owner
Data
Collection
Model Training
Trained
Model
Deployed
Model
Hyperparameters
User
Machine Learning Service
API
User

39. Privacy Mechanisms
38
Data Subjects
Data
Collection
Data Owner
Data
Collection
Model Training
Trained
Model
Deployed
Model
Hyperparameters
User
Machine Learning Service
API
User
Randomized Response,
Local Differential Privacy
Output
Perturbation
Objective Perturbation

40. 39
Image: https://en.wikipedia.org/wiki/Laplace_distribution
Laplace Distribution
!"#\$,&
' =
1
2+
,-
&-.
\$

41. Charge
Project Proposals due Tonight, 8:59pm
40