Concepts on AI Fairness

Transcript

1. Concepts on AI Fairness Yoriyuki Yamagata

2. Group discrimination vs. Individual discrimination • Group discrimination • eg.

   Men are more/less hired than women • Individual discrimination • eg. A man/woman with same skill is hired while a woman /man is not • Can be contradictory

3. Motivating Example • Gender discrimination on employment • Men may

   be hired more/less likely than women • Extreme measure: hire 50/50% of men/women • The extreme measure may cause individual discrimination • Women with the same skill may be more/less likely hired than men

4. Direct discrimination vs. indirect discrimination • Direct discrimination • eg.

   Hire employers based on gender • Indirect discrimination • eg. Hire based on features which correlated to gender

5. Motivating example • Hire people with longer experience of full-time

   employment • This can be legitimate • Full-time employment may indicate higher skill • But can adversely affect women • Women tend to have more part-time job • Family responsibility, child birth, etc...

6. Formal definition of group fairness

7. General settings • S: sensitive features (gender, nationality, religion...) •

   X: other features • Y: output of the algorithm • Can be deterministic/probabilistic • Assume Y: binary

8. Direct / Indirect discrimination • Fairness against direct discrimination •

   Pr[ Y = 1 | X, S ] = Pr[ Y = 1 | X ] • Fairness against indirect discrimination • Pr[ Y = 1 | S ] = Pr[ Y = 1 ] • X0: explainable feature • Pr[ Y = 1 | X0, S ] = Pr[ Y = 1 | X0 ]

9. Motivating examples • Fairness against direct discrimination • Hire regardless

   of gender if an applicant has the same skill • Fairness against indirect discrimination • If 25% of applicants are women, 25% of hired applicants are also women (regardless of relative skill level between men/women)

10. Problem of group fairness • Hire men by their skill,

    but women by the length of hair • Hire men randomly, but women by their skill • Probability can be same • More generally, it is based on statistical properties • address “disparate impact” • do not address “disparate treatment”

11. Formal definition of individual fairness - following Dwork et al.

    2012

12. Idea • “Similar” person (for a given task) must be

    treated similarly • eg. A person with the same skill level is treated in the same way, regardless of gender

13. Formal setting • V : set of individual (= X,

    S) • Y : outcome (Y = 0 or 1) • d : similarity measure on V (real number) • d(x, y) >= 0, d(x, y) = d(y, x), d(x, x) = 0 • M : Assignment for Y to V (M : V -> distributions on Y) • may use lottery (eg. choosing jury) • D : measure of difference between two distribution on Y

14. (D, d) - Lipschitz property • D(Mx, My) <= d(x,

    y) for all pairs of individuals x, y • • Dtv (P, Q) = 1 2 ∑ Y=0,1 |P(Y) − Q(Y)| D∞ (P, Q) = sup Y=0,1 log(max{ P(Y) Q(Y) , Q(Y) P(Y) })

15. Relation to group fairness • : Average provability of Y

    = 1 for an individual belonging to S • : the largest for any M with (D, d) - Lipschitz property • Theorem: • : Earth mover’s distance between S and T μS (1) biasD,d (S, T) μS (1) − μT (1) biasD,d (S, T) ≤ dEM (S, T) dEM (S, T)

16. Individual fairness contradicts group fairness S T High Skill Low

    Skill Everyone must have the same result

17. Appendix: Distinction and Discrimination

18. Reflective equilibrium • Methodology of modern ethics • Start from

    intuition • Find a governing principle • Explanatory power • Internal consistency • consistency with other principles • Change intuition Principle Intuition

19. When does distinction become discrimination? • Competing views (* are

    popular views) • If it causes disadvantage to one of the groups* • If it shows/expresses exclusion of one of the groups* • If it is not justified rationally • If it is based on one’s nature which cannot be changed

20. References • ػցֶशɾσʔλϚΠχϯάʹ͓͚Δެฏੑ, Kamishima & Komiyama, ਓ޻஌ೳ34ר2߸, 2019 • Fairness

    Through Awareness, Dwork et al., ITCS 2012 • FlipTest: Fairness Testing via Optimal Transport, Black et al. ACM FAT 2020 • Altman, "Discrimination", The Stanford Encyclopedia of Philosophy (Winter 2016 Edition)

21. Other resources • ACM Conference on Fairness, Accountability, and Transparency

    (ACM FAccT) • https://facctconference.org