Algorithmic Bias in Machine Learning

Machine learning algorithms are susceptible to both intentional and unintentional bias. Relying on biased algorithms to drive decisions can lead to unfair outcomes that have serious consequences affecting underrepresented groups of people. In this talk, we'll walk through examples of algorithmic bias in machine learning algorithms and explore tools (in Python) that can measure this bias.

Transcript

1. Algorithmic Bias in Machine Learning Jill Cates Data Scientist at BioSymetrics November 16, 2019 PyCon Canada, Toronto Image source: Wired
2. Machine Learning Trends
3. PyCon U.S. PyCon Canada Machine Learning Trends
4. PyCon Canada 2019 Machine Learning Trends PyCon U.S.
5. Machine Learning Trends Decision-Making Facial Recognition Recommendations Financial Investing Real Estate Political Campaigns
6. Algorithmic Bias “systematic and repeatable errors in a computer system that create unfair outcomes, such as privileging one arbitrary group of users over others” “Our own values and desires influence our choices, from the data we choose to collect to the questions we ask. Models are opinions embedded in mathematics.” - Cathy O’Neil (Weapons of Math Destruction) An unfortunate by-product of machine learning
7. Algorithmic Bias An unfortunate by-product of machine learning Online Advertising Legal System HR Recruitment Facial Recognition Healthcare Credit Scores
8. Algorithmic Bias An unfortunate by-product of machine learning Future Implications: MIT Media Lab's Moral Machine: http://moralmachine.mit.edu/ Source: MIT Technology Review
9. Building a Shoe Classifier
10. Building a Shoe Classifier Data Collection
11. ??? Machine Learning Algorithm Building a Shoe Classifier Model Training Biased data = Biased model
12. ??? Machine Learning Algorithm Building a President Classifier Model Training (Zuzana Čaputová, president of Slovakia) Biased data = Biased model
13. Clinical Decision-Making • Recent paper published in Science (Oct 25, 2019) • Assessed a U.S. healthcare decision-making algorithm • Looked at 50,000 records from an academic hospital • Found that white patients were given higher risk scores and received more care than black patients • Clinical diagnostic tools and early detection of disease • When to admit or discharge a patient from the hospital • Automated triaging of patients • Assessing patient risk
14. • MIMIC-III is a widely used healthcare dataset in machine learning research • Developed and de-identified by the MIT Laboratory for Computational Physiology • Contains electronic medical record data of 50,000 hospital admissions for 40,000 critical care patients • Collected at Beth Israel Deaconess Medical Centre between 2001 and 2012 Clinical Decision-Making Some papers that use MIMIC III:
15. 70% of patients are white 47% of patients are insured by Medicare 35% of patients are Catholic 41% of patients are married MIMIC-III Dataset Demographics (mean age of adult patients is 62.5 years old)
16. Language Translation Translating gender neutral languages • Gender neutral languages: Malay, Farsi (Persian), Hungarian, Armenian, Tagalog, etc. • Google Translate determines which gender should be assigned to which role • Trained on examples of translations from the web
17. Language Translation • Words are represented as vectors • Similar words have similar representations Word Embeddings Word2Vec • Generates word embeddings (gensim) • Reveals semantic relations (associations) between words
18. Language Translation • gender bias is captured by a direction in the word embedding • gender-neutral words are distinct from gender-definition words in the word embedding number of generated analogies number of stereotypic analogies
19. Mitigating the Risk of Bias • Contains metrics to test for “unwarranted associations between an algorithm's outputs and certain user subpopulations identified by protected features” • Identifies subpopulations with disproportionately high error rates, assesses offensive labeling, and detects uneven rates of algorithmic error FairTest by Columbia University Some included metrics: Normalized Mutual Information, Normalized Conditional Mutual Information, Binary Ratio, Binary Difference
20. Mitigating the Risk of Bias • Fairness metrics to test for biases (e.g., Generalized Entropy Index evaluates inequality in a dataset) • Bias mitigation algorithms (e.g., Adversarial Debiasing, Learning Fair Representations, Disparate Impact Remover, etc.) AIF360 by IBM AI 360
21. Mitigating the Risk of Bias • Deep learning models are difficult to interpret (can’t extract feature importances) • Lime provides explanations of a black-box model’s predictions • Can be used to interpret image and text classifiers Explaining Models with Lime
22. Achieving Algorithmic Fairness Conference on Fairness, Accountability, and Transparency (FAT) Algorithmic Accountability
23. Jill Cates Data Scientist at BioSymetrics github: @topspinj / twitter: @jillacates cates.jill@gmail.com Thank you!