Machine Learning Bias

We have been constantly told this statement “Computers don’t lie”. Yes in fact Computers don’t lie, but neither does it speak the truth. A computer does what its Master programs it to do. Similarly, A model wouldn’t lie unless the Machine Learning Engineer want it to lie. Humans are filled with unconscious biases and when these are fed into Machine to Learn in the form of Data, the resulting AI model wouldn't be `fair` enough without Biases. This deck tries to introduce you to the world of Machine Learning Bias.

Transcript

1. Machine Learning Bias
   AbdulMajedRaja RS
   

   View Slide

2. Outline
   ● Recognizing the Problem
   ● What’s Machine Learning Bias?
   ● Definition of “Fairness”
   ● Interpretable Machine Learning
   

   View Slide

3. Thoughts?
   What if I told you Computers can lie?
   Would you believe me?
   

   View Slide

4. Biased-Google Translation at Work
   

   View Slide

5. The Problem - Samples
   

   View Slide

6. But Wait, Why is this concerning?
   After all, This is just Google Translate
   

   View Slide

7. Biased-Google Photos App at Work
   

   View Slide

8. Perhaps, That’s just Google.
   Two instances can account for the entire
   industry, Huh?
   

   View Slide

9. Microsoft’s super-cool Teen Tweeting Bot Tay
   

   View Slide

10. Much more!
    

    View Slide

11. Oops, Got it!
    There, definitely, is Bias!
    What’s next?
    

    View Slide

12. ML Bias - What
    

    View Slide

13. What’s Machine Learning Bias?
    A Machine Learning Algorithm being
    “unfair” with its Predictions
    A Machine Learning Algorithm missing
    “Fairness”
    

    View Slide

14. ML Bias - (un)Fairness
    

    View Slide

15. Disclaimer
    No Common Consensus / Standard
    definition of Fairness
    

    View Slide

16. ML Bias - un(Fairness)
    ● Group Fairness
    ● Individual Fairness
    

    View Slide

17. ML Bias - Causes
    

    View Slide

18. ML Bias - Causes
    ● Skewed sample
    ● Tainted examples
    ● Limited features
    ● Sample size disparity
    ● Proxies
    

    View Slide

19. ML Bias - Mitigate
    

    View Slide

20. Mitigation
    Also means, Improving Fairness
    

    View Slide

21. ML Bias - Improving Fairness
    Pre-Processing Training (Optimization) Post-Processing
    Learn a New
    Representation - Free
    from Sensitive Variable
    Yet, preserving the
    Information
    Add a constraint or a
    regularization term
    Find a proper threshold
    using the original score
    function
    

    View Slide

22. ML Bias - Happening
    

    View Slide

23. Mention of ML Fairness in Research Papers
    

    View Slide

24. Difficulties in ensuring ML Algorithm is unbiased
    

    View Slide

25. Interpretable
    Machine Learning
    

    View Slide

26. Today - Modelling Architecture
    

    View Slide

27. IML - Definition
    Interpretable Machine Learning refers to methods and
    models that make the behavior and predictions of
    machine learning systems understandable
    to humans.
    

    View Slide

28. IML - Benefits
    ● Fairness: Ensuring that predictions are unbiased and do not implicitly or explicitly
    discriminate against protected groups. An interpretable model can tell you why it
    has decided that a certain person should not get a loan, and it becomes easier for a
    human to judge whether the decision is based on a learned demographic (e.g.
    racial) bias.
    ● Privacy: Ensuring that sensitive information in the data is protected.
    ● Reliability or Robustness: Ensuring that small changes in the input do not lead to
    large changes in the prediction.
    ● Causality: Check that only causal relationships are picked up.
    ● Trust: It is easier for humans to trust a system that explains its decisions
    compared to a black box.
    

    View Slide

29. Modelling Architecture - with IML
    

    View Slide

30. Preferred Explaining - Model Interpretation
    

    View Slide

31. References
    ● https://developers.google.com/machine-learning/fairness-overview/
    ● https://towardsdatascience.com/a-tutorial-on-fairness-in-machine-learning-3ff8ba1040cb
    ● https://www.youtube.com/watch?v=fMym_BKWQzk
    ● https://www.kaggle.com/nulldata/ml-bias-iml-perspective-recommendation#media-coverage-about-bias-i
    ml
    ● Doshi-Velez, Finale, and Been Kim. “Towards a rigorous science of interpretable machine learning,” no.
    Ml: 1–13. http://arxiv.org/abs/1702.08608 ( 2017)
    ● https://christophm.github.io/interpretable-ml-book/
    ● https://github.com/adebayoj/fairml/
    

    View Slide

32. Thank you!
    

    View Slide