The AI 'Black Box'

As humans we're inherently biased. Sometimes it's explicit and other times it's unconscious, but as we move forward with technology how do we keep our biases out of the algorithms we create? We need to have a conversation about how AI 'Black Box' should be governed and ask who is responsible for overseeing the ethical standards of these algorithms; and the need for gender balance — having access to large and diverse points of view helps to train algorithms to maintain the principle of fairness.

Transcript

1. The Artificial
   Intelligence “Black-box”
   Carla Vieira
   @carlaprvieira
   Ilustração: Hanne Mostard
   

   View Slide

2. About me
   Carla Vieira
   Information Systems – USP
   Artificial Intelligence Evangelist
   Community Manager perifaCode
   @car lap r vie ir a @car lap r v@h o tmail.co m
   

   View Slide

3. View Slide

4. View Slide

5. View Slide

6. View Slide

7. Tech Conferences
   

   View Slide

8. bias
   data
   privacy law
   ethics
   

   View Slide

9. View Slide

10. We need to talk less about Artificial
    Intelligence hype …
    … and more about how we are using this
    technology.
    

    View Slide

11. #1 Google Photos
    

    View Slide

12. #2 Gender Shades Article
    http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf
    

    View Slide

13. “Whether AI will help us reach our
    aspirations or reinforce the unjust
    inequalities is ultimately up to us.”
    Joy Buolamwini
    

    View Slide

14. #3 Tweet
    

    View Slide

15. #4 Google’s Algorithm
    

    View Slide

16. • 46% false positives for
    African American
    • African American authors are
    1.5 times more likely to be
    labelled “offensive”
    https://homes.cs.washington.edu/~msap/pdfs/sap2019risk.pdf
    

    View Slide

17. Software COMPAS
    #5 COMPAS Software
    

    View Slide

18. https://www.research.ibm.com/artificial-intelligence/trusted-ai/diversity-in-faces/
    

    View Slide

19. Artificial Intelligence needs to
    learn from the real world.
    Creating a smart computer is
    not enough, you need to teach
    it the right thing.
    https://about.google/stories/gender-balance-diversity-important-to-
    machine-learning/?hl=pt-BR
    

    View Slide

20. Gender Gap in Artificial Intelligence
    “Only 22% of AI professionals globally are female,
    compared to 78% who are male.”
    (The Global Gender Gap Report 2018 - p.28)
    

    View Slide

21. Human Bias Technology
    Bias
    

    View Slide

22. View Slide

23. Even though these decisions affect humans, to
    optimize task performance ML models often
    become too complex to be intelligible to
    humans: black-box models
    

    View Slide

24. BLACK BOX
    INPUT OUTPUT
    

    View Slide

25. BLACK BOX
    INPUT OUTPUT
    

    View Slide

26. View Slide

27. JUSTICE MATH
    

    View Slide

28. https://www.artificiallawyer.com/2019/06/04/france-bans-judge-analytics-5-years-in-prison-for-rule-breakers/
    "This new law is a complete
    shame for our democracy.”
    Louis Larret Chahine
    Co-founder PREDICTICE
    

    View Slide

29. https://edition.cnn.com/2019/05/14/tech/san-francisco-facial-recognition-ban/index.html
    

    View Slide

30. How to open this black-box?
    TRANSPARENCY
    EXPLAINABLE ARTIFICIAL
    INTELLIGENCE
    (XAI)
    TRUST
    

    View Slide

31. XAI intends to create a new suite of
    ML techniques that produce more
    interpretable ML models
    

    View Slide

32. Accuracy vs. Interpretability trade-off
    

    View Slide

33. Pre-modelling
    explainability
    Goal
    Understand/describe data
    used to develop models
    Methodologies
    • Exploratory data analysis
    • Dataset description
    standardization
    • Dataset summarization
    • Explainable feature
    engineering
    Explainable
    modelling
    Goal
    Develop inherently more
    explainable models
    Methodologies
    • Adopt explainable model
    family
    • Hybrid models
    • Joint prediction and
    explanation
    • Architectural adjustments
    • Regularization
    Post-modelling
    explainability
    Goal
    Extract explanations to
    describe pre-developed
    models
    Methodologies
    • Perturbation mechanism
    • Backward propagation
    • Proxy models
    • Activation optimization
    Explainability
    

    View Slide

34. View Slide

35. Post-modelling explainability
    The proposed taxonomy of the post-hoc explainability methods including the four aspects of
    target, drivers, explanation family, and estimator.
    

    View Slide

36. Post-modelling explainability
    first a perturbation model is used to obtain perturbed versions of the input sequence. Next,
    associations between input and predicted sequence are inferred using a causal inference model.
    Finally, the obtained associations are partitioned and the most relevant sets are selected.
    

    View Slide

37. View Slide

38. http://www.portaltransparencia.gov.br/download-de-dados
    

    View Slide

39. If we want AI to really benefit
    people, we need to find a way to
    get people to trust it.
    

    View Slide

40. https://brasil.io/home
    https://serenata.ai/
    https://colaboradados.github.io/
    

    View Slide

41. Less machines that are going to take our
    jobs and more about what technology can
    actually achieve…
    

    View Slide

42. The choices we are making today
    about Artificial Intelligence are going
    to define our future.
    

    View Slide

43. Thank you!
    Carla Vieira
    @carlaprvieira
    carlaprv@hotmail.com
    

    View Slide

44. Useful links
    − AI NOW
    − Racial and Gender bias in Amazon Rekognition
    − Diversity in faces (IBM)
    − Google video – Machine Learning and Human Bias
    − Visão Computacional e Vieses Racializados
    − Machine Bias on Compas
    − Machine Learning Explainability Kaggle
    − Predictive modeling: striking a balance between accuracy and interpretability
    

    View Slide

45. Useful links
    −Racismo Algorítmico em Plataformas Digitais: microagressões e
    discriminação em código
    −Metrics for Explainable AI: Challenges and Prospects
    −The Mythos of Model Interpretability
    −Towards Robust Interpretability with Self-Explaining Neural
    Networks
    −The How of Explainable AI: Post-modelling Explainability
    

    View Slide