The AI 'Black Box'

The AI 'Black Box'

As humans we're inherently biased. Sometimes it's explicit and other times it's unconscious, but as we move forward with technology how do we keep our biases out of the algorithms we create? We need to have a conversation about how AI 'Black Box' should be governed and ask who is responsible for overseeing the ethical standards of these algorithms; and the need for gender balance — having access to large and diverse points of view helps to train algorithms to maintain the principle of fairness.


Carla Vieira

August 16, 2019


  1. The Artificial Intelligence “Black-box” Carla Vieira @carlaprvieira Ilustração: Hanne Mostard

  2. About me Carla Vieira Information Systems – USP Artificial Intelligence

    Evangelist Community Manager perifaCode @car lap r vie ir a @car lap r v@h o m
  3. None
  4. None
  5. None
  6. None
  7. Tech Conferences

  8. bias data privacy law ethics

  9. None
  10. We need to talk less about Artificial Intelligence hype …

    … and more about how we are using this technology.
  11. #1 Google Photos

  12. #2 Gender Shades Article

  13. “Whether AI will help us reach our aspirations or reinforce

    the unjust inequalities is ultimately up to us.” Joy Buolamwini
  14. #3 Tweet

  15. #4 Google’s Algorithm

  16. • 46% false positives for African American • African American

    authors are 1.5 times more likely to be labelled “offensive”
  17. Software COMPAS #5 COMPAS Software


  19. Artificial Intelligence needs to learn from the real world. Creating

    a smart computer is not enough, you need to teach it the right thing. machine-learning/?hl=pt-BR
  20. Gender Gap in Artificial Intelligence “Only 22% of AI professionals

    globally are female, compared to 78% who are male.” (The Global Gender Gap Report 2018 - p.28)
  21. Human Bias Technology Bias

  22. None
  23. Even though these decisions affect humans, to optimize task performance

    ML models often become too complex to be intelligible to humans: black-box models


  26. None

  28. "This new law is a complete shame for our

    democracy.” Louis Larret Chahine Co-founder PREDICTICE


  31. XAI intends to create a new suite of ML techniques

    that produce more interpretable ML models
  32. Accuracy vs. Interpretability trade-off

  33. Pre-modelling explainability Goal Understand/describe data used to develop models Methodologies

    • Exploratory data analysis • Dataset description standardization • Dataset summarization • Explainable feature engineering Explainable modelling Goal Develop inherently more explainable models Methodologies • Adopt explainable model family • Hybrid models • Joint prediction and explanation • Architectural adjustments • Regularization Post-modelling explainability Goal Extract explanations to describe pre-developed models Methodologies • Perturbation mechanism • Backward propagation • Proxy models • Activation optimization Explainability
  34. None
  35. Post-modelling explainability The proposed taxonomy of the post-hoc explainability methods

    including the four aspects of target, drivers, explanation family, and estimator.
  36. Post-modelling explainability first a perturbation model is used to obtain

    perturbed versions of the input sequence. Next, associations between input and predicted sequence are inferred using a causal inference model. Finally, the obtained associations are partitioned and the most relevant sets are selected.
  37. None

  39. If we want AI to really benefit people, we need

    to find a way to get people to trust it.

  41. Less machines that are going to take our jobs and

    more about what technology can actually achieve…
  42. The choices we are making today about Artificial Intelligence are

    going to define our future.
  43. Thank you! Carla Vieira @carlaprvieira

  44. Useful links − AI NOW − Racial and Gender bias

    in Amazon Rekognition − Diversity in faces (IBM) − Google video – Machine Learning and Human Bias − Visão Computacional e Vieses Racializados − Machine Bias on Compas − Machine Learning Explainability Kaggle − Predictive modeling: striking a balance between accuracy and interpretability
  45. Useful links −Racismo Algorítmico em Plataformas Digitais: microagressões e discriminação

    em código −Metrics for Explainable AI: Challenges and Prospects −The Mythos of Model Interpretability −Towards Robust Interpretability with Self-Explaining Neural Networks −The How of Explainable AI: Post-modelling Explainability