OS@IBM - The Machine in Sheep’s Clothing: Trust and Transparency in the ML Lifecycle

Tools/Communities:
AI Fairness 360 Toolkit: http://aif360.mybluemix.net/
Model Asset Exchange: http://ibm.biz/model-exchange
IBM's Center for Open Source Data and AI Technologies: http://ibm.biz/codait-projects
IBM's Internal Call for Code Competition: https://w3.ibm.com/developer/callforcode/
IBM's Public Call for Code Competition: https://callforcode.org/

Talk Sources:

Podcasts/Tweets
https://leanin.org/podcast-episodes/siri-is-artificial-intelligence-biased
https://art19.com/shows/the-ezra-klein-show/episodes/663fd0b7-ee60-4e3e-b2cb-4fcb4040eef1
https://twitter.com/alexisohanian/status/1087973027055316994

Amazon
https://www.aclu.org/blog/privacy-technology/surveillance-technologies/amazons-face-recognition-falsely-matched-28
https://www.openmic.org/news/2019/1/16/halt-rekognition

Google
https://motherboard.vice.com/en_us/article/j5jmj8/google-artificial-intelligence-bias

COMPAS
https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
https://www.technologyreview.com/s/612775/algorithms-criminal-justice-ai/

Data for Black Lives
http://d4bl.org/about.html
2019 Conference Notes: https://docs.google.com/document/d/1E1mfgTp73QFRmNBunl8cIpyUmDos28rekidux0voTsg/edit?ts=5c39f92e

Gender Shades Project
http://news.mit.edu/2018/study-finds-gender-skin-type-bias-artificial-intelligence-systems-0212
https://www.youtube.com/watch?time_continue=1&v=TWWsW1w-BVo
https://www.ajlunited.org/fight

Other resources referenced in this talk:
https://www.nytimes.com/2018/02/12/business/computer-science-ethics-courses.html
https://www.vox.com/science-and-health/2017/4/17/15322378/how-artificial-intelligence-learns-how-to-be-racist
https://www.engadget.com/2019/01/24/pinterest-skin-tone-search-diversity/

Transcript

1. Open Source @ IBM
   Blueprint Talk
   The Machine in Sheep’s
   Clothing
   Trust and Transparency in
   the ML Lifecycle
   May 8th, 2019
   

   View Slide

2. Open Source @ IBM
   May 8, 2019 / © 2019 IBM Corporation 2
   JEFFREY BOREK
   WW Program Director, Open Technology
   & IP Mgt.
   IBM Cognitive Applications
   jborek@us.ibm.com
   @jeffborek
   

   View Slide

3. The Machine in Sheep’s Clothing
   Trust and Transparency in the ML
   Lifecycle
   Maureen McElaney
   Developer Advocate
   IBM Center for Open Source Data and AI
   Technologies
   mmcelaney@us.ibm.com
   @Mo_Mack
   May 8, 2019 / © 2019 IBM Corporation
   

   View Slide

4. The Machine in Sheep’s
   Clothing
   Building Trust and Transparency
   into the ML Lifecycle
   4
   May 8, 2019 / © 2019 IBM Corporation
   

   View Slide

5. May 8, 2019 / © 2019 IBM Corporation 5
   

   View Slide

6. “A cognitive bias is a systematic pattern
   of deviation from norm or rationality in
   judgment. Individuals create their own
   "subjective social reality" from their
   perception of the input.”
   - Wikipedia
   6
   

   View Slide

7. May 8, 2019 / © 2019 IBM Corporation 7
   

   View Slide

8. Examples of bias in machine
   learning.
   8
   May 8, 2019 / © 2019 IBM Corporation
   

   View Slide

9. Google’s Cloud
   Natural
   Language API
   9 Image Credit: #WOCinTech
   

   View Slide

10. October 2017 - Google Natural Language API
    https://cloud.google.com/natural-language/
    10
    Source: https://motherboard.vice.com/en_us/article/j5jmj8/google-artificial-intelligence-bias
    

    View Slide

11. October 2017 - Google Natural Language API
    https://cloud.google.com/natural-language/
    11
    Source: https://motherboard.vice.com/en_us/article/j5jmj8/google-artificial-intelligence-bias
    

    View Slide

12. October 2017 - Google Natural Language API
    https://cloud.google.com/natural-language/
    12
    Source: https://motherboard.vice.com/en_us/article/j5jmj8/google-artificial-intelligence-bias
    

    View Slide

13. “We will correct this
    specific case, and, more
    broadly, building more
    inclusive algorithms is
    crucial to bringing the
    benefits of machine
    learning to everyone.”
    13
    

    View Slide

14. NorthPointe’s
    COMPAS
    Algorithm
    14 Image Credit: #WOCinTech
    

    View Slide

15. Source: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
    May 2016 - Northpointe’s COMPAS Algorithm
    http://www.equivant.com/solutions/inmate-
    classification
    15
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

16. May 2016 - Northpointe’s COMPAS Algorithm
    http://www.equivant.com/solutions/inmate-
    classification
    Source: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

17. May 2016 - Northpointe’s COMPAS Algorithm
    http://www.equivant.com/solutions/inmate-
    classification
    Source: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

18. May 2016 - Northpointe’s COMPAS Algorithm
    http://www.equivant.com/solutions/inmate-
    classification
    Source: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

19. Source: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
    19
    Black Defendant’s Risk Scores
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

20. Source: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
    20
    White Defendant’s Risk Scores
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

21. BLACK VS. WHITE
    DEFENDANTS
    ○ Falsely labeled black defendants as likely
    of future crime at twice the rate as white
    defendants.
    ○ White defendants mislabeled as low risk
    more than black defendants
    ○ Pegged Black defendants 77% more likely
    to be at risk of committing future violent
    crime
    21
    

    View Slide

22. 22
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

23. Amazon
    Rekognition
    23 Image Credit: #WOCinTech
    

    View Slide

24. July 2018 - Amazon Rekognition
    https://aws.amazon.com/rekognition/
    24
    Source: https://www.aclu.org/blog/privacy-technology/surveillance-technologies/amazons-face-recognition-falsely-matched-28
    

    View Slide

25. April 2019 - Amazon Rekognition
    https://aws.amazon.com/rekognition/
    25
    Source: https://www.openmic.org/news/2019/4/4/a-win-for-shareholders-amazon
    

    View Slide

26. Joy Buolamwini,
    Algorithmic Justice League
    Gender Shades Project
    Released February 2018
    26
    

    View Slide

27. View Slide

28. “If we fail to make
    ethical and inclusive
    artificial intelligence
    we risk losing gains
    made in civil rights
    and gender equity
    under the guise of
    machine neutrality.”
    28
    - Joy Boulamwini
    @jovialjoy
    

    View Slide

29. Solutions?
    What can we do
    to combat bias
    in AI?
    29
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

30. 30
    https://www.vox.com/ezra-klein-show-podcast
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

31. “Coders are the
    most empowered
    laborers that have
    ever existed.”
    31
    - Anil Dash
    @anildash
    

    View Slide

32. EDUCATION IS
    KEY
    32 Image Credit: #WOCinTech
    

    View Slide

33. https://www.nytimes.com/2018/02/12/business/computer-science-
    ethics-courses.html
    

    View Slide

34. Questions
    posed to
    students
    in these
    courses...
    Is the
    technology
    fair?
    How do you
    make sure
    that the
    data is not
    biased?
    Should
    machines
    be judging
    humans?
    34
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

35. 35
    https://twitter.com/Neurosarda/status/1084198368526680064
    

    View Slide

36. FIX THE
    PIPELINE?
    36 Image Credit: #WOCinTech
    

    View Slide

37. “Cognitive bias in
    machine learning is
    human bias on
    steroids.”
    37
    - Rediet Abebe
    @red_abebe
    

    View Slide

38. January 2019 - New Search Feature on...
    https://www.pinterest.com
    Source:
    https://www.engadget.com/2019/01/24/pinterest-skin-tone-search-diversity/
    

    View Slide

39. “By combining the
    latest in machine
    learning and inclusive
    product development,
    we're able to directly
    respond to Pinner
    feedback and build a
    more useful product.”
    39
    - Candice Morgan
    @Candice_MMorgan
    

    View Slide

40. TOOLS TO
    COMBAT BIAS
    40 Image Credit: #WOCinTech
    

    View Slide

41. Tool #1:
    AI Fairness
    360 Toolkit
    Open Source Library
    41
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

42. http://aif360.mybluemix.net/
    

    View Slide

43. http://aif360.mybluemix.net/
    

    View Slide

44. TYPES OF METRICS
    ○ Individual vs. Group Fairness, or Both
    ○ Group Fairness: Data vs Model
    ○ Group Fairness: We’re All Equal vs What
    You See is What You Get
    ○ Group Fairness: Ratios vs Differences
    44
    

    View Slide

45. View Slide

46. View Slide

47. Machine Learning
    Pipeline
    In-
    Processing
    Pre-
    Processing
    Post-
    Processing
    47
    Modifying the
    training data.
    Modifying the
    learning
    algorithm.
    Modifying the
    predictions (or
    outcomes.)
    

    View Slide

48. View Slide

49. http://aif360.mybluemix.net/
    Demos
    

    View Slide

50. https://github.com/IBM/AIF360
    AI Fairness 360 Toolkit Public Repo
    50
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

51. http://aif360.mybluemix.net/community
    AI Fairness 360 Toolkit Slack
    51
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

52. Tool #2:
    Model Asset
    Exchange
    Open Source Pre-Trained
    Deep Learning Models
    52
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

53. Step 1: Find a model
    ...that does what you need
    ...that is free to use
    ...that is performant enough
    53
    

    View Slide

54. Step 2: Get the code
    Is there a good implementation available?
    ...that does what you need
    ...that is free to use
    ...that is performant enough
    54
    

    View Slide

55. Step 3: Verify the
    model
    ○ Does it do what you need?
    ○ Is it free to use (license)?
    ○ Is it performant enough?
    ○ Accuracy?
    55
    

    View Slide

56. Step 4: Train the model
    56
    

    View Slide

57. Step 4: Train the model
    57
    

    View Slide

58. Step 5: Deploy your
    model
    ○ Adjust inference code (or write from
    scratch)
    ○ Package inference code, model code, and
    pre-trained weights together
    ○ Deploy your package
    58
    

    View Slide

59. Step 6: Consume your
    model
    59
    

    View Slide

60. Model Asset
    Exchange
    The Model Asset Exchange (MAX) is a one
    stop shop for developers/data scientists to
    find and use free and open source deep
    learning models
    ibm.biz/model-exchange
    60
    

    View Slide

61. ○ Wide variety of domains (text, audio,
    images, etc)
    ○ Multiple deep learning frameworks
    ○ Vetted and tested code/IP
    ○ Build and deploy a model web service in
    seconds
    61
    Model Asset
    Exchange
    

    View Slide

62. ibm.biz/model-exchange
    62
    

    View Slide

63. http://ibm.biz/model-exchange
    http://ibm.biz/max-slack
    Model Asset eXchange (MAX)
    63
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

64. Dedicated Open
    Source Efforts
    IBM Center for
    Open-Source Data and AI
    Technologies (CODAIT)
    64
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

65. Center for Open
    Source Data and AI
    Technologies
    CODAIT aims to make AI
    solutions dramatically easier
    to create, deploy, and manage
    in the enterprise.
    40 open source
    developers!
    65
    CODAIT
    codait.org
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

66. Improving Enterprise AI lifecycle in Open
    Source
    

    View Slide

67. 67
    Or is it?
    That’s a lot of
    open source
    developers!
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

68. Active IBM Contributors to Open Source
    (Committed code on Github in 2018)
    68
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

69. Active IBM Users of Open Source
    (Certified to consume and/or contribute open source
    in 2018)
    69
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

70. http://ibm.biz/codait-projects
    IBM Center for Open Source Data and
    AI Technologies (CODAIT)
    70
    May 8, 2019 / © 2019 IBM Corporation
    

    View Slide

71. UPDATE TO
    THE GENDER
    SHADES
    PROJECT
    71 Image Credit: #WOCinTech
    

    View Slide

72. 72
    http://www.aies-conference.com/wp-content/uploads/2019/01/AIES-19_paper_223.pdf
    

    View Slide

73. 73
    https://www.ajlunited.org/fight
    

    View Slide

74. 74
    Photo by rawpixel on Unsplash
    No matter what it is our
    responsibility to build
    systems that are fair.
    

    View Slide

75. 75
    https://w3.ibm.com/developer/callforcode/
    

    View Slide

76. THANKS!
    Any questions?
    @Mo_Mack
    76
    

    View Slide