Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Behind the AI Curtain - Designing for Machine Learning Products

Behind the AI Curtain - Designing for Machine Learning Products

When startups first launch, they can make the news with application of cutting edge AI - but convincing users to trust the AI is often another story. There's often also no process for integrating future AI development into product roadmaps.

This session covers three key principles for how design and data science teams can work together better to build greater trust among users. Additionally, a case study on how a design and data science team partnered to redesign predictive analytics scores powered by machine learning will illustrate those principles in practice.

Crystal C. Yan

April 13, 2017
Tweet

More Decks by Crystal C. Yan

Other Decks in Technology

Transcript

  1. Behind the AI Curtain:
    Designing for Machine Learning Products
    #aicurtain
    Crystal Yan

    View Slide

  2. I’m Crystal Yan
    I’m a designer and product manager who works
    with data scientists every day.
    You can find me at @crystalcy
    Hello! ✋
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  3. Terminology Cheatsheet
    Data Science
    Getting insights from
    data
    (everything from business
    analytics and statistics to
    machine learning)
    Artificial Intelligence
    Machines have
    intelligent behavior
    (goals and methods include
    machine learning, natural
    language processing,
    computer vision, facial
    recognition)
    Machine Learning
    Computers learn on
    their own without
    explicit programming,
    requires lots of data
    (ML is a method that can be
    applied to create models that
    will predict, aka predictive
    analytics)
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  4. Today’s Agenda
    Introduction
    Why this matters
    Principles
    1. Less is more
    2. Ask the right
    questions
    3. Writing well
    matters
    Case Study
    Redesigning
    predictive analytics
    scores
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  5. Artificial intelligence is
    changing the
    It’s everywhere, whether you see it or not. We interact with more
    systems powered by AI each day.
    I work with data scientists and often meet designers and clients who
    ask, “Are algorithms here to take my job?! ”
    1
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  6. The New Yorker
    What happens when
    machines out-diagnose
    doctors?
    From medicine to SaaS
    VC blogs
    Will your users trust your
    analysis / will they pay for it?
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide


  7. “Artificial intelligence will have reached human
    levels by 2029. Follow that out further to say,
    2045, we will have multiplied the intelligence, the
    human biological machine intelligence of our
    civilization a billion-fold.”
    -Ray Kurzweil
    Inventor
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide


  8. “The field of AI has traditionally been focused on
    computational intelligence, not on social or
    emotional intelligence. Yet being deficient in EQ
    can be a great disadvantage in society.”
    -Rana el Kaliouby
    Cofounder of Affectiva
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide


  9. “We all have a responsibility to make sure
    everyone - including companies, governments
    and researchers - develop AI with diversity in
    mind.”
    -Fei-Fei Li
    Professor at Stanford University and Director of AI Lab
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  10. Three Principles
    Here are three principles for establishing greater trust in the machine
    learning behind your design.
    2
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  11. Principle #1:
    Less is more.
    Sometimes...it pays to hide the numbers. @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  12. Principle #2:
    Ask the right questions
    Just right:
    How would you
    explain this?
    Too leading:
    Does this make
    sense?
    Do you like this?
    Too
    open-ended:
    What would you
    do? What would
    you call this?
    Most importantly: listen to their questions. @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  13. Principle #3:
    Writing well matters.
    ◉ Define your audience and purpose.
    ◉ Set tone/personality and match to brand.
    ◉ Be concise. Solution first, evidence after (for those who seek it).
    Good writing is concise, scannable, objective, and actionable.
    Resources: plainlanguage.gov, Letting Go of the Words (book)
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  14. Redesigning Predictive
    Analytics Scores
    Case Study: FiscalNote
    3
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  15. test
    Our design process
    dev &
    release
    iterate
    define
    problem
    ideate/
    prototype
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  16. We promised the world.
    But people had trouble
    understanding this score,
    and the company strategy
    changed.
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  17. But...why?
    The to the
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  18. The redesign.
    We hid the numbers
    We gave an explanation
    We adopted a
    conversational tone
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  19. $MONEY
    a lot of revenue at the time attributed to predictive analytics
    140+
    number of training docs we created on our internal drive to try to
    explain the scores
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  20. 5/5
    everyone could concisely articulate how they would explain the new scores
    to a coworker
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  21. Recap
    Less is more
    In our case, it made sense to
    hide the numbers.
    Ask the right questions
    What you ask defines what
    you’ll get. Listen for insights from
    the questions users ask.
    Writing well matters
    Brush up on your writing skills,
    or risk getting left behind.
    Adapt
    Algorithms might not take your
    job, but you must adapt.
    Why matters more than what
    People wanted to know why we
    gave a particular score. In
    general, they preferred a less
    accurate human analyst over a
    more accurate black box.
    Copy > graphics
    Our redesign shifted focus from
    charts to copy.
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide

  22. Any questions?
    You can find me at
    ◉ @crystalcy
    ◉ crystalcyan.github.io
    Thanks!
    @crystalcy | #aicurtain
    crystalcyan.github.io

    View Slide