Slide 1

Slide 1 text

Behind the AI Curtain: Designing for Machine Learning Products #aicurtain Crystal Yan

Slide 2

Slide 2 text

I’m Crystal Yan I’m a designer and product manager who works with data scientists every day. You can find me at @crystalcy Hello! ✋ @crystalcy | #aicurtain

Slide 3

Slide 3 text

Terminology Cheatsheet Data Science Getting insights from data (everything from business analytics and statistics to machine learning) Artificial Intelligence Machines have intelligent behavior (goals and methods include machine learning, natural language processing, computer vision, facial recognition) Machine Learning Computers learn on their own without explicit programming, requires lots of data (ML is a method that can be applied to create models that will predict, aka predictive analytics) @crystalcy | #aicurtain

Slide 4

Slide 4 text

Today’s Agenda Introduction Why this matters Principles 1. Less is more 2. Ask the right questions 3. Writing well matters Case Study Redesigning predictive analytics scores @crystalcy | #aicurtain

Slide 5

Slide 5 text

Artificial intelligence is changing the It’s everywhere, whether you see it or not. We interact with more systems powered by AI each day. I work with data scientists and often meet designers and clients who ask, “Are algorithms here to take my job?! ” 1 @crystalcy | #aicurtain

Slide 6

Slide 6 text

The New Yorker What happens when machines out-diagnose doctors? From medicine to SaaS VC blogs Will your users trust your analysis / will they pay for it? @crystalcy | #aicurtain

Slide 7

Slide 7 text

“ “Artificial intelligence will have reached human levels by 2029. Follow that out further to say, 2045, we will have multiplied the intelligence, the human biological machine intelligence of our civilization a billion-fold.” -Ray Kurzweil Inventor @crystalcy | #aicurtain

Slide 8

Slide 8 text

“ “The field of AI has traditionally been focused on computational intelligence, not on social or emotional intelligence. Yet being deficient in EQ can be a great disadvantage in society.” -Rana el Kaliouby Cofounder of Affectiva @crystalcy | #aicurtain

Slide 9

Slide 9 text

“ “We all have a responsibility to make sure everyone - including companies, governments and researchers - develop AI with diversity in mind.” -Fei-Fei Li Professor at Stanford University and Director of AI Lab @crystalcy | #aicurtain

Slide 10

Slide 10 text

Three Principles Here are three principles for establishing greater trust in the machine learning behind your design. 2 @crystalcy | #aicurtain

Slide 11

Slide 11 text

Principle #1: Less is more. pays to hide the numbers. @crystalcy | #aicurtain

Slide 12

Slide 12 text

Principle #2: Ask the right questions Just right: How would you explain this? Too leading: Does this make sense? Do you like this? Too open-ended: What would you do? What would you call this? Most importantly: listen to their questions. @crystalcy | #aicurtain

Slide 13

Slide 13 text

Principle #3: Writing well matters. ◉ Define your audience and purpose. ◉ Set tone/personality and match to brand. ◉ Be concise. Solution first, evidence after (for those who seek it). Good writing is concise, scannable, objective, and actionable. Resources:, Letting Go of the Words (book) @crystalcy | #aicurtain

Slide 14

Slide 14 text

Redesigning Predictive Analytics Scores Case Study: FiscalNote 3 @crystalcy | #aicurtain

Slide 15

Slide 15 text

test Our design process dev & release iterate define problem ideate/ prototype @crystalcy | #aicurtain

Slide 16

Slide 16 text

We promised the world. But people had trouble understanding this score, and the company strategy changed. @crystalcy | #aicurtain

Slide 17

Slide 17 text

But...why? The to the @crystalcy | #aicurtain

Slide 18

Slide 18 text

The redesign. We hid the numbers We gave an explanation We adopted a conversational tone @crystalcy | #aicurtain

Slide 19

Slide 19 text

$MONEY a lot of revenue at the time attributed to predictive analytics 140+ number of training docs we created on our internal drive to try to explain the scores @crystalcy | #aicurtain

Slide 20

Slide 20 text

5/5 everyone could concisely articulate how they would explain the new scores to a coworker @crystalcy | #aicurtain

Slide 21

Slide 21 text

Recap Less is more In our case, it made sense to hide the numbers. Ask the right questions What you ask defines what you’ll get. Listen for insights from the questions users ask. Writing well matters Brush up on your writing skills, or risk getting left behind. Adapt Algorithms might not take your job, but you must adapt. Why matters more than what People wanted to know why we gave a particular score. In general, they preferred a less accurate human analyst over a more accurate black box. Copy > graphics Our redesign shifted focus from charts to copy. @crystalcy | #aicurtain

Slide 22

Slide 22 text

Any questions? You can find me at ◉ @crystalcy ◉ Thanks! @crystalcy | #aicurtain