Sorry, I can’t help with that: Content strategy for chatbots

It feels like everywhere you look, someone is spinning up a chatbot team. They’re captivating executives with stories of how they’ll use AI and machine learning to deliver cost savings and better customer service.

Despite all the excitement, building a good one is harder than it looks. Figuring out what types of problems your chatbot can help with is the most important part of building one, but teams often rush through that process.

Learn how to make sure your chatbot meets user and business needs. Don’t build an expensive FAQ.

In this session, you'll:
- Understand how to approach research and discovery for chatbots, based on experience leading conversation design teams for a large insurance company.
- Get frameworks and templates that will help you evaluate whether a chatbot can help in the first place.
- Learn methods for creating a chatbot strategy that will lead to a useful, usable product.

Chatbot evaluator: https://docs.google.com/spreadsheets/d/1shjUnx-xvGZ6vara8yq3B8wSTS70GVfdb65ORNXWd3w/edit?usp=sharing

Transcript

1. "Sorry, I can’t help with that."
   Content strategy for chatbots
   
   Michael J. Metts | he/him | Co-author, Writing is Designing | @mjmetts | mjmetts.com
   

   View Slide

2. Where do chatbots come from?
   

   View Slide

3. People love AI.
   Business people love being
   able to say they work on AI.
   

   View Slide

4. BotBuilder
   12:42 83%
   Hi, I'm an automated assistant. Why would
   you like to build a bot today?
   Amazon makes Alexa and Amazon is very
   successful!
   I'm sorry. It doesn't work like that.
   

   View Slide

5. 12:42 83%
   BotBuilder
   Hi, I'm an automated assistant. Why would
   you like to build a bot today?
   Google makes conversational AI and
   Google is very successful!
   I'm sorry, I don't understand. You can say
   things like "cost savings".
   12:42 83%
   

   View Slide

6. 12:42 83%
   BotBuilder
   Hi, I'm an automated assistant. Why would
   you like to build a bot today?
   We need to meet consumer expectations.
   I'm sorry, I can't help with that. Would you
   like to take a short survey about your
   experience?
   12:42 83%
   

   View Slide

7. 12:42 83%
   BotBuilder
   Hi, I'm an automated assistant. Why would
   you like to build a bot today?
   Lots of these customer service tasks seem
   easy. Can't we just automate them?
   I can help with that. I'll need to ask you a
   few questions to get started.
   12:42 83%
   

   View Slide

8. “Content strategy plans for the creation, publication,
   and governance of useful, usable content.”
   - Kristina Halvorson
   Source: https://alistapart.com/article/thedisciplineofcontentstrategy/
   

   View Slide

9. "I'm having some
   trouble right now."
   Deciding whether a
   chatbot will help.
   
   

   View Slide

10. What are our goals?
    Writing and design decisions are inseparable
    from business goals.
    

    View Slide

11. Customer service
    Types of chatbots
    Example: T-Mobile's Virtual
    Assistant
    • Responds to any kind of
    natural language
    • Helps with customer service
    requests
    • Can escalate to a human
    

    View Slide

12. Customer service
    Types of chatbots
    Possible goals:
    • Increase speed of service
    • Reduce contact center costs
    

    View Slide

13. Distinct product
    Types of chatbots
    Example: Capital One's Eno
    • Responds to natural
    language text messages
    • Heavily marketed
    • Self-contained (doesn't
    escalate to human support
    via chat)
    

    View Slide

14. Distinct product
    Types of chatbots
    Possible goals:
    • Portray Capital One as a
    technology leader
    • Increase number of
    interactions with customers
    

    View Slide

15. Routing and sales
    Types of chatbots
    Example: LiveEngage
    automated assistant
    • Gives the user pre-defined
    options to choose from
    • Goal is to connect you with
    a human
    • Offers multiple channels
    

    View Slide

16. Routing and sales
    Types of chatbots
    Possible goals:
    • Lead generation and
    retention
    • Revenue targets
    

    View Slide

17. Is it going to be useful?
    “Content strategy plans for the creation, publication,
    and governance of useful, usable content.”
    - Kristina Halvorson
    I wasn’t kidding about
    content strategy!
    Source: https://alistapart.com/article/thedisciplineofcontentstrategy/
    

    View Slide

18. How does this fit into other channels?
    Should this be
    handled on a website
    or mobile app?
    Should it be handled by
    humans via phone,
    email, or social media?
    

    View Slide

19. Pick an
    evaluation
    method.
    Focus on the
    user need.
    Source: https://www.gartner.com/en/documents/3978596/customer-service-issues-to-channel-mapping-tool
    

    View Slide

20. What could go wrong?
    Source: May-Li Khoe on Twitter - https://twitter.com/mayli/status/1305952098996097025?s=21
    

    View Slide

21. What could go wrong?
    Source: Ayodele Odubela on Twitter - https://twitter.com/DataSciBae/status/1308839841606533122
    

    View Slide

22. Ayodele Odubela’s upcoming book
    Book details: dazzlingdata.science/uncovering-bias-in-ml
    

    View Slide

23. Meet Jessica Zhang Koehler
    • Design leader
    • PhD in Cultural Theory of
    Technical Communication
    • Experience helping
    business navigate
    emerging technologies
    

    View Slide

24. So, if you're in the deciding stage, ask
    • What are our goals?
    • Is it going to be useful?
    • How does it fit into other channels?
    • What could go wrong?
    

    View Slide

25. "Here are some things
    I can help with."
    Figuring out what your
    chatbot should do.
    
    

    View Slide

26. What problem are we solving?
    Get involved in the problem space. Your
    perspective as a writer is critical.
    

    View Slide

27. Example:
    Customer service bot for
    an electric utility.
    X
    X
    Hi, I'm ElectricCo’s
    automated assistant. How
    can I help you today?
    Chat Support
    • Responds to natural language
    • Goal is cost savings and
    increased service speed
    • Available when customers are
    signed in to their accounts
    

    View Slide

28. What do people need?
    • Chat logs
    • Call recordings
    • Support emails and social
    media messages
    • Forums, reviews, and blog posts
    • Help content analytics
    How are they describing it?
    Protect users’
    privacy when
    gathering data
    

    View Slide

29. “I need to reset my password.”
    “We just moved to a new house and I need to change the
    service address but I can’t log in.”
    “CAN YOU PLEASE RESET MY LOGIN?????”
    “can’t log on to my account”
    “I need a copy of my latest bill but I can’t figure
    out how to sign in.”
    

    View Slide

30. Change billing
    due date Route to
    correct
    department
    Get help
    logging in
    Move service
    to a new
    address
    

    View Slide

31. How will we make decisions?
    Create criteria for evaluating chatbot features
    as a team.
    

    View Slide

32. Alexa guidelines
    Feature evaluation
    Voice-specific, but contains
    principles that apply to chat.
    • Is specifically about
    Amazon’s technology
    • Checklist gives you a way to
    check in
    Source: https://developer.amazon.com/en-US/docs/alexa/alexa-design/
    checklists.html
    

    View Slide

33. Google conversation
    design process
    Feature evaluation
    Designed to help teams
    understand whether
    conversation is appropriate.
    • Ties questions the team can
    discuss to benefits of
    conversational UI
    • Applies to chat and voice
    Source: https://designguidelines.withgoogle.com/conversation/conversation-
    design-process/is-conversation-the-right-fit.html#is-conversation-the-right-
    fit-take-the-quiz
    

    View Slide

34. Build your own evaluator
    Get this at go.mjmetts.com/chatbotevaluator
    

    View Slide

35. What’s the bot’s personality?
    Heavy
    personification
    No
    personification
    Capital
    One’s Eno
    Customer
    service for
    a utility
    

    View Slide

36. Where should we start?
    Get the Mural template: mural.co/templates/importance-difficulty-matrix
    

    View Slide

37. Impact
    Difficulty
    Change
    billing due
    date
    Route to
    correct
    department
    Get help
    logging in
    Move service
    to a new
    address
    

    View Slide

38. Impact
    Difficulty
    Change
    billing due
    date
    Route to
    correct
    department
    Get help
    logging in
    Move service
    to a new
    address
    

    View Slide

39. Strategic
    High value
    Short term
    Nice to have
    Impact
    Difficulty
    Change
    billing due
    date
    Route to
    correct
    department
    Get help
    logging in
    Move service
    to a new
    address 1
    2
    3
    

    View Slide

40. Meet Rebecca Evanhoe
    • Conversation designer
    • Has worked with
    conversational technology
    since 2011
    • Co-authoring a book with
    Diana Diebel called
    ‘Conversations with
    Things’ coming in 2021
    from Rosenfeld Media
    Book details: rosenfeldmedia.com/books/conversations-with-things/
    

    View Slide

41. So, if you're in the building stage, ask
    • What problem are we solving?
    • What do people need?
    • How will we make decisions?
    • What’s the bot’s personality?
    • Where should we start?
    

    View Slide

42. Thank you!
    • Check out ‘Writing is
    Designing’ by Andy Welfle
    and myself at
    writingisdesigning.com
    • See upcoming remote
    workshops at mjmetts.com
    

    View Slide