Upgrade to Pro — share decks privately, control downloads, hide ads and more …

3 Paths to Build a Conversational App

elainedb
December 07, 2017

3 Paths to Build a Conversational App

Talk given at Sfeir on December 07th 2017

elainedb

December 07, 2017
Tweet

More Decks by elainedb

Other Decks in Technology

Transcript

  1. 3 Paths to Build a
    Conversational App
    And some building blocks too…!
    Elaine Dias Batista
    07-dec-2017

    View full-size slide

  2. Classic ways
    Noob ways
    (no code required)
    Non Google Assistant
    ways (but intelligent
    nonetheless!)

    View full-size slide

  3. Classic ways

    View full-size slide

  4. Source: https://dialogflow.com/docs/agents

    View full-size slide

  5. User Input + Device
    Output + Device(s)
    NLP / NLU
    Fulfillment

    View full-size slide

  6. "Order a cheese pizza"
    "Pizza ordered"
    Intent: order
    Entity: cheese pizza
    Pizzeria backend Webhook

    View full-size slide

  7. User Input + Device
    Output + Device(s)
    NLP / NLU
    Fulfillment

    View full-size slide

  8. User Input + Device

    View full-size slide

  9. NLP / NLU
    ➔ Intent detection
    ➔ Entity extraction
    "Order a cheese pizza"
    Intent: order
    Entity: cheese pizza

    View full-size slide

  10. NLP? / NLU?
    Actions SDK
    ➔ Intent detection
    ➔ Entity extraction
    One shot, done by the
    Google Assistant itself
    ➔ Custom NLP / NLU algorithm
    ➔ Library
    Just before your app is launched Inside your app

    View full-size slide

  11. Fulfillment?
    ActionsSdkApp
    Conversation
    Request
    Conversation
    Response
    DialogflowApp
    Conversation
    Request
    Dialogflow
    Request
    Dialogflow
    Response
    Conversation
    Response

    View full-size slide

  12. Noob ways
    (no code required)

    View full-size slide

  13. if this then that (IFTTT)

    View full-size slide

  14. https://ifttt.com/create/if-google_assistant?sid=3

    View full-size slide

  15. IFTTT / Template
    User Input + Device
    Output + Device(s)

    View full-size slide

  16. Non Google Assistant
    ways (but intelligent
    nonetheless!)

    View full-size slide

  17. https://www.tensorflow.org/mobile/tflite/
    https://research.googleblog.com/2017/11/on-device-conversational-modeling-with.html

    View full-size slide

  18. TensorFlow Lite / Dialogflow SDK
    User Input + Mobile Device
    Output + Mobile Device

    View full-size slide

  19. Classic ways:
    . Actions SDK without TensorFlow
    . Actions SDK with TensorFlow
    . Dialogflow without webhook +
    Google Assistant integration
    . Dialogflow with webhook +
    Google Assistant integration
    Noob ways
    (no code required)
    . IFTTT
    . Templates
    Non Google Assistant
    ways (but intelligent
    nonetheless!)
    . Dialogflow SDK + mobile device,
    Slack, FB messenger...
    . TensorFlow Lite + mobile device
    Assistant SDK + Raspberry Pi

    View full-size slide

  20. Which path to choose?
    - Do you need an app for yourself, for others? (IFTTT or not)
    - Do you want it to work on every possible device? (design for voice only or
    screen too)
    - Google Home (voice only) / Phone, wear, chromecast (voice + text, images)
    - Does your app require a natural conversation?
    - Automation, one shot stuff: no (IFTTT, Actions SDK)
    - Games, news and others: yes (Dialogflow, Actions SDK + TensorFlow)
    - Do you need information from your DB or an API? (Webhook)
    - Do you want to integrate the Assistant on your own device? (Assistant SDK)
    - Do you want to integrate a conversational experience on a Android or iOS
    app? On Slack? On Facebook messenger? (Dialogflow SDK)

    View full-size slide

  21. Building Blocks
    1) Device that is going to detect the input
    a) Spoken voice (natural or not)
    b) Text (natural or not)
    c) Multiple choice
    Examples: Smart Speaker, Smartphone (screen), Computer (keyboard)
    2) Input. If voice → Speech to Text ("voice recognition") provided by: Google,
    Siri, Cortana, Web stuff. (otherwise → text, button input).
    3) If natural language → NLP / NLU
    - On device or Hosted
    4) Webhook
    5) Output. If voice → Text to Speech ("voice synthesis") Otherwise: text,
    image, cards...

    View full-size slide

  22. Raspberry Pi 3
    Android, iOS devices
    (app integration)
    Google Assistant devices
    (actions on google)
    Dialogflow +
    Google
    Assistant
    integration
    Actions
    SDK
    Assistant SDK
    IFTTT
    Dialogflow
    SDK
    TensorFlow
    Lite
    Templat
    e

    View full-size slide

  23. Dialogflow +
    Google
    Assistant
    integration
    Actions
    SDK
    Assistant SDK
    IFTTT
    Dialogflow
    SDK
    TensorFlow
    Lite
    Templat
    e

    View full-size slide

  24. Android, iOS devices
    (app integration)
    Google Assistant devices
    (actions on google)
    Dialogflow +
    Google
    Assistant
    integration
    Actions
    SDK
    Dialogflow
    SDK
    TensorFlow
    Lite
    Templat
    e

    View full-size slide

  25. Why do you want an Assistant / Assistive app?
    Do you just want to surf on the Assistant wave?
    Does it have intelligence (NLP/NLU) or not? Do we care?
    Is a conversational experience important for your app? Is your public ready for
    it? Can we work in an "hybrid" experience?
    Does simple voice recognition suffice? (if you already have an app)

    View full-size slide

  26. Intelligent apps can have an Assistant behaviour, but not necessarily all intelligent
    apps are assistant apps
    and vice versa.

    View full-size slide

  27. Il est beau ce cube

    View full-size slide

  28. Building blocks of an
    Assistant App
    And how to build apps with them
    Elaine Dias Batista
    07-dec-2017

    View full-size slide

  29. 9 ways to build an
    Assistant App
    (not necessarily a Google Assistant app)
    Elaine Dias Batista
    07-dec-2017

    View full-size slide

  30. Tips to Build an
    Assistant App
    Elaine Dias Batista
    07-dec-2017

    View full-size slide

  31. X ways to build an
    Intelligent App
    Elaine Dias Batista
    07-dec-2017

    View full-size slide

  32. Assistant Apps vs.
    Intelligent Apps
    Elaine Dias Batista
    07-dec-2017

    View full-size slide