Conversational AI and Serverless

Conversational AI and Serverless

Conversational Interfaces are rapidly taking over the world. In this talk we’ll look at how Serverless technologies can be used for rapid chatbot development, specifically with Amazon Lex and Amazon Alexa. We’ll talk about lessons learned from my own experiences building an enterprise digital assistant, and how using a Natural Language Understanding interface changes how we think about and design our applications.

705df02f597ff36f900251c749e879cf?s=128

Gillian Armstrong

February 16, 2018
Tweet

Transcript

  1. Conversational AI and Serverless Gillian Armstrong

  2. 2 Liberty Information Technology Gillian Armstrong @virtualgill // Technologist, Cognitive

    Technologies – Liberty IT // Conversational AI Specialist – Workgrid Software
  3. We’re going to look specifically at Amazon Alexa and Lex,

    but lots of other chatbot platforms and cloud providers exist Disclaimer Not all Chatbots use Artificial Intelligence, but today we’re only going to talk about those that use Natural Language Understanding Gillian Armstrong @virtualgill
  4. Amazon Alexa Alexa is Amazon’s voice service, and the brain

    behind devices like Amazon Echo. Alexa provides Skills that allow developers to extent Alexa’s capabilities and provide customers a more personalised experience. Gillian Armstrong @virtualgill
  5. Amazon Lex Amazon Lex is a AWS service for building

    conversational interfaces for any application using voice and text. It uses the same conversational engine that powers Amazon Alexa, making it available to any developer to use to create their own natural language chatbots. Gillian Armstrong @virtualgill
  6. Note When you build an Alexa Skill you extend the

    functionality of Alexa, so the users have all the functionality of Alexa as well as what you add. When you build a chatbot using Amazon Lex your users only have the functionality that you provide available to them. Gillian Armstrong @virtualgill
  7. Why might you build a Chatbot? Expand call centre capacity

    on demand Chatbots don’t sleep, they can be available 24/7 Save time and money offloading simple questions or tasks to a chatbot Collect data in a friendlier format than the traditional web form Gillian Armstrong @virtualgill
  8. Because Voice is the Future User Interface Gillian Armstrong @virtualgill

  9. Why did we build a Chatbot? We work in a

    large Fortune 100 Company, this means lots of big legacy internal systems and information in lots of different places. We wanted to provide “one place to go” where you could ask your question just the same as you would to a colleague or the help desk…and get an answer, or even some get tasks done for you. Gillian Armstrong @virtualgill
  10. Let’s take a step back… Gillian Armstrong @virtualgill

  11. Utterance What the user said Slot Entities like names, places,

    dates or times Intents What the user means Fulfilment What action to take Response Natural Language Processing Natural Language Understanding Text to Speech Automatic Speech Recognition Gillian Armstrong @virtualgill
  12. Bot Training: Creating your Interaction Model Gillian Armstrong @virtualgill

  13. Lex Console Gillian Armstrong @virtualgill

  14. Lex Console Intents (functionality) available in your bot Gillian Armstrong

    @virtualgill
  15. Lex Console Sample Utterances show examples of what a user

    might say to trigger this Intent. This is not a regex match – these will be generalized, and so if the user says something similar it should still match Gillian Armstrong @virtualgill
  16. Lex Console Gillian Armstrong @virtualgill

  17. Lex Console Slots are defined showing type (place, name, time,

    custom, etc). They are also included in the sample utterances to show where in the sentence they would be used so that Lex can fill them in without prompting. Gillian Armstrong @virtualgill
  18. Fulfillment code defined. Intent and Slot matches will be sent

    to this lambda or returned directly to client. Lex Console Gillian Armstrong @virtualgill
  19. Lex Console Build to create the Interaction Model Publish to

    make the functionality available. Built-in Test Console allows you to test directly here after building, but before publishing. Gillian Armstrong @virtualgill
  20. Amazon Lex Amazon Polly Echo Lambda function INTERFACE NLP/NLU FULFILLMENT

    Alexa Skill Gillian Armstrong @virtualgill
  21. Amazon Lex Amazon Polly Echo INTERFACE NLP/NLU FULFILLMENT EXTERNAL SERVICES

    Alexa Skill API API Gillian Armstrong @virtualgill
  22. Amazon Lex Amazon Polly Echo Lambda function INTERFACE NLP/NLU FULFILLMENT

    Alexa Skill Gillian Armstrong @virtualgill
  23. Amazon Lex Amazon Polly Echo Lambda function INTERFACE NLP/NLU FULFILLMENT

    EXTERNAL SERVICES Alexa Skill API API Gillian Armstrong @virtualgill
  24. Amazon Lex Amazon Polly Echo Lambda function INTERFACE NLP/NLU FULFILLMENT

    EXTERNAL SERVICES Alexa Skill API Amazon DynamoDB API Amazon S3 Amazon CloudWatch AWS X-Ray Amazon ES Amazon Athena Amazon SNS Gillian Armstrong @virtualgill
  25. So, what does that look like for our chatbot? Gillian

    Armstrong @virtualgill
  26. CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING

    FULFILLMENT … Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE Gillian Armstrong @virtualgill
  27. Conversation as Code Gillian Armstrong @virtualgill

  28. Gillian Armstrong @virtualgill

  29. Conversation as Code - Amazon Lex Fully Validated and Repeatable

    Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments AWS CloudFormation Templates Amazon S3 deployment bucket Amazon Lex Model Building Service Botbuilder Lambda function Packaged Fulfillment Lambdas Bot JSON files Gillian Armstrong @virtualgill
  30. Conversation as Code - Amazon Lex Fully Validated and Repeatable

    Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Fully tested new functionality available in minutes Live Analytics Gillian Armstrong @virtualgill
  31. Conversation as Code - Amazon Alexa • Alexa Skill can

    be created through code, but requires the use of CLI for automation. • Alexa Skills have great analytics dashboards built in for chatbot analytics. • All backend code can still be created through CFN. • You should still handle all the monitoring of your lambdas, databases, etc.… Gillian Armstrong @virtualgill
  32. Why Serverless? Gillian Armstrong @virtualgill

  33. Serverless means you don’t pay if no-one talks to your

    bot… Gillian Armstrong @virtualgill
  34. Serverless means you don’t pay if no-one talks to your

    bot… …and lets you scale seamlessly if everyone wants to talk to your bot! Gillian Armstrong @virtualgill
  35. Lambdas are quick to develop, isolate small pieces of functionality

    and allow you to scale different parts of the conversation independently. Gillian Armstrong @virtualgill
  36. • You do not need an SSL certificate. • You

    do not need to verify that requests are coming from the Alexa service yourself. Access to execute your function is controlled by permissions within AWS instead. • Alexa encrypts its communications with Lambda utilizing TLS. For Alexa, using a lambda has additional benefits other than being low cost and having automatic scaling: Gillian Armstrong @virtualgill
  37. Serverless lets you build your architecture up from lots of

    different pieces… …and just as important remove pieces that aren’t needed! …which lets you change it around and add new pieces easily (especially when a cool new thing comes out)… Gillian Armstrong @virtualgill
  38. Being able to change quickly is vital, because you won’t

    get it right first time. Users will surprise you, you’ll want to keep adding new functionality, and getting natural language understanding to work well takes effort. Gillian Armstrong @virtualgill
  39. Be careful! Gillian Armstrong @virtualgill

  40. It’s important you know the SLA for any APIs you

    call – how fast do they return? API Lambda function Conversation (especially with Voice) needs to be fast Gillian Armstrong @virtualgill
  41. Lambda function invoke invoke invoke Cold starts can matter –

    especially if you chain Lambdas you can incur “spin up” time for each one, as well time to invoke. Conversation (especially with Voice) needs to be fast Gillian Armstrong @virtualgill
  42. Conversation (especially with Voice) needs to be fast Make sure

    everything in your architecture scales as well as your lambdas – performance test to verify! API Lambda functions Amazon DynamoDB Gillian Armstrong @virtualgill
  43. Monitor, no, seriously, monitor If your chatbot stopped working, would

    you know? If you have to wait for user complaints you’ve already lost users. Gillian Armstrong @virtualgill
  44. Chatbot Tips Gillian Armstrong @virtualgill

  45. Every time you change your model, you essentially have a

    whole new bot, no previous behaviours can be assumed, everything must be retested. Natural Language Understand is tricky. Getting a good bot interaction model with the right training data is hard. Gillian Armstrong @virtualgill
  46. Gillian Armstrong @virtualgill

  47. Gillian Armstrong @virtualgill

  48. Writing a good Conversation script is hard. You need to

    think through every flow, including error flows or the user saying something unexpected. Practice the bot conversations by ‘acting out’ the bot with another person. Design for voice first before other interfaces. Gillian Armstrong @virtualgill
  49. Getting Feedback is essential. Work to rapidly be able to

    fix what isn’t working, and add more of what is! You won’t get it right first time – listen to your users, and monitor how well intent recognition is working. Gillian Armstrong @virtualgill
  50. 50 Liberty Information Technology Gillian Armstrong @virtualgill // Technologist, Cognitive

    Technologies – Liberty IT // Conversational AI Specialist – Workgrid Software