Conversational AI and Serverless

Conversational Interfaces are rapidly taking over the world. In this talk we’ll look at how Serverless technologies can be used for rapid chatbot development, specifically with Amazon Lex and Amazon Alexa. We’ll talk about lessons learned from my own experiences building an enterprise digital assistant, and how using a Natural Language Understanding interface changes how we think about and design our applications.

Transcript

1. Conversational AI and Serverless Gillian Armstrong
2. 2 Liberty Information Technology Gillian Armstrong @virtualgill // Technologist, Cognitive Technologies – Liberty IT // Conversational AI Specialist – Workgrid Software
3. We’re going to look specifically at Amazon Alexa and Lex, but lots of other chatbot platforms and cloud providers exist Disclaimer Not all Chatbots use Artificial Intelligence, but today we’re only going to talk about those that use Natural Language Understanding Gillian Armstrong @virtualgill
4. Amazon Alexa Alexa is Amazon’s voice service, and the brain behind devices like Amazon Echo. Alexa provides Skills that allow developers to extent Alexa’s capabilities and provide customers a more personalised experience. Gillian Armstrong @virtualgill
5. Amazon Lex Amazon Lex is a AWS service for building conversational interfaces for any application using voice and text. It uses the same conversational engine that powers Amazon Alexa, making it available to any developer to use to create their own natural language chatbots. Gillian Armstrong @virtualgill
6. Note When you build an Alexa Skill you extend the functionality of Alexa, so the users have all the functionality of Alexa as well as what you add. When you build a chatbot using Amazon Lex your users only have the functionality that you provide available to them. Gillian Armstrong @virtualgill
7. Why might you build a Chatbot? Expand call centre capacity on demand Chatbots don’t sleep, they can be available 24/7 Save time and money offloading simple questions or tasks to a chatbot Collect data in a friendlier format than the traditional web form Gillian Armstrong @virtualgill
8. Because Voice is the Future User Interface Gillian Armstrong @virtualgill
9. Why did we build a Chatbot? We work in a large Fortune 100 Company, this means lots of big legacy internal systems and information in lots of different places. We wanted to provide “one place to go” where you could ask your question just the same as you would to a colleague or the help desk…and get an answer, or even some get tasks done for you. Gillian Armstrong @virtualgill
10. Let’s take a step back… Gillian Armstrong @virtualgill
11. Utterance What the user said Slot Entities like names, places, dates or times Intents What the user means Fulfilment What action to take Response Natural Language Processing Natural Language Understanding Text to Speech Automatic Speech Recognition Gillian Armstrong @virtualgill
12. Bot Training: Creating your Interaction Model Gillian Armstrong @virtualgill
13. Lex Console Gillian Armstrong @virtualgill
14. Lex Console Intents (functionality) available in your bot Gillian Armstrong @virtualgill
15. Lex Console Sample Utterances show examples of what a user might say to trigger this Intent. This is not a regex match – these will be generalized, and so if the user says something similar it should still match Gillian Armstrong @virtualgill
16. Lex Console Gillian Armstrong @virtualgill
17. Lex Console Slots are defined showing type (place, name, time, custom, etc). They are also included in the sample utterances to show where in the sentence they would be used so that Lex can fill them in without prompting. Gillian Armstrong @virtualgill
18. Fulfillment code defined. Intent and Slot matches will be sent to this lambda or returned directly to client. Lex Console Gillian Armstrong @virtualgill
19. Lex Console Build to create the Interaction Model Publish to make the functionality available. Built-in Test Console allows you to test directly here after building, but before publishing. Gillian Armstrong @virtualgill
20. Amazon Lex Amazon Polly Echo Lambda function INTERFACE NLP/NLU FULFILLMENT Alexa Skill Gillian Armstrong @virtualgill
21. Amazon Lex Amazon Polly Echo INTERFACE NLP/NLU FULFILLMENT EXTERNAL SERVICES Alexa Skill API API Gillian Armstrong @virtualgill
22. Amazon Lex Amazon Polly Echo Lambda function INTERFACE NLP/NLU FULFILLMENT Alexa Skill Gillian Armstrong @virtualgill
23. Amazon Lex Amazon Polly Echo Lambda function INTERFACE NLP/NLU FULFILLMENT EXTERNAL SERVICES Alexa Skill API API Gillian Armstrong @virtualgill
24. Amazon Lex Amazon Polly Echo Lambda function INTERFACE NLP/NLU FULFILLMENT EXTERNAL SERVICES Alexa Skill API Amazon DynamoDB API Amazon S3 Amazon CloudWatch AWS X-Ray Amazon ES Amazon Athena Amazon SNS Gillian Armstrong @virtualgill
25. So, what does that look like for our chatbot? Gillian Armstrong @virtualgill
26. CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING FULFILLMENT … Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE Gillian Armstrong @virtualgill
27. Conversation as Code Gillian Armstrong @virtualgill
28. Gillian Armstrong @virtualgill
29. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments AWS CloudFormation Templates Amazon S3 deployment bucket Amazon Lex Model Building Service Botbuilder Lambda function Packaged Fulfillment Lambdas Bot JSON files Gillian Armstrong @virtualgill
30. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Fully tested new functionality available in minutes Live Analytics Gillian Armstrong @virtualgill
31. Conversation as Code - Amazon Alexa • Alexa Skill can be created through code, but requires the use of CLI for automation. • Alexa Skills have great analytics dashboards built in for chatbot analytics. • All backend code can still be created through CFN. • You should still handle all the monitoring of your lambdas, databases, etc.… Gillian Armstrong @virtualgill
32. Why Serverless? Gillian Armstrong @virtualgill
33. Serverless means you don’t pay if no-one talks to your bot… Gillian Armstrong @virtualgill
34. Serverless means you don’t pay if no-one talks to your bot… …and lets you scale seamlessly if everyone wants to talk to your bot! Gillian Armstrong @virtualgill
35. Lambdas are quick to develop, isolate small pieces of functionality and allow you to scale different parts of the conversation independently. Gillian Armstrong @virtualgill
36. • You do not need an SSL certificate. • You do not need to verify that requests are coming from the Alexa service yourself. Access to execute your function is controlled by permissions within AWS instead. • Alexa encrypts its communications with Lambda utilizing TLS. For Alexa, using a lambda has additional benefits other than being low cost and having automatic scaling: Gillian Armstrong @virtualgill
37. Serverless lets you build your architecture up from lots of different pieces… …and just as important remove pieces that aren’t needed! …which lets you change it around and add new pieces easily (especially when a cool new thing comes out)… Gillian Armstrong @virtualgill
38. Being able to change quickly is vital, because you won’t get it right first time. Users will surprise you, you’ll want to keep adding new functionality, and getting natural language understanding to work well takes effort. Gillian Armstrong @virtualgill
39. Be careful! Gillian Armstrong @virtualgill
40. It’s important you know the SLA for any APIs you call – how fast do they return? API Lambda function Conversation (especially with Voice) needs to be fast Gillian Armstrong @virtualgill
41. Lambda function invoke invoke invoke Cold starts can matter – especially if you chain Lambdas you can incur “spin up” time for each one, as well time to invoke. Conversation (especially with Voice) needs to be fast Gillian Armstrong @virtualgill
42. Conversation (especially with Voice) needs to be fast Make sure everything in your architecture scales as well as your lambdas – performance test to verify! API Lambda functions Amazon DynamoDB Gillian Armstrong @virtualgill
43. Monitor, no, seriously, monitor If your chatbot stopped working, would you know? If you have to wait for user complaints you’ve already lost users. Gillian Armstrong @virtualgill
44. Chatbot Tips Gillian Armstrong @virtualgill
45. Every time you change your model, you essentially have a whole new bot, no previous behaviours can be assumed, everything must be retested. Natural Language Understand is tricky. Getting a good bot interaction model with the right training data is hard. Gillian Armstrong @virtualgill
46. Gillian Armstrong @virtualgill
47. Gillian Armstrong @virtualgill
48. Writing a good Conversation script is hard. You need to think through every flow, including error flows or the user saying something unexpected. Practice the bot conversations by ‘acting out’ the bot with another person. Design for voice first before other interfaces. Gillian Armstrong @virtualgill
49. Getting Feedback is essential. Work to rapidly be able to fix what isn’t working, and add more of what is! You won’t get it right first time – listen to your users, and monitor how well intent recognition is working. Gillian Armstrong @virtualgill
50. 50 Liberty Information Technology Gillian Armstrong @virtualgill // Technologist, Cognitive Technologies – Liberty IT // Conversational AI Specialist – Workgrid Software