Slide 1

Slide 1 text

Conversational AI and Serverless Gillian Armstrong

Slide 2

Slide 2 text

2 Liberty Information Technology Gillian Armstrong @virtualgill // Technologist, Cognitive Technologies – Liberty IT // Conversational AI Specialist – Workgrid Software

Slide 3

Slide 3 text

We’re going to look specifically at Amazon Alexa and Lex, but lots of other chatbot platforms and cloud providers exist Disclaimer Not all Chatbots use Artificial Intelligence, but today we’re only going to talk about those that use Natural Language Understanding Gillian Armstrong @virtualgill

Slide 4

Slide 4 text

Amazon Alexa Alexa is Amazon’s voice service, and the brain behind devices like Amazon Echo. Alexa provides Skills that allow developers to extent Alexa’s capabilities and provide customers a more personalised experience. Gillian Armstrong @virtualgill

Slide 5

Slide 5 text

Amazon Lex Amazon Lex is a AWS service for building conversational interfaces for any application using voice and text. It uses the same conversational engine that powers Amazon Alexa, making it available to any developer to use to create their own natural language chatbots. Gillian Armstrong @virtualgill

Slide 6

Slide 6 text

Note When you build an Alexa Skill you extend the functionality of Alexa, so the users have all the functionality of Alexa as well as what you add. When you build a chatbot using Amazon Lex your users only have the functionality that you provide available to them. Gillian Armstrong @virtualgill

Slide 7

Slide 7 text

Why might you build a Chatbot? Expand call centre capacity on demand Chatbots don’t sleep, they can be available 24/7 Save time and money offloading simple questions or tasks to a chatbot Collect data in a friendlier format than the traditional web form Gillian Armstrong @virtualgill

Slide 8

Slide 8 text

Because Voice is the Future User Interface Gillian Armstrong @virtualgill

Slide 9

Slide 9 text

Why did we build a Chatbot? We work in a large Fortune 100 Company, this means lots of big legacy internal systems and information in lots of different places. We wanted to provide “one place to go” where you could ask your question just the same as you would to a colleague or the help desk…and get an answer, or even some get tasks done for you. Gillian Armstrong @virtualgill

Slide 10

Slide 10 text

Let’s take a step back… Gillian Armstrong @virtualgill

Slide 11

Slide 11 text

Utterance What the user said Slot Entities like names, places, dates or times Intents What the user means Fulfilment What action to take Response Natural Language Processing Natural Language Understanding Text to Speech Automatic Speech Recognition Gillian Armstrong @virtualgill

Slide 12

Slide 12 text

Bot Training: Creating your Interaction Model Gillian Armstrong @virtualgill

Slide 13

Slide 13 text

Lex Console Gillian Armstrong @virtualgill

Slide 14

Slide 14 text

Lex Console Intents (functionality) available in your bot Gillian Armstrong @virtualgill

Slide 15

Slide 15 text

Lex Console Sample Utterances show examples of what a user might say to trigger this Intent. This is not a regex match – these will be generalized, and so if the user says something similar it should still match Gillian Armstrong @virtualgill

Slide 16

Slide 16 text

Lex Console Gillian Armstrong @virtualgill

Slide 17

Slide 17 text

Lex Console Slots are defined showing type (place, name, time, custom, etc). They are also included in the sample utterances to show where in the sentence they would be used so that Lex can fill them in without prompting. Gillian Armstrong @virtualgill

Slide 18

Slide 18 text

Fulfillment code defined. Intent and Slot matches will be sent to this lambda or returned directly to client. Lex Console Gillian Armstrong @virtualgill

Slide 19

Slide 19 text

Lex Console Build to create the Interaction Model Publish to make the functionality available. Built-in Test Console allows you to test directly here after building, but before publishing. Gillian Armstrong @virtualgill

Slide 20

Slide 20 text

Amazon Lex Amazon Polly Echo Lambda function INTERFACE NLP/NLU FULFILLMENT Alexa Skill Gillian Armstrong @virtualgill

Slide 21

Slide 21 text

Amazon Lex Amazon Polly Echo INTERFACE NLP/NLU FULFILLMENT EXTERNAL SERVICES Alexa Skill API API Gillian Armstrong @virtualgill

Slide 22

Slide 22 text

Amazon Lex Amazon Polly Echo Lambda function INTERFACE NLP/NLU FULFILLMENT Alexa Skill Gillian Armstrong @virtualgill

Slide 23

Slide 23 text

Amazon Lex Amazon Polly Echo Lambda function INTERFACE NLP/NLU FULFILLMENT EXTERNAL SERVICES Alexa Skill API API Gillian Armstrong @virtualgill

Slide 24

Slide 24 text

Amazon Lex Amazon Polly Echo Lambda function INTERFACE NLP/NLU FULFILLMENT EXTERNAL SERVICES Alexa Skill API Amazon DynamoDB API Amazon S3 Amazon CloudWatch AWS X-Ray Amazon ES Amazon Athena Amazon SNS Gillian Armstrong @virtualgill

Slide 25

Slide 25 text

So, what does that look like for our chatbot? Gillian Armstrong @virtualgill

Slide 26

Slide 26 text

CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING FULFILLMENT … Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE Gillian Armstrong @virtualgill

Slide 27

Slide 27 text

Conversation as Code Gillian Armstrong @virtualgill

Slide 28

Slide 28 text

Gillian Armstrong @virtualgill

Slide 29

Slide 29 text

Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments AWS CloudFormation Templates Amazon S3 deployment bucket Amazon Lex Model Building Service Botbuilder Lambda function Packaged Fulfillment Lambdas Bot JSON files Gillian Armstrong @virtualgill

Slide 30

Slide 30 text

Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Fully tested new functionality available in minutes Live Analytics Gillian Armstrong @virtualgill

Slide 31

Slide 31 text

Conversation as Code - Amazon Alexa • Alexa Skill can be created through code, but requires the use of CLI for automation. • Alexa Skills have great analytics dashboards built in for chatbot analytics. • All backend code can still be created through CFN. • You should still handle all the monitoring of your lambdas, databases, etc.… Gillian Armstrong @virtualgill

Slide 32

Slide 32 text

Why Serverless? Gillian Armstrong @virtualgill

Slide 33

Slide 33 text

Serverless means you don’t pay if no-one talks to your bot… Gillian Armstrong @virtualgill

Slide 34

Slide 34 text

Serverless means you don’t pay if no-one talks to your bot… …and lets you scale seamlessly if everyone wants to talk to your bot! Gillian Armstrong @virtualgill

Slide 35

Slide 35 text

Lambdas are quick to develop, isolate small pieces of functionality and allow you to scale different parts of the conversation independently. Gillian Armstrong @virtualgill

Slide 36

Slide 36 text

• You do not need an SSL certificate. • You do not need to verify that requests are coming from the Alexa service yourself. Access to execute your function is controlled by permissions within AWS instead. • Alexa encrypts its communications with Lambda utilizing TLS. For Alexa, using a lambda has additional benefits other than being low cost and having automatic scaling: Gillian Armstrong @virtualgill

Slide 37

Slide 37 text

Serverless lets you build your architecture up from lots of different pieces… …and just as important remove pieces that aren’t needed! …which lets you change it around and add new pieces easily (especially when a cool new thing comes out)… Gillian Armstrong @virtualgill

Slide 38

Slide 38 text

Being able to change quickly is vital, because you won’t get it right first time. Users will surprise you, you’ll want to keep adding new functionality, and getting natural language understanding to work well takes effort. Gillian Armstrong @virtualgill

Slide 39

Slide 39 text

Be careful! Gillian Armstrong @virtualgill

Slide 40

Slide 40 text

It’s important you know the SLA for any APIs you call – how fast do they return? API Lambda function Conversation (especially with Voice) needs to be fast Gillian Armstrong @virtualgill

Slide 41

Slide 41 text

Lambda function invoke invoke invoke Cold starts can matter – especially if you chain Lambdas you can incur “spin up” time for each one, as well time to invoke. Conversation (especially with Voice) needs to be fast Gillian Armstrong @virtualgill

Slide 42

Slide 42 text

Conversation (especially with Voice) needs to be fast Make sure everything in your architecture scales as well as your lambdas – performance test to verify! API Lambda functions Amazon DynamoDB Gillian Armstrong @virtualgill

Slide 43

Slide 43 text

Monitor, no, seriously, monitor If your chatbot stopped working, would you know? If you have to wait for user complaints you’ve already lost users. Gillian Armstrong @virtualgill

Slide 44

Slide 44 text

Chatbot Tips Gillian Armstrong @virtualgill

Slide 45

Slide 45 text

Every time you change your model, you essentially have a whole new bot, no previous behaviours can be assumed, everything must be retested. Natural Language Understand is tricky. Getting a good bot interaction model with the right training data is hard. Gillian Armstrong @virtualgill

Slide 46

Slide 46 text

Gillian Armstrong @virtualgill

Slide 47

Slide 47 text

Gillian Armstrong @virtualgill

Slide 48

Slide 48 text

Writing a good Conversation script is hard. You need to think through every flow, including error flows or the user saying something unexpected. Practice the bot conversations by ‘acting out’ the bot with another person. Design for voice first before other interfaces. Gillian Armstrong @virtualgill

Slide 49

Slide 49 text

Getting Feedback is essential. Work to rapidly be able to fix what isn’t working, and add more of what is! You won’t get it right first time – listen to your users, and monitor how well intent recognition is working. Gillian Armstrong @virtualgill

Slide 50

Slide 50 text

50 Liberty Information Technology Gillian Armstrong @virtualgill // Technologist, Cognitive Technologies – Liberty IT // Conversational AI Specialist – Workgrid Software