Serverless and Chatbots (Cardiff)

The Internet tutorials promised that building a chatbot was simple, and that the magic of Serverless would make it even simpler. And it was true. But it turns out that the simple runs out very quick. By the time I discovered this I was already hooked on both Conversational AI and Serverless technologies. Conversational Interfaces let users interact with technology in a new way - and Serverless lets developers build software in a new way. Combined they bring something brand new that requires both learning new skills and applying existing skills in new ways.

Transcript

1. Chatbots and Serverless: A match made in the Cloud Gillian Armstrong @virtualgill Cardiff ServerlessDays
3. Gillian Armstrong @virtualgill Technologist Emerging Technology Liberty IT
4. 2019 5% SOURCE: YouGov Custom Research, eMarketer Smart Speaker Ownership UK Households 10% 2018 2017 22%
5. 2019 5% Smart Speaker Ownership UK Households 10% 2018 2017 22% Activations of Alexa devices crashed the servers on Christmas Day SOURCE: YouGov Custom Research, eMarketer
6. 2019 5% Smart Speaker Ownership UK Households 10% 2018 2017 22% The world is changing fast ?? SOURCE: YouGov Custom Research, eMarketer
7. So we need to learn fast The world is changing fast
8. Change Fast Learn Fast use what we learn to so we can keep on learning
9. Change Fast Learn Fast Both Chatbots and Serverless let us and
10. What is a Chatbot??? ??? ??? ??? ???
11. For the purposes of this presentation... A Chatbot is something you can interact with conversationally using natural language - me
13. CHANNEL/DEVICE NLU NLP INTENTS ENTITIES I n t e r a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX A very quick overview of a Chatbot using Conversational AI
14. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Machine Learning ASR TTS Utterance (“what the user said”) Conversational User Interface Fulfillment CUX/VUX ENTITIES
15. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Conversational User Interface ENTITIES
16. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Conversational User Interface ENTITIES
17. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Intent maps to Fulfillment (“what we need to do”) Conversational User Interface ENTITIES
18. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Insert cool serverless architecture here Conversational User Interface ENTITIES Intent maps to Fulfillment (“what we need to do”)
19. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Oh hey, there’s still UI design Insert cool serverless architecture here Conversational / Voice User Experience (“how we sound and the words we say”) Conversational User Interface ENTITIES Intent maps to Fulfillment (“what we need to do”)
20. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Text to Speech Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Oh hey, there’s still UI design Insert cool serverless architecture here Conversational / Voice User Experience (“how we sound and the words we say”) Conversational User Interface ENTITIES Intent maps to Fulfillment (“what we need to do”)
21. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Text to Speech Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Oh hey, there’s still UI design Insert cool serverless architecture here Conversational / Voice User Experience (“how we sound and the words we say”) Conversational User Interface ENTITIES Intent maps to Fulfillment (“what we need to do”)
22. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Design it so a Human can understand it Train your Bot to understand Humans Write code to make it do something! 1 2 3 What we need to do… ENTITIES
23. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Design it so a Human can understand it Train your Bot to understand Humans Write code to make it do something! 1 2 3 What we need to do… Often starting point and focus ENTITIES
24. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Design it so a Human can understand it Train your Bot to understand Humans Write code to make it do something! 1 2 3 What we need to do… Often starting point and focus Hardcoded and then panic-written after that well-received demo ENTITIES
25. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Design it so a Human can understand it Train your Bot to understand Humans Write code to make it do something! 1 2 3 What we need to do… Often starting point and focus Hardcoded and then panic-written after that well-received demo Forgotten ENTITIES
26. Design it so a Human can understand it 1
27. In a chatbot, Conversation is the Interface
28. Source: http://hosanna1.com/ Not a CSS and HTML problem! Design Matters!
29. Up to now we’ve kept designing based on what came before
30. We’ve carried the concept of menus and commands into our graphical user interfaces
31. We base our web forms on the paper forms that came before
33. Model your design on a human, not a website People are the original conversational interface
34. https://bit.ly/2S8UzCc 1-minute Conversational Design Guide:
35. Train your Bot to understand Humans 2
36. Amazon Lex Console What the Web Tutorials told me building a chatbot looked like…
37. BOOK MEETING Intent Utterance SAY HELLO GET HELP Fulfillment
38. Intent Utterance As simple providing a set of sample utterances! Book Meeting at <TIME> on <DATE> Set up Appointment Create event on <DATE> Set up Meeting Book me a meeting at <TIME> … Hello Hi there Hey Good Morning … Help What can you do Help me I need help … BOOK MEETING SAY HELLO GET HELP
39. Amazon Lex Console SET OF INTENTS Represents the functionality the bot has SAMPLE UTTERANCES Examples of what a user might say to trigger this Intent. This is training data - not a regex match.
40. Amazon Lex Console SLOTS (ENTITIES) are defined showing type (place, name, time, custom, etc.) If the user does not supply this data the bot knows to prompt for it. The sample utterances show where to expect the data to be supplied in an utterance
41. FULFILLMENT location defined. Intent and Slot matches will be sent to this lambda or returned directly to client. Amazon Lex Console
42. Amazon Lex Console Publish to make the functionality live to users. Build to create the Interaction Model
43. Amazon Lex Console Built-in Test Console allows you to test directly here after building, but before publishing.
44. Amazon Lex Console Easy to connect your bot to popular channels through console.
45. Amazon Lex Console Chatbot Done…?
46. Sorry, I’m just a baby bot. I’m still learning.
47. Sorry, I’m just a baby bot. I’m still learning. are you?
48. Intent Utterance As simple providing a set of sample utterances! Book Meeting at <TIME> on <DATE> Set up Appointment Create event on <DATE> Set up Meeting Book me a meeting at <TIME> … Hello Hi there Hey Good Morning … Help What can you do Help me I need help … BOOK MEETING SAY HELLO GET HELP
49. Intent Utterance As complex as providing a set of sample utterances! Book Meeting at <TIME> on <DATE> Set up Appointment Create event on <DATE> Set up Meeting Book me a meeting at <TIME> … Hello Hi there Hey Good Morning … Help What can you do Help me I need help … BOOK MEETING SAY HELLO GET HELP Model your design on a human Get your sample utterances from your users… you wouldn’t make up other machine learning training data!
50. Track metrics and review conversation to improve your Bot Interaction Model. Learn Fast Sorry, I’m just a baby bot. I’m still learning. Change Fast
51. Look for Vocabulary used Should we add new synonyms Look for the Utterance Structure Should we add new training data Look for Interaction Patterns Can we improve the conversation flow Learn Fast Change Fast Reviewing Conversation Transcripts…
52. Write code to make it do something! 3
53. Pop a lambda here – and you’re done!
56. CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING FULFILLMENT … Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
57. CONTEXT CONVERSATION SESSION … AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB IAM AWS X-Ray Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
58. CONTEXT CONVERSATION SESSION NLP + INTENT MAPPING … Amazon Lex AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB IAM Amazon Polly AWS X-Ray Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
59. CONTEXT CONVERSATION SESSION NLP + INTENT MAPPING FULFILLMENT … Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
60. CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING FULFILLMENT … Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
62. Number of conversations Length of conversations Functionality being used Number of Users What people want us to add Where we are getting it right Where we are getting it wrong Examples of our Conversational Analytics
63. CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING FULFILLMENT … Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
64. Observability is critical This lets you quickly learn where there might be problems with your system.
65. Why Serverless??
66. Lower Cost lets us Change Fast Learn Fast Off-loading Maintenance lets us
67. Being able to change quickly is vital, so you can keep improving your bot’s understanding and add new functionality.
68. Serverless means you don’t pay if no-one talks to your bot…
69. Serverless means you don’t pay if no-one talks to your bot… …and lets you scale seamlessly if everyone wants to talk to your bot!
70. Lambdas are quick to develop, isolate small pieces of functionality and allow you to scale different parts of the conversation independently.
71. Both Serverless and Chatbots might require new ways of thinking about your architecture.
72. A good human conversation is synchronous
73. A good human conversation is synchronous You wait for a response
74. A good human conversation is synchronous You wait for a response You expect a response
75. A good human conversation is synchronous You wait for a response You expect a response immediately
76. A Conversational Architecture needs to - Be fast! - Appear synchronous to the user (there should be a response) This is particularly critical for a voice-based bot
77. invoke invoke invoke Combining the time to invoke and the time to ‘spin up’ can hit performance and add complexity to your system. Before you “keep warm” look at combining lambdas or pulling reusable code into libraries rather separate lambdas Be careful how far you decompose…
78. Ensure that everything scales - or at least know your limits. API Lambda functions Amazon DynamoDB
79. CHANGE FAST LEARN FAST We get there iteratively by being able to and
80. CHANGE FAST LEARN FAST We get there iteratively by being able to and but Good Supporting Engineering Practices are needed to do both with confidence and at scale
81. Amazon Lex Console What the Web Tutorials told me building a chatbot looked like…
82. CODE HERE Fulfillment in Code
83. CODE HERE CODE HERE Bot Interaction Model in Code
84. CODE HERE CODE HERE CODE HERE Design in Code
85. CODE HERE CODE HERE CODE HERE CODE HERE CODE HERE CODE HERE CODE HERE Infrastructure in Code
86. CODE HERE EVERYTHING in Code CODE HERE CODE HERE CODE HERE CODE HERE CODE HERE CODE HERE
88. CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING FULFILLMENT … Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
89. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Local and Pre-Commit Hook Tests • Unit Test • Lambdas • Libraries • Static analysis • Cloud Formation • Code • JSON for API calls
90. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments Build Tests • Unit Test • Lambdas • Libraries • Static analysis • Cloud Formation • Code • JSON for API calls
91. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately
92. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments AWS CloudFormation Templates Amazon S3 deployment bucket Amazon Lex Model Building Service Botbuilder Lambda function Packaged Fulfillment Lambdas Bot JSON files
93. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Deploy Tests • Smoke Tests • Component Tests (Lex) Non-Prod Only
94. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Fully tested new functionality available in minutes
95. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Live Analytics ‘Live’ Tests - APM • Alarms • Cloudwatch metrics and Dashboards • Centralized Logging • Tracing
96. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments Live Analytics ‘Live’ Tests - APM • Alarms • Cloudwatch metrics and Dashboards • Centralized Logging • Tracing
97. Know Fast Code Quality Conversational Analytics User Feedback Application Monitoring Learn Fast FAST FEEDBACK LOOP
98. Conversational UI is about Listening to what the user wants to do… CHANGE FAST LEARN FAST
99. Conversational UI is about Listening to what the user wants to do… CHANGE FAST LEARN FAST …and that’s the secret to a great chatbot!
100. Got questions? Let’s chat! @virtualgill KEY TAKEAW AYS • Model your Chatbot UI and Architecture based on observing Humans and real Conversation. Break out of old mindsets! • New tech still needs the good engineering practices you already know virtualgill.io