Serverless and Chatbots (Cardiff)

Serverless and Chatbots (Cardiff)

The Internet tutorials promised that building a chatbot was simple, and that the magic of Serverless would make it even simpler. And it was true. But it turns out that the simple runs out very quick. By the time I discovered this I was already hooked on both Conversational AI and Serverless technologies. Conversational Interfaces let users interact with technology in a new way - and Serverless lets developers build software in a new way. Combined they bring something brand new that requires both learning new skills and applying existing skills in new ways.

705df02f597ff36f900251c749e879cf?s=128

Gillian Armstrong

January 30, 2019
Tweet

Transcript

  1. Chatbots and Serverless: A match made in the Cloud Gillian

    Armstrong @virtualgill Cardiff ServerlessDays
  2. None
  3. Gillian Armstrong @virtualgill Technologist Emerging Technology Liberty IT

  4. 2019 5% SOURCE: YouGov Custom Research, eMarketer Smart Speaker Ownership

    UK Households 10% 2018 2017 22%
  5. 2019 5% Smart Speaker Ownership UK Households 10% 2018 2017

    22% Activations of Alexa devices crashed the servers on Christmas Day SOURCE: YouGov Custom Research, eMarketer
  6. 2019 5% Smart Speaker Ownership UK Households 10% 2018 2017

    22% The world is changing fast ?? SOURCE: YouGov Custom Research, eMarketer
  7. So we need to learn fast The world is changing

    fast
  8. Change Fast Learn Fast use what we learn to so

    we can keep on learning
  9. Change Fast Learn Fast Both Chatbots and Serverless let us

    and
  10. What is a Chatbot??? ??? ??? ??? ???

  11. For the purposes of this presentation... A Chatbot is something

    you can interact with conversationally using natural language - me
  12. None
  13. CHANNEL/DEVICE NLU NLP INTENTS ENTITIES I n t e r

    a c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX A very quick overview of a Chatbot using Conversational AI
  14. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a

    c t i o n M o d e l Machine Learning ASR TTS Utterance (“what the user said”) Conversational User Interface Fulfillment CUX/VUX ENTITIES
  15. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a

    c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Conversational User Interface ENTITIES
  16. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a

    c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Conversational User Interface ENTITIES
  17. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a

    c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Intent maps to Fulfillment (“what we need to do”) Conversational User Interface ENTITIES
  18. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a

    c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Insert cool serverless architecture here Conversational User Interface ENTITIES Intent maps to Fulfillment (“what we need to do”)
  19. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a

    c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Oh hey, there’s still UI design Insert cool serverless architecture here Conversational / Voice User Experience (“how we sound and the words we say”) Conversational User Interface ENTITIES Intent maps to Fulfillment (“what we need to do”)
  20. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a

    c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Text to Speech Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Oh hey, there’s still UI design Insert cool serverless architecture here Conversational / Voice User Experience (“how we sound and the words we say”) Conversational User Interface ENTITIES Intent maps to Fulfillment (“what we need to do”)
  21. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a

    c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Text to Speech Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Oh hey, there’s still UI design Insert cool serverless architecture here Conversational / Voice User Experience (“how we sound and the words we say”) Conversational User Interface ENTITIES Intent maps to Fulfillment (“what we need to do”)
  22. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a

    c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Design it so a Human can understand it Train your Bot to understand Humans Write code to make it do something! 1 2 3 What we need to do… ENTITIES
  23. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a

    c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Design it so a Human can understand it Train your Bot to understand Humans Write code to make it do something! 1 2 3 What we need to do… Often starting point and focus ENTITIES
  24. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a

    c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Design it so a Human can understand it Train your Bot to understand Humans Write code to make it do something! 1 2 3 What we need to do… Often starting point and focus Hardcoded and then panic-written after that well-received demo ENTITIES
  25. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a

    c t i o n M o d e l Fulfillment Machine Learning ASR TTS CUX/VUX Design it so a Human can understand it Train your Bot to understand Humans Write code to make it do something! 1 2 3 What we need to do… Often starting point and focus Hardcoded and then panic-written after that well-received demo Forgotten ENTITIES
  26. Design it so a Human can understand it 1

  27. In a chatbot, Conversation is the Interface

  28. Source: http://hosanna1.com/ Not a CSS and HTML problem! Design Matters!

  29. Up to now we’ve kept designing based on what came

    before
  30. We’ve carried the concept of menus and commands into our

    graphical user interfaces
  31. We base our web forms on the paper forms that

    came before
  32. None
  33. Model your design on a human, not a website People

    are the original conversational interface
  34. https://bit.ly/2S8UzCc 1-minute Conversational Design Guide:

  35. Train your Bot to understand Humans 2

  36. Amazon Lex Console What the Web Tutorials told me building

    a chatbot looked like…
  37. BOOK MEETING Intent Utterance SAY HELLO GET HELP Fulfillment

  38. Intent Utterance As simple providing a set of sample utterances!

    Book Meeting at <TIME> on <DATE> Set up Appointment Create event on <DATE> Set up Meeting Book me a meeting at <TIME> … Hello Hi there Hey Good Morning … Help What can you do Help me I need help … BOOK MEETING SAY HELLO GET HELP
  39. Amazon Lex Console SET OF INTENTS Represents the functionality the

    bot has SAMPLE UTTERANCES Examples of what a user might say to trigger this Intent. This is training data - not a regex match.
  40. Amazon Lex Console SLOTS (ENTITIES) are defined showing type (place,

    name, time, custom, etc.) If the user does not supply this data the bot knows to prompt for it. The sample utterances show where to expect the data to be supplied in an utterance
  41. FULFILLMENT location defined. Intent and Slot matches will be sent

    to this lambda or returned directly to client. Amazon Lex Console
  42. Amazon Lex Console Publish to make the functionality live to

    users. Build to create the Interaction Model
  43. Amazon Lex Console Built-in Test Console allows you to test

    directly here after building, but before publishing.
  44. Amazon Lex Console Easy to connect your bot to popular

    channels through console.
  45. Amazon Lex Console Chatbot Done…?

  46. Sorry, I’m just a baby bot. I’m still learning.

  47. Sorry, I’m just a baby bot. I’m still learning. are

    you?
  48. Intent Utterance As simple providing a set of sample utterances!

    Book Meeting at <TIME> on <DATE> Set up Appointment Create event on <DATE> Set up Meeting Book me a meeting at <TIME> … Hello Hi there Hey Good Morning … Help What can you do Help me I need help … BOOK MEETING SAY HELLO GET HELP
  49. Intent Utterance As complex as providing a set of sample

    utterances! Book Meeting at <TIME> on <DATE> Set up Appointment Create event on <DATE> Set up Meeting Book me a meeting at <TIME> … Hello Hi there Hey Good Morning … Help What can you do Help me I need help … BOOK MEETING SAY HELLO GET HELP Model your design on a human Get your sample utterances from your users… you wouldn’t make up other machine learning training data!
  50. Track metrics and review conversation to improve your Bot Interaction

    Model. Learn Fast Sorry, I’m just a baby bot. I’m still learning. Change Fast
  51. Look for Vocabulary used Should we add new synonyms Look

    for the Utterance Structure Should we add new training data Look for Interaction Patterns Can we improve the conversation flow Learn Fast Change Fast Reviewing Conversation Transcripts…
  52. Write code to make it do something! 3

  53. Pop a lambda here – and you’re done!

  54. None
  55. None
  56. CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING

    FULFILLMENT … Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
  57. CONTEXT CONVERSATION SESSION … AWS Lambda AWS Lambda Amazon DynamoDB

    Amazon DynamoDB IAM AWS X-Ray Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
  58. CONTEXT CONVERSATION SESSION NLP + INTENT MAPPING … Amazon Lex

    AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB IAM Amazon Polly AWS X-Ray Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
  59. CONTEXT CONVERSATION SESSION NLP + INTENT MAPPING FULFILLMENT … Amazon

    Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
  60. CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING

    FULFILLMENT … Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
  61. None
  62. Number of conversations Length of conversations Functionality being used Number

    of Users What people want us to add Where we are getting it right Where we are getting it wrong Examples of our Conversational Analytics
  63. CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING

    FULFILLMENT … Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
  64. Observability is critical This lets you quickly learn where there

    might be problems with your system.
  65. Why Serverless??

  66. Lower Cost lets us Change Fast Learn Fast Off-loading Maintenance

    lets us
  67. Being able to change quickly is vital, so you can

    keep improving your bot’s understanding and add new functionality.
  68. Serverless means you don’t pay if no-one talks to your

    bot…
  69. Serverless means you don’t pay if no-one talks to your

    bot… …and lets you scale seamlessly if everyone wants to talk to your bot!
  70. Lambdas are quick to develop, isolate small pieces of functionality

    and allow you to scale different parts of the conversation independently.
  71. Both Serverless and Chatbots might require new ways of thinking

    about your architecture.
  72. A good human conversation is synchronous

  73. A good human conversation is synchronous You wait for a

    response
  74. A good human conversation is synchronous You wait for a

    response You expect a response
  75. A good human conversation is synchronous You wait for a

    response You expect a response immediately
  76. A Conversational Architecture needs to - Be fast! - Appear

    synchronous to the user (there should be a response) This is particularly critical for a voice-based bot
  77. invoke invoke invoke Combining the time to invoke and the

    time to ‘spin up’ can hit performance and add complexity to your system. Before you “keep warm” look at combining lambdas or pulling reusable code into libraries rather separate lambdas Be careful how far you decompose…
  78. Ensure that everything scales - or at least know your

    limits. API Lambda functions Amazon DynamoDB
  79. CHANGE FAST LEARN FAST We get there iteratively by being

    able to and
  80. CHANGE FAST LEARN FAST We get there iteratively by being

    able to and but Good Supporting Engineering Practices are needed to do both with confidence and at scale
  81. Amazon Lex Console What the Web Tutorials told me building

    a chatbot looked like…
  82. CODE HERE Fulfillment in Code

  83. CODE HERE CODE HERE Bot Interaction Model in Code

  84. CODE HERE CODE HERE CODE HERE Design in Code

  85. CODE HERE CODE HERE CODE HERE CODE HERE CODE HERE

    CODE HERE CODE HERE Infrastructure in Code
  86. CODE HERE EVERYTHING in Code CODE HERE CODE HERE CODE

    HERE CODE HERE CODE HERE CODE HERE
  87. None
  88. CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING

    FULFILLMENT … Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
  89. Conversation as Code - Amazon Lex Fully Validated and Repeatable

    Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Local and Pre-Commit Hook Tests • Unit Test • Lambdas • Libraries • Static analysis • Cloud Formation • Code • JSON for API calls
  90. Conversation as Code - Amazon Lex Fully Validated and Repeatable

    Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments Build Tests • Unit Test • Lambdas • Libraries • Static analysis • Cloud Formation • Code • JSON for API calls
  91. Conversation as Code - Amazon Lex Fully Validated and Repeatable

    Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately
  92. Conversation as Code - Amazon Lex Fully Validated and Repeatable

    Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments AWS CloudFormation Templates Amazon S3 deployment bucket Amazon Lex Model Building Service Botbuilder Lambda function Packaged Fulfillment Lambdas Bot JSON files
  93. Conversation as Code - Amazon Lex Fully Validated and Repeatable

    Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Deploy Tests • Smoke Tests • Component Tests (Lex) Non-Prod Only
  94. Conversation as Code - Amazon Lex Fully Validated and Repeatable

    Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Fully tested new functionality available in minutes
  95. Conversation as Code - Amazon Lex Fully Validated and Repeatable

    Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Live Analytics ‘Live’ Tests - APM • Alarms • Cloudwatch metrics and Dashboards • Centralized Logging • Tracing
  96. Conversation as Code - Amazon Lex Fully Validated and Repeatable

    Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments Live Analytics ‘Live’ Tests - APM • Alarms • Cloudwatch metrics and Dashboards • Centralized Logging • Tracing
  97. Know Fast Code Quality Conversational Analytics User Feedback Application Monitoring

    Learn Fast FAST FEEDBACK LOOP
  98. Conversational UI is about Listening to what the user wants

    to do… CHANGE FAST LEARN FAST
  99. Conversational UI is about Listening to what the user wants

    to do… CHANGE FAST LEARN FAST …and that’s the secret to a great chatbot!
  100. Got questions? Let’s chat! @virtualgill KEY TAKEAW AYS • Model

    your Chatbot UI and Architecture based on observing Humans and real Conversation. Break out of old mindsets! • New tech still needs the good engineering practices you already know virtualgill.io