Building Intelligent and Conversational Apps in the Cloud

How do you build an application that is at the same time scalable, maintainable, cost-effective, smart, and conversational? The Cloud has the potential to bring natural language processing, automatic speech recognition, text-to-speech, and neural machine translation technologies within reach of every developer.

In this session, I will discuss how you can easily add intelligence to any application with solution-oriented machine learning services that provide speech, language, and chatbot functionalities on AWS. Additionally, we will explore the capabilities of new platforms and devices such as Amazon Alexa and AWS DeepLens.

Transcript

1. Alex Casalboni Technical Evangelist, AWS @alex_casalboni @ 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved Building Intelligent and Conversational Apps in the Cloud
2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. About me • Software Engineer & Web Developer • Serverless Lover & AI Enthusiast • AWS Customer since 2013
3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Human-machine interfaces (HMI / HCI)
5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Language capabilities Speech-to-Text (STT) Text-to-Speech (TTS) Natural Language Understanding Language Translation Uniform programming interface for text and voice apps
6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Language capabilities on AWS Speech-to-Text (STT) Text-to-Speech (TTS) Natural Language Understanding Language Translation Uniform programming interface for text and voice apps Amazon Transcribe Amazon Polly Amazon Comprehend Amazon Translate Amazon Lex
7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Common language use cases Information Bots Education Accessibility Knowledge Management Voice of Customer Applications Customer Service/ Call Centers Enterprise Digital Assistant Semantic Search Captioning Workflows Localization Personalization
8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. The AWS machine learning stack FRAMEWORKS / INTERFACES KERAS PLATFORMS APPLICATION SERVICES A M A Z O N R E K O G N I T I O N A M A Z O N R E K O G N I T I O N V I D E O A M A Z O N P O L L Y A M A Z O N T R A N S C R I B E A M A Z O N T R A N S L A T E A M A Z O N C O M P R E H E N D A M A Z O N L E X AMAZON SAGEMAKER AMAZON MECHANICAL TURK
9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Put AI in the hands of every developer and data scientist Machine Learning at AWS: Our mission
10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do you make your applications conversational?
11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Lex A service for building conversational interfaces into your applications using voice and text
12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Lex End to End Speech to Intent ASR+NLU integrated into one API Dialog Management Native support & maintains context Text to Speech Amazon Polly integrated into API Business Logic Native integration with AWS Lambda Deployment One click deployment Security Encrypted data in transit & at rest Scale Completely managed service Analytics Monitor and improve End to End
13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Lex Social Intelligence Personality Modality Dynamic Conversation • Built-in error handling for unexpected scenarios • Intent chaining lets you build a conversation flow based on user input • Rich formatting capabilities for a more engaging experience • Create conversation combining speech and text in the same session • Select from multiple prompts to give your bot a personality • Choose from a range of voices depending on target audience • Use AWS Lambda to personalize the conversation based on user’s profile • Context is maintained allowing you to build intelligent conversations
14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Lex – use cases Informational Bots Chatbots for everyday consumer requests Application Bots Build powerful interfaces to mobile applications • News updates • Weather information • Game scores …. • Book tickets • Order food • Manage bank accounts …. Enterprise Productivity Bots Streamline enterprise work activities and improve efficiencies • Check sales numbers • Marketing performance • Inventory status …. Internet of Things (IoT) Bots Enable conversational interfaces for device interactions • Wearables • Appliances • Auto …. Contact Center Bots Chatbots for customer service IVR • Account inquiries • Bill payment • Service update ….
15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
16. https://www.capitalone.com/applications/alexa/
17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. < demo time >
18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do you give your applications a voice?
19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Polly Text-to-speech service based on advanced deep learning technologies to synthesize speech that sounds like a human voice
20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Polly
21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Polly • Content creation • Mobile & desktop applications • Internet of Things (IoT) • Education & e-learning • Telephony • Game development Use cases • 54 voices across 26 languages • Lip-syncing & text highlighting • Fine-grained voice control • Custom vocabularies • Available in 16 AWS Regions Key features
22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Polly – Use cases Voiced Notifications AI Radio Announcer Special Needs AI Assistant Amazon Polly Navigation Video Creation Podcasting
23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. “With Amazon Polly our users benefit from the most lifelike Text-to-Speech voices available on the market.” Severin Hacker CTO, Duolingo
24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do you extract insights from unstructured text?
25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Comprehend A fully managed and continuously trained service that helps you extract insights from unstructured text
26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Comprehend Sentiment Entities Languages Keyphrases Topic modeling Syntax
27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Comprehend – Natural Language Processing A m a z o n . c o m , I n c . i s l o c a t e d i n S e a t t l e , W A a n d w a s f o u n d e d J u l y 5 , 1 9 9 4 b y J e f f B e z o s . O u r c u s t o m e r s l o v e b u y i n g e v e r y t h i n g f r o m b o o k s t o b l e n d e r s a t g r e a t p r i c e s N a m e d E n t i t i e s • A m a z o n . c o m : O r g a n i z a t i o n • S e a t t l e , W A : L o c a t i o n • J u l y 5 t h , 1 9 9 4 : D a t e • J e f f B e z o s : P e r s o n K e y p h r a s e s • O u r c u s t o m e r s • b o o k s • b l e n d e r s • g r e a t p r i c e s S e n t i m e n t • P o s i t i v e L a n g u a g e • E n g l i s h
28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Comprehend – Syntax API O u r c u s t o m e r s l o v e b u y i n g e v e r y t h i n g f r o m b o o k s t o b l e n d e r s a t g r e a t p r i c e s Token (word) Part of Speech customers Noun love Verb books Noun great Adjective prices Noun
29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Supported parts of speech ADJ – Adjective ADP – Adposition ADV – Adverb AUX – Auxiliary CCONJ – Coordinating Conjunction DET – Determiner INTJ - Interjection NOUN - Noun NUM – Numeral O – Other PART – Particle PRON – Pronoun PROPN – Proper Noun PUNCT – Punctuation SCONJ – Subordinating Conjunction SYM – Symbol VERB – Verb
30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Syntax detection $ aws comprehend detect-syntax --language-code 'en' --text 'I love cloud!’ { "SyntaxTokens": [ { "TokenId": 1, "Text": "I", "BeginOffset": 0, "EndOffset": 1, "PartOfSpeech": { "Tag": "PRON", "Score": 0.9999802112579346 } }, ...
31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Sentiment Analysis $ aws comprehend detect-sentiment --language-code 'en' --text 'I love cloud!’ { "Sentiment": "POSITIVE”, "SentimentScore": { "Mixed": 0.012617903761565685, "Positive": 0.9599817991256714, "Neutral": 0.021758323535323143, "Negative": 0.005641999188810587 } }
32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Popular text analytics use cases Content Personalization • Understand related documents based on entities, phrases or even topic similarities for trends analysis, to drive content personalization and recommendations Semantic Search • Index entities for boosting and ranking search results Intelligent data warehouse • Query unstructured data in relational databases, processing data within the data lake (Amazon S3) and then inserting it back into the data warehouse Social Analytics • Ingest, process and analyze trends from entities and sentiment from social media posts across Twitter and Facebook
33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Text analytics workload Amazon Kinesis Amazon ES Amazon Redshift Amazon EMR • Semantic • Rich filtering • Grouping, trends • Joining, correlating • Clustering • Graph, search • Near real-time • Alerts Amazon S3 Social media, support Amazon Aurora Articles, documents
34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. « Amazon Comprehend helps us analyze the key sentiments, objects, and geos in our 30 million plus reviews & testimonies. Now we are able to discover new insights into the unique experiences available at each property, so our customers can make the best decision possible for their travel.” Matt Fryer, VP and Chief Data Science Officer, Hotels.com
35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do you make your applications listen?
36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Transcribe A fully managed and continuously trained automatic speech recognition (ASR) service that takes in audio and automatically generates accurate transcripts
37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Transcribe – automatic speech recognition Time stamps and confidence scores Support for both regular and telephony audio Punctuation § Amazon S3 integration Hello Hola English and Spanish with more to come Detect multiple speakers Custom vocabulary
38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Transcribe – use cases Call centers Subtitles for VOD Broadcast closed captions Transcribe meetings
39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Audio Input Amazon S3 AWS Lambda Amazon Athena Amazon QuickSight Amazon Comprehend Amazon Transcribe – media workloads
40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. • End-to-end communications platform for sales teams. • Hundreds of enterprise organizations use RingDNA to dramatically increase productivity, engage in smarter sales conversations, gain predictive sales insights and improve their win rate. ringDNA Speech to Text "A critical component of RingDNA’s Conversation AI requires best of breed speech-to-text to deliver transcriptions of every phone call. RingDNA is excited about Amazon Transcribe since it provides high-quality speech recognition at scale, helping us to better transcribe every call to text " Howard Brown, CEO & Founder, RingDNA https://www.youtube.com/watch?v=1ZJ_f1bDdog
41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do you make your applications multilingual?
42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Translate A fully managed and continuously trained neural machine translation service that translates text from one language to another
43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Translate – features Translate text input Real-time Translation 12 languages & more to come Language detection via Amazon Comprehend AWS security standards Available in us-east-1, us-east-2, us-west-2, and eu-west-1 Easy to use (CLI & SDKs) Cost effective
44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Translate – supported languages * Arabic Simplified Chinese French German Spanish Portuguese Japanese Traditional Chinese Italian Russian Turkish Czech * Coming soon: Danish, Dutch, Finnish, Swedish, Hebrew, and Polish
45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. Use for high volume and time-sensitive information Share of worldwide digital content Machine translation • User-authored content • Text Analytics • Real-time communication • Content discovery Post-edited machine translation • Legal documents • Healthcare documents • Advertising materials
46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. «We operate 90 localized websites in 41 languages. (…) Having evaluated Amazon Translate and several other solutions, we believe that Amazon Translate presents a quick, efficient and most importantly, accurate solution. » Matt Fryer, VP and Chief Data Science Officer, Hotels.com
47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. < demo time >
48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. There’s Never Been A Better Time To Build New Businesses
49. Alex Casalboni Technical Evangelist, AWS Thank you! @alex_casalboni @ 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved