Building Smart Applications with Conversational Interfaces and Language Services [AWS Summit @ Berlin]

How do you build an application that is at the same time scalable, maintainable, cost-effective, smart, and conversational? The Cloud now brings natural language processing, automatic speech recognition, text-to-speech, and neural machine translation technologies within reach of every developer. In this session, I will discuss how you can easily add intelligence to any application with solution-oriented machine learning services that provide speech, language, and chatbot functionalities on AWS.

Transcript

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Building Smart Applications with Conversational Interfaces and Language Services Alex Casalboni Technical Evangelist, AWS
2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T About me • Software Engineer & Web Developer • Data Science background • Startupper for 4.5 years • ServerlessDays Organizer • AWS Customer since 2013
3. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Put machine learning in the hands of every developer Our mission
4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Some of our machine learning customers…
5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T M L F R A M E W O R K S & I N F R A S T R U C T U R E Amazon ML Stack A I S E R V I C E S R E K O G N I T I O N I M A G E P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E X R E K O G N I T I O N V I D E O Vision Speech Chatbots A M A Z O N S A G E M A K E R B U I L D T R A I N F O R E C A S T T E X T R A C T P E R S O N A L I Z E D E P L O Y M L O P S & A L G O S F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e E C 2 P 3 & P 3 d n E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C I N F E R E N C E Language Forecasting Recommendations NEW NEW NEW NEW
6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Language capabilities Vision (OCR) Speech-to-Text (STT) Text-to-Speech (TTS) Natural Language Understanding Language Translation Chatbot Interface
7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Language capabilities on AWS Vision (OCR) Speech-to-Text (STT) Text-to-Speech (TTS) Natural Language Understanding Language Translation Chatbot Interface Amazon Rekognition & Textract Amazon Transcribe Amazon Polly Amazon Comprehend Amazon Translate Amazon Lex
8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Audio Input Example: End-to-end audio analysis store Amazon S3 trigger AWS Lambda call Amazon Transcribe in cascade Amazon Comprehend aggregate Amazon Athena analyze Amazon QuickSight
9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Example: Media analysis solution S3: Web UI aws.amazon.com/answers/media-entertainment/media-analysis-solution/ S3: Media storage Step Functions: Orchestrate analysis Elasticsearch: Search index Amazon Rekognition Video: Detect objects, scenes, faces, & celebrities AWS Elemental MediaConvert: Transcode videos Transcribe Comprehend Lambda API Gateway: REST API Lambda
10. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Rekognition + Amazon Textract
11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon Rekognition – Image and Video Analysis
12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Optical Character Recognition (OCR)
13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon Textract – OCR++
14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon Textract – How it works
15. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Turn text into lifelike speech using deep learning
16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon Polly • Content creation • Mobile & desktop applications • Internet of Things (IoT) • Education & e-learning • Telephony • Game development Use cases • 58 voices across 28 languages • Lip-syncing & text highlighting • Fine-grained voice control • Custom vocabularies • Available in 18 AWS Regions Key features
17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon Polly “Hallo! Mein Name ist…” Text-to-speech (TTS)
18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Synthesize Speech API $ aws polly synthesize-speech --text "hello" --voice-id Matthew --output-format mp3 [--lexicon-names mylex1 mylex2] output.mp3 { "ContentType": "audio/mpeg", "RequestCharacters": "11" }
19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T “With Amazon Polly our users benefit from the most lifelike Text-to-Speech voices available on the market.” Severin Hacker CTO, Duolingo
20. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Turn speech into text
21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T “Hello, this is Allan speaking” Amazon Transcribe Speech-to-text (STT)
22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T English Italian French Spanish Portuguese New! Supported languages * * more languages coming soon!
23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Use cases Call centers Subtitles for VOD Broadcast closed captions Transcribe meetings
24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T ringDNA End-to-end communications platform for sales teams Enterprise organizations use RingDNA to dramatically increase productivity, engage in smarter sales conversations, gain predictive sales insights and improve their win rate Speech to Text "A critical component of RingDNA’s Conversation AI requires best of breed speech-to-text to deliver transcriptions of every phone call. RingDNA is excited about Amazon Transcribe since it provides high-quality speech recognition at scale, helping us to better transcribe every call to text " Howard Brown, CEO & Founder, RingDNA https://www.youtube.com/watch?v=1ZJ_f1bDdog
25. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Discover insights and relationships in text
26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Sentiment Entities Languages Keyphrases Topic modeling Syntax Natural Language Understanding (NLU)
27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T English Italian French German Portuguese New! Supported Languages
28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Syntax detection $ aws comprehend detect-syntax --language-code 'en' --text 'I love cloud!’ { "SyntaxTokens": [ { "TokenId": 1, "Text": "I", "BeginOffset": 0, "EndOffset": 1, "PartOfSpeech": { "Tag": "PRON", "Score": 0.9999802112579346 } }, ...
29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Sentiment Analysis $ aws comprehend detect-sentiment --language-code 'en' --text 'I love cloud!’ { "Sentiment": "POSITIVE”, "SentimentScore": { "Mixed": 0.012617903761565685, "Positive": 0.9599817991256714, "Neutral": 0.021758323535323143, "Negative": 0.005641999188810587 } }
30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T « Amazon Comprehend helps us analyze the key sentiments, objects, and geos in our 30 million plus reviews & testimonies. Now we are able to discover new insights into the unique experiences available at each property, so our customers can make the best decision possible for their travel.” Matt Fryer, VP and Chief Data Science Officer, Hotels.com
31. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Extract health information from patient notes, clinical trial reports, and other electronic health records
32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Comprehend Medical
33. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yes, natural language translation J
34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Supported languages * Arabic Simplified Chinese French German Spanish Portuguese Japanese Traditional Chinese Italian Russian Turkish Czech * 417 translation combinations Danish Dutch Finnish Swedish Polish Hebrew
35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T «We operate 90 localized websites in 41 languages. (…) Having evaluated Amazon Translate and several other solutions, we believe that Amazon Translate presents a quick, efficient and most importantly, accurate solution. » Matt Fryer, VP and Chief Data Science Officer, Hotels.com
36. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Conversational interfaces for your applications powered by the same deep learning technologies as Alexa
37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon Lex End to End Speech to Intent ASR+NLU integrated into one API Dialog Management Native support & maintains context Text to Speech Amazon Polly integrated into API Business Logic Native integration with AWS Lambda Deployment One click deployment Security Encrypted data in transit & at rest Scale Completely managed service Analytics Monitor and improve End to End
38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T
39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon Connect Self-service, cloud-based contact center service Real time and historical analytics High-quality voice capability Call recording Skills-based routing [Automatic Call Distribution (ACD)]
40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Intelligent call center chatbot Amazon Connect Customer Amazon Lex Lambda: Fulfillment DynamoDB: Customer Data SNS: SMS Messaging Customer calls Connect to reschedule an appointment Connect calls Lex chatbot Lex chatbot calls Lambda function to get customer preferences and fulfil Intents Lambda function sends text message confirmation via SNS Customer receives appointment confirmation text message Lambda function writes updates to DynamoDB
41. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
42. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
43. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.