Save 37% off PRO during our Black Friday Sale! »

Building Intelligent and Conversational Apps in the Cloud

Building Intelligent and Conversational Apps in the Cloud

How do you build an application that is at the same time scalable, maintainable, cost-effective, smart, and conversational? The Cloud has the potential to bring natural language processing, automatic speech recognition, text-to-speech, and neural machine translation technologies within reach of every developer.

In this session, I will discuss how you can easily add intelligence to any application with solution-oriented machine learning services that provide speech, language, and chatbot functionalities on AWS. Additionally, we will explore the capabilities of new platforms and devices such as Amazon Alexa and AWS DeepLens.

36ca8ae7e19067021d0e39c0b72acc2a?s=128

Alex Casalboni

October 03, 2018
Tweet

Transcript

  1. Alex Casalboni Technical Evangelist, AWS @alex_casalboni @ 2018, Amazon Web

    Services, Inc. or its Affiliates. All rights reserved Building Intelligent and Conversational Apps in the Cloud
  2. © 2018, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. About me • Software Engineer & Web Developer • Serverless Lover & AI Enthusiast • AWS Customer since 2013
  3. © 2018, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Human-machine interfaces (HMI / HCI)
  4. None
  5. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Language capabilities Speech-to-Text (STT) Text-to-Speech (TTS) Natural Language Understanding Language Translation Uniform programming interface for text and voice apps
  6. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Language capabilities on AWS Speech-to-Text (STT) Text-to-Speech (TTS) Natural Language Understanding Language Translation Uniform programming interface for text and voice apps Amazon Transcribe Amazon Polly Amazon Comprehend Amazon Translate Amazon Lex
  7. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Common language use cases Information Bots Education Accessibility Knowledge Management Voice of Customer Applications Customer Service/ Call Centers Enterprise Digital Assistant Semantic Search Captioning Workflows Localization Personalization
  8. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. The AWS machine learning stack FRAMEWORKS / INTERFACES KERAS PLATFORMS APPLICATION SERVICES A M A Z O N R E K O G N I T I O N A M A Z O N R E K O G N I T I O N V I D E O A M A Z O N P O L L Y A M A Z O N T R A N S C R I B E A M A Z O N T R A N S L A T E A M A Z O N C O M P R E H E N D A M A Z O N L E X AMAZON SAGEMAKER AMAZON MECHANICAL TURK
  9. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Put AI in the hands of every developer and data scientist Machine Learning at AWS: Our mission
  10. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How do you make your applications conversational?
  11. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Lex A service for building conversational interfaces into your applications using voice and text
  12. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Lex End to End Speech to Intent ASR+NLU integrated into one API Dialog Management Native support & maintains context Text to Speech Amazon Polly integrated into API Business Logic Native integration with AWS Lambda Deployment One click deployment Security Encrypted data in transit & at rest Scale Completely managed service Analytics Monitor and improve End to End
  13. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Lex Social Intelligence Personality Modality Dynamic Conversation • Built-in error handling for unexpected scenarios • Intent chaining lets you build a conversation flow based on user input • Rich formatting capabilities for a more engaging experience • Create conversation combining speech and text in the same session • Select from multiple prompts to give your bot a personality • Choose from a range of voices depending on target audience • Use AWS Lambda to personalize the conversation based on user’s profile • Context is maintained allowing you to build intelligent conversations
  14. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Lex – use cases Informational Bots Chatbots for everyday consumer requests Application Bots Build powerful interfaces to mobile applications • News updates • Weather information • Game scores …. • Book tickets • Order food • Manage bank accounts …. Enterprise Productivity Bots Streamline enterprise work activities and improve efficiencies • Check sales numbers • Marketing performance • Inventory status …. Internet of Things (IoT) Bots Enable conversational interfaces for device interactions • Wearables • Appliances • Auto …. Contact Center Bots Chatbots for customer service IVR • Account inquiries • Bill payment • Service update ….
  15. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved.
  16. https://www.capitalone.com/applications/alexa/

  17. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. < demo time >
  18. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How do you give your applications a voice?
  19. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Polly Text-to-speech service based on advanced deep learning technologies to synthesize speech that sounds like a human voice
  20. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Polly
  21. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Polly • Content creation • Mobile & desktop applications • Internet of Things (IoT) • Education & e-learning • Telephony • Game development Use cases • 54 voices across 26 languages • Lip-syncing & text highlighting • Fine-grained voice control • Custom vocabularies • Available in 16 AWS Regions Key features
  22. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Polly – Use cases Voiced Notifications AI Radio Announcer Special Needs AI Assistant Amazon Polly Navigation Video Creation Podcasting
  23. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. “With Amazon Polly our users benefit from the most lifelike Text-to-Speech voices available on the market.” Severin Hacker CTO, Duolingo
  24. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How do you extract insights from unstructured text?
  25. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Comprehend A fully managed and continuously trained service that helps you extract insights from unstructured text
  26. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Comprehend Sentiment Entities Languages Keyphrases Topic modeling Syntax
  27. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Comprehend – Natural Language Processing A m a z o n . c o m , I n c . i s l o c a t e d i n S e a t t l e , W A a n d w a s f o u n d e d J u l y 5 , 1 9 9 4 b y J e f f B e z o s . O u r c u s t o m e r s l o v e b u y i n g e v e r y t h i n g f r o m b o o k s t o b l e n d e r s a t g r e a t p r i c e s N a m e d E n t i t i e s • A m a z o n . c o m : O r g a n i z a t i o n • S e a t t l e , W A : L o c a t i o n • J u l y 5 t h , 1 9 9 4 : D a t e • J e f f B e z o s : P e r s o n K e y p h r a s e s • O u r c u s t o m e r s • b o o k s • b l e n d e r s • g r e a t p r i c e s S e n t i m e n t • P o s i t i v e L a n g u a g e • E n g l i s h
  28. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Comprehend – Syntax API O u r c u s t o m e r s l o v e b u y i n g e v e r y t h i n g f r o m b o o k s t o b l e n d e r s a t g r e a t p r i c e s Token (word) Part of Speech customers Noun love Verb books Noun great Adjective prices Noun
  29. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Supported parts of speech ADJ – Adjective ADP – Adposition ADV – Adverb AUX – Auxiliary CCONJ – Coordinating Conjunction DET – Determiner INTJ - Interjection NOUN - Noun NUM – Numeral O – Other PART – Particle PRON – Pronoun PROPN – Proper Noun PUNCT – Punctuation SCONJ – Subordinating Conjunction SYM – Symbol VERB – Verb
  30. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Syntax detection $ aws comprehend detect-syntax --language-code 'en' --text 'I love cloud!’ { "SyntaxTokens": [ { "TokenId": 1, "Text": "I", "BeginOffset": 0, "EndOffset": 1, "PartOfSpeech": { "Tag": "PRON", "Score": 0.9999802112579346 } }, ...
  31. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Sentiment Analysis $ aws comprehend detect-sentiment --language-code 'en' --text 'I love cloud!’ { "Sentiment": "POSITIVE”, "SentimentScore": { "Mixed": 0.012617903761565685, "Positive": 0.9599817991256714, "Neutral": 0.021758323535323143, "Negative": 0.005641999188810587 } }
  32. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Popular text analytics use cases Content Personalization • Understand related documents based on entities, phrases or even topic similarities for trends analysis, to drive content personalization and recommendations Semantic Search • Index entities for boosting and ranking search results Intelligent data warehouse • Query unstructured data in relational databases, processing data within the data lake (Amazon S3) and then inserting it back into the data warehouse Social Analytics • Ingest, process and analyze trends from entities and sentiment from social media posts across Twitter and Facebook
  33. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Text analytics workload Amazon Kinesis Amazon ES Amazon Redshift Amazon EMR • Semantic • Rich filtering • Grouping, trends • Joining, correlating • Clustering • Graph, search • Near real-time • Alerts Amazon S3 Social media, support Amazon Aurora Articles, documents
  34. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. « Amazon Comprehend helps us analyze the key sentiments, objects, and geos in our 30 million plus reviews & testimonies. Now we are able to discover new insights into the unique experiences available at each property, so our customers can make the best decision possible for their travel.” Matt Fryer, VP and Chief Data Science Officer, Hotels.com
  35. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How do you make your applications listen?
  36. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Transcribe A fully managed and continuously trained automatic speech recognition (ASR) service that takes in audio and automatically generates accurate transcripts
  37. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Transcribe – automatic speech recognition Time stamps and confidence scores Support for both regular and telephony audio Punctuation § Amazon S3 integration Hello Hola English and Spanish with more to come Detect multiple speakers Custom vocabulary
  38. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Transcribe – use cases Call centers Subtitles for VOD Broadcast closed captions Transcribe meetings
  39. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Audio Input Amazon S3 AWS Lambda Amazon Athena Amazon QuickSight Amazon Comprehend Amazon Transcribe – media workloads
  40. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. • End-to-end communications platform for sales teams. • Hundreds of enterprise organizations use RingDNA to dramatically increase productivity, engage in smarter sales conversations, gain predictive sales insights and improve their win rate. ringDNA Speech to Text "A critical component of RingDNA’s Conversation AI requires best of breed speech-to-text to deliver transcriptions of every phone call. RingDNA is excited about Amazon Transcribe since it provides high-quality speech recognition at scale, helping us to better transcribe every call to text " Howard Brown, CEO & Founder, RingDNA https://www.youtube.com/watch?v=1ZJ_f1bDdog
  41. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How do you make your applications multilingual?
  42. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Translate A fully managed and continuously trained neural machine translation service that translates text from one language to another
  43. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Translate – features Translate text input Real-time Translation 12 languages & more to come Language detection via Amazon Comprehend AWS security standards Available in us-east-1, us-east-2, us-west-2, and eu-west-1 Easy to use (CLI & SDKs) Cost effective
  44. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Translate – supported languages * Arabic Simplified Chinese French German Spanish Portuguese Japanese Traditional Chinese Italian Russian Turkish Czech * Coming soon: Danish, Dutch, Finnish, Swedish, Hebrew, and Polish
  45. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Use for high volume and time-sensitive information Share of worldwide digital content Machine translation • User-authored content • Text Analytics • Real-time communication • Content discovery Post-edited machine translation • Legal documents • Healthcare documents • Advertising materials
  46. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. «We operate 90 localized websites in 41 languages. (…) Having evaluated Amazon Translate and several other solutions, we believe that Amazon Translate presents a quick, efficient and most importantly, accurate solution. » Matt Fryer, VP and Chief Data Science Officer, Hotels.com
  47. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. < demo time >
  48. © 2018, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. There’s Never Been A Better Time To Build New Businesses
  49. Alex Casalboni Technical Evangelist, AWS Thank you! @alex_casalboni @ 2018,

    Amazon Web Services, Inc. or its Affiliates. All rights reserved