Serverless AI Services From AWS: Polly, Lex, Rekognition and Translate

AI is penetrating into our lives, but developing AI is hard. Using AI services like Polly, Lex, Rekognition and Translate from AWS, it is super easy to add AI to your mobile and web applications.

These are slides of my talk at DevOps Turkey Meetup on Jul 12th 2018,
https://www.meetup.com/tr-TR/Istanbul-DevOps-Turkey-Meetup/events/251715924/

For more information on , see my blog
https://medium.com/@ceyhun.ozgun/you-cant-fail-to-learn-a-new-language-with-these-aws-ai-services-translate-rekognition-polly-caa5807dec36

Transcript

1. Serverless AI Services from AWS Polly, Lex, Rekognition and Translate Ceyhun ÖZGÜN
2. Cloud Architect at CyberSoft 20+ years of experience Developed in-house infrastructure frameworks Hacker Noon writer Not an AI Expert !!! ceyhunozgun.blogspot.com.tr www.linkedin.com/in/ceyhun-ozgun medium.com/@ceyhun.ozgun github.com/ceyhunozgun Who am I?
3. What will I tell? What is (not) Serverless? What is AI? Serverless AI Services Amazon Polly Amazon Lex Amazon Rekognition Amazon Translate
4. What is Serverless? Serverless architecture, refers to applications that • Use third-party (managed) services, • Run code in ephemeral containers (Functions as a Service) In both cases, we don’t Create and Manage Servers.
5. What is (not) Serverless? Layer Not Serverless Serverless Compute EC2 Lambda Database Cassandra on EC2 MySql on EC2 or RDS DynamoDB Messaging ActiveMQ on EC2 SQS Artificial Intelligence Apache MXNet on EC2 Polly, Lex and Rekognition Whole App Compute Serverless + Database on EC2 Serverless on all Layers
6. What is AI? Software that can perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages. Examples include: • Driverless cars • Virtual assistants (Apple Siri, Amazon Alexa) • AI that beats humans in chess, Go, Dota 2, Ms Pac-Man, StarCraft 2
7. Serverless AI Services Service Function Introduced in Polly Text To Speech re:Invent 2016 Lex Speech Recognition & Chat Bot re:Invent 2016 Rekognition Face Recognition, Object Detection re:Invent 2016 Comprehend Natural Language Processing re:Invent 2017 Translate Language Translation re:Invent 2017 Transcribe Speech To Text re:Invent 2017
8. Amazon Polly
9. Life like text to speech service 25 Languages, 50 Voices Easy to use Smart (dates, acronyms, etc) Speech Synthesis Markup Language (SSML) support Amazon Polly
10. • “St John and his son broadcasts live from Paris, but they live in London St” • “Market grew by > 20%” • “Merhaba, ben Filiz. 03/04/2018 tarihinde ODTÜ Teknokent'te olacağım.” How AI is used?
11. Use cases • News reader (Washington Post) • E-Learning and education (Duolingo) • Games • Call centers • IoT devices • Notifications Sample use case: Notifying a user with voice message in a web app. For more info, see my blog post Look Who's Talking!. Creating a Web Application That Talks with Amazon Polly
12. How does it work? $ aws polly synthesize-speech \ --output-format mp3 \ --voice-id Filiz \ --text "Merhaba, ben Filiz. 03/04/2018 tarihinde Teknokent'te olacağım." \ hello.mp3
13. Demo
14. Amazon Lex
15. Amazon Lex Service for building conversational interfaces using voice and text, ie chatbots Same deep learning engine that powered Alexa For web and mobile apps Integrated with messaging platforms like Facebook, Slack and Twilio Integrated with AWS Lambda English only, for now
16. How is AI used? • Automated Speech Recognition (ASR) • Natural Language Understanding (NLU)
17. Use cases • Informational bots ◦ News updates, weather information, ... • Application bots ◦ Book a hotel, order food, … • Enterprise productivity bots ◦ Sales numbers, inventory status, … • Customer support Sample use case: Using a web app with voice commands. For more information, see my blog post Nobody Understands You? There's An App For That, with Amazon Lex
18. Bot Structure
19. Flow Amazon Lex
20. Demo BookRegistry Bot for registering the books Saves book name and page count read
21. BookRegistryIntent Utterances
22. BookRegistryIntent Slots
23. BookRegistryIntent Confirmation Prompt
24. BookRegistryIntent Fulfillment
25. BookRegistry Bot Test
26. BookRegistry Bot Test with AWS CLI $ aws polly synthesize-speech --output-format pcm --voice-id "Joanna" \ --text "I have read Three Little Pigs" lexInput.mpg $ aws lex-runtime post-content --bot-name BookRegistry \ --bot-alias "\$LATEST" --region eu-west-1 --user-id UserOne \ --content-type "audio/l16; rate=16000; channels=1" \ --input-stream lexInput.mpg lexOutput.mpg { "contentType": "audio/mpeg", "slotToElicit": "PageCount", "dialogState": "ElicitSlot", "intentName": "BookRegistryIntent", "inputTranscript": "i have read three little pigs", "slots": { "BookName": "three little pigs", "PageCount": null }, "message": "How many pages?" }
27. Amazon Rekognition
28. Deep learning based image and video analysis service • Object and scene detection, facial analysis, facial recognition, face comparison, text detection • Person tracking and activity detection Integrated with S3 Fast, near real time face recognition Easy to use Amazon Rekognition
29. How is AI used? • Deep Learning • Convolutional neural networks (CNNs)
30. Use cases • Image classification (Pinterest) • Amscreen analyses the age and gender of shoppers at checkout • Alipay is aiming to use faces like passwords. • Marinus Analytics uses to find missing people • C-SPAN indexes people talking in government proceedings Sample use case: Recording allergy information with an image of child and checking quickly. For more info, see my blog post Serverless Allergy Checker with Amazon Rekognition, Lex, Polly, DynamoDB, S3 and Lambda
32. DetectLabels with AWS CLI $ aws rekognition detect-labels --image="S3Object={Bucket=x,Name=skateboard.jpg}" { "Labels": [ { "Confidence": 99.25074005126953, "Name": "Human" }, { "Confidence": 99.25072479248047, "Name": "People" }, { "Confidence": 99.25074005126953, "Name": "Person" }, { "Confidence": 99.23777770996094, "Name": "Skateboard" }, { "Confidence": 99.23777770996094, "Name": "Sport" }, ...
34. DetectFaces with AWS CLI $ aws rekognition detect-faces --image="S3Object={Bucket=x,Name=driving_woman.jpg}" --attributes=ALL { "FaceDetails": [ { "Confidence": 99.9611587524414, "Eyeglasses": { "Confidence": 99.99998474121094, "Value": true }, "Sunglasses": { "Confidence": 87.86084747314453, "Value": true }, "Gender": { "Confidence": 100.0, "Value": "Female" }, ...
37. DetectText with AWS CLI $ aws rekognition detect-text --image="S3Object={Bucket=x, Name=exit-sign.png}" { "TextDetections": [ { ... "Confidence": 99.69648742675781, "DetectedText": "EMERGENCY", "Type": "LINE", "Id": 0 }, { ... "Confidence": 99.76488494873047, "DetectedText": "EXIT", "Type": "LINE", "Id": 1 } ...
38. IndexFaces with AWS CLI $ aws rekognition create-collection --collection-id s $ aws rekognition index-faces --collection-id=s --image="S3Object={Bucket=xxx, Name=faces.jpg}" { "FaceRecords": [ { "Face": { "FaceId": "c11e5baf-b96a-4be0-bc17-d3e01ac33b89", } }, { "Face": { "FaceId": "0e7558d0-4b07-4687-94ba-4a41742014f2", } }, { "Face": { "FaceId": "8b369be6-c8fa-4620-9dc3-ee3d1d862288", } } ] }
39. SearchFacesByImage with AWS CLI $ aws rekognition search-faces-by-image --collection-id=s --image="S3Object={Bucket=x, Name=face.jpg}" { "FaceMatches": [ { "Face": { "FaceId": "c11e5baf-b96a-4be0-bc17-d3e01ac33b89", }, "Similarity": 97.85618591308594 } ] }
40. Demo Recording allergy information with an image of child and checking quickly. For more info, see my blog post Serverless Allergy Checker with Amazon Rekognition, Lex, Polly, DynamoDB, S3 and Lambda
41. Amazon Translate
42. Deep learning based machine translation service Real-time English to Arabic, Chinese, French, German, Portuguese, Spanish Language detection Easy API, one method only Amazon Translate
43. How is AI used? • Based on Amazon Sockeye framework • Uses seq2seq, a supervised learning Deep Learning algorithm
44. Use cases • Live chat for non-English users • News reading apps • Draft translation for translation agencies Sample use case: E-learning web app for translating speech into another language. For more info, see my blog post You Can’t Fail To Learn A New Language With These AWS AI Services: Translate, Rekognition, Polly
45. TranslateText with AWS CLI $ aws translate translate-text --text "good evening" \--source-language-code en --target-language-code es { "TargetLanguageCode": "es", "TranslatedText": "Buenas noches", "SourceLanguageCode": "en" }
46. Demo E-learning web app for translating speech into another language and translating texts in pictures. For more info, see my blog post You Can’t Fail To Learn A New Language With These AWS AI Services: Translate, Rekognition, Polly
47. For more information, see my blog posts Look Who's Talking!. Creating a Web Application That Talks with Amazon Polly Nobody Understands You? There's An App For That, with Amazon Lex Serverless Allergy Checker with Amazon Rekognition, Lex, Polly, DynamoDB, S3 and Lambda You Can’t Fail To Learn A New Language With These AWS AI Services: Translate, Rekognition, Polly