Building voice enabled experiences with Alexa

Transcript

3. People converse automatically. Computers don’t.
4. What distinguishes conversation from simple communication?
5. Human conversational partners can: • construct meaning based on shared context • evolve the data sets upon which they draw • converge on increased understanding of each other's intentions • remember novel information beyond a given conversational session
6. VOICE REPRESENTS THE NEXT MAJOR DISRUPTION IN COMPUTING
7. WHAT IS VOICE COMPUTING?
8. Voice computing is the discipline that develops hardware or software to process voice inputs.
9. WHAT IS ALEX A?
10. Amazon Alexa, also known simply as Alexa, is a virtual assistant AI technology which can play audio, control your smart home, answer questions, engage your favorite services and much more.
11. THEN WHAT ABOUT CONVERSATIONAL AI?
12. Conversational AI is the set of technologies behind automated messaging and speech-enabled applications that offer human-like interactions bet ween computers and humans.
13. NOW, TELL ME SOMETHING ABOUT VUI.
14. A Voice-User Inter face( VUI) makes spoken human interaction with computers possible, using speech recognition to understand spoken commands and answer questions, and t ypically tex t to speech to give a reply.
15. WHAT IS ASK & SDK THEN?
16. The Alexa Skills Kit (ASK) is a sof t ware development framework that provides self-ser vice APIs and tools which you can use to build Alexa skills. Sof t ware Development Kits (SDKs) are sof t ware development tools and libraries that give you programmatic access to Alexa features for building custom skills.
17. HOW ABOUT A QUICK INTRODUCTION OF WAKE WORD, INVOCATION NAME, INTENT, SLOT, UT TER ANCE ?
18. WE ARE ALL SET TO EXPLORE NOW…
19. 70s 80s 90s 00s Present Graphical User Interface (GUI) Character Mode Voice User Interface (VUI) Mobile Web
20. HUMAN ACCURACY 50% 55% 60% 62% 70% 95% ASR accuracy has dramatically increased in the last 4-5 years. This inflection point has created sustained momentum in consumer adoption of voice technology Source: MindMeld
21. Alexa is Growing Fast P R O D U C T S W I T H A L E X A B U I L T - I N S U P P O R T E D C O U N T R I E S A L E X A S K I L L S A L E X A D E V E L O P E R S P R O D U C T S T H A T W O R K W I T H A L E X A 1,00,000+ HUNDREDS OF THOUSANDS 150+ 28,000+ 15 Numbers as of Dec 2019
23. Building for Voice with Alexa
24. Creating an Alexa Skill Voice User Interface Programming Logic +
25. Creating an Alexa Skill Voice User Interface Programming Logic + developer.amazon.com aws.amazon.com
26. Alexa Skills Kit Request Audio Response Your Service Text to Speech Machine Learning Natural Language Understanding Speech Recognition Cards
27. Wake word detection Beam forming Cards Your Service Text to Speech Machine Learning Natural Language Understanding Speech Recognition Request Response Alexa Skills Kit: Signal Processing
28. Alexa Skills Kit: Speech to text Your Service Text to Speech Machine Learning Natural Language Understanding Speech Recognition Response Cards Utterances Intents
29. ASR – Automatic Speech Recognition fo ̇ r tē tīmz
30. fo ̇ r tē tīmz Forty Times? 40x ASR – Automatic Speech Recognition
31. fo ̇ r tē tīmz Forty Times? 40x For Tea Times? ASR – Automatic Speech Recognition
32. fo ̇ r tē tīmz Forty Times? For Tea Times? For Tee Times? 40x ASR – Automatic Speech Recognition
33. fo ̇ r tē tīmz Forty Times? For Tea Times? Four Tee Times? 40x ASR – Automatic Speech Recognition
34. Alexa Skills Kit: Utterances Into Intents Your Service Text to Speech Machine Learning Natural Language Understanding Speech Recognition Response Cards Utterances Intents
35. Alexa, wake word S P E A K I N G W I T H A L E X A
36. Alexa, ask restaurant finder for breakfast wake word Invocation name launch S P E A K I N G W I T H A L E X A utterance for a good breakfast place to find places to eat in the morning a spot that serves the most important meal of the day
37. Utterances Intent Utterances and Intents RecommendationIntent to find places to eat in the morning a good breakfast place where to eat breakfast
38. Alexa, open restaurant finder invocation name Intents and Slots Recommend a restaurant within four miles slot utterance RecommendationIntent {distance: “4”} slot value intent launch wake word
39. Tell me about Italian food within four miles slot RecommendationIntent {distance: “4”} slot value intent slot {cuisine: “Italian food”} Intents and Slots
40. Alexa Skills Kit: Requests and Responses Your Service Text to Speech Machine Learning Natural Language Understanding Speech Recognition Response Cards Audio Intents
41. Alexa Skills Kit: Output Your Service Text to Speech SSML, streaming audio Machine Learning Natural Language Understanding Speech Recognition Response Cards Audio Intents
42. SSML • Speech Synthesis Markup Language • TTS (Text-to-Speech)
43. Under the hood On-Device Processing Cloud Processing Wake Word Detection Automatic Speech Recognition Natural Language Understanding Skill Text to Speech Signal Processing User: “Alexa, is it hot outside?” Audio Echo: “The weather in Delhi is:” Beamformed Signal Audio Stream Speech to Text Intent: GetWeather Location: Delhi, IN
45. Ask yourself. Why are you designing the skill?
46. While designing a skill, ensure that it should be - • User friendly
47. While designing a skill, ensure that it should be - • User friendly • Easy to remember
48. While designing a skill, ensure that it should be - • User friendly • Easy to remember • Personable
49. Prioritizing your audience
50. It’s always good to have - • General idea of the audience
51. It’s always good to have - • General idea of the audience
52. It’s always good to have - • General idea of the audience
54. Starting with the favorite thing of developers - Hello World
55. Let’s go to - https://developer.amazon.com/
56. Congratulations! You just created your first Alexa Skill.
57. Now, let’s start with Cake Walk.
58. Companion Course alexa.design/cakewalk
59. Cake Walk is a skill that celebrates your birthday! Tell it your birthday to have it count down the days. Interact with the skill on your special day to hear a happy birthday message.
61. How it all Works Your Service Text to Speech Machine Learning Natural Language Understanding Speech Recognition Response Visual Audio Request
62. Flowcharts
63. Designing for Conversation: Start with the Script
64. Designing for Conversation: Start with the Script
65. Who led the conversation? Alexa? The Customer?
66. The Customer Leads the Conversation First Turn Second Turn Situation
67. Represent Each Turn with a Card
68. Situation Card • Utterance • Situation • Response • Prompt
69. Represent Each Turn with a Card
70. From Script to Storyboard
71. Solution: Birthday Unknown - Partial
72. Identifying Necessary Information
73. Solution: Underlined Slots for Partial
75. What you’ll create 1. Cake Walk Skill 2. Interaction Model 3. Backend Resource
76. How to Host your Backend Resource • Node.js SDK • Best with Node.js v8.10 or higher • Python SDK • Java SDK • AWS Lambda function • AWS CloudWatch Logs • Amazon S3 bucket • Amazon IAM role • Amazon DynamoDB table
77. Provisioning Your Webservice
78. const handler = { canHandle(handlerInput) { // Returns true if the handler can service the request }, handle(handlerInput) { // Code to handle the request } } Understanding the SDK
80. Ask Follow-up Questions
81. How to Activate Auto-delegation 1. Define Slots 2. Mark them Required 3. Define Prompts 4. Choose a Delegation Strategy
82. Define Slots
83. Mark Required
84. Define Prompts
85. Choose a Delegation Strategy
87. Publishing Skills 1. Describe your skill 2. Provide up to three example phrases 3. Provide large and small icon • Must prove you own any IP or copyrighted material
88. Invest Time in Your Metadata Skill Description for instructions, tips, FAQs, context Skill Icon Skill Keywords for discoverability Skill Example Utterances for helping users start out
89. Leverage the Beta Functionality Give up to 500 users early access to your Skill Even just a few users will catch majority of issues – use it!
90. Alexa Skill Certification Guidelines Amazon will test your skill for: • Functionality • Content Guidelines / Policy • Security (HTTPS) • Voice Interface
91. Common Certification Failures • Incorrectly worded Example phrases • Intent response errors • Infringement on Intellectual Property • Invocation name guidelines not met • One word invocation names like “kiddo” not allowed • Doesn’t follow tokenization guidelines e.g., use k. b. c. quiz instead of kbc quiz
93. alexa.design/AWScredits AWS Promotional Credits Alexa Developer Rewards Monthly Incentive Programs alexa.design/india_rewards alexa.design/offers    Developer Incentives
94. alexa.design/AWScredits • $100 in credits when you publish your first skill • Additional $100/month if your skill incurs AWS usage charges 1 AWS Promotional Credits
95. alexa.design/india_rewards • Developers of skills with the highest customer engagement paid monthly • Only Indian developers publishing for the India Alexa Skills Store qualify • 8 eligible categories, engagement calculated based on factors like minutes of usage, new customers, recurring customers, sessions, and more 2 Developer Rewards Program
96. alexa.design/offers • Build new engaging skills and earn swag every month • Bonus Incentive – Earn Sony PlayStation 4 if your new skill published in March gets over n unique customers 3 Monthly Developer Incentives
98. Proactive Events API alexa.design/proactive
99. Reminders API alexa.design/reminders
100. Customer Profile API alexa.design/profile-api
101. Customer Profile API SnapTravel KptnCook Recipes Sends SMS to confirm travel arrangements Sends Recipe recommendations to email alexa.design/profile-api
102. All New Python SDK alexa.design/python-sdk Brand New!
103. ASK Toolkit for Visual Studio Code (Beta) alexa.design/vscode
104. Fallback Intent alexa.design/fallback
105. Resources: • alexa.design/cdw • alexa.design/guide Follow Us: @AlexaDevs twitch.tv/amazonalexa
108. Thank you!