Voice Interface Project

Uma visão geral sobre Intelligent Personal Assistants (IPA, Assistentes Pessoais Inteligentes), mais especificamente sobre interação por comandos de voz, de uma perspectiva do desenvolvimento de software e interface de usuário com as tecnologias atuais.

João Ricardo Mattos e Silva
github.com/joaoricardo000
linkedin.com/in/joaoricardo000

Transcript

1. Voice Interface Project João Ricardo
2. Apresentação João Ricardo Mattos e Silva 1988 @ Florianópolis Ciência da Computação - UFSC Backend Developer @ Cheesecake Labs github.com/joaoricardo000 joaoricardo@ckl.io
3. Intelligent Personal Assistants - IPA " A virtual assistant is a software agent that can perform tasks or services for an individual." - Wikipedia Microsoft's Office Assistant and later Cortana, Apple's Siri, Google's Google Home, Google Now (earlier Android versions had voice search via Google Search; later Google introduced Google Assistant). Amazon Alexa, Amazon's Evi (branded as Evi), the open source Lucida, Braina (application developed by Brainasoft for Microsoft Windows), Samsung's Bixby, LG G3's Voice Mate, BlackBerry's Assistant, SILVIA, HTC's Hidi, IBM's Watson (computer), Facebook's M (app) and One Voice Technologies (IVAN).
4. Intelligent Personal Assistants - IPA " A virtual assistant is a software agent that can perform tasks or services for an individual." - Wikipedia Microsoft's Office Assistant and later Cortana, Apple's Siri, Google's Google Home, Google Now (earlier Android versions had voice search via Google Search; later Google introduced Google Assistant). Amazon Alexa, Amazon's Evi (branded as Evi), the open source Lucida, Braina (application developed by Brainasoft for Microsoft Windows),Samsung's Bixby, LG G3's Voice Mate, BlackBerry's Assistant, SILVIA, HTC's Hidi, IBM's Watson (computer), Facebook's M (app) and One Voice Technologies (IVAN).
5. Utterance "Alexa, ask cheesecake who is awesome" User Interface
6. Intent "Alexa, ask cheesecake who is awesome" User Interface > Random User
7. User express an intent with an utterance. Utterances and Intents User Interface
8. "Alexa, ask cheesecake who is awesome" Software Interface
9. "Alexa, ask cheesecake who is awesome" Software Interface "wake up" word invocation name utterance
10. Software Interface > Random User Intent "Alexa, ask cheesecake who is awesome" "wake up" word invocation name utterance
11. Software Interface > Random User Intent > Server query CKL API for a random User < "João Ricardo" "Alexa, ask cheesecake who is awesome" "wake up" word invocation name utterance
12. "Alexa, ask cheesecake who is awesome" Software Interface utterances > Random User Intent who is the best caker who should win the ticket
13. Alexa, ask home bond to set speed 5 on the kitchen fan Software Interface utterance > Command Intent
14. Alexa, ask home bond to set speed 5 on the kitchen fan Software Interface utterance > Command Intent > Send RF command from the bond to the respective user's matching home appliances < "Ok"
15. Alexa, ask home bond to set speed 5 on the kitchen fan Software Interface utterance < "Ok"
16. Alexa, ask home bond to set speed 5 on the kitchen fan Software Interface utterance > Command Intent turn off the lights in the kitchen shades up turn all lights on ...
17. Alexa, ask home bond to set speed 5 on the kitchen fan Software Interface utterance > Command Intent
18. Alexa, ask home bond to set speed 5 on the kitchen fan Software Interface utterance > Command Intent {Location} Kitchen Office Garage Bedroom .... {Device} Fan Shade Light Air Conditioner ... {Command} Speed 1-5 On/Off Up/Down Reverse … Slots (or entities or parameters)
19. Alexa, ask home bond to set speed 5 on the kitchen fan Software Interface utterance > Command Intent turn off the lights in the kitchen shades up turn all lights on {Location} {Device} {Command}
20. Software Interface > Command Intent turn {Command} the {Device} in the {Location} {Device} {Command} set {Location} {Device} {Command} {Device} at the {Location} to {Command} ...
21. Software Interface POST { "user": "user_id", "intent": "CommandIntent", "parameters":{ "command": "up", "device": "shades", "location": "" } }
22. Software Interface Responses! > "Ok!" > "42" > "Sorry, could not do that" > "Please, login in the app to link your account"
23. Software Interface Questions?! < "Alexa, ask home bond shades up!" > "Which shade, the one in the office or bedroom?" < "Bedroom" > "Ok!"
24. Software Interface Questions?! < "Alexa, start cheesecake" > "Here is cheesecake, what do you want?" Welcome Intent
25. Software Interface Smart Home Skill < "Alexa, discover devices" > [{ 'type': 'fan', 'name':'kitchen fan', 'Commands':['power toggle', 'light toggle'] }]
26. Software Interface Smart Home Skill < "Alexa, turn off kitchen fan" > POST { 'name': 'kitchen fan', 'command': 'power off' }
28. Skill (or assistant) Interaction Model - Utterances - Intents - Slots
29. Thank you!
30. Voice Interface Project João Ricardo