The History and Future of Speaking with Machines

Ubiquitous voice interfaces like the Google Home and Amazon Alexa became available only recently. The history of humanity's attempts to converse with machines goes back centuries, however. We review this history of the effort to give machines voice. This analysis will provide context to the core concepts of modern conversational design, like Natural Language Processing and Voice User Experience. We introduce design and development concepts for building voice interfaces that work.

Transcript

1. By Matt Buck & Lauren Golembiewski The History and Future of Speaking with Machines #givevoice
2. Overview • The history of speaking with machines • The principles of conversational design • The future of speaking with machines • Demonstration of building a voice interface
3. THE HISTORY OF SPEAKING WITH MACHINES #givevoice
4. Talking with Machines Speech recognition Speech synthesis
5. • c.1219 - c.1292 • English philosopher and Franciscan friar • Proponent of the scientific method Roger Bacon
6. Brazen Heads • Automatons • Not real (obvs) • Could answer any question put to them https://en.wikipedia.org/wiki/Speech_synthesis#History
7. 1700s: Early speech synthesis • 1779: Christian Gottlieb Kratzenstein models vocal tract • 1791: Wolfgang von Kempelen’s “acoustic- mechanical speech machine” https://en.wikipedia.org/wiki/ Wolfgang_von_Kempelen's_Speaking_Machine
9. 1846: Euphonia • Created by Joseph Faber • Also played like an organ • Modeled entire head • Spoke three languages https://irrationalgeographic.wordpress.com/ 2009/06/24/joseph-fabers-talking-euphonia/
10. https://irrationalgeographic.wordpress.com/2009/06/24/joseph-fabers-talking-euphonia/ Alexander Graham Bell
11. https://www.flickr.com/photos/ 64416865@N00/5451405887/in/photostream/
12. https://www.flickr.com/photos/ 64416865@N00/5451405887/in/photostream/
13. https://www.flickr.com/photos/ 64416865@N00/5451405887/in/photostream/
14. 99% Invisible EP. 208 “VOX EX MACHINA” http://99percentinvisible.org/episode/vox-ex-machina/
15. https://www.youtube.com/watch?v=0rAyrmm7vv0 1940: Voder World’s Fair Demo http://hackaday.com/2014/08/12/retrotechtacular-the-voder-from-bell-labs/
16. https://www.youtube.com/watch?v=41U78QP8nBk 1961: The First Machine to Sing http://speechstones.com/milestones.html
17. https://www.youtube.com/watch?v=OuEN5TjYRCE 2001: Still Singing http://speechstones.com/milestones.html
18. • One of the most influential engineers of the 20th century • Inventor of the term “transistor” • “There are strong reasons for believing that spoken English is… not recognizable phoneme by phoneme or word by word.” 1969: John Robinson Pierce https://en.wikipedia.org/wiki/Speech_recognition#History
19. • Used TI’s Solid State Speech • First toy to use speech that was synthesized 1978: The Texas Instruments Speak & Spell https://commons.wikimedia.org/wiki/File:TI_SpeakSpell_no_shadow.jpg
20. 1978: The TI Speak & Spell https://www.youtube.com/watch?v=qM8FcN0aAvU
21. • Interest in speech recognition reignited by DARPA grants in early 70s • Tangora: A voice-activated word processor with a 20,000 word vocabulary 1986: IBM Tangora
23. I V R
24. Interactive Voice Response
25. • Integrated with core Apple apps + Wolfram Alpha • Solved real problems 2011: Siri
26. • Neural network capable of generating speech • Can mimic any human voice • Reduces gap in performance by over 50% 2016: Google WaveNet https://deepmind.com/blog/wavenet-generative-model-raw-audio/
27. • Ubiquitous voice interfaces • Always-on, react to “wake words” • Siri-like functionality • Control smart-home devices 2014-2016: Amazon Echo & Google Home
28. Tea. Earl Grey. Hot.
29. The History of Speaking with Machines • Roger Bacon’s Brazen Head • Early attempts at speech synthesis • The 20th century: • Demonstration of building a voice interface
30. THE PRINCIPLES OF VOICE INTERFACE DESIGN #givevoice
31. The Core Technology The Voice User Interface Automatic Speech Recognition (ASR) Natural Language Understanding (NLU) Bot Intelligence Text to Speech (TTS)
32. The Core Technology Automatic Speech Recognition (ASR) Takes the spoken word and turns it into text.
34. The Core Technology Natural Language Understanding (NLU) Gives the text meaning by turning it into structured data
35. The Core Technology Bot Intelligence Bot Intelligence manages the context of the user, the application, and the conversation.
36. The Core Technology Text to Speech (TTS) The result generated by the intelligence is spoken back to the user.
37. User Experience 
 Principles of VUIs • Define the job your bot will do • Map the conversation flow • Help users create a mental model • Define how the machine handles the job
38. What job is your bot being hired to do? USER EXPERIENCE OF VUIS https://www.intercom.com/books/jobs-to-be-done
39. Model the Conversation Flow “Show me flights leaving Atlanta next Friday after 4pm.” Sure thing! Would you like to fly first class, business class, or coach? First class Classy! And would you like a meal on this flight? Perfect! Here are a list of flights that match your criteria. Yes, of course. find flights first class first class, yes to meal departure city: Austin
40. Give Structure to your Bot https://www.nngroup.com/articles/mental-models/ implementation model user’s mental model representation model
41. Booking a Flight and a Hotel • Can user’s book a flight and a hotel with your product? • Is it only flights? If so, what if I have a problem with my flight can I contact your bot?
42. Structure into Machine Language “Show me flights from Austin to Atlanta leaving next Friday 
 after 4pm.” departure city destination city date time INTENT: FLIGHTSEARCH
43. User Interface Elements of VUIs • Words, Words, Words • SSML - Speech Synthesis Markup Language • Media • Hardware
44. Words, Words, Words • Write for speaking and listening • Create a script and act it out
45. Voice, Tone, and Persona • Voice is the quality of your words • Tone is how the words are modulated for different situations • Persona is the character that embodies the voice and tone
46. Speech Synthesis Markup Language (SSML) • Style • Emphasis • Breaks • Prosody https://www.w3.org/TR/speech-synthesis/
47. Media & Hardware • Audio • Screens and Sensors • Hardware
48. THE FUTURE OF SPEAKING WITH MACHINES #givevoice
49. 1. More emotional awareness 2. Social attitudes toward voice 3. Better integration of voice and visual 4. Improvements in underlying tech THE FUTURE OF SPEAKING WITH MACHINES
50. HOW TO BUILD A VOICE INTERFACE IN TEN MINUTES #givevoice