Tweet
Tweet

Slide 1

Slide 1 text

IN CONVERSATION WITH A BROWSER @philnash

Slide 2

Slide 2 text

Phil Nash @philnash http://philna.sh [email protected]

Slide 3

Slide 3 text

BOTS @philnash

Slide 4

Slide 4 text

ELIZA: Please tell me what's been bothering you. YOU: I'm scared of public speaking. ELIZA: Do you know anyone else who is scared of public speaking ? YOU: Lot's of people ELIZA: Do you feel strongly about discussing such things ? YOU: Only with you ELIZA: You're not really talking about me -- are you ? YOU: No, I'm the one who is scared. ELIZA: Why 'no' ? @philnash

Slide 5

Slide 5 text

@philnash

Slide 6

Slide 6 text

*´¯`*.¸¸.*´¯`*ÃŇĞẸĹŜĎẸÃĎƑÃĹĹ*´¯`*.¸¸.*´¯`* @philnash

Slide 7

Slide 7 text

IRC > SMS > SLACK @philnash

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

HOW DO WE BUILD OUR OWN? @philnash

Slide 10

Slide 10 text

WEB APIS @philnash

Slide 11

Slide 11 text

WEB SPEECH @philnash

Slide 12

Slide 12 text

Speech Synthesis Hello everyone const utterance = new SpeechSynthesisUtterance(text); speechSynthesis.speak(utterance); 01. 02. @philnash

Slide 13

Slide 13 text

Speech Synthesis https://glitch.com/~browser-voices @philnash

Slide 14

Slide 14 text

@philnash

Slide 15

Slide 15 text

Speech Recognition Start Speech Recognition start listening to me and probably putting things into that textarea there eventually I can get it going again I can see he @philnash

Slide 16

Slide 16 text

Speech Recognition const recognition = new webkitSpeechRecognition(); recognition.addEventListener('result', event => { const result = event.results[0][0].transcript; console.log(result); }); recognition.start(); 01. 02. 03. 04. 05. 06. @philnash

Slide 17

Slide 17 text

@philnash

Slide 18

Slide 18 text

Speech Recognition Sends all the data to Google Cloud Speech API @philnash

Slide 19

Slide 19 text

MEDIARECORDER API @philnash

Slide 20

Slide 20 text

MediaRecorder API Start recording 0:00 0:00 @philnash

Slide 21

Slide 21 text

MediaRecorder API const stream = await navigator.mediaDevices.getUserMedia(); const recorder = new MediaRecorder(stream, { type: 'audio/webm' }); const chunks = []; 01. 02. 03. @philnash

Slide 22

Slide 22 text

MediaRecorder API recorder.addEventListener('dataavailable', event => { if (typeof event.data === 'undefined') return; if (event.data.size === 0) return; chunks.push(event.data); }); 01. 02. 03. 04. 05. @philnash

Slide 23

Slide 23 text

MediaRecorder API recorder.addEventListener('stop', event => { const recording = new Blob(chunks, { type: 'audio/webm' }); }); 01. 02. 03. 04. 05. @philnash

Slide 24

Slide 24 text

@philnash

Slide 25

Slide 25 text

MediaRecorder API https://glitch.com/~web-recorder @philnash

Slide 26

Slide 26 text

Then what? Send the file to a speech to text service • Google Cloud Speech • Azure Cognitive Services • IBM Watson @philnash

Slide 27

Slide 27 text

WEBAUDIO API @philnash

Slide 28

Slide 28 text

@philnash

Slide 29

Slide 29 text

AUDIOWORKLET + WEBSOCKETS @philnash

Slide 30

Slide 30 text

DEMO @philnash

Slide 31

Slide 31 text

Web Speech Alternatives/Polyfills https://github.com/watson-developer-cloud/speech-javascript-sdk https://github.com/anteloe/speech-polyfill https://github.com/compulim/web-speech-cognitive-services @philnash

Slide 32

Slide 32 text

THIS IS ALL GREAT... BUT @philnash

Slide 33

Slide 33 text

IT'S SENDING ALL THE MIC DATA TO A THIRD PARTY SERVICE @philnash

Slide 34

Slide 34 text

WAKE WORDS @philnash

Slide 35

Slide 35 text

MACHINE LEARNING @philnash

Slide 36

Slide 36 text

Machine Learning TensorFlow.js ml5.js @philnash

Slide 37

Slide 37 text

DEMO @philnash

Slide 38

Slide 38 text

CONVERSATION DESIGN @philnash

Slide 39

Slide 39 text

SPEAK YOUR BOT CONVERSATIONS OUT LOUD WITH SOMEONE ELSE @philnash

Slide 40

Slide 40 text

WHAT DO WE DO WITH THIS? @philnash

Slide 41

Slide 41 text

TECHNICAL JOURNEY @philnash

Slide 42

Slide 42 text

WEB PLATFORM @philnash

Slide 43

Slide 43 text

EXPERIMENTATION + FREEDOM @philnash

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

@philnash

Slide 46

Slide 46 text

WEB ASSISTANT @philnash

Slide 47

Slide 47 text

Web Assistant https://github.com/philnash/web-assistant/ @philnash

Slide 48

Slide 48 text

THIS IS JUST THE START OF THE JOURNEY @philnash

Slide 49

Slide 49 text

@philnash

Slide 50

Slide 50 text

Thanks! @philnash http://philna.sh [email protected]