Slide 1
Slide 1 text
Conversational
Interfaces
in iOS
Slide 2
Slide 2 text
No content
Slide 3
Slide 3 text
“Conversation as a Platform”
Slide 4
Slide 4 text
What are conversational
interfaces?
տ扖ࣳαЀόЄϢδαφ;΅Ҙ
Slide 5
Slide 5 text
Onus on the software,
not the user
ϳЄσЄͽ΅̵ͥ
ϊϢϕγδί揗Ψͧ͡͵
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
No content
Slide 9
Slide 9 text
iOS Speech Recognition API
Amazon Lex
Google Speech API
OpenEars
Nuance
Slide 10
Slide 10 text
Speech Recognition API
ᶪ्扯挷API
Slide 11
Slide 11 text
Server-side recognition
Slide 12
Slide 12 text
Free, but not unlimited
Slide 13
Slide 13 text
Pre-recorded or live
audio
Slide 14
Slide 14 text
Over 50 languages and
dialects
50զӤ΄ࢵ;ො
Slide 15
Slide 15 text
iOS 10+
Internet connection
required
Slide 16
Slide 16 text
func recognizeRecording() {
guard let url = Bundle.main.url(forResource: "hi", withExtension: "m4a") else {
return
}
guard let recognizer = SFSpeechRecognizer() else {
// Device or locale not supported
return
}
if !recognizer.isAvailable {
// Internet connection may not be available
return
}
let request = SFSpeechURLRecognitionRequest(url: url)
recognizer.recognitionTask(with: request) { (result, error) in
guard let result = result else {
return
}
print("result: \(result.bestTranscription.formattedString)")
if result.isFinal {
print("final result: \(result.bestTranscription.formattedString)")
}
}
}
Slide 17
Slide 17 text
let audioEngine = AVAudioEngine()
let speechRecognizer = SFSpeechRecognizer()
let request = SFSpeechAudioBufferRecognitionRequest()
func startRecording() throws {
let node = audioEngine.inputNode
let recordingFormat = node.outputFormat(forBus: 0)
node.installTap(onBus: 0, bufferSize: 1024, format: recordingFormat) { [weak self]
(buffer, _) in
self?.request.append(buffer)
}
audioEngine.prepare()
try audioEngine.start()
speechRecognizer?.recognitionTask(with: request, resultHandler: { (result, error) in
guard let result = result else {
return
}
print("result: \(result.bestTranscription.formattedString)")
})
}
func stopRecording() {
audioEngine.stop()
request.endAudio()
}
Slide 18
Slide 18 text
Google Speech API
Bing Speech API
OpenEars
Nuance
Amazon Lex
Slide 19
Slide 19 text
Cost
Free tier + paid
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
No content
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
No content
Slide 24
Slide 24 text
No content
Slide 25
Slide 25 text
No content
Slide 26
Slide 26 text
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions:
[UIApplicationLaunchOptionsKey: Any]?) -> Bool {
let credentialsProvider = AWSCognitoCredentialsProvider(regionType:
AWSRegionType.USEast1, identityPoolId:"your-pool-id")
let serviceConfiguration = AWSServiceConfiguration(region: AWSRegionType.USEast1,
credentialsProvider:credentialsProvider)
AWSServiceManager.default().defaultServiceConfiguration = serviceConfiguration
let config = AWSLexInteractionKitConfig.defaultInteractionKitConfig(withBotName:
"RecipeBot", botAlias:"Prod")
// 5000 seconds before timeout
config.noSpeechTimeoutInterval = 5000
config.maxSpeechTimeoutInterval = 5000
// We will use this key to retrieve the interaction kit in our view controller
AWSLexInteractionKit.register(with: serviceConfiguration!,
interactionKitConfiguration: config, forKey:"USEast1InteractionKit")
return true
}
Slide 27
Slide 27 text
Listen for input
Slide 28
Slide 28 text
let interactionKit = AWSLexInteractionKit(forKey: "USEast1InteractionKit")
interactionKit.audioInAudioOut()
Slide 29
Slide 29 text
interactionKit.audioInTextOut()
interactionKit.textInTextOut()
interactionKit.textInAudioOut()
Slide 30
Slide 30 text
private func interactionKit(_ interactionKit: AWSLexInteractionKit,
onDialogReadyForFulfillmentForIntent intent: String, slots: Dictionary) {
print("Intent fulfilled: \(intent)")
}
Slide 31
Slide 31 text
internal func interactionKit(onAudioPlaybackStarted _ : AWSLexInteractionKit) {
spinner.startAnimating()
}
internal func interactionKit(onAudioPlaybackFinished _ : AWSLexInteractionKit) {
spinner.stopAnimating()
}
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
func interactionKit(_ interactionKit: AWSLexInteractionKit, onError error: Error) {
interactionKit.audioInAudioOut()
}
Slide 34
Slide 34 text
•Completely free, but
not unlimited
Speech Recognition API
•Built right into iOS
Slide 35
Slide 35 text
•Higher-level abstraction
(text parsing, error
handling)
Lex
•Broad match of phrases
to intents
Slide 36
Slide 36 text
Information
•Cross Platform
Lex
•Not free past first 5000
requests/month
Slide 37
Slide 37 text
Best Practices
Slide 38
Slide 38 text
Transparency!
᭐กѺ
Slide 39
Slide 39 text
No content
Slide 40
Slide 40 text
Sensitive information
䱛ੂఘ䁭
Slide 41
Slide 41 text
Be creative!
Slide 42
Slide 42 text
Thanks!
Wendy Lu
@wendyluwho