But now, we really use them! ! • Voice-based customer service phone tree • Text chat with automated customer service agent • Automated telephone survey • IM messaging spam • Email spam
What is it? Conversational UI Reality Check(list): • Messaging? Evolving new interaction model. No magic AI. • Chatbots? Still quite limited. No magic AI. (afaict). • Voice? Yes! Significant breakthroughs thanks to neural nets.
What If? by Anne Bernays & Pamela Painter • Exercises for characterization • What does the character want? • Remember most from childhood? • Usually feel two hours after lunch? • What are they in denial of?
Voice UI is opposite of Message Thread UI Thread Voice history accessible no history identity visible no identity rich media text only guided input freeform text
Rochefoucauld • François VI, Duc de la Rouchefoucauld, Prince de Marcillac • Born 1613, Died 1680 • Wealthy, noble, good-looking • Unhappily married, imprisoned, libeled, exiled, shot in the eye
Rochefoucauld • Maxims and other works • 500+ aphorisms on love, ambition, self-delusion, ennui, envy, esteem, evils, the exchange of secrets, etc. • character: cynical, worldly wise, witty • domain: life's sorrows
Once, create and/or configure • AVAudioEngine, AVAudioSession, SFSpeechRecognizer Per utterance, request stream of live partial results • AVAudioNodeTapBlock, SFSpeechAudioBufferRecognitionRequest, SFSpeechAudioBufferRecognitionTask During utterances, handle callbacks with results • SFSpeechRecognitionResult • but SFSpeechRecognitionResult.isFinal is never
/** Operation which recognizes speech and finishes after it has reached an "endpoint", an interval over which no speech has been recognized (e.g., a period of silence or of noise). */ public class SpeechRecognitionOperation: Operation, SFSpeechRecognitionTaskDelegate { open override var isAsynchronous: Bool { return true } public var output:String? = nil private var endpointTimer:Timer? private var request:SFSpeechAudioBufferRecognitionRequest? // ... public init(engine e:AVAudioEngine, recognizer r:SFSpeechRecognizer) { /* ... */ } // ... }
// Represents a bot, e.g., Eliza, Rochefoucauld, etc.. protocol Interlocutor : class { init() func respondTo(saying:String) -> String } // wraps any Interlocutor in a speech rec/synth UI public class VoiceChatter : NSObject { public init(interlocutor:Interlocutor) { /* ... */ } } // receives a callback when a line of dialog is recognized/spoken. protocol VoiceChatterDelegate { func engineDidUpdateDialog(engine:VoiceChatter, dialogLines:[String]) }
Tree, where every node is a Q&A or a set of quotations: let rocheTree:ConversationNode = .Question( question: "Is your problem with your own feelings, or with other people?", answerPatterns: [ ("feelings", .Question( question: "And do you suffer from love, or from ambition?", answerPatterns:[ ("love", .Statement(quotations:feelingsLoveQuotes)), ("ambition", .Statement(quotations:feelingsAmbitionQuotes)) ]) ), // ... ])
Amazon Alexa Custom Skills • 1000s of them, some quite fun! • deployed on the Amazon Echo device • most easily hosted on Amazon Lambda ... • ... which only supports NodeJS, Python, Java
Conclusions: • Yes, Swift can go anywhere! ! • ... even into conversational UI • Real hardware/software advances in speech • Works best in constrained domains • An interesting world where design is writing • Don't believe the hype (except on neural nets)
Acknowledgements • Nick Jackson (@sheriffjackson), docker and terraform magic, in https://github.com/algal/SwiftOnLambda • Claus Höfele (@claushoefele), Alexa wrapper • Xavier Schott, Objective-C Eliza implementation • Swift@IBM, for Kitura and helpful docker images
Swift Package Manager + Xcode: • isolate cross platform in modules • use SPM to manage its build • use SPM to generate xcodeproj files • contain iOS app in Xcode workspace • SPM-generated project is one imported module • iOS project imports and uses it
What If? by Anne Bernays & Pamela Painter • Exercises for characterization • What does the character want? • Remember most from childhood? • Usually feel two hours after lunch? • What are they in denial of?
cgi-bin, all over again /** Reads all of stdin in a String buffer. Transforms the string. Prints result to stdout. */ func readTransformPrint(transform:(String)->String) { var input:String = "" for line in lineGenerator(file: stdin) { input += line } let result = transform(input) print(result, separator: "", terminator: "") }
algal
peisuke
kei_s
onlinesalon
yamamuteki
sansantech
seosoft
rilmayer
oracle4engineer
codenote
keiko713
smasato
kneath
notwaldorf
rasmusluckow
ammeep
philnash
productmarketing
malarkey
maltzj
chrislema
colly
myddelton
addyosmani