Siri Shortcuts - NSSpain, Logroño, Spain, September 2018

My talk on the history of Shortcuts, my co-worker named Siri, and the new iOS 12 APIs you can use to start creating your own (semi-limited) shortcuts.

Work-In-Progress sample Code: https://github.com/designatednerd/TravelPlanner

Transcript

1. SIRI SHORTCUTS NSSPAIN | LOGROÑO, SPAIN | SEPTEMBER 2018 @DESIGNATEDNERD | BAKKENBAECK.COM | JUSTHUM.COM
2. !
4. "HEY SIRI!"
5. Photo illustration via http://thetechnews.com/2016/12/07/not-only-samsung-iphone-explodes-too/
6. Photo via [http://www.gizmodo.co.uk/2013/07/apparently-people-hated-the-iphone-5-the-most/
8. ! " #
12. MACSTORIES
13. MACSTORIES
19. WORKFLOW -> SHORTCUTS
21. [https://twitter.com/AdamFootUK/status/1028760230304333824]
23. https://twitter.com/bpmarkowitz/status/1015303601839837186
24. https://twitter.com/bpmarkowitz/status/1015303601839837186
25. https://twitter.com/davedelong/status/1015445465062502400
26. https://twitter.com/AdrianEves07/status/1017032252629319680
27. https://twitter.com/AdrianEves07/status/1017032252629319680
28. https://twitter.com/AdrianEves07/status/1017032252629319680
29. https://twitter.com/AdrianEves07/status/1017032252629319680
30. https://twitter.com/thomaswood/status/1017428845748768769]
32. WHAT ABOUT MY APP?
35. 1. CREATE SHORTCUT
36. 2. DONATE SHORTCUT TO THE SYSTEM FOR LATER USE
38. 3. HANDLE SHORTCUT
39. CREATING A SHORTCUT
40. (SOME OF) TEH CODEZ! https://github.com/designatednerd TravelPlanner
41. EASY MODE: NSUserActivity
42. NSUserActivity
43. NSUserActivity SETUP let userActivity = NSUserActivity(activityType: "com.yourco.YourApp.ActivityType") userActivity.isEligibleForSearch = true userActivity.isEligibleForPrediction = true userActivity.title = "View \(trip.name ?? "") trip" userActivity.suggestedInvocationPhrase = "View \(trip.name ?? "") trip" userActivity.requiredUserInfoKeys = [ UserActivityInfoKey.trip.rawValue ] userActivity.userInfo = [ UserActivityInfoKey.trip.rawValue: trip.id ] userActivity.persistentIdentifier = trip.id viewController.userActivity = userActivity userActivity.becomeCurrent()
44. NSUserActivity SETUP let userActivity = NSUserActivity(activityType: "com.yourco.YourApp.ActivityType") userActivity.isEligibleForSearch = true userActivity.isEligibleForPrediction = true userActivity.title = "View \(trip.name ?? "") trip" userActivity.suggestedInvocationPhrase = "View \(trip.name ?? "") trip" userActivity.requiredUserInfoKeys = [ UserActivityInfoKey.trip.rawValue ] userActivity.userInfo = [ UserActivityInfoKey.trip.rawValue: trip.id ] userActivity.persistentIdentifier = trip.id viewController.userActivity = userActivity userActivity.becomeCurrent()
45. NSUserActivity SETUP let userActivity = NSUserActivity(activityType: "com.yourco.YourApp.ActivityType") userActivity.isEligibleForSearch = true userActivity.isEligibleForPrediction = true userActivity.title = "View \(trip.name ?? "") trip" userActivity.suggestedInvocationPhrase = "View \(trip.name ?? "") trip" userActivity.requiredUserInfoKeys = [ UserActivityInfoKey.trip.rawValue ] userActivity.userInfo = [ UserActivityInfoKey.trip.rawValue: trip.id ] userActivity.persistentIdentifier = trip.id viewController.userActivity = userActivity userActivity.becomeCurrent()
46. NSUserActivity SETUP let userActivity = NSUserActivity(activityType: "com.yourco.YourApp.ActivityType") userActivity.isEligibleForSearch = true userActivity.isEligibleForPrediction = true userActivity.title = "View \(trip.name ?? "") trip" userActivity.suggestedInvocationPhrase = "View \(trip.name ?? "") trip" userActivity.requiredUserInfoKeys = [ UserActivityInfoKey.trip.rawValue ] userActivity.userInfo = [ UserActivityInfoKey.trip.rawValue: trip.id ] userActivity.persistentIdentifier = trip.id viewController.userActivity = userActivity userActivity.becomeCurrent()
47. NSUserActivity SETUP let userActivity = NSUserActivity(activityType: "com.yourco.YourApp.ActivityType") userActivity.isEligibleForSearch = true userActivity.isEligibleForPrediction = true userActivity.title = "View \(trip.name ?? "") trip" userActivity.suggestedInvocationPhrase = "View \(trip.name ?? "") trip" userActivity.requiredUserInfoKeys = [ UserActivityInfoKey.trip.rawValue ] userActivity.userInfo = [ UserActivityInfoKey.trip.rawValue: trip.id ] userActivity.persistentIdentifier = trip.id viewController.userActivity = userActivity userActivity.becomeCurrent()
48. NSUserActivity SETUP let userActivity = NSUserActivity(activityType: "com.yourco.YourApp.ActivityType") userActivity.isEligibleForSearch = true userActivity.isEligibleForPrediction = true userActivity.title = "View \(trip.name ?? "") trip" userActivity.suggestedInvocationPhrase = "View \(trip.name ?? "") trip" userActivity.requiredUserInfoKeys = [ UserActivityInfoKey.trip.rawValue ] userActivity.userInfo = [ UserActivityInfoKey.trip.rawValue: trip.id ] userActivity.persistentIdentifier = trip.id viewController.userActivity = userActivity userActivity.becomeCurrent()
49. NSUserActivity SETUP let userActivity = NSUserActivity(activityType: "com.yourco.YourApp.ActivityType") userActivity.isEligibleForSearch = true userActivity.isEligibleForPrediction = true userActivity.title = "View \(trip.name ?? "") trip" userActivity.suggestedInvocationPhrase = "View \(trip.name ?? "") trip" userActivity.requiredUserInfoKeys = [ UserActivityInfoKey.trip.rawValue ] userActivity.userInfo = [ UserActivityInfoKey.trip.rawValue: trip.id ] userActivity.persistentIdentifier = trip.id viewController.userActivity = userActivity userActivity.becomeCurrent()
50. NSUserActivity SETUP let userActivity = NSUserActivity(activityType: "com.yourco.YourApp.ActivityType") userActivity.isEligibleForSearch = true userActivity.isEligibleForPrediction = true userActivity.title = "View \(trip.name ?? "") trip" userActivity.suggestedInvocationPhrase = "View \(trip.name ?? "") trip" userActivity.requiredUserInfoKeys = [ UserActivityInfoKey.trip.rawValue ] userActivity.userInfo = [ UserActivityInfoKey.trip.rawValue: trip.id ] userActivity.persistentIdentifier = trip.id viewController.userActivity = userActivity userActivity.becomeCurrent()
51. NSUserActivity SETUP let userActivity = NSUserActivity(activityType: "com.yourco.YourApp.ActivityType") userActivity.isEligibleForSearch = true userActivity.isEligibleForPrediction = true userActivity.title = "View \(trip.name ?? "") trip" userActivity.suggestedInvocationPhrase = "View \(trip.name ?? "") trip" userActivity.requiredUserInfoKeys = [ UserActivityInfoKey.trip.rawValue ] userActivity.userInfo = [ UserActivityInfoKey.trip.rawValue: trip.id ] userActivity.persistentIdentifier = trip.id viewController.userActivity = userActivity // IMPORTANT userActivity.becomeCurrent()
52. NSUserActivity SETUP let userActivity = NSUserActivity(activityType: "com.yourco.YourApp.ActivityType") userActivity.isEligibleForSearch = true userActivity.isEligibleForPrediction = true userActivity.title = "View \(trip.name ?? "") trip" userActivity.suggestedInvocationPhrase = "View \(trip.name ?? "") trip" userActivity.requiredUserInfoKeys = [ UserActivityInfoKey.trip.rawValue ] userActivity.userInfo = [ UserActivityInfoKey.trip.rawValue: trip.id ] userActivity.persistentIdentifier = trip.id viewController.userActivity = userActivity // IMPORTANT userActivity.becomeCurrent()
57. ! SPOTLIGHT
58. SPOTLIGHT SETUP let attributes = CSSearchableItemAttributeSet (itemContentType: kUTTypeContent as String) if let destination = trip.destination { attributes.contentDescription = "View details for your trip to \(destination)" } attributes.thumbnailData = UIImage(named: "trip")?.pngData() userActivity.contentAttributeSet = attributes
59. SPOTLIGHT SETUP let attributes = CSSearchableItemAttributeSet (itemContentType: kUTTypeContent as String) if let destination = trip.destination { attributes.contentDescription = "View details for your trip to \(destination)" } attributes.thumbnailData = UIImage(named: "trip")?.pngData() userActivity.contentAttributeSet = attributes
60. SPOTLIGHT SETUP let attributes = CSSearchableItemAttributeSet (itemContentType: kUTTypeContent as String) if let destination = trip.destination { attributes.contentDescription = "View details for your trip to \(destination)" } attributes.thumbnailData = UIImage(named: "trip")?.pngData() userActivity.contentAttributeSet = attributes
61. SPOTLIGHT SETUP let attributes = CSSearchableItemAttributeSet (itemContentType: kUTTypeContent as String) if let destination = trip.destination { attributes.contentDescription = "View details for your trip to \(destination)" } attributes.thumbnailData = UIImage(named: "trip")?.pngData() userActivity.contentAttributeSet = attributes
62. SPOTLIGHT SETUP let attributes = CSSearchableItemAttributeSet (itemContentType: kUTTypeContent as String) if let destination = trip.destination { attributes.contentDescription = "View details for your trip to \(destination)" } // TODO: Y U NO WORK?! attributes.thumbnailData = UIImage(named: "trip")?.pngData() userActivity.contentAttributeSet = attributes
63. SPOTLIGHT SETUP let attributes = CSSearchableItemAttributeSet (itemContentType: kUTTypeContent as String) if let destination = trip.destination { attributes.contentDescription = "View details for your trip to \(destination)" } // TODO: Y U NO WORK?! attributes.thumbnailData = UIImage(named: "trip")?.pngData() userActivity.contentAttributeSet = attributes
66. DON'T TRY TO CRAM WAY TOO MUCH REPETITIVE INFORMATION INTO THE CONTENT DESCRIPTION
67. DON'T TRY TO CRAM WAY TOO MUCH REPETITIVE INFORMATION INTO THE CONTENT DESCRIPTION ⬆ (LIKE THIS)
68. SPOTLIGHT SETUP let attributes = CSSearchableItemAttributeSet (itemContentType: kUTTypeContent as String) if let destination = trip.destination { attributes.contentDescription = "\(destination) || \(trip.formattedDateInterval)" } attributes.thumbnailData = UIImage(named: "trip")?.pngData() userActivity.contentAttributeSet = attributes
70. SPOTLIGHT SETUP let attributes = CSSearchableItemAttributeSet (itemContentType: kUTTypeContent as String) if let destination = trip.destination { attributes.contentDescription = """ ! \(destination) " \(trip.formattedTripInterval) """ } attributes.thumbnailData = UIImage(named: "trip")?.pngData() userActivity.contentAttributeSet = attributes
72. HANDLING SHORTCUT SELECTION
73. UIApplicationDelegate
74. UIApplicationDelegate SETUP func application(_ application: UIApplication, continue userActivity: NSUserActivity, restorationHandler: @escaping ([UIUserActivityRestoring]?) -> Void) -> Bool { guard userActivity.activityType == "com.yourco.YourApp.ActivityType" else { return false } let handlingVC = // Omitted for length restorationHandler(handlingVC) return true }
75. UIApplicationDelegate SETUP func application(_ application: UIApplication, continue userActivity: NSUserActivity, restorationHandler: @escaping ([UIUserActivityRestoring]?) -> Void) -> Bool { guard userActivity.activityType == "com.yourco.YourApp.ActivityType" else { return false } let handlingVC = // Omitted for length restorationHandler(handlingVC) return true }
76. UIApplicationDelegate SETUP func application(_ application: UIApplication, continue userActivity: NSUserActivity, restorationHandler: @escaping ([UIUserActivityRestoring]?) -> Void) -> Bool { guard userActivity.activityType == "com.yourco.YourApp.ActivityType" else { return false } let handlingVC = // Omitted for length restorationHandler(handlingVC) return true }
77. UIApplicationDelegate SETUP func application(_ application: UIApplication, continue userActivity: NSUserActivity, restorationHandler: @escaping ([UIUserActivityRestoring]?) -> Void) -> Bool { guard userActivity.activityType == "com.yourco.YourApp.ActivityType" else { return false } let handlingVC = // Omitted for length restorationHandler(handlingVC) return true }
79. HARD MODE: Intents FRAMEWORK
80. INIntent + INInteraction
96. DONATING AN INTERACTION private func donateEditTripIntent() { let intent = EditTripIntent() intent.name = self.trip.name intent.destination = self.trip.destination intent.suggestedInvocationPhrase = "Edit \(intent.destination ?? intent.name!) trip" let interaction = INInteraction(intent: intent, response: nil) interaction.donate { error in if let error = error { debugPrint("Error donating intent: \(error)") } } }
97. DONATING AN INTERACTION private func donateEditTripIntent() { let intent = EditTripIntent() intent.name = self.trip.name intent.destination = self.trip.destination intent.suggestedInvocationPhrase = "Edit \(intent.destination ?? intent.name!) trip" let interaction = INInteraction(intent: intent, response: nil) interaction.donate { error in if let error = error { debugPrint("Error donating intent: \(error)") } } }
98. DONATING AN INTERACTION private func donateEditTripIntent() { let intent = EditTripIntent() intent.name = self.trip.name intent.destination = self.trip.destination intent.suggestedInvocationPhrase = "Edit \(intent.destination ?? intent.name!) trip" let interaction = INInteraction(intent: intent, response: nil) interaction.donate { error in if let error = error { debugPrint("Error donating intent: \(error)") } } }
101. DONATING AN INTERACTION private func donateEditTripIntent() { let intent = EditTripIntent() intent.name = self.trip.name intent.destination = self.trip.destination intent.suggestedInvocationPhrase = "Edit \(intent.destination ?? intent.name!) trip" let interaction = INInteraction(intent: intent, response: nil) interaction.donate { error in if let error = error { debugPrint("Error donating intent: \(error)") } } }
102. DONATING AN INTERACTION private func donateEditTripIntent() { let intent = EditTripIntent() intent.name = self.trip.name intent.destination = self.trip.destination intent.suggestedInvocationPhrase = "Edit \(intent.destination ?? intent.name!) trip" let interaction = INInteraction(intent: intent, response: nil) interaction.donate { error in if let error = error { debugPrint("Error donating intent: \(error)") } } }
103. AppDelegate INTENT SETUP func application(_ application: UIApplication, continue userActivity: NSUserActivity, restorationHandler: @escaping ([UIUserActivityRestoring]?) -> Void) -> Bool { if let intent = userActivity.interaction?.intent as? EditTripIntent { guard let trip = self.getTrip(from: intent) else { return false } self.coordinator.editTrip(trip) return true } // Rest of method we saw earlier }
104. AppDelegate INTENT SETUP func application(_ application: UIApplication, continue userActivity: NSUserActivity, restorationHandler: @escaping ([UIUserActivityRestoring]?) -> Void) -> Bool { if let intent = userActivity.interaction?.intent as? EditTripIntent { guard let trip = self.getTrip(from: intent) else { return false } self.coordinator.editTrip(trip) return true } // Rest of method we saw earlier }
105. AppDelegate INTENT SETUP func application(_ application: UIApplication, continue userActivity: NSUserActivity, restorationHandler: @escaping ([UIUserActivityRestoring]?) -> Void) -> Bool { if let intent = userActivity.interaction?.intent as? EditTripIntent { guard let trip = self.getTrip(from: intent) else { return false } self.coordinator.editTrip(trip) return true } // Rest of method we saw earlier }
106. AppDelegate INTENT SETUP func application(_ application: UIApplication, continue userActivity: NSUserActivity, restorationHandler: @escaping ([UIUserActivityRestoring]?) -> Void) -> Bool { if let intent = userActivity.interaction?.intent as? EditTripIntent { guard let trip = self.getTrip(from: intent) else { return false } self.coordinator.editTrip(trip) return true } // Rest of method we saw earlier }
107. AppDelegate INTENT SETUP func application(_ application: UIApplication, continue userActivity: NSUserActivity, restorationHandler: @escaping ([UIUserActivityRestoring]?) -> Void) -> Bool { if let intent = userActivity.interaction?.intent as? EditTripIntent { guard let trip = self.getTrip(from: intent) else { return false } self.coordinator.editTrip(trip) return true } // Rest of method we saw earlier }
108. EXPERT MODE: RESPONSES + INTENT EXTENSION
110. INTENT HANDLER SETUP public class AddTripIntentHandler: NSObject, EditTripIntentHandling { // Optional! public func confirm(intent: EditTripIntent, completion: @escaping (EditTriptIntentResponse) -> Void) { // Figure out if the thing can be done } public func handle(intent: EditTripIntent, completion: @escaping (EditTripIntentResponse) -> Void) { // Do the thing! completion(.success) } }
111. INTENT HANDLER SETUP public class AddTripIntentHandler: NSObject, EditTripIntentHandling { // Optional! public func confirm(intent: EditTripIntent, completion: @escaping (EditTriptIntentResponse) -> Void) { // Figure out if the thing can be done } public func handle(intent: EditTripIntent, completion: @escaping (EditTripIntentResponse) -> Void) { // Do the thing! completion(.success) } }
112. INTENT HANDLER SETUP public class AddTripIntentHandler: NSObject, EditTripIntentHandling { // Optional! public func confirm(intent: EditTripIntent, completion: @escaping (EditTriptIntentResponse) -> Void) { // Figure out if the thing can be done } public func handle(intent: EditTripIntent, completion: @escaping (EditTripIntentResponse) -> Void) { // Do the thing! completion(.success) } }
116. INTENT HANDLER SETUP public class AddTripIntentHandler: NSObject, EditTripIntentHandling { // Optional! public func confirm(intent: EditTripIntent, completion: @escaping (EditTriptIntentResponse) -> Void) { // Figure out if the thing can be done } public func handle(intent: EditTripIntent, completion: @escaping (EditTripIntentResponse) -> Void) { // Do the thing! completion(.success(name: name, destination: destination)) } }
117. GALAXY BRAIN MODE: IntentsUI EXTENSION
118. 404 - Time Not Found
119. UX FOR VOICE
120. VoiceX
121. VoiceX (TOO ELON MUSK)
122. VUX
123. VUX (SOUNDS TOO MUCH LIKE VOX)
124. VX
125. VX (IS NERVE GAS)
126. UX FOR VOICE
127. KEEP SUGGESTED INVOCATION PHRASES SHORT AND SIMPLE
128. HEY SIRI, WHEN IS MY NEXT FLIGHT?
130. HEY SIRI, WHEN IS MY NEXT FLIGHT?
131. WHEN IS MY NEXT FLIGHT?
132. WHEN IS MY NEXT FLIGHT?
133. WHEN IS MY NEXT FLIGHT?
134. NEXT FLIGHT
135. OBLIGATORY SUMMARY SLIDE
136. OBLIGATORY SUMMARY SLIDE > Shortcuts can be a powerful tool for automation
137. OBLIGATORY SUMMARY SLIDE > Shortcuts can be a powerful tool for automation > your shortcuts are a bit more limited
138. OBLIGATORY SUMMARY SLIDE > Shortcuts can be a powerful tool for automation > your shortcuts are a bit more limited > Use NSUserActivity for basic action shortcuts
139. OBLIGATORY SUMMARY SLIDE > Shortcuts can be a powerful tool for automation > your shortcuts are a bit more limited > Use NSUserActivity for basic action shortcuts > Use Custom Intents for more involved shortcuts
140. OBLIGATORY SUMMARY SLIDE > Shortcuts can be a powerful tool for automation > your shortcuts are a bit more limited > Use NSUserActivity for basic action shortcuts > Use Custom Intents for more involved shortcuts > Keep titles and suggested invocation phrases short
141. OBLIGATORY SUMMARY SLIDE > Shortcuts can be a powerful tool for automation > your shortcuts are a bit more limited > Use NSUserActivity for basic action shortcuts > Use Custom Intents for more involved shortcuts > Keep titles and suggested invocation phrases short > Responses + Intent Extension + Intent UI = users
142. OBLIGATORY SUMMARY SLIDE > Shortcuts can be a powerful tool for automation > Use NSUserActivity for basic action shortcuts > Use Custom Intents for more involved shortcuts > Keep titles and suggested invocation phrases short > Use custom responses to handle errors more clearly
143. WWDC VIDEOS! > Session 211: Introduction to Siri Shortcuts https://developer.apple.com/videos/play/ wwdc2018/211/ > Session 214: Building for Voice with Siri Shortcuts https://developer.apple.com/videos/play/ wwdc2018/214/
144. MOAR WWDC VIDEOS! > Session 217: Siri Shortcuts On The Watch Face https://developer.apple.com/videos/play/ wwdc2018/217/ > Session 404: New Localization Workflows in Xcode 10 https://developer.apple.com/videos/play/ wwdc2018/404/
145. LINKS! > Original MacStories Workflow Review https://www.macstories.net/reviews/ workflow-review-integrated-automation-for- ios-8/ > Original Workflow Teaser video https://www.youtube.com/watch? v=fSh6Q0F4d9Q