The Amazing Powers and Mesmerizing Secrets of 3D Touch

A somewhat mad science experiment where I explore the 3D touch features of the iPhone 6s, by building a "touch pedestal." (Also, I explain the 3D Touch API introduced in iOS 9.)

PDF deck of the talk given at: https://realm.io/news/alexis-gallagher-3d-touch-swift/

Transcript

1. The Amazing Powers
   & Mesmerizing Secrets
   of 3D Touch
   @alexisgallagher
   

   View Slide

2. First New Input Method
   • iPhone 3G : GPS
   • iPhone 3GS : Digital Compass
   • iPhone 4 : Front Camera
   • iPhone 5S : Touch ID
   • iPhone 6 : Barometer
   • iPhone 6s : 3D Touch !"
   

   View Slide

3. Three new APIs
   

   View Slide

4. Three new APIs
   1.Home Screen Quick Actions
   2.Peek and Pop
   3.Force Properties !
   

   View Slide

5. 1. Home Screen
   Quick Actions
   

   View Slide

6. Quick Actions
   Designed for:
   purposeful launch
   • Shortcut to launch an app
   with an action
   • Static actions
   (e.g., "New Message")
   • Dynamic actions
   (e.g., "Johnny Appleseed")
   

   View Slide

7. View Slide

8. Configuring Quick
   Actions
   Your app provides
   UIApplicationShortcutItems:
   class UIApplicationShortcutItem {
   // app-specific action identifier & payload
   var type: String { get }
   var userInfo: [String : NSSecureCoding]? { get }
   // title, maybe subtitle & icon
   var localizedTitle: String { get }
   var localizedSubtitle: String? { get }
   var icon: UIApplicationShortcutIcon? { get }
   }
   

   View Slide

9. Providing Quick Actions
   Static actions go on the Info.plist:
   

   View Slide

10. Providing Quick Actions
    Dynamic actions go in the UIApplication object:
    let quickAction = UIMutableApplicationShortcutItem(type: "mytype",
    localizedTitle: "Play", localizedSubtitle: nil,
    icon: UIApplicationShortcutIcon(type: .Play),
    userInfo: nil)
    UIApplication.sharedApplication().shortcutItems = [quickAction]
    

    View Slide

11. Responding to Quick Actions
    On launch, via app delegate launch methods:
    func application(application: UIApplication,
    didFinishLaunchingWithOptions launchOptions: [NSObject: AnyObject]?) -> Bool {
    // If a shortcut was launched, display its information and take the appropriate action
    if let shortcutItem = launchOptions?[UIApplicationLaunchOptionsShortcutItemKey] as? UIApplicationShortcutItem
    where shortcutItem.type == "myType" {
    // do something associated with "myType" shortcut and its userInfo data payload
    }
    // ...
    return true
    }
    On activation or if launch methods returned true,
    via the new method, performActionForShortcut:
    

    View Slide

12. Home Screen Quick Actions
    Key Points
    • Fixed format: title, icon, maybe subtitle
    • Configured statically in Info.plist, or
    dynamically at runtime
    • Passed to the app delegate launch methods, and/
    or a dedicated protocol method
    

    View Slide

13. Home Screen Quick Actions
    Gotchas
    • Constraints on icon imagery
    (system-provided, static template, or contact
    photo)
    • When to update dynamic items
    (Accounting for all states produced by app
    lifecycle, app update, etc..)
    • Confusing API for app delegate launch methods
    

    View Slide

14. 2. Peek and Pop
    

    View Slide

15. Peek and Pop
    • Pressing triggers preview of
    tappable content
    • Pressing triggers stages:
    1.Hint
    (blur background)
    2.Peek
    (content preview)
    3.Pop
    (navigate as if tapped)
    

    View Slide

16. press
    Hint
    press press
    Peek Pop
    

    View Slide

17. tap
    press
    Hint
    press press
    Peek Pop
    

    View Slide

18. tap
    release
    press
    Hint
    press press
    Peek Pop
    

    View Slide

19. tap
    press
    Hint
    press press
    Peek Pop
    Quick Actions
    swipe
    

    View Slide

20. Peek and Pop
    Designed for: faster navigation
    • Pop navigates just like a tap
    • Peek previews content reachable by tap
    • Peek Quick Action is a shortcut to content
    beyond a tap
    

    View Slide

21. tap
    press
    Hint
    press press
    Peek Pop
    Quick Actions
    swipe
    

    View Slide

22. tap
    press
    Hint
    press press
    Peek Pop
    Quick Actions
    swipe
    DestinationViewController
    PreviewViewController
    previewActionItems()
    

    View Slide

23. tap
    press
    Hint
    press press
    Peek Pop
    Quick Actions
    swipe
    DestinationViewController
    PreviewViewController
    previewActionItems()
    UIViewController
    registerForPreviewingWithDelegate(_:sourceView:)
    UIViewControllerPreviewDelegate
    previewingContext(_:viewControllerForLocation:)
    previewingContext(_:commitViewController:)
    

    View Slide

24. Provides Peek View Controller:
    UIViewControllerPreviewingDelegate
    .previewingContext(_:viewControllerForLocation:) -> UIViewController?
    Performs Pop Navigation:
    UIViewControllerPreviewingDelegate
    .previewingContext(_:commitViewController:) -> Void
    Provides Peek Quick Actions:
    UIViewController.previewActionItems() -> [UIPreviewActionItem]
    

    View Slide

25. 3. Force
    Properties
    !
    

    View Slide

26. Force Properties
    

    View Slide

27. Force Properties
    Designed for:
    adventure!
    • No design direction & few
    examples from Apple
    • Extremely capable hardware
    

    View Slide

28. Force Properties
    Adopting the API
    • UITouch.force : CGFloat
    • UITouch.maximumPossibleForce
    • UITraitCollection.forceTouchCapability
    • UIResponder.estimatedPropertiesUpdated(_:)
    

    View Slide

29. 3D Touch Gesture Recognizers
    class ALGSqueezeGestureRecognizer: UIGestureRecognizer
    {
    var squeezeThreshhold:CGFloat = 0.5
    override func touchesMoved(touches:Set, withEvent:UIEvent) {
    super.touchesMoved(touches, withEvent: event)
    if touches.count == 1 && UIScreen.mainScreen().traitCollection.forceTouchCapability == .Available {
    let normalizedForce = touches.first!.force / touches.first!.maximumPossibleForce
    if normalizedForce >= self.squeezeThreshhold {
    self.state = .Recognized
    }
    }
    else { self.state = .Failed }
    }
    // etc. at
    }
    

    View Slide

30. Force Properties
    Gotchas
    • Do check UITraitCollection.forceTouchCapability
    • Define your own gesture recognizers and
    controls
    • SqueezeButton, ForceSqueezeGestureRecognizer,
    PeekPopGestureRecognizer, etc..
    

    View Slide

31. Force Properties
    Precision & Accuracy
    • Can access a touch's force,
    radius, and radius error
    • But what is the reported
    resolution of the force
    sensor?
    

    View Slide

32. 0.2 0.4 0.6 0.8 1.0
    force
    maxForce
    0
    20
    40
    60
    80
    100
    count
    Force values (n ≈ 2000)
    

    View Slide

33. Force Properties
    Precision
    • Reported force resolution is very precise
    (600 possible values, or ≈0.2% resolution)
    • Much more precise than UITouch.radius
    information
    • But is it just noise? How accurate?
    

    View Slide

34. My Quest For a Touch Pedestal
    • UIKit API: all force detection requires
    capacitive touch detection
    • To measure force (e.g., to weigh an object) we
    need a weightless finger to activate touch, a
    touch pedestal
    • Many things don't work
    

    View Slide

35. View Slide

36. View Slide

37. View Slide

38. View Slide

39. View Slide

40. View Slide

41. View Slide

42. View Slide

43. View Slide

44. View Slide

45. View Slide

46. View Slide

47. View Slide

48. View Slide

49. uh, just how does
    capacitive touch
    work anyway?
    

    View Slide

50. View Slide

51. View Slide

52. View Slide

53. View Slide

54. View Slide

55. View Slide

56. View Slide

57. View Slide

58. View Slide

59. View Slide

60. 1 2 3 4 5 6
    CGFloat
    0
    1
    2
    3
    4
    Newtons
    physicalforce vs UITouch.force
    

    View Slide

61. View Slide

62. Force Properties
    • UITouch.force seems to measure actual force
    • UIKit's minimum reported force values seem
    physically meaningful (< 5 grams, e.g., 3
    playing cards)
    • maximumPossibleForce is around 0.5 kg
    

    View Slide

63. Force Properties
    Let a thousand user
    experiences bloom!
    !"#$%
    

    View Slide

64. • Drawing
    • Music
    • Game controls
    • Text selection (press to expand selection)
    • Whacky easter eggs (press to "shatter", to
    "wobble")
    • Measure the physical world:
    • postal scale, breath strength, etc.
    

    View Slide

65. But… is this just
    right click?
    

    View Slide

66. Resources
    • Apple's "Adopting 3D Touch on iPhone" and
    sample code
    • Tools and demos:
    https://github.com/algal/TouchVisualizer
    

    View Slide

67. The Amazing Powers
    & Mesmerizing Secrets
    of 3D Touch
    @alexisgallagher
    

    View Slide