The Generalized Perceived Input Point Model and How to Double Touch Accuracy by Extracting Fingerprints

It is generally assumed that touch input cannot be accurate because of the fat finger problem, i.e., the softness of the fingertip combined with the occlusion of the target by the finger. In this paper, we show that this is not the case. We base our argument on a new model of touch inaccuracy. Our model is not based on the fat finger problem, but on the perceived input point model. In its published form, this model states that touch screens report touch location at an offset from the intended target. We generalize this model so that it represents offsets for individual finger postures and users. We thereby switch from the traditional 2D model of touch to a model that considers touch a phenomenon in 3-space. We report a user study, in which the generalized model explained 67% of the touch inaccuracy that was previously attributed to the fat finger problem.

In the second half of this paper, we present two devices that exploit the new model in order to improve touch accuracy. Both model touch on per-posture and per-user basis in order to increase accuracy by applying respective offsets. Our RidgePad prototype extracts posture and user ID from the user’s fingerprint during each touch interaction. In a user study, it achieved 1.8 times higher accuracy than a simulated capacitive baseline condition. A prototype based on optical tracking achieved even 3.3 times higher accuracy. The increase in accuracy can be used to make touch interfaces more reliable, to pack up to 3.3^2 > 10 times more controls into the same surface, or to bring touch input to very small mobile devices.

More information on http://www.christianholz.net/generalized_perceived_input_point_model.html

Transcript

1. The Generalized Perceived Input Point Model and How to Double Touch Accuracy by Extracting Fingerprints christian holz | patrick baudisch
2. fat nger
3. fat nger
4. so touch is inaccurate or is it?
5. could it be that it is not the ngers but our touch devices that are wrong?
6. Part 1 (science): even though screens are 2D, pointing is not Part 2 (engineering): sensing ngers in 3D ! highly accurate touch
7. no fat nger we claim there is problem
8. perceived instead, almost all observed targeting error comes from problem input point
9. perceived input point problem target [Vogel&Baudisch 2007]
10. perceived input point problem target [Vogel&Baudisch 2007] touch device perceives
11. why we hope it’s the perceived input point problem?
12. o set why we hope it’s the perceived input point problem?
13. o set why we hope it’s the perceived input point problem? we can correct for it
14. o set why we hope it’s the perceived input point problem? the fat nger problem, in contrast is always noise = error we can correct for it
15. while there is always an o set, we hypothesize that the o set depends on the pointing situation our main hypothesis
16. so what does “pointing situation” mean?
17. ≠ [iPhone, Wang et al. ’09] 1yaw
18. ≠ [Forlines et al. CHI’07] 2pitch
19. ≠ 3roll
20. ≠ 4users: nger shape
21. ≠ target target 4users: mental model
22. (… and there might be more e.g., head position/parallax…)
23. a non 2D-model
24. x/y current model screen
25. x/y x/y current model touch pad screen
26. nD touch pad screen proposed model x/y
27. user study 1
28. task
29. 2. acquire target precisely 1. push okay task
30. footswitch on-screen instructions controlled head position ! parallax capacitive touch pad
31. 1 yaw
32. !"# $%# &%# "# '&%# 2roll pitch constant
33. 3pitch !"# (%# $%# )%# &%# roll constant
34. 4user 12 participants all students, so di erences among them will be lower bound
35. *+*,-./,012.!"#$!%"%&'(&'&%$)&1/./3*./4563.2461748 dependent
36. *+*,-./,012.!"#$!%"%&'(&'&%$)&1/./3*./4563.2461748 dependent
37. *+*,-./,012.!"#$!%"%&'(&'&%$)&1/./3*./4563.2461748 dependent 648/1089.!%: 4;.122.91<=2*9>
38. main e ects for roll, pitch, yaw, & participant ID hypotheses
39. if the additional IVs had no impact, we would expect to see something like this
40. or touch locations indeed fall into clusters…
41. 2 yaw × 2 sessions (pitch, roll) × 5 angles × 6 repetitions per angle × 5 blocks = 600 trials / participant 12 participants design
42. results
43. !"" #"" $"" %"" &"" '!"" '#"" '$"" '%"" '&"" ()*+,(,-.*/0(-1230,.41( 45)0165) 45)0)-//:4,(23 45)0165)90)-//:4,(23 45)07*8 45)0*// 45)07*890165) 45)07*890)-//:4,(23 ?5@48.90A*.;4,.!%:.1665,16- results
44. !"" #"" $"" %"" &"" '!"" '#"" '$"" '%"" '&"" ()*+,(,-.*/0(-1230,.41( 45)0165) 45)0)-//:4,(23 45)0165)90)-//:4,(23 45)07*8 45)0*// 45)07*890165) 45)07*890)-//:4,(23 ?5@48.90A*.;4,.!%:.1665,16- results requires 15mm button
45. !"" #"" $"" %"" &"" '!"" '#"" '$"" '%"" '&"" ()*+,(,-.*/0(-1230,.41( 45)0165) 45)0)-//:4,(23 45)0165)90)-//:4,(23 45)07*8 45)0*// 45)07*890165) 45)07*890)-//:4,(23 ?5@48.90A*.;4,.!%:.1665,16- results requires 5.4mm button requires 15mm button
46. !"" #"" $"" %"" &"" '!"" '#"" '$"" '%"" '&"" ()*+,(,-.*/0(-1230,.41( 45)0165) 45)0)-//:4,(23 45)0165)90)-//:4,(23 45)07*8 45)0*// 45)07*890165) 45)07*890)-//:4,(23 ~three times more accurate ?5@48.90A*.;4,.!%:.1665,16- results requires 5.4mm button requires 15mm button
47. !"" #"" $"" %"" &"" '!"" '#"" '$"" '%"" '&"" ()*+,(,-.*/0(-1230,.41( 45)0165) 45)0)-//:4,(23 45)0165)90)-//:4,(23 45)07*8 45)0*// 45)07*890165) 45)07*890)-//:4,(23 ~three times more accurate allows 3x smaller devices ?5@48.90A*.;4,.!%:.1665,16- results requires 5.4mm button requires 15mm button
48. target 1yaw 1cm (participant #4, roll varied)
49. target 1yaw 1cm (participant #4, roll varied)
50. 2pitch 90° 65° 45° 25° 15° 1cm (participant #10, tilt varied)
51. (participant #1, roll varied) 3roll 1cm 15° 15° 0° 45 90 -
52. ! ! " " # # $ $ % % & & ' ' ( ( )* )* )) )) )! )! ) ) all data by participant #1-#12 tilt 4users 1cm roll
53. ! ! " " # # $ $ % % & & ' ' ( ( )* )* )) )) )! )! ) ) all data by participant #1-#12 tilt 4users 1cm roll
54. participant #3 and #4 4users !"# $%# &"# '"# ("# '"# $%# ("# !"# &"# 1cm
55. !"" #"" $"" %"" &"" '!"" '#"" '$"" '%"" '&"" ()*+,(,-.*/0(-1230,.41( 45)0165) 45)0)-//74,(23 45)0165)80)-//74,(23 45)09*: 45)0*// 45)09*:80165) 45)09*:80)-//74,(23 results ?5@48.90A*.;4,.!%:.1665,16- requires 15mm button requires 5.4mm button
57. how (in)accurate current devices are (button must be that big)
58. if we knew the pad orientation
59. if we knew nger angles
61. also need to know user ID, or we will overcompensate for people like this one
62. !"" #"" $"" %"" &"" '!"" '#"" '$"" '%"" '&"" ()*+,(,-.*/0(-1230,.41( 45)0165) 45)0)-//74,(23 45)0165)80)-//74,(23 45)09*: 45)0*// 45)09*:80165) 45)09*:80)-//74,(23 ?5@48.90A*.;4,.!%:.1665,16- requires 15mm button requires 5.4mm button results
63. !"" #"" $"" %"" &"" '!"" '#"" '$"" '%"" '&"" ()*+,(,-.*/0(-1230,.41( 45)0165) 45)0)-//74,(23 45)0165)80)-//74,(23 45)09*: 45)0*// 45)09*:80165) 45)09*:80)-//74,(23 ?5@48.90A*.;4,.!%:.1665,16- requires 15mm button requires 5.4mm button results fat nger problem compensation for respectively perceived input point
64. !"" #"" $"" %"" &"" '!"" '#"" '$"" '%"" '&"" ()*+,(,-.*/0(-1230,.41( 45)0165) 45)0)-//74,(23 45)0165)80)-//74,(23 45)09*: 45)0*// 45)09*:80165) 45)09*:80)-//74,(23 ?5@48.90A*.;4,.!%:.1665,16- requires 15mm button requires 5.4mm button shouldn’t we be able to make such a device? results fat nger problem compensation for respectively perceived input point
65. Part 1 (science): the Generalized Perceived Input Point Model Part 2 (engineering): sensing ngers in 3D ! highly accurate touch
67. optical tracker
68. optical tracker what do you mean: “not very practical”? retro re ective markers on nger 8 camera setup makes a great “gold standard” implementation to test the concept
69. mobile okay, maybe something a bit more
74. devices that sense touch and ngerprints already exist
75. gets everything a traditional touchpad gets + roll, pitch, yaw, & participant ID
76. ! this is very di erent from MicroRolls [Roudaut et al. CHI’09]
77. calibration have user touch a known target repeatedly and with di erent nger postures ! create database of ( ngerprint, target o set) use obtain ngerprint as user touches the device look up similar ngerprints in the database aggregate associated o sets (k nearest neighbor) and apply it algorithm
78. user study 2
79. 0 tracking device ngerprint optical tracker “simulated capacitive” (just contact area)
80. 1 yaw
81. 2 roll & ,422 ,422 '&%# "# &%# $%# !"# =0/63 &%# ! ! ! ! ! )%# ! $%# ! ! ! ! ! (%# ! !"# ! 3 pitch
82. optical beats simulated capacitive by ~3x (based on user study 1) ngerprint beats simulated capacitive (let’s nd out by how much) hypotheses
83. 2 rotations × 13 angles × 5 repetitions per angle × 5 blocks = 650 trials / participant 12 participants design
84. results
85. 0 mm 2 mm 4 mm 6 mm 8 mm 10 mm 12 mm 14 mm 16 mm contact area ngerprint scanner optical tracker results
86. as expected a factor of 3x 0 mm 2 mm 4 mm 6 mm 8 mm 10 mm 12 mm 14 mm 16 mm contact area ngerprint scanner optical tracker results
87. as expected a factor of 3x factor 1.8 0 mm 2 mm 4 mm 6 mm 8 mm 10 mm 12 mm 14 mm 16 mm contact area ngerprint scanner optical tracker results
88. as expected a factor of 3x factor 1.8 potential 0 mm 2 mm 4 mm 6 mm 8 mm 10 mm 12 mm 14 mm 16 mm contact area ngerprint scanner optical tracker results
89. Part 1 (science): the Generalized Perceived Input Point Model Part 2 (engineering): sensing ngers in 3D ! highly accurate touch
90. science 2D model of touch is an oversimpli cation 2/3 of touch inaccuracy is not the fat nger problem but a systematic e ect we call this: generalized perceived input point model
91. engineering bene ts 1. make more reliable touch input devices 2. use targeting aids (shift...) less often 3. make smaller mobile touch devices
92. future work speed up learning make real-time interactive prototype miniaturize technology for mobile use
93. human-computer interaction
94. ! ! " " # # $ $ % % & & ' ' ( ( )* )* )) )) )! )! ) ) tilt 1cm roll Q?