Multitoe

Traditional touch screens suffer from a limitation of size—tabletops for example can’t become larger than arm’s length because you couldn’t reach the objects in the middle. With Multitoe, we propose a touch screen floor that could fill an entire room and allows for many more objects by using high-resultion multi-touch tracking while still allowing direct manipulation by touching. We used frustrated total internal reflection due to its ability to sense pressure. This allows us to see the users’ soles while they are walking over the floor. Based on this, we can recognize the foot posture and identify users, or control applications with a high degree of freedom.

Transcript

1. Thomas Augsten Konstantin Käfer René Meusel Caroline Fetzer Dorian Kanitz Thomas Stoff Torsten Becker Christian Holz Patrick Baudisch Multitoe
2. how to extend direct manipulation to 10,000 of objects we think that interaction using feet could play a role here
4. what if my application
 produces more data?
5. unreachable
6. 1.  allow manipulating 10,000 of objects 2.  maintain direct manipulation from tabletop Goal:
7. related
 work
8. Cruz-Neira, C. CAVE SIGGRAPH 1993
9. Help me pull that cursor. Krogh, P. G. OZCH 2004
10. Grønbæk, K. ACE 2007 iGameFloor
11. Luminvision AdVis floor
12. we were surprised how limited
 interaction is on these floors…
13. touch is limited… Buxton, Interact b90
14. Table: through the air
15. porting this to the floor…
17. hard to escape gravity for >1sec
19. for direct manipulation we need multiple states
20. (we are still in related work)
 borrowing from
 gait analysis
21. Expressive Footwear Paradiso J., ISWC 1997
22. The Smart Floor Orr, R. J. CHI 2000
23. Choi/Ricci, IEEE ICSMC 1997 Toe Heel
24. combine floors + gait analysis
25. hardware
26. iGameFloor uses Front DI
27. key decision behind multitoe:
 add FTIR
31. 1. support glass (34mm-60mm) bringing FTIR to floor
 requires almost no modification
32. 0   20   40   60   80   100   pixel brightness [%] [kg/cm2] 1   0   log 2. need a harder compliant surface
33. Prototype
34. direct manipulation
35. as the name says:
 mani-pulation is about hands
36. what does direct manipulation
 mean when using feet?
37. à we conducted a few studies
 (not scientific, no hypotheses,
 but to inform design)
38. interacting with many objects 3 stepping on things 2 invoking menus 4 creating a passive tracking state 1
39. how to walk across a floor
 without setting things off
40. (as mentioned earlier)
 we distinguish lgaitsz ok, but there are lots of gaits which on discoverable & ergonomic?
41. how do users want to walk across a button
 without activating it study 1:
42. not all are equally plausible… activation on dwell à  users  can  never  stop  
43. Strategy To activate To not activate # Part of foot tap (ball only) walk 8 walk tiptoe 1 walk on ball walk on heel 1 Amount of pressure stomp walk 5 jump onto walk 2 Temporal double-tap walk 2 Dwell (with both feet) walk quickly 5 Left-right right foot left foot 1 Spatial walk across centre walk edge of button 5
46. gait recognition only possible because of FTIR
47. interacting with many objects 3 stepping on objects 2 invoking menus 4 creating a passive tracking state 1
48. here  are  three  bu,ons.   which  ones  do  users  expect  to  be  depressed?
49. If  these  hexes  were  bu,ons,   which  should  be  depressed?
50. participants painted in the buttons
 they thought their shoe should depress study 2:
51. only 2 referred to actual contact area
52. only 2 referred to actual contact area
53. majority of users: projection of shoe
54. majority of users: projection of shoe
55. same thing but only those >50% covered
56. for most not contact area, but projection
57. ok, so projection is harder with FTIR reconstruct by completing shoeprint front di ftir
58. interacting with many objects 3 stepping on objects 2 invoking menus 4 creating a passive tracking state 1
59. if we go with the projection model,
 interfaces become huge (keyboard)
60. small objects if we want 10,000 objects,
 large objects are not what we want
63. hotspot we need to reduce the foot to a
64. targeting using a hotspot study 3:
65. 3.1cm     3.5cm     Medium   1.5cm     1.7cm     Small   5.3cm     5.8cm     Large   smaller than
67. error rate per character Large 0 % 10 % 20 % 30 % Medium Small 3.0% 9.5% 28.6% outside keyboard neighboring key wrong key <10% error on 3cm buttons
 < 3% error on 5cm buttons
68. 3x4 cm target 10.000  3x4cm  objects   we  need  a  3m  x  4m  floor this is quite feasible…
 (we are building one right now)
69. locking hotspot to a specific position on the sole requires FTIR
70. interacting with many objects 3 stepping on objects 2 invoking menus 4 creating a passive tracking state 1 (still in)
71. this was under the assumption that users agree on where the hotspot is. Do they?
72. step here
74. no, they disagree pretty massively
76. 8.4cm
77. 8.4cm   3.5cm   2.2cm   5.0cm   free
 choice shoe tip big toe ball
78. à let every user pick their hotspot
81. personalization works
 because of FTIR
82. interacting with many objects 3 stepping on objects 2 invoking menus 4 creating a passive tracking state 1
83. distances can get long… menu
84. Tap and wait Jump Heel in Front Stomp Tap with heel Clutching heels Edge of foot Slide with Foot how do users invoke a context menu study 4:
86. reliable decision contact detection is better with FTIR
87. interacting with many objects 3 stepping on objects 2 invoking menus 4 creating a passive tracking state 1 these four things together enable direct manipulation, ftir is key
88. algorithms
89. sole recognition
90. raw - DI raw we use frontDI to locate the foot
91. raw FTIR we use FTIR to identify user and gait
92. User Identification User identification:
 compare to user database: based on
 center points of blobs—not great
93. log polar of Sensed image DFT exploring: comparing lfingerprintsl in frequency domain, seems more robust
94. log polar of reference image DFT exploring: comparing lfingerprintsl in frequency domain, seems more robust
95. what else…
96. backwards strafe right strafe left forward fire look left
98. head tracking
101. future work
102. head tracking got us inspired. What else can we infer about space above the floor?
103. smart rooms based on multitouch
 Can we look after inhabitants?
108. conclusion
109. 1.  allow manipulating 10,000 of objects 2.  maintain direct manipulation from tabletop 3.  gravity kills the passive tracking state 4.  FTIR à per-pxl pressure à gait recognition 5.  gravity brings in a ubicomp perspective summary:
110. the team: thomas, thomas, dorian, & rene
111. thanks to Christian, Torsten, Patrick Baudisch
112. Human-Computer Interaction Lab Thanks to everyone at the
113. Thomas Augsten Konstantin Käfer René Meusel Caroline Fetzer Dorian Kanitz Thomas Stoff Torsten Becker Christian Holz Patrick Baudisch thanks!