Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Multitoe

 Multitoe

Traditional touch screens suffer from a limitation of size—tabletops for example can’t become larger than arm’s length because you couldn’t reach the objects in the middle. With Multitoe, we propose a touch screen floor that could fill an entire room and allows for many more objects by using high-resultion multi-touch tracking while still allowing direct manipulation by touching. We used frustrated total internal reflection due to its ability to sense pressure. This allows us to see the users’ soles while they are walking over the floor. Based on this, we can recognize the foot posture and identify users, or control applications with a high degree of freedom.

1eb73b0f6071b878e2333cd625199f31?s=128

Konstantin Käfer

October 05, 2010
Tweet

Transcript

  1. Thomas Augsten Konstantin Käfer René Meusel Caroline Fetzer Dorian Kanitz

    Thomas Stoff Torsten Becker Christian Holz Patrick Baudisch Multitoe
  2. how to extend direct manipulation to 10,000 of objects we

    think that interaction using feet could play a role here
  3. None
  4. what if my application
 produces more data?

  5. unreachable

  6. 1.  allow manipulating 10,000 of objects 2.  maintain direct manipulation

    from tabletop Goal:
  7. related
 work

  8. Cruz-Neira, C. CAVE SIGGRAPH 1993

  9. Help me pull that cursor. Krogh, P. G. OZCH 2004

  10. Grønbæk, K. ACE 2007 iGameFloor

  11. Luminvision AdVis floor

  12. we were surprised how limited
 interaction is on these floors…

  13. touch is limited… Buxton, Interact b90

  14. Table: through the air

  15. porting this to the floor…

  16. None
  17. hard to escape gravity for >1sec

  18. None
  19. for direct manipulation we need multiple states

  20. (we are still in related work)
 borrowing from
 gait analysis

  21. Expressive Footwear Paradiso J., ISWC 1997

  22. The Smart Floor Orr, R. J. CHI 2000

  23. Choi/Ricci, IEEE ICSMC 1997 Toe Heel

  24. combine floors + gait analysis

  25. hardware

  26. iGameFloor uses Front DI

  27. key decision behind multitoe:
 add FTIR

  28. None
  29. None
  30. None
  31. 1. support glass (34mm-60mm) bringing FTIR to floor
 requires almost

    no modification
  32. 0   20   40   60   80  

    100   pixel brightness [%] [kg/cm2] 1   0   log 2. need a harder compliant surface
  33. Prototype

  34. direct manipulation

  35. as the name says:
 mani-pulation is about hands

  36. what does direct manipulation
 mean when using feet?

  37. à we conducted a few studies
 (not scientific, no hypotheses,


    but to inform design)
  38. interacting with many objects 3 stepping on things 2 invoking

    menus 4 creating a passive tracking state 1
  39. how to walk across a floor
 without setting things off

  40. (as mentioned earlier)
 we distinguish lgaitsz ok, but there are

    lots of gaits which on discoverable & ergonomic?
  41. how do users want to walk across a button
 without

    activating it study 1:
  42. not all are equally plausible… activation on dwell à  users

     can  never  stop  
  43. Strategy To activate To not activate # Part of foot

    tap (ball only) walk 8 walk tiptoe 1 walk on ball walk on heel 1 Amount of pressure stomp walk 5 jump onto walk 2 Temporal double-tap walk 2 Dwell (with both feet) walk quickly 5 Left-right right foot left foot 1 Spatial walk across centre walk edge of button 5
  44. None
  45. None
  46. gait recognition only possible because of FTIR

  47. interacting with many objects 3 stepping on objects 2 invoking

    menus 4 creating a passive tracking state 1
  48. here  are  three  bu,ons.   which  ones  do  users  expect

     to  be  depressed?
  49. If  these  hexes  were  bu,ons,   which  should  be  depressed?

  50. participants painted in the buttons
 they thought their shoe should

    depress study 2:
  51. only 2 referred to actual contact area

  52. only 2 referred to actual contact area

  53. majority of users: projection of shoe

  54. majority of users: projection of shoe

  55. same thing but only those >50% covered

  56. for most not contact area, but projection

  57. ok, so projection is harder with FTIR reconstruct by completing

    shoeprint front di ftir
  58. interacting with many objects 3 stepping on objects 2 invoking

    menus 4 creating a passive tracking state 1
  59. if we go with the projection model,
 interfaces become huge

    (keyboard)
  60. small objects if we want 10,000 objects,
 large objects are

    not what we want
  61. None
  62. None
  63. hotspot we need to reduce the foot to a

  64. targeting using a hotspot study 3:

  65. 3.1cm     3.5cm     Medium   1.5cm  

      1.7cm     Small   5.3cm     5.8cm     Large   smaller than
  66. None
  67. error rate per character Large 0 % 10 % 20

    % 30 % Medium Small 3.0% 9.5% 28.6% outside keyboard neighboring key wrong key <10% error on 3cm buttons
 < 3% error on 5cm buttons
  68. 3x4 cm target 10.000  3x4cm  objects   we  need  a

     3m  x  4m  floor this is quite feasible…
 (we are building one right now)
  69. locking hotspot to a specific position on the sole requires

    FTIR
  70. interacting with many objects 3 stepping on objects 2 invoking

    menus 4 creating a passive tracking state 1 (still in)
  71. this was under the assumption that users agree on where

    the hotspot is. Do they?
  72. step here

  73. None
  74. no, they disagree pretty massively

  75. None
  76. 8.4cm

  77. 8.4cm   3.5cm   2.2cm   5.0cm   free
 choice

    shoe tip big toe ball
  78. à let every user pick their hotspot

  79. None
  80. None
  81. personalization works
 because of FTIR

  82. interacting with many objects 3 stepping on objects 2 invoking

    menus 4 creating a passive tracking state 1
  83. distances can get long… menu

  84. Tap and wait Jump Heel in Front Stomp Tap with

    heel Clutching heels Edge of foot Slide with Foot how do users invoke a context menu study 4:
  85. None
  86. reliable decision contact detection is better with FTIR

  87. interacting with many objects 3 stepping on objects 2 invoking

    menus 4 creating a passive tracking state 1 these four things together enable direct manipulation, ftir is key
  88. algorithms

  89. sole recognition

  90. raw - DI raw we use frontDI to locate the

    foot
  91. raw FTIR we use FTIR to identify user and gait

  92. User Identification User identification:
 compare to user database: based on


    center points of blobs—not great
  93. log polar of Sensed image DFT exploring: comparing lfingerprintsl in

    frequency domain, seems more robust
  94. log polar of reference image DFT exploring: comparing lfingerprintsl in

    frequency domain, seems more robust
  95. what else…

  96. backwards strafe right strafe left forward fire look left

  97. None
  98. head tracking

  99. None
  100. None
  101. future work

  102. head tracking got us inspired. What else can we infer

    about space above the floor?
  103. smart rooms based on multitouch
 Can we look after inhabitants?

  104. None
  105. None
  106. None
  107. None
  108. conclusion

  109. 1.  allow manipulating 10,000 of objects 2.  maintain direct manipulation

    from tabletop 3.  gravity kills the passive tracking state 4.  FTIR à per-pxl pressure à gait recognition 5.  gravity brings in a ubicomp perspective summary:
  110. the team: thomas, thomas, dorian, & rene

  111. thanks to Christian, Torsten, Patrick Baudisch

  112. Human-Computer Interaction Lab Thanks to everyone at the

  113. Thomas Augsten Konstantin Käfer René Meusel Caroline Fetzer Dorian Kanitz

    Thomas Stoff Torsten Becker Christian Holz Patrick Baudisch thanks!