Thomas Augsten
Konstantin Käfer
René Meusel
Caroline Fetzer
Dorian Kanitz
Thomas Stoff
Torsten Becker
Christian Holz
Patrick Baudisch
Multitoe
Slide 2
Slide 2 text
how to extend direct manipulation to
10,000 of objects
we think that interaction using feet
could play a role here
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
what if my application
produces more data?
Slide 5
Slide 5 text
unreachable
Slide 6
Slide 6 text
1. allow manipulating 10,000 of objects
2. maintain direct manipulation from tabletop
Goal:
Slide 7
Slide 7 text
related
work
Slide 8
Slide 8 text
Cruz-Neira, C. CAVE SIGGRAPH 1993
Slide 9
Slide 9 text
Help me pull that cursor.
Krogh, P. G. OZCH 2004
Slide 10
Slide 10 text
Grønbæk, K. ACE 2007
iGameFloor
Slide 11
Slide 11 text
Luminvision AdVis floor
Slide 12
Slide 12 text
we were surprised how limited
interaction is on these floors…
Slide 13
Slide 13 text
touch is limited…
Buxton, Interact b90
Slide 14
Slide 14 text
Table: through the air
Slide 15
Slide 15 text
porting this to the floor…
Slide 16
Slide 16 text
No content
Slide 17
Slide 17 text
hard to escape gravity for >1sec
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
for direct manipulation
we need multiple states
Slide 20
Slide 20 text
(we are still in related work)
borrowing from
gait analysis
Slide 21
Slide 21 text
Expressive Footwear
Paradiso J., ISWC 1997
Slide 22
Slide 22 text
The Smart Floor
Orr, R. J. CHI 2000
Slide 23
Slide 23 text
Choi/Ricci, IEEE ICSMC 1997
Toe
Heel
Slide 24
Slide 24 text
combine floors + gait analysis
Slide 25
Slide 25 text
hardware
Slide 26
Slide 26 text
iGameFloor uses Front DI
Slide 27
Slide 27 text
key decision behind multitoe:
add FTIR
Slide 28
Slide 28 text
No content
Slide 29
Slide 29 text
No content
Slide 30
Slide 30 text
No content
Slide 31
Slide 31 text
1. support glass
(34mm-60mm)
bringing FTIR to floor
requires almost no modification
Slide 32
Slide 32 text
0
20
40
60
80
100
pixel brightness [%]
[kg/cm2]
1
0
log
2. need a harder compliant surface
Slide 33
Slide 33 text
Prototype
Slide 34
Slide 34 text
direct
manipulation
Slide 35
Slide 35 text
as the name says:
mani-pulation is about hands
Slide 36
Slide 36 text
what does direct manipulation
mean when using feet?
Slide 37
Slide 37 text
à we conducted a few studies
(not scientific, no hypotheses,
but to inform design)
Slide 38
Slide 38 text
interacting with many objects
3
stepping on things
2
invoking menus
4
creating a passive tracking state
1
Slide 39
Slide 39 text
how to walk across a floor
without setting things off
Slide 40
Slide 40 text
(as mentioned earlier)
we distinguish lgaitsz
ok, but there are lots of gaits
which on discoverable & ergonomic?
Slide 41
Slide 41 text
how do users want to walk across a button
without activating it
study 1:
Slide 42
Slide 42 text
not all are equally plausible…
activation on dwell à
users
can
never
stop
Slide 43
Slide 43 text
Strategy To activate To not activate #
Part of foot
tap (ball only) walk 8
walk tiptoe 1
walk on ball walk on heel 1
Amount of pressure
stomp walk 5
jump onto walk 2
Temporal
double-tap walk 2
Dwell (with both
feet)
walk quickly 5
Left-right right foot left foot 1
Spatial walk across centre walk edge of button 5
Slide 44
Slide 44 text
No content
Slide 45
Slide 45 text
No content
Slide 46
Slide 46 text
gait recognition only possible
because of FTIR
Slide 47
Slide 47 text
interacting with many objects
3
stepping on objects
2
invoking menus
4
creating a passive tracking state
1
Slide 48
Slide 48 text
here
are
three
bu,ons.
which
ones
do
users
expect
to
be
depressed?
Slide 49
Slide 49 text
If
these
hexes
were
bu,ons,
which
should
be
depressed?
Slide 50
Slide 50 text
participants painted in the buttons
they thought their shoe should depress
study 2:
Slide 51
Slide 51 text
only 2 referred to actual contact area
Slide 52
Slide 52 text
only 2 referred to actual contact area
Slide 53
Slide 53 text
majority of users: projection of shoe
Slide 54
Slide 54 text
majority of users: projection of shoe
Slide 55
Slide 55 text
same thing but only those >50% covered
Slide 56
Slide 56 text
for most not contact area, but projection
Slide 57
Slide 57 text
ok, so projection is harder with FTIR
reconstruct by completing shoeprint
front di
ftir
Slide 58
Slide 58 text
interacting with many objects
3
stepping on objects
2
invoking menus
4
creating a passive tracking state
1
Slide 59
Slide 59 text
if we go with the projection model,
interfaces become huge (keyboard)
Slide 60
Slide 60 text
small objects
if we want 10,000 objects,
large objects are not what we want
Slide 61
Slide 61 text
No content
Slide 62
Slide 62 text
No content
Slide 63
Slide 63 text
hotspot
we need to reduce the foot to a
Slide 64
Slide 64 text
targeting using a hotspot
study 3:
Slide 65
Slide 65 text
3.1cm
3.5cm
Medium
1.5cm
1.7cm
Small
5.3cm
5.8cm
Large
smaller than
Slide 66
Slide 66 text
No content
Slide 67
Slide 67 text
error rate per character
Large
0 %
10 %
20 %
30 %
Medium Small
3.0%
9.5%
28.6%
outside keyboard
neighboring key
wrong key
<10% error on 3cm buttons
< 3% error on 5cm buttons
Slide 68
Slide 68 text
3x4 cm target
10.000
3x4cm
objects
we
need
a
3m
x
4m
floor
this is quite feasible…
(we are building one right now)
Slide 69
Slide 69 text
locking hotspot to a specific
position on the sole requires FTIR
Slide 70
Slide 70 text
interacting with many objects
3
stepping on objects
2
invoking menus
4
creating a passive tracking state
1
(still in)
Slide 71
Slide 71 text
this was under the assumption that users
agree on where the hotspot is. Do they?
Slide 72
Slide 72 text
step here
Slide 73
Slide 73 text
No content
Slide 74
Slide 74 text
no, they disagree pretty massively
Slide 75
Slide 75 text
No content
Slide 76
Slide 76 text
8.4cm
Slide 77
Slide 77 text
8.4cm
3.5cm
2.2cm
5.0cm
free
choice
shoe tip big toe ball
Slide 78
Slide 78 text
à let every user pick their
hotspot
Slide 79
Slide 79 text
No content
Slide 80
Slide 80 text
No content
Slide 81
Slide 81 text
personalization works
because of FTIR
Slide 82
Slide 82 text
interacting with many objects
3
stepping on objects
2
invoking menus
4
creating a passive tracking state
1
Slide 83
Slide 83 text
distances can get long…
menu
Slide 84
Slide 84 text
Tap and wait
Jump
Heel in Front
Stomp
Tap with heel
Clutching heels
Edge of foot
Slide with Foot
how do users invoke a context menu
study 4:
Slide 85
Slide 85 text
No content
Slide 86
Slide 86 text
reliable decision contact detection
is better with FTIR
Slide 87
Slide 87 text
interacting with many objects
3
stepping on objects
2
invoking menus
4
creating a passive tracking state
1
these four things together enable direct
manipulation, ftir is key
Slide 88
Slide 88 text
algorithms
Slide 89
Slide 89 text
sole recognition
Slide 90
Slide 90 text
raw - DI
raw
we use frontDI to locate the foot
Slide 91
Slide 91 text
raw
FTIR
we use FTIR to identify user and gait
Slide 92
Slide 92 text
User Identification
User identification:
compare to user database: based on
center points of blobs—not great
Slide 93
Slide 93 text
log polar of Sensed image DFT
exploring: comparing lfingerprintsl in
frequency domain, seems more robust
Slide 94
Slide 94 text
log polar of reference image DFT
exploring: comparing lfingerprintsl in
frequency domain, seems more robust
Slide 95
Slide 95 text
what else…
Slide 96
Slide 96 text
backwards strafe right
strafe left
forward fire look left
Slide 97
Slide 97 text
No content
Slide 98
Slide 98 text
head tracking
Slide 99
Slide 99 text
No content
Slide 100
Slide 100 text
No content
Slide 101
Slide 101 text
future work
Slide 102
Slide 102 text
head tracking got us inspired.
What else can we infer about
space above the floor?
Slide 103
Slide 103 text
smart rooms based on multitouch
Can we look after inhabitants?
Slide 104
Slide 104 text
No content
Slide 105
Slide 105 text
No content
Slide 106
Slide 106 text
No content
Slide 107
Slide 107 text
No content
Slide 108
Slide 108 text
conclusion
Slide 109
Slide 109 text
1. allow manipulating 10,000 of objects
2. maintain direct manipulation from tabletop
3. gravity kills the passive tracking state
4. FTIR à per-pxl pressure à gait recognition
5. gravity brings in a ubicomp perspective
summary:
Slide 110
Slide 110 text
the team: thomas, thomas, dorian, & rene
Slide 111
Slide 111 text
thanks to Christian, Torsten, Patrick Baudisch
Slide 112
Slide 112 text
Human-Computer Interaction Lab
Thanks to everyone at the
Slide 113
Slide 113 text
Thomas Augsten
Konstantin Käfer
René Meusel
Caroline Fetzer
Dorian Kanitz
Thomas Stoff
Torsten Becker
Christian Holz
Patrick Baudisch
thanks!