Augmented Typing: 映像と音の演出付与によるキーボードタイピング体験の拡張 / Augmented Typing

2017年6月2日(金)に東京大学で開催されたEC44の発表資料。論文はこちら → https://goo.gl/hrJ1pl

Transcript

1. Augmented Typing: ө૾ͱԻͷԋग़෇༩ʹΑΔ
   ΩʔϘʔυλΠϐϯάମݧͷ֦ு
   ౬ଜ ཌྷ†
   ɼதଜ ૱࢙‡
   † ๺཮ઌ୺Պֶٕज़େֶӃେֶ
   ‡ ໌࣏େֶ
   ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
   

   View Slide

2. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
   ͸͡ΊʹɿΩʔϘʔυ
   • PCͷจࣈೖྗσόΠεͷओྲྀ͸ΩʔϘʔυ
   • Ωʔ഑ྻɺετϩʔΫɺଧ伴ԻͳͲͷ༷ʑͳ৚͔݅Β
   ࣗ෼ͷ޷Έʹ߹ͬͨ΋ͷΛ࢖༻͢Δ
   2
   Happy Hacking Keyboard[1] Kinesis[2]
   

   View Slide

3. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
   ͸͡ΊʹɿσόΠεͷιϑτ΢ΣΞͰͷΞϨϯδ
   • PC΍εϚϗͰนࢴ΍ணϝϩΛม͑Δ
   • Emacs΍VimΛ࢖͍΍͍͢Α͏ʹઃఆ͢Δ
   3
   Apple iPhone Vim
   

   View Slide

4. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
   ͸͡Ίʹ
   • ΩʔϘʔυʹιϑτ΢ΣΞͰͷΞϨϯδΛՃ͑Δ͜ͱʹΑͬ
   ͯλΠϐϯάମݧΛ֦ு͢ΔγεςϜAugmented TypingΛ
   ఏҊ
   4
   ΩʔϘʔυ
   λΠϐϯά
   ιϑτ΢ΣΞ
   Ͱͷ
   ΞϨϯδ
   

   View Slide

5. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
   ؔ࿈ݚڀɿө૾දࣔʹΑΔΠϯλϑΣʔε֦ு
   5
   Optimus[3] Touch-Display Keyboard(Block et al. 2010)[4]
   Touch Bar[5]
   

   View Slide

6. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
   ؔ࿈ݚڀɿԋग़෇༩ʹΑΔମݧ֦ு
   6
   PingPongPlus(Ishii et al.,1999)[8]
   OpenPool(Kato et al., 2013)[9]
   ө૾૷ஔͱͯ͠ͷϐΞϊ(ؠҪ,1995)[6]
   

   View Slide

7. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
   ؔ࿈ݚڀɿԋग़෇༩ʹΑΔֶशࢧԉͱ࡞ۀࢧԉ
   7
   ϐΞϊֶशࢧԉ(Takegawa et al., 2013)[12] ᝨԋ૗ֶशࢧԉ(౔ҪͱٶԼ, 2017)[14]
   ిࢠ޻࡞ֶशࢧԉ
   (Akiyama and Miyashita, 2014)[16]
   ݐங෺଄࡞ࢧԉ
   (Yoshida et al., 2015)[17]
   ϒϩοΫ଄࡞ࢧԉ
   (ڮຊͱ௣ඌ, 2016)[18]
   

   View Slide

8. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
   Augmented Typing
   • ࣗ෼͚ͩͷ૊Έ߹ΘͤʹΑΓॴ༗ཉΛຬͨ͢
   • ૊Έ߹Θͤͷڞ༗΍ެ։
   8
   ଧ伴
   ө૾ԋग़
   ߴ༲ײ
   ຅ೖײ
   ༊΍͠
   Իڹԋग़
   

   View Slide

9. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
   ࣮૷
   • ϋʔυ΢ΣΞɿMacBook + ϓϩδΣΫλʔ + ΩʔϘʔυ
   • ιϑτ΢ΣΞɿJavaScript (p5.js)
   9
   ιʔείʔυ͸GitHubʹͯެ։
   https://github.com/yumu19/augmentedtyping
   

   View Slide

10. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ධՁ࣮ݧ
    • ʮҹ৅ධՁʯͱʮύϥϝʔλ࠷దԽʯͷ2छྨͷ࣮ݧ
    • ͍ͣΕ΋ࢴͷΞϯέʔτʹΑΔճ౴
    • ඃݧऀ͸৘ใܥͷେֶੜ͓ΑͼେֶӃੜ11໊
    • େֶ಺ͷֶੜډࣨͰ࣮ࢪ
    • ө૾ԋग़ͱԻڹԋग़ͷධՁΛಠཱ࣮ͯ͠ࢪ
    • ө૾ԋग़ͷධՁ࣌ʹ͸ແԻ
    • Իڹԋग़ͷධՁ࣌ʹ͸ޮՌө૾ͳ͠
    10
    

    View Slide

11. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ධՁ࣮ݧɿө૾ԋग़
    • 5छྨͷө૾ԋग़ΛධՁ
    11
    particle ray character
    wave stellar
    

    View Slide

12. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ධՁ࣮ݧɿԻڹԋग़
    • 6छྨͷԻڹԋग़ΛධՁ
    12
    keyboard digital1
    typewriter
    digital2 cat
    wadaiko
    

    View Slide

13. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ҹ৅ධՁ
    • 7߲໨ʹ͍ͭͯͷ7ஈ֊ϦοΧʔτई౓ʹΑΔධՁ
    • ઒࡚ͱҪखޱʹΑΔಈըө૾ͷҹ৅ධՁ[21]Λࢀߟ
    13
    ҹ৅߲໨ ධՁई౓ʢ1ʙ7ʣ
    ഭྗ ഭྗ͕ͳ͍ ʙ ഭྗ͕͋Δ
    ܰշ͞ ॏʑ͍͠ ʙ ܰշ
    Թྫྷ ྫྷ͍ͨ ʙ Թ͔͍
    ᓔಃ͠͞ ͏ͬͱ͏͍͠ ʙ ͏ͬͱ͏͘͠ͳ͍
    ඒ͠͞ ඒ͘͠ͳ͍ ʙ ඒ͍͠
    ༂ಈײ ༂ಈײ͕ͳ͍ ʙ ༂ಈײ͕͋Δ
    ೿ख͞ ஍ຯ ʙ ೿ख
    

    View Slide

14. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ҹ৅ධՁ݁Ռʢശͻ͛ਤʣ
    • ө૾ʹൺ΂Իڹԋग़ͷํ͕શମతʹ͹Β͖͕ͭେ͖͘ɼҹ৅ͷݸਓ͕ࠩେ͖͍ɽ
    • ᓔಃ͠͞ɼඒ͠͞ͷ2߲໨Ͱҹ৅ධՁͷ෼෍ʹ༗ҙ͕ࠩݟΒΕͨ
    • ͍ͣΕͷ߲໨΋ө૾ԋग़ͷํ͕ධՁ͕ߴ͍ʢᓔಃ͘͠ͳ͘ɺඒ͍͠ʣ
    14
    ө૾ԋग़ Իڹԋग़
    

    View Slide

15. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ҹ৅ධՁ݁ՌʢϨʔμʔνϟʔτʣ
    15
    ө૾ԋग़ Իڹԋग़
    ԋग़ʹΑΒͣൺֱతۉ౳ ԋग़ʹΑͬͯಛ௃͕ҟͳΔ
    

    View Slide

16. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ޮՌө૾ͷ࠷΋৺஍Α͍ύϥϝʔλͷௐࠪ
    • 4ͭͷޮՌө૾ʹ͍ͭͯ1ͭͳ͍͠2ͭͷύϥϝʔλΛʮ࠷΋৺஍Α͍ͱ
    ײ͡Δʯ΋ͷʹௐ੔
    • ύϥϝʔλมߋ͸ϒϥ΢β্ͷεϥΠμʔΛඃݧऀࣗ਎Ͱௐ੔͢Δ͜ͱ
    ʹΑΓ࣮ࢪ
    16
    ޮՌө૾ ύϥϝʔλ1 ύϥϝʔλ2
    particle ཻࢠ਺ ֦ࢄൣғ
    ray ଎౓ ֦ࢄൣғ
    character ଎౓ -
    wave ଎౓ ֦ࢄൣғ
    

    View Slide

17. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࠷΋৺஍Α͍ύϥϝʔλ(particle)
    17
    άϧʔϓB
    ཻࢠີ౓͕Ұఆ
    άϧʔϓA
    ֦ࢄൣғ͕Ұఆ
    R2=0.19899
    R2=0.94539
    

    View Slide

18. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࠷΋৺஍Α͍ύϥϝʔλ(wave)
    ʮ೾໲ͷੜ੒͔Βফ໓·Ͱͷ࣌ؒʯ͕ҹ৅ʹӨڹΛ༩͑ΔཁҼͱͳ͍ͬͯΔ
    ͜ͱΛࣔࠦ͢Δ
    18
    R2=0.31252
    

    View Slide

19. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࠷దύϥϝʔλͷௐࠪ݁Ռ(ray,character)
    ෼෍͔ΒͦͷಛੑΛݟग़ͤͣ
    19
    ray character
    

    View Slide

20. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࣗ༝هड़ʹΑΔײ૝ʢө૾ԋग़ʣ
    20
    ્֐ੑʹ͍ͭͯ
    ө૾ԋग़΁ͷ஫໨ʹ͍ͭͯ
    • ը໘ݟͳ͕ΒଧͭͱͲ͏ײ͡Δͷ͔ؾʹͳΓ·ͨ͠(all)
    • ΩʔϘʔυΛݟͳ͍Ͱࢹքͷ୺ʹগ͠͏ͭΔ͘Β͍ͩͱָ͍͠(wave)
    • ൣғΛ޿͛͗͢ΔͱɺଞͷΩʔ͕ݟͮΒ͘ͳΔ(wave)
    • ଞͷΤϑΣΫτʹൺ΂ɺλΠϐϯάΛ્֐͢Δ͜ͱͳ͘ΤϑΣΫτΛָ͠Ή͜
    ͱ͕Ͱ͖ͯྑ͔ͬͨɽ(character) ޮՌͷදࣔɾਐߦํ޲͕Өڹ
    

    View Slide

21. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࣗ༝هड़ʹΑΔײ૝ʢԻڹԋग़ʣ
    21
    λΠϐϯάମݧͷ֦ு
    • iPhoneͳͲͷԾ૝ͷΩʔϘʔυͱ͔ʹ͜͏͍͏Ի͕ͭ͘ͱɺଧ͍ͬͯΔײ͕͡ग़
    ΔͷͰྑ͍(keyboard)
    • ࡞ۀ͕͸͔ͲΔΑ͏ͳײ͕֮ಘΒΕͦ͏ɽ(keyboard)
    • ະདྷͷPCΛ࢖͍ͬͯΔײָ͕͍ͯ͡͠͠ɻөըͷϫϯγʔϯʹೖ͍ͬͯΔؾ
    ෼ɻ(digital1)
    • ະདྷײͷ͋ΔԻͱΩʔϘʔυͷݟͨ໨͕Ϛον͍ͯͯ͠ɺ͔ͳΓؾ͍͍࣋ͪϑΟʔ
    υόοΫʹײ͡·ͨ͠(digital2)
    ҧ࿨ײ
    • λΠϓϥΠλʔͷԻ͕͜ͷΩʔϘʔυΛଧ͍ͬͯΔͱ͖ʹௌ͑͜Δ͜ͱʹଟগҧ
    ࿨ײ͕͋Γ·ͨ͠ɻ(typewriter)
    • ͜ͷΩʔϘʔυͷݟͨ໨ɺUI͔Β͜ͷԻ͕ग़Δؾ͕͠ͳ͍ͷͰҧ࿨ײ͕͋Γ·
    ͨ͠ɻ(digital1)
    

    View Slide

22. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    σϞϯετϨʔγϣϯ
    • ܭ8ճͷσϞϯετϨʔγϣϯలࣔΛ࣮
    ࢪ
    • ଟ͘ͷମݧऀʹ޷ධ
    • Twitter౤ߘɼTV঺հ΁ͷ൓ԠΛTogetter
    ʹ·ͱΊͨ[22]
    22
    Πϕϯτ໊ ৔ॴ లࣔ೔࣌
    TMCN Vol.9 ೔ຊϚΠΫϩιϑτຊࣾ 2014/10/01
    GUGEN2014 ෋࢜ιϑτΞΩόϓϥβ 2014/12/13-14
    WIDE߹॓2016य़ ৴भদ୅ϩΠϠϧϗςϧ 2016/03/10
    Maker Faire Singapore 2016 Singapore University of Technology and Design 2016/06/25-26
    NTۚ୔2016 ۚ୔Ӻ΋ͯͳ͠υʔϜ 2016/07/16-18
    Maker Faire Tokyo 2016 ౦ژϏοάαΠτ 2016/08/07
    ळ༿ݪϝΠΧʔζ۞ָ෦ mogura 2016/08/07
    NICOGRAPH2016 ෋ࢁେֶ 2016/11/04
    

    View Slide

23. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ٞ࿦ɿ࣮૷ํ๏ʹ͍ͭͯ
    • ຊݚڀͰ༻͍ͨϓϩδΣΫγϣϯϚοϐϯάʹΑΔ࣮૷ͷ୹ॴ
    • ө૾͕ࢦʹөΔ
    • දࣔ૷ஔ(ࡾ٭+ϓϩδΣΫλ)͕अຐ
    • ͦͷଞͷ࣮૷ํ๏
    • ΩʔτοϓσΟεϓϨΠ෇͖ΩʔϘʔυ
    • λονύωϧࣜσΟεϓϨΠ
    • ARʹΑΔॏ৞දࣔ
    23
    Optimus[3] iPad HoloLens[20]
    

    View Slide

24. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ٞ࿦ɿλΠϐϯάֶशࢧԉ΁ͷԠ༻
    • ೔ຊޠೖྗࢧԉػೳ
    • ೖྗ͞ΕΔจࣈΛΩʔτοϓʹදࣔ
    • τϙϩδදࣔʹΑΔ৽͍͠λΠϐϯάֶश
    24
    

    View Slide

25. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ͓ΘΓʹ
    • Augmented Typingͱ͍͏ө૾ͱԻͷԋग़෇༩ʹΑΔΩʔϘʔ
    υλΠϐϯάମݧͷ֦ுΛఏҊͨ͠
    • γεςϜͷϓϩτλΠϓ࣮૷Λߦͬͨ
    • ҹ৅ධՁͷ݁Ռ
    • ޮՌө૾ͷ࠷΋৺஍Α͍ύϥϝʔλͷௐࠪͰ͸ɺ͍͔ͭ͘ͷ
    ύϥϝʔλ͕৺஍Α͞Λࢧ഑͢Δ͜ͱΛࣔࠦ͢Δ݁ՌΛಘͨ
    25
    ө૾ Ի
    ޮՌ͝ͱͷҹ৅ͷ͹Β͖ͭ খ͍͞ େ͖͍
    ᓔಃ͠͞ ᓔಃ͘͠ͳ͍
    ᓔಃ͍͠
    ඒ͠͞ ඒ͍͠ ඒ͘͠ͳ͍
    ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    

    View Slide

26. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࢀߟจݙ
    [1] PFU: Happy Hacking Keyboard, http://www.pfu.fujitsu.com/hhkeyboard/
    [2] Kinesis: Ergonomic keyboards and accessories, https://www.kinesis-ergo.com/
    [3] Optimus: Optimus Maximus Keyboard, http://www.artlebedev.com/everything/optimus/
    [4] Block, F., Gellersen, H. and Villar, N.: Touch-display keyboards: transforming keyboards into interactive sur- faces., Proceedings of the 28th
    International Conference on Human Factors in Computing Systems (CHI 2010), pp. 1145–1154 (2010).
    [5] Apple: MacBook Pro - Apple, https://www.apple.com/macbook-pro/ (2016).
    [6] ؠҪढ़༤: ө૾૷ஔͱͯ͠ͷϐΞϊ(1995)
    [8] Ishii, H., Wisneski, C., Orbanes, J., Chun, B. and Paradiso, J.: PingPongPlus: design of an athletic-tangible interface for computer-
    supported cooperative, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, pp. 394–401
    (1999).
    [9] Kato, J., Nakashima, T., Takeoka, H., Ogasawara, K., Murao, K., Shimokawa, T. and Sugimoto, M.: OpenPool: Community-based
    Prototyping of Digitally- augmented Billiard Table, GCCE ’13: Proceedings of the 2nd IEEE Global Conference on Consumer Electronics, pp.
    175–176 (2013).
    [12] Takegawa, Y., Tayanagi, E., Tsubakimoto, M. and Hi- rata, M.: Evaluation of a Piano Learning Support Sys- tem Focusing on the Learning
    Process, Proceeding of World Conference on Educational Media and Technol- ogy (EdMedia2013), pp. 2306–2314 (2013).
    [14]౔Ҫຑ༝Ղ, ٶԼ๕໌: ϓϩδΣΫγϣϯϚοϐϯάʹΑΔᝨԋ૗ֶशࢧԉγεςϜ, ৘ใॲཧֶձݚڀใࠂώϡʔϚϯίϯϐϡʔλΠ
    ϯλϥΫγϣϯ(HCI), Vol. 2017, N0. 15, pp. 1-8 (2017).
    [16] Akiyama, Y. and Miyashita, H.: Projectron mapping: The exercise and extension of augmented workspaces for learning electronic
    modeling through projection map- ping, Proceedings of the adjunct publication of the 27th annual ACM symposium on User interface software
    and technology, ACM, pp. 57–58 (2014).
    [17] Yoshida, H., Igarashi, T., Obuchi, Y., Takami, Y., Sato, J., Araki, M., Miki, M., Nagata, K., Sakai, K. and Igarashi, S.: Architecture-Scale
    Human-Assisted Addi- tive Manufacturing, Acm Transactions on Graphics, Vol. 34, No. 4, pp. 1–8 (2015).
    [18] ڮຊࡊఠ, ௣ඌҰ࿠: StudI/O: ϓϩδΣΫγϣϯϚοϐϯάʹΑΔτΠϒϩοΫͷ૊Έཱͯɾه࿥ࢧԉ, ৘ใॲཧֶձ࿦จࢽ, Vol. 57,
    No. 12, pp. 2577-2588 (2016).
    [20] Microsoft: HoloLens, https://www.microsoft.com/hololens (2016).
    [22] yumu19: cyberkeyboard൓Ԡ·ͱΊ - Togetter·ͱΊ, https://togetter.com/li/946782 (2016).
    26
    

    View Slide