Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Augmented Typing: 映像と音の演出付与によるキーボードタイピング体験の拡張 / Augmented Typing

Augmented Typing: 映像と音の演出付与によるキーボードタイピング体験の拡張 / Augmented Typing

2017年6月2日(金)に東京大学で開催されたEC44の発表資料。論文はこちら → https://goo.gl/hrJ1pl

Tsubasa Yumura

June 02, 2017
Tweet

More Decks by Tsubasa Yumura

Other Decks in Research

Transcript

  1. Augmented Typing: ө૾ͱԻͷԋग़෇༩ʹΑΔ
    ΩʔϘʔυλΠϐϯάମݧͷ֦ு
    ౬ଜ ཌྷ†
    ɼதଜ ૱࢙‡
    † ๺཮ઌ୺Պֶٕज़େֶӃେֶ
    ‡ ໌࣏େֶ
    ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02

    View full-size slide

  2. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ͸͡ΊʹɿΩʔϘʔυ
    • PCͷจࣈೖྗσόΠεͷओྲྀ͸ΩʔϘʔυ
    • Ωʔ഑ྻɺετϩʔΫɺଧ伴ԻͳͲͷ༷ʑͳ৚͔݅Β
    ࣗ෼ͷ޷Έʹ߹ͬͨ΋ͷΛ࢖༻͢Δ
    2
    Happy Hacking Keyboard[1] Kinesis[2]

    View full-size slide

  3. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ͸͡ΊʹɿσόΠεͷιϑτ΢ΣΞͰͷΞϨϯδ
    • PC΍εϚϗͰนࢴ΍ணϝϩΛม͑Δ
    • Emacs΍VimΛ࢖͍΍͍͢Α͏ʹઃఆ͢Δ
    3
    Apple iPhone Vim

    View full-size slide

  4. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ͸͡Ίʹ
    • ΩʔϘʔυʹιϑτ΢ΣΞͰͷΞϨϯδΛՃ͑Δ͜ͱʹΑͬ
    ͯλΠϐϯάମݧΛ֦ு͢ΔγεςϜAugmented TypingΛ
    ఏҊ
    4
    ΩʔϘʔυ
    λΠϐϯά
    ιϑτ΢ΣΞ
    Ͱͷ
    ΞϨϯδ

    View full-size slide

  5. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ؔ࿈ݚڀɿө૾දࣔʹΑΔΠϯλϑΣʔε֦ு
    5
    Optimus[3] Touch-Display Keyboard(Block et al. 2010)[4]
    Touch Bar[5]

    View full-size slide

  6. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ؔ࿈ݚڀɿԋग़෇༩ʹΑΔମݧ֦ு
    6
    PingPongPlus(Ishii et al.,1999)[8]
    OpenPool(Kato et al., 2013)[9]
    ө૾૷ஔͱͯ͠ͷϐΞϊ(ؠҪ,1995)[6]

    View full-size slide

  7. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ؔ࿈ݚڀɿԋग़෇༩ʹΑΔֶशࢧԉͱ࡞ۀࢧԉ
    7
    ϐΞϊֶशࢧԉ(Takegawa et al., 2013)[12] ᝨԋ૗ֶशࢧԉ(౔ҪͱٶԼ, 2017)[14]
    ిࢠ޻࡞ֶशࢧԉ
    (Akiyama and Miyashita, 2014)[16]
    ݐங෺଄࡞ࢧԉ
    (Yoshida et al., 2015)[17]
    ϒϩοΫ଄࡞ࢧԉ
    (ڮຊͱ௣ඌ, 2016)[18]

    View full-size slide

  8. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    Augmented Typing
    • ࣗ෼͚ͩͷ૊Έ߹ΘͤʹΑΓॴ༗ཉΛຬͨ͢
    • ૊Έ߹Θͤͷڞ༗΍ެ։
    8
    ଧ伴
    ө૾ԋग़
    ߴ༲ײ
    ຅ೖײ
    ༊΍͠
    Իڹԋग़

    View full-size slide

  9. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࣮૷
    • ϋʔυ΢ΣΞɿMacBook + ϓϩδΣΫλʔ + ΩʔϘʔυ
    • ιϑτ΢ΣΞɿJavaScript (p5.js)
    9
    ιʔείʔυ͸GitHubʹͯެ։
    https://github.com/yumu19/augmentedtyping

    View full-size slide

  10. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ධՁ࣮ݧ
    • ʮҹ৅ධՁʯͱʮύϥϝʔλ࠷దԽʯͷ2छྨͷ࣮ݧ
    • ͍ͣΕ΋ࢴͷΞϯέʔτʹΑΔճ౴
    • ඃݧऀ͸৘ใܥͷେֶੜ͓ΑͼେֶӃੜ11໊
    • େֶ಺ͷֶੜډࣨͰ࣮ࢪ
    • ө૾ԋग़ͱԻڹԋग़ͷධՁΛಠཱ࣮ͯ͠ࢪ
    • ө૾ԋग़ͷධՁ࣌ʹ͸ແԻ
    • Իڹԋग़ͷධՁ࣌ʹ͸ޮՌө૾ͳ͠
    10

    View full-size slide

  11. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ධՁ࣮ݧɿө૾ԋग़
    • 5छྨͷө૾ԋग़ΛධՁ
    11
    particle ray character
    wave stellar

    View full-size slide

  12. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ධՁ࣮ݧɿԻڹԋग़
    • 6छྨͷԻڹԋग़ΛධՁ
    12
    keyboard digital1
    typewriter
    digital2 cat
    wadaiko

    View full-size slide

  13. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ҹ৅ධՁ
    • 7߲໨ʹ͍ͭͯͷ7ஈ֊ϦοΧʔτई౓ʹΑΔධՁ
    • ઒࡚ͱҪखޱʹΑΔಈըө૾ͷҹ৅ධՁ[21]Λࢀߟ
    13
    ҹ৅߲໨ ධՁई౓ʢ1ʙ7ʣ
    ഭྗ ഭྗ͕ͳ͍ ʙ ഭྗ͕͋Δ
    ܰշ͞ ॏʑ͍͠ ʙ ܰշ
    Թྫྷ ྫྷ͍ͨ ʙ Թ͔͍
    ᓔಃ͠͞ ͏ͬͱ͏͍͠ ʙ ͏ͬͱ͏͘͠ͳ͍
    ඒ͠͞ ඒ͘͠ͳ͍ ʙ ඒ͍͠
    ༂ಈײ ༂ಈײ͕ͳ͍ ʙ ༂ಈײ͕͋Δ
    ೿ख͞ ஍ຯ ʙ ೿ख

    View full-size slide

  14. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ҹ৅ධՁ݁Ռʢശͻ͛ਤʣ
    • ө૾ʹൺ΂Իڹԋग़ͷํ͕શମతʹ͹Β͖͕ͭେ͖͘ɼҹ৅ͷݸਓ͕ࠩେ͖͍ɽ
    • ᓔಃ͠͞ɼඒ͠͞ͷ2߲໨Ͱҹ৅ධՁͷ෼෍ʹ༗ҙ͕ࠩݟΒΕͨ
    • ͍ͣΕͷ߲໨΋ө૾ԋग़ͷํ͕ධՁ͕ߴ͍ʢᓔಃ͘͠ͳ͘ɺඒ͍͠ʣ
    14
    ө૾ԋग़ Իڹԋग़

    View full-size slide

  15. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ҹ৅ධՁ݁ՌʢϨʔμʔνϟʔτʣ
    15
    ө૾ԋग़ Իڹԋग़
    ԋग़ʹΑΒͣൺֱతۉ౳ ԋग़ʹΑͬͯಛ௃͕ҟͳΔ

    View full-size slide

  16. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ޮՌө૾ͷ࠷΋৺஍Α͍ύϥϝʔλͷௐࠪ
    • 4ͭͷޮՌө૾ʹ͍ͭͯ1ͭͳ͍͠2ͭͷύϥϝʔλΛʮ࠷΋৺஍Α͍ͱ
    ײ͡Δʯ΋ͷʹௐ੔
    • ύϥϝʔλมߋ͸ϒϥ΢β্ͷεϥΠμʔΛඃݧऀࣗ਎Ͱௐ੔͢Δ͜ͱ
    ʹΑΓ࣮ࢪ
    16
    ޮՌө૾ ύϥϝʔλ1 ύϥϝʔλ2
    particle ཻࢠ਺ ֦ࢄൣғ
    ray ଎౓ ֦ࢄൣғ
    character ଎౓ -
    wave ଎౓ ֦ࢄൣғ

    View full-size slide

  17. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࠷΋৺஍Α͍ύϥϝʔλ(particle)
    17
    άϧʔϓB
    ཻࢠີ౓͕Ұఆ
    άϧʔϓA
    ֦ࢄൣғ͕Ұఆ
    R2=0.19899
    R2=0.94539

    View full-size slide

  18. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࠷΋৺஍Α͍ύϥϝʔλ(wave)
    ʮ೾໲ͷੜ੒͔Βফ໓·Ͱͷ࣌ؒʯ͕ҹ৅ʹӨڹΛ༩͑ΔཁҼͱͳ͍ͬͯΔ
    ͜ͱΛࣔࠦ͢Δ
    18
    R2=0.31252

    View full-size slide

  19. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࠷దύϥϝʔλͷௐࠪ݁Ռ(ray,character)
    ෼෍͔ΒͦͷಛੑΛݟग़ͤͣ
    19
    ray character

    View full-size slide

  20. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࣗ༝هड़ʹΑΔײ૝ʢө૾ԋग़ʣ
    20
    ્֐ੑʹ͍ͭͯ
    ө૾ԋग़΁ͷ஫໨ʹ͍ͭͯ
    • ը໘ݟͳ͕ΒଧͭͱͲ͏ײ͡Δͷ͔ؾʹͳΓ·ͨ͠(all)
    • ΩʔϘʔυΛݟͳ͍Ͱࢹքͷ୺ʹগ͠͏ͭΔ͘Β͍ͩͱָ͍͠(wave)
    • ൣғΛ޿͛͗͢ΔͱɺଞͷΩʔ͕ݟͮΒ͘ͳΔ(wave)
    • ଞͷΤϑΣΫτʹൺ΂ɺλΠϐϯάΛ્֐͢Δ͜ͱͳ͘ΤϑΣΫτΛָ͠Ή͜
    ͱ͕Ͱ͖ͯྑ͔ͬͨɽ(character) ޮՌͷදࣔɾਐߦํ޲͕Өڹ

    View full-size slide

  21. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࣗ༝هड़ʹΑΔײ૝ʢԻڹԋग़ʣ
    21
    λΠϐϯάମݧͷ֦ு
    • iPhoneͳͲͷԾ૝ͷΩʔϘʔυͱ͔ʹ͜͏͍͏Ի͕ͭ͘ͱɺଧ͍ͬͯΔײ͕͡ग़
    ΔͷͰྑ͍(keyboard)
    • ࡞ۀ͕͸͔ͲΔΑ͏ͳײ͕֮ಘΒΕͦ͏ɽ(keyboard)
    • ະདྷͷPCΛ࢖͍ͬͯΔײָ͕͍ͯ͡͠͠ɻөըͷϫϯγʔϯʹೖ͍ͬͯΔؾ
    ෼ɻ(digital1)
    • ະདྷײͷ͋ΔԻͱΩʔϘʔυͷݟͨ໨͕Ϛον͍ͯͯ͠ɺ͔ͳΓؾ͍͍࣋ͪϑΟʔ
    υόοΫʹײ͡·ͨ͠(digital2)
    ҧ࿨ײ
    • λΠϓϥΠλʔͷԻ͕͜ͷΩʔϘʔυΛଧ͍ͬͯΔͱ͖ʹௌ͑͜Δ͜ͱʹଟগҧ
    ࿨ײ͕͋Γ·ͨ͠ɻ(typewriter)
    • ͜ͷΩʔϘʔυͷݟͨ໨ɺUI͔Β͜ͷԻ͕ग़Δؾ͕͠ͳ͍ͷͰҧ࿨ײ͕͋Γ·
    ͨ͠ɻ(digital1)

    View full-size slide

  22. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    σϞϯετϨʔγϣϯ
    • ܭ8ճͷσϞϯετϨʔγϣϯలࣔΛ࣮

    • ଟ͘ͷମݧऀʹ޷ධ
    • Twitter౤ߘɼTV঺հ΁ͷ൓ԠΛTogetter
    ʹ·ͱΊͨ[22]
    22
    Πϕϯτ໊ ৔ॴ లࣔ೔࣌
    TMCN Vol.9 ೔ຊϚΠΫϩιϑτຊࣾ 2014/10/01
    GUGEN2014 ෋࢜ιϑτΞΩόϓϥβ 2014/12/13-14
    WIDE߹॓2016य़ ৴भদ୅ϩΠϠϧϗςϧ 2016/03/10
    Maker Faire Singapore 2016 Singapore University of Technology and Design 2016/06/25-26
    NTۚ୔2016 ۚ୔Ӻ΋ͯͳ͠υʔϜ 2016/07/16-18
    Maker Faire Tokyo 2016 ౦ژϏοάαΠτ 2016/08/07
    ळ༿ݪϝΠΧʔζ۞ָ෦ mogura 2016/08/07
    NICOGRAPH2016 ෋ࢁେֶ 2016/11/04

    View full-size slide

  23. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ٞ࿦ɿ࣮૷ํ๏ʹ͍ͭͯ
    • ຊݚڀͰ༻͍ͨϓϩδΣΫγϣϯϚοϐϯάʹΑΔ࣮૷ͷ୹ॴ
    • ө૾͕ࢦʹөΔ
    • දࣔ૷ஔ(ࡾ٭+ϓϩδΣΫλ)͕अຐ
    • ͦͷଞͷ࣮૷ํ๏
    • ΩʔτοϓσΟεϓϨΠ෇͖ΩʔϘʔυ
    • λονύωϧࣜσΟεϓϨΠ
    • ARʹΑΔॏ৞දࣔ
    23
    Optimus[3] iPad HoloLens[20]

    View full-size slide

  24. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ٞ࿦ɿλΠϐϯάֶशࢧԉ΁ͷԠ༻
    • ೔ຊޠೖྗࢧԉػೳ
    • ೖྗ͞ΕΔจࣈΛΩʔτοϓʹදࣔ
    • τϙϩδදࣔʹΑΔ৽͍͠λΠϐϯάֶश
    24

    View full-size slide

  25. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ͓ΘΓʹ
    • Augmented Typingͱ͍͏ө૾ͱԻͷԋग़෇༩ʹΑΔΩʔϘʔ
    υλΠϐϯάମݧͷ֦ுΛఏҊͨ͠
    • γεςϜͷϓϩτλΠϓ࣮૷Λߦͬͨ
    • ҹ৅ධՁͷ݁Ռ
    • ޮՌө૾ͷ࠷΋৺஍Α͍ύϥϝʔλͷௐࠪͰ͸ɺ͍͔ͭ͘ͷ
    ύϥϝʔλ͕৺஍Α͞Λࢧ഑͢Δ͜ͱΛࣔࠦ͢Δ݁ՌΛಘͨ
    25
    ө૾ Ի
    ޮՌ͝ͱͷҹ৅ͷ͹Β͖ͭ খ͍͞ େ͖͍
    ᓔಃ͠͞ ᓔಃ͘͠ͳ͍
    ᓔಃ͍͠
    ඒ͠͞ ඒ͍͠ ඒ͘͠ͳ͍
    ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02

    View full-size slide

  26. ৘ใॲཧֶձ ୈ44ճΤϯλςΠϯϝϯτίϯϐϡʔςΟϯάݚڀձˏ౦ژେֶɹ2017/06/02
    ࢀߟจݙ
    [1] PFU: Happy Hacking Keyboard, http://www.pfu.fujitsu.com/hhkeyboard/
    [2] Kinesis: Ergonomic keyboards and accessories, https://www.kinesis-ergo.com/
    [3] Optimus: Optimus Maximus Keyboard, http://www.artlebedev.com/everything/optimus/
    [4] Block, F., Gellersen, H. and Villar, N.: Touch-display keyboards: transforming keyboards into interactive sur- faces., Proceedings of the 28th
    International Conference on Human Factors in Computing Systems (CHI 2010), pp. 1145–1154 (2010).
    [5] Apple: MacBook Pro - Apple, https://www.apple.com/macbook-pro/ (2016).
    [6] ؠҪढ़༤: ө૾૷ஔͱͯ͠ͷϐΞϊ(1995)
    [8] Ishii, H., Wisneski, C., Orbanes, J., Chun, B. and Paradiso, J.: PingPongPlus: design of an athletic-tangible interface for computer-
    supported cooperative, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, pp. 394–401
    (1999).
    [9] Kato, J., Nakashima, T., Takeoka, H., Ogasawara, K., Murao, K., Shimokawa, T. and Sugimoto, M.: OpenPool: Community-based
    Prototyping of Digitally- augmented Billiard Table, GCCE ’13: Proceedings of the 2nd IEEE Global Conference on Consumer Electronics, pp.
    175–176 (2013).
    [12] Takegawa, Y., Tayanagi, E., Tsubakimoto, M. and Hi- rata, M.: Evaluation of a Piano Learning Support Sys- tem Focusing on the Learning
    Process, Proceeding of World Conference on Educational Media and Technol- ogy (EdMedia2013), pp. 2306–2314 (2013).
    [14]౔Ҫຑ༝Ղ, ٶԼ๕໌: ϓϩδΣΫγϣϯϚοϐϯάʹΑΔᝨԋ૗ֶशࢧԉγεςϜ, ৘ใॲཧֶձݚڀใࠂώϡʔϚϯίϯϐϡʔλΠ
    ϯλϥΫγϣϯ(HCI), Vol. 2017, N0. 15, pp. 1-8 (2017).
    [16] Akiyama, Y. and Miyashita, H.: Projectron mapping: The exercise and extension of augmented workspaces for learning electronic
    modeling through projection map- ping, Proceedings of the adjunct publication of the 27th annual ACM symposium on User interface software
    and technology, ACM, pp. 57–58 (2014).
    [17] Yoshida, H., Igarashi, T., Obuchi, Y., Takami, Y., Sato, J., Araki, M., Miki, M., Nagata, K., Sakai, K. and Igarashi, S.: Architecture-Scale
    Human-Assisted Addi- tive Manufacturing, Acm Transactions on Graphics, Vol. 34, No. 4, pp. 1–8 (2015).
    [18] ڮຊࡊఠ, ௣ඌҰ࿠: StudI/O: ϓϩδΣΫγϣϯϚοϐϯάʹΑΔτΠϒϩοΫͷ૊Έཱͯɾه࿥ࢧԉ, ৘ใॲཧֶձ࿦จࢽ, Vol. 57,
    No. 12, pp. 2577-2588 (2016).
    [20] Microsoft: HoloLens, https://www.microsoft.com/hololens (2016).
    [22] yumu19: cyberkeyboard൓Ԡ·ͱΊ - Togetter·ͱΊ, https://togetter.com/li/946782 (2016).
    26

    View full-size slide