Upgrade to Pro — share decks privately, control downloads, hide ads and more …

人工知能研究の最前線と中学・高校で得た経験

 人工知能研究の最前線と中学・高校で得た経験

2017年7月3日に愛媛県立松山西中等教育学校で行った講演で使用したスライドです。

Shoya Ishimaru

July 03, 2017
Tweet

More Decks by Shoya Ishimaru

Other Decks in Technology

Transcript

  1. ࣗݾ঺հੴؙᠳ໵ • ೥দࢁ੢த౳ڭҭֶߍଔۀ ̎ظੜ • ೥େࡕ෎ཱେֶେֶӃ޻ֶݚڀՊमྃ ܦྺ ೥݄ݱࡏ • ܦ࢈লɾ*1"ೝఆະ౿εʔύʔΫϦΤʔλ

    • %',* υΠπਓ޻஌ೳݚڀηϯλʔ ݚڀһ • ΧΠβʔεϥ΢ςϧϯ޻Պେֶ
 ϦαʔνɾΞιγΤΠτ • େࡕ෎ཱେֶ٬һݚڀһ • ܚጯٛक़େֶ,.%ݚڀॴݚڀһ
  2. ࣗݾ঺հੴؙᠳ໵ • ೥দࢁ੢த౳ڭҭֶߍଔۀ ̎ظੜ • ೥େࡕ෎ཱେֶେֶӃ޻ֶݚڀՊमྃ ܦྺ ೥݄ݱࡏ • ܦ࢈লɾ*1"ೝఆະ౿εʔύʔΫϦΤʔλ

    • %',* υΠπਓ޻஌ೳݚڀηϯλʔ ݚڀһ • ΧΠβʔεϥ΢ςϧϯ޻Պେֶ
 ϦαʔνɾΞιγΤΠτ • େࡕ෎ཱେֶ٬һݚڀһ • ܚጯٛक़େֶ,.%ݚڀॴݚڀһ ͲΜͳݚڀΛͯ͠ΔͷʁݚڀऀͬͯͲΜͳ࢓ࣄʁ ͲΜͳܦҢͰݚڀऀʹͳͬͨͷʁ
  3. ਓ޻஌ೳͷ͜Ε͔ΒͱγϯΪϡϥϦςΟ αΠϘ΢ζɾϥϘ੢ඌହ࿨͞ΜͷൃදΛ΋ͱʹ࡞੒
 ʮ*5ۀքͷاۀͰಇ͘ʯݚڀऀ͕ߟ͍͑ͯΔ͜ͱ!ट౎େֶ౦ژ
 IUUQTXXXTMJEFTIBSFOFUOJTIJP ਓؒ
 "* "* ਓؒ ະདྷ ݱࡏ

    γϯΪϡϥϦςΟ • "*ਓͷ࣌୅
 "*͕ਓʹউΔ෼໺͕૿͖͍͑ͯͯΔ • ਓؒ "*"*ਓ
 ྫ ΞυόϯευɾνΣε "*ʹΑͬͯਓͷೳྗΛ֦ுɾڧԽ "VHNFOUFE)VNBO ਓͷߦಈ΍ҙਤͷೝ͕ࣝෆՄܽ
  4. ೴ͷηϯγϯά [1] Shoya Ishimaru, Kai Kunze, Koichi Kise and Masahiko

    Inami. Position Paper: Brain Teasers - Toward Wearable Computing that Engages Our Mind. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct Publication (UbiComp2014). September 2014. 1-back 2-back 4-back n-backهԱήʔϜதͷ೴׆ੑ ෛՙ 3-back
  5. هԱྗΛ஁͑Δͷʹ࠷దͳ೉қ౓͕෼͔Δ [1] Shoya Ishimaru, Kai Kunze, Koichi Kise and Masahiko

    Inami. Position Paper: Brain Teasers - Toward Wearable Computing that Engages Our Mind. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct Publication (UbiComp2014). September 2014. 1-back 2-back 4-back n-backهԱήʔϜதͷ೴׆ੑ ෛՙ 3-back
  6. ϞʔγϣϯηϯαΛ࢖ͬͨߦಈೝࣝ ਭ຾ Ҡಈ(ెา) Ҡಈ(ిं) ಡॻ ன৯ ߨٛΛड͚Δ Ҡಈ(ెา) ༦৯ ςϨϏΛݟΔ

    07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 06:00
  7. ϞʔγϣϯηϯαΛ࢖ͬͨߦಈೝࣝ ਭ຾ Ҡಈ(ెา) Ҡಈ(ిं) ಡॻ ன৯ ߨٛΛड͚Δ Ҡಈ(ెา) ༦৯ ςϨϏΛݟΔ

    07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 06:00
  8. ߦಈೝࣝʹ༻͍͍ͯΔ༷ʑͳηϯα ҆Ձ ߴՁ 1ઍສ 100ສ 10ສ 1ສԁ fNIRS SMI Mobile

    Eye Tracking glasses Google Glass Software based eye tracking J!NS MEME Tobii e4c E4 wristband Pupil Emotiv EEG FLIR One Fitbit
  9. ॠ͖ͷηϯγϯά [2] Shoya Ishimaru, Jens Weppner, Kai Kunze, Andreas Bulling,

    Koichi Kise, Andreas Dengel and Paul Lukowicz. In the Blink of an Eye - Combining Head Motion and Eye Blink Frequency for Activity Recognition with Google Glass. Proceedings of the 5th Augmented Human International Conference (AH2014). March 2014.
  10. ॠ͖ͷηϯγϯά reading watching solving sawing talking [2] Shoya Ishimaru, Jens

    Weppner, Kai Kunze, Andreas Bulling, Koichi Kise, Andreas Dengel and Paul Lukowicz. In the Blink of an Eye - Combining Head Motion and Eye Blink Frequency for Activity Recognition with Google Glass. Proceedings of the 5th Augmented Human International Conference (AH2014). March 2014.
  11. ॠ͖ͷηϯγϯά [2] Shoya Ishimaru, Jens Weppner, Kai Kunze, Andreas Bulling,

    Koichi Kise, Andreas Dengel and Paul Lukowicz. In the Blink of an Eye - Combining Head Motion and Eye Blink Frequency for Activity Recognition with Google Glass. Proceedings of the 5th Augmented Human International Conference (AH2014). March 2014. reading watching solving sawing talking
  12. ॠ͖ͷύλʔϯͰ૷ணऀ͕ࠓԿΛ͍ͯ͠Δ͔෼͔Δ [2] Shoya Ishimaru, Jens Weppner, Kai Kunze, Andreas Bulling,

    Koichi Kise, Andreas Dengel and Paul Lukowicz. In the Blink of an Eye - Combining Head Motion and Eye Blink Frequency for Activity Recognition with Google Glass. Proceedings of the 5th Augmented Human International Conference (AH2014). March 2014. reading watching solving sawing talking ॠ͖͚ͩͰ ಄ͷಈ͖ΛՃ͑Δͱͷࣝผਫ਼౓
  13. ʮ৺ԹܭʯߦಈͷมԽ͔Βؾ෼ΛਪఆͰ͖Δ [7] Shoya Ishimaru and Koichi Kise. Quantifying the Mental

    State on the Basis of Physical and Social Activities. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct Publication (UbiComp2015). September 2015.
  14. ʮສޠܭʯ؟ٿӡಈ͔ΒಡΜͰ͍Δจࣈͷྔ͕෼͔Δ [3] Shoya Ishimaru, Kai Kunze, Koichi Kise, and Andreas

    Dengel. The Wordometer 2.0 - Estimating the Number of Words You Read in Real Life using Commercial EOG Glasses. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct Publication (UbiComp2016). September 2016.
  15. ࢹઢͷηϯγϯά 3 4 5 5 5 6 7 7 novice(ॳ৺ऀ)

    intermediate(தڃऀ) expert(্ڃऀ) reading solving score (out of 14)
  16. ڭՊॻͷಡΈํ͔Β಺༰ͷཧղ౓͕෼͔Δ intro. def. appl. intro. def. appl. intro. def. appl.

    intro. def. appl. intro. def. appl. intro. def. appl. Expert Intermediate Novice reading solving reading solving reading solving B SFBEJOH C TPMWJOH F SFBEJOH G TPMWJOH /PWJDF D SFBEJOH E TPMWJOH *OUFSNFEJBUF &YQFSU Introduction Definitions Applications (a) text [4] Shoya Ishimaru, Syed Saqib Bukhari, Carina Heisel, Jochen Kuhn, and Andreas Dengel. Towards an Intelligent Textbook: Eye Gaze Based Attention Extraction on Materials for Learning and Instruction in Physics. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct Publication (UbiComp2016). September 2016.
  17. ڭՊॻͷಡΈํ͔Β಺༰ͷཧղ౓͕෼͔Δ intro. def. appl. intro. def. appl. intro. def. appl.

    intro. def. appl. intro. def. appl. intro. def. appl. Expert Intermediate Novice reading solving reading solving reading solving B SFBEJOH C TPMWJOH F SFBEJOH G TPMWJOH /PWJDF D SFBEJOH E TPMWJOH *OUFSNFEJBUF &YQFSU Introduction Definitions Applications (a) text [4] Shoya Ishimaru, Syed Saqib Bukhari, Carina Heisel, Jochen Kuhn, and Andreas Dengel. Towards an Intelligent Textbook: Eye Gaze Based Attention Extraction on Materials for Learning and Instruction in Physics. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct Publication (UbiComp2016). September 2016.
  18. ڭՊॻͷಡΈํ͔Β಺༰ͷཧղ౓͕෼͔Δ intro. def. appl. intro. def. appl. intro. def. appl.

    intro. def. appl. intro. def. appl. intro. def. appl. Expert Intermediate Novice reading solving reading solving reading solving B SFBEJOH C TPMWJOH F SFBEJOH G TPMWJOH /PWJDF D SFBEJOH E TPMWJOH *OUFSNFEJBUF &YQFSU Introduction Definitions Applications (a) text [4] Shoya Ishimaru, Syed Saqib Bukhari, Carina Heisel, Jochen Kuhn, and Andreas Dengel. Towards an Intelligent Textbook: Eye Gaze Based Attention Extraction on Materials for Learning and Instruction in Physics. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct Publication (UbiComp2016). September 2016.
  19. ࢹઢ͔Βʮ೉͍͠ͱײͨ͡୯ޠʯ͕෼͔Δ ܭଌ͞Εͨࢹઢ ԁͷେ͖͞஫ࢹ࣌ؒ ೉͍͠ͱײͨ͡୯ޠͷਪఆ ΦϨϯδਖ਼ղ৘ใਫ৭ਪఆ݁Ռ [5] େࣾ ҁ೫, ੴؙ ᠳ໵,

    Olivier Augereau, ԫ੉ ߒҰ. ࢹ఺৘ใΛ༻͍ͨओ؍తߴ೉қ౓୯ޠͷਪఆ. ిࢠ৘ใ௨৴ֶձٕज़ ݚڀใࠂ, vol. 115, no. 517, PRMU2015-189, pp. 149-153, March 2016.
  20. ࢹઢ͔Β50&*$είΞ͕෼͔Δ ௕จΛ୊ಡΉͱฏۉઈରޡࠩ఺ ຬ఺த ͰਪఆՄೳ େࡕ෎େɺӳޠशख़౓Λਪఆ͢Δٕज़Λ։ൃ−ӳจ໰୊ΛಡΈղ͘؟ٿͷಈ͖ղੳ ೔ץ޻ۀ৽ฉ 2016/8/3 https://www.nikkan.co.jp/articles/view/00394843 
 [6]

    ౻޷ ޺थ, ੴؙ ᠳ໵, Olivier Augereau, ԫ੉ ߒҰ. ࢹ఺৘ใΛ༻͍ͨӳޠशख़౓ਪఆ๏ͷ࣮ݧతݕ౼. ిࢠ৘ใ௨৴ֶձ ٕज़ݚڀใࠂ, vol. 115, no. 517, PRMU2015-195, pp. 185-190, March 2016.
  21. ࣗ཯ਆܦͷಇ͖ͱηϯγϯά [7] Robert M. Sapolsky "Why Zebras Don't Get Ulcers."

    WH Freeman, 1994. ަײਆܦ ෭ަײਆܦ ɾٳଉ ɾϦϥοΫε ɾ׆ಈ ɾूத ɾετϨε ɾۓு ೒࠼͕େ͖͘ͳΔ ݂؅͕ऩॖ͢Δ ඓͷԹ౓͕Լ͕Δ ৺ഥ਺্͕͕Δ ೒࠼͕খ͘͞ͳΔ ݂؅͕๲ு͢Δ ඓͷԹ౓্͕͕Δ ৺ഥ਺͕Լ͕Δ
  22. ࿦จΛಡΉ Figure 2. Proximity sensor value and ground truth of

    2 participants. a video, solving a mathematical problem and sawing. Dis- tinguishing between these activities involves not only recog- nizing physical actions (that can easily be captured using for example on body motion sensors) but also a cognitive com- ponent which is what we hypothesize eye blinking frequency and head motion correlate with. We evaluate our method on a data set containing eight par- ticipants demonstrating an average classification accuracy of 67% using blink features only and 82% using blink and mo- tion features. Related Work There is a large corpus of work to recognize human activities. A variety of physical activities can be recognized using body- mounted sensors [5]. On the other hand, some researchers focus on our cognitive activities. Bentivoglio et al. have stud- ied the relation between sitting activities and blink patterns [3]. They described that the blink rate changes when partic- ipants were reading, talking and resting. Acosta et al. have presented that working with computers causes a reduction of blink [1]. Haak et al. have described that emotion, especially stress, effects blink frequency [9]. Therefore, blink pattern should be one of the important features to recognize our ac- tivities. Some researchers have applied an image processing method [6] and an eye tracking approach [4] to detect blinks. As far as we know, we are the first to use a simple proximity sensor embedded in a commercial wearable computing sys- tem for activity recognition and to combine it with head mo- tion patterns. APPROACH We believe that blink patterns can give a lot of insights about the user’s mental state (drowsiness etc.) and the user’s ac- tivity. To show this we use an infrared proximity sensor on Google Glass (see Figure 1). It monitors the distance between the Google Glass and the eye. Figure 2 shows the raw values of the sensor. While the main function of this sensor is to detect if the user wears the device, when the user blinks, a peak value appears due to the eye lid and eyelashes move- ment. Our algorithm is based on two stages. The first stage is the pre-processing stage of the raw sensor signal. The pre- processing stage extracts the time of blinks. We validate the pre-processing results with ground truth blink information. Q Q Q Q Q Q Q Q Q from peak to average from peak to average Figure 3. Blink detection by calculating peak value. Secondly, the main part of our algorithm calculates features based on the detected blinks. Getting raw data of infrared proximity sensor on Google Glass is not provided in an of- ficial way. We rooted (customized) our Glass on the basis of Glass hacking tutorial [7] and installed our own logging application [8] for the experiment. Blink detection During pre-processing blinks are detected based on the raw infrared proximity sensor signal. We move a sliding win- dow on the sensor data stream and monitor whether the cen- ter point of each window is a peak or not according to the following definition. We calculate the distance from one sen- sor value of the center point in the window (p5 in Figure 3) to the average value of other points (p1 , p2 , p3 , p7 , p8 and p9 ). The preceding and subsequent points of the center (p4 and p6 ) are excluded from the average calculation because their sensor values are often affected by the center point. If the distance is larger than a threshold ranging from 3.0 - 7.0 we define the center point as a blink. Because the shape of the face and eye location vary, the best threshold for the peak detection varies for each user. Figure 2 with the same scale for each sub-graphic also demonstrates different signal vari- ations for different users. We calculate the best threshold (in 0.1 steps ranging from 3.0 to 7.0) by evaluating the accuracy based on the ground truth information. This approach can be applied only in off-line evaluation. In on-line usage, we need a few seconds for calibration before detection. During the calibration term, Glass urges the user to blink as matching some timing. We get sensor values and actual blink timing from calibration and evaluate the best threshold. Blink frequency based activity recognition As an output of our pre-processing step we extract the times- tamps of blinks and compute a three-dimensional feature vec- tor. One is the mean blink frequency which describes the number of blinks during a period divided by the length of a period. Two other features are based on the distribution of blinks. Graphically, this can be understood as the histogram of the blink frequency. Figure 5 shows five histograms with a period of 5 minutes. The x-axis describes the mean blink frequency (0.0 - 1.0 Hz) and the y-axis describes the blink counts of each frequency. The number of specified bins per histogram is 20 having a resolution of 0.05 Hz. The frequency value is calculated as inverse value of the interval between two blinks. The second and third features are defined as the x-center of mass and the y-center of mass of the histogram. 2 ֶज़࿦จͱ͸ • ԿΛ໌Β͔ʹ͍ͨ͠ͷ͔ • ͦΕ͕෼͔Δͱͳͥخ͍͠ͷ͔ • ઌߦݚڀͱൺ΂ͯԿ͕͍͔͢͝ • Ͳ͏΍ͬͯ໌Β͔ʹ͢Δͷ͔ • Ͳ͏΍ͬͯ༗ޮͩͱݕূ͔ͨ͠ • ͜ͷݚڀ෼໺΁ͷߩݙ͸Կ͔ ͜ΕΒͷ৘ใ͕ߴ଎ͰಡΈऔΕΔ ϑΥʔϚοτʹͳ͍ͬͯΔจॻ
  23. ͳͥ࿦จΛॻ͘ͷ͔ Shimon Petyon Jones (Microsoft Research Cambridge). "How to Write

    a Great Research Paper."
 https://www.microsoft.com/en-us/research/academic-program/write-great-research-paper/ • ൃݟΛଟ͘ͷਓʹ఻͑ΔͨΊ
 ֶձൃද͚ͩͰ͸ͦͷ৔ʹډ߹Θͤͨਓʹ͔͠఻ΘΒͳ͍ɻ
 ࿦จ͕ࡶࢽʹࡌΕ͹ੈքதͷਓ͕ಡΉ͜ͱ͕Ͱ͖Δɻ • ݚڀ੒ՌΛ࣌୅Λ௒͑ͯ఻͑ΔͨΊ
 1BQFSTBSFGBSNPSFEVSBCMFUIBOQSPHSBNT
 ʮ࿦จ͸ϓϩάϥϜΑΓང͔ʹ௕࣋ͪ͢Δʯ
  24. ݚڀऀʹඞཁͱ͞ΕΔ ͱੴؙ͕ࢥ͏ εΩϧͱ͸ʁ  ࿦จΛಡΜͰઌߦݚڀΛௐࠪ͢Δ  ݚڀςʔϚ ղܾ͍ͨ͠໰୊ ΛܾΊΔ 

    ख๏ΛߟҊ࣮ͯ͠૷͢Δ  ࣮ݧͯ͠༗ޮੑΛݕূ͢Δ  ࿦จΛॻ͘  ֶձͰݚڀ੒ՌΛ఻͑Δ
  25. ݚڀऀʹඞཁͱ͞ΕΔ ͱੴؙ͕ࢥ͏ εΩϧͱ͸ʁ  ࿦จΛಡΜͰઌߦݚڀΛௐࠪ͢Δ  ݚڀςʔϚ ղܾ͍ͨ͠໰୊ ΛܾΊΔ 

    ख๏ΛߟҊ࣮ͯ͠૷͢Δ  ࣮ݧͯ͠༗ޮੑΛݕূ͢Δ  ࿦จΛॻ͘  ֶձͰݚڀ੒ՌΛ఻͑Δ ӳޠྗ ࣮૷ྗ ೜଱ྗ